SAM SOLIS

Estimated reading time: 3 minutes, 13 seconds. Contains 646 words

In terms of development, I have always been a fan of simply using Linux.

I always installed a Linux distribution alongside my Windows OS. Although I had a habit of distro hopping before, I always find myself going back to an ubuntu-based distribution (Ubuntu or PopOS) cause of the resourceful forums and active community. Also, I much prefer gnome-style desktop but I am aware that it is resource hungry.

I dual-boot Linux for the following reasons:

• I didn't want to go through the pain of adding extra apps on top of a terminal (I didn't want so much apps to eat my memory)
• Most servers are linux-based and I want to familiarize myself with the environment
• I'm an advocate of open-source products (a.k.a. I'm really cheap and I always try to find free alternative software first)
• I've never upgraded my laptop's HDD to an SSD so I always had to endure slow boot time if I wanted to use Windows (again I'm very frugal 😂)

As a result, I use Linux about 95% of the time when I'm using my laptop and I rarely boot Windows, usually when I really need to open a file or an application that is not compatible with Linux or when I play games.

But eventually, I was gifted with an SSD and I saw how it significantly made a difference in boot time and opening applications. Since then, I considered migrating my workspace and adjusting my workflow. I never really committed to it until I recognized that some of my responsibilities would be more conveniently and quickly accomplished if I find a sweet balance inside Windows OS. Some of my roles require Exploratory Data Analysis. Although, this is still possible in Linux, intuitive software tools are more available and supported in Windows, free or proprietary.

And so my set up is shown as seen in the figure: (pink apps are paid)

Essentially, I wanted to highlight that my drives are partitioned to separate System Files and Data Files. I do this for the following reasons:

• If system files partition occurs an error, Data Files will not be affected
• Accessibility of Data Files to any OS that I boot (much applicable when I still dual-boot)
• Save memory allocation in SSD for app installation (for faster boot times of applications)

Subsequently, I kept all database servers within Windows OS and did not install them in my WSL. I did this for the following reasons:

From my experience in R&D, data is prime factor to consider before finalizing a plan of action when researching. Because without data, how can we empirically and justifiably show valuable information to other people or stakeholders?

There are many times that I have ambitious research topics in mind only to be pulled back to reality that I need to find a way to collect data either existing or non-existing.

Taking on various projects opened my eyes to the endless possibilities of data sources. And these are methods that have really helped me in the past:

•  Utilizing the search engine. The first thing I do is to type keywords in related to the topic at hand. You'd be surprised with the vast availability of sources out there if you dig hard enough.
  
• Scouring the internet for possible reports that can be downloaded (usually published reports can be found in respective websites of different private and public companies). You shouldn't just settle with what search engines immediately return. It pays to check these sites out because when you view these reports, you might sense patterns that could help you extract information repeatedly.
  
• Personally contact and request data from organizations and institutions. Both public and private companies, may not publish raw data, however, it's worth the effort to  approach them in terms of the articles they publish in their sites or publicly. You don't know, they might just write back and help you with the data you need.
  
• Scrape a data from their website. This isn't a common knowledge for some people but people can actually 'save' information from websites. Like a screenshot of their website but already organized through tables. Web scraping is the term for it and there are also existing software services that offer doing this for you!
  
• Experimentation. Collect data through empirical methods. Especially for scenarios where data is really non-existent. Personally, this has always been my last option unless my research requires it. This is because experimentation adds extended time needed before data processing and analysis.
  

I feel like some of the bullets above should have been common information. But I observe that some people already decide that some methods are unfavorable - especially if it means re-encoding or re-organizing the data again. As a result, they turn away these feasible data sources and return searching for alternative sources in hope that they could find one that is more convenient to work with.

But reality is, technology has far advanced that we really need to appreciate how it can help us in cases like this.

Just like how people should be aware of different sources for data, people should also be aware of the different tools that's available that could help them extract and systematize their data from more difficult providers. Most times, data coming from the internet would be published in PDF documents or image content. And these are the most difficult sources to manage since immediately, people would accept defeat encoding mountain-size image data.

But if we are resourceful, we can be creative with the ways we can extract information from these examples. There are many image-to-text or pdf-to-text converters available online. Check them out and see if they can help you! Some that I normally use are the following:

•  PDF Tables
• Online OCR
• Python - given that some sites have paid services, when I handle large amount files, I do code my own text extractors. There are packages out there that are available to help you out:
• Tabula-py
• Py-Tesseract

With these pointed out, I would want to raise it as well that data owners should lessen inconvenience if they are hiring analysts to extract information from their data and in a time-constrained schedule. It would lessen data processing period if they provide raw files. It regularly happens that a client would provide tabular reports in PDF format only to discover that they have the raw files in document, excel or CSV format. The process could have been more efficient if raw files were submitted early on for data processing. It will significantly reduce the time needed to encode accurate the characters for analysis and more effort can be allotted in the analysis itself.

Estimated reading time: 2 minutes, 35 seconds.

There's something about programming for the clever, critical and logical person that makes it alluring and addicting. There are people out there that will always think that there is a better or efficient way to handle a task at hand.

I personally have frequent anecdotes wherein I really don't want to do redundant tasks and I use coding to automate these tasks for me. And just that mentality influenced my interest in wanting a career that encourages me to code. From then on, I know I wanted a career in Software Development or Software Engineering.

I eventually got a job in this field but I really had no clue about the practices and tools being used for this discipline. I essentially know how to code, but some of the terms aren't even familiar to me. But there really is a lot to consider before you can finally deploy your code.

I'm grateful that fresh hires are trained in my previous job and they accept that new recruits may come from different backgrounds. They know that a baseline or synchronization is needed before we are tasked with serious work. During our onboarding training, I'm very intimidated by my colleagues at that time cause I felt like I was the odd one out. I had an undergraduate degree for Electronics and Communications Engineering, not Computer Science. So in some sessions they would already know what are being taught and I would be in awe with the different tools used.

So for anyone who is feeling the same I did, I'd like to give a gist of what was being taught to us during our onboarding. I'm not going to write it because, well, we're tech people. We don't like to read a lot. So I'd like to promote this video playlist that essentially discusses the different tools and different practices in the field. I hope this gives you an insight of what you can expect if you take a Software Engineer or Developer job.

The YouTube channel is named Missing Semester. This is a class taught at MIT and in their site they shared their intent for coming up with the lessons:

Classes teach you all about advanced topics within CS, from operating systems to machine learning, but there’s one critical subject that’s rarely covered, and is instead left to students to figure out on their own: proficiency with their tools. We’ll teach you how to master the command-line, use a powerful text editor, use fancy features of version control systems, and much more!

To get an idea of what they teach, these is the title of the videos the published:

• Course overview + the shell
• Shell Tools and Scripting
• Editors (Vim)
• Data Wrangling
• Command-line Environment
• Version Control (Git)
• Debugging and Profiling
• Metaprogramming
• Security and Cryptography
• Potpourri

Don't be intimidated if you don't know any of the words! Test it out by watching the first video!

I bombed my first interview. Granted, I don't really know much about coding then but I'd like to believe I have a passion for it. Among all the courses I took while I was taking my undergraduate degree, I loved my coding courses and most excelled in them compared to my other majors. This gave me a false confidence and convinced me that I can rely on stock knowledge alone.

I feel like it's standard to use coding test platforms to prepare for an interview for software development jobs. Most times corporate recruiters test a candidate's knowledge of algorithms and data structures. This is why hackerrank.com and other similar platforms are alluring for last minute preparations.

I attended the test and interview while only having practiced a few rounds in hackerrank.com. Although they've acknowledged the result of the coding exam, my interview didn't go so well. The interview dug deeper to my knowledge and practice of the craft - to which I have no experience in except for my class projects which are decided and scoped by the professor. I was not really equipped for standard practices in a work setting in my undergraduate course. Our education was really concentrated on theories since we are a research institution. 

In retrospect from that experience, I appreciate my interviewers. They treated me kindly despite my inexperience. And I was left with a wonderful piece of advice as we ended. They said:

try developing a project you're interested on.

Eventually, I was able to take an entry level job as a software developer, but not with a portfolio yet. While I was at my first job, I was also taking up my graduate studies which motivated me to do a mini project of developing a Point-of-Sale (POS) system for my parents' restaurant business. I submitted it to my professor and was impressed by my commitment to the project. After that, I was offered an academic research project to which I accepted. From then on, that decision has opened more opportunities for me that I am grateful for today.

Going back to the piece of advice my interviewers gave me, I never really appreciated the advice until I started doing the mini project.

The difference between coding tests vs. mini projects

• Coding tests are focused on algorithms and structures
• Coding tests are focused on your conceptual knowledge of a programming language
• Coding tests are formulated such that they are solvable in a limited period of time  
• Doing mini project is multidisciplinary
  
• Doing mini project is an experience-based type of learning
• Mini projects have a relatively larger time frame than coding tests 

My ultimate case is that doing the mini project has really helped me to easily retain software development concepts and practices over practicing on coding tests. It did not box me to focus only on the programming language nor focus only on how to write a block of code that would work to solve a problem. It enlightened me on all aspects of software development.

Software development is a large discipline that does not only involve programming and running your code and hope it works. It's a pipeline that involves planning, implementation, testing and debugging.

To end, I am not discouraging the use of coding tests. The topic it focuses on is inevitable in job interviews and is usually the first phase of a candidacy after all. But having a portfolio of mini projects and an arsenal of experience will help you in the final interview.

I just wanted to share a piece of myself in this blog. In my next posts, I plan to be more objective and share a lot of materials I used for self-learning to become a better software developer. I plan to share some standard practices in software engineering that was not taught to me before I was trained at my first job.

The tech industry is perpetually evolving. Moreover, with any other career paths, we will always learn something new in our respective fields.

This page serves as a blog about my past experiences and what I learned in retrospect throughout my career and my projects.

This is not intended to be a diary, but rather, a collection of notes that I would like to impart to those who are very much interested in the field that I am in.