Aligning learnings from Software Engineering Degree with real job

It has been a quite a while since I wrote a blog on this site. Recently I have worked on a few projects that changed my view on how software should be developed. Based on my experience in startups, initially the main goal is to release the app as long as it works due to tight deadlines. The project was a fullstack Flutter app with Express backend developed in 2021, where I was tasked with fixing Typescript errors and versioning issues.

The codebase was huge, with a lot of spaghetti code and uncommented code blocks. It was a clear sign of tech debt. But the client's idea was to reuse the same widgets and backend code for the new app, meaning even more tech debt. I tried to find out which parts of the code can be reused by creating 87 use case diagrams, with ChatGPT's help and PlantUML. This helped me understand how the codebase works while also documenting it. Then I looked at the architecture diagram and realised the whole backend is dependent on Azure. All the code was written while only thinking about Azure services and did not follow any good design patterns. The client claimed that the app was very slow, and I told him it was because of Node.js single thread nature while being non-blocking for IO operations. Video files were being uploaded to a single instance of the server and then ffmpeg was used to format it, in the same thread. All of this processing meant the server will be busy handling all the IO operations and then processing the videos which is a CPU intensive task. I knew setting a job queue with Redis or BullMQ would kind of fix this issue, where the processing is done in a background worker and using signed uploads to directly upload to Azure bucket as well as load balancers with multiple instances of the server. Or even a worker thread may work but I didn't have time to figure it out. Then came the loading of the videos in the app. It was downloading one MP4 video at a time as you scroll xD. The way to fix this is to preload videos and using HLS encoding to stream the videos, which meant another service had to be developed. The backend had to be ditched as there was no time to migrate it from Azure to another cloud provider and a lot of code had to be modified (didn't follow Open Closed Principle).

I was doing my Software Design and Architecture course at that time and tried to see if there were any patterns used from the course such as adapter, factory, builder patterns etc. But there were none except singleton for database connections and this made realise the hard truth. Unopinionated backend frameworks such as Express are more likely to have tech debt in the long run due to its flexibility in allowing developers to make their own decisions when structuring their code and also when they are inexperienced. Laravel, Spring Boot or even Ruby On Rails enforce a certain structure, meaning no matter how experienced the developer is, they will follow the same convensions provided by these opinionated frameworks. It is easier to get started with Express but in the long run, if there were poor initial planning, it will be very difficult to maintain.

The Flutter app had a lot of patterns which helped me practise the design patterns I learned from the course. However the app was three years old with no version upgrades. Upgrading the Dart and Flutter version was a nightmare. A lot of the packages became deprecated and there were no alternatives. This meant the new app had to be rewritten from scratch. So I started recruiting developers and interviewed over ten candidates on Upwork. In the end I selected three, one of whom now works at Google in Silicon Valley. They chose the Clean Architecture with a two weeks deadline, and I got assigned to setup Firebase for backend with Sports API. I setup the database after creating UML diagrams then spent the rest of the days on studies.

The app did not meet all the requirements, and I blame it on Clean Architecture. Its a good way to structure your code but not when you are on a deadline. I think a simpler pattern such as bloc or MVP would have been a lot easier.

When designing projects, we often think of how many design patterns or architecture we can include, focusing on DRY, SOLID and all the other terms you find in books, often leading to overengineered project designs. But in reality, these are guidelines, not laws. It is not always possible to follow them, but thinking about it when writing the code or even passing the terms as prompts can greatly improve your code quality.