Architecting enterprise Nextjs web applications with Directus and Refine.dev

When it comes to architecture, we often think about MVC, microservices, monoliths, etc. Traditionally, a monolith setup would take you very far, take a look at Shopify for example. Most frameworks such as Laravel, Ruby On Rails have been battle tested and have proven itself as tools that will scale. As recommended by Martin Fowler, we should start with monolith and only start decoupling code by services when the complexity increases. This blog explains how I used my software engineering knowledge to maintain a 150k+ LOC Nextjs application.

When setting up Nextjs, you would just run the commands from docs in the terminal and it would setup a full stack application. Other parts of SDLC, such as testing, deployment would be automated using CICD tools such as Github Actions. This would go a long way if you are using the api route for backend or server actions which is code that runs on the server side similar to PHP. For database ORMs usually Prisma or Drizzle is used. Trpc can also be used where you get type safe client out of the box but it always seemed limiting as it requires a lot of common code to be written such fetching data by id, list etc. In Spring Boot for example, you can use the CRUD repository to generate CRUD operations. Testing can be done using Cypress, Playwright or Jest.

The tooling of Nextjs made me think of only using its ecosystem which are Next Auth, server actions, api route, and Prisma/Drizzle. But this made me realize a lot of stuff had to be reinvented, such as authentication with RBAC and also ABAC, database triggers, and a custom CMS. Previously the client was using Directus CMS to manage the database so I recommended that we use it as API and internal management while the Nextjs application will be used for public access and some role specific operations. This cut down a lot of time for development as I had to only worry about database design and the API query logic for the directus API and the database triggers. The app currently has 5 roles. Organizing code based on role is the most time consuming task I have ever done especially when there is column level access control. Thankfully Directus has a library to generate types similar to Supabase. From the base types, I created role specific types for each collection. But this was not enough. Each page had fields specific to a role so I had to create data fetching logic per role and also do checks in the UI to show or hide fields based on role, giving rise to spaghetti code. All the design patterns I had known came to no use in this case. If there were different views per role, then it would be easily separated. But since the views were the same, I had to use a lot of conditional rendering.

Then I discovered Refine.dev, a framework built on top of React. This is an opinionated framework that uses provider pattern to modularize the codebase as well as abstractions for data fetching hooks using Tanstack Query. This was a goldmine for me. With Refine.dev, the auth, access control, data, live and notification providers are easily replaceable. Need to change an Express API client to Firebase? Just add a new provider! If there are too many conditional checks for roles, Refine provides CanAccess Higher Order Component as well as usePermissions and useCan hooks. These can be used to reduce the spaghetti code as all the checks are made in the access control provider, for which I used Casbin. The clear separation of concerns out of the box is what makes Refine.dev powerful.

But with great power comes great responsibility, and in this case, manual typing of responses, the second most time consuming task I have done for this project. I have used GraphQL extensively in the past years and got used to autogenerated typings. But in Refine, due to the data provider being separate, you cannot even pass the client SDK types back to the response in the hook (maybe someday this becomes possible). REST APIs are a pain, but thanks to Directus API, I decided to use the good old typing the JSON response (thanks to Maximiliano Firtman for teaching this on FrontendMasters Flutter course). You may think why I didn't just use the types generated by Directus. There are somewhere around 20-100 fields among each page, this means if I were to reuse the types for each role, I would have to use Omit and Pick just to get the correct schema. I also got this motivation to manually type the data from one of Brian Holt's courses where he mentioned he found it easier to manually maintain the types.

This workflow lasted for a long time and still is in use. I have always preferred to use Test Driven Development in mobile and web apps. But when there is a lot of code to write, I would prefer to focus on the use cases and test that first followed by any core logic not part of the use cases. Snapshot testing can also be kept as an option. In my case, I decided to carry out end to end tests from the beginning with a Directus docker container, covering the use cases only. Another tool I recommend is Storybook. This will force you to think in components instead of business logic, a common mistake of mine. You can also develop the components in isolation without having to run the entire application.