Over-Engineered

Scalability is one of those things that is expected from senior software engineers, and frequently appear on senior level job descriptions. As good as this may sound, it’s something that can easily lead to over engineered codebases, leading to many abstractions in the code that result from something I that call premature optimization. Many companies, especially enterprise companies, have the tendency to expect their engineers to build a product with scalability in mind, without even knowing if the product will ever scale.

Yes, it’s important to design software with scalability in mind, but it’s important not to create a solution to a problem that does not exist. That’s called putting the cart before the horse. It results in creating problems instead of solving them. One of these problems is a codebase that is over engineered.

While I was at Endava as a senior engineer, I came across several codebases from their clients that were sorely over engineered as a result of trying to be “scalable” which led to very ambiguous, and very hard to maintain code. The result was that the codebases were not scalable at all, which was the exact opposite of the original intention of a scalable codebase.

An example

Here's an example of premature optimization. Let's say that you have a React app that uses global state. You installed a package to manage global state like Jotai. Jotai exposes primitives that allow you to set and get state, like useAtom. This api is so straightforward, that you can easily just plug it into any component where you need the state in:

'use server'
import { useAtom } from 'jotai'

export async function UserAccount({ id }: UserProps){
  // use the primitives directly
  const [user, setUser] = useAtom(userID)
  const { data } = await fetch('/api/user/${id}')
  setUser(data.user)
  ...
}

A premature optimization in a codebase would abstract the setUser call into a function called setUser into a separate file called 'setters' where a bunch of similar setters live and define it as such:

import { useAtom } from 'jotai'

// unnecessary optimization
export function setUser(user: User): User {
  const [user, setUser] = useAtom(user)
  return user
}
...

This may seem hardly believable, but I assure you, this kind of code exists in real codebases, even in big tech companies. This is the type of code that usually is produced by junior developers, and even some seniors. This is why code reviews are crucial. Imagine a ton of code like this in a very large codebase. I've seen it, it's not good.

When to optimize

Optimization is not a bad thing. It's not that you shouldn't optimize. You should optimize when you need to, not when you don't need to. If you find a piece of code that is repeated several times throughout the codebase, then extract it for code reuse. If you see a reference being used multiple times in the same function or file, then create a variable. It's more efficient. But there's usually no need to create a variable for something that is used only once.

If all software engineers were to keep this in mind, there'd be a lot more cleaner and leaner codebases out there. Anyway, this article does a great job explaining this issue.