Ask HN: What practices can we use to avoid overengineering in our projects?

About a year ago, I joined a web dev team that at the time seemed very close to an ideal work environment that you only see in books.

Every good practice is implemented: strict version control using git flow, standarized linters, unit and integraton tests that are required to pass to get code in our repositories, code review with a minimum of two approvals required, and a long etc.

Over time, however, we've seen the project slow to a crawl due to the amount of overengineering that is commonplace: If a new need arises, it's usually met with a new dockerized service that will have a three layer structure, testing, handles things in an abstract and generic way for future proofing....

It's come to a point where we have about 70 repositories (mostly just generic modules like a logger or an error handling library, but still) to manage a project that, to be honest, it's way too simple for the amount of code we maintain.

We all agree that we've fallen for overengineering, but merely being aware of this fact hasn't made us any more effective at tackling the problem.

Do you guys have any suggestion about how we could both mitigate future overengineering and reducing the current codebase to acceptable levels? Anything from methodologies to rules of thumb or recommendations of books would be welcome at this point.

  • I actually appreciate this question for multiple reasons. Earlier in my career, I was praised for creating highly flexible solutions that could be easily customized. As a result, you can become conditioned to overengineer a solution.

    Here's my advice on how to avoid it. I learned to rephrase the following question: Do I need this now?

    At face value, this is a pretty easy question to answer, right? Here's the mental trap. I can always create a false scenario that "justifies" certain choices. "What if we have a spike of 1,000 users because of an announcement?" or "We need to make this system elastic now so we can scale up and down easily!"

    Although these questions are valid, they don't need to be addressed immediately. This goes back to "Bring the pain forward". Instead, try to rephrase the question of "Do I need this now?" to become "Can I build this out later?"

    This simple rephrasing does a few things.

    a. It tells you and the team that you're not avoiding it, but you're prioritizing it. The last thing you want to do is solve a problem that isn't a problem that needs to be solved now.

    b. It keeps important things on the radar. Very much like you do with Scrum and the backlog.

    It's all about reconditioning engineers to make choices that are aligned with the business's immediate needs vs. "Wouldn't our system be amazing to the engineering team if we did this?" As engineers, our job is to support the business. Or let me rephrase that more strongly, our jobs as engineers is to allow our business to be so amazingly awesome in how we help our customers.

    Any excessive code is distracting and introduces latency in development. Remember, we want to build fast scalable solutions. That means having a fast and scalable development process.

  • > very close to an ideal work environment that you only see in books.

    > Every good practice is implemented

    There's no such thing as perfect. Pretty much everything has a tradeoff in coding. What are you optimising for? You need to work out what you're optimising for and drop practices that are getting in the way of that.

    For example, if you're optimising for time to market and cost where a few bugs isn't a big deal then avoiding extensive testing could be a good tradeoff.

  • Always ask why and why not to do X. Talk in terms of value add and intrinsic costs. Don't pitch building a lawnmower if you just want to trim a tiny grass stem.

    The key here is simplicity. You want a system that 1) does what you want it to do 2) is simple and 3) easy to maintain. These points may mean different things to people within different domains, but for me, having fewer dependencies mean less points of failure, easier to understand/document and less prone to failures imo.

    Hah, I had to deal with a code review comment recently where someone pitched creating a whole new C++ container to track just a minor exception case. Doing it this way is gonna cost a lot of engineering time plus added risk of a regression, on top of insignificant value add to the existing implementation. Realistically speaking, we could've just limp on that edge case if statement check that's well documented to describe what its intent was, and if more hairy edge cases come up down the road, we can throw up our hands and circle back to implementing THAT "more ideal" solution. (I doubt we'll even get there. I have questions for Product if we have such hairy requirements before I touch code tbh).

  • - You should establish a list of the desired properties of the finished product.

    1] slices

    2] dices

    3] indicates time

    4] dishwasher safe

      When you achieve these properties, stop its done, but be ready for a request to develop V2.0
    
    maintain a >set< of codebases or snippet libraries each one specific to each property of a finished project.

    Continually review the efficacy efficiency and utility of each said codebase.

    Continually Review code for desireable properties [meta- engineering]

  • > It's come to a point where we have about 70 repositories (mostly just generic modules like a logger or an error handling library, but still) to manage a project that, to be honest, it's way too simple for the amount of code we maintain.

    It sounds like your application should be a monolith rather than a collection of microservices.

  • Reading “A Philosophy of Software Design” now, which answers these questions. The author states that raised red flags apply to systems too.

    So without context is hard to tell, but it’s a quick read, you might want to take a look and spot possible improvements without sacrificing quality.

  • Focus on the exact problem being solved, start small, refactor progressively as new requirements come in.

  • Maybe you have to try underengineering for a while, in order to find the middle path.