Why software engineering sucks

According to the 2024 stack overflow survey, only 20.2% of developers are happy at work. For 62.4%, their biggest frustration is technical debt, but we also often find the complexity of the stack (32.9% for the building stack and 32.3% for the deployment stack) and the amount of software used (22.8%), which could be summarized as tech bloat.

How can we end up in a such stupid situation?

For tech bloat, I think the developers have partly made their misfortune on their own. Over the past fifteen years, there has been an inordinate explosion in the number of tools. Marketing has mixed with information. We have lost the sense of lean. And we tend to use all the existing tools, just to follow the trend, even when they are not adapted to our project or the size of our organization.

But more generally, the problem comes from poor communication between two worlds that do not know how to talk to each other.

On the one hand, we have the world of the product/market/project. And on the other, the world of tech.

As I was a solo entrepreneur for 13 years and created about thirty products, I have a foot in each world. And I wonder how such an absurd situation can be so widespread.

Most people who do engineering are passionate. More than that, they are addicted. When we create code or tinker with a system, we enter in a flow state. This state makes us happy. And most techs are addicted to that. You can leave them alone and you are sure that they will work hard, just because it makes them happy.

To illustrate this subject, I had an edifying experience. In 2001, I was working for an American company. Following 9/11, the company ended up in chapter 11 and decided to withdraw from Europe.

We were doing internet hosting, and we didn’t have an autonomous architecture. So we had to recreate everything, the network, the security, the monitoring, customer extranets… Our directors and managers thought it was impossible and completely abandoned the idea of saving the day.

But in the time allotted before being disconnected from the United States, we recreated everything with only 4 of us and our customers were never disconnected. And even if the tools we offered them were more primitive, we always provided the same quality of service.

Without micromanagement, a team of engineers is capable of miracles. But the lack of trust means that engineers generally do not have the necessary ownership to be at the top of their abilities.

However, engineers have two big flaws, which explains this lack of trust.

First of all, they tend to be too perfectionist and to create while being in a bubble, without ever showing what they do, because “it’s too early”.

The second problem is related to the first. They also tend to create stuff because it makes them happy, without wondering if it might interest users.

On the other side, we have business and market.

The secret to successful business is to start before you’re ready.

And of course, a product, no matter how cool, is almost unsellable if it doesn’t address a customer problem. It takes a lot of influence to create a need and it’s easier to have a product that people want.

To solve the engineers’ problem, most companies have chosen micromanagement. But it doesn’t work. Decisions made by people who don’t practice engineering on a daily basis are always stupid, no matter how intelligent and competent they are. And the more an engineer is told how to do his job, the less effective he is.

Most of the time, decision-makers have no long-term technical vision. They don’t know how to go beyond the MVP mindset. This is why technical debt is neglected.

Once you’ve reached the product market fit (let’s say that a product brought in $1M), it’s time to think about how to scale. And when you’re looking to scale a trash can, the best you can get is a big trash can.

To put it more concretely, when technical debt is not managed, it turns into technical bankruptcy. The product becomes unmanageable. The engineering team is running at a slower pace and development costs are exploding. And of course, the quality of the product is poor.

However, the solution seems extremely simple to me. All you have to do is make engineers aware of marketing and entrepreneurship.

Explain to them that most ideas that sound good don’t survive the market. They will understand the value of quickly confronting an idea with the market. Explain to them that a feature that doesn’t meet a real customer need, brings little or no value to the product.

Most of them won’t become marketing geniuses. But they will become super-autonomous engineers. Then, just give them the main directions. For example, “the customer would like this or that”. Or “it would be cool to test this idea”. And then, all you have to do is leave them alone.

And miracles will happen.