The stupidity of the AI trend

There are two main types of reactions to the AI hype, with progressives on one hand who see AI in a positive light. And on the other hand, the conservatives who think that either it will lead nowhere, or it will cause the apocalypse.

If you follow my content, you’re probably perplexed, because on the one hand I’m debunking the AI hype. But at the same time, I’m promoting this technology. This is simply because the reality of AI and the hype about AI have little to do with each other.

I’ve been studying AI and machine learning for 30 years. And the biggest revolution, the one that creates value, did not take place with the release of ChatGPT. It took place in 2000, when Google started offering contextual advertising based on semantic algorithms. But no one noticed.

If you listen to the lemmings who follow the AI hype, they will tell you that at that time, it was the winter of AI. This is because they confuse AI and machine learning with generative AI. Generative AI did not create AI. It just made it sexy and now everyone is interested in it.

Now we have machines that speak our language and everyone is becoming aware that AI exists. But AI was already here, everywhere. Social networks tend to lock us into algorithmic spheres, but no one realizes it. For the past ten years, DeepMind has been creating revolutions, in the field of strategy games, but also in the more serious fields of medicine and chemistry. But it’s not very fun and people prefer to be interested in OpenAI.

This AI trend is based on transformers, a new type of neural networks created in 2017 by Google for machine translation. This neural network architecture was organized in two parts. On one hand, an encoder network that transforms the text into a vector that represents its meaning. On the other hand, a decoder network that takes this vector as input and generates text as output, in another language. AI chatbots use the decoder part of a transformer network.

One of the problems that arises right now is that people who have discovered AI with ChatGPT think that AI is all about it and would want to do everything with this tool, even when it is not suitable. For example, it is very complicated to develop a specific chatbot on a particular topic, without hallucination. It is much easier to develop a semantic search engine based on classic NLP algorithms. And the result is much more reliable. But no one is interested in that. Everyone wants their AI chatbot.

And beyond that, there is the hidden side of AI. There are those who really know the subject and who set up reliable solutions with the right tools.

I was talking earlier about transformers. Even if the hype is around decoders, most of the companies that create products around this tech lose money. Besides that, in the shadows, many companies use encoders for embedding and create some value with it. But almost no one knows.

And while the AI giants (with feet of clay) are spending more and more money to create bigger and bigger models, for a tiny performance gain, some companies still use gpt2 successfully for very specific needs.

Generalist chatbots reached their peak 2 years ago. Since then, it’s mostly a lot of misleading marketing, and the improvements are minimal. But the future of LLMs lies in small, lightweight models specifically trained for a particular task. There are still endless things to explore. But most AI companies today are just wrappers of OpenAI, Anthropic, and others… And outside the open source world, fine tuning is still very little used. People dream of a generalist model that can do everything without specific training.

However, I think it’s not a good idea to use gpt2. It is preferable to use same size models, in the state of the art, rather than gpt2. Even if gpt2 is a little easier to implement, with RoPE (Rotary Position Embedding), flash attention, and RMSNorm (Root Mean Square Normalization), for an equivalent size, we get much more efficient models than gpt2. And it’s not that complicated in the end.

But that’s not really the point. Now that everyone knows that AI exists, it needs to be democratized. Really. This means knowing tools and methods, which have sometimes existed for ten or twenty years, which have completely gone under the radar. Last week, I posted an article about spam detection, which uses a statistics algorithm that is over two and a half centuries old and is super efficient.

For example, about ten years ago, I created an SEO tool that automatically creates the internal linking of a website based on the thematic proximity of the articles, using the LSI (Latent semantic indexing) algorithm. But this kind of tech, although simple to implement, has never become mainstream. In the SEO sphere at the time, everyone dreamed of semantic analysis. But no one has experienced it.

It’s time to democratize all this. LLMs are great, but they are not an universal tool. They are sometimes complex to set up for certain tasks, and inefficient in terms of accuracy or performance.

So, yes, I’m an AI enthusiast. But that doesn’t stop me from taking a critical look at the current trend. In the short term, the impact of AI is overestimated. But at the same time, we underestimate what we can do with today’s tech.