The secret history of OpenAI

Since GPT 4 included, OpenAI announces a new revolution before each of their releases. But when you are in close contact with the world of open source AI, you understand this narrative quite differently. And if you’re a little clever, you’ve probably realized that it’s hot air. This is what I would like to detail here.

When ChatGPT was released on November 30, 2022, OpenAI did explain that it was going to release GPT 4 soon, and then quickly GPT 5. And two years later, GPT 5 is still not released. Why?

The origin of LLMs

To explain what happened, I have to go back to the invention of transformers. They were invented by Google for machine translation. The original transformers architecture was based on two interconnected neural networks. One part encoder and one part decoder. The decoder part transforms a text into a vector. This vector represents the semantics of the text. And the decoder part made it possible to convert a vector back into natural language, in the language of our choice.

The models we use today for generative AI are based on the decoder part of the transformers. As input we use a tokenizer to translate the text into numbers. And with the right training, the model is able to complete a text, for example, respond to instructions.

The magic happens

This feature was not initially planned at all. This is an emerging feature that was later discovered. Another surprising fact is that the more the number of parameters is increased (from GPT2 to GPT3), the more the model is capable of reasoning.

Following this logic, OpenAI thought that they would achieve AGI (Artificial general intelligence) by increasing the number of parameters (GPT4, then GPT5).

The lie about AGI

But this phenomenon very quickly reached its limits. The cost/benefit ratio of GPT4 is not great. And OpenAI realized that they wouldn’t achieve AGI with this transformer-based strategy.

In reality, having some knowledge of how intelligence works, this was predictable. I served for a year in the French Air Force. I was working in a psychological research center on the computerization of intelligence tests and on associated statistical models.

The nature of intelligence

And with my little knowledge on the subject, I know that intelligence is multi-factorial. With text, we focus mainly on verbal intelligence. That’s why the ability of an LLM to do mathematics is mediocre. They struggle with calculation, abstract reasoning…

With some knowledge of intelligence, it is obvious that AGI, if it ever exists, will not come from an LLM.

Lagging behind open source

With the release of LLaMA in February 2023 and the excitement that followed, even though OpenAI’s communication seeks to put them in a leadership position, they are only following open source.

Even if it is announced as a revolution, the release of GPTs in November 2023 simply takes up the principle of RAG (Retrieval Augmented Generation ) that has been in vogue in the open source world in months.

When they say that they have evolved their model to be faster and consume less energy, they have just implemented quantization. They say that now, the results are better, but all the benchmarkers see the opposite.

Even though AI outsiders raved when Sora was announced in February 2024, it was just a remake of Stable Video. And while Sora has still not been released, many open source alternatives have appeared since then (for example, Pyramid Flow).

GTP-4o in May 2024 implements the CoT (Chain of Thought) that has been popular in open source for more than a year. It must be said that it is a little more complex to implement than a RAG and that it took them time.

Conclusion

Today, OpenAI’s communication claims that AGI will arrive in 2025. But in reality, we are far from that and finally, this race for AGI and this communication war prevents us from seeing the true power of the LLM.

Like existing IT tools, LLMs are not going to replace us. But they could have given us superpowers. Excell has not replaced the accountants. In the same way, assistant coders are sometimes good for completion or for interactively querying documentation. But they are incapable of software engineering.

In all this confusion, many people are looking to create tools to replace us, with mediocre results. As a matter of principle, LLMs lack autonomy and need a human to lead them. To take advantage of the full potential of LLMs, it is more effective to create tools that strengthen our capabilities rather than dreaming of an autonomous AGI.

Disclaimer: I have no vision from the inside. Everything I’m describing here is my interpretation, according to the facts.