All products
The Elusive Goal of AGI: Understanding the Challenges of Controlling Language Models
When Altman celebrates finally getting GPT to avoid em dashes, he’s really celebrating that OpenAI has tuned the latest version of GPT-5.1 (probably through reinforcement learning or fine-tuning) to weight custom instructions more heavily in its probability calculations. This achievement may seem minor, but it highlights the significant challenges of controlling language models and the long road ahead for achieving Artificial General Intelligence (AGI).
The irony of control lies in the probabilistic nature of language models. Despite the latest updates, there’s no guarantee that the issue will stay fixed. OpenAI continuously updates its models behind the scenes, even within the same version number, adjusting outputs based on user feedback and new training runs. Each update arrives with different output characteristics that can undo previous behavioral tuning, a phenomenon researchers call the “alignment tax.”
The Complexity of Neural Networks
Precisely tuning a neural network’s behavior is not yet an exact science. Since all concepts encoded in the network are interconnected by values called weights, adjusting one behavior can alter others in unintended ways. Fix em dash overuse today, and tomorrow’s update (aimed at improving, say, coding capabilities) might inadvertently bring them back, not because OpenAI wants them there, but because that’s the nature of trying to steer a statistical system with millions of competing influences.
This complexity raises questions about the feasibility of achieving AGI, which would require true understanding and self-reflective intentional action, not statistical pattern matching that sometimes aligns with instructions if you happen to get lucky. As researchers continue to push the boundaries of language models, it becomes increasingly clear that AGI may not emerge from a large language model alone.
The Limits of Current Technology
Some users still aren’t having luck with controlling em dash use outside of the “custom instructions” feature. Upon being told in-chat to not use em dashes within a chat, ChatGPT updated a saved memory and replied to one X user, “Got it—I’ll stick strictly to short hyphens from now on.” While this may seem like a minor victory, it highlights the limitations of current technology and the need for continued research and development.
As we strive to create more advanced language models, it’s essential to acknowledge the challenges and complexities involved. By understanding the intricacies of neural networks and the probabilistic nature of language models, we can better appreciate the significant progress being made and the long road ahead for achieving AGI. For more information on this topic, visit Here
Image Credit: arstechnica.com