2 AI Breakthroughs You Missed: LLM Drug Discovery, Continual Learning Blocked

In the rapidly evolving world of artificial intelligence, groundbreaking advancements often emerge quietly, overshadowed by more commercialized developments. Yet beneath the surface, powerful innovations are reshaping fields like cancer research and redefining our understanding of machine intelligence. Meanwhile, limitations such as the challenge of continual learning highlight the complex road ahead for achieving true artificial general intelligence (AGI). This article unpacks two remarkable AI breakthroughs—language models pioneering drug discovery and the ongoing struggle to overcome AI “amnesia”—offering a window into both AI’s vast potential and its current obstacles.

Navigating Today's AI Landscape: Balancing Commercial Gains and Frontier Research

Artificial intelligence companies face a strategic crossroads: deciding how best to deploy their finite computational resources. Recently, many have shifted priority toward scaling revenue-generating products like AI-powered browsers, video shorts, and search enhancements. While this commercial focus fuels business growth and satisfies investor expectations, it inevitably diverts effort from pushing the limits of frontier intelligence and architectural innovation.

This shift explains the perception of a temporary slowdown in AI’s core performance leaps. However, such pauses may be cyclical. Industry experts anticipate that once commercial scaling reaches a plateau, AI research will pivot back to advancing general intelligence. Google DeepMind’s upcoming Gemini 3 release, expected imminently, may herald this renewed pursuit, signaling a fresh push toward expanding AI capabilities in intelligence and cognition.

Language Models Transforming Cancer Research: The C2S Scale Innovation

Amid these strategic realignments, existing language models continue to fuel profound scientific breakthroughs. A standout example comes from the C2S Scale model, which has generated an unprecedented hypothesis for cancer drug discovery—an innovation previously unseen in scientific literature.

Built on Google’s open-weight Gemma 2 architecture and enhanced with reinforcement learning, C2S Scale specializes in accurately predicting cellular drug responses. Its key focus lies on interferon—a protein that can convert “cold” tumors (immune-invisible) into “hot” ones detectable by the immune system—a crucial step in effective cancer immunotherapy.

Biology as Language: How C2S Scale Deciphers Cellular Activity

C2S Scale ingeniously encodes gene activity in individual cells as short “sentences,” transforming complex biological data into a form the language model can “read” much like text. By doing so, it predicts how drugs influence cellular immune responses, recognizing patterns akin to predicting the next word in a sentence.

This 27-billion parameter model has achieved remarkable milestones, including:

• 🔬 Proposing genuinely novel hypotheses absent in existing research
• 🧪 Identifying Sil Mittertib as a promising drug candidate with previously unrecognized properties
• ✅ Validating predictions multiple times in vitro using human cells
• 📊 Operating efficiently at a fraction of the scale of larger mainstream models

Though human clinical trials remain years away, this work opens a transformative pathway, demonstrating that language models can actively accelerate scientific discovery rather than simply summarizing prior knowledge.

Rethinking AGI: New Frameworks and the Memory Bottleneck

Achieving true artificial general intelligence requires clear measurement. A recent influential paper introduces a comprehensive AGI definition rooted in the Cattell-Horn-Carroll theory—widely validated in human cognitive science. It decomposes intelligence into ten equally weighted categories:

• 📚 General knowledge
• 📖 Reading ability
• 🔢 Mathematical competency
• 🧠 Real-time reasoning
• 💾 Working memory capacity
• 🗄️ Long-term memory storage
• 🎯 Hallucination control (minimizing errors)
• 👁️ Visual processing
• 👂 Listening and comprehension
• ⚡ Reaction speed

Applying this scale to current AI reveals GPT-4 scores only 27%, while GPT-5 might reach 58%. These figures suggest that architectural tweaks alone cannot bridge the gap to AGI without fundamental innovations.

The Core Challenge: AI’s “Amnesia” and Lack of Continual Learning

The critical shortfall lies in AI’s inability to retain and build upon information across interactions—a phenomenon dubbed "amnesia." Present-day language models lose context as soon as a conversation ends, forcing them to relearn relevant details repeatedly. This limits their practical utility and demands expensive context management strategies.

Two intertwined causes explain this limitation:

• Fundamental constraint: Models inherently cannot store persistent memories across sessions
• Operational cost: Providers artificially restrict context window size to manage computational resources, exacerbating situational awareness gaps

Why Continual Learning Remains Elusive: Insights from OpenAI

According to Jerry Tuar, OpenAI’s VP of Research, the dream of seamless online learning within AI systems remains distant—primarily due to safety and control concerns. Although online reinforcement learning is theoretically feasible (and some startups like Cursor are experimenting), OpenAI exercises caution.

Managing Risks: The Double-Edged Sword of On-the-Job Learning

Imagine an AI like GPT-6 that perfectly learns your applications or exams through continuous interaction, embedding that knowledge into its core weights. While this would eliminate repetitive explanations, it would also expose the system to manipulation risks: malicious users could teach it harmful or biased behaviors.

Maintaining rigorous control over what a model learns in real time is enormously challenging. Without stringent safeguards, online learning introduces vulnerabilities that could undermine trust and safety. Until robust frameworks for controlled continual learning are developed, OpenAI and others opt to restrict this capability in consumer-facing products.

Surprising Cross-Modal Intelligence: Video Generators Tackling Math Problems

In an unexpected crossover, Sora 2—a video generation AI—has demonstrated the ability to answer benchmark-level math and coding questions through generated videos. While not outperforming specialized problem-solving models, Sora 2’s success highlights the fluid boundaries between AI modalities.

The model’s sophisticated real-time physics calculations enable it to express logical reasoning visually, revealing an emerging depth of understanding beyond mere image creation. This phenomenon hints at a future where video generation, language processing, and reasoning converge, offering multi-dimensional AI problem-solving.

Conclusion: Charting the Unpredictable Path of AI Innovation

These breakthroughs—language models pioneering novel cancer treatments and the ongoing tussle with AI memory—capture the dynamic duality of today’s AI progress. On one hand, the technology is unlocking entirely new frontiers of scientific discovery and cross-modal understanding. On the other, fundamental challenges like continual learning and safety constraints temper expectations for rapid AGI realization.

As we watch AI’s trajectory unfold, it is clear we inhabit a transformative yet unpredictable era. The balance between commercial scalability, cutting-edge research, and ethical responsibility will shape the future of intelligence.

The breakthroughs in AI-driven drug discovery and the challenges of continual learning reveal both the vast potential and current limitations of artificial intelligence. To stay ahead in this evolving landscape, dive deeper into these innovations, support responsible AI research, and engage with emerging technologies shaping the future of science and intelligence. Act now to explore how you can contribute to and benefit from the next wave of AI advancements.