AI for the 21st Century

What is recursive learning and what are the types, trends and roadblocks? That’s exactly what I’ll explain below. We don’t need to fear recursive learning, but we do need caution.

How Recursive Learning Works

To start, let’s define recursive learning:

Definition: Recursive learning in AI refers to a system’s ability to learn from its own outputs, without anyone stepping in. The AI becomes autonomous and it takes previously run tasks as new training data and refines its own capabilities.

The process flows like this:

• Execute – AI performs a task or generates a solution
• Evaluate – The output from the execution is analyzed for quality and success
• Refine – The evaluation is then fed back to the AI to modify its rules, update its context or even generate newer and better versions of itself.

Types of Recursive Learning

Let’s explore the different types of recursive learning for AI.

• Recursive Self Improvement (RSI) – The intent is to have AI completely analyze, rewrite and optimize its own programming. The result would be a fully self autonomous, self-improving AI model.
• Iterative Prompting and Search – In conversational AI, the model autonomously questions, evaluates and refines its own search criteria. The model queries data, synthesizes what it finds, identify gaps and uses that to query again until it reaches a highly accurate answer.
• Self Play – This approach is used heavily in reinforcement learning where the AI plays millions of games against itself. Every win or loss is recursively fed into the model to continuously evolve without a person intervening,
• Recursive Synthetic Data Generation – As high quality data sets begin to run out, this technique generates complex math, coding or logic problems, filters out the wrong answers and then uses this corrected AI generated data to train the next generation of the model.

Trends

The recursive learning technique is used to achieve something called Artificial General Intelligence or AGI for short. Unlike today, an AGI model would understand, learn and apply knowledge across any intellectual tasks. It would also have the ability to adapt to the unfamiliar without needing task specific programming.

Some of the worlds brightest are working towards this.

• 2026 – 2029: Dario Amodei (CEO of Anthropic) and Elon Musk thin it could be here this year! Similar predictions forecast a timeline near 2027.
• 2030’s: Demis Hassabis (CEO of Google DeepMind) point towards the 2030’s stating there are still hurdles to overcome in areas such as memory and reasoning.
• 2040’s: Yann LeCun at Meta believes we’re still decades away. Some give a 50% probability of reaching this level in the late 2040’s.

Whichever timeline you choose, the trend for recursive learning is certainly trending up

Roadblocks

Like most things in the technical pipeline, there are roadblocks that need to be overcome.

1. The Ouroboros problem – This is when the data generated by the AI (synthetic data) without careful filtering can cause model collapse. Errors, bias and statistical anomalies can compound over time. For recursive learning to work, the synthetic data generated must be better that the current, not just regurgitating some shallow subset of what it already knows.
2. Absence of reliable ground truth evaluators – In general reasoning evaluating superiority is incredibly difficult. If an AI is already at the height of human capability, it cannot easily verify the newly generated data is brilliant or a very convincing hoax, sometimes called a hallucination or divergent output.
3. Contextual and memory drifts – AI doesn’t have an immune system. When the AI recursively patches its own code over weeks and months, small errors in context can occur. Without an immune system, the AI cannot fight against those errors. There’s no automatic audit or rollbacks. When this happens, newer models inherits these errors and they can eventually derail the architecture.

Future Direction

With all the breakthroughs in AI, there are actually other tools and fields of study that can get us closer to achieving recursive learning in AI. First, there is the new field of study under AI called Mechanistic Interpretability. This new field of study focuses on the neural-net and employs tools such as the sparse autoencoders. You can read more in my article titled Mechanistic Interpretablity and Sparse Autoencoders.

Next, we add in another training method called Adversarial Training. You can review my article called Adversarial Training – The First Line of AI Defense.

Our last new piece addresses processing power. This is another newer piece of technology that combines light and silicon to produce a hybrid photonic/silicon chip. You can read about that in my article called Computer Chip Manufacturing.

Let’s identify our new pieces.

• Adversarial Training – Here the edge cases are generated to stress the system.
• Sparse Autoencoders – Identifies, measures and map the internal concepts
• Hybrid Photonic Chips – Running an adversarial loop while also mapping everything through sparse autoencoders is a huge computational hit and can take literally weeks of hyper-expensive GPU (graphics processing unit) cluster time.

Time to put this all together in one possible future implementation.

1. The Automated Audit (Offline Continuous Improvement) – Before a new recursive patch is accepted, an adversarial generator attacks it with trillions of edge cases. The sparse autoencoders monitor the internal activation switches. Should a gender bias or manipulative logic starts to form under adversarial stress, it flags the specific weight layer and triggers an automated rewrite.
2. The Real-Time Shunt (Live Production Safeguard) – Once deployed, should a novel adversarial attack occur trying to trick the model into being prejudice, the photonic sparse autoencoder detects the hidden intent in real-time. Here the hardware actually acts a circuit breaker and suppresses the bias instantly.

Summary

There’s a lot to unpack when learning the concept of recursive learning by itself. It becomes even more complicated when we factor newer tools that can aid in achieving this. Recursive learning is something to be feared, rather I think we should embrace it. Having said that, it will take a diverse team of experts to get this to work and caution needs to be applied every step of the way.