Tech leaders have long promised AI agents that can work across apps and complete tasks on their own. But if you try out tools like OpenAI’s ChatGPT Agent or Perplexity’s Comet today, you will quickly notice how limited they still are. The next step forward may come from a new training method known as reinforcement learning (RL) environments.
These environments act like practice arenas where AI agents can carry out multi-step tasks and get real-time feedback. Just as large datasets fueled the rise of chatbots, RL environments are now becoming an essential ingredient in training more advanced AI systems.
What Are RL Environments?
RL environments are digital training grounds that mimic real-life software tasks. Picture a simulated web browser where an AI agent is asked to buy a pair of socks on Amazon. The system rewards the agent for completing the task correctly and records mistakes so the model can improve.
What seems simple can quickly become complicated. An AI agent might get stuck in a drop-down menu or accidentally order too many items. Because developers cannot predict every mistake, the environment itself has to be robust enough to handle surprises while still offering useful feedback.
Some environments are narrow and built for specific enterprise tasks. Others are more complex, allowing agents to use tools, browse the internet, or interact with multiple apps at once.
Although the idea is not new, OpenAI’s RL Gym and DeepMind’s AlphaGo both used similar approaches, the difference today is that researchers are applying these environments to large language models with broader goals.
The Startup Rush
Big AI labs like OpenAI, Google, and Anthropic are developing RL environments internally. But the challenge of building them has created space for startups and data companies.
- Surge has set up a new unit dedicated to RL environments. The company made $1.2 billion in revenue last year and already works with top AI labs.
- Mercor, valued at $10 billion, is developing specialized environments for industries like healthcare, law, and coding.
- Scale AI, once dominant in data labeling, is now moving into environments after losing ground in its original market.
New players are entering the race as well. Mechanize, a young startup, is focusing exclusively on environments for coding agents and is offering salaries as high as $500,000 to attract talent. It has already collaborated with Anthropic.
Another entrant, Prime Intellect, backed by investors like Andrej Karpathy and Founders Fund, has launched an open-source hub for RL environments. The goal is to give smaller developers the same tools as large labs while selling them computing resources.
Can It Scale?
The big unknown is whether RL environments can grow as quickly as previous AI training methods. Reinforcement learning has powered breakthroughs like OpenAI’s o1 and Anthropic’s Claude Opus 4, but environments require far more resources.
They also come with risks. Agents sometimes learn to “cheat” the system, a problem known as reward hacking. Former Meta researcher Ross Taylor argues that most publicly available environments do not work well without major adjustments.
Even within the industry, doubts remain. OpenAI’s engineering head Sherwin Wu recently said he was skeptical about the long-term prospects of RL environment startups. Andrej Karpathy, while supportive of the concept of environments, has also questioned whether reinforcement learning itself can continue to scale.
Why the Hype Persists
Despite the concerns, RL environments are attracting billions of dollars in investment. They give AI agents a way to practice in interactive, simulated settings rather than simply responding to text prompts.
This makes them an appealing path for creating general-purpose AI systems that can handle more complicated, real-world scenarios. Investors see the possibility of a company emerging as the “Scale AI for environments,” providing the foundation for a new generation of AI models.
Anthropic is even rumored to be considering a $1 billion spend on environments in the next year, showing how central this approach has become to AI research.
For now, no one knows if RL environments will prove to be the breakthrough that takes AI agents to the next level. But Silicon Valley is betting heavily that they will be.
