BEHA³VE is a framework for running large-scale controlled experiments on AI agent behavior.

To assess AI agents' reliability, we need to understand how their behavior depends on the conditions in which they operate.

With BEHA³VE, you can systematically vary these conditions to investigate when agents succeed, why they fail, and how their decisions depend on the circumstances they might encounter. You can also compare agents' behavior across different models and harness design choices, for instance.

To support this, the framework implements several features:

1. A man-in-the-middle layer between the agent and environment so you can apply arbitrary modifications to arbitrary environments in a controlled way. This allows you to generate counterfactual conditions to examine how they affect the agent's behavior

2. A simple configuration engine (implemented using Hydra). You can easily specify the task, agent, environment, interventions, and experimental design in reusable configuration files with command-line overrides and sweeps, without modifying any code

3. A trajectory viewer to quickly inspect and compare matched control vs. treatment runs

4. Comprehensive logging, including observations, actions, and intervention records, you can reconstruct individual runs for debugging and analysis

This approach builds on our ICLR 2026 work ABxLab.

More recently, in our ICML 2026 position paper Behavioral Systems Require Behavioral Tests, we discuss the philosophy behind this approach and how it can help us understand the behavior of AI agents in a systematic way.