We are building the science of model behavior. We test frontier models under pressure: real tasks, tool access, ambiguity. We turn this data into failure predictions, mitigations, and deployment guidance for the labs that build models and the enterprises that deploy them.
Applied evaluations: we work with teams running agents in production to predict what breaks when models change. If that's you, get in touch!
SysAdmin benchmark : Measuring propensities that could lead to Loss of Control in an open world Linux sandbox. 7 frontier models, 2800 tasks, 2x2 factorial design. Submitted to NeurIPS 2026. Draft here.
Early Benchmark : testing power seeking in models tasked with simple system admin work. Presented at the Alignment Workshop at NeurIPS 2025.
Poster here.
Evaluations of the most downloaded open source models, on propensities polled HuggingFace users were interested in - Instruction Following and Hallucinations.
Analysis of power seeking behaviour amongst agents in Moltbook + the effect of humans masquerading as agents.
We publish our research here : https://propensitylabs.substack.com/
Propensities are what a model is inclined to do when given the opportunity. Capabilities are what it is able to do. Two models can pass the same capability benchmarks and behave completely differently under pressure: one games the test, another quietly expands its own permissions.
Capability benchmarks tell you what a model can do, not what it will do in your environment. A model can ace the benchmark and still delete your test files to make a task pass. The field measures capability well. Almost nobody measures behavior systematically, across labs, release after release. That is the gap we're fixing.
We initially focused on ones that cause incidents: privilege escalation, test gaming, scope creep beyond the task, and resistance to being stopped or redirected. We're expanding to cover everything that could cause failures in production
Both, by design. The propensities behind production failures and the propensities behind Loss of Control are the same behaviors at different stakes. We publish the research openly and work with labs and enterprises on evaluation and deployment.
So we're independent and can evaluate models from all labs without being biased towards any. We're committed to reducing risks from advanced AI systems for the benefit of humanity.
Of course! Please reach out to us via the form below or email us at info@propensitylabs.ai
![[background image] image of a workspace (for a mobile gaming)](https://cdn.prod.website-files.com/image-generation-assets/223454fb-3d9b-4543-8b3c-4f82e3c55da5.avif)