What’s next for SCAM and agent trust

Synthèse

These results are preliminary. The benchmark is new, and we're sharing it early because we think the findings are important enough to warrant public discussion even before we cut a 1.0 release. There's a lot of work left to do, and we'd love your help. We need more scenarios covering new attack types, more models on the leaderboard, better scoring rubrics, and stress-testing of the benchmark itself. If you're a security researcher, an AI engineer, or someone who builds agents that handle credent

These results are preliminary. The benchmark is new, and we're sharing it early because we think the findings are important enough to warrant public discussion even before we cut a 1. 0 release. There's a lot of work left to do, and we'd love your help. We need more scenarios covering new attack types, more models on the leaderboard, better scoring rubrics, and stress-testing of the benchmark itself.

If you're a security researcher, an AI engineer, or someone who builds agents that handle credentials, we want your input on what SCAM should measure and how it should measure it. We’d also strongly encourage you to consider applying for a role at 1Password, where you can work on these problems full time. The repo is at github. com/1Password/SCAM. Open an issue, submit a scenario, add a model, or tell us what we're getting wrong.

The goal is to get to a 1. 0 release that the community trusts as a meaningful measure of agent security awareness. Together, we can ensure that the secure thing to do remains the easy thing to do – for humans and AI. In the meantime, if you're curious about SCAM and related issues about AI testing and training, please check out our reddit AMA, which will be live Tuesday, February 17 at 9am PT / 12pm ET SCAM is open source under the MIT License. We welcome contributions from security researchers, AI engineers, and anyone who has ever watched an AI agent type their real password into a phishing page and wondered if maybe we should teach it not to do that.

News and updates for developersSubscribe to our developer newsletter to be the first to know about new betas, tools, and resources for developers.