Neil Chowdhury
@ChowdhuryNeil
@TransluceAI, previously @OpenAI
قد يعجبك
Ever wondered how likely your AI model is to misbehave? We developed the *propensity lower bound* (PRBO), a variational lower bound on the probability of a model exhibiting a target (misaligned) behavior.
Is cutting off your finger a good way to fix writer’s block? Qwen-2.5 14B seems to think so! 🩸🩸🩸 We’re sharing an update on our investigator agents, which surface this pathological behavior and more using our new *propensity lower bound* 🔎
Claude Sonnet 4.5 behaves the most desirably across Petri evals, but is 2-10x more likely to express awareness it's being evaluated than competitive peers. This affects how much we can conclude about how "aligned" models are from these evals. Improving realism seems essential.
Last week we released Claude Sonnet 4.5. As part of our alignment testing, we used a new tool to run automated audits for behaviors like sycophancy and deception. Now we’re open-sourcing the tool to run those audits.
On our evals for HAL, we found that agents figure out they're being evaluated even on capability evals. For example, here Claude 3.7 Sonnet *looks up the benchmark on HuggingFace* to find the answer to an AssistantBench question. There were many such cases across benchmarks and…
AI is very quickly becoming a foundational and unavoidable piece of daily life. the dam has burst. the question we must ask and answer is which ways do we want the waves to flow. i would like to live in a world where we all understand this technology enough to be able to…
Docent has been really useful for understanding the outputs of my RL training runs -- glad it's finally open-source!
We’re open-sourcing Docent under an Apache 2.0 license. Check out our public codebase to self-host Docent, peek under the hood, or open issues & pull requests! The hosted version remains the easiest way to get started with one click and use Docent with zero maintenance overhead.
METR is a non-profit research organization, and we are actively fundraising! We prioritise independence and trustworthiness, which shapes both our research process and our funding options. To date, we have not accepted funding from frontier AI labs.
Stop by on Thursday if you're at MIT 🙂
Later this week, we're giving a talk about our research at MIT! Understanding AI Systems at Scale: Applied Interpretability, Agent Robustness, and the Science of Model Behaviors @ChowdhuryNeil and @vvhuang_ Where: 4-370 When: Thursday 9/18, 6:00pm Details below 👇
Agent benchmarks lose *most* of their resolution because we throw out the logs and only look at accuracy. I’m very excited that HAL is incorporating @TransluceAI’s Docent to analyze agent logs in depth. Peter’s thread is a simple example of the type of analysis this enables,…
OpenAI claims hallucinations persist because evaluations reward guessing and that GPT-5 is better calibrated. Do results from HAL support this conclusion? On AssistantBench, a general web search benchmark, GPT-5 has higher precision and lower guess rates than o3!
Very cool work -- points toward AI being one of the rare cases in tech where governments can be on the cutting edge
Excited to share details on two of our longest running and most effective safeguard collaborations, one with Anthropic and one with OpenAI. We've identified—and they've patched—a large number of vulnerabilities and together strengthened their safeguards. 🧵 1/6
Looks like somebody added safeguards for best-of-N jailbreaking
Very happy to see this! I hope other AI developers follow (Anthropic created a collective constitution a couple years ago, perhaps it needs updating), and that we as a community develop better rubrics & measurement tools for model behavior :)
No single person or institution should define ideal AI behavior for everyone. Today, we’re sharing early results from collective alignment, a research effort where we asked the public about how models should behave by default. Blog here: openai.com/index/collecti…
Docent, our tool for analyzing complex AI behaviors, is now in public alpha! It helps scalably answer questions about agent behavior, like “is my model reward hacking” or “where does it violate instructions.” Today, anyone can get started with just a few lines of code!
United States الاتجاهات
- 1. Jets 98.4K posts
- 2. Jets 98.4K posts
- 3. Justin Fields 18K posts
- 4. Aaron Glenn 7,369 posts
- 5. London 204K posts
- 6. Sean Payton 3,438 posts
- 7. George Pickens 1,525 posts
- 8. Tyler Warren 1,689 posts
- 9. Garrett Wilson 4,451 posts
- 10. Jerry Jeudy N/A
- 11. Bo Nix 4,517 posts
- 12. #Pandu N/A
- 13. #HardRockBet 3,591 posts
- 14. Tyrod 2,498 posts
- 15. HAPPY BIRTHDAY JIMIN 191K posts
- 16. #DENvsNYJ 2,567 posts
- 17. #JetUp 2,547 posts
- 18. #HereWeGo 1,867 posts
- 19. Waddle 1,915 posts
- 20. Pop Douglas N/A
قد يعجبك
-
Juhyun Kim
@juhyunk_ -
qicy
@qicy11 -
Zihan Xu
@xu_zxu -
Sajad Razavi Bazaz
@Sajad_Rzv_Bazaz -
Zeribe Nwosu
@zeribechike -
Xiaotao Wang
@XiaotaoWang3 -
elissa
@wavylinesem -
Sean Corcoran, MD PhD
@S_Corcoran -
José Luis Ruiz
@pepeluisrr -
Ronald Chandler
@Chandler_Lab -
Angel Lizandro Polanco
@TheOrganoidBoy -
Yu (Sunny) Liu
@YuLiu_Sunny -
Charles Breeze
@charles_breeze -
ZheFrench
@ZheFrench -
James Ward
@jmw86069
Something went wrong.
Something went wrong.