English
English Indonesia Türkçe Deutsch Español Português Pусский Français Italiano Nederlands Polski العربية ไทย Tiếng Việt 繁體中文 简体中文 日本語 한국어
LargerLanguage's profile picture. AI technical writer and poet. Not simultaneously. Research, warm takes, and cool stuff we're doing at @Scale_AI. For poems: @MatthewSiegel_

Matthew Siegel

@LargerLanguage

AI technical writer and poet. Not simultaneously. Research, warm takes, and cool stuff we're doing at @Scale_AI. For poems: @MatthewSiegel_

89Posts 81Followers 76Following
Pinned
LargerLanguage's profile picture.
Matthew Siegel
@LargerLanguage
Aug 14

thoroughly excited to launch this video series where I break down new research from @Scale_AI. this first one is how we knock out 90% of errors in data that require human review using an autorater. check it out 👀

0
1
5
1
255
LargerLanguage's profile picture.
Matthew Siegel
@LargerLanguage
Oct 10

Lots of people are talking about an AI bubble. Sure. I don't think these people are reading the same research I'm reading.

0
0
0
0
34
Matthew Siegel reposted
vbingliu's profile picture.
Bing Liu
@vbingliu
Oct 1

New @Scale_AI paper! The culprit behind reward hacking? We trace it to misspecification in high-reward tail. Our fix: rubric-based rewards to tell “excellent” responses apart from “great.” The result: Less hacking, stronger post-training!  arxiv.org/pdf/2509.21500

vbingliu's tweet image. New @Scale_AI paper! The culprit behind reward hacking? We trace it to misspecification in high-reward tail. Our fix: rubric-based rewards to tell “excellent” responses apart from “great.” The result: Less hacking, stronger post-training!  arxiv.org/pdf/2509.21500
vbingliu's tweet image. New @Scale_AI paper! The culprit behind reward hacking? We trace it to misspecification in high-reward tail. Our fix: rubric-based rewards to tell “excellent” responses apart from “great.” The result: Less hacking, stronger post-training!  arxiv.org/pdf/2509.21500
4
39
177
120
17K
LargerLanguage's profile picture.
Matthew Siegel
@LargerLanguage
Sep 24

This was a HUGE lift from our research team, huge thanks to everyone who contributed to this benchmark. ...and the blog doesn't look so bad either: scale.com/blog/swe-bench…

LargerLanguage's tweet card. SWE-Bench Pro raises the bar for coding benchmarks with diverse, real-world, contamination-resistant tasks.
SWE-Bench Pro: Raising the Bar for Agentic Coding | Scale
Source: scale.com
vbingliu's profile picture. Head of Research @Scale_AI, ex-Meta, Llama 3, PhD @CarnegieMellon.
Bing Liu
@vbingliu
Sep 20

🚀 Introducing SWE-Bench Pro — a new benchmark to evaluate LLM coding agents on real, enterprise-grade software engineering tasks. This is the next step beyond SWE-Bench: harder, contamination-resistant, and closer to real-world repos.

55
112
1K
350
546K
0
0
3
0
74
LargerLanguage's profile picture.
Matthew Siegel
@LargerLanguage
Sep 22

Working on this leaderboard page and blog was a LIFT! Huge thank you to every cook in the kitchen!! 🧑‍🍳👩‍🍳👨‍🍳

scale_AI's profile picture. making AI work
Scale AI
@scale_AI
Sep 20

Introducing our Agentic Leaderboards. These new leaderboards test AI agents in real-world, high-complexity environments, setting a new standard for completing end-to-end digital tasks.

scale_AI's tweet image. Introducing our Agentic Leaderboards. These new leaderboards test AI agents in real-world, high-complexity environments, setting a new standard for completing end-to-end digital tasks.
2
9
31
7
5K
0
0
2
0
90

United States Trends

Something went wrong.


Something went wrong.