harshitj__'s profile picture. CS phd @StanfordNLP, @StanfordOVAL  | prev: @MSFTResearch | LLM systems for knowledge access, discovery and curation

Harshit Joshi

@harshitj__

CS phd @StanfordNLP, @StanfordOVAL | prev: @MSFTResearch | LLM systems for knowledge access, discovery and curation

Such a nicely written blogpost, even a non-interp person can understand why you should use circuits rather than using learned features.

Is your LM secretly an SAE? Most circuit-finding interpretability methods use learned features rather than raw activations, based on the belief that neurons do not cleanly decompose computation. In our new work, we show MLP neurons actually do support sparse, faithful circuits!

TransluceAI's tweet image. Is your LM secretly an SAE?

Most circuit-finding interpretability methods use learned features rather than raw activations, based on the belief that neurons do not cleanly decompose computation. In our new work, we show MLP neurons actually do support sparse, faithful circuits!


Harshit Joshi 已轉發

New eval! Code duels for LMs ⚔️ Current evals test LMs on *tasks*: "fix this bug," "write a test" But we code to achieve *goals*: maximize revenue, cut costs, win users Meet CodeClash: LMs compete via their codebases across multi-round tournaments to achieve high-level goals


had fun using burst!! also my new name is Sanjay!

Do you hold back from posting because the audience never feels right? Small groups feel safe but limiting. Public platforms feel risky and performative. Our #CSCW2025 paper introduces Burst: a design that connects private and public spaces. We found that posters felt safer and…

zhangyt0704's tweet image. Do you hold back from posting because the audience never feels right?
Small groups feel safe but limiting.
Public platforms feel risky and performative.

Our #CSCW2025 paper introduces Burst: a design that connects private and public spaces.
We found that posters felt safer and…


Harshit Joshi 已轉發

Interested in building and benchmarking deep research systems? Excited to introduce DeepScholar-Bench, a live benchmark for generative research synthesis, from our team at Stanford and Berkeley! 🏆Live Leaderboard guestrin-lab.github.io/deepscholar-le… 📚 Paper: arxiv.org/abs/2508.20033 🛠️…

lianapatel_'s tweet image. Interested in building and benchmarking deep research systems?

Excited to introduce DeepScholar-Bench, a live benchmark for generative research synthesis, from our team at Stanford and Berkeley!

🏆Live Leaderboard guestrin-lab.github.io/deepscholar-le…
📚 Paper: arxiv.org/abs/2508.20033
🛠️…

Harshit Joshi 已轉發

Introducing Generative Interfaces - a new paradigm beyond chatbots. We generate interfaces on the fly to better facilitate LLM interaction, so no more passive reading of long text blocks. Adaptive and Interactive: creates the form that best adapts to your goals and needs!


which model will first get to 10%? taking wagers

New paper! We explore a radical paradigm for AI evals: assessing LLMs on *unsolved* questions. Instead of contrived exams where progress ≠ value, we eval LLMs on organic, unsolved problems via reference-free LLM validation & community verification. LLMs solved ~10/500 so far:

kenziyuliu's tweet image. New paper! We explore a radical paradigm for AI evals: assessing LLMs on *unsolved* questions.

Instead of contrived exams where progress ≠ value, we eval LLMs on organic, unsolved problems via reference-free LLM validation & community verification. LLMs solved ~10/500 so far:


Harshit Joshi 已轉發

Can AI solve open problems in math, physics, coding, medical sciences & beyond? We collected unsolved questions (UQ) & tested frontier LLMs. Some solutions passed expert validation…

Muennighoff's tweet image. Can AI solve open problems in math, physics, coding, medical sciences & beyond?

We collected unsolved questions (UQ) & tested frontier LLMs. Some solutions passed expert validation…

it would be nice to have personal agents that can take care of mundane/complex works for us while interacting with other agents — but how much can we rely on them? how can I trust them to not share my personal details? GO CHECK OUT THIS CRAZY WORK!

Soon, AI agents will act for us—collaborating, negotiating, and sharing data. But can they truly protect our privacy? We simulate privacy-critical scenarios, using alternating search to evolve attacks and defenses, uncovering severe vulnerabilities and building protections.



nothing is constant in travels, except desi family kalesh


Harshit Joshi 已轉發

if you find this post interesting, please also check out AxBench, @aryaman2020 and I have a few more steering options to show. these are supervised methods, but automated with less than 50 LM generated examples to train.

ZhengxuanZenWu's tweet image. if you find this post interesting, please also check out AxBench, @aryaman2020 and I have a few more steering options to show. these are supervised methods, but automated with less than 50 LM generated examples to train.

New Anthropic research: Persona vectors. Language models sometimes go haywire and slip into weird and unsettling personas. Why? In a new paper, we find “persona vectors"—neural activity patterns controlling traits like evil, sycophancy, or hallucination.

AnthropicAI's tweet image. New Anthropic research: Persona vectors.

Language models sometimes go haywire and slip into weird and unsettling personas. Why? In a new paper, we find “persona vectors"—neural activity patterns controlling traits like evil, sycophancy, or hallucination.


Harshit Joshi 已轉發

.@stanfordnlp papers at @aclmeeting in Vienna next week: • HumT DumT: Measuring and controlling human-like language in LLMs @chengmyra1 @sunnyyuych @jurafsky • Controllable and Reliable Knowledge-Intensive Task Agents with Declarative GenieWorksheets @harshitj__ @ShichengGLiu

stanfordnlp's tweet image. .@stanfordnlp papers at @aclmeeting in Vienna next week:
• HumT DumT: Measuring and controlling human-like language in LLMs @chengmyra1 @sunnyyuych @jurafsky
• Controllable and Reliable Knowledge-Intensive Task Agents with Declarative GenieWorksheets
@harshitj__ @ShichengGLiu…
stanfordnlp's tweet image. .@stanfordnlp papers at @aclmeeting in Vienna next week:
• HumT DumT: Measuring and controlling human-like language in LLMs @chengmyra1 @sunnyyuych @jurafsky
• Controllable and Reliable Knowledge-Intensive Task Agents with Declarative GenieWorksheets
@harshitj__ @ShichengGLiu…
stanfordnlp's tweet image. .@stanfordnlp papers at @aclmeeting in Vienna next week:
• HumT DumT: Measuring and controlling human-like language in LLMs @chengmyra1 @sunnyyuych @jurafsky
• Controllable and Reliable Knowledge-Intensive Task Agents with Declarative GenieWorksheets
@harshitj__ @ShichengGLiu…
stanfordnlp's tweet image. .@stanfordnlp papers at @aclmeeting in Vienna next week:
• HumT DumT: Measuring and controlling human-like language in LLMs @chengmyra1 @sunnyyuych @jurafsky
• Controllable and Reliable Knowledge-Intensive Task Agents with Declarative GenieWorksheets
@harshitj__ @ShichengGLiu…

Claude code just added a bunch of `pytest.skip` lol 😭

harshitj__'s tweet image. Claude code just added a bunch of `pytest.skip` lol 😭

Harshit Joshi 已轉發

Are AI scientists already better than human researchers? We recruited 43 PhD students to spend 3 months executing research ideas proposed by an LLM agent vs human experts. Main finding: LLM ideas result in worse projects than human ideas.

ChengleiSi's tweet image. Are AI scientists already better than human researchers?

We recruited 43 PhD students to spend 3 months executing research ideas proposed by an LLM agent vs human experts.

Main finding: LLM ideas result in worse projects than human ideas.

Harshit Joshi 已轉發

AI companions aren’t science fiction anymore 🤖💬❤️ Thousands are turning to AI chatbots for emotional connection – finding comfort, sharing secrets, and even falling in love. But as AI companionship grows, the line between real and artificial relationships blurs. 📰 “Can A.I.…

zhangyt0704's tweet image. AI companions aren’t science fiction anymore 🤖💬❤️
Thousands are turning to AI chatbots for emotional connection – finding comfort, sharing secrets, and even falling in love. But as AI companionship grows, the line between real and artificial relationships blurs.

📰 “Can A.I.…

Loading...

Something went wrong.


Something went wrong.