josephcc's profile picture. 😉💻🔄 Research Scientist @ AI2/Semantic Scholar | Artisanal small-batch handcrafted tweets with no added llms. @joseph@hci.social @josephc.bsky.social

Joseph Chee Chang

@josephcc

😉💻🔄 Research Scientist @ AI2/Semantic Scholar | Artisanal small-batch handcrafted tweets with no added llms. @[email protected] @josephc.bsky.social

Joseph Chee Chang รีโพสต์แล้ว

Last year I worked at @Adobe @AdobeResearch and @allen_ai, exploring how we can help users read, organize and understand long documents. This piece covers what we learned on modelling user intent and combining LLMs with principled tools when building complex pipelines for it!

CDS PhD alum Vishakh Padmakumar (@vishakh_pk), now at @Stanford, tackled the hard part of summarization — deciding what matters. At @Adobe, he built diversity-aware summarizers; at AI2 (@allen_ai), intent-based tools for literature review tables. nyudatascience.medium.com/supercharged-i…



Joseph Chee Chang รีโพสต์แล้ว

Introducing Asta—our bold initiative to accelerate science with trustworthy, capable agents, benchmarks, & developer resources that bring clarity to the landscape of scientific AI + agents. 🧵


Joseph Chee Chang รีโพสต์แล้ว

Producing reasoning texts boosts the capabilities of AI models, but do we humans correctly understand these texts? Our latest research suggests that we do not. This highlights a new angle on the "Are they transparent?" debate: they might be, but we misinterpret them. 🧵

mosh_levy's tweet image. Producing reasoning texts boosts the capabilities of AI models, but do we humans correctly understand these texts? Our latest research suggests that we do not.
This highlights a new angle on the "Are they transparent?" debate: they might be, but we misinterpret them. 🧵

Joseph Chee Chang รีโพสต์แล้ว

People at #ACL2025, come drop by our poster today & chat with me about how context matters for reliable language model evaluations! Jul 30, 11:00-12:30 at Hall 4X, board 424.

Excited to share ✨ Contextualized Evaluations ✨! Benchmarks like Chatbot Arena contain underspecified queries, which can lead to arbitrary eval judgments. What happens if we provide evaluators with context (e.g who's the user, what's their intent) when judging LM outputs? 🧵↓

cmalaviya11's tweet image. Excited to share ✨ Contextualized Evaluations ✨!

Benchmarks like Chatbot Arena contain underspecified queries, which can lead to arbitrary eval judgments. What happens if we provide evaluators with context (e.g who's the user, what's their intent) when judging LM outputs? 🧵↓


Joseph Chee Chang รีโพสต์แล้ว

Ai2 is excited to be at #ACL2025 in Vienna, Austria this week. Come say hello, meet the team, and chat about the future of NLP. See you there! 🤝📚

allen_ai's tweet image. Ai2 is excited to be at #ACL2025 in Vienna, Austria this week. Come say hello, meet the team, and chat about the future of NLP. See you there! 🤝📚
allen_ai's tweet image. Ai2 is excited to be at #ACL2025 in Vienna, Austria this week. Come say hello, meet the team, and chat about the future of NLP. See you there! 🤝📚

Joseph Chee Chang รีโพสต์แล้ว

Thank you all for joining us!! The Q&A were all very insightful 😀😀 Here's the link to the slides: bit.ly/acl25-hai-team


Joseph Chee Chang รีโพสต์แล้ว

In Vienna for #ACL2025NLP this week! @josephcc, @aps6992 and I will present the Ai2 ScholarQA scientific QA system on Wed. I’ll also be at @sdpworkshop on Thurs! Hit me up if you’d like to chat about agents for science and post-training, or explore cafes in Vienna 🥐


Joseph Chee Chang รีโพสต์แล้ว

We all agree that AI models/agents should augment humans instead of replace us in many cases. But how do we pick when to have AI collaborators, and how do we build them? Come check out our #ACL2025NLP tutorial on Human-AI Collaboration w/ @Diyi_Yang @josephcc, 📍7/27 9am@ Hall N!

tongshuangwu's tweet image. We all agree that AI models/agents should augment humans instead of replace us in many cases. But how do we pick when to have AI collaborators, and how do we build them? Come check out our #ACL2025NLP tutorial on Human-AI Collaboration w/ @Diyi_Yang @josephcc, 📍7/27 9am@ Hall N!
tongshuangwu's tweet image. We all agree that AI models/agents should augment humans instead of replace us in many cases. But how do we pick when to have AI collaborators, and how do we build them? Come check out our #ACL2025NLP tutorial on Human-AI Collaboration w/ @Diyi_Yang @josephcc, 📍7/27 9am@ Hall N!

Joseph Chee Chang รีโพสต์แล้ว

In our new paper, “Contextualized Evaluations: Judging Language Model Responses to Underspecified Queries,” we find that adding just a bit of missing context can reorder model leaderboards—and surface hidden biases. 🧵👇

allen_ai's tweet image. In our new paper, “Contextualized Evaluations: Judging Language Model Responses to Underspecified Queries,” we find that adding just a bit of missing context can reorder model leaderboards—and surface hidden biases. 🧵👇

Joseph Chee Chang รีโพสต์แล้ว

This new ScholarQA capability works for most openly licensed papers. It’s part of our commitment to transparency in science and making it easier to verify, trace, and build trusted AI.


You can now jump from Scholar QA answers to highlighted evidence in the source paper's pdf : )

We’ve upgraded ScholarQA, our agent that helps researchers conduct literature reviews efficiently by providing detailed answers. Now, when ScholarQA cites a source, it won’t just tell you which paper it came from–you’ll see the exact quote, highlighted in the original PDF. 🧵

allen_ai's tweet image. We’ve upgraded ScholarQA, our agent that helps researchers conduct literature reviews efficiently by providing detailed answers. Now, when ScholarQA cites a source, it won’t just tell you which paper it came from–you’ll see the exact quote, highlighted in the original PDF. 🧵


Joseph Chee Chang รีโพสต์แล้ว

Introducing SciArena, a platform for benchmarking models across scientific literature tasks. Inspired by Chatbot Arena, SciArena applies a crowdsourced LLM evaluation approach to the scientific domain. 🧵

allen_ai's tweet image. Introducing SciArena, a platform for benchmarking models across scientific literature tasks. Inspired by Chatbot Arena, SciArena applies a crowdsourced LLM evaluation approach to the scientific domain. 🧵

Joseph Chee Chang รีโพสต์แล้ว

@allen_ai @SemanticScholar is hiring an #ml #nlp #ai reasoning researcher for a Research Scientist, Agents for Science position with target start dates in 2025. Excited about developing AI systems with deep reasoning capabilities for science? Send an application our way!

ai2_s2research's tweet image. @allen_ai @SemanticScholar
is hiring an #ml #nlp #ai reasoning researcher for a Research Scientist, Agents for Science position with target start dates in 2025. Excited about developing AI systems with deep reasoning capabilities for science? Send an application our way!

Joseph Chee Chang รีโพสต์แล้ว

🆕paper: LLMs Get Lost in Multi-Turn Conversation In real life, people don’t speak in perfect prompts. So we simulate multi-turn conversations — less lab-like, more like real use. We find that LLMs get lost in conversation. 👀What does that mean? 🧵1/N 📄arxiv.org/abs/2505.06120

PhilippeLaban's tweet image. 🆕paper: LLMs Get Lost in Multi-Turn Conversation

In real life, people don’t speak in perfect prompts.
So we simulate multi-turn conversations — less lab-like, more like real use.

We find that LLMs get lost in conversation.
👀What does that mean? 🧵1/N
📄arxiv.org/abs/2505.06120
PhilippeLaban's tweet image. 🆕paper: LLMs Get Lost in Multi-Turn Conversation

In real life, people don’t speak in perfect prompts.
So we simulate multi-turn conversations — less lab-like, more like real use.

We find that LLMs get lost in conversation.
👀What does that mean? 🧵1/N
📄arxiv.org/abs/2505.06120

Joseph Chee Chang รีโพสต์แล้ว

I am presenting IdeaSynth in the last paper session at #CHI2025 right now! Feel free to come by G314-315 to learn about how we utilize LLM to provide literature-grounded assistance for research idea development! The talk is happening at around 10:12 AM.

🔬Research ideation is hard: After the spark of a brilliant initial idea, much work is still needed to further develop it into a well-thoughtout project by iteratively expanding and refining the initial idea and grounding it to relevant literature. How can we better support this?

kevpjk's tweet image. 🔬Research ideation is hard: After the spark of a brilliant initial idea, much work is still needed to further develop it into a well-thoughtout project by iteratively expanding and refining the initial idea and grounding it to relevant literature. How can we better support this?


Joseph Chee Chang รีโพสต์แล้ว

AI agents are entering online social spaces, but often their messages feel generic or intrusive. In our #CHI25 paper, we introduce Social-RAG, a workflow that grounds AI generations in the specific group context by retrieving from the group’s interaction history. 🧵(1/9)

RuotongWang1's tweet image. AI agents are entering online social spaces, but often their messages feel generic or intrusive. In our #CHI25 paper, we introduce Social-RAG, a workflow that grounds AI generations in the specific group context by retrieving from the group’s interaction history. 🧵(1/9)

Joseph Chee Chang รีโพสต์แล้ว

Meet Ai2 Paper Finder, an LLM-powered literature search system. Searching for relevant work is a multi-step process that requires iteration. Paper Finder mimics this workflow — and helps researchers find more papers than ever 🔍

allen_ai's tweet image. Meet Ai2 Paper Finder, an LLM-powered literature search system.

Searching for relevant work is a multi-step process that requires iteration. Paper Finder mimics this workflow — and helps researchers find more papers than ever 🔍

Joseph Chee Chang รีโพสต์แล้ว

We are looking for CS researchers to participate in a study exploring how AI can change the way we do literature reviews. 📚🧑‍🎓 Time: ~90 min, remote Compensation: $60 USD Sign up here: forms.gle/Pzw6YUhVUaZsS6… @dsweld @amyxzh @josephcc @marissa_rad @Siangliulue @turingmusician


Loading...

Something went wrong.


Something went wrong.