alex_ander's profile picture. Interested in how & what the brain computes. Professor of Stats & Neuro UC Berkeley. Married to the incredible @Libertysays. he/him

Alexander Huth

@alex_ander

Interested in how & what the brain computes. Professor of Stats & Neuro UC Berkeley. Married to the incredible @Libertysays. he/him

Alexander Huth reposted

Our "mind captioning" paper is now published in @ScienceAdvances. The method generates descriptive text of what we perceive and recall from brain activity — a linguistic interpretation of nonverbal mental content rather than language decoding. doi.org/10.1126/sciadv…

Our new paper is on bioRxiv. We present a novel generative decoding method, called Mind Captioning, and demonstrate the generation of descriptive text of viewed and imagined content from human brain activity. The video shows text generated for viewed content during optimization.



Oops—to be clear this data is @zaidzada_'s wonderful Podcast ECoG dataset, not new ECoG data! Thanks to those authors for releasing this great corpus @samnastase @HassonLab

Finally, we tested whether the same interpretable embeddings could also be used to model ECoG data from Nima Mesgarani's lab. Despite the fact that our features are less well-localized in time than LLM embeddings, this still works quite well!

alex_ander's tweet image. Finally, we tested whether the same interpretable embeddings could also be used to model ECoG data from Nima Mesgarani's lab. Despite the fact that our features are less well-localized in time than LLM embeddings, this still works quite well!


Alexander Huth reposted

In our new paper, we explore how we can build encoding models that are both powerful and understandable. Our model uses an LLM to answer 35 questions about a sentence's content. The answers linearly contribute to our prediction of how the brain will respond to that sentence. 1/6


Alexander Huth reposted

New paper: Ask 35 simple questions about sentences in a story and use the answers to predict brain responses. Interpretable. Compact. Surprisingly high performance in both fMRI and ECoG. biorxiv.org/content/10.110…

csinva's tweet image. New paper: Ask 35 simple questions about sentences in a story and use the answers to predict brain responses. Interpretable. Compact. Surprisingly high performance in both fMRI and ECoG. biorxiv.org/content/10.110…

Loading...

Something went wrong.


Something went wrong.