Abhilasha Ravichander

Publications

"*" denotes equal contribution

In Agents We Trust, but Who Do Agents Trust? Latent Preferences Steer LLM Generations
Mohammad Aflah Khan, Mahsa Amani, Soumi Das, Bishwamittra Ghosh, Qinyuan Wu, Krishna P. Gummadi, Manish Gupta, Abhilasha Ravichander
ICLR 2026
PDF Code/Data Long
Revisiting the Past: Data Unlearning with Model State History
Keivan Rezaei, Mehrdad Saberi, Soheil Feizi, Abhilasha Ravichander
ICLR 2026
PDF Code/Data Long
The Curious Case of Factuality Finetuning: Models' Internal Beliefs Can Improve Factuality
Benjamin Newman, Abhilasha Ravichander, Jaehun Jung, Rui Xin, Hamish Ivison, Yegor Kuznetsov, Pang Wei Koh, Yejin Choi
arxiv
PDF Code/Data Long
🏆 HALoGEN: Fantastic LLM Hallucinations and Where To Find Them ACL Outstanding Paper Award/TrustNLP Workshop Best Paper Award
Abhilasha Ravichander*, Shrusti Ghela*, David Wadden, Yejin Choi
ACL 2025
PDF Code/Data Website Long
The Surprising Effectiveness of Membership Inference with Simple N-Gram Coverage
Skyler Hallinan, Jaehun Jung, Melanie Sclar, Ximing Lu, Abhilasha Ravichander, Sahana Ramnath, Yejin Choi, Sai Praneeth Karimireddy, Niloofar Mireshghallah, Xiang Ren
COLM 2025
PDF Code/Data Long
Information-Guided Identification of Training Data Imprint in (Proprietary) Large Language Models Nominated for Outstanding Paper Award
Abhilasha Ravichander, Jillian Fisher, Taylor Sorensen, Ximing Lu, Maria Antoniak, Bill Yuchen Lin, Niloofar Mireshghallah, Chandra Bhagavatula, Yejin Choi
NAACL 2025
PDF Code/Data Long
📰 Press: Techcrunch Mint
RESTOR: Knowledge Recovery through Machine Unlearning
Keivan Rezaei, Khyathi Chandu, Soheil Feizi, Yejin Choi, Faeze Brahman, Abhilasha Ravichander
TMLR 2025
PDF Code/Data Long
Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations
Yiyou Sun, Yu Gai, Lijie Chen, Abhilasha Ravichander, Yejin Choi, Dawn Song
NeurIPS 2025
PDF Code/Data Long
What Has Been Lost with Synthetic Evaluation?
Alex Gill, Abhilasha Ravichander, Ana Marasović
EMNLP Findings 2025
PDF Long
Reverse Question Answering: Can an LLM Write a Question so Hard (or Bad) that it Can't Answer?
Nishant Balepur, Feng Gu, Abhilasha Ravichander, Shi Feng, Jordan Boyd-Graber, Rachel Rudinger
NAACL 2025
PDF Code/Data Short
⭐ WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild Spotlight
Bill Yuchen Lin, Yuntian Deng, Khyathi Chandu, Faeze Brahman, Abhilasha Ravichander, Valentina Pyatkin, Nouha Dziri, Ronan Le Bras, Yejin Choi
ICLR 2025
PDF Code/Data Long
The Art of Saying No: Contextual Noncompliance in Language Models
Faeze Brahman*, Sachin Kumar*, Abhilasha Ravichander‡, Vidhisha Balachandran‡, Pradeep Dasigi‡, Valentina Pyatkin‡, Sarah Wiegreffe‡, Nouha Dziri, Khyathi Chandu, Jack Hessel, Yulia Tsvetkov, Noah A Smith, Yejin Choi, Hannaneh Hajishirzi
NeurIPS 2024 Datasets and Benchmarks
PDF Code/Data Long
🏆 Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question? MASC-SLL 2024 Best Paper Award
Nishant Balepur, Abhilasha Ravichander, Rachel Rudinger
ACL 2024
PDF Code/Data Long
🏆 OLMo: Accelerating the Science of Language Models ACL Best Theme Paper Award/GeekWire Innovation of the Year Award
Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, Hannaneh Hajishirzi
62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024)
PDF Code/Data Long
📰 Press: TechCrunch VentureBeat Forbes GeekWire Axios SD Times Fast Company
🏆 Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research ACL Best Resource Paper Award
Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A. Smith, Hannaneh Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, Kyle Lo
62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024)
PDF Code/Data Long
📰 Press: TechCrunch Marktechpost Voicebot
Agent Lumos: Unified and Modular Training for Open-Source Language Agents
Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, Bill Yuchen Lin
62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024)
PDF Code/Data Long Website
📰 Press: Marktechpost
MacGyver: Are Large Language Models Creative Problem Solvers? Nominated for Outstanding Paper Award
Yufei Tian, Abhilasha Ravichander, Lianhui Qin, Ronan Le Bras, Raja Marjieh, Nanyun Peng, Yejin Choi, Thomas L Griffiths, Faeze Brahman
NAACL 2024
PDF Code/Data Long
WildHallucinations: Evaluating Long-form Factuality in LLMs with Real-World Entity Queries
Wenting Zhao, Tanya Goyal, Yu Ying Chiu, Liwei Jiang, Benjamin Newman, Abhilasha Ravichander, Khyathi Chandu, Ronan Le Bras, Claire Cardie, Yuntian Deng, Yejin Choi
arXiv
PDF Code/Data Long
⭐ What's In My Big Data? Spotlight
Yanai Elazar, Akshita Bhagia, Ian Magnusson, Abhilasha Ravichander, Dustin Schwenk, Alane Suhr, Evan Pete Walsh, Dirk Groeneveld, Luca Soldaini, Sameer Singh, Hannaneh Hajishirzi, Noah A. Smith, Jesse Dodge
ICLR 2024
PDF Code/Data Long Website
📰 Press: Marktechpost
The Generative AI Paradox: "What It Can Create, It May Not Understand"
Peter West, Ximing Lu, Nouha Dziri, Faeze Brahman, Linjie Li, Jena D. Hwang, Liwei Jiang, Jillian Fisher, Abhilasha Ravichander, Khyathi Chandu, Benjamin Newman, Pang Wei Koh, Allyson Ettinger, Yejin Choi
2024 International Conference on Learning Representations (ICLR 2024)
PDF Long
The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning
Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, Yejin Choi
2024 International Conference on Learning Representations (ICLR 2024)
PDF Code/Data Long Website
Understanding How to Inform Blind and Low-Vision Users about Data Privacy through Privacy Question Answering Assistants
Yuanyuan Feng, Abhilasha Ravichander, Yaxing Yao, Shikun Zhang, Rex Chen, Shomir Wilson, Norman Sadeh
USENIX Security 2024
PDF Long
Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuning
Ximing Lu, Faeze Brahman, Peter West, Jaehun Jang, Khyathi Chandu, Abhilasha Ravichander, Lianhui Qin, Prithviraj Ammanabrolu, Liwei Jiang, Sahana Ramnath, Nouha Dziri, Jillian Fisher, Bill Yuchen Lin, Skyler Hallinan, Xiang Ren, Sean Welleck, Yejin Choi
EMNLP 2023
PDF Code/Data Long
When and Why Does Bias Mitigation Work?
Abhilasha Ravichander*, Joe Stacey*, Marek Rei
EMNLP Findings 2023
PDF Code/Data Long
🏆 CondaQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation SoCal NLP Symposium Best Paper Award
Abhilasha Ravichander, Matt Gardner, Ana Marasović
EMNLP 2022
PDF Code/Data Long
Measuring Causal Effects of Data Statistics on Language Model's 'Factual' Predictions
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Amir Feder, Abhilasha Ravichander, Marius Mosbach, Yonatan Belinkov, Hinrich Schütze, Yoav Goldberg
arXiv
PDF
A Tale of Two Regulatory Regimes: Creation and Analysis of a Bilingual Privacy Policy Corpus
Siddhant Arora, Henry Hosseini, Christine Utz, Vinayshekhar Bannihatti Kumar, Tristan Dhellemmes, Abhilasha Ravichander, Peter Story, Jasmine Mangat, Rex Chen, Martin Degeling, Thomas Norton, Thomas Hupperich, Shomir Wilson, Norman Sadeh
LREC 2022
PDF Code/Data Long
Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?
Abhilasha Ravichander, Yonatan Belinkov, Eduard Hovy
EACL 2021
PDF Code/Data Long
NoiseQA: Challenge Set Evaluation for User-Centric Question Answering
Abhilasha Ravichander, Siddharth Dalmia, Maria Ryskina, Florian Metze, Eduard Hovy, Alan W Black
16th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2021)
PDF Code/Data Long Website
Measuring and Improving Consistency in Pretrained Language Models
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, Yoav Goldberg
TACL 2021
PDF Code/Data Long
Breaking Down Walls of Text: How Can NLP Benefit Consumer Privacy?
Abhilasha Ravichander, Alan W Black, Thomas Norton, Shomir Wilson and Norman Sadeh
ACL 2021
PDF Long
On the Systematicity of Probing Contextualized Word Representations: The Case of Hypernymy in BERT
Abhilasha Ravichander, Eduard Hovy, Kaheer Suleman, Adam Trischler, Jackie Chi Kit Cheung
*SEM 2020
PDF Code/Data Long
EQUATE: A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference
Abhilasha Ravichander*, Aakanksha Naik*, Carolyn Rose, Eduard Hovy
CoNLL 2019
PDF Code/Data Long
Question Answering for Privacy Policies: Combining Computational and Legal Perspectives
Abhilasha Ravichander, Alan W Black, Shomir Wilson, Thomas Norton and Norman Sadeh
EMNLP 2019
PDF Code/Data Long
Exploring Numeracy in Word Embeddings
Aakanksha Naik*, Abhilasha Ravichander*, Carolyn Rose, Eduard Hovy
ACL 2019
PDF Short
Evaluating How Global Privacy Principles Answer Consumers' Questions About Mobile App Privacy
Thomas Norton, Joel Reidenberg, Norman Sadeh and Abhilasha Ravichander
PLSC 2019
Challenges in Automated Question Answering for Privacy Policies
Abhilasha Ravichander, Alan Black, Eduard Hovy, Joel Reidenberg, N. Cameron Russell and Norman Sadeh
AAAI Spring Symposium Series, 2019
PDF Long
MAPS: Scaling Privacy Compliance Analysis to a Million Apps
Peter Story, Sebastian Zimmeck, Daniel Smullen, Abhilasha Ravichander, Ziqi Wang, Joel Reidenberg, N. Cameron Russell and Norman Sadeh
PETS 2019
PDF Long
🏆 Stress Test Evaluation for Natural Language Inference Area Chair Favorite Paper Prize
Aakanksha Naik*, Abhilasha Ravichander*, Norman Sadeh, Carolyn Rose, Graham Neubig
COLING 2018
PDF Code/Data Long Website Slides
An Empirical Study of Self-Disclosure in Spoken Dialogue Systems
Abhilasha Ravichander, Alan Black
SIGDIAL 2018
PDF Long
Does the Geometry of Word Embeddings Help Document Classification? A Case Study on Persistent Homology-Based Representations
Paul Michel*, Abhilasha Ravichander*, Shruti Rijhwani*
ACL 2017 Workshop
PDF Short
How Would You Say It? Eliciting Lexically Diverse Data for Supervised Semantic Parsing
Abhilasha Ravichander*, Thomas Manzini*, Matthias Grabmair, Jonathan Francis, Graham Neubig, Eric Nyberg
SIGDIAL 2017
PDF Code/Data Long