Pluralistic Alignment
@ ICML 2026 Workshop
July 11, 2026.
Room 403, COEX Convention & Exhibition Center, Seoul, South Korea.
Pluralistic AI: Aligning with the Diversity of Human Values
Welcome to the Pluralistic Alignment Workshop! Aligning AI systems with human preferences and societal values has become a critical challenge as these technologies grow more powerful and pervasive. However, current AI alignment methods have proven insufficient for capturing the full spectrum of complex—and often conflicting—real-world values held across diverse populations. This workshop addresses this gap by examining how to integrate diverse perspectives, values, and expertise into pluralistic AI alignment frameworks. We will explore novel approaches to multi-objective alignment, drawing inspiration from established governance mechanisms and consensus-building practices to navigate the value conflicts inherent in pluralistic societies. The workshop will cover technical innovations in preference elicitation and dataset collection, algorithm development for multi-stakeholder optimization, and the design of human-AI interaction workflows that authentically reflect pluralistic values across diverse communities. By convening researchers, practitioners, and domain experts from AI safety, political philosophy, social science, and human-computer interaction, this workshop aims to foster interdisciplinary collaboration that advances both the theoretical foundations and practical implementation of pluralistic AI alignment.
Stay tuned by following us on Twitter @pluralistic_ai.
Speakers
Alice Oh
KAIST
Atoosa Kasirzadeh
Carnegie Mellon University
Mitchell Gordon
Massachusetts Institute of Technology
Xiaoyuan Yi
Microsoft Research Asia
Panel Discussion
The workshop will feature several panel discussions on pluralistic AI, alignment, and diversity of human values. We expect invited speakers and additional researchers to participate in the panels. Participation details will be updated as invitations are finalized.
Jenny Ni
Naman Goyal
Google DeepMind
Nazneen Rajani
Collinear
Wei Xu
Georgia Institute of Technology
Schedule
| Time | Program |
|---|---|
| 09:15 - 09:30 | Opening remarks |
| 09:30 - 10:15 | Keynote Talk: Alice Oh |
| 10:15 - 11:00 | Keynote Talk: Xiaoyuan Yi |
| 11:00 - 11:15 | Break & Networking |
| 11:15 - 12:00 | Keynote Talk: Mitchell Gordon |
| 12:00 - 13:30 | Lunch Break |
| 13:30 - 14:15 | Keynote Talk: Atoosa Kasirzadeh |
| 14:15 - 15:00 | Panel Discussion |
| 15:00 - 16:30 | Poster Presentation & Coffee Break |
| 16:30 - 17:00 | Closing Remarks & Next Steps |
Keynote Talks
Alignment for Code-Switching
Speaker: Alice Oh
All multilingual speakers code-switch, mixing two or more languages intra- or inter-sententially. While people respond appropriately to code-switching, LLMs do not. I will first present our research on how code-switching causes problems for LLMs, and then I will present research on how code-switching can offer solutions to improve multilingual capabilities of LLMs.
I Contain Multitudes: Why Value Pluralism Matters for AI
Speaker: Xiaoyuan Yi
As AI evolves from a tool into a social actor deeply embedded in human life, aligning frontier AI with pluralistic human values becomes increasingly important, whether through Overton pluralism, where AI can express multiple values, or steerable pluralism, where values can be customized, as human values are inherently diverse. However, the concrete impact of value pluralism on AI remains underexplored. This talk examines why pluralism matters for AI through interdisciplinary social simulation experiments. We demonstrate that, in an AI community composed of multiple LLM agents, (1) value diversity can enhance collective creativity and facilitate the emergence of more complex, human-like social structures; (2) while homogeneous or biased value configurations can trigger irreversible, catastrophic community collapse (x-risk). Building on these findings, we further present an effective in-context-learning-based approach to pluralistic alignment.
Pluralistic and Collective Alignment, End-To-End
Speaker: Mitchell Gordon
How should we align AI systems when people genuinely disagree about how those systems should behave? In this talk, I will discuss challenges that arise at several different parts of an end-to-end collective alignment process, from theory and definitions, to gathering and acting on people's input, to applying behavioral guidance in practice. I will first examine what it actually means for a model to be pluralistically aligned and how different forms of pluralism can be defined and evaluated. I will then discuss our work on collective alignment at OpenAI, including how we gathered participants' preferences about model behavior and translated their feedback into proposed changes to the Model Spec. Finally, I will consider what happens when a model spec or constitution has to be applied to a new problem, where many important judgments remain open, and introduce our work on making those judgments easier to examine.
Scientific alignment: How could AI in science go wrong & how can we prevent it?
Speaker: Atoosa Kasirzadeh
In this talk, I investigate the applications of AI agents in conducting machine learning and social science research. I will highlight common failure modes and methodological pitfalls, and outline practical mitigation strategies.
Call for Papers
Our workshop aims to bring together researchers with diverse scientific backgrounds, including (but not limited to) machine learning, human-computer interaction, philosophy, and policy studies. More broadly, our workshop lies at the intersection of computer and social sciences. We welcome all interested researchers to discuss the aspects of pluralistic AI, from its definition to the technical pipeline to broad deployment and social acceptance.
We invite submissions that discuss the technical, philosophical, and societal aspects of pluralistic AI. We provide a non-exhaustive list of topics we hope to cover below. We also broadly welcome any submissions which are broadly relevant to pluralistic alignment.
- Philosophy:
- Definitions and frameworks for Pluralistic Alignment
- Ethical considerations in aligning AI with diverse human values
- Machine learning:
- Methods for pluralistic ML training and learning algorithms
- Methods for handling annotation disagreements
- Evaluation metrics and datasets suitable for pluralistic AI
- Human-computer interaction:
- Designing human-AI interaction that reflects diverse user experiences and values
- Integrating existing surveys on human values into AI design
- Navigating privacy challenges in pluralistic AI systems
- Social sciences:
- Methods for achieving consensus and different forms of aggregation
- Assessment and measurement of the social impact of pluralistic AI
- Dealing with pluralistic AI representing values that are offensive to some cultural groups
- Policy studies:
- Policy and laws for the deployment of pluralistic AI
- Democratic processes for incorporating diverse values into AI systems on a broad scale
- Applications:
- Case studies in areas such as hate speech mitigation and public health
Submission Instructions
We invite authors to submit anonymized papers at least 4 pages and up to 8 pages of content, plus unlimited pages of references and appendices. Submissions must follow the ICML 2026 template and be made through OpenReview submission portal. Checklists are not required for submissions. Reviews will be double-blind, with at least three reviewers assigned to each paper to ensure a thorough evaluation process.
We welcome various types of papers including works in progress, position papers, policy papers, academic papers. All accepted papers will be available on the workshop website, but are to be considered non-archival.
Due to the high volume of submissions (thank you for your interest), we will adopt the ICML policy. All submissions must include at least one author who agrees to serve as a reviewer for the Pluralistic Alignment Workshop.
Important Dates
All deadlines are 11:59 pm UTC-12h (“Anywhere on Earth”).
| March 22, 2026 | Call for Workshop Papers |
| May 8, 2026 | Paper Submission Deadline |
| May 22, 2026 | Notification of Acceptance |
| June 10, 2026 | Camera-Ready Version Due |
| July 11, 2026 | Workshop Date |
|---|
Accepted Papers
- Adaptive Pluralistic Alignment: a pipeline for dynamic artificial democracy (Poster, Video)
- AI Pluralism and the Worlds It Misses
- Algorithmic Approaches to Opinion Selection for Online Deliberation: A Comparative Study (Poster, Video)
- Benchmarking Pluralistic Alignment Through Persona-Conditioned Behavioral Evaluation
- Beyond the Mean: Three-Axis Fidelity for Aligning LLM-Based Survey Simulators from Small Pilot Data
- Bosses, Kings, and the Commons: Cooperation Under Power Asymmetry in LLM Societies
- Changing Tunes: A Longitudinal Study of Political Drift in LLMs
- ConstitutionMAS-EC: Peer Constitutional Critique for Aligned Emergent Communication in Decentralized Multi-Agent LLMs
- Data Mixing for Group Preference Heterogeneity in Collaborative Filtering
- Deference by Design: Pluralistic Alignment Is an Interface Problem (Poster, Video)
- Directional Influence and Consensus Formation in Multi-Agent Systems
- Diversifying Multiple Generative Agents by Aligning with Human Populations
- Do LLMs Acknowledge Disputed Facts? A Benchmark for Factual Pluralism in LLMs
- Does AI Assistance Preserve or Collapse Disagreement? A Study of Pre-Annotations in Ambiguous Video Labeling (Poster)
- Does Privacy Always Harm Fairness? Data-Dependent Trade-offs via Chernoff Information Neural Estimation (Poster, Video)
- Dual Mechanisms of Value Expression: Intrinsic vs. Prompted Values in Large Language Models (Poster, Video)
- Efficient Rashomon Set Approximation for Decision Trees
- EGGROLL-IPO: Pluralistic Alignment via Decentralised Post-Training with Population Preferences
- Evaluating Pluralism in LLMs through Latent Perspectives
- Event-Driven Reinforcement Learning for Pluralistic Alignment
- For Questions of Ought, AI Could Use Some SAGE Advice
- FRAGILE: Benchmarking Framing Sensitivity in High-Stakes Decision-Making (Poster)
- Geometry of Values: Task Vector Composition for Ethical Preference Alignment in Language Models
- HEARSAYBENCH: Can LLMs Navigate from Abstract Human Rights to Lived Lives?
- Helpful or Safe? UltraFeedback's Binarized Labels Encode a Value Tradeoff
- Inference-Time Alignment Control for Diffusion Models with Reinforcement Learning Guidance
- Innocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMs
- Investigating Alignment Conflicts in Tool-Calling Enabled LLMs
- It's Up to Interpretation: Aligning to One's Ever-Shifting Internal State (Poster)
- Learning to Retrieve User History and Generate User Profiles for Personalized Persuasiveness Prediction
- Learning Unanimously Acceptable Lotteries via Queries
- LLM Human Response Alignment: A Multi-Sample Debiasing Framework
- Majority Vote Silences Minority Values: Annotator Disagreement at the Hate/Offensive Boundary in HateXplain
- Memetic Capture: A Pluralistic Policy Framework for Governing AI-Driven Cultural Disempowerment
- Memetic Drift in Multi-Agent LLMs: Scaling Laws for Consensus Under Pluralistic Uncertainty
- Mission Impossible: Universal Moral Alignment
- Modeling diverse preferences in movie artwork personalization with large language models
- Moral Orientation and Calibration: Coupled in Human Annotators, Separable in Judge LLMs
- PEBS: Per-rater Empirical-Bayes Shrinkage for RLHF Reward-Model Calibration
- Pedagogical Games: Paths to Generalisation for Agentic Moral Alignment (Poster)
- Personalization, Personas, and Forecasting in Value Alignment
- PIPE: Personalized Image-generation via Preference Encoding (Poster)
- Playing Devil's Advocate: Off-the-Shelf Persona Vectors Rival Targeted Steering for Sycophancy
- Pluralistic AI Alignment Requires Inference-Time Multi-Objective Control
- Pluralistic On-Policy Self-Distillation
- Pluralistic Preference Alignment via Sortition-Weighted RLHF
- Position: Aggregate Preference Optimization Hides a Posterior Identifiability Failure for Pluralistic Alignment
- Position: Align AI to Our Aspirations, Not Our Flaws (Poster, Video)
- Position: LLM alignment data should be regulated as mass media
- Position: Why LLMs Should Be Reasonably Morally Inconsistent (Poster)
- PRISM: When Agents Provably Learn from Pluralistic Human Feedback (Poster)
- Provably Efficient Regularized Online RLHF with Generalized Bilinear Preferences
- Reasoning Models Generate Societies of Thought
- Reducing Supervision Uncertainty Induces Model Miscalibration
- Response-Aware User Memory Selection for LLM Personalization
- Rethinking AI Alignment: From Static Rewards to Social Reinforcement Learning
- Rethinking Diversity-Preserving RL for Pluralistic Alignment: Empirical Evidence from Rubric-Grounded Moral Reasoning
- Rethinking Scaffolding in LLM Tutors: The Interactional Mismatch Between Benchmarks and Real-World Deployments (Poster)
- RobotValues: Evaluating Household Robots When Human Values Conflict (Poster)
- RouterJudge: Preference-Based Evaluation of LLM Routers under Pluralistic User Preferences (Poster)
- Same Facts, Different Updates: Inference Setup Shapes LLM Behavior in Medical Allocation
- Separating Value Disagreement from Data Uncertainty in Pluralistic Preference Data
- Side Effects of Character Training: Quantifying Cross-Constitution Drift in LLMs (Poster)
- Social Choice Foundations for Simulation-Augmented Generation
- Socially Grounded Agentic AI: Coordinating Plural Perspectives through Social Theory
- Steerable Cultural Preference Optimization of Reward Models (Poster)
- The Homogenization Problem in LLMs: Towards Meaningful Diversity in AI Safety (Poster)
- The Language of Bargaining: Linguistic Effects in LLM Negotiations (Poster)
- The Persona Fidelity Gap: Behaviorally Grounded LLM Personas Still Compress Real-User Preference Diversity
- The Wedge Questions: Latent Cultural Boundaries in LLMs via Persona Projection Divergence
- ThoughtTrace: Understanding User Thoughts in Real-World LLM Interactions
- To Whom Do Language Models Align? Measuring Principal Hierarchies Under High-Stakes Competing Demands (Poster)
- Toward Deployable Pluralistic Alignment in Robotics: Learning Similarity-Grouped Rewards from Diverse Human Preferences
- Universal Alignment Fails in Global Classrooms: Cross-Cultural Blind Spots in EdTech AI (Poster, Video)
- What Aggregate Accuracy Hides: Cultural Affective Inequity in Multilingual LLMs
- What Does the AI Doctor Value? Auditing Pluralism in the Clinical Ethics of Language Models
- When Disagreement Matters: Friction, Pluralistic Alignment, and National-Security AI
- When We Don't See The Same Picture: Aligning Agents with Divergent Visual Spaces (Poster, Video)
- Where Models Concentrate and Humans Spread: A Coverage Framework for Distributional Pluralism in Open-Ended Generation
- Whose Alignment? Comparing LLM Process Alignment Across Diverse Organizational Decision Contexts
Organization
Organizing Committee
JinYeong Bak
Sungkyunkwan University
Yohan Jo
Seoul National University
Ruyuan Wan
Pennsylvania State University
Liwei Jiang
University of Washington
Maarten Sap
CMU LTI
Dongyeop Kang
University of Minnesota
Taylor Sorensen
University of Washington
Kshitish Ghate
University of Washington
Amy Zhang
University of Washington
Reviewers
| Abhilekh Borah | Abhinav Chinta | Abhishek Bakshi | Agamdeep Singh |
| Ahmad A Rushdi | Akash Das | Alexander Nemecek | Alexandra Neagu |
| Ali Iranmanesh | Amrita Bhattacharjee | Ananya Sutradhar | Andrea Cavallaro |
| Andrea Miele | Andrew Aquilina | Andrew M. Bean | Anushri Iyer |
| Archie Chaudhury | Arjun Nichani | Arnav Raj | Aryan Raju Mandapati |
| Ashish Mehta | Ashish Tiwari | Avinab Saha | Ayush Noori |
| Ayushi Mehta | Bhagyesh Kumar | Bidipta Sarkar | Catherine Ge-Wang |
| Cen Lu | Chaewon Heo | Cheyon Jin | Chhavi Yadav |
| Chih-Wei Hsu | Chuanyang Jin | Daiwei Chen | David Mingfei Liu |
| David Scott Lewis | E. Glen Weyl | Edith Elkind | Edward Stevinson |
| Elinor Poole-Dayan | Emilio Barkett | Enfa Fane | Enoch H. Kang |
| Eun Cheol Choi | Fangyi Yu | Guannan Lai | Gul Zain Khan |
| Gyubin Choi | Haemin Jung | Haesung Pyun | Haewon Park |
| Hakeem Hannoon | Hanxuan Chen | Haopeng Xie | Hayden McTavish |
| HAYOUNG OH | Heejae Suh | Hidenori Tanaka | Hiro Taiyo Hamada |
| Hiroshi Yamakawa | Hwanhee Lee | Hyeongjin Kim | Hyoung Jo Bhang |
| Hyunbin Song | HyunJi Nam | HyunJin Kim | Hyunjong Kim |
| Ian Rios-Sialer | Injin Kong | Itamar Pres | Jaehyeok Lee |
| Jaewoong Han | Jakob Stenseke | James Wedgwood | Jennifer Neville |
| Jeonghoon Shim | Jillian Fisher | Jingyi Zhang | JINSU SHIN |
| Jiwon Park | João Gonçalves | Jonathn Chang | Jongeun Choi |
| Jonggeun Lee | Jonghyun Song | Jongwon Lim | Jongwook Han |
| Jooyoung Lim | Joshua Ashkinaze | Joshua Muhumuza | Joy Olusanya |
| Juan Gutiérrez | Junghyun Lee | Jungwoo Yang | Junsol Kim |
| Kaustubha V | Kenji Doya | Keun Hee Park | Keyu Zhu |
| Krish Sen | Krishna Reddy Kesari | Leixin Zhang | Leo Lorence George |
| Li Jiang | Lionel Levine | Luozhijie Jin | Maël Donoso |
| Maja Pavlovic | Majid Ghasemi | Manh Hung Nguyen | Martin T. Wells |
| Martin Tutek | Massimo Poesio | Matt Ratto | Minjae Oh |
| Minsik Oh | Mirae Kim | Mithilesh Kumar Singh | Monojit Choudhury |
| Moonkyung Ryu | Morgan Livingston | Muhammad Bilal Zafar | Nabeel Seedat |
| Naeun Lee | Nicholas Teh | Nikita Kazeev | Payal Chandak |
| Payel Das | Peter Vamplew | Poojak Patel | Praveenkumar Kondikoppa |
| Prisha Priyadarshini | Quan Ze Chen | Rachel Freedman | Raghav Gupta |
| Ran Wei | Rashid Mushkani | Rishav Kumar | Rishi Ashish Shah |
| Rob Sneiderman | Roberto-Rafael Maura-Rivero | Saimun Habib | Saket Reddy |
| Salim Hafid | Sangjun Song | Saurav Panigrahi | Sebastian Tschiatschek |
| Sejun Park | Seojin Hwang | Seongjun Yang | Sharon T Itonyo |
| Sharvin Goyal | Shreyas Kadekodi | Shuo Yang | Simret A Gebreegziabher |
| Smitha Milli | Sonja M. Schmer-Galunder | Soyoung Yun | Spencer Gibson |
| Steven Lawrence Fernandes | Steven Molotnikov | Subramanyam Sahoo | Suhyun Han |
| Sungjib Lim | Suvadip Sana | Taehyung Kim | Taichiro Endo |
| Taposh Dutta Roy | Tianyi Qiu | Tiffany Wang | Toryn Q. Klassen |
| Tyna Eloundou | Tzu-Sheng Kuo | Utkarsh Agarwal | Vincent Huang |
| Weichen Li | Woojung Song | Xiao Liu | Xing Zhang |
| Yadnyesh Chakane | Yanan Long | Yejin Son | YeongJun Hwang |
| Yi Chen | Yiran Jenny Shen | Yoonah Park | Youngjin Lee |
| Youngsam Chun | YuXin Song | Zakk Heile | Zezheng Lin |
| Zhaowei Zhang | Zhen Chen | Zijie Qiu | Zini Yang |
Contact us
Please email pluralistic-alignment@googlegroups.com if you have any questions.
Sponsors
