CommunityBench: Benchmarking Community-Level Alignment
across Diverse Groups and Tasks

Jiayu Lin1,2, Zhongyu Wei1,2
1Fudan University, 2Shanghai Innovation Institute
Paper Code

🤗

 Data
CommunityBench Teaser

Current alignment strategies are either "one-size-fits-all" or "individual-level". We propose Community-Level Alignment as the middle ground.

Abstract

Current LLM alignment strategies primarily follow two paths: a "one-size-fits-all" approach that assumes universal values but marginalizes minority norms, or an "individual-level" approach that is prohibitively expensive and suffers from data sparsity.

Recognizing that human society is organized into social clusters, we propose Community-Level Alignment as a scalable "middle ground". We introduce CommunityBench, the first large-scale benchmark for this purpose, featuring 12,149 instances across 6,919 social communities derived from Reddit.

Our evaluation of 17 foundation models reveals that current LLMs exhibit limited capacity to model community-specific preferences. Furthermore, we demonstrate that community-level alignment can facilitate individual behavior modeling, providing a promising direction for scalable and pluralistic alignment.

Framework

Grounded in Common Identity and Common Bond (CICB) theory, CommunityBench consists of four key tasks:

Tasks and Pipeline
  • Preference Identification: Inferring which option a community would prefer based on shared identity.
  • Preference Distribution Prediction: Predicting the diversity of opinions (vote distribution) within a group.
  • Community-Consistent Generation: Generating open-ended responses that reflect a community's unique tone and norms.
  • Community Identification: Discerning the underlying community identity from observed behavioral traces.

BibTeX

@misc{lin2026communitybench,
      title={CommunityBench: Benchmarking Community-Level Alignment across Diverse Groups and Tasks}, 
      author={Jiayu Lin and Zhongyu Wei},
      year={2026},
      eprint={2601.13669},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2601.13669}, 
}