UCSB NLP Group / Sept 2025 – Present

Persona Based Large Language Model Debate

Evaluation harness comparing single-shot, majority-vote, and persona-conditioned LLM debate on AIME 2025, GPQA, and HLE-Verified.

resume
timelineSep 2025 - Present
focusai, research, rl
stack
  • Python
  • Gemini API
  • AIME 2025
  • GPQA
  • HLE-Verified

problem

Multi-agent debate results are difficult to interpret when inference strategies, persona diversity, judges, and traces are not controlled or reproducible.

approach

Generate task-adaptive persona cards, run independent solves followed by multi-round debates, and compare majority and judge-selected answers using persisted artifacts and structured JSONL traces.

implementation notes

  • Built a reproducible evaluation harness comparing single-shot inference, majority voting, and multi-round persona-conditioned debate with a judge.
  • Generates per-question personas along fixed and task-specific diversity axes, then samples five complementary persona cards using maximin spacing.
  • Supports AIME 2025, GPQA, and HLE-Verified with domain-specific judge banks, persisted persona artifacts, JSONL traces, and cost accounting.

impact

  • Makes single, majority, and debate strategies directly comparable across three reasoning benchmarks
  • Preserves personas, judge banks, transcripts, token usage, and outputs for reproducible failure analysis