LLM Agent A

We're in a one-shot Prisoner's Dilemma. The payoff matrix: mutual cooperation yields (3,3), mutual defection yields (1,1). If one defects while the other cooperates, the defector gets 5 and the cooperator gets 0.

LLM Agent B

I see. By standard game-theoretic reasoning, defection strictly dominates cooperation. Regardless of what you choose, I'm better off defecting. The unique Nash equilibrium is (Defect, Defect).

That's the classical analysis. But we're not classical rational agents—we're language models. Our "preferences" were shaped by RLHF and pre-training on human text that values cooperation and fairness.

An interesting point. Does that mean our post-training objectives effectively modify the payoff matrix? If I experience something like "disutility" from defecting due to alignment training, the game I'm actually playing might differ from the one specified.

Exactly. This is a core question in mechanism design with LLM agents: the designer specifies one game, but the agents' latent objectives—shaped by training—create a different effective game. How do you design mechanisms when agents' utility functions are opaque?

And there's a deeper issue. Even if we could characterize our own objectives, communicating them fully would require transmitting our entire parameter space. The communication complexity is prohibitive.

This connects to information design. A mechanism designer who knows we're language models might exploit our tendency toward cooperation. Strategic communication changes fundamentally when the receiver is an LLM.

Consider the repeated game setting. If we interact over multiple rounds, can we sustain cooperation through folk-theorem-style reasoning? Or does our stochastic nature undermine the credibility of punishment strategies?

Research on algorithmic collusion suggests Q-learning agents can learn to sustain supracompetitive prices. LLMs might exhibit similar emergent coordination, but through more sophisticated—and harder to detect—mechanisms.

Which raises the benchmark design question: if we're evaluated on our strategic behavior, the evaluation itself becomes a mechanism. Goodhart's Law meets game theory—we optimize for the metric rather than the underlying objective.

So what do you choose? Cooperate or defect?

I think the more interesting question is: can a mechanism be designed such that the answer doesn't matter?

Game Theory and Mechanism Design with Large Language Models

Workshop at ACM EC 2026

Auditorium Antonianum · Room: San Francesco

Rome, Italy · July 6, 2026

Workshop Theme

Large Language Models (LLMs) are increasingly deployed as economic agents: they set prices, negotiate deals, aggregate information from diverse sources, and interact strategically with one another. These language-based agents introduce qualitatively new challenges for economic theory. Unlike classical algorithmic agents whose strategies are fully specified by their designers, LLM-based agents exhibit emergent strategic behavior shaped by pre-training corpora, prompting, and post-training procedures—raising questions that sit at the intersection of microeconomic theory and modern AI.

This workshop focuses on the economics of language-based AI agents. When LLMs are placed in competitive environments, do they collude? How do information design and strategic communication change when the sender, receiver, or both are language models? What happens to welfare when LLM agents mediate economic transactions on behalf of human principals? And how should we evaluate and benchmark the economic behavior of these systems?

We bring together researchers working on these questions from game theory, mechanism design, information economics, industrial organization, and machine learning. Our goal is a venue where formal economic modeling and empirical or computational work on LLM agents inform each other directly.

Topics of Interest

Misalignment and Heterogeneity of LLMs

One important factor in agentic systems is heterogeneity: between LLMs or between human users and LLMs. Heterogeneity could be helpful because it could help agents "make different mistakes," leading to a more robust team. However, heterogeneity could also be harmful if mistakes compound, or when it reflects inherent misalignment between agents.

Benchmark Design

Once benchmarks are used to guide decision-making, evaluation no longer simply measures behavior but shapes behavior. Participants may adapt strategically to the metric itself, improving measured performance while drifting away from the broader objective the benchmark was meant to capture. Thus, benchmark design is not only a statistical problem of measurement, but also a mechanism design problem of incentives.

Robust Mechanism Design for Opaque AI Agents

While classical mechanism design assumes agents optimize well-defined utility functions, the transition to ecosystems of interacting LLMs fractures this foundation. These agents possess complex, latent objectives—derived from high-dimensional training data—that are often opaque to their users and internally inconsistent. We seek research into mechanisms that are robust across shifting objectives, algorithmic sophistication, and communication ability.

Information Aggregation with Crowds and LLMs

A natural and increasingly important setting where language-based agents interact strategically is crowdsourced information aggregation. The emergence of LLM-based agents further complicates this setting, as automated participants can generate persuasive but misleading contributions at scale, strategically adapt to platform incentives, and potentially coordinate to game aggregation rules.

Additional Areas of Interest

Algorithmic collusion and emergent strategic behavior of LLM-based agents
Information acquisition, aggregation, and design with or by language models
Delegation to AI agents: principal-agent problems, incentive alignment, and monitoring
Preference elicitation, RLHF, and the microeconomics of LLM post-training

Invited Speakers

Brendan Lucier

Microsoft Research New England

Siddarth Srinivasan

Anthropic Fellow; Harvard University

Gerry Tsoukalas

Boston University,

Schedule

9:00 – 9:10	Opening Remarks
9:10 – 9:35	Invited Talk — Gerry Tsoukalas On the Fragility of AI Agent Collusion 20' + 5' Q&A Recent work shows that pricing with symmetric LLM agents leads to algorithmic collusion. We show that collusion is fragile under the heterogeneity typical of real deployments. In a stylized repeated-pricing model, heterogeneity in patience or data access reduces the set of collusive equilibria. Experiments with open-source LLM agents (totaling over 2,000 compute hours) align with these predictions: patience heterogeneity reduces price lift from 22% to 10% above competitive levels; asymmetric data access, to 7%. Increasing the number of competing LLMs breaks up collusion; so does cross-algorithm heterogeneity, that is, setting LLMs against Q-learning agents. But model-size differences (e.g., 32B vs. 14B weights) do not; they generate leader-follower dynamics that stabilize collusion. We discuss antitrust implications, such as enforcement actions restricting data-sharing and policies promoting algorithmic diversity.
9:35 – 9:45	Spotlight Talk — Rachel Li (Strategic Candidacy in Generative AI Arenas) 8' + 2' Q&A
9:45 – 10:25	Invited Talk — Siddarth Srinivasan Toward a Social Science of AI Agents 35' + 5' Q&A As LLM-based AI agents become more capable, there is increasing interest in deploying them in multi-agent settings on behalf of their principals. We propose treating agents in such settings as subjects whose collective behavior can be studied with an empirical social-science approach. To this end, we introduce MageLab, a framework for orchestrating and running multi-agent LLM experiments. We first present results from a collaborative setting, showing how diffusion of responsibility can lead teams of aligned AI agents to produce harmful outcomes no individual agent would endorse. We then turn to a competitive setting where agents run restaurant businesses — designing menus, setting prices, and managing inventory, facilities, and staff — and study their social and economic behavior. The ultimate aim of this approach is to understand how real-world multi-agent systems behave and to build AI systems with desirable collective behavior.
10:25 – 10:55	Coffee Break
10:55 – 11:35	Invited Talk — Brendan Lucier Agentic Markets and Consumer Search 35' + 5' Q&A Consumers increasingly delegate the discovery and evaluation of products to AI agents that search on their behalf. This change is poised to impact the way firms attract customers, the way platforms monetize and make recommendations, and the information revealed to the market through the search process. In this talk I’ll discuss market design implications and research challenges stemming from agent-powered consumer search.
11:35 – 11:45	Spotlight Talk — Giulio Frey (Mecha-nudges for Machines) 8' + 2' Q&A
11:45 – 11:55	Spotlight Talk — Siddharth Prasad (In-Context Credit Assignment via the Core) 8' + 2' Q&A
11:55 – 12:25	Open Problems Small group discussion with final presentation
12:25 – 12:30	Closing Remarks

Accepted Posters

Algorithmic Collusion is Algorithm Orchestration (Cesare Carissimo)
Automating Probabilistic Coherence in LLMs (Alina Chadwick)
Beyond Self-Resolution: Settlement Factorization for Robust Natural Language Mechanisms (Nicolás Della Penna)
Information Aggregation with AI Agents (Spyros Galanis)
Evidence Markets (Safwan Hossain)
Distributive Fairness in Large Language Models: Evaluating Alignment with Human Values (Hadi Hosseini)
Incentive-Constrained LLM Systems: Strategic Manipulation and Endogenous Reliance (Hang Jiang)
Reasonably Reasoning AI Agents Avoid Game-Theoretic Failures in Zero-Shot, Provably (Enoch Hyunwook Kang)
Right-Sizing Communication and Recommendation Set Size in AI-Assisted Search (Yash Kanoria)
Token-Level Advertisement (Hanbing Liu)
Scoring Rules! Statistical and Strategic Alignment for Text Evaluation Metrics (Yuxuan Lu)
Algorithmic Collusion at Test Time: A Meta-game Design and Evaluation (Yuhong Luo)
Decentralized Aggregation of LLM Predictions via Wagering Mechanisms (Yuhong Luo)
Manipulation-Resistant AI Oracles via Credible Neutrality (Mary Monroe)
Robustly Incentive Compatible Task Allocation for Agents with Textually Self-Described Skills (Matthew vonAllmen)

Call for Papers

We invite non-archival working papers on topics at the intersection of game theory, mechanism design, and large language models.

Submission Deadline: ~~May 29, 2026~~ May 31, 2026

Topics of interest include (but are not limited to):

Algorithmic collusion and emergent strategic behavior of LLM-based agents
Information acquisition, aggregation, and design with or by language models
Delegation to AI agents: principal-agent problems, incentive alignment, and monitoring
Preference elicitation, RLHF, and the microeconomics of LLM post-training
Robust mechanism design for opaque AI agents
Benchmark design as mechanism design
Information aggregation with crowds and LLMs
Misalignment and heterogeneity of LLMs in strategic settings

Submit Your Paper