返回首页
arXiv AI··论文与技术

Cooperative Multi-Agent Reinforcement Learning via Epigraph-Form Guided Exploration

中文摘要

EFXPLORER通过上图形式引导探索,解决了多智能体强化学习中协作行为发现难的问题,避免了内在奖励对学习信号的干扰。

English Summary

EFXPLORER uses epigraph-form guided exploration to help multi-agent reinforcement learning discover cooperative behaviors while avoiding the learning signal distortion caused by naive intrinsic rewards.

原文节选

arXiv:2611.07066v1 Announce Type: new Abstract: Discovering cooperative behavior in multi-agent reinforcement learning (MARL) is challenging due to the combinatorial complexity of joint state-action spaces, which hinders the emergence of coordinated behaviors from trial-and-error alone. Intrinsic rewards are often used to aid discovery, but naively combining them with team objectives can distort the learning signal, compromising task performance. In this paper, we propose EFXPLORER, a constrained exploration framework that maximizes exploration objectives subject to a constraint that preserves established task performance. We solve the resulting problem using an epigraph reformulation that introduces adaptive exploration budgets. This approach separates intrinsic rewards from task objectives and regulates exploration through task feasibility. To further encourage diverse and temporally extended exploration, we incorporate a successor distance-based intrinsic reward that captures long-horizon dependencies. Empirically, our method outperforms strong baselines and induces novel cooperative strategies across SMAX, VMAS, and MPE benchmark suites.