返回首页
arXiv AI··论文与技术

Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds

中文摘要

该研究探讨通过持续预训练(CPT)在数据量、多样性和模型规模上进行扩展,以提升具身智能体在多样化虚拟世界中的性能。

English Summary

Research explores how scaling data, diversity, and model size via continued pre-training (CPT) enhances embodied agent learning across diverse virtual worlds with varying styles and dynamics.

原文节选

arXiv:2610.28698v1 Announce Type: new Abstract: Scaling data, diversity and model capacity has improved embodied learning in physical environments, where diverse experiences remain grounded in shared physical laws. Whether these gains extend to virtual worlds with different visual styles, dynamics, controls, and objectives remains unclear. We investigate this question through a controlled study of continued pre-training (CPT) across three axes: data volume, diversity, and model size. Our 120 training configurations span four model sizes from 0.8B to 27B, six data budgets from 50 to 1,600 hours, and five mixtures ranging from Minecraft-only data to diverse multi-world corpora. We assess performance and generalization using three-level validation losses and gameplay benchmarks from in-corpus worlds to unseen worlds. Our results show: (1) more data improves in-corpus performance, diversity improves transfer to related worlds with limited gains in distant worlds, and model size improves data efficiency but generalizes worse to distant worlds under single-world training; (2) we observe an early interface phase with broad cross-world benefits and a later world phase with gains narrowed t…