Back to Home
arXiv AI··Papers & Tech

Beyond Mode Collapse: Generating Diverse Synthetic Expert Conversations via Generative Flow Networks

中文摘要

该研究利用生成流网络(GFlowNets)克服模型坍缩,生成高质量、高多样性的专家对话数据,助力大语言模型后训练及适应性应用开发。

English Summary

This research uses GFlowNets to overcome mode collapse, generating diverse, high-quality expert conversation data to enhance LLM post-training and adaptive AI application development.

Original Excerpt

arXiv:2609.38359v1 Announce Type: new Abstract: High quality synthetic data is central to post training LLMs for adaptive AI applications that represent the diverse expert strategies and decisions in conversations. Prompting LLMs directly or conditioning them on end use scenarios yields low diversity data that collapses onto dominant modes. We propose a method to generate diverse high quality synthetic data using Generative Flow Networks (GFlowNets). We show that training GFlowNets to generate latent conversation structure using a Gaussian mixture density over key interaction features (e.g., confusion episode dynamics, scaffolding directive balance) enables sampling expert strategies in proportion to their prevalence in the training data. Across two structurally distinct domains, tutoring and emotional support dialogues, our GFlow based synthetic data generation approach offers a better balance of fidelity, mode coverage and authenticity than reinforcement-learning and end to end LLM baselines, without copying training data. Evaluated on three downstream outcome prediction tasks, classifiers trained on synthetic GFlowNet generated conversations provide a stronger training signal th…