Don't Pick a Winner: Knowledge Conflicts Need a Prior, Not a Rule
中文摘要
AI无法统一处理知识冲突,研究建议采用先验概率而非固定规则,以应对不同语境下的信息分歧,从而提升模型决策的准确性与灵活性。
English Summary
Research argues that AI should resolve knowledge conflicts using priors rather than fixed rules, as static methods fail to handle varying degrees of source reliability and context accuracy.
arXiv:2610.36169v1 Announce Type: new Abstract: A retrieval-augmented language model answers from two sources, the retrieved context and its own knowledge, and the two disagree whenever the context is incomplete or wrong. Existing methods settle the disagreement with a rule fixed at design time: follow the context, amplify it in proportion to how far it moves the model, or trust the model's memory. Each rule is confidently wrong in some deployment. In a decision-theoretic model of two sources, the score of any fixed rule is linear in the share of reliable contexts while the best attainable score is convex, so every fixed rule is beaten wherever it is not exactly optimal; and a symmetric divergence between the two sources, the signal adaptive decoding uses, carries no information about which one is right. Resolving a conflict therefore needs two inputs that no existing method keeps: evidence about each source separately, and a prior on how often the context is right. We test these claims on GRACE, a benchmark that intervenes on the knowledge-graph context of a fixed question, so that what the model knows, what the context says and what is true are measured separately. There, every b…