Why a Tiny GPT Can Know Two Facts and Still Not Connect Them
中文摘要
小型GPT能记住单个事实却无法将它们联系起来回答问题,且近期提出的修复方案效果有限。
English Summary
A tiny GPT could memorize individual facts but failed to connect them for multi-fact questions, and a proposed fix only partially worked.
原文节选
A tiny GPT memorized every fact I gave it, then couldn’t answer a single question that needed two of them. I tested the one-line fix from a recent paper, and it only half worked.