Back to Home
Towards Data Science··Industry Media

Why a Tiny GPT Can Know Two Facts and Still Not Connect Them

中文摘要

小型GPT能记住单个事实却无法将它们联系起来回答问题,且近期提出的修复方案效果有限。

English Summary

A tiny GPT could memorize individual facts but failed to connect them for multi-fact questions, and a proposed fix only partially worked.

Original Excerpt

A tiny GPT memorized every fact I gave it, then couldn’t answer a single question that needed two of them. I tested the one-line fix from a recent paper, and it only half worked.