返回首页
arXiv AI··论文与技术

Tracing Psychometric Inference in Large Language Models

中文摘要

研究探讨大语言模型是内部表征心理构念还是仅模仿其结构,并测试其通过人类数据预测多种心理量表的能力。

English Summary

Researchers explored whether LLMs internally represent psychological constructs or merely mimic psychometric structures, testing their ability to predict various scales from human data.

原文节选

arXiv:2611.12937v1 Announce Type: new Abstract: Do large language models represent psychological constructs internally, or do they only generate outputs that resemble psychometric structure? We address this question using a cross-persona paradigm across 14 model variants from two families (Llama 3 and Qwen 2.5, 0.5B-14B, base and instruct). Given responses on one psychological scale from real individuals (N = 272 Taiwanese university students), models predict responses on six other scales. At the behavioral level, larger variants reproduce and amplify human cross-scale correlations; amplification persists after correcting the seven-scale human correlations for measurement attenuation. We then examine whether this structure is reflected internally. Contrastive direction analysis finds geometry aligned with the human matrix in larger instruct variants, but little alignment in base variants under this assay. Across models, geometry strength is associated with behavioral amplification (r = 0.68, p = 0.008; partial r = 0.69 controlling for log size). To relate internal structure to output behavior, a matched activation-probing paradigm shows that probe slopes vary less than behavioral s…