Measuring Where Intent Gets Lost in Agentic Smart Contract Synthesis
中文摘要
本研究探讨 AI 智能体合成智能合约时意图丢失的问题,旨在通过评估代码行为来识别用户意图与实际执行间的偏差。
English Summary
Researchers examine the gap between user intent and AI-generated smart contracts, developing methods to identify where behavioral discrepancies occur during automated code synthesis.
arXiv:2610.37830v1 Announce Type: new Abstract: AI agents can now write software from a plain-English description, but code that runs is not the same as code that does what was asked. This gap matters most for smart contracts, the programs behind Web3 applications that move money on a blockchain and cannot be fixed once deployed. We ask how to check whether an AI-written contract behaves as it should, and where the requested behavior gets lost. We design an evaluation built on execution rather than opinion. We replay the same transactions on an AI-written contract and on the original it should reproduce and measure how often they agree. We test every grading tool on planted mistakes, compare against an answer key written without the description, and audit our own scorer until known wrong answers cannot pass. Common methods, such as AI judges and standard analysis tools, miss obvious mistakes, and scoring choices matter too. Checking only the functions every system includes hid about half the difference between pipelines. We then propose an elicitation-first architecture. Before writing code, the agent asks the client a short checklist of questions, checks the contract against an in…