Prompt Caching Worked Perfectly. One Dynamic Timestamp Destroyed Every Cache Hit.
中文摘要
提示词缓存因前缀中包含动态时间戳而失效，即使看似可复用，服务器也会将其视为不同内容。
English Summary
Prompt caching fails when dynamic timestamps in prefixes make seemingly reusable prompts appear unique to inference servers.
原文节选
The expensive prefix looked reusable to humans and completely different to the inference server. Continue reading on Stackademic »