Back to Home
AWS ML Blog··Official Labs

Use prompt caching with open-weight models on Amazon Bedrock

中文摘要

Amazon Bedrock 为开源权重模型引入提示词缓存,通过 Chat Completions API 降低长上下文的输入成本与延迟。

English Summary

Amazon Bedrock now supports prompt caching for open-weight models via the Chat Completions API to reduce input costs and latency for long contexts.

Original Excerpt

Many generative AI applications resend the same long context on every request. Prompt caching lets open-weight models on Amazon Bedrock reuse it to reduce input cost and latency. This post shows implicit and explicit caching through the Chat Completions API and applies caching to an OpenCode coding agent session.