GenAI Developer
Amazon Bedrock Provisioned Throughput vs Cross-Region Inference, Batch and Prompt Caching
Provisioned Throughput buys dedicated model units at a fixed hourly price; cross-Region inference spreads on-demand requests across Regions at no extra routing cost; batch inference runs prompts asynchronously from S3; prompt caching reuses a repeated prompt prefix; intelligent prompt routing picks between two models of a family. Inference profiles currently do not support Provisioned Throughput.
The options at a glance
| Option | Use it when | Key facts |
|---|---|---|
| Provisioned Throughput | You need guaranteed capacity or must use a customized model | Hourly price per model unit; no commitment, 1-month or 6-month terms; required for customized models. |
| Cross-Region inference | On-demand traffic needs more capacity across Regions | Inference profile; geographic (US, EU, APAC) or global; priced by the source Region. |
| Batch inference | Results are not needed in real time | Input files in S3, asynchronous job, output in S3; not supported for provisioned models. |
| Prompt caching | Many requests share a long, static prefix | Lower latency and input-token cost; on-demand only, not with batch inference. |
| Intelligent prompt routing | Some requests are simple and some are complex | Routes between two models of the same family; optimized for English prompts. |
Source: docs.aws.amazon.com/bedrock/latest/userguide/prov-throughput.html
Provisioned Throughput
You buy model units (MUs); each MU delivers a set number of input and output tokens per minute. Billing is hourly and continues until you delete it. A longer commitment (1 or 6 months) lowers the hourly price, but you cannot delete it before the term ends. A customized model must use Provisioned Throughput.
Cross-Region inference: geographic vs global
| Geographic | Global | |
|---|---|---|
| Data residency | Inside the geography (US, EU, APAC) | Any supported commercial Region |
| Cost | Standard pricing | About 10% savings |
| SCP requirement | Allow every destination Region in the profile | Allow aws:RequestedRegion equal to unspecified |
Source: docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html
There is no extra routing cost, data stays on the AWS network, and CloudTrail logs the request in the source Region with `additionalEventData.inferenceRegion`. Inference profiles currently do not support Provisioned Throughput.
Prompt caching
- Put static content first and dynamic content last; any change to the cached prefix is a cache miss.
- Checkpoints have a model-specific minimum (512 to 4,096 tokens) and the common 5-minute TTL resets on every hit.
- Checkpoints are processed tools, then system, then messages; changing an earlier section invalidates later ones.
- Total input tokens = inputTokens + cacheReadInputTokens + cacheWriteInputTokens.
Intelligent prompt routing and batch inference
A prompt router predicts the response quality of each model for a request and sends it to the better quality-and-cost choice. Default routers use Amazon Nova and Meta Llama models; a configured router sets a fallback model and a response quality difference. Batch inference suits large offline jobs: upload inputs to S3, create a job and read the outputs from S3.
Why it matters for AIP-C01
Domain 4 (12%) covers cost and performance, and Domain 2 (26%) covers model deployment strategies. See Domain 4 and the study topic throughput, cross-Region inference and routing.
Frequently asked questions
Can I use Provisioned Throughput with a cross-Region inference profile?
No. AWS says inference profiles currently don't support Provisioned Throughput.
Does cross-Region inference cost more?
No extra routing cost: you pay the price of the Region you call from. Global profiles save about 10%.
Can I use prompt caching with batch inference?
No. Prompt caching is supported for on-demand inference, not with the batch inference API.
Do I need Provisioned Throughput for a customized model?
Yes. AWS says you must purchase Provisioned Throughput to use a model you customized.
Sources
- Provisioned Throughput (AWS)
- Cross-Region inference (AWS)
- Prompt caching (AWS)
- Intelligent prompt routing (AWS)
- Batch inference (AWS)
AWS updates its programs and documentation. Check the linked official pages for the latest details before you decide or pay.
Practice for AIP-C01
Free AWS Generative AI Developer – Professional practice questions with an explanation for every option. No sign-up.
Keep reading
- GenAI Developer
AIP-C01 Study Plan: Pass AWS GenAI Developer Pro
A 4-week AIP-C01 study plan built on the official exam guide: domain weights, what is out of scope, the format and how to practice for each domain.
3 min read - GenAI Developer
Amazon Bedrock Guardrails Explained: Filters, PII, Grounding
How Amazon Bedrock Guardrails work: content filters, denied topics, word filters, PII block or mask, contextual grounding thresholds and IAM enforcement.
3 min read - GenAI Developer
Bedrock Knowledge Base Chunking Strategies Compared
Compare chunking in Amazon Bedrock Knowledge Bases: default, fixed-size, hierarchical, semantic and no chunking, plus hybrid search, reranking and filters.
2 min read