From c4ba2fdf5fa5d693e82ca4d7fede2f7fa96788d1 Mon Sep 17 00:00:00 2001 From: C Naresh Date: Wed, 2 Sep 2026 23:58:32 +0530 Subject: [PATCH 1/5] docs: add cookbook for AWS Bedrock cost attribution with STS session tags Add a comprehensive guide covering end-to-end setup for forwarding Portkey metadata as AWS STS session tags during Bedrock AssumeRole calls. Covers gateway configuration, IAM permissions (caller + target trust policy), SDK/cURL examples, CloudTrail verification, Cost Explorer activation, tag sanitisation constraints, credential caching, and troubleshooting common errors. Also updates the existing assumed-role doc with minimum gateway version, clearer IAM guidance, and a cross-link to the new cookbook. --- docs.json | 2 + ...cost-attribution-with-sts-session-tags.mdx | 376 ++++++++++++++++++ ...nnect-bedrock-with-amazon-assumed-role.mdx | 8 +- 3 files changed, 382 insertions(+), 4 deletions(-) create mode 100644 guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx diff --git a/docs.json b/docs.json index 1170f317b..1d85a43ae 100644 --- a/docs.json +++ b/docs.json @@ -1365,6 +1365,7 @@ "guides/use-cases/comparing-top10-lmsys-models-with-portkey", "guides/use-cases/metadata-use-cases", "guides/use-cases/track-costs-using-metadata", + "guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags", "guides/use-cases/deepseek-r1", "guides/use-cases/openai-computer-use", "guides/use-cases/private-mcp-servers", @@ -2375,6 +2376,7 @@ "guides/use-cases/comparing-top10-lmsys-models-with-portkey", "guides/use-cases/metadata-use-cases", "guides/use-cases/track-costs-using-metadata", + "guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags", "guides/use-cases/enforcing-limits-and-budgets", "guides/use-cases/deepseek-r1", "guides/use-cases/openai-computer-use", diff --git a/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx b/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx new file mode 100644 index 000000000..3708934d4 --- /dev/null +++ b/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx @@ -0,0 +1,376 @@ +--- +title: "AWS Bedrock Cost Attribution with STS Session Tags" +description: "Attribute Amazon Bedrock costs per application, team, or environment by forwarding Portkey metadata as AWS STS session tags through a shared assumed role." +--- + +LLM gateways typically share a single IAM role across all tenants. Without per-request tagging, every Bedrock call attributes to the same identity in AWS Cost Explorer — making cost breakdowns impossible. + +Portkey solves this by forwarding the `x-portkey-metadata` header as [AWS STS session tags](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html) during `AssumeRole`. Each unique tag set produces a distinct STS session, so AWS records cost and usage per application, team, or environment — no extra IAM roles required. + +## How It Works + +``` +┌──────────┐ x-portkey-metadata ┌─────────────────┐ Tags.member.N ┌─────────┐ +│ Client │ ───────────────────────▶ │ Portkey Gateway │ ───────────────▶ │ AWS STS │ +│ │ {"app":"chatbot", │ │ AssumeRole + │ │ +│ │ "team":"platform"} │ (self-hosted) │ SessionTags │ │ +└──────────┘ └─────────────────┘ └────┬─────┘ + │ + Temp credentials + with session tags + │ + ▼ + ┌──────────────┐ + │ Amazon Bedrock │ + │ │ + │ InvokeModel │ + └──────────────┘ + │ + Tags land in CUR 2.0 & Cost Explorer +``` + +1. Client sends `x-portkey-metadata` with key-value pairs on each request. +2. Gateway parses the metadata, sanitises it (drops reserved `aws:` prefixes, enforces AWS limits), and passes the pairs as `Tags.member.N.Key / Tags.member.N.Value` in the STS `AssumeRole` POST. +3. AWS STS issues temporary credentials tagged with those values. +4. Gateway uses those credentials to call Bedrock `InvokeModel`. +5. Tags appear in **CUR 2.0** (`iamPrincipal/` prefix), **CloudTrail**, and **Cost Explorer**. + +## Prerequisites + +| Requirement | Details | +|---|---| +| **Gateway version** | `≥ 2.19.0` | +| **Auth type** | Bedrock with **AWS Assumed Role** ([setup guide](/product/model-catalog/connect-bedrock-with-amazon-assumed-role)) | +| **Deployment** | Self-hosted (ECS, EKS, EC2, or Docker) | + +--- + +## Step 1: Enable Session Tags on the Gateway + +Set the environment variable on the gateway container: + +```sh +AWS_BEDROCK_STS_SESSION_TAGS_ENABLED=true +``` + + +```json +{ + "containerDefinitions": [{ + "name": "portkey-gateway", + "environment": [ + {"name": "AWS_BEDROCK_STS_SESSION_TAGS_ENABLED", "value": "true"} + ] + }] +} +``` + + + +```yaml +containers: + - name: portkey-gateway + env: + - name: AWS_BEDROCK_STS_SESSION_TAGS_ENABLED + value: "true" +``` + + + +```yaml +services: + portkey-gateway: + environment: + - AWS_BEDROCK_STS_SESSION_TAGS_ENABLED=true +``` + + +Restart or redeploy the gateway after adding the variable. + +--- + +## Step 2: Configure IAM Permissions on AWS + +Session tags require `sts:TagSession` in **two places**: the caller's IAM policy and the target role's trust policy. Missing either one causes an `AccessDenied` error. + +### 2a. Caller principal (the gateway's execution role) + +The IAM role the gateway runs under (e.g. an ECS task role, an EC2 instance profile, or a Kubernetes service account role) must be allowed to both assume and tag the target Bedrock role. + +Add or update the policy attached to the **caller role**: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "sts:AssumeRole", + "sts:TagSession" + ], + "Resource": "arn:aws:iam:::role/" + } + ] +} +``` + +Replace `` and `` with the actual values for the role Portkey assumes to invoke Bedrock. + + +**Cross-account setups:** When the gateway and the Bedrock role live in different AWS accounts, both the caller permissions (in the gateway account) and the target trust policy (in the Bedrock account) must allow `sts:TagSession`. This is the most common source of `AccessDenied` errors. + + +### 2b. Target role trust policy (the Bedrock invocation role) + +Open the target role in IAM, go to **Trust relationships → Edit trust policy**, and ensure the `Action` includes both `sts:AssumeRole` and `sts:TagSession`: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "AWS": "arn:aws:iam:::role/" + }, + "Action": [ + "sts:AssumeRole", + "sts:TagSession" + ], + "Condition": { + "StringEquals": { + "sts:ExternalId": "" + } + } + } + ] +} +``` + + +The `Condition` block is optional but recommended. If you configured an external ID when setting up the assumed role in Portkey, include it here. + + +### 2c. Bedrock invocation permissions (no change needed) + +The existing permission policy on the target role (`bedrock:InvokeModel`, `bedrock:InvokeModelWithResponseStream`) does not need modification. Session tags only affect the `AssumeRole` call, not the Bedrock API calls themselves. + +--- + +## Step 3: Send Metadata with Requests + +Pass key-value pairs in the `x-portkey-metadata` header (or the SDK `metadata` option). Every key-value pair becomes an STS session tag. + + + +```python Python +from portkey_ai import Portkey + +portkey = Portkey( + api_key="PORTKEY_API_KEY", + provider="bedrock" +) + +response = portkey.with_options( + metadata={ + "app": "chatbot", + "team": "ai-platform", + "env": "production", + "cost_center": "CC-1234" + } +).chat.completions.create( + model="anthropic.claude-sonnet-4-20250514-v1:0", + messages=[{"role": "user", "content": "Hello"}] +) + +print(response.choices[0].message) +``` + +```js JavaScript +import Portkey from "portkey-ai" + +const portkey = new Portkey({ + apiKey: "PORTKEY_API_KEY", + provider: "bedrock" +}) + +const response = await portkey.chat.completions.create({ + model: "anthropic.claude-sonnet-4-20250514-v1:0", + messages: [{ role: "user", content: "Hello" }] +}, { + metadata: { + app: "chatbot", + team: "ai-platform", + env: "production", + cost_center: "CC-1234" + } +}) + +console.log(response.choices) +``` + +```sh cURL +curl https://your-gateway.example.com/v1/chat/completions \ + -H "Content-Type: application/json" \ + -H "x-portkey-api-key: $PORTKEY_API_KEY" \ + -H "x-portkey-provider: bedrock" \ + -H 'x-portkey-metadata: {"app":"chatbot","team":"ai-platform","env":"production","cost_center":"CC-1234"}' \ + -d '{ + "model": "anthropic.claude-sonnet-4-20250514-v1:0", + "messages": [{"role": "user", "content": "Hello"}] + }' +``` + + + +--- + +## Step 4: Verify Tags in CloudTrail + +After sending a tagged request, confirm the tags appear in the `AssumeRole` CloudTrail event: + +```sh +aws cloudtrail lookup-events \ + --region \ + --lookup-attributes AttributeKey=EventName,AttributeValue=AssumeRole \ + --max-results 5 \ + --output json \ + --query "Events[?contains(CloudTrailEvent, '')].CloudTrailEvent" \ + | jq -r '.[] | fromjson | {eventTime, requestParameters}' +``` + +A successful tagged request shows a `tags` array in `requestParameters`: + +```json +{ + "eventTime": "2026-09-02T10:15:30Z", + "requestParameters": { + "roleArn": "arn:aws:iam::123456789012:role/portkey-bedrock-invokemodel", + "roleSessionName": "202692", + "externalId": "...", + "tags": [ + {"key": "app", "value": "chatbot"}, + {"key": "team", "value": "ai-platform"}, + {"key": "env", "value": "production"}, + {"key": "cost_center", "value": "CC-1234"} + ] + } +} +``` + +If the `tags` field is absent, double-check: +- Gateway version is `≥ 2.19.0` +- `AWS_BEDROCK_STS_SESSION_TAGS_ENABLED=true` is set +- The gateway was restarted after adding the variable + +--- + +## Step 5: View Costs in AWS Cost Explorer + +Session tags flow into **AWS Cost and Usage Reports (CUR 2.0)** with the `iamPrincipal/` prefix. To view them in Cost Explorer: + +1. Open **Billing and Cost Management → Cost Allocation Tags**. +2. Filter for **User-defined cost allocation tags**. +3. Locate your tag keys (e.g. `app`, `team`, `env`) and click **Activate**. + + +Tags take up to **24 hours** to appear after the first tagged API call, and a further **24 hours** to activate in Cost Explorer. + + +4. In **Cost Explorer**, group or filter by **Tag** → select your activated tag key. + +This breaks down Bedrock spend by application, team, environment, or any other dimension passed in metadata — all through a single shared IAM role. + +--- + +## Tag Constraints and Sanitisation + +The gateway automatically sanitises metadata before passing it to AWS STS. Understanding the constraints helps avoid silent tag drops. + +| Constraint | Limit | Gateway behavior | +|---|---|---| +| Maximum tags per request | 50 | Excess tags silently dropped | +| Key length | 128 characters | Truncated to 128 | +| Value length | 256 characters | Truncated to 256 | +| `aws:` key prefix | Reserved by AWS | Dropped silently | +| Empty keys or values | Not allowed | Dropped silently | +| Non-scalar values (objects, arrays) | Not supported | Dropped during metadata parsing | +| Value types | String only | Numbers and booleans auto-coerced to strings | + +**Source:** [AWS STS session tag limits](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_iam-quotas.html), [AssumeRole API reference](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) + +--- + +## Credential Caching + +The gateway caches STS credentials per unique combination of `(roleArn, externalId, region, sourceCredentialHash, tagsHash)`. Different metadata tag sets produce different cache keys, so: + +- Two requests with **identical metadata** reuse the same cached credentials (no extra STS call). +- Two requests with **different metadata** trigger separate `AssumeRole` calls and produce distinct STS sessions. + +This ensures cost attribution accuracy while minimizing STS API usage. + +--- + +## Troubleshooting + + + +This is the most common error. The full message looks like: + +``` +User: arn:aws:sts::884994594706:assumed-role/ecsTaskContainerRole/... +is not authorized to perform: sts:TagSession on resource: +arn:aws:iam::642137768492:role/portkey-bedrock-invokemodel +``` + +**Fix:** Add `sts:TagSession` to **both** the caller's IAM policy (Step 2a) **and** the target role's trust policy (Step 2b). In cross-account setups, both accounts must allow the action. + + + + +Possible causes: + +1. **Gateway version too old** — Confirm `>= 2.19.0` by calling `GET /health`. +2. **Feature flag not set** — Verify `AWS_BEDROCK_STS_SESSION_TAGS_ENABLED=true` is in the container environment. +3. **Gateway not restarted** — Environment variables are read at startup. Redeploy after adding the variable. +4. **Metadata not sent** — Confirm the `x-portkey-metadata` header is present in Portkey's request inspector. +5. **Cached credentials** — Previously cached (untagged) credentials may still be in use. Wait for the credential TTL to expire or restart the gateway to clear the cache. + + + + +1. Tags must be **activated** as cost allocation tags in the Billing console (Step 5). +2. Activation takes up to **24 hours** after the first tagged Bedrock call. +3. Cost Explorer data may lag an additional **24 hours**. +4. Ensure you are looking under **User-defined cost allocation tags**, not IAM principal type tags. + + + + +If Bedrock requests fail immediately after upgrading with the feature flag enabled, the most likely cause is missing `sts:TagSession` permissions. As a quick mitigation: + +1. Disable the feature flag: set `AWS_BEDROCK_STS_SESSION_TAGS_ENABLED` to `false` (or remove it). +2. Restart the gateway. +3. Fix the IAM permissions per Step 2, then re-enable. + + + + +STS session tags do **not** survive role chaining by default. If the gateway performs a two-hop assumption (source role → target role), tags passed on the first hop are dropped on the second unless marked as [transitive](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html#id_session-tags_role-chaining). The gateway does not currently set `TransitiveTagKeys`. + + +--- + +## Reference + +- [AWS: Pass session tags in STS](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html) +- [AWS: AssumeRole API — Tags parameter](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) +- [AWS: IAM principal cost attribution for Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/cost-mgmt-iam-principal-tracking.html) +- [AWS: Using IAM principal for cost allocation](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/iam-principal-cost-allocation.html) +- [Portkey: Connect Bedrock with Assumed Role](/product/model-catalog/connect-bedrock-with-amazon-assumed-role) + +import PrismaAirsCta from "/snippets/prisma-airs-cta.mdx"; + + diff --git a/product/model-catalog/connect-bedrock-with-amazon-assumed-role.mdx b/product/model-catalog/connect-bedrock-with-amazon-assumed-role.mdx index 0b7e58547..d1d6b4062 100644 --- a/product/model-catalog/connect-bedrock-with-amazon-assumed-role.mdx +++ b/product/model-catalog/connect-bedrock-with-amazon-assumed-role.mdx @@ -125,10 +125,10 @@ Once the role is created, copy the role ARN and paste it into the Bedrock integr ## Attribute Cost and Usage with Session Tags -You can attribute Bedrock cost and audit trails to individual applications, teams, or environments while sharing a single assumed role. Any metadata you send on a request is forwarded to AWS as [STS session tags](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html), so you can segregate spend in AWS Cost Explorer and CloudTrail without provisioning a separate IAM role per application. +Attribute Bedrock cost and audit trails to individual applications, teams, or environments while sharing a single assumed role. Any metadata sent on a request is forwarded to AWS as [STS session tags](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html), so spend can be segregated in AWS Cost Explorer and CloudTrail without provisioning a separate IAM role per application. -This feature requires the `AWS_BEDROCK_STS_SESSION_TAGS_ENABLED` environment variable to be set to `true` on the gateway container. +This feature requires gateway version **≥ 2.19.0** and the `AWS_BEDROCK_STS_SESSION_TAGS_ENABLED` environment variable set to `true` on the gateway container. Pass the tags using the `metadata` field (or the `x-portkey-metadata` header): @@ -139,11 +139,11 @@ curl https://api.portkey.ai/v1/chat/completions \ -H "x-portkey-api-key: $PORTKEY_API_KEY" \ -H "x-portkey-provider: $PROVIDER_SLUG" \ -H 'x-portkey-metadata: {"app": "chatbot", "team": "platform", "env": "prod"}' \ - -d '{ "model": "anthropic.claude-3-5-sonnet-20240620-v1:0", "messages": [{"role": "user", "content": "Hello"}] }' + -d '{ "model": "anthropic.claude-sonnet-4-20250514-v1:0", "messages": [{"role": "user", "content": "Hello"}] }' ``` -For AWS to accept session tags, add the `sts:TagSession` action to the trust policy of the role Portkey assumes, alongside the existing `sts:AssumeRole` action. +For AWS to accept session tags, add the `sts:TagSession` action to **both** the caller principal's IAM policy **and** the target role's trust policy. See the [full setup cookbook](/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags) for step-by-step IAM configuration, CloudTrail verification, and Cost Explorer activation. From a2b3920a0ef91e24597d80836a66a70d258cfa6a Mon Sep 17 00:00:00 2001 From: C Naresh Date: Thu, 3 Sep 2026 00:16:50 +0530 Subject: [PATCH 2/5] docs: redact client ARNs and expand credential caching section - Replace real AWS account IDs and role names in the troubleshooting error example with generic placeholders - Expand the Credential Caching section with: cache key composition table, TTL details, cache backend (Redis vs in-memory), when new AssumeRole calls happen, STS call volume estimation formula, and a note about throttling for high-cardinality metadata --- ...cost-attribution-with-sts-session-tags.mdx | 56 +++++++++++++++++-- 1 file changed, 50 insertions(+), 6 deletions(-) diff --git a/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx b/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx index 3708934d4..896b3f63f 100644 --- a/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx +++ b/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx @@ -304,12 +304,56 @@ The gateway automatically sanitises metadata before passing it to AWS STS. Under ## Credential Caching -The gateway caches STS credentials per unique combination of `(roleArn, externalId, region, sourceCredentialHash, tagsHash)`. Different metadata tag sets produce different cache keys, so: +The gateway caches STS temporary credentials to minimize `AssumeRole` API calls. No custom cache configuration is needed — caching works out of the box. -- Two requests with **identical metadata** reuse the same cached credentials (no extra STS call). -- Two requests with **different metadata** trigger separate `AssumeRole` calls and produce distinct STS sessions. +### Cache key composition -This ensures cost attribution accuracy while minimizing STS API usage. +Each cached credential is keyed by the combination of: + +| Parameter | Source | Effect on cache | +|---|---|---| +| `roleArn` | Virtual key / provider config | Different target roles → separate cache entries | +| `externalId` | Virtual key / provider config | Different external IDs → separate cache entries | +| `region` | Virtual key / provider config / `AWS_ASSUME_ROLE_REGION` env | Different regions → separate cache entries | +| `sourceCredentialHash` | SHA-256 of the gateway's own credentials (access key + secret key + session token) | If the gateway's source credentials rotate, the cache naturally invalidates | +| `tagsHash` | FNV-1a hash of the sanitised metadata key-value pairs | **Different metadata → different cache entries → separate STS sessions** | + +The cache key format is: + +``` +assumed-sts-//// +``` + +Parameters **not** in the cache key: model name, Portkey API key, virtual key slug, request body. These do not affect the STS call and are irrelevant to credential identity. + +### TTL and reuse + +| Setting | Value | Configurable? | +|---|---|---| +| Credential cache TTL | **5 minutes** (300 seconds) | Not configurable via env — hardcoded in the gateway | +| Cache backend | **Redis** (with 30-second local in-process cache) when Redis is configured; **in-memory** otherwise | Follows the gateway's `CACHE_STORE` setting | +| Local cache layer | When using Redis, a 30-second in-process local cache avoids a Redis round-trip on every request | Automatic when Redis is configured | + +### When does a new AssumeRole call happen? + +A new STS `AssumeRole` call is made **only** when no cached credential exists for the computed cache key. In practice: + +- **Same metadata, same role** — Reuses cached credentials for up to 5 minutes. One STS call per 5-minute window. +- **Different metadata, same role** — Each unique metadata set triggers its own STS call and produces a separate cached entry. For example, 3 distinct `team` values means 3 STS calls (then cached for 5 min each). +- **Gateway restart** — Clears the in-memory cache. Redis cache (if configured) survives restarts. +- **Source credential rotation** — When the gateway's own credentials change (e.g. ECS task role refresh), the `sourceCredentialHash` changes and new STS calls are made. + +### Estimating STS call volume + +``` +STS calls ≈ (unique metadata combinations × unique roles × unique regions) / 5 min +``` + +For a gateway serving 3 teams (`ai-platform`, `mobile`, `backend`) through 1 Bedrock role in 1 region, expect ~3 STS calls every 5 minutes — regardless of request volume. + + +AWS STS has no hard per-second rate limit, but sustained high call rates may trigger throttling. The 5-minute cache keeps call volume well within safe bounds for typical deployments. If you have hundreds of unique metadata combinations, consider limiting the metadata keys sent as session tags to a small set of high-cardinality dimensions (e.g. `team` and `env`, not per-request IDs). + --- @@ -320,9 +364,9 @@ This ensures cost attribution accuracy while minimizing STS API usage. This is the most common error. The full message looks like: ``` -User: arn:aws:sts::884994594706:assumed-role/ecsTaskContainerRole/... +User: arn:aws:sts:::assumed-role//... is not authorized to perform: sts:TagSession on resource: -arn:aws:iam::642137768492:role/portkey-bedrock-invokemodel +arn:aws:iam:::role/ ``` **Fix:** Add `sts:TagSession` to **both** the caller's IAM policy (Step 2a) **and** the target role's trust policy (Step 2b). In cross-account setups, both accounts must allow the action. From eac9d20db09db02c85d5d1ae771181c1b43c3416 Mon Sep 17 00:00:00 2001 From: C Naresh Date: Mon, 7 Sep 2026 21:26:40 +0530 Subject: [PATCH 3/5] docs: remove internal cache key format per review feedback Remove the raw cache key format string (internal implementation detail) while keeping the parameter table and 'not in cache key' clarification that are useful for users. --- .../aws-bedrock-cost-attribution-with-sts-session-tags.mdx | 6 ------ 1 file changed, 6 deletions(-) diff --git a/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx b/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx index 896b3f63f..014f29fb8 100644 --- a/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx +++ b/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx @@ -318,12 +318,6 @@ Each cached credential is keyed by the combination of: | `sourceCredentialHash` | SHA-256 of the gateway's own credentials (access key + secret key + session token) | If the gateway's source credentials rotate, the cache naturally invalidates | | `tagsHash` | FNV-1a hash of the sanitised metadata key-value pairs | **Different metadata → different cache entries → separate STS sessions** | -The cache key format is: - -``` -assumed-sts-//// -``` - Parameters **not** in the cache key: model name, Portkey API key, virtual key slug, request body. These do not affect the STS call and are irrelevant to credential identity. ### TTL and reuse From 579146f1c56345dbc70a2c4f5585a8af17e088de Mon Sep 17 00:00:00 2001 From: C Naresh Date: Tue, 8 Sep 2026 04:11:24 +0530 Subject: [PATCH 4/5] docs: replace ASCII art with Mermaid sequence diagram Replace the box-drawing character diagram with a Mermaid sequenceDiagram that Mintlify renders natively with interactive zoom/pan controls. --- ...cost-attribution-with-sts-session-tags.mdx | 33 +++++++++---------- 1 file changed, 15 insertions(+), 18 deletions(-) diff --git a/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx b/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx index 014f29fb8..6eab3be91 100644 --- a/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx +++ b/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx @@ -9,24 +9,21 @@ Portkey solves this by forwarding the `x-portkey-metadata` header as [AWS STS se ## How It Works -``` -┌──────────┐ x-portkey-metadata ┌─────────────────┐ Tags.member.N ┌─────────┐ -│ Client │ ───────────────────────▶ │ Portkey Gateway │ ───────────────▶ │ AWS STS │ -│ │ {"app":"chatbot", │ │ AssumeRole + │ │ -│ │ "team":"platform"} │ (self-hosted) │ SessionTags │ │ -└──────────┘ └─────────────────┘ └────┬─────┘ - │ - Temp credentials - with session tags - │ - ▼ - ┌──────────────┐ - │ Amazon Bedrock │ - │ │ - │ InvokeModel │ - └──────────────┘ - │ - Tags land in CUR 2.0 & Cost Explorer +```mermaid +sequenceDiagram + participant Client + participant Gateway as Portkey Gateway
(self-hosted) + participant STS as AWS STS + participant Bedrock as Amazon Bedrock + + Client->>Gateway: Request + x-portkey-metadata
{"app":"chatbot", "team":"platform"} + Gateway->>Gateway: Parse & sanitise metadata + Gateway->>STS: AssumeRole + Tags.member.N
(Key/Value pairs from metadata) + STS-->>Gateway: Temporary credentials
(tagged session) + Gateway->>Bedrock: InvokeModel
(using tagged credentials) + Bedrock-->>Gateway: Response + Gateway-->>Client: Response + Note over STS,Bedrock: Tags land in CUR 2.0 (iamPrincipal/ prefix),
CloudTrail, and Cost Explorer ``` 1. Client sends `x-portkey-metadata` with key-value pairs on each request. From 567ffc598cbcba0a2ae80da4d6bb272d9b32497d Mon Sep 17 00:00:00 2001 From: C Naresh Date: Wed, 9 Sep 2026 23:32:28 +0530 Subject: [PATCH 5/5] =?UTF-8?q?docs:=20reorder=20steps=20=E2=80=94=20IAM?= =?UTF-8?q?=20setup=20first,=20gateway=20enablement=20after?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Move IAM permission setup (caller policy + trust policy) to Step 1 - Move gateway env flag enablement to Step 2 with a warning to complete IAM setup first to avoid AccessDenied on all requests - Add enterprise self-hosted note at the top - Update all step references in troubleshooting section --- ...cost-attribution-with-sts-session-tags.mdx | 118 ++++++++++-------- 1 file changed, 63 insertions(+), 55 deletions(-) diff --git a/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx b/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx index 6eab3be91..799f0ad79 100644 --- a/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx +++ b/guides/use-cases/aws-bedrock-cost-attribution-with-sts-session-tags.mdx @@ -7,6 +7,10 @@ LLM gateways typically share a single IAM role across all tenants. Without per-r Portkey solves this by forwarding the `x-portkey-metadata` header as [AWS STS session tags](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html) during `AssumeRole`. Each unique tag set produces a distinct STS session, so AWS records cost and usage per application, team, or environment — no extra IAM roles required. + +This feature is available exclusively for **Enterprise self-hosted gateway** customers. It is not available on the Portkey managed cloud. + + ## How It Works ```mermaid @@ -38,59 +42,15 @@ sequenceDiagram |---|---| | **Gateway version** | `≥ 2.19.0` | | **Auth type** | Bedrock with **AWS Assumed Role** ([setup guide](/product/model-catalog/connect-bedrock-with-amazon-assumed-role)) | -| **Deployment** | Self-hosted (ECS, EKS, EC2, or Docker) | - ---- - -## Step 1: Enable Session Tags on the Gateway - -Set the environment variable on the gateway container: - -```sh -AWS_BEDROCK_STS_SESSION_TAGS_ENABLED=true -``` - - -```json -{ - "containerDefinitions": [{ - "name": "portkey-gateway", - "environment": [ - {"name": "AWS_BEDROCK_STS_SESSION_TAGS_ENABLED", "value": "true"} - ] - }] -} -``` - - - -```yaml -containers: - - name: portkey-gateway - env: - - name: AWS_BEDROCK_STS_SESSION_TAGS_ENABLED - value: "true" -``` - - - -```yaml -services: - portkey-gateway: - environment: - - AWS_BEDROCK_STS_SESSION_TAGS_ENABLED=true -``` - - -Restart or redeploy the gateway after adding the variable. +| **Deployment** | Enterprise self-hosted (ECS, EKS, EC2, or Docker) | --- -## Step 2: Configure IAM Permissions on AWS +## Step 1: Configure IAM Permissions on AWS -Session tags require `sts:TagSession` in **two places**: the caller's IAM policy and the target role's trust policy. Missing either one causes an `AccessDenied` error. +Session tags require `sts:TagSession` in **two places**: the caller's IAM policy and the target role's trust policy. Missing either one causes an `AccessDenied` error. Complete this setup **before** enabling the feature on the gateway. -### 2a. Caller principal (the gateway's execution role) +### 1a. Caller principal (the gateway's execution role) The IAM role the gateway runs under (e.g. an ECS task role, an EC2 instance profile, or a Kubernetes service account role) must be allowed to both assume and tag the target Bedrock role. @@ -118,7 +78,7 @@ Replace `` and `` with the actual values f **Cross-account setups:** When the gateway and the Bedrock role live in different AWS accounts, both the caller permissions (in the gateway account) and the target trust policy (in the Bedrock account) must allow `sts:TagSession`. This is the most common source of `AccessDenied` errors. -### 2b. Target role trust policy (the Bedrock invocation role) +### 1b. Target role trust policy (the Bedrock invocation role) Open the target role in IAM, go to **Trust relationships → Edit trust policy**, and ensure the `Action` includes both `sts:AssumeRole` and `sts:TagSession`: @@ -149,12 +109,60 @@ Open the target role in IAM, go to **Trust relationships → Edit trust policy** The `Condition` block is optional but recommended. If you configured an external ID when setting up the assumed role in Portkey, include it here. -### 2c. Bedrock invocation permissions (no change needed) +### 1c. Bedrock invocation permissions (no change needed) The existing permission policy on the target role (`bedrock:InvokeModel`, `bedrock:InvokeModelWithResponseStream`) does not need modification. Session tags only affect the `AssumeRole` call, not the Bedrock API calls themselves. --- +## Step 2: Enable Session Tags on the Gateway + +Once IAM permissions are in place, enable the feature by setting the environment variable on the gateway container: + +```sh +AWS_BEDROCK_STS_SESSION_TAGS_ENABLED=true +``` + + +```json +{ + "containerDefinitions": [{ + "name": "portkey-gateway", + "environment": [ + {"name": "AWS_BEDROCK_STS_SESSION_TAGS_ENABLED", "value": "true"} + ] + }] +} +``` + + + +```yaml +containers: + - name: portkey-gateway + env: + - name: AWS_BEDROCK_STS_SESSION_TAGS_ENABLED + value: "true" +``` + + + +```yaml +services: + portkey-gateway: + environment: + - AWS_BEDROCK_STS_SESSION_TAGS_ENABLED=true +``` + + + +Enable this flag **only after** completing the IAM setup in Step 1. If `sts:TagSession` is not permitted on both the caller and target roles, all Bedrock requests through assumed roles will fail with `AccessDenied` once the flag is enabled. + + +Restart or redeploy the gateway after adding the variable. + +--- + ## Step 3: Send Metadata with Requests Pass key-value pairs in the `x-portkey-metadata` header (or the SDK `metadata` option). Every key-value pair becomes an STS session tag. @@ -360,7 +368,7 @@ is not authorized to perform: sts:TagSession on resource: arn:aws:iam:::role/ ``` -**Fix:** Add `sts:TagSession` to **both** the caller's IAM policy (Step 2a) **and** the target role's trust policy (Step 2b). In cross-account setups, both accounts must allow the action. +**Fix:** Add `sts:TagSession` to **both** the caller's IAM policy (Step 1a) **and** the target role's trust policy (Step 1b). In cross-account setups, both accounts must allow the action. @@ -368,7 +376,7 @@ arn:aws:iam:::role/ Possible causes: 1. **Gateway version too old** — Confirm `>= 2.19.0` by calling `GET /health`. -2. **Feature flag not set** — Verify `AWS_BEDROCK_STS_SESSION_TAGS_ENABLED=true` is in the container environment. +2. **Feature flag not set** — Verify `AWS_BEDROCK_STS_SESSION_TAGS_ENABLED=true` is in the container environment (Step 2). 3. **Gateway not restarted** — Environment variables are read at startup. Redeploy after adding the variable. 4. **Metadata not sent** — Confirm the `x-portkey-metadata` header is present in Portkey's request inspector. 5. **Cached credentials** — Previously cached (untagged) credentials may still be in use. Wait for the credential TTL to expire or restart the gateway to clear the cache. @@ -382,13 +390,13 @@ Possible causes: 4. Ensure you are looking under **User-defined cost allocation tags**, not IAM principal type tags. - + -If Bedrock requests fail immediately after upgrading with the feature flag enabled, the most likely cause is missing `sts:TagSession` permissions. As a quick mitigation: +If Bedrock requests fail immediately after enabling `AWS_BEDROCK_STS_SESSION_TAGS_ENABLED`, the most likely cause is missing `sts:TagSession` permissions. As a quick mitigation: 1. Disable the feature flag: set `AWS_BEDROCK_STS_SESSION_TAGS_ENABLED` to `false` (or remove it). 2. Restart the gateway. -3. Fix the IAM permissions per Step 2, then re-enable. +3. Complete the IAM setup in Step 1, then re-enable.