---
title: Troubleshooting - Common stdapi.ai deployment issues
description: Fixes for the most common errors encountered when deploying and running
  stdapi.ai - Terraform failures, 400/401/403/404/429/503 responses, Bedrock throttling,
  IAM permission errors, S3 bucket errors, VPC connectivity, and more.
keywords: stdapi.ai troubleshooting, CannotPullContainerError, ECS task cannot pull
  image, no internet access deployment, container_image_repository_url, ECR mirror,
  ghcr.io, AWS Bedrock errors, Terraform apply failed, 503 ECS service, 401 API key,
  403 permission IAM, AccessDeniedException, 404 model not found, model does not exist
  or you do not have access to it, model_not_found, gpt-4o 404, gpt-4o not found,
  gpt-3.5-turbo not found, text-embedding-3-small 404, dall-e-3 model not found, OpenAI
  model name rejected, which models are available, list available models, ThrottlingException
  Bedrock, S3 bucket region, ElastiCache capacity, VPC endpoint timeout, podman SELinux,
  ECONNREFUSED IPv6 service discovery, drop_params ValidationException, ECS task unhealthy
  restart loop, container healthCheck command, 502 Bad Gateway ECS, 504 Gateway Time-out
  ECS, ECS task restarts under load, ECS task killed OutOfMemoryError, container killed
  due to memory usage, MemoryUtilization near 100 percent, Essential container in
  task exited, ECS stopped task reason OutOfMemoryError, task out of memory audio
  video, increase ECS task memory, Fargate task memory too small, text to speech input
  too long, speech input limited to 3000 characters, speech input limited to 20000
  characters, generative voice long input, StartSpeechSynthesisStream AccessDenied,
  service_tier ignored, guardrail ignored, model parameters ignored, tool_choice none
  ignored, tool_choice none returns tool_calls, model calls a tool I removed, removed
  tool still called, tools changed between turns, mid-conversation tool changes, model
  alias configuration, MODEL_ALIASES validation error, extra inputs are not permitted
  alias, 413 attachment too large, a part must not exceed bytes, upload part too large,
  413 uploads part rejected, large image rejected, large PDF rejected, attachments
  larger than not available, attached files are too large per request, attachment
  total too large, AWS_S3_REGIONAL_BUCKETS attachment staging, web search returns
  no results, no error when web search permission missing, external_web_access rejected,
  external_web_access not available with this model, web search allowed_domains rejected,
  web search user_location rejected, web_search_options not available with this model,
  web_search_options rejected chat completions, web_search_options 400, vector store
  search responses, batch result files never expire, batch output_expires_after, batch
  results file 404 after expiry, bedrock-websearch InvokeSearch AccessDenied, web
  search stale cached results, mcp_servers ignored, mcp_toolset dropped, MCP tools
  never run, MCP connector not supported, browser_toolset_20260801 not available,
  computer_toolset not available for this model, toolset is not available for this
  model, browser_state tool result refused, Agent Skills container ignored, container
  skills never loaded, valid Cognito token rejected 401, Cognito JWT unauthorized,
  aws_cognito_client_ids required, server will not start after enabling Cognito, server
  will not start after enabling authentication, API key secret is empty, API key secret
  stored as binary, secret holds no text value, API_KEY_SECRETSMANAGER_KEY does not
  hold text, plain string Secrets Manager secret, API_KEY_SSM_PARAMETER empty value,
  AUTHENTICATION_MODE method not enabled, issuer-cognito-idp issuer mismatch, aws.cognito.signin.user.admin
  scope missing, WWW-Authenticate Bearer challenge, MCP client cannot discover authentication,
  oauth-protected-resource 404, protected resource does not match expected, OAUTH_RESOURCE_IDENTIFIER
  invalid, openid-configuration not served, OAUTH_AUTHORIZATION_SERVERS must name
  the issuer, published issuer does not match user pool, authorization server derived
  from Cognito, response format srt not available, verbose_json not available transcription,
  streamed transcription delivers nothing until the end, transcription stream hangs,
  stream=true events arrive all at once, transcript.text.delta arrives at the end,
  StartStreamTranscription missing streamed transcription served as job, StartMedicalStreamTranscription
  AccessDenied, AWS_TRANSCRIBE_STREAM_LANGUAGES, MaxSpeakerLabels streamed transcription
  not phrase by phrase, timestamp_granularities rejected, 503 transcription not available,
  transcription bucket missing, 503 url response format not available, image url response
  bucket missing, audio too long 10 minutes, nova-2-sonic model not found, InvokeModelWithBidirectionalStream
  AccessDenied, 503 live conversation not available, tenant API key 503, sk-std key
  rejected, TENANT_API_KEYS, tenant key not in Parameter Store, revoked tenant key
  still accepted, TENANT_KEY_CACHE_SECONDS, tenant key rotation, rotated tenant key
  refused, superseded tenant key still accepted, AWSPREVIOUS tenant key, AWSCURRENT
  tenant key, TENANT_KEY_SECRETSMANAGER_PREFIX, TENANT_KEY_ROTATION_DAYS, TENANT_KEY_ROTATION_OVERLAP_SECONDS,
  key_generation ignored, tenant key never rotated, secretsmanager PutSecretValue
  AccessDenied, tenant model 404, models_allow empty, tenant rate limit, 429 rate_limit_error
  tenant API key, rate_limit_exceeded tenant, requests_per_minute tenant record, tokens_per_minute
  tenant record, TENANT_RATE_LIMIT_REQUESTS_PER_MINUTE, TENANT_RATE_LIMIT_TOKENS_PER_MINUTE,
  x-ratelimit-remaining-requests, anthropic-ratelimit-requests-remaining, retry-after
  tenant, tenant refused below requests_per_minute, rate limit reached before the
  limit several instances, tenant 429 no tokens billed, new tenant key rate limited
  after deploy, token limit refuses eight concurrent requests, token limit 64 requests
  in flight, latency after enabling tenant rate limits, first request of each minute
  slow, tokens_per_minute exceeded by a concurrent burst, realtime handshake rate
  limited, websocket close 3000 rate_limit_exceeded, invalid_request_error rate_limit_exceeded
  realtime, realtime 429 no retry-after, Ollama 429 no x-ratelimit headers, Cohere
  429 no rate limit headers, dynamodb UpdateItem AccessDenied, counter permission
  probed at startup only, tenant rate limiting not available 503, rate limit counter
  unavailable 529, tenant AWS credentials, TENANT_AWS_CREDENTIALS, aws_role_arn refused
  503, tenant role 403, ExternalId mismatch, sts:AssumeRole tenant role AccessDenied,
  tenant credential could not be used, tenant account does not have access to this
  model, per-user cost attribution, sts:TagSession AccessDenied, AWS_BEDROCK_USER_ROLE_ARN,
  all Bedrock spend on one identity, line_item_iam_principal missing, cost allocation
  tag IAM principal, requests fail after enabling per-user roles, end user role ApplyGuardrail
  AccessDenied, 403 guardrail per-user cost attribution, end user role missing model
  ARN form, 503 speech to speech request fails after a few seconds, real-time audio
  session timeout, AWS_CONNECT_TIMEOUT generative voice, conversation items missing,
  conversation 404 after 30 days, cannot add items to conversation, bedrock UpdateSession
  AccessDenied, conversation metadata update 503, streamed response conversation items
  delayed, batch stays validating for minutes, batch rejected fewer than 100 requests,
  batch minimum 100 requests per model, minimum number of records per batch inference
  job quota, batch model marked legacy 30 days, 503 batch creation refused ValidationException,
  search_models batch false, model missing from batch=true, batch flag absent COST_TRACKING,
  batch role not configured, 503 batch API disabled, AWS_BEDROCK_BATCH_ROLE_ARN, iam:PassRole
  bedrock.amazonaws.com, CreateModelInvocationJob AccessDenied, 503 batch creation
  denied, batch line failed tool use not supported, batch never reports failed, batch
  status failed never reached, batch create returns 400 instead of a batch, invalid_json_line,
  duplicate_custom_id, batch errors data line null, batch results out of order, batch
  output_file_id missing, batch cached_tokens 0, prompt caching ignored in a batch,
  cache_control dropped batch, prompt_cache_key batch no effect, batch cache_read_input_tokens
  zero, vector store file stays in_progress, vector store unsupported_file, attaching
  a PDF fails unsupported_file, search returns nothing after attaching, 503 vector
  stores disabled, AWS_S3_VECTORS_BUCKET, vector bucket missing, vector bucket wrong
  region, s3vectors AccessDenied, 503 vector store operation denied, vector store
  409 conflict, vs_kb vector store 404, knowledge base vector store not found, AWS_BEDROCK_KNOWLEDGE_BASE_IDS,
  knowledge base not allowlisted, bedrock GetKnowledgeBase AccessDenied, 503 attaching
  a file to a knowledge base store, knowledge base multiple data sources, dataSourceId
  ambiguous, 400 files cannot be attached to this vector store, knowledge base data
  source is not a custom one, attach refused knowledge base store, cannot delete a
  file a search returned, kbdoc file cannot be removed, 400 cannot delete knowledge
  base vector store, chunking_strategy rejected knowledge base, score_threshold rejected
  knowledge base, file_batches rejected knowledge base, search query too long knowledge
  base, realtime websocket upgrade 404, WS /v1/realtime not found, POST /v1/realtime/calls
  404, realtime calls endpoint not found, realtime WebRTC not supported, realtime
  SIP not supported, WebRTC transport realtime API, SIP transport realtime API, SDP
  offer realtime 404, ALB does not carry UDP, WebRTC media ingress NLB, REALTIME_WEBRTC_ENABLED,
  WebRTC call no audio, SDP answer silent call, iceConnectionState failed gateway,
  WebRTC call one way audio, REALTIME_WEBRTC_STUN_SERVER private address candidates,
  TURN relay coturn UDP blocked, WebRTC hangup 404 another instance, call_id sideband
  404 instance, WebRTC call drops 8 minutes, invalid_offer failed to parse offer,
  unsupported_content_type realtime calls JSON, oai-events data channel no events,
  realtime session closes after 8 minutes, session_expired close code, realtime session
  closes mid-conversation, realtime session idle timeout, ALBRequestCountPerTarget
  realtime autoscaling, 403 NoUserAgent_HEADER websocket, WAF blocks websocket client
  no user agent, ephemeral client secret rejected, ek_ secret invalid on other instance,
  REALTIME_CLIENT_SECRET_KEY, realtime client secret multi-instance, 503 feature_unavailable
  AccessDeniedException, IAM permission denied 503 not available on the current server,
  model access not enabled 503, 403 permission_error end user role, access denied
  reading the input s3 URI, 400 input_access_denied, s3 input AccessDenied not 503,
  accepted bucket GetObject denied, Mantle models missing from catalog, bedrock-mantle
  unreachable, api.aws endpoint blocked, HTTPS_PROXY not honored, proxy environment
  variables ignored, NO_PROXY api.aws, com.amazonaws bedrock-mantle VPC endpoint,
  netrc credentials sent to AWS endpoints, bedrock_mantle_regions_without_endpoint,
  Mantle unreachable in every region, MantleError service is temporarily unavailable,
  bedrock-mantle Domain name not found, ClientConnectorDNSError bedrock-mantle, NXDOMAIN
  bedrock-mantle api.aws, region does not offer Bedrock Mantle, AWS_BEDROCK_MANTLE_REGIONS
  default, Mantle models missing in one region, server_start_time_ms slow startup,
  startup takes 30 seconds, ECS container metadata endpoint answered after attempts,
  ECS task metadata slow startup, unreachable region slows startup, AWS_CONNECT_TIMEOUT
  startup time, abandoned_background_tasks, background work lost on deploy, vector
  store file stuck in_progress after deploy, cleanup skipped on shutdown, SHUTDOWN_DRAIN_TIMEOUT,
  container stop timeout SIGKILL, graceful shutdown drain, work abandoned on scale-in,
  vector store file failed indexing was interrupted, attach the file again to index
  it, vector store indexing waits its turn, AWS_SQS_VECTOR_STORE_QUEUE_URL, vector
  store indexing queue, durable vector store indexing, sqs:SendMessage AccessDenied,
  must be an Amazon SQS queue URL, must name a standard queue, FIFO queue refused
  indexing, indexing queue has no dead-letter queue, vector store indexing job abandoned,
  indexing not picked up after deploy, another server finishes the indexing, tenant
  key indexing not queued, per-user role indexing not queued, tenant vector store
  file failed after deploy, aws_bedrock_mantle_preferred_models is incompatible with
  Amazon Bedrock Guardrails, server will not start guardrail Mantle, guardrail preferred
  models startup error, AWS_BEDROCK_MANTLE_PREFERRED_MODELS empty, GPT-5.6 more expensive,
  gpt-5.6 price increased 10%, GPT-5.6 billed under Bedrock Mantle, global cross-region
  discount lost, restore bedrock-runtime routing dual-homed, Marketplace model endpoint
  missing from model list, AWS_BEDROCK_MARKETPLACE_ENDPOINTS_ENABLED, AWS_BEDROCK_MARKETPLACE_ENDPOINT_REGIONS,
  marketplace model endpoint not in service, marketplace model endpoint deployment
  not registered, marketplace model endpoint requests refused, marketplace/model-endpoint
  ARN missing from policy, foundation-model wildcard denies marketplace endpoint,
  marketplace model endpoint reports usage but no cost, SageMaker hosting instance-hour
  billing, no scale to zero marketplace endpoint, two marketplace endpoints same listing
  one model missing, first SageMaker request takes minutes, SageMaker endpoint cold
  start slow, scale to zero SageMaker first request, 503 model is starting up and
  did not become available in time, AWS_SAGEMAKER_WARMUP_TIMEOUT, NoCapacityInvocationFailures
  alarm missing, SageMaker endpoint never scales up from zero, inference component
  CurrentCopyCount stays 0, there is not enough hardware resources on the instances
  for this endpoint, SageMaker AI endpoint missing from model list, AWS_SAGEMAKER_ENDPOINTS,
  sagemaker:CallWithBearerToken AccessDenied, sagemaker InvokeEndpoint 403 bearer
  token, SageMaker AI endpoint reports usage but no cost, wildcard model pattern ambiguous,
  400 model pattern matches several models released same date, wildcard model pattern
  skips a listed model, model release date unknown never selected by pattern, glob
  model name 400, model pattern fewer than three characters, bare asterisk model not
  accepted, organization usage endpoints 503, /v1/organization/usage 503, /v1/organization/costs
  503, usage API not enabled, USAGE_API, GET /v1/usage not served, retired usage endpoint,
  usage endpoints empty buckets, no usage reported, organization costs empty, per-endpoint
  usage buckets missing, Operation dimension missing, CLOUDWATCH_METRICS_REGION, usage
  reports one region only, usage endpoints 403, tenant API key refused usage endpoints,
  USAGE_API_ADMIN_SCOPES, usage admin scope missing, usage query too large, too many
  metric series, USAGE_API_MAX_METRICS, USAGE_API_MAX_RANGE_DAYS, usage range too
  long, bucket_width 1m refused, one-minute buckets 15 days, num_model_requests, Ollama
  client shows no models, Ollama connection empty model list, ollama list size 0,
  ollama pull does nothing, Ollama 404 llama3.2, Ollama tokens per second 0, eval_duration
  missing, load_duration missing, Ollama is running not returned, Ollama is running
  base URL, Ollama test connection fails, /api/version probe, Ollama thinking empty,
  Ollama 401 API key, Unable to locate credentials docker run, ~/.aws mount permission
  denied container, container runs as uid 65532, nonroot 65532 cannot read credentials,
  docker run --user HOME /home/nonroot, userns keep-id uid 65532, marketplace image
  no shell, community image no shell, docker run sh not found image, entrypoint python
  command override ignored, bedrock_regions_missing_iam_permission, region reported
  unreachable but permission denied, missing IAM permission named in startup log,
  bedrock:ListProvisionedModelThroughputs AccessDenied, models missing in one region
  only, 403 Forbidden served by the load balancer, 403 not from stdapi.ai, AWSManagedRulesAnonymousIpList,
  alb_waf_block_anonymous_ips, WAF blocks VPN, WAF blocks CloudShell, WAF blocks cloud
  host, anonymous IP list 403, terraform apply hangs aws_acm_certificate_validation,
  ACM DNS validation stuck, PENDING_VALIDATION certificate, no matching Route 53 Zone
  found, alb_route53_zone_name, hosted zone not found terraform, private hosted zone
  refused, Anthropic use case form, PutUseCaseForModelAccess, Bedrock model access,
  Claude model missing from catalogue, unentitled unavailable_models, Bedrock model
  catalog use case details, aws-marketplace:Subscribe first use, realtime session
  silent after function call, function_call_output no answer, realtime tools ignored,
  realtime tool_choice, realtime session.update tools refused, moderation flagged
  true no category, moderation flagged with every category false, moderations flags
  harmless text, prompt attack detection moderations, PII detection moderations, AWS_BEDROCK_GUARDRAIL_CHECKS_PII_ENTITIES,
  AWS_BEDROCK_GUARDRAIL_CHECKS_PROMPT_ATTACK, guardrail checks billed per check, AWS_BEDROCK_GUARDRAIL_SCOPE_TURNS,
  guardrail no longer flags conversation history, guardrail misses earlier turns,
  guardrail stopped checking an image, tenant rate limit slow requests, rate-limited
  tenant latency on every request, requests_per_minute below 16 table write, tool_choice
  required 400 Claude Opus 5.5, forced tool use refused, reasoning_effort none still
  reasons, disabled thinking still returns thinking, reasoning cannot be disabled
  Opus 5.5, find your symptom troubleshooting index, gpt-6 web_search 400, GPT-6 billed
  under Bedrock Mantle, gpt-6 price increased 10%, guardrail refused GPT-6 Mantle,
  gpt-6 code_interpreter refused, Server tool 'web_search' is not supported by this
  model on this server, GPT-6 on Bedrock Mantle, reasoning effort none still reasons
  responses, context_management ignored, context_management no effect, applied_edits
  missing, compact_20260112 400, clear_tool_uses_20250919, clear_thinking_20251015
  requires thinking, srt not supported amazon.transcribe-medical, medical transcription
  language not supported, Specialty only available with stream=true, 503 medical transcription
  not available, 503 live transcription not available medical, StartMedicalTranscriptionJob
  not authorized region, context_length_exceeded, Your input exceeds the context window
  of this model, Input tokens exceed the configured limit, context_length_exceeded
  chat completions, prompt is too long anthropic messages, input exceeds context window,
  prompt is too long, streamed response server_error context window, truncation auto,
  context_management compaction, answer shorter than max_output_tokens, response cut
  short before max_tokens, incomplete max_output_tokens below the limit, stop_reason
  model_context_window_exceeded, input length and max_tokens exceed context limit,
  The requested maximum output tokens leave no room for the input, Forcing tool use
  (tool_choice or function_call) is not available with this model, forced function_call
  refused, reasoning_effort ignored Claude 3.7 4.5, no reasoning_content small max_tokens,
  reasoning off with max_completion_tokens 1024, empty thinking text Claude, thinking
  display summarized, display omitted default, reasoning summary empty Claude, reasoning_content
  empty Opus 5, thinking field empty with signature, gpt-oss reasoning cannot be disabled,
  gpt-6 astra reasoning_effort none, thinking display updates 400, display updates
  not available, unknown_parameter, Unknown parameter, STRICT_INPUT_VALIDATION 400,
  Please reduce your prompt; or completion length, legacy completions context length
---

# :material-wrench: Troubleshooting

Common issues when deploying stdapi.ai for the first time. If your error isn't listed here, see the [Contact](contact.md) page, or open an issue on [GitHub](https://github.com/stdapi-ai/stdapi.ai/issues).

---

## :material-magnify: Find your symptom

Search the page with Ctrl-F for the exact error string, or start from the row that matches what you see.

| You see | Start here |
|---|---|
| `terraform apply` fails with `AccessDenied`, or on `no matching Route 53 Zone found`, or hangs on `aws_acm_certificate_validation` | [Terraform / Deployment](#terraform-deployment) |
| `terraform apply` succeeded, but nothing is reachable | [Terraform / Deployment](#terraform-deployment) · [Nothing answers](#reaching-the-gateway) |
| `CannotPullContainerError`: ECS tasks cannot pull the image without internet access | [Terraform / Deployment](#terraform-deployment) |
| `Unable to locate credentials`, or a volume mount fails under SELinux | [Local Docker / Podman](#local-docker-podman) |
| `503 Service Unavailable` on `/docs` or any endpoint, `ECONNREFUSED`, a task that never becomes healthy | [Nothing answers](#reaching-the-gateway) |
| `403 Forbidden` with no JSON body — served by the load balancer, not by stdapi.ai | [Nothing answers](#reaching-the-gateway) |
| `502 Bad Gateway` or `504 Gateway Time-out` under load, tasks restarting, ECS stopped reason `OutOfMemoryError` | [Nothing answers](#reaching-the-gateway) |
| `401 Unauthorized` | [Authentication & Identity](#authentication-identity) |
| `403 permission_error` on a model call | [Cost attribution and usage](#cost-and-usage) · [Authentication & Identity](#authentication-identity) |
| `404 Not Found` — `The model ... does not exist or you do not have access to it` | [`404` model not found](#models-and-the-catalogue) |
| A Claude model is missing from the catalogue | [`404` model not found](#models-and-the-catalogue) |
| A Marketplace, SageMaker AI or Bedrock Mantle model is missing, slow or uncosted | [Your own models](#your-own-models) |
| `503 feature_unavailable`, or `AccessDeniedException` in the server log | [Startup and shutdown](#startup-and-shutdown) |
| Startup takes tens of seconds, or a deploy leaves work unfinished | [Startup and shutdown](#startup-and-shutdown) |
| `400 Bad Request` from a model call, a parameter that is silently ignored, or a tool called after the request stopped declaring it | [`400` and `429`](#bedrock-request-errors) |
| `429 Too Many Requests`, `ThrottlingException` | [`400` and `429`](#bedrock-request-errors) |
| `context_management` has no effect, `applied_edits` is missing, or `compact_20260112` is refused | [`400` and `429`](#bedrock-request-errors) |
| Claude thinking blocks or reasoning items come back with empty text | [`400` and `429`](#bedrock-request-errors) |
| `429 rate_limit_error` naming a tenant API key, `x-ratelimit-*` or `anthropic-ratelimit-*` headers, a Realtime session closed with `rate_limit_exceeded` | [Authentication & Identity](#authentication-identity) |
| A moderation result is `flagged` with every category `false` | [`400` and `429`](#bedrock-request-errors) |
| Text-to-speech, transcription, a realtime session or a WebRTC call misbehaves | [Audio, speech and realtime](#audio-speech-and-realtime) |
| A file, batch, vector store or conversation never finishes | [Files, batches, vector stores](#files-batches-and-vector-stores) |
| All spend lands on one identity, or `/v1/organization/usage` answers `503`, `403` or empty buckets | [Cost attribution and usage](#cost-and-usage) |
| Web search returns nothing, MCP tools are never called, a browser or computer toolset is refused, or an Agent Skills `container` changes nothing | [Web search and MCP tools](#tools-and-search) |
| An Ollama client shows no models, `0` tokens per second, or no thinking text | [Ollama clients](#ollama-clients) |
| `503`, `504` or `408` mid-stream, `413 Payload Too Large`, `400 context_length_exceeded` or `prompt is too long`, an answer cut short below its output limit | [Long generations](#long-generations) |

Every response the gateway returns for an AWS failure is listed in [AWS error → HTTP status mapping](#aws-error-http-status-mapping).

---

## :material-cloud-upload: Terraform / Deployment { #terraform-deployment }

??? failure "`terraform apply` fails with AccessDenied on IAM, KMS, or ECS actions"
    Your AWS profile does not have sufficient permissions. The stdapi.ai Terraform module provisions IAM roles, KMS keys, ECS, ALB, Route53 records, an optional WAF, and (for some samples) RDS and ElastiCache.

    - Use an **administrator-level** AWS profile for the evaluation deployment.
    - **Recommended**: deploy into a sandbox/non-production AWS account first, then replicate into your target account with scoped-down principals once validated.
    - Verify your active identity: `aws sts get-caller-identity`.

??? failure "`terraform apply` fails with `no matching Route 53 Zone found`"
    The deployment serves the API from a name you control, so the module needs a **public** Route 53 hosted zone, **in the same AWS account**, that is authoritative for that name. It looks the zone up from `alb_domain_name` by taking the most specific parent zone it can find; when nothing matches, the plan fails before anything is created.

    - Name the zone explicitly when it is not the immediate parent of the domain — `api.eu.example.com` served from the `example.com` zone needs `-var alb_route53_zone_name=example.com`.
    - Confirm the zone exists in this account and is public:
      ```bash
      aws route53 list-hosted-zones-by-name --dns-name example.com \
        --query 'HostedZones[].{Name:Name,Private:Config.PrivateZone,Id:Id}'
      ```
      A private zone never matches: the module validates the name on the public internet.
    - Wrong account or wrong profile produces the same message — check with `aws sts get-caller-identity` before re-applying.
    - Bringing your own certificate and DNS instead: pass `alb_certificate_arn` and leave `alb_domain_name` unset, then point your own record at the ALB. See [Advanced Deployment](operations_deploy_advanced.md).

??? failure "`terraform apply` hangs on `aws_acm_certificate_validation`"
    ACM issues the certificate only once the DNS validation record it asked for resolves on the public internet. The module writes that record into the zone it found, then waits. A few minutes is normal; longer than that means the record is not being answered, and because the resource waits rather than fails, it reads as a hang.

    - Check what ACM is still waiting for:
      ```bash
      aws acm describe-certificate --certificate-arn <arn> \
        --query 'Certificate.DomainValidationOptions'
      ```
      `ValidationStatus: PENDING_VALIDATION` with a `ResourceRecord` names the exact record that must resolve.
    - Resolve that record from outside AWS: `dig +short <name> CNAME`. Nothing back means the record is in a zone that does not serve the domain publicly.
    - The usual cause is a zone found by name that is not the authoritative one — a duplicate zone in this account, or a registrar delegating elsewhere. Compare the registrar's name servers with the zone's: `dig +short NS example.com` against the zone's `NS` record set, and pass the right zone with `-var alb_route53_zone_name=...`.
    - Cancelling the apply leaves the certificate `PENDING_VALIDATION`; it is picked up again by the next apply once DNS answers, and no other resource depends on it having been re-created.

??? failure "`terraform apply` succeeds but nothing is reachable"
    Terraform does not wait for the service to stabilise, so a successful apply means the resources exist, not that a task is running. The ALB returns `503 Service Unavailable` until tasks pass health checks — either because the service is still coming up, or because no task can start at all.

    - Wait 2–3 minutes after `terraform apply` completes.
    - Check ECS service status: `aws ecs describe-services --cluster <cluster> --services <service>`. A `CannotPullContainerError` in its events means the tasks cannot pull the container image: see the next entry.
    - Check task logs in CloudWatch: `/aws/ecs/<service-name>`.

??? failure "ECS tasks never start: `CannotPullContainerError` in a deployment without internet access"
    The Terraform module deploys the hardened image `ghcr.io/stdapi-ai/stdapi.ai-community`, which the ECS tasks pull from GitHub Container Registry at startup. A deployment with no internet egress (no NAT gateway, no proxy) cannot reach it: the stopped task reason reads `CannotPullContainerError`, and the ALB answers `503` because no task becomes healthy.

    - Mirror the image into an Amazon ECR repository of your own, **in the deployment Region**, with the multi-arch tag `<version>`, copied with `skopeo copy --all` so that both the amd64 and arm64 images are mirrored, then set `container_image_repository_url` on the module to that repository URL, without a tag. The image is then pulled through the module's Amazon ECR VPC endpoints. The mirror alone does not remove internet access: also set `aws_bedrock_marketplace_auto_subscribe = false`, keep every AWS service in the deployment Region and WebRTC media mode off, and let the module create the VPC. See [Deployment without internet access](operations_deploy_advanced.md#deployment-without-internet-access).
    - Mirror each new version before you bump the module version: a task definition that names a tag missing from your repository fails the same way.
    - Or allow internet access to `ghcr.io` from the task subnets (a NAT gateway or an HTTP proxy) and leave `container_image_repository_url` unset.
    - Confirm which image the service uses: `aws ecs describe-task-definition --task-definition <family> --query 'taskDefinition.containerDefinitions[].image'`.

??? failure "Wrong AWS region or profile used by Terraform"
    The AWS provider uses the region/profile from your environment, not a Terraform variable.

    - Confirm before applying:
      ```bash
      aws sts get-caller-identity
      aws configure get region
      ```
    - Set explicitly with `AWS_PROFILE=... AWS_REGION=... terraform apply` if needed.

??? failure "ElastiCache creation failed — insufficient capacity in AZ (Open WebUI sample)"
    The ElastiCache Valkey cache occasionally fails to create when the target availability zone is out of capacity.

    ```text
    Error: waiting for ElastiCache Replication Group ... create: unexpected state 'create-failed',
    wanted target 'available'
    ```

    - Remove the failed Valkey cache from the ElastiCache console (disable backups first, then wait for full deletion) and re-run `terraform apply`.
    - If the problem persists, change `node_type` in `valkey.tf` (e.g. `cache.t4g.micro` → `cache.t3.micro`) and retry.

---

## :material-docker: Local Docker / Podman { #local-docker-podman }

??? failure "Podman volume mount fails on Fedora/RHEL with SELinux (local Docker)"
    SELinux blocks container access to `~/.aws` without a relabel.

    - Add `:z` (or `:Z` for exclusive use) to the volume and `--userns=keep-id:uid=65532,gid=65532`:
      ```bash
      podman run --rm -p 8000:8000 \
        --userns=keep-id:uid=65532,gid=65532 \
        -v ~/.aws:/home/nonroot/.aws:ro,z \
        -e AWS_BEDROCK_REGIONS=us-east-1,us-west-2 \
        -e ENABLE_DOCS=true \
        ghcr.io/stdapi-ai/stdapi.ai-community:latest
      ```
    - See [Local Development](operations_getting_started_local.md#run-it) for the full run command.

??? failure "`Unable to locate credentials` with `~/.aws` mounted into the container"
    The image runs as the unprivileged user `nonroot`, uid/gid **65532**, while
    the files under `~/.aws` belong to your own account and are typically
    readable by it alone. The mount succeeds, every file inside it is
    unreadable, and the server exits as if no credentials had been given.

    - Run the container as yourself: `--user "$(id -u):$(id -g)" -e HOME=/home/nonroot`. The `HOME` value matters — without it the AWS SDK looks for `.aws` in the wrong place. Run this from your own shell, never under `sudo` — under `sudo`, `$(id -u):$(id -g)` resolves to `0:0` and the container silently runs as root instead of `nonroot`.
    - On rootless Podman use `--userns=keep-id:uid=65532,gid=65532` instead, which maps your account onto the image's user.
    - Or skip the mount entirely and pass `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_SESSION_TOKEN` as environment variables — see [Local Development](operations_getting_started_local.md#run-it).

??? failure "A `docker run` command override stopped working after upgrading the image"
    The image declares `python` as its entry point, so anything after the
    image name is passed to the interpreter rather than run as a program. The
    image ships **no shell and no package manager**, so `sh`, `bash` and any
    install command have nothing to run there.

    - Pass Python arguments directly: `docker run ... ghcr.io/stdapi-ai/stdapi.ai-community:latest -c "import stdapi; print(stdapi)"`.
    - To run another binary in the image, name it explicitly: `--entrypoint /usr/bin/ffmpeg`.
    - Server options are environment variables, not command arguments — see [Configuration](operations_configuration.md).

---

## :material-api: Runtime / First API call { #runtime-first-api-call }

Grouped by what you see. The two `401` entries live under [Authentication & Identity](#authentication-identity).

### :material-lan-connect: Nothing answers — `503` on `/docs`, `ECONNREFUSED`, unhealthy tasks { #reaching-the-gateway }

??? failure "`503 Service Unavailable` — on the /docs page or any endpoint"
    The ECS service is still starting up. Health checks take a few minutes.

    - Wait 2–3 minutes after deployment and refresh.
    - Check the ALB target group health in the AWS console.
    - If it persists longer than 5 minutes, inspect CloudWatch logs for the ECS task. No log group and no task at all points at the image pull rather than the application — see [`terraform apply` succeeds but nothing is reachable](#terraform-deployment).

??? failure "`ECONNREFUSED` — connection refused, but only from some clients"
    The image binds IPv4 only (`GRANIAN_HOST=0.0.0.0`), so a client that resolves the server to an IPv6 address reaches a port nothing is listening on. Clients disagree about which address to try first, which is why the same deployment looks reachable from one language and dead from another: Node.js prefers the `AAAA` record and fails outright, while most Python clients fall back to the `A` record and hide the problem.

    - Typically hit with **ECS service discovery**, which publishes an `AAAA` record for every task in an IPv6-enabled subnet. Deployments fronted by an ALB are unaffected — the load balancer terminates the client connection itself and reaches the task over IPv4.
    - Set `GRANIAN_HOST=::` for a dual-stack socket answering both families; see the Container Runtime note in [Configuration](operations_configuration.md). The [Terraform module](https://github.com/stdapi-ai/terraform-aws-stdapi-ai) sets it when the VPC has IPv6 enabled.
    - After switching, extend [`PROXY_TRUSTED_HOSTS`](operations_configuration_server.md#proxy-trusted-hosts) with the IPv4-mapped form of each range (`::ffff:10.0.0.0/112`) — a dual-stack listener reports IPv4 peers in that form, and an untrusted proxy means `X-Forwarded-For` is ignored and the load balancer's own address is logged as the client IP.

??? failure "Targets never become healthy after setting `TRUSTED_HOSTS`"
    Host header validation applies to `/health` as well. A load balancer health check addresses the target directly, so its `Host` header carries the target's IP address — which a list of domain names does not match, and every probe is answered with `400`. The target group stays unhealthy and the ALB keeps returning `503`.

    - Prefer host validation at the load balancer: an ALB listener rule on the `Host` header, with `TRUSTED_HOSTS` left unset.
    - If the application-level allow-list is required, include the address the health check actually sends.
    - The container's own `HEALTHCHECK` is unaffected: it derives its `Host` header from `TRUSTED_HOSTS`. See [`TRUSTED_HOSTS`](operations_configuration_server.md#trusted-hosts).

??? failure "The ECS task never reports healthy, or restarts in a loop"
    ECS ignores the image's own `HEALTHCHECK`, so a task definition that declares no `healthCheck` gets no container-level probe at all, and one that declares a probe carried over from an earlier version runs a command the current image no longer provides. Either way the container is reported unhealthy and the service replaces it.

    - Copy the `healthCheck` block from the [ECS task definition example](operations_deploy_advanced.md#ecs-task-definition-example), which declares the image's own probe, and re-copy it when upgrading.
    - The [Terraform module](https://github.com/stdapi-ai/terraform-aws-stdapi-ai) declares it for you.
    - Do not substitute a `curl` or `urllib` one-liner: it sends an untrusted `Host` and is answered with `400` as soon as [`TRUSTED_HOSTS`](operations_configuration_server.md#trusted-hosts) is set.

??? failure "`502 Bad Gateway` or `504 Gateway Time-out` under load, tasks restarting — the container was OOM-killed"
    Neither status comes from stdapi.ai: the load balancer answers them itself when a request arrives with no healthy target to forward to, because the container behind it was just killed for using too much memory. The Terraform module's default Fargate task (512 MiB) sizes the task for text generation and embeddings, where it holds little more than the request in flight — it is not enough for the paths that hold bytes in memory: audio and video through the ffmpeg pipeline, inline input files up to [`MAX_INPUT_FILE_SIZE`](operations_configuration_server.md#max-input-file-size) each, and [`MAX_CONCURRENT_INPUT_DOWNLOADS`](operations_configuration_server.md#max-concurrent-input-downloads) of them fetched at once.

    - **Confirm the diagnosis**: in the ECS console, open the service's stopped tasks — a `stoppedReason` of `Essential container in task exited`, with the container's own reason `OutOfMemoryError: container killed due to memory usage`, is this failure, not a crash in the application. The same command works from the CLI:
      ```bash
      aws ecs describe-tasks --cluster <cluster> --tasks <stopped-task-arn> \
        --query 'tasks[].{Stopped:stoppedReason,Containers:containers[].reason}'
      ```
    - Cross-check with CloudWatch: the service's `MemoryUtilization` metric peaking near 100% in the same window rules out a health-check or networking cause.
    - **Fix**: raise the Terraform module's `memory` variable — `1024` or higher for a deployment that serves audio, video or image generation under real load. Valid `cpu`/`memory` pairs are constrained by Fargate; see [Amazon ECS task CPU and memory](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/fargate-tasks-services.html#fargate-tasks-size). See [Cost-Optimized Deployment](operations_deploy_advanced.md#cost-optimized-deployment) for the same sizing note in context.

??? failure "The API endpoint is `http://`, not `https://`"
    Neither `alb_domain_name` nor `alb_certificate_arn` was set, so the [Terraform module](https://github.com/stdapi-ai/terraform-aws-stdapi-ai) creates no HTTPS listener at all and forwards port 80 straight to the tasks. Everything works, but the API — including every API key sent to it — crosses the network in the clear. Never send an API key to that endpoint.

    - Set `alb_domain_name` to a name in a public Route 53 hosted zone in the account: the module then issues and renews an ACM certificate, serves HTTPS on port 443, and redirects port 80 to it.
    - Or supply `alb_certificate_arn` for a certificate you already manage. Supplying it *without* a domain leaves the ALB on its generated `*.elb.amazonaws.com` name, which no certificate can cover — that is the one configuration that does produce a browser certificate warning.

??? failure "`403 Forbidden` served by the load balancer, not by stdapi.ai — a VPN, cloud shell or proxy address"
    With `alb_waf_enabled = true` and `alb_waf_block_anonymous_ips = true` (both set by the production getting-started sample), the AWS-managed `AWSManagedRulesAnonymousIpList` rule group rejects requests from VPNs, proxies, Tor exit nodes and hosting-provider address ranges. Your address is allowed by the security group and *then* blocked by the WAF, so the `403` is produced before the request reaches stdapi.ai — it is not an API-key problem, and no amount of key fixing changes it.

    - Tell the two apart by the body: stdapi.ai answers every refusal with a JSON `error` object, the WAF does not. A `403` with no JSON body came from the load balancer.
    - Confirm the rule that matched: **WAF & Shield → Web ACLs → `<deployment>` → Sampled requests** in the console, or the `aws-waf-logs-<deployment>` CloudWatch log group when `alb_waf_logging_enabled = true` (the default).
    - Testing from a corporate VPN, AWS CloudShell, an EC2 instance, a CI runner or any cloud host is the common way to hit this — every one of those is an address range the rule group covers. Retry from an ordinary network first.
    - Keep the rule and allow yourself through, or set `alb_waf_block_anonymous_ips = false` in the sample's `main.tf` and re-apply. The rest of the WAF, including the rate limit, is unaffected.
    - A `403` with no `User-Agent` header on the request is the *other* WAF rule, `NoUserAgent_HEADER` — see [AWS WAF's `NoUserAgent_HEADER` rule reads like an auth failure](operations_deploy_advanced.md#aws-wafs-nouseragent_header-rule-reads-like-an-auth-failure). A `403` that *does* carry a JSON `permission_error` body is [per-user cost attribution](operations_cost_management.md#per-user-attribution), not the WAF.

??? failure "Connection timeout to AWS services from ECS"
    Outbound traffic to AWS endpoints is blocked.

    - Confirm the ECS task's security group allows outbound HTTPS (port 443).
    - If using **VPC endpoints** (the Terraform module default), verify the endpoint security groups and policies permit traffic from the ECS task subnet.
    - If ECS runs in a private subnet without VPC endpoints, confirm the NAT gateway / route table is configured.

### :material-format-list-bulleted: `404` model not found, wildcards and the model list { #models-and-the-catalogue }

??? failure "`404 Not Found` — `The model ... does not exist or you do not have access to it`"
    The name in the request is not one this deployment serves. Most names need no change when you adopt the gateway: the Anthropic, OpenAI and Cohere models Bedrock serves are published under their providers' own names as well as their Bedrock IDs, derived mechanically from the ID, so `anthropic.claude-opus-5` also answers to `claude-opus-5`, `openai.gpt-oss-120b-1:0` to `gpt-oss-120b` and `cohere.rerank-v3-5:0` to `rerank-v3.5`. This `404` is what a name that *does* differ looks like: one your application hard-codes for a model this deployment does not serve (`gpt-4o`, `gpt-3.5-turbo`, `text-embedding-3-small`, `dall-e-3`) resolves only if you map it yourself with [`MODEL_ALIASES`](operations_configuration_models.md#model-aliases). Nothing is substituted on your behalf, because a lookalike would serve a different model than the one you asked for.

    - List what this deployment actually serves: `GET /search_models` (the default model-discovery endpoint). Filter by capability with query parameters — e.g. `GET /search_models?input_modalities=IMAGE&route=/v1/chat/completions` returns only vision-capable chat models. See the [Search Models API](api_search_models.md) reference.
    - `GET /v1/models` is also available for strict OpenAI SDK compatibility (lighter payload, no capability metadata).
    - Confirm the pipeline itself with a low-friction model: `amazon.nova-micro-v1:0` (available in all standard Bedrock regions).
    - **Keep the name your application already sends** by mapping it onto a served model with [`MODEL_ALIASES`](operations_configuration_models.md#model-aliases) — an alias only ever points at a model this deployment serves, so what it resolves to is your choice, not a guess.
    - Try the name with and without its `anthropic.` / `openai.` / `cohere.` prefix: both forms resolve for those three families, so `anthropic.claude-fable-5` and `claude-fable-5` reach the same model, as do `openai.gpt-oss-120b-1:0` and `gpt-oss-120b`. Cohere additionally spells its versions with a dot, so `cohere.embed-english-v3` answers to `embed-english-v3.0` and `cohere.rerank-v3-5:0` to `rerank-v3.5` — not to `embed-english-v3` or `rerank-v3-5`. Models from other providers are served under their Bedrock IDs only, so name the ID or alias it. A model version Bedrock has retired needs a current one, whichever form you use.
    - Only if the model *is* one Bedrock serves and it is still missing: verify `AWS_BEDROCK_REGIONS` includes a region that offers it — see the [Bedrock model availability table](https://docs.aws.amazon.com/bedrock/latest/userguide/models-regions.html). Adding regions never makes a name Bedrock does not serve resolve.

??? failure "A Claude model is absent from the catalogue and answers `404` — the Anthropic use case form has not been submitted"
    Anthropic models are the one family Bedrock gates behind a one-time **use case form**, submitted once per AWS account or Organization. Until it is accepted, the gateway's startup availability check finds the model unentitled and never adds it to the served catalogue: it is missing from `GET /v1/models` and `GET /search_models`, and naming it answers `404 Not Found` — not a `503`. Every other Bedrock foundation model is enabled by default in commercial Regions, so there is no per-model switch to hunt for.

    1. Confirm the diagnosis in the startup log: the model is listed under `unavailable_models` with an `unentitled` issue.
    2. Open any Anthropic model in the **Bedrock console → Model catalog** and submit the use case details, or call [`PutUseCaseForModelAccess`](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_PutUseCaseForModelAccess.html). The form asks for your company name, website, industry and intended users, so no deployment can fill it in on your behalf.
    3. Restart the deployment — the catalogue is built at startup, so an accepted form takes effect on the next start, not on the next request.
    4. Verify with `GET /search_models`; the model now appears with its capabilities.

    - Submit it in **every** account that runs a deployment: entitlement follows the account or Organization, not the region or the gateway.
    - A model listed through [AWS Marketplace](operations_cost_management.md#bedrock-marketplace-model-endpoints) is gated differently: AWS creates the subscription on first invocation, which needs `aws-marketplace:Subscribe` and `aws-marketplace:ViewSubscriptions` on the task role and [`AWS_BEDROCK_MARKETPLACE_AUTO_SUBSCRIBE`](operations_configuration_models.md#bedrock-marketplace-auto-subscribe) left enabled (the default). Without those permissions the first call fails with `AccessDeniedException`.
    - A model that is entitled but still missing is a region problem, not an access one — see the `404` entry above and check `AWS_BEDROCK_REGIONS`.

??? failure "`400 Bad Request` — a wildcard model pattern is refused as ambiguous"
    Two or more of its matches were released on the same date, so the server refuses to guess which one you meant, rather than spend your money on a model you never named — sibling models released together are often priced differently. The message names the tied models. `openai.gpt-5.6-*` (Sol, Terra, Luna) and `stability.*` on the image routes are real examples of ties.

    - Name the model you meant explicitly, or narrow the pattern so it matches only one of the tied models.
    - See everything a pattern matches, newest first, before relying on it: `GET /search_models?model=<pattern>` — see [Search Models](api_search_models.md#query-parameters).
    - See [Model Wildcard Patterns](operations_configuration_models.md#model-wildcard-patterns) for the full resolution rule.

??? failure "A wildcard model pattern never picks a model that is listed in `/v1/models` or `/search_models`"
    A pattern only selects from what it can order by release date, and four kinds of match are skipped even when they are listed and match the glob:

    - **A model with no known release date.** The server cannot order it, so it is never selected by a pattern, whatever else it matches — a property of that model, not a bug in the pattern or a gap in a particular backend's catalogue. It also never makes an otherwise-unique match ambiguous.
    - **A legacy model**, one [`AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK`](operations_configuration_models.md#bedrock-deprecated-model-fallback) covers, or one whose first use would open a paid Marketplace subscription.

    Name any of these explicitly and it still resolves exactly as it does today — only pattern resolution skips them. `GET /search_models?model=<pattern>` returns the whole match set a pattern draws from, newest first with unknown-release-date models last, so you can see what it will do before it surprises you — see [Model Wildcard Patterns](operations_configuration_models.md#model-wildcard-patterns).

??? failure "A model that has been removed still appears in `/v1/models`, or a new one takes minutes to show up"
    Expected, within a bounded window. The model list is discovered from Amazon Bedrock and kept for [`MODEL_CACHE_SECONDS`](operations_configuration_models.md#model-cache-seconds) (15 minutes by default); once it expires the request that notices is answered from the list in hand and the refresh runs behind it, so no request pays for the discovery pass. A model AWS has withdrawn — or that this account has lost access to — can therefore stay listed until that refresh lands, and a request naming it is accepted and then fails at the backend with the same `404 model_not_found` as any unknown model. See [Model List Refresh](operations_resilience.md#model-list-refresh).

    - **A model that is new** is never served from an expired list: naming one the list does not know makes the server refresh before it answers, so it is usable as soon as it exists.
    - **Shorten the window** with a lower `MODEL_CACHE_SECONDS`, at the cost of more discovery calls.
    - **Remove it entirely** with `MODEL_CACHE_MAX_STALE_SECONDS=0`, which makes every expiry refresh synchronously — the freshest and the slowest setting.
    - Only if the list is *very* old, or a model that exists never appears, is this a fault rather than the window — see the next entry.

??? failure "The model list never changes, or keeps advertising models that are gone"
    A refresh that keeps failing leaves the list frozen. It is reported in the server log rather than in a response, because the request that triggered it was answered from the list already in memory.

    - Look for `Refreshing the model list from AWS Bedrock failed` in the server log. It is logged at `warning`, and at `error` once the list is more than two [`MODEL_CACHE_SECONDS`](operations_configuration_models.md#model-cache-seconds) old — the signal that the list is drifting toward the ceiling.
    - The usual causes are a revoked `bedrock:ListFoundationModels` or `bedrock:GetFoundationModelAvailability` permission and a prolonged outage in every configured region. Check [Bedrock IAM](operations_iam_permissions.md#bedrock-iam) first.
    - Beyond [`MODEL_CACHE_MAX_STALE_SECONDS`](operations_configuration_models.md#model-cache-max-stale-seconds) (24 hours by default) requests stop being answered from the frozen list and wait for a refresh instead, so a deployment in this state eventually surfaces the real error to clients as a `503` rather than serving a list it cannot confirm.

??? failure "Enabling `MODEL_CACHE_SHARED` did not make startup any faster"
    A published model list is only read by servers running the **same version, in the same AWS account, with the same `AWS_BEDROCK_*` and `AWS_SAGEMAKER_*` configuration** — every setting under those prefixes, not only the ones discovery reads. Anything else reads as an empty cache and the server discovers the catalogue itself, exactly as it would with the feature off, so changing one of those settings costs the fleet one discovery pass each rather than pointing at a table or permission fault.

    - **During a rolling deployment this is expected**: the new version recognises nothing the old one published, so the first server of each version performs a full discovery pass and publishes for the rest. Changing any `AWS_BEDROCK_*` setting has the same effect, once.
    - **If it never gets faster**, the table itself is the suspect. Every failure is reported in the server log at `WARNING`, naming the DynamoDB action, the table and the setting to fix — a missing `dynamodb:GetItem`/`PutItem`/`Query` on the table ARN is the common one, see [Shared Table IAM](operations_iam_permissions.md#shared-table).
    - **A single-container deployment gains nothing** by design: there is no second server to share with, and the first one still has to discover the catalogue.
    - Setting it without [`AWS_DYNAMODB_TABLE`](operations_configuration_storage.md#aws-dynamodb-table) fails startup with a message naming both settings.

??? failure "Startup fails with a validation error naming a model alias"
    An alias in [`MODEL_ALIASES`](operations_configuration_models.md#model-aliases-configuration) maps to an object that is not a valid alias configuration, so the server refuses to start rather than ignore it.

    - `Extra inputs are not permitted` names a field that does not exist — check its spelling against the [alias fields](operations_configuration_models.md#model-aliases-configuration).
    - `Field required` on `model` means the object gives configuration but no target model.
    - A guardrail needs both `guardrail_id` and `guardrail_version`.
    - An alias that only maps a name to a model stays a plain string: `{"my-model": "amazon.nova-lite-v1:0"}`.

??? failure "`amazon.nova-2-sonic-v1:0` is missing from `/v1/models` or returns `404`"
    The model is not offered in every AWS Region, and the catalog only lists what the configured Regions serve. [Check the model's regions](models.md).

    - Add a Region that offers it to [`AWS_BEDROCK_REGIONS`](operations_configuration_aws.md#aws-bedrock-regions); `us-east-1`, `us-west-2`, `ap-northeast-1` and `eu-north-1` did at the time of writing, while `eu-west-3` and `eu-central-1` did not.
    - Request model access for it in the Amazon Bedrock console for that Region.
    - Grant the task role `bedrock:InvokeModelWithBidirectionalStream`, which this model needs on top of the usual invoke permissions — see [IAM Permissions](operations_iam_permissions.md). Without it the session opens and then ends with a `503` saying live conversation is not available on this server; the server log names the permission.
    - `/search_models?route=openai_audio_transcription` lists what this deployment can actually reach.

### :material-package-variant: Your own models — Marketplace, SageMaker AI and Bedrock Mantle { #your-own-models }

??? failure "A deployed Marketplace model endpoint does not appear in the model list"
    A [Bedrock Marketplace model endpoint](operations_cost_management.md#bedrock-marketplace-model-endpoints) is discovered, never created — the gateway only ever lists an endpoint that already exists in your account. Work through the checks in order:

    - **The feature is off.** [`AWS_BEDROCK_MARKETPLACE_ENDPOINTS_ENABLED`](operations_configuration_models.md#bedrock-marketplace-endpoints-enabled) defaults to `false`.
    - **The endpoint is in a region the server does not serve.** It must be in [`AWS_BEDROCK_REGIONS`](operations_configuration_aws.md#aws-bedrock-regions), and in [`AWS_BEDROCK_MARKETPLACE_ENDPOINT_REGIONS`](operations_configuration_models.md#bedrock-marketplace-endpoint-regions) too when that is set. A Marketplace model endpoint has no cross-region form, so the server cannot reach one outside the regions it is configured to call.
    - **The endpoint is not yet in service.** Deployment takes roughly 10–15 minutes; it appears the next time the model cache refreshes, not the instant deployment finishes. An endpoint you *update* — a scale, an instance change — is a different case: SageMaker keeps serving it on the previous configuration throughout, so it stays listed and keeps answering.
    - **Amazon Bedrock could not register the endpoint.** Its own console reports this — check there before assuming a gateway-side gap.
    - **The server role is missing the discovery permissions.** The server reports this itself, in its own log at `WARNING` — see [Bedrock Marketplace Model Endpoints IAM](operations_iam_permissions.md#bedrock-marketplace-endpoints-iam).

    Two endpoints deployed from the same Marketplace listing in the same region publish only one model, because both carry the listing's name. Set [`AWS_BEDROCK_ALLOW_MARKETPLACE_ENDPOINT_ARN`](operations_configuration_models.md#bedrock-allow-marketplace-endpoint-arn) to let a client name the other by its ARN, or delete the duplicate.

??? failure "A Marketplace model endpoint reports usage but no cost"
    Not a fault. AWS bills a [Bedrock Marketplace model endpoint](operations_cost_management.md#bedrock-marketplace-model-endpoints) by the instance-hour, whether or not it ever serves a request, so there is no per-token rate to report against the requests that use it — nothing is invented in its place. Read the real charge from AWS Cost Explorer or the SageMaker hosting line of your bill. The same holds for a [SageMaker AI endpoint](operations_cost_management.md#sagemaker-endpoints-cost).

??? failure "The first request to a SageMaker AI model takes minutes"
    Expected, and it is the feature working. An [endpoint scaled to zero](https://docs.aws.amazon.com/sagemaker/latest/dg/endpoint-auto-scaling-zero-instances.html) has no capacity to answer with, and the request itself is what makes AWS provision an instance again — a few minutes for the scaling alarm, the instance and the model to load. stdapi.ai holds the connection and retries rather than failing, so the caller gets a slow answer instead of an error, and callers arriving during the wait share it rather than each starting one.

    - Bound it with [`AWS_SAGEMAKER_WARMUP_TIMEOUT`](operations_configuration_models.md#aws-sagemaker-warmup-timeout), or set it to `0` to fail fast instead of waiting.
    - Keep the endpoint warm by giving its Application Auto Scaling target a minimum capacity of one — at the cost of an instance-hour bill that never stops.
    - Check that any load balancer, proxy or client SDK in front of the deployment tolerates that much silence before the first byte.

??? failure "Every request to a SageMaker AI model answers 503 after several minutes"
    The endpoint never came back up, and the server waited its whole [warm-up budget](operations_configuration_models.md#aws-sagemaker-warmup-timeout) before saying so. The message to the caller is deliberately generic; the server log carries the real cause at `WARNING`, naming the endpoint and the elapsed budget.

    **By far the most likely cause is the scaling alarm.** An endpoint with a minimum instance count of zero and no CloudWatch alarm on `NoCapacityInvocationFailures` never scales back up at all: every request fails, forever. Check the alarm exists, that its step-scaling policy is its alarm action, and that it is not stuck in `INSUFFICIENT_DATA`:

    ```bash
    aws cloudwatch describe-alarms --region <region> \
      --query "MetricAlarms[?MetricName=='NoCapacityInvocationFailures'].[AlarmName,StateValue,AlarmActions]"
    ```

    Then check the component is actually being asked to grow, and why it might not be:

    ```bash
    aws sagemaker describe-inference-component --region <region> \
      --inference-component-name <name> --query 'RuntimeConfig'
    ```

    A `DesiredCopyCount` of 1 with a `CurrentCopyCount` of 0 that never moves means the copy cannot be placed — most often the component asks for more memory or accelerators than the instance type has, which `CreateInferenceComponent` accepts and the placement then fails on. A longer budget will not fix either case.

??? failure "A declared SageMaker AI endpoint does not appear in the model list"
    An endpoint is served because you named it, so the entry is the first thing to check.

    - **The model ID collides with a model already in the catalogue.** The declaration is ignored and the reason is reported in the startup log; give the endpoint a model ID of its own.
    - **The declaration is malformed.** [`AWS_SAGEMAKER_ENDPOINTS`](operations_configuration_models.md#aws-sagemaker-endpoints) is a JSON object and an unknown field is refused at startup rather than ignored, so the server would not have started.
    - **The catalogue has not refreshed.** A declaration is picked up when the server starts; changing the environment needs a restart, as every setting does.

??? failure "A SageMaker AI model answers 503 immediately, and the log names two permissions"
    The endpoint refused the server's credentials. The OpenAI-compatible route takes a short-term API key derived from the server's own role, which needs `sagemaker:CallWithBearerToken` on `*` **in addition to** `sagemaker:InvokeEndpoint` on the endpoint ARN — a policy carrying only the second is refused with a `403` the server reports as a feature being unavailable. See [SageMaker AI Endpoints IAM](operations_iam_permissions.md#sagemaker-endpoints-iam).

??? failure "Every Bedrock Mantle model is missing, while the classic Bedrock models are all there"
    The catalog lists the classic Bedrock models and none of the ones served through Amazon Bedrock Mantle, in every configured region at once. A per-region or per-model gap looks different: this is the shape of a network policy that reaches one endpoint and not the other.

    Bedrock Mantle is served from `bedrock-mantle.<region>.api.aws` — a different domain from classic Bedrock's `bedrock-runtime.<region>.amazonaws.com`, so an allowlist, firewall rule, proxy exception or VPC endpoint written for `amazonaws.com` does not cover it.

    - Allow `bedrock-mantle.<region>.api.aws` outbound, for every region in `AWS_BEDROCK_REGIONS`.
    - On a private deployment, create the interface VPC endpoint `com.amazonaws.<region>.bedrock-mantle` with private DNS enabled.
    - Behind a proxy, make sure `HTTPS_PROXY` is set in the task environment and that `NO_PROXY` does not exclude `api.aws`.
    - Confirm the task role carries the Bedrock Mantle permissions — without them the models are simply not listed rather than refused. See [IAM Permissions](operations_iam_permissions.md).
    - Check what was discovered: `GET /search_models` returns every model the server found, with its regions.

    See [Outbound Network Requirements](operations_deploy_advanced.md#outbound-network-requirements) for the full destination list.

    The startup warning names the region, the endpoint address and the exception chain behind the failure, so a blocked route (`ConnectionTimeoutError`), a refused connection (`ConnectionRefusedError`), an intercepting proxy's certificate (`SSLCertVerificationError`) and an unresolvable address (`ClientConnectorDNSError`) are told apart without further instrumentation.

??? failure "Bedrock Mantle models are missing in one region only — `bedrock_mantle_regions_without_endpoint`"
    The startup log lists the region under `bedrock_mantle_regions_without_endpoint` instead of `unreachable_bedrock_regions`. Bedrock Mantle is offered in fewer regions than classic Bedrock, and where it is not offered `bedrock-mantle.<region>.api.aws` has no DNS record at all — nothing to retry, and no network policy to change. See [model availability by endpoint](https://docs.aws.amazon.com/bedrock/latest/userguide/models-endpoint-availability.html).

    - Remove the region from [`AWS_BEDROCK_MANTLE_REGIONS`](operations_configuration_aws.md#bedrock-mantle-regions), or unset it to fall back to the regions of `AWS_BEDROCK_REGIONS` that offer Mantle. Classic Bedrock in that region is unaffected either way.
    - If no configured region offers Mantle, the log says so as well and no Mantle model is served — set [`AWS_BEDROCK_MANTLE_ENABLED`](operations_configuration_aws.md#bedrock-mantle-enabled) to `false` to stop the warning.
    - If AWS has since added the region, list it explicitly in `AWS_BEDROCK_MANTLE_REGIONS`: an explicit list is used as given.
    - If the address should resolve because a VPC endpoint provides it, this is instead the private-DNS case covered by *Every Bedrock Mantle model is missing* above.

### :material-restart: Startup and shutdown — `503 feature_unavailable`, slow starts, lost work { #startup-and-shutdown }

??? failure "`503` saying the feature is not available on the current server — IAM permission denied on an AWS call"
    The gateway reached the AWS service, but the **ECS task role** (or your local AWS credentials) lacks permission for the action it used. AWS returns `AccessDeniedException`, which stdapi.ai answers as a feature this deployment cannot run: HTTP `503`, error code `feature_unavailable`, and the same generic message whatever is missing. This is an IAM misconfiguration, **not** a client API-key problem, and the client is deliberately told nothing about it — **the server log names the operation, the model and the permission AWS refused**, under `error_detail`.

    - Confirm the task role grants `bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream` (and the `bedrock:Converse*` actions) for the target model ARNs.
    - For models invoked through an inference profile, allow both the profile ARN and the underlying foundation-model ARNs in the policy.
    - **By a distance the most likely cause for a [Marketplace model endpoint](operations_cost_management.md#bedrock-marketplace-model-endpoints) that is listed but every call to it is refused**: a policy scoped to `foundation-model/*` and `inference-profile/*` — copied from an example that predates Marketplace model endpoints — denies every call to one. It needs `arn:aws:bedrock:*:ACCOUNT_ID:marketplace/model-endpoint/*` as well, on the task role and, when [per-user cost attribution](operations_cost_management.md#per-user-attribution) is enabled, on the end user role too — see [Bedrock Marketplace Model Endpoints IAM](operations_iam_permissions.md#bedrock-marketplace-endpoints-iam).
    - **Anthropic models only, and not this `503`:** Claude requires a one-time use case form per AWS account or Organization. Without it the model never enters the served catalogue and naming it answers `404 Not Found` — see [the use case form entry](#models-and-the-catalogue) for the steps.
    - Audio, embeddings, and file features need permissions for the relevant services (Polly, Transcribe, Translate, Comprehend, S3) — see [IAM Permissions](operations_iam_permissions.md) for the full IAM reference.
    - A `403 permission_error` on a model call means the opposite: [per-user cost attribution](operations_cost_management.md#per-user-attribution) is enabled and the **end user's** role was denied, so the policy to fix is that role's, not the task role's.
    - A `400 invalid_request_error` naming an `s3://` input also means the opposite: the denial is on the object *the request pointed at*, in one of the external buckets declared in [`AWS_S3_ACCEPTED_BUCKETS`](operations_configuration_storage.md#aws-s3-accepted-buckets). Only the caller can fix that one — a wrong key, or a bucket policy that does not grant this deployment's role `s3:GetObject` on the object. Objects in the deployment's own buckets keep the `503` above.

??? failure "A permission is named against a region — `bedrock_regions_missing_iam_permission`"
    The startup log lists the region under `bedrock_regions_missing_iam_permission`, never under `unreachable_bedrock_regions`: AWS answered, and refused. The entry names the IAM action, the region, and the resource the call was attempted on when AWS supplied one.

    - **Read the wording first, it says whose problem it is.** `the server role is missing the IAM permission <action>` means IAM evaluated a policy and denied it — grant the action and restart. `AWS denied <action> … unless the service does not offer the operation there` means AWS refused without naming an action, which is equally how a region that does not offer the operation answers; check regional availability before editing a policy.
    - `provisioned model discovery skipped: …` is not a lost region. Provisioned throughput is offered in a subset of regions and the listing is refused in the others, so the region keeps its on-demand models and loses only any model it serves exclusively as provisioned throughput. As of 2026-08-31 that is the answer in `af-south-1`, `ap-east-2`, `ap-southeast-5`, `ap-southeast-6`, `ap-southeast-7`, `ca-west-1`, `il-central-1` and `mx-central-1`.
    - Model discovery calls `bedrock:ListFoundationModels`, `bedrock:ListProvisionedModelThroughputs` and `bedrock:ListInferenceProfiles` in every configured region. Only the provisioned-throughput listing degrades; a denial on either of the other two costs the whole region's models. See [IAM Permissions](operations_iam_permissions.md).
    - With [`AWS_BEDROCK_MARKETPLACE_ENDPOINTS_ENABLED`](operations_configuration_models.md#bedrock-marketplace-endpoints-enabled) set, `bedrock:ListMarketplaceModelEndpoints` and `bedrock:GetMarketplaceModelEndpoint` are called too, and a denial is reported the same way under `<region> (Marketplace endpoints)`. It costs only the endpoints of that region, never a foundation model.
    - Do not read any of this as a model retirement: the models are still there, this deployment just may not list them.
    - The same distinction applies per request. A denied call names its permission in the request log's `error_detail` and answers `503 feature_unavailable`, never a region failover.

??? failure "Startup takes tens of seconds — `server_start_time_ms` far above the usual few seconds"
    The `start` log event reports `server_start_time_ms` in the tens of thousands where a healthy deployment reports a few thousand. Startup reads the model catalogs of every configured region, and the ECS task metadata endpoint before them; a destination that never answers is only given up on after a timeout.

    - Read the `server_warnings` of the same event first. `ECS container metadata endpoint answered after N attempts in X s` accounts for that many seconds on its own: the endpoint is served by the ECS agent over the task ENI and answers slowly when the task is CPU-starved at boot. Raise the task CPU, or the `cpu` of the Fargate task definition, so the agent is scheduled promptly.
    - `unreachable_bedrock_regions` and `bedrock_mantle_regions_without_endpoint` each name a region that spent its full timeout budget. Removing the region from `AWS_BEDROCK_REGIONS` or `AWS_BEDROCK_MANTLE_REGIONS` removes the delay.
    - [`AWS_CONNECT_TIMEOUT`](operations_configuration_aws.md#aws-connect-timeout) bounds each connection attempt and the model-catalog fetch that follows it, so lowering it lowers what an unreachable region can cost. It also bounds failover between healthy regions, so keep it above your real inter-region latency.
    - Model discovery is per region and runs in parallel, so the count of regions costs far less than one unreachable region does.

??? failure "A deploy leaves work unfinished — `abandoned_background_tasks` in the `stop` log event"
    Some work is started outside the request that asked for it, so the caller is answered without waiting: temporary file cleanups, vector store file indexing, and the release of live audio sessions. When a task is stopped, the server waits [`SHUTDOWN_DRAIN_TIMEOUT`](operations_configuration_server.md#shutdown-drain-timeout) seconds for that work, then cancels the rest — and reports the counts as `abandoned_background_tasks` in a `stop` log event raised to `warning`.

    - A vector store file reported `failed` after a deployment, with `last_error` saying the indexing was interrupted, or a temporary object that outlived its request, is what those counts look like from the outside. Attach the file again, or let the object's lifecycle rule expire it. A file is never left `in_progress`: one whose indexing is gone is settled the next time it, its store, or the store's file list is read.
    - Raise [`SHUTDOWN_DRAIN_TIMEOUT`](operations_configuration_server.md#shutdown-drain-timeout) **and** the container stop timeout together — the container runtime sends `SIGKILL` a fixed delay after the stop signal (30 seconds by default on Amazon ECS), so raising the wait alone changes nothing past that delay.
    - Counts that persist after raising both mean the work itself is long, not that the wait is short: indexing a large file outlasts any stop timeout. Attach large files when a deployment is not in flight, and treat the operation as retryable.
    - The wait is best effort in every deployment: a Spot interruption or a hard kill ends the process regardless, so never rely on it for anything whose completion matters.

??? failure "The server refuses to start: `aws_bedrock_mantle_preferred_models is incompatible with Amazon Bedrock Guardrails`"
    A guardrail is configured — [`AWS_BEDROCK_GUARDRAIL_IDENTIFIER`](operations_configuration_bedrock.md#aws-bedrock-guardrail-identifier) or a [`MODEL_ALIASES`](operations_configuration_models.md#model-aliases-configuration) entry carrying one — while [`AWS_BEDROCK_MANTLE_PREFERRED_MODELS`](operations_configuration_models.md#bedrock-mantle-preferred-models) routes models to Amazon Bedrock Mantle, which cannot apply guardrails. Those models would be served unfiltered, so startup stops instead. The setting defaults to `openai.gpt-5.6,openai.gpt-6`, so a guardrailed deployment meets this without having set it.

    - Keep the guardrail: `export AWS_BEDROCK_MANTLE_PREFERRED_MODELS=` (empty). Every dual-homed model, the GPT-5.6 and GPT-6 families included, is then served by the classic endpoint under the guardrail — and `web_search` and `code_interpreter` are refused with a `400` for those families.
    - Keep Mantle routing: remove the guardrail configuration, or point the offending alias at a model that stays on the classic endpoint.
    - The message names the routed entries, so a deployment that listed several sees which ones are at stake.

### :material-alert-circle-outline: `400` and `429` from a model call — Bedrock request errors { #bedrock-request-errors }

??? failure "`429 Too Many Requests` — Bedrock throttling / quota"
    AWS returned `ThrottlingException`, `TooManyRequestsException`, or `ServiceQuotaExceededException` — mapped to HTTP `429` with error type `rate_limit_error`. You've hit the per-region Bedrock quota.

    - Add more regions to `AWS_BEDROCK_REGIONS`. Each region has its own independent quota — three regions ≈ triple the throughput.
    - See [Resilience & Failover](operations_resilience.md) for multi-region routing configuration.
    - Check quotas in the AWS Service Quotas console for **Amazon Bedrock**.
    - When the router put a region on a quota backoff while serving the request, the response carries a `retry-after` header (in seconds) telling the client exactly how long to wait — OpenAI, Anthropic and Cohere SDKs honour it automatically instead of guessing an exponential backoff, up to the 60 s ceiling their retry loops apply to server-supplied delays.

??? failure "`400 Bad Request` — This model is not available under data retention mode 'default'."
    A specific model is unavailable or requests to it are rejected because your account's data retention mode is incompatible with what that model requires.

    Amazon Bedrock enforces retention compatibility at invocation time: each model declares the retention modes it accepts, and if your effective mode is not among them, the request is blocked.

    **Common scenarios:**

    - Your account is set to **zero data retention (`none`)** but the model requires `default` or `provider_data_share` for safety or abuse-prevention purposes. Bedrock blocks the request to honour your retention policy. To access the model, either switch to a compatible retention mode or contact your AWS account manager to request ZDR eligibility for that specific model.
    - Your account is set to **`default`** but the model exclusively requires `provider_data_share` (typically models with mandatory provider-side safety review). The model will appear as unavailable. Enabling `provider_data_share` grants access but means AWS will share your inference data with the model provider — see [Data Privacy](operations_compliance.md#data-privacy) before enabling it.

??? failure "`400 Bad Request` — invalid parameters from Bedrock"
    Bedrock rejected the request parameters (`ValidationException` / `BadRequestException`), mapped to HTTP `400` with error type `invalid_request_error` — for example an unsupported parameter for the chosen model, an out-of-range value, or content that exceeds the model's limits.

    - Read the message detail returned in the response (correlate with `x-request-id` in the server logs). A parameter the model provider rejects carries the provider's own message.
    - Confirm the parameter is supported by the model — see the per-API **Feature Compatibility** tables.

??? failure "`400 Unknown parameter` — a field is refused under `STRICT_INPUT_VALIDATION`"
    With [`STRICT_INPUT_VALIDATION`](operations_configuration_observability.md#strict-input-validation) on, a field the gateway does not declare is refused with `400`, code `unknown_parameter`, and the field in `error.param` (e.g., `Unknown parameter: 'messages[0].x'.`). On OpenAI-compatible routes this happens only where OpenAI itself refuses the field — a `developer` message, a file or audio content part, a Responses input item — and the places OpenAI ignores one stay ignored.

    - Remove the field from the request, or move it to where the API accepts it.
    - If a client you do not control sends it, turn `STRICT_INPUT_VALIDATION` off so the gateway ignores it.

??? failure "`400 Bad Request` — a client's own control flag reaches Bedrock as a model parameter"
    Request fields stdapi.ai does not declare are forwarded to Amazon Bedrock as provider-specific inference parameters, so any parameter a model accepts can be passed through — including the highly model-specific ones no common API surface exposes. Some OpenAI-SDK-based clients also use that same channel for their *client-side* settings — LiteLLM-derived ones send `drop_params`, `api_key` or `custom_llm_provider` in `extra_body` — and Bedrock answers `ValidationException` for a field no model declares. The symptom is a route that fails for one client and works for every other.

    - The known LiteLLM control parameters are stripped by a built-in denylist, so this only appears for a name it does not yet cover. The rejected field is in the Bedrock message detail, correlated via `x-request-id` in the server logs.
    - Add that name to [`EXTRA_MODEL_PARAMS_DENYLIST`](operations_configuration_models.md#extra-model-params-denylist) — it is merged with the built-in list, and every other extra parameter keeps being forwarded.
    - If no client needs the passthrough, [`EXTRA_MODEL_PARAMS_DROP_ALL`](operations_configuration_models.md#extra-model-params-drop-all) disables it outright. Per-model defaults set through [`DEFAULT_MODEL_PARAMS`](operations_configuration_models.md#default-model-params) are unaffected — only request-supplied extras are dropped.

??? failure "A tool the request no longer declares is called anyway, or `tool_choice: \"none\"` still returns a tool call"
    Changing the tool set between the turns of one conversation works: the tools a turn declares are the only ones the model may call, and a tool dropped from that list is not offered again. The exception is the empty tool set — a turn that declares no tool at all, or sets `tool_choice` to `none` (`{"type": "none"}` on `/v1/messages`), while the conversation it replays already contains a tool call. Those earlier tools stay available for the turn, so the answer can be a call to one of them.

    - Remove the tool turns from the history — the earlier `tool_calls`/`tool` messages, `tool_use`/`tool_result` blocks or `function_call`/`function_call_output` items — and send the conversation as plain text. Nothing then keeps a tool available.
    - Or keep the history and ignore the returned call: the message is otherwise a normal answer, and the call names a tool the conversation already used, never an unknown one.
    - Declaring a *different* tool set on that turn does not help: `none` withdraws the tools the request declares, whichever they are, and only the conversation's own tools remain.
    - No setting changes this, and it is the same on `/v1/chat/completions`, `/v1/responses` and `/v1/messages`.

??? failure "`400` — `Forcing tool use (tool_choice or function_call) is not available with this model`"
    The model does not let a request force tool use, as the `tool_choice` row of the [Chat Completions compatibility table](api_openai_chat_completions.md#feature-compatibility) notes. The forced forms are `tool_choice: "required"` or a named function on `/v1/chat/completions` and `/v1/responses`, a named legacy `function_call`, and `{"type": "any"}` or `{"type": "tool"}` on `/v1/messages`.

    - Send `tool_choice: "auto"` (`{"type": "auto"}` on `/v1/messages`), or omit it, and name the tool to call in the prompt.
    - Or pick a model that accepts a forced choice, such as Claude Opus 5 or Claude Fable 5.

??? failure "Reasoning set to `none` or disabled still returns reasoning, and its tokens are billed"
    The model always reasons, so `reasoning_effort: "none"` (Chat Completions), `reasoning: {"effort": "none"}` (Responses) or `thinking: {"type": "disabled"}` (Messages) is served at its default reasoning level and a warning is recorded in the request log. Besides the Claude models that always reason, this includes OpenAI GPT-6 Astra, GPT-6.1 and later, and gpt-oss. Claude Sonnet 5.5 is served with `between_tools` instead: no thinking before the answer, but short progress updates between tool calls still arrive as summarized `thinking` blocks, billed as output tokens. See [Disabled Thinking Not Honored](api_anthropic_messages.md#extended-thinking) for the models concerned, and the `none` level under [Reasoning Control](api_openai_chat_completions.md#reasoning-control).

    - Lower the depth instead: `reasoning_effort: "low"` on `/v1/chat/completions`, `reasoning: {"effort": "low"}` on `/v1/responses`, `output_config: {"effort": "low"}` on `/v1/messages`.
    - Or pick a model that can disable reasoning, such as Claude Opus 5 or Claude Sonnet 5.

??? failure "`reasoning_effort` returns no reasoning on Claude 3.7 - 4.5"
    These models spend a thinking budget of at least 1,024 tokens out of the output limit, so a limit of 1,024 or less leaves no room for it: the request is served without reasoning and a warning is logged. See [the note under Reasoning Control](api_openai_chat_completions.md#reasoning-control).

    - Raise `max_completion_tokens` (or `max_tokens`, `max_output_tokens` on `/v1/responses`) above 1,024.

??? failure "Thinking blocks or reasoning items come back with empty text on recent Claude models"
    Claude Opus 4.7 and later, Sonnet 5, Fable and Mythos reason without returning the text by default: the thinking block keeps its signature, and the reasoning tokens are billed in `output_tokens`. Ask for a summary to read it — see [Extended Thinking](api_anthropic_messages.md#extended-thinking) and [Extended Reasoning](api_openai_responses.md#extended-reasoning).

    - `/v1/messages`: set `"display": "summarized"` in `thinking`, for example `{"type": "adaptive", "display": "summarized"}`.
    - `/v1/responses`: set `reasoning.summary` to `auto`, `concise` or `detailed`; the text comes back in the reasoning item's `summary`, not `content`.
    - `/v1/chat/completions` has no summary request, so `reasoning_content` stays empty on these models; use one of the two routes above.
    - A `400` on `"display": "updates"`: that beta value is not available; use `summarized` or `omitted`.

??? failure "A request's `service_tier`, guardrail or model parameters are ignored"
    Some configuration reaches the model that the client did not send, or the value the client sent is not the one applied. Two layers of server-side configuration sit behind every request, and both are deliberate.

    - The model name may be an alias carrying its own configuration — check the entry in [`MODEL_ALIASES`](operations_configuration_models.md#model-aliases-configuration) for that name. Requests naming it get its service tier, guardrail, metadata and model parameters; requests naming the target model directly do not.
    - The reverse also happens: alias and server-wide configuration apply to models served through Amazon Bedrock's Converse and InvokeModel operations. A model served through Amazon Bedrock Mantle applies the request's own values only, so a configured service tier, metadata or model parameters are ignored there by design — see [the scope note](operations_configuration_models.md#model-aliases-configuration).
    - A request value is discarded on purpose when its override setting is disabled: [`AWS_BEDROCK_ALLOW_SERVICE_TIER_OVERRIDE`](operations_configuration_models.md#aws-bedrock-allow-service-tier-override) for the tier, [`AWS_BEDROCK_ALLOW_GUARDRAIL_OVERRIDE`](operations_configuration_bedrock.md#aws-bedrock-allow-guardrail-override) for the guardrail headers.
    - Otherwise the value comes from the server-wide setting for that field — [`DEFAULT_MODEL_SERVICE_TIERS`](operations_configuration_models.md#default-model-service-tiers), [`DEFAULT_MODEL_PARAMS`](operations_configuration_models.md#default-model-params) or [`AWS_BEDROCK_GUARDRAIL_IDENTIFIER`](operations_configuration_bedrock.md#aws-bedrock-guardrail-identifier). The order is always the request, then the alias, then the setting.

??? failure "`context_management` has no effect, `applied_edits` is missing, or `compact_20260112` is refused with `400`"
    [Context editing](api_anthropic_messages.md#context-editing) is applied by Claude models only. On any other model the field is accepted and ignored rather than refused, because clients such as Claude Code send it on every turn with thinking on: the answer is an ordinary `200`, `usage.input_tokens` keeps growing with the conversation, and the response carries no `context_management`.

    - The server log carries a `warning` saying the `context_management` of the request was ignored, correlated with `x-request-id` — the only place this is reported. Send the request to a Claude model to have the edits applied.
    - On a Claude model, `applied_edits` lists only the edits that cleared something; an empty list means the trigger was not reached — the default `clear_tool_uses_20250919` trigger is 100,000 input tokens — or that the tool results were too short to be worth clearing, which the Anthropic API skips too.
    - Batch results never report `applied_edits`.
    - **`400` naming `compact_20260112`**: compaction is not available, and is refused as an unknown edit type. Only `clear_tool_uses_20250919` and `clear_thinking_20251015` are accepted.
    - **`400` saying `clear_thinking_20251015` requires `thinking`, or must be the first edit**: the Anthropic API's own rule — enable `thinking` (or adaptive thinking) and list that edit first.

??? failure "A guardrail stopped flagging content that appears earlier in the conversation, or an image in the latest turn"
    [`AWS_BEDROCK_GUARDRAIL_SCOPE_TURNS`](operations_configuration_bedrock.md#aws-bedrock-guardrail-scope-turns) is set. On the chat routes the guardrail then evaluates the text of that many trailing user turns only — the saving it buys is paid for in detection; see [Evaluation Scope](operations_configuration_bedrock.md#bedrock-guardrails) for exactly what stops being checked.

    - **The conversation history is no longer submitted** — the earlier user turns, and the answers the client attributes to the model. A forbidden word or denied topic placed there is not detected. Raise the number of turns to widen the window, or unset the setting to evaluate the whole conversation again.
    - **An image sent alongside the newest question is not submitted either**: only the *text* of a scoped turn is. Unset the setting on a deployment that relies on the guardrail to screen images on the chat routes.
    - Model output is always evaluated in full, and the other routes — like a chat model served through InvokeModel rather than Converse — check the content of the request they are given, whatever this is set to.

??? failure "`/v1/moderations` returns `flagged: true` with every category `false` and every score `0.0`"
    The result is correct: the classification hit something the OpenAI moderation categories have no field for, so it is reported on the overall `flagged` field alone. Which one depends on the [model that served it](api_openai_moderations.md#model-support).

    - A guardrail resource: a denied topic, a word filter, a sensitive-information policy, a prompt attack filter or contextual grounding — every policy other than the content filters. Read the guardrail's own configuration in Amazon Bedrock to see which ones it applies.
    - `amazon.bedrock-runtime-guardrail-checks`: the prompt attack or personal data check, enabled with [`AWS_BEDROCK_GUARDRAIL_CHECKS_PROMPT_ATTACK`](operations_configuration_bedrock.md#aws-bedrock-guardrail-checks-prompt-attack) or [`AWS_BEDROCK_GUARDRAIL_CHECKS_PII_ENTITIES`](operations_configuration_bedrock.md#aws-bedrock-guardrail-checks-pii-entities). Broad PII entity types flag ordinary prose — a city name is an `ADDRESS` — so narrow the list to the types your policy covers. Each enabled check is also billed separately, which is the other symptom it produces.
    - `amazon.comprehend-toxicity`: the overall toxicity score, or a label with no OpenAI counterpart such as profanity.

??? failure "The OpenAI GPT-5.6 or GPT-6 models cost about 10% more than before, with no configuration change"
    They are served by Amazon Bedrock Mantle by default, so that their `web_search` and `code_interpreter` tools work. Mantle has no cross-region inference profiles, so those requests no longer ride the Global profile the classic endpoint uses by default, and pay the In-Region rate — exactly 10% above the Global one, on every token dimension, for every model with a published rate. GPT-6 Sol and Luna have none yet, so their usage is recorded without a cost.

    - Accept the routing and the rate: nothing to do. It is what makes [OpenAI GPT web search](api_openai_responses.md#openai-gpt-web-search) available. The same routing has two other effects: a configured Amazon Bedrock Guardrail stops the server at startup (see the entry above), and usage is billed under Bedrock Mantle rather than Bedrock in [cost reporting](operations_cost_management.md), attributed by [project](operations_configuration_aws.md#bedrock-mantle-project) rather than by IAM principal.
    - Prefer the Global rate: `export AWS_BEDROCK_MANTLE_PREFERRED_MODELS=` (empty). Both families return to the classic endpoint, and their server tools are refused with a `400`.
    - A deployment that had already disabled [`AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL`](operations_configuration_aws.md#cross-region-global) paid the In-Region rate before and pays it now: nothing changed for it.

??? failure "`400` — `web_search` or `code_interpreter` refused for an OpenAI GPT-6 model"
    Amazon Bedrock serves these tools on the Mantle endpoint only. The GPT-6 models are routed there by default, but only where a configured Mantle region lists them, and Mantle offers them in only a few US regions: a deployment none of whose regions does keeps serving them on the classic endpoint, where no search or code interpreter can run.

    - Set [`AWS_BEDROCK_MANTLE_REGIONS`](operations_configuration_aws.md#bedrock-mantle-regions) to your current Mantle regions plus one whose Mantle endpoint lists the model. An explicit value replaces the default (the Mantle-capable subset of `AWS_BEDROCK_REGIONS`) rather than adding to it, so list every region you still want. Every request for the model is then served from that region, which may sit outside your usual geography, at the In-Region rate; other Mantle-served models may be served from it too.
    - Check that [`AWS_BEDROCK_MANTLE_PREFERRED_MODELS`](operations_configuration_models.md#bedrock-mantle-preferred-models) still names `openai.gpt-6`: an explicit value replaces the default.
    - A request authenticated with a [tenant AWS credential](operations_authentication_security.md#tenant-aws-credentials) is always served by the classic endpoint, and a deployment with a guardrail cannot route to Mantle at all.

### :material-waveform: Audio, speech and realtime — transcription, text-to-speech, WebRTC, WebSocket { #audio-speech-and-realtime }

??? failure "`400 Bad Request` — text-to-speech rejects a long `input`"
    The message states the length the server accepts (`'input' is limited to 3,000 characters…`, or 20,000 with a generative voice). A generative voice speaks up to 20,000 characters unaided; every other voice, and longer generative input, is synthesized into an S3 bucket co-located with the region serving the request, and none is configured there.

    - Set [`AWS_S3_BUCKET`](operations_configuration_storage.md#aws-s3-bucket) for the first region of `AWS_BEDROCK_REGIONS`, and an [`AWS_S3_REGIONAL_BUCKETS`](operations_configuration_storage.md#aws-s3-regional-buckets) entry for every other region that may serve speech — the server log names the one that was missing.
    - Grant the task role `polly:StartSpeechSynthesisStream` (generative voices beyond 3,000 characters), `polly:StartSpeechSynthesisTask`, `polly:GetSpeechSynthesisTask`, and S3 read/write/delete on those buckets — see [IAM Permissions](operations_iam_permissions.md#text-to-speech-optional).
    - A generative request that still behaves like the others — rejected without a bucket, or slower than expected with one — is missing `polly:StartSpeechSynthesisStream`; the server log names the failure.
    - Up to 3,000 characters (6,000 including SSML markup) never needs a bucket; the limits and the expected latency are in [Long Input](api_openai_audio_speech.md#long-input).

??? failure "`400 Bad Request` — a transcription is rejected only for one model"
    The message names the condition: `amazon.nova-2-sonic-v1:0` serves `json` and `text` only, and accepts at most 10 minutes of audio per request. It returns no timestamps and does not report a detected language, so subtitles, `verbose_json`, `diarized_json` and `timestamp_granularities` cannot be produced from it.

    - Request `json` or `text`, or send the same audio to `amazon.transcribe`, which produces timestamps, SRT/VTT subtitles, speaker diarization and longer recordings — see [Transcriptions](api_openai_audio_transcriptions.md).
    - The same model and the same limits apply on [Translations](api_openai_audio_translations.md).
    - `amazon.transcribe-medical` refuses `srt` and `vtt` (request `verbose_json` for timed segments), any `language` other than US English, and a `Specialty` other than `PRIMARYCARE` without `stream=true` — see [Medical transcription](api_openai_audio_transcriptions.md#medical-transcription).

??? failure "`503 Service Unavailable` — medical (or live) transcription is not available on the current server"
    Every configured region refused `amazon.transcribe-medical`: medical transcription runs in fewer regions than `amazon.transcribe`, and streamed medical transcription in fewer still, so a deployment whose regions all lack it answers `503` while standard transcription works. A streamed request falls back to a non-streamed transcription when no region offers live medical transcription, except with a `Specialty` other than `PRIMARYCARE`, which then reports *live transcription* as not available. The server log names the refused action.

    - Add a region that offers it to [`AWS_BEDROCK_REGIONS`](operations_configuration_aws.md#aws-bedrock-regions), with a co-located bucket for non-streamed requests — see the [supported regions](https://docs.aws.amazon.com/general/latest/gr/transcribe.html).
    - Setting [`AWS_TRANSCRIBE_REGION`](operations_configuration_aws.md#aws-transcribe-region) to such a region also works, but it pins **all** transcription, `amazon.transcribe` included, to that one region with no failover, and needs a bucket there — check it against your data-residency requirements first.
    - Grant the task role the medical actions — `transcribe:StartMedicalTranscriptionJob`, `transcribe:StartMedicalStreamTranscription`, `transcribe:GetMedicalTranscriptionJob`, `transcribe:DeleteMedicalTranscriptionJob` — see [Speech-to-Text](operations_iam_permissions.md#speech-to-text-optional).

??? failure "`400 Bad Request` — every transcription fails after setting an output encryption key"
    The message is Amazon Transcribe's own failure reason for the job, and it names KMS. Only requests that stage audio in a bucket are affected: [`AWS_TRANSCRIBE_OUTPUT_ENCRYPTION_KEY_ARN`](operations_configuration_storage.md#aws-transcribe-output-encryption-key-arn) encrypts the job's output, so streamed transcriptions, which write nothing, keep working — which is what makes the failure look model-specific at first.

    - Grant the task role `kms:GenerateDataKey` and `kms:Decrypt` on that key, **and** allow the same role in the key's own policy — a grant on only one of the two denies the job. See [Speech-to-Text](operations_iam_permissions.md#speech-to-text-optional).
    - With [`AWS_TRANSCRIBE_REGION`](operations_configuration_aws.md#aws-transcribe-region) unset, a job runs in whichever candidate Region has a co-located bucket, so a single-Region key fails as soon as failover moves the job. Use a [multi-Region key](https://docs.aws.amazon.com/kms/latest/developerguide/multi-region-keys-overview.html), or pin the Region.
    - A key policy conditioned on the [encryption context](operations_configuration_storage.md#aws-transcribe-output-encryption-key-arn) must not require `stdapi-ai.user_id`: it is sent only when the request identifies an end user, so requiring it denies every anonymous call.

??? failure "A streamed transcription delivers nothing until the recording ends"
    `stream=true` always answers with server-sent events, but they arrive phrase by phrase only when the request can be served that way. Otherwise every event is delivered at once, after the whole recording has been read — which reads as a stream that hangs and then completes in a single burst. See [Streaming](api_openai_audio_transcriptions.md#streaming).

    - **Name the language expected**: send `language`, or two or more `languages`. A request naming neither has to read the recording before it can tell which language it is in.
    - **Set [`AWS_TRANSCRIBE_STREAM_LANGUAGES`](operations_configuration_storage.md#aws-transcribe-stream-languages)** to the languages your callers actually send, which gives phrase-by-phrase delivery to requests that name none.
    - **Drop provider-specific parameters other than `VocabularyName`, `VocabularyFilterName` and `VocabularyFilterMethod`.** `MaxSpeakerLabels`, `ShowSpeakerLabels`, `ChannelIdentification`, `MaxAlternatives`, `ToxicityDetection`, `ContentRedaction`, `IdentifyMultipleLanguages` and the rest are honoured in full, but the request that uses one is answered in a single burst rather than phrase by phrase.
    - **Check the live-session permission**: when no configured region can open a live session — `transcribe:StartStreamTranscription` (`transcribe:StartMedicalStreamTranscription` for the medical model) is missing, or no region offers streaming — the request is served as a transcription job, which needs a transcription bucket. Grant the [permission](operations_iam_permissions.md#speech-to-text-optional) or add a region that streams.
    - `response_format=diarized_json` on its own does not cost the phrase-by-phrase delivery: its `transcript.text.segment` events are interleaved with the deltas like any other.

??? failure "`503 Service Unavailable` — a speech or speech-to-speech request fails a few seconds after it starts"
    Real-time audio requests must become ready within [`AWS_CONNECT_TIMEOUT`](operations_configuration_aws.md#aws-connect-timeout) in each candidate region — that budget covers the connection, the initial handshake and the first response together, not the connection alone. On a high-latency or NAT-fronted network the default of 5 seconds can expire in every region, and the request then ends as a `503`.

    - Raise [`AWS_CONNECT_TIMEOUT`](operations_configuration_aws.md#aws-connect-timeout) to 10 seconds or more; `AI_RESPONSE_TIMEOUT` governs the response itself and has no effect here.
    - The server log names each Region that was abandoned and why.

??? failure "A WebSocket upgrade to `/v1/realtime` answers `404`"
    The upgrade request never reached the [Realtime API](api_openai_realtime.md) route — either the URL is wrong, or something in front of the deployment does not forward a WebSocket upgrade at all.

    - Check the path: the route lives at `${OPENAI_ROUTES_PREFIX}/v1/realtime` (`/v1/realtime` with the default empty prefix). A client dialling the wrong prefix reaches no route and gets `404` from the framework itself, before authentication is even checked.
    - An Application Load Balancer forwards a WebSocket upgrade by default, but anything placed **in front of** it — a CDN, an API Gateway REST API (which has no WebSocket support at all), a reverse proxy that does not pass through `Connection: Upgrade` / `Upgrade: websocket` — answers its own `404` or refuses the upgrade first. See [WebSocket-Capable Deployment](operations_deploy_advanced.md#websocket-capable-deployment-realtime-api).
    - Confirm the deployment actually ships the Realtime API: this is a versioned feature, not every earlier release includes it.

??? failure "`POST /v1/realtime/calls` answers `404`"
    The WebRTC transport is an **operator opt-in**: without [`REALTIME_WEBRTC_ENABLED`](operations_configuration_bedrock.md#realtime-webrtc-enabled) the endpoint answers `404`, and the error message says so. Enabling it is a deployment-architecture decision — a UDP media path straight to the task, a single instance — not a flag to flip casually; see [WebRTC calls terminated by the gateway](api_openai_realtime.md#webrtc-calls) and [the deployment section](operations_deploy_advanced.md#webrtc-and-sip-need-their-own-ingress) first.

    - **A browser does not need it.** It connects to the same `WS /v1/realtime` with an [ephemeral client secret](api_openai_realtime.md#ephemeral-client-secrets) in the `Sec-WebSocket-Protocol` list, and captures and plays the audio itself.
    - **A client that must speak SIP** — a phone line — belongs behind a voice-agent framework that terminates the media itself and reaches this API over the WebSocket; the SIP call verbs (`accept`, `reject`, `refer`) answer `400` on every deployment. See [Put WebRTC or a phone line in front of the gateway](api_openai_realtime.md#put-webrtc-or-a-phone-line-in-front-of-the-gateway).
    - **Do not try to route the media UDP through the deployment's ALB**: its listeners carry HTTP and HTTPS only. The [Terraform module's media mode](operations_deploy_advanced.md#webrtc-and-sip-need-their-own-ingress) gives the task the public UDP path instead.

??? failure "A WebRTC call gets an SDP answer but carries no audio"
    The SDP exchange rides ordinary HTTPS through the ALB, so it succeeds even when the media path is broken: the failure only appears as a call that connects and stays silent (the browser's `iceConnectionState` sticks at `checking` or `failed`).

    - **The task has no public UDP path.** The default deployment keeps tasks in private subnets behind an ALB that cannot carry UDP. Enable the module's `realtime_webrtc_media_enabled`, which assigns the task a public IP and opens the UDP port range in both directions — see [the deployment section](operations_deploy_advanced.md#webrtc-and-sip-need-their-own-ingress).
    - **The subnets' network ACLs drop the media.** On the VPC the module creates, media mode widens the application subnets' NACLs itself, so this only bites on subnets you brought through `subnet_ids` — the module cannot write rules in a VPC it did not create. Allow the UDP media range inbound and the ephemeral range `1024-65535` in both directions on those subnets; without it the SDP exchange still succeeds over TCP and no media ever arrives, which is exactly this symptom.
    - **The answer advertises a private address.** Behind 1:1 NAT (an ECS task with a public IP) the process sees only its private address; without [`REALTIME_WEBRTC_STUN_SERVER`](operations_configuration_bedrock.md#realtime-webrtc-stun-server) the ICE candidates it sends are unreachable. The module's media mode sets it.
    - **The security group does not cover the negotiated port.** Media binds OS-assigned ephemeral UDP ports; the opened range must match the kernel's ephemeral range (the module opens `32768-60999` by default).
    - **The caller's network blocks UDP.** Corporate networks often do; only a TURN relay you run gets media through — see [`REALTIME_WEBRTC_TURN_SERVER`](operations_configuration_bedrock.md#realtime-webrtc-turn-server).
    - **Call control lands on another instance.** `hangup` or `WS /v1/realtime?call_id=...` answering `404` naming the instance means the deployment runs more than one task; media mode requires a single instance.

??? failure "`POST /v1/realtime/calls` answers `400 invalid_offer` — no reachable ICE candidate"
    The offer named only addresses the gateway will not probe. It screens the candidates a caller offers before answering, because those addresses are where its own ICE checks go — see [What a WebRTC call demands of the deployment](api_openai_realtime.md#webrtc-calls).

    - **The caller is on the deployment's own network** (same VPC, on-premises, a LAN), so every candidate it has is a private address. Set [`REALTIME_WEBRTC_ALLOW_PRIVATE_CANDIDATES`](operations_configuration_bedrock.md#realtime-webrtc-allow-private-candidates); it is off by default so an untrusted caller cannot aim UDP probes inside the VPC.
    - **The caller offered only mDNS (`.local`) candidates**, which browsers use to hide the host address. Those are dropped whatever the setting is — nothing resolves them from a server — and the caller needs a server-reflexive candidate: configure a STUN server in the client's own `RTCPeerConnection`, or a TURN relay.

??? failure "A WebRTC call drops at exactly 8 minutes"
    Expected: the model session behind the call is capped at 480 seconds by Amazon Nova Sonic, and when it ends the peer connection is torn down with it — see [Session Lifecycle and Limits](api_openai_realtime.md#session-lifecycle-and-limits). A phone-length conversation needs the caller to reconnect with a fresh offer; context does not carry over.

??? failure "A realtime session closes exactly at 8 minutes, or unexpectedly earlier"
    Two different things end a [Realtime](api_openai_realtime.md) session, and the close code tells them apart — inspect it in the client's WebSocket close handler.

    - **Close code `1000`, reason `session_expired`**: expected. Every session is capped at 8 minutes; reconnect to continue the conversation — see [Session Lifecycle and Limits](api_openai_realtime.md#session-lifecycle-and-limits).
    - **Closes with no code from the gateway at all, well before 8 minutes**: the load balancer's idle timeout fired on a quiet stretch between spoken turns. Raise `alb_idle_timeout` to at least 8 minutes (480 seconds) — see [The idle timeout bounds a session](operations_deploy_advanced.md#the-idle-timeout-bounds-a-session).
    - **Closes during a deployment, scale-in, or Spot interruption**: the ECS task holding the session was replaced. There is no live handoff between tasks — see [A deploy truncates open sessions](operations_deploy_advanced.md#a-deploy-truncates-open-sessions).
    - **Close code `1001`, reason `server_shutdown`**: the deployment was shutting down when the session was still open; reconnect once it is back.
    - **Close code `3000`**: a fatal error, not a limit. The reason is `<error type>.<error code>`, and a terminal `error` event carrying the same detail was sent just before the close frame.

??? failure "A realtime session goes silent after the model calls a function"
    The model is waiting for the result. A [function call](api_openai_realtime.md#function-calling) suspends the conversation until a `function_call_output` item naming the same `call_id` comes back, and nothing else is spoken in the meantime.

    - **Answer every call, including a failed one.** Return `{"error": "..."}` rather than nothing; there is no timeout that resumes the conversation for you.
    - **Reply to the `call_id` the call carried**, as `response.function_call_arguments.done` reports it. Any `call_id` is accepted and carried to the model, so a wrong one is not refused — it simply leaves the real call unanswered and the session silent.
    - **Answer each call once.** A repeated answer is carried to the model too, which then sees a turn the application did not intend.
    - A `session.update` that changes `tools` or `tool_choice` after the model has answered anything is refused: declare them before the conversation opens, as with the voice and the audio formats.

??? failure "A raw WebSocket client gets `403` that looks like an authentication failure"
    If the deployment's WAF is enabled with the AWS-managed Common Rule Set (`alb_waf_enabled = true`), its `NoUserAgent_HEADER` rule blocks any request — including a WebSocket upgrade — that carries no `User-Agent` header, with a plain `403` that is easy to mistake for a rejected credential.

    - Every mainstream WebSocket client library sets a `User-Agent` automatically; this only surfaces with a hand-rolled client (a bespoke SIP/telephony bridge, a minimal test script).
    - Check the WAF sampled requests in the console to confirm `NoUserAgent_HEADER` is the rule that matched, before assuming the API key or ephemeral secret is wrong.
    - Have the client send any non-empty `User-Agent`, or exclude the rule for the Realtime path. See [AWS WAF's `NoUserAgent_HEADER` rule reads like an auth failure](operations_deploy_advanced.md#aws-wafs-nouseragent_header-rule-reads-like-an-auth-failure).

??? failure "An ephemeral client secret works on one instance and is rejected on another"
    A [Realtime API ephemeral client secret](api_openai_realtime.md#ephemeral-client-secrets) is a signed token with nothing stored server-side, verified by re-checking its signature against a shared key — every instance must sign with the **same** key for that to work.

    - With no [`API_KEY`](operations_configuration_authentication.md#api-key)-family setting configured at all, each instance falls back to a **random signing key generated per process**, so a secret minted by one instance never verifies on another — the symptom is intermittent rejection that tracks which instance the client's connection happened to land on.
    - Set [`REALTIME_CLIENT_SECRET_KEY`](operations_configuration_bedrock.md#realtime-client-secret-key) explicitly to a value shared by every instance; this also covers a deployment with no API key by design (e.g. behind an IP-restricted ALB).
    - A deployment that already configures an API key is unaffected: the signing key is derived from it automatically, and that same key is already shared across instances.
    - Rotating the API key or `REALTIME_CLIENT_SECRET_KEY` invalidates every client secret minted before the change, the same as an expired one.

### :material-file-multiple: Files, batches, vector stores and conversations { #files-batches-and-vector-stores }

??? failure "S3 error on image generation or audio transcription"
    The S3 bucket is missing, unreachable, or in the wrong region.

    - The Terraform module creates the bucket automatically unless you pass your own via `aws_s3_bucket`.
    - If you're using your own bucket: `AWS_S3_BUCKET` must point to a bucket in the same region as the **first** entry in `AWS_BEDROCK_REGIONS`.
    - Verify the ECS task IAM role has `s3:PutObject` / `s3:GetObject` on the bucket.
    - **`503` saying transcription is not available on the current server** means no region that can run a transcription has a bucket at all: set [`AWS_S3_BUCKET`](operations_configuration_storage.md#aws-s3-bucket), [`AWS_TRANSCRIBE_S3_BUCKET`](operations_configuration_storage.md#aws-transcribe-s3-bucket) or an [`AWS_S3_REGIONAL_BUCKETS`](operations_configuration_storage.md#aws-s3-regional-buckets) entry for it. The server log names the settings.
    - **`503` saying the `'url'` response format is not available on the current server** means there is no bucket to host the images a `url` response points at: set [`AWS_S3_BUCKET`](operations_configuration_storage.md#aws-s3-bucket), or request `response_format="b64_json"`, which needs no storage. Image requests are refused before anything is generated, so a misconfigured deployment is never billed for an image it cannot serve; with a guardrail configured, the prompt's check runs first and is billed, because a prompt the guardrail blocks is answered as such rather than as a missing bucket.

??? failure "A batch is refused, stuck, or its requests fail one by one"
    The [Batch API](api_openai_batches.md) and the [Message Batches API](api_anthropic_batches.md) refuse at submit what the backend would otherwise fail hours later, so most surprises land in the create call.

    - **`503` (or `529` on `/anthropic/...`) on every batch endpoint**: the deployment declares no batch service role. Set [`AWS_BEDROCK_BATCH_ROLE_ARN`](operations_configuration_bedrock.md#aws-bedrock-batch-role-arn) and grant the policies in [Batch Inference IAM](operations_iam_permissions.md#batch-inference). The server also reports the disabled feature in the `server_warnings` field of its `start` log event.
    - **`400` naming a minimum of 100 requests**: batches run on a backend with a floor of 100 requests **per model**. A batch naming several models must reach it for each of them; combine the small ones or send them without batching. The 100 is the default of the per-model Amazon Bedrock quota *Minimum number of records per batch inference job* ([Amazon Bedrock quotas](https://docs.aws.amazon.com/general/latest/gr/bedrock.html)), and it is the value checked here whatever your own account's quota says.
    - **The batch stays `validating` (`in_progress`) for several minutes**: expected. Validation alone takes a few minutes before any request runs, and the whole batch has a 24-hour window. Poll rather than resubmit — a resubmission is a second, separately billed batch.
    - **The create call fails where you expected a batch that later reports `failed`**: every problem in the input file — an unparseable or truncated line, a `url` that is not the batch's endpoint, a method other than `POST`, a missing `body`, a repeated `custom_id`, a file uploaded with another `purpose` — is answered by `POST /v1/batches` itself, with a `400` naming what to fix. No batch is created, so there is none to poll and no `errors.data[]` to read; [What the input file is checked for](api_openai_batches.md#input-file-validation) lists every case. `failed` is reported only for a batch that was accepted and then could not run at all.
    - **`400` naming tool use or a structured output schema**: neither is available in a batch. Remove `tools`/`tool_choice` and `response_format` of type `json_schema`, or send those requests without batching.
    - **`400` saying the model is not available for batched requests**: not every model can run batched. Pick another one; when the batch named several models, no sibling job is left running.
    - **A model that batches fine is missing from `search_models?batch=true`, or reports `batch: false`**: that flag is a discovery hint published on a best-effort basis and is never used to reject anything — submit the batch and let the answer decide. It is reported for no model at all while [`COST_TRACKING`](operations_configuration_observability.md#cost-tracking) is disabled, and for a few seconds after startup while the catalogue is still being built.
    - **`503` on creation, with nothing wrong with the request**: the backend refused the job for a reason that is not the model — the account's batch quota for that model, the service role, or a restriction such as a model the provider marked legacy and the account has not used in the last 30 days. The client message is deliberately generic; the server log carries the reason the backend gave, as a warning.
    - **`503` on creation, after the endpoints answered normally**: the task role is missing `bedrock:CreateModelInvocationJob` or the `iam:PassRole` statement on the batch service role; the server log names which. A batch that starts and then fails without results usually means the service role itself cannot read or write the bucket under [`AWS_S3_BATCHES_PREFIX`](operations_configuration_storage.md#aws-s3-batches-prefix) — the reason the backend gives is logged as a warning when a job reports `Failed`.
    - **A batch reports no cached tokens, whatever its requests asked for**: prompt caching does not apply to batched requests, on any model. A cache hint — `cache_control` on `/anthropic/v1/messages/batches`, `prompt_cache_key` or `prompt_cache_breakpoint` on `/v1/batches` — is accepted and dropped rather than refused, so the request is answered normally and no cached tokens are reported for it. There is no discount to lose: batched requests are billed at the batch rate already.
    - **The first read after the batch ends is slow**: the results are translated and published on that read. Later reads are immediate.
    - **The result files are gone, or never go away**: a batch created with `output_expires_after` deletes both files that long after they are written, and one created without it keeps them until they are deleted with the [Files API](api_openai_files.md). The clock starts when the results are published, not when the batch was created.

??? failure "A batch is missing from the listing, but still answers when retrieved by ID"
    The listing answers from a window of the most recent batches, found by a seek bounded to a fixed number of storage requests. A burst of thousands of batches created inside the same minute can outrun that budget, and the ones beyond it fall outside the window.

    - The record is intact: [`GET /v1/batches/{batch_id}`](api_openai_batches.md) returns it, and so does cancelling or reading its output. Only the *listing* is bounded — see [Listing Order](api_openai_batches.md#listing-order).
    - A cursor that points outside the current window returns an empty page rather than an error, so a paginating client stops early instead of failing.
    - Record the `id` each `POST /v1/batches` returns and address batches by it, rather than rediscovering them through the listing. Amazon Bedrock's own batch quotas bound how fast batches can realistically be created, so this density is hard to reach by accident.

??? failure "A vector store file stays in progress, fails, or returns nothing"
    Indexing runs after the response is sent, so a file is `in_progress` for a moment by design — see [Vector Stores](api_openai_vector_stores.md#indexing-is-asynchronous).

    - **`503` on every vector store endpoint**: the deployment declares no vector storage. Set [`AWS_S3_VECTORS_BUCKET`](operations_configuration_storage.md#aws-s3-vectors-bucket) and [`AWS_S3_VECTORS_REGION`](operations_configuration_storage.md#aws-s3-vectors-region), keep [`AWS_S3_BUCKET`](operations_configuration_storage.md#aws-s3-bucket) set, and grant the [Vector Stores IAM permissions](operations_iam_permissions.md#vector-stores-optional).
    - **`503` on one operation only** (creating a store, searching, deleting): a single `s3vectors` action is missing from the task role. The client message is deliberately the same as above; the server log names the action and the bucket.
    - **Creating a store fails**: the vector bucket must already exist, in the Region named by [`AWS_S3_VECTORS_REGION`](operations_configuration_storage.md#aws-s3-vectors-region). A vector bucket is a Region-local resource with no failover, so a bucket in another Region is not reachable at all.
    - **`400` on the attach, or a file settling as `failed` with `unsupported_file`**: the store does not index that file type. A file attached on its own is refused there and then, since its content type answers the question; one attached in a batch is reported on the file. Read the message —  it names what **this** store indexes, since that differs per store. A store the server owns indexes text only, so a PDF or an office document settles here; convert it first — [RAG Pipelines](use_cases_rag.md#document-parsing) shows a conversion stage — or attach it to a [knowledge base store](api_openai_vector_stores.md#knowledge-base-stores), which indexes those formats as they stand. When the message names formats and the file is already one of them, its bytes are not what the content type claims.
    - **A file settles as `failed` with `server_error`**: indexing hit a backend error, or was interrupted before it finished — `last_error.message` says which. Interrupted means the server was replaced, scaled in or killed while it was indexing, and nothing else is wrong. Check the `background` log event sharing the request's `id` for the backend case; attach the file again in both. To stop losing that work at every deployment, give the deployment an indexing queue — [`AWS_SQS_VECTOR_STORE_QUEUE_URL`](operations_configuration_storage.md#aws-sqs-vector-store-queue-url) — and another server finishes the job instead.
    - **A file stays `in_progress` far longer than the others**: a large file is many passages, each embedded in turn, and indexing is bounded server-wide, so a file attached while others are being indexed waits its turn. A file no server is indexing any more settles as `failed` rather than waiting for good, so an unchanging `in_progress` is work that is still queued.
    - **A search returns nothing after attaching**: the store is still indexing (`status` is `in_progress`), the store has passed its expiration (`status` is `expired`), or the `filters` match no file. A filter applies to the file's `attributes`, never to its content.
    - **`409` on an update**: several requests are changing the same store at once. Retry the request.

??? failure "Vector store indexing is not picked up by the queue, or a queued file never settles"
    Only deployments that set [`AWS_SQS_VECTOR_STORE_QUEUE_URL`](operations_configuration_storage.md#aws-sqs-vector-store-queue-url) hand indexing to a queue; without it, indexing runs in the server that accepted the request and everything below is expected behaviour rather than a fault.

    - **The server refuses to start, naming the setting**: the URL is not an Amazon SQS queue URL (`https://sqs.<region>.amazonaws.com/<account-id>/<queue-name>`), it names a FIFO queue, or [`AWS_S3_VECTORS_BUCKET`](operations_configuration_storage.md#aws-s3-vectors-bucket) is unset. FIFO is refused because its deduplication would silently drop a legitimate re-attach of the same files.
    - **A `start` log warning says the queue could not be described**: the queue does not exist, or the task role lacks `sqs:GetQueueAttributes`. The deployment still runs and still queues, but it cannot read your redrive policy, so it falls back to its own retry count. Grant the [Durable Vector Store Indexing permissions](operations_iam_permissions.md#durable-vector-store-indexing).
    - **A `start` log warning says the queue has no dead-letter queue**: add a redrive policy. Without one, the message of a file that cannot be indexed is dropped once its retries run out instead of being kept for inspection.
    - **Every file still settles as `failed` after a deployment**: the send is failing, which the server log reports at `error` naming `sqs:SendMessage`. A deployment that cannot queue keeps indexing in-process, which is exactly the behaviour the setting was meant to replace, so the symptom looks like the setting doing nothing.
    - **Files sit `in_progress` for minutes under load**: a server only takes jobs off the queue while it is not busy answering requests, so indexing yields to clients by design. Scale out, or wait.
    - **A file settles as `failed` although the queue is configured**: the job ran out of deliveries. Its message is in your dead-letter queue; the server log says so at `error`. Attach the file again once the underlying cause is fixed.
    - **Files attached with a tenant key, or by an identified end user, never reach the queue**: a queued job runs under the server's own identity, so a request whose embeddings are billed to a [tenant AWS credential](operations_authentication_security.md#tenant-aws-credentials) or attributed to an end user under [`AWS_BEDROCK_USER_ROLE_ARN`](operations_configuration_bedrock.md#aws-bedrock-user-role-arn) is indexed in the server that accepted it, keeping the spend where the request put it. Those files settle as `failed` when that server stops first; attach them again.

??? failure "A `vs_kb_...` vector store answers `404`, or refuses a file attached to it"
    A [knowledge base store](api_openai_vector_stores.md#knowledge-base-stores) is addressed, never created, so most of these are configuration rather than a bad request.

    - **`404` on every route of a `vs_kb_...` identifier**: the knowledge base is not listed in [`AWS_BEDROCK_KNOWLEDGE_BASE_IDS`](operations_configuration_storage.md#aws-bedrock-knowledge-base-ids), or it does not exist in the first [`AWS_BEDROCK_REGIONS`](operations_configuration_aws.md#aws-bedrock-regions) entry, or the task role lacks the read permission on it. The three cases answer identically **by design**, so the allowlist cannot be probed by a client; the server log says which one it was. Check the setting first, then grant the [Knowledge Base Vector Stores permissions](operations_iam_permissions.md#knowledge-base-vector-stores) on the knowledge base ARN.
    - **`503` when attaching a file, while search and listing work**: the knowledge base has more than one data source, so which one a document belongs to is ambiguous. Name it in the setting as `<knowledgeBaseId>/<dataSourceId>`.
    - **`400` saying files cannot be attached to the store, while search and listing work**: the allowlisted data source keeps its corpus in sync from somewhere else — a bucket, or another connected service — and takes no file handed to it. Only a **custom** data source does. Point the entry at one, as `<knowledgeBaseId>/<dataSourceId>`; the server log names the data source that refused. A knowledge base can hold both kinds, and the store keeps serving search and listing meanwhile.
    - **`400` when deleting a file a search returned**: that document belongs to the corpus behind the store rather than to the files attached here, so it is readable and never removable. Remove it where the corpus comes from.
    - **`400` on an update, a delete, a chunking strategy, a file batch, a `score_threshold` or a file's `content`**: none of these apply to a store managed outside the server. The [refusal table](api_openai_vector_stores.md#knowledge-base-stores) lists each one and what to do instead.

??? failure "Conversation items are missing, or cannot be added"
    A [conversation](api_openai_conversations.md) has a bounded lifetime and a bounded number of writes, and both are reached silently.

    - **After 30 days**, a conversation and its items are removed and every route on it returns `404`. Long-lived agents must create a new conversation rather than reusing one indefinitely.
    - **1,000 requests that add or delete items** is the per-conversation ceiling; a response bound to a conversation counts as one, whatever its number of output items. Past it, a listing stops early rather than adding failing: the gateway reads at most 1,000 invocation steps, and a single large item spans several. Start a new conversation, seeding it with the items you still need.
    - **`503` saying the API is not available on the current server** means the IAM role is missing the [Bedrock Session Storage permissions](operations_iam_permissions.md#bedrock-session-storage-optional), including `bedrock:UpdateSession`, which only the metadata update uses — a deployment created before conversations shipped fails on `POST /v1/conversations/{id}` alone. The client message is the same whichever one is absent; the server log names it.
    - **Items added by a streamed response appear when the stream ends**, not while it runs; a client that reads them from a callback fired on the terminal event must wait for the stream to close.

### :material-currency-usd: Cost attribution and the usage endpoints { #cost-and-usage }

??? failure "All Bedrock spend still lands on one identity"
    [Per-user attribution](operations_cost_management.md#per-user-attribution) reaches the AWS bill through two AWS-side steps that are easy to miss, and neither is instant:

    - **The Cost and Usage Report export must include caller identity.** Create a Data Exports CUR 2.0 export with *Include caller identity (IAM principal) allocation data* enabled; an existing export cannot be changed and must be re-created. The identity then appears in `line_item_iam_principal` as `assumed-role/<role>/<session>`.
    - **The session tag must be activated as a cost allocation tag**, in the AWS Billing console under **Cost allocation tags**, filtered by type **IAM principal**. It is only listed there after that identity has made at least one call, and takes up to 24 hours to appear in Cost Explorer.
    - **Requests that identify no end user are billed to the server**, by design. The request log's `aws_role_session_name` field is absent on exactly those requests — use it to find the clients that send no identifier, then enable [`AWS_BEDROCK_USER_ROLE_REQUIRE_IDENTITY`](operations_configuration_bedrock.md#aws-bedrock-user-role-require-identity).
    - **Only model invocations are attributed.** Video generation, guardrail evaluations, speech, transcription and translation stay on the server's own identity.

??? failure "Every request fails after enabling per-user cost attribution"
    Model calls run under a session of [`AWS_BEDROCK_USER_ROLE_ARN`](operations_configuration_bedrock.md#aws-bedrock-user-role-arn), and a session that cannot be opened fails the request rather than silently falling back to the server's identity. The server also reports this at startup, in the `server_warnings` field of its `start` log event. Five causes, in order of likelihood:

    - **The trust policy allows only `sts:AssumeRole`.** Tagging the session is a separate action: add `sts:TagSession` to both the trust policy of the end user role and the server's own policy — see [Per-User Cost Attribution IAM](operations_iam_permissions.md#per-user-cost-attribution). Setting [`AWS_BEDROCK_USER_ROLE_TAG_KEY`](operations_configuration_bedrock.md#aws-bedrock-user-role-tag-key) to null removes the need for it, at the cost of Cost Explorer grouping.
    - **The role was just created.** A new or newly-edited trust policy takes a few seconds to propagate; a task started immediately after logs the startup warning and recovers on its own.
    - **`403` on every request**: the end user role lacks `bedrock:InvokeModel` or `bedrock:InvokeModelWithResponseStream`, or its `Resource` list misses an ARN form requests actually reach. A cross-region inference profile also needs `arn:aws:bedrock:*::foundation-model/...` for every Region it routes to, and an application inference profile, a prompt router or a prompt ARN each has to be named in its own right — see [Per-User Cost Attribution IAM](operations_iam_permissions.md#per-user-cost-attribution).
    - **`403` once a guardrail is configured**: a guardrail applied during an invocation — [`AWS_BEDROCK_GUARDRAIL_IDENTIFIER`](operations_configuration_bedrock.md#aws-bedrock-guardrail-identifier), a model alias carrying one, or a request-level `moderation` parameter — is evaluated as part of the call the end user signed, so the end user role needs `bedrock:ApplyGuardrail` on the guardrail ARN as well.
    - **`400` naming `safety_identifier`**: [`AWS_BEDROCK_USER_ROLE_REQUIRE_IDENTITY`](operations_configuration_bedrock.md#aws-bedrock-user-role-require-identity) is enabled and the client sends no end user identifier. Either have the client send one, or disable that setting.

??? failure "The organization usage endpoints answer `503 feature_unavailable`"
    [`USAGE_API`](operations_configuration_observability.md#usage-api) is off, which is the default — the endpoints exist and refuse, so this is neither a wrong path nor a rejected credential.

    - Set `USAGE_API=true` and restart — together with [`CLOUDWATCH_METRICS`](operations_configuration_observability.md#cloudwatch-metrics), which publishes the metrics these endpoints are answered from: enabling `USAGE_API` without it fails startup rather than serving endpoints that could never answer. `/v1/organization/costs` additionally needs [`COST_TRACKING`](operations_configuration_observability.md#cost-tracking), which is only a startup warning when missing. The server log names whichever one is missing.
    - Read [what a query costs](operations_cost_management.md#usage-api-cost) before turning it on: every query is billed per metric read, and enabling it also stores additional metric series.
    - A `404` instead means the prefix is wrong: the routes live under `${OPENAI_ROUTES_PREFIX}/v1/organization/...` — see [`OPENAI_ROUTES_PREFIX`](operations_configuration_server.md#openai-routes-prefix).
    - `GET /v1/usage` is not served at any setting: the retired endpoint is absent from OpenAI's current API surface and from the `openai` SDK. Use [`GET /v1/organization/usage/...`](api_openai_organization_usage.md) instead.

??? failure "The usage endpoints answer, but every bucket is empty"
    Nothing was refused — the query simply found no published metric in the range it covers. In order of likelihood:

    - **`/v1/organization/costs` alone is empty**: [`COST_TRACKING`](operations_configuration_observability.md#cost-tracking) is disabled, so no cost is computed and no `Cost` metric is published. The usage endpoints are unaffected — [`CLOUDWATCH_METRICS`](operations_configuration_observability.md#cloudwatch-metrics) cannot be disabled while these endpoints answer at all, since `USAGE_API` now refuses to start without it.
    - **The range predates the feature**: usage exists only from the moment `CLOUDWATCH_METRICS` was enabled, and nothing is backfilled. Per-endpoint buckets start later still — only from the moment [`USAGE_API`](operations_configuration_observability.md#usage-api) was enabled, since that is what publishes the `Operation` dimension, so a query grouped or filtered by endpoint is empty over traffic served before it.
    - **A multi-region deployment reports one region's traffic**: the endpoints read a single Amazon CloudWatch region, [`CLOUDWATCH_METRICS_REGION`](operations_configuration_observability.md#cloudwatch-metrics-region), defaulting to the first entry of [`AWS_BEDROCK_REGIONS`](operations_configuration_aws.md#aws-bedrock-regions). Point it at the region the metrics are actually ingested in.

??? failure "The organization usage endpoints answer `403`"
    They are an administrator surface, so the credential that calls the models is not the credential that reads them.

    - **A tenant API key is never accepted**, whatever its scope: these endpoints report the whole deployment's consumption and spend, not one tenant's. Read them with the deployment's own [API key](operations_configuration_authentication.md#api-key).
    - **A user pool token must carry every scope** named in [`USAGE_API_ADMIN_SCOPES`](operations_configuration_observability.md#usage-api-admin-scopes) — all of them, not one of them.
    - **With that list empty** (the default) no token is accepted at all, and only the deployment's own API key may read them. Name a scope there to let an operator's token in.

??? failure "A usage query is refused as too large, or `bucket_width=1m` is refused over an older range"
    A query outside these bounds is refused rather than truncated or quietly answered at a resolution it did not ask for — each one is billed by Amazon CloudWatch per metric read, and a partial answer would be indistinguishable from a quiet period.

    - **Too many metric series**: the query matched more than [`USAGE_API_MAX_METRICS`](operations_configuration_observability.md#usage-api-max-metrics) (500 by default, which is also CloudWatch's own per-request maximum). Narrow it with `models`, or ask for fewer `group_by` keys.
    - **The range is too long**: it spans more than [`USAGE_API_MAX_RANGE_DAYS`](operations_configuration_observability.md#usage-api-max-range-days) (92 days by default). Split it across several queries, or raise the setting.
    - **`bucket_width=1m` over an older range**: one-minute buckets are reported for the last 15 days only, one-hour and one-day buckets for the last 455. Request `1h` or `1d`, or move the range inside 15 days.

### :material-tools: Web search and MCP tools { #tools-and-search }

??? failure "Web search returns nothing, stale results, or is rejected"
    The built-in [web search tool](api_openai_responses.md#openai-gpt-web-search) is gated by both an IAM permission and a server setting, and each failure looks different.

    - **The model answers from its training data and says it could not search**: the task role is missing `bedrock-websearch:InvokeSearch` / `bedrock-websearch:InvokeFetch`. Add them — see [Web Search IAM](operations_iam_permissions.md#web-search-iam). The request itself still succeeds, so this shows up as a weak answer rather than an error.
    - **`400` on `external_web_access`**: the request asked for a value the server does not allow. By default searches stay inside the AWS boundary; set [`AWS_BEDROCK_EXTERNAL_WEB_ACCESS`](operations_configuration_models.md#bedrock-external-web-access) to change what the server does, or [`AWS_BEDROCK_ALLOW_EXTERNAL_WEB_ACCESS_OVERRIDE`](operations_configuration_models.md#bedrock-allow-external-web-access-override) to let requests choose. It travels as an extra model parameter (a top-level request field), not as a field of the tool: a client that sets it on the tool changes nothing.
    - **`400` on `external_web_access` saying it is not available with this model**: the override is enabled, but only the models that serve web search natively — the OpenAI GPT-5.x family — take a web access choice per request. Everywhere else the parameter must match the configured value, and is refused rather than accepted and ignored. Send it only with those models, or drop it.
    - **External web access was enabled but results still look cached**: `bedrock-websearch:ExternalWebAccess` is missing from the task role, so the search falls back to the Amazon Bedrock web index.
    - **`400` naming `filters.allowed_domains` or `user_location`**: the model's own search cannot restrict which sources it uses, and running it unrestricted would answer from the very sources the request excluded. Drop the restriction, or send the request to a model that serves web search natively — the OpenAI GPT-5.x family. `web_search_options.user_location` on `/v1/chat/completions` is refused for the same reason.
    - **`400` saying `web_search_options` is not available with this model**: the [Chat Completions](api_openai_chat_completions.md#web-grounding) parameter reaches the model's own search, and this model runs none — answering it without searching would return something other than what was asked. Send it to an Amazon Nova model that supports web grounding, or ask for the search with the `web_search` tool of [`/v1/responses`](api_openai_responses.md#openai-gpt-web-search).
    - **The tool is rejected outright**: web search is served on `/v1/responses` for the OpenAI GPT-5.x family, in `us-east-1`, `us-east-2` and `us-west-2`. On `/v1/messages` and `/v1/chat/completions` it is not available for these models.

??? failure "MCP tools declared with `mcp_servers` / `mcp_toolset` are never called"
    The request succeeds and the model answers plausibly, but no MCP server was ever contacted. The [MCP connector](api_anthropic_messages.md#mcp-connector) asks the **model** to act as an MCP client during the turn, and the models this API serves do not open those connections. The connector is therefore accepted and ignored rather than refused, so nothing in the response says it was dropped — no setting enables it.

    - The server log carries a `warning` naming exactly what was ignored (`mcp_servers`, `mcp_toolset`), correlated with `x-request-id` — the caller sees an ordinary `200`, so that line is the only place this is reported.
    - Run the MCP client yourself: connect to the server, declare its tools in `tools`, and return each result as a `tool_result` block. Every other tool in `tools` is kept and behaves normally.
    - A `tool_choice` is dropped with the toolsets when they were the only entries in `tools`, and a `cache_control` breakpoint carried by a dropped `mcp_toolset` goes with it — move the breakpoint to a tool that survives, or the cached prefix is shorter than the one the request paid to write.
    - `mcp_tool_use` and `mcp_tool_result` blocks replayed from a connector-enabled transcript are read as an ordinary `tool_use` and `tool_result`, so a repeated call comes back as a plain `tool_use` block for the client to run.
    - Unrelated to this deployment being an [MCP server](features.md#mcp-model-context-protocol) itself, which is the opposite direction: that lets an AI agent call these endpoints as tools, and is unaffected.

??? failure "A browser or computer toolset is refused, or an Agent Skills `container` changes nothing"
    Both are Anthropic Messages features whose backing runs outside this gateway, and they fail in opposite ways — one loudly, one silently.

    - **`400 The 'browser_toolset_20260801' toolset is not available for this model.`** (`computer_toolset_*` reads the same): the [toolsets](api_anthropic_messages.md#browser-and-computer-toolsets) run only on [Bedrock Mantle](api_anthropic_messages.md#bedrock-mantle)-served Claude models, which forward them to the upstream Messages API. A Converse-served model, and a Mantle model whose request is converted to an OpenAI shape, refuse them rather than answer without the browser or the desktop. A replayed `browser_state` tool result is refused the same way, on the same paths.
    - **[`AWS_BEDROCK_EXTERNAL_WEB_ACCESS`](operations_configuration_models.md#bedrock-external-web-access) neither enables nor blocks a toolset**: it governs the built-in web search only. Where a Mantle Claude model serves the toolset, turning [`AWS_BEDROCK_MANTLE_ENABLED`](operations_configuration_aws.md#bedrock-mantle-enabled) off is what leaves no path serving it.
    - **`container` is accepted and the skills are never loaded**: both the identifier string and the `{id, skills}` object [Agent Skills](api_anthropic_messages.md#limits-and-behaviour-to-know) requests use validate, and neither reaches any backend — no container is created or returned, and the response answers as if the field had not been sent. Nothing reports it: unlike the MCP connector above, no warning is logged, so an answer that ignored a skill's instructions is the only signal.

### :material-robot-outline: Ollama clients { #ollama-clients }

??? failure "An Ollama client shows no models, or refuses to connect at all"
    The Ollama-compatible endpoints answer at `/api/*` on the deployment's base URL, and the models they list are this deployment's, not the ones a local Ollama had pulled.

    - Point the client at the deployment URL followed by [`OLLAMA_ROUTES_PREFIX`](operations_configuration_server.md#ollama-routes-prefix) — `https://your-host/ollama` by default, not `https://your-host/v1`. Set `OLLAMA_ROUTES_PREFIX` to an empty value to mount at the root instead, for a drop-in swap with a stock Ollama host.
    - Send the deployment's API key as a Bearer token: a local Ollama needs no credentials, so a client configured against one usually has no field filled in, and every endpoint here answers `401` without it.
    - Choose a model from [`GET /api/tags`](api_ollama_models.md). A name learned from ollama.com such as `llama3.2:3b` is not served and answers `404`; a trailing `:latest` on a name that *is* served is accepted.
    - A client that probes its base URL for `Ollama is running` is answered at the [routes prefix](api_ollama_models.md#base-url-probe) — `https://your-host/ollama`, with or without a trailing slash — not at `/`, which serves the deployment's own root document. With `OLLAMA_ROUTES_PREFIX` set to an empty value the two collide and the probe is not served at all; have the client probe [`GET /api/version`](api_ollama_models.md) instead.

??? failure "An Ollama client shows `0` or `NaN` tokens per second"
    Tokens per second is computed from `eval_count` divided by `eval_duration`, and `eval_duration` is only reported when the response streamed.

    - Request the response with `"stream": true` (the default on `/api/chat` and `/api/generate`) and the durations are measured and reported.
    - On a non-streamed response `prompt_eval_duration` and `eval_duration` are **omitted**, because a buffered answer carries no split between reading the prompt and generating the answer, and a number with nothing behind it would be an invention. `load_duration` is never reported for the same reason: nothing is loaded, since models are served on demand.
    - The token counts themselves (`prompt_eval_count`, `eval_count`) and `total_duration` are always reported.

??? failure "`ollama pull` appears to do nothing, and `ollama list` shows every model at size 0"
    Both are correct. Models here are served on demand and none is stored on the deployment.

    - [`POST /api/pull`](api_ollama_models.md) reports success immediately for any model `/api/tags` lists, because it is already usable — there is nothing to transfer. A model this deployment does not serve answers `404` instead.
    - `size` is `0` and the parameter-count, quantization and format details are empty because they describe a model file that does not exist here. `digest` is a stable identifier derived from the model name, usable as a cache key but not a hash of any content.
    - `create`, `copy`, `push` and `delete` answer `403`: there is no model store for them to change, and reporting success would tell the client that state changed when nothing did.

??? failure "An Ollama client shows no thinking text on a reasoning model"
    `message.thinking` follows the deployment-wide reasoning setting.

    - [`CHAT_COMPLETIONS_REASONING_FIELD`](operations_configuration_observability.md#chat-completions-reasoning-field) set to `none` suppresses the reasoning text on every dialect, including this one. Set it back to `reasoning_content` or `reasoning` to have it emitted.
    - `think` must also be set on the request; without it a model returns its answer only.
    - The `capabilities` list from `/api/show` never advertises `thinking`, so a client gating its toggle on that list will not offer it. `think` can still be sent to any model — one that does not reason simply returns no thinking text.

### :material-timer-sand: Long generations and oversized requests — `504`, `408`, `413` { #long-generations }

??? failure "`503`/`504`/`408` — request times out mid-stream on long generations"
    A slow or hung generation exceeded a timeout somewhere between the model and the client. The status code tells you where: `503` comes from stdapi.ai's own gateway timeout; `504`/`408` come from an edge or proxy timeout in front of it.

    - **`503` from the gateway**: Check [`AI_RESPONSE_TIMEOUT`](operations_configuration_server.md#ai-response-timeout) — it closes stalled upstream model connections; raise it for workloads with long-running generations. The request is **not** retried in another region: the model already ran and AWS bills it either way, so a failover would pay twice for the same generation.
    - **`504`/`408` from the edge/proxy**: Check the Terraform module's `alb_idle_timeout` (default: 3600 s) — if you lowered it, or front the deployment with your own load balancer or reverse proxy at a shorter idle timeout, streaming responses can be cut off mid-flight before the gateway's own timeout fires. See [ALB Resilience](operations_resilience.md#alb-resilience).

??? failure "`400 context_length_exceeded`, `prompt is too long`, or an answer cut short below its output limit — the context window is full"
    The conversation no longer fits the model's context window. Each API words the refusal as its vendor does: `context_length_exceeded` on Chat Completions (`param: messages`) and Responses (`param: input`), a message beginning `This model's maximum context length`, stating the token counts when the model reported them, with no code on legacy Completions, an `invalid_request_error` reading `prompt is too long` on Anthropic Messages, and `{"error": "Your input exceeds the context window of this model…"}` on the Ollama routes. A streamed request gets the same `400` before its stream starts, which the OpenAI and Anthropic SDKs raise as `BadRequestError` and the Ollama Python library as `ResponseError`; only a streamed Responses request gets the refusal as an `error` event followed by `response.failed`, as the OpenAI API streams it.

    - Responses API only: send `truncation: "auto"` to drop the oldest turns instead, or `context_management` to compact the conversation before it reaches the window — see [Context Window Management](api_openai_responses.md#context-window-management). On Anthropic Messages, [context editing](api_anthropic_messages.md#context-editing) clears old tool results.
    - An input that fits the window alone but not beside the output limit is answered with a shorter output on Responses and Anthropic Messages — see [Context Window Management](api_openai_responses.md#context-window-management) and [Limits and behaviour to know](api_anthropic_messages.md#limits-and-behaviour-to-know) — and refused with `context_length_exceeded` naming the completion on Chat Completions, as OpenAI does. An output limit that fills the window on its own is a plain `400` asking to lower it.
    - Choose a model with a larger context window — [`search_models`](api_search_models.md) lists the available models.

??? failure "`413 Payload Too Large` — request or file rejected as oversized"
    Either an attachment exceeds what the chosen model reads, the application-level file-size cap, or an edge control rejected the request.

    - `Attachments larger than … are not available on the current server` means the attachment is too large to travel inside the request and there is nowhere to stage it: no region able to serve that model has an S3 bucket. Set [`AWS_S3_BUCKET`](operations_configuration_storage.md#aws-s3-bucket) for the first region of `AWS_BEDROCK_REGIONS` and an [`AWS_S3_REGIONAL_BUCKETS`](operations_configuration_storage.md#aws-s3-regional-buckets) entry for every other region that may serve the model — the server log names the one that was missing. The same request succeeds unchanged once a bucket exists.
    - `An attached file is too large: this model accepts at most … bytes per file` means that model only reads attachments sent inside the request. Send a smaller file, or choose a model that reads attachments from storage — see [Attachment Size](features.md#attachment-size).
    - `The attached files are too large: this model accepts at most … bytes of attachments per request` means the same, for the request as a whole: each file fits on its own, but their total does not. Split them across several requests, or choose a model that reads attachments from storage.
    - `A part sent as an inline JSON body must not exceed … bytes` means one part of an [upload session](api_openai_files.md#uploads-api) was sent in the JSON form and carried more than the 64 MiB that form may hold — it is decoded in memory, so it is capped. Send the part as a binary `multipart/form-data` upload, which is streamed and bounded only by S3's 5 GiB per part, or split the content into more parts. The binary form never raises this error; the official OpenAI client uses it and already splits a file at 64 MiB.
    - Check [`MAX_INPUT_FILE_SIZE`](operations_configuration_server.md#max-input-file-size) — it caps the bytes of any single file loaded into memory for model input; disabled by default, so if it's set and the error appears, raise it or reduce the input size.
    - If the deployment sits behind the Terraform module's WAF (`alb_waf_enabled=true`), check for a `SizeConstraintStatement` rule on the request body — see [Request Size & Resource Limits](operations_authentication_security.md#request-size-resource-limits).
    - If fronted by Amazon API Gateway instead of an ALB, remember its hard 10 MB payload limit.

### AWS error → HTTP status mapping { #aws-error-http-status-mapping }

stdapi.ai translates upstream AWS error codes into standard HTTP responses with an OpenAI/Anthropic-style error type. Use this table to map a status code back to its likely AWS cause. HTTP status and error type are as returned on OpenAI-compatible routes (`/v1/...`); Anthropic-compatible routes (`/anthropic/...`) diverge on the footnoted rows.

| HTTP  | Error type                  | AWS error codes                                                                                         | Typical cause                                 |
|-------|------------------------------|---------------------------------------------------------------------------------------------------------|-----------------------------------------------|
| `400` | `invalid_request_error`     | `ValidationException`, `BadRequestException`, `EntityTooSmall`, `InvalidPart`                           | Unsupported/invalid request parameters; the last two when completing an [upload](api_openai_files.md#uploads-api) |
| `400` | `invalid_request_error`     | `AccessDenied` — on the object an `s3://` input named                                                   | The caller's own object cannot be read[^4]    |
| `401` | `authentication_error`      | `UnrecognizedClientException`, `InvalidSignatureException`, `ExpiredTokenException`                     | stdapi.ai's AWS credentials missing/expired   |
| `403` | `permission_error`          | `AccessDeniedException` — on a model call an end user's own role signed                                 | That end user is not allowed that model[^3]   |
| `404` | `invalid_request_error`[^1] | `ResourceNotFoundException`, `NotFoundException`                                                        | Model or resource not available in the region |
| `409` | `conflict_error`[^5]        | `ConflictException`                                                                                     | Another request is changing the same resource — retry |
| `429` | `rate_limit_error`          | `ThrottlingException`, `TooManyRequestsException`, `ServiceQuotaExceededException`                      | Bedrock quota / throttling                    |
| `503` | `service_unavailable_error`<br>(code `feature_unavailable`) | `AccessDeniedException`, `AccessDenied` — every other denial                            | IAM task role lacks permission / model access |
| `503` | `service_unavailable_error`[^2] | `ServiceUnavailableException`, `InternalServerException`, `ServiceFailureException`, `ReadTimeoutError` | Transient AWS-side error — retry          |

[^1]: Anthropic-compatible routes return `not_found_error` instead.
[^2]: Anthropic-compatible routes return HTTP `529` with error type `overloaded_error` instead.
[^3]: Only when [per-user cost attribution](operations_cost_management.md#per-user-attribution) is enabled: the call then carries the end user's identity, and AWS evaluated a policy written about them.
[^4]: Only for a bucket declared in [`AWS_S3_ACCEPTED_BUCKETS`](operations_configuration_storage.md#aws-s3-accepted-buckets), which the deployment reads but does not own — so the refused object is the one the request named. The message names that input, and nothing else. A denial on the deployment's own buckets stays `feature_unavailable`.
[^5]: Anthropic-compatible routes return `invalid_request_error` instead. The gateway also answers `409` itself when concurrent requests update the same vector store.

!!! note "Where to find the detail"
    For security, `401`, `403` and `feature_unavailable` responses returned to clients contain only a generic message — the same one whatever is missing, so that the difference between "no permission" and "not configured" is not disclosed. The full diagnostic detail is captured in the server logs under `error_detail` and can be correlated via the `x-request-id` response header (`request-id` on Anthropic-compatible `/anthropic/...` routes) — see [Logging & Monitoring](operations_logging_monitoring.md).

---

## :material-key-variant: Authentication & Identity { #authentication-identity }

??? failure "`401 Unauthorized` — client API key missing or wrong"
    The API key is missing, wrong, or not configured.

    - Pass the key in the `Authorization: Bearer <key>` header (OpenAI-style) or `X-API-Key` header.
    - Retrieve the generated key with `terraform output -raw api_key`.
    - If `api_key_create = true` was not set, no API key is configured and requests pass through without authentication by default (useful for testing behind IP-restricted ALB, not for production).
    - See [Authentication & Security](operations_authentication_security.md) for all options.

??? failure "`401 Unauthorized` — AWS credentials invalid or expired (often local Docker)"
    stdapi.ai's **own** AWS credentials are missing, invalid, or expired — AWS returns `UnrecognizedClientException`, `InvalidSignatureException`, or `ExpiredTokenException`, which stdapi.ai maps to HTTP `401` with error type `authentication_error`. This is distinct from the client-facing API-key `401` above (which concerns your `Authorization` / `X-API-Key` header).

    - Locally: refresh with `aws sso login` (or update `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / `AWS_SESSION_TOKEN`) and restart the container.
    - On ECS: confirm the task is assuming its IAM role rather than relying on stale static keys.

??? failure "Bearer token works, but Anthropic SDK requests fail"
    The Anthropic SDK uses a different auth header than OpenAI.

    - Use `x-api-key: <your-key>` (not `Authorization: Bearer`).
    - Set the base URL to `https://<your-endpoint>/anthropic` (not `/v1`).
    - See [API Overview → Anthropic-Compatible API](api_overview.md#using-the-anthropic-compatible-api).

??? failure "A valid Amazon Cognito token is rejected with 401"
    The response body is always the same opaque `Unauthorized`; the check that failed is in the server's request log (`error_detail`). Work through the checks in order:

    - **Wrong app client**: the token's app client must be listed in [`AWS_COGNITO_CLIENT_IDS`](operations_configuration_authentication.md#aws-cognito-client-ids).
    - **Wrong token**: an identity token is rejected unless [`AWS_COGNITO_ACCEPT_ID_TOKEN`](operations_configuration_authentication.md#aws-cognito-accept-id-token) is enabled. Send the **access** token.
    - **Wrong issuer**: pools on the Essentials and Plus tiers can issue `https://issuer-cognito-idp.<region>.amazonaws.com/...`. Set [`AWS_COGNITO_ISSUER_TYPE`](operations_configuration_authentication.md#aws-cognito-issuer-type) to `updated` for those, and confirm the pool ID matches the pool that minted the token.
    - **Missing scope**: a token obtained by signing in with a username and password carries only `aws.cognito.signin.user.admin`. Clear [`AWS_COGNITO_REQUIRED_SCOPES`](operations_configuration_authentication.md#aws-cognito-required-scopes), or have clients obtain tokens from the pool's OAuth 2.0 token endpoint.
    - **Expired token**: tokens are accepted up to one minute past expiry only. Refresh the token, and check the container clock if expiry errors are constant.
    - See [Authentication & Security → Amazon Cognito User Pool Tokens](operations_authentication_security.md#amazon-cognito-user-pool-tokens).

??? failure "The server does not start after enabling authentication"
    A half-applied credential configuration is refused rather than accepted, so the deployment never runs unauthenticated by accident. The startup log's `error_detail` names the exact rule:

    - **Missing allowlist**: [`AWS_COGNITO_CLIENT_IDS`](operations_configuration_authentication.md#aws-cognito-client-ids) is required with a pool.
    - **Setting without a pool**: any other `AWS_COGNITO_*` variable requires [`AWS_COGNITO_USER_POOL_ID`](operations_configuration_authentication.md#aws-cognito-user-pool-id).
    - **Mode conflict**: [`AUTHENTICATION_MODE`](operations_configuration_authentication.md#authentication-mode) must not demand a method that is unconfigured, nor ignore one that is configured — use `any` to accept both.
    - **Empty API key**: the SSM parameter or Secrets Manager secret named by [`API_KEY_SSM_PARAMETER`](operations_configuration_authentication.md#api-key-ssm) or [`API_KEY_SECRETSMANAGER_SECRET`](operations_configuration_authentication.md#api-key-secretsmanager-secret) exists but holds an empty value — populate it, or unset the setting to run without an API key deliberately.
    - **API key secret is not text**: the secret named by [`API_KEY_SECRETSMANAGER_SECRET`](operations_configuration_authentication.md#api-key-secretsmanager-secret) holds binary data, or the field selected by [`API_KEY_SECRETSMANAGER_KEY`](operations_configuration_authentication.md#api-key-secretsmanager-key) is not a string. Store the key as a plain string, or as a JSON object whose selected field is a string.
    - **Signing keys unreachable**: the pool's public keys are read at startup over HTTPS. Check the pool ID, and that the task can reach the internet or a suitable endpoint for outbound HTTPS.

??? failure "An MCP client or agent cannot discover how to authenticate"
    Discovery is off until it is configured, and a client that finds nothing falls back to asking the user for a key.

    - **Nothing published**: `GET /.well-known/oauth-protected-resource` answering `404` means [`OAUTH_RESOURCE_IDENTIFIER`](operations_configuration_authentication.md#oauth-resource-identifier) is unset. That is the default, not a fault — set it to turn discovery on. With an [`AWS_COGNITO_USER_POOL_ID`](operations_configuration_authentication.md#aws-cognito-user-pool-id) configured it is the only setting needed; otherwise set [`OAUTH_AUTHORIZATION_SERVERS`](operations_configuration_authentication.md#oauth-authorization-servers) too.
    - **Nothing in the challenge**: a `401` carrying a bare `WWW-Authenticate: Bearer` means the same thing. Once configured it also carries `resource_metadata="…"` and, with [`OAUTH_SCOPES_SUPPORTED`](operations_configuration_authentication.md#oauth-scopes-supported), `scope="…"`.
    - **Browser-hosted client**: a client running in a page reads the document cross-origin and needs its origin in [`CORS_ALLOW_ORIGINS`](operations_configuration_server.md#cors-allow-origins).
    - **The client asks for `/.well-known/openid-configuration`**: stdapi.ai is a resource server and deliberately does not serve it; the client should follow `authorization_servers` to the issuer's own document.
    - See [Authentication & Security → Authentication Discovery for Agents](operations_authentication_security.md#authentication-discovery-for-agents).

??? failure "A client reports the protected resource does not match the expected URL"
    Clients compare the published `resource` against the URL they dialled character by character, so any difference aborts the flow.

    - **Scheme**: set [`OAUTH_RESOURCE_IDENTIFIER`](operations_configuration_authentication.md#oauth-resource-identifier) to `https://…` when clients reach the deployment over TLS, even though the container itself listens on plain HTTP behind the load balancer.
    - **Host**: it must be the public hostname clients use, not the internal service or task address.
    - **Port**: an explicit `:443` on the client's URL does not match an identifier without one, and vice versa. Drop the default port on both sides.
    - **Path**: the identifier is an origin. Do not append `/mcp`, `/v1`, or a trailing slash — one document at the root already covers every surface of the deployment.

??? failure "The server does not start after configuring authentication discovery"
    The three settings describe one document, so an incomplete set is refused rather than published half-formed. The startup log's `error_detail` names the rule:

    - **No authorization server**: [`OAUTH_AUTHORIZATION_SERVERS`](operations_configuration_authentication.md#oauth-authorization-servers) is required with [`OAUTH_RESOURCE_IDENTIFIER`](operations_configuration_authentication.md#oauth-resource-identifier) unless [`AWS_COGNITO_USER_POOL_ID`](operations_configuration_authentication.md#aws-cognito-user-pool-id) is set, which supplies the issuer; a document naming none leaves a client unable to obtain a token.
    - **Issuer contradicting the pool**: with a user pool configured, the published issuers must include the pool's own — otherwise clients are sent to an authorization server whose tokens every request refuses. Leave [`OAUTH_AUTHORIZATION_SERVERS`](operations_configuration_authentication.md#oauth-authorization-servers) empty to publish exactly the pool issuer, or list it alongside the others.
    - **Setting without an identifier**: the authorization servers and the scopes describe a document that is not published without [`OAUTH_RESOURCE_IDENTIFIER`](operations_configuration_authentication.md#oauth-resource-identifier).
    - **Malformed value**: the identifier is an origin with no path or query, each issuer is an `https` URL with no query or fragment, and a scope carries no space or quote.

??? failure "A tenant API key is refused with 503"
    A `503` for an `sk-std-...` credential means the key could not be *checked*, not that it is wrong: the tenant records in the DynamoDB table are unreachable, and the gateway fails closed rather than guessing. The server log's `error_detail` names what to fix:

    - **Missing permission**: the task role needs the [shared table permissions](operations_iam_permissions.md#shared-table) (`dynamodb:GetItem` in particular) on the table ARN.
    - **Missing or wrong table**: [`AWS_DYNAMODB_TABLE`](operations_configuration_storage.md#aws-dynamodb-table) and [`AWS_DYNAMODB_REGION`](operations_configuration_storage.md#aws-dynamodb-region) must name the table holding the tenant records.
    - **Record from a newer build**: during a rolling deployment, an instance on the previous version refuses records written with a newer layout; the refusals stop when the rollout completes.
    - **A rate-limited tenant whose counter cannot be written** — the log names `dynamodb:UpdateItem`, or a throttled table: a limit the operator declared is never silently dropped, so the tenant's requests are refused once the slots already reserved are used. Grant the [counter statement](operations_iam_permissions.md#shared-table) on the `LIMIT#*` items. The permission is probed at startup only, over the limits declared at that moment: a limit added to a tenant record later is not probed until the service restarts, and the request log's `error_detail` names the action either way. Anthropic-compatible routes answer `529`; a tenant limited on tokens alone has no reserved requests to serve first and is refused at once.
    - **A record whose `requests_per_minute` or `tokens_per_minute` is not a whole number of at least 1**: zero is refused rather than read as unlimited, so a malformed limit never opens the door. The log names the attribute.
    - Other credential kinds are unaffected: the deployment API key and Cognito tokens keep working through the outage.

??? failure "A tenant API key is refused with `429 rate_limit_error`, or a Realtime session closes with `rate_limit_exceeded`"
    The key reached its [per-tenant limit](operations_authentication_security.md#tenant-rate-limits) — its record's `requests_per_minute` or `tokens_per_minute`, or the deployment's [`TENANT_RATE_LIMIT_REQUESTS_PER_MINUTE`](operations_configuration_authentication.md#tenant-rate-limit-requests-per-minute) / [`TENANT_RATE_LIMIT_TOKENS_PER_MINUTE`](operations_configuration_authentication.md#tenant-rate-limit-tokens-per-minute). The message names the key ID and the limit; the `retry-after` header names the seconds left in the minute, and on OpenAI- and Anthropic-compatible routes the `x-ratelimit-*` or `anthropic-ratelimit-*` headers on every response of that key show how much of the minute is left before the refusal. The Cohere- and Ollama-compatible routes answer their own plain `429` envelope with `retry-after` and no rate-limit headers.

    - **Raise or remove the limit**: the record's value wins over the deployment default, so one tenant can be widened without touching the others; a record change reaches every instance within [`TENANT_KEY_CACHE_SECONDS`](operations_configuration_authentication.md#tenant-key-cache-seconds).
    - **Refused before the declared number of requests, on several instances**: request slots are reserved per instance in batches and never returned within the minute, so a tenant whose traffic is spread thin — or moved by a scale event or a rolling deployment — can be refused below its `requests_per_minute`, the more so the more instances it spans. Declare the limit with headroom for the instance count, or keep the tenant's traffic on fewer instances.
    - **A token limit refused a request that was well under it, with nothing billed yet**: on an instance that has not billed a request of the key — every instance, after a deployment or a scale-out — each request in flight counts as an eighth of the token limit, so about eight concurrent requests are admitted whatever their size and the next is refused while the minute's total reads zero. Raising the limit does not change that ratio; the refusals stop as soon as one request of the key bills on that instance. A request limit sized for the tenant's concurrency is what raises the admission bound.
    - **A token limit refused a request that was well under it, mid-minute**: requests are admitted on the key's mean tokens per billed request and reconciled from what the model billed, so a burst of large responses in flight can exhaust the minute before their totals are known — and a token limit alone admits at most 64 requests in flight per instance, refused with the same `429`. The refusal lasts until the minute ends. Pair the token limit with a request limit: it replaces the in-flight ceiling, and the pair bounds the minute's spend.
    - **Only one client is loud**: the limit is per key, not per client — every client of the tenant shares it. Give a noisy integration its own tenant record.
    - **A Realtime session opened but the next connection is refused**: the session's turns count against the token limit and the open session is never cut; the *next* handshake is what the limit refuses — and on the wire it is not a `429`. The upgrade succeeds (`101`), then the server sends one `error` event of type `invalid_request_error` with code `rate_limit_exceeded` and closes with code `3000`, reason `invalid_request_error.rate_limit_exceeded`. No `retry-after` reaches a WebSocket client, so the client backs off itself; only the server request log shows the `429`. A session opened with an [ephemeral client secret](api_openai_realtime.md#ephemeral-client-secrets) also costs a second request, for the `POST /v1/realtime/client_secrets` that mints it.

??? failure "A rate-limited tenant's requests wait on the shared table"
    Declaring a [per-tenant limit](operations_authentication_security.md#tenant-rate-limits) puts writes to the [shared table](operations_configuration_storage.md#aws-dynamodb-table) on the admission path, and a request finding no slot reserved waits for one. Which requests wait follows the limit that is declared:

    - **A request limit of `16` or more**: slots are reserved in batches that start at one and double within the minute, capped at an eighth of the limit, so the first two requests of each minute for that key on each instance wait, and a later one waits whenever the background refill has not landed yet. The rest of the minute is admitted from what the instance holds.
    - **A request limit below `16`**: an eighth of the limit rounds down to a single slot, so **every** request of that tenant waits for a write. Declare at least `16` where the table's round trip must stay off the request path.
    - **A token limit alone**: one write per minute per key per instance, on the request that reads the window's totals; billed tokens are flushed in the background.

    A latency step of the table's round trip on those requests is expected; more than that is the table answering slowly:

    - **Throttling**: a provisioned table whose write capacity is below the number of limited keys times instances makes the write retry with backoff before it succeeds, or fails it — the request log then names `dynamodb:UpdateItem` with `ProvisionedThroughputExceededException`. The Terraform module creates the table on-demand, which scales with the load.
    - **A distant region**: [`AWS_DYNAMODB_REGION`](operations_configuration_storage.md#aws-dynamodb-region) is far from the instances; keep the table in the deployment's own region.
    - **A table that does not answer** is given up on within seconds, and the request is then refused with `503` rather than served unlimited.

??? failure "A newly declared tenant has no key in Parameter Store"
    The server mints pending tenants at startup and then once a minute, so the parameter appears within about a minute of `terraform apply` — if it can:

    - **Missing permission**: minting needs `ssm:PutParameter` and `ssm:GetParameter` on the [delivery prefix](operations_iam_permissions.md#tenant-key-delivery), and the [shared table permissions](operations_iam_permissions.md#shared-table) to record the hash. The refusal is in the server log.
    - **A parameter already exists at that name** with something that is not this tenant's key: the server refuses to adopt it. Delete the parameter and let the next cycle mint a fresh key.
    - **The feature is off**: [`TENANT_API_KEYS`](operations_configuration_authentication.md#tenant-api-keys) must be `true` on the running service, not only in the table.

??? failure "A revoked or re-scoped tenant key still works"
    Each instance caches a validated key for [`TENANT_KEY_CACHE_SECONDS`](operations_configuration_authentication.md#tenant-key-cache-seconds) — 60 seconds by default — so a revocation, a `disabled = true` or a scope change takes up to that long to reach every instance. That window is the documented trade against a table read per request; lower the setting if a minute is too long, `0` disables the cache entirely.

??? failure "A superseded tenant key is still accepted after a rotation"
    By design, for [`TENANT_KEY_ROTATION_OVERLAP_SECONDS`](operations_configuration_authentication.md#tenant-key-rotation-overlap-seconds) — 7 days by default: the key a [rotation](operations_authentication_security.md#rotating-tenant-keys) superseded stays readable as the secret's `AWSPREVIOUS` version and keeps authenticating, so a client that has not re-read its secret yet is not locked out. The cutoff is exact, whatever the cache: the key stops at the end of the window on every instance at once. Only the last superseded key is kept, so a further rotation inside the window — raising `key_generation` again — retires it there and then.

    - **Shorter window**: lower the setting; `0` refuses the superseded key as soon as the new one is promoted to `AWSCURRENT`.
    - **Compromised key**: set `disabled = true` on the tenant, which refuses both keys within [`TENANT_KEY_CACHE_SECONDS`](operations_configuration_authentication.md#tenant-key-cache-seconds), whatever the overlap.

??? failure "A rotated tenant key is refused, or a tenant key is never rotated"
    A rotation writes the new key as a pending version of the tenant's secret, records it, then moves the `AWSCURRENT` label — the gateway accepts the new key **before** it becomes current, so a client re-reading the secret is never handed a key that is refused. When a key nonetheless does not rotate, or the rotated key answers `401`:

    - **Keys are delivered through Parameter Store**: rotation needs the Secrets Manager store — [`TENANT_KEY_SECRETSMANAGER_PREFIX`](operations_configuration_authentication.md#tenant-key-secretsmanager-prefix) on the service, or `tenant_key_rotation_days` / `key_generation` in the Terraform module. A `key_generation` raised without it is reported once in the server log and ignored.
    - **Missing permission**: the task role needs the [rotation permissions](operations_iam_permissions.md#tenant-key-rotation) on the secret prefix, and `kms:GenerateDataKey` and `kms:Decrypt` on the key encrypting the secrets. The refusal names the action in the server log; the key keeps working unchanged.
    - **The rotation is not due**: [`TENANT_KEY_ROTATION_DAYS`](operations_configuration_authentication.md#tenant-key-rotation-days) counts from the mint or the last rotation, and `key_generation` only rotates when it exceeds the generation recorded at the previous rotation — raise it again for another rotation.
    - **The client read the secret before the promotion**: a read that names no version stage answers `AWSCURRENT`, which moves within a second of the record being written; a client that reads `AWSPENDING` explicitly can read a key that is not recorded yet. Read the current version, or nothing more specific.
    - **A mixed fleet**: instances on a release without rotation accept the new key but refuse the superseded one at once.
    - **The promotion was not confirmed**: the server log warns that a version *could not be confirmed as `AWSCURRENT`*, naming the secret and the key. The superseded key keeps working and the promotion is retried at the next reconciliation.

??? failure "A tenant with a registered AWS role gets 403 on every model call"
    The fixed message *"The AWS credential registered for this API key could not be used"* means the gateway could not open (or keep) a session of the tenant's role; *"Your AWS account does not have access to this model"* means the session opened but the tenant's account refused the invocation. The full AWS detail is in the server log only. In order of likelihood:

    - **Trust policy**: the tenant role must trust the deployment's account with `sts:AssumeRole`, conditioned on the exact `ExternalId` the server minted — read it from the `external_id` attribute of the tenant's `secret#<key id>` record. A wrong or missing ExternalId is indistinguishable from a revoked trust, on purpose.
    - **Gateway-side permission**: the task role needs [`sts:AssumeRole` on the tenant role](operations_iam_permissions.md#tenant-aws-credentials).
    - **Model access in the tenant's account**: the tenant must have been granted access to the model in *its* account, in the serving Region — including [enabling opt-in Regions](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-support.html) a cross-Region profile routes to.
    - **The role's own policy**: it must allow the Bedrock invocation actions on the model (and its inference profiles).

    A `503` *"…could not be used right now. Retry the request"* is **not** one of these: it means AWS STS itself was throttled or unreachable, or the deployment's own session had expired. Nothing on the tenant's side is wrong, and the request is worth retrying — a client SDK retries it on its own.

??? failure "A tenant record declaring aws_role_arn is refused with 503"
    A declared role is never silently ignored — the gateway refuses the key rather than billing the deployment for a tenant that expects its own account. The server log names which of these it is:

    - **The feature is off**: [`TENANT_AWS_CREDENTIALS`](operations_configuration_authentication.md#tenant-aws-credentials) must be `true`, or the attribute removed.
    - **The ExternalId is not minted yet**: for a tenant created before this feature existed, the server mints one within a minute of the role being declared; the refusal covers that window.
    - **The ARN is malformed**: `aws_role_arn` must be an IAM role ARN, `arn:aws:iam::<account>:role/<name>`.
    - **A guardrail is configured**: the combination is refused at startup — see [the incompatibility](operations_authentication_security.md#tenant-aws-credentials).

??? failure "One tenant gets 404 for a model that works for everyone else"
    A model outside a tenant's scope answers the standard `model_not_found`, indistinguishable from a model that does not exist — by design, so the catalogue leaks nothing. Check the tenant's `models_allow` and `models_deny` patterns against the **resolved** model ID (after aliases — the ID the working requests are logged with), and remember an **empty** `models_allow` list allows nothing, while an absent one restricts nothing. `GET /v1/models` is not filtered per tenant, so a model appearing there can still be refused at invocation.

??? failure "OIDC/Cognito redirect loop or 401 from the ALB"
    Authentication is enforced by the ALB listener, not stdapi.ai.

    - Verify the OIDC issuer URL, client ID, client secret, and redirect URI in the ALB listener rule.
    - For Cognito, confirm the app client is configured as a "confidential" client with a client secret.
    - See [Authentication & Security → via Application Load Balancer (ALB)](operations_authentication_security.md#via-application-load-balancer-alb).

---

## :material-arrow-right: Next Steps

<div class="grid cards" markdown>

- :material-cog: [**Configuration Reference**](operations_configuration.md) — Every environment variable
- :material-server-network: [**Advanced Deployment**](operations_deploy_advanced.md) — VPC integration, manual ECS, multi-region
- :material-lock: [**Authentication & Security**](operations_authentication_security.md) — API keys, OIDC/Cognito, IAM
- :material-terraform: [**Terraform Module Docs**](https://github.com/stdapi-ai/terraform-aws-stdapi-ai) — All module inputs and outputs
- :material-github: [**GitHub Issues**](https://github.com/stdapi-ai/stdapi.ai/issues) — Report a bug or ask a question
- :material-email-outline: [**Contact**](contact.md) — Bug reports on GitHub issues (project discontinued)

</div>
