One user action can trigger several model requests, but a connection error alone does not prove duplicate billing. Count attempts across your SDK, application and queue, then reconcile each request with provider usage. Protect actions such as publishing a listing separately from text generation.
This guide covers server-side LLM API troubleshooting. Official documentation was reviewed on 2026-10-06. All counts, deadlines and log entries below are illustrative, not paid inference results, provider limits or savings claims. For the broader purchasing decision, see OpenRouter versus direct provider APIs.
Separate requests, charges and business actions
“Generate and publish a product description” contains three operations: request text, validate it and publish it. Count each independently.
| Observation | Missing evidence | Next step |
|---|---|---|
| One click produces many backend calls | Actual attempts at each layer | Trace who owns each retry loop |
| Client times out without a complete answer | Whether the server accepted it and recorded usage | Look up identifiers; mark cost unknown |
| Two answers arrive | Billing records for both requests | Reconcile both, not only the final success |
| Two copies appear in the shop | Business action identifiers and execution history | Repair action deduplication |
A lost response, duplicate action and billing discrepancy are different findings. Showing one answer in the interface does not establish that the backend ran once.
Find retry multiplication across layers
Inspect the chain: queue or scheduler → application loop → SDK → gateway or provider. Record client versions, explicit settings and defaults; searching only your own code for retry misses SDK behavior.
Consider a hypothetical worst-case path: three SDK attempts per call, three application calls per delivery and two queue deliveries. If every failure path exhausts its allowance, the client could send 3 × 3 × 2 = 18 requests. Each number includes the initial attempt. This is a request ceiling for this example, not proof of eighteen charges.
Terminology matters. The Claude error documentation describes two automatic SDK retries by default, configurable through max_retries. The AWS SDK retry reference counts the initial request in max_attempts. That AWS page also flags opt-in requirements for newer behavior; do not transfer its defaults to another client.
Fill in this worksheet before changing configuration:
| Layer | What to establish | Evidence |
|---|---|---|
| Queue / scheduler | Delivery limit and redelivery rules | Delivery ID and task state |
| Application | Total attempt limit and stop conditions | Task ID and attempt sequence |
| SDK / HTTP client | Version, retry defaults and overrides | Outgoing request count |
| Gateway / routing | Internal retry and fallback rules | Gateway records and processing provider |
Write “unknown” when gateway internals are not visible. Client counts do not necessarily cover the entire route. If using a router, also inspect provider and model fallback boundaries.
Classify the failure before replaying it
This table is a proposed application policy. Apply it to the service’s error type and body, not the HTTP number alone.
| Failure | Default response | Condition for continuing |
|---|---|---|
| Invalid input, authentication or permission | Stop unchanged replay | Correct the request or access first |
| Explicit temporary throttling | Wait within the total budget | Respect server timing and remaining deadline |
| Exhausted credits or spending allowance | Pause and report | Access or allowance has recovered |
| Temporary service fault | Bounded backoff | Replay is safe and budget remains |
| Connection error or read timeout | Treat the result as potentially unknown | Check acceptance and available result records |
| Interrupted stream | Mark the answer incomplete | Never append a new generation as if it continued the old one |
Claude documents that 429 can involve a spend cap and that a stream can fail after HTTP 200. Neither “retry every 429” nor “200 means the task completed” is sufficient. Consult the relevant error contract alongside your logs.
Give one layer ownership of the total budget
For queued workflows, the application can own a persistent budget while internal automatic retries are disabled. For simpler calls, the SDK can own retries if the outer application does not loop unconditionally. A redelivery must not reset the same task’s allowance.
An illustrative interactive-task policy might allow three outgoing attempts within 45 seconds of enqueueing. Longer reasoning jobs need a different budget; these are not universal defaults. Implement these conditions:
- Atomically reserve an attempt before sending. Persist the counter and deadline so concurrent consumers cannot each spend a full allowance.
- Classify the failure. Move an uncertain, unsafe-to-replay operation into reconciliation instead of automatically resending it.
- Use backoff with jitter and respect valid server-directed waits. If the required wait exceeds the remaining deadline, stop this run; do not shorten it to retry sooner.
- Bound each request timeout by the remaining task time. Check the deadline again after sleeping. Client cancellation does not establish that server processing or billing stopped.
- Return a clear completed, failed or needs-reconciliation state. A manual retry must not bypass an unresolved business action.
Attempt and time limits are not a monetary hard cap. Concurrent spending reservations, settlement and provider controls require separate design. “At most three attempts” alone cannot guarantee a particular bill.
Keep three identifiers and preserve unknown costs
Keep the business task ID stable across retries. Allocate a new attempt ID for each outgoing request, then attach any provider request or generation ID. Do not log API keys or unnecessary original user text.
The following is a fictional record layout, not a reported API run:
| Task | Attempt / event | Client result | Provider reference | Cost state | Publishing action |
|---|---|---|---|---|---|
| task-demo | attempt-1 | Read timeout | Not received | Unknown | Not executed |
| task-demo | attempt-2 | Validated answer | Example reference A | Awaiting reconciliation | action-demo completed |
| task-demo | Queue redelivery | Existing completion found | No new request | Earlier unknown remains | Return stored result |
The second success does not erase the first unknown. When an OpenRouter generation ID is available, its Generation lookup endpoint, GET /api/v1/generation?id=<generation-id>, exposes fields including model, provider_name and total_cost. Without an ID, retain the time window, account and client records and use an authorized dashboard or support channel; do not invent an identifier.
Deduplicate repeated imports of the same provider record, but retain distinct model requests under the same business task. Track request count, unresolved cost entries, confirmed costs and accepted tasks separately. Do not call the cost complete while unresolved entries remain.
Idempotency needs persistent state and recovery
A task ID in a log is not deduplication. For a publishing action, a stable key such as tenant + action + content revision can support a persistent uniqueness constraint and states for in-progress, completed and uncertain work. Return stored results for completed duplicates; prevent concurrent execution. A legitimate content revision needs a distinct identity.
Account for the crash between remote success and saving local completion. Simply repeating the action after recovery can publish twice. Reuse a downstream idempotency key only where its API contract supports it; otherwise query the business object or require reconciliation. Do not assume every LLM endpoint implements Idempotency-Key. Generation deduplication and publishing deduplication are separate responsibilities.
Acceptance cases to run against a local stub
These are proposed project checks. We have not executed these fault injections or paid API tests for this article. Stubs and controlled clocks check application behavior, not a provider’s real billing policy.
| Injected condition | Expected observation |
|---|---|
| Explicit invalid credentials or input | No unchanged replay; error category retained |
| Temporary throttling with a wait beyond the deadline | Run stops without an early retry |
| Repeated transient failures followed by queue redelivery | Persistent total attempt allowance is respected |
| Client timeout after server acceptance | Cost remains unknown; unsafe actions are not blindly replayed |
| Stream fails partway through | Partial output is neither accepted nor joined to a new answer |
| Two consumers process the same task | Only one executes the same business action |
| Crash after remote action but before local completion | Recovery uses downstream idempotency or reconciliation |
Save before-and-after settings, stub request counts and task logs, then reconcile a small real workload with provider records. Compare accepted tasks, attempts, unknown outcomes and confirmed costs together: dropping work can lower a bill without improving efficiency. For the budgeting formula, see the API cost worksheet (Chinese).