Skip to main content
hastekit.NewLLMClient takes provider configuration and optional provider middleware. A bound model uses a Provider/model identifier, such as OpenAI/gpt-4.1-mini. Construct clients and bound models at startup and reuse them.

​
Configure credentials

Load keys from your application’s secret source. With two environment variables set:
Multiple keys are selected randomly in proportion to their weights. Use non-negative weights. When all weights are zero, selection is uniform. Selection alone does not guarantee quota enforcement, health checking, or failover; do not treat RateLimits, Enabled, or IsDefault fields as an enforcement policy in the direct SDK configuration store. ProviderConfig.BaseURL and CustomHeaders customize provider routing. Use the exact provider name constants rather than guessing capitalization. The SDK has adapters for OpenAI, Anthropic, Gemini, Bedrock, xAI, OpenRouter, Ollama, DeepSeek, Moonshot, Z.ai, Sarvam, and ElevenLabs, but supported methods and model features differ.

​
Configure retries and fallback

Import github.com/hastekit/agent-sdk-go/pkg/gateway/middleware. Given providerConfigs containing every target provider:
Middleware is ordered outermost first. With fallback outside retry, each provider gets its retry budget before the chain moves to another target. Neither retries nor fallback are installed unless configured. RetryConfig.MaxAttempts includes the initial call. Streaming retries/fallback can occur only before the caller receives its first chunk; once output has been delivered, a later failure ends that stream rather than mixing two attempts. A model inherits its client’s middleware by default. Passing WithMiddleware(...) to client.Model replaces the whole chain for that model. WithoutMiddleware() removes the configurable chain. Bind such models once rather than rebuilding their internal configuration on every request.

​
Per-run configuration keys

gateway.WithProviderConfigKey passes a key to a configured ConfigStore. A remote or custom store may use it for credential lookup, and the agent durable runtimes propagate it across execution boundaries. The direct in-memory store used by hastekit.NewLLMClient ignores that lookup key and selects from its configured keys. It does not implement virtual keys or pin a specific per-run credential through this context value. Use a suitable custom/remote configuration store, or separately configured clients, for tenant-specific credentials.