Managing Multiple Keys

HasteKit supports managing multiple API keys for a single provider. This is useful for:

Load Balancing: Distribute requests across multiple API keys to avoid rate limits.
Redundancy: Ensure high availability by having fallback keys.
Cost Management: Allocate traffic based on usage quotas or costs associated with different keys.

Weighted Random Selection

When multiple API keys with weights are configured for a provider, HasteKit uses weighted random selection to choose which key to use for each request. Each API key must be assigned a Weight. The probability of a key being selected is proportional to its weight relative to the sum of all weights for that provider.

How it Works

HasteKit retrieves all the configured API keys for the requested provider.
It extracts the Weight from each APIKeyConfig.
If multiple keys exist, it performs a weighted random selection.
The request is then executed using the selected API key.

If all keys have the same weight (e.g., all set to 1 or not specified), the selection becomes uniformly random.

Configuration Example

When using the HasteKit SDK in direct mode, you can configure multiple keys by passing a []hastekit.ProviderConfig to hastekit.NewLLMClient, assigning each APIKeyConfig a Weight.

package main

import (
	hastekit "github.com/hastekit/hastekit-sdk-go"
)

func main() {
	// Define multiple API keys for OpenAI with different weights
	providerConfigs := []hastekit.ProviderConfig{{
		ProviderName: hastekit.ProviderOpenAI,
		ApiKeys: []*hastekit.APIKeyConfig{
			{
				APIKey: "sk-openai-key-1",
				Weight: 70, // This key will be used ~70% of the time
			},
			{
				APIKey: "sk-openai-key-2",
				Weight: 30, // This key will be used ~30% of the time
			},
		},
	}}

	// Create the HasteKit client
	client := hastekit.NewLLMClient(providerConfigs)
}

SDK

LLM Gateway

Agents

Workflows

Managing Multiple Keys

Weighted Random Selection

How it Works

Configuration Example

​Weighted Random Selection

​How it Works

​Configuration Example

Weighted Random Selection

How it Works

Configuration Example