Skip to main content

Documentation Index

Fetch the complete documentation index at: https://hastekit.ai/docs/llms.txt

Use this file to discover all available pages before exploring further.

HasteKit supports managing multiple API keys for a single provider. This is useful for:
  • Load Balancing: Distribute requests across multiple API keys to avoid rate limits.
  • Redundancy: Ensure high availability by having fallback keys.
  • Cost Management: Allocate traffic based on usage quotas or costs associated with different keys.

​
Weighted Random Selection

When multiple API keys with weights are configured for a provider, HasteKit uses weighted random selection to choose which key to use for each request. Each API key must be assigned a Weight. The probability of a key being selected is proportional to its weight relative to the sum of all weights for that provider.

​
How it Works

  1. HasteKit retrieves all enabled API keys for the requested provider.
  2. It extracts the Weight from each APIKeyConfig.
  3. If multiple keys exist, it performs a weighted random selection.
  4. The request is then executed using the selected API key.
If all keys have the same weight (e.g., all set to 1 or not specified), the selection becomes uniformly random.

​
Configuration Example

When using the HasteKit SDK in direct mode, you can configure multiple keys using the InMemoryConfigStore. 
package main

import (
	"context"
	"fmt"
	"log"

	"github.com/hastekit/hastekit-sdk-go/pkg/gateway"
	"github.com/hastekit/hastekit-sdk-go/pkg/gateway/llm"
	hastekit "github.com/hastekit/hastekit-sdk-go"
)

func main() {
	// Define multiple API keys for OpenAI with different weights
	providerConfigs := []gateway.ProviderConfig{{
		ProviderName: llm.ProviderNameOpenAI,
		ApiKeys: []*gateway.APIKeyConfig{
			{
				APIKey: "sk-openai-key-1",
				Weight: 70, // This key will be used ~70% of the time
			},
			{
				APIKey: "sk-openai-key-2",
				Weight: 30, // This key will be used ~30% of the time
			},
		},
	}}

	// Create the HasteKit client
	HasteKit, err := hastekit.New(&hastekit.ClientOptions{
		ProviderConfigs: providerConfigs,
	})
	if err != nil {
		log.Fatalf("failed to create HasteKit client: %v", err)
	}
}