Summarization

Conversation summarization helps manage long conversation histories by condensing older messages while preserving important context. This is essential for maintaining context in extended conversations without exceeding token limits or performance constraints.

Overview

The SDK provides two types of summarizers:

LLM Summarizer: Uses an LLM to intelligently summarize conversation history while preserving important context
Sliding Window Summarizer: Keeps only the most recent N conversation runs and discards older messages

Summarizers are integrated with conversation managers and automatically trigger when certain conditions are met (e.g., token threshold for LLM summarizer, or when the number of runs exceeds the window size for sliding window).

LLM Summarizer

The LLM summarizer uses an LLM to create intelligent summaries of conversation history. It triggers when the total token count exceeds a specified threshold, keeping recent messages intact and summarizing older ones.

Configuration

import (
    "github.com/hastekit/hastekit-sdk-go/pkg/agents/history/summariser"
    "github.com/hastekit/hastekit-sdk-go/pkg/agents/history"
    "github.com/hastekit/hastekit-sdk-go/pkg/agents/prompts"
)

// Create a summarizer LLM (can be different from the agent's LLM)
summarizerLLM := client.Model("OpenAI/gpt-4o-mini")

// Create an instruction provider for the summarizer
summarizerInstruction := hastekit.NewPrompt("You are a conversation summarizer. Create concise summaries that preserve important context, decisions, and information.")

// Create the LLM summarizer
summarizer := summariser.NewLLMHistorySummarizer(&summariser.LLMHistorySummarizerOptions{
    LLM:                 summarizerLLM,
    Instruction:         summarizerInstruction,
    TokenThreshold:      1000,        // Summarize when total tokens exceed this
    KeepRecentCount:     5,            // Keep the last 5 runs unsummarized (default: 5)
    Parameters:          responses.Parameters{}, // Optional LLM parameters
})

Parameters

LLM: The LLM provider to use for summarization (can be different from the agent’s LLM)
Instruction: System prompt that defines how the summarizer should summarize conversations
TokenThreshold: The token count threshold at which summarization triggers
KeepRecentCount: Number of recent conversation runs to keep unsummarized (default: 5)
Parameters: Optional LLM parameters (temperature, etc.) for the summarization call

How It Works

When messages are loaded, the summarizer checks if the total token count exceeds the threshold
If the threshold is exceeded and there are enough messages, it groups messages by run ID
It keeps the most recent KeepRecentCount runs intact
Older runs are summarized into a single system message using the LLM
The summary replaces the old messages, preserving context while reducing token usage

Sliding Window Summarizer

The sliding window summarizer keeps only the most recent N conversation runs and discards older ones. This is a simple, cost-effective approach that doesn’t require an LLM.

Configuration

import "github.com/hastekit/hastekit-sdk-go/pkg/agents/history/summariser"

summarizer := summariser.NewSlidingWindowHistorySummarizer(&summariser.SlidingWindowHistorySummarizerOptions{
    KeepCount: 10, // Keep only the last 10 conversation runs
})

Parameters

KeepCount: The number of recent conversation runs to retain. Older runs are discarded.

How It Works

Messages are grouped by their run ID
If the number of runs exceeds KeepCount, only the most recent KeepCount runs are kept
Older runs are discarded without creating a summary
This approach is simple and cost-effective but loses older context completely

Using Summarizers with Conversation Managers

To use a summarizer, pass it to the conversation manager using history.WithSummarizer():

import (
    "github.com/hastekit/hastekit-sdk-go/pkg/agents/history"
    "github.com/hastekit/hastekit-sdk-go/pkg/agents/history/summariser"
)

// Create a summarizer
summarizer := summariser.NewLLMHistorySummarizer(&summariser.LLMHistorySummarizerOptions{
    LLM:                 summarizerLLM,
    Instruction:         summarizerInstruction,
    TokenThreshold:      1000,
    KeepRecentCount:     5,
})

// Create conversation manager with summarizer
p, _ := history.NewFileConversationPersistence("./conversations")
cm := history.NewConversationManager(
    p,
    history.WithSummarizer(summarizer),
)

// Use with agent
agent := hastekit.NewAgent(&hastekit.AgentConfig{
    Name:        "Assistant",
    Instruction: hastekit.NewPrompt("You are a helpful assistant."),
    LLM:         model,
    History:     cm,
})

Complete Example: LLM Summarizer

Here’s a complete example using an LLM summarizer:

package main

import (
	"context"
	"log"
	"os"

	hastekit "github.com/hastekit/hastekit-sdk-go"
	"github.com/hastekit/hastekit-sdk-go/pkg/agents"
	"github.com/hastekit/hastekit-sdk-go/pkg/agents/history"
	"github.com/hastekit/hastekit-sdk-go/pkg/agents/history/summariser"
	"github.com/hastekit/hastekit-sdk-go/pkg/gateway/llm/responses"
)

func main() {
	// Configure an LLM client
    client := hastekit.NewLLMClient([]hastekit.ProviderConfig{
        {
            ProviderName:  hastekit.ProviderOpenAI,
            BaseURL:       "",
            CustomHeaders: nil,
            ApiKeys: []*hastekit.APIKeyConfig{
                {
                    Name:   "Key 1",
                    APIKey: os.Getenv("OPENAI_API_KEY"),
                },
            },
        },
    })

	// Create main agent LLM
	model := client.Model("OpenAI/gpt-4o-mini")

	// Create summarizer LLM (can use a cheaper/faster model)
	summarizerLLM := client.Model("OpenAI/gpt-4o-mini")

	// Create summarizer instruction
	summarizerInstruction := hastekit.NewPrompt(
		"You are a conversation summarizer. Create concise summaries that preserve important context, decisions, and information needed for future interactions.",
	)

	// Create LLM summarizer
	summarizer := summariser.NewLLMHistorySummarizer(&summariser.LLMHistorySummarizerOptions{
		LLM:             summarizerLLM,
		Instruction:     summarizerInstruction,
		TokenThreshold:  1000, // Summarize when tokens exceed 1000
		KeepRecentCount: 5,    // Keep last 5 runs
		Parameters:      responses.Parameters{},
	})

	// Create conversation manager with summarizer
	p, _ := history.NewFileConversationPersistence("./conversations")
	cm := history.NewConversationManager(
		p,
		history.WithSummarizer(summarizer),
	)

	// Create agent with history
	agent := hastekit.NewAgent(&hastekit.AgentConfig{
		Name:        "Assistant",
		Instruction: hastekit.NewPrompt("You are a helpful assistant."),
		LLM:         model,
		History:     cm,
	})

	// Execute agent (summarization happens automatically when threshold is exceeded)
	handle, err := agent.Execute(context.Background(), &agents.AgentInput{
		Message: history.Message{
            Messages: []responses.InputMessageUnion{
			    responses.UserMessage("Hello!"),
			},
		},
	})
	if err != nil {
		log.Fatal(err)
	}
	out, err := handle.Result()
	if err != nil {
		log.Fatal(err)
	}

	log.Println(out.Output[0].OfOutputMessage.Content[0].OfOutputText.Text)
}

Complete Example: Sliding Window Summarizer

Here’s a complete example using a sliding window summarizer:

package main

import (
	"context"
	"log"
	"os"

	hastekit "github.com/hastekit/hastekit-sdk-go"
	"github.com/hastekit/hastekit-sdk-go/pkg/agents"
	"github.com/hastekit/hastekit-sdk-go/pkg/agents/history"
	"github.com/hastekit/hastekit-sdk-go/pkg/agents/history/summariser"
	"github.com/hastekit/hastekit-sdk-go/pkg/gateway/llm/responses"
)

func main() {
	// Configure an LLM client
    client := hastekit.NewLLMClient([]hastekit.ProviderConfig{
        {
            ProviderName:  hastekit.ProviderOpenAI,
            BaseURL:       "",
            CustomHeaders: nil,
            ApiKeys: []*hastekit.APIKeyConfig{
                {
                    Name:   "Key 1",
                    APIKey: os.Getenv("OPENAI_API_KEY"),
                },
            },
        },
    })

	model := client.Model("OpenAI/gpt-4o-mini")

	// Create sliding window summarizer
	summarizer := summariser.NewSlidingWindowHistorySummarizer(&summariser.SlidingWindowHistorySummarizerOptions{
		KeepCount: 10, // Keep only the last 10 conversation runs
	})

	// Create conversation manager with summarizer
	p, _ := history.NewFileConversationPersistence("./conversations")
	cm := history.NewConversationManager(
		p,
		history.WithSummarizer(summarizer),
	)

	// Create agent
	agent := hastekit.NewAgent(&hastekit.AgentConfig{
		Name:        "Assistant",
		Instruction: hastekit.NewPrompt("You are a helpful assistant."),
		LLM:         model,
		History:     cm,
	})

	// Execute agent
	handle, err := agent.Execute(context.Background(), &agents.AgentInput{
		Message: history.Message{
            Messages: []responses.InputMessageUnion{
			    responses.UserMessage("Hello!"),
			},
		},
	})
	if err != nil {
		log.Fatal(err)
	}
	out, err := handle.Result()
	if err != nil {
		log.Fatal(err)
	}

	log.Println(out.Output[0].OfOutputMessage.Content[0].OfOutputText.Text)
}

SDK

LLM Gateway

Agents

Workflows

Overview

LLM Summarizer

Configuration

Parameters

How It Works

Sliding Window Summarizer

Configuration

Parameters

How It Works

Using Summarizers with Conversation Managers

Complete Example: LLM Summarizer

Complete Example: Sliding Window Summarizer

​Overview

​LLM Summarizer

​Configuration

​Parameters

​How It Works

​Sliding Window Summarizer

​Configuration

​Parameters

​How It Works

​Using Summarizers with Conversation Managers

​Complete Example: LLM Summarizer

​Complete Example: Sliding Window Summarizer

Overview

LLM Summarizer

Configuration

Parameters

How It Works

Sliding Window Summarizer

Configuration

Parameters

How It Works

Using Summarizers with Conversation Managers

Complete Example: LLM Summarizer

Complete Example: Sliding Window Summarizer