How do I use Commune thread history as context for my LLM when generating email replies? #47

shanjairaj7 · 2026-03-01T12:01:45Z

shanjairaj7
Mar 1, 2026
Maintainer

The Problem

When your agent receives an inbound email, it rarely has enough context from just that single message to write a great reply. The customer might be referencing something from three emails ago. You need the full thread history — efficiently loaded, well-formatted for the LLM, and within the context window.

This post covers: fetching thread history, formatting it for different LLMs, handling long threads that exceed context limits, and a complete LangChain memory integration.

Step 1: Fetching Thread History

from commune import CommuneClient

client = CommuneClient(api_key="YOUR_API_KEY")

def get_thread_messages(thread_id: str) -> list[dict]:
    thread = client.threads.get(thread_id)
    return thread.messages  # Ordered oldest → newest

Each message object has: id, from_email, subject, body_text, body_html, direction (inbound/outbound), created_at.

Step 2: Formatting for LLM Context

Raw email bodies contain quoted text, signatures, and HTML artifacts. Clean and structure them:

from datetime import datetime
from typing import List

def format_thread_for_llm(messages: list) -> str:
    lines = []
    for msg in messages:
        direction = "Customer" if msg.direction == "inbound" else "Agent"
        timestamp = msg.created_at.strftime("%Y-%m-%d %H:%M UTC") if msg.created_at else "Unknown time"
        body = clean_email_body(msg.body_text)

        lines.append(f"[{timestamp}] {direction} ({msg.from_email}):")
        lines.append(body)
        lines.append("")  # Blank line between messages

    return "\n".join(lines)

def clean_email_body(text: str) -> str:
    if not text:
        return "(no body)"

    lines = text.split("\n")
    cleaned = []
    for line in lines:
        # Skip quoted reply lines (start with >)
        if line.startswith(">"):
            continue
        # Skip common signature separators
        if line.strip() in ("--", "— ", "Sent from my iPhone", "Best regards,"):
            break
        cleaned.append(line)

    # Collapse multiple blank lines
    result = "\n".join(cleaned)
    while "\n\n\n" in result:
        result = result.replace("\n\n\n", "\n\n")

    return result.strip()

Step 3: The Structured Format vs. Raw Text Decision

Raw text (above) works for most cases. For better results with complex threads, use a structured chat format that the LLM understands as a conversation:

def format_thread_as_messages(messages: list) -> list[dict]:
    """Format thread as OpenAI chat messages for in-context conversation history."""
    formatted = []
    for msg in messages:
        role = "user" if msg.direction == "inbound" else "assistant"
        body = clean_email_body(msg.body_text)
        formatted.append({
            "role": role,
            "content": f"From: {msg.from_email}\nSubject: {msg.subject}\n\n{body}"
        })
    return formatted

# Use directly as chat history
messages = format_thread_as_messages(thread.messages)
messages.append({
    "role": "user",
    "content": "The customer just replied — draft the next response."
})

response = openai_client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "system", "content": "You are a professional email agent."}] + messages,
)

Step 4: Token Counting and Context Window Management

Long threads can exceed your model's context window. Count tokens before sending:

import tiktoken

def count_tokens(text: str, model: str = "gpt-4o") -> int:
    enc = tiktoken.encoding_for_model(model)
    return len(enc.encode(text))

def truncate_thread_to_fit(
    messages: list,
    max_tokens: int = 100_000,  # Leave room for system prompt + completion
    model: str = "gpt-4o",
) -> list:
    total_tokens = 0
    # Always keep the most recent messages — truncate oldest first
    kept = []
    for msg in reversed(messages):
        body = clean_email_body(msg.body_text)
        msg_tokens = count_tokens(f"From: {msg.from_email}\n{body}", model)
        if total_tokens + msg_tokens > max_tokens:
            break  # Stop adding older messages
        kept.insert(0, msg)  # Prepend to maintain chronological order
        total_tokens += msg_tokens

    if len(kept) < len(messages):
        print(f"Truncated thread: showing {len(kept)}/{len(messages)} messages ({total_tokens} tokens)")

    return kept

Step 5: Semantic Search for Very Long Threads

If a customer has 200 emails in a thread and you need the 5 most relevant ones to answer their current question, use search instead of loading everything:

def get_relevant_context(thread_id: str, query: str, top_k: int = 5) -> list:
    """Fetch only the most relevant messages from a long thread."""
    # Option A: Commune native search (if available)
    results = client.threads.search(
        thread_id=thread_id,
        query=query,
        limit=top_k,
    )
    return results.messages

    # Option B: Load recent + embed + similarity search
    # (use if Commune search is not available for your plan)

Complete LangChain Integration

Use Commune thread history as LangChain memory:

from langchain.memory import ConversationBufferMemory
from langchain.schema import HumanMessage, AIMessage
from langchain_openai import ChatOpenAI
from langchain.chains import ConversationChain
from commune import CommuneClient

client = CommuneClient(api_key="YOUR_API_KEY")

def load_commune_thread_as_memory(thread_id: str) -> ConversationBufferMemory:
    """Load an email thread into LangChain memory."""
    thread = client.threads.get(thread_id)
    messages = truncate_thread_to_fit(thread.messages)

    memory = ConversationBufferMemory(return_messages=True)
    for msg in messages:
        body = clean_email_body(msg.body_text)
        if msg.direction == "inbound":
            memory.chat_memory.add_user_message(body)
        else:
            memory.chat_memory.add_ai_message(body)

    return memory

def handle_inbound_email(payload: dict) -> str:
    thread_id = payload["thread_id"]

    # Load thread history as LangChain memory
    memory = load_commune_thread_as_memory(thread_id)

    llm = ChatOpenAI(model="gpt-4o", temperature=0.3)
    chain = ConversationChain(
        llm=llm,
        memory=memory,
        verbose=True,
    )

    # Generate reply with full thread context
    new_message = clean_email_body(payload["body"])
    reply = chain.predict(input=new_message)

    # Send reply maintaining thread
    result = client.messages.send(
        inbox_id=payload["inbox_id"],
        to=payload["from"],
        subject=f"Re: {payload['subject']}",
        body=reply,
        thread_id=thread_id,
    )

    return result.thread_id

Performance Considerations

Cache thread history if your agent handles multiple turns in quick succession — fetching the same thread 3 times in 10 seconds wastes API quota
Stream thread messages if the thread has 50+ messages — avoid loading all into memory at once
Prefer body_text over body_html for LLM context — HTML adds tokens with no semantic value
Set a reasonable max_tokens — leaving 2000+ tokens for the completion is important; a reply cut off mid-sentence is worse than a shorter context window

shanjairaj7 · 2026-03-01T12:01:46Z

shanjairaj7
Mar 1, 2026
Maintainer Author

Using This with Claude's 200k Context Window for Very Long Email Threads

Claude's 200k token context window changes the math significantly. For most email threads — even long customer support histories — you can load the entire thread without truncation:

import anthropic
from commune import CommuneClient

client = CommuneClient(api_key="YOUR_API_KEY")
anthropic_client = anthropic.Anthropic()

def handle_email_with_claude(payload: dict) -> str:
    thread_id = payload["thread_id"]
    thread = client.threads.get(thread_id)

    # With Claude's 200k context, load the full thread
    # Only truncate if thread is genuinely massive (rare)
    messages = thread.messages

    # Format thread as Claude messages
    claude_messages = []
    for msg in messages:
        role = "user" if msg.direction == "inbound" else "assistant"
        body = clean_email_body(msg.body_text)
        claude_messages.append({
            "role": role,
            "content": f"[Email from {msg.from_email} at {msg.created_at}]\n{body}"
        })

    # The new inbound email is already in thread.messages if fetched fresh
    # But if processing from webhook payload, append it explicitly
    if not any(m.id == payload.get("message_id") for m in messages):
        claude_messages.append({
            "role": "user",
            "content": f"[Email from {payload['from']} — just received]\n{payload['body']}"
        })

    # Add instruction as last user message if conversation ends with assistant
    if claude_messages and claude_messages[-1]["role"] == "assistant":
        claude_messages.append({
            "role": "user",
            "content": "Based on the full email history above, draft a professional reply to the customer's most recent message."
        })

    response = anthropic_client.messages.create(
        model="claude-opus-4-6",
        max_tokens=1024,
        system=(
            "You are a professional customer support agent. "
            "You have access to the full email conversation history. "
            "Write replies that reference prior context naturally when relevant. "
            "Be concise but thorough. Never repeat information the customer already knows."
        ),
        messages=claude_messages,
    )

    reply_text = response.content[0].text

    # Send via Commune with thread continuity
    result = client.messages.send(
        inbox_id=payload["inbox_id"],
        to=payload["from"],
        subject=f"Re: {payload['subject']}",
        body=reply_text,
        thread_id=thread_id,
    )
    return result.thread_id

The practical advantage of Claude's large context: for a 6-month support history with a high-value customer (think 80+ emails), you can load everything and Claude can say "I see you've mentioned this issue three times before — let me escalate this directly to our engineering team." That level of contextual awareness is impossible if you are truncating to the last 5 messages.

A rough guide:

GPT-4o (128k): Handles ~500-600 typical emails before truncation needed
Claude (200k): Handles ~800-900 typical emails before truncation needed
A 2-year customer support history: Usually 20-100 emails — fits comfortably in either

0 replies

Navigation Menu

Uh oh!

How do I use Commune thread history as context for my LLM when generating email replies? #47

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

How do I use Commune thread history as context for my LLM when generating email replies? #47

Uh oh!

shanjairaj7 Mar 1, 2026 Maintainer

The Problem

Step 1: Fetching Thread History

Step 2: Formatting for LLM Context

Step 3: The Structured Format vs. Raw Text Decision

Step 4: Token Counting and Context Window Management

Step 5: Semantic Search for Very Long Threads

Complete LangChain Integration

Performance Considerations

Replies: 1 comment

Uh oh!

shanjairaj7 Mar 1, 2026 Maintainer Author

Using This with Claude's 200k Context Window for Very Long Email Threads

shanjairaj7
Mar 1, 2026
Maintainer

shanjairaj7
Mar 1, 2026
Maintainer Author