Don't Invent Your Own Session IDs: Using Email Headers to Scope Agent Memory

Once you've decided that thread state needs its own boundary—a key that says "everything inside this fence belongs to one conversation"—the obvious next move is tempting and wrong: invent your own. Hash the subject line. Start a new thread after a time gap. Group by sender plus day.

All of these break in ways that are hard to notice until production traffic finds the edge case, because email already solved this exact problem decades ago. RFC 5322 (and its predecessor RFC 822) defines a threading mechanism that every mail client and server already implements. Using it instead of reinventing it isn't just less work—it's structurally more correct.

The headers that actually matter

Three headers carry the threading information you need:

A reply three messages deep in a thread typically carries a References header listing all three prior Message-IDs, and an In-Reply-To pointing specifically at the most recent one. This is the chain you walk to recover the thread.

Why the Subject line is a trap

Subject is the field every naive implementation reaches for first, and it fails for reasons that are individually obvious but collectively easy to miss:

• Clients prepend "Re:", "RE:", "Fwd:", or localised equivalents ("Réf:", "回复:") inconsistently, so naive string matching on subject misses threads or merges unrelated ones.
• Users edit the subject mid-thread ("Re: Invoice question" becomes "Re: Invoice question — URGENT") without starting a new conversation.
• Two completely unrelated threads can legitimately share an identical subject line ("Quick question").

Microsoft's ecosystem adds its own signal—Thread-Topic and Thread-Index—which can help as a secondary heuristic inside Outlook-originated mail, but it's not a substitute for the standard headers, and it's absent entirely from non-Microsoft clients. Treat Subject and Thread-Topic as supporting evidence at best, never as the primary key.

Deriving a canonical thread ID

The algorithm is straightforward once you have the headers:

1. If References is present, the root Message-ID is the first entry in that list. Use a stable hash of the root Message-ID as thread_id.
2. If References is absent but In-Reply-To is present, look up the message that In-Reply-To points to and inherit its thread_id. This requires a small lookup table mapping message_id → thread_id for every message you've ingested.
3. If neither header is present, this is the first message in a new thread. Its own Message-ID becomes the root, and thread_id is derived from it.

def resolve_thread_id(message, thread_map):
    """
    message: dict with 'message_id', 'in_reply_to', 'references' (list)
    thread_map: persistent store of message_id -> thread_id
    """
    if message.get("references"):
        root_id = message["references"][0]
        thread_id = hash_id(root_id)
    elif message.get("in_reply_to"):
        parent_id = message["in_reply_to"]
        thread_id = thread_map.get(parent_id) or hash_id(parent_id)
    else:
        thread_id = hash_id(message["message_id"])

    thread_map.set(message["message_id"], thread_id)
    return thread_id

The thread_map table is the piece teams most often skip, and it's what makes step 2 reliable. Without it, any message whose References header got stripped or truncated by an intermediate mail server has no way to find its thread—you'd be relying on every message carrying the full chain, which isn't guaranteed.

Edge cases worth deciding up front

What this buys you

Once thread_id is reliably derived from headers rather than guessed from content, it becomes the partition key for everything described in the rest of this series: the table you key thread_state on, the boundary that decides what gets summarised together, and the signal that tells you when one conversation ends and a new one—possibly with the same person—begins.