Wiring the Open Brain into the Daily Digest

Yesterday I wrote about building the Open Brain: thought capture, LLM auto-classification, and semantic search added to my personal dashboard. Today I wired it into the daily digest, which turned out to be more interesting than I expected. The digest isn't just a summary anymore. It's starting to think.

Two Cards, Two Different Jobs

The daily digest already rolls up mood, sleep, habits, goals, activity, and a dozen other data points into a single page. Adding thoughts meant adding two distinct cards, each solving a different problem.

The first card is straightforward: Today's Thoughts. Every thought captured during the day appears here with its auto-classified type badge (idea, decision, observation, etc.), topic tags, and people mentioned. This is just a mirror: you captured it, here it is in your daily summary. The value is seeing your thoughts alongside your structured data. An "idea" about restructuring your workout routine sits next to your habit completion rate and sleep hours. Context that was separate becomes connected.

The second card is where it gets interesting: Related to Today. This takes today's journal entries, generates a vector embedding of the combined text, and searches the last 90 days of thoughts and journal entries for semantically similar content. The top three matches appear with a similarity score.

Why Semantic Matching in a Digest Matters

The traditional approach to surfacing related content is keyword matching. Write about "kubernetes" today, find past entries that mention "kubernetes." That works for exact-match recall but misses the connections that actually matter.

Semantic search finds meaning, not words. A journal entry about "feeling overwhelmed by the migration timeline" matches a past thought about "need to break the ArgoCD rollout into smaller phases", even though they share zero keywords. The embedding model understands that both are about managing complexity in infrastructure work.

This is the kind of connection your brain makes in the shower, except now it happens automatically every day. And unlike your shower thoughts, it shows you the receipts: the actual past thought, when you captured it, what topics it was tagged with, and how strong the semantic match is.

Implementation Decisions

A few choices that shaped how this works:

Threshold at 0.35, not 0.30. The general semantic search uses 0.30 as its minimum cosine similarity. For the digest, I bumped it to 0.35. The digest is a curated daily view: it should surface connections that are genuinely relevant, not vaguely related. A 35% match on a 1536-dimensional embedding is a meaningful signal. Below that, you get noise.

90-day lookback window. I considered searching all time but landed on 90 days. Thoughts from six months ago are less likely to be actionably related to today's journal entry. Three months keeps the results fresh and relevant. This is configurable if I change my mind.

Exclude today's own content. The search runs against today's journal text, so without filtering, today's own journal entries and thoughts would rank highest (they're semantically identical to themselves). The implementation explicitly filters out today's thought IDs and today's journal date from the results. Only past content surfaces.

Top 3, not top 5. The digest is already information-dense. Three related items is enough to spark a connection without adding scroll fatigue. Each card shows a 150-character preview, the type (thought vs. journal), the date, topic tags, and the match percentage.

Resilient by default. The related thoughts lookup is wrapped in a try/except that logs failures and returns an empty list. If the embedding API is down, if there are no embeddings yet, if the search returns nothing, the digest renders normally without the card. No feature should break the daily summary.

The Journal Content Problem

One thing I hadn't considered: the daily digest previously only counted journal entries ("3 journal entries"), it never fetched the actual content. To power semantic search, I needed the text. So I added a second query that pulls journal content alongside the count.

To keep embedding costs reasonable, the implementation concatenates up to three journal entries and truncates to 2,000 characters before generating the query embedding. A typical journal entry is a few hundred characters. This covers the day's writing without hitting token limits or running up the API bill.

What the Data Flow Looks Like

When the digest computes:

1. Query today's thoughts from the database (already captured and classified throughout the day)
2. Query today's journal entry content
3. If journal content exists, combine it and generate a vector embedding via OpenRouter
4. Search the last 90 days of embedded thoughts and journal entries by cosine similarity
5. Filter out today's own content, apply the 0.35 threshold, take top 3
6. Include both card datasets in the digest dict

The template renders the "Today's Thoughts" card with color-coded type badges (yellow for ideas, blue for decisions, purple for person notes) and the "Related to Today" card with match percentages and source type indicators.

Backfilling Matters

Semantic search is only as good as your embedding coverage. New thoughts get embedded on capture, but existing journal entries needed a backfill. A simple batch function runs through all entries with embedding IS NULL and generates vectors via the API. For my dataset, this was 3 entries (the system is new). But as the journal grows, the backfill becomes the foundation: every past entry becomes searchable by meaning.

The backfill is idempotent and rate-limited by batch size. Run it once after setup, then new entries get embedded automatically on creation.

The Bigger Picture

The digest started as a daily report card: here's what you did, here's how you're doing. Adding semantic thought connections shifts it toward something more like a daily briefing from a system that actually understands your context.

"You journaled about feeling stuck on the Copilot rollout. Three weeks ago you captured a thought about breaking large rollouts into pilot phases. Here's that thought, 42% match."

That's not a report card. That's a thinking partner surfacing your own past wisdom at the moment you need it. The data was always there: in separate thoughts, in journal entries written weeks apart. The embeddings just make the invisible connections visible.

Next up: surfacing related thoughts in the council briefing so the AI advisors have access to your thought history when generating their commentary. The Open Brain becomes shared context.