Subscribe

Building an Open Brain: Thought Capture + Semantic Search in a Personal Dashboard

Adding unstructured thought capture, LLM auto-classification, and vector-based semantic search to a personal FastAPI + SQLite dashboard. No vector database required.

Building an Open Brain: Thought Capture + Semantic Search in a Personal Dashboard

My personal dashboard tracks the structured stuff well: habits, mood, sleep, goals, journal entries. But the unstructured stuff? Fleeting ideas, decisions made during a walk, mental notes about people. Those disappeared into thin air. I wanted a way to capture raw thoughts, have them auto-classified, and then search them by meaning, not just keywords.

Inspired by Nate B Jones' "Open Brain" concept, I added three capabilities to my existing FastAPI + SQLite dashboard in a single build session: quick thought capture, LLM auto-classification, and vector-based semantic search.

What It Does

The system accepts a raw thought from any interface: the web UI, the REST API, or an MCP tool accessible from Claude conversations. On capture, it:

  1. Auto-classifies the thought type (idea, decision, person_note, observation, action_item, reflection)
  2. Extracts people mentioned, topic tags, and action items
  3. Generates a 1536-dimensional vector embedding for semantic search
  4. Stores everything in SQLite

Later, you can search across all thoughts and journal entries by meaning. "kubernetes secrets management" finds a thought about "Decided to use Sealed Secrets instead of SOPS for the ArgoCD integration", even though the words barely overlap.

The Data Model

The Thought table is intentionally simple. Metadata (people, topics, action items) is stored as JSON strings, and the embedding vector is a JSON array of 1536 floats stored as text:

class Thought(Base):
    __tablename__ = "thoughts"

    id: Mapped[int] = mapped_column(primary_key=True)
    content: Mapped[str] = mapped_column(nullable=False)
    thought_type: Mapped[str] = mapped_column(default="observation")
    people: Mapped[str | None]      # JSON array
    topics: Mapped[str | None]      # JSON array
    action_items: Mapped[str | None] # JSON array
    embedding: Mapped[str | None]   # JSON array of 1536 floats
    source: Mapped[str] = mapped_column(default="api")
    created_at: Mapped[str]

JSON-in-SQLite works perfectly at personal scale. No need for PostgreSQL with pgvector, no need for a dedicated vector database. Thousands of records with 1536-float vectors? SQLite handles it without breaking a sweat.

Auto-Classification with Claude Haiku

When a thought comes in without explicit metadata, it gets routed through Claude Haiku for classification. The system prompt is straightforward: extract the type, people, topics, and action items:

_SYSTEM_PROMPT = """You classify captured thoughts and extract structured metadata.
Given a raw thought, return a JSON object with exactly these keys:

- "thought_type": one of: idea, decision, person_note, observation,
                  action_item, reflection
- "people": array of person names mentioned (empty array if none)
- "topics": array of 1-3 short topic tags (lowercase)
- "action_items": array of actionable items extracted (empty array if none)

Return ONLY valid JSON, no explanation."""

The classification adds about half a second of latency. For a brain-dump tool, that's acceptable: you get immediate feedback on what the system understood. If Haiku is unavailable, the thought still gets stored with a default type of "observation" and empty metadata. No data lost.

Every LLM response gets validated: if the returned thought_type isn't in the valid set, it falls back to "observation". If people isn't an array, it becomes an empty array. Defense in depth against LLM unpredictability.

Vector Embeddings Without the Complexity

For embeddings, I'm using OpenAI's text-embedding-3-small model (1536 dimensions) routed through OpenRouter. The embedding generation is a single async HTTP POST:

async def generate_embedding(text: str) -> list[float] | None:
    async with httpx.AsyncClient(timeout=30) as client:
        resp = await client.post(
            "https://openrouter.ai/api/v1/embeddings",
            headers={
                "Authorization": f"Bearer {api_key}",
                "Content-Type": "application/json",
            },
            json={
                "model": "openai/text-embedding-3-small",
                "input": text[:32000],
            },
        )
        resp.raise_for_status()
        return resp.json()["data"][0]["embedding"]

The interesting decision was cosine similarity. I wrote it in pure Python (no numpy, no scipy, no additional dependencies):

def cosine_similarity(a: list[float], b: list[float]) -> float:
    dot = sum(x * y for x, y in zip(a, b))
    mag_a = math.sqrt(sum(x * x for x in a))
    mag_b = math.sqrt(sum(x * x for x in b))
    if mag_a == 0 or mag_b == 0:
        return 0.0
    return dot / (mag_a * mag_b)

This is a brute-force scan: load all embeddings, compute similarity against each one, sort, return the top results. At personal scale (fewer than 10,000 records), this runs in well under a second. A vector database like Pinecone or Weaviate would be engineering theater for this use case.

The search also indexes journal entries alongside thoughts. One search query, two data sources, ranked by semantic similarity. If the embedding API is unavailable, the system falls back to SQL ILIKE keyword search: graceful degradation, not a crash.

MCP Integration: The Killer Feature

The MCP (Model Context Protocol) tools are where this gets genuinely useful. Two tools make the system accessible from any Claude conversation:

capture_thought: dump a thought into the system with a single function call. Auto-classifies, auto-embeds, returns the extracted metadata as confirmation:

@mcp.tool()
async def capture_thought(
    content: str,
    thought_type: str | None = None,
    source: str = "mcp",
) -> dict:
    """Capture a quick thought, idea, or decision.
    Auto-classifies and extracts people, topics, action items."""

search_brain: semantic search across everything. "What did I think about container networking?" finds relevant thoughts and journal entries even if I never used those exact words:

@mcp.tool()
async def search_brain(
    query: str,
    limit: int = 10,
    days: int | None = None,
) -> dict:
    """Semantic search across all thoughts and journal entries
    by meaning, not keywords."""

This turns every Claude conversation into a bidirectional knowledge interface. I can capture context from a conversation ("capture this decision we just made") and retrieve context for a conversation ("what have I thought about this topic before?").

What Bit Me

A few lessons from the build:

ORM models must match the database schema. I added the embedding column to the journal_entries table via a migration but forgot to add the corresponding attribute to the SQLAlchemy ORM model class. The column existed in the database, but SQLAlchemy didn't know about it, resulting in an AttributeError in production when semantic search tried to read journal embeddings. Migration and model are two separate things; both need updating.

Secret rotation is not optional. I accidentally committed an API key to source control during initial setup. Even though it was removed in the next commit, the key was in git history. Rotated the key immediately through the provider, updated the SOPS-encrypted Kubernetes secret, and redeployed. The whole rotation took about 5 minutes, but only because the secret management pipeline (SOPS + Age + Flux) was already in place.

JSON embedding storage is fine. Storing 1536-float vectors as JSON text in SQLite sounds wasteful. It is: each embedding is about 12KB of JSON text. But for personal scale, storage is irrelevant. 10,000 thoughts × 12KB = 120MB. My SQLite database has room. The simplicity of not running a vector database outweighs the storage overhead by a wide margin.

What's Next

The immediate plan is to backfill embeddings on all existing journal entries so semantic search covers historical content. Beyond that, I want to auto-surface related thoughts in the daily digest: "you captured a thought about X last week, and today's journal entry is about Y, which is semantically similar." Cross-pollination between past and present thinking.

The broader lesson: you don't need a complex stack to build a personal knowledge system with semantic search. SQLite, a few hundred lines of Python, an embedding API, and an LLM for classification. The "Open Brain" isn't an architecture. It's a feature you add to whatever you're already running.