SQL-based approach for AI memory outperforms vector and graph alternatives

What Happened

A HackerNews thread on SQL-based AI memory systems drew 136 upvotes, with commenters reporting that relational approaches outperformed vector and graph databases on retrieval tasks for agent memory. The discussion centered on architectures that store conversational and episodic memory as structured rows, using indexed queries for deterministic recall rather than embedding similarity search. Several builders described production deployments where latency and cost dropped after migrating portions of their memory layer off vector stores.

Why It Matters

Vector databases became the default substrate for AI memory on the assumption that semantic similarity is the primary retrieval primitive. The thread surfaces a counterexample: many production memory queries are actually structured — fetch the last N turns for a user, filter by session, join against metadata, or retrieve facts by entity key. These patterns map cleanly to B-tree indexes and SQL execution plans, which avoid embedding generation cost, approximate nearest neighbor overhead, and index rebuild churn. For teams running cost-sensitive agents at volume, shifting deterministic recall to Postgres-class storage reduces both per-query latency and the vector throughput bill. It also narrows the surface area where an external vector service sits on the critical path.

Technical Details

Reported architectures combine a relational table for structured memory (user_id, session_id, timestamp, role, content) with optional pgvector or a separate ANN store for semantic recall. Deterministic lookups hit standard indexes; only queries requiring fuzzy matching route through embeddings. Commenters cited p99 latencies in the low single-digit milliseconds for indexed SQL recall versus tens of milliseconds for vector round-trips, though these numbers are workload-dependent and not benchmarked under controlled conditions. The approach requires schema design upfront — entity modeling, retention policies, and migration handling — which vector-only stacks defer. Limitations: SQL does not natively express semantic similarity, so hybrid stacks still carry two query paths and must reconcile ranking across them. Graph approaches were discussed but drew less support for the specific recall patterns cited.

Operational Impact

Builders can consolidate memory storage into an existing Postgres instance, removing a managed vector service from the dependency graph and simplifying backup, auth, and observability. Cost profiles change: embedding API calls drop for structured queries, and vector index size shrinks when only semantic-relevant content is embedded. Recall logic becomes testable as SQL — deterministic, diffable, and debuggable with standard tooling — rather than tuned against embedding model versions. Teams already running managed Postgres may find the marginal infrastructure cost of adding memory is near zero, shifting the build-versus-buy calculus away from dedicated vector vendors for non-semantic workloads. The tradeoff is schema rigidity: memory shape changes now require migrations.