Vector, Graph, and SQL: Comparison of AI memory architectures

What Happened

A HackerNews thread challenging the assumption that vector databases are the default choice for AI memory reached 136 points, with the submitter arguing that SQL databases handle many retrieval workloads more efficiently. The discussion centered on a comparison of vector, graph, and relational architectures for storing and querying AI context, with multiple commenters reporting production deployments where Postgres with pgvector or plain relational tables outperformed dedicated vector stores on cost and latency. The thread reflects a broader reexamination of vector-first stack decisions that became standard practice following the 2023 embedding boom.

Why It Matters

Memory architecture determines query patterns, infrastructure spend, and the operational surface area a team must maintain. Vector databases optimize for approximate nearest-neighbor search over high-dimensional embeddings, but many production AI workloads — conversational state, user history, tool outputs, session recall — are structured lookups with conditional filters, not pure semantic search. When similarity ranking is not the primary access pattern, a relational system with indexed columns can answer the same query at lower cost and with fewer moving parts. The discussion matters because it questions a default that many teams adopted without benchmarking against incumbent infrastructure they already operate.

Technical Details

Vector databases index embeddings using HNSW or IVF structures to support approximate cosine or dot-product similarity at scale, typically trading recall for latency. Relational systems like Postgres with pgvector now support the same distance operators, and recent versions (pgvector 0.7+) added half-precision and binary quantization that reduce index size and memory footprint. Graph databases model explicit relationships between entities, which suits multi-hop reasoning over knowledge structures but adds query-language and schema overhead. The trade-off is structural: vector stores win on high-recall semantic retrieval over large corpora; SQL wins on filtered, ordered, transactional access; graphs win on traversal-heavy dependency queries. Hybrid retrieval — filtering in SQL, then reranking by embedding similarity — is the pattern most commenters converged on.

Operational Impact

Teams running conversational agents, session memory, or tool-call history can often consolidate onto an existing Postgres instance rather than provisioning a separate vector service, removing a network hop and a vendor from the critical path. This reduces sync complexity — no dual-write between application state and a vector store — and keeps transactional consistency for memory writes and reads. Query costs shift from per-vector-operation pricing to fixed database compute, which favors predictable workloads and penalizes bursty ones. Engineering hiring also shifts: Postgres and SQL expertise is more broadly available than specialized vector DB operations, lowering onboarding friction. The trade-off is that teams lose managed scaling behavior for embedding-heavy read patterns and must tune indexes themselves.