Below roughly a million vectors, pgvector in your existing Postgres is fast enough, and it keeps embeddings in the same transaction and the same backup as the rows they describe. A dedicated vector database buys scale you do not have yet and costs you consistency you already needed.
For corpora below roughly a million vectors, pgvector in your existing Postgres returns similarity results in single-digit milliseconds. It also keeps the embedding in the same transaction and the same backup as the row it describes, which removes an entire class of consistency bug that a second datastore introduces on day one.
Nearly every AI proposal that crosses our desk includes a dedicated vector database on the architecture diagram. In two years of scoping, we have needed one twice.
#What the actual corpus size tends to be
A mid-market GCC company\'s entire document estate — policies, product data, support history, contracts — chunks down to somewhere between thirty thousand and three hundred thousand vectors. That is not a scale problem. An HNSW index over that in Postgres answers in a handful of milliseconds on hardware you are already paying for.
The vector databases are genuinely excellent, and they are built for a scale most of these projects will not reach for years, if ever.
#The consistency argument, which matters more than speed
Two datastores means two writes. A document updates in Postgres, and its embedding updates in the vector store — in a second request, over a network, with its own failure mode.
When that second write fails, nothing errors. The system keeps answering. It just answers using the embedding of the previous version of a policy that has since changed, and it does so confidently, and nobody finds out until somebody acts on it. Reconciling that is a background job you now have to write, monitor and be paged for.
With pgvector, the row and its embedding update in one transaction. The failure mode is a rolled-back write, which is a failure mode you already understand and already alert on.
#The filtering argument
Real retrieval queries are rarely pure similarity. They are "the closest matches, in this tenant, that this user is allowed to see, from the current version of the document, excluding drafts".
In Postgres that is one query with a WHERE clause. Across two stores it is a similarity search, a round trip, an ID list, a second query, and a decision about what to do when post-filtering leaves you with three results instead of the ten you asked for.
#When to actually move
When you have measured the ceiling, not when you have imagined it. Watch p95 query time as the corpus grows. When it stops being comfortable, migrate — it is roughly a day of work, and by then you will have real numbers to choose and size the replacement with, instead of a guess made before the first document was indexed.
- Most GCC mid-market projects hold tens of thousands of chunks, not tens of millions.
- One datastore means one transaction: a document and its embedding cannot get out of sync.
- You can filter by tenant, permission and date in the same query as the similarity search.
- Move when you actually measure the ceiling — the migration is a day, and you will have real numbers to size it with.
