Verdict: For most AI applications in 2026 — RAG, code search, recommendation engines, multi-tenant SaaS — storing vectors on object storage (S3, GCS, Azure Blob) with a cache in front costs 10x to 100x less than keeping everything in RAM, with latency that is "good enough" for the majority of query patterns. The architecture became mainstream when AWS shipped S3 Vectors as a first-class service in December 2025, and companies like Cursor, Notion, and Linear have already migrated and cut costs by 70–95%. You should use RAM-resident vector databases only when you need guaranteed sub-10ms latency on every single query regardless of access pattern.
Last verified: 2026-08-04 · Object-storage vector search is a mainstream pattern in 2026, not a hack · Cost savings of 10–100x are real and independently documented · Pricing/limits change often — re-check vendor pages before committing.
Why Would You Put Vectors on Object Storage?
Because the economics are not even close. A conventional vector database keeps every vector (plus graph structure) in memory or on local NVMe, triple-replicated for durability. You pay for peak capacity around the clock, whether or not anyone is querying. Object storage inverts the model: durability is the storage layer's problem (eleven nines for S3, versioning, cross-region replication), compute scales to zero when idle, and capacity is effectively unbounded.
The cost gap is staggering. AWS S3 Standard costs $0.023 per GB-month in us-east-1 (AWS S3 pricing page). A cloud instance with RAM provisioned for a vector index costs roughly $2–5 per GB-month depending on the instance type and whether you use spot or on-demand pricing. That is a 100–200x difference per GB of stored data. Even comparing S3 to locally-attached NVMe SSDs, the gap is roughly 10x, because SSDs are still ephemeral, capacity-bound, and triple-replicated for durability in a production setup.
This is why the pattern took over the long tail of workloads: massive corpora, bursty or infrequent queries, and multi-tenant platforms where most tenants are cold at any given moment.
How Does Vector Search on Object Storage Actually Work?
Every serious implementation of object-storage-backed vector search converges on the same four architectural ideas. Understanding them is the difference between a system that works and one that is agonizingly slow.
1. Immutable Segments (Write-Once, Compact Later)
Vectors land in append-only, LSM-style files. Updates write new segments, and background jobs compact old ones. This is necessary because object storage cannot cheaply mutate bytes in place — you cannot do an in-place update on an S3 object. Instead, you write a new version and the old one becomes stale until compaction sweeps it up. This is the same log-structured approach that powers Cassandra, LevelDB, and modern storage engines.
The implication for builders: writes are fast and durable (committed directly to object storage), but reads may need to merge multiple segments until compaction runs. Most implementations hide this from you, but if you are building your own, plan for a compaction strategy from day one.
2. Coarse Partitioning (Fetch Less, Not Faster)
Each query is a network round-trip to object storage, which takes 50–200ms depending on the provider and region. The game is not making that round-trip faster — it is fetching less data per query. Coarse partitioning (typically IVF-style centroid clustering) groups nearby vectors into partitions so a query touches a handful of partitions instead of the whole index.
A well-tuned system with IVF partitioning and a thousand partitions will fetch roughly 0.1% of the index per query. Without partitioning, you are scanning the entire vector corpus on every request, which defeats the purpose.
3. Quantization Plus Rescoring (Smaller Vectors, Same Recall)
Quantization shrinks vectors before storing them. Int8 quantization cuts storage 4x with minimal recall loss. Binary quantization cuts it up to 32x. The trick is to store both the quantized vectors (for fast candidate generation) and the full-precision vectors (for accurate rescoring of the top candidates). You fetch a large candidate set using the tiny quantized vectors, then rescore the top-K with full precision.
This is not optional at scale. A billion 768-dimensional float32 vectors is about 3TB raw. With int8 quantization, it drops to 750GB. With binary, it drops to 94GB. The storage savings compound directly into cost savings when you are paying per GB-month on object storage.
4. Three-Tier Cache Hierarchy (Hot, Warm, Cold)
This is the core innovation. Data lives in one of three tiers:
| Tier | Medium | Latency | Cost per GB-month |
|---|---|---|---|
| Hot | RAM | < 10ms | $2–5 (instance memory) |
| Warm | NVMe SSD | 10–50ms | $0.10–0.25 (attached SSD) |
| Cold | Object storage (S3/GCS/Azure Blob) | 100–500ms | $0.023 (S3 Standard) |
When a query arrives, the system checks RAM first, then SSD, then object storage. Frequently queried namespaces "inflate" into the hot tier. Inactive ones "deflate" back to cold storage and cost almost nothing. This is the "pufferfish" pattern — data expands into faster tiers when accessed and contracts when idle.
The first query against a cold namespace is slow (potentially 300–500ms). Subsequent queries in the same session hit cache and are fast (sub-10ms from RAM). Whether that cold-start latency matters depends entirely on your access pattern.
How Much Does It Actually Cost? A Worked Example
Let us do the napkin math that separates a sensible architecture decision from a benchmark-driven guess. Consider a realistic workload:
- 10 million vectors at 768 dimensions (typical for OpenAI text-embedding-3-small)
- 5 million queries per month
- Storage capacity needed: 10M × 768 × 4 bytes × 1.5 (index overhead) = ~46 GB
Object-Storage Architecture (S3 + cache)
Storage: 46 GB × $0.023/GB-month = ~$1.06/month for S3 Standard
Queries: $2.00 per million queries (typical S3-marginal vector DB pricing) × 5 = $10/month
Total: ~$11/month for the data layer, plus a modest compute budget for the cache nodes (a single 8-core node with 16GB RAM handles this load easily at ~$50/month)
RAM-Resident Architecture (conventional vector DB)
The same 10M vectors at 768-dim float32 needs roughly a 64GB RAM tier including overhead. A managed cluster provisioned with 64GB RAM runs $200–400/month depending on the vendor and replication factor.
The object-storage approach is roughly 10–20x cheaper at this scale, and the gap widens dramatically as your corpus grows. At a billion vectors (3TB), the RAM approach requires a large multi-node cluster ($2,000–5,000/month), while the S3 approach adds roughly $69/month in storage (3TB × $0.023) plus the same modest compute budget.
When Should You NOT Use Object Storage for Vectors?
Object-storage vector search is not the right answer for every workload. You should stick with a RAM-resident database when:
You need guaranteed sub-10ms latency on every query. If you are building a real-time recommendation system for a high-traffic e-commerce site where every millisecond of latency directly impacts conversion, the cold-start penalty is unacceptable.
Your workload is steady high-throughput, not bursty. If queries come in at a constant high rate 24/7 (not a few hours a day per user), your data is always warm anyway, and you are paying for SSD cache you never benefit from. A provisioned RAM cluster wins here.
Your corpus is small (under 1M vectors). At this scale, any free tier (Pinecone Starter, Qdrant free 1GB, or pgvector on an existing Postgres instance) handles the workload. The architectural savings of object storage do not justify the operational complexity below roughly 5M vectors.
You need maximum recall with no latency variance. Object-storage systems trade tail latency for cost. If you need every query to return in under 50ms, and you cannot tolerate a 300ms outlier when a namespace goes cold, stick with RAM.
For more on when to choose different AI infrastructure patterns based on your specific use case — including when to build your own pipeline versus use a managed service — see our guide on how to build an AI agent OS that ships any tool you ask for.
How Do the Major Object-Storage Vector Databases Compare?
The landscape in 2026 has settled into a clear set of options. Here is a comparison built from primary vendor pricing pages and independently reported customer data:
| Database | Architecture | Entry Price | Cold Query Latency | Notable Customers |
|---|---|---|---|---|
| Turbopuffer | S3/GCS/Azure Blob + RAM/SSD cache | $16/month minimum | ~300–500ms p99 | Cursor, Notion, Linear |
| S3 Vectors (AWS native) | S3-native, no separate cluster | Pay-per-use (PUTs + queries + storage) | < 1 second infrequent, ~100ms frequent | Integrated with Bedrock, OpenSearch |
| LanceDB | Open Lance columnar format on any S3-compatible store | Free (self-hosted) / managed cloud beta | Higher (few hundred ms) | Netflix, Runway, Character.ai |
| Pinecone Serverless | SSD-tiered (not pure object storage) | Free tier, then usage-based | ~50–100ms (SSD-backed) | General purpose |
| pgvector (Postgres) | Disk-resident with optional RAM caching | Included with Postgres hosting | High (full scan at scale) | Small-to-medium workloads |
A few notes on the comparison:
Turbopuffer is the breakout company of this category. Cursor's official case study reports a 95% cost reduction after migrating in November 2023, scaling to 1T+ documents and 80M+ namespaces, with 10GB/s write peaks (turbopuffer.com/customers/cursor). Notion reports 80% cost reduction across 10B+ vectors (turbopuffer.com/customers/notion). Linear replaced both Elasticsearch and pgvector, cutting costs 70% (turbopuffer.com/customers/linear).
S3 Vectors went generally available on December 2, 2025 at re:Invent, with support for 2 billion vectors per index and 10,000 indexes per vector bucket, claiming up to 90% cost reduction compared to specialized vector database solutions (AWS News Blog, December 2 2025). It is the lowest-friction option if you are already on AWS, but it is newer and less feature-rich than Turbopuffer.
LanceDB is the open-source alternative. It runs embedded (like "SQLite for vectors") on any S3-compatible store and uses the open Lance columnar format. It is the right choice if you want to own the format outright and avoid vendor lock-in.
For a broader comparison of AI infrastructure costs and how they interact — including free AI API providers that can power your embedding pipeline without upfront spend — see our list of 12 free AI API providers in 2026.
What Made Object-Storage Vector Search Possible? Two Key Changes
The architecture of storing vectors on S3 is not new in concept — the idea of tiering data between cheap and fast storage is as old as database engines. But two specific technical changes made it practical for vector search specifically.
Change 1: S3 Strong Consistency (December 2020)
On December 1, 2020, AWS announced that all S3 GET, PUT, and LIST operations became strongly consistent — "what you write is what you will read" (AWS News Blog, December 1 2020). Before this, S3 used eventual consistency for overwrites and deletes, which meant a vector database could not safely read an index segment back immediately after writing it. Every object-storage-backed database had to work around stale reads, which added complexity and latency.
Strong consistency eliminated that workaround entirely. A vector index can now write a segment to S3 and immediately serve reads from it, which is a fundamental requirement for any database that claims ACID-like durability on object storage.
Change 2: S3 Vectors as a Native Service (December 2025)
When AWS shipped S3 Vectors as a generally available service on December 2, 2025, it did not just add a feature — it validated the entire architecture. If the largest cloud provider builds native vector storage into S3 itself, object-storage-backed vector search is no longer a niche bet. It is the default.
S3 Vectors supports 2 billion vectors per index, query latency under 100ms for frequent queries and under 1 second for infrequent ones, and up to 90% cost reduction versus specialized vector databases (AWS re:Invent 2025 announcement).
How to Choose: A Decision Framework
For builders and small teams evaluating vector database architecture, here is the decision tree:
Under 1M vectors? Use pgvector on your existing Postgres, or Pinecone's free tier. Do not overthink it.
1M–10M vectors with steady high QPS? A provisioned RAM cluster (Qdrant, Milvus, or pgvector with enough RAM) is simpler and has predictable latency.
10M+ vectors OR massive multi-tenancy (many small namespaces)? Object-storage-backed is the right default. The cost savings at this scale are too large to ignore.
Building a product where each user/customer has their own corpus (code search, document Q&A, per-user RAG)? Object-storage with per-namespace partitioning is the architecture. The "pufferfish" pattern — each user's data inflates when active and deflates when idle — is exactly what makes this economically viable.
Need guaranteed single-digit-millisecond latency on every single query? Stay on RAM. There is no way around it. Object storage adds a network round-trip, and no amount of caching eliminates the cold-start tail.
For more on how to think about the strategic implications of choosing AI infrastructure — especially as a small team or solo builder — the framework in the 5 levels of AI building applies directly here: do not compete on infrastructure you cannot win; choose the architecture that matches your workload and cost constraints.
What This Means for You
If you are building an AI product in 2026 — a RAG chatbot, a code search tool, a recommendation engine, or a multi-tenant SaaS that lets each customer query their own data — the default vector database architecture has shifted. Object-storage-first is no longer experimental. AWS, Turbopuffer, and LanceDB have proven it in production at companies processing billions of vectors and trillions of documents.
The practical takeaway: start with the free tier of whatever is easiest (pgvector on an existing Postgres, or Pinecone Starter). When your bill crosses roughly $200/month or your corpus crosses 10M vectors, evaluate an object-storage-backed architecture. The migration is usually straightforward because the data is just vectors and metadata — you export, re-ingest, and switch the query endpoint. The cost difference at scale is not a theoretical argument. It is the difference between a product whose unit economics work and one that does not.
For related infrastructure decisions you will face as you scale — including how to give your AI agents persistent memory without provisioning a database for each one — see our guide on the three layers every AI agent OS needs, which covers the broader memory architecture pattern this fits into.
FAQ
Q: Can you really run a vector database on S3?
A: Yes. As of 2026 it is a mainstream architecture, not a hack. Turbopuffer, LanceDB, and AWS S3 Vectors all store vectors in object storage (S3, GCS, or Azure Blob) with a cache hierarchy in front. AWS made the pattern official when S3 Vectors went GA in December 2025. The tradeoff is single-digit-millisecond latency for roughly an order of magnitude lower cost.
Q: How much cheaper is object-storage vector search versus RAM?
A: Roughly 10x to 100x cheaper for storage-heavy, moderate-throughput workloads. S3 Standard costs $0.023/GB-month. RAM provisioned in a cloud instance costs roughly $2–5/GB-month. A 10M-vector workload at 768 dimensions costs about $11/month in S3 storage versus $200–400/month for a RAM cluster. Cursor reported a 95% cost reduction after migrating to Turbopuffer, per their official case study on turbopuffer.com.
Q: What is the latency of querying vectors stored on S3?
A: It depends on the cache state. A cold query (data has been idle and needs to load from S3) takes 100–500ms. A warm query (data is cached on SSD) takes 10–50ms. A hot query (data is cached in RAM) takes under 10ms. S3 Vectors, which is the AWS-native option, reports ~100ms for frequent queries and under 1 second for infrequent ones, per AWS's December 2025 GA announcement.
Q: When should I avoid object-storage vector search?
A: Avoid it when you need guaranteed sub-10ms latency on every query regardless of access pattern, when your workload is steady high-throughput (24/7 constant QPS), when your corpus is under 1M vectors (use a free tier instead), or when you cannot tolerate latency variance. In these cases, a provisioned RAM cluster is the right choice.
Q: What changed in 2020 that made object-storage vector databases possible?
A: AWS made S3 strongly consistent on December 1, 2020, which eliminated stale reads after writes. Before that, every object-storage-backed database had to work around eventual consistency for overwrites and deletes. Strong consistency means a vector index can write a segment to S3 and immediately serve reads from it with confidence that the data is current.
Q: Is Turbopuffer the only option, or are there alternatives?
A: No, there are several. AWS S3 Vectors is the native option on AWS (GA December 2025). LanceDB is the open-source, self-hostable alternative using the open Lance columnar format. Pinecone Serverless uses an SSD-tiered architecture (not pure object storage) but offers similar cost benefits for some workloads. pgvector on Postgres works for small-to-medium workloads without needing a separate database. Choose based on your cloud, your need for open-source versus managed, and your scale.
Discussion
0 comments