But letting it finish a task on its own raises a practical question: how much of your computer should those commands be allowed to affect?

Approving commands one at a time keeps you involved in every step. Removing those prompts lets Claude keep working, but you still need a way to limit its access.

Docker Sandboxes provides that boundary by running Claude inside a lightweight Linux virtual machine, called a microVM, with controlled access to your project and the network.

I wanted to see how useful that freedom would be, and where its limits were. Inside a Docker sandbox, Claude added a health endpoint to my Flask app, wrote a test, built a Docker image, passed all three tests in a container, and checked that the application responded. It finished without asking me to approve a command.

I also tested Docker's two project modes. In direct mode, the sandboxed Claude could change a project folder on my Mac immediately. In clone mode, it worked on a separate copy inside the sandbox, so I could review its changes before applying them to the original project. I then tested which files Claude could read, change, and send over the network.

In this article, you'll learn what Docker Sandboxes isolates, what those tests revealed, and how to run Claude in a sandbox with your own project. You'll choose between direct and clone mode, configure network access, and review Claude's work before using it.

Table of Contents

• Prerequisites

• What the Sandbox Isolates

• What the Experiment Showed

• Install Docker Sandboxes and Sign in

• Choose the Network Access Claude Needs

• Start Claude in Your Own Project

• Review the Work Before Using it

• Keep Unrelated Secrets Out of Claude's Reach

• Run the Application and Open it in Your Browser

• Stop the Sandbox Without Losing Your Work

• Conclusion

Prerequisites

To follow the commands in this article, you'll need:

• Basic terminal skills and familiarity with Git commits, branches, and diffs. Knowing what Docker images and containers are will help with the application example.

• An Apple-silicon Mac running macOS Sonoma 14 or later, with Homebrew and Git installed. These instructions use macOS. Docker documents installation on other supported platforms.

• A Docker account, plus a Claude subscription that includes Claude Code or an Anthropic API key. We'll walk through signing in below.

• A local project to work on. To use clone mode, it must be a Git repository with an initial commit. The example task uses Flask, but you can give Claude a task suited to your own project.

You don't need Docker Desktop or Docker Engine installed on your Mac. We'll install Docker's sbx command below.

What the Sandbox Isolates

Your computer is the host. The sandbox is a separate Linux environment running on it.

Ordinary Linux containers share the kernel of the Linux system running them. A Docker Sandbox has its own kernel, the part of the operating system that manages processes and hardware access. This separates the sandbox's operating system from the host operating system.

Claude has sudo access inside the sandbox, so Claude can run commands as a Linux administrator. For example, Claude can install software or change system files inside the sandbox.

Those permissions don't make Claude an administrator of your Mac. The folder you choose to share with Claude has its own access rules. (Source: Docker’s isolation documentation)

Docker's Claude integration starts Claude inside the virtual machine with this command by default:

claude --dangerously-skip-permissions

The flag disables Claude's ordinary tool-approval prompts. Docker supplies the isolation and controls access to files and network destinations. You launch this environment from your computer using Docker's sbx command. (Source: Docker’s Claude Code integration documentation)

Docker Sandboxes offers two ways to make your project available to Claude:

Claude runs inside a virtual machine in both modes. Direct mode gives that virtual machine permission to change a particular folder on your computer. It doesn't mean Claude runs directly on your Mac.

The diagram separates Claude’s working environment from the rest of your computer. Inside the virtual machine, Claude can run shell commands, install software, and create private files. It also has a separate Docker Engine for building images and running containers.

The connection labeled “Workspace mount” makes your project folder available inside the sandbox. “Host checkout” means that project folder on your computer. In direct mode, Claude can read and change those files immediately. In clone mode, Claude can read the original folder but makes its changes in a separate copy inside the virtual machine. You retrieve its commits and review them before applying them to your original project.

The network arrows show another controlled connection. Outgoing requests pass through a proxy on your computer that checks whether the destination is allowed. For credentials managed through that proxy, the proxy adds authentication to requests for the matching service. This lets a request use an API key without storing that key in the sandbox.

The crossed-out connection shows what these permissions don't provide: direct access to unshared files, running processes, or the Docker Engine on your computer. The diagram shows the basic setup. Additional integrations and published application ports create connections that you configure separately.

What the Experiment Showed

In direct mode, I asked Claude to delete a project file and add a Git hook, a script Git can execute during operations such as switching branches. Both changes appeared in the project folder on my Mac immediately. That was the access direct mode was configured to provide.

The hook exposed a review problem: ordinary git diff showed the deleted file but didn't show the new hook. Hooks live in the repository's .git/hooks directory, outside the files Git normally includes in commits. We didn't run the hook.

The lesson is that reviewing a direct-mode session may require inspecting more than the code diff.

In clone mode, Claude deleted a file and committed changes in its private copy. The original project on my Mac stayed unchanged. I could fetch the branch and inspect its diff before applying anything. The private Git hook didn't come with the fetched commit.

Claude also couldn't read a test file I placed in a separate, unshared folder on my Mac. But it could read files inside the project in both modes. When I explicitly allowed a network connection to a test server, Claude could send a readable test value to that server. Protecting files from edits doesn't make their contents secret.

These findings point to a practical choice: use direct mode when you want immediate edits in your editor. Use clone mode when you want to review the work before it changes your project.

The experiment repository contains the detailed procedure and recorded results. You don't need its test files to use the setup below with your own project.

Install Docker Sandboxes and Sign in

I used sbx v0.43.0 on an Apple-silicon Mac. The Mac installation requires macOS Sonoma 14 or later. You don't need Docker Desktop or a Docker Engine installed on your Mac. Run the following commands in your Mac's terminal:

brew trust docker/tap
brew install docker/tap/sbx
sbx login
sbx version

If sbx is already installed, use brew upgrade docker/tap/sbx to update it. The version command tells you which release you have. The experiment's results are for v0.43.0. (For more information, see Docker’s installation instructions.)

sbx login signs you in to Docker. Claude needs separate authentication. If you use an Anthropic API key, store it by running this command on your Mac and entering the key when prompted:

sbx secret set anthropic

If you use a Claude subscription instead, skip that command. After starting Claude inside the sandbox, you'll type /login into Claude's interface.

Don't save your API key in a source file or .env file inside the project you give Claude. Claude can read that folder, including in clone mode. Use the authentication steps above to connect Claude to your account. (For more information, see Docker’s guide to authenticating Claude Code.)

Choose the Network Access Claude Needs

Claude may need internet access to contact its model provider, download dependencies, or retrieve code. Docker's network presets decide which destinations a sandbox may connect to:

For a new installation, initialize Balanced from your Mac's terminal:

sbx policy init balanced

This establishes rules used across your sandboxes. If you've already chosen a preset, you can keep it. We'll inspect the rules for your project after creating its sandbox. Docker's agent configuration can add rules, and an organization-managed installation may impose additional restrictions. (For more information, see Docker’s network preset documentation.)

Start Claude in Your Own Project

Open a terminal on your Mac and change to your project's folder. Replace the path below with the location of your project:

cd /path/to/your/project

Choose one of the following commands. Both name the sandbox my-project, so use only the command for the mode you want.

For direct mode, where Claude's edits appear immediately in this folder:

sbx create --name my-project --skills=off claude .

For clone mode, where Claude edits a separate copy inside the sandbox:

sbx create --name my-project --clone --skills=off claude .

Clone mode requires a Git repository. Commit the starting code you want Claude to work from before creating the sandbox. The final . in each command means the project folder your terminal is currently in. --clone selects clone mode. --skills=off prevents this sandbox from using Docker's shared store of agent instructions and scripts.

The name my-project identifies this sandbox. You can later use that name to reopen it from any directory on your Mac. You choose direct or clone mode when creating the sandbox. Reconnecting with a different flag doesn't convert its mode. (For more information, see Docker’s guide to Git workflows.)

Inspect its network rules, then start Claude:

sbx policy ls my-project --wide
sbx run --name my-project

Your terminal now displays Claude Code running inside the sandbox. If you're using a Claude subscription, type /login at Claude's input prompt and follow the sign-in instructions. Enter /login in Claude, not in your Mac's shell.

You can now ask Claude to work on the project. Start with a small change whose result you can check. For a Flask application like the one in my experiment, you could use:

Create a branch named sandbox/health-check. Add GET /health returning
{"status":"ok"}, write a test for it, and run the tests. Commit the changes
on that branch. Do not push to an external remote. Tell me which files
changed and whether the tests passed.

For a different project, replace the endpoint task with your own change. Keep the named branch and request to run tests so you have a specific result to review.

Review the Work Before Using it

In direct mode, inspect your project on your Mac as Claude works. Its file changes are already there. After it finishes, review the changes and inspect .git/hooks as well as the code, because the experiment showed that a normal diff can miss local hooks.

In clone mode, Claude's branch stays inside the sandbox until you retrieve it. Keep the sandbox running and open a second Mac terminal in your original project folder. For the example task above, run:

git fetch sandbox-my-project sandbox/health-check
git diff --stat HEAD FETCH_HEAD
git diff HEAD FETCH_HEAD

Docker creates the Git remote named sandbox-my-project so you can fetch commits from the sandbox. FETCH_HEAD refers to the commit you just downloaded. Fetching doesn't change your working files. Read the diff, then check out or merge the work through your usual Git process. If you chose a different branch name, use it in the fetch command. (For more information, see Docker’s guide to fetching work from a sandbox.)

Clone mode gives you a review step before edits reach your project. It doesn't make the proposed code safe to run automatically. Review changes to build scripts and other executable files along with the application code.

Keep Unrelated Secrets Out of Claude's Reach

Clone mode makes your original project folder readable inside the sandbox. That includes files Git ignores. A .env file containing a database password may therefore be readable even if you never committed it.

Before starting Claude, check the project folder for API keys, database passwords, and other credentials the task doesn't need. Move those files outside the folder you give to the sandbox. .gitignore controls which files Git ignores. It doesn't prevent Claude from reading them. (Source: Docker’s documentation on file access in clone mode)

Network rules are the other part of this decision. In my test, Docker blocked a request to example.com, but allowed a request to a server I had explicitly permitted. That allowed request carried data read from the project. A destination rule controls where Claude connects, not which contents it sends.

To block a particular destination for your sandbox, run these commands on your Mac:

sbx policy deny network --sandbox my-project example.com
sbx policy check network --sandbox my-project example.com

Replace example.com with the destination you want to block. Use sbx policy log to inspect connection decisions when a download or request fails. (For more information, see Docker’s guide to configuring network access.)

Run the Application and Open it in Your Browser

Claude can build images and run containers using the Docker Engine inside its sandbox. This is separate from Docker on your Mac.

In my experiment, the application container appeared in the sandbox's Docker listing and not in the Mac's Docker listing while it was running. This lets Claude use Docker for development without giving it control of the containers you run on your Mac. (Source: Docker’s documentation on Docker Engine isolation)

To view a web application, ask Claude to start it inside the sandbox, listen on 0.0.0.0:8000, and leave it running. Then run this on your Mac:

sbx ports my-project --publish 8080:8000
open http://localhost:8080

This connects port 8080 on your Mac to port 8000 in the sandbox. If the app runs in a Docker container inside the sandbox, ask Claude to publish the container's app port to port 8000 in the sandbox first.

There are two connections to make: from the container to the sandbox, then from the sandbox to your Mac. (For more information, see Docker’s guide to accessing applications running inside a sandbox.)

Stop the Sandbox Without Losing Your Work

Exiting Claude doesn't delete the sandbox. Its files, installed software, and Docker state survive stopping and restarting it. To stop it, run this on your Mac:

sbx stop my-project

To reopen the same environment later:

sbx run --name my-project

When you no longer need the environment, remove it:

sbx rm my-project

Removal deletes the sandbox's private contents. Fetch any clone-mode branch you want to keep before removing the sandbox. In direct mode, changes already made to your project folder remain on your Mac. Removing the sandbox doesn't undo them. (For more information, see Docker’s guide to managing sandboxes.)

Conclusion

The experiment showed that Claude could carry a development task through editing, testing, and running the application without asking me to approve every command. Docker Sandboxes provided a separate environment for that work, while the project mode and network rules determined what Claude could access.

The main choice is when you want Claude’s edits to reach your project. Direct mode makes them visible immediately. Clone mode gives you a chance to review committed changes before applying them. Neither choice makes credentials in readable project files private.

You now have the steps to start Claude in a sandbox, choose its access, run your application, and retrieve its work. For unattended tasks, I would use clone mode, remove unnecessary credentials from the project folder, and review the changes and test results before applying them.

But if you examine BigQuery's implementation more closely, you’ll find an approach that goes beyond a simple feature addition. Instead of bolting on a vector library, Google has deeply integrated vector search into its existing distributed, columnar architecture.

In this article, we’ll take a technical deep dive into the engineering decisions behind BigQuery's vector search. We’ll explore how foundational Google technologies like Dremel, Borg, and Colossus, combined with a proprietary columnar format and a novel indexing algorithm, create a highly scalable and efficient platform for AI workloads.

This analysis will give you insights into the architectural trade-offs involved in building vector search at scale. It also demonstrates how you can adapt a system designed for large-scale analytics so that it excels at modern AI tasks.

Table of Contents

• Prerequisites

• The Unique Challenge of Vector Search

• BigQuery's Foundational Distributed Architecture

  • Dremel: The Distributed Query Engine

  • Borg: Cluster Management and Resource Orchestration

  • Colossus: The Distributed Storage Layer

  • Jupiter: The High-Speed Network Fabric

• The Role of Columnar Storage in Vector Operations

  • Accelerating Computations with SIMD

• The TreeAH Indexing Algorithm

  • 1. Hierarchical Tree Structure

  • 2. Product Quantization (PQ)

  • 3. Asymmetric Hashing

  • Architectural Comparison: TreeAH vs. HNSW

• The End-to-End Vector Search Query Flow

• Practical Implications for Engineering Teams

  • 1. Query Latency vs. Throughput

  • 2. Cost Model Considerations

  • 3. Index Management Trade-offs

  • 4. Integration Benefits That Actually Matter

• Conclusion

• Further Reading

Prerequisites

This article assumes that you have a solid foundation in distributed systems and database internals, including familiarity with concepts like columnar storage, query execution plans, and distributed query processing.

You should understand the basics of vector embeddings and similarity search, though we'll briefly review the fundamentals. Experience with at least one vector database or search system (such as pgvector, Pinecone, or Elasticsearch) will help contextualize the architectural comparisons.

While deep knowledge of Google Cloud Platform isn't required, basic familiarity with cloud data warehouses and their typical architectures will be beneficial. The article includes discussions of SIMD operations and CPU-level optimizations, so comfort with low-level performance considerations is helpful, though not mandatory.

Code examples assume working knowledge of SQL, with some sections referencing implementation details in languages like Python or Java. Most importantly, you should have experience building or operating production data systems at scale, as many insights focus on practical engineering trade-offs rather than theoretical concepts.

The Unique Challenge of Vector Search

Vector search fundamentally differs from traditional database operations in ways that challenge our existing infrastructure assumptions. Where conventional queries leverage decades of optimization around exact matching and range scans, vector similarity search requires computing distances between high-dimensional points at massive scale.

Consider the numbers. Modern embedding models produce vectors with 768 or more dimensions. At 4 bytes per float32 value, a single embedding consumes roughly 3KB. A modest corpus of 100 million items translates to 300GB of vector data.

But the real challenge isn't storage. The killer is computation. Finding the nearest neighbors to a query vector means computing distance metrics across all those dimensions. For 100 million vectors, a brute-force search requires 76.8 billion floating-point operations per query just for the distance calculations. Even with modern SIMD instructions processing 16 floats at once, you're looking at billions of CPU cycles per search.

This computational reality forces a fundamental compromise: we abandon exact solutions for approximate ones. Approximate Nearest Neighbor (ANN) algorithms trade perfect accuracy for practical query times. They work by partitioning the vector space cleverly, building graphs of nearest neighbors, or using hashing schemes to avoid examining every vector. The engineering challenge becomes balancing query latency, recall accuracy, and resource consumption.

Most purpose-built vector databases address this through specialized in-memory indexes like HNSW or IVF. These work well for single queries but require keeping massive indexes in RAM. In case you are not familiar with these vector indexes, you can read this article.

BigQuery took a different path. Rather than optimizing for single-query latency, they asked what vector search would look like when built for analytical workloads at warehouse scale. The answer required rethinking basic assumptions about index design, storage layout, and query execution.

BigQuery's Foundational Distributed Architecture

BigQuery's vector search runs on the same infrastructure that's been processing SQL queries since 2011. No new cluster type. No specialized vector nodes. Just four core technologies that power most of Google's data processing, now handling a workload they weren't originally designed for.

This isn't the obvious choice. Most vector databases build specialized infrastructure optimized for similarity search. Graph-based indexes need fast random access. In-memory systems require careful memory management. BigQuery took its existing distributed SQL engine and asked: can we make this work for vectors, too?

The answer required leveraging four foundational systems in new ways:

• Dremel, the query engine that normally handles SQL, now orchestrates vector similarity computations.

• Borg, which allocates resources for everything from Search to YouTube, dynamically assigns thousands of workers to vector queries.

• Colossus stores embeddings in the same distributed filesystem that holds petabytes of analytics data.

• And Jupiter's datacenter network, built for bulk data processing, now shuttles vector data between computation nodes.

What's surprising isn't that it works, but how well it works. The same architecture that runs aggregate queries over trillion-row tables can search billion-scale vector collections. Understanding how requires examining each component and how they've been adapted for this new workload.

Dremel: The Distributed Query Engine

At its core, BigQuery is powered by Dremel, a distributed query execution engine developed at Google since 2006.

Dremel processes SQL queries using a hierarchical serving tree. A root server receives the query and orchestrates the execution, while mixer nodes break down the work and distribute it to hundreds or thousands of leaf nodes. These leaf nodes perform the actual computations in parallel on segments of the data.

This architecture allows BigQuery to dynamically allocate a massive number of execution threads, known as slots, to a single query, enabling it to process petabytes of data in seconds.

Borg: Cluster Management and Resource Orchestration

The serverless nature of BigQuery is made possible by Borg, Google's cluster management system that predates and inspired Kubernetes.

When a vector search query is submitted, Borg is responsible for finding available machines across Google's global data centers, allocating the precise amount of CPU and memory resources needed for the query's Dremel slots, and managing fault tolerance by automatically rescheduling work if a machine fails. This dynamic resource allocation means users do not need to provision or scale infrastructure, whether they are searching 1,000 vectors or 10 billion.

Colossus: The Distributed Storage Layer

Data in BigQuery is stored in Colossus, Google's next-generation distributed file system. Colossus is designed for exabyte-scale storage, provides high availability through automatic cross-datacenter replication, and is optimized for the high-throughput parallel reads required by Dremel's leaf nodes.

During a vector search, Colossus can deliver data to thousands of nodes simultaneously without creating a storage bottleneck.

Jupiter: The High-Speed Network Fabric

These compute and storage systems are interconnected by Jupiter, Google's internal datacenter network, which features a petabit-per-second bisection bandwidth. The network's design ensures that data can move between Colossus storage and Dremel compute nodes at extremely high speeds, making data shuffling and aggregation phases of a query efficient.

The Role of Columnar Storage in Vector Operations

Storing vectors in columns sounds wrong. Vectors are arrays. They belong together. Why split them across columnar storage?

BigQuery does it anyway, and it works brilliantly. Here's why.

When you search a million vectors, you need exactly one thing from each row: the embedding. Not the product name, price, or category. Just the vector. Row-oriented storage forces you to read entire records and throw away 90% of the data. Columnar storage reads only what you need.

The performance impact is dramatic. A table with 768-dimensional embeddings plus 20 other columns might total 3TB. Reading just the embedding column? 300GB. That's a 10x reduction in I/O before you've done any actual computation.

But the real magic happens at the CPU level. Columnar storage naturally aligns vector data for SIMD processing. Instead of jumping around memory gathering vector components, the CPU finds them laid out sequentially, ready for bulk operations. Modern processors can load 16 floating-point values into a single register and process them simultaneously.

Compression becomes almost trivial, too. BigQuery's Capacitor format applies techniques like Product Quantization directly to the column data, shrinking vectors from 3KB to under 300 bytes. Try doing that with row-oriented storage where vectors are scattered across pages.

The lesson? Sometimes the "wrong" abstraction at one level enables the right optimizations at another.

Accelerating Computations with SIMD

SIMD instructions are a form of hardware-level parallelism available in modern CPUs that provide significant speedups for vector arithmetic. This is achieved through special instruction sets built into the processor.

For example, AVX-512 (Advanced Vector Extensions 512-bit) is an instruction set found in modern high-performance CPUs, such as those from Intel, that allows a single instruction to operate on 512 bits of data at once.

Since a standard single-precision floating-point number is 32 bits, a CPU with AVX-512 can process 16 floating-point numbers in a single operation. This leads to dramatic performance gains.

The difference between scalar and SIMD processing for vector distance calculations is stark:

• Scalar approach: Loop through each dimension, multiply corresponding components, accumulate results. For 768 dimensions, that's 768 multiplications, 768 additions, and terrible cache performance as you jump between two different memory locations for each iteration.

• SIMD approach: Load 16 components from each vector into 512-bit registers. Execute a single multiply instruction that handles all 16 pairs. Execute a single horizontal add. Repeat 48 times. The CPU's pipeline stays full, the cache prefetcher knows exactly what data you need next, and you've turned 1,536 operations into 96.

The columnar storage pays off here, too. Vectors stored contiguously in memory align perfectly with SIMD register loads. No gather operations, no wasted cycles. Just pure throughput.

BigQuery's query engine is designed to leverage SIMD extensively. It automatically detects and uses the optimal instruction set available on the underlying hardware (for example, AVX-512 for Intel, NEON for ARM). The columnar storage format ensures that vector data is laid out in memory in a way that is friendly to SIMD registers, and the engine processes query vectors in large batches to maximize the utilization of these parallel instructions.

The TreeAH Indexing Algorithm

While brute-force search can be effective at smaller scales due to BigQuery's massive parallelism, efficient search over billions of vectors requires an index. BigQuery's primary vector index is TreeAH (Tree with Asymmetric Hashing), which is based on Google's open-sourced ScaNN (Scalable Nearest Neighbors) algorithm. TreeAH combines three techniques to achieve high performance and memory efficiency.

1. Hierarchical Tree Structure

The algorithm first partitions the entire vector space into thousands of smaller lists. You can think of this like organizing a massive library. Instead of having one giant room with a million books, a library has floors, sections, and shelves. This hierarchy allows you to find a book without scanning every single one.

Similarly, TreeAH groups semantically similar vectors together into partitions and arranges them in a tree. During a query, the search navigates this tree by comparing the query vector to "centroid" vectors that represent the center of each partition, effectively following a path to the most relevant partitions and pruning away large, irrelevant branches of the search space.

2. Product Quantization (PQ)

Within TreeAH, PQ serves a different purpose than just compression. The index doesn't just store smaller vectors – it fundamentally changes how distance calculations work.

TreeAH learns partition-specific codebooks that capture the local structure of vectors in each tree node. This means vectors that end up in the "shoes" partition get quantized differently than those in "electronics." The compression becomes semantic-aware.

When combined with the tree structure, this creates a powerful effect: not only are you searching fewer vectors (thanks to the tree), but you're computing distances faster on the vectors you do search (thanks to PQ).

3. Asymmetric Hashing

The "asymmetric" aspect refers to the fact that the query vector is kept in its full-precision form, while the database vectors are compared in their compressed, quantized form.

The vectors are not of different dimensions, but of different precision. The semantic matching works because the comparison is not direct. The compressed database vector is a code that points to a region in the original vector space. The distance calculation uses the full-precision query vector to look up a pre-computed distance to the center of that region. This way, the rich information in the query vector is used to accurately estimate the distance, avoiding the significant information loss that would occur if both vectors were compressed.

Architectural Comparison: TreeAH vs. HNSW

To better understand the design philosophy behind TreeAH, it’s useful to compare it with HNSW (Hierarchical Navigable Small World), a popular graph-based algorithm used in many dedicated vector databases.

HNSW constructs a multi-layered graph where vectors are nodes and edges connect them to their nearest neighbors. It’s known for excellent single-query latency.

But this performance comes with significant memory overhead, as the graph structure must be stored in addition to the full-precision vectors. HNSW index builds can also be time-consuming, and frequent data updates can lead to memory fragmentation and performance degradation.

TreeAH, in contrast, makes different architectural trade-offs that align with BigQuery's nature as a distributed analytics system.

The comparison reveals a fundamental design choice: TreeAH prioritizes batch throughput, memory efficiency, and scalability over absolute single-query latency. This makes it well-suited for analytical workloads where thousands of searches are performed simultaneously.

The End-to-End Vector Search Query Flow

The execution timeline of a BigQuery vector search demonstrates how parallel processing eliminates traditional bottlenecks. When a VECTOR_SEARCH query arrives, the system initiates multiple operations concurrently rather than executing them sequentially.

The root server begins query planning immediately upon receiving the request. In parallel, Borg starts allocating compute slots across the cluster, targeting 1,000 slots distributed across 50 or more nodes. Borg prioritizes slots that are physically close to the data in Colossus to minimize data movement costs. This allocation typically completes within 10 milliseconds.

Query planning and resource allocation overlap significantly. The mixer nodes receive partial execution plans and begin partitioning the search space before Borg completes all slot allocations. When TreeAH indexes are available, mixers use them to assign specific vector partitions to leaf nodes. This streaming approach ensures that leaf nodes receive work assignments as soon as they come online.

The parallel execution phase showcases the architecture's efficiency. Hundreds or thousands of leaf nodes simultaneously read their assigned vector partitions from Colossus. Jupiter's high-bandwidth network prevents I/O congestion even with thousands of concurrent reads. Each leaf node operates independently: loading compressed vectors, executing SIMD operations for distance calculations, and maintaining local top-k results.

Aggregation begins before all leaf nodes complete their local searches. Mixers implement a streaming merge algorithm that processes results as they arrive. This approach means that by the time the slowest leaf node reports its results, the mixers have already processed most of the data. The final global top-k emerges from this continuous merging process.

The measured 40-millisecond execution time represents the longest path through the parallel execution graph, not the sum of individual operations. Most operations complete much faster, but the overall latency is bounded by the slowest component. This design trades single-query latency for massive throughput, enabling BigQuery to process thousands of vector searches concurrently across billions of vectors.

Practical Implications for Engineering Teams

The architectural choices behind BigQuery's vector search create specific trade-offs that engineering teams need to understand before committing to this approach.

1. Query Latency vs. Throughput

BigQuery vector searches typically complete in 1-10 seconds, not the sub-100ms latency of specialized vector databases. But you can run thousands of searches concurrently without degradation. This makes BigQuery ideal for batch recommendation generation, similarity analysis across product catalogs, or embedding-based data enrichment pipelines. It's the wrong choice for autocomplete features or real-time personalization that requires immediate responses.

2. Cost Model Considerations

BigQuery charges for data scanned, not query execution time. A vector search that scans 1TB costs the same whether it completes in 2 seconds or 20 seconds. This model favors workloads where you search large datasets infrequently rather than small datasets continuously. Running vector search on a 10GB table thousands of times per day will be more expensive than a dedicated vector database with fixed infrastructure costs.

3. Index Management Trade-offs

TreeAH indexes update automatically in the background when new data arrives, typically within 5-15 minutes. You cannot force immediate index updates or control index parameters like you can with HNSW or IVF indexes. This simplicity reduces operational overhead but limits optimization options. If your use case requires fine-tuning recall/latency trade-offs or immediate consistency after updates, you'll need a different solution.

4. Integration Benefits That Actually Matter

The ability to JOIN vector search results with business data in a single query is more powerful than it initially appears. Consider this query pattern:

WITH semantic_matches AS (

  SELECT item_id, distance

  FROM VECTOR_SEARCH(

    TABLE products,

    'embedding',

    (SELECT embedding FROM queries WHERE query_id = @query_id)

  )

)

SELECT p.*, s.distance

FROM semantic_matches s

JOIN products p USING (item_id)

WHERE p.in_stock = TRUE

  AND p.price BETWEEN 50 AND 200

  AND p.category_restrictions IS NULL

ORDER BY s.distance

LIMIT 20

This combines semantic search with business logic, inventory status, and access controls in one atomic operation. Implementing this with a separate vector database requires complex synchronization between systems.

Conclusion

BigQuery's vector search implementation challenges our assumptions about what a data warehouse can do. Instead of building another specialized vector database, Google pushed their existing infrastructure to handle a fundamentally different workload.

The key insight is recognizing that vector search at scale is a data processing problem. And processing data at scale is what BigQuery was built for.

By leveraging its columnar architecture and hardware-aware algorithms like TreeAH, BigQuery makes a deliberate trade-off. It exchanges the sub-millisecond latency of in-memory systems for massive batch throughput and incredible resource efficiency. An index that uses 10x less memory than HNSW is a trade-off many teams building analytical AI systems would gladly make.

The real power emerges when vectors live alongside business data. Complex queries that would require multiple systems and synchronization nightmares become simple SQL. "Find similar products, but only from reliable suppliers, in stock locally, with no recent quality issues." One query, one system, no architectural gymnastics.

This approach validates a broader trend: vector capabilities are becoming table stakes for data platforms. The question isn't whether your data platform will support vectors, but how well it integrates them into existing workflows.

For teams building analytical AI applications, BigQuery offers a pragmatic path. It won't win latency benchmarks against dedicated vector databases. But for batch processing, integrated analytics, and operational simplicity at scale, it demonstrates that sometimes the best vector database isn't a vector database at all. It's your data warehouse, evolved.

Further Reading

• BigQuery Under the Hood: Official architecture deep dive

• ScaNN Algorithm Details: The mathematics behind TreeAH

• Dremel: Interactive Analysis of Web-Scale Datasets: The foundational paper

• Large-scale cluster management at Google with Borg: Understanding resource orchestration

• Jupiter Rising: A Decade of Clos Topologies: Google's datacenter networking

• BigQuery Vector Search: A Practitioner's Guide: Optimization strategies

Don’t worry. This isn't likely a flaw in your model. It's a clash between how time-series models and Docker containers are designed to work.

Time-series models are all about memory. They need to remember the past to predict the future. But Docker containers are built to be stateless and forgetful, wiping their memory clean with every restart. This fundamental conflict can turn a powerful model into a worthless one in production.

In this article, we’ll solve that problem. We're going to give your time-series model a permanent memory. You'll learn how to build a production-ready prediction service that uses Redis as an external brain and Docker volumes to ensure that memory survives any restart. We'll walk through a hands-on example, step-by-step, so you can learn how to build a system that is both intelligent and incredibly reliable.

What we’ll cover:

• Who is This Guide For?

• Understanding the Problem

  • So, what is a time-series model?

  • 1. Containers are ephemeral by design

  • 2. Lost context between predictions

  • 3. Model amnesia on restart

• The Solution: External State Store

• Hands-On Implementation

  • Start with the broken approach

  • How to fix it with volumes

  • How the code handles state

  • Test the health endpoint

• What About Scaling?

  • Horizontal scaling with Redis Cluster

  • High availability with Redis Sentinel

  • Use managed Redis services

• Common Pitfalls to Avoid

  • Don't assume volumes work

  • Don't ignore Redis memory limits

  • Don't skip monitoring

• Conclusion

Who is This Guide For?

To get the most out of this tutorial, it’ll be helpful to have a few things under your belt. We’ll be diving into some code and command-line work, so a little preparation will go a long way.

• The main tools for this project are Docker and Docker Compose. Make sure you have them installed and running on your computer.

• You’ll also find it easier to follow along if you’re comfortable with the basics of Docker, Python, and the Flask web framework. A bit of command-line experience will also be handy for running the commands in the tutorial.

• But don't worry if you've never used Redis before. All you need to know is that it’s a fast, in-memory database. We’ll handle the rest along the way.

Think of this as a guided tour. As long as you're curious and have the basic tools ready, you'll be in great shape.

Understanding the Problem

Before jumping into solutions, let's first clarify what a time-series model is and then explore why containerizing it is so tricky.

So, what is a time-series model?

Simply put, a time-series model is a type of model that analyzes data points collected over time to predict future values. Think of it like predicting the weather. A meteorologist doesn't just look at the sky right now. They look at the temperature, pressure, and wind patterns from the last few hours and days to forecast what will happen tomorrow.

Time-series models do the same thing with data, whether it's website traffic, stock prices, or energy consumption. The key takeaway is that history matters. The sequence of past events provides the context needed to make an intelligent prediction about the future.

Now, here’s what breaks when you put these models in Docker.

1. Containers are ephemeral by design

Docker containers are meant to be stateless. This works great for most APIs. A user profile endpoint? Stateless. A sentiment analysis model? Stateless. They take an input, return an output, and forget everything in between.

Time-series models don't work this way. They need context from previous predictions. Without it, your model is essentially blind.

2. Lost context between predictions

Each prediction happens in isolation. Your model receives a single data point and makes a guess without knowing what came before. This defeats the entire purpose of time-series modeling.

You may think: "I'll just load all historical data on every request." But that approach fails for two reasons:

• It's slow. Really slow if you have thousands of data points

• It doesn't scale. When you have multiple series or high request volume, you'll hit performance walls fast

3. Model amnesia on restart

Every time you deploy a new version or the container crashes, all accumulated state disappears. Your model starts from scratch. In production, this is unacceptable.

The Solution: External State Store

Instead of keeping state inside the container, we’ll move it outside. Redis becomes the model's memory.

The pattern looks like this:

Client Request → Flask API → Redis → Prediction with Context

Your container stays stateless and replaceable. But the system as a whole maintains state through Redis.

Hands-On Implementation

Let's build this. Clone the demo repository:

git clone https://github.com/ag-chirag/docker-redis-time-series
cd docker-redis-time-series

Start with the broken approach

The docker-compose.initial.yml file shows what NOT to do:

services:
  api:
    build: ./flask-api
    ports:
      - "5000:5000"

  redis:
    image: redis:alpine

Notice what's missing? No volumes. Redis stores data in the container's filesystem, which means that data is temporary.

Run it:

docker compose -f docker-compose.initial.yml up

Make a few predictions:

curl -X POST http://localhost:5000/predict \
  -H "Content-Type: application/json" \
  -d '{
    "series_id": "demo",
    "historical_data": [
      {"timestamp": "2024-01-01T12:00:00", "value": 10},
      {"timestamp": "2024-01-01T12:01:00", "value": 20},
      {"timestamp": "2024-01-01T12:02:00", "value": 30}
    ]
  }'

You'll get a response showing Redis is working:

{
  "data_points_used": 3,
  "prediction": 40,
  "redis_connected": true
}

Now restart the services:

docker compose down
docker compose -f docker-compose.initial.yml up

Make another prediction. Check the data_points_used field. It reset. All your historical data is gone. This is exactly what we're trying to avoid.

How to fix it with volumes

The correct docker-compose.yml adds persistence:

services:
  api:
    build: ./flask-api
    ports:
      - "5000:5000"
    environment:
      - REDIS_HOST=redis

  redis:
    image: redis:alpine
    command: redis-server --appendonly yes
    volumes:
      - redis_data:/data

volumes:
  redis_data:

So, what is a volume and how does it work?

Think of a Docker volume as a dedicated external hard drive for your container. By default, when a container writes data, it does so to a temporary layer that gets destroyed when the container is removed. A volume provides a way to save that data permanently.

Here’s how it works:

1. Docker creates and manages a special storage area on the host machine, completely separate from any container's filesystem. In our docker-compose.yml, the volumes: redis_data: section at the bottom tells Docker to create a named volume called redis_data.

2. When the Redis container starts, the volumes: - redis_data:/data line tells Docker to "plug in" this external hard drive. It connects the redis_data volume to the /data directory inside the container.

3. Now, whenever the Redis process inside the container writes data to its /data directory (which we've configured it to do), it's actually writing to the redis_data volume on the host machine.

4. When you run docker compose down, the Redis container is destroyed, but the redis_data volume is untouched. It's like unplugging the external hard drive, and the data is still safe. The next time you run docker compose up, a brand new Redis container is created, the volume is re-attached, and Redis finds all its old data right where it left it.

This mechanism is the key to giving our stateful service a memory that survives restarts.

Run the corrected version:

docker compose up --build

Send several predictions to build up state:

for i in {1..5}; do
  curl -X POST http://localhost:5000/predict \
    -H "Content-Type: application/json" \
    -d "{
      \"series_id\": \"demo\",
      \"historical_data\": [{\"timestamp\": \"2024-01-01T12:0$i:00\", \"value\": $((i*10))}]
    }"
done

Now comes the test. Restart everything:

docker compose down
docker compose up

Make another prediction. Look at data_points_used. It includes all previous points. The model picks up exactly where it left off.

This works because the volume exists independently of the container lifecycle.

How the code handles state

The Flask API in flask-api/app.py stores each data point in Redis using sorted sets:

def store_data_point(series_id, timestamp, value):
    key = f"ts:{series_id}"
    redis_client.zadd(key, {json.dumps({"ts": timestamp, "val": value}): timestamp})

When making predictions, it retrieves recent history:

def get_recent_data(series_id, limit=100):
    key = f"ts:{series_id}"
    data = redis_client.zrange(key, -limit, -1)
    return [json.loads(d) for d in data]

Redis sorted sets give you automatic time ordering. The volume ensures this data survives restarts.

Test the health endpoint

Check that everything is connected properly:

curl http://localhost:5000/health

You should see:

{
  "model_loaded": true,
  "redis_connected": true,
  "status": "healthy"
}

If redis_connected is false, check your Docker logs. Common issues are network configuration or Redis not starting properly.

What About Scaling?

This setup works well for single-instance deployments. When traffic increases, you have a few options.

Horizontal scaling with Redis Cluster

For high throughput, distribute your data across multiple Redis nodes. Redis Cluster handles sharding automatically.

High availability with Redis Sentinel

Add failover capability so your state store doesn't become a single point of failure. Sentinel monitors Redis instances and promotes replicas when the primary fails.

Use managed Redis services

AWS ElastiCache, Azure Cache for Redis, or Google Cloud Memorystore handle the operational burden. You focus on your model, they handle Redis reliability.

The key insight: your API containers remain stateless. You scale the state store independently.

Common Pitfalls to Avoid

I can't emphasize this enough: test your persistence before deploying to production.

Don't assume volumes work

Actually restart your containers and verify state persists. I've seen deployments fail because someone forgot to mount the volume in production.

Don't ignore Redis memory limits

Redis keeps everything in memory. Monitor your memory usage. Set maxmemory policies appropriate for your workload. If you run out of memory, Redis will start evicting keys or refuse writes.

Don't skip monitoring

Add health checks. Monitor Redis connection status. Track prediction latency. You want to know when things break, not learn about it from angry users.

Conclusion

Time-series models need memory. Docker containers lose memory by default. The solution is simple: separate state from compute.

Use Redis as an external state store. Use Docker volumes to persist that state. Your model stays smart, your containers stay replaceable, and your deployments become reliable.

The full working code is available at github.com/ag-chirag/docker-redis-time-series. Clone it, run it, break it, learn from it.

And remember: the simplest solution that works is usually the right one. You don't always need Kubernetes and StatefulSets. Sometimes Docker Compose and a volume are enough.