<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/"
    xmlns:atom="http://www.w3.org/2005/Atom" xmlns:media="http://search.yahoo.com/mrss/" version="2.0">
    <channel>
        
        <title>
            <![CDATA[ PersistentVolumes - freeCodeCamp.org ]]>
        </title>
        <description>
            <![CDATA[ Browse thousands of programming tutorials written by experts. Learn Web Development, Data Science, DevOps, Security, and get developer career advice. ]]>
        </description>
        <link>https://www.freecodecamp.org/news/</link>
        <image>
            <url>https://cdn.freecodecamp.org/universal/favicons/favicon.png</url>
            <title>
                <![CDATA[ PersistentVolumes - freeCodeCamp.org ]]>
            </title>
            <link>https://www.freecodecamp.org/news/</link>
        </image>
        <generator>Eleventy</generator>
        <lastBuildDate>Tue, 25 Aug 2026 22:04:14 +0000</lastBuildDate>
        <atom:link href="https://www.freecodecamp.org/news/tag/persistentvolumes/rss.xml" rel="self" type="application/rss+xml" />
        <ttl>60</ttl>
        
            <item>
                <title>
                    <![CDATA[ How to Persist State in Time-Series Models with Docker and Redis ]]>
                </title>
                <description>
                    <![CDATA[ Have you ever built a brilliant time-series model, one that could forecast sales or predict stock prices, only to watch it fail in the real world? Well, this is a common frustration. Your model works perfectly on your machine, but the moment you depl... ]]>
                </description>
                <link>https://www.freecodecamp.org/news/how-to-persist-state-in-time-series-models-with-docker-and-redis/</link>
                <guid isPermaLink="false">68e70d838fa4b92d9a027ebe</guid>
                
                    <category>
                        <![CDATA[ Docker ]]>
                    </category>
                
                    <category>
                        <![CDATA[ Time Series Forecasting ]]>
                    </category>
                
                    <category>
                        <![CDATA[ Redis ]]>
                    </category>
                
                    <category>
                        <![CDATA[ Machine Learning ]]>
                    </category>
                
                    <category>
                        <![CDATA[ PersistentVolumes ]]>
                    </category>
                
                <dc:creator>
                    <![CDATA[ Chirag Agrawal ]]>
                </dc:creator>
                <pubDate>Thu, 09 Oct 2025 01:18:59 +0000</pubDate>
                <media:content url="https://cdn.hashnode.com/res/hashnode/image/upload/v1759972706788/66d45afa-f86b-4365-8a55-8b6873df718b.png" medium="image" />
                <content:encoded>
                    <![CDATA[ <p>Have you ever built a brilliant time-series model, one that could forecast sales or predict stock prices, only to watch it fail in the real world? Well, this is a common frustration. Your model works perfectly on your machine, but the moment you deploy it in a Docker container, it seems to develop amnesia. It forgets everything it knew yesterday, making its predictions for tomorrow useless.</p>
<p>Don’t worry. This isn't likely a flaw in your model. It's a clash between how time-series models and Docker containers are designed to work.</p>
<p>Time-series models are all about memory. They need to remember the past to predict the future. But Docker containers are built to be stateless and forgetful, wiping their memory clean with every restart. This fundamental conflict can turn a powerful model into a worthless one in production.</p>
<p>In this article, we’ll solve that problem. We're going to give your time-series model a permanent memory. You'll learn how to build a production-ready prediction service that uses Redis as an external brain and Docker volumes to ensure that memory survives any restart. We'll walk through a hands-on example, step-by-step, so you can learn how to build a system that is both intelligent and incredibly reliable.</p>
<h3 id="heading-what-well-cover">What we’ll cover:</h3>
<ul>
<li><p><a class="post-section-overview" href="#heading-who-is-this-guide-for">Who is This Guide For?</a></p>
</li>
<li><p><a class="post-section-overview" href="#heading-understanding-the-problem">Understanding the Problem</a></p>
<ul>
<li><p><a class="post-section-overview" href="#heading-so-what-is-a-time-series-model">So, what is a time-series model?</a></p>
</li>
<li><p><a class="post-section-overview" href="#heading-1-containers-are-ephemeral-by-design">1. Containers are ephemeral by design</a></p>
</li>
<li><p><a class="post-section-overview" href="#heading-2-lost-context-between-predictions">2. Lost context between predictions</a></p>
</li>
<li><p><a class="post-section-overview" href="#heading-3-model-amnesia-on-restart">3. Model amnesia on restart</a></p>
</li>
</ul>
</li>
<li><p><a class="post-section-overview" href="#heading-the-solution-external-state-store">The Solution: External State Store</a></p>
</li>
<li><p><a class="post-section-overview" href="#heading-hands-on-implementation">Hands-On Implementation</a></p>
<ul>
<li><p><a class="post-section-overview" href="#heading-start-with-the-broken-approach">Start with the broken approach</a></p>
</li>
<li><p><a class="post-section-overview" href="#heading-how-to-fix-it-with-volumes">How to fix it with volumes</a></p>
</li>
<li><p><a class="post-section-overview" href="#heading-how-the-code-handles-state">How the code handles state</a></p>
</li>
<li><p><a class="post-section-overview" href="#heading-test-the-health-endpoint">Test the health endpoint</a></p>
</li>
</ul>
</li>
<li><p><a class="post-section-overview" href="#heading-what-about-scaling">What About Scaling?</a></p>
<ul>
<li><p><a class="post-section-overview" href="#heading-horizontal-scaling-with-redis-cluster">Horizontal scaling with Redis Cluster</a></p>
</li>
<li><p><a class="post-section-overview" href="#heading-high-availability-with-redis-sentinel">High availability with Redis Sentinel</a></p>
</li>
<li><p><a class="post-section-overview" href="#heading-use-managed-redis-services">Use managed Redis services</a></p>
</li>
</ul>
</li>
<li><p><a class="post-section-overview" href="#heading-common-pitfalls-to-avoid">Common Pitfalls to Avoid</a></p>
<ul>
<li><p><a class="post-section-overview" href="#heading-dont-assume-volumes-work">Don't assume volumes work</a></p>
</li>
<li><p><a class="post-section-overview" href="#heading-dont-ignore-redis-memory-limits">Don't ignore Redis memory limits</a></p>
</li>
<li><p><a class="post-section-overview" href="#heading-dont-skip-monitoring">Don't skip monitoring</a></p>
</li>
</ul>
</li>
<li><p><a class="post-section-overview" href="#heading-conclusion">Conclusion</a></p>
</li>
</ul>
<h2 id="heading-who-is-this-guide-for">Who is This Guide For?</h2>
<p>To get the most out of this tutorial, it’ll be helpful to have a few things under your belt. We’ll be diving into some code and command-line work, so a little preparation will go a long way.</p>
<ul>
<li><p>The main tools for this project are <a target="_blank" href="https://docs.docker.com/get-started/get-docker/">Docker</a> and <a target="_blank" href="https://docs.docker.com/compose/">Docker Compose</a>. Make sure you have them installed and running on your computer.</p>
</li>
<li><p>You’ll also find it easier to follow along if you’re comfortable with the basics of Docker, Python, and the <a target="_blank" href="http://flask.palletsprojects.com/en/stable/quickstart/">Flask</a> web framework. A bit of command-line experience will also be handy for running the commands in the tutorial.</p>
</li>
<li><p>But don't worry if you've never used <a target="_blank" href="https://redis.io/docs/latest/">Redis</a> before. All you need to know is that it’s a fast, in-memory database. We’ll handle the rest along the way.</p>
</li>
</ul>
<p>Think of this as a guided tour. As long as you're curious and have the basic tools ready, you'll be in great shape.</p>
<h2 id="heading-understanding-the-problem">Understanding the Problem</h2>
<p>Before jumping into solutions, let's first clarify what a time-series model is and then explore why containerizing it is so tricky.</p>
<h3 id="heading-so-what-is-a-time-series-model">So, what is a time-series model?</h3>
<p>Simply put, a time-series model is a type of model that analyzes data points collected over time to predict future values. Think of it like predicting the weather. A meteorologist doesn't just look at the sky right now. They look at the temperature, pressure, and wind patterns from the last few hours and days to forecast what will happen tomorrow.</p>
<p>Time-series models do the same thing with data, whether it's website traffic, stock prices, or energy consumption. The key takeaway is that history matters. The sequence of past events provides the context needed to make an intelligent prediction about the future.</p>
<p>Now, here’s what breaks when you put these models in Docker.</p>
<h3 id="heading-1-containers-are-ephemeral-by-design">1. Containers are ephemeral by design</h3>
<p>Docker containers are meant to be stateless. This works great for most APIs. A user profile endpoint? Stateless. A sentiment analysis model? Stateless. They take an input, return an output, and forget everything in between.</p>
<p>Time-series models don't work this way. They need context from previous predictions. Without it, your model is essentially blind.</p>
<h3 id="heading-2-lost-context-between-predictions">2. Lost context between predictions</h3>
<p>Each prediction happens in isolation. Your model receives a single data point and makes a guess without knowing what came before. This defeats the entire purpose of time-series modeling.</p>
<p>You may think: "I'll just load all historical data on every request." But that approach fails for two reasons:</p>
<ul>
<li><p>It's slow. Really slow if you have thousands of data points</p>
</li>
<li><p>It doesn't scale. When you have multiple series or high request volume, you'll hit performance walls fast</p>
</li>
</ul>
<h3 id="heading-3-model-amnesia-on-restart">3. Model amnesia on restart</h3>
<p>Every time you deploy a new version or the container crashes, all accumulated state disappears. Your model starts from scratch. In production, this is unacceptable.</p>
<h2 id="heading-the-solution-external-state-store">The Solution: External State Store</h2>
<p>Instead of keeping state inside the container, we’ll move it outside. Redis becomes the model's memory.</p>
<p>The pattern looks like this:</p>
<pre><code class="lang-plaintext">Client Request → Flask API → Redis → Prediction with Context
</code></pre>
<p>Your container stays stateless and replaceable. But the system as a whole maintains state through Redis.</p>
<h2 id="heading-hands-on-implementation">Hands-On Implementation</h2>
<p>Let's build this. Clone the demo repository:</p>
<pre><code class="lang-bash">git <span class="hljs-built_in">clone</span> https://github.com/ag-chirag/docker-redis-time-series
<span class="hljs-built_in">cd</span> docker-redis-time-series
</code></pre>
<h3 id="heading-start-with-the-broken-approach">Start with the broken approach</h3>
<p>The <code>docker-compose.initial.yml</code> file shows what NOT to do:</p>
<pre><code class="lang-yaml"><span class="hljs-attr">services:</span>
  <span class="hljs-attr">api:</span>
    <span class="hljs-attr">build:</span> <span class="hljs-string">./flask-api</span>
    <span class="hljs-attr">ports:</span>
      <span class="hljs-bullet">-</span> <span class="hljs-string">"5000:5000"</span>

  <span class="hljs-attr">redis:</span>
    <span class="hljs-attr">image:</span> <span class="hljs-string">redis:alpine</span>
</code></pre>
<p>Notice what's missing? No volumes. Redis stores data in the container's filesystem, which means that data is temporary.</p>
<p>Run it:</p>
<pre><code class="lang-bash">docker compose -f docker-compose.initial.yml up
</code></pre>
<p>Make a few predictions:</p>
<pre><code class="lang-bash">curl -X POST http://localhost:5000/predict \
  -H <span class="hljs-string">"Content-Type: application/json"</span> \
  -d <span class="hljs-string">'{
    "series_id": "demo",
    "historical_data": [
      {"timestamp": "2024-01-01T12:00:00", "value": 10},
      {"timestamp": "2024-01-01T12:01:00", "value": 20},
      {"timestamp": "2024-01-01T12:02:00", "value": 30}
    ]
  }'</span>
</code></pre>
<p>You'll get a response showing Redis is working:</p>
<pre><code class="lang-json">{
  <span class="hljs-attr">"data_points_used"</span>: <span class="hljs-number">3</span>,
  <span class="hljs-attr">"prediction"</span>: <span class="hljs-number">40</span>,
  <span class="hljs-attr">"redis_connected"</span>: <span class="hljs-literal">true</span>
}
</code></pre>
<p>Now restart the services:</p>
<pre><code class="lang-bash">docker compose down
docker compose -f docker-compose.initial.yml up
</code></pre>
<p>Make another prediction. Check the <code>data_points_used</code> field. It reset. All your historical data is gone. This is exactly what we're trying to avoid.</p>
<h3 id="heading-how-to-fix-it-with-volumes">How to fix it with volumes</h3>
<p>The correct <code>docker-compose.yml</code> adds persistence:</p>
<pre><code class="lang-yaml"><span class="hljs-attr">services:</span>
  <span class="hljs-attr">api:</span>
    <span class="hljs-attr">build:</span> <span class="hljs-string">./flask-api</span>
    <span class="hljs-attr">ports:</span>
      <span class="hljs-bullet">-</span> <span class="hljs-string">"5000:5000"</span>
    <span class="hljs-attr">environment:</span>
      <span class="hljs-bullet">-</span> <span class="hljs-string">REDIS_HOST=redis</span>

  <span class="hljs-attr">redis:</span>
    <span class="hljs-attr">image:</span> <span class="hljs-string">redis:alpine</span>
    <span class="hljs-attr">command:</span> <span class="hljs-string">redis-server</span> <span class="hljs-string">--appendonly</span> <span class="hljs-literal">yes</span>
    <span class="hljs-attr">volumes:</span>
      <span class="hljs-bullet">-</span> <span class="hljs-string">redis_data:/data</span>

<span class="hljs-attr">volumes:</span>
  <span class="hljs-attr">redis_data:</span>
</code></pre>
<h4 id="heading-so-what-is-a-volume-and-how-does-it-work">So, what is a volume and how does it work?</h4>
<p>Think of a Docker volume as a dedicated external hard drive for your container. By default, when a container writes data, it does so to a temporary layer that gets destroyed when the container is removed. A volume provides a way to save that data permanently.</p>
<p>Here’s how it works:</p>
<ol>
<li><p>Docker creates and manages a special storage area on the host machine, completely separate from any container's filesystem. In our docker-compose.yml, the <code>volumes: redis_data:</code> section at the bottom tells Docker to create a named volume called <code>redis_data</code>.</p>
</li>
<li><p>When the Redis container starts, the <code>volumes: - redis_data:/data</code> line tells Docker to "plug in" this external hard drive. It connects the <code>redis_data</code> volume to the <code>/data</code> directory inside the container.</p>
</li>
<li><p>Now, whenever the Redis process inside the container writes data to its <code>/data</code> directory (which we've configured it to do), it's actually writing to the <code>redis_data</code> volume on the host machine.</p>
</li>
<li><p>When you run docker compose down, the Redis container is destroyed, but the <code>redis_data</code> volume is untouched. It's like unplugging the external hard drive, and the data is still safe. The next time you run docker compose up, a brand new Redis container is created, the volume is re-attached, and Redis finds all its old data right where it left it.</p>
</li>
</ol>
<p>This mechanism is the key to giving our stateful service a memory that survives restarts.</p>
<p>Run the corrected version:</p>
<pre><code class="lang-bash">docker compose up --build
</code></pre>
<p>Send several predictions to build up state:</p>
<pre><code class="lang-bash"><span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> {1..5}; <span class="hljs-keyword">do</span>
  curl -X POST http://localhost:5000/predict \
    -H <span class="hljs-string">"Content-Type: application/json"</span> \
    -d <span class="hljs-string">"{
      \"series_id\": \"demo\",
      \"historical_data\": [{\"timestamp\": \"2024-01-01T12:0<span class="hljs-variable">$i</span>:00\", \"value\": <span class="hljs-subst">$((i*10)</span>)}]
    }"</span>
<span class="hljs-keyword">done</span>
</code></pre>
<p>Now comes the test. Restart everything:</p>
<pre><code class="lang-bash">docker compose down
docker compose up
</code></pre>
<p>Make another prediction. Look at <code>data_points_used</code>. It includes all previous points. The model picks up exactly where it left off.</p>
<p>This works because the volume exists independently of the container lifecycle.</p>
<h3 id="heading-how-the-code-handles-state">How the code handles state</h3>
<p>The Flask API in <code>flask-api/app.py</code> stores each data point in Redis using sorted sets:</p>
<pre><code class="lang-python"><span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">store_data_point</span>(<span class="hljs-params">series_id, timestamp, value</span>):</span>
    key = <span class="hljs-string">f"ts:<span class="hljs-subst">{series_id}</span>"</span>
    redis_client.zadd(key, {json.dumps({<span class="hljs-string">"ts"</span>: timestamp, <span class="hljs-string">"val"</span>: value}): timestamp})
</code></pre>
<p>When making predictions, it retrieves recent history:</p>
<pre><code class="lang-python"><span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_recent_data</span>(<span class="hljs-params">series_id, limit=<span class="hljs-number">100</span></span>):</span>
    key = <span class="hljs-string">f"ts:<span class="hljs-subst">{series_id}</span>"</span>
    data = redis_client.zrange(key, -limit, <span class="hljs-number">-1</span>)
    <span class="hljs-keyword">return</span> [json.loads(d) <span class="hljs-keyword">for</span> d <span class="hljs-keyword">in</span> data]
</code></pre>
<p>Redis sorted sets give you automatic time ordering. The volume ensures this data survives restarts.</p>
<h3 id="heading-test-the-health-endpoint">Test the health endpoint</h3>
<p>Check that everything is connected properly:</p>
<pre><code class="lang-bash">curl http://localhost:5000/health
</code></pre>
<p>You should see:</p>
<pre><code class="lang-json">{
  <span class="hljs-attr">"model_loaded"</span>: <span class="hljs-literal">true</span>,
  <span class="hljs-attr">"redis_connected"</span>: <span class="hljs-literal">true</span>,
  <span class="hljs-attr">"status"</span>: <span class="hljs-string">"healthy"</span>
}
</code></pre>
<p>If <code>redis_connected</code> is false, check your Docker logs. Common issues are network configuration or Redis not starting properly.</p>
<h2 id="heading-what-about-scaling">What About Scaling?</h2>
<p>This setup works well for single-instance deployments. When traffic increases, you have a few options.</p>
<h3 id="heading-horizontal-scaling-with-redis-cluster">Horizontal scaling with Redis Cluster</h3>
<p>For high throughput, distribute your data across multiple Redis nodes. Redis Cluster handles sharding automatically.</p>
<h3 id="heading-high-availability-with-redis-sentinel">High availability with Redis Sentinel</h3>
<p>Add failover capability so your state store doesn't become a single point of failure. Sentinel monitors Redis instances and promotes replicas when the primary fails.</p>
<h3 id="heading-use-managed-redis-services">Use managed Redis services</h3>
<p>AWS ElastiCache, Azure Cache for Redis, or Google Cloud Memorystore handle the operational burden. You focus on your model, they handle Redis reliability.</p>
<p>The key insight: your API containers remain stateless. You scale the state store independently.</p>
<h2 id="heading-common-pitfalls-to-avoid">Common Pitfalls to Avoid</h2>
<p>I can't emphasize this enough: test your persistence before deploying to production.</p>
<h3 id="heading-dont-assume-volumes-work">Don't assume volumes work</h3>
<p>Actually restart your containers and verify state persists. I've seen deployments fail because someone forgot to mount the volume in production.</p>
<h3 id="heading-dont-ignore-redis-memory-limits">Don't ignore Redis memory limits</h3>
<p>Redis keeps everything in memory. Monitor your memory usage. Set maxmemory policies appropriate for your workload. If you run out of memory, Redis will start evicting keys or refuse writes.</p>
<h3 id="heading-dont-skip-monitoring">Don't skip monitoring</h3>
<p>Add health checks. Monitor Redis connection status. Track prediction latency. You want to know when things break, not learn about it from angry users.</p>
<h2 id="heading-conclusion">Conclusion</h2>
<p>Time-series models need memory. Docker containers lose memory by default. The solution is simple: separate state from compute.</p>
<p>Use Redis as an external state store. Use Docker volumes to persist that state. Your model stays smart, your containers stay replaceable, and your deployments become reliable.</p>
<p>The full working code is available at <a target="_blank" href="https://github.com/ag-chirag/docker-redis-time-series">github.com/ag-chirag/docker-redis-time-series</a>. Clone it, run it, break it, learn from it.</p>
<p>And remember: the simplest solution that works is usually the right one. You don't always need Kubernetes and StatefulSets. Sometimes Docker Compose and a volume are enough.</p>
 ]]>
                </content:encoded>
            </item>
        
    </channel>
</rss>
