<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/"
    xmlns:atom="http://www.w3.org/2005/Atom" xmlns:media="http://search.yahoo.com/mrss/" version="2.0">
    <channel>
        
        <title>
            <![CDATA[ cloud-storage - freeCodeCamp.org ]]>
        </title>
        <description>
            <![CDATA[ Browse thousands of programming tutorials written by experts. Learn Web Development, Data Science, DevOps, Security, and get developer career advice. ]]>
        </description>
        <link>https://www.freecodecamp.org/news/</link>
        <image>
            <url>https://cdn.freecodecamp.org/universal/favicons/favicon.png</url>
            <title>
                <![CDATA[ cloud-storage - freeCodeCamp.org ]]>
            </title>
            <link>https://www.freecodecamp.org/news/</link>
        </image>
        <generator>Eleventy</generator>
        <lastBuildDate>Fri, 26 Jun 2026 22:46:55 +0000</lastBuildDate>
        <atom:link href="https://www.freecodecamp.org/news/tag/cloud-storage/rss.xml" rel="self" type="application/rss+xml" />
        <ttl>60</ttl>
        
            <item>
                <title>
                    <![CDATA[ How to Self‑Host an S3‑Compatible Object Store with MinIO on Your Staging Server (and Save Hundreds of Dollars a Month) ]]>
                </title>
                <description>
                    <![CDATA[ This article is a complete copy‑paste guide to running MinIO behind Traefik with HTTPS, custom domains, and pre-signed upload/download URLs — using only Docker Compose. Your production will keep using ]]>
                </description>
                <link>https://www.freecodecamp.org/news/how-to-self-host-an-s3-compatible-object-store-with-minio-on-your-staging-server/</link>
                <guid isPermaLink="false">6a1d99eb2f5663bb4c520a8f</guid>
                
                    <category>
                        <![CDATA[ Devops ]]>
                    </category>
                
                    <category>
                        <![CDATA[ Docker ]]>
                    </category>
                
                    <category>
                        <![CDATA[ cloud-storage ]]>
                    </category>
                
                    <category>
                        <![CDATA[ S3 ]]>
                    </category>
                
                <dc:creator>
                    <![CDATA[ Md Tarikul Islam ]]>
                </dc:creator>
                <pubDate>Mon, 01 Jun 2026 14:40:43 +0000</pubDate>
                <media:content url="https://cdn.hashnode.com/uploads/covers/5e1e335a7a1d3fcc59028c64/a7e1dd1d-2e31-4d80-ae9b-10242588a5e1.png" medium="image" />
                <content:encoded>
                    <![CDATA[ <p>This article is a complete copy‑paste guide to running MinIO behind Traefik with HTTPS, custom domains, and pre-signed upload/download URLs — using only Docker Compose.</p>
<p>Your production will keep using a managed S3 / Cloudflare R2 / Hetzner Object Storage, while every staging upload, download, and pre-signed URL goes to your <strong>own</strong> server for free.</p>
<h2 id="heading-table-of-contents">Table of Contents</h2>
<ul>
<li><p><a href="#heading-1-why-selfhost-object-storage-on-staging">1. Why Self‑Host Object Storage on Staging?</a></p>
</li>
<li><p><a href="#heading-2-the-architecture-production-vs-staging">2. The Architecture: Production vs. Staging</a></p>
</li>
<li><p><a href="#heading-3-prerequisites">3. Prerequisites</a></p>
</li>
<li><p><a href="#heading-4-step-1-dns-point-your-domains-to-the-staging-server">4. Step 1 — DNS: Point Your Domains to the Staging Server</a></p>
</li>
<li><p><a href="#heading-5-step-2-run-minio-with-docker-compose">5. Step 2 — Run MinIO with Docker Compose</a></p>
</li>
<li><p><a href="#heading-6-step-3-expose-minio-over-https-with-traefik">6. Step 3 — Expose MinIO over HTTPS with Traefik</a></p>
</li>
<li><p><a href="#heading-7-step-4-create-the-bucket-and-access-keys">7. Step 4 — Create the Bucket and Access Keys</a></p>
</li>
<li><p><a href="#heading-8-step-5-configure-your-app-to-use-minio-on-staging-only">8. Step 5 — Configure Your App to Use MinIO on Staging Only</a></p>
</li>
<li><p><a href="#heading-9-step-6-upload-files-3-ways">9. Step 6 — Upload Files (3 Ways)</a></p>
</li>
<li><p><a href="#heading-10-step-7-generate-presigned-urls-put-and-get">10. Step 7 — Generate Presigned URLs (PUT and GET)</a></p>
</li>
<li><p><a href="#heading-11-step-8-get-public-urls-for-documents">11. Step 8 — Get Public URLs for Documents</a></p>
</li>
<li><p><a href="#heading-12-step-9-lock-down-cors-lifecycle-and-security">12. Step 9 — Lock Down CORS, Lifecycle, and Security</a></p>
</li>
<li><p><a href="#heading-13-step-10-backups-and-monitoring">13. Step 10 — Backups and Monitoring</a></p>
</li>
<li><p><a href="#heading-14-troubleshooting-cheat-sheet">14. Troubleshooting Cheat Sheet</a></p>
</li>
<li><p><a href="#heading-15-wrapping-up">15. Wrapping Up</a></p>
</li>
</ul>
<h2 id="heading-1-why-selfhost-object-storage-on-staging">1. Why Self‑Host Object Storage on Staging?</h2>
<p>If your app handles documents — PDFs, profile pictures, application transcripts, recordings — every test upload your QA team makes costs real money on AWS S3, Cloudflare R2, or Hetzner Object Storage. The price isn't huge per file, but staging is where you:</p>
<ul>
<li><p>run automated end‑to‑end tests that upload thousands of dummy files,</p>
</li>
<li><p>reset databases nightly (which leaves orphan objects behind),</p>
</li>
<li><p>let developers experiment with broken code that re‑uploads the same files,</p>
</li>
<li><p>and hold months of test data nobody ever deletes.</p>
</li>
</ul>
<p>In production those costs are justified. Managed storage gives you replication, availability, and someone else's pager. In staging, those costs are pure waste.</p>
<p><a href="https://min.io/"><strong>MinIO</strong></a> is a free, open‑source, S3‑compatible object server. Same API, same SDKs, same presigned URLs, same <code>mc</code>/<code>aws s3</code> CLIs — but running on your own VPS, billed at $0 per gigabyte. Point your staging app at MinIO, point your production app at S3/R2, and the only thing that changes is an environment variable.</p>
<p><strong>The result:</strong> identical code paths in both environments, zero storage bill on staging, and a nice fallback if your cloud provider ever has an outage.</p>
<h2 id="heading-2-the-architecture-production-vs-staging">2. The Architecture: Production vs. Staging</h2>
<p>In real-world applications, you usually don’t want your development or staging environment writing directly to production storage.</p>
<p>A common and cost-effective setup is:</p>
<ul>
<li><p><strong>Production</strong>: managed cloud object storage</p>
</li>
<li><p><strong>Staging / Development</strong>: self-hosted S3-compatible storage</p>
</li>
</ul>
<p>The good part is that your application code doesn't need to change.</p>
<p>As long as both services are S3-compatible, the same SDK and upload logic work everywhere. Only the environment variables differ.</p>
<h3 id="heading-high-level-architecture">High-Level Architecture</h3>
<img src="https://cdn.hashnode.com/uploads/covers/66cb39fcaa2a09f9a8d691c1/01ddeefd-8a67-42e3-a3af-9b1d3664bdb2.png" alt="High-level architecture showing a Next.js application uploading files to Cloudflare R2 in production and MinIO in staging through the same S3-compatible API." style="display:block;margin:0 auto" width="426" height="421" loading="lazy">

<p>The above diagram illustrates how the same application can communicate with different storage providers depending on the deployment environment.</p>
<p>In the <strong>production environment</strong>, uploads are stored in a managed object storage service such as AWS S3, Cloudflare R2, or Hetzner Object Storage. These services handle durability, scalability, backups, and infrastructure management.</p>
<p>In the <strong>staging environment</strong>, uploads are directed to a self-hosted MinIO instance running inside Docker on a VPS. MinIO implements the S3 API, making it behave similarly to production storage while keeping costs low.</p>
<p>Because both storage systems are S3-compatible, the application uses the same upload logic in every environment. The only difference is the configuration provided through environment variables.</p>
<h3 id="heading-why-this-architecture-is-useful">Why This Architecture Is Useful</h3>
<p>This setup gives you:</p>
<ul>
<li><p>A cheap staging environment</p>
</li>
<li><p>Production-like testing</p>
</li>
<li><p>Zero storage vendor lock-in</p>
</li>
<li><p>The ability to switch providers without rewriting application code</p>
</li>
</ul>
<p>Because both environments speak the S3 protocol, your upload logic remains identical.</p>
<h3 id="heading-example-environment-variables">Example Environment Variables</h3>
<p>Your application only reads environment variables like these:</p>
<pre><code class="language-xml">S3_ENDPOINT=
S3_REGION=
S3_ACCESS_KEY=
S3_SECRET_KEY=
S3_BUCKET=
</code></pre>
<p>Switch the values, and the exact same application now uploads files to a different backend.</p>
<h3 id="heading-production-storage-example">Production Storage Example</h3>
<p>In production, you typically use managed object storage providers such as:</p>
<ul>
<li><p>AWS S3</p>
</li>
<li><p>Cloudflare R2</p>
</li>
<li><p>Hetzner Object Storage</p>
</li>
</ul>
<p>Example:</p>
<pre><code class="language-plaintext">S3_ENDPOINT=https://&lt;region&gt;.r2.cloudflarestorage.com
</code></pre>
<p>The benefits are that it's highly scalable, globally available, durable, has managed backups, and doesn't have infrastructure maintenance.</p>
<h3 id="heading-staging-environment-example">Staging Environment Example</h3>
<p>For staging, a lightweight self-hosted MinIO container is often enough.</p>
<pre><code class="language-plaintext">Next.js App
     ↓
MinIO Container (inside Docker on VPS)
</code></pre>
<p>Example domains:</p>
<table>
<thead>
<tr>
<th>Service</th>
<th>Domain</th>
<th>Internal Port</th>
</tr>
</thead>
<tbody><tr>
<td>MinIO S3 API</td>
<td><a href="http://minio-staging.domain.com"><code>minio-staging.domain.com</code></a></td>
<td><code>9000</code></td>
</tr>
<tr>
<td>MinIO Web Console</td>
<td><a href="http://minio-console-staging.domain.com"><code>minio-console-staging.domain.com</code></a></td>
<td><code>9001</code></td>
</tr>
</tbody></table>
<p>This allows you to:</p>
<ul>
<li><p>Test uploads safely</p>
</li>
<li><p>Avoid production storage costs</p>
</li>
<li><p>Reproduce production-like behavior locally</p>
</li>
</ul>
<h2 id="heading-3-prerequisites">3. Prerequisites</h2>
<p>You'll need:</p>
<ul>
<li><p>A Linux VPS (Hetzner, DigitalOcean, Contabo, OVH — anything with a public IP).</p>
</li>
<li><p>Two A records pointing at that IP (we'll register them next).</p>
</li>
<li><p>Docker + Docker Compose v2.</p>
</li>
<li><p><a href="https://traefik.io/">Traefik</a> v2 in front, with Let's Encrypt configured (any reverse proxy works&nbsp;– the labels below are Traefik's flavor).</p>
</li>
<li><p>Open ports <code>80</code> and <code>443</code> on the firewall for Let's Encrypt + HTTPS.</p>
</li>
<li><p>~10 GB free disk for the MinIO data volume to start.</p>
</li>
</ul>
<p>If Docker isn't installed:</p>
<pre><code class="language-bash">curl -fsSL https://get.docker.com | sh
sudo apt-get install -y docker-compose-plugin
docker --version &amp;&amp; docker compose version
</code></pre>
<h2 id="heading-4-step-1-dns-point-your-domains-to-the-staging-server">4. Step 1 — DNS: Point Your Domains to the Staging Server</h2>
<p>In your DNS provider (Cloudflare, Route 53, Namecheap, and so on), create two <strong>A records</strong> pointing at your staging server's public IP:</p>
<pre><code class="language-plaintext">minio-staging.domain.com           A    203.0.113.45
minio-console-staging.domain.com   A    203.0.113.45
</code></pre>
<p>If you use Cloudflare, set the proxy status to <strong>DNS only</strong> (gray cloud) for <code>minio-staging.*</code>. Cloudflare's free plan caps uploads at 100 MB, and you don't want it stripping S3 signing headers. The console subdomain can stay proxied if you want a WAF in front of it.</p>
<p>Wait a minute and verify:</p>
<pre><code class="language-bash">dig +short minio-staging.domain.com
# 203.0.113.45
</code></pre>
<h2 id="heading-5-step-2-run-minio-with-docker-compose">5. Step 2 — Run MinIO with Docker Compose</h2>
<p>Add this service to your staging compose file (<code>docker-compose.staging.yml</code>). MinIO is just one container — the disk is mounted as a Docker volume so data survives upgrades.</p>
<pre><code class="language-yaml"># docker-compose.staging.yml
networks:
  proxy:
    external: true
    name: proxy
  internal:
    name: internal

volumes:
  minio-data:

services:
  minio:
    image: minio/minio:latest
    container_name: minio-staging
    restart: unless-stopped
    environment:
      - MINIO_ROOT_USER=${MINIO_ROOT_USER:-admin}
      - MINIO_ROOT_PASSWORD=${MINIO_ROOT_PASSWORD:-change-me-please}
      # Tell MinIO which public domain to sign URLs with
      - MINIO_SERVER_URL=https://minio-staging.domain.com
      - MINIO_BROWSER_REDIRECT_URL=https://minio-console-staging.domain.com
    command: server /data --console-address ":9001"
    volumes:
      - minio-data:/data
    networks:
      - proxy
      - internal
    ports:
      - "9000:9000"  # S3 API
      - "9001:9001"  # Web console
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 30s
</code></pre>
<p>Two things deserve attention:</p>
<ul>
<li><p><code>MINIO_SERVER_URL</code> is the secret sauce. Without it, MinIO signs presigned URLs using its internal hostname (<code>http://minio:9000</code>), which then fails verification when the browser hits the public domain. Set it to the exact HTTPS URL clients will use.</p>
</li>
<li><p><code>MINIO_BROWSER_REDIRECT_URL</code> does the same for the web console (login redirects, OIDC callbacks, and so on).</p>
</li>
</ul>
<p>Bring it up:</p>
<pre><code class="language-bash">docker compose -f docker-compose.staging.yml up -d minio
docker compose -f docker-compose.staging.yml logs -f minio
</code></pre>
<p>You should see <code>API: http://...</code> and <code>Console: http://...</code> lines.</p>
<h2 id="heading-6-step-3-expose-minio-over-https-with-traefik">6. Step 3 — Expose MinIO over HTTPS with Traefik</h2>
<p>We don't expose ports <code>9000</code>/<code>9001</code> to the world directly — Traefik does that for us, terminating TLS with a free Let's Encrypt certificate.</p>
<p>Add these labels to the <code>minio</code> service:</p>
<pre><code class="language-yaml">    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=proxy"

      # ---- S3 API (port 9000) ----
      - "traefik.http.routers.minio-staging.rule=Host(`minio-staging.domain.com`)"
      - "traefik.http.routers.minio-staging.entrypoints=websecure"
      - "traefik.http.routers.minio-staging.tls.certresolver=letsencrypt"
      - "traefik.http.routers.minio-staging.service=minio-staging"
      - "traefik.http.services.minio-staging.loadbalancer.server.port=9000"

      # ---- Web Console (port 9001) ----
      - "traefik.http.routers.minio-console-staging.rule=Host(`minio-console-staging.domain.com`)"
      - "traefik.http.routers.minio-console-staging.entrypoints=websecure"
      - "traefik.http.routers.minio-console-staging.tls.certresolver=letsencrypt"
      - "traefik.http.routers.minio-console-staging.service=minio-console-staging"
      - "traefik.http.services.minio-console-staging.loadbalancer.server.port=9001"
</code></pre>
<p>You also need an <code>entrypoint</code> for <code>:443</code> and a <code>certificatesresolver</code> named <code>letsencrypt</code>. Here's the minimum Traefik config (<code>traefik.staging.yml</code>):</p>
<pre><code class="language-yaml">api:
  dashboard: true

entryPoints:
  web:
    address: ":80"
  websecure:
    address: ":443"

certificatesResolvers:
  letsencrypt:
    acme:
      httpChallenge:
        entryPoint: web
      email: admin@domain.com
      storage: /etc/traefik/acme.json

providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false
    network: proxy
</code></pre>
<p>Restart and watch the cert get issued:</p>
<pre><code class="language-bash">docker compose -f docker-compose.staging.yml up -d
docker compose -f docker-compose.staging.yml logs -f traefik | grep -i acme
</code></pre>
<p>Sanity check from your laptop:</p>
<pre><code class="language-bash">curl -I https://minio-staging.domain.com/minio/health/live
# HTTP/2 200
</code></pre>
<p>You can now log in to the <strong>web console</strong> at <code>https://minio-console-staging.domain.com</code> with <code>admin</code> / <code>change-me-please</code>.</p>
<p><strong>Important upload size tweak:</strong> if you're behind Cloudflare or NGINX in front of Traefik, raise the request body limit. Traefik itself has no default limit, but Cloudflare's free plan refuses anything over 100 MB. For self‑hosted edge proxies, set <code>client_max_body_size 0;</code> (NGINX) or the equivalent.</p>
<h2 id="heading-7-step-4-create-the-bucket-and-access-keys">7. Step 4 — Create the Bucket and Access Keys</h2>
<p>Anything that speaks S3 can talk to MinIO. The easiest tool is <code>mc</code> (the official MinIO client), shipped inside the same image.</p>
<h3 id="heading-71-connect-mc-to-your-server">7.1 Connect mc to your server</h3>
<pre><code class="language-bash">docker exec -it minio-staging \
  mc alias set local http://localhost:9000 admin change-me-please
</code></pre>
<h3 id="heading-72-create-a-bucket">7.2 Create a bucket</h3>
<pre><code class="language-bash">docker exec -it minio-staging mc mb local/domain-files-staging
</code></pre>
<h3 id="heading-73-choose-a-bucket-policy">7.3 Choose a bucket policy</h3>
<p>You have three choices, so just pick based on what you store:</p>
<table>
<thead>
<tr>
<th>Policy</th>
<th>When to use</th>
</tr>
</thead>
<tbody><tr>
<td><code>private</code> (default)</td>
<td>Anything sensitive — student transcripts, contracts, internal docs. Reads only via presigned URL.</td>
</tr>
<tr>
<td><code>download</code></td>
<td>Public read, no listing. Good for CDN‑style assets like avatars.</td>
</tr>
<tr>
<td><code>public</code></td>
<td>Anyone can read AND list. Use only for truly public content.</td>
</tr>
</tbody></table>
<p>Set one:</p>
<pre><code class="language-bash"># Private (recommended for documents)
docker exec -it minio-staging \
  mc anonymous set none local/domain-files-staging

# OR public read for static assets only:
docker exec -it minio-staging \
  mc anonymous set download local/domain-files-staging
</code></pre>
<h3 id="heading-74-create-a-dedicated-app-user-dont-use-root-keys">7.4 Create a dedicated app user (don't use root keys!)</h3>
<p>The <code>admin</code> account can wipe everything. Make a least‑privilege user for your app:</p>
<pre><code class="language-bash">docker exec -it minio-staging mc admin user add local \
  domain-app a-long-random-secret-key

# Attach the built-in read/write policy, scoped to one bucket via JSON:
cat &gt; /tmp/policy.json &lt;&lt;'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:*"],
      "Resource": [
        "arn:aws:s3:::domain-files-staging",
        "arn:aws:s3:::domain-files-staging/*"
      ]
    }
  ]
}
EOF

docker cp /tmp/policy.json minio-staging:/tmp/policy.json
docker exec -it minio-staging \
  mc admin policy create local domain-rw /tmp/policy.json
docker exec -it minio-staging \
  mc admin policy attach local domain-rw --user domain-app
</code></pre>
<p>Save those two values — they are your <code>S3_ACCESS_KEY</code> and <code>S3_SECRET_KEY</code>.</p>
<h2 id="heading-8-step-5-configure-your-app-to-use-minio-on-staging-only">8. Step 5 — Configure Your App to Use MinIO on Staging Only</h2>
<p>The trick to "MinIO in staging, real S3 in prod" is to use the <strong>same S3 client</strong> in your code and only swap the env vars.</p>
<p>Your <code>staging.env</code> (loaded by your staging compose stack):</p>
<pre><code class="language-env"># ---- Staging: self-hosted MinIO ----
STORAGE_ENABLED=true
S3_ENDPOINT=https://minio-staging.domain.com
S3_PUBLIC_ENDPOINT=https://minio-staging.domain.com
S3_BUCKET=domain-files-staging
S3_ACCESS_KEY=domain-app
S3_SECRET_KEY=a-long-random-secret-key
S3_REGION=us-east-1
S3_FORCE_PATH_STYLE=true
</code></pre>
<p>Your <code>production.env</code>:</p>
<pre><code class="language-env"># ---- Production: Cloudflare R2 ----
STORAGE_ENABLED=true
S3_ENDPOINT=https://&lt;account-id&gt;.r2.cloudflarestorage.com
S3_PUBLIC_ENDPOINT=https://files.domain.com
S3_BUCKET=domain-files
S3_ACCESS_KEY=&lt;r2-access-key&gt;
S3_SECRET_KEY=&lt;r2-secret-key&gt;
S3_REGION=auto
S3_FORCE_PATH_STYLE=true
</code></pre>
<p><code>S3_FORCE_PATH_STYLE=true</code> is critical for both MinIO <strong>and</strong> R2/Hetzner. Without it, the SDK tries <code>https://bucket.minio-staging.domain.com</code> (virtual‑host style), which won't resolve.</p>
<p>Now in your application code (Node.js example using AWS SDK v3):</p>
<pre><code class="language-javascript">// src/lib/s3.js
import { S3Client } from "@aws-sdk/client-s3";

export const s3 = new S3Client({
  endpoint: process.env.S3_ENDPOINT,
  region: process.env.S3_REGION,
  credentials: {
    accessKeyId: process.env.S3_ACCESS_KEY,
    secretAccessKey: process.env.S3_SECRET_KEY,
  },
  forcePathStyle: process.env.S3_FORCE_PATH_STYLE === "true",
});

export const BUCKET = process.env.S3_BUCKET;
export const PUBLIC_ENDPOINT = process.env.S3_PUBLIC_ENDPOINT;
</code></pre>
<p>The same <code>s3</code> instance now talks to MinIO on staging and to R2 in production with no code change.</p>
<h2 id="heading-9-step-6-upload-files-3-ways">9. Step 6 — Upload Files (3 Ways)</h2>
<h3 id="heading-91-from-a-server-best-for-trusted-backends">9.1 From a server (best for trusted backends)</h3>
<pre><code class="language-javascript">import { PutObjectCommand } from "@aws-sdk/client-s3";
import { s3, BUCKET } from "./lib/s3.js";
import { readFile } from "node:fs/promises";

export async function uploadDocument(localPath, key, contentType) {
  const Body = await readFile(localPath);
  await s3.send(new PutObjectCommand({
    Bucket: BUCKET,
    Key: key,
    Body,
    ContentType: contentType,
    // Optional: per-object metadata, useful for audits
    Metadata: { uploadedBy: "system", env: process.env.NODE_ENV },
  }));
  return key;
}
</code></pre>
<h3 id="heading-92-with-the-mc-cli-good-for-oneoff-uploads-migrations">9.2 With the mc CLI (good for one‑off uploads / migrations)</h3>
<pre><code class="language-bash">mc alias set staging https://minio-staging.domain.com domain-app a-long-random-secret-key
mc cp ./report.pdf staging/domain-files-staging/reports/2026/report.pdf
mc ls staging/domain-files-staging --recursive
</code></pre>
<h3 id="heading-93-directly-from-the-browser-via-a-presigned-put-url">9.3 Directly from the browser via a presigned PUT URL</h3>
<p>The recommended pattern for user uploads is: the file goes from the browser to MinIO with <strong>zero</strong> bytes touching your API server.</p>
<p>We'll cover this in detail next.</p>
<h2 id="heading-10-step-7-generate-presigned-urls-put-and-get">10. Step 7 — Generate Presigned URLs (PUT and GET)</h2>
<p>A <strong>presigned URL</strong> is a regular HTTPS URL with a time‑limited signature in the query string. Anyone with the URL can do exactly the action it was signed for (PUT this object, or GET that object) for the next N minutes — and nothing else.</p>
<p>This is what makes "users upload directly to storage" safe.</p>
<h3 id="heading-101-presigned-put-for-uploads">10.1 Presigned PUT (for uploads)</h3>
<pre><code class="language-javascript">// src/lib/presign.js
import { PutObjectCommand, GetObjectCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import { s3, BUCKET } from "./s3.js";
import { randomUUID } from "node:crypto";

export async function presignUpload({ filename, contentType, userId }) {
  const key = `users/\({userId}/\){randomUUID()}-${filename}`;
  const cmd = new PutObjectCommand({
    Bucket: BUCKET,
    Key: key,
    ContentType: contentType,
  });
  const uploadUrl = await getSignedUrl(s3, cmd, { expiresIn: 60 * 5 }); // 5 min
  return { uploadUrl, key };
}
</code></pre>
<p>Wire it to your API:</p>
<pre><code class="language-javascript">// POST /api/uploads/presign
app.post("/api/uploads/presign", requireAuth, async (req, res) =&gt; {
  const { filename, contentType } = req.body;
  const result = await presignUpload({
    filename,
    contentType,
    userId: req.user.id,
  });
  res.json(result); // { uploadUrl, key }
});
</code></pre>
<p>The browser uploads straight to MinIO:</p>
<pre><code class="language-javascript">// In your frontend
async function uploadFile(file) {
  const { uploadUrl, key } = await fetch("/api/uploads/presign", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ filename: file.name, contentType: file.type }),
  }).then(r =&gt; r.json());

  await fetch(uploadUrl, {
    method: "PUT",
    headers: { "Content-Type": file.type },
    body: file,
  });

  // Persist `key` in your DB so you can retrieve it later
  await fetch("/api/documents", {
    method: "POST",
    body: JSON.stringify({ key, originalName: file.name }),
  });
}
</code></pre>
<p>The <code>Content-Type</code> you send during PUT <strong>must match</strong> the one you signed with, or MinIO will reject the request with <code>SignatureDoesNotMatch</code>. This catches everyone the first time.</p>
<h3 id="heading-102-presigned-get-for-downloads">10.2 Presigned GET (for downloads)</h3>
<p>Same idea, but with <code>GetObjectCommand</code>:</p>
<pre><code class="language-javascript">export async function presignDownload(key, expiresIn = 60 * 10) {
  const cmd = new GetObjectCommand({ Bucket: BUCKET, Key: key });
  return getSignedUrl(s3, cmd, { expiresIn });
}
</code></pre>
<p>A typical "view document" endpoint:</p>
<pre><code class="language-javascript">app.get("/api/documents/:id/url", requireAuth, async (req, res) =&gt; {
  const doc = await db.documents.findById(req.params.id);
  if (!doc || !canUserSee(req.user, doc)) return res.sendStatus(403);
  const url = await presignDownload(doc.key, 600);
  res.json({ url });
});
</code></pre>
<p>The frontend just opens that URL — the file streams from MinIO directly to the user.</p>
<h3 id="heading-103-why-presigned-urls-beat-proxy-through-the-api">10.3 Why presigned URLs beat "proxy through the API"</h3>
<table>
<thead>
<tr>
<th></th>
<th>Proxy through API</th>
<th>Presigned URL</th>
</tr>
</thead>
<tbody><tr>
<td>Bytes through your app</td>
<td>All of them</td>
<td>Zero</td>
</tr>
<tr>
<td>API CPU/RAM cost</td>
<td>High</td>
<td>None</td>
</tr>
<tr>
<td>Throughput limit</td>
<td>Your API</td>
<td>MinIO's NIC</td>
</tr>
<tr>
<td>Auth check</td>
<td>Your code</td>
<td>Your code (still — check before signing)</td>
</tr>
</tbody></table>
<h2 id="heading-11-step-8-get-public-urls-for-documents">11. Step 8 — Get Public URLs for Documents</h2>
<p>Sometimes you want a permanent, unauthenticated URL — for example public profile pictures.</p>
<p>If the bucket policy allows anonymous reads (<code>mc anonymous set download …</code>), the public URL pattern is:</p>
<pre><code class="language-plaintext">https://minio-staging.domain.com/&lt;bucket&gt;/&lt;key&gt;
</code></pre>
<p>So <code>users/42/avatar.png</code> becomes:</p>
<pre><code class="language-plaintext">https://minio-staging.domain.com/domain-files-staging/users/42/avatar.png
</code></pre>
<p>In code:</p>
<pre><code class="language-javascript">export function publicUrl(key) {
  return `\({process.env.S3_PUBLIC_ENDPOINT}/\){BUCKET}/${key}`;
}
</code></pre>
<p>For <strong>private</strong> buckets (most documents), don't use public URLs at all — always go through <code>presignDownload(key)</code> so you can re‑check authorization on every request and expire links.</p>
<h2 id="heading-12-step-9-lock-down-cors-lifecycle-and-security">12. Step 9 — Lock Down CORS, Lifecycle, and Security</h2>
<h3 id="heading-121-allow-your-frontend-origins-cors">12.1 Allow your frontend origins (CORS)</h3>
<p>Browser uploads need CORS rules on the bucket. Drop this JSON via <code>mc</code>:</p>
<pre><code class="language-bash">cat &gt; /tmp/cors.json &lt;&lt;'EOF'
{
  "CORSRules": [
    {
      "AllowedOrigins": [
        "https://crm-staging.domain.com",
        "http://localhost:3000"
      ],
      "AllowedMethods": ["GET", "PUT", "POST", "HEAD"],
      "AllowedHeaders": ["*"],
      "ExposeHeaders": ["ETag"],
      "MaxAgeSeconds": 3000
    }
  ]
}
EOF

docker cp /tmp/cors.json minio-staging:/tmp/cors.json
docker exec -it minio-staging \
  mc cors set local/domain-files-staging /tmp/cors.json
</code></pre>
<h3 id="heading-122-autodelete-old-test-files-lifecycle">12.2 Auto‑delete old test files (lifecycle)</h3>
<p>Staging accumulates junk. Tell MinIO to expire anything older than 30 days:</p>
<pre><code class="language-bash">docker exec -it minio-staging \
  mc ilm rule add --expire-days 30 local/domain-files-staging
</code></pre>
<h3 id="heading-123-encrypt-at-rest">12.3 Encrypt at rest</h3>
<pre><code class="language-bash">docker exec -it minio-staging \
  mc encrypt set sse-s3 local/domain-files-staging
</code></pre>
<h3 id="heading-124-hard-rules">12.4 Hard rules</h3>
<ul>
<li><p><strong>Never</strong> ship <code>MINIO_ROOT_USER=admin</code> / <code>MINIO_ROOT_PASSWORD=admin123</code> to a server reachable from the internet. Generate strong values and store them in your secret manager.</p>
</li>
<li><p>The root account should be used only by <code>mc admin</code>, never by your app. The app uses a scoped IAM user (Step 7.4).</p>
</li>
<li><p>Keep the <strong>console</strong> subdomain behind an IP allow‑list or basic auth via Traefik middleware if it's truly public.</p>
</li>
<li><p>Rotate the app access keys at least every 90 days.</p>
</li>
</ul>
<h2 id="heading-13-step-10-backups-and-monitoring">13. Step 10 — Backups and Monitoring</h2>
<h3 id="heading-131-backups-mirror-to-a-cheap-cold-bucket-weekly">13.1 Backups: mirror to a cheap cold bucket weekly</h3>
<p>Set up a tiny cron job that uses <code>mc mirror</code> to push to Backblaze B2, R2, or another cheap S3 endpoint:</p>
<pre><code class="language-bash">mc alias set b2 https://s3.us-east-005.backblazeb2.com \(B2_KEY \)B2_SECRET
mc mirror --overwrite --remove \
  staging/domain-files-staging \
  b2/domain-staging-backup
</code></pre>
<p>Even at $6/TB/month this is essentially free for staging volumes.</p>
<h3 id="heading-132-monitoring-with-prometheus">13.2 Monitoring with Prometheus</h3>
<p>MinIO exposes Prometheus metrics out of the box at <code>/minio/v2/metrics/cluster</code>. Scrape with:</p>
<pre><code class="language-yaml">scrape_configs:
  - job_name: minio
    metrics_path: /minio/v2/metrics/cluster
    scheme: https
    static_configs:
      - targets: ["minio-staging.domain.com"]
</code></pre>
<p>If you have Grafana, import dashboard ID <strong>13502</strong> for an instant overview (capacity, request rates, latency, error counts).</p>
<h2 id="heading-14-troubleshooting-cheat-sheet">14. Troubleshooting Cheat Sheet</h2>
<table>
<thead>
<tr>
<th>Symptom</th>
<th>Likely cause</th>
<th>Fix</th>
</tr>
</thead>
<tbody><tr>
<td><code>SignatureDoesNotMatch</code> on presigned PUT</td>
<td>Browser sent a different <code>Content-Type</code> than what was signed</td>
<td>Send the exact same <code>Content-Type</code> header during PUT</td>
</tr>
<tr>
<td>Presigned URL works locally but not in browser</td>
<td><code>MINIO_SERVER_URL</code> not set, so URLs are signed for <code>minio:9000</code></td>
<td>Set <code>MINIO_SERVER_URL=https://minio-staging.domain.com</code> and restart</td>
</tr>
<tr>
<td><code>403 SignatureDoesNotMatch</code> after going through Cloudflare</td>
<td>Cloudflare strips/modifies headers</td>
<td>Set the DNS record to <strong>DNS‑only</strong> (gray cloud)</td>
</tr>
<tr>
<td><code>NoSuchBucket</code></td>
<td>App pointing at the wrong endpoint or bucket</td>
<td>Re‑check <code>S3_ENDPOINT</code> and <code>S3_BUCKET</code> in env</td>
</tr>
<tr>
<td>Browser CORS preflight fails</td>
<td>No CORS rule on the bucket</td>
<td>Apply the CORS JSON from §12.1</td>
</tr>
<tr>
<td>Upload works for small files, fails at 100 MB</td>
<td>Cloudflare free plan body limit</td>
<td>Use Cloudflare paid plan, or skip CF proxy</td>
</tr>
<tr>
<td><code>x509: certificate signed by unknown authority</code> from your app</td>
<td>App container doesn't trust Let's Encrypt</td>
<td>Update CA bundle (<code>apt install ca-certificates</code>) or use HTTP inside the Docker network</td>
</tr>
<tr>
<td>Web console redirects to <code>http://minio:9001/login</code></td>
<td><code>MINIO_BROWSER_REDIRECT_URL</code> missing</td>
<td>Set it to <code>https://minio-console-staging.domain.com</code></td>
</tr>
</tbody></table>
<p>Useful diagnostics:</p>
<pre><code class="language-bash"># Check MinIO health
curl -I https://minio-staging.domain.com/minio/health/live

# List all objects in a bucket
docker exec -it minio-staging mc ls --recursive local/domain-files-staging

# Tail MinIO logs
docker compose -f docker-compose.staging.yml logs -f minio

# Decode a presigned URL to see what it was signed for
echo "&lt;paste url&gt;" | tr '&amp;' '\n'
</code></pre>
<h2 id="heading-15-wrapping-up">15. Wrapping Up</h2>
<p>Here's what you have now:</p>
<ul>
<li><p>A free, S3‑compatible object store running on your own staging server.</p>
</li>
<li><p>Real HTTPS on a real domain (<code>https://minio-staging.domain.com</code>), thanks to Traefik + Let's Encrypt.</p>
</li>
<li><p>A scoped, least‑privilege application user — root keys stay locked away.</p>
</li>
<li><p>The same exact code paths in staging and production. Switching between MinIO / R2 / Hetzner / AWS S3 is a four‑variable change in the env file.</p>
</li>
<li><p>Presigned PUT URLs so users upload straight to storage, bypassing your API.</p>
</li>
<li><p>Presigned GET URLs so private documents are short‑lived and authorization‑gated.</p>
</li>
<li><p>Lifecycle rules that nuke old test files automatically.</p>
</li>
<li><p>Optional weekly mirror to a cold backup bucket.</p>
</li>
</ul>
<p>Production keeps running on managed storage where the SLA matters. Staging now costs you exactly <strong>$0 per month per gigabyte uploaded</strong> — and you can finally stop telling QA to "delete the test files when you're done."</p>
<h3 id="heading-further-reading">Further Reading</h3>
<ul>
<li><p><a href="https://min.io/docs/minio/container/index.html">MinIO Documentation</a></p>
</li>
<li><p><a href="https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-s3-request-presigner/">AWS SDK v3 — <code>getSignedUrl</code></a></p>
</li>
<li><p><a href="https://doc.traefik.io/traefik/providers/docker/">Traefik v2 Docker provider</a></p>
</li>
<li><p><a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-policies.html">S3 bucket policy reference</a></p>
</li>
</ul>
<p>If this guide saved your team a few dollars, share it with another team that's still uploading test PDFs to a $90/month S3 bucket. Happy shipping.</p>
 ]]>
                </content:encoded>
            </item>
        
    </channel>
</rss>
