<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/"
    xmlns:atom="http://www.w3.org/2005/Atom" xmlns:media="http://search.yahoo.com/mrss/" version="2.0">
    <channel>
        
        <title>
            <![CDATA[ cronjob - freeCodeCamp.org ]]>
        </title>
        <description>
            <![CDATA[ Browse thousands of programming tutorials written by experts. Learn Web Development, Data Science, DevOps, Security, and get developer career advice. ]]>
        </description>
        <link>https://www.freecodecamp.org/news/</link>
        <image>
            <url>https://cdn.freecodecamp.org/universal/favicons/favicon.png</url>
            <title>
                <![CDATA[ cronjob - freeCodeCamp.org ]]>
            </title>
            <link>https://www.freecodecamp.org/news/</link>
        </image>
        <generator>Eleventy</generator>
        <lastBuildDate>Sun, 26 Jul 2026 16:37:19 +0000</lastBuildDate>
        <atom:link href="https://www.freecodecamp.org/news/tag/cronjob/rss.xml" rel="self" type="application/rss+xml" />
        <ttl>60</ttl>
        
            <item>
                <title>
                    <![CDATA[ What to Do if You've Outgrown Your Cron Job Scheduler ]]>
                </title>
                <description>
                    <![CDATA[ Most developers begin their automation journey similarly. They create a script that performs a helpful task, such as pulling data from an API, resizing a batch of images, or emailing a report, and the ]]>
                </description>
                <link>https://www.freecodecamp.org/news/what-to-do-if-you-ve-outgrown-your-cron-job-scheduler/</link>
                <guid isPermaLink="false">6a63cd4ce956973f287e1685</guid>
                
                    <category>
                        <![CDATA[ Workflow Automation ]]>
                    </category>
                
                    <category>
                        <![CDATA[ Orchestration ]]>
                    </category>
                
                    <category>
                        <![CDATA[ orchestration-platform ]]>
                    </category>
                
                    <category>
                        <![CDATA[ Python automation ]]>
                    </category>
                
                    <category>
                        <![CDATA[ it automation tools ]]>
                    </category>
                
                    <category>
                        <![CDATA[ cron ]]>
                    </category>
                
                    <category>
                        <![CDATA[ cronjob ]]>
                    </category>
                
                    <category>
                        <![CDATA[ Cron Job Scheduling ]]>
                    </category>
                
                    <category>
                        <![CDATA[ AI-automation ]]>
                    </category>
                
                <dc:creator>
                    <![CDATA[ Rob Walters ]]>
                </dc:creator>
                <pubDate>Fri, 24 Jul 2026 20:38:36 +0000</pubDate>
                <media:content url="https://cdn.hashnode.com/uploads/covers/5e1e335a7a1d3fcc59028c64/af95f928-4ab0-4e7d-99c9-fae3fb328a83.png" medium="image" />
                <content:encoded>
                    <![CDATA[ <p>Most developers begin their automation journey similarly. They create a script that performs a helpful task, such as pulling data from an API, resizing a batch of images, or emailing a report, and then schedule it to run each morning.</p>
<p>This leads them to add a line to their crontab, which gives them a sense of control. Now, their computer runs the script automatically while they sleep.</p>
<p>For a while, that's enough. Then it isn't.</p>
<p>Maybe the backup script failed silently at 3 a.m., and you didn't find out until you needed it later that day, or you developed the perfect script that ran smoothly in your terminal to find it failing when scheduled in cron.</p>
<p>If any of that sounds familiar, congratulations: you've outgrown cron. You're not alone, and this article aims to help you feel understood and ready for better solutions.</p>
<p>This article is about what comes next. We'll look at exactly where cron runs out of road, what 'workflow orchestration' actually means underneath the buzzword, and then build a real workflow step by step so the concepts stick.</p>
<p>By the end, you'll feel more confident and empowered to choose the right tools for complex scheduling challenges.</p>
<h2 id="heading-what-cron-is-and-what-its-genuinely-good-at">What Cron is, and What it's Genuinely Good at</h2>
<p><strong>Cron</strong> is the system background service (daemon) that runs scheduled tasks. Crontab (cron table) is the configuration file or command utility used to write and manage those task schedules' time-based job scheduler that has shipped with Unix-like systems since the 1970s. It's available on today’s Linux distributions as well as on Macs.</p>
<p>With cron, you specify a schedule and a command, and it runs the command at that time. The schedule uses the famous five-field syntax:</p>
<pre><code class="language-plaintext">┌───────────── minute (0–59)
│ ┌───────────── hour (0–23)
│ │ ┌───────────── day of month (1–31)
│ │ │ ┌───────────── month (1–12)
│ │ │ │ ┌───────────── day of week (0–6)
│ │ │ │ │
0  9  *  1 5  /usr/bin/python3 /home/me/daily_report.py
</code></pre>
<p>Note: The line <code>0 9 * 1 5&nbsp; /usr/bin/python3 /home/me/daily_report.py</code> means "run daily_report.py at 9:00 a.m. on weekdays." The syntax is terse, ubiquitous, and (credit where it's due) rock-solid for what it does.</p>
<p>And here's the important part: <strong>cron is not bad.</strong> For a single, self-contained, failure-tolerant task on one machine, it's the right tool. But for more complex workflows, a purpose-built orchestration tool can provide the reliability and visibility you need, helping you feel more in control of your automation.</p>
<h2 id="heading-the-four-walls-youll-hit-with-cron"><strong>The Four Walls You'll Hit with Cron</strong></h2>
<p>The trouble with cron starts when your automation stops being a single self-contained task. As your scripts grow, they form more complex systems where you run into the same four limitations, more or less in this order.</p>
<h3 id="heading-wall-1-dependencies-between-steps">Wall 1: Dependencies Between Steps</h3>
<p>Your morning routine grows from one script into three. Consider an ETL scenario with three scripts:</p>
<ol>
<li><p><strong>extract.py</strong> pulls yesterday's orders from an API.</p>
</li>
<li><p><strong>transform.py</strong> cleans the data and computes totals.</p>
</li>
<li><p><strong>load.py</strong> writes the result into the analytics database.</p>
</li>
</ol>
<p>Each step depends on the one before it. The obvious cron approach is to guess at the timing:</p>
<pre><code class="language-plaintext">0 2 * * *  python extract.py
0 3 * * *  python transform.py
0 4 * * *  python load.py
</code></pre>
<p>You're now hoping the extract finishes within an hour, so that the transform has something to work with. On the day the API is slow, and extract takes 70 minutes, transform runs against stale or missing data and quietly produces garbage. Cron has no concept of "run B only after A succeeds." It only knows wall-clock time.</p>
<h3 id="heading-wall-2-failure-handling-and-retries">Wall 2: Failure Handling and Retries</h3>
<p>Networks blip. APIs return 503 status codes (Service Unavailable). Databases drop connections. A robust job needs to detect a failure and retry. Maybe three times, or with increasing delays between attempts, so you don't hammer a struggling service.</p>
<p>With cron, retry logic is your problem. You end up bolting it onto every script by hand: try/except blocks, sleep calls, a counter, and a flag file so the next cron tick knows whether the previous one finished. Multiply that across a dozen jobs, and you've written a small, buggy, undocumented orchestration engine.</p>
<h3 id="heading-wall-3-visibility">Wall 3: Visibility</h3>
<p>Ask yourself, did your jobs run last night? Which ones succeeded? How long did each take? Is the slowdown in the extract step or the load step?</p>
<p>With cron, the honest answer is "I'd have to go read some log files, if the script even wrote any." The first task is finding the cron job log files themselves.</p>
<p>Finding them can be complicated: cron's logs can be in /var/log/syslog on one system and /var/log/cron on another, assuming logging is enabled. Meanwhile, your script's output only exists if you explicitly redirect stdout and stderr. As a result, you have to search through system logs to verify the job ran, then locate any separate output files that captured its print statements.</p>
<p>When a job fails, identifying the issue is even harder, as you must sift through interleaved logs from multiple runs, trying to determine which timestamp corresponds to the last execution and where it went wrong, often relying only on a non-zero exit code as a clue. There's no dashboard, no run history, no record of how long things took, and no alert when something breaks.</p>
<p>Failures are silent by default, the single most dangerous property a background job can have. You find out your pipeline has been broken for a week when someone downstream notices the numbers stopped updating.</p>
<h3 id="heading-wall-4-backfills-and-re-runs">Wall 4: Backfills and Re-runs</h3>
<p>Your analytics database has been live for two months when you discover a bug in <strong>transform.py</strong> that miscalculated totals. You've fixed the code. Now you need to rerun the pipeline for each day in those two months, processing its own slice of data.</p>
<p>This is a backfill, and with cron, it's a nightmare. Cron only ever runs "now." There's no built-in notion of "run this job as if it were March 14th, then March 15th, then..." So you write yet another throwaway script with a date loop, pray it's idempotent, and babysit it.</p>
<p>Notice the pattern across all four walls: each time, you end up reimplementing something poorly that a category of tools already solves well. That category is workflow orchestration.</p>
<h2 id="heading-can-workflow-orchestration-save-the-day"><strong>Can Workflow Orchestration Save the Day?</strong></h2>
<p>A workflow orchestrator manages workflows, which are sets of tasks with specific relationships, triggers, and failure protocols, along with observability for tracking outcomes.</p>
<p>While many solutions like Airflow, Dagster, Prefect, and Temporal are available, we'll focus on <a href="https://kestra.io">Kestra</a>, an open-source orchestrator. Kestra stands out from the pack by enabling developers to run, monitor, and manage workflows all from a single declarative layer compatible with any programming language and infrastructure, including public, private, or even air-gapped networks.</p>
<p>Kestra also provides enterprises with the control needed to ensure insights and regulatory compliance. With over 1,600 connectors available, you can build almost any data, infrastructure, or AI workflow you can imagine.</p>
<p>In this tutorial, we'll continue with the cron scheduler theme and build a simple ETL workflow that uses a cron-like schedule while addressing some of cron's shortcomings, such as execution sequence and error handling.</p>
<h2 id="heading-kestra-primer">Kestra Primer</h2>
<p>Kestra was born out of the engineering pain of creating Python code to achieve proper workflow orchestration.&nbsp;Rather than spend engineering hours crafting the right Python code for your Apache Airflow DAG, with Kestra, your workflow is a YAML file. Simply describe what should run in plain, declarative syntax. Store the file in Git, and deploy it like any other code.</p>
<p>There's no obscure UI logic building and no hidden state management. Just simple text that can be easily reviewed and diffed.&nbsp;The workflow you read is the workflow that runs.</p>
<p>The example YAML file in Figure 1 illustrates a scenario in which you want to move NoSQL data into an analytics-ready warehouse.</p>
<pre><code class="language-yaml">id: cassandra-to-bigquery
namespace: company.team

tasks:
  - id: query_cassandra
    type: io.kestra.plugin.cassandra.Query
    session:
      endpoints:
        - hostname: localhost
          port: 9042
      localDatacenter: datacenter1
    cql: |
      SELECT salary_id, work_year, experience_level, employment_type,
      job_title, salary, salary_currency, salary_in_usd, employee_residence,
      remote_ratio, company_location, company_size
      FROM test.salary
    fetchType: STORE

  - id: write_to_csv
    type: io.kestra.plugin.serdes.csv.IonToCsv
    from: "{{ outputs.query_cassandra.uri }}"

  - id: load_bigquery
    type: io.kestra.plugin.gcp.bigquery.Load
    from: "{{ outputs.write_to_csv.uri }}"
    destinationTable: my_project.my_dataset.my_table
    serviceAccount: "{{ secret('GCP_SERVICE_ACCOUNT_JSON') }}"
    projectId: my_project
    format: CSV
    csvOptions:
      fieldDelimiter: ","
      skipLeadingRows: 1
</code></pre>
<p>Figure 1: Cassandra to BigQuery Example</p>
<p>Without even knowing much about Kestra, the YAML file is simple and self-explanatory. Later in this post, we'll create a basic ETL flow and explain the significance of fields such as id and type. But first, let’s get an instance of Kestra running on your local machine.</p>
<h3 id="heading-how-to-set-up-kestra">How to Set Up Kestra</h3>
<p>Kestra is available as an open-source platform licensed under the Apache 2.0 license as well as an Enterprise offering <a href="https://kestra.io/docs/enterprise">additional features</a> and product support. In this tutorial, we'll use Docker to run Kestra locally using the latest version.</p>
<p>To spin up Kestra, run the following Docker command:</p>
<pre><code class="language-shell">docker run --pull=always --rm -it -p 8080:8080 
  --user=root \
  --name kestra \
  -v kestra_data:/app/storage \
  -v kestra_db:/app/data \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /tmp:/tmp \
  kestra/kestra:latest server local
</code></pre>
<p>For other platforms such as Windows and Linux check out <a href="https://kestra.io/get-started">https://kestra.io/get-started</a>.</p>
<p>Once the containers are loaded, navigate to the Kestra UI at <a href="http://localhost:8080">http://localhost:8080</a>. The welcome screen will ask you to create an administrator.&nbsp;Create the user and finish the initial launch wizard.</p>
<p>Once that's complete, navigate to the Flows tab in the left panel, then click the Create button at the top right of the page. This will create a new flow using a sample template, as shown in the following figure:</p>
<img src="https://cdn.hashnode.com/uploads/covers/67c9ccd99fc59c702c3edd73/0ed945f2-ae0b-4f4f-b274-a4f61d3fc255.png" alt="Kestra Flows showing Flow Code Editor" style="display:block;margin:0 auto" width="1734" height="814" loading="lazy">

<p>Figure 2: Flows page showing the new flow template</p>
<h3 id="heading-flows-in-kestra">Flows in Kestra</h3>
<p>In Kestra you define your workflow orchestration through Flows. You can create these Flows using YAML syntax in the UI, through a no-code editor in the UI, or programmatically through an API. In this tutorial, we'll create flows using YAML.</p>
<p>In Figure 2 above, you'll notice that a sample flow is already created to get you started.</p>
<p>The <strong>_id</strong>, <strong>namespace,</strong> and <strong>tasks</strong> are three required fields and are used to identify the flow within the Kestra environment and the task the flow should execute. Each flow lives in one namespace. Namespaces are like folders in a filesystem and are used to group flows and provide structure. Note that you can't change a flow’s namespace after creation.</p>
<h2 id="heading-how-to-use-kestra">How to Use Kestra</h2>
<h3 id="heading-step-1-a-simple-task-executed-on-a-schedule">Step 1: A Simple Task Executed on a Schedule</h3>
<p>Let's start by erasing the sample flow provided and replacing it with the following:</p>
<pre><code class="language-yaml">id: morning_report
namespace: tutorial

tasks:
  - id: say_hello
    type: io.kestra.plugin.core.log.Log
    message: "Good morning — the pipeline ran at {{ execution.startDate }}"
</code></pre>
<p>This example has the required id, namespace, and has one tasks field that logs a message. The message displayed uses a <a href="https://kestra.io/docs/expressions">pebble expression</a> (the command inside the&nbsp; {{ }} brackets) to show the startDate of the execution.</p>
<p>Pebble expressions are used to set values dynamically in flows. In this example, the start date of the execution will be inserted within the string.</p>
<p>To execute this flow, we need to first save it by clicking the Save button in the upper-right corner of the page. Once saved, click the Play button. You'll then be presented with the execute options page, as shown in Figure 3 below:</p>
<img src="https://cdn.hashnode.com/uploads/covers/67c9ccd99fc59c702c3edd73/9e31e532-f41b-4921-916f-e803c7bcc900.png" alt="Execute Flow input dialog" style="display:block;margin:0 auto" width="1058" height="490" loading="lazy">

<p>Figure 3: Execute flow options</p>
<p>If this workflow had inputs such as a filename or URL, you can manually enter them here to test your flow. The modal also provides the curl command if you’d like to run the flow via the API instead of the UI.&nbsp;Click the Execute button to run the flow:</p>
<img src="https://cdn.hashnode.com/uploads/covers/67c9ccd99fc59c702c3edd73/a7a2e91c-13f3-4c82-b704-3f2812348a09.png" alt="Flow execution log dialog" style="display:block;margin:0 auto" width="1592" height="940" loading="lazy">

<p>Figure 4: Flow execution log</p>
<p>The flow was simple and wrote the info message to the log file. If the flow had errors or warnings, you'd be able to see detailed execution logs on this page.</p>
<p>Now that we’ve created our first task, let’s schedule it with a cron mask. Add the following to the flow by clicking on the “Edit Flow” button at the top of the page.</p>
<p>Next, add the triggers section to the flow:</p>
<pre><code class="language-yaml">triggers:
  - id: every_minute
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "* * * * *"
</code></pre>
<p>Execute the flow.&nbsp; After a few minutes, check out the execution history by clicking on the “Execution” tab on the left side navbar.</p>
<img src="https://cdn.hashnode.com/uploads/covers/67c9ccd99fc59c702c3edd73/e421c8b1-5aac-4524-934a-f0cc70cb5f8a.png" alt="Flow execution page" style="display:block;margin:0 auto" width="2048" height="359" loading="lazy">

<p>Figure 5: Flow execution page</p>
<p>Our flow now behaves similarly to a single task cron job. The flow has a triggers block with a Schedule trigger whose cron field is the exact same five-field syntax you already know. That last point matters: you're not throwing away what you learned. You're wrapping it in something that can grow as your needs change.</p>
<p>Notice that even in this simple Kestra example, you’ve gained something cron didn't offer. Every time this runs, it's recorded as an execution with a timestamp, a duration, and a status, and all logs are visible in the UI. That's Wall 3 (visibility) handled before we've even done anything interesting.</p>
<h3 id="heading-step-2-real-work-and-a-dependency">Step 2: Real Work, and a Dependency</h3>
<p>Now let's replace the simple task with the three-step extract/transform/load and have the orchestrator enforce the ordering rather than timing offsets.</p>
<pre><code class="language-yaml">id: csv_to_parquet
namespace: company.team
description: Download orders CSV, transform it with a Python script, and write the result to a Parquet file.

tasks:
  # Download a public CSV file into Kestra's internal storage
  - id: download_csv
    type: io.kestra.plugin.core.http.Download
    uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv

  # Transform the CSV with a simple Python script and write it out as Parquet
  - id: transform_to_parquet
    type: io.kestra.plugin.scripts.python.Script
    containerImage: ghcr.io/kestra-io/pydata:latest
    inputFiles:
      input.csv: "{{ outputs.download_csv.uri }}"
    outputFiles:
      - orders.parquet
    script: |
      import pandas as pd

      # Read the downloaded CSV
      df = pd.read_csv("input.csv")

      # --- simple transformation ---
      # Ensure numeric types and add a computed column
      df["total"] = df["quantity"] * df["price"]

      # Keep only orders above a small threshold as an example filter
      df = df[df["total"] &gt; 0]

      print(f"Rows after transform: {len(df)}")

      # Write the result to Parquet
      df.to_parquet("orders.parquet", index=False)

  # Log that the Parquet file was produced
  - id: log_output
    type: io.kestra.plugin.core.log.Log
    message: "Parquet file created: {{ outputs.transform_to_parquet.outputFiles['orders.parquet'] }}"
# Expose the Parquet file as a downloadable flow output.
# FILE-typed flow outputs appear on the execution's Overview tab with a download button.
outputs:
  - id: parquet_file
    type: FILE
    value: "{{ outputs.transform_to_parquet.outputFiles['orders.parquet'] }}"
</code></pre>
<p>Save the flow, then execute.</p>
<img src="https://cdn.hashnode.com/uploads/covers/67c9ccd99fc59c702c3edd73/cd4e0202-4769-4e83-bf1d-c1dec91bae8e.png" alt="Execution results" style="display:block;margin:0 auto" width="2048" height="928" loading="lazy">

<p>Figure 6: Execution results</p>
<p>Two things just happened. First, tasks listed in sequence run in sequence. Transform_to_parquet only starts after download_csv succeeds, and log_output only after transform_to_parquet succeeds.&nbsp;If a failure occurs, the execution stops, and transform_to_parquet never touches stale data. That's <strong>Wall 1 (dependencies)</strong> gone, with no guesswork about timing.</p>
<p>Second, notice the expression within the flow {{ outputs.download_csv.uri }}. Tasks can pass data and metadata to subsequent tasks using expressions like this. That wiring turns a list of scripts into an actual pipeline.</p>
<h3 id="heading-step-3-surviving-failure-with-retries">Step 3: Surviving Failure with Retries</h3>
<p>Now consider the scenario where the download_csv task encounters a network issue, and the flow is unable to download the latest data. Let's make that task resilient declaratively by adding a retry section to the download_csv task:</p>
<pre><code class="language-yaml">- id: download_csv
    type: io.kestra.plugin.core.http.Download
    uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv
    retry:
      type: constant
      maxAttempts: 5
      interval: PT10S
</code></pre>
<p>That's the whole retry policy. If the download fails, Kestra waits and tries again up to 5 times, with a 10-second delay (PT10S is ISO-8601 for "10 seconds"). There are no counters, sleep calls, or flag files. <strong>Wall 2 (failure handling)</strong> is handled in 3 lines that read like a sentence.</p>
<p>To test this, drop the “s” from orders.csv and rerun the flow. You can see the execution showing retrying.</p>
<img src="https://cdn.hashnode.com/uploads/covers/67c9ccd99fc59c702c3edd73/4ab1a70c-9ada-4c99-aec0-5f5ccc48975a.png" alt="Execution dialog showing retry" style="display:block;margin:0 auto" width="2048" height="552" loading="lazy">

<p>Figure 7: Execution showing retry</p>
<p>If all attempts fail, you'll probably want to be notified. Let’s add a flow-level error handler that runs only when something in the workflow fails:</p>
<pre><code class="language-yaml">errors:
  - id: notify_failure
    type: io.kestra.plugin.notifications.slack.SlackIncomingWebhook
    url: "{{ secret('SLACK_WEBHOOK') }}"
    payload: |
      {"text": "orders_pipeline failed on execution {{ execution.id }}"}
</code></pre>
<p>Now a broken pipeline pings a Slack channel instead of failing silently at 3 a.m. Note that secrets are protected within flows via the <a href="https://kestra.io/docs/how-to-guides/secrets">secret</a> expression.</p>
<h3 id="heading-step-4-triggering-on-events-not-just-time">Step 4: Triggering on Events, Not Just Time</h3>
<p>Schedules are only one kind of trigger. Suppose orders don't arrive on a fixed timetable. Instead, a file lands in cloud storage whenever an upstream system feels like it.</p>
<p>Polling on a cron schedule ("check every 5 minutes, exit if nothing's there") is wasteful and laggy. Event triggers are the better model: run the workflow when the thing happens.</p>
<p>Conceptually, instead of adding a scheduled trigger similar to the one added in the previous example:</p>
<pre><code class="language-yaml">triggers:
  - id: every_minute
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "* * * * *"
</code></pre>
<p>You can add a trigger that fires when a new object appears in an S3 bucket:</p>
<pre><code class="language-yaml">triggers:
  - id: new_s3_object
    type: io.kestra.plugin.aws.s3.Trigger
    interval: "PT1M"
    accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
    secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
    region: "eu-central-1"
    bucket: "my-bucket"
    prefix: "incoming/"
    on: CREATE
    action: NONE
</code></pre>
<p>Alternatively, you could have a trigger that exposes a webhook URL you can POST to start execution, or <a href="https://kestra.io/docs/how-to-guides/realtime-triggers">real-time triggers</a> that listen to streaming services such as Kafka queues. In any of these trigger scenarios, the workflow body stays identical. Your automation can now respond to the world rather than just watching the clock.</p>
<h3 id="heading-step-5-backfilling-the-past">Step 5: Backfilling the Past</h3>
<p>Finally, consider the bug-in-transform scenario. You've fixed the calculation and need to re-run the pipeline for every day of the last two months.&nbsp;This would be painful in cron.</p>
<p>In an orchestrator, a <a href="https://kestra.io/docs/concepts/backfill">backfill</a> is a first-class operation on a scheduled workflow: you pick a start and end date, and it generates one execution per scheduled interval across that range. Each execution is aware of the date it represents, through an expression like {{ trigger.date }}. Your transform step can use that date to fetch and process the correct slice of data.</p>
<p>This is where idempotency stops being academic. Because a backfill re-runs days you may have already processed, your data load step should write with an "insert or replace" semantic keyed on the date, so running March 14th, for example, twice leaves the database in the same state as running it once.</p>
<p>Design for that and backfills become routine instead of terrifying. <strong>Wall 4 (backfills)</strong> is handled, but only if you've held up your end with idempotent tasks.</p>
<h2 id="heading-where-to-go-next"><strong>Where to Go Next</strong></h2>
<p>The best way to internalize all of this is to take an existing cron job and rebuild it as a proper workflow. Start with the one-task version, confirm it runs and appears in the run history, then add a second dependent task, a retry policy, and a failure alert.</p>
<p>Each step maps to one of the four walls, and you'll feel the difference immediately the first time a task fails, retries itself, and recovers without waking you up.</p>
<p>You don't need to write every line from scratch. Kestra provides a library of <strong>Blueprints</strong> that are hundreds of ready-made, copy-pasteable flows that you can browse at <a href="https://kestra.io/blueprints">kestra.io/blueprints</a> or access directly in your instance under the Blueprints tab.</p>
<p>Each Blueprint is a complete, executable example with an explanation of its functionality and how to extend it. This allows you to start with a close version of your goal and modify it rather than guessing at the syntax.</p>
<p>Cron showed that computers can operate while you're asleep. Orchestration takes it further, ensuring they do so reliably, in order, and visibly. This shift transforms the idea from merely writing a script to managing a full system.</p>
 ]]>
                </content:encoded>
            </item>
        
            <item>
                <title>
                    <![CDATA[ How to Schedule Jobs in PostgreSQL with pg_cron  ]]>
                </title>
                <description>
                    <![CDATA[ Every backend system eventually needs something to run on a schedule. Old sessions need deleting, summary tables need rebuilding, materialized views need refreshing, and maintenance tasks need to happ ]]>
                </description>
                <link>https://www.freecodecamp.org/news/how-to-schedule-jobs-in-postgresql-with-pg-cron/</link>
                <guid isPermaLink="false">6a32f343a137d9657aba1b9e</guid>
                
                    <category>
                        <![CDATA[ PostgreSQL ]]>
                    </category>
                
                    <category>
                        <![CDATA[ cronjob ]]>
                    </category>
                
                    <category>
                        <![CDATA[ jobs ]]>
                    </category>
                
                <dc:creator>
                    <![CDATA[ iyiola ]]>
                </dc:creator>
                <pubDate>Wed, 17 Jun 2026 19:19:31 +0000</pubDate>
                <media:content url="https://cdn.hashnode.com/uploads/covers/5e1e335a7a1d3fcc59028c64/4f2b2004-5710-43db-9db9-0603bf757d9b.png" medium="image" />
                <content:encoded>
                    <![CDATA[ <p>Every backend system eventually needs something to run on a schedule. Old sessions need deleting, summary tables need rebuilding, materialized views need refreshing, and maintenance tasks need to happen while everyone is asleep.</p>
<p>The usual answer is to reach outside the database: a system crontab, a Kubernetes CronJob, a Celery beat worker, or a scheduler service. All of these work, but they add moving parts. Now you have credentials to manage, a separate process to monitor, and one more thing that can silently stop running.</p>
<p>pg_cron takes a different approach. It's a PostgreSQL extension that runs a cron-style scheduler <em>inside</em> the database itself. You schedule jobs with plain SQL, the database executes them, and the run history lands in a table you can query like anything else.</p>
<p>In this tutorial, you'll learn how pg_cron works, how to install and configure it, and how to use it for real maintenance tasks. You'll also learn how to monitor jobs, manage permissions, and decide when pg_cron is the right tool — and when it isn't.</p>
<h2 id="heading-table-of-contents">Table of Contents</h2>
<ul>
<li><p><a href="#heading-prerequisites">Prerequisites</a></p>
</li>
<li><p><a href="#heading-what-is-pgcron">What Is pg_cron?</a></p>
</li>
<li><p><a href="#heading-how-pgcron-works">How pg_cron Works</a></p>
</li>
<li><p><a href="#heading-how-to-install-and-set-up-pgcron">How to Install and Set Up pg_cron</a></p>
</li>
<li><p><a href="#heading-a-quick-refresher-on-cron-syntax">A Quick Refresher on Cron Syntax</a></p>
</li>
<li><p><a href="#heading-how-to-schedule-your-first-job">How to Schedule Your First Job</a></p>
</li>
<li><p><a href="#heading-practical-pgcron-examples">Practical pg_cron Examples</a></p>
</li>
<li><p><a href="#heading-how-to-view-and-monitor-your-jobs">How to View and Monitor Your Jobs</a></p>
</li>
<li><p><a href="#heading-how-to-update-and-remove-jobs">How to Update and Remove Jobs</a></p>
</li>
<li><p><a href="#heading-how-to-run-jobs-in-other-databases">How to Run Jobs in Other Databases</a></p>
</li>
<li><p><a href="#heading-how-to-let-other-users-schedule-jobs">How to Let Other Users Schedule Jobs</a></p>
</li>
<li><p><a href="#heading-when-to-use-pgcron-and-when-to-avoid-it">When to Use pg_cron (and When to Avoid It)</a></p>
</li>
<li><p><a href="#heading-best-practices-for-working-with-pgcron">Best Practices for Working with pg_cron</a></p>
</li>
<li><p><a href="#heading-conclusion">Conclusion</a></p>
</li>
</ul>
<h2 id="heading-prerequisites">Prerequisites</h2>
<p>To follow along with the examples, you'll need:</p>
<ul>
<li><p>Basic knowledge of SQL (SELECT, INSERT, UPDATE, DELETE)</p>
</li>
<li><p>A running PostgreSQL instance (version 13 or later is ideal, though pg_cron supports version 10 and up)</p>
</li>
<li><p>Superuser or admin access to that instance, since installing the extension requires it</p>
</li>
<li><p>A SQL client like <code>psql</code>, pgAdmin, or DBeaver</p>
</li>
</ul>
<p>If you don't run your own server, that's fine too. Most managed PostgreSQL services — including Amazon RDS, Azure Database for PostgreSQL, Google Cloud SQL, Supabase, and Neon — support pg_cron. I'll cover how to enable it on those later in the tutorial.</p>
<h2 id="heading-what-is-pgcron">What Is pg_cron?</h2>
<p>pg_cron is an open source PostgreSQL extension, originally built by the team at Citus Data, that lets you schedule SQL commands using the familiar cron syntax.</p>
<p>Instead of writing a crontab entry on a server, you write a SQL statement:</p>
<pre><code class="language-sql">SELECT cron.schedule(
  'nightly-cleanup',
  '0 3 * * *',
  $$DELETE FROM sessions WHERE expires_at &lt; now()$$
);
</code></pre>
<p>That single statement tells PostgreSQL to delete expired sessions every day at 3 AM. No external process, no shell script, no extra credentials. The job definition lives in the database, version-controlled alongside your migrations if you want it to be.</p>
<p>Because the scheduler is just another extension, your jobs travel with the database. Anyone who can connect and query can see exactly what's scheduled, when it last ran, and whether it succeeded.</p>
<h2 id="heading-how-pgcron-works">How pg_cron Works</h2>
<p>When PostgreSQL starts with pg_cron enabled, the extension launches a background worker. This worker has one job: watch the <code>cron.job</code> table, which holds every scheduled job along with its schedule, command, target database, and the user it runs as.</p>
<p>When a job's scheduled time arrives, the worker executes the command. By default it does this by opening a new local connection to the database, just as your application would. You can also configure it to use PostgreSQL background workers instead of connections, which I'll show you in the setup section.</p>
<p>Two behaviors are worth knowing up front:</p>
<p>First, pg_cron can run many <em>different</em> jobs in parallel, but it never runs two instances of the <em>same</em> job at once. If a job is still running when its next scheduled time arrives, the new run waits in a queue and starts as soon as the current one finishes. This protects you from a slow cleanup job piling up on top of itself.</p>
<p>Second, pg_cron doesn't run jobs while a server is in hot standby mode. If you use streaming replication, jobs only execute on the primary. When a replica gets promoted, the scheduler starts up automatically — so failover doesn't leave you without your scheduled jobs.</p>
<h2 id="heading-how-to-install-and-set-up-pgcron">How to Install and Set Up pg_cron</h2>
<p>Setting up pg_cron on a self-managed server takes three steps: install the package, update the configuration, and create the extension.</p>
<h3 id="heading-step-1-install-the-package">Step 1: Install the Package</h3>
<p>On Debian or Ubuntu using the official PostgreSQL apt repository, install the package that matches your PostgreSQL major version. For PostgreSQL 17, that's:</p>
<pre><code class="language-bash">sudo apt-get install postgresql-17-cron
</code></pre>
<p>On Red Hat-based systems using the PGDG yum repository:</p>
<pre><code class="language-bash">sudo yum install pg_cron_17
</code></pre>
<p>If you're on PostgreSQL 16 or 18, swap the version number accordingly. You can also build the extension from source if your platform doesn't have a package.</p>
<h3 id="heading-step-2-update-postgresqlconf">Step 2: Update postgresql.conf</h3>
<p>pg_cron needs to start its background worker when PostgreSQL boots, so it must be preloaded. Add it to <code>shared_preload_libraries</code> in your <code>postgresql.conf</code>:</p>
<pre><code class="language-ini">shared_preload_libraries = 'pg_cron'
</code></pre>
<p>If that setting already lists other libraries, add pg_cron to the comma-separated list rather than replacing them.</p>
<p>By default, the scheduler stores its metadata in the database named <code>postgres</code>. If your application lives in a different database and you want the jobs there, set:</p>
<pre><code class="language-ini">cron.database_name = 'app_db'
</code></pre>
<p>One more setting worth knowing: pg_cron interprets all schedules in GMT by default. If you want your "3 AM cleanup" to actually run at 3 AM local time, set the timezone explicitly:</p>
<pre><code class="language-ini">cron.timezone = 'Africa/Lagos'
</code></pre>
<p>These settings require a server restart to take effect:</p>
<pre><code class="language-bash">sudo systemctl restart postgresql
</code></pre>
<h3 id="heading-step-3-create-the-extension">Step 3: Create the Extension</h3>
<p>Connect to the database you configured in <code>cron.database_name</code> and create the extension as a superuser:</p>
<pre><code class="language-sql">CREATE EXTENSION pg_cron;
</code></pre>
<p>This creates the <code>cron</code> schema, the metadata tables, and the scheduling functions. You're ready to schedule jobs.</p>
<p>Note that pg_cron can only be <em>installed</em> in one database per PostgreSQL instance. That sounds limiting, but it isn't. You can still run jobs in any database on the instance using <code>cron.schedule_in_database()</code>, which we'll cover later.</p>
<h3 id="heading-a-note-on-how-jobs-connect">A Note on How Jobs Connect</h3>
<p>Since pg_cron opens local connections by default, your <code>pg_hba.conf</code> needs to allow them. The common approaches are enabling <code>trust</code> authentication for localhost connections for the job's user, or putting the password in a <code>.pgpass</code> file.</p>
<p>If you'd rather avoid connection authentication entirely, tell pg_cron to use background workers instead:</p>
<pre><code class="language-ini">cron.use_background_workers = on
max_worker_processes = 20
</code></pre>
<p>With background workers, the number of jobs that can run concurrently is capped by <code>max_worker_processes</code>, so raise it if you schedule a lot of overlapping jobs.</p>
<h3 id="heading-using-pgcron-on-managed-database-services">Using pg_cron on Managed Database Services</h3>
<p>If you're on a managed service, you usually can't edit <code>postgresql.conf</code> directly, but the providers expose the same settings through their own mechanisms:</p>
<ul>
<li><p><strong>Amazon RDS and Aurora PostgreSQL</strong>: add <code>pg_cron</code> to the <code>shared_preload_libraries</code> parameter in your DB parameter group, reboot the instance, then run <code>CREATE EXTENSION pg_cron;</code> as a user with <code>rds_superuser</code>. The scheduler runs in the <code>postgres</code> database.</p>
</li>
<li><p><strong>Azure Database for PostgreSQL</strong>: enable pg_cron under server parameters (<code>shared_preload_libraries</code> and <code>azure.extensions</code>), restart, then create the extension.</p>
</li>
<li><p><strong>Google Cloud SQL</strong>: set the <code>cloudsql.enable_pg_cron</code> flag, restart, then create the extension.</p>
</li>
<li><p><strong>Supabase</strong>: enable the pg_cron extension with a single toggle in the dashboard under Database → Extensions.</p>
</li>
<li><p><strong>Neon</strong>: pg_cron is available as a supported extension you can enable per project.</p>
</li>
</ul>
<p>The SQL you write afterward is identical everywhere, which is part of the appeal.</p>
<h2 id="heading-a-quick-refresher-on-cron-syntax">A Quick Refresher on Cron Syntax</h2>
<p>pg_cron uses the same five-field schedule format as classic Unix cron:</p>
<pre><code class="language-plaintext">┌──────────── minute (0–59)
│ ┌────────── hour (0–23)
│ │ ┌──────── day of month (1–31, or $ for the last day)
│ │ │ ┌────── month (1–12)
│ │ │ │ ┌──── day of week (0–6, Sunday = 0)
│ │ │ │ │
* * * * *
</code></pre>
<p>An asterisk means "every value". You can combine values with commas, ranges with hyphens, and steps with slashes. Some schedules you'll use constantly:</p>
<pre><code class="language-plaintext">*/5 * * * *    every 5 minutes
0 * * * *      every hour, on the hour
0 3 * * *      every day at 3:00 AM
0 3 * * 1-5    3:00 AM on weekdays
30 1 * * 0     1:30 AM every Sunday
0 0 1 * *      midnight on the 1st of each month
</code></pre>
<p>pg_cron also adds two extensions to the standard syntax that regular cron doesn't have.</p>
<p>You can use <code>$</code> in the day-of-month field to mean the last day of the month, which is genuinely painful to express in standard cron:</p>
<pre><code class="language-plaintext">0 23 $ * *     11:00 PM on the last day of every month
</code></pre>
<p>And for jobs that need to run more often than once a minute, you can use a plain interval between 1 and 59 seconds:</p>
<pre><code class="language-plaintext">'30 seconds'   every 30 seconds
</code></pre>
<p>The seconds syntax stands alone — you can't mix it with the five-field format.</p>
<p>If you ever doubt what a schedule means, <a href="https://crontab.guru">crontab.guru</a> translates cron expressions into plain English. Just remember that pg_cron evaluates schedules in the timezone set by <code>cron.timezone</code>, which defaults to GMT.</p>
<h2 id="heading-how-to-schedule-your-first-job">How to Schedule Your First Job</h2>
<p>The core function is <code>cron.schedule()</code>. It comes in two forms: one with a name and one without.</p>
<p>The named form is the one you should use, because names make jobs easy to find, update, and remove:</p>
<pre><code class="language-sql">SELECT cron.schedule(
  'delete-expired-sessions',                          -- job name
  '0 3 * * *',                                        -- schedule
  $$DELETE FROM sessions WHERE expires_at &lt; now()$$   -- command
);
</code></pre>
<p>The function returns the job's ID:</p>
<pre><code class="language-plaintext"> schedule
----------
        1
(1 row)
</code></pre>
<p>A few details worth noticing:</p>
<p>The command is wrapped in <code>$$ ... $$</code>, PostgreSQL's dollar quoting. This saves you from escaping the single quotes inside the SQL. For commands without quotes, regular string literals work fine.</p>
<p>The job runs in the database where you called <code>cron.schedule()</code>, as the user you called it with, using that user's normal permissions. There's no privilege escalation hiding in the scheduler — if your user can't delete from <code>sessions</code>, neither can the job.</p>
<p>And if you call <code>cron.schedule()</code> again with the same job name, pg_cron updates the existing job instead of creating a duplicate. That makes schedules idempotent, which is handy if you define jobs inside database migrations.</p>
<h2 id="heading-practical-pgcron-examples">Practical pg_cron Examples</h2>
<p>Let's walk through the patterns that cover most real-world use. Each example is something you can adapt directly.</p>
<h3 id="heading-example-1-clean-up-old-rows-every-night">Example 1: Clean Up Old Rows Every Night</h3>
<p>Tables that collect transient data — sessions, tokens, audit events, notification logs — grow forever unless something prunes them. A nightly delete is the classic first pg_cron job:</p>
<pre><code class="language-sql">SELECT cron.schedule(
  'purge-old-events',
  '0 2 * * *',
  $$DELETE FROM events WHERE created_at &lt; now() - interval '90 days'$$
);
</code></pre>
<p>Every night at 2:00 AM, rows older than 90 days disappear. If the table is large, consider batching the delete inside a function so each run stays short, then schedule the function instead.</p>
<h3 id="heading-example-2-refresh-a-materialized-view-every-hour">Example 2: Refresh a Materialized View Every Hour</h3>
<p>Materialized views are a great way to cache expensive aggregations, but PostgreSQL never refreshes them on its own. pg_cron fixes that:</p>
<pre><code class="language-sql">SELECT cron.schedule(
  'refresh-daily-sales',
  '5 * * * *',
  'REFRESH MATERIALIZED VIEW CONCURRENTLY daily_sales_summary'
);
</code></pre>
<p>This refreshes the view at five minutes past every hour. The <code>CONCURRENTLY</code> option lets reads continue during the refresh, as long as the view has a unique index.</p>
<h3 id="heading-example-3-build-a-daily-summary-table">Example 3: Build a Daily Summary Table</h3>
<p>Rollup tables are another common pattern: instead of aggregating millions of raw rows on every dashboard load, you precompute the numbers once a day.</p>
<pre><code class="language-sql">SELECT cron.schedule(
  'rollup-daily-orders',
  '15 0 * * *',
  $$
  INSERT INTO daily_order_stats (day, order_count, total_amount)
  SELECT created_at::date, count(*), sum(amount)
  FROM orders
  WHERE created_at &gt;= current_date - 1
    AND created_at &lt; current_date
  GROUP BY created_at::date
  ON CONFLICT (day) DO UPDATE
    SET order_count = EXCLUDED.order_count,
        total_amount = EXCLUDED.total_amount
  $$
);
</code></pre>
<p>At fifteen minutes past midnight, yesterday's orders get summarized into one row. The <code>ON CONFLICT</code> clause makes the job safe to re-run — if it executes twice, it overwrites rather than duplicates.</p>
<h3 id="heading-example-4-run-a-job-every-30-seconds">Example 4: Run a Job Every 30 Seconds</h3>
<p>Some work needs to happen more often than cron's one-minute floor allows: flushing a buffer table, picking up rows from an outbox, advancing a lightweight queue. The seconds syntax handles this:</p>
<pre><code class="language-sql">SELECT cron.schedule(
  'process-outbox',
  '30 seconds',
  'CALL process_outbox_batch()'
);
</code></pre>
<p>Remember the guarantee from earlier: pg_cron won't start a second instance of this job while the first is still running. If a batch occasionally takes 45 seconds, the next run simply waits its turn instead of stampeding.</p>
<h3 id="heading-example-5-run-maintenance-on-the-last-day-of-the-month">Example 5: Run Maintenance on the Last Day of the Month</h3>
<p>Month-end jobs are awkward in standard cron because months have different lengths. pg_cron's <code>$</code> makes it trivial:</p>
<pre><code class="language-sql">SELECT cron.schedule(
  'month-end-vacuum',
  '0 23 $ * *',
  'VACUUM ANALYZE orders'
);
</code></pre>
<p>This runs <code>VACUUM ANALYZE</code> on the <code>orders</code> table at 11:00 PM on the 28th, 29th, 30th, or 31st — whichever happens to be the last day of that month.</p>
<h2 id="heading-how-to-view-and-monitor-your-jobs">How to View and Monitor Your Jobs</h2>
<p>Everything pg_cron knows lives in two tables in the <code>cron</code> schema, and you query them like any other tables.</p>
<p>To see what's scheduled, look at <code>cron.job</code>:</p>
<pre><code class="language-sql">SELECT jobid, jobname, schedule, command, active
FROM cron.job;
</code></pre>
<pre><code class="language-plaintext"> jobid |         jobname         |  schedule  |            command             | active
-------+-------------------------+------------+--------------------------------+--------
     1 | delete-expired-sessions | 0 3 * * *  | DELETE FROM sessions WHERE ... | t
     2 | refresh-daily-sales     | 5 * * * *  | REFRESH MATERIALIZED VIEW ...  | t
(2 rows)
</code></pre>
<p>To see how jobs have actually been running, query <code>cron.job_run_details</code>:</p>
<pre><code class="language-sql">SELECT jobid, status, return_message, start_time, end_time
FROM cron.job_run_details
ORDER BY start_time DESC
LIMIT 10;
</code></pre>
<p>Each row records one execution: whether it succeeded or failed, the message it returned, and exactly when it started and ended. A failed job shows <code>status = 'failed'</code> along with the error message, so debugging usually starts and ends with this table.</p>
<p>One important catch: <strong>pg_cron never cleans this table up by itself</strong>. A job running every 30 seconds writes almost three thousand rows a day. The standard fix is delightfully recursive — schedule a pg_cron job to prune pg_cron's own history:</p>
<pre><code class="language-sql">SELECT cron.schedule(
  'purge-cron-history',
  '0 12 * * *',
  $$DELETE FROM cron.job_run_details
    WHERE end_time &lt; now() - interval '14 days'$$
);
</code></pre>
<p>If you don't want run history recorded at all, set <code>cron.log_run = off</code> in your configuration.</p>
<h2 id="heading-how-to-update-and-remove-jobs">How to Update and Remove Jobs</h2>
<p>To change an existing job, use <code>cron.alter_job()</code> with the job's ID. Only the parameters you pass get changed — everything else stays as it was:</p>
<pre><code class="language-sql">-- Move job 1 from 3 AM to 4 AM
SELECT cron.alter_job(1, schedule := '0 4 * * *');

-- Pause a job without deleting it
SELECT cron.alter_job(1, active := false);

-- Resume it later
SELECT cron.alter_job(1, active := true);
</code></pre>
<p>Pausing with <code>active := false</code> is underrated. During an incident or a big migration, you can switch off a noisy job and switch it back on afterward, without losing its definition.</p>
<p>To remove a job permanently, use <code>cron.unschedule()</code> with either the name or the ID:</p>
<pre><code class="language-sql">SELECT cron.unschedule('delete-expired-sessions');
-- or
SELECT cron.unschedule(1);
</code></pre>
<p>Both return <code>true</code> when the job was found and removed.</p>
<h2 id="heading-how-to-run-jobs-in-other-databases">How to Run Jobs in Other Databases</h2>
<p>Remember that pg_cron is installed in exactly one database per instance, usually <code>postgres</code>. If your instance hosts several databases, you don't install pg_cron in each one — you schedule cross-database jobs from the one place it lives, using <code>cron.schedule_in_database()</code>:</p>
<pre><code class="language-sql">SELECT cron.schedule_in_database(
  'analytics-nightly-vacuum',
  '0 4 * * *',
  'VACUUM ANALYZE page_views',
  'analytics_db'
);
</code></pre>
<p>The job is stored centrally but executes inside <code>analytics_db</code>. The function also accepts an optional username if the job should run as a different user, and an <code>active</code> flag if you want to create it paused.</p>
<p>This pattern keeps all scheduling in one schema on one database, which makes auditing simple: a single <code>SELECT * FROM cron.job</code> shows every scheduled job across the whole instance.</p>
<h2 id="heading-how-to-let-other-users-schedule-jobs">How to Let Other Users Schedule Jobs</h2>
<p>Out of the box, only superusers can call the scheduling functions. To let an application role manage its own jobs, grant it usage on the <code>cron</code> schema:</p>
<pre><code class="language-sql">GRANT USAGE ON SCHEMA cron TO app_user;
</code></pre>
<p>The permission model after that is sensible and safe:</p>
<ul>
<li><p>Jobs run with the permissions of the user who scheduled them, nothing more.</p>
</li>
<li><p>A row-level security policy on <code>cron.job</code> means users only see and modify their own jobs. Superusers see everything.</p>
</li>
<li><p>Each user can also delete their own rows from <code>cron.job_run_details</code>, so the cleanup job from earlier works without superuser rights.</p>
</li>
</ul>
<p>In practice, I recommend creating a dedicated role for scheduled work rather than scheduling jobs as a personal account. When the engineer who scheduled everything leaves and their role gets dropped, you don't want the nightly rollups going with them.</p>
<h2 id="heading-when-to-use-pgcron-and-when-to-avoid-it">When to Use pg_cron (and When to Avoid It)</h2>
<p>pg_cron shines when the work is <em>database work</em>. Use it for:</p>
<ul>
<li><p><strong>Data retention</strong>: pruning old rows from sessions, logs, events, and token tables.</p>
</li>
<li><p><strong>Aggregations</strong>: refreshing materialized views and building rollup tables.</p>
</li>
<li><p><strong>Maintenance</strong>: targeted <code>VACUUM ANALYZE</code>, rebuilding statistics, managing partitions (it pairs beautifully with pg_partman).</p>
</li>
<li><p><strong>Lightweight pipelines</strong>: moving rows between tables, processing outbox patterns, expiring soft-deleted records.</p>
</li>
</ul>
<p>The common thread: the entire job is expressible as SQL or a stored procedure, and it touches nothing outside the database.</p>
<p>You should reach for something else when:</p>
<ul>
<li><p><strong>The job needs to call external systems.</strong> pg_cron runs SQL. It can't send an HTTP request, push to a queue, or send an email on its own. Jobs like that belong in your application or a workflow engine.</p>
</li>
<li><p><strong>You need retries, backoff, and alerting built in.</strong> pg_cron records failures but won't retry them or page you. For workflows that must complete, tools like Temporal or a proper job queue earn their complexity.</p>
</li>
<li><p><strong>The work is heavy and long-running.</strong> A four-hour batch job running inside your primary OLTP database competes with your application for CPU, memory, and locks. Schedule heavy compute elsewhere.</p>
</li>
<li><p><strong>Jobs need complex dependencies.</strong> "Run B only after A succeeds, then fan out to C and D" is orchestration. That's Airflow territory, not cron territory.</p>
</li>
</ul>
<p>A reasonable rule of thumb: pg_cron replaces the crontab entry that used to run <code>psql -c "..."</code> on some forgotten server. It doesn't replace your job queue or your workflow orchestrator.</p>
<h2 id="heading-best-practices-for-working-with-pgcron">Best Practices for Working with pg_cron</h2>
<p>A handful of habits will keep your scheduled jobs boring, in the best sense of the word:</p>
<p><strong>Name every job:</strong> Anonymous jobs identified only by an ID are painful to manage six months later. Names also make <code>cron.schedule()</code> idempotent, which lets you define jobs safely in migrations.</p>
<p><strong>Set the timezone deliberately:</strong> The default is GMT, and "why does the 3 AM job run at 4 AM?" is a rite of passage you can skip by setting <code>cron.timezone</code> on day one.</p>
<p><strong>Keep individual runs short:</strong> Wrap big deletes in batched stored procedures. A job that finishes in seconds holds locks briefly and queues less behind itself.</p>
<p><strong>Make jobs idempotent:</strong> Servers restart, and a job can fail halfway. Use <code>ON CONFLICT</code>, time-window predicates, and other patterns that make a re-run harmless.</p>
<p><strong>Prune</strong> <code>cron.job_run_details</code><strong>:</strong> Schedule the cleanup job from the monitoring section before the table grows large enough that you notice it the hard way.</p>
<p><strong>Monitor for silence, not just failure:</strong> A failed run appears in <code>job_run_details</code>, but a job that stopped being scheduled at all leaves no trace. A periodic check that each critical job has a recent successful run catches both cases:</p>
<pre><code class="language-sql">SELECT j.jobname, max(d.end_time) AS last_success
FROM cron.job j
LEFT JOIN cron.job_run_details d
  ON d.jobid = j.jobid AND d.status = 'succeeded'
GROUP BY j.jobname
HAVING max(d.end_time) &lt; now() - interval '1 day'
   OR max(d.end_time) IS NULL;
</code></pre>
<p>Any job this query returns hasn't succeeded in over a day, and deserves a look.</p>
<h2 id="heading-conclusion">Conclusion</h2>
<p>pg_cron turns PostgreSQL into its own scheduler. You define jobs in SQL, the database runs them, and the entire system — definitions, history, failures — is visible through ordinary queries.</p>
<p>In this tutorial, you learned how the extension works under the hood, how to install it on your own servers and on managed services, how to write schedules (including pg_cron's seconds and last-day-of-month extensions), and how to apply it to the maintenance work every real database accumulates: pruning, rollups, refreshes, and vacuums. You also saw how to monitor jobs, manage permissions, and recognize the point where a real job queue or orchestrator becomes the better tool.</p>
<p>If your infrastructure currently has a lonely server whose only purpose is running <code>psql</code> from a crontab, you now know how to retire it.</p>
<p>Thanks for reading! I write about PostgreSQL and backend engineering. You can connect with me on <a href="https://linkedin.com/in/iyioladev">LinkedIn</a> and <a href="x.com/iyiola_dev_">X</a>.</p>
 ]]>
                </content:encoded>
            </item>
        
    </channel>
</rss>
