This leads them to add a line to their crontab, which gives them a sense of control. Now, their computer runs the script automatically while they sleep.

For a while, that's enough. Then it isn't.

Maybe the backup script failed silently at 3 a.m., and you didn't find out until you needed it later that day, or you developed the perfect script that ran smoothly in your terminal to find it failing when scheduled in cron.

If any of that sounds familiar, congratulations: you've outgrown cron. You're not alone, and this article aims to help you feel understood and ready for better solutions.

This article is about what comes next. We'll look at exactly where cron runs out of road, what 'workflow orchestration' actually means underneath the buzzword, and then build a real workflow step by step so the concepts stick.

By the end, you'll feel more confident and empowered to choose the right tools for complex scheduling challenges.

What We'll Cover:

• What Cron is, and What it's Genuinely Good at

• The Four Walls You'll Hit with Cron

• Can Workflow Orchestration Save the Day?

• Kestra Primer

• How to Use Kestra

• Where to Go Next

What Cron is, and What it's Genuinely Good at

Cron is the system background service (daemon) that runs scheduled tasks. Crontab (cron table) is the configuration file or command utility used to write and manage those task schedules' time-based job scheduler that has shipped with Unix-like systems since the 1970s. It's available on today’s Linux distributions as well as on Macs.

With cron, you specify a schedule and a command, and it runs the command at that time. The schedule uses the famous five-field syntax:

┌───────────── minute (0–59)
│ ┌───────────── hour (0–23)
│ │ ┌───────────── day of month (1–31)
│ │ │ ┌───────────── month (1–12)
│ │ │ │ ┌───────────── day of week (0–6)
│ │ │ │ │
0  9  *  1 5  /usr/bin/python3 /home/me/daily_report.py

Note: The line 0 9 * 1 5  /usr/bin/python3 /home/me/daily_report.py means "run daily_report.py at 9:00 a.m. on weekdays." The syntax is terse, ubiquitous, and (credit where it's due) rock-solid for what it does.

And here's the important part: cron is not bad. For a single, self-contained, failure-tolerant task on one machine, it's the right tool. But for more complex workflows, a purpose-built orchestration tool can provide the reliability and visibility you need, helping you feel more in control of your automation.

The Four Walls You'll Hit with Cron

The trouble with cron starts when your automation stops being a single self-contained task. As your scripts grow, they form more complex systems where you run into the same four limitations, more or less in this order.

Wall 1: Dependencies Between Steps

Your morning routine grows from one script into three. Consider an ETL scenario with three scripts:

1. extract.py pulls yesterday's orders from an API.

2. transform.py cleans the data and computes totals.

3. load.py writes the result into the analytics database.

Each step depends on the one before it. The obvious cron approach is to guess at the timing:

0 2 * * *  python extract.py
0 3 * * *  python transform.py
0 4 * * *  python load.py

You're now hoping the extract finishes within an hour, so that the transform has something to work with. On the day the API is slow, and extract takes 70 minutes, transform runs against stale or missing data and quietly produces garbage. Cron has no concept of "run B only after A succeeds." It only knows wall-clock time.

Wall 2: Failure Handling and Retries

Networks blip. APIs return 503 status codes (Service Unavailable). Databases drop connections. A robust job needs to detect a failure and retry. Maybe three times, or with increasing delays between attempts, so you don't hammer a struggling service.

With cron, retry logic is your problem. You end up bolting it onto every script by hand: try/except blocks, sleep calls, a counter, and a flag file so the next cron tick knows whether the previous one finished. Multiply that across a dozen jobs, and you've written a small, buggy, undocumented orchestration engine.

Wall 3: Visibility

Ask yourself, did your jobs run last night? Which ones succeeded? How long did each take? Is the slowdown in the extract step or the load step?

With cron, the honest answer is "I'd have to go read some log files, if the script even wrote any." The first task is finding the cron job log files themselves.

Finding them can be complicated: cron's logs can be in /var/log/syslog on one system and /var/log/cron on another, assuming logging is enabled. Meanwhile, your script's output only exists if you explicitly redirect stdout and stderr. As a result, you have to search through system logs to verify the job ran, then locate any separate output files that captured its print statements.

When a job fails, identifying the issue is even harder, as you must sift through interleaved logs from multiple runs, trying to determine which timestamp corresponds to the last execution and where it went wrong, often relying only on a non-zero exit code as a clue. There's no dashboard, no run history, no record of how long things took, and no alert when something breaks.

Failures are silent by default, the single most dangerous property a background job can have. You find out your pipeline has been broken for a week when someone downstream notices the numbers stopped updating.

Wall 4: Backfills and Re-runs

Your analytics database has been live for two months when you discover a bug in transform.py that miscalculated totals. You've fixed the code. Now you need to rerun the pipeline for each day in those two months, processing its own slice of data.

This is a backfill, and with cron, it's a nightmare. Cron only ever runs "now." There's no built-in notion of "run this job as if it were March 14th, then March 15th, then..." So you write yet another throwaway script with a date loop, pray it's idempotent, and babysit it.

Notice the pattern across all four walls: each time, you end up reimplementing something poorly that a category of tools already solves well. That category is workflow orchestration.

Can Workflow Orchestration Save the Day?

A workflow orchestrator manages workflows, which are sets of tasks with specific relationships, triggers, and failure protocols, along with observability for tracking outcomes.

While many solutions like Airflow, Dagster, Prefect, and Temporal are available, we'll focus on Kestra, an open-source orchestrator. Kestra stands out from the pack by enabling developers to run, monitor, and manage workflows all from a single declarative layer compatible with any programming language and infrastructure, including public, private, or even air-gapped networks.

Kestra also provides enterprises with the control needed to ensure insights and regulatory compliance. With over 1,600 connectors available, you can build almost any data, infrastructure, or AI workflow you can imagine.

In this tutorial, we'll continue with the cron scheduler theme and build a simple ETL workflow that uses a cron-like schedule while addressing some of cron's shortcomings, such as execution sequence and error handling.

Kestra Primer

Kestra was born out of the engineering pain of creating Python code to achieve proper workflow orchestration. Rather than spend engineering hours crafting the right Python code for your Apache Airflow DAG, with Kestra, your workflow is a YAML file. Simply describe what should run in plain, declarative syntax. Store the file in Git, and deploy it like any other code.

There's no obscure UI logic building and no hidden state management. Just simple text that can be easily reviewed and diffed. The workflow you read is the workflow that runs.

The example YAML file in Figure 1 illustrates a scenario in which you want to move NoSQL data into an analytics-ready warehouse.

id: cassandra-to-bigquery
namespace: company.team

tasks:
  - id: query_cassandra
    type: io.kestra.plugin.cassandra.Query
    session:
      endpoints:
        - hostname: localhost
          port: 9042
      localDatacenter: datacenter1
    cql: |
      SELECT salary_id, work_year, experience_level, employment_type,
      job_title, salary, salary_currency, salary_in_usd, employee_residence,
      remote_ratio, company_location, company_size
      FROM test.salary
    fetchType: STORE

  - id: write_to_csv
    type: io.kestra.plugin.serdes.csv.IonToCsv
    from: "{{ outputs.query_cassandra.uri }}"

  - id: load_bigquery
    type: io.kestra.plugin.gcp.bigquery.Load
    from: "{{ outputs.write_to_csv.uri }}"
    destinationTable: my_project.my_dataset.my_table
    serviceAccount: "{{ secret('GCP_SERVICE_ACCOUNT_JSON') }}"
    projectId: my_project
    format: CSV
    csvOptions:
      fieldDelimiter: ","
      skipLeadingRows: 1

Figure 1: Cassandra to BigQuery Example

Without even knowing much about Kestra, the YAML file is simple and self-explanatory. Later in this post, we'll create a basic ETL flow and explain the significance of fields such as id and type. But first, let’s get an instance of Kestra running on your local machine.

How to Set Up Kestra

Kestra is available as an open-source platform licensed under the Apache 2.0 license as well as an Enterprise offering additional features and product support. In this tutorial, we'll use Docker to run Kestra locally using the latest version.

To spin up Kestra, run the following Docker command:

docker run --pull=always --rm -it -p 8080:8080 
  --user=root \
  --name kestra \
  -v kestra_data:/app/storage \
  -v kestra_db:/app/data \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /tmp:/tmp \
  kestra/kestra:latest server local

For other platforms such as Windows and Linux check out https://kestra.io/get-started.

Once the containers are loaded, navigate to the Kestra UI at http://localhost:8080. The welcome screen will ask you to create an administrator. Create the user and finish the initial launch wizard.

Once that's complete, navigate to the Flows tab in the left panel, then click the Create button at the top right of the page. This will create a new flow using a sample template, as shown in the following figure:

Figure 2: Flows page showing the new flow template

Flows in Kestra

In Kestra you define your workflow orchestration through Flows. You can create these Flows using YAML syntax in the UI, through a no-code editor in the UI, or programmatically through an API. In this tutorial, we'll create flows using YAML.

In Figure 2 above, you'll notice that a sample flow is already created to get you started.

The _id, namespace, and tasks are three required fields and are used to identify the flow within the Kestra environment and the task the flow should execute. Each flow lives in one namespace. Namespaces are like folders in a filesystem and are used to group flows and provide structure. Note that you can't change a flow’s namespace after creation.

How to Use Kestra

Step 1: A Simple Task Executed on a Schedule

Let's start by erasing the sample flow provided and replacing it with the following:

id: morning_report
namespace: tutorial

tasks:
  - id: say_hello
    type: io.kestra.plugin.core.log.Log
    message: "Good morning — the pipeline ran at {{ execution.startDate }}"

This example has the required id, namespace, and has one tasks field that logs a message. The message displayed uses a pebble expression (the command inside the  {{ }} brackets) to show the startDate of the execution.

Pebble expressions are used to set values dynamically in flows. In this example, the start date of the execution will be inserted within the string.

To execute this flow, we need to first save it by clicking the Save button in the upper-right corner of the page. Once saved, click the Play button. You'll then be presented with the execute options page, as shown in Figure 3 below:

Figure 3: Execute flow options

If this workflow had inputs such as a filename or URL, you can manually enter them here to test your flow. The modal also provides the curl command if you’d like to run the flow via the API instead of the UI. Click the Execute button to run the flow:

Figure 4: Flow execution log

The flow was simple and wrote the info message to the log file. If the flow had errors or warnings, you'd be able to see detailed execution logs on this page.

Now that we’ve created our first task, let’s schedule it with a cron mask. Add the following to the flow by clicking on the “Edit Flow” button at the top of the page.

Next, add the triggers section to the flow:

triggers:
  - id: every_minute
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "* * * * *"

Execute the flow.  After a few minutes, check out the execution history by clicking on the “Execution” tab on the left side navbar.

Figure 5: Flow execution page

Our flow now behaves similarly to a single task cron job. The flow has a triggers block with a Schedule trigger whose cron field is the exact same five-field syntax you already know. That last point matters: you're not throwing away what you learned. You're wrapping it in something that can grow as your needs change.

Notice that even in this simple Kestra example, you’ve gained something cron didn't offer. Every time this runs, it's recorded as an execution with a timestamp, a duration, and a status, and all logs are visible in the UI. That's Wall 3 (visibility) handled before we've even done anything interesting.

Step 2: Real Work, and a Dependency

Now let's replace the simple task with the three-step extract/transform/load and have the orchestrator enforce the ordering rather than timing offsets.

id: csv_to_parquet
namespace: company.team
description: Download orders CSV, transform it with a Python script, and write the result to a Parquet file.

tasks:
  # Download a public CSV file into Kestra's internal storage
  - id: download_csv
    type: io.kestra.plugin.core.http.Download
    uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv

  # Transform the CSV with a simple Python script and write it out as Parquet
  - id: transform_to_parquet
    type: io.kestra.plugin.scripts.python.Script
    containerImage: ghcr.io/kestra-io/pydata:latest
    inputFiles:
      input.csv: "{{ outputs.download_csv.uri }}"
    outputFiles:
      - orders.parquet
    script: |
      import pandas as pd

      # Read the downloaded CSV
      df = pd.read_csv("input.csv")

      # --- simple transformation ---
      # Ensure numeric types and add a computed column
      df["total"] = df["quantity"] * df["price"]

      # Keep only orders above a small threshold as an example filter
      df = df[df["total"] > 0]

      print(f"Rows after transform: {len(df)}")

      # Write the result to Parquet
      df.to_parquet("orders.parquet", index=False)

  # Log that the Parquet file was produced
  - id: log_output
    type: io.kestra.plugin.core.log.Log
    message: "Parquet file created: {{ outputs.transform_to_parquet.outputFiles['orders.parquet'] }}"
# Expose the Parquet file as a downloadable flow output.
# FILE-typed flow outputs appear on the execution's Overview tab with a download button.
outputs:
  - id: parquet_file
    type: FILE
    value: "{{ outputs.transform_to_parquet.outputFiles['orders.parquet'] }}"

Save the flow, then execute.

Figure 6: Execution results

Two things just happened. First, tasks listed in sequence run in sequence. Transform_to_parquet only starts after download_csv succeeds, and log_output only after transform_to_parquet succeeds. If a failure occurs, the execution stops, and transform_to_parquet never touches stale data. That's Wall 1 (dependencies) gone, with no guesswork about timing.

Second, notice the expression within the flow {{ outputs.download_csv.uri }}. Tasks can pass data and metadata to subsequent tasks using expressions like this. That wiring turns a list of scripts into an actual pipeline.

Step 3: Surviving Failure with Retries

Now consider the scenario where the download_csv task encounters a network issue, and the flow is unable to download the latest data. Let's make that task resilient declaratively by adding a retry section to the download_csv task:

- id: download_csv
    type: io.kestra.plugin.core.http.Download
    uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv
    retry:
      type: constant
      maxAttempts: 5
      interval: PT10S

That's the whole retry policy. If the download fails, Kestra waits and tries again up to 5 times, with a 10-second delay (PT10S is ISO-8601 for "10 seconds"). There are no counters, sleep calls, or flag files. Wall 2 (failure handling) is handled in 3 lines that read like a sentence.

To test this, drop the “s” from orders.csv and rerun the flow. You can see the execution showing retrying.

Figure 7: Execution showing retry

If all attempts fail, you'll probably want to be notified. Let’s add a flow-level error handler that runs only when something in the workflow fails:

errors:
  - id: notify_failure
    type: io.kestra.plugin.notifications.slack.SlackIncomingWebhook
    url: "{{ secret('SLACK_WEBHOOK') }}"
    payload: |
      {"text": "orders_pipeline failed on execution {{ execution.id }}"}

Now a broken pipeline pings a Slack channel instead of failing silently at 3 a.m. Note that secrets are protected within flows via the secret expression.

Step 4: Triggering on Events, Not Just Time

Schedules are only one kind of trigger. Suppose orders don't arrive on a fixed timetable. Instead, a file lands in cloud storage whenever an upstream system feels like it.

Polling on a cron schedule ("check every 5 minutes, exit if nothing's there") is wasteful and laggy. Event triggers are the better model: run the workflow when the thing happens.

Conceptually, instead of adding a scheduled trigger similar to the one added in the previous example:

triggers:
  - id: every_minute
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "* * * * *"

You can add a trigger that fires when a new object appears in an S3 bucket:

triggers:
  - id: new_s3_object
    type: io.kestra.plugin.aws.s3.Trigger
    interval: "PT1M"
    accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
    secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
    region: "eu-central-1"
    bucket: "my-bucket"
    prefix: "incoming/"
    on: CREATE
    action: NONE

Alternatively, you could have a trigger that exposes a webhook URL you can POST to start execution, or real-time triggers that listen to streaming services such as Kafka queues. In any of these trigger scenarios, the workflow body stays identical. Your automation can now respond to the world rather than just watching the clock.

Step 5: Backfilling the Past

Finally, consider the bug-in-transform scenario. You've fixed the calculation and need to re-run the pipeline for every day of the last two months. This would be painful in cron.

In an orchestrator, a backfill is a first-class operation on a scheduled workflow: you pick a start and end date, and it generates one execution per scheduled interval across that range. Each execution is aware of the date it represents, through an expression like {{ trigger.date }}. Your transform step can use that date to fetch and process the correct slice of data.

This is where idempotency stops being academic. Because a backfill re-runs days you may have already processed, your data load step should write with an "insert or replace" semantic keyed on the date, so running March 14th, for example, twice leaves the database in the same state as running it once.

Design for that and backfills become routine instead of terrifying. Wall 4 (backfills) is handled, but only if you've held up your end with idempotent tasks.

Where to Go Next

The best way to internalize all of this is to take an existing cron job and rebuild it as a proper workflow. Start with the one-task version, confirm it runs and appears in the run history, then add a second dependent task, a retry policy, and a failure alert.

Each step maps to one of the four walls, and you'll feel the difference immediately the first time a task fails, retries itself, and recovers without waking you up.

You don't need to write every line from scratch. Kestra provides a library of Blueprints that are hundreds of ready-made, copy-pasteable flows that you can browse at kestra.io/blueprints or access directly in your instance under the Blueprints tab.

Each Blueprint is a complete, executable example with an explanation of its functionality and how to extend it. This allows you to start with a close version of your goal and modify it rather than guessing at the syntax.

Cron showed that computers can operate while you're asleep. Orchestration takes it further, ensuring they do so reliably, in order, and visibly. This shift transforms the idea from merely writing a script to managing a full system.

AWS also offers an API Gateway, allowing developers to invoke AWS Lambda functions, which provides enhanced security and performance features like rate limiting. But even with the API Gateway, you have to coordinate these microservices, as your client applications likely each have unique data needs. Data might need to be transformed, filtered, or combined before it is returned to the client.

These orchestration tasks can reduce your productivity and take time and effort away from solving the business problem your application is trying to solve.

Apollo GraphQL is an API orchestration layer that helps teams ship new features faster and more independently by composing any number of underlying services and data sources into a single endpoint. This allows clients on-demand access to precisely what the experience needs, regardless of the source of that data.

This article will teach you how to orchestrate AWS Lambda functions using Apollo GraphQL. Specifically, here’s what we will cover:

• GraphQL Primer

• Tutorial Overview

• Prerequisites

• Section 1: Create the AWS Resources

• Section 2: Create an Apollo Connector

• Section 3: How to Use Apollo Sandbox

• Summary

GraphQL Primer

For those unfamiliar with GraphQL, here’s a primer that offers some background on the challenges GraphQL addresses and how data is typically managed through REST APIs in GraphQL before the emergence of Apollo Connectors. If you’re familiar with GraphQL, feel free to skip this section.

GraphQL is a query language for APIs. This query language and corresponding runtime enable clients to specify exactly the data they require, minimizing over-fetching and under-fetching.

In contrast to REST, which necessitates multiple endpoints for various data requirements, GraphQL streamlines queries into a single request, enhancing performance and reducing network latency.

GraphQL also uses a strongly typed schema. This improves API documentation and makes validation, early error detection, and immersive developer tooling easy.

To illustrate the difference between REST APIs and GraphQL, consider the following REST API call: /user/123

Response:

{
  "id": 123,
  "name": "Alice Johnson",
  "email": "alice@example.com",
  "phone": "555-1234",
  "address": {
    "street": "123 Main St",
    "city": "Springfield",
    "state": "IL",
    "zip": "62704"
  },
  "createdAt": "2022-01-01T12:00:00Z",
  "updatedAt": "2022-05-15T14:30:00Z",
  "isAdmin": false
}

If you were only interested in the name and email, using REST would be a lot of data returned from the network to the client for no reason. Using GraphQL, the GraphQL query to return the name and email would be the following:

query {
  user(id: 123) {
    name
    email
  }
}

The result set is just the data the client needs:

{
  "data": {
    "user": {
      "name": "Alice Johnson",
      "email": "alice@example.com"
    }
  }
}

This is a simple example showing the benefit of not over-fetching data, but GraphQL has many other advantages. One of them is the separation between client and server. Since both parties leverage and respect the GraphQL type schema, both teams can operate more independently with the back end defining where the data resides and the front end only asking for data it needs.

So how does GraphQL know how to populate data for every field in your schema? It does this through resolvers. Resolvers can fetch data from a back-end databases or third-party API such as REST APIs, gRPC, and so on. These functions comprise procedural code compiled and maintained for each field in the schema. Thus, one field can have a resolver that queries a REST API and another can query a gRPC endpoint.

To illustrate resolvers, consider the example above. Let’s add a field, status, that queries a REST API to determine if the user is full-time, part-time, or terminated.

First we have defined our schema as:

type User {
  id: ID!
  name: String!
  email: String!
  status: String!  # Need this from an external REST API
}

type Query {
  user(id: ID!): User
}

The user query in this case will accept a user id and return a type User. The resolver function to support the data fetching resembles the following:

const resolvers = {
  Query: {
    user: async (_, { id }) => {
      // Fetch user details from one REST API
      const userResponse = await fetch(`https://api.company.com/users/${id}`);
      const userData = await userResponse.json();

      // Fetch employee status from another REST API
      const statusResponse = await fetch(`https://api.company.com/employees/${id}/status`);
      const statusData = await statusResponse.json();

      return {
        id: userData.id,
        name: userData.name,
        email: userData.email,
        status: statusData.status, // e.g., "Full-Time", "Part-Time", "Terminated"
      };
    },
  },
};

Notice that not only are there two fetches needed to obtain the information the query requires, but we also need to write procedural code and deploy it.

A better approach would be to declaratively specify to GraphQL where the REST API is located and what data to return. Apollo Connectors is the solution to this challenge, simplifying the process and allowing you to declaratively integrate REST API data without requiring code compilation and maintenance.

Now that you have a general idea of GraphQL and the challenges it addresses, let’s delve into the example we will build out.

Tutorial Overview

In this tutorial, you will create two AWS Lambda functions that return product information, which are described as follows:

Products Request:

POST /2015-03-31/functions/products/invocations

Response:

{
  "statusCode": 200,
  "body": [
    {
      "id": "RANQi6AZkUXCbZ",
      "name": "OG Olive Putter - Blade",
      "description": "The traditional Block in a blade shape is made from a solid block of Olive wood. The head weight is approximately 360 grams with the addition of pure tungsten weights. Paired with a walnut center-line and white accents colors.",
      "image": "https://keynote-strapi-production.up.railway.app/uploads/thumbnail_IMG_9102_3119483fac.png"
    },
    {
      "id": "RANYrWRy876AA5",
      "name": "Butter Knife Olive Putter- Blade",
      "description": "The traditional Block in a extremely thin blade shape (~1\") is made from a solid block of Olive wood. The head weight is approximately 330 grams with the addition of pure tungsten weights.",
      "image": "https://keynote-strapi-production.up.railway.app/uploads/thumbnail_IMG_9104_97c221e79c.png"
    },...

Product-price request:

POST: /2015-03-31/functions/product-price/invocations

Response:

{
  "default_price": 49900,
  "is_active": true,
  "currency": "usd",
  "billing_schema": "per_unit",
  "recurring": {
    "interval": 0,
    "interval_count": 3
  }
}

To expose these two lambda microservices, you need to create API Gateway triggers. This involves either setting up a distinct API Gateway for each lambda or consolidating them under one or a few API Gateway instances with specified routes for each lambda.

Creating a trigger may feel tedious and repetitive in a microservices setup. But there is an alternative available. You could directly invoke those functions via REST using the InvokeFunction permission assigned to an IAM user. This article will show you this method and guide you through function creation, necessary AWS IAM permissions, and configuring the Apollo Connector to invoke the function.

Prerequisites

To follow along in this tutorial, you will need to have a basic understanding of AWS Lambda functions as well as AWS security. You’ll also need access to the following:

• An AWS account with permissions to create IAM Users and Policies

• An Apollo GraphQL account, you can sign up for a free plan here.

We will also use the following tools:

• VS Code: Microsoft VS Code is a free source code editor from Microsoft

• Apollo Rover CLI: Rover is the command-line interface for managing and maintaining graphs

• Apollo Studio: A web-based portal used for managing all aspects of your graph

• Apollo Connectors Mapping Playground: A website that takes a JSON document and helps developers create the selection mapping used with Apollo Connectors

Section 1: Create the AWS Resources

First, let’s configure our AWS environment, starting with security. In our scenario, we will create an IAM User, “ConnectorUser,” with access to an AWS Policy, “ConnectorLambdaPolicy,” with the minimum permissions needed to access the AWS Lambda functions.

Note that you could create user groups and assign permission policies to those groups in a production environment. But for this article, we are reducing the number of administrative steps to focus on the core integration with GraphQL.

Step 1: Create an AWS Policy

To create a policy, navigate to IAM within the AWS Management console, then select “Policies” under Access Management. Click “Create Policy”. This will open the policy editor page, as shown below:

Choose the “Lambda” service and under the Access level select “InvokeFunction” from the Write drop down menu as shown below:

Under the Resources menu, you can choose either All ARNs or a specific option. It's a best practice to be as granular as possible when defining security configurations. In this example, let’s limit our selection to the “us-east-1” region by clicking on the “Specific” option and then “Add ARNs.” Enter “us-east-1” in the resource region and select “Any function name.”

With the policy created, we can assign an IAM user to that policy.

Step 2: Create the IAM User and Attach a Policy

Click on Users under “Access Management” then Create User. Provide a name for the user, “ConnectorUser”.

Next, select “Attach policies directly,” choose the policy we just created, “ConnectorLambdaPolicy,” and click “Create User.”

Step 3: Create AWS Lambda Functions

In your AWS console, create a new NodeJS AWS Lambda function, “products”.

Select “Node.JS” for the runtime then click “Create function”. Once created, paste in the the function code from this Gist.

Repeat this process, creating another function for, “product-price” and use the function code from this Gist.

Section 2: Create an Apollo Connector

In this section, we will install the Apollo Rover CLI tool, create an Apollo Studio free tier account, and clone the Apollo Connectors repository. If you already have an Apollo environment available, you can skip steps 1 and 2.

Step 1: Install Rover

Rover is the command-line interface for managing and maintaining graphs. It also provides a modern hot-reloading experience for developing and running your connectors locally. If you don’t have Rover installed, install it by following the steps here.

Step 2: Create an Apollo Studio Free Tier Account

Apollo Studio is a cloud-based management platform designed to explore, deliver, and collaborate on graphs. If you do not have an Apollo Studio account, create one on a free plan by navigating here.

Step 3: Clone the Apollo Connectors Repository

To help you start your first Apollo Connector, a GitHub repository provides sample connectors and a template script. When run, this script will create all the necessary files and configurations you need to begin.

Go ahead and clone the repository from here.

Note: While not required, I recommended using VS Code, as this repo leverages VS Code-specific settings files.

Step 4: Create a .env File

Before you run the Create Connectors template script, create a .env locally with a user API key from your Apollo Studio. You can create and obtain this key here. Populating this .env file will add this API key to the connector template you create in the next step.

Step 5: Create Your New Connector from a Template

Execute npm start and provide a location to create the connector template. You can use default values for the remaining questions.

This script will create all the necessary files to run a local Apollo GraphQL instance in the specified directory. Load the newly created connector using VS Code or your preferred code editor. You will return to this editor soon, but first, we need to obtain some access keys from AWS.

Step 6: Create an AWS Access Key

Since we connect to AWS using SigV4, we must create an AWS access key and enter the KEY values in the settings.json file. Return to the AWS IAM Console and select the ConnectorUser you created in Step 1. Create a new access key by clicking on “Create access key”.

You will be presented with multiple options as far as where the use of this key will originate. Since we are first running locally, select “Third-party service” and then continue the wizard until you are presented with the key and secret key as shown below:

Add the access key and secret access key to the settings.json file as “AWS_ACCESS_KEY_ID” and “AWS_SECRET_ACCESS_KEY” respectively.

You'll need to reload the window since VS Code only loads these files under the .vscode directory once.

Note: In this step, we saved the key to the settings.json file. While this is acceptable for development, consider saving environment variables in .env files.

Step 7: Configure the Graph

The supergraph.yaml file is used to define all the subgraphs that are part of this federation. Modify the supergraph.yaml file as follows:

federation_version: =2.10.0
subgraphs:
  awsconnector:
    routing_url: http://lambda
    schema:
      file: connector.graphql

Step 8: Configure Apollo Router

Apollo Router supports AWS SigV4 authentication. To configure the connector to use this, modify the router.yaml file and add an authentication section as follows:

authentication:
  connector:
    sources:
      awsconnector.lambda:   # subgraph name . connector source name
        aws_sig_v4:
          default_chain:
            region: "us-east-1"
            service_name: "lambda"

There are other AWS security configuration options available, including using assume role. The full documentation for subgraph authentication is available here.

Step 9: Build the connector

Now that we have configured the environment variables and authentication information, we are ready to build the connector. Open the connector.graphql file and erase the contents. Next, copy the following extend schema:

extend schema
  @link(
    url: "https://specs.apollo.dev/federation/v2.10"
    import: ["@key"]
  )
  @link(
    url: "https://specs.apollo.dev/connect/v0.1"
    import: ["@source", "@connect"]
  )

  @source(
    name: "lambda"
    http: { baseURL: "https://lambda.us-east-1.amazonaws.com" }
  )

Extend schema is used to link the Apollo Connectors directives into the current schema. In this article we are defining the base URL of our lambda function. If your REST API has HTTP headers that apply to all references of this source, such as Content-Length restrictions, you can add them here in the @source declaration. Next, let’s define the Product schema:

type Product {
  id: ID!
  name: String
  description: String
  image: String
  price: Price
    @connect(
      source: "lambda"
      http: {
        POST: "/2015-03-31/functions/product-price/invocations"
        body: """
        product_id: $this.id
        """
      }
      selection: """
      amount: default_price
      isActive: is_active
      currency
      recurringInterval: recurring.interval -> match(
        [0,"ONE_TIME"],
        [1,"DAILY"],
        [2,"MONTHLY"],
        [3,"ANNUALLY"],
      )
      recurringCount: recurring.interval_count
      """
    )
}

Notice our query Products has an @connect directive that defines, at a minimum, the source name. Here, you can add the HTTP-specific configuration you need for this field, such as Authorizations headers. In this scenario, since we only defined a baseUrl in the extend schema section, we need to put the specific URL for the InvokeFunction, which is /2015-03-31/functions/product-price/invocations.

The selection field allows you to transform and map values returned from the REST API using the mapping definition defined in the selection field. While a complete discussion of selection mapping is beyond the scope of this article, check out the documentation for a detailed look at Mapping GraphQL Responses. Apollo provides a free online tool that makes building mappings intuitive and fast.

Next, let’s define the Price schema and products Query.

type Price {
  amount: Float
  isActive: Boolean
  currency: String
  recurringInterval: RecurringInterval
  recurringCount: Int
}
enum RecurringInterval {
  ONE_TIME
  DAILY
  MONTHLY
  ANNUALLY
}

type Query {
  products: [Product]
    # https://docs.aws.amazon.com/lambda/latest/api/API_Invoke.html
    @connect(
      source: "lambda"
      http: { POST: "/2015-03-31/functions/products/invocations" }
      selection: """
      $.body {
        id
        name
        description
        image
      }
      """
    )
}

Now we're ready to run our connector and issue queries to our graph! The complete configuration script is available at this Gist.

Step 10: Run the Connector

If you're using VS Code, the repository includes a tasks.json file that adds a “rover dev” task, which launches Rover locally.

{
    "version": "2.0.0",
    "tasks": [{
        "label": "rover dev",
        "command": "rover", // Could be any other shell command
        "args": ["dev", "--supergraph-config","supergraph.yaml", "--router-config","router.yaml"],
        "type": "shell",
        "problemMatcher": [],
    }]
}

If you are not using VS Code, you can start your graph by executing rover dev –supergraph-config supergraph.yaml –router-config router.yaml from a terminal window.

If everything is configured correctly, you’ll see the following:

Section 3: How to Use Apollo Sandbox

The rover dev command you launched in the previous step configures a local Apollo Router instance for development mode. This mode makes it easy for developers to create, execute, and debug ad-hoc GraphQL queries using the Apollo Sandbox web portal. This portal is located at http://localhost:4000 by default.

Launch the portal and click on the products field. This will populate the Operation pane with all the available fields in the schema. In the operation pane, you can modify and build your GraphQL query. Clicking the Run button (which displays the query name, Products, in our example) will execute the query and show the results in the Response panel, as illustrated in the figure above.

In this example, you can see that data has been returned from our AWS Lambda function. To confirm, you can view the query plan by selecting "Query Plan” from the Response drop-down menu.

The query plan illustrates the orchestration of our two AWS Lambda functions that fetch product and product price data.

A helpful debugging feature is the Connectors Debugger, available in the drop-down as shown in the previous figure.

The Connection Debugger provides a comprehensive view of the HTTP request, including headers, body, response code, and the selection mapping used in the query. If you’re experiencing difficulties running queries, use this debugger – it will save you a lot of time.

Summary

In this article, you learned how to:

• Set up AWS IAM User, Policies, and Lambda functions

• Create an Apollo Connector to obtain data from an AWS Lambda function

• Configure the Apollo Router

• Execute and debug queries using Apollo Sandbox

Integrating AWS Lambda with Apollo Connectors offers a simplified, resolver-free method for incorporating cloud functions into your GraphQL API. By utilizing Apollo Connectors, you can declaratively link REST-based Lambda functions to your supergraph while ensuring secure authentication with AWS SigV4.

You can learn more about Apollo Connectors from the following resources:

1. Tutorial: GraphQL meets REST, with Apollo Connectors

2. Blog: Discover how Apollo Connectors integrate with Apollo Federation through insights from Apollo's Founder & CTO: REST API Orchestration with GraphQL.

3. Blog: Delve into the engineering journey behind Apollo Connectors and the process of their creation: Our Journey to Apollo Connectors

4. Webinar: Apollo Connectors GA Launch Webinar