Workflows are set steps by which individual jobs are executed. You can use Argo Workflows for various purposes, such as data processing, machine learning, CI/CD, and infrastructure automation.

In this article, you will set up Argo Workflows on a Kubernetes cluster and use available templates to create a Workflow to manage in a Kubernetes cluster.

Argo Workflows Key Concepts

As I mentioned above, Argo Workflows is implemented as a Kubernetes custom resource definition (CRD) by its own controller. Argo Workflows is made up of two main concepts: workflow and Template.

Workflow

A workflow is a central resource in Argo Workflows that defines the workflow to be executed and stores the workflow’s state.

A workflow consists of a specification that contains an entrypoint and a list of templates.

A workflow can model complex logic using directed acyclic graphs (DAGs) or steps to capture the dependencies or sequences between the templates.

Template

A template is a core concept in Argo Workflows that defines the instructions to execute in a workflow step. A template can be one of the following types:

• Container: Allows you to be able to define the container. Here is an example:

  - name: hello-world
    container:
      image: alpine:latest
      command: [sh, -c]
      args: ["echo hello world"]

• Script: A template that runs a script in a container image. This is similar to the container type, but allows you to write the script inline instead of using a separate file. Here is an example:

  - name: factorial
    inputs:
      parameters:
      - name: num
    script:
      image: python:alpine 3.6
      command: [python]
      source: |
        def factorial(n):
          if n == 0:
            return 1
          else:
            return n * factorial(n-1)
        print(factorial(int({{inputs.parameters.num}})))

• Resource: This template allows direct manipulation of cluster resources. It can be used for operations such as retrieval, creation, modification, or deletion, including GET, CREATE, APPLY, PATCH, REPLACE, or DELETE requests. Here is an example:

  - name: create-configmap
    resource:
      action: create
      manifest: |
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: my-config
        data:
          foo: bar
          hello: world

• DAG: This template defines a directed acyclic graph of other templates. In this example, the DAG has three tasks: A, B, and C. Task A runs first, then tasks B and C run in parallel. Here is an example:

  - name: my-dag
    dag:
      tasks:
      - name: A
        template: echo
        arguments:
          parameters: [{name: message, value: A}]
      - name: B
        dependencies: [A]
        template: echo
        arguments:
          parameters: [{name: message, value: B}]
      - name: C
        dependencies: [A]
        template: echo
        arguments:
          parameters: [{name: message, value: C}]

Now, let's get started.

Prerequisites

To follow this guide, make sure you have the following:

• A Kubernetes cluster. To create a new Kubernetes cluster, follow this guide

• Basic knowledge of what Kubernetes is. You can learn more about Kubernetes from their official docs

• The Kubectl CLI

Step 1 - Install Argo Workflows

1. Using Kubectl, create a namespace for Argo Workflows to segregate your kubernetes cluster resources.

$ kubectl create namespace argo

2. Install the latest Argo Workflows release file from Argo Workflows github page.

$ kubectl apply -n argo -f https://github.com/argoproj/argo-workflows/releases/download/v<VERSION>/install.yaml

The version of Argo Workflows used in this guide is v3.5.0.

3. Check if all the resources have been installed correctly.

$ kubectl get all -n argo

Output:

NAME                                      READY   STATUS    RESTARTS   AGE
pod/workflow-controller-7f8c9f8f5-9qj2l   1/1     Running   0          2m
pod/argo-server-6f8f9c9f8f-6kx4d          1/1     Running   0          2m

NAME                                  TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/argo-server                   ClusterIP   10.3.240.123    <none>        2746/TCP   2m
service/workflow-controller-metrics   ClusterIP   10.3.240.124    <none>        9090/TCP   2m

NAME                                 READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/workflow-controller   1/1     1            1           3m05s
deployment.apps/argo-server           1/1     1            1           3m07s

NAME                                            DESIRED   CURRENT   READY   AGE
replicaset.apps/workflow-controller-7f8c9f8f5   1         1         1       3m33s
replicaset.apps/argo-server-6f8f9c9f8f          1         1         1       2m33s

Step 2 - Start the Argo UI for Monitoring

Argo Server has a graphical user interface that you can use to manage and monitor your Kubernetes cluster Workflows.

In this guide, you will bypass the authentication process of requesting a token to access the Argo web interface because it cannot be accessed publicly. This is called the Server Authentication mode, although you can use the other mode, Client Authentication, which requires that you request a token to be able to access the web interface.

1. Change the authentication mode to Server Authentication.

$ kubectl patch deployment \
  argo-server \
  --namespace argo \
  --type='json' \
  -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/args", "value": [
  "server",
  "--auth-mode=server"
]}]'

Output:

deployment.apps/argo-server patched

Note that this mode is not recommended for production environments, as it creates a significant security risks. A more secure option is to use the client authentication mode, which require clients to provide their Kubernetes bearer token.

2. Configure Kubernetes Role-Based Access Control (RBAC) to grant Argo Admin-level permissions for managing resources within the argo namespace.

$ kubectl create rolebinding argo-default-admin --clusterrole=admin --serviceaccount=argo:default -n argo

3. Forward the Argo server web interface port with kubectl port-forward.

$ kubectl -n argo port-forward deployment/argo-server 2746:2746

Using a browser like Chrome, visit the IP Address http://localhost:2746.

Create a New Workflow

You can use a YAML manifest to define Agro Workflows and apply to your Kubernetes cluster.

1. Create a new Workflow file.

Nano argo-workflow.yaml

2. Add the following to the file:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
    name: demo-workflow
spec:
    entrypoint: main
    templates:
    - name: main
    container:
        image: busybox
        command: ["/bin/sh"]
        args: ["-c", "echo 'The first step of the Workflow'"]

Here is a quick breakdown of the components of the file:

• entrypoint specifies the entry point for the workflow, which is defined as main.

• templates contains a list of templates, which define the steps or tasks within the workflow.

• name is the name of the template, which is set as main.

• container specifies a container-based step within the workflow.

• image specifies the Docker image to use for the container, set here as busybox.

• command specifies the command to be executed inside the container, set as ["/bin/sh"].

• args specifies the arguments to be passed to the command inside the container, set as ["-c", "echo 'The first step of the Workflow'"]. This command will run echo to print "The first step of the Workflow".

3. Apply the Workflow to your cluster:

$ kubectl -n argo create -f argo-workflow.yaml

Here's the output:

workflow.argoproj.io/hello-world-nb42c created

How to Manage Argo Workflows

1. To list all workflows within the argo namespace, do the following:

$ kubectl -n argo get wf

Here's the output:

NAME                      STATUS        AGE     MESSAGE
demo-workflow             Succeeded     4m23s

2. To see the pod logs for your Workflow, do the following:

$ kubectl -n argo logs demo-workflow

Here's the output:

This template is the first step of the Workflow
time="2024-02-13T19:56:54.629Z" level=info msg="sub-process exited" argo=true error="<nil>"

3. To delete a workflow, do this:

$ kubectl -n argo delete wf workflow-name

4. To suspend or resume a workflow, do this:

$ kubectl -n argo suspend wf workflow-name
$ kubectl -n argo resume wf workflow-name

5. To submit a workflow using the Argo CLI, do this:

$ argo submit -n argo workflow.yaml

You can learn more about Argo Workflows on their official documentation.

Conclusion

You have now explored Argo Workflows and successfully set it up. This powerful tool enables you to create logic using DAGs, or individual steps, helping you execute various tasks through different templates. You can also interact with your workflows and keep track of their progress by utilizing tools like Argo CLI, Argo UI, and Argo Events.

By using Argo Workflows, you can take advantage of Kubernetes scalability and flexibility to ensure reliable execution of tasks.

Luckily, there's Docker, a popular containerization technology. It works by packing up all the components needed to run the application and then executing the application in an isolated environment. This helps tackle the problem of "it works on my machine but not yours."

You just have to save the application in the Docker registry so that it can be downloaded and used by other developers. In this article, you will learn how to build a web application Docker image and push it to an Azure private registry.

Table of Contents

• How to Get Started
• How to Build a Docker image
• How to Run and Test a New Container Locally from the Image
• How to Push the Image to Azure Registry
• Clean Up
• Summary

How to Get Started

Before attempting to follow along with this tutorial, you should have the following:

• Docker installed – You can install Docker from the Docker website.
• An Azure Subscription – You can sign up for a free Azure account if you don’t already have one.
• Azure CLI – This tutorial requires the Azure command-line interface (CLI) to interact with the Azure container registry.
• Basic understanding of Docker commands and containers.

How to Build a Docker Image

A Docker image is a fundamental component and consists of a text file called a Dockerfile. This contains the application files and binaries that you wish to containerize.

In this guide, you will examine an example that lets you create a Docker image that has an Apache web server and a web application in order to learn how to create a Docker image.

How to Create a Docker image from a Dockerfile

You will first develop an HTML page that serves as your web application before writing a Dockerfile. As a result, you will create a new directory called appdocker and within it, create a new file called “index.html”. Add the following code to index.html:

<html>
    <body>
        <h1>Welcome to my web app</h1>
        <p>This page demo how to create, build and push a dockerized application to azure container registrar, (ACR) </p>

    </body>
</html>

You should now create a Dockerfile (without an extension) in the same directory with the content listed below to build your web image, which we will cover in more detail next.

FROM httpd:latest
LABEL Owner 'Destiny Erhabor'
COPY index.html /usr/local/apache2/htdocs/

• Every Docker image is created from another Docker image. The FROM httpd:latest at the start of the Dockerfile is necessary and indicates the Apache base image you will use for your own.
• The LABEL Owner 'Destiny Erhabor' gives you an option to specify the image owner. In this case you should use your name.
• The image creation procedure is then carried out using the COPY index.html /etc/nginx/nginx.conf instruction. The local index.html page that you created is copied by Docker into the image's /usr/local/apache2/htdocs/ directory.

Now, to build the Docker image, open a terminal to where the Dockerfile is located in the folder “appdocker”, and then run the docker build command:

docker build -t mywebapp:v1 .

docker build

You can also check list of the images created by executing the command:

docker images

docker images

The three steps of the Docker image builder are as follows, which you can see in the execution above:

1. The base image is downloaded by Docker in step one.
2. Docker copies the image's index.html file.
3. The image is created and tagged by Docker.

How to Run and Test a New Container Locally from the Image.

You will use the docker run command in your terminal to create a new container using your Docker image as follows:

docker run -d --name mywebapp -p 8080:80 mywebapp:v1

• The name of the container you desire is specified in the --name parameter.
• You define the desired port translation with the -p argument. In your example, this would entail that our local machine's port 8080 would be translated from port 80 of the container.
• The name of the image and its tag make up the command's final parameter.

You can test your container on your local system on the port translation that we previously performed. To achieve this, launch a browser and type http://localhost:8080.

testing image locally

How to Push the Image to Azure Registry

The primary repository for shared Docker images is called the Docker Registry. This registry can either be public, such as Docker Hub or private, such as Azure Container Registry.

Private Docker container images and other relevant artifacts can be stored and managed on the Microsoft-owned Azure Container Registry (ACR). These images can then be downloaded and used for local or hosting platform container-based deployments.

You'll need to go through the following steps to push your image into ACR:

First, you will launch the azure CLI command below in your terminal to connect to your Azure account:

az login

Next, you’ll use the Azure CLI to create a resource group, a container that manages multiple Azure resources using az group create and the azure container registry using the az acr commands:

az group create --name RG-ACR --location eastus

The --name tag specifies the resource group name as RG-ACR and --location gives the region where it is located as eastus.

Resource group

az acr create --resource-group RG-ACR --name mywebappacrdemo --sku Basic

The resource group you created earlier is required to manage the acr resource – hence the --resource-group tag. The --name specifies the name of the registry (the name must be globally unique) and the --sku provides the several service tiers.

These tiers offer a range of options for the capacity and use of your private Docker registry together with predictable pricing. Here you are using Basic tiers.

ACR resources creation

Now that you are logged in, use the command to establish a connection to the ACR resource you created above by passing the --name parameter as the name of that resource:

az acr login --name mywebappacrdemo

When the command is successful, a Login Succeeded message is returned.

Finally, run a few instructions to push the mywebapp Docker image into your ACR resource.

The initial action is to tag the image with the fully qualified name of the login server for the registry. The format of the fully qualified name is . azurecr.io, as show below:

docker tag mywebapp:v1 mywebappacrdemo.azurecr.io/mywebapp:v1

Now push the image into the ACR resource.

docker push mywebappacrdemo.azurecr.io/mywebapp:v1

ACR resource in azure portal

Clean Up

To avoid paying for the underlying Azure cost, you should clean up your resources if they're not being used.

To clean up your Azure resources, in the Azure portal go to or search for resources group, find the one you just created, and delete it. This will delete all resources in the resource group.

Summary

In this tutorial, we looked at the primary tasks that go into building and running a Docker image for a web application.

You also learned how to push a Docker image to the Azure container registry using the Azure CLI.

As always, I hope you enjoyed the article and learned something new. If you want, you can follow me on LinkedIn or Twitter.

The Docker logo actually symbolizes software that brings together a huge amount of organized information, hinting at its convenience.

Many enterprise applications use Docker-based deployments (also called containerized deployments) these days. So it's a good skill to have as a developer.

This tutorial will be the best fit for those who know nothing apart from the term "Docker".

In this article, you'll learn about the fundamentals of Docker, build your own Docker image, and publish it to the Docker Hub.

Why Do You Need Docker?

Let's understand why we need Docker with a simple example.

Let's assume you're joining a new company and you have been assigned to an extremely huge project to work on. You received a brand new laptop and are ready to put on your development shoes.

The first step whenever you are onboarded into a project is to set up the development environment. Since it's a huge enterprise project, it consumes a bunch of time to set up the development environment. You may need to install project-specific dependencies, tools, and many more.

In the middle, you may face errors that you can mostly solve by following the README guide, but a couple of them might happen for the first time (might happen due to laptop configuration) and you have to solve them on your own.

Imagine if you have to follow the same process for large teams with tons of members. It would be horrible to handle right?

This is where Docker comes in really handy. Docker will create containerized applications that run on any type of environment that have all the dependencies within it. So setting up the development environment is just one command away. Docker has many use cases – this scenario is just one of them.

By using Docker we can standardize application operations, ship code faster, and seamlessly deploy to production.

How Does Docker Work?

Docker provides a standardized way to develop, test, and deploy code.

You can think about Docker as a super-powered advanced virtual machine. Let’s learn more about it with an example.

Comparison between Virtual Machine and Docker

Before getting started to work on Docker, we should understand some fundamentals of how it works. They are:

1. Docker Engine
2. Docker Container
3. Docker Image

What is Docker Engine?

The Docker engine is an open-source containerization technology you can use to build a containerizing application. W

e can use the Docker engine on a variety of platforms through Docker Desktop or Binary Installation. To simplify, the software that hosts the containers is called Docker Engine.

What is a Docker Container?

A container is a standard unit of software that packages up the code and all required dependencies needed to run an application on different platforms or computing environments.

Containers are fully isolated from the computing environment, so applications run quickly and reliably from one computing environment to another.

Docker containers are built from container images.

What is a Docker image?

A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application. It includes code, runtime, system tools, system libraries, and settings.

As we discussed, containers are built from images. Images become containers when they starts running on a Docker Engine.

Use Cases for Docker

Docker gives you the ability to run an application in any type of environment which is completely isolated from the current environment. This isolated environment is called a container.

This isolation and security allow us to run as many containers as we want in a host. Docker helps developers to work in a standardized environment using local containers which provide all the packages and dependencies to run an application.

Let's see a few use cases of Docker:

1. Developers can write code locally and share their work using Docker containers
2. You can use Docker to deploy your applications into a test/production environment and execute automated and manual tests
3. When developers need to fix something, they can easily make the changes and push the Docker image to the testing or production environment

Just like Git, we can use Docker when we're making changes to our projects. If we make any changes we can just push the Docker images and pull them into the host machine. No more changes need to be done in the host server.

Here are the deployment steps you'd go through if you're not using Docker:

1. Pull/clone the code from Git
2. Install dependencies, run migrations, and so on in the host machine.
3. Start the application

These steps have to be repeated on every server whether it is a testing or production environment.

Here is the deployment steps using Docker:

1. Pull the docker image (docker pull)
2. Run the container in the host machine (docker run)

What is Docker Hub?

Docker Hub is the largest community for Docker images. It allows you to share the container images among your team, or the Docker community. Docker Hub is sort of like GitHub. Here, Docker images reside instead of the project's code.

You can pull the open-source Docker images also. To use this you have to create an account in Docker hub.

Now that you know a bit about how Docker works and why it's useful, let's learn how to containerize an application.

How to Containerize a NodeJS Application using Docker

Prerequisites:

1. A Basic understanding of NodeJS Applications
2. Docker desktop application

You can install the Docker Desktop application by following their original documentation.

You can verify if Docker is installed on your machine by querying it's version like this:

docker -v

Find Docker Version

Let's start our implementation.

Instead of starting over from scratch, I have created a super simple Express API that exposes only one endpoint. I pushed the code to a public repo on GitHub. You can clone the repo by running the below command:

git clone https://github.com/5minslearn/node_with_docker.git

This is the project's structure:

Simple NodeJS Project Structure

I have created only one endpoint ("/") and calling it will return "Greeting from 5minslearn". I've simplified this as much as possible to allow us to focus more on Docker.

How to Create a Dockerfile

Now, you'll need to create a file named "Dockerfile" in the root directory. It is the default file name for the Docker engine. Paste the following code into the file:

FROM node:latest
WORKDIR /app
COPY . /app
RUN npm install
EXPOSE 8000
CMD ["npm","start"]

Let's understand these commands line by line.

Defining the parent image is the first step of Docker engine. You have to define the parent image from which you're building the project. In our case it's Node.

There are a lot of parent images available in Docker Hub. You have to define the Image Variant next to the parent image. I always prefer to use the latest node image.

FROM node:latest

The second step is to define the working directory in Docker. Let's define our working directory as the app directory.

WORKDIR /app

Copy the project to the app directory. Make sure you exclude the node_modules directory. We will see how to ignore files/folders in the upcoming steps.

COPY . /app

In the above command, the . indicates that all the files and directories are to be copied to the app folder.

The next step is to install the required dependencies with this command:

RUN npm install

RUN is an image build step. The state of the container after a RUN command will be committed to the container image. A Dockerfile can have many RUN steps that can layer on top of one another to build the image.

Expose the port in which application should run.

EXPOSE 8000

The EXPOSE instruction informs Docker that the container listens on the specified network ports at runtime.

Finally run the execution command:

CMD ts-node src/index.ts

CMD is the command the container executes by default when we launch the built image.

Tip: I recommend that you install the Docker extension if you're using VS Code. It'll help you with its solid suggestions.

VS Code Docker extension

How to Ignore Files So They're Not Copied into the Docker Container

You have to exclude the unwanted files from being copied into the container. The .dockerignore file helps you with that. It works like .gitignore for Git.

You can define all the files you have to ignore and don't want to copy.

Using .dockerignore to exclude the unwanted files from being copied to the container

Great! You've completed the Docker configuration. Let's run the application.

How to Build Docker Images

You can build a Docker image by running the docker build command. Here's the syntax for it:

docker build -t image_name:version_number .

image_name indicates the container image name and version_number indicates the image version. Did you notice the dot (.) at the end of the command? It indicates that the Docker image should be built from the current directory.

I decided to set the image name as node_with_docker. Remember that the image name has to be prefixed with the Docker hub username. I've created my Docker hub account with the aanandggs username. So, my image name is aanandggs/node_with_docker. You may choose to enter whatever you want.

docker build -t aanandggs/node_with_docker:0.0.1 .

Sample output of building docker image

Once you've built your Docker image, you'll be able to see it on your Docker Desktop.

Docker Image in Docker Desktop

How to Run the Docker Image

After we build the Docker image, the next step is to run it. When an image starts running on a Docker Engine, it'll become a container.

Let's run our image which we built in the previous step:

docker container run -d --name <name_of_app> -p <local_port>:<docker_port> <image_name>:<version>

docker container run -d --name docker_with_node -p 8000:8000 aanandggs/node_with_docker:0.0.1

Docker Command to create and run a container

Let's understand the above command.

We use the docker container run command to create and run a new container from an image.

The -d flag instructs the container to run in the background (detach). It prints the container id.

The --name parameter gives a name to our container.

We use the -p parameter to publish a container’s port to the host. It'll bind the port 8000 of your local machine with the port 8000 of the Docker container.

image_name:version indicates the image and its version that this container should run.

You can have a look at the running containers in the Docker Desktop app.

Running Container in Docker Desktop

The Dashboard shows that our app is running. Let's check the output on the browser by trying to access the "/" endpoint. Hit locahost:8000/ on your browser. You should see a message similar to the one in the below screenshot:

NodeJS App running output

You have successfully run an app with a Docker container.

Note: You can bind any local port to the Docker port. Ensure your local port is not used by any other process.

How to Push the Image to the Docker Hub

Create your profile on Docker Hub. Come back to terminal and run the below command to login to Docker CLI:

docker login

If you face any issues with login, follow this doc to login to the Dockerhub on your machine.

After a successful login, you can push the image to Docker hub.

docker push <docker_image>:<image_version>

docker push aanandggs/node_with_docker:0.0.1

Sample output of pushing image to docker

Hurray! Docker images are uploaded to Docker hub.

You'll be able to see them in the Docker hub console.

Pushed Docker image in DockerHub

Since our image is public, anyone on the internet can pull the image and run it on their machines without any third-party installation.

Conclusion

In this article, we have learnt the very basics of Docker. Docker is an ocean of information. To fully learn it, you'll need to do more than just reading – you'll need to practice working with it.

Hope you folks enjoyed reading this one. Will see you in another interesting tutorial. Feel free to reach out to me on LinkedIn if you have any queries.

To learn more about Docker, subscribe to my email newsletter on my site (https://5minslearn.gogosoon.com) and follow me on social media.

Both virtual machines and containers help replicate the development environment, and manage dependencies and configurations better. But there are certain differences you should be aware of that will help you choose a VM or a Docker container depending on the application.

Over the next few minutes, we'll go over how virtual machines and Docker containers work, and then summarize the key differences between the two.

Let's begin!

Challenges in Application Development and Deployment

When you work as part of a development team, each application requires installation of multiple third-party software and packages. In order to collaborate and work together, every developer on the team should configure their local development environment.

However, setting up the development environment is a tedious process. The installation steps can be potentially different depending on the operating system and system configuration. Even during deployment, you have to configure the same environment on the server.

Different applications also require multiple versions of a specific software, say, PostgreSQL. In such cases, managing dependencies across applications becomes difficult.

To address the above challenges, it really helps if the applications run in isolated environments that you can replicate easily—independent of the system configuration. Both Virtual Machines (VMs) and Docker containers help you achieve this. Let's learn how!

How Does a Virtual Machine Work?

A Virtual Machine or VM is the emulation of a physical computer inside a host machine.

How a VM works (image by the author)

Running on top of the host operating system is a piece of software called a hypervisor that controls the VM instances. Each VM instance has its own guest operating system. The applications run inside this isolated environment.

You can have multiple VMs, each running a different application on a different operating system.

How Does a Docker Container Work?

Recently, container technology has revolutionized the software development process and the way development and operation teams work together. With time, Docker has become the go-to choice for containerizing applications.

Dockers containers are analogous to physical containers that you can use to store, package, and transport goods. But instead of tangible goods, they’re containers for software applications. 🙂

A docker container is a portable unit of software—that has the application—along with the associated dependency and configuration.

How containers work (image by the author)

Unlike a VM, Docker containers do not boot up their own guest OS. Rather, they run on top of the host operating system. This is facilitated by a container engine.

Docker vs VM – A Comprehensive Comparison

1️⃣ Virtualization

From our understanding thus far, both virtual machines and Docker containers provide isolated environments to run applications. The key difference between the two is in how they facilitate this isolation.

Recall that a VM boots up its own guest OS. Therefore, it virtualizes both the operating system kernel and the application layer.

A Docker container virtualizes only the application layer, and runs on top of the host operating system.

Container vs VM (image by the author)

2️⃣ Compatibility

A virtual machine uses its own operating system and is independent of the host operating system that it’s running on. Therefore, a VM is compatible with all operating systems.

A Docker container, on the other hand, is compatible with any Linux distribution. You may run into some problems running Docker on a Windows machine or an older Mac.

3️⃣ Size

A Docker image is lightweight and is typically in the order of kilobytes.

💡 Note: A Docker image denotes the artifact containing the application, its associated dependencies, and configuration. A running instance of the Docker image is called a container.

A VM instance can be as large as a few gigabytes or even terabytes.

4️⃣ Performance

In terms of performance, Docker containers provide near-native performance. Because they are lightweight, you can start them in a few milliseconds.

Starting a VM is equivalent to setting up a standalone machine inside your computer. It can take as long as a few minutes to start a VM instance.

5️⃣ Security

Docker containers run on top of the host operating system. Therefore, if the host OS is susceptible to security vulnerabilities, so are the Docker containers.

Virtual machines, on the other hand, boot up their own operating system, and are more secure. Recall: each virtual machine is a fully blown machine running inside another. If you have stringent security constraints to be met for sensitive applications, you should consider using a virtual machine instead.

6️⃣ Replicability

The next factor we'll consider is the ease with which you can replicate the isolated environments provided by VMs and containers. We can infer the ease of replicability from our earlier discussions on size and performance.

When there are multiple applications, each of which should run on a VM instance, using VMs can be inefficient and resource intensive. Docker containers, by virtue of being lightweight and performant, are preferred when you need to run multiple applications. ✅

Summing Up

I hope this tutorial helped you understand how Docker containers and VMs work, and the key differences between the two.

Here's a summary of what you've learned:

Thank you for reading this far. See you all soon in another tutorial! 😄

If you’re a cloud-based entrepreneur, you should have a Kubernetes platform for your enterprise. But adopting this six-year-old technology can be tough if there are gaps in your operations.

Improving the operational quality of your Kubernetes cluster is important in Kubernetes adoption. This article can help you achieve a secure, efficient, and reliable Kubernetes environment.

But first, you must understand what Kubernetes is. Read on to get important tips for your Kubernetes journey.

What is Kubernetes?

Kubernetes is an extensible, open-source platform built to manage containerized workloads and services. Google designed the technology, and the Cloud Native Computing Foundation maintains it.

Some of the features of the platform include:

• Storage orchestration
• Service discovery and load balancing
• Electronic rollouts and rollbacks
• Automatic bin packing
• Secret and configuration management

Don Foster, VP at Commvault, highlighted the importance of Kubernetes in digitalization. He outlined the value of containers to modern firms in an Executive Q&A session with TDWI.

According to Foster, containers introduce applications that can deploy new digital services. DevOps can work on apps without worrying whether they'll work in other environments.

He added that containers also speed up companies' digital transformation efforts.

To promote the use of the platform, Fairwinds developed the Kubernetes Maturity Model. Fairwinds is a Kubernetes enablement company.

What is the Kubernetes Maturity Model?

This model determines what stage you are in along your journey towards becoming cloud-native. It has seven phases:

• (Phase 1) Preparation
• (Phase 2) Transformation
• (Phase 3) Deployment
• (Phase 4) Confidence Building
• (Phase 5) Operational Improvement
• (Phase 6) Measurement and Control
• (Phase 7) Optimization and Automation

All these phases are important in Kubernetes adoption. But, you must pay special attention to possible issues during the fifth phase.

During this operation improvement phase, you may struggle to see the areas where you need to improve. Read on to know how to make your platform more secure, efficient, and reliable.

How to Achieve Kubernetes Security, Efficiency, and Reliability

After the confidence-building phase, the confidence may make you feel too reassured. This may cause a sense of false optimization to permeate throughout your organization.

You may get too comfortable with the technicalities and terminologies. You may miss issues that need tweaking and fortifications.

After building confidence, you must remember to conduct meticulous monitoring. Check for potential issues, responsibilities, and expectations.

How to Make Kubernetes Secure

Below are some points to remember to help you develop a secure Kubernetes environment.

Remember to follow standard Kubernetes security procedures. This includes setting limits for individual paths and hostnames against the following:

• The number of simultaneous connections for IP addresses
• The number of requests allowed for every user over a specified period
• Request body sizes

You must also make sure that you’re keeping your enterprise’s secrets a secret. You should encrypt confidential info, such as passwords and encryption keys. You must encrypt these data before you put them into the infrastructure repository.

You should also have a point person for cluster security and management. The point person will be responsible for enforcing Kubernetes security measures.

You also need to identify the officer or staff responsible for securing your Kubernetes platform. You must have a written list of obligations and expectations for the security staff.

A security point person must be well-versed when it comes to finding misconfigurations. This can prevent security gaps in the implementation of Kubernetes.

They must also be able to spot problems that create vulnerabilities in the system.

How to Make Kubernetes Efficient

Your Kubernetes environment isn't mature until your containers operate efficiently.

In order to make this happen, you need to assign someone to take charge of the monitoring of resources. They must be well-versed in the Kubernetes best practices for efficient resource usage.

The resource staff must clarify the scope of applications or services. They must always apportion the right amount of resources.

Always check if you're under-provisioning or over-provisioning resources. Neither is acceptable.

Finding the right values for efficient resource provisioning can be tricky. But, you can always use tools that can speed up the process of finding the best values to use.

Other things to keep in mind to ensure Kubernetes cluster efficiency include:

• Using the latest version of Kubernetes
• Employing namespaces to achieve isolation for those trying to access the same clusters
• Using small container images
• Taking advantage of the Kubernetes Check Probes to prevent pod failures
• Autoscaling
• Tracking control plane components
• Instituting Git-based workflow setups
• Monitoring high disk usage
• Policy log auditing

How to Make Kubernetes Reliable

You must have the right configurations to ensure stable operation and streamlined development of your Kubernetes environment.

Misconfigurations are not uncommon in Kubernetes. Below is a guide with helpful tips to prevent misconfigurations.

• Prefer ephemerality

You should use cloud-native architecture to adopt the ephemeral nature of Kubernetes.

• Take advantage of redundancy

Kubernetes supports redundancy to prevent single points of failure. You can use node or anti-affinity selection to achieve high availability.

• Use traffic readiness checks

You must take advantage of the platform’s liveness and readiness probes. These probes can regulate the appropriate sending of traffic to app containers.

Kubernetes, by default, sends traffic to containers immediately. This can become a problem when application pods aren't ready to accept traffic.

Misconfigurations can result in downtime challenges or the possibility of applications becoming unresponsive.

You should take advantage of the platform’s self-healing and auto-scaling functions.

You can also consider using a Kubernetes-native troubleshooting solution. This can help you address problems faster and prevent other reliability issues.

Recap

The adoption of Kubernetes technology can be a challenging feat for some entrepreneurs. Without the right skills, you can experience issues caused by gaps in operations.

But, there are steps you can take to ensure that your Kubernetes platform is working well.

You must make sure to follow standard Kubernetes security procedures. Hire a point person to be responsible for cluster security and management.

You can use tools to avoid overprovisioning or underprovisioning. Remember to keep in mind the importance of finding the optimal value of resources to use.

Take advantage of the Kubernetes Check Probes to promote improved efficiency and reliability.

You can also use a Kubernetes-native troubleshooting solution. Using this allows you to address issues faster and prevent reliability problems.

You can always fix issues in your operations in the later phase of Kubernetes maturity. But, remember that doing so could have repercussions to your enterprise in the long run.

Containers have exploded in popularity in recent years. And as more developers are using these containers, they need more tools to manage them and their interactions with them effectively.

To help manage all this, many devs use Kubernetes. And it has become the de facto standard for container orchestration.

While containers help make the software development and deployment lifecycle more efficient, they can also expand the possible ways hackers can attack your organization.

And while Kubernetes definitely simplifies the process of managing containers, it too has security vulnerabilities.

Given the popularity of Kubernetes, cybercriminals have put a great deal of effort into exploiting those vulnerabilities. As a result, attacks on the supply chain have risen significantly over the last year.

And so securing the Kubernetes supply chain is a high priority for many organizations.

One important security feature that you should pay close attention to if you're a Kubernetes user is dynamic admission control.

In this article, we'll discuss Kubernetes supply chain vulnerabilities and how to address them with dynamic admission control.

Vulnerabilities in the Kubernetes Supply Chain

Recently, nearly all Kubernetes users experienced a security incident caused by various vulnerabilities, ranging from misconfiguration to failed audits. Because of this, security concerns have started to take their toll on deployment practices.

More than half of organizations that use Kubernetes have delayed deploying an application solely based on security issues.

If you want to secure your Kubernetes supply chain, you should have an understanding of the supply chain components for containerized applications and their associated vulnerabilities.

The supply chain extends well beyond Kubernetes itself and includes the contents of the containers that Kubernetes manages as well as the container host.

Within the container, you'll typically have a bunch of code from a variety of sources (both internal and external). This gives attackers numerous opportunities to get creative.

To protect against these threats, you need to properly secure all code sets regardless of their source – and this can be challenging.

For example, securing code sourced from Linux distributions such as openssl libraries or glibc may only require you to apply the most recent patch.

But code from other external sources, such as upstream open-source libraries or internal development processes, can be more difficult to secure.

Internal development is perhaps the biggest organizational threat, particularly when developers prioritize speed of release over security.

Many organizations have decentralized responsibility for container security, with everyone from developers to DevOps getting involved. But more organizations are building DevSecOps units and placing Kubernetes security in their hands.

How to Secure the Kubernetes Supply Chain with Dynamic Admission Control

In Kubernetes, dynamic admission control involves user-defined or configured admission controllers rather than standard built-in controllers.

What are admission controllers?

Admission controllers take over after the Kubernetes API server receives an authentication and authorization of a request. These pieces of code intercept the request before a container gets initialized and is added as a pod to the Kubernetes cluster.

Essentially, the admission controller attempts to verify that the image is safe.

Image Source

Admission controllers implement a variety of functions, from enforcing resource quotas to running cluster-critical tasks. You'll need to have properly configured admission controllers to properly operate many of Kubernetes' advanced features.

What are admission webhooks?

Dynamic admission controllers rely on admission webhooks, which are user-defined HTTP callbacks that process admission requests.

In Kubernetes, there are two types of admission webhooks: validating admission webhooks and mutating admission webhooks.

In the admission control process, mutating admission controls run before validating controls. Both types of webhooks are essentially self-explanatory in principle, although their specific operation requires some explanation.

Validating admission webhooks

Validating admission webhooks intercept and validate requests to the Kubernetes API using an external webhook. Importantly, though, they can not mutate requests.

All validating webhooks matching a request run in parallel (because no potential modification can occur), and the controller rejects the request on the failure of any matching webhook.

Validation admission webhooks are all-or-nothing – if the request fails to match precisely, the control rejects it.

Mutating admission webhooks

In contrast, mutating admission webhooks are able to modify requests, allowing requests that are only slightly non-compliant with the rule to process.

If multiple webhooks match a request, they run serially, and each may modify the request. Because mutating controllers run first, mutated requests can still be rejected by validating webhooks.

The joint operation of mutating and validating admission webhooks allows Kubernetes developers to ensure that requests are compliant and valid before instantiation of containers.

Kubernetes Pod Security Policies

Pod security policies (or PSPs) are a Kubernetes security feature that relies on the implementation of admission controls.

PSPs set conditions and defaults that Kubernetes pods must meet in order to be accepted into the container system.

PSPs can enforce such policies as disabling privileged containers, preventing privilege escalation, and preventing containers from running as root.

PSPs allow admins to enforce organizational security policies across an entire namespace easily. While PSPs require enabling admission controllers, they must be separately enabled.

Under the Kubernetes Pod Security Standards, there are three types of policies:

• The Baseline Policy has minimal restrictions, although it does not allow privilege escalation.
• For the lowest trust users, there is the Restricted Policy which disables certain functions in accordance with pod hardening best practices.
• At the highest level is the Privileged Policy, which is the broadest, allowing the most permissions and privilege escalation.

Kubernetes started deprecating PSPs with the release of version 1.21. PSPs will be removed entirely in 2022 with the release of version 1.25. As a result, if you use Kubernetes you should carefully consider alternative security options for all future container applications.

Don’t Neglect Standard Kubernetes Security Tools

If you use containers, you should be aware of common vulnerabilities throughout your Kubernetes environment.

To achieve maximum security, both developers and users must apply Kubernetes-specific security features such as dynamic admission control and standard network security features like VPNs.

Image Source

A recent Kubernetes vulnerability involved attackers who had access to the API and who could obtain complete administrator access to a Kubernetes cluster and all associated resources.

A VPN can help avoid this and other similar vulnerabilities where the API server is exposed. But it's important to select the right VPN for your needs.

How to Choose a VPN

According to cybersecurity expert Ludovic Rembert of Privacy Canada, encryption protocols are the most important factor to look for in a VPN.

“A VPN protocol determines how your data gets routed between your machine and the server. Different protocols have different costs and benefits depending on what you need. For example, some prioritize privacy and security, while others prioritize speed….a PE Provider Edge device is a single device, or multiple devices, at the edge of the provider’s network. This device then connects through Consumer Edge devices. In this setup, users can view a website, while the provider device is only aware of the VPN device.” – Rembert

Conclusion

Containerized applications will continue to become more widely used in coming years, as will container management resources such as Kubernetes.

As the popularity of these tools increases, the number of attacks at all points in the supply chain will also increase.

So if you work with Kubernetes, you need to take advantage of every security resource available to ensure maximum application security and reliability.

And applying dynamic application controls to verify that the requests are compliant with security policies is one important step in this process.

Containers have become the de-facto way to develop applications nowadays. They provide a standard way to package all the dependencies that your application needs.

But how you deploy a containerized application to the cloud? The cloud offers scalability, elasticity, and a "pay for what you use" model that is very desirable in modern applications.

Let's imagine that you have developed your application and packaged it in a Docker container. This application can be your website, some software that you develop for your company, or anything really.

This point is where most basic container and Docker tutorials end. But you want to deploy your application somewhere so others can use it. Now you'll have to start learning about Kubernetes and all the complex orchestration systems. They seem so complicated for just deploying a simple application.

You know that the cloud is the best place to deploy your application. Yet, most of the cloud services out there are complex.

To use them, you need to know specific cloud concepts such as networking, instance types, and others. So many challenges, and you just want to deploy a simple web application.

What if I tell you that there is a cloud service you can use to deploy your containers in a simple way? A service that provides all the benefits from the cloud, and you don't need to use any complex orchestration system to manage the container workloads.

If this sounds interesting, keep on reading. In this post, I will introduce you to Amazon Lightsail. And then I will show you a demo on how to deploy a container application to AWS using Lightsail.

What is AWS?

AWS stands for Amazon Web Services and is the most widely-adopted cloud platform. It has lots of different services that help you develop and host your applications.

Using the cloud for your applications has many benefits over using your own on-premise servers. For example, it helps you lower the costs of your application, become more agile, and innovate faster.

What is Amazon Lightsail?

Amazon Lightsail is part of AWS's cloud-based offerings. It is a service that provides everything you need to deploy applications and websites to the cloud in a simple and cost effective way.

Even the pricing is made simpler – you know every month exactly what you are paying. Amazon Lightsail is an ideal way to deploy simple applications and websites and get started with AWS.

Lightsail is powered under the hood by AWS services such as virtual machines (Amazon EC2), relational databases (Amazon RDS), and other services. It offers the same level of scalability, reliability and security that you expect from any other AWS service.

At the end of 2020, Lightsail added support for deploying containers to the cloud. To do this, all you need is to provide a Docker image for your containers and Lightsail will automatically deploy it for you.

Lightsail provides an HTTPS endpoint that is ready to serve your application. It also takes care of load balancing and orchestrating the application.

How to Deploy an Application with Lightsail

Let's see how Lightsail works by deploying a simple NodeJS application packaged as a container image. This image is the one that Docker Desktop provides for learning their platform.

We will start this demo where most tutorials end – when your application image is hosted in Docker Hub.

Step 1 - Setup your AWS account

The first step in this tutorial is to get an AWS account. In this AWS account you will be deploying your containers.

If you are just creating your account, the free tier should be enough for this project. The free tier will give you access to a lot of AWS services for free for the first 12 months. And you will get one month of Amazon Lightsail for free.

Keep in mind that having an AWS account is free if you don’t use any services. You won’t be charged for creating the account, and if you don’t use the account, nothing will be charged.

To create an AWS account you can follow the steps in this video:

Step 2 - Create your container service

By now you should have an AWS account and your application in Docker Hub - as this Docker Desktop tutorial shows.

Log into your AWS account and go to Amazon Lightsail.

Amazon's Lightsail interface is quite different from the regular AWS interface. You can see that this interface has many tabs available. The one we are interested in for this post is Containers. But in a similar way you can create virtual instances, databases, and other cloud components using Lightsail.

Now we can create a container service, and that will take us to a form where we need to make some simple decisions.

The first one is in which region we want to deploy our container image. Amazon Lightsail is available in all these regions, so you can pick the one that is the best for your application.

After that, we need to choose how much power to give to the machine that will be running our containerized application. We need to decide on the size of the machine, and how many of them we need.

You can see how much it will cost per machine and how much memory and CPU each of them have. In addition, after you select the scale, you can see exactly how much this service will cost.

Lightsail is very clear when it comes to cost. We can see the end cost on the screen. That includes storage, load balancing, networking, and whatever this container needs to run.

Then we need to setup our deployment. The container service we are creating can hold up to 10 container's images.

For each of the containers, we need to define a name, where the image is (the URL from Docker Hub) and how we'll run and access this application.

In our case, we will open port 3000 with the protocol HTTP so the application can be accessed via that URL.

The last thing we need to configure is a public endpoint. You can choose from your deployment what container you want to make a public endpoint on the internet.

After that, you are ready to start the deployment. This takes a couple of minutes. After you are done you can access the public endpoint for this service.

At the top of the container service page, you can see the public domain. When you click that URL, you will be accessing the application you defined in the public endpoint.

If you need your containers to talk to each other without making them public, use the private domain.

Conclusion

Now you have a container application deployed in the cloud. This application is scalable. You can monitor the usage of the power you defined before in the Metrics tab. You can always modify these specs if you see that you need more or less power.

In addition, if you need a domain for your application, you can get one from the Amazon Lightsail console.

If you want to watch this blog post as a video you can check it out here where we do the same together. This video is also available in Spanish if you prefer.

Thanks for reading.

I’m Marcia Villalba, Developer Advocate for AWS and the host of a YouTube channel called FooBar. There I published over 300 video tutorials about Serverless, AWS and software engineer practices.

• Twitter: https://twitter.com/mavi888uy
• Youtube: https://youtube.com/foobar_codes
• Spanish Youtube channel: https://bit.ly/aws-esp-yt