exam - freeCodeCamp.org

How to Pass the Certified Kubernetes Security Specialist Exam – Cheat sheet and Study Guide

freeCodeCamp — Tue, 08 Mar 2022 17:40:39 +0000

By Faizan Bashir

This article is based on my experience studying for and passing the Certified Kubernetes Security Specialist exam. I passed the exam on my first attempt in Sep 2021.

I passed the Certified Kubernetes Application Developer exam back in Feb 2020, followed by Certified Kubernetes Administrator in March 2020.

The Certified Kubernetes Security Specialist or CKS exam was released around November, 2020, but I didn't have a chance to take that exam before Sep 2021.

As a bit of background information, I have been working with Kubernetes for the past 3 years almost on a day-to-day basis and that experience was an added advantage in helping me pass the CKS.

In this article, I'll share some resources that should help you study for and pass the exam, along with a helpful cheatsheet you can use while preparing. I'll also share some advice that should help you along the way.

What is Kubernetes?

Kubernetes is the most evolved and feature-rich Container Orchestration system out there, and it keeps getting better.

It has an enormous community to support, and it's always building new features and resolving issues. Kubernetes is certainly evolving at a breakneck pace, and it becomes a challenge to keep up with its pace of development. This makes it the best bet for a container orchestration solution.

Resources for the Exam

The following are a few awesome resources available on passing the CKS exam:

The courses for KodeKloud and Killer.sh provide mock exam simulators which are very helpful in preparing for the exam, and provide a pretty good idea of what the exam looks like. I strongly suggest enrolling in one or both courses.

Purchasing the exam from the Linux Foundation gives you 2 free attempts at the exam simulator from killer.sh. That way if you are well-versed with the contents of the curriculum you can skip the courses and directly go for the exam simulator provided with the exam.

The exam costs $375 but there are offers and deals available, and if you look for them you might be able to get a better price. The duration of the exam is 2 hours and is valid for 2 years, unlike the CKA and CKAD which are valid for 3 years.

Aliases

The CKS is a performance-based exam where you are provided with an exam simulator in which you have to work out the problems. You are allowed to open only one tab apart from the exam tab.

Since this exam requires you to write a lot of commands, I figured early on that I'd have to rely on aliases to reduce the number of keystrokes to save time.

I used the vi editor during the exam, so here I will share some useful tips for this editor.

vi defaults for ~/.vimrc:

vi ~/.vimrc
---
:set number
:set et
:set sw=2 ts=2 sts=2
---
^: Start of word in line
0: Start of line
$: End of line
w: End of word
GG: End of file

kubectl defaults for ~/.bashrc:

vi ~/.bashrc
---
alias k='kubectl'
alias kg='k get'
alias kd='k describe'
alias kl='k logs'
alias ke='k explain'
alias kr='k replace'
alias kc='k create'
alias kgp='k get po'
alias kgn='k get no'
alias kge='k get ev'
alias kex='k exec -it'
alias kgc='k config get-contexts'
alias ksn='k config set-context --current --namespace'
alias kuc='k config use-context'
alias krun='k run'
export do='--dry-run=client -oyaml'
export force='--grace-period=0 --force'

source <(kubectl completion bash)
source <(kubectl completion bash | sed 's/kubectl/k/g' )
complete -F __start_kubectl k


alias krp='k run test --image=busybox --restart=Never'
alias kuc='k config use-context'
---

Shortcuts

The kubectl get command provides short catchy names for accessing resources and like pvc for persistentstorageclaim. These can help save a lot of keystrokes and valuable time during the exam.

po for pods
rs for replicasets
deploy for deployments
svc for services
ns for namespace
netpol for networkpolicy
pv for persistentstorage
pvc for persistentstorageclaim
sa for serviceaccounts

Kubernetes Cheat Sheet

kubectl run command

The kubectl run command provides a flag --restart which allows you to create different kinds of Kubernetes objects from a Deployment to CronJob.

The below snippet shows the different options available for the --restart flag.

k run:
--restart=Always                             #Creates a deployment
--restart=Never                              #Creates a Pod
--restart=OnFailure                          #Creates a Job
--restart=OnFailure --schedule="*/1 * * * *" #Creates a CronJob

How to generate yaml spec from an existing pod

Sometimes it's easier to generate a spec from an existing pod and make changes to it than to create a new one from scratch. The kubectl get pod command provides us with the required flags to output the pod spec in the format we want.

kgp  -o wide

# Generating YAML Pod spec
kgp  -o yaml
kgp  -o yaml > .yaml

# Get a pod's YAML spec without cluster specific information
kgp my-pod -o yaml --export > .yaml

kubectl pod commands

The kubectl run command provides a lot of options, like specifying requests and the limits a pod is supposed to use or the commands a container should run once created.

# Output YAML for a nginx pod running an echo command
krun nginx --image=nginx --restart=Never --dry-run -o yaml -- /bin/sh -c 'echo Hello World!'
# Output YAML for a busybox pod running a sleep command
krun busybox --image=busybox:1.28 --restart=Never --dry-run -o yaml -- /bin/sh -c 'while true; do echo sleep; sleep 10; done'
# Run a pod with set requests and limits
krun nginx --image=nginx --restart=Never --requests='cpu=100m,memory=512Mi' --limits='cpu=300m,memory=1Gi'
# Delete pod without delay
k delete po busybox --grace-period=0 --force

How to print logs and export them

Logs are the fundamental source of information when it comes to debugging an application. The kubectl logs command provides the functionality to check the logs of a given pod. You can use the below commands to check the logs of a given pod.

kubectl logs deploy/
kubectl logs deployment/
#Follow logs
kubectl logs deploy/ --tail 1 --follow

Apart from just looking at logs, we can also export logs to a file for further debugging of sharing the same with anyone.

kubectl logs  --namespace  > /path/to/file.format

How to create config maps and secrets

The kubectl create command lets us create ConfigMaps and Secrets from the command line. We can also use the YAML file to create the same resources and by using kubectl apply -f we can apply the commands.

kc cm my-cm --from-literal=APP_ENV=dev
kc cm my-cm --from-file=test.txt
kc cm my-cm --from-env-file=config.env

kc secret generic my-secret --from-literal=APP_SECRET=sdcdcsdcsdcsdc
kc secret generic my-secret --from-file=secret.txt
kc secret generic my-secret --from-env-file=secret.env

Helpful commands for debugging

Debugging is a very important skill when you're facing issues and errors both in our day jobs and while solving problems in the CKS exam.

Apart from the ability to output logs from a container, the kubectl exec commands lets you log in to a running container and debug issues. While inside the container you can also use utilities like nc and nslookup to diagnose networking-related issues.

# Run busybox container
k run busybox --image=busybox:1.28 --rm --restart=Never -it sh
# Connect to a specific container in a Pod
k exec -it busybox -c busybox2 -- /bin/sh
# adding limits and requests in command
kubectl run nginx --image=nginx --restart=Never --requests='cpu=100m,memory=256Mi' --limits='cpu=200m,memory=512Mi'
# Create a Pod with a service
kubectl run nginx --image=nginx --restart=Never --port=80 --expose
# Check port
nc -z -v -w 2  
# NSLookup
nslookup 
nslookup 10-32-0-10.default.pod

Rolling updates and rollouts

The kubectl rollout command provides the ability to check for the status of updates and, if required, roll back to a previous version.

k set image deploy/nginx nginx=nginx:1.17.0 --record
k rollout status deploy/nginx
k rollout history deploy/nginx
# Rollback to previous version
k rollout undo deploy/nginx
# Rollback to revision number
k rollout undo deploy/nginx --to-revision=2
k rollout pause deploy/nginx
k rollout resume deploy/nginx
k rollout restart deploy/nginx
kubectl run nginx-deploy --image=nginx:1.16 --replias=1 --record

Scale and autoscale command

The kubectl scale command provides the functionality to scale up or scale down pods in a given deployment.

Using the kubectl autoscale command we can define the minimum number of pods that should be running for a given deployment and the maximum numbers of pods the deployment can scale to along with the scaling criteria like CPU percentage.

k scale deploy/nginx --replicas=6
k autoscale deploy/nginx --min=3 --max=9 --cpu-percent=80

Network policy

In a Kubernetes cluster, all pods can communicate with all pods by default, which can be a security issue in some implementations.

To get around this issue, Kubernetes introduced Network Policies to allow or deny traffic to and from pods based on pod labels which are part of the pod spec.

The below example denies both the Ingress and Egress traffic for pods running in all namespaces.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: example
  namespace: default
spec:
  podSelector: {}
  policyTypes:
  - Egress
  - Ingress

The below example denies both the Ingress and Egress traffic for pods running in all namespaces. But it allows access to DNS resolution services running on port 53.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny
  namespace: default
spec:
  podSelector: {}
  policyTypes:
  - Egress
  - Ingress
  egress:
  - to:
    ports:
      - port: 53
        protocol: TCP
      - port: 53
        protocol: UDP

The below example denies Egress access to the metadata server running on IP address 169.256.169.256 in AWS EC2 Instances.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name:cloud-metadata-deny
  namespace: default
spec:
  podSelector: {}
  policyTypes:
  - Egress
  egress:
  - to:
      - ipBlock: 
          cidr: 0.0.0.0/0
          except:
          - 169.256.169.256/32

The below example allows Egress access to the metadata server running on IP address 169.256.169.256 in AWS EC2 Instances.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: cloud-metadata-accessor
  namespace: default
spec:
  podSelector:
    matchLabels:
      role: metadata-accessor
  policyTypes:
  - Egress
  egress:
  - to:
    - ipBlock:
        cidr: 169.256.169.256/32

Static analysis using Kubesec

Kubesec is a Static Analysis tool for analyzing the YAML files to find issues with the files.

kubesec scan pod.yaml

# Using online kubesec API
curl -sSX POST --data-binary @pod.yaml https://v2.kubesec.io/scan

# Running the API locally
kubesec http 8080 &

kubesec scan pod.yaml -o pod_report.json -o json

Vulnerability scanning using Trivvy

Trivvy is a Vulnerability Scanning tool that scans container images for security issues.

trivy image nginx:1.18.0
trivy image --severity CRITICAL nginx:1.18.0
trivy image --severity CRITICAL, HIGH nginx:1.18.0
trivy image --ignore-unfixed nginx:1.18.0

# Scanning image tarball
docker save nginx:1.18.0 > nginx.tar
trivy image --input archive.tar

# Scan and output results to file
trivy image --output python_alpine.txt python:3.10.0a4-alpine
trivy image --severity HIGH --output /root/python.txt python:3.10.0a4-alpine

# Scan image tarball
trivy image --input alpine.tar --format json --output /root/alpine.json

How to remove unwanted services

The systemctl exposes the capabilities to start, stop, enable, disable and list services running on a Linux Virtual Machine.

List services:

systemctl list-units --type service

Stop Service:

systemctl stop apache2

Disable Service:

systemctl disable apache2

Remove Service:

apt remove apache2

Runtime classes

Kubernetes introduced the RuntimeClass feature in version v1.12 for selecting the container runtime configuration. The container runtime configuration is used to run a pod's underlying containers.

Most Kubernetes clusters use the dockershim as the Runtime class for the running containers, but you can use different container Runtimes.

The dockershim has been deprecated in Kubernetes version v1.20, and will be removed in v1.24.

How to create a Runtime Class:

apiversion: node.k8s.io/v1beta1
kind: RuntimeClass
metadata:
  name: gvisor
handler: runsc

How to use a runtime class for any given pod:

apiVersion: v1
kind: Pod
metadata:
  labels:
    run: nginx
  name: nginx
spec:
  runtimeClassName: gvisor
  containers:
  - name: nginx
    image: nginx

RBAC commands

In Kubernetes,

Role-based access control (RBAC) commands provide a method of regulating access to Kubernetes resources based on the roles of individual users or service accounts. (Source)

Here's how to create a role:

kubectl create role developer --resource=pods --verb=create,list,get,update,delete --namespace=development

How to create a role binding:

kubectl create rolebinding developer-role-binding --role=developer --user=faizan --namespace=development

How to validate:

kubectl auth can-i update pods --namespace=development --as=faizan

How to create a cluster role:

kubectl create clusterrole pvviewer-role --resource=persistentvolumes --verb=list

And how to create a Clusterrole Binding association with a service account:

kubectl create clusterrolebinding pvviewer-role-binding --clusterrole=pvviewer-role --serviceaccount=default:pvviewer

Cluster maintenance

You use the kubectl drain command to remove all running workloads (pods) from a given Node.

You use the kubectl cordon command to cordon a node to mark it as schedulable.

Ands you use the kubectl uncordon command to set the node as schedulable, meaning the Controller Manager can schedule new pods to the given node.

How to drain a node of all pods:

kubectl drain node-1

How to drain a node and ignore daemonsets:

kubectl drain node01 --ignore-daemonsets

How to force drain:

kubectl drain node02 --ignore-daemonsets --force

How to mark a node un-schedulable, so that no new pods can be scheduled on this node:

kubectl cordon node-1

Mark a node schedulable

kubectl uncordon node-1

CKS Exam Tips

The Kubernetes kubectl get command provides the user with an output flag, -o or --output, which helps us format the output in the form of JSON, yaml, wide, or custom-columns.

JSON and JSONPath

How to output the contents of all the pods in the form of a JSON Object:

kubectl get pods -o json

The JSONPath outputs a specific key from the JSON Object:

kubectl get pods -o=jsonpath='{@}'
kubectl get pods -o=jsonpath='{.items[0]}'

The .items[*] is used where we have multiple objects, for instance multiple containers with a pod config:

# For list of items use .items[*]
k get pods -o 'jsonpath={.items[*].metadata.labels.version}'
# For single item
k get po busybox -o jsonpath='{.metadata}'
k get po busybox -o jsonpath="{['.metadata.name', '.metadata.namespace']}{'\n'}"

The command returns the internal IP of a Node using JSONPath:

kubectl get nodes -o=jsonpath='{.items[*].status.addresses[?(@.type=="InternalIP")].address}'

The command checks for equality on a specific key:

kubectl get pod api-stag-765797cf-lrd8q -o=jsonpath='{.spec.volumes[?(@.name=="api-data")].persistentVolumeClaim.claimName}'
kubectl get pod -o=jsonpath='{.items[*].spec.tolerations[?(@.effect=="NoSchedule")].key}'

Custom Columns are helpful in order to output specific fields:

kubectl get pods -o='custom-columns=PODS:.metadata.name,Images:.spec.containers[*].image'

CKS Exam Topics

The CKS exam covers topics related to security in the Kubernetes ecosystem. Kubernetes security is a vast topic to cover in one article, so this article contains some of the topics covered in the exam.

How to secure and harden container images

While designing container images to run your code, pay special attention to securing and hardening measures in order to prevent hacks and privilege escalation attacks. Keep the below points in mind while building the container images:

Use specific package versions like alpine:3.13.
Don't run as root – use the USER to block root access.
Make filesystem read-only in the securityContext using readOnlyRootFilesystem: true
Remove shell access using RUN rm -rf /bin/*

How to minimise OS footprint

Conatiner layers

The instructions RUN, COPY, and ADD create container layers. Other instructions create temporary intermediate images and do not increase the size of the build. Instructions that create layers add to the size of the resulting image.

A typical Dockerfile looks like the one given below. It adds a single layer using the RUN instruction.

FROM ubuntu

RUN apt-get update && apt-get install -y golang-go

CMD ["sh"]

Multi-stage builds

Multi-Stage builds leverage multiple FROM statements in the Dockerfile. The FROM instruction marks a new stage in the build. It combines multiple FROM statements allow to leverage from the previous build in order to selectively copy binaries over to the new build stage omitting the unnecessary binaries. The resulting Docker image is considerably smaller in size with a drastically reduced attack surface.

FROM ubuntu:20.04 AS build
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y golang-go
COPY app.go .
RUN CGO_ENABLED=0 go build app.go

FROM alpine:3.13
RUN chmod a-w /etc
RUN addgroup -S appgroup && adduser -S appuser -G appgroup -h /home/appuser
RUN rm -rf /bin/*
COPY --from=build /app /home/appuser/
USER appuser
CMD ["/home/appuser/app"]

How to limit node access

Access Control files contain sensitive information about users/groups in the Linux OS.

#Stores information about the UID/GID, user shell, and home directory for a user
/etc/passwd
#Stores the user password in a hashed format
/etc/shadow
#Stores information about the group a user belongs
/etc/group
#Stored information about the Sudoers present in the system
/etc/sudoers

Disabling a user account helps in securing access to a Node by disabling login to a given user account.

usermod -s /bin/nologin

Disabling the root user account is of special significance, as the root account has all the capabilities.

usermod -s /bin/nologin root

Here's how to add a user with a home directory and shell:

adduser --home /opt/faizanbashir --shell /bin/bash --uid 2328 --ingroup admin faizanbashir
useradd -d /opt/faizanbashir -s /bin/bash -G admin -u 2328 faizanbashir

How to delete the user account:

userdel

How to delete a group:

groupdel

How to add a user to a group:

adduser  <groupname>

How to remove a user from a group:

#deluser faizanbashir admin
deluser  <groupname>

How to set a password for a user:

passwd

How to elevate a user to sudoer:

vim /etc/sudoers
>>>
faizanbashir ALL=(ALL:ALL) ALL

How to enable sudo with no password:

vim /etc/sudoers
>>>
faizanbashir ALL=(ALL) NOPASSWD:ALL

visudo
usermod -aG sudo faizanbashir
usermod faizanbashir -G admin

SSH hardening

How to disable SSH

The configuration given in the /etc/ssh/sshd_config can be leveraged to secure SSH access to Linux nodes. Setting the PermitRootLogin to no disables the root login on a node.

To enforce using a key to login and disabling login using passwords to nodes, you can set the PasswordAuthentication to no.

vim /etc/ssh/sshd_config
>>
PermitRootLogin no
PasswordAuthentication no
<<
# Restart SSHD Service
systemctl restart sshd

How to set no login for the root user:

usermod -s /bin/nologin root

SSH Copy user key / Passwordless SSH:

ssh-copy-id -i ~/.ssh/id_rsa.pub faizanbashir@node01
ssh faizanbashir@node01

How to remove obsolete packages and services

Here's how you can list all services running on an Ubuntu machine:

systemctl list-units --type service
systemctl list-units --type service --state running

How to stop, disable, and remove a service:

systemctl stop apache2
systemctl disable apache2
apt remove apache2

How to restrict kernel modules

In Linux, Kernel modules are pieces of code that can be loaded and unloaded into the kernel upon demand. They extend the functionality of the kernel without the need to reboot the system. A module can be configured as built-in or loadable.

How to list all Kernel Modules:

lsmod

How to manually load modules into a Kernel:

modprobe pcspkr

How to blacklist a module: (Reference: CIS Benchmarks -> 3.4 Uncommon Network Protocols)

cat /etc/modprobe.d/blacklist.conf
>>>
blacklist sctp
blacklist dccp

# Shutdown for changes to take effect
shutdown -r now

# Verify
lsmod | grep dccp

How to identify and disable open ports

How to check for open ports:

netstat -an | grep -w LISTEN
netstat -natp | grep 9090

nc -zv  22
nc -zv  10-22

ufw deny 8080

How to check port usage:

/etc/services | grep -w 53

Here's the reference doc for a list of open ports.

How to restrict network access

How to identity a service running on port:

systemctl status ssh
cat /etc/services | grep ssh
netstat -an | grep 22 | grep -w LISTEN

UFW firewall

Uncomplicated Fire Wall (UFW) is a tool for managing firewall rules in Arch Linux, Debian, or Ubuntu. UFW lets you allow and block traffic on a given port and from a given source.

Here's how to install UFW Firewall:

apt-get update
apt-get install ufw
systemctl enable ufw
systemctl start ufw
ufw status
ufw status numbered

How to allow all outbound and inbound connections:

ufw default allow outgoing
ufw default allow incoming

How to allow rules:

ufw allow 22
ufw allow 1000:2000/tcp
ufw allow from 172.16.238.5 to any port 22 proto tcp
ufw allow from 172.16.238.5 to any port 80 proto tcp
ufw allow from 172.16.100.0/28 to any port 80 proto tcp

How to deny rules:

ufw deny 8080

How to enable and activate the Firewall:

ufw enable

How to delete rules:

ufw delete deny 8080
ufw delete

How to reset rules:

ufw reset

Linux Syscalls

Linux Syscalls are used to make requests from user space into the Linux kernel. For instance, while creating a file, the userspace makes a request to the Linux Kernel to create the file.

Kernel Space has the following:

Kernel Code
Kernel Extensions
Device Drivers

How to trace Syscalls using Strace

Here's how you can trace syscalls using strace:

which strace
strace touch /tmp/error.log

How to get the PID of a service:

pidof sshd
strace -p

How to list all syscalls made during an operation:

strace -c touch /tmp/error.log

How to consolidate listing syscalls: (Count and summarise)

strace -cw ls /

How to follow a PID and consolidate:

strace -p 3502 -f -cw

AquaSec Tracee

AquaSec Tracee was created by Aqua Security which uses eBPF to trace events in containers. Tracee uses eBPF (Extended Berkeley Packet Filter) at runtime directly in the kernel space without interfering with the kernel source or loading any kernel modules.

Binary stored at /tmp/tracee
Needs access to the following, in read-only mode if run using a container with --privileged capability:
- /tmp/tracee -> Default workspace
- /lib/modules -> Kernel Headers
- /usr/src -> Kernel Headers

How to fun Tracee in a Docker container:

docker run --name tracee --rm --privileged --pid=host \
  -v /lib/modules/:/lib/modules/:ro -v /usr/src/:/usr/src/ro \
  -v /tmp/tracee:/tmp/tracee aquasec/tracee:0.4.0 --trace comm=ls

# List syscalls made by all the new process on the host
docker run --name tracee --rm --privileged --pid=host \
  -v /lib/modules/:/lib/modules/:ro -v /usr/src/:/usr/src/ro \
  -v /tmp/tracee:/tmp/tracee aquasec/tracee:0.4.0 --trace pid=new

# List syscalls made from any new container
docker run --name tracee --rm --privileged --pid=host \
  -v /lib/modules/:/lib/modules/:ro -v /usr/src/:/usr/src/ro \
  -v /tmp/tracee:/tmp/tracee aquasec/tracee:0.4.0 --trace container=new

How to restrict Syscalls with Seccomp

SECCOMP – Secure Computing Mode – is a Linux Kernel level feature that you can use to sandbox applications to only use the syscalls they need.

How to check support for seccomp:

grep -i seccomp /boot/config-$(uname -r)

How to test to change system time:

docker run -it --rm docker/whalesay /bin/sh
# date -s '19 APR 2013 22:00:00'

ps -ef

How to check seccomp status for any PID:

grep -i seccomp /proc/1/status

Seccomp modes:

Mode 0: Disabled
Mode 1: Strict
Mode 2: Filtered

The following configuration is used to whitelist syscalls. The whitelist profile is secure but syscalls have to be selectively enabled as it blocks all syscalls by default.

{
  "defaultAction": "SCMP_ACT_ERRNO",
  "architectures": [
    "SCMP_ARCH_X86_64",
    "SCMP_ARCH_X86",
    "SCMP_ARCH_X32"
  ],
  "syscalls": [
    {
      "names": [
        "",
        "",
        ""
      ],
      "action": "SCMP_ACT_ALLOW"
    }
  ]
}

The following configuration is used to blacklist syscalls. The blacklist profile has a greater attack surface than the whitelist.

{
  "defaultAction": "SCMP_ACT_ALLOW",
  "architectures": [
    "SCMP_ARCH_X86_64",
    "SCMP_ARCH_X86",
    "SCMP_ARCH_X32"
  ],
  "syscalls": [
    {
      "names": [
        "",
        "",
        ""
      ],
      "action": "SCMP_ACT_ERRNO"
    }
  ]
}

The Docker seccomp profile blocks 60 of the 300+ syscalls on the x86 architecture.

How to use seccomp profiles with Docker:

docker run -it --rm --security-opt seccomp=/root/custom.json docker/whalesay /bin/sh

How to allow all syscalls with the container:

docker run -it --rm --security-opt seccomp=unconfined docker/whalesay /bin/sh

# Verify
grep -i seccomp /proc/1/status

# Output should be:
Seccomp:         0

How to use Docker container to get container runtime related information:

docker run r.j3ss.co/amicontained amicontained

Seccomp in Kubernetes

Secure computing mode (SECCOMP) is a Linux kernel feature. You can use it to restrict the actions available within the container. Seccomp documentation

How to run amicontained in Kubernetes:

kubectl run amicontained --image r.j3ss.co/amicontained amicontained -- amicontained

As of version v1.20 Kubernetes does not implement seccomp by default.

Seccomp 'RuntimeDefault' docker profile in Kubernetes:

apiVersion: v1
kind: Pod
metadata:
  labels:
    run: amicontained
  name: amicontained
spec:
  securityContext:
    seccompProfile:
      type: RuntimeDefault
  containers:
  - args:
    - amicontained
    image: r.j3ss.co/amicontained
    name: amicontained
    securityContext:
      allowPrivilegeEscalation: false

Default seccomp location in kubelets

/var/lib/kubelet/seccomp

How to create a seccomp profile in node:

mkdir -p /var/lib/kubelet/seccomp/profiles

# Add a profile for audit
vim /var/lib/kubelet/seccomp/profiles/audit.json
>>>
{
  defaultAction: "SCMP_ACT_LOG"
}

# Add a profile for violations (Blocks all syscalls by default, will let nothing run)
vim /var/lib/kubelet/seccomp/profiles/violation.json
>>>
{
  defaultAction: "SCMP_ACT_ERRNO"
}

Local seccomp profile – this file should exist locally on a node to be able to work:

...
securityContext:
  seccompProfile:
    type: Localhost
    localhostProfile: profiles/audit.json
...

The above profile will enable syscalls to be saved to a file.

grep syscall /var/log/syslog

How to map syscall numbers to syscall name:

grep -w 35 /usr/include/asm/unistd_64.h

# OR
grep -w 35 /usr/include/asm-generic/unistd.h

AppArmor

AppArmor is a Linux security module that is used to confine a program to a limited set of resources.

How to install AppArmor utils:

apt-get install apparmor-utils

How to check if AppArmor is running and enabled:

systemctl status apparmor

cat /sys/module/apparmor/parameters/enabled
Y

The AppArmor profiles are stored at:

cat /etc/apparmor.d/root.add_data.sh

How to list AppArmor profiles:

cat /sys/kernel/security/apparmor/profiles

How to deny all file write profiles:

profile apparmor-deny-write flags=(attach_disconnected) {
  file,
  # Deny all file writes.
  deny /** w,
}

How to deny write to /proc files:

profile apparmor-deny-proc-write flags=(attach_disconnected) {
  file,
  # Deny all file writes.
  deny /proc/* w,
}

How to deny remount root FS:

profile apparmor-deny-remount-root flags=(attach_disconnected) {

  # Deny all file writes.
  deny mount options=(ro, remount) -> /,
}

How to check profile status:

aa-status

Profile load modes

Enforce, monitor and enforce the rules
Complain, will not enforce the rules but logs them as events
Unconfined, will not enforce or log events

How to check if a profile is valid:

apparmor_parser /etc/apparmor.d/root.add_data.sh

How to disable a profile:

apparmor_parser -R /etc/apparmor.d/root.add_data.sh
ln -s /etc/apparmor.d/root.add_data.sh /etc/apparmor.d/disable/

How to generate a profile and answer the series of questions that follow:

aa-genprof /root/add_data.sh

How to generate a profile for a command:

aa-genprof curl

How to disable profile from logs:

aa-logprof

How to use AppArmor in Kubernetes

To use AppArmor with Kubernetes, the following prerequisites must be met:

Kubernetes version should be greater than 1.4
AppArmor Kernel module should be enabled
AppArmor profile should be loaded in the kernel
Container runtime should be supported

Sample usage in Kubernetes:

apiVersion: v1
kind: Pod
metadata:
  name: ubuntu-sleeper
  annotations:
    container.apparmor.security.beta.kubernetes.io/: localhost/
spec:
  containers:
  - name: ubuntu-sleeper
    image: ubuntu
    command: ["sh", "-c", "echo 'Sleeping for an hour!' && sleep 1h"]

Note: The container should run in a node containing the AppArmor profile.

Linux capabilities

The Linux capabilities feature breaks up the privileges available to processes run as the root user into smaller groups of privileges. This way a process running with root privilege can be limited to get only the minimal permissions it needs to perform its operation.

Docker supports the Linux capabilities as part of the Docker run command: with --cap-add and --cap-drop. By default, a container is started with several capabilities that are allowed by default and can be dropped. Other permissions can be added manually.

Both --cap-add and --cap-drop support the ALL value, to allow or drop all capabilities. By default Docker containers run with 14 capabilities.

Kernel < 2.2
- Privileged Process
- Unprivileged Process
Kernel >= 2.2
- Privileged Process
  - CAP_CHOWN
  - CAP_SYS_TIME
  - CAP_SYS_BOOT
  - CAP_NET_ADMIN

Refer to this document for the full list of Linux Capabilities.

How to check what capabilities a command needs:

getcap /usr/bin/ping

How to get process capabilities:

getpcaps

How to add security capabilities:

apiVersion: v1
kind: Pod
metadata:
  name: ubuntu-sleeper
spec:
  containers:
  - name: ubuntu-sleeper
    image: ubuntu
    command: ["sleep", "1000"]
    securityContext:
      capabilities:
        add: ["SYS_TIME"]
        drop: ["CHOWN"]

How to Prepare for the Exam

CKS is considered a pretty tough exam. But based on my experience I think that, given good enough practice and if you understand the concepts the exam covers, it'll be pretty manageable within two hours.

You definitely need to understand the underlying Kubernetes concepts. And since a prerequisite for CKS is to pass the CKA exam, you should have a strong understanding of Kubernetes and how it functions before attempting the CKS.

In addition, to pass the CKS, you need to understand the threats and security implications introduced by container orchestration.

The introduction of the CKS exam is an indication that the security of containers should not be taken lightly. Security mechanisms should always be in place to thwart attacks on Kubernetes clusters.

The Tesla cryptocurrency hack that was thanks to an unprotected Kubernetes dashboard brings to light the risks associated with Kubernetes or any other container orchestration engine. Hackerone has a Kubernetes bounty page listing the source code repos used in a standard Kubernetes cluster.

Practice, Practice, and Practice!

Practice is the key to cracking the exam, I personally found that the exam simulators by KodeKloud and Killer.sh were immensely helpful for me.

I didn't have as much time to prepare for the CKS exam as I had for the CKA exam, but I was working on Kubernetes in my day job so I'd become really comfortable with it.

Practice is the key to success. Best of luck with the exam!

You can read this and other articles at faizanbashir.me

How I Passed the CompTIA Linux+ Exam

freeCodeCamp — Thu, 30 Apr 2020 10:01:32 +0000

By Clark Jason Ngo

Summary of this article:

The backstory: why CompTIA Linux+ exam.
Review of learning resources: what is good and what is not good.
Retrospective on studying: what worked and did not work.
My experience during the exam: how the test was conducted and how I felt.
My approach during the exam: what strategies to use and mindset to have.
Getting an awesome badge: what it looks like and what's in it for me?
Bonus: as most of my materials are sponsored by the school, I've looked for free content for you to study for the exam, such as filesystem hierarchy, basic linux commands, using AWS EC2 Ubuntu as your Linux environment. I also created a flash card in Trello, available here. Disclaimer: free content might not be sufficient.
Exam objectives: Official CompTIA Linux+ Exam Objectives.

TLDR exam tips (I had no professional experience with Linux):

Focus on the Activity Section of the CompTIA Study Guide and follow along with your own Linux environment.
Practice on Performance-Based Questions (PBQs). They show around 10 choices that all look correct.
Play around with Linux command combinations using piping (|)
Configure IP tables, firewall, and troubleshooting.
Learn the configuration steps, boot process, and backing up files.
Basic bash scripting and regular expressions.

Read until the end for tips on how to have the correct mindset and strategies to answer difficult questions. Also, a video of me summarizing this article during a club meeting at our school is available at the bottom of the article. Do check the video out as it has a Q&A section towards the end.

I'd like to start off with a big thank you to School of Technology and Computing of City University of Seattle (CityU). They provided me with a job, career growth, and sponsored my CompTIA learning materials and exam voucher.

As an F-1 Student Visa holder with limited job opportunities, this allowed me to earn extra income, of which I donate a portion to freeCodeCamp on a monthly basis.

This is a loose continuation from my story: Why I abandoned my MBA to get a master's in Computer Science.

City University of Seattle Campus

Why CompTIA Linux+?

In July 2019, our school, with the help of our director of external relations, landed a contract with Amazon Web Services (AWS) and Washington Technology Industry Association (WTIA), called the AWS Apprenticeship Program (AAP).

We had the opportunity to train 23 military veterans and veteran spouses. Their requirement was to pass CompTIA Linux+ Exam and CityU's Full Stack System Developer Program which includes CompTIA Linux+ Exam preparation. Once they met both criteria, they'd be able to transition to on-the-job training with AWS.

My role at the time was a Lead Teaching Assistant (TA) and I was tasked to help build the Dean's program proposal. I coordinated with 6 other TAs to build the courses, which consisted the following: Linux I, Linux II, Networking, Web Development, JavaScript/TypeScript, Full Stack - MEAN Stack, Python, and Full Stack - Django.

As this was our first contract, we were building the plane while trying to fly it.

Picture looks fun! But not in real life lol!

We utilized the following resources for our Linux Operating System course:

Textbook: Pro Bash Programming: Scripting the GNU/Linux Shell, Second Edition, by Chris F. A. Johnson, Jayant Varma
Student's Linux environment: AWS EC2 Instance with Ubuntu
Virtual Lab: InfoSec Learning

What I learned

A lot of bash scripting exercises and coding challenges
How to SSH to a remote server (EC2 Instance)
Various Linux commands and configuration setups related to CompTIA Linux+

After a few weeks into the program, we researched more and listened to student feedback. Turns out we were doing too much bash scripting and lacked on CompTIA Linux+ Exam preparation.

The students were in hell during the first few weeks of bash scripting

We immediately partnered with CompTIA to get CompTIA Linux+ Exam Study materials at bulk price. We pivoted hard and changed parts of our program to integrate the new study materials.

Cool badge

Good thing our department was practicing Agile methodology and it made us very nimble.

The new resources are as follows:

CertMaster Learn - includes eBook, Performance-Based Question (PBQ), and CompTIA Videos (Note: This was last year. If you purchased this year, you won't have the PBQs, but you get a new interface, and an additional practice test).
CertMaster Practice - multiple-choice questions on steroids. Once you've accumulated a good amount of wrong answers, you'd get feedback on why your answers are wrong and it explains the other choices as well. Then, it pushes your progress bar back.
CertMaster Labs - shorter lab activities compared to InfoSec Labs.

The CompTIA Linux+ Learning bundle

My thoughts on the new resources

I read the eBook from top to bottom and it was helpful to obtain an overview. However, I wasn't able to retain any knowledge from reading it. I breezed through both the CertMaster Practice and CertMaster Labs and I gained nothing as well.

What's wrong with me? Hmm..

Because of an opportunity to teach the JavaScript/TypeScript course in the AAP program and the unfortunate event of my dad's passing, my road to Linux+ Exam was eventually derailed and forgotten.

Man.... What a bummer...

Fast forward to the end of the AWS Apprenticeship Program

In November 2019, we successfully transitioned all 23 students! A 100%! We were the only partner school to have done that.

Most credit goes to the School of Technology & Computing Dean's repeated quotes of "leave no man behind" - similar to the soldier's creed: "I will never leave a fallen comrade behind", and "we stay together, we survive" (from the movie Gladiator).

The program's success was largely attributed to TAs, who supported the students since Day 1.

We asked for feedback from our students on what worked and what did not in regards to the CompTIA Linux+ materials. We got the following:

Focus on Activity Section of the Study Guide
Do the virtual labs repeatedly
Going through the practice exam is not effective
Watch ITPro.tv videos (which we were not able to provide that time)

I did not study for the Linux+ Exam for several months as I focused on learning the MEAN Stack (especially the Angular front end framework). I regularly tackled leetcode coding challenges to prepare for software developer job interviews.

I was also finishing my capstone project: Software Documentation and Architectural Analysis of Full Stack Development, and teaching two classes: Information Systems, and Data Management Communications and Networking.

No time... no time at all... or was it just excuses?

When I was nearing the end of my last quarter, I felt guilty and started studying again for the Linux+ Exam.

This time, a new plan of attack.

I started reading the eBook again from top to bottom. What was different the second time around was that for every theory, definition, command, option, subcommands, etc. that I'd encounter, I would create a flash card in Trello (available here). I used this to train with friends to do a pop quiz and execute commands in the Linux terminal as well.

On my own, I would have my own Linux environment following along with the Activity Section of the eBook. This really helped me retain my knowledge on Linux commands and configurations. I also incorporated using Linux commands in my personal laptop with MacBook Pro, as MacOS has Unix-like terminal.

I utilized the CertMaster Practice Exam again. And this time, I had a friend to go over and discuss why each choice was right or wrong, using the process of elimination.

It seems like doing hands-on and having a discussion with a friend truly helps you master a topic.

I really didn't like the CertMaster Labs and did not use it during my study.

I hesitated to study the following:

bash scripting - I'm a software developer and I know how to code already. I have built bash scripts to boost my productivity, from class notes to web applications.
version control - I've been using git and GitHub in my classes and personal projects.
containers and orchestration - I have a good foundation as I've been reading DevOps books.

The Binge-Worthy ITPro.tv Videos

I accidentally discovered it. It was timely and effective.

where you can binge-watch IT videos

At the start of April 2020, we got news of incoming AAP students. The Dean asked me to check the CompTIA store again for materials, and find out what these ITPro.tv videos were all about.

At first, I was reluctant to even purchase the videos because I remembered the CompTIA Study Guide having the ITPro.tv logo at the start of their videos. I started calculating the total hours of the CompTIA Study Guide videos. The total duration was only 22 minutes.

This was a big difference compared to the standalone ITPro.tv videos with 16 hours of content. I went ahead and purchased it (with reimbursement of course).

And boy, I did not regret it. Those videos were fantastic. They dove deeper into the topics and explained the commands throughly with the help of a co-host to ask why we needed to use those commands.

The average duration of a video is about 15 minutes. A big contrast compared to the other resource, which averaged about 2 minutes per video.

I started picking videos to watch based on the topics that I knew I couldn't explain to anyone. I combined this strategy with the multiple-choice-on-steroids CertMaster Practice to fill in my knowledge gaps.

In the middle of April 2020, I scheduled my CompTIA Linux+ Exam Online Testing. They created the online testing environment due to the pandemic. As it was nearing the scheduled date, I relaxed my study patterns and limited myself to watching the ITPro.tv videos.

Exam Procedures

The online exam requires you to abide by the following procedures:

have an internet connection
have an external webcam or laptop webcam
have an external microphone or laptop microphone
have a photo of valid ID
have pictures of your front, back, left, and right of your own testing environment
No headphones, and smartphones and electronics should be out of reach

View the full details here.

Exam Time

instagram post of my ever supportive partner

I totally appreciate the support!

My Experience with the Online Test App

The exam was scheduled 12:15 midnight. This was my intention to avoid any noise in the house.
The proctor can see you through the webcam, and you can't see them.
The online testing app slowed and crashed several times. The proctor said it was due to the high volume of test takers.

What I felt during the exam

The first few questions I got in the exam were Performance-Based Questions. It was freaking hard. I really couldn't answer with 100% confidence. I encountered a problem using Linux commands grep and awk with piping to another command.

Another problem was about investigating an SELinux policy violation, which requires you have to inspect each command and their respective output to pinpoint which file or directory needed additional configuration.

If only I had that professional experience, this would have been much easier.

As I went through more questions, I really felt so dumb and lost my confidence. I didn't know how to answer the questions about network, system, or user configurations. In the bash scripting portion, I knew I wasn't really good at RegEx (regular expressions).

I had 30 minutes left in my exam and felt like failing...

My thoughts of plan B were coming back – I would just take the exam again. This was partly due to my own perspective in life. I always tell people I'm used to failure and I'm fine failing as long as I can be persistent until I reach my goal.

Around the 28-minute mark I was about to call it quits, when I heard another voice in my brain (I'm not crazy I tell you. It's the Jedi Mind trick, read How to teach yourself to learn again).

The voice said: "don't think about failure now, use the remaining minutes and do your best!"

So I used the remaining time to jump around to different questions until I'd covered them all. This time with more focus. I changed almost 80% of my answers and I started to do the following strategies to increase my chances of getting the right answer:

Process of elimination - I removed choices that I knew were totally wrong.
I stuck with what I knew - I didn't choose an answer if I didn't know what it meant.
Deeper focus - I read word by word and tried to use both comprehension and logical thinking.
Eyes closed, mind opened - I convinced myself that I'd studied really hard for this exam so it must be stored in my brain somewhere. With that in mind, I tried to retrieve those memories to achieve 100% confidence in some questions that I had around 70% confidence answering.

I kept repeating these strategies until the last minute.

Exam Results

The exam ended and I passed – 739 out of 1000! A passing grade point is 720. After 30 minutes, I got an email from CompTIA.

my cool instagram post celebrating my achievement

I will still be humbled and say to people that I got lucky. But for myself, I won't say I got lucky because I changed most of my answers using a better mindset and more effective strategies.

Lessons Learned

I should really stop taking these exams when I don't have professional experience yet. Why? I encountered the same kind of difficulty and failed my AWS Developer Certification Exam before (here's the article: I failed my AWS Developer Exam. What now?).
In studying, I should've done more assessment early on to focus on my knowledge gap instead of speeding through all the materials.
During the exam, I needed to remind myself this is my reality now. What can I do to be great in this situation? This mindset is something I can apply to my current work as well.
Remember to have study sessions with friends or colleagues, as you'll have the chance to grow your knowledge exponentially.

quote from Gladiator (2000) movie

What now, Clark?

I rejoined CityU as a full-timer with a position of Program Manager for AWS Apprentice Program (Full Stack Web Systems Developer) last April 20, 2019.

With the Linux+ Certified under my belt, I have gained additional experience, along with my previous experience being a TA Lead with the same program. It's allowed me to better lead, teach, and mentor the new cohort. This includes my team that includes new TAs that I know for sure will be more successful than I was.

I will do my best to keep teaching and mentoring students while sharpening my software development skills, as I'm still on a journey to find a software developer job in the tech industry. I know it's hard coming from a non-tech background to transition, but we'll all get there in time.

To my readers, thank you for reading my article. I really want to focus more on technical articles but I couldn't let a good story slip away.=)

Connect with me in LinkedIn here.

I couldn't decide on my LinkedIn headline lol

Article Video Summary and Q&A