3. Kata Containers - VM-Level Isolation

Time to Complete

Planned time: ~40 minutes

Traditional containers share the host kernel, which means a kernel vulnerability can affect all containers on the node. Kata Containers solves this by running each container (or pod) inside a lightweight virtual machine, providing hardware-level isolation while maintaining container-like performance and usability.

In this lab, you’ll install Kata Containers on a Kubernetes cluster, create RuntimeClasses, and compare the isolation between standard containers and Kata-based containers.

What You’ll Learn

Why shared kernel isolation is a security concern in multi-tenant clusters
How Kata Containers provides VM-level isolation for pods
How to install Kata Containers using kata-deploy
How to create and use RuntimeClasses in Kubernetes
How to verify that pods are running in lightweight VMs
How to compare isolation between runc and Kata
(Bonus) How to use different hypervisors (QEMU, Cloud Hypervisor)
(Bonus) How to customize Kata VMs with annotations

Trainer Instructions

Tested versions:

Kata Containers / Helm Chart: 3.26.0
Kubernetes: 1.32.x
containerd: 1.7.x
nginx image: 1.27.3
alpine image: 3.21

Hardware requirements:

Nodes must support hardware virtualization (Intel VT-x or AMD-V)
For cloud VMs, nested virtualization must be enabled
kind/minikube on laptops will NOT work unless nested virtualization is supported

Supported environments:

Bare metal servers with virtualization support
Cloud VMs with nested virtualization (GCP, Azure, some AWS instance types)
kubeadm clusters on compatible hardware

Not supported:

Standard kind clusters (no nested virtualization)
Most managed Kubernetes services (EKS, GKE, AKS) - use node pools with Kata support

Info

We are in the cluster created by hand: kx c<x>-byhand

Prerequisites

Hardware Virtualization Check

Before starting, verify that your nodes support hardware virtualization:

# On each worker node, check for virtualization support
ssh azuser@NODE-0 # and again for ssh azuser@NODE-1

# 1 
sudo apt install cpu-checker
sudo /usr/sbin/kvm-ok

# should be
INFO: /dev/kvm exists
KVM acceleration can be used

# or 2
grep -E '(vmx|svm)' /proc/cpuinfo
# which should result in:

vmx = Intel VT-x
svm = AMD-V

If no output, your nodes don’t support hardware virtualization and cannot run Kata Containers.

Warning

This lab requires hardware virtualization support. If your cluster doesn’t support it, you can still learn the concepts by reviewing the configurations and understanding the architecture.

1. Understanding Container Isolation

Before installing Kata, let’s understand why VM-level isolation matters.

The Shared Kernel Problem

Info

Traditional containers (using runc) share the host kernel. All containers on a node make system calls to the same kernel. If an attacker exploits a kernel vulnerability, they can potentially escape the container and access other containers or the host.

How Kata Containers Solves This

Kata Containers runs each pod inside a lightweight virtual machine (microVM):

┌─────────────────────────────────────────────────────┐
│                    Host Node                         │
│  ┌──────────────┐  ┌──────────────┐                │
│  │   Kata VM    │  │   Kata VM    │                │
│  │ ┌──────────┐ │  │ ┌──────────┐ │                │
│  │ │Container │ │  │ │Container │ │                │
│  │ └──────────┘ │  │ └──────────┘ │                │
│  │ Guest Kernel │  │ Guest Kernel │                │
│  └──────────────┘  └──────────────┘                │
│         │                  │                        │
│         └────────┬─────────┘                        │
│              Hypervisor (QEMU/Cloud-Hypervisor)     │
│                    Host Kernel                       │
└─────────────────────────────────────────────────────┘

Questions

What is the default container runtime in Kubernetes?
Why does Kata use a separate kernel for each pod?

Answers

The default runtime is containerd with runc as the low-level runtime.
Each pod gets its own kernel, so a kernel vulnerability in one pod cannot affect other pods. The attack surface is the hypervisor, which is much smaller than the kernel.

2. Install Kata Containers

Kata Containers can be installed using the kata-deploy Helm chart, which automatically installs the Kata runtime on all nodes and creates the necessary RuntimeClasses.

Install with Helm

Install kata-deploy from the OCI registry with version 3.26.0:

Solution

# Install kata-deploy from OCI registry with pinned version
helm install kata-deploy oci://ghcr.io/kata-containers/kata-deploy-charts/kata-deploy \
  --namespace kube-system \
  --version 3.26.0 \
  --wait --timeout 10m

Wait for Installation

Wait for the kata-deploy pods to be ready:

kubectl -n kube-system wait --timeout=10m --for=condition=Ready -l name=kata-deploy pod

Verify RuntimeClasses

After installation, kata-deploy creates several RuntimeClasses. Display them with kubectl get ...

Solution

kubectl get runtimeclass

Expected output:

NAME             HANDLER          AGE
kata-clh         kata-clh         1m
kata-qemu        kata-qemu        1m
kata-dragonball  kata-dragonball  1m

Info

Different RuntimeClasses use different hypervisors:

kata-qemu: Uses QEMU (most compatible)
kata-clh: Uses Cloud Hypervisor (faster startup)
kata-dragonball: Uses Dragonball (lightweight)
kata-fc: Uses Firecracker (AWS microVM, if available)

3. Create a RuntimeClass (Manual Method)

If kata-deploy doesn’t create RuntimeClasses automatically, or you want to understand the process, you can create them manually.

Task

Create a RuntimeClass named kata that uses the Kata Qemu runtime.

Hint

A RuntimeClass needs a handler field that matches the containerd configuration.

Solution

Create the RuntimeClass (~/exercise/kubernetes/kata-containers/runtimeclass-kata.yaml):

apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: kata
# Handler must match the containerd runtime configuration name
# Common handlers: kata, kata-qemu, kata-clh (Cloud Hypervisor), kata-fc (Firecracker)
handler: kata-qemu

Apply:

kubectl apply -f ~/exercise/kubernetes/kata-containers/runtimeclass-kata.yaml

Questions

What is the purpose of the handler field?
How does Kubernetes know which runtime to use for a pod?

Answers

The handler field maps to a runtime configured in containerd’s config (/etc/containerd/config.toml). The handler name must match exactly.
Kubernetes reads the pod’s runtimeClassName field and uses the corresponding RuntimeClass to determine which containerd runtime to invoke.

4. Run a Pod with Kata

Now let’s run a pod using the Kata runtime.

Task

Deploy a pod using the kata RuntimeClass and the nginx:1.27.3 image
Verify the pod is running

Hint

Your best friend are the docs https://kubernetes.io

Solution

Create the pod (~/exercise/kubernetes/kata-containers/pod-kata.yaml):

apiVersion: v1
kind: Pod
metadata:
  name: nginx-kata
  labels:
    app: nginx
    runtime: kata
spec:
  runtimeClassName: kata
  containers:
  - name: nginx
    image: nginx:1.27.3
    ports:
    - containerPort: 80

Apply and verify:

kubectl apply -f ~/exercise/kubernetes/kata-containers/pod-kata.yaml
kubectl get pod nginx-kata -o wide

Verify Kata is Being Used

Check that the pod is running in a Kata VM:

Solution

# Check the runtime class in the pod spec
kubectl get pod nginx-kata -o jsonpath='{.spec.runtimeClassName}'

# and on which node it is running
kubectl get pod nginx-kata -o jsonpath='{.spec.nodeName}'

# On the node, you can see the QEMU process (requires node access)
# ssh to node
ssh azuser@NODE

# and run:
azuser@NODE:~$ sudo crictl ps
CONTAINER      IMAGE         STATE   NAME   POD ID              POD                 NAMESPACE
247f073f93f0d  c59e925d63f3a Running nginx  c09cb4f0858d5       nginx-kata          default

ps aux | grep qemu
root        9837  2.1  7.3 2955360 295344 ?      Sl   06:25   0:04 /opt/kata/bin/qemu-system-x86_64 -name sandbox-c09cb4f

Questions

How can you confirm that a pod is using Kata?
Why is this useful in multi-tenant clusters?

Answers

Check runtimeClassName in the pod spec, or look for QEMU/hypervisor processes on the node.
Multi-tenant clusters can run untrusted workloads in Kata VMs, preventing them from affecting other tenants even if they exploit kernel vulnerabilities.

5. Compare Isolation

Let’s compare the isolation between a standard container and a Kata container.

Deploy Both Pod Types

Deploy two pods - one with the default runtime, one with Kata:

apiVersion: v1
kind: Pod
metadata:
  name: isolation-test-default
  labels:
    app: isolation-test
    runtime: runc
spec:
  # Default runtime (runc) for comparison
  containers:
  - name: alpine
    image: alpine:3.21
    command: ["sleep", "3600"]

apiVersion: v1
kind: Pod
metadata:
  name: isolation-test-kata
  labels:
    app: isolation-test
    runtime: kata
spec:
  runtimeClassName: kata
  containers:
  - name: alpine
    image: alpine:3.21
    command: ["sleep", "3600"]

Solution

kubectl apply -f ~/exercise/kubernetes/kata-containers/pod-isolation-test-default.yaml
kubectl apply -f ~/exercise/kubernetes/kata-containers/pod-isolation-test.yaml

# Wait for pods to be ready
kubectl wait --for=condition=Ready pod/isolation-test-default pod/isolation-test-kata --timeout=60s

Compare Kernel Information

Check the kernel version inside each pod:

Solution

echo "=== Default runtime (runc) - shares host kernel ==="
kubectl exec isolation-test-default -- uname -a

echo ""
echo "=== Kata runtime - separate guest kernel ==="
kubectl exec isolation-test-kata -- uname -a

Compare Process Visibility

Check what processes are visible from inside each pod:

Solution

echo "=== Default runtime - can see host PIDs with hostPID ==="
kubectl exec isolation-test-default -- ps aux

echo ""
echo "=== Kata runtime - only VM processes visible ==="
kubectl exec isolation-test-kata -- ps aux

Questions

What differences do you observe in the kernel versions?
What are the trade-offs of using Kata?

Answers

The default pod shows the host kernel version. The Kata pod shows a guest kernel version (typically different, optimized for VMs).
Trade-offs:
- Pros: Stronger isolation, kernel independence, better security for untrusted workloads
- Cons: Higher memory overhead (~20-50MB per pod), slower startup time, requires virtualization support

6. Bonus: Alternative Hypervisors

Bonus Exercise

This section explores using different hypervisors with Kata.

Kata supports multiple hypervisors:

Hypervisor	RuntimeClass	Startup Time	Memory	Use Case
QEMU	`kata-qemu`	~500ms	Higher	Most compatible, feature-rich
Cloud Hypervisor	`kata-clh`	~150ms	Lower	Fast startup, modern
Firecracker	`kata-fc`	~125ms	Lowest	AWS, minimal footprint

Task

Create a RuntimeClass for Cloud Hypervisor
Deploy a pod using Cloud Hypervisor
Compare startup time with QEMU

Solution

Create the RuntimeClass (~/exercise/kubernetes/kata-containers/runtimeclass-kata-clh.yaml):

apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: kata-clh
# Use Cloud Hypervisor (faster startup than QEMU)
handler: kata-clh

Deploy a pod using Cloud Hypervisor:

apiVersion: v1
kind: Pod
metadata:
  name: nginx-kata-clh
  labels:
    app: nginx
    runtime: kata-clh
spec:
  # Use Cloud Hypervisor for faster startup
  runtimeClassName: kata-clh
  containers:
  - name: nginx
    image: nginx:1.27.3
    ports:
    - containerPort: 80

Apply and compare:

kubectl apply -f ~/exercise/kubernetes/kata-containers/runtimeclass-kata-clh.yaml

# Time QEMU startup
time kubectl run test-qemu --image=alpine:3.21 --restart=Never --rm -it \
  --overrides='{"spec":{"runtimeClassName":"kata-qemu"}}' -- echo "QEMU started"

# Time Cloud Hypervisor startup
time kubectl run test-clh --image=alpine:3.21 --restart=Never --rm -it \
  --overrides='{"spec":{"runtimeClassName":"kata-clh"}}' -- echo "CLH started"

7. Clean Up

Remove the resources created during this lab:

kubectl delete pod nginx-kata nginx-default isolation-test-kata isolation-test-default nginx-kata-clh nginx-kata-custom --ignore-not-found
kubectl delete deployment nginx-kata-deployment --ignore-not-found

Optional: Remove Kata Containers

To completely remove Kata from your cluster:

helm uninstall kata-deploy -n kube-system
kubectl delete runtimeclass kata kata-qemu kata-clh kata-dragonball kata-fc --ignore-not-found

Recap

You have:

Understood why shared kernel isolation is a security concern
Learned how Kata Containers provides VM-level isolation
Installed Kata Containers 3.26.0 using the kata-deploy Helm chart
Created RuntimeClasses for different hypervisors
Deployed pods using the Kata runtime
Compared isolation between standard containers and Kata VMs
(Bonus) Used Cloud Hypervisor for faster startup
(Bonus) Customized Kata VMs using annotations

Wrap-Up Questions

Discussion

When would you use Kata Containers vs standard containers?
How does Kata fit into a defense-in-depth strategy?
What are the cost implications of running Kata in production?

Discussion Points

When to use Kata: Multi-tenant clusters, running untrusted code, CI/CD pipelines with user-submitted code, compliance requirements for workload isolation.
Defense in Depth: Kata adds VM-level isolation on top of namespace isolation, seccomp, AppArmor/SELinux, and network policies. It protects against kernel exploits that bypass container isolation.
Cost implications: Higher memory overhead (20-50MB per pod), slightly higher CPU usage, requires hardware virtualization. Consider using Kata only for workloads that need strong isolation.

3. Kata Containers - VM-Level Isolation

What You’ll Learn

Prerequisites

Hardware Virtualization Check

1. Understanding Container Isolation

The Shared Kernel Problem

How Kata Containers Solves This

2. Install Kata Containers

Install with Helm

Wait for Installation

Verify RuntimeClasses

3. Create a RuntimeClass (Manual Method)

Task

4. Run a Pod with Kata

Task

Verify Kata is Being Used

5. Compare Isolation

Deploy Both Pod Types

Compare Kernel Information

Compare Process Visibility

6. Bonus: Alternative Hypervisors

Task

7. Clean Up

Recap

Wrap-Up Questions

Further Reading