Symmetric Encryption — AES and the Shared Secret

You need to send a Kubernetes Secret — a database password — from one pod to another. The Secrets manifest is base64-encoded, not encrypted. If anyone intercepts the traffic between your pods, they read the password in plain text.

You know you need encryption. But what does that actually mean? What happens inside the algorithm? And why can't you just encrypt everything with one key and call it a day?

Before you can understand TLS, cert-manager, or mTLS in a service mesh, you need to understand the building block that encrypts every byte of data on the internet: symmetric encryption.

---

Part 1: What Encryption Actually Means

At its core, encryption is a reversible transformation. You take readable data (plaintext), combine it with a secret (the key), and produce unreadable data (ciphertext). Anyone with the key can reverse the process and recover the plaintext. Anyone without the key gets meaningless noise.

Encryption:   plaintext + key → ciphertext
Decryption:   ciphertext + key → plaintext

Symmetric encryption means the same key is used for both encryption and decryption. The sender and receiver share one secret. If you know the key, you can both encrypt and decrypt. If an attacker gets the key, all data encrypted with it is compromised — past, present, and future.

The word "symmetric" comes from this symmetry — the encrypt and decrypt operations are mirrors of each other, using the same key. This is in contrast to asymmetric encryption (Lesson 2), where two different keys are used.

Why Kubernetes Engineers Care

Every TLS connection protecting your cluster — ingress traffic, pod-to-pod mTLS, etcd communication, webhook calls — uses symmetric encryption for the actual data transfer. The TLS handshake negotiates which symmetric cipher to use and exchanges the key. Once the handshake is done, every byte flows through AES (or ChaCha20). Understanding symmetric encryption is understanding what happens to your data after the handshake completes.

---

Part 2: AES — The Industry Standard

The Advanced Encryption Standard (AES) is the symmetric cipher used by TLS, Kubernetes Secrets encryption at rest, disk encryption, VPNs, and virtually every modern encrypted system. It was selected by NIST in 2001 after a five-year international competition. The winning algorithm, Rijndael, was designed by two Belgian cryptographers.

Key Sizes

AES supports three key sizes:

Key Size	Rounds	Security Level	Common Usage
128-bit	10	Secure for most applications	TLS default in many configs
192-bit	12	Higher security margin	Rarely used in practice
256-bit	14	Government/military grade	TLS 1.3 preferred, etcd encryption

A 256-bit key means there are 2^256 possible keys. To put this in perspective: there are roughly 2^80 atoms in the observable universe. Brute-forcing a 256-bit key is not a matter of computing power — it is physically impossible with any conceivable technology.

How AES Works (Simplified)

AES operates on fixed-size blocks of 16 bytes (128 bits). It transforms each block through multiple rounds of four operations:

Input: 16-byte plaintext block + key
│
├── Round 1 through Round N (10/12/14 depending on key size):
│   ├── SubBytes    — substitute each byte using a lookup table (the S-box)
│   ├── ShiftRows   — shift rows of the 4×4 byte matrix by different offsets
│   ├── MixColumns  — matrix multiplication on each column (skipped in last round)
│   └── AddRoundKey — XOR the block with a round-specific subkey derived from the main key
│
Output: 16-byte ciphertext block

Each round's operations serve a specific purpose. SubBytes provides confusion (making the relationship between key and ciphertext complex). ShiftRows and MixColumns provide diffusion (spreading the influence of each plaintext bit across the entire block). AddRoundKey introduces the actual secret key material.

After 10 rounds (for AES-128) or 14 rounds (for AES-256), the output is completely scrambled. Change one bit in the input and roughly half the output bits change — this is called the avalanche effect.

AES Is Fast — Really Fast

Modern CPUs include dedicated hardware instructions for AES. Intel's AES-NI (AES New Instructions) and ARM's equivalent perform AES rounds in silicon, not software.

# Check if your CPU supports AES-NI
grep -o aes /proc/cpuinfo | head -1
# aes

# Benchmark AES speed with OpenSSL
openssl speed -evp aes-256-gcm
# type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
# aes-256-gcm     832697.27k  2551498.37k  5765498.88k  8553310.21k  9648209.92k
#                                                                     ~9.6 GB/s

At 9+ GB/s, AES with hardware acceleration is fast enough that encryption adds negligible latency to your network traffic. This is why TLS has virtually no performance penalty on modern hardware — the symmetric encryption phase (which handles all actual data) runs at wire speed.

---

Part 3: Block Modes — Why AES Alone Is Not Enough

AES encrypts 16-byte blocks. Your data is almost never exactly 16 bytes. A block cipher mode defines how to encrypt data larger than one block. The mode you choose has enormous security implications.

ECB — Electronic Codebook (Never Use This)

ECB encrypts each 16-byte block independently with the same key. Identical plaintext blocks produce identical ciphertext blocks.

Plaintext:  [Block 1] [Block 2] [Block 3] [Block 1]  ← Block 1 repeats
                ↓          ↓         ↓          ↓
Key →       [Cipher 1] [Cipher 2] [Cipher 3] [Cipher 1]  ← Same ciphertext!

This means patterns in the plaintext are visible in the ciphertext. The classic demonstration: encrypt a bitmap image with ECB and the outline of the image is still visible because large areas of identical color produce identical ciphertext blocks.

CBC — Cipher Block Chaining (Acceptable)

CBC fixes ECB's pattern problem by XORing each plaintext block with the previous ciphertext block before encryption. The first block is XORed with a random Initialization Vector (IV).

IV ──XOR──→ [Block 1] ──AES──→ [Cipher 1] ──XOR──→ [Block 2] ──AES──→ [Cipher 2]
                                     │
                                     └── feeds into next block

Identical plaintext blocks now produce different ciphertext because each block depends on all previous blocks. CBC is secure when used correctly, but it has two problems:

1. Sequential processing — each block depends on the previous one, so you cannot parallelize encryption
2. Padding oracle attacks — if the server reveals whether decryption padding is valid, an attacker can decrypt the entire message one byte at a time (this is the POODLE attack that killed SSL 3.0 and weakened TLS 1.0)

GCM — Galois/Counter Mode (Use This)

GCM combines Counter mode (CTR) for encryption with Galois field multiplication for authentication. It provides authenticated encryption — both confidentiality and integrity in a single operation.

Nonce + Counter ──AES──→ Keystream ──XOR──→ Ciphertext
                                              │
                                    GHASH (Galois auth) → Authentication Tag

GCM advantages over CBC:

Property	CBC	GCM
Confidentiality	Yes	Yes
Integrity/Authentication	No (needs separate HMAC)	Built-in (auth tag)
Parallelizable	No (sequential)	Yes (counter mode)
Padding required	Yes (vulnerable to oracles)	No
Performance with AES-NI	Fast	Faster (parallelism + CLMUL)

GCM Nonce Reuse — The One Fatal Mistake

GCM has one critical requirement: the nonce (IV) must never repeat with the same key. If you encrypt two different messages with the same key and nonce, an attacker can XOR the two ciphertexts together, cancel out the keystream, and recover both plaintexts. Worse, they can also recover the authentication key.

# TLS handles nonce generation automatically — each record gets
# a sequential nonce. You do not manage this. But if you are
# implementing encryption at the application level (encrypting
# Kubernetes Secrets, for example), use a random 96-bit nonce
# and never reuse it.

# Generate a random nonce with OpenSSL
openssl rand -hex 12
# a3f7c2e891b4d0165c8e2f7a

---

Part 4: Symmetric Encryption with OpenSSL

Every Kubernetes node has OpenSSL installed. Here are the commands you need to know:

# Encrypt a file with AES-256-GCM
# -aes-256-gcm: cipher
# -pbkdf2: derive key from password using PBKDF2
# -iter 100000: 100k iterations of key derivation (slow down brute force)
openssl enc -aes-256-gcm -pbkdf2 -iter 100000 \
  -in secret.txt -out secret.enc

# Decrypt
openssl enc -d -aes-256-gcm -pbkdf2 -iter 100000 \
  -in secret.enc -out secret.txt

# Encrypt with a specific key and IV (hex-encoded)
# 256-bit key = 32 bytes = 64 hex chars
# 96-bit IV = 12 bytes = 24 hex chars
openssl enc -aes-256-gcm \
  -K 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef \
  -iv 0123456789abcdef01234567 \
  -in secret.txt -out secret.enc

# List all available ciphers
openssl enc -list | grep -i gcm
# -aes-128-gcm
# -aes-192-gcm
# -aes-256-gcm

# Kubernetes etcd encryption at rest — EncryptionConfiguration
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - secrets
    providers:
      - aesgcm:
          keys:
            - name: key1
              # 32 bytes = 256-bit AES key, base64-encoded
              secret: c2VjcmV0LWtleS1oZXJlLXRoaXJ0eS10d28tYnl0ZXM=
      - identity: {}  # fallback to plaintext for reading old secrets

---

Part 5: The Fundamental Problem — Key Distribution

Symmetric encryption is fast, secure, and battle-tested. AES-256-GCM at hardware speed can encrypt your entire cluster's traffic without breaking a sweat. There is just one problem: both sides need the same key.

If you and the server already share a secret key, AES works perfectly. But how did you get that key to the server in the first place?

• You cannot send the key over the network — it is not encrypted yet (that is the whole point)
• You cannot embed the key in the software — anyone who decompiles the binary gets it
• You cannot use a different password for every server — it does not scale

This is the key distribution problem, and it is the reason the rest of this module exists.

ChaCha20-Poly1305 — The Alternative to AES-GCM

TLS 1.3 supports one non-AES cipher: ChaCha20-Poly1305. ChaCha20 is a stream cipher (not a block cipher) designed by Daniel Bernstein. Poly1305 provides authentication (similar to GCM's GHASH).

Why does it exist if AES-GCM is so good? Performance on devices without AES hardware acceleration. On mobile phones and IoT devices with ARM processors lacking AES instructions, ChaCha20 is 3-5x faster than software AES.

TLS 1.3 symmetric ciphers:
├── AES-128-GCM        — fast on hardware with AES-NI
├── AES-256-GCM        — same, higher security margin
└── ChaCha20-Poly1305  — fast on devices without AES-NI

---

Key Concepts Summary

• Symmetric encryption uses the same key to encrypt and decrypt — fast, simple, but requires both parties to share a secret key
• AES is the industry-standard symmetric cipher — 128/192/256-bit keys, selected by NIST in 2001, used in TLS, disk encryption, and Kubernetes etcd encryption
• AES processes 16-byte blocks through 10-14 rounds of substitution, shifting, mixing, and key addition — each round increases scrambling
• Block modes determine how to encrypt data larger than 16 bytes — ECB leaks patterns (never use), CBC is sequential and vulnerable to padding oracles, GCM provides authenticated encryption and is parallelizable
• AES-256-GCM is the standard for TLS — encrypts, authenticates, and runs at 9+ GB/s with hardware acceleration (AES-NI)
• GCM nonce reuse is catastrophic — it leaks plaintext and the authentication key; TLS handles nonces automatically but application-level encryption must guarantee uniqueness
• The key distribution problem is the fundamental limitation of symmetric encryption — you need the key to start encrypting, but you need encryption to safely send the key
• ChaCha20-Poly1305 is the non-AES option in TLS 1.3, faster on hardware without AES-NI

Common Mistakes

• Using ECB mode for anything — it leaks plaintext patterns and is never appropriate for security
• Confusing AES key size with block size — AES always uses 128-bit blocks regardless of whether the key is 128, 192, or 256 bits
• Reusing nonces with AES-GCM — this is not a minor bug but a complete break of both confidentiality and authentication
• Assuming encryption provides integrity — AES-CBC encrypts but does not prove the data was not tampered with; only authenticated modes (GCM) or separate HMACs provide integrity
• Using password-derived keys without a proper KDF — if you derive an AES key from a password, use PBKDF2, scrypt, or Argon2 with high iteration counts, not a simple hash
• Confusing base64 encoding with encryption — Kubernetes Secrets are base64-encoded, not encrypted; anyone who can read the Secret object gets the plaintext

---