Bash & Shell Scripting for Engineers

Variables, Functions, and Patterns

Makefiles have variables, functions, and pattern rules — a whole small language that most engineers have only seen glimpses of. Learning the pieces beyond "targets and recipes" is the difference between writing Makefiles that scale and copy-pasting 100 lines of repetitive rules.

This lesson covers the four assignment operators (and why they're not the same), the built-in functions you will actually use, pattern rules, and the self-documenting help target that professional Makefiles ship with.

KEY CONCEPT

The assignment operators matter: = is lazy, := is immediate, ?= sets if unset, += appends. Pick the wrong one and your variables expand at the wrong time, usually breaking things in subtle ways.


The four assignment operators

NAME = value     # lazy — re-evaluates every time NAME is expanded
NAME := value    # immediate — evaluates once, when this line is read
NAME ?= value    # conditional — sets only if NAME was previously unset
NAME += value    # append — adds to the existing value (space-separated)

They look almost identical and do very different things.

= (lazy / recursive)

Value is re-expanded every time $(NAME) is referenced:

FOO = hello
BAR = $(FOO) world

FOO = bye       # redefine

build:
	@echo $(BAR)    # prints "bye world" — $(FOO) is looked up NOW

Useful for "late-binding" variables whose value should reflect whatever is current at use time. Dangerous when you want a fixed value.

:= (immediate / simple)

Value is evaluated once, at the time of assignment:

FOO := hello
BAR := $(FOO) world

FOO := bye       # redefine

build:
	@echo $(BAR)    # prints "hello world" — BAR was set when FOO was "hello"

Almost always what you want. Predictable. Makes subsequent changes to FOO not affect BAR.

Prefer := over = as a default. Use = only when you specifically want late binding.

?= (conditional)

Only assigns if the variable is not already set:

ENV ?= dev
# If ENV was set externally (e.g. `make ENV=prod build`), it stays prod.
# Otherwise it's dev.

The idiomatic way to supply a default that can be overridden from the command line or the environment.

+= (append)

Adds to the existing value, space-separated:

FLAGS := -Wall
FLAGS += -O2
FLAGS += -g
# FLAGS is now "-Wall -O2 -g"

Useful for building flag lists progressively.

WARNING

If you use = with +=, the appended value is also lazy. Stick with := + += for simple-expansion variables — the most predictable combo.


Referencing variables

VAR := hello
$(VAR)     # in a recipe or variable, expands to "hello"
${VAR}     # equivalent — some people prefer curly braces

Variables can be referenced in targets, prerequisites, and recipes. In recipes, they expand when the recipe runs.

Variables vs shell variables

Inside a recipe, Make variables are expanded first; then the resulting text is passed to the shell, which handles $ itself.

build:
	echo $(FOO)    # Make expands $(FOO), shell sees the literal value
	echo $$FOO     # $$ is Make-escape for a literal $, shell sees $FOO

To pass a literal $ to the shell, double it. This is how you reference shell variables vs Make variables.


Built-in functions — the useful ones

Make has ~30 built-in functions. The ones you'll use most:

$(shell cmd) — run a shell command

Captures the output of a shell command at parse time:

VERSION := $(shell git describe --tags --always)
NUM_CORES := $(shell nproc)

build:
	@echo "building version $(VERSION) with $(NUM_CORES) cores"

Runs once when the Makefile is parsed. The result is stored as a string.

$(wildcard pattern) — glob matching

Expand a glob to a space-separated list:

SOURCES := $(wildcard src/*.c)
# e.g. "src/main.c src/utils.c src/io.c"

$(patsubst pattern,replacement,text) — pattern substitution

SOURCES := src/main.c src/utils.c
OBJECTS := $(patsubst %.c,%.o,$(SOURCES))
# OBJECTS = "src/main.o src/utils.o"

Turns sources into corresponding object files. The % is a wildcard that matches anything.

Shorthand form:

OBJECTS := $(SOURCES:.c=.o)
# Same result, shorter syntax

$(foreach var,list,expr) — iterate

DIRS := src test lib
RESULT := $(foreach d,$(DIRS),$(d)/out)
# RESULT = "src/out test/out lib/out"

$(addprefix prefix,list) and $(addsuffix suffix,list)

FILES := main utils
OBJECTS := $(addsuffix .o,$(FILES))    # main.o utils.o
PATHS := $(addprefix build/,$(OBJECTS)) # build/main.o build/utils.o

$(dir names) and $(notdir names) and $(basename names) and $(suffix names)

PATH := src/sub/file.c
$(dir $(PATH))      # "src/sub/"
$(notdir $(PATH))   # "file.c"
$(basename $(PATH)) # "src/sub/file"
$(suffix $(PATH))   # ".c"

$(filter pattern,list) and $(filter-out pattern,list)

ALL := a.c b.h c.c d.h
$(filter %.c,$(ALL))      # "a.c c.c"
$(filter-out %.h,$(ALL))  # "a.c c.c"

$(strip text)

Removes leading/trailing whitespace and collapses internal whitespace.

$(strip  a   b  c )    # "a b c"

Pattern rules

Pattern rules let one rule cover many targets. The % is a wildcard.

# Build any .o from the corresponding .c
%.o: %.c
	gcc -c -o $@ $<

Used by Make automatically: when you ask for main.o, Make looks at the pattern rule, matches %.o to main.o (so % is main), and applies the rule with main.c as $<.

Common recipe with $@ and $<

%.min.js: %.js
	terser $< -o $@

%.gz: %
	gzip -k $<

Reading: target uses $@; first dep uses $<. The pattern rule matches any target ending in .min.js and builds from the same base with .js.

Using pattern rules for a list of targets

SOURCES := $(wildcard src/*.c)
OBJECTS := $(SOURCES:.c=.o)

all: $(OBJECTS)

%.o: %.c
	gcc -c -o $@ $<

Make walks OBJECTS, notices each is a .o, applies the pattern rule, and builds from its .c counterpart. One rule, N targets.


Conditional directives

ifeq ($(ENV),prod)
  FLAGS := -O2 -DNDEBUG
else
  FLAGS := -g -O0
endif

ifneq ($(OS),Windows_NT)
  RM := rm -f
else
  RM := del /Q
endif

ifdef DEBUG
  EXTRA := --verbose
endif

Conditional variables — pick values based on environment, target OS, feature flags, etc.


.PHONY meta-target

Already covered. Here's the common pattern:

.PHONY: help build test lint clean install run

One line near the top, every command-like target listed. Updated as targets are added.


The self-documenting help target — the canonical pattern

Every good Makefile has a help target that lists all targets with their descriptions. The magic: the help target reads the Makefile itself and picks out ## description comments.

.PHONY: help
.DEFAULT_GOAL := help

help: ## Show this help message
	@awk 'BEGIN {FS = ":.*?## "} \
	  /^[a-zA-Z_-]+:.*?## / \
	  {printf "  \033[36m%-20s\033[0m %s\n", $$1, $$2}' \
	  $(MAKEFILE_LIST)

build: ## Build the binary
	@go build -o bin/app .

test: ## Run tests with race detection
	@go test -race ./...

lint: ## Run golangci-lint
	@golangci-lint run

clean: ## Remove build artifacts
	@rm -rf bin/

deploy: ## Deploy to production (requires KUBECONFIG)
	@kubectl apply -f manifests/

Running make:

  build                Build the binary
  test                 Run tests with race detection
  lint                 Run golangci-lint
  clean                Remove build artifacts
  deploy               Deploy to production (requires KUBECONFIG)
  help                 Show this help message

The awk command parses any line matching target: ... ## description and formats it. New targets appear automatically once you add the ## doc comment.

PRO TIP

Copy the help target into every Makefile you write. It takes 30 seconds and the payoff is huge — every new team member finds the targets without reading the Makefile.


Including variables from the environment

All environment variables are automatically available as Make variables:

build:
	@echo "USER is $(USER)"
	@echo "PATH is $(PATH)"

You can override any Make variable from the command line:

make VERSION=v2.0 build

Inside the Makefile, that overrides whatever VERSION := says. Command-line assignments win.

Exporting to subprocesses

Variables in a Makefile are NOT exported to recipe shells by default:

NAME := myapp

show:
	@echo "NAME in make: $(NAME)"
	@echo "NAME in shell: $$NAME"    # empty!

Export with export:

export NAME := myapp

show:
	@echo "NAME in shell: $$NAME"    # now "myapp"

Putting it together — a realistic Makefile

# Project Makefile

# --- variables ---
BINARY := myapp
VERSION := $(shell git describe --tags --always --dirty)
GO_FILES := $(shell find . -name '*.go' -not -path './vendor/*')
BUILD_DIR := bin
GOFLAGS := -ldflags "-X main.Version=$(VERSION)"

ENV ?= dev   # default; override with `make ENV=prod ...`

# --- meta ---
.PHONY: help build test lint clean run docker-build install
.DEFAULT_GOAL := help

# --- targets ---
help: ## Show this help
	@awk 'BEGIN {FS = ":.*?## "} \
	  /^[a-zA-Z_-]+:.*?## / {printf "  \033[36m%-20s\033[0m %s\n", $$1, $$2}' \
	  $(MAKEFILE_LIST)

build: $(BUILD_DIR)/$(BINARY) ## Build the binary

$(BUILD_DIR)/$(BINARY): $(GO_FILES) | $(BUILD_DIR)
	@echo "building $(VERSION)"
	@go build $(GOFLAGS) -o $@ .

$(BUILD_DIR):
	@mkdir -p $@

test: ## Run tests
	@go test -race ./...

lint: ## Run linters
	@golangci-lint run

clean: ## Remove build artifacts
	@rm -rf $(BUILD_DIR)

run: build ## Run the app
	@./$(BUILD_DIR)/$(BINARY)

docker-build: ## Build the Docker image
	@docker build -t $(BINARY):$(VERSION) -t $(BINARY):latest .

install: build ## Install to /usr/local/bin
	@install -m 0755 $(BUILD_DIR)/$(BINARY) /usr/local/bin/

Features:

  • := for all the immediate assignments.
  • ?= for ENV (overridable).
  • $(shell ...) for version from git.
  • $(wildcard) via find (covers subdirectories).
  • Proper file target for the binary (caches correctly).
  • | $(BUILD_DIR) as an order-only prerequisite so the dir exists.
  • .PHONY listed at the top.
  • Help target self-documents from ## comments.

This is the shape of a mid-sized project Makefile.


Pattern rules in practice — a multi-file build

Build every .c to .o, then link them into a binary:

SOURCES := $(wildcard src/*.c)
OBJECTS := $(SOURCES:.c=.o)
BINARY := app

.PHONY: all clean
.DEFAULT_GOAL := all

all: $(BINARY)

$(BINARY): $(OBJECTS)
	gcc -o $@ $^

%.o: %.c
	gcc -c -o $@ $<

clean:
	rm -f $(OBJECTS) $(BINARY)

No matter how many .c files are added to src/, this Makefile builds them all without modification. That is the power of pattern rules.

Header dependencies — the -MMD trick

A C file depends on its headers, but you shouldn't hand-list every header. gcc -MMD generates dependency files automatically:

SOURCES := $(wildcard src/*.c)
OBJECTS := $(SOURCES:.c=.o)
DEPS := $(OBJECTS:.o=.d)

%.o: %.c
	gcc -c -MMD -MP -o $@ $<

-include $(DEPS)    # include auto-generated .d files

Now when a header changes, Make correctly rebuilds the .o files that use it. Advanced, but ubiquitous in serious C/C++ builds.


Common variable mistakes

Mistake 1: = where you meant :=

VERSION = $(shell git describe)    # re-runs git every time VERSION is expanded!

If you reference $(VERSION) in 10 places, you run git describe 10 times. Use :=:

VERSION := $(shell git describe)    # runs once

Mistake 2: unquoted shell variables in recipes

NAME := my file
deploy:
	cp $(NAME) /target    # word-splits on space

Variables can contain spaces. Quote them as you would shell vars:

deploy:
	cp "$(NAME)" /target

Mistake 3: forgetting $$ for shell variables

greet:
	name="Alice"
	echo $name    # Make saw $n and expanded it to empty; shell sees "ame"

Double the $:

greet:
	name="Alice"; echo $$name

And chain them with ; or &&, since each line is a new shell:

greet:
	name="Alice" && echo $$name

Quiz

KNOWLEDGE CHECK

You write: VERSION = $(shell git describe --tags) and reference $(VERSION) four times in different targets. What happens each time you run make?


What to take away

  • := is the right default. Use = only when you want late binding, ?= for overridable defaults, += to append.
  • $(shell cmd) runs commands at parse time. Use := so it runs once.
  • $(wildcard), $(patsubst), $(foreach), $(addprefix), $(filter) — memorize these. They cover 90% of Make function usage.
  • Pattern rules (%.o: %.c) let one rule cover N similar targets. Combined with $(wildcard), they scale to any number of files.
  • Self-documenting help target via awk parsing ## description comments. Copy it into every Makefile.
  • Order-only prerequisites (| dir) for directories and other things that should exist but shouldn't trigger rebuilds.
  • $$ to escape $ for the shell. Shell variables in recipes need doubling.
  • Chain recipe lines with && if they share state, or use .ONESHELL:.

Next lesson: production Makefile conventions — the build/test/lint/ci/deploy pattern, Docker integration, and when to reach for just or task instead.