Bash & Shell Scripting for Engineers

Variables, Functions, and Patterns

Makefiles have variables, functions, and pattern rules, a whole small language that most engineers have only seen glimpses of. Learning the pieces beyond "targets and recipes" is the difference between writing Makefiles that scale and copy-pasting 100 lines of repetitive rules.

This lesson covers the four assignment operators (and why they're not the same), the built-in functions you will actually use, pattern rules, and the self-documenting help target that professional Makefiles ship with.

KEY CONCEPT

The assignment operators matter: = is lazy, := is immediate, ?= sets if unset, += appends. Pick the wrong one and your variables expand at the wrong time, usually breaking things in subtle ways.


The four assignment operators

NAME = value     # lazy — re-evaluates every time NAME is expanded
NAME := value    # immediate — evaluates once, when this line is read
NAME ?= value    # conditional — sets only if NAME was previously unset
NAME += value    # append — adds to the existing value (space-separated)

They look almost identical and do very different things.

= (lazy / recursive)

Value is re-expanded every time $(NAME) is referenced:

FOO = hello
BAR = $(FOO) world

FOO = bye       # redefine

build:
	@echo $(BAR)    # prints "bye world" — $(FOO) is looked up NOW

Useful for "late-binding" variables whose value should reflect whatever is current at use time. Dangerous when you want a fixed value.

:= (immediate / simple)

Value is evaluated once, at the time of assignment:

FOO := hello
BAR := $(FOO) world

FOO := bye       # redefine

build:
	@echo $(BAR)    # prints "hello world" — BAR was set when FOO was "hello"

Almost always what you want. Predictable. Makes subsequent changes to FOO not affect BAR.

Prefer := over = as a default. Use = only when you specifically want late binding.

?= (conditional)

Only assigns if the variable is not already set:

ENV ?= dev
# If ENV was set externally (e.g. `make ENV=prod build`), it stays prod.
# Otherwise it's dev.

The idiomatic way to supply a default that can be overridden from the command line or the environment.

+= (append)

Adds to the existing value, space-separated:

FLAGS := -Wall
FLAGS += -O2
FLAGS += -g
# FLAGS is now "-Wall -O2 -g"

Useful for building flag lists progressively.

WARNING

If you use = with +=, the appended value is also lazy. Stick with := + += for simple-expansion variables, the most predictable combo.


Referencing variables

VAR := hello
$(VAR)     # in a recipe or variable, expands to "hello"
${VAR}     # equivalent — some people prefer curly braces

Variables can be referenced in targets, prerequisites, and recipes. In recipes, they expand when the recipe runs.

Variables vs shell variables

Inside a recipe, Make variables are expanded first; then the resulting text is passed to the shell, which handles $ itself.

build:
	echo $(FOO)    # Make expands $(FOO), shell sees the literal value
	echo $$FOO     # $$ is Make-escape for a literal $, shell sees $FOO

To pass a literal $ to the shell, double it. This is how you reference shell variables vs Make variables.


Built-in functions: the useful ones

Make has ~30 built-in functions. The ones you'll use most:

$(shell cmd): run a shell command

Captures the output of a shell command at parse time:

VERSION := $(shell git describe --tags --always)
NUM_CORES := $(shell nproc)

build:
	@echo "building version $(VERSION) with $(NUM_CORES) cores"

Runs once when the Makefile is parsed. The result is stored as a string.

$(wildcard pattern): glob matching

Expand a glob to a space-separated list:

SOURCES := $(wildcard src/*.c)
# e.g. "src/main.c src/utils.c src/io.c"

$(patsubst pattern,replacement,text): pattern substitution

SOURCES := src/main.c src/utils.c
OBJECTS := $(patsubst %.c,%.o,$(SOURCES))
# OBJECTS = "src/main.o src/utils.o"

Turns sources into corresponding object files. The % is a wildcard that matches anything.

Shorthand form:

OBJECTS := $(SOURCES:.c=.o)
# Same result, shorter syntax

$(foreach var,list,expr): iterate

DIRS := src test lib
RESULT := $(foreach d,$(DIRS),$(d)/out)
# RESULT = "src/out test/out lib/out"

$(addprefix prefix,list) and $(addsuffix suffix,list)

FILES := main utils
OBJECTS := $(addsuffix .o,$(FILES))    # main.o utils.o
PATHS := $(addprefix build/,$(OBJECTS)) # build/main.o build/utils.o

$(dir names) and $(notdir names) and $(basename names) and $(suffix names)

PATH := src/sub/file.c
$(dir $(PATH))      # "src/sub/"
$(notdir $(PATH))   # "file.c"
$(basename $(PATH)) # "src/sub/file"
$(suffix $(PATH))   # ".c"

$(filter pattern,list) and $(filter-out pattern,list)

ALL := a.c b.h c.c d.h
$(filter %.c,$(ALL))      # "a.c c.c"
$(filter-out %.h,$(ALL))  # "a.c c.c"

$(strip text)

Removes leading/trailing whitespace and collapses internal whitespace.

$(strip  a   b  c )    # "a b c"

Pattern rules

Pattern rules let one rule cover many targets. The % is a wildcard.

# Build any .o from the corresponding .c
%.o: %.c
	gcc -c -o $@ $<

Used by Make automatically: when you ask for main.o, Make looks at the pattern rule, matches %.o to main.o (so % is main), and applies the rule with main.c as $<.

Common recipe with $@ and $<

%.min.js: %.js
	terser $< -o $@

%.gz: %
	gzip -k $<

Reading: target uses $@; first dep uses $<. The pattern rule matches any target ending in .min.js and builds from the same base with .js.

Using pattern rules for a list of targets

SOURCES := $(wildcard src/*.c)
OBJECTS := $(SOURCES:.c=.o)

all: $(OBJECTS)

%.o: %.c
	gcc -c -o $@ $<

Make walks OBJECTS, notices each is a .o, applies the pattern rule, and builds from its .c counterpart. One rule, N targets.


Conditional directives

ifeq ($(ENV),prod)
  FLAGS := -O2 -DNDEBUG
else
  FLAGS := -g -O0
endif

ifneq ($(OS),Windows_NT)
  RM := rm -f
else
  RM := del /Q
endif

ifdef DEBUG
  EXTRA := --verbose
endif

Conditional variables: pick values based on environment, target OS, feature flags, etc.


.PHONY meta-target

Already covered. Here's the common pattern:

.PHONY: help build test lint clean install run

One line near the top, every command-like target listed. Updated as targets are added.


The self-documenting help target: the canonical pattern

Every good Makefile has a help target that lists all targets with their descriptions. The magic: the help target reads the Makefile itself and picks out ## description comments.

.PHONY: help
.DEFAULT_GOAL := help

help: ## Show this help message
	@awk 'BEGIN {FS = ":.*?## "} \
	  /^[a-zA-Z_-]+:.*?## / \
	  {printf "  \033[36m%-20s\033[0m %s\n", $$1, $$2}' \
	  $(MAKEFILE_LIST)

build: ## Build the binary
	@go build -o bin/app .

test: ## Run tests with race detection
	@go test -race ./...

lint: ## Run golangci-lint
	@golangci-lint run

clean: ## Remove build artifacts
	@rm -rf bin/

deploy: ## Deploy to production (requires KUBECONFIG)
	@kubectl apply -f manifests/

Running make:

  build                Build the binary
  test                 Run tests with race detection
  lint                 Run golangci-lint
  clean                Remove build artifacts
  deploy               Deploy to production (requires KUBECONFIG)
  help                 Show this help message

The awk command parses any line matching target: ... ## description and formats it. New targets appear automatically once you add the ## doc comment.

PRO TIP

Copy the help target into every Makefile you write. It takes 30 seconds and the payoff is huge, every new team member finds the targets without reading the Makefile.


Including variables from the environment

All environment variables are automatically available as Make variables:

build:
	@echo "USER is $(USER)"
	@echo "PATH is $(PATH)"

You can override any Make variable from the command line:

make VERSION=v2.0 build

Inside the Makefile, that overrides whatever VERSION := says. Command-line assignments win.

Exporting to subprocesses

Variables in a Makefile are NOT exported to recipe shells by default:

NAME := myapp

show:
	@echo "NAME in make: $(NAME)"
	@echo "NAME in shell: $$NAME"    # empty!

Export with export:

export NAME := myapp

show:
	@echo "NAME in shell: $$NAME"    # now "myapp"

Putting it together: a realistic Makefile

# Project Makefile

# --- variables ---
BINARY := myapp
VERSION := $(shell git describe --tags --always --dirty)
GO_FILES := $(shell find . -name '*.go' -not -path './vendor/*')
BUILD_DIR := bin
GOFLAGS := -ldflags "-X main.Version=$(VERSION)"

ENV ?= dev   # default; override with `make ENV=prod ...`

# --- meta ---
.PHONY: help build test lint clean run docker-build install
.DEFAULT_GOAL := help

# --- targets ---
help: ## Show this help
	@awk 'BEGIN {FS = ":.*?## "} \
	  /^[a-zA-Z_-]+:.*?## / {printf "  \033[36m%-20s\033[0m %s\n", $$1, $$2}' \
	  $(MAKEFILE_LIST)

build: $(BUILD_DIR)/$(BINARY) ## Build the binary

$(BUILD_DIR)/$(BINARY): $(GO_FILES) | $(BUILD_DIR)
	@echo "building $(VERSION)"
	@go build $(GOFLAGS) -o $@ .

$(BUILD_DIR):
	@mkdir -p $@

test: ## Run tests
	@go test -race ./...

lint: ## Run linters
	@golangci-lint run

clean: ## Remove build artifacts
	@rm -rf $(BUILD_DIR)

run: build ## Run the app
	@./$(BUILD_DIR)/$(BINARY)

docker-build: ## Build the Docker image
	@docker build -t $(BINARY):$(VERSION) -t $(BINARY):latest .

install: build ## Install to /usr/local/bin
	@install -m 0755 $(BUILD_DIR)/$(BINARY) /usr/local/bin/

Features:

  • := for all the immediate assignments.
  • ?= for ENV (overridable).
  • $(shell ...) for version from git.
  • $(wildcard) via find (covers subdirectories).
  • Proper file target for the binary (caches correctly).
  • | $(BUILD_DIR) as an order-only prerequisite so the dir exists.
  • .PHONY listed at the top.
  • Help target self-documents from ## comments.

This is the shape of a mid-sized project Makefile.


Pattern rules in practice: a multi-file build

Build every .c to .o, then link them into a binary:

SOURCES := $(wildcard src/*.c)
OBJECTS := $(SOURCES:.c=.o)
BINARY := app

.PHONY: all clean
.DEFAULT_GOAL := all

all: $(BINARY)

$(BINARY): $(OBJECTS)
	gcc -o $@ $^

%.o: %.c
	gcc -c -o $@ $<

clean:
	rm -f $(OBJECTS) $(BINARY)

No matter how many .c files are added to src/, this Makefile builds them all without modification. That is the power of pattern rules.

Header dependencies: the -MMD trick

A C file depends on its headers, but you shouldn't hand-list every header. gcc -MMD generates dependency files automatically:

SOURCES := $(wildcard src/*.c)
OBJECTS := $(SOURCES:.c=.o)
DEPS := $(OBJECTS:.o=.d)

%.o: %.c
	gcc -c -MMD -MP -o $@ $<

-include $(DEPS)    # include auto-generated .d files

Now when a header changes, Make correctly rebuilds the .o files that use it. Advanced, but ubiquitous in serious C/C++ builds.


Common variable mistakes

Mistake 1: = where you meant :=

VERSION = $(shell git describe)    # re-runs git every time VERSION is expanded!

If you reference $(VERSION) in 10 places, you run git describe 10 times. Use :=:

VERSION := $(shell git describe)    # runs once

Mistake 2: unquoted shell variables in recipes

NAME := my file
deploy:
	cp $(NAME) /target    # word-splits on space

Variables can contain spaces. Quote them as you would shell vars:

deploy:
	cp "$(NAME)" /target

Mistake 3: forgetting $$ for shell variables

greet:
	name="Alice"
	echo $name    # Make saw $n and expanded it to empty; shell sees "ame"

Double the $:

greet:
	name="Alice"; echo $$name

And chain them with ; or &&, since each line is a new shell:

greet:
	name="Alice" && echo $$name

Quiz

KNOWLEDGE CHECK

You write: VERSION = $(shell git describe --tags) and reference $(VERSION) four times in different targets. What happens each time you run make?


What to take away

  • := is the right default. Use = only when you want late binding, ?= for overridable defaults, += to append.
  • $(shell cmd) runs commands at parse time. Use := so it runs once.
  • $(wildcard), $(patsubst), $(foreach), $(addprefix), $(filter), memorize these. They cover 90% of Make function usage.
  • Pattern rules (%.o: %.c) let one rule cover N similar targets. Combined with $(wildcard), they scale to any number of files.
  • Self-documenting help target via awk parsing ## description comments. Copy it into every Makefile.
  • Order-only prerequisites (| dir) for directories and other things that should exist but shouldn't trigger rebuilds.
  • $$ to escape $ for the shell. Shell variables in recipes need doubling.
  • Chain recipe lines with && if they share state, or use .ONESHELL:.

Next lesson: production Makefile conventions, the build/test/lint/ci/deploy pattern, Docker integration, and when to reach for just or task instead.