01 · FoundationsLab 5 / 59
Guided lablinux
Bash Automation: A Script You Can Trust
Write a backup script that fails loudly instead of silently, and schedule it so a missed run does not go unnoticed.
- Time
- 40 min
- Level
- Beginner
- Objectives
- 4 objectives
- Cost
- Free
Where this fits in the platform
This lab adds
- A script that fails loudly instead of half-running
Which lets you
—
Before you start
You will need
- bash 4+
- systemd or cron
You do not need these already — the lab environment below provides them.
You will be able to
- Write a script that stops at the first real error
- Quote variables so a space cannot become a second argument
- Exit with codes that CI and systemd can act on
- Schedule work and detect a run that never happened
Cost — Free
— no cloud resources
Nothing to pay in the browser. Open the terminal runs this against a simulated cloud — the same API calls and the same commands, with no account and no bill. The figure above applies only if you build it in your own.
The scenario#
There is a backup script on the server. It has 'run successfully' every night for eight months. The backup directory is empty.
It has been exiting 0 the whole time, because nothing in it ever checked whether anything worked.
Hands-on environment
Run this lab in a real terminal, free and in your browser. The environment is temporary and yours alone — break it as much as you like.
Open the terminalOpens in Killercoda, in a new tab — keep this page open for the steps.
Run it on your own machine
Run this lab on your own machine. One command starts the environment, with everything the lab needs already installed:
You will need:
- bash
git clone https://github.com/EgyKode/EgyKode-lab.git
cd EgyKode-lab
./egykode start
./egykode shell
ssh node1You need Docker and Git installed. Everything else runs inside the environment. The first start downloads it and takes a few minutes; later starts are seconds.
Not sure what you already have? Run: npm run doctor — it checks and changes nothing.
Anything you tick here is your own record. EgyKode cannot see inside that terminal, so the success criteria stay self-assessed even when the environment checks your work for you.
The four lines that make a script trustworthy
Step 1 of 3
What you are proving: You can make a script stop at the first real failure, and prove it by breaking a step on purpose
Marking this settles success criterion 1.
Terminal
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'-e— exit on the first failing command. Without it, a script whosepg_dumpfailed carries on to upload an empty file and reports success.-u— an unset variable is an error. This is what stopsrm -rf "$BACKUP_DIR"/*becomingrm -rf /*when the variable was never set.-o pipefail— ina | b, fail if any stage failed. Without it,pg_dump | gzip > out.gzreports success whenevergzipsucceeds, which it does even when it compresses nothing.IFS— split on newlines and tabs, not spaces, so a filename with a space stays one filename.
What you are proving: You can write a backup that can run twice without duplicating or corrupting the last one, and that prunes what is past its retention
Marking this settles success criteria 2 and 3.
Terminal
#!/usr/bin/env bash
set -euo pipefail
BACKUP_DIR="${BACKUP_DIR:-/var/backups/app}"
RETENTION_DAYS="${RETENTION_DAYS:-7}"
STAMP="$(date +%Y-%m-%dT%H-%M-%S)"
TARGET="${BACKUP_DIR}/db-${STAMP}.sql.gz"
log() { printf '%s %s\n' "$(date -Is)" "$*" >&2; }
die() { log "FAILED: $*"; exit 1; }
mkdir -p "$BACKUP_DIR"
log "starting backup -> ${TARGET}"
# Write to a temporary name first. A partial file that is never renamed can
# never be mistaken for a good backup.
tmp="${TARGET}.partial"
pg_dump --no-owner "$DATABASE_URL" | gzip -9 > "$tmp" || die "pg_dump failed"
# A dump that produced nothing is a failure, even though every command exited 0.
size=$(stat -c %s "$tmp")
[ "$size" -gt 1024 ] || die "backup is only ${size} bytes — refusing to keep it"
mv "$tmp" "$TARGET"
log "wrote ${TARGET} (${size} bytes)"
# Retention. -mtime is whole days; this deletes nothing on the first week.
deleted=$(find "$BACKUP_DIR" -name 'db-*.sql.gz' -mtime "+${RETENTION_DAYS}" -print -delete | wc -l)
log "removed ${deleted} backup(s) older than ${RETENTION_DAYS} days"Two details carry most of the value.
Write to .partial, then rename. A rename is atomic. If the machine dies
mid-dump, you are left with a .partial file that no restore will ever pick up
— rather than a truncated file that looks like a backup.
Check the size. This is what the eight-months-of-nothing script was missing. Every command exited 0; the dump was simply empty. A backup that is not checked is a hope, not a backup.
What you are proving: You can schedule it so a run that never happened is visible without anyone reading a log
Marking this settles success criterion 4.
A systemd timer over cron, for one reason — Persistent=true:
ini
# /etc/systemd/system/db-backup.timer
[Unit]
Description=Nightly database backup
[Timer]
OnCalendar=*-*-* 02:30:00
Persistent=true
[Install]
WantedBy=timers.targetPersistent=true runs a missed job when the machine comes back. A cron job on a
machine that was asleep at 02:30 simply never runs, and nothing says so.
Terminal
sudo systemctl enable --now db-backup.timer
systemctl list-timers db-backup.timer
journalctl -u db-backup.service -n 20OnFailure= on the service unit turns a failure into an alert rather than a
log line nobody reads.
The failure is where the learning is. These are the ones that actually happen:
The script exits 0 but the backup is empty
pipefail is not set, so only gzip's exit code was checked. Add set -o pipefail, and check the file size explicitly.
rm deleted more than expected
An unquoted or unset variable. set -u catches the unset case; quoting "$VAR" catches the space case.
The timer never fired
systemctl list-timers shows the next run. If the unit is not listed, it was created but not enabled.
Retention deletes nothing
find -mtime +7 means strictly more than 7×24 hours. On day 7 there is nothing to delete yet — that is correct, not broken.
Success criteria
0 of 4
The concept behind it
Next up
Lab 5 of 59 on the project path