DEV Community: Anguishe

find -perm 777 Missed a World-Writable File in My Test Tree. -perm -o+w Didn't.

Anguishe — Thu, 29 Oct 2026 13:00:00 +0000

Most homegrown permission audits start with the search everybody remembers: find /var/www -type f -perm 777. I built a three-file scratch tree on my own machine to see what that search actually covers. index.html at a normal 644, w666.txt at 666, and w777.sh at 777 — the classic leftover from a chmod 777 thrown at an error to make it go away.

find . -type f -perm 777 found one file: w777.sh. It did not find w666.txt, which any account on the machine can write to.

find . -type f -perm -o+w found both.

That one-character difference is the whole audit. Without a leading dash, -perm matches files whose mode is exactly the one you typed and nothing else; a world-writable file that is not also executable sits at 666 and is invisible to it. With the dash, -perm switches to mask semantics: -perm -o+w, or -perm -002, matches any file with the other-write bit set, whatever the remaining bits say. Reach for the exact form when you are hunting your own 777 leftovers specifically. Reach for the mask when the question is "what can other people write to", because that question does not care about the other eight bits.

Why the other-write bit is the one that matters

A Unix permission check sorts every process into owner, group or other, and a world-writable file hands write access to the last bucket: every account on the machine, including the ones you forgot exist. On a web server that includes the account the server itself runs as. A compromised plugin executing as www-data needs no root and no stolen password — a world-writable file in the web root is one it can rewrite at will, whether that means defacing a page or planting a backdoor in a cache directory. There is no exploit chain in that story. There is an open write bit and something hostile enough to use it.

The fix that locks you out

The tempting cleanup after an audit like that is one recursive command: chmod -R 644 across the whole tree. I ran it on a copy of the scratch tree.

chmod complained before it finished — cannot access 'site2/index.html': Permission denied, once per file — because it stripped the directory's execute bit first and then could not get back in to reach the files inside. Afterwards ls site2 still listed all three names, and cat site2/index.html failed with Permission denied. The files exist, have sane modes, and cannot be reached.

On a directory, the execute bit does not mean "run this". It means "you may enter and traverse this". Take it away recursively and every directory becomes a wall; on a web server that shows up as a wave of 403s for content that is sitting right there on disk. Files and directories need different modes, which means two passes:

find /var/www -type f -perm -o+w 2>/dev/null    # any world-writable file
find /var/www -type f -exec chmod 644 {} \;     # files: rw-r--r--
find /var/www -type d -exec chmod 755 {} \;     # dirs keep x, or nobody gets in

The first line is the audit, the second and third are the repair. Files go to 644, directories to 755, and nothing is walled off.

Keys go further, and ssh enforces it

Secrets go down to 600, and for SSH private keys that is not advice — the tools enforce it. I generated a throwaway ed25519 key in the scratch directory, set it to 644, and asked ssh-keygen -y to read it. It printed the WARNING: UNPROTECTED PRIVATE KEY FILE! banner, Permissions 0644 for 'testkey' are too open, then This private key will be ignored, and exited 255. At 600 the same command loaded the key without comment.

The ssh client makes the same check when it loads a key during a login. It ignores the key and carries on with whatever other methods it has, so what you see depends on what else was available: the banner followed by a password prompt, or the banner followed by Permission denied. Either way the answer is in ls -l, four characters wide, and the fix is chmod 600.

Let the scan remember

None of these three runs involved a malfunction. The exact-mode search did exactly what it says, chmod did exactly what it was told, and ssh refusing a readable private key is the permission model working. What fails is memory: which 777 did I throw at which error, and did I ever close it? A scan with the mask form of -perm answers that in under a second, and a saved report makes the answer comparable from one week to the next.

The full audit script — with the saved report file, the SUID scan for a deeper security pass, and the recommended-modes reference for files, directories, keys and private dirs — is at https://bashsnippets.xyz/snippets/file-permissions-security

If your next move after fixing a key is generating one properly, the SSH key setup script covers that end to end, and when you need a mode you do not have memorized, the Chmod Permissions Builder turns checkboxes into the octal value and the exact chmod command. The rest of the library is at https://bashsnippets.xyz

I Deleted a 48M Log to Free Space. The Next Write Still Hit 'No space left on device'.

Anguishe — Wed, 28 Oct 2026 13:00:00 +0000

A 64M filesystem, a worker that appends to a log and keeps it open the way a daemon does, and the reflex everyone has when a disk fills up: delete the log. I built it on a tmpfs mounted on /srv inside unshare -rm, a private mount namespace with no sudo, where I show up as root. The worker is a bash process that writes 48M to worker.log through file descriptor 3 and then turns into sleep, still holding the descriptor:

$ df -h /srv
Filesystem      Size  Used Avail Use% Mounted on
tmpfs            64M   48M   16M  75% /srv
$ du -sh /srv/app
48M /srv/app
$ rm /srv/app/worker.log
$ du -sh /srv/app
0   /srv/app
$ df -h /srv
Filesystem      Size  Used Avail Use% Mounted on
tmpfs            64M   48M   16M  75% /srv
$ head -c 32M /dev/zero > /srv/app/export.bin
head: error writing 'standard output': No space left on device
exit=1
$ du -sh /srv/app
16M /srv/app
$ df -h /srv
Filesystem      Size  Used Avail Use% Mounted on
tmpfs            64M   64M     0 100% /srv

du went to zero the moment the log was gone. df did not move. Then a 32M write failed at 16M, and the filesystem reported itself 100% full while du could only find the 16M half-file from the write that failed.

Every check you would normally run after a cleanup says the cleanup worked. ls shows no log. du shows almost nothing. Only df, the one number that matters to the next write, disagrees, and it does not say why.

rm removes a name, not the data

A file is an inode, and a directory entry is a name pointing at it. rm removes the name. The kernel frees the blocks only when the inode has no names left and no process has it open. The worker still had descriptor 3 open, so 48M stayed allocated to a file nobody can see. du walks names, so it stopped counting the moment the name went. df asks the filesystem how many blocks are in use, so it kept counting.

lsof +L1 lists open files with a link count below one, which is the definition of deleted-but-open:

$ lsof -w +L1 /srv
COMMAND    PID USER FD   TYPE DEVICE SIZE/OFF NLINK NODE NAME
sleep   608657 root 3w   REG   0,78 50331648     0    3 /srv/app/worker.log (deleted)
exit=0
$ ls -l /proc/$PID/fd | grep deleted
l-wx------ 1 root root 64 Oct  3 18:18 3 -> /srv/app/worker.log (deleted)
$ : > /proc/$PID/fd/3
$ df -h /srv
Filesystem      Size  Used Avail Use% Mounted on
tmpfs            64M   16M   48M  25% /srv
$ ps -o pid,stat,cmd -p $PID
    PID STAT CMD
 608657 S    sleep 600

NLINK 0, (deleted), 50,331,648 bytes (48M exactly), held open for writing (3w) by PID 608657. /proc/PID/fd/3 is a live handle on the deleted file, so : > /proc/$PID/fd/3 truncates it in place. df dropped by 48M and the process kept running with its descriptor open, now pointing at an empty file. For a service, restarting it is the cleaner fix; truncation is for when you cannot, and it throws away whatever was in the file, so aim it at a runaway log and never at a database.

One thing I did not expect: lsof exited 0 here. On a second demo mount holding another deleted file, lsof -w +L1 data, run from the parent directory, and lsof -w +L1 /srv/demo/data printed the same row. The relative path exited 1, the absolute one 0. Same lsof 4.99.4, same file. Read its output, never its exit status.

The next time: truncate, and know how the log was opened

The fix for next time is to empty a live log instead of deleting it: : > /srv/app/worker.log keeps the inode and every open descriptor, and the space comes back at once. There is a catch that depends on how the writer opened the file, so I ran two writers side by side, one opened with >> (O_APPEND) and one with >, each wrote 16M, I truncated both, and each wrote 1M more:

$ : > /srv/app/append.log; : > /srv/app/plain.log
$ ls -lsh /srv/app
total 2.0M
1.0M -rw-rw-r-- 1 root root 1.0M Oct  3 18:18 append.log
1.0M -rw-rw-r-- 1 root root  17M Oct  3 18:18 plain.log

The append writer started again at the new end of the file: 1.0M on disk, 1.0M long. The other writer kept its own offset, 16M, and wrote its next megabyte there. plain.log now claims to be 17M long with only 1.0M allocated, a sparse file whose first 16M is a hole. It costs no disk, but ls -l reports 17M, anything that copies it naively gets 16M of zeros, and anyone sizing logs by ls -l will chase a problem that is not there. That is why logrotate's copytruncate suits append-mode logs (its man page warns it can still lose the lines written between the copy and the truncate), and why for anything else the right move is to make the program reopen its log, usually with SIGHUP or a restart.

The script that does the looking

On a real machine lsof +L1 prints dozens of rows, and a deleted file shared by many processes appears once per process. The page's deleted-open-files.sh groups by inode, skips anything under 1M, sorts biggest first and totals it. On the same demo it printed ✗ 48M /srv/app/worker.log (deleted), held by: sleep[620003] fd 3, a total of 48M and the two ways to free it, and exited 1. After the truncate it printed ✓ no deleted-but-open files over 1.0M on /srv (as root) and exited 0, which makes it usable as a check in a disk alert.

The 48M log was never gone. It had lost its name and kept its blocks, and lsof +L1 was the one command that could see both.

lsof for deleted files, ports and busy mounts, with the grouping script: https://bashsnippets.xyz/snippets/lsof-command-examples

To hear about a filling disk before a write fails, the disk space warning script alerts at a threshold you pick, and Find Large Files on Linux covers what is still on the disk with a name. The rest of the library is at https://bashsnippets.xyz

I Tested a jq Alert Against a Down Status. It Printed 'no alert'.

Anguishe — Tue, 27 Oct 2026 13:00:00 +0000

The shape is everywhere in monitoring scripts: curl a status endpoint, pull one field out with jq, compare it, fire an alert on a match. I wrote the smallest honest version of it on my own machine, jq 1.8.1, against a response that said the service was down:

resp='{"status":"down"}'
status=$(echo "$resp" | jq '.status')
echo "[$status]"                                                 # ["down"]
if [ "$status" = "down" ]; then echo ALERT; else echo "no alert"; fi   # no alert

The service was down. The script said no alert, and exited 0. The echo "[$status]" line shows why: the variable holds "down", quote characters included. jq handed bash a six-character string, bash compared it byte for byte against the four characters in the test, and they never match. Put that in a five-minute cron job and it stays quiet through every outage it exists to report.

Adding two characters, jq -r, changed the output to [down] and the script to ALERT.

jq speaks JSON, bash speaks bytes

jq's default output format is JSON, and a JSON string includes its own quotes, so printing "down" is jq being correct. Bash's = has no idea that quotes might be markup; it compares the bytes it was given. -r tells jq to emit the string's contents rather than its JSON representation, which is why nearly every value headed into a shell variable, a filename or a test wants it.

That is the first of three ways a jq one-liner fails while looking right. The second: asking for a key the response does not have is not an error. On the same response, jq -r '.region' printed null and exited 0. In bash that is the literal four-character string null, which passes [ -n ] checks and walks straight into filenames and log lines. The // operator is the guard: jq -r '.region // "unknown"' printed unknown.

The third is select(), and here the folklore is half wrong. Forget the double quotes around a string inside the filter — select(.name == redis) — and jq does not quietly match nothing. It refuses to compile, prints redis/0 is not defined, and exits 3. That one is loud, and set -e will catch it. The silent version is a value that is spelled right but does not exist: select(.name == "nginx") against a list with no nginx in it printed nothing and exited 0. So did a case mismatch. Your script then treats that silence however it treats an empty string.

Three traps, and two of them share one property: plausible output, zero exit code.

Build the filter against the response you already have

The usual way to debug all three is trial and error against the live API: tweak the filter, re-run the curl, squint, repeat, often against a rate-limited endpoint. The jq Filter Builder turns that around, because you already have the JSON.

Paste a real response, or load one of the bundled samples, and it renders the parsed structure as a clickable tree. Click a field and the path is built for you, however deep it is nested. Arrays get the full treatment: switch on per-element iteration, attach a select() whose field name comes from a dropdown filled with the keys that actually exist on the elements, so a misspelled key stops being possible, and project one field out of each match. The comparison value is written the way jq reads literals — true, false and numbers bare, everything else double-quoted for you inside the single-quoted program. The quoting decision is made mechanically.

Below that sit two copyable outputs: the bare filter for a script, and the full curl -s … | jq command with your URL in place, -r included when you ask for raw output. And a live preview, evaluated in the browser against the JSON you pasted, on every change. Leave -r off and the quotes are right there in the preview. When the result is empty, the preview says why — No elements matched (jq would output nothing), Path not found, or that a // empty default is deliberately producing nothing — so empty output stops being ambiguous.

It runs client-side: nothing is uploaded and no remote jq is invoked, so a response from a production system stays in your browser. Scope, honestly: it builds the extraction patterns that cover most API scripting — nested access, array indexing, iteration with select() and projection, // defaults, raw output. Reduce, string interpolation and arithmetic still belong to the manual.

The four-line check at the top of this post could not have shipped from the builder. The quotes show up in the preview before the script exists.

Build a filter against your own API response: https://bashsnippets.xyz/tools/jq-filter-builder

The parse JSON with jq snippet covers these patterns in script form with the hardening around them, curl for API requests is the other half of the same pipeline, and the rest of the library is at https://bashsnippets.xyz

grep "error|failed" Found Nothing in a Log That Said 'backup failed'

Anguishe — Thu, 22 Oct 2026 13:00:00 +0000

I wrote a three-line backup log on my own machine: a start line, tar: /var/www/uploads: Cannot open: Permission denied, and backup failed (exit 2). Then I ran the check a lot of people keep as a weekly ritual, the one that looks like it asks "did anything go wrong?": grep "error|failed" backup.log.

GNU grep 3.12 printed nothing and exited 1. The last line of the log says failed in plain text.

grep -E "error|failed" backup.log printed that line and exited 0.

Nothing about the first command is malformed, so there is no error message. Exit 1 means "no lines matched", which is exactly what grep returns for a genuinely clean log. A monitoring check wired to that exit code reports all clear on a log full of failures, every single run, and the empty output does not start an investigation. It ends one.

The pipe that isn't a pipe

grep's default engine is Basic Regular Expressions, and in BRE the pipe is not alternation. It is a literal character. error|failed does not mean "error or failed"; it means the twelve-character string error|failed, pipe included, which no logger has ever written. The same goes for +, ? and parentheses: in BRE they are ordinary characters unless you backslash-escape them. Alternation the way every other regex engine behaves needs -E, Extended Regular Expressions — the mode egrep has been shorthand for since long before most of us started typing it.

# BRE (default): the pipe is a literal character — matches nothing, exits 1
grep "error|failed" backup.log

# ERE: the pipe means OR — the search you actually meant
grep -E "error|failed" backup.log

GNU grep also accepts grep "error\|failed" in BRE mode — on my machine that matched and exited 0 — but the escaped pipe is a GNU extension that POSIX does not define, so it is a habit that stops working on the systems that do not carry GNU grep. -E is the portable spelling.

The deeper trap is in the exit codes. grep exits 0 on a match, 1 on no match and 2 on an actual error; pointed at a file that did not exist, it printed No such file or directory and exited 2. An impossible pattern is not an error. "No matching lines" and "you searched for a string that cannot exist" produce identical output: none, with the same status. A false negative is the worst failure a search tool has, because nobody goes looking for a problem that grep has already said is not there.

The alias is a crutch

The tempting fix is alias grep='grep -E' in your shell rc file, and it does make the interactive ritual behave. It also does nothing anywhere that matters. Aliases apply only to interactive shells, so every cron job and script on the same box still runs BRE grep. The alias does not exist on any machine you have not configured, which is most of the machines you will ever SSH into. And it trains your fingers to type patterns that silently degrade the moment it is absent. The durable fix is knowing which engine a grep runs under and what its silence means before you trust it — and writing -E into the script itself.

So the builder shows the engine and the empty result

The grep Pattern Builder assembles the command live as you type: pattern, path, and an optional comma-separated list of file types that it turns into --include flags, so a log search does not wade through binaries. Toggles cover the flags that earn their keep — case-insensitive, recursive, line numbers, invert, count, filenames-only, whole-word, quiet and -B/-A context lines — and every combination produces a plain-English sentence saying what the command will do before you run it.

The engine picker sits front and centre: BRE, -E and -P, each with a note about what is literal where. The BRE note says it outright: + ? | ( ) are literal unless escaped. The -P note warns that Perl-compatible regex is GNU-only and fails on macOS and BSD grep, the portability cousin of the same trap: a \d pattern that works on a Debian server errors out on a Mac.

There are guardrails for mistakes people ship. It turns on -r when the path looks like a directory, because plain grep pointed at a directory refuses with "Is a directory", and in a script with stderr redirected that refusal is invisible. It drops -n when -c, -l or -q is active, because those flags change the output mode and line numbers stop meaning anything.

The panel that makes the ritual trustworthy is the live tester. Paste a few sample lines, including one real failure line from your actual log, and it highlights which lines the current pattern matches, with a count. In BRE mode it treats a bare pipe literally, the way grep does, so error|failed against a pasted backup failed line highlights nothing: zero out of one, a number that makes you look up. Switch to -E and the line lights. The preview runs on JavaScript's regex engine, which is close to ERE and PCRE for everyday patterns — treat it as a check on your pattern, not a replacement for running grep.

One unescaped character turned a failure search into a search for a string that cannot exist, and grep answered that question with perfect accuracy. The command was valid, ran clean, exited 1, and reported nothing wrong.

Build the command and read the explanation before you trust it: https://bashsnippets.xyz/tools/grep-pattern-builder

The Search Files for Text snippet covers the grep patterns worth keeping, Delete Old Log Files handles the other half of log hygiene, and the rest of the library is at https://bashsnippets.xyz

I Moved a Script to Another Directory on PATH. bash Said 'No such file or directory' and Exited 127.

Anguishe — Wed, 21 Oct 2026 13:00:00 +0000

One bash process, one small demo script called report, and a mv between two directories that are both on PATH. I set it up in a scratch tmpfs mounted over /opt inside unshare -rm, with PATH starting /opt/jobs/bin:/opt/jobs/newbin, and ran it as a script. Inside the namespace I appear as root, which is why ls says root root:

$ report
report 2.1: 14 jobs ok
exit=0
$ mv /opt/jobs/bin/report /opt/jobs/newbin/report
$ ls -l /opt/jobs/newbin/report
-rwxrwxr-x 1 root root 42 Oct  3 18:26 /opt/jobs/newbin/report
$ report
session.sh: line 5: /opt/jobs/bin/report: No such file or directory
exit=127
$ type -a report
report is /opt/jobs/newbin/report
$ hash -t report
/opt/jobs/bin/report
$ command -v report
/opt/jobs/bin/report
$ hash -r
$ report
report 2.1: 14 jobs ok
exit=0

The file exists, it is executable, and its new directory is on PATH. bash ran the old path anyway, said No such file or directory, and exited 127, the same code a typo gets. I ran the same sequence through an interactive bash -i to be sure it was not a script quirk, and got bash: /opt/jobs/bin/report: No such file or directory, exit 127.

What makes it hard to trust your own eyes is the middle of that output. type -a names the new location. command -v, the check most scripts use to ask "is this installed", names the old one.

bash remembers where it found things

The first time bash runs a command through a PATH search, it stores the full path in a hash table, so the next call can skip the search. That is what hash -t report prints. When the file moves, bash does not search again; it executes the remembered path, the kernel answers ENOENT, and bash reports No such file or directory with the stale path in the message.

type -a searches PATH fresh every time, because its job is to list every match. command -v reports what bash would run, and bash would run the hashed path. So the two most common "where is it" checks disagree, and the one that agrees with the error is the one that looks wrong.

hash -r empties the table. A new terminal starts with an empty one. And there is a shell option made for exactly this:

$ shopt -s checkhash
$ report
report 2.1: 14 jobs ok
exit=0
$ mv /opt/jobs/bin/report /opt/jobs/newbin/report
$ report
report 2.1: 14 jobs ok
exit=0
$ hash -t report
/opt/jobs/newbin/report

With checkhash set, bash checks that a hashed path still exists before running it, searches PATH again when it does not, and updates the table. In the wild this shows up after an upgrade moves a binary between /usr/local/bin and /usr/bin, or after swapping a pip-installed tool for the distro package, both of which the page describes.

Six causes, one exit code

command not found and its 127 cover more than a stale hash. The page reproduces the rest: a typo, a program installed outside PATH, a script in the current directory run without ./, an alias a script cannot see, a found file whose shebang names a missing interpreter (a CRLF line ending does that), and the cron job whose PATH is shorter than your terminal's.

The cron one is worth reproducing, because it never fails while you are watching. Below, a demo tool in a scratch directory on my interactive PATH, then the same call with cron's two-directory PATH, which env -i reproduces without waiting for the schedule. After that, the page's diagnosis script, why-command-not-found.sh, run in the namespace with a home directory of its own, shown as ~, and the same tool in ~/.local/bin:

$ nightly-sync
nightly-sync 0.4
exit=0
$ env -i PATH=/usr/bin:/bin bash -c 'nightly-sync'
bash: line 1: nightly-sync: command not found
exit=127

$ ./why-command-not-found.sh nightly-sync
✓ nightly-sync resolves to ~/.local/bin/nightly-sync
✗ cron's default PATH (/usr/bin:/bin) will NOT find it: use ~/.local/bin/nightly-sync in the crontab, or set PATH= at its top
  if your terminal still says 'not found' or 'No such file', its hash table is stale: run hash -r
exit=0
$ ./why-command-not-found.sh gti
✗ gti is not on PATH
  did you mean git? (/usr/bin/git)
exit=1

It uses type -P, which searches PATH for files only and ignores aliases and functions, so it sees what any other script would see. It repeats the lookup with PATH=/usr/bin:/bin to catch the cron case, reads the first line of the file for a carriage return, checks ./NAME, looks in the usual places installers put binaries without touching PATH (~/.local/bin, ~/.cargo/bin, /usr/sbin, /opt/*/bin), and tries every adjacent-letter swap of the name, which is how gti became git. It exits 1 for every cause it finds, so ./why-command-not-found.sh rsync || exit 1 can gate a deploy script.

The order that separates them

type -a NAME for everything bash can see. hash -t NAME when the message says No such file or directory about a path. echo "$PATH" | tr : '\n' when the answer is "nothing". And env -i PATH=/usr/bin:/bin bash -c NAME before anything goes in a crontab.

The stale hash is the one cause where everything you check says the tool is fine: ls and type -a both show it right where it should be. One hash -r, or one new terminal, is the whole fix.

All six causes reproduced, with the diagnosis script and what each check does: https://bashsnippets.xyz/snippets/bash-command-not-found

If 127 is all a log gives you, the Bash Exit Code Lookup explains it next to 126, and the Bash $PATH Debugger checks a pasted PATH for missing directories and duplicates. The rest of the library is at https://bashsnippets.xyz

I Pointed rsync --delete at a Nearly Empty Directory. It Deleted 3,800 Files and Exited 0.

Anguishe — Tue, 20 Oct 2026 13:00:00 +0000

The rsync accident that ends backups is not an error. It is rsync doing exactly what it was told, successfully, to the wrong directory. I wanted to see how quiet that is, so I rebuilt it in a scratch directory on my own machine with rsync 3.5.0.

The destination held 3,800 files, standing in for a nightly mirror. The source held one file, index.nginx-debian.html — what Debian's nginx package leaves behind in /var/www/html after you move a site's web root somewhere else and forget that the backup script still points at the old path. Then the mirror command anyone would have in that script: rsync -a --delete src/ dst/.

It exited 0. Afterwards the destination held one file: index.nginx-debian.html. Three thousand eight hundred files gone, in a run that any cron wrapper, healthcheck ping or ✓ backup complete log line would have recorded as a success. Run that nightly and every green checkmark in the dashboard is a record of the deletion.

Exit 0 doesn't mean what your monitoring thinks

The bug is not in rsync. It is in what we assume its exit code promises. rsync reserves non-zero codes for failures on its own terms: an unreachable host, a protocol error, an I/O failure, source files vanishing mid-transfer (code 24), a partial transfer (code 23). Making a destination match a near-empty source is none of those. It is the requested operation, delivered without incident. A monitor that checks the exit code answers "did rsync finish?" and most of us have labelled that answer "is the data safe?" Those are different questions.

rsync's sharp edges fail by succeeding

Once you see that shape, you find it all over the command. A trailing slash on the source means "copy the contents"; no trailing slash means "copy the directory itself", which nests a second directory inside your destination and, on a --delete mirror, changes what gets compared and therefore what gets removed. Both versions run clean. Drop -a and rsync stops preserving permissions, symlinks and timestamps, and because its quick check decides "already transferred" by size and modification time, a destination full of transfer-time mtimes never matches again, so every run quietly becomes a full re-copy that still exits 0. Hand a comma-separated list to a single --exclude and it is treated as one pattern with a comma in it, so it matches nothing and node_modules rides along every night. None of these produce an error. Each completes a different operation from the one in your head.

The flag that shows the damage first

The habit that prevents the whole class is previewing every destructive sync. Same scratch directories, same command, with --dry-run and -v:

$ rsync -a --delete --dry-run -v src/ dst/
sending incremental file list
deleting photo-999.jpg
deleting photo-998.jpg
deleting photo-997.jpg
…

Thousands of deleting lines, and afterwards the destination still held all 3,800 files. A wall of deletions in a preview you are reading is impossible to miss. The same wall executed in a log nobody opens is a green checkmark.

One detail the documentation does not shout: the -v is not optional. I ran --dry-run without it against a second scratch directory and it printed nothing at all and exited 0. A dry run you cannot see is not a preview. Use -v, or --itemize-changes if you want one line per file with a change code.

So the builder makes the preview the default

The Rsync Command Builder assembles the command in a live preview while you toggle the real options: archive mode, verbose output, compression, resume for unreliable links (--partial --progress), SSH transport, a bandwidth cap for syncs that share a link with people trying to use it, and exclude patterns. You type the excludes comma-separated and it emits one individually quoted --exclude flag per pattern, so the shell never expands your glob and the comma mistake cannot happen.

The opinions built into it are the point. Turn on --delete without dry-run and a red warning appears: those deletions are permanent, preview them first. Turn dry-run on and it switches to a green note telling you to remove --dry-run only after you have read the output. The Mirror preset ships with dry-run already enabled — mirroring is the one mode where you switch the safety off on purpose instead of remembering to switch it on. The whole thing runs in your browser; nothing you type leaves the page.

The scratch run took a fraction of a second to delete 3,800 files and report success. The preview took the same fraction of a second to list every one of them. The only difference was a flag.

Build the command with the guardrails on: https://bashsnippets.xyz/tools/rsync-command-builder

For the script around it — cron scheduling, SSH keys, a hardened nightly job — the Rsync Remote Backup snippet pairs with this tool, Automated File Backup covers the local variant, and the rest of the library is at https://bashsnippets.xyz

My Machine Has No mail Command. That's Where Most Disk Alerts Die.

Anguishe — Thu, 15 Oct 2026 13:00:00 +0000

I recently found that the disk check on my own Linux machine had been printing WARNING: Disk at 88% into nothing: cron's journal said No MTA installed, discarding output beside every run. So I checked what it would take for that warning to reach me by email. mail: not installed. mailx: not installed. sendmail: not installed. msmtp: not installed.

That is normal for a desktop install and common on minimal server images. It also means every monitoring script that ends in | mail -s "…" you@example.com is a script whose last line fails. Run it by hand and you would at least see command not found. Run it from cron and the error goes the same way as the output: cron tries to mail it to you, has no mail agent to do it with, and discards it. Two layers of silence, and the check itself is working perfectly.

Detection and delivery are different jobs

The detection logic is the part everybody polishes: read the number, compare it, decide. Telling a person is a separate job, and a monitor that only appends to a log file or prints to stdout has not done it — it has relocated the problem. Somebody still has to poll that output, and that somebody is a person with better things to do at 2am. The fix is to push the news to the place you already check compulsively, which for most of us is an inbox.

The core is small enough to memorize:

#!/bin/bash
set -euo pipefail

USAGE=$(df / | awk 'NR==2 {print $5}' | tr -d '%')
if [ "$USAGE" -gt 80 ]; then
  printf 'Disk on %s hit %s%% at %s\n' "$(hostname)" "$USAGE" "$(date '+%F %T')" \
    | mail -s "[ALERT] disk space on $(hostname)" you@example.com
fi

Read the number, compare it, and when it is over the line, pipe a message into mail with a subject that names the machine — because once the same check runs on several hosts, an alert that does not say which server is on fire is a puzzle, not an alert.

The alert you "sent" that never arrived

Installing mail fixes the first layer and exposes the next one. mail exiting 0 does not mean your alert reached anyone. It means the message was handed to whatever mail transfer agent lives on the box; delivery is the MTA's problem, and the exit code you can observe stops at the handoff. On a server with no MTA configured for the outside world, that handoff often drops the message into a local mailbox under /var/mail — a file on the same machine that no human reads. The script reports success, and the alert sits a few directories away from the log it was supposed to replace.

Cron adds one more trap on top. Jobs run with a minimal PATH, so a mail that resolves in your login shell can be command not found under cron. The test that proves anything is not running the script by hand and receiving an email. It is letting cron run it and receiving an email, then reading the MTA's log to see what it actually did.

Real delivery means giving the box a real path out. On Debian or Ubuntu, mailutils pulls in an MTA, and the "Internet Site" option covers a machine that is allowed to send directly. Plenty are not — outbound mail from cloud IP ranges gets filtered or spam-binned constantly — so relay through a provider instead: msmtp pointed at a Gmail account with an app password is a few lines of config, and curl can speak SMTP to a relay on boxes where you would rather not install an MTA at all. Either turns "handed to the MTA" into "accepted by a server whose whole job is delivering".

The opposite failure: the alert that cries wolf

Once delivery works you meet the inverse problem. A threshold does not trip once; it stays tripped. Disk at 81 % at nine o'clock is still 81 % at ten, and an hourly cron job will mail you the same fact twenty-four times a day until you fix the disk or stop reading, which comes first. An alert channel you have learned to skim is a silent failure with extra steps.

The guard is a dedupe sentinel: on send, touch a marker file; before sending, check the marker's age with find "$MARKER" -mmin -1440. I ran that logic three times on my machine. The first call sent. The second, a moment later, was suppressed. Then I backdated the marker 25 hours with touch -d and the third call sent again. One email per incident per day; the condition is still measured every hour, only the nagging is suppressed.

Delivery is the whole difference

The check on my machine had the right threshold and the right schedule-shaped intentions. What it lacked was any route from the machine to me, and nothing in its own output could say so, because its output had nowhere to go either. Before you trust an alert, send one on purpose, from cron, and watch it arrive.

The full script — the alert body with the top disk consumers included, the msmtp config for a Gmail relay, the once-per-24-hours dedupe guard, and the cron traps that make alerts vanish — is here: https://bashsnippets.xyz/snippets/bash-send-email-alert

An email alert is the last guard an unattended job needs, not the first: the Cron Wrapper Generator wires alert-on-failure in alongside the lock, timeout and retry it generates, and Bash Scripts That Survive Cron is the full playbook for jobs nobody watches. The rest of the library is at https://bashsnippets.xyz

Bash Said '-eq: unary operator expected'. The Script Still Exited 0.

Anguishe — Wed, 14 Oct 2026 13:00:00 +0000

Four lines of bash, one empty variable, two tests of it. I wrote it on purpose, in a scratch directory, to capture the error for a guide. The output is not what I expected to capture:

$ cat queue-check.sh
#!/bin/bash
count=""
if [ $count -eq 0 ]; then echo "queue empty"; fi
if [[ $count -eq 0 ]]; then echo "queue empty ([[ ]])"; fi

$ bash queue-check.sh
queue-check.sh: line 3: [: -eq: unary operator expected
queue empty ([[ ]])
exit=0

The first test printed an error. The second printed an answer, and the answer is wrong: count is not zero, it is empty. Then the script exited 0, so anything watching the exit code, a cron wrapper or a CI step, saw a clean run.

That is three separate lies from one bug, and only one of them is on stderr. The part I keep coming back to: if I had only written the [[ ]] line, the way most modern bash guides tell you to, there would have been no message at all. I would have trusted "queue empty".

The message names the check, not the cause

unary operator expected points at -eq. The bug is the variable before it. $count is unquoted, and an empty unquoted variable is removed by word splitting before [ ever runs. So [ received -eq 0 instead of "" -eq 0, tried to read -eq as a one-argument test like -f, and gave up with exit status 2. Inside an if, status 2 counts as "not true", so the branch was skipped and the script carried on. [[ ]] does no word splitting, so it saw an empty operand, and in an arithmetic comparison an empty string evaluates to 0. True.

The fix is to quote it, [ "$count" -eq 0 ], which turns the error into an honest integer expected, or to give it a default with "${count:-0}" when empty really does mean zero.

That pattern, a message that names the check that failed rather than the reason, holds for almost every bash error I reproduced for the guide. Here is the syntax-error version, from a script missing one semicolon:

$ bash missing-then.sh
missing-then.sh: line 4: syntax error near unexpected token `fi'
missing-then.sh: line 4: `fi'
exit=2

$ shellcheck missing-then.sh

In missing-then.sh line 2:
if [ -f /etc/hostname ] then
                        ^-- SC1010 (warning): Use semicolon or linefeed before 'then' (or quote to make it literal).

bash blames line 4. The mistake is on line 2, where then without a ; before it became one more argument to [. bash parses a whole if ... fi block before running it, and it reports the first token that cannot fit, which is the fi with no then open. The line number is where bash gave up, not where you went wrong. ShellCheck pointed at the right line on the first try.

Permission denied, after chmod +x

The one that wastes the most time is Permission denied when the execute bit is already set. I reproduced it on a tmpfs mounted noexec, inside a throwaway mount namespace so nothing real was touched:

$ chmod +x backup.sh
$ ./backup.sh
bash: line 1: ./backup.sh: Permission denied
exit=126
$ findmnt -no OPTIONS --target backup.sh
rw,noexec,relatime,uid=1000,gid=1000,inode64

chmod succeeded, the mode says executable, and the kernel still refused, because the filesystem forbids executing anything on it, root included. Exit 126 means "found it, could not run it". The other causes chmod cannot reach are a parent directory missing its x bit and a file owned by someone else, and namei -l on the full path shows both in one listing.

The rest of the nine

The guide covers nine messages, each reproduced on my machine with the output pasted as it came out. command not found has six causes behind one exit code, 127, and type -a plus a look at cron's two-directory PATH separates them. bad interpreter: No such file or directory complains about a file that plainly exists, because the shebang ends in a Windows carriage return. No space left on device can fire on a filesystem df -h reports as 0% used, when the inodes ran out instead of the blocks. Connection refused can come from a service that is listening, on 127.0.0.1 only. And Argument list too long means rm never started, so nothing was deleted.

The order I read any error in now

The message first, word for word, with the variable parts removed. Then the exit code: 127 not found, 126 found but not executable, 2 usually a syntax or usage error, 128+N killed by signal N. Then bash -x, which prints each command after expansion, so you see the [ -eq 0 ] bash actually ran instead of the [ $count -eq 0 ] you wrote. Then ShellCheck, which flags the unquoted $count as SC2086 before the script runs at all.

Then validate the value before you test it. I re-ran the same check under set -euo pipefail, and strict mode did nothing: count="" is set, so set -u stays quiet, and set -e ignores a failing command inside an if condition by design. The script printed the same error and then still running. What stopped it was one line that says what the variable must look like, [[ $count =~ ^[0-9]+$ ]] || { echo "count is not a number: '$count'" >&2; exit 1; }, which exited 1 and printed count is not a number: ''. A quoted variable makes the error honest. A validated one makes the script stop.

Full guide with all nine messages, the one check that tells each one's causes apart, and links to the fix for every case: https://bashsnippets.xyz/guides/bash-error-messages

If an exit code is all you have, the Bash Exit Code Lookup decodes it, and the ShellCheck Error Decoder explains the common SC codes ShellCheck prints. The rest of the library is at https://bashsnippets.xyz

grep Found 5 Matches for a Hostname I Expected 3 Times. sed Would Have Rewritten All 5.

Anguishe — Tue, 13 Oct 2026 13:00:00 +0000

To see how a routine rename goes wrong, I built a small config tree on my own machine. conf.d/app.conf points an upstream at api.internal:8080. conf.d/health.conf has a health-check URL on the same host. samples/example.conf is a sample shipped as documentation and should never change. conf.d/docs.conf sets doc_host = apixinternal.example.com, a different host that happens to look similar. And cache.bin is a binary file that contains the string. The job: move the live config from api.internal to api-v2.internal.

The reflex is one line: sed -i "s/api.internal/api-v2.internal/g" $(grep -rl "api.internal" myapp). It exited 0. Afterwards the two live files were correct — and docs.conf now read doc_host = api-v2.internal.example.com, the sample had been rewritten, and the binary cache had been edited in place. Five files touched, two intended, and nothing in the output to say so.

Two mechanisms, one bad afternoon

The shallow one is regex. In api.internal that dot is not punctuation; it is a wildcard matching any single character, in grep and sed alike. So the pattern does not mean "this hostname", it means "this substring, with anything in the middle, wherever it appears", which is how apixinternal matched. When you mean a literal dot, escape it (api\.internal); when you mean a literal string, grep -F says so outright.

The deep one is order. sed -i writes over the original file the instant the pattern matches — no preview, no confirmation, no undo unless you supply one. Text processing on a live tree is a pipeline with a sequence: find decides which files may be touched, grep shows what will actually change, sed or awk makes the change, and a diff earns your trust afterwards. The one-liner started at step three and pointed it at everything grep could see.

grep: count the matches before you rewrite them

The search that should come first is the same one the one-liner hid inside $( ). Run on its own, grep -rn "api.internal" myapp printed five hits: the two live lines, the sample, the apixinternal line in docs.conf, and binary file matches for the cache. I expected three. That mismatch is the entire value of the step: the extra two lines are the damage, caught at read time instead of write time.

With the dot escaped and the search limited to config files — grep -rn 'api\.internal' myapp --include="*.conf" — the count dropped to three: the two live lines and the sample. Closer, and still one file too many.

find: print the blast radius

Every bulk edit begins with a file list, and the default list, "everything under this directory, including whatever I forgot lives there", is the dangerous one. find myapp -type f -name "*.conf" -not -path "*/samples/*" printed three candidates: app.conf, docs.conf and health.conf. The -not -path is the single flag that keeps documentation out of a rename. Read that output like a checklist; anything destructive should only ever receive filenames that appeared on it.

sed: transform with an undo you did not have to build

-i.bak edits in place and leaves a .bak copy of every original: a full rollback for the price of four characters. It is also the portable spelling. GNU sed treats the backup suffix as optional, but BSD and macOS sed require one, so a bare sed -i 's/old/new/' that works on Linux swallows your expression as the suffix on a Mac.

Composed, on a fresh copy of the same tree:

# Scope with find, confirm with grep, transform with a per-file undo
find myapp -type f -name "*.conf" -not -path "*/samples/*" \
  -exec grep -l 'api\.internal' {} + \
  | xargs sed -i.bak 's/api\.internal/api-v2.internal/g'

It edited app.conf and health.conf and left a .bak beside each. docs.conf was a candidate from find but grep -l with the escaped dot never selected it, so apixinternal.example.com survived. The sample was never a candidate. The binary cache was never a candidate. Two files changed, the two I meant. (If the tree can hold filenames with spaces, grep -lZ into xargs -0 closes the word-splitting gap.) Then diff one file against its .bak before believing the run.

awk: when the question is a column, not a line

grep and sed think in lines. The moment your question is about a field — total bytes served from column 10 of an access log, requests grouped by status code — that is awk, which splits every line into $1 through $NF and runs a small program per line. Its END block fires once after the last line, which is how a million-line log becomes a single number. Chain all four stages and you get the command worth keeping for incidents: find scopes to today's logs, grep pulls the error lines, awk strips the timestamp fields so identical errors collapse together, and sort | uniq -c | sort -rn | head ranks them.

The list worth printing

The unscoped one-liner did nothing wrong by its own rules: pattern found, substitution made, exit 0, five times. The difference between five files and two was an explicit file list, an escaped dot, and one grep read before the write. Scope, confirm, transform, verify. In that order this work is boring, and boring is the goal.

The full guide, with each stage's flags and the incident-triage pipeline in copy-paste form: https://bashsnippets.xyz/guides/bash-text-processing

If you would rather assemble the flags than memorize them, the find command builder and grep pattern builder construct and explain the exact invocation before you run it. The rest of the library is at https://bashsnippets.xyz

find -mtime +30 Kept a File That Was 30.5 Days Old. I Checked With touch -d.

Anguishe — Thu, 08 Oct 2026 13:00:00 +0000

Nothing on my Linux machine deletes anything by age unless I tell it to. ~/.cache alone holds 47 GB right now — 13 GB of pip downloads, 11 GB of uv, 5 GB of browser cache — each one a program doing exactly its job, none of them with a retention policy. Application logs have the same shape: a writer that never stops and no reader. That is how a disk goes from fine to full without a single thing malfunctioning.

The standard fix is an age-based cleanup with find. It is also a command that deletes files, so before trusting it with anything real, I tested its three sharp edges on files I backdated with touch -d.

Why a full disk lies to you

The reason prevention beats diagnosis here is what a full disk looks like when it arrives. The moment the filesystem runs out of blocks, every process that tries to write gets ENOSPC back from the kernel, and almost no application reports that as "the disk is full". You get a failed save in one program, a crashed build in another and a database complaint in a third, all worded differently, while the process that filled the disk goes quiet because it cannot write either. By the time symptoms appear they point everywhere except the cause.

Aging files out with find

The tool has shipped with every Linux and macOS box for decades:

LOG_DIR="/var/log/myapp"
DAYS=30

# Preview pass — read this list before letting anything delete
find "$LOG_DIR" -type f -name "*.log" -mtime +"$DAYS" -print

Run that, read the output, and when the list holds exactly what you expect, swap -print for -delete and run it again. The three runs below are why the preview is not optional.

Edge one: -mtime +30 does not mean 30 days

I created four logs dated 29 days, exactly 30 days, 30 and a half days, and 31 days back. find . -name "*.log" -mtime +30 -print printed one of them: the 31-day file. The 30.5-day file survived.

find measures age in whole 24-hour periods and throws the fraction away, so 30.5 days old counts as 30, and the + means strictly greater than. -mtime +29 printed three files: 30, 30.5 and 31 days. If your policy says "keep 30 days", +29 is the flag that enforces it. The off-by-one is invisible in a quick test and surfaces a month later as "why is that file still there".

Edge two: -delete goes last, always

On a copy of the same directory I put the predicates in the wrong order: find . -delete -name "*.log". Afterwards the directory had zero entries. Every log, the non-log keep.txt, everything.

find evaluates its expression strictly left to right, and -delete is an action that runs and returns true. Put it first and it executes on every file find walks, before -name ever gets a vote. There is no warning and no prompt. The fix is order: tests first, -delete at the very end.

Edge three: never pipe filenames to rm

The reflex is find … | xargs rm. I made four files — app v2.log, app, v2.log and keep.txt — and ran find . -name "app v2.log" | xargs rm. When it finished, app and v2.log were gone. app v2.log, the one file the command was aimed at, was still there.

xargs split the filename on the space and handed rm two arguments, both of which happened to exist. That is the worst possible outcome: the target survives, two innocent files die, and rm reports nothing unusual. -delete never leaves find — no pipe, no word splitting — so it cannot do that. If you genuinely need another command, -print0 | xargs -0 keeps each name in one piece.

The blast-radius controls

-type f keeps directories and symlinks out of the match. And the path should name one application's log directory — /var/log/nginx, /var/log/myapp — never /var/log wholesale, which holds logs the OS still needs and files that must never be deleted by age. Retention for a database transaction log is a backup problem, not a find problem.

Then take yourself out of the loop

The version that actually prevents a full disk is the one that runs without you: the path and the age limit in named variables, a crontab line running it weekly, and the cleanup's own output appended to a log with a timestamp, so every run leaves a record. Whatever grows without bound gets aged out on a schedule, long before it matters.

Three runs on backdated files took less time than reading this post. Each one would have been an incident on a real directory.

Full script with the multi-directory loop, the .gz variant for rotated logs, a before/after disk-usage report and ready-made cron lines: https://bashsnippets.xyz/snippets/delete-old-log-files

Deleting old logs is the recovery; hearing about a filling disk before it hits 100 % is the upgrade — the disk space warning script does that, and when the space thief is not a log at all, find large files names it. The rest of the library is at https://bashsnippets.xyz

My Disk Check Prints WARNING at 88%. Cron Throws It Away Every Monday.

Anguishe — Tue, 06 Oct 2026 13:00:00 +0000

There is a small disk check in my home directory that I wrote months ago. It reads / with df, strips the percent sign, and prints WARNING: Disk at N% once usage passes 80. A crontab line runs it on Mondays. Today I ran it by hand for the first time in a while, and it answered WARNING: Disk at 88%.

Then I went looking for where that warning goes when cron runs it, and the answer was nowhere. The log file sitting next to the script was last written on May 2, and its newest line says OK: Disk at 30% — those were manual runs, because the crontab line never redirected anything into it. Cron's own journal filled in the rest. Last Monday it ran the script 54 times between 05:00 and 05:59, because the schedule reads * 5 * * 1, every minute of hour five, where I meant once. And beside those runs, the same journal says No MTA installed, discarding output. Every warning that check has produced under cron went into a mail system this machine does not have.

Nothing about that is exotic. It is the most common way a monitoring script fails: it runs, it detects the condition, and the channel between it and a person does not exist.

A full disk never introduces itself

The reason that channel matters is what the failure looks like when it arrives. When a filesystem hits 100 %, nothing says "the disk is full." Every process that needs a write fails in its own vocabulary: the database complains about its own files, the app dies persisting a session, and every error points at the component that surfaced it rather than the cause underneath. One root cause, several convincing disguises.

Logging is a write too, so the logs stop at the moment the disk fills, and the recorded evidence ends right before the incident begins. And ext4 holds back about 5 % of blocks for root, so a root shell keeps working on a disk where every non-root service is starving. The machine feels healthy to the person debugging it.

There is a second trap on the way back down, and this box taught me that one too. On August 28 I deleted a 44.6 GB video file here. du dropped by 44 GB; df moved by 5. XFCE's thumbnailer daemon, tumblerd, had the file open, and the kernel does not release blocks while any process still holds a descriptor on the deleted inode. pkill -x tumblerd gave the space back instantly. Judge a cleanup by df, never by du: du counts names that are gone, df counts blocks that are actually free.

The check, done so a person sees it

# 80% is a calendar item; 100% is an outage
df -h | awk 'NR>1 && $5+0 >= 80 {print $0}'

NR>1 skips df's header row. $5+0 is the part worth stealing: the Use% column is a string like 82%, and awk's numeric coercion reads the leading digits and drops the rest, so adding zero turns it into 82 with no cut or sed. It prints only the filesystems over the line, which means its output is the alert.

That is also where mine broke, so check the last hop before you trust any of it. | mail -s only works on a machine with a working mail agent; test it once by hand. If there is no mail agent, send the output to a webhook or at least to a log line with a timestamp. And read the schedule field twice: 0 5 * * 1 is once a week, * 5 * * 1 is sixty times.

That one-liner is the first of the 25 scripts in the guide this post is drawn from, and the shape repeats across all of them: a small check, run on cron, that turns a silent slow failure into a loud early one.

The other 24 aim the same idea at different failures

The guide groups them into the five ways servers actually go down: disk growth, backup gaps, service crashes, network problems and security drift.

The disk section continues from the warning. du -ah | sort -rh | head -20 names what grew, and a find-based pruner clears the predictable offenders. One flag in it deserves a label: -mtime +30 does not mean "30 days or older" — the + means strictly more than 30 twenty-four-hour periods, and misreading it changes what gets deleted.

The backup section checks the thing a timestamp cannot. A dump that has been failing quietly since a credential rotation is not a backup, it is confidence with a date on it. The guide's mysqldump script checks that the output file is non-empty before it finishes, so a broken backup fails loudly on night one instead of on restore day.

The service section caps undetected downtime: systemctl is-active --quiet nginx || systemctl restart nginx on a five-minute cron means a crash costs five minutes. On one condition — log every restart. A watchdog that does not log will re-heal a memory leak every few hours and hide it by suppressing its only symptom.

The SSL check reads the certificate the server is actually presenting over the wire, not the file on disk, which catches the renewed-but-never-reloaded case. And ss -tlnp compared against a known baseline turns "unexpected listener on a public port" from a post-incident discovery into a line item.

The closing section wires everything into /etc/cron.d with staggered minutes and one shared log, and that log turns out to be the real product: after a week you know your normal. A deviation from a known baseline takes seconds to spot. My own disk check had a threshold and a schedule and no way to reach anyone, which is the gap the cron section exists to close.

All 25 scripts, grouped by failure category, each with its core command and a link to the hardened version with thresholds, alerting and cron lines: https://bashsnippets.xyz/guides/bash-scripts-every-sysadmin-needs

Once they are scheduled, the jobs themselves have to survive running unattended. Bash Scripts That Survive Cron covers the locks, timeouts and retries, and the Cron Wrapper Generator wraps those guards around any command you paste in. The rest of the library is at https://bashsnippets.xyz

sed 's/port/listen_port/g' Changed Four Lines. I Wanted One.

Anguishe — Thu, 01 Oct 2026 13:00:00 +0000

Renaming a config key is the job sed exists for, so I gave it the smallest honest test I could build on my own machine: a four-line YAML file and the rename most people type first. The file held port: 8080, support_email: ops@example.com, transport: tcp and report_dir: /var/reports. The command was sed -i 's/port/listen_port/g' app.yml.

GNU sed 4.9 returned immediately, exit 0, no output. -i edits in place and prints nothing, so there was nothing to read until I opened the file:

listen_port: 8080
suplisten_port_email: ops@example.com
translisten_port: tcp
relisten_port_dir: /var/relisten_ports

One line was meant to change. All four did. The email key is now a word no program will look up, the transport key is gone, and report_dir points at /var/relisten_ports, a directory that does not exist. Nothing failed. sed did what the command said, and the command said something different from what I meant. Across a service tree of forty config files that is not four bad lines; it is every file that happens to contain the letters, and you find out from whatever reads them next.

sed matches characters, not words

s/port/listen_port/g means: wherever the four characters p-o-r-t appear, replace them. Inside support. Inside transport. Inside report, and twice on the report_dir line because reports carries them again. sed has no idea what a word is unless the pattern tells it.

Three defaults stack into the failure, and none of them is a sed bug. An unanchored pattern that matches more than you pictured. -i, so there is no output to review. And a glob like *.yml or a find | xargs that reaches more files than you have opened. Each one is a default you have to refuse on purpose.

Fix one: tell sed where the word ends

GNU sed understands \b as a word boundary, a zero-width match between a word character and a non-word character. On the same original file, sed -i 's/\bport\b/listen_port/g' app.yml changed exactly one line, listen_port: 8080, and left the other three alone. In support, the p of port sits right after another word character, so \b refuses to match there.

Two details matter before you trust it. Underscore counts as a word character, so \bport\b will not touch the port inside db_port either — usually what you want, occasionally not. And \b is a GNU extension: BSD sed on macOS documents [[:<:]] and [[:>:]] for word start and end instead. A script that has to run on both platforms should not assume either spelling.

Fix two: read the diff before anything is written

The habit that matters more than any regex is refusing to let the first run be the in-place one. Drop -i and pipe sed's output into diff against the original: sed 's/port/listen_port/g' app.yml | diff app.yml -. On my test file that printed all four lines as changed, suplisten_port_email included, while the file on disk stayed intact. The whole disaster, on screen, costing nothing.

diff exits 1 when it finds differences, so the check works in a script as well as at a prompt. For more than one file, the shape that holds up is three steps. Build the file list with grep -rl first, so sed only ever touches files that contain the pattern and everything else keeps its modification time — build caches and rsync care about that. Show a diff per file. Then require an explicit --apply before a single byte is written. The default fails toward reading, not writing. If you forget the flag, the worst outcome is a diff you did not want to see.

Three more places the same command bites

The replacement side has its own special character. An unescaped & in the replacement means "the entire match", so s/CEO/Smith & Sons/ turns CEO into Smith CEO Sons. Write \& when you mean an ampersand.

The delimiter is yours to choose. s|/var/www/html|/srv/www|g is the same command as the version with every slash escaped, and it is the one you can still read under pressure. Pick a character that appears in neither side.

When the pattern comes from a shell variable, it is still a regex. OLD=staging.internal matches stagingXinternal too, because the dot means any character, and a | or & inside the value breaks the command outright. For values you control, a delimiter the value cannot contain is enough. For arbitrary input — filenames, anything a user typed — escape the variable first or use a tool that does literal replacement.

And the macOS trap that eventually costs every mixed team an afternoon: GNU -i takes an optional suffix, BSD -i requires one. On a Mac, sed -i 's/old/new/g' file treats the script as the backup suffix and then tries to run the filename as the program. The BSD spelling is sed -i '' 's/old/new/g' file.

Where sed stops

sed works one line at a time, so a plain s/// never matches across a newline. For multi-line surgery, awk or perl -0pe are the honest tools. And if the file is JSON or YAML with real structure, stop doing regex on it; a field-aware tool such as jq parses instead of pattern-matching, which is the entire reason it exists.

The rename I wanted needed two characters of anchoring and one pipe to diff. The version without them exited 0 and rewrote four lines out of four.

The full bulk-replace script — grep -rl file list, per-file diff, --apply gate, GNU/BSD -i detection — with the line-by-line breakdown and the FAQ on delimiters and multi-line matches: https://bashsnippets.xyz/snippets/bash-sed-find-replace

For the discovery half of the job, searching files with grep covers finding the text before you replace it, and the Grep Pattern Builder builds anchored patterns interactively. The rest of the library is at https://bashsnippets.xyz