Sed Search and Replace: Master Advanced Techniques

Need to find and replace text in files without opening an editor? Sed’s s command handles sed search and replace in one line. Good for config updates, data cleaning, or batch edits across essential Linux commands workflows.

What is sed?

sed (Stream Editor) processes text line by line without loading the whole file into memory. Good for:

• Automation and scripts
• Large files
• Piping with other Unix tools
• Batch processing

This guide focuses on GNU sed (the default on Linux) but covers macOS/BSD and Alpine/busybox differences in a dedicated portability section.

Other sed guides:

• Delete lines with sed
• Insert or append text with sed
• Transform text case with sed

Basic syntax

The substitute (s) command

sed 's/old/new/' file.txt        # Replace first occurrence per line
sed 's/old/new/g' file.txt       # Replace all occurrences (global)
sed 's/old/new/I' file.txt       # Ignore case (use capital I, portable)

Delimiters

Use different delimiters when your pattern contains slashes:

sed 's|/old/path|/new/path|g' config.txt
sed 's#http://old#https://new#g' urls.txt

Key options

Important: Test without -i first. Sed outputs to stdout by default.

Common examples

Config files:

sed 's/localhost/production-server/g' config.ini
sed 's/port=8080/port=80/g' server.conf

Code refactoring:

sed 's/oldFunction/newFunction/g' script.js
sed 's/var /let /g' legacy.js

Data cleanup:

sed 's/,/;/g' data.csv              # Change delimiter
sed 's/  / /g' document.txt         # Fix double spaces
sed 's/\t/ /g' file.txt             # Tabs to spaces (GNU-only, see portability section)

File operations

sed 's/old/new/g' file.txt > newfile.txt   # Save to new file
sed -i 's/old/new/g' file.txt              # Edit in place
sed -i.bak 's/old/new/g' file.txt          # Edit with backup

Multiple replacements

sed -e 's/old1/new1/g' -e 's/old2/new2/g' file.txt
sed 's/old1/new1/g; s/old2/new2/g' file.txt

Safety tips

1. Test without -i first
2. Use -i.bak for backups
3. Verify with diff original.txt modified.txt
4. Never combine -n with -i (file truncation, see the warning above)

GNU vs BSD vs busybox: Portable sed

Your dev laptop probably runs macOS (BSD sed), your production VPS runs Linux (GNU sed), and your CI/CD might run Alpine containers (busybox sed). A sed script that works on one can silently fail on another. This section covers the differences that actually break things.

sed -i 's/foo/bar/g' file.txt           # GNU/busybox: works
sed -i.bak 's/foo/bar/g' file.txt       # GNU/busybox: works, creates backup

sed -i '' 's/foo/bar/g' file.txt        # macOS: no backup
sed -i '.bak' 's/foo/bar/g' file.txt    # macOS: with backup

sed -i.bak 's/foo/bar/' config.conf && rm config.conf.bak

Universal in-place wrapper for scripts

If you write sed scripts that run on multiple platforms (CI/CD, shared dotfiles, Docker commands on Alpine), use this pattern:

# Works on GNU, macOS, and Alpine busybox
sed -i.bak 's/foo/bar/' config.conf && rm config.conf.bak

For CI pipelines, wrap it in a function:

sed_inplace() {
    sed -i.bak "$@" && rm -f "${@: -1}.bak"
}

Advanced pattern matching with regular expressions

Regular expressions are where sed goes from simple find-and-replace to real text processing. The patterns below cover what you will actually use.

Basic regular expression elements

Wildcard character (.):

sed 's/t.st/test/g' filename        # Matches "test", "tast", "t3st", etc.
sed 's/c.t/cat/g' pets.txt          # Matches "cat", "cut", "cot", etc.

Character classes:

sed 's/[aeiou]/X/g' filename        # Replace any vowel with X
sed 's/[0-9]/N/g' filename          # Replace any digit with N
sed 's/[A-Z]/L/g' filename          # Replace uppercase letters with L

Predefined character classes:

sed 's/[[:digit:]]/N/g' filename    # Replace digits (same as [0-9])
sed 's/[[:alpha:]]/L/g' filename    # Replace letters
sed 's/[[:space:]]/X/g' filename    # Replace whitespace characters

Quantifiers

Zero or more (*):

sed 's/ab*c/X/g' filename          # Matches "ac", "abc", "abbc", "abbbc"
sed 's/[0-9]*/NUM/g' filename      # Matches empty string or any digits

One or more (\+ in BRE):

sed 's/[0-9]\+/NUM/g' filename     # Matches one or more digits (GNU BRE extension)
sed 's/a\+/A/g' filename           # Matches "a", "aa", "aaa", etc.

Exact occurrences (\{n\}):

sed 's/[0-9]\{3\}/XXX/g' filename  # Matches exactly 3 digits
sed 's/a\{2,4\}/A/g' filename      # Matches 2 to 4 'a' characters

Anchors and boundaries

Line anchors:

sed 's/^Error/WARNING/' filename    # Replace "Error" at line start
sed 's/end$/END/' filename          # Replace "end" at line end
sed 's/^$/EMPTY/' filename          # Replace empty lines

Word boundaries:

sed 's/\bcat\b/dog/g' filename      # Replace whole word "cat" only
sed 's/\btest\b/exam/g' filename    # Avoids matching "testing" or "retest"

Practical advanced examples

Phone number formatting:

# Transform (123) 456-7890 to 123-456-7890
sed 's/(\([0-9]\{3\}\)) \([0-9]\{3\}\)-\([0-9]\{4\}\)/\1-\2-\3/g' contacts.txt

Email extraction and masking:

# Replace email addresses with [EMAIL]
sed 's/[a-zA-Z0-9._%+-]\+@[a-zA-Z0-9.-]\+\.[a-zA-Z]\{2,\}/[EMAIL]/g' data.txt

Date format conversion:

# Convert MM/DD/YYYY to YYYY-MM-DD
sed 's/\([0-9]\{2\}\)\/\([0-9]\{2\}\)\/\([0-9]\{4\}\)/\3-\1-\2/g' dates.txt

URL protocol updates:

# Change HTTP to HTTPS
sed 's|http://\([^[:space:]]*\)|https://\1|g' urls.txt

Grouping and back-references

Capture groups with \(\) and back-references with \1, \2:

# Swap first and last names
sed 's/\([A-Za-z]*\) \([A-Za-z]*\)/\2, \1/g' names.txt

# Duplicate words detection and removal (GNU-only: uses \b and \+)
sed 's/\b\([a-zA-Z]\+\) \1\b/\1/g' text.txt

# Extract filename from path
sed 's/.*\/\([^\/]*\)$/\1/' paths.txt

Complex pattern examples

Log processing:

# Extract timestamp from log entries
sed 's/^\[\([0-9-: ]*\)\] .*/\1/' server.log

# Replace IP addresses with [IP]
sed 's/\([0-9]\{1,3\}\.\)\{3\}[0-9]\{1,3\}/[IP]/g' access.log

Code refactoring:

# Update function calls: oldFunc(param) -> newFunc(param)
sed 's/oldFunc(\([^)]*\))/newFunc(\1)/g' code.js

# Convert 'single-quoted' strings to "double-quoted" (simple case)
sed -E "s/'([^']*)'/\"\1\"/g" script.js

The single-to-double-quote example above handles simple cases. It does not handle escaped single quotes (\') inside strings. For that, reach for a proper parser or perl.

Extended regular expressions

Using -E flag for enhanced patterns:

# Alternation with |
sed -E 's/(cat|dog|bird)/animal/g' pets.txt

# Simplified quantifiers (no escaping needed)
sed -E 's/[0-9]{3}-[0-9]{2}-[0-9]{4}/XXX-XX-XXXX/g' ssn.txt

# Matching URL protocols
sed -E 's/(http|https):\/\/[^[:space:]]*/[URL]/g' text.txt

Note: sed ERE has no non-capturing groups ((?:...)). Every () group captures. Use -E for cleaner syntax, not for non-capturing behavior.

Testing and debugging regular expressions

Preview matches before replacement:

# Show what would be matched
grep 'pattern' filename

# Show line numbers with matches
grep -n 'pattern' filename

# Test with sed's print flag
sed -n 's/pattern/replacement/p' filename

Build patterns incrementally:

# Start simple
sed 's/[0-9]/X/' filename

# Add complexity gradually
sed 's/[0-9]\+/NUM/' filename

# Final complex pattern
sed 's/[0-9]\{3\}-[0-9]\{2\}-[0-9]\{4\}/XXX-XX-XXXX/' filename

Use --debug to trace execution (GNU sed 4.6+):

echo 1 | sed --debug 's/1/3/'

This prints the sed program, annotates each input line showing the PATTERN space, and traces every command. It is the single most useful tool for “why did my sed command do that?” The --debug flag is mature in GNU sed 4.10. A crash that occurred when a label preceded the option has been fixed.

External debugger: sedsed is a debugger and formatter for sed scripts. Useful for stepping through complex multi-command scripts.

Online testers caveat: regex101 and similar tools use PCRE, not POSIX BRE/ERE. Features like lookarounds, non-greedy quantifiers (*?), and \d will not transfer to sed. Always verify patterns with actual sed.

Advanced sed techniques and best practices

Targeted line processing

Limit operations to specific line ranges:

sed '1,10s/old/new/g' file.txt      # Replace only in lines 1-10
sed '5,$s/old/new/g' file.txt       # Replace from line 5 to end
sed '10s/old/new/g' file.txt        # Replace only on line 10

Target lines by pattern:

sed '/pattern/s/old/new/g' file.txt     # Replace only in lines containing "pattern"
sed '/^#/s/old/new/g' file.txt          # Replace only in comment lines
sed '/ERROR/s/old/new/g' log.txt        # Replace only in error lines

Preview and testing techniques

Preview changes before applying:

sed -n 's/old/new/p' file.txt           # Show only changed lines
sed 's/old/new/g' file.txt | head -20   # Preview first 20 lines
sed 's/old/new/g' file.txt | diff file.txt -  # Show differences

Test with line numbers:

nl file.txt | sed 's/old/new/g'        # Show line numbers for context

Working with special characters

Escape literal characters:

sed 's/\./DOT/g' file.txt              # Escape literal dots
sed 's/\*/STAR/g' file.txt             # Escape literal asterisks
sed 's/\$/DOLLAR/g' file.txt           # Escape literal dollar signs
sed 's/\//SLASH/g' file.txt            # Escape literal forward slashes

Use alternative delimiters:

sed 's|/old/path|/new/path|g' file.txt     # Use | for paths
sed 's#http://old#https://new#g' file.txt  # Use # for URLs
sed 's@old@new@g' file.txt                 # Use @ as delimiter

Escape & in the replacement string:

& in the replacement means “the whole match.” A literal & in a URL or query-string rewrite will corrupt your output:

echo 'foo' | sed 's/foo/A & B/'    # → "A foo B" (probably not intended)
echo 'foo' | sed 's/foo/A \& B/'   # → "A & B" (literal ampersand)

This bites people when rewriting URLs with query parameters (&param=value). Always escape literal & in the replacement with \&.

Shell quoting and variable substitution

When sed patterns come from variables (common in config-update scripts), shell quoting gets tricky:

# Single quotes: no variable expansion (safe for literal patterns)
sed 's/old/new/g' file.txt

# Double quotes: variables expand, but /, &, \ in values collide with sed
KEY="old/path"
VAL="new/path"
sed "s|$KEY|$VAL|g" config.txt      # Use | delimiter when values contain /

# Always quote filenames in batch loops
for file in *.txt; do
    sed -i.bak 's/old/new/g' "$file"
done

# Use -- to stop option parsing for filenames starting with -
sed -i.bak 's/old/new/g' -- "-weird-name.txt"

For more on shell quoting differences, see shell quoting across Bash and Zsh.

Batch processing multiple files

Process all files of a type:

find . -name "*.txt" -exec sed -i 's/old/new/g' {} \;
find . -name "*.conf" -exec sed -i.bak 's/old/new/g' {} \;

Using xargs for efficiency:

find . -name "*.txt" | xargs sed -i 's/old/new/g'
find . -name "*.js" -print0 | xargs -0 sed -i 's/console.log/logger.debug/g'

Loop through files:

for file in *.txt; do
    sed -i.bak 's/old/new/g' "$file"
    echo "Processed: $file"
done

For more Linux text processing tips, see how to extract text on the Linux command line.

Advanced pattern techniques

Using back-references for complex replacements:

# Swap two words
sed 's/\(foo\) \(bar\)/\2 \1/g' file.txt

# Duplicate text
sed 's/\(important\)/\1 \1/g' file.txt

# Rearrange data fields
sed 's/\([^,]*\),\([^,]*\),\([^,]*\)/\3,\1,\2/' data.csv

Multiple operations in sequence:

sed -e 's/old1/new1/g' -e 's/old2/new2/g' -e 's/old3/new3/g' file.txt

Conditional replacements:

# Replace only if line contains specific pattern
sed '/contains_this/{s/old/new/g;}' file.txt

# Replace in specific sections
sed '/START/,/END/{s/old/new/g;}' file.txt

Performance optimization

Use the C locale for ASCII-heavy work:

LC_ALL=C sed -E 's/[0-9]+/NUM/g' bigfile.txt

The default locale does Unicode-aware character matching, which is slower. Setting LC_ALL=C forces ASCII byte matching and can be significantly faster on large files with regex-heavy substitutions. Benchmark on your own data.

Other performance tips:

# Skip the g flag when you only need the first match per line
sed 's/old/new/' file.txt

# One sed pass with semicolons beats multiple sed processes
sed 's/old1/new1/g; s/old2/new2/g; s/old3/new3/g' file.txt

# Use specific patterns to reduce processing
sed '/pattern/s/old/new/g' file.txt

Safety and backup strategies

Always backup important files:

cp original.txt original.txt.backup
sed -i.$(date +%Y%m%d) 's/old/new/g' original.txt

Test on sample data first:

head -100 largefile.txt > sample.txt
sed 's/old/new/g' sample.txt          # Test your pattern
# If good, apply to original:
sed -i.backup 's/old/new/g' largefile.txt

Atomic in-place editing (temp + mv):

sed -i is not truly atomic. It writes to a temp file and renames. For critical config files, the explicit temp + mv pattern gives you more control:

sed 's/old/new/' config.conf > config.conf.tmp && mv config.conf.tmp config.conf

Wrap it with a trap for cleanup on failure:

trap 'rm -f config.conf.tmp' EXIT
sed 's/old/new/' config.conf > config.conf.tmp && mv config.conf.tmp config.conf
trap - EXIT

Trade-off: mv changes the inode, which matters if another process has the file mmap-ed or if a file watcher is monitoring inode changes.

Use version control:

git add file.txt                      # Stage current version
sed -i 's/old/new/g' file.txt         # Make changes
git diff                              # Review changes

See Git commands for more on using version control as a safety net.

CI/CD and idempotency

Exit codes:

• 0: success
• 1: invalid command or regex
• 2: can’t open input file
• 4: I/O or serious runtime error

You can set custom exit codes with q N or Q N (GNU extension):

# Exit with code 2 if "ERROR" is found
sed '/ERROR/{q 2;}' logfile.txt

Sandbox mode (GNU sed):

sed --sandbox 's/old/new/g' untrusted.txt

--sandbox rejects the e (execute), w (write), and r (read) commands. Use it when running sed on untrusted input in pipelines.

Null-delimited processing:

find . -name "*.txt" -print0 | xargs -0 sed -i.bak 's/old/new/g'

The -z / --null-data flag (GNU sed) processes NUL-delimited input, which pairs with find -print0 and sort -z for safe handling of filenames with spaces.

Practical workflow examples

Configuration file updates:

# Update server configuration across multiple files
find /etc/nginx -name "*.conf" -exec sed -i.backup \
    -e 's/old-server.com/new-server.com/g' \
    -e 's/port 8080/port 80/g' {} \;

Code refactoring:

# Update function names in JavaScript files
find ./src -name "*.js" -exec sed -i \
    's/oldFunction/newFunction/g' {} \;

Log processing:

# Clean and standardize log files
sed -e 's/DEBUG/[DEBUG]/g' \
    -e 's/ERROR/[ERROR]/g' \
    -e 's/INFO/[INFO]/g' app.log > standardized.log

Common pitfalls to avoid

1. Forgetting to escape special characters. ., *, $, [, \ all have special meaning in regex.
2. Not testing patterns before applying to important files. Always preview with stdout first.
3. Using global replacement when you only want first occurrence. Omit the g flag.
4. Not backing up files before in-place editing. Use -i.bak or cp.
5. Making patterns too broad. Matching unintended text with .* or unescaped ..
6. Using GNU-only escapes in portable scripts. \b, \t, \+ silently fail on macOS/Alpine.
7. Combining -n with -i. Truncates the file to zero bytes.
8. Forgetting that & in replacement means “whole match”. Escape literal & as \&.

Pro tips

• Start with simple patterns and add complexity gradually
• Use grep to test your patterns before using in sed
• Use --debug (GNU sed 4.6+) to trace execution when patterns misbehave
• Use LC_ALL=C for faster processing of ASCII data
• Keep a collection of tested sed patterns for reuse
• Document complex regular expressions for future reference
• Know when to stop using sed. For branches, hold space, or multi-line slurps, reach for awk or Python.

Conclusion

Sed’s s command is the right tool for one-line, line-oriented text edits. It handles config updates, data cleanup, and batch replacements without loading the whole file into memory.

The key things to remember: test without -i first, use -i.bak for backups, and know which sed flavor you are running. GNU sed on Linux, BSD sed on macOS, and busybox sed on Alpine all behave differently for -i, \b, \t, and the I flag. Write the sed for the box it runs on.

For complex transformations involving branches, hold space, or multi-line processing, reach for awk or Python instead. Always back up your data before in-place edits, and practice incrementally on sample files.