Sed

Getting Started

Introduction to sed and basic concepts

What is Sed

Understanding sed and its capabilities as a stream editor

Sed overview and basic concepts

Sed reads input line by line, applies the substitution command to match patterns, and outputs the result.

Code

# Sed stands for: Stream Editor
# Processes text line by line, applying commands to matching lines

# Basic syntax: sed [options] 'command' file
# Most common: sed 's/pattern/replacement/' file

# Key features:
# - Non-interactive text transformations
# - Pattern matching with regular expressions
# - Substitution, deletion, insertion, transformation
# - Process files in-place or to stdout
# - Powerful scripting capabilities
# - Efficient stream processing

Execution

echo -e "apple\nbanana\ncherry" | sed 's/apple/orange/'

Output

orange
banana
cherry

Sed is line-oriented; processes one line at a time
Pattern/replacement is most common use case
Non-destructive by default (modifies stdout, not file)
Use -i flag for in-place file modification

Sed vs other text processing tools

Sed excels at simple but powerful text transformations using substitution and command chaining.

Code

# Compare sed with similar tools:
# sed: Stream transformations (substitution, deletion, etc.)
# awk: Full text processing language with patterns and fields
# grep: Search for patterns in lines (filtering)
# perl: Programming language for text manipulation

# Use sed when you need:
# - Simple text transformations
# - Substitutions and replacements
# - Deletion of lines or parts
# - Regular find-and-replace operations
# - Lightweight processing without awk/perl

Execution

echo -e "hello world\ntest string" | sed 's/world/universe/; s/test/sample/'

Output

hello universe
sample string

Sed is lighter than awk/perl for simple operations
Faster processing for large files
More portable across Unix systems
CLI-friendly for bash scripting

Installation and Setup

Installing sed and verifying functionality

Verify sed installation

Sed is typically pre-installed on Linux. Verify version with --version flag.

Code

# Check if sed is installed
which sed

# Display sed version
sed --version

# Show sed help
sed --help | head -20

Execution

sed --version | head -3

Output

GNU sed, version 4.9
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.

Sed is standard on all Unix-like systems
GNU sed and BSD sed have minor differences
GNU sed has more features and extensions
Most scripts work on both versions

Install sed on different systems

Sed is universally available on Linux. macOS users may prefer GNU sed for compatibility.

Code

# Ubuntu/Debian
sudo apt-get update
sudo apt-get install -y sed

# CentOS/RHEL
sudo yum install -y sed

# macOS (comes with BSD sed)
brew install gnu-sed  # Installs GNU sed as gsed

# Alpine Linux
apk add sed

# Arch Linux
sudo pacman -S sed

Execution

which sed && sed --version | head -1

Output

/usr/bin/sed
GNU sed, version 4.9

GNU sed has more robust features
BSD sed works for basic operations
macOS comes with BSD sed by default
GNU sed available as gsed on macOS

Basic Substitution

Simple text replacements and substitution patterns

Simple Replacements

Basic substitution with literal text

Simple text substitution

The s command (substitution) replaces the first occurrence of pattern with replacement on each line.

Code

# Replace first occurrence on each line

# Simple example
sed 's/old/new/' file.txt

# Replace on specific line
sed '3s/foo/bar/' file.txt

# Multiple substitutions
sed 's/a/b/; s/c/d/' file.txt

Execution

echo -e "hello world\nhello there" | sed 's/hello/hi/'

Output

hi world
hi there

Only replaces first occurrence per line by default
Pattern can be literal text or regex
Original file is not modified unless using -i
Delimiters (/) can be any character

Using different delimiters

Using alternative delimiters prevents escaping forward slashes in patterns.

Code

# Use different delimiters for readability
# Useful when pattern contains /

# Colon delimiter
sed 's:pattern:replacement:' file.txt

# Pipe delimiter
sed 's|pattern|replacement|' file.txt

# Hash delimiter
sed 's#pattern#replacement#' file.txt

# Useful with paths
sed 's|/old/path|/new/path|' file.txt

Execution

echo "/usr/local/bin" | sed 's|/local||'

Output

/usr/bin

Any character can be delimiter (avoid regex metacharacters)
Improves readability with paths
Reduces escaping requirements

Global Replacements

Replace all occurrences using g flag

Global replacement flag

The g flag replaces all occurrences of pattern on each line, not just the first.

Code

# g flag replaces all occurrences on each line
# Replace all
sed 's/foo/bar/g' file.txt

# Global with different delimiter
sed 's|pattern|replacement|g' file.txt

# Multiple substitutions, all global
sed 's/a/b/g; s/c/d/g' file.txt

Execution

echo "cat dog cat bird cat" | sed 's/cat/mouse/g'

Output

mouse dog mouse bird mouse

Without g flag, only first occurrence is replaced
g flag is most common use case
Works with case-insensitive flag (i)
Can be combined with other flags

Partial global replacement

Numeric flags target specific occurrences; 2g means 2nd and following.

Code

# Replace only 2nd and subsequent occurrences
sed 's/the/THE/2g' file.txt

# Replace starting at 3rd occurrence
sed 's/a/A/3g' file.txt

# Replace 2nd occurrence only
sed 's/foo/bar/2' file.txt

Execution

echo "the cat in the hat" | sed 's/the/THE/2g'

Output

the cat in THE hat

Number specifies starting occurrence
2g means 2nd, 3rd, 4th, etc.
Without g, only that specific occurrence replaced

Flags and Options

Substitution flags and command-line options

Substitution flags

Multiple flags can be combined; i makes pattern case-insensitive, g replaces all.

Code

# Common substitution flags:
# g = global (all occurrences)
# i = case-insensitive
# p = print matched line
# e = execute replacement as shell command
# number = replace only nth occurrence

# Case-insensitive replacement
sed 's/hello/hi/i' file.txt

# Print matching lines with -n
sed -n 's/pattern/replacement/p' file.txt

# Combine flags
sed 's/HELLO/hi/gi' file.txt  # global + case-insensitive

Execution

echo -e "Hello\nHELLO\nhello" | sed 's/hello/hi/gi'

Output

hi
hi
hi

Flags can be combined (gip, etc.)
Flag order doesn't matter for most flags
g flag most useful with others
p flag useful with -n option

Command-line options

The -n option suppresses default line printing; p flag prints only lines with substitutions.

Code

# sed command-line options:
# -n = suppress automatic printing (quiet mode)
# -e = specify script
# -f = read script from file
# -i = in-place file editing
# -E/-r = use extended regular expressions

# Quiet mode: only print matched lines
sed -n 's/pattern/replacement/p' file.txt

# Extended regex
sed -E 's/(old)(text)/\2-\1/' file.txt

# Multiple scripts
sed -e 's/a/b/' -e 's/c/d/' file.txt

Execution

echo -e "match1\nnomatch\nmatch2" | sed -n 's/match/MATCH/p'

Output

MATCH1
MATCH2

-n is combination of quiet mode and p flag
Very useful for filtering by pattern
Different from grep; modifies matched portion

Addressing and Ranges

Target specific lines or ranges of lines

Line-based Addressing

Target operations to specific line numbers

Address specific lines

Line numbers target specific lines; 2 means line 2 only.

Code

# Operate on specific line number
# Syntax: sed 'LineNums command' file

# Line 1 only
sed '1s/foo/bar/' file.txt

# Line 3 only
sed '3s/old/new/' file.txt

# Last line ($)
sed '$s/end/END/' file.txt

# Lines 1-5
sed '1,5s/a/b/' file.txt

Execution

echo -e "line1\nline2\nline3\nline4" | sed '2s/line/LINE/'

Output

line1
LINE2
line3
line4

Line numbers start at 1
$ is last line
Multiple lines use comma: 1,5
Range is inclusive

Line ranges and intervals

Range 2,5 targets lines 2 through 5 inclusive.

Code

# Range syntax: startLine,endLine command

# Lines 5 to 10
sed '5,10s/old/new/' file.txt

# Lines 1 to last line
sed '1,$s/a/b/' file.txt

# Every 5th line starting at line 1
sed '1~5d' file.txt  # GNU sed only

# From specific line to end
sed '100,$s/text/TEXT/' file.txt

Execution

seq 1 6 | sed '2,5s/.*/[LINE]/'

Output

1
[LINE]
[LINE]
[LINE]
[LINE]
6

Ranges are inclusive on both ends
1~5 means every 5th line (GNU sed)
Intervals: line~step (GNU sed extension)

Regex-based Addressing

Target lines matching regular expressions

Target lines by pattern

/mid/ matches lines containing "mid" string; operation applies only to matched lines.

Code

# Address lines matching pattern
# Syntax: sed '/pattern/command' file

# Lines containing "error"
sed '/error/s/old/new/' file.txt

# Lines starting with #
sed '/^#/d' file.txt

# Lines ending with semicolon
sed '/;$/s/;//' file.txt

# Case-insensitive pattern
sed '/ERROR/Iy s/msg/MESSAGE/' file.txt

Execution

echo -e "start\nmiddle\nend" | sed '/mid/s/./X/'

Output

start
Xiddle
end

Pattern is regex; can use anchors
Matches any line containing pattern substring
Case-sensitive by default
I flag makes pattern case-insensitive

Range by regex patterns

Range /START/,/END/ targets all lines from pattern to pattern inclusive.

Code

# Range from start pattern to end pattern
# Syntax: /startPattern/,/endPattern/command

# Delete from "start" to "end"
sed '/start/,/end/d' file.txt

# Substitute in pattern ranges
sed '/BEGIN/,/END/s/old/new/' file.txt

# Print only section between patterns
sed -n '/start/,/stop/p' file.txt

# Multiple ranges
sed '/foo/,/bar/d; /baz/,/qux/d' file.txt

Execution

echo -e "a\nSTART\nb\nc\nEND\nd" | sed '/START/,/END/s/./X/'

Output

a
X
X
X
X
d

Ranges inclusive on both ends
Patterns are regex patterns
Single occurrence per file for regex ranges
Useful for multi-line operations

Range Specifications

Combining addresses for flexible targeting

Step addressing

1~3 means line 1, then every 3rd line after (1, 4, 7, 10...).

Code

# Every nth line (GNU sed)
# Syntax: first~step

# Every 2nd line starting from line 1
sed '1~2d' file.txt

# Every 3rd line starting from line 2
sed '2~3s/x/y/' file.txt

# Every 5th line
sed '0~5d' file.txt  # 5, 10, 15, 20...

# Process in chunks
sed '1~10s/^/> /' file.txt  # prefix every 10th

Execution

seq 1 10 | sed '1~3s/^/LINE:/'

Output

LINE:1
2
3
LINE:4
5
6
LINE:7
8
9
LINE:10

Syntax: first~step (GNU sed extension)
Step interval starting from first line
Very useful for pattern-based processing
Not available in BSD sed

Negation and exclusion

The ! operator inverts the address; operation applies to non-matching lines.

Code

# Negate address with !
# Apply command to all lines EXCEPT matched

# Delete all lines except those matching
sed '/pattern/!d' file.txt

# Modify all lines except specified range
sed '10,20!d' file.txt

# Apply to all non-matching lines
sed '/^#/!s/old/new/' file.txt

# Keep only non-empty lines
sed '/^$/!s/a/b/' file.txt

Execution

echo -e "keep\ndelete\nkeep" | sed '/keep/!d'

Output

keep
keep

! means NOT or inverse
Keep lines matching pattern by deleting non-matches
Can use with line numbers or regex

Deletion and Insertion

Delete, append, insert, and change lines

Delete Commands

Remove lines from output

Delete lines with d command

The d command deletes (removes) the specified line from output.

Code

# d command deletes matching lines
# Syntax: [address]d

# Delete specific line
sed '5d' file.txt

# Delete pattern-matching lines
sed '/error/d' file.txt

# Delete range of lines
sed '10,20d' file.txt

# Delete empty lines
sed '/^$/d' file.txt

# Delete lines starting with #
sed '/^#/d' file.txt

Execution

echo -e "line1\nline2\nline3\nline4" | sed '2d'

Output

line1
line3
line4

d stops processing line; next line is read
Original file not modified (unless -i used)
Pattern matching deletes all matching lines
Useful for filtering

Delete with pattern ranges

Range 2,4d deletes lines 2 through 4 inclusive.

Code

# Delete line ranges
sed '/start/,/end/d' file.txt

# Delete first 10 lines
sed '1,10d' file.txt

# Delete last line
sed '$d' file.txt

# Delete all lines except first
sed '1!d' file.txt

# Keep only lines matching pattern
sed '/keep/!d' file.txt

Execution

seq 1 5 | sed '2,4d'

Output

1
5

Range includes both start and end lines
d with ! inverts; deletes non-matching lines
$ = last line

Append and Insert

Add lines before or after matched lines

Append and insert commands

The a command appends (adds) a new line after the matched line.

Code

# a = append (add after line)
# i = insert (add before line)
# Syntax: [address]a\text or [address]i\text

# Append text after line 3
sed '3a\NEW LINE' file.txt

# Insert text before line 5
sed '5i\INSERTED' file.txt

# Append after pattern match
sed '/pattern/a\APPENDED TEXT' file.txt

# Insert before first line
sed '1i\HEADER' file.txt

# Append to last line
sed '$a\FOOTER' file.txt

Execution

echo -e "line1\nline2" | sed '2a\APPENDED'

Output

line1
line2
APPENDED

i inserts before; a appends after
Text must follow backslash on same line
Multiline text requires backslash continuation
No substitution in appended text

Multiline append and insert

Chaining insert (i) and append (a) commands adds multiple lines.

Code

# Multiple lines with continuation
sed '3a\line1\
line2\
line3' file.txt

# Using printf for readability
sed '2a\'"$(printf 'line1\nline2')"'' file.txt

# Append blank line
sed '/pattern/a\' file.txt

# Insert multiple blocks
sed '/start/i\BEGIN\nCONTENT' file.txt

Execution

echo "middle" | sed 'i\START' | sed 'a\END'

Output

START
middle
END

Each line continuation needs backslash
Piping sed commands adds processing overhead
Better to use multiple -e flags

Change Command

Replace entire lines with new content

Change lines with c command

The c command replaces the entire matched line with the specified text.

Code

# c = change (replace entire line)
# Change line 3
sed '3c\NEW LINE' file.txt

# Change pattern-matching lines
sed '/pattern/c\REPLACEMENT' file.txt

# Change range of lines
sed '5,10c\CHANGED' file.txt

# Change first occurrence of pattern
sed '/error/c\ERROR HANDLED' file.txt

Execution

echo -e "line1\nline2\nline3" | sed '2c\CHANGED'

Output

line1
CHANGED
line3

Entire line is replaced, not just pattern
Range with c replaces all with single line
Pattern matching replaces each matching line
Original line content is lost

Change with ranges

Range 2,4c replaces all lines in range with single line.

Code

# Range with change: all lines become one
sed '5,10c\SECTION REPLACED' file.txt

# Pattern range change
sed '/begin/,/end/c\BLOCK' file.txt

# Selective change with condition
sed '/^old/c\new' file.txt

# Preserve format with change
sed '/find/c\  replaced' file.txt  # indentation

Execution

seq 1 5 | sed '2,4c\[REMOVED]'

Output

1
[REMOVED]
5

Range replaced with one line (unlike delete)
Useful for removing blocks
Preserves position in sequence

Text Transformation

Advanced text transformation operations

Transliteration (y command)

Character-by-character translation

Transliteration with y command

The y command maps each character in source-chars to matching position in dest-chars.

Code

# y command: transliteration (character mapping)
# Convert lowercase to uppercase
sed 'y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/' file.txt

# Simple character mapping
sed 'y/abc/xyz/' file.txt  # a->x, b->y, c->z

# ROT13 encoding (example)
sed 'y/abcdefghijklmnopqrstuvwxyz/nopqrstuvwxyzabcdefghijklm/' file.txt

# Map symbols
sed 'y/_-/ /' file.txt  # underscore and dash to space

Execution

echo "hello WORLD" | sed 'y/hello/HELLO/'

Output

HELLO woRLD

Source and destination must be same length
Maps character by position, not by pattern
Global by nature (all occurrences)
Cannot use regex or special patterns

Practical transliteration uses

y command translates each character in source to corresponding destination.

Code

# Normalize whitespace characters
sed 'y/\t/ /' file.txt  # tabs to spaces

# Date format conversion
sed 'y/-/ /' file.txt  # dashes to spaces

# Simple encoding
sed 'y/!@#$%/12345/' file.txt

# Phone number normalization
echo "123-456-7890" | sed 'y/()-/ /'

# Keyboard mapping simulation
echo "hello" | sed 'y/helo/hdlo/'  # swap e and l

Execution

echo "a-b-c" | sed 'y/-/ /'

Output

a b c

All matching characters are replaced
Order matters; source[n] -> dest[n]
Source and dest lengths must match exactly

Hold Space Operations

Use hold space for advanced transformations

Pattern and hold space basics

Complex hold space operations can reverse line order and enable advanced transformations.

Code

# Sed maintains two buffers:
# Pattern space = current line being processed
# Hold space = auxiliary buffer for storage

# h = copy pattern space to hold space
# g = copy hold space to pattern space
# x = exchange pattern and hold spaces
# H = append pattern space to hold space
# G = append hold space to pattern space

# Swap adjacent lines
sed 'N;h;s/\n/ /;x;s/ /\n/;U' file.txt

# Reverse entire file
sed '1h;1!H;$!d;x' file.txt

Execution

echo -e "A\nB\nC" | sed '1h;1!H;$!d;x'

Output

C
B
A

Hold space starts empty
Advanced feature; complex to master
Enables multi-line transformations
Powerful for sophisticated operations

Practical hold space examples

N appends next line to pattern space; useful for multi-line operations.

Code

# Print lines in reverse order
sed -n '1h;1!H;$!d;x;p' file.txt

# Print every other line
sed -n 'N;P' file.txt

# Duplicate each line
sed 'h;p' file.txt

# Process pairs of lines
sed 'N;s/\n/ /' file.txt  # merge pairs

Execution

echo -e "1\n2\n3\n4" | sed 'N;s/\n/-/'

Output

1-2
3-4

N = read next line into pattern space
Creates multi-line patterns
Enables cross-line substitutions

Print Commands

Various line printing and display options

Print commands

The l command shows whitespace and line endings explicitly.

Code

# p = print current pattern space
# l = list line (shows non-printing chars)
# = = print line number
# Combine with -n for selective printing

# Print specific lines
sed -n '5p' file.txt

# Print pattern-matching lines
sed -n '/error/p' file.txt

# Print with line numbers
sed -n '=/error/p' file.txt  # prints line number then line

# Print non-printing characters visible
sed -n 'l' file.txt  # shows tabs, spaces, newlines

Execution

echo -e "hello\tworld\ntest" | sed -n 'l'

Output

hello\tworld$
test$

p prints line (use with -n to avoid duplication)
l shows visible representation (tabs, spaces, newlines)
= prints current line number
Useful for debugging and visualization

Selective output with options

-n with p prints only matching lines; useful instead of grep for modifications.

Code

# Duplicate matching lines
sed '/pattern/p' file.txt

# Print line numbers for matches
sed -n '/error/=' file.txt

# Print line number and content
sed -n '/pattern/={=;p;}' file.txt

# Show matching context
sed -n '/start/,/end/p' file.txt

# Count and show matches
sed -n '/pattern/p' file.txt | wc -l

Execution

echo -e "match\nnomatch\nmatch" | sed -n '/match/p'

Output

match
match

p without -n duplicates lines
-n alone suppresses all output
p flag in s command (s///p) prints substitutions

Advanced Operations

Branches, labels, and complex operations

Branches and Labels

Control flow with branches and labels

Branch and label commands

The t command branches to label if substitution succeeded; loop removes double 'a's.

Code

# b [label] = branch to label or end
# t [label] = branch if substitution made
# T [label] = branch if no substitution (GNU sed)
# : label = define label name

# Basic branch
sed ':start; s/aa/a/; t start' file.txt

# Repeat substitution until no match
sed ':a; s/  / /g; t a' file.txt

# Conditional branch
sed '/pattern/{=; b end}; d; :end' file.txt

# Skip section
sed '/skip/b skip; s/a/b/g; :skip' file.txt

Execution

echo "aaa" | sed ':a; s/aa/a/; ta'

Output

b branches unconditionally
t branches only if substitution made
Labels defined with: notation
Creates loops for repeated operations

Practical branch patterns

t skips further processing if substitution succeeds, implementing conditional logic.

Code

# Multiple substitutions in sequence
sed 's/foo/bar/; s/old/new/; s/a/b/' file.txt

# Skip certain lines
sed '/^#/{N; b}; s/^/  /' file.txt

# Process based on condition
sed '/match/{s/a/b/; b skip}; s/x/y/; :skip' file.txt

# Complex multi-pattern logic
sed '/start/,:end {s/x/y/; /end/b; n; b;} d' file.txt

Execution

echo -e "foo\nbar\nbaz" | sed 's/foo/FOO/; t skip; s/./X/; :skip'

Output

FOO
XXX
XXX

Enables complex multi-step logic
Useful for avoiding redundant operations
More efficient than multiple passes

Multiple Scripts

Combining multiple sed commands and scripts

Multiple -e scripts

Multiple -e flags apply each script sequentially to each line.

Code

# Multiple -e flags for separate commands
sed -e 's/a/b/' -e 's/c/d/' file.txt

# Equivalent to semicolon separation
sed 's/a/b/; s/c/d/' file.txt

# Multiple -e with complex operations
sed -e '/pattern/d' -e 's/x/y/' file.txt

# Each -e is processed in order
sed -e '1s/^/HEAD: /' -e '$a\TAIL'  file.txt

Execution

echo -e "a\nb\nc" | sed -e 's/a/A/' -e 's/b/B/'

Output

A
B
c

-e and ; are equivalent for multiple commands
Order matters; first command output feeds to next
Each line processed through all commands

Script files with -f

Script files organize complex sed operations for reusability.

Code

# Save script to file, use with -f
# File: myscript.sed
# Contents:
# /pattern/d
# /^#/d

# Execute script file
sed -f myscript.sed file.txt

# Multiple script files
sed -f script1.sed -f script2.sed file.txt

# Create and run script
cat > /tmp/script.sed << 'EOF'
s/foo/bar/g
s/hello/hi/
/^$/d
EOF
sed -f /tmp/script.sed file.txt

Execution

cat > /tmp/test.sed << 'EOF'
s/x/X/g
/^$/d
EOF
echo -e "x\n\nY" | sed -f /tmp/test.sed

Output

X
Y

-f allows script in file
Easier to manage complex scripts
Can version control scripts
Comments with # in script files

Regular Expressions

Advanced regex patterns and options

Basic vs extended regex

Extended regex (-E) provides modern regex syntax with fewer escapes.

Code

# Basic regex: sed 's/pattern/replacement/'
# Extended regex: sed -E 's/pattern/replacement/'

# Basic: escape special chars
sed 's/\(.*\)/[\1]/' file.txt

# Extended: no escaping needed
sed -E 's/(.*)/[\1]/' file.txt

# Extended groups
sed -E 's/([0-9]+)-([0-9]+)/\2-\1/' file.txt

# Use -E for clarity and consistency
sed -E 's/(foo|bar)/FOUND/' file.txt

Execution

echo "123-456" | sed -E 's/([0-9]+)-([0-9]+)/\2-\1/'

Output

456-123

-E (GNU) or -r (BSD) enables extended regex
Less escaping in extended mode
More readable patterns
Recommended for consistency

Advanced pattern matching

Regex groups and backreferences enable sophisticated text rearrangement.

Code

# Capture groups and backreferences
sed -E 's/([a-z]+) ([a-z]+)/\2 \1/' file.txt

# Lookahead-like patterns
sed -E 's/([0-9]+)([a-z])/\1 \2/' file.txt

# Complex patterns
sed -E 's/^([^:]*):(.*)$/\2:\1/' file.txt

# Word boundaries (in extended mode)
sed -E 's/\bword\b/WORD/' file.txt

# Multiple alternatives
sed -E 's/(foo|bar|baz)/MATCH/' file.txt

Execution

echo "John Smith" | sed -E 's/([A-Za-z]+) ([A-Za-z]+)/\2, \1/'

Output

Smith, John

\1, \2... reference captured groups
Groups defined by ( ) in extended mode
Powerful for text reformatting

In-Place Editing

Modify files directly with sed

File Modification

Edit files in place with -i flag

In-place file editing

The -i flag modifies files in place; changes are written to original file.

Code

# -i flag: edit files in-place
# Changes written back to original file

# Simple in-place substitution
sed -i 's/old/new/' file.txt

# In-place with extended regex
sed -i -E 's/(pattern)/replacement/' file.txt

# In-place on multiple files
sed -i 's/find/replace/' *.txt

# Apply to specific line
sed -i '5s/text/TEXT/' file.txt

# Note: -i modifies original file directly
sed -i 's/a/b/' important.txt  # NO BACKUP!

Execution

echo -e "test\ntext" > /tmp/test.txt
sed -i 's/test/TEST/' /tmp/test.txt
cat /tmp/test.txt

Output

TEST
text

Dangerous without backup; file is modified immediately
No recovery possible if mistake made
Use backup option (-i.bak) when testing

Backup before modification

Backup extension creates copy of original before modification for safety.

Code

# Create backup with -i flag
# -i.bak creates backup with .bak extension

# Backup example
sed -i.bak 's/old/new/' file.txt
# Creates: file.txt (modified) and file.txt.bak (original)

# Custom backup extension
sed -i.orig 's/pattern/replacement/' file.txt
# Creates: file.txt (modified) and file.txt.orig (original)

# Backup with timestamp
sed -i.$(date +%s).bak 's/a/b/' file.txt

# Backup multiple files
sed -i.bak 's/find/replace/' *.conf

Execution

echo "original" > /tmp/backup_test.txt
sed -i.backup 's/original/modified/' /tmp/backup_test.txt
echo "Modified:" && cat /tmp/backup_test.txt
echo "Backup:" && cat /tmp/backup_test.txt.backup

Output

Modified:
modified
Backup:
original

Always use backup for important files
Backup contains original unmodified content
Can verify changes by comparing files

Platform Differences

GNU sed vs BSD sed variations

GNU vs BSD sed differences

Checking sed version helps determine feature availability.

Code

# GNU sed (Linux) vs BSD sed (macOS default)
# Key differences:

# 1. In-place editing syntax:
# GNU: sed -i 's/a/b/' file.txt (no space)
# BSD: sed -i '' 's/a/b/' file.txt or sed -i.bak 's/a/b/' file.txt

# 2. Extended regex flags:
# GNU: sed -r 's/(a)/[\1]/' file.txt
# BSD: sed -E 's/(a)/[\1]/' file.txt

# 3. GNU-only features:
# BSD doesn't support: 0~step, e flag, T command

# 4. Escape sequences in replacement:
# GNU: \L (lowercase), \U (uppercase)
# BSD: Not supported; use tr instead

Execution

sed --version 2>&1 | head -1 || echo "BSD sed (or compatible)"

Output

GNU sed, version 4.9

GNU sed is more feature-rich
BSD sed is simpler but less capable
macOS ships with BSD sed by default
Install GNU sed via Homebrew on macOS

Writing portable sed scripts

Using -E and avoiding GNU-specific features ensures portable scripts.

Code

# Portable sed patterns (work on both GNU/BSD):

# Use -e for clarity (both support it)
sed -e 's/old/new/' -e 's/foo/bar/' file.txt

# Use -E instead of -r (more portable)
sed -E 's/(pattern)/[\1]/' file.txt

# Avoid GNU-only features:
# Don't use: 0~step, T, L, U escape sequences, e flag

# For macOS/BSD, install GNU sed:
# brew install gnu-sed
# Use as: gsed command file.txt

# Portable in-place editing:
sed -i.bak 's/pattern/replacement/' file.txt
rm file.txt.bak  # cleanup

Execution

echo "TEST" | sed -E 's/([A-Z]+)/[\1]/g'

Output

[T][E][S][T]

-E works on both GNU and BSD
-r is GNU-only (not portable)
Think about target systems before scripting

Practical Examples

Real-world sed usage scenarios

Real-World Use Cases

Practical application examples

Log file processing

Sed efficiently filters and processes log files for analysis.

Code

# Extract error lines from logs
sed -n '/ERROR/p' app.log

# Remove timestamp prefix
sed 's/^\[[^]]*\] //' app.log

# Filter and format errors
sed -n '/ERROR/{s/ERROR/[!]/p;}' app.log

# Gzip compress, preserving original processing
sed 's/\.log$/.log.gz/' <<<log_list.txt | xargs gzip

# Remove ANSI color codes
sed 's/\x1b\[[0-9;]*m//g' colored.log

Execution

echo -e "[2025-01-01 10:00:00] Starting\n[2025-01-01 10:00:01] ERROR: Failed\n[2025-01-01 10:00:02] Done" | sed -n '/ERROR/p'

Output

[2025-01-01 10:00:01] ERROR: Failed

Fast for large log processing
Combine with pipes for complex workflows
Better than loading entire file in memory

Configuration file editing

Sed automates configuration file updates without manual editing.

Code

# Enable/disable settings
sed -i 's/^#\(enable_feature\)/\1/' config.conf

# Change parameter values
sed -i 's/^\(parameter\)=.*/\1=newvalue/' config.conf

# Add configuration if missing
sed -i '/^\[section\]/a\new_option=value' config.ini

# Update port in config
sed -i 's/port=[0-9]*/port=8080/' app.conf

# Comment out deprecated options
sed -i 's/^\(deprecated_option\)/#\1/' config.conf

Execution

echo "enable_feature=false" > /tmp/config.conf
sed -i 's/enable_feature=.*/enable_feature=true/' /tmp/config.conf
cat /tmp/config.conf

Output

enable_feature=true

Safer than manual editing
Scriptable for automation
Essential for infrastructure as code

Data format conversion

Sed converts between data formats efficiently.

Code

# CSV to space-separated
sed 's/,/ /g' data.csv

# URL decode (simple version)
sed 's/%20/ /g; s/%2F/\//g' encoded.txt

# Convert tabs to spaces
sed 's/\t/  /g' data.tsv

# Format SQL queries
sed 's/ where / WHERE /i; s/ and / AND /i' query.sql

# Convert JSON keys to uppercase
sed 's/"[^"]*":/\U&/g' data.json

Execution

echo "a,b,c" | sed 's/,/ /g'

Output

a b c

Quick format transformations
Better for simple conversions than scripting

Code refactoring

Sed automates code refactoring across multiple files.

Code

# Rename function/variable
sed -i 's/\boldName\b/newName/g' *.js

# Add copyright header to files
sed -i '1i/* Copyright 2025 */' *.js

# Convert var to const
sed -i 's/^var /const /g' script.js

# Update import paths
sed -i "s|from './old|from './new|g" *.ts

# Remove console.log statements (careful!)
sed -i '/console\.log/d' app.js

Execution

echo "var x = 5;" | sed 's/^var /const /'

Output

const x = 5;

Use -i.bak for safety
Test patterns first
Great for large codebase refactoring

Text formatting

Sed formats text output efficiently.

Code

# Add line numbers
sed = file.txt | sed 'N;s/\n/: /'

# Add indentation
sed 's/^/  /' file.txt

# Remove empty lines
sed '/^$/d' file.txt

# Wrap long lines
sed 's/\(.\{1,80\}\) /\1\n/g' file.txt

# Capitalize first letter
sed 's/^./\U&/' file.txt

Execution

echo -e "first\nsecond\nthird" | sed = | sed 'N;s/\n/. /'

Output

1. first
2. second
3. third

Useful for report generation
Combines well with other tools

File cleanup and normalization

Sed cleans whitespace and normalizes file encoding.

Code

# Remove trailing whitespace
sed 's/[[:space:]]*$//' file.txt

# Remove duplicate blank lines
sed '/^$/N;/^\n$/!P;D' file.txt

# Normalize line endings (CRLF to LF)
sed 's/\r$//' windows.txt

# Remove control characters
sed 's/\x00//g' file.txt

# Convert DOS to Unix
sed 's/\r$//' dos_file.txt > unix_file.txt

Execution

echo -e "line1  \nline2 \t" | sed 's/[[:space:]]*$//'

Output

line1
line2

Essential for data preprocessing
Improves data quality

Performance Tips

Optimization and performance best practices

Performance considerations

Sed processes efficiently even with large inputs.

Code

# Use sed for line-by-line processing
# Better than loading entire file in memory

# Avoid unnecessary commands
# This is slow: multiple passes
sed 's/a/b/' file | sed 's/c/d/' | sed 's/e/f/'

# This is faster: combine operations
sed 's/a/b/; s/c/d/; s/e/f/' file

# Limit address ranges
# Slow: search entire file
sed 's/pattern/replacement/' huge.txt

# Faster: limit to relevant lines
sed '100,200s/pattern/replacement/' file.txt

# Use -n with p for filtering
# Better than piping to grep
sed -n '/pattern/p' file | wc -l  # Slow
sed -n '/pattern/p' file | wc -l  # Still works

Execution

time seq 1 1000 | sed 's/1/2/'

Output

real  0m0.005s
user  0m0.003s
sys  0m0.001s

Stream processing avoids memory overhead
Single pass generally fastest
Combining operations reduces overhead

Optimization techniques

Efficient filtering removes unnecessary processing of lines.

Code

# Use specific addresses to reduce work
# Instead of: sed 's/a/b/g' (every line)
sed '1,100s/a/b/g' file.txt  # only first 100 lines

# Use simpler regex patterns
# Complex: sed 's/^[[:space:]]*([a-zA-Z_][a-zA-Z0-9_]*).*$/\1/'
# Simpler: sed 's/^[^:]*:'  # if suitable

# Avoid expensive regex operations
# Slow: sed 's/.*\(pattern\).*/\1/'
# Better: sed -n 's/.*\(pattern\).*/\1/p'

# Use -i for in-place without temp files
# Avoids: cp file file.bak; sed ... file.bak > file; rm file.bak

# Avoid recursive holds/branching
# Test termination conditions carefully

Execution

# Demonstrate efficient skipping
seq 1 5 | sed '/3/d'  # Skip specific lines

Output

Plan logic carefully to minimize work
Profile with time command for large files
Consider awk/perl for very complex operations

Large file handling

Sed processes large files efficiently through streaming.

Code

# Processing large files efficiently

# Split large file, process in parallel
split -n l/4 large.txt tmp_

# Process each chunk
for chunk in tmp_*; do
  sed 's/pattern/replacement/' "$chunk" > "$chunk.out"
done

# Merge results
cat tmp_*.out > output.txt

# Use sed with tail for end-of-file operations
tail -100 large.txt | sed 's/a/b/'

# Stream processing (best for large files)
sed 's/old/new/' input.txt > output.txt  # Better than loading all in memory

Execution

# Create sample data
seq 1 10 > /tmp/large.txt
# Process with sed
sed 's/1/ONE/g' /tmp/large.txt | head -5

Output

ONE
2
3
4
5

Stream processing avoids memory issues
Process in chunks for parallelization
Pipe directly without temp files when possible

Sed

Getting Started

What is Sed

Accessibility

Best Practices

Common Errors

Keywords

Sed overview and basic concepts

Sed vs other text processing tools

Installation and Setup

Accessibility

Best Practices

Common Errors

Keywords

Verify sed installation

Install sed on different systems

Basic Substitution

Simple Replacements

Accessibility

Best Practices

Common Errors

Keywords

Simple text substitution

Using different delimiters

Global Replacements

Accessibility

Best Practices

Common Errors

Keywords

Global replacement flag

Partial global replacement

Flags and Options

Accessibility

Best Practices

Common Errors

Keywords

Substitution flags

Command-line options

Addressing and Ranges

Line-based Addressing

Accessibility

Best Practices

Common Errors

Keywords

Address specific lines

Line ranges and intervals

Regex-based Addressing

Accessibility

Best Practices

Common Errors

Keywords

Target lines by pattern

Range by regex patterns

Range Specifications

Accessibility

Best Practices

Common Errors

Keywords

Step addressing

Negation and exclusion

Deletion and Insertion

Delete Commands

Accessibility

Best Practices

Common Errors

Keywords

Delete lines with d command

Delete with pattern ranges

Append and Insert

Accessibility

Best Practices

Common Errors

Keywords

Append and insert commands

Multiline append and insert

Change Command

Accessibility

Best Practices

Common Errors

Keywords