Curl

Getting Started

Learn what curl is and make your first HTTP requests

What is Curl

Understand curl and its capabilities

Install curl and check version

Installs curl and displays version information showing libcurl version and supported protocols.

Code

# On macOS with Homebrew
brew install curl

# On Debian/Ubuntu
sudo apt-get install curl

# On CentOS/RHEL
sudo yum install curl

# Check version
curl --version

Execution

curl --version

Output

curl 7.88.1 (x86_64-pc-linux-gnu) libcurl/7.88.1
Release-Date: 2023-02-20

curl is usually pre-installed on most Unix-like systems
If missing, install via package manager (apt, yum, brew, etc.)
curl supports HTTP, HTTPS, FTP, FTPS, SCP, SFTP, TFTP, TELNET, LDAP, LDAPS, and many more

Basic curl command structure

Demonstrates basic curl syntax and how to retrieve HTTP status codes from endpoints.

Code

# Basic syntax
curl [options] [URL]

# Simplest request - GET a URL
curl https://example.com

# Get just the response code
curl -o /dev/null -s -w "%{http_code}" https://example.com

# Show response headers
curl -i https://example.com

Execution

curl -s -w "%{http_code}\n" https://httpbin.org/get

Output

By default, curl sends a GET request to the URL
-s flag enables silent mode (no progress meter)
-w flag allows custom output formatting with HTTP response codes
Response is written to stdout unless redirected

Quick API testing examples

Quickly test if APIs are responding without downloading full response body.

Code

# Test if a server is up
curl -I https://api.example.com

# Get response with timing information
curl -w "Response time: %{time_total}s\n" https://example.com

# Test followed by status code only
curl -s -o /dev/null -w "HTTP Status: %{http_code}\n" https://api.example.com

Execution

curl -I -s https://httpbin.org/get

Output

HTTP/2 200
Date: Fri, 28 Feb 2025 12:00:00 GMT
Content-Type: application/json
Server: gunicorn/19.9.0

-I flag retrieves headers only (HEAD request)
Useful for health checks and availability testing
-w provides detailed timing information

HTTP Methods Fundamentals

Understanding and using different HTTP methods

Specify HTTP method with -X flag

Use the -X flag to explicitly specify the HTTP method for your request.

Code

# GET request (default)
curl https://api.example.com/users

# Explicitly specify GET
curl -X GET https://api.example.com/users

# Use different methods
curl -X POST https://api.example.com/users
curl -X PUT https://api.example.com/users/1
curl -X DELETE https://api.example.com/users/1
curl -X PATCH https://api.example.com/users/1

Execution

curl -X GET https://httpbin.org/get

Output

{
  "args": {},
  "headers": {
    "Host": "httpbin.org",
    "User-Agent": "curl/7.88.1"
  },
  "origin": "192.0.2.1",
  "url": "https://httpbin.org/get"
}

GET is the default method (retrieves data)
POST creates new resources
PUT replaces entire resource
DELETE removes resources
PATCH modifies part of resource

Method-specific behavior

Different HTTP methods handle data and requests differently, with appropriate use cases.

Code

# GET - no body sent
curl https://api.example.com/users?page=1

# POST - requires body
curl -X POST -d "name=John" https://api.example.com/users

# PUT - replace entire resource
curl -X PUT -d '{"name":"Jane","email":"jane@example.com"}' https://api.example.com/users/1

# DELETE - no body, just delete
curl -X DELETE https://api.example.com/users/1

Execution

curl -X POST -d "key=value" https://httpbin.org/post

Output

{
  "form": {
    "key": "value"
  },
  "headers": {
    "Content-Type": "application/x-www-form-urlencoded"
  }
}

GET requests are cacheable and idempotent
POST creates new resources and is not idempotent
Methods determine server behavior and response

Simple Requests

Make your first requests to APIs and websites

Get a web page HTML content

Fetch web content using curl and save or display the response.

Code

# Fetch HTML from a website
curl https://example.com

# Save HTML to file
curl https://example.com > page.html

# Get with all headers
curl -i https://example.com

# Save to file with proper name
curl -o index.html https://example.com

Execution

curl -s https://httpbin.org/html | head -20

Output

<!DOCTYPE html>
<html>
  <head>
  </head>
  <body>
      <h1>Herman Melville - Moby-Dick</h1>

Without flags, response is printed to stdout
Use > to redirect to file or -o to save
-i includes HTTP headers in response

Query parameters and URLs

Pass query parameters to APIs using URL query strings.

Code

# Simple URL with query parameters
curl "https://api.example.com/search?q=curl&page=1"

# Handle special characters in parameters
curl "https://api.example.com/search?q=hello%20world"

# URL encoding alternative
curl --data-urlencode "q=hello world" https://api.example.com/search

# Test with httpbin
curl "https://httpbin.org/get?name=John&age=30"

Execution

curl -s "https://httpbin.org/get?name=Alice&age=25" | grep -A5 args

Output

"args": {
  "age": "25",
  "name": "Alice"
}

Use quotes around URLs with special characters
Query parameters start with ?
Multiple parameters separated by &
URL encode special characters

Follow redirects

Use -L flag to automatically follow HTTP redirects (301, 302, etc.).

Code

# Follow HTTP redirects
curl -L https://example.com

# Follow max 5 redirects
curl -L --max-redirs 5 https://example.com

# Show redirect chain
curl -L -v https://example.com 2>&1 | grep ">"

Execution

curl -L -s -w "Final URL: %{url_effective}\n" https://httpbin.org/redirect/2

Output

Final URL: https://httpbin.org/relative-redirect/1

Default curl doesn't follow redirects
-L (--location) enables automatic redirect following
--max-redirs limits maximum redirects to prevent loops

Request Methods

Detailed usage of different HTTP request methods

GET Requests

Retrieve data from servers with GET requests

Basic GET requests with parameters

GET requests retrieve data with parameters appended to the URL.

Code

# Get with parameters
curl https://api.example.com/users?limit=10&offset=0

# Get specific resource
curl https://api.example.com/users/123

# Get with multiple parameters
curl "https://api.example.com/posts?author=john&status=published&sort=date"

# Use -G to send POST data as GET
curl -G https://api.example.com/search -d "q=api" -d "limit=10"

Execution

curl -s "https://httpbin.org/get?id=1&name=test" | python3 -m json.tool

Output

{
  "args": {
    "id": "1",
    "name": "test"
  },
  "url": "https://httpbin.org/get?id=1&name=test"
}

GET parameters are visible in URL
Use for non-sensitive data
Parameters passed via query string

Conditional GET requests

Optimize GET requests with conditional headers and timeouts.

Code

# Get only if modified since date (caching)
curl -H "If-Modified-Since: Mon, 12 Dec 2022 13:00:00 GMT" https://api.example.com/data

# Check if resource exists (HEAD request for headers only)
curl -I https://api.example.com/users/123

# Get with timeout
curl --max-time 5 https://api.example.com/data

Execution

curl -I -s "https://httpbin.org/get" | head -5

Output

HTTP/1.1 200 OK
Date: Fri, 28 Feb 2025 12:00:00 GMT
Content-Type: application/json
Content-Length: 347

-I gets headers only (more efficient)
--max-time prevents hanging on slow servers
If-Modified-Since improves caching

Handle large responses

Download large files efficiently with progress tracking and resume capability.

Code

# Get large file and show progress
curl -# https://api.example.com/large-dataset.json -o data.json

# Resume interrupted download
curl -C - -o data.json https://api.example.com/large-file.json

# Stream response (don't wait for completion)
curl -N https://api.example.com/stream

# Get with progress bar
curl -# -O https://example.com/file.zip

Execution

curl -s "https://httpbin.org/get" | wc -c

Output

-# shows simple progress bar
-C - resumes partial downloads
-N enables streaming responses
-O saves with original filename

POST Requests

Send data to servers with POST requests

POST with form data

Send form data using POST requests with URL-encoded parameters.

Code

# POST with form data (URL encoded)
curl -X POST -d "name=John&email=john@example.com" https://api.example.com/users

# POST with multiple form fields
curl -d "user=john" -d "password=secret" -d "login=true" https://api.example.com/login

# Post data from file
curl -d @data.txt https://api.example.com/submit

# Alternative form data syntax
curl -X POST --data "key1=value1&key2=value2" https://api.example.com/create

Execution

curl -s -X POST -d "user=john&age=30" https://httpbin.org/post | python3 -m json.tool

Output

{
  "form": {
    "age": "30",
    "user": "john"
  }
}

-d automatically sets Content-Type to application/x-www-form-urlencoded
Multiple -d flags combine parameters
@filename sends file contents as data

POST with JSON data

Send JSON data using proper Content-Type header to APIs expecting JSON.

Code

# POST JSON data
curl -X POST -H "Content-Type: application/json" \
  -d '{"name":"John","email":"john@example.com"}' \
  https://api.example.com/users

# POST JSON from file
curl -X POST -H "Content-Type: application/json" \
  -d @user.json https://api.example.com/users

# Using jq to format JSON
curl -X POST -H "Content-Type: application/json" \
  -d "$(echo '{"name":"John"}' | jq .)" \
  https://api.example.com/users

Execution

curl -s -X POST -H "Content-Type: application/json" -d '{"key":"value"}' https://httpbin.org/post | python3 -m json.tool

Output

{
  "json": {
    "key": "value"
  }
}

Must set Content-Type: application/json header
Payload sent as JSON string
Use single quotes to preserve JSON structure

POST with file upload

Upload files using multipart/form-data encoding with optional additional fields.

Code

# Upload file with multipart form
curl -F "file=@image.png" https://api.example.com/upload

# Upload with additional fields
curl -F "file=@photo.jpg" -F "title=My Photo" -F "description=A beautiful photo" https://api.example.com/upload

# Upload multiple files
curl -F "files=@file1.txt" -F "files=@file2.txt" https://api.example.com/upload-multiple

# Upload with HTTP Basic auth
curl -F "file=@document.pdf" -u user:pass https://api.example.com/upload

Execution

curl -s -F "file=@/etc/hosts" -F "title=test" https://httpbin.org/post

Output

{
  "files": {
    "file": "[binary content]"
  },
  "form": {
    "title": "test"
  }
}

-F flag sets Content-Type to multipart/form-data automatically
@ symbol indicates file path
Can combine multiple files and form fields

PUT, PATCH, and DELETE Requests

Update and delete resources with different methods

PUT and PATCH requests

Use PUT to replace entire resources and PATCH to partially update them.

Code

# PUT - Replace entire resource
curl -X PUT -H "Content-Type: application/json" \
  -d '{"name":"Jane","email":"jane@example.com","age":28}' \
  https://api.example.com/users/1

# PATCH - Update specific fields only
curl -X PATCH -H "Content-Type: application/json" \
  -d '{"email":"newemail@example.com"}' \
  https://api.example.com/users/1

# PUT with form data
curl -X PUT -d "name=John&status=active" https://api.example.com/users/1

# PATCH with file
curl -X PATCH -H "Content-Type: application/json" \
  -d @user-update.json https://api.example.com/users/1

Execution

curl -s -X PATCH -H "Content-Type: application/json" -d '{"status":"updated"}' https://httpbin.org/patch | python3 -m json.tool

Output

{
  "json": {
    "status": "updated"
  }
}

PUT replaces entire resource (must provide all fields)
PATCH updates only specified fields
Both typically require authentication

DELETE requests

Remove resources from the server using DELETE requests.

Code

# Simple DELETE
curl -X DELETE https://api.example.com/users/1

# DELETE with response verification
curl -X DELETE -w "Status: %{http_code}\n" https://api.example.com/users/1

# DELETE with authorization
curl -X DELETE -H "Authorization: Bearer token123" https://api.example.com/users/1

# DELETE with query parameters
curl -X DELETE "https://api.example.com/posts?id=123&confirm=true"

Execution

curl -X DELETE -w "\nHTTP Status: %{http_code}\n" https://httpbin.org/delete

Output

HTTP Status: 200

DELETE is idempotent (multiple calls have same effect)
Typically returns 204 (No Content) or 200 (OK)
May require authentication

Request with body and status verification

Combine requests with response verification and output handling.

Code

# Update and verify response
curl -X PUT \
  -H "Content-Type: application/json" \
  -d '{"status":"active"}' \
  -w "\nStatus: %{http_code}\nTime: %{time_total}s\n" \
  https://api.example.com/users/1

# Delete and check status
curl -i -X DELETE https://api.example.com/users/1

# Save response to file on successful update
curl -X PATCH -d "status=completed" \
  -o response.json \
  -w "%{http_code}" \
  https://api.example.com/tasks/1

Execution

curl -i -X DELETE https://httpbin.org/delete | head -10

Output

HTTP/2 200
Date: Fri, 28 Feb 2025 12:00:00 GMT
Content-Type: application/json

-i includes headers and response
-w allows custom output formatting
Combine with -o to save to file

Headers & Authentication

Work with HTTP headers and authentication mechanisms

Custom Headers

Add custom headers to requests

Set custom headers with -H flag

Add custom headers to requests using the -H flag (multiple times if needed).

Code

# Single header
curl -H "X-API-Key: abc123" https://api.example.com/data

# Multiple headers
curl -H "X-API-Key: abc123" -H "X-Request-ID: req-123" https://api.example.com/data

# Content-Type for JSON
curl -H "Content-Type: application/json" \
  -d '{"key":"value"}' https://api.example.com/submit

# Accept header for response format
curl -H "Accept: application/json" https://api.example.com/data

Execution

curl -s -H "X-Custom: hello" https://httpbin.org/headers | python3 -m json.tool

Output

{
  "headers": {
    "X-Custom": "hello",
    "User-Agent": "curl/7.88.1"
  }
}

Format: -H "Header-Name: Header-Value"
Multiple headers require multiple -H flags
Headers are case-insensitive

User-Agent and Referer headers

Set User-Agent and Referer headers to identify your request origin and client.

Code

# Set user agent (identify as browser)
curl -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36" https://example.com

# Alternative syntax for user agent
curl -H "User-Agent: MyApp/1.0" https://api.example.com/data

# Set referer (HTTP_REFERER)
curl -e "https://google.com" https://example.com

# Combined user agent and referer
curl -A "Mozilla/5.0" -e "https://google.com" https://example.com

Execution

curl -s -A "CustomBot/1.0" https://httpbin.org/user-agent

Output

{
  "user-agent": "CustomBot/1.0"
}

-A is shortcut for setting User-Agent
-e sets Referer header (note spelling)
Some sites may block requests without proper User-Agent

Request header inspection

View outgoing request headers using -v (verbose) flag.

Code

# View all request headers sent
curl -v https://api.example.com/data 2>&1 | grep ">"

# Show comprehensive header information
curl -v -H "Authorization: Bearer token" https://api.example.com/data 2>&1 | head -20

# Test with custom headers and see what was sent
curl -H "X-Test: value1" -H "X-Test2: value2" -v https://httpbin.org/headers 2>&1 | grep ">"

Execution

curl -v https://httpbin.org/headers 2>&1 | grep ">"

Output

> GET /headers HTTP/1.1
> Host: httpbin.org
> User-Agent: curl/7.88.1
> Accept: */*

Lines starting with > show request headers
Lines starting with < show response headers
2>&1 redirects stderr to stdout

Authentication Methods

Authenticate requests with various methods

HTTP Basic Authentication

Use HTTP Basic Authentication with -u flag (username:password).

Code

# Basic auth with -u flag
curl -u username:password https://api.example.com/protected

# Basic auth alternative with header
curl -H "Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=" https://api.example.com/protected

# Base64 encode credentials
echo -n "username:password" | base64

# Prompt for password (don't show in history)
curl -u username https://api.example.com/protected

Execution

curl -s -u "user:pass" https://httpbin.org/basic-auth/user/pass

Output

{
  "authenticated": true,
  "user": "user"
}

Credentials are Base64 encoded, not encrypted
Always use HTTPS with Basic Auth (not HTTP)
-u without password prompts for it securely

Bearer token and API key authentication

Authenticate with bearer tokens (JWT, OAuth2) and API keys.

Code

# Bearer token (OAuth2, JWT, etc)
curl -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIs..." https://api.example.com/data

# API key in header
curl -H "X-API-Key: sk_live_abc123xyz789" https://api.example.com/data

# API key as query parameter
curl "https://api.example.com/data?api_key=sk_live_abc123"

# Store token in variable and use it
TOKEN="eyJhbGciOiJIUzI1NiIs..."
curl -H "Authorization: Bearer $TOKEN" https://api.example.com/data

Execution

curl -s -H "Authorization: Bearer test-token" https://httpbin.org/bearer | python3 -m json.tool

Output

{
  "authenticated": true,
  "token": "test-token"
}

Bearer tokens include "Bearer " prefix
API keys are commonly passed in headers or query params
Tokens are sensitive, avoid including in logs

Complex authentication scenarios

Implement more advanced authentication flows and secure credential handling.

Code

# Digest authentication (more secure than basic)
curl --digest -u username:password https://api.example.com/protected

# Get JWT token then use it
TOKEN=$(curl -s -X POST -d "user=john&pass=secret" https://api.example.com/auth | jq -r '.token')
curl -H "Authorization: Bearer $TOKEN" https://api.example.com/data

# Use netrc file for credentials (~/.netrc)
curl --netrc https://api.example.com/protected

# Pass credentials in URL (deprecated, use -u instead)
curl "https://user:pass@api.example.com/protected"

Execution

curl -s --digest -u "user:passwd" https://httpbin.org/digest-auth/auth/user/passwd

Output

{
  "authenticated": true,
  "user": "user"
}

Digest auth is more secure than Basic auth
Never include passwords in URLs
Use environment variables or .netrc for sensitive credentials

Cookies & Sessions

Handle cookies and maintain sessions

Set and send cookies

Send cookies with requests using -b flag.

Code

# Send cookie with request
curl -b "session=abc123" https://api.example.com/data

# Multiple cookies
curl -b "session=abc123" -b "user_id=456" https://api.example.com/data

# Cookie from file
curl -b cookies.txt https://api.example.com/data

# Send raw cookie string
curl -b "name1=value1; name2=value2" https://example.com

Execution

curl -s -b "test=value123" https://httpbin.org/cookies | python3 -m json.tool

Output

{
  "cookies": {
    "test": "value123"
  }
}

-b sends cookies, -B sends from file, -c saves to file
Multiple -b flags combine cookies
Cookie format: name=value

Save and load cookies

Save cookies from responses and reuse them in subsequent requests for session management.

Code

# Save response cookies to file
curl -c cookies.txt https://api.example.com/login

# Load cookies from file for subsequent requests
curl -b cookies.txt https://api.example.com/protected

# Save and load in single command
curl -c cookies.txt -b cookies.txt https://api.example.com/data

# Enable cookie jar for session persistence
curl -c ~/.curl_cookies -b ~/.curl_cookies https://api.example.com/login

Execution

curl -s https://httpbin.org/cookies/set?test=value123 -c /tmp/cookies.txt && cat /tmp/cookies.txt

Output

# Netscape HTTP Cookie File
.httpbin.org  TRUE  /  FALSE  0  test  value123

-c saves response cookies to file
Netscape HTTP Cookie File format is standard
Useful for maintaining sessions across multiple requests

Session workflow example

Manage complete session workflow from login to authenticated requests to logout.

Code

# Typical workflow: login, then use session

# 1. Login and save session cookie
curl -c cookies.txt -X POST \
  -d "username=john&password=secret" \
  https://api.example.com/login

# 2. Use session for authenticated requests
curl -b cookies.txt https://api.example.com/profile

# 3. Make multiple requests with same session
curl -b cookies.txt https://api.example.com/posts
curl -b cookies.txt -X POST -d "content=hello" https://api.example.com/posts

# 4. Logout and clear cookies
curl -b cookies.txt -X POST https://api.example.com/logout
rm cookies.txt

Execution

curl -s -c - https://httpbin.org/cookies/set?session=xyz123 | grep -i httpbin

Output

.httpbin.org  TRUE  /  FALSE  0  session  xyz123

Always save and load cookies for stateful APIs
Use same cookie jar across related requests
Clear cookies after logout for security

Data Transfer

Handle different data formats and file operations

JSON Data

Work with JSON data in requests and responses

Send JSON requests

Send properly formatted JSON data to APIs.

Code

# Basic JSON POST
curl -X POST -H "Content-Type: application/json" \
  -d '{"name":"John","email":"john@example.com"}' \
  https://api.example.com/users

# Pretty JSON with jq
curl -X POST -H "Content-Type: application/json" \
  -d "$(jq -n --arg name "John" --arg email "john@example.com" '{name: $name, email: $email}')" \
  https://api.example.com/users

# Nested JSON
curl -X POST -H "Content-Type: application/json" \
  -d '{"user":{"name":"John","address":{"city":"NYC"}},"active":true}' \
  https://api.example.com/users

Execution

curl -s -X POST -H "Content-Type: application/json" -d '{"test":"data"}' https://httpbin.org/post | python3 -m json.tool

Output

{
  "json": {
    "test": "data"
  }
}

Always set Content-Type: application/json
Single quotes preserve special characters in JSON
Escape quotes in JSON: \" for double quotes

Parse and modify JSON responses

Parse, filter, and manipulate JSON responses using jq for complex data operations.

Code

# Get JSON and parse with jq
curl -s https://api.example.com/users | jq '.'

# Extract specific field
curl -s https://api.example.com/users | jq '.users[0].name'

# Filter and map responses
curl -s https://api.example.com/users | jq '[.users[] | select(.active) | {name, email}]'

# Modify response and send as next request
curl -s https://api.example.com/users/1 | jq '.status = "updated"' | \
  curl -X PATCH -H "Content-Type: application/json" -d @- https://api.example.com/users/1

Execution

curl -s https://httpbin.org/json | python3 -m json.tool | head -10

Output

{
  "slideshow": {
    "author": "Yours Truly",
    "date": "2003-05-20",
    "slides": [
      {
        "title": "Wake up to WonderWidgets!"
      }
    ]
  }
}

Install jq: apt-get install jq (or brew install jq on macOS)
jq is powerful for JSON transformation
(hyphen) in curl -d @- reads from stdin

JSON to form data conversion

Choose correct Content-Type based on API requirements.

Code

# Convert JSON to form data
curl -X POST -H "Content-Type: application/json" \
  -d '{"username":"john","password":"secret"}' \
  https://api.example.com/api-submit

# Compare with form data version
curl -X POST \
  -d "username=john&password=secret" \
  https://api.example.com/form-submit

# Use correct Content-Type based on API requirements
# JSON APIs need: application/json
# Form APIs need: application/x-www-form-urlencoded
# Multipart APIs need: multipart/form-data (auto with -F)

Execution

curl -s -X POST -d "key=value&test=data" https://httpbin.org/post | python3 -m json.tool | grep -A5 form

Output

"form": {
  "key": "value",
  "test": "data"
}

JSON uses application/json
Form data uses application/x-www-form-urlencoded (default -d)
Multipart forms use multipart/form-data (-F)

File Uploads & Downloads

Upload and download files efficiently

Upload files with multipart form

Upload files with multipart form encoding for APIs that accept file uploads.

Code

# Upload single file
curl -F "file=@/path/to/file.txt" https://api.example.com/upload

# Upload with additional metadata
curl -F "file=@/path/to/image.png" \
  -F "title=My Image" \
  -F "description=Beautiful image" \
  https://api.example.com/upload

# Upload multiple files
curl -F "files=@file1.txt" -F "files=@file2.txt" -F "files=@file3.txt" \
  https://api.example.com/upload-batch

# Upload with authentication
curl -F "file=@document.pdf" -u user:pass https://api.example.com/upload

Execution

curl -s -F "file=@/etc/hostname" https://httpbin.org/post | python3 -m json.tool | head -15

Output

{
  "files": {
    "file": "[binary content]"
  }
}

-F flag automatically sets multipart/form-data
@ prefix indicates file path
Repeat -F for multiple files with same field name

Download and resume downloads

Download files efficiently with progress tracking and resume capability.

Code

# Download file
curl -o filename.zip https://example.com/large-file.zip

# Download with original filename
curl -O https://example.com/file.pdf

# Resume interrupted download
curl -C - -o file.zip https://example.com/file.zip

# Show download progress
curl -# -o file.zip https://example.com/file.zip

# Download multiple files
curl -o file1.txt https://example.com/file1.txt -o file2.txt https://example.com/file2.txt

Execution

curl -s -I https://httpbin.org/image/jpeg | grep Content-Length

Output

Content-Length: 8633

-o specifies output filename
-O uses original filename from URL
-C - resumes partial downloads
# shows simple progress bar

Advanced file operations

Combine curl with other tools for complex file operations and error handling.

Code

# Download and pipe to another command
curl -s https://example.com/archive.tar.gz | tar xz

# Download multiple files from listing
curl -s https://example.com/files.json | jq -r '.[] | .url' | \
  xargs -I {} curl -O {}

# Upload file with progress and error handling
if curl -F "file=@largefile.bin" \
  --progress-bar \
  https://api.example.com/upload; then
  echo "Upload successful"
else
  echo "Upload failed"
fi

# Download with custom headers and authentication
curl -H "Authorization: Bearer token" \
  -H "Accept: application/octet-stream" \
  -o downloaded-file \
  https://api.example.com/download

Execution

curl -s https://httpbin.org/image/png | file -

Output

standard input: PNG image data, 200 x 200...

Pipe output to tar, unzip, etc. for automated extraction
Use xargs to download multiple files in parallel
Check status codes ($?) for error handling

Streaming & Chunked Data

Handle streaming and chunked data transfer

Stream responses

Receive streaming responses and process them line-by-line without waiting.

Code

# Stream without buffering (don't wait for complete response)
curl -N https://api.example.com/stream

# Process streaming JSON lines
curl -N https://example.com/events | while read line; do
  echo "$line" | jq '.event'
done

# Stream with timeout
curl -N --max-time 30 https://api.example.com/stream

# Save stream content
curl -N https://example.com/stream -o stream.log

Execution

timeout 2 curl -s -N https://httpbin.org/stream-bytes/100 | xxd | head -5

Output

00000000: 8d5e 16cf 9cac 098e e3dc 1fa3 5f7e 9bb7  .^.........._~..
00000010: 89e9 7c5a ed66 7fa5 e901 40b5 0b46 6903  ..|Z.f....@..Fi.

-N enables streaming (no buffering)
Useful for long-lived streams and event feeds
Can save to file or pipe to other commands

Chunked uploads

Upload large files in chunks and streams for efficient bandwidth usage.

Code

# Upload in chunks (for large files)
# Using HTTP/1.1 Transfer-Encoding: chunked
curl -H "Transfer-Encoding: chunked" \
  -d @largefile.bin \
  https://api.example.com/upload

# Stream file content
curl -X PUT -H "Content-Type: application/octet-stream" \
  --data-binary @file.bin \
  https://api.example.com/objects/file.bin

# Split large file and upload parts
split -b 1M largefile.bin part_
for part in part_*; do
  curl -X POST -F "file=@$part" \
    https://api.example.com/upload-part
done

Execution

echo "test data" | curl -s -X POST -d @- https://httpbin.org/post

Output

{
  "data": "test data"
}

Transfer-Encoding: chunked for chunked uploads
--data-binary preserves binary file content
Split and reassemble for multi-part uploads

Output & Debugging

Inspect responses and debug requests

Verbose Mode & Inspection

Debug requests and responses with verbose output

Enable verbose mode for troubleshooting

Use verbose mode to see all request and response details for debugging.

Code

# Basic verbose mode (shows headers)
curl -v https://api.example.com/data

# Extra verbose (shows all details)
curl -vv https://api.example.com/data

# Verbose to file
curl -v https://api.example.com/data > response.html 2> debug.txt

# Show request headers only
curl -v https://api.example.com/data 2>&1 | grep ">"

# Show response headers only
curl -v https://api.example.com/data 2>&1 | grep "<"

Execution

curl -v https://httpbin.org/get 2>&1 | head -15

Output

*   Trying 34.201.40.105:443...
* Connected to httpbin.org (34.201.40.105) port 443 (#0)
* ALPN, offering h2
> GET /get HTTP/1.1
> Host: httpbin.org
> User-Agent: curl/7.88.1

Lines starting with > are request headers
Lines starting with < are response headers
Lines starting with * are curl debug info

Show headers without body

Inspect response headers without downloading body content.

Code

# Headers only (no body)
curl -i https://api.example.com/data

# Headers from HEAD request only
curl -I https://api.example.com/data

# Just response headers with body as well
curl -D - https://api.example.com/data

# Dump headers to file
curl -D headers.txt https://api.example.com/data

Execution

curl -I -s https://httpbin.org/json 2>&1

Output

HTTP/2 200
date: Fri, 28 Feb 2025 12:00:00 GMT
content-type: application/json
content-length: 429

-i includes headers and body
-I gets headers only (HEAD request)
-D saves headers to file

Detailed timing and trace information

Measure performance metrics and timing information for API requests.

Code

# Show timing breakdown
curl -w "@curl-format.txt" -o /dev/null https://api.example.com/data

# Simple timing format
curl -w "\nTime: %{time_total}s\nSpeed: %{speed_download} bytes/s\n" -o /dev/null https://example.com

# Comprehensive timing
curl -w "\n\nDNS Lookup: %{time_namelookup}s\nTCP Connect: %{time_connect}s\nFirst Byte: %{time_starttransfer}s\nTotal Time: %{time_total}s\n" https://api.example.com/data

# Trace connection information
curl --trace-ascii debug.txt https://api.example.com/data

Execution

curl -w "\nStatus: %{http_code}\nTime: %{time_total}s\n" -o /dev/null -s https://httpbin.org/get

Output

Status: 200
Time: 0.452341s

%{time_total} total request time
%{time_namelookup} DNS resolution time
%{http_code} HTTP response status code
--trace-ascii for detailed debugging

Filter & Customize Output

Filter responses and customize output format

Extract specific data from responses

Extract specific fields from API responses using jq, grep, and other tools.

Code

# Extract JSON field with jq
curl -s https://api.example.com/users | jq '.users[0].email'

# Extract HTML content with grep
curl -s https://example.com | grep -oP '(?<=<title>)[^<]*'

# Extract HTTP header value
curl -s -I https://api.example.com/data | grep -i "content-type"

# Count items in response
curl -s https://api.example.com/items | jq '.items | length'

Execution

curl -s https://httpbin.org/get | python3 -c "import json, sys; data=json.load(sys.stdin); print(data['url'])"

Output

https://httpbin.org/get

jq best for JSON
grep with -oP for regex extraction
pipe | allows chaining commands

Format output with -w flag

Customize output format using -w flag for scripting and batch operations.

Code

# Status code only
curl -w "%{http_code}\n" -o /dev/null https://api.example.com/data

# URL and status
curl -w "%{url_effective}\t%{http_code}\n" -o /dev/null https://api.example.com

# Multiple values separated by tabs
curl -w "%{http_code}\t%{time_total}\t%{size_download}\n" -o /dev/null https://api.example.com/data

# Custom format for batch processing
curl -w "%{filename_effective},%{http_code},%{time_total}\n" -o data.json https://api.example.com/data

Execution

curl -s -w "\nURL: %{url_effective}\nStatus: %{http_code}\nSize: %{size_download} bytes\n" https://httpbin.org/get -o /dev/null

Output

URL: https://httpbin.org/get
Status: 200
Size: 389 bytes

%{http_code} HTTP status code
%{time_total} total time
%{size_download} downloaded size

Batch processing and response validation

Validate responses and process multiple items from API responses.

Code

# Check multiple endpoints
for url in https://api.example.com/users https://api.example.com/posts https://api.example.com/comments; do
  status=$(curl -s -o /dev/null -w "%{http_code}" "$url")
  echo "$url: $status"
done

# Validate JSON response
response=$(curl -s https://api.example.com/data)
if echo "$response" | jq empty 2>/dev/null; then
  echo "Valid JSON"
else
  echo "Invalid JSON: $response"
fi

# Extract and process multiple values
curl -s https://api.example.com/users | jq -r '.[] | "ID: \(.id), Name: \(.name), Email: \(.email)"'

Execution

curl -s https://httpbin.org/get | python3 -c "import json, sys; json.load(sys.stdin); print('Valid JSON')"

Output

Valid JSON

Use jq empty to validate JSON without output
Process arrays with jq .[]
Combine with bash loops for batch operations

Error Handling & Status

Handle errors and check response status

Check HTTP status codes

Extract and evaluate HTTP status codes for error handling.

Code

# Get HTTP status code
status=$(curl -s -o /dev/null -w "%{http_code}" https://api.example.com/data)

# Check if successful (2xx)
if [ $status -ge 200 ] && [ $status -lt 300 ]; then
  echo "Success: $status"
elif [ $status -ge 400 ] && [ $status -lt 500 ]; then
  echo "Client error: $status"
else
  echo "Server error: $status"
fi

# Fail on non-2xx status
curl -f https://api.example.com/data || echo "Request failed"

Execution

curl -f -s https://httpbin.org/status/404 2>&1 || echo "Exit code: $?"

Output

Exit code: 22

-f flag returns non-zero exit on HTTP errors
2xx = success, 4xx = client error, 5xx = server error
Check $? for curl exit code

Retry logic for failed requests

Implement retry logic for handling transient failures.

Code

# Simple retry with exponential backoff
retry_count=0
max_retries=3
while [ $retry_count -lt $max_retries ]; do
  if curl -f https://api.example.com/data; then
    echo "Success"
    break
  fi
  retry_count=$((retry_count + 1))
  sleep $((2 ** retry_count))
done

# Retry on timeout
curl --connect-timeout 5 --max-time 10 https://api.example.com/data || \
  (sleep 2 && curl --connect-timeout 5 --max-time 10 https://api.example.com/data)

# Retry with custom exit code check
for attempt in {1..3}; do
  curl -f https://api.example.com/data && break
  [ $attempt -lt 3 ] && sleep $((attempt * 2))
done

Execution

curl -s --connect-timeout 2 https://httpbin.org/delay/1 | head -20

Output

{
  "args": {},
  "delay": 1,
  "headers": {

--connect-timeout limits connection time
--max-time limits total request time
Exponential backoff prevents overwhelming server

Capture errors and responses

Capture status codes and error messages for proper error handling.

Code

# Capture both success and error responses
response=$(curl -s -w "\nSTATUS:%{http_code}" https://api.example.com/data)
body=$(echo "$response" | sed '$d')
status=$(echo "$response" | tail -1 | cut -d: -f2)

if [ "$status" -eq 200 ]; then
  echo "Success response: $body"
else
  echo "Error ($status): $body"
fi

# Save error responses
curl -s -w "%{http_code}" https://api.example.com/data -o response.json
if [ $? -ne 0 ]; then
  echo "Curl failed" >&2
  cat response.json >&2
  exit 1
fi

Execution

curl -s -w "\n%{http_code}" https://httpbin.org/get | tail -1

Output

Use -w to append status code to output
sed '$d' removes last line
tail -1 gets last line
check $? for curl exit code

SSL/TLS & Security

Configure SSL/TLS and handle secure connections

SSL Certificates

Work with SSL/TLS certificates and HTTPS

Handle SSL certificate issues

Configure certificate handling for various HTTPS scenarios.

Code

# Ignore SSL certificate validation (not recommended)
curl -k https://self-signed-cert.example.com

# Use custom CA certificate
curl --cacert /path/to/ca-cert.pem https://api.example.com/data

# Specify client certificate
curl --cert client-cert.pem --key client-key.pem https://api.example.com/data

# Use PKCS12 certificate file
curl --cert certificate.p12:password https://api.example.com/data

Execution

curl -I https://httpbin.org 2>&1 | head -5

Output

HTTP/2 200
date: Fri, 28 Feb 2025 12:00:00 GMT
content-type: application/json

-k ignores certificate verification (unsafe)
--cacert uses custom CA certificate
--cert specifies client certificate

View certificate information

Inspect SSL/TLS certificate information for debugging connection issues.

Code

# Show certificate details
echo | openssl s_client -connect api.example.com:443 2>/dev/null | openssl x509 -text

# Extract certificate from curl verbose
curl -v https://api.example.com/data 2>&1 | grep "subject:"

# Save certificate to file
echo | openssl s_client -connect api.example.com:443 -showcerts | \
  openssl x509 -out server-cert.pem

# Check certificate expiration
curl --insecure -I https://api.example.com 2>&1 | head -5 && \
  echo | openssl s_client -connect api.example.com:443 2>/dev/null | \
  openssl x509 -noout -dates

Execution

echo | openssl s_client -connect httpbin.org:443 -showcerts 2>/dev/null | grep subject

Output

subject=CN = httpbin.org

openssl s_client connects and shows certificate
openssl x509 displays certificate details
Check dates for expiration

Mutual TLS (mTLS) configuration

Configure mutual TLS authentication with client and server certificates.

Code

# mTLS with client and CA certificates
curl --cert client-cert.pem \
  --key client-key.pem \
  --cacert ca-cert.pem \
  https://api.example.com/secure

# Using certificate bundle
curl --cert /etc/ssl/certs/client-cert.pem \
  --key /etc/ssl/private/client-key.pem \
  --cacert /etc/ssl/certs/ca-bundle.crt \
  https://secure-api.example.com

# Verify certificate against hostname
curl --cert-type PEM \
  --key-type PEM \
  --cacert ca.pem \
  https://api.example.com/data

Execution

curl -i https://httpbin.org 2>&1 | grep -i "certificate\|subject"

Output

* Certificate verified successfully

--cert client certificate file
--key private key file
--cacert CA certificate for verification

Security Best Practices

Secure curl usage for APIs and services

Protect sensitive data

Safely handle sensitive credentials using environment variables and secure storage.

Code

# Store credentials in environment variables
API_KEY="$GITHUB_TOKEN"
curl -H "Authorization: token $API_KEY" https://api.github.com/user

# Use .netrc file for credentials (~/.netrc)
# Format: machine api.example.com login user password secret
# Permissions: chmod 600 ~/.netrc
curl --netrc https://api.example.com/data

# Read password securely without logging
read -sp "Enter API Key: " API_KEY
curl -H "X-API-Key: $API_KEY" https://api.example.com/data

# Don't include credentials in commands (use processes)
API_KEY=$(cat ~/.config/api-key)
curl -H "Authorization: Bearer $API_KEY" https://api.example.com

Execution

echo 'Best practice: Use environment variables for secrets'

Output

Best practice: Use environment variables for secrets

Never hardcode credentials in scripts
Use environment variables
.netrc file for reusable credentials

Network security measures

Implement security best practices for network communication.

Code

# Use timeouts to prevent hanging requests
curl --connect-timeout 5 --max-time 30 https://api.example.com/data

# Limit supported protocols
curl --tlsv1.2 --tlsv1.3 https://api.example.com/data

# Disable insecure protocols
curl --disable https://api.example.com/data

# Use IPv4 or IPv6 only
curl -4 https://api.example.com  # IPv4 only
curl -6 https://api.example.com  # IPv6 only

# Restrict to specific IP ranges
curl --resolve api.example.com:443:192.0.2.1 https://api.example.com/data

Execution

curl --connect-timeout 2 -m 5 https://httpbin.org/delay/1 2>&1 | head -3

Output

{
  "delayed": true,
  "sleep": 1

--connect-timeout and --max-time prevent DoS
Explicit TLS versions increase security
IP restrictions prevent DNS spoofing

Logging and monitoring security

Log and monitor API requests while protecting sensitive information.

Code

# Log requests without sensitive headers
curl -v https://api.example.com/data 2>&1 | sed 's/Authorization:.*/Authorization: [REDACTED]/'

# Monitor failed authentication attempts
log_file="/var/log/api-requests.log"
if ! curl -f -H "Authorization: Bearer $TOKEN" https://api.example.com/data >> "$log_file" 2>&1; then
  echo "Failed auth attempt at $(date)" >> "$log_file"
fi

# Audit API requests
{
  echo "Request to: https://api.example.com"
  echo "Time: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
  echo "Status: $(curl -s -o /dev/null -w '%{http_code}' https://api.example.com)"
} >> api-audit.log

# Redact sensitive data from responses
curl -s https://api.example.com/data | jq '.password = "[REDACTED]" | .token = "[REDACTED]"'

Execution

echo "curl -v <url> 2>&1 | sed 's/Authorization:.*/Authorization: [REDACTED]/'"

Output

curl -v <url> 2>&1 | sed 's/Authorization:.*/Authorization: [REDACTED]/'

Redact credentials in logs
Monitor for failed authentication
Audit API usage for security analysis

Advanced Features

Advanced curl options and techniques

Proxies & Redirects

Configure proxies and handle redirects

Configure HTTP and SOCKS proxies

Route requests through HTTP or SOCKS proxies for privacy or network access.

Code

# HTTP proxy
curl -x http://proxy.example.com:8080 https://api.example.com/data

# SOCKS5 proxy
curl -x socks5://proxy.example.com:1080 https://api.example.com/data

# Proxy with authentication
curl -x http://user:pass@proxy.example.com:8080 https://api.example.com/data

# Use environment variable for proxy
export http_proxy="http://proxy.example.com:8080"
curl https://api.example.com/data

Execution

curl -x "http://127.0.0.1:3128" https://httpbin.org/ip 2>&1 | head -5

Output

* Connected to 127.0.0.1 (127.0.0.1) via SOCKS proxy
(Connection refused)

-x specifies proxy
Format: protocol://[user:pass@]host:port
Environment variables: http_proxy, https_proxy, no_proxy

Redirect handling

Control redirect behavior and trace redirect chains.

Code

# Follow redirects
curl -L https://api.example.com/data

# Follow max 5 redirects
curl -L --max-redirs 5 https://api.example.com/data

# Don't follow redirects (default)
curl https://api.example.com/data

# Show redirect chain
curl -L -v https://example.com 2>&1 | grep -E "^> |< HTTP"

# POST data preserved across redirects (except method change)
curl -L -X POST -d "data=value" https://api.example.com/endpoint

Execution

curl -L -s -w "\nFinal URL: %{url_effective}\n" https://httpbin.org/redirect/2 | tail -5

Output

Final URL: https://httpbin.org/get

-L enables automatic redirect following
--max-redirs limits maximum redirects
%{url_effective} shows final URL after redirects

Advanced redirect scenarios

Implement complex redirect scenarios with method preservation and tracing.

Code

# Preserve method on redirect
curl -L -X PUT -H "Content-Type: application/json" \
  -d '{"key":"value"}' https://api.example.com/endpoint

# Manual redirect handling
location=$(curl -s -I https://api.example.com/redirect | grep location | cut -d' ' -f2)
curl "$location"

# Redirect with cookie preservation
curl -b cookies.txt -L https://example.com/login
curl -b cookies.txt https://example.com/protected

# Debug redirects in detail
curl -L -v https://example.com 2>&1 | grep -E "^> GET|^< HTTP|^< location"

Execution

curl -L -i https://httpbin.org/redirect-to?url=https://httpbin.org/get 2>&1 | grep -E "^HTTP|^location"

Output

HTTP/1.1 302 FOUND
location: https://httpbin.org/get
HTTP/2 200

curl preserves POST/PUT on redirects only with -L
Manual handling gives more control
Trace redirects with -v flag

Compression & Performance Optimization

Optimize requests with compression and smart techniques

Enable compression

Enable compression for reduced bandwidth and faster transfers.

Code

# Request compression (ask server to compress response)
curl --compressed https://api.example.com/data

# Specify accepted compression methods
curl -H "Accept-Encoding: gzip, deflate, br" https://api.example.com/data

# Decompress and view
curl -s --compressed https://api.example.com/large-file | gzip -d

# Compare sizes
curl -s https://api.example.com/data | wc -c
curl -s --compressed https://api.example.com/data | wc -c

Execution

curl -s --compressed https://httpbin.org/gzip | head -10

Output

{
  "gzipped": true,
  "method": "GET",
  "origin": "192.0.2.1"
}

--compressed requests gzip/deflate compression
Automatic decompression by curl
Significant savings for large responses

Connection pooling and optimization

Optimize connection handling for better performance.

Code

# Reuse connection for multiple requests
url="https://api.example.com"
for i in {1..5}; do
  curl -s "$url/endpoint$i" &
done
wait

# Keep connection alive
curl -H "Connection: keep-alive" https://api.example.com/data

# Disable keep-alive if needed
curl -H "Connection: close" https://api.example.com/data

# Batch multiple requests efficiently
cat urls.txt | xargs -I {} curl -s {} > combined-output.txt

Execution

echo "Connection optimization via HTTP/2, HTTP/1.1 default settings"

Output

Connection optimization via HTTP/2, HTTP/1.1 default settings

HTTP/2 multiplexes multiple requests
Keep-alive reduces connection overhead
Parallel requests with & and wait

Performance measurement and optimization

Measure and optimize request performance with timing analysis.

Code

# Measure and compare performance
time curl -s https://api.example.com/data > /dev/null
time curl -s --compressed https://api.example.com/data > /dev/null

# Monitor bandwidth
curl -# https://api.example.com/large-file -o output

# Optimize with early termination
curl -s https://api.example.com/stream | head -100

# Measure request timing
curl -w "\nTime breakdown:\nDNS: %{time_namelookup}s\nConnect: %{time_connect}s\nTransfer: %{time_starttransfer}s\nTotal: %{time_total}s\n" https://api.example.com/data

Execution

curl -w "\n\nTotal: %{time_total}s\n" -o /dev/null -s https://httpbin.org/delay/1

Output

Total: 1.234s

Use time for overall performance
-w provides detailed breakdown
Compression saves bandwidth significantly

Curl

Getting Started

What is Curl

Accessibility

Best Practices

Common Errors

Keywords

Install curl and check version

Basic curl command structure

Quick API testing examples

HTTP Methods Fundamentals

Accessibility

Best Practices

Common Errors

Keywords

Specify HTTP method with -X flag

Method-specific behavior

Simple Requests

Accessibility

Best Practices

Common Errors

Keywords

Get a web page HTML content

Query parameters and URLs

Follow redirects

Request Methods

GET Requests

Accessibility

Best Practices

Common Errors

Keywords

Basic GET requests with parameters

Conditional GET requests

Handle large responses

POST Requests

Accessibility

Best Practices

Common Errors

Keywords

POST with form data

POST with JSON data

POST with file upload

PUT, PATCH, and DELETE Requests

Accessibility

Best Practices

Common Errors

Keywords

PUT and PATCH requests

DELETE requests

Request with body and status verification

Headers & Authentication

Custom Headers

Accessibility

Best Practices

Common Errors

Keywords

Set custom headers with -H flag

User-Agent and Referer headers

Request header inspection

Authentication Methods

Accessibility

Best Practices

Common Errors

Keywords

HTTP Basic Authentication

Bearer token and API key authentication

Complex authentication scenarios

Cookies & Sessions

Accessibility

Best Practices

Common Errors

Keywords

Set and send cookies

Save and load cookies

Session workflow example

Data Transfer

JSON Data

Accessibility

Best Practices

Common Errors