yq — Command-Line Data File Processor

Examples:

# YAML to JSON
yq -o json file.yaml

# JSON to YAML (pretty-printed)
yq -P -o yaml file.json

# XML to YAML
yq -p xml -o yaml file.xml

# YAML to XML
yq -o xml file.yaml

# CSV to YAML (first row becomes keys)
yq -p csv file.csv

# YAML to CSV (construct the rows you need)
yq -o csv '[["Name","Age"]] + [.people[] | [.name, .age]]' file.yaml

# TOML to JSON
yq -p toml -o json config.toml

Reading values

# Simple path
yq '.metadata.name' deployment.yaml

# Array element
yq '.spec.containers[0].image' deployment.yaml

# All array elements
yq '.items[].name' file.yaml

# Recursive descent — find a key anywhere in the tree
yq '.. | select(has("password")).password' config.yaml

# Length of an array
yq '.items | length' file.yaml

# Keys of a map
yq '.settings | keys' file.yaml

Updating values

# Set a value
yq -i '.spec.replicas = 3' deployment.yaml

# Set a nested value (creates intermediate nodes)
yq -i '.a.b.c = "new"' file.yaml

# Update using the current value
yq -i '.metadata.name = .metadata.name + "-staging"' file.yaml

# Delete a field
yq -i 'del(.metadata.annotations)' file.yaml

# Add an element to an array
yq -i '.spec.containers[0].ports += [{"containerPort": 8080}]' deployment.yaml

# Update in place with environment variable
NAME="myapp" yq -i '.metadata.name = strenv(NAME)' deployment.yaml

Filtering and selecting

# Select array items by field value
yq '.users[] | select(.role == "admin")' users.yaml

# Select with regex
yq '.[] | select(.name | test("^prod-"))' services.yaml

# Select with wildcards
yq '.[] | select(.host == "*.example.com")' ingresses.yaml

# Filter and update matching items
yq '(.users[] | select(.role == "admin") | .active) = true' users.yaml

# Filter map keys
yq 'with_entries(select(.key | test("^app_")))' config.yaml

# Filter, flatten, sort, unique
yq '[.[] | select(.type == "foo") | .names] | flatten | sort | unique' file.yaml

Merging

# Merge two files (file2 values override file1)
yq -n 'load("file1.yaml") * load("file2.yaml")'

# Merge two files in place (file2 into file1)
yq ea -i 'select(fi == 0) * select(fi == 1)' file1.yaml file2.yaml

# Merge all given files
yq ea '. as $item ireduce ({}; . * $item)' file1.yaml file2.yaml file3.yaml

# Append arrays instead of replacing (use *+)
yq -n 'load("file1.yaml") *+ load("file2.yaml")'

# Merge only existing fields (use *?)
yq -n 'load("base.yaml") *? load("overrides.yaml")'

# Merge only new fields (use *n)
yq -n 'load("base.yaml") *n load("extras.yaml")'

Multi-document YAML

Kubernetes manifests and other files with --- separators contain multiple documents.

# Apply expression to each document independently (default eval behavior)
yq '.metadata.name' multi-doc.yaml

# Process all documents at once (eval-all)
yq ea 'select(.kind == "Deployment")' multi-doc.yaml

# Update all documents
yq -i '.metadata.namespace = "production"' multi-doc.yaml

# Merge all documents into one
yq ea '. as $item ireduce ({}; . * $item)' multi-doc.yaml

# Combine separate files into one multi-doc file
yq '.' file1.yaml file2.yaml file3.yaml > combined.yaml

Sorting

# Sort an array by a field
yq '.users |= sort_by(.name)' users.yaml

# Sort keys of a map (recursively)
yq -P 'sort_keys(..)' file.yaml

# Compare two YAML files (normalize then diff)
diff <(yq -P 'sort_keys(..)' file1.yaml) <(yq -P 'sort_keys(..)' file2.yaml)

Creating YAML from scratch

# Create a simple document
yq -n '.name = "myapp" | .version = "1.0"'

# Create nested structure
yq -n '.metadata.name = "app" | .metadata.labels.env = "prod"'

# Create an array
yq -n '.items = ["a", "b", "c"]'

# Build from env vars
APP="myapp" VER="2.0" yq -n '.app = strenv(APP) | .version = strenv(VER)'

Working with CSV/TSV

# Parse CSV (first row = headers)
yq -p csv data.csv

# Parse TSV
yq -p tsv data.tsv

# Modify CSV in place
yq -p csv -o csv '(.[] | select(.name == "Gary") | .score) = "100"' data.csv

# Custom headers when outputting CSV
yq -o csv '[["Name", "Cats"]] + [.[] | [.name, .numberOfCats]]' file.yaml

# Disable auto-parsing of values (keep everything as strings)
yq -p csv --csv-auto-parse=false data.csv

Working with XML

# Parse XML to YAML
yq -p xml file.xml

# Encode YAML as XML
yq -o xml file.yaml

# XML attributes use +@ prefix
# <item id="5"> becomes: item: { +@id: "5" }

# XML content uses +content
# <p>hello <b>world</b></p> becomes: p: { +content: hello, b: world }

# Auto-parse string values into typed values
yq -p xml '(.. | select(tag == "!!str")) |= from_yaml' file.xml

The with operator for complex updates

with scopes an expression to a path, keeping the surrounding document intact. It's much cleaner than deeply nested paths for multiple updates.

# Multiple updates to a nested path
yq -i '
  with(.spec.template.spec.containers[0];
    .image = "nginx:latest" |
    .resources.limits.memory = "256Mi"
  )
' deployment.yaml

# Update each array element
yq 'with(.myArray[]; .name = .name + " - " + .type)' file.yaml

Useful patterns

# Validate YAML (exit 0 if valid map or seq, 1 otherwise)
yq --exit-status 'tag == "!!map" or tag == "!!seq"' file.yaml > /dev/null

# Read YAML array into bash array
readarray items < <(yq '.items[]' file.yaml)

# Loop over entries as JSON objects (safe for complex values)
readarray entries < <(yq -o=j -I=0 '.entries[]' file.yaml)
for entry in "${entries[@]}"; do
    name=$(echo "$entry" | yq '.name')
done

# Update many files at once
find . -name "*.yaml" -exec yq -i '.metadata.namespace = "prod"' {} \;

# Handle special characters in values using strenv
VAL='has "quotes" & <symbols>' yq -i '.raw = strenv(VAL)' file.yaml

# Pretty print and normalize
yq -P file.yaml

Key flags reference

Common pitfalls

• Forgetting -i for in-place edits — without it, yq prints to stdout and the file is unchanged.
• Using eval-all when eval suffices — ea loads every document into memory at once. Only use it when your expression needs to reference across documents (like merging files). The default eval streams documents one at a time and uses less memory.
• Array merging with * — by default, * replaces arrays entirely rather than appending. Use *+ to concatenate, or *d to deep-merge by index.
• Expression vs filename ambiguity — if yq misinterprets your expression as a filename, use --expression to force it: yq --expression '.foo.bar' file.yaml.
• Quoting — use single quotes around expressions to prevent shell expansion of special characters like $, *, !.