The awk command
The awk command works with text in columns…
The awk command works with text in columns: awk '{print $1}' prints the first column of every line, and awk can filter, count and add up as it goes.
Taught in Advanced, lesson 36; more in Ninja
A practice shell in your browser that behaves like bash on Linux for these commands. Nothing you type leaves the page.
Usage
awk [-F SEP] [-v VAR=VALUE] 'PROGRAM' [FILE]...
Scan text for patterns and work with its columns. A small language for text in columns. awk splits each line into fields: $1 is the first, $2 the second, $NF the last and $0 the whole line, separated by spaces unless you give -F. A program is a list of pattern { action } pairs: the action runs on every line that matches the pattern, such as /error/, $3 > 50 or NR > 1. With no pattern it runs on every line; with no action it prints the line. BEGIN { } runs before the first line and END { } after the last, which is where totals go. Useful names: NR is the line number, NF the number of fields, and print, printf, length(), substr(), toupper(), split() and gsub() work as in other languages. Put the program in single quotes.
How to use awk
Print a column
Fields are split on spaces: $1 is the first, $NF the last and $0 the whole line.
awk '{print $1}' app.log08:00 08:05 08:06 08:07 08:10 08:15
awk '{print $NF}' app.logstarted found retrying timeout ready request
Use another separator: awk -F
awk -F, '{print $1}' data.csvname Ava Leo Mia
awk -F: '{print $1, $7}' /etc/passwdroot /bin/bash www-data /usr/sbin/nologin learner /bin/bash
Filter rows
A pattern before the braces picks the lines: a /regex/ or a comparison. NR > 1 skips the header.
awk -F, 'NR > 1 && $2 >= 80' data.csvAva,82 Leo,91
awk '/ERROR|WARN/' app.log08:05 ERROR disk not found 08:15 WARN slow request
Add up a column
END runs after the last line, which is where totals go.
awk -F, 'NR > 1 {sum += $2} END {print "total:", sum, "average:", sum / (NR - 1)}' data.csvtotal: 250 average: 83.3333
Count things
An array keyed by a field counts how often each value appears.
awk '{count[$2]++} END {for (level in count) print level, count[level]}' app.logINFO 3 ERROR 1 error: 1 WARN 1
Format the output: printf
awk -F, 'NR > 1 {printf "%-5s %5.1f%%\n", $1, $2}' data.csvAva 82.0% Leo 91.0% Mia 77.0%
Options
| -F SEP | fields are separated by SEP, such as -F, for CSV or -F: for /etc/passwd |
|---|---|
| -v VAR=VALUE | set a variable before the program starts |
| $1, $2, $NF | the first, second and last field |
| NR, NF | the line number and the number of fields on the line |
| BEGIN { }, END { } | run before the first line and after the last |
| printf "%-10s %5.1f\n", $1, $2 | print in columns (like printf in C) |
More examples
awk -F, 'NR > 1 && $2 > 80 {print $1}' data.csvAva Leo
awk -F, 'NR > 1 {sum += $2} END {print sum / (NR - 1)}' data.csv83.3333
awk '{count[$1]++} END {for (f in count) print f, count[f]}' fruit.txtpear 3 apple 2 fig 1
awk 'length > 20' notes.txtNote 2: renew the library books Note 3: back up the laptop Note 4: book the dentist Note 5: pay the electricity bill Note 7: water the plants Note 9: renew the library books Note 10: back up the laptop Note 11: book the dentist Note 12: pay the electricity bill Note 14: water the plants
Questions about awk
What is the difference between awk and cut?
cut picks columns by one delimiter character. awk splits on any run of spaces by default, can reorder columns, filter rows, do sums and format output.
What do NR and NF mean in awk?
NR is the number of the current line (record), counting from 1; NF is the number of fields on it, so $NF is the last field.
Is awk a programming language?
Yes, a small one, with variables, arrays, if, loops and functions. Most awk programs are one line, but it can do much more.
Learn it step by step
Related commands
Every everyday command on one page, to print or keep open: the Linux commands cheat sheet.