Skip to content

The awk command

The awk command works with text in columns…

The awk command works with text in columns: awk '{print $1}' prints the first column of every line, and awk can filter, count and add up as it goes.

Taught in Advanced, lesson 36; more in Ninja

A practice shell in your browser that behaves like bash on Linux for these commands. Nothing you type leaves the page.

Usage

awk [-F SEP] [-v VAR=VALUE] 'PROGRAM' [FILE]...

Scan text for patterns and work with its columns. A small language for text in columns. awk splits each line into fields: $1 is the first, $2 the second, $NF the last and $0 the whole line, separated by spaces unless you give -F. A program is a list of pattern { action } pairs: the action runs on every line that matches the pattern, such as /error/, $3 > 50 or NR > 1. With no pattern it runs on every line; with no action it prints the line. BEGIN { } runs before the first line and END { } after the last, which is where totals go. Useful names: NR is the line number, NF the number of fields, and print, printf, length(), substr(), toupper(), split() and gsub() work as in other languages. Put the program in single quotes.

How to use awk

Fields are split on spaces: $1 is the first, $NF the last and $0 the whole line.

awk '{print $1}' app.log
08:00
08:05
08:06
08:07
08:10
08:15
awk '{print $NF}' app.log
started
found
retrying
timeout
ready
request

Use another separator: awk -F

awk -F, '{print $1}' data.csv
name
Ava
Leo
Mia
awk -F: '{print $1, $7}' /etc/passwd
root /bin/bash
www-data /usr/sbin/nologin
learner /bin/bash

Filter rows

A pattern before the braces picks the lines: a /regex/ or a comparison. NR > 1 skips the header.

awk -F, 'NR > 1 && $2 >= 80' data.csv
Ava,82
Leo,91
awk '/ERROR|WARN/' app.log
08:05 ERROR disk not found
08:15 WARN slow request

Add up a column

END runs after the last line, which is where totals go.

awk -F, 'NR > 1 {sum += $2} END {print "total:", sum, "average:", sum / (NR - 1)}' data.csv
total: 250 average: 83.3333

Count things

An array keyed by a field counts how often each value appears.

awk '{count[$2]++} END {for (level in count) print level, count[level]}' app.log
INFO 3
ERROR 1
error: 1
WARN 1

Format the output: printf

awk -F, 'NR > 1 {printf "%-5s %5.1f%%\n", $1, $2}' data.csv
Ava    82.0%
Leo    91.0%
Mia    77.0%

Options

-F SEPfields are separated by SEP, such as -F, for CSV or -F: for /etc/passwd
-v VAR=VALUEset a variable before the program starts
$1, $2, $NFthe first, second and last field
NR, NFthe line number and the number of fields on the line
BEGIN { }, END { }run before the first line and after the last
printf "%-10s %5.1f\n", $1, $2print in columns (like printf in C)

More examples

awk -F, 'NR > 1 && $2 > 80 {print $1}' data.csv
Names with a score over 80
Ava
Leo
awk -F, 'NR > 1 {sum += $2} END {print sum / (NR - 1)}' data.csv
The average score
83.3333
awk '{count[$1]++} END {for (f in count) print f, count[f]}' fruit.txt
Count each word
pear 3
apple 2
fig 1
awk 'length > 20' notes.txt
Lines longer than 20 characters
Note 2: renew the library books
Note 3: back up the laptop
Note 4: book the dentist
Note 5: pay the electricity bill
Note 7: water the plants
Note 9: renew the library books
Note 10: back up the laptop
Note 11: book the dentist
Note 12: pay the electricity bill
Note 14: water the plants

Questions about awk

What is the difference between awk and cut?

cut picks columns by one delimiter character. awk splits on any run of spaces by default, can reorder columns, filter rows, do sums and format output.

What do NR and NF mean in awk?

NR is the number of the current line (record), counting from 1; NF is the number of fields on it, so $NF is the last field.

Is awk a programming language?

Yes, a small one, with variables, arrays, if, loops and functions. Most awk programs are one line, but it can do much more.

Learn it step by step

Every everyday command on one page, to print or keep open: the Linux commands cheat sheet.