HowTo Handle Mass Output - The awk Command

This is part of a tutorial series. The main thread begins here:

Hi guys,

Sorry for going silent past couple months! My daughter underwent surgery and she's fine now. So I'm back writing tutorials on my blog.

The awk command allows us to extract information by separation per column or row.

Let's say you have a file named "" with content like below

Hostname      Old IP          New IP
========      =============   ============

You want to look into this list and display only the hostnames plus the New IP Addresses.

$ cat | awk '{ print $1, $3 }'
Hostname        New IP
========        ============

The awk command by default uses white spaces as delimiter. Here in this example, $1 and $3 represent first and third columns.  The columns are separated by white spaces, so they got printed correctly.

Specifying Delimiter

In other cases where the delimiter is not a white space, for example the /etc/passwd file. The delimiter there is the colon:


If you want to print the username and the userID only, they would be the 1st and 3rd column, but here the syntax needs to be a bit different.

$ cat /etc/passwd | awk -f : '{ print $1, $3 }'
root 0
bin 1
daemon 2
adm 3
lp 4
sync 5
shutdown 6
halt 7
mail 8
news 9
uucp 10

This syntax allows us to parse data files and extract wanted information regardless of what delimiter being used in the file.

Last Element

When you need to print the last element but don't know how many elements there are in the data, you can use $NF.

Using /etc/passwd again to see how $NF works

$ cat /etc/passwd | awk -f : '{ print $NF }'

More Options

awk has lots and lots of options that allow us to do much more with text strings. We stop here to use only the above basic options allowing us enough ability to extract data out of a large amount of output and present them in a much more human-readable format.

Back to the IP Change example text above; we would like to write a script that prepares a network address change for the 3 machines in the list, here's a skeleton of the script

Here's the configuration file
Hostname      Old IP          New IP
========      =============   ============

We will drill into this list, get the hostname, old and new address, then download the current config file from the actual server, modify the address and upload it back to the server.


HOSTS=`cat | awk '{ print $1 }' | grep -v Hostname | grep -v '===='`

for HOST in $HOSTS; do
   scp username@$HOST:/etc/sysconfig/network-scripts/ifcfg-eth0 ./backup/$HOST.ifcfg-eth0
   cat ./backup/$HOST_ifcfg-eth0 | sed 's/$OLD_IP/$NEW_IP/g' > ./output/$HOST.ifcfg-eth0
   scp ./output/$HOST.ifcfg-eth0 username@$HOST:/etc/sysconfig/network-scripts/ifcfg-eth0

In the code example above, we use grep to extract rows related to certain hostnames that we need to act on and used awk to extract columns to get specific information related to a host that has already been processed by grep.

You noticed that there was also a sed command? That's next

Articles in the Series

Part 1: The grep Command
Part 2: The awk Command 
Part 3: The sed Command
Part 4: Miscellaneous Commands

1 comment:

  1. Our entire choice of slots is equipped with leading edge technology for seamless action. It originated in 토토사이트 Ancient China and is played in most Asian countries. Translated into English, Pai Gow means "to make 9." It is played with 32 dominoes, additionally called tiles, eleven of which are similar pairs.


Help a friend, share your knowledge