awk

31/07/2006
+91.9911064207 12:58:46 PM
awk -- 11
awk
Copyright/Acknowledgements

This document has been authored by Atanu Mandal

(atanu@aclabs.net) and contains original material.
Some of the slides may contain material sourced from various
other sources under the fair use principle. Wherever possible,
the author has attributed these source(s). Any such credit
omission may please be excused as human error. The author
would be thankful if such omissions are brought to his notice
and he will correct them in later versions of this document.
Any one may reproduce completely or in part, any portion(s)
or all of this document for spreading knowledge. The only
requirement is that this page may not be removed or modified in
the reproduction(s) and the author (Atanu Mandal) is given due
credit for his hard work.
Refer to the end portions of this document for more details

12:58:47PM
12:58:47
12:58:47 PM
PM
info@ackabs.net +91.9911064207 11/04/2014
Module
Module -- 22
Introduction

Awk - "Al Aho, Peter Weinberger, & Brian Kernighan"

Awk is a utility for performing text-processing tasks
It can be used from the command line
Or complex awk programs can be written as awk scripts.
Awk has two purposes:
- it is a utility for performing simple text-processing tasks,
- it is a programming language for performing complex
text-processing tasks.

31/07/2006
+91.9911064207 12:58:47 PM
awk -- 33
awk
History

1977: The original version of awk was written in at AT&T Bell Laboratories
1985: A new version introduced user-defined functions, multiple input
streams and computed expressions.
Generally became available with Unix System V Release 3.1
The awk version in System V Release 4.0 added some new features and also
cleaned up the language.
The specification for awk in the POSIX Command Language and Utilities
standard further clarified the language based on feedback from both the
gawk designers and the original Bell Labs awk designers
1986: The GNU gawk was written by Paul Rubin and Jay Fenlason with
advice from Richard Stallman. John Woods contributed parts of the code.
1988 and 1989: David Trueman and Arnold Robbins thoroughly reworked
gawk for compatibility with the newer awk.

31/07/2006
+91.9911064207 12:58:47 PM
awk -- 44
awk
Introduction

Brian Kernighan said about AWK:

It was originally for writing these one and two line programs. It
really was. I think it's very seductive because it does so many
things automatically. It handles strings and numbers smoothly. It
is an interpreter and there's no baggage, no derived object files.
People start to write a one and two line program that just grows
and grows; some of them grow unbelievably large: tens of
thousands of lines -- which is nonsense.
Brian Kernighan, cited by Peter H. Salus (who in turn cites
Peter Collinson from the ".EXE" magazine). From
A Quarter Century of UNIX, pp. 103-104.

31/07/2006
+91.9911064207 12:58:47 PM
awk -- 55
awk
Versions
oawk (old AWK)
The first AWK version. It is still around on many UNIX systems. If
you have an oawk on your system, you probably have nawk, too.
Only required if you have older AWK scripts that require the older
AWK version.
nawk (new AWK)
Extension of OAWK. This is now the standard AWK version. Any
references made here to this version apply to the GNU AWK gawk
as well.
gawk (GNU AWK)
Similar to NAWK but entirely re-written from scratch and open
source.
awk
Generic or default name which is usually linked with nawk or
gawk.
awk is also available for different OS like DOS, Windows, etc.

31/07/2006
+91.9911064207 12:58:48 PM
awk -- 66
awk
Command Line
Name Phone Address City State Country
==============================================================
Vinamra 98212344 16,Sec 24 Noida UP India
Atanu 31087031 20B,Pkt C NewDelhi Delhi India
Abhi 29374563 RatHole NewDelhi Delhi India
Britney 32325723 HerHome City State USA

awk '/Atanu/' myfile.txt

Print out lines in the file (myfile.txt) containing the string

"atanu"
awk '/Atanu/ {print $3,$5,$6}' myfile.txt
Print out 3rd, 5th & 6th columns of the lines in the file (myfile.txt)
containing the string "atanu"

31/07/2006
+91.9911064207 12:58:48 PM
awk -- 77
awk
Command Line

$ awk '{ print }' /etc/passwd

equivalent to
$ awk '{ print $0 }' /etc/passwd

$0 refers to the entire line

31/07/2006
+91.9911064207 12:58:48 PM
awk -- 88
awk
Command Line
Suppose /etc/passwd contains:
root:x:0:0:root:/root:/bin/bash
nfsnobody:x:65534:65534:AnonNFS
User:/var/lib/nfs:/sbin/nologin
atanu:x:0:0::/home/atanu:/bin/bash
rat:x:500:501::/home/rat:/bin/bash
test:x:501:502::/home/test:/bin/bash

$ awk '{ print "" }' /etc/passwd

$ awk '{ print "hello" }' /etc/passwd

Try
Try the
the above
above two
two awk
awk commands
commands and
and describe
describe what
what you
you observe.
observe.

31/07/2006
+91.9911064207 12:58:48 PM
awk -- 99
awk
Command Line

$ awk '{ print "" }' /etc/passwd | wc -l

$ awk '{ print "hello" }' /etc/passwd | wc -l

Now
Now try
try the
the above
above modified
modified commands
commands and
and describe
describe what
what you
you
observe.
observe.

31/07/2006
+91.9911064207 12:58:48 PM
awk -- 10
awk 10
 $ awk -F":" '{ print $1 }' /etc/passwd

Explain
Explain What
What isis happening
happening above
above

31/07/2006
+91.9911064207 12:58:48 PM
awk -- 11
awk 11
Field Separator

Field Separator

$ awk -F":" '{ print $1 }' /etc/passwd

Print the user names since $1 refers to the first field

Discuss
Discuss tabular
tabular data
data storage
storage

31/07/2006
+91.9911064207 12:58:48 PM
awk -- 12
awk 12
 Command Line

awk -F":" '{ print $1 $3 }' /etc/passwd

awk -F":" '{ print $1 " " $3 }' /etc/passwd

awk -F":" '{ print "username: " $1 "\t\tuid:" $3" }' /etc/passwd

Try
Try the
the above
above variations
variations and
and explain
explain what
what isis happening
happening

31/07/2006
+91.9911064207 12:58:48 PM
awk -- 13
awk 13
Command Line
Suppose /etc/passwd contains:
root:x:0:0:root:/root:/bin/bash
nfsnobody:x:65534:65534:AnonNFS
User:/var/lib/nfs:/sbin/nologin
atanu:x:0:0::/home/atanu:/bin/bash
rat:x:500:501::/home/rat:/bin/bash
test:x:501:502::/home/test:/bin/bash

awk '/atanu/ { print $1 $3 }' /etc/passwd

awk -F":" '/atanu/ { print $1 $3 }' /etc/passwd

awk -F":" '/atanu/ { print $1 "-" $3 }' /etc/passwd

Try
Try the
the above
above three
three variations
variations and
and explain
explain what
what isis happening
happening

31/07/2006
+91.9911064207 12:58:48 PM
awk -- 14
awk 14
 Patterns
ADV

Several patterns may be put together, separated by logical operators-

"!" (logical negation),

"&&" (logical and),

"||" (logical or).

Typical combined pattern may look like:

/Smith / || /Jones /

31/07/2006
+91.9911064207 12:58:48 PM
awk -- 15
awk 15
 Built In variables
ADV

NF number of words on this line

NR number of record, ie. line number
FILENAME name of the input file
FS input field separator (space or tab character)
RS input record (line) separator (newline)

/Smith / || /Jones / { print NR, $0 }

31/07/2006
+91.9911064207 12:58:48 PM
awk -- 16
awk 16
Comparsions
== equal to
!= not equal to
> greater than
< less than
>= greater than or equal to
<= less than or equal to

awk '$1 == 45' filename

awk -F':' '{if ($3 > 500) print}' /etc/passwd

awk -F':' '{if ($3 > 500) print $3}' /etc/passwd

31/07/2006
+91.9911064207 12:58:48 PM
awk -- 17
awk 17
References

https://www.gnu.org/software/gawk/manual/gawk.html

12:58:47PM
12:58:47
12:58:47 PM
PM
info@ackabs.net +91.9911064207 11/04/2014
Module
Module -- 18
18
Copyright/Acknowledgements

This document has been authored by Atanu Mandal

(atanu@aclabs.net) and contains original material.
Some of the slides may contain material sourced from various
other sources under the fair use principle. Wherever possible,
the author has attributed these source(s). Any such credit
omission may please be excused as human error. The author
would be thankful if such omissions are brought to his notice
and he will correct them in later versions of this document.
Any one may reproduce completely or in part, any portion(s)
or all of this document for spreading knowledge. The only
requirement is that this page may not be removed or modified in
the reproduction(s) and the author (Atanu Mandal) is given due
credit for his hard work.

12:58:47PM
12:58:47
12:58:47 PM
PM
info@ackabs.net +91.9911064207 11/04/2014
Module
Module -- 19
19
Acknowledgements

Linux/Unix man pages and their authors

12:58:47PM
12:58:47
12:58:47 PM
PM
info@ackabs.net +91.9911064207 11/04/2014
Module
Module -- 20
20
Versions

No record of prior versions

31/07/2006
11/04/2014
05/May.2015

12:58:47PM
12:58:47
12:58:47 PM
PM
info@ackabs.net +91.9911064207 11/04/2014
Module
Module -- 21
21
That's all, folks!!
...till we meet again...

31/07/2006
+91.9911064207 12:58:48 PM
awk -- 22
awk 22