Power Filters and File redirection

Two powerful commands that we learn in this section:
awk:helps you get specific columns of information, modify text, swap the order of column of information in a file
tee: enable you to save a copy of the data being transmitted through a pipeline

awk
Syntax: awk '{commands}'
flags:
-f file  specifies that the instruction should be read from the file rather than from command line
-Fc indicates that the program should consider the letter c as the separator between fields of

using print with awk
bash-3.00$ whobuild      pts/2        Oct 17 11:28    (10.222.46.189)
root       pts/10       Jul 12 18:45    (10.209.188.122)
scholm  pts/12       Oct 18 23:46    (10.203.54.223)
jmadh  pts/15       Oct 19 11:06    (10.203.54.225)

bash-3.00$ who | awk '{print}'
build      pts/2        Oct 17 11:28    (10.222.46.189)
root       pts/10       Jul 12 18:45    (10.209.188.122)
scholm  pts/12       Oct 18 23:46    (10.203.54.223)
jmadh  pts/15       Oct 19 11:06    (10.203.54.225)

bash-3.00$ who | awk '{print$1}'
build
root
scholm
jmadh

bash-3.00$ who | awk '{print$2}'pts/2
pts/10
pts/12
pts/15

bash-3.00$ who | awk '{print$3}'Oct
Jul
Oct
Oct
Here what you have seen is each column is considered as field and each field is represented by $1, $2, $3 etc...for field one, field two, field three respectively.

Much better customization:
bash-3.00$  who | awk '{ print "User " $1 " is on terminal line " $2 }'User build     is on terminal line pts/2
User root     is on terminal line pts/10
User scholm is on terminal line pts/12
User jmadh is on terminal line pts/15
Note: You need to be careful to place the strings to be printed in double quotes, else throw error as below, due to conflict with single quotes of the awk command.

bash-3.00$  who | awk '{ print 'User' $1 ' is on terminal line ' $2 }'awk: syntax error near line 1
awk: illegal statement near line 1

NF Variable:
NF indicates number of fields in a line.
When we use this variable without a $ sign it give us number of fields in each line

bash-3.00$ who | awk '{print NF}'6
6
6
6
When used with $ it displays the value at last field of each line
bash-3.00$ who | awk '{print $NF}'(10.222.46.189)
(10.209.188.122)
(10.203.54.223)
(10.203.54.225)
similarly NR stands for number of records.

Sum up the file size with the awk command:
bash-3.00$ ls -ltotal 80
-rw-r--r--   1 fwire    fwire         12 Oct 18 22:40 MyWords.txt
-rw-r--r--   1 fwire    fwire       2958 Oct 19 00:02 def.txt
-rw-r--r--   1 fwire    fwire       2976 Oct 19 00:01 def_new.txt
-rwxr-xr-x   1 fwire    fwire        141 Oct 18 23:40 search
-rw-r--r--   1 fwire    fwire       2958 Oct 18 21:57 what is unix.txt
file size is the fifthe field
bash-3.00$ ls -l | awk '{print $5}'
12
2958
2976
141
2958
Now we need to sum up all these size

bash-3.00$ ls -l | awk '{total=total+$5;print total}'0
12
2970
5946
6087
9045
To get the summed up vlaue alone use command below
bash-3.00$ ls -l | awk '{total+=$5;print total}'| tail -19045

Much better way is using END command
bash-3.00$  ls -l | awk '{total+=$5} END {print total}'9045
It will be better if we can put this code into some file so that our command line will be less lengthier
% cat << EOF > script
{ totalsize += $4 } END { print “You have a total of “totalsize .”}
EOF
The above command will create a file called script in your current directory with belwo content
{ totalsize += $4 } END { print “You have a total of “totalsize .”}

bash-3.00$ ls -l | awk -f scriptYou have a size of 9150
In above command -f tells awk command to look into the file script

 We can use awk commnad for conditional execution of statement as well, as shown in the example below
% awk -F: ‘{ if (length($1) == 2) print $0 }’ /etc/passwd | wc -l

Here the passwd file is given as input for the awk command, where it says the field seperator is ':' and if the length of first field, ic account is two character lenght, then print the entire lines from the passwd file and this output  is then supplied to wc where we added -l flag, which gives us the total number of lines.

Say we recived an output of 5, this indicates that there are totally 5 accounts which are two character length. 
tee  command has only one option, -a, which appends the stream data/ output to a specified file.
bash-3.00$ who | tee who.outfwire      pts/10       Jul 12 18:45    (10.209.188.122)
fwire      pts/12       Oct 18 23:46    (10.203.54.223)
fwire      pts/15       Oct 19 14:33    (10.203.54.224)

bash-3.00$ cat who.outfwire      pts/10       Jul 12 18:45    (10.209.188.122)
fwire      pts/12       Oct 18 23:46    (10.203.54.223)
fwire      pts/15       Oct 19 14:33    (10.203.54.224)

Here who | tee who.out placed the output of who to who.out file, which can also be achieved by the command who > who.out. Hence, what is the difference between these two.

In the first type of command the primary thing is we are displaying who are all there in the system and in the background the contents are saved into a specified file, here who.out (you can see the output in the screen for first type of command whereas you cant see for the second type). In the second type of command, the action is writing the output to  the specified file but not been displayed in the screen.


 




tee command:
information, rather than the default of white space