Here "sed" stands for "stream edit"
The main use of sed is to replace the string provided
bash-3.00$ cat def_new.txt | sed -e 's/LINUX/UNIX/g'
In the above example the word LINUX is replaced by UNIX in the file def_new.txt and is displayed.
s stands for substitute.
What if you need replace multiple strings?
bash-3.00$ cat def_new.txt | sed -e 's/LINUX/UNIX/g;s/system/COMPUTER'
sed command can also be used to delete
bash-3.00$ whobuild pts/2 Oct 17 11:28 (10.222.46.189)
rmc pts/10 Jul 12 18:45 (10.209.188.122)
root pts/12 Oct 18 23:46 (10.203.54.223)
bash-3.00$ who | sed 'd'
wont display anything as it deletes everything
bash-3.00$ who | sed '1d'
delete the first line
rmc pts/10 Jul 12 18:45 (10.209.188.122)
root pts/12 Oct 18 23:46 (10.203.54.223)
bash-3.00$ who | sed '1,2d'
delete the first two lines
root pts/12 Oct 18 23:46 (10.203.54.223)
bash-3.00$ who | sed '/build/d'delete only the user build
rmc pts/10 Jul 12 18:45 (10.209.188.122)
root pts/12 Oct 18 23:46 (10.203.54.223)
bash-3.00$ who | sed '/rmc/,/build/d'
delete both rmc and build
root pts/12 Oct 18 23:46 (10.203.54.223)
bash-3.00$ who | sed '1,/rmc/d'root pts/12 Oct 18 23:46 (10.203.54.223)
delete from line 1 till the line has rmc
Tuesday, October 18, 2011
UNIX "grep" command Examples
What is grep command used for?
Ans: grep command can be considered as a "search" command for UNIX/LINUX systems. Used for searching text strings and some regular expressions.
Examples:
1) Search for a text string in a file
above command search for all the occurence of the text 'shiyas' in the file passwd and print the lines having those text on the screen.
2) Search for the string in multiple files
grep 'shiyas' *
above command search for all the occurence of the text 'shiyas' in all the files in current dierectory and print the lines having those text on the screen along with the filenames. Here the wildcard character * looks for all the files in the current directory.
grep 'shiyas' *.txt
this command will search for the pattern in all txt files.
grep 'SHIYAS' /etc/passwd
wont display anything as it doesnt have any string SHIYAS, and UNIX is very sensitive for this case
inorder to avoid this case sensitive issue you can use below command which will help you to get the output you need irrespective of case
grep -i 'SHIYAS' /etc/passwd
4) what if I need to display all the lines which doesnt have the string called 'shiyas'
grep -v 'shiyas' /etc/passwd
Note: It is not necessary to place the string in single quotes, but mandatory when a space is included.
5) find all the sub-directories in the current directory
ls -al | grep '^d'
6) search for multiple patterns at one time (egrep)
egrep 'and|loop|cursor' example.txt
Note: egrep stand for "extended grep"
7) suppose you want to search for the strings "Foo" or "Goo" in all files in the current directory. That grep command would be:
grep '[FG]oo' *
8) Search for the string 'bond' but not 'jamesbond'
grep '^bond' /etc/password
9) Display the files that has the search string?
grep -l 'bond' /etc/password
10) Display the line number as well along withe lines that has the search string
grep -n 'bond' /etc/password
11) Display the lines before/after your search pattern
grep -B 4 'bond' /etc/password
this command displays 4 lines before the search pattern
grep -A 4 'bond' /etc/password
this command diplays 4 lines after the search pattern
12) List down all files which has pattern 'bond' in all the subdirectories in the current directory tree
find . -type f -exec grep -il 'foo' {} \;
This command will list down all the files which has the given search pattern, no matter at what level of directory it is
13) What is the grep command that returns specific number of rows for the search pattern provided.
For ex :- if your file has 10 row with the given string and you want to display only 5 out of 10 rows use below command
grep text 'string' | head -5
What if i want to search for multiple patterns?
Use egrep. egrep stands for "extended grep".
How it works?
What if you have to search for a pattern either this"" or this "", you have rely on "egrep" command which has more powerful notational scheme than "grep" command.
Application of egrep is explained as below.
cat passwd | head-10
root:x:0:0:Super-User:/:/sbin/sh
daemon:x:1:1::/:
bin:x:2:2::/usr/bin:
sys:x:3:3::/:
adm:x:4:4:Admin:/var/adm:
lp:x:71:8:Line Printer Admin:/usr/spool/lp:
uucp:x:5:5:uucp Admin:/usr/lib/uucp:
nuucp:x:9:9:uucp Admin:/var/spool/uucppublic:/usr/lib/uucp/uucico
smmsp:x:25:25:SendMail Message Submission Program:/:
listen:x:37:4:Network Admin:/usr/net/nls:
When I am looking for one or more occurence of string 'c' in /etc/passwd file, using grep the command is as below
grep 'cc*' /etc/passwd
uucp:x:5:5:uucp Admin:/usr/lib/uucp:
nuucp:x:9:9:uucp Admin:/var/spool/uucppublic:/usr/lib/uucp/uucico
what if i am entering the command as
grep 'c*' /etc/passwd
this will list down all those lines which has zero or more occurence of the string 'c' which is equal to listing down the entire content of the file.
root:x:0:0:Super-User:/:/sbin/sh
daemon:x:1:1::/:
bin:x:2:2::/usr/bin:
sys:x:3:3::/:
adm:x:4:4:Admin:/var/adm:
lp:x:71:8:Line Printer Admin:/usr/spool/lp:
uucp:x:5:5:uucp Admin:/usr/lib/uucp:
nuucp:x:9:9:uucp Admin:/var/spool/uucppublic:/usr/lib/uucp/uucico
smmsp:x:25:25:SendMail Message Submission Program:/:
listen:x:37:4:Network Admin:/usr/net/nls:
whereas while using egrep it is as simple as,
egrep 'u+' /etc/passwd
root:x:0:0:Super-User:/:/sbin/sh
bin:x:2:2::/usr/bin:
lp:x:71:8:Line Printer Admin:/usr/spool/lp:
uucp:x:5:5:uucp Admin:/usr/lib/uucp:
nuucp:x:9:9:uucp Admin:/var/spool/uucppublic:/usr/lib/uucp/uucico
smmsp:x:25:25:SendMail Message Submission Program:/:
listen:x:37:4:Network Admin:/usr/net/nls:
then what if I am giving the command as
egrep 'cc+' /etc/passwd
this will list down all those lines which has two or more occurence of the charater 'c'
uucp:x:5:5:uucp Admin:/usr/lib/uucp:
nuucp:x:9:9:uucp Admin:/var/spool/uucppublic:/usr/lib/uucp/uucico
Now let us look into the question asked above, how to search for multiple patterns.
It is as simple as this
bash-3.00$ cat passwd |head -10 | egrep 'uu|mm'uucp:x:5:5:uucp Admin:/usr/lib/uucp:
nuucp:x:9:9:uucp Admin:/var/spool/uucppublic:/usr/lib/uucp/uucico
smmsp:x:25:25:SendMail Message Submission Program:/:
I want to list down those lines which starts with a
bash-3.00$ cat passwd |head -10 | egrep '^a' passwdadm:x:4:4:Admin:/var/adm:
apache:x:104:103:Apache User:/export/apache:/bin/bash
What if I want to list down those lines which starts with a,b,c,d
bash-3.00$ cat passwd |head -10 | egrep '^[a-d]' passwddaemon:x:1:1::/:
bin:x:2:2::/usr/bin:
adm:x:4:4:Admin:/var/adm:
build:x:102:103:Build User:/export/build:/bin/bash
apache:x:104:103:Apache User:/export/apache:/bin/bash
csvn:x:204:204:CollabNet Subversion User:/opt/CollabNet_Subversion:/bin/sh
What is fgrep and how it is used?
fgrep stands for 'file based grep'. For example I have a file of search strings say MyWords.txt and what I need is to list down the lines in 'appreport.txt' file having these search strings.
For this let me go the directory
bash-3.00$ cd /tmp/shiyas/skills/unix/
bash-3.00$ lswhat is unix.txt
bash-3.00$ cat what\ is\ unix.txt | head -5X is a computer operating system, a control program that works with users to run
programs, manage resources, and communicate with other computer systems. Several people
can use a UNIX computer at the same time; hence UNIX is called a multiuser system. Any
of these users can also run multiple programs at the same time; hence UNIX is called
multitasking. Because UNIX is such a pastiche.a patchwork of development.it.s a lot
let me create one file Mywords.txt having words manage & such
bash-3.00$ vi MyWords.txtmanage
such
~
~
~
:wq!
"MyWords.txt" [New file] 2 lines, 12 characters
bash-3.00$ cat MyWords.txtmanage
such
bash-3.00$ fgrep -f MyWords.txt what\ is\ unix.txtprograms, manage resources, and communicate with other computer systems. Several people
multitasking. Because UNIX is such a pastiche.a patchwork of development.it.s a lot
used in high-speed networking, file revision management, and software development.
Why is having all this choice such a big deal? Think about why Microsoft MS-DOS and the
Is there any alernative for typing down such a long command?
Ofcourse you have, and it is using 'alias'
the command for alias is
bash-3.00$ alias search='fgrep -i -f MyWords.txt'
you have to be very careful with the syntax else will through error.
the space after '=' will give error as below
bash-3.00$ alias search= 'fgrep -i -f MyWords.txt'
bash: alias: fgrep -i -f MyWords.txt: not found
Now lets see how we can use this alias
bash-3.00$ search what\ is\ unix.txtprograms, manage resources, and communicate with other computer systems. Several people
multitasking. Because UNIX is such a pastiche.a patchwork of development.it.s a lot
used in high-speed networking, file revision management, and software development.
Why is having all this choice such a big deal? Think about why Microsoft MS-DOS and the
bash-3.00$
Now I am removing the alias function
bash-3.00$ unalias search
bash-3.00$ search what\ is\ unix.txtbash: search: command not foundbash-3.00$
I need to display only the words that match instead of the entire line, what should I do?
To achieve this we have to use the 'awk' command , for which below is sample
bash-3.00$ echo 'My name is shiyas' | awk '{for (i=1;i<=NF;i++) print $i}'
My
name
is
shiyas
bash-3.00$
NF stands for number of fields (here it is 4)
Now lets work on displaying only the matching word alone, not the entire line
Logic we are going to implement is:
Make the content of file a list of one word each as above and now search for the pattern
step1: awk '{for (i=1;i<=NF;i++) print $i}'
step2: fgrep -i -f MyWords.txt what\ is\ unix.txtFor this lets code one shell script to incorporate both above commands.
bash-3.00$ cat search
#Wrongwords - show a list of commonly misused words in the file
cat $* | \
awk .{for (i=1;i<=NF;i++) print $i}. |\
fgrep -i -f MyWords.txt
bash-3.00$ unalias search
Give execute permission for the above shell script
bash-3.00$ chmod +x search
****
Hope this is helpful. Thanks Phoenix
Ans: grep command can be considered as a "search" command for UNIX/LINUX systems. Used for searching text strings and some regular expressions.
Examples:
1) Search for a text string in a file
grep 'shiyas' /etc/passwd
above command search for all the occurence of the text 'shiyas' in the file passwd and print the lines having those text on the screen.
2) Search for the string in multiple files
grep 'shiyas' *
above command search for all the occurence of the text 'shiyas' in all the files in current dierectory and print the lines having those text on the screen along with the filenames. Here the wildcard character * looks for all the files in the current directory.
grep 'shiyas' *.txt
this command will search for the pattern in all txt files.
grep 'SHIYAS' /etc/passwd
wont display anything as it doesnt have any string SHIYAS, and UNIX is very sensitive for this case
inorder to avoid this case sensitive issue you can use below command which will help you to get the output you need irrespective of case
grep -i 'SHIYAS' /etc/passwd
4) what if I need to display all the lines which doesnt have the string called 'shiyas'
grep -v 'shiyas' /etc/passwd
Note: It is not necessary to place the string in single quotes, but mandatory when a space is included.
5) find all the sub-directories in the current directory
ls -al | grep '^d'
6) search for multiple patterns at one time (egrep)
egrep 'and|loop|cursor' example.txt
Note: egrep stand for "extended grep"
7) suppose you want to search for the strings "Foo" or "Goo" in all files in the current directory. That grep command would be:
grep '[FG]oo' *
8) Search for the string 'bond' but not 'jamesbond'
grep '^bond' /etc/password
9) Display the files that has the search string?
grep -l 'bond' /etc/password
10) Display the line number as well along withe lines that has the search string
grep -n 'bond' /etc/password
11) Display the lines before/after your search pattern
grep -B 4 'bond' /etc/password
this command displays 4 lines before the search pattern
grep -A 4 'bond' /etc/password
this command diplays 4 lines after the search pattern
12) List down all files which has pattern 'bond' in all the subdirectories in the current directory tree
find . -type f -exec grep -il 'foo' {} \;
This command will list down all the files which has the given search pattern, no matter at what level of directory it is
13) What is the grep command that returns specific number of rows for the search pattern provided.
For ex :- if your file has 10 row with the given string and you want to display only 5 out of 10 rows use below command
grep text 'string' | head -5
What if i want to search for multiple patterns?
Use egrep. egrep stands for "extended grep".
How it works?
What if you have to search for a pattern either this"" or this "", you have rely on "egrep" command which has more powerful notational scheme than "grep" command.
| Notation | Meaning |
| c | Matches the character c |
| \c | Forces c to be read as the letter c, not as another meaning the character might have |
| ^ | Beginning of the line |
| $ | End of the line |
| . | Any single character |
| [xy] | Any single character in the set specified |
| [^xy] | Any single character not in the set specified |
| c* | Zero or more occurrences of character c |
| c+ | One or more occurrences of character c |
| c? | Zero or one occurrences of character c |
| a|b | Either a or b |
| (a) | Regular expression |
Application of egrep is explained as below.
cat passwd | head-10
root:x:0:0:Super-User:/:/sbin/sh
daemon:x:1:1::/:
bin:x:2:2::/usr/bin:
sys:x:3:3::/:
adm:x:4:4:Admin:/var/adm:
lp:x:71:8:Line Printer Admin:/usr/spool/lp:
uucp:x:5:5:uucp Admin:/usr/lib/uucp:
nuucp:x:9:9:uucp Admin:/var/spool/uucppublic:/usr/lib/uucp/uucico
smmsp:x:25:25:SendMail Message Submission Program:/:
listen:x:37:4:Network Admin:/usr/net/nls:
When I am looking for one or more occurence of string 'c' in /etc/passwd file, using grep the command is as below
grep 'cc*' /etc/passwd
uucp:x:5:5:uucp Admin:/usr/lib/uucp:
nuucp:x:9:9:uucp Admin:/var/spool/uucppublic:/usr/lib/uucp/uucico
what if i am entering the command as
grep 'c*' /etc/passwd
this will list down all those lines which has zero or more occurence of the string 'c' which is equal to listing down the entire content of the file.
root:x:0:0:Super-User:/:/sbin/sh
daemon:x:1:1::/:
bin:x:2:2::/usr/bin:
sys:x:3:3::/:
adm:x:4:4:Admin:/var/adm:
lp:x:71:8:Line Printer Admin:/usr/spool/lp:
uucp:x:5:5:uucp Admin:/usr/lib/uucp:
nuucp:x:9:9:uucp Admin:/var/spool/uucppublic:/usr/lib/uucp/uucico
smmsp:x:25:25:SendMail Message Submission Program:/:
listen:x:37:4:Network Admin:/usr/net/nls:
whereas while using egrep it is as simple as,
egrep 'u+' /etc/passwd
root:x:0:0:Super-User:/:/sbin/sh
bin:x:2:2::/usr/bin:
lp:x:71:8:Line Printer Admin:/usr/spool/lp:
uucp:x:5:5:uucp Admin:/usr/lib/uucp:
nuucp:x:9:9:uucp Admin:/var/spool/uucppublic:/usr/lib/uucp/uucico
smmsp:x:25:25:SendMail Message Submission Program:/:
listen:x:37:4:Network Admin:/usr/net/nls:
then what if I am giving the command as
egrep 'cc+' /etc/passwd
this will list down all those lines which has two or more occurence of the charater 'c'
uucp:x:5:5:uucp Admin:/usr/lib/uucp:
nuucp:x:9:9:uucp Admin:/var/spool/uucppublic:/usr/lib/uucp/uucico
Now let us look into the question asked above, how to search for multiple patterns.
It is as simple as this
bash-3.00$ cat passwd |head -10 | egrep 'uu|mm'uucp:x:5:5:uucp Admin:/usr/lib/uucp:
nuucp:x:9:9:uucp Admin:/var/spool/uucppublic:/usr/lib/uucp/uucico
smmsp:x:25:25:SendMail Message Submission Program:/:
I want to list down those lines which starts with a
bash-3.00$ cat passwd |head -10 | egrep '^a' passwdadm:x:4:4:Admin:/var/adm:
apache:x:104:103:Apache User:/export/apache:/bin/bash
What if I want to list down those lines which starts with a,b,c,d
bash-3.00$ cat passwd |head -10 | egrep '^[a-d]' passwddaemon:x:1:1::/:
bin:x:2:2::/usr/bin:
adm:x:4:4:Admin:/var/adm:
build:x:102:103:Build User:/export/build:/bin/bash
apache:x:104:103:Apache User:/export/apache:/bin/bash
csvn:x:204:204:CollabNet Subversion User:/opt/CollabNet_Subversion:/bin/sh
What is fgrep and how it is used?
fgrep stands for 'file based grep'. For example I have a file of search strings say MyWords.txt and what I need is to list down the lines in 'appreport.txt' file having these search strings.
For this let me go the directory
bash-3.00$ cd /tmp/shiyas/skills/unix/
bash-3.00$ lswhat is unix.txt
bash-3.00$ cat what\ is\ unix.txt | head -5X is a computer operating system, a control program that works with users to run
programs, manage resources, and communicate with other computer systems. Several people
can use a UNIX computer at the same time; hence UNIX is called a multiuser system. Any
of these users can also run multiple programs at the same time; hence UNIX is called
multitasking. Because UNIX is such a pastiche.a patchwork of development.it.s a lot
let me create one file Mywords.txt having words manage & such
bash-3.00$ vi MyWords.txtmanage
such
~
~
~
:wq!
"MyWords.txt" [New file] 2 lines, 12 characters
bash-3.00$ cat MyWords.txtmanage
such
bash-3.00$ fgrep -f MyWords.txt what\ is\ unix.txtprograms, manage resources, and communicate with other computer systems. Several people
multitasking. Because UNIX is such a pastiche.a patchwork of development.it.s a lot
used in high-speed networking, file revision management, and software development.
Why is having all this choice such a big deal? Think about why Microsoft MS-DOS and the
Is there any alernative for typing down such a long command?
Ofcourse you have, and it is using 'alias'
the command for alias is
bash-3.00$ alias search='fgrep -i -f MyWords.txt'
you have to be very careful with the syntax else will through error.
the space after '=' will give error as below
bash-3.00$ alias search= 'fgrep -i -f MyWords.txt'
bash: alias: fgrep -i -f MyWords.txt: not found
Now lets see how we can use this alias
bash-3.00$ search what\ is\ unix.txtprograms, manage resources, and communicate with other computer systems. Several people
multitasking. Because UNIX is such a pastiche.a patchwork of development.it.s a lot
used in high-speed networking, file revision management, and software development.
Why is having all this choice such a big deal? Think about why Microsoft MS-DOS and the
bash-3.00$
Now I am removing the alias function
bash-3.00$ unalias search
bash-3.00$ search what\ is\ unix.txtbash: search: command not foundbash-3.00$
I need to display only the words that match instead of the entire line, what should I do?
To achieve this we have to use the 'awk' command , for which below is sample
bash-3.00$ echo 'My name is shiyas' | awk '{for (i=1;i<=NF;i++) print $i}'
My
name
is
shiyas
bash-3.00$
NF stands for number of fields (here it is 4)
Now lets work on displaying only the matching word alone, not the entire line
Logic we are going to implement is:
Make the content of file a list of one word each as above and now search for the pattern
step1: awk '{for (i=1;i<=NF;i++) print $i}'
step2: fgrep -i -f MyWords.txt what\ is\ unix.txtFor this lets code one shell script to incorporate both above commands.
bash-3.00$ cat search
#Wrongwords - show a list of commonly misused words in the file
cat $* | \
awk .{for (i=1;i<=NF;i++) print $i}. |\
fgrep -i -f MyWords.txt
bash-3.00$ unalias search
Give execute permission for the above shell script
bash-3.00$ chmod +x search
****
Hope this is helpful. Thanks Phoenix
Saturday, October 15, 2011
Unix Commands
> To sort filenames alphabetically regardless of case
% ls -1 | sort -f
ls -1 will list the files in one single column, this output is passed to sorting, where sort -f makes sure that the list is sorted not considering the case.
> Sorting lines of a file?
sort < shiyas.txt
> find the largest file in your directory
bash-3.00$ ls -s | sort -nr
ls -s list the files and folder along with size
pass this to sort numerically and in reverse order.
the first line gives you the largest file along with its size in blocks.
Here if we need to see only the highest 5 files add head -5
bash-3.00$ ls -s | sort -nr | head -5
I have a file in which there is space after each line. what should be the unix command to remove these spaces?
I i am using "uniq" command for this purpose, it wont work, as uniq command will consider the duplication of lines only if the duplicants are adjacent. What should i do in this case?
sort text.txt | uniq > new.txt
here the sorting will sort your file having space between each line, in a manner that the blank line will come first. Supply this output to the uniq command and now the blank lines are adjacent, hence uniq command take one among them and place this new content to a new file.
Problem solved !!
I want to display the contents of my file along with the line number. How could I achieve that?
% cat -n text.txt
here the flag n will do the job of giving line numbers
We have one alternative for this
% nl text.txt
"nl" will do the same job of "cat -n"
> test.txt has 10 lines of which 5 are blank lines. When nl test.txt is fired, it is expected that the output is a numbered lines. What will be the last number? 5 or 10
ans: 5. nl by default only numbers the lines that are not blank.
So, what should I do to number the blank lines as well?
% nl -ba test.txt
will number all the lines.
Is there any alternativ for nl command?
% nl -bt test.txt
will number only printable text
I want to number all those lines which has specific pattern.
% nl -bpORA -s: shiyas_new.txt
here "bpORA" looks for the word ORA in each line
"-s:" is used for seperator which seperates number and line like 1:
search for the line in a file for a given pattern:
%grep STATUS ../bin/fatwire.sh
here STATUS is the word you are looking for in the fatwire.sh
Output:
echo "$STATUS"
export STATUS
List down all the files under the directory which has a particular pattern in it
grep 'INGESTION' * */*
* expands your search beyond the files in the current dirctory
and */* expands your search to all files contained one directory below the current point.
I have a filename called 'wha is unix.txt' and I need to display the content of this file in screen for which I am using the command
cat 'what is unix.txt',
will this work and is there any other alternative for this?
Ofcourse this will work and you have one alternative as well. The alternative is:
cat what\ is\ unix.txt
Here what we have done is, we have used the escape character which tells the unix system to interpret the space as space itself.
% ls -1 | sort -f
ls -1 will list the files in one single column, this output is passed to sorting, where sort -f makes sure that the list is sorted not considering the case.
> Sorting lines of a file?
sort < shiyas.txt
> find the largest file in your directory
bash-3.00$ ls -s | sort -nr
ls -s list the files and folder along with size
pass this to sort numerically and in reverse order.
the first line gives you the largest file along with its size in blocks.
Here if we need to see only the highest 5 files add head -5
bash-3.00$ ls -s | sort -nr | head -5
I have a file in which there is space after each line. what should be the unix command to remove these spaces?
I i am using "uniq" command for this purpose, it wont work, as uniq command will consider the duplication of lines only if the duplicants are adjacent. What should i do in this case?
sort text.txt | uniq > new.txt
here the sorting will sort your file having space between each line, in a manner that the blank line will come first. Supply this output to the uniq command and now the blank lines are adjacent, hence uniq command take one among them and place this new content to a new file.
Problem solved !!
I want to display the contents of my file along with the line number. How could I achieve that?
% cat -n text.txt
here the flag n will do the job of giving line numbers
We have one alternative for this
% nl text.txt
"nl" will do the same job of "cat -n"
> test.txt has 10 lines of which 5 are blank lines. When nl test.txt is fired, it is expected that the output is a numbered lines. What will be the last number? 5 or 10
ans: 5. nl by default only numbers the lines that are not blank.
So, what should I do to number the blank lines as well?
% nl -ba test.txt
will number all the lines.
Is there any alternativ for nl command?
% nl -bt test.txt
will number only printable text
I want to number all those lines which has specific pattern.
% nl -bpORA -s: shiyas_new.txt
here "bpORA" looks for the word ORA in each line
"-s:" is used for seperator which seperates number and line like 1:
search for the line in a file for a given pattern:
%grep STATUS ../bin/fatwire.sh
here STATUS is the word you are looking for in the fatwire.sh
Output:
echo "$STATUS"
export STATUS
List down all the files under the directory which has a particular pattern in it
grep 'INGESTION' * */*
* expands your search beyond the files in the current dirctory
and */* expands your search to all files contained one directory below the current point.
I have a filename called 'wha is unix.txt' and I need to display the content of this file in screen for which I am using the command
cat 'what is unix.txt',
will this work and is there any other alternative for this?
Ofcourse this will work and you have one alternative as well. The alternative is:
cat what\ is\ unix.txt
Here what we have done is, we have used the escape character which tells the unix system to interpret the space as space itself.
Tuesday, September 20, 2011
UNIX Command: Looking into files.
As it is important to know how to navigate in UNIX to find files it is important to know how to navigate within files. There are obviously few commands in UNIX that helps you to navigate within the files or view the required contend of your files without much difficulty.
Below are the few commands which are helpful to navigate through files in UNIX.
file - to identify file types
head- peak at the first few lines
tail - view last few lines
cat - view content of file
more - view larger files.
file:
A program that can easily offer you a good hint as to the contents of a file by looking at the first few lines. The disadvantage of this command is, it is not 100% accurate
eg:- A text file has executable permission and the initial contents of the files look like a C program the file command interpret it as an executable program rather that an English text file.
% file newtext.txt helloworld.txt 'shiyas resume.doc'
newtext.txt: ascii text
....
...
photos: directory
% file *
will analyze all files in home directory
head: Use it to view up to the first few hundred lines of a very long file, actually. You can specify the number as well.
% head newfile.txt
first few lines are displayed
% head -4 newfile.txt
first 4 lines are displayed
Can supply multiple files as well
% head -4 newfile.txt passwd
=====>passwd<====
......................................
=====>newfile.txt<===
.......................................
Can be used along with pipes (|):
% who | head -5
the output of who is supplied to head -5 and lists down first 5 users in the list.
tail:
Provide last lines of the file.
%
cat:
flags: -v -shead -12 .cshrc | tail -3
Below are the few commands which are helpful to navigate through files in UNIX.
file - to identify file types
head- peak at the first few lines
tail - view last few lines
cat - view content of file
more - view larger files.
file:
A program that can easily offer you a good hint as to the contents of a file by looking at the first few lines. The disadvantage of this command is, it is not 100% accurate
eg:- A text file has executable permission and the initial contents of the files look like a C program the file command interpret it as an executable program rather that an English text file.
% file newtext.txt helloworld.txt 'shiyas resume.doc'
newtext.txt: ascii text
....
...
photos: directory
% file *
will analyze all files in home directory
head: Use it to view up to the first few hundred lines of a very long file, actually. You can specify the number as well.
% head newfile.txt
first few lines are displayed
% head -4 newfile.txt
first 4 lines are displayed
Can supply multiple files as well
% head -4 newfile.txt passwd
=====>passwd<====
......................................
=====>newfile.txt<===
.......................................
Can be used along with pipes (|):
% who | head -5
the output of who is supplied to head -5 and lists down first 5 users in the list.
tail:
Provide last lines of the file.
%
cat:
flags: -v -shead -12 .cshrc | tail -3
combine the two, head and tail, so you can see just the tenth, eleventh, and twelfth lines of a file?
Monday, September 19, 2011
UNIX questions
Q: What is the output of % echo $DATE?
Ans: Nothing is displayed.
Note: Here we are trying to display the value stored in the variable DATE, which not declared nor initialized, hence nothing is displayed
Q: What is the output of %date?
Ans: date with time
Q: In my directory /u/home/phoenix I have one file name magazine.txt and a directory magazine. What will be the output for ls -l magaz* ? Will it list both the file and directory or only one among them? If one, which one?
Ans: ls -l magaz* will list the magazine.txt file only.
If you need to see the directory then you should run ls -ld magaz*
Q: Does the command mkdir has arguments?
Ans: No
Q: What if you create a directory with same name as one that exist already?
Ans: Will throw error saying file already exist
Q: umask is set to 0222, decode the default permission for this value.
Ans:
The value for umask is just opposite of what the real privilege is set, hence
0 indicates owner has all access except execute : rw-
2 indicates group has all privilege except write and execute: r--
2:r--
Hence finally we have: -rw-r--r--
Q: What is command use for moving a file?
Ans: mv
Q: What is the command used for renaming a file?
Ans: mv
Q: I have a directory in which there are many directories and files and some are as below:
directories:DIR1, DIR2,DIR3
files:DIR1.txt, DIR2.txt
there are no files in DIR1 and DIR2, whereas DIR3 has two files hello.txt and hi.txt
Now i ran
ls- l DI*, what will be the output?
ls -ld DI* what will be the output?
Q: What is the result, cp newdir latestdir, where newdir is a directory?
cp: omitting directory 'newdir'
Q: what is the resutl, cp newfile.txt
Ans: cp: missing destination file operand after newfile.txt
Try 'cp --help' for more information.
What is the result on executing umask 77777?
bash: umask: 777777: octal number out of range
I have changed the umask value from the default one (022) to 777 and then I exit from the terminal and logged in again. What will the value of umask.
ans:022.
Each time you login to the terminal umask value get reset always.
****
Hope this is helpful. Thanks Phoenix..
Ans: Nothing is displayed.
Note: Here we are trying to display the value stored in the variable DATE, which not declared nor initialized, hence nothing is displayed
Q: What is the output of %date?
Ans: date with time
Q: In my directory /u/home/phoenix I have one file name magazine.txt and a directory magazine. What will be the output for ls -l magaz* ? Will it list both the file and directory or only one among them? If one, which one?
Ans: ls -l magaz* will list the magazine.txt file only.
If you need to see the directory then you should run ls -ld magaz*
Q: Does the command mkdir has arguments?
Ans: No
Q: What if you create a directory with same name as one that exist already?
Ans: Will throw error saying file already exist
Q: umask is set to 0222, decode the default permission for this value.
Ans:
The value for umask is just opposite of what the real privilege is set, hence
0 indicates owner has all access except execute : rw-
2 indicates group has all privilege except write and execute: r--
2:r--
Hence finally we have: -rw-r--r--
Q: What is command use for moving a file?
Ans: mv
Q: What is the command used for renaming a file?
Ans: mv
Q: I have a directory in which there are many directories and files and some are as below:
directories:DIR1, DIR2,DIR3
files:DIR1.txt, DIR2.txt
there are no files in DIR1 and DIR2, whereas DIR3 has two files hello.txt and hi.txt
Now i ran
ls- l DI*, what will be the output?
ls -ld DI* what will be the output?
Q: What is the result, cp newdir latestdir, where newdir is a directory?
cp: omitting directory 'newdir'
Q: what is the resutl, cp newfile.txt
Ans: cp: missing destination file operand after newfile.txt
Try 'cp --help' for more information.
What is the result on executing umask 77777?
bash: umask: 777777: octal number out of range
I have changed the umask value from the default one (022) to 777 and then I exit from the terminal and logged in again. What will the value of umask.
ans:022.
Each time you login to the terminal umask value get reset always.
****
Hope this is helpful. Thanks Phoenix..
Why Shell Scripts always require both read and execute permission?
Any program works fine in UNIX even it has only execute permission. By giving execute privilege alone for any program we make sure that, the users of this particular program has the permission to run the program but not to examine or copy or modify the code that we have written.
Shell scripts belongs to a special class of programs, which fails to execute if only execute privilege is granted on those. Why is this becasue, shell script, which acts as a UNIX command line macro facility, which helps in clubbing togther a series of commands into a single file and run them as a single program. Hence for the shell to execute this program (or commands inside the file), it need to read the file first and then execute.
So shell script always need both read and execute permissions!!
Shell scripts belongs to a special class of programs, which fails to execute if only execute privilege is granted on those. Why is this becasue, shell script, which acts as a UNIX command line macro facility, which helps in clubbing togther a series of commands into a single file and run them as a single program. Hence for the shell to execute this program (or commands inside the file), it need to read the file first and then execute.
So shell script always need both read and execute permissions!!
UNIX Commands
ls : List files in a directory
ls -a : List hidden dot files as well in a directory
ls -C -F: List files with slash ('/') for current driectory
ls -C -F /: List files with slash ('/') for root directory
ls -l: List files in long format.
ls -lr: List files in long format and in reverse order based on file/directory name.
ls -lt: List files in long format and ordered based on file/directory modified date. The most recently accessed files.
ls -lrt: List files in long format and in reverse order based on file/directory modified date.
ls -s: List the filenames along with its size. The total size of all files in the directory is given in the first line. Size is given in Kilobites rounded upwards.
ls -s will list all the files under the current directory and its size as well. If you are intereseted to see size details of any specific directory you can give the command as below
ls -s <dirname>
we can specify the filename as well
ls -s <filename>
You can specify multiple filenames or dierctories as well and seperate them by space.
ls -s <dir1> <dir2> <file1>
Output:
10 file1
dir1:
total 456
2 file1
4 file2
etc...
dir2:
total 100
45 file1
50 file2
etc...
ls -f: Suffices the file type at the end of the filename
Different types of suffices
- / indicates directory
- * Indicates program
- @ indicates a symbolic link to another file or directory.
ls -m: List the contents of a directory comma seperated.
ls -1: List the content of directory in a single column.
ls -x: Change the default sorting order from column first then row to row first then column
ls -
pwd- stands for present working directory and will display the absolute path of your current directory.
bc: Used for infix mathematical calcualtions.
$bc
12*4
48
quit
$
Touch Command:
helps you create new files on the system
Compress:
compress textfile.txt
compress -v textfile.txt
Uncompress:
uncompres textfile.txt / uncompress textfile.txt.z
touch command on a file that doesn’t exist, the programSort output in row-first order.
ls -a : List hidden dot files as well in a directory
ls -C -F: List files with slash ('/') for current driectory
ls -C -F /: List files with slash ('/') for root directory
ls -l: List files in long format.
ls -lr: List files in long format and in reverse order based on file/directory name.
ls -lt: List files in long format and ordered based on file/directory modified date. The most recently accessed files.
ls -lrt: List files in long format and in reverse order based on file/directory modified date.
ls -s: List the filenames along with its size. The total size of all files in the directory is given in the first line. Size is given in Kilobites rounded upwards.
ls -s will list all the files under the current directory and its size as well. If you are intereseted to see size details of any specific directory you can give the command as below
ls -s <dirname>
we can specify the filename as well
ls -s <filename>
You can specify multiple filenames or dierctories as well and seperate them by space.
ls -s <dir1> <dir2> <file1>
Output:
10 file1
dir1:
total 456
2 file1
4 file2
etc...
dir2:
total 100
45 file1
50 file2
etc...
ls -f: Suffices the file type at the end of the filename
Different types of suffices
- / indicates directory
- * Indicates program
- @ indicates a symbolic link to another file or directory.
ls -m: List the contents of a directory comma seperated.
ls -1: List the content of directory in a single column.
-a
List all files, including any dot files. -F
Indicate file types; / = directory, * = executable. -m
Show files as a comma-separated list. -s
Show size of files, in blocks (typically, 1 block = 1,024 bytes). -C
Force multiple-column output on listings.-1 Force single-column output on listings.ls -x: Change the default sorting order from column first then row to row first then column
ls -
pwd- stands for present working directory and will display the absolute path of your current directory.
bc: Used for infix mathematical calcualtions.
$bc
12*4
48
quit
$
-1
Force single-column output on listings.-a
List all files, including any dot files.-C Force multiple-column output on listings-d
List directories rather than their contents.-F
Indicate file types; / = directory, * = executable.-l
Generate a long listing of files and directories.-m
Show files as a comma-separated list.-r
Reverse the order of any file sorting.-R
Recursively show directories and their contents.-s
Show size of files, in blocks (typically 1 block = 1,024 bytes).-t
Sort output in most-recently-modified order.-x Touch Command:
helps you create new files on the system
The main reason that
updated, as the following example demonstrates.
touch is used in UNIX is to force the last-modified time of a file to be%
ls -l iecc.list-rw------- 1 taylor 3843 Oct 6 18:02 iecc.list
%
touch iecc.list%
ls -l iecc.list-rw------- 1 taylor 3843 Oct 10 16:22 iecc.listIf you try to use the
creates the file:Compress:
compress textfile.txt
compress -v textfile.txt
Uncompress:
uncompres textfile.txt / uncompress textfile.txt.z
touch command on a file that doesn’t exist, the programSort output in row-first order.
Flag Meaning
mv
% ls -1
Payroll
newfile.txt
photos
newphotos
% mv newfile.txt latestfile.txt
% ls -1
Payroll
latestfile.txt
photos
newphotos
All the contents of the newfile.txt is moved into the latestfile.txt and latestfile.txt is removed from directory. The same is applicable in the case of directories also. We can entirely move the content of a directory into a new one. If you look closer you can find that this is nothing but just renaming the folder or file. Then what is actual moving of file or dirctory?
% mv newphotos ./Photos
% ls -1
Payroll
latestfile.txt
helloworld.txt
Photos
% ls -1 ./Photos
newphotos
Here the entire newphotos directory is moved into Photos directory. Similarly we can move the files as well from directory to directory.
What if I execute the below query?
% mv latestfile.txt helloworld.txt
ans: The contents of latestfile.txt is overwritten into helloworld.txt. A popup will come before overriding for which if give 'y' as input itwill override else wont.
rmdir:
rmdir is used to remove any directory. One advantage with rmdir is that, it removes only empty data, hence no need to worry on deleting the content of a directory unknowigly.
On executing rmdir against a directory which is not empty we get a message as below.
rmdir: failed to remove 'dir': Directory not empty
rm:
Used to remove files. Cant be used for directory (will throw error). This command removes any file. This command removes the file permanantly, cant be restored at any point of time. You can remove multiple files at a time.
For the safe side always use -i flag along with rm command, where i stands for interactive, which will ask us before removing the file
rm -r is a dangerous one as it will delete files recursively
How could you check, what type of login shell you're running?
% grep pnaraya /etc/passwd
pnaraya:x:1000:1000:pnaraya,,,:/u/pnaraya:/bin/bash
on the password entry the login shell is specified. (/bin/bash)
mv
% ls -1
Payroll
newfile.txt
photos
newphotos
% mv newfile.txt latestfile.txt
% ls -1
Payroll
latestfile.txt
photos
newphotos
All the contents of the newfile.txt is moved into the latestfile.txt and latestfile.txt is removed from directory. The same is applicable in the case of directories also. We can entirely move the content of a directory into a new one. If you look closer you can find that this is nothing but just renaming the folder or file. Then what is actual moving of file or dirctory?
% mv newphotos ./Photos
% ls -1
Payroll
latestfile.txt
helloworld.txt
Photos
% ls -1 ./Photos
newphotos
Here the entire newphotos directory is moved into Photos directory. Similarly we can move the files as well from directory to directory.
What if I execute the below query?
% mv latestfile.txt helloworld.txt
ans: The contents of latestfile.txt is overwritten into helloworld.txt. A popup will come before overriding for which if give 'y' as input itwill override else wont.
rmdir:
rmdir is used to remove any directory. One advantage with rmdir is that, it removes only empty data, hence no need to worry on deleting the content of a directory unknowigly.
On executing rmdir against a directory which is not empty we get a message as below.
rmdir: failed to remove 'dir': Directory not empty
rm:
Used to remove files. Cant be used for directory (will throw error). This command removes any file. This command removes the file permanantly, cant be restored at any point of time. You can remove multiple files at a time.
For the safe side always use -i flag along with rm command, where i stands for interactive, which will ask us before removing the file
rm -r is a dangerous one as it will delete files recursively
How could you check, what type of login shell you're running?
% grep pnaraya /etc/passwd
pnaraya:x:1000:1000:pnaraya,,,:/u/pnaraya:/bin/bash
on the password entry the login shell is specified. (/bin/bash)
Subscribe to:
Posts (Atom)