UNIX-1

 

 

1 : How to ignore some words while doing search using grep in UNIX

grep Exception logfile.txt | grep -v ERROR

This grep command example will search for word “Exception” in logfile.txt and print them but since we have piped out of first grep command to second grep command which will exclude all lines which match world “ERROR”.

$ cat example.txt
UNIX operating system
UNIX and Linux operating system
Linux operation system

Now we want to search all lines in file example.txt which contains word UNIX but same time doesn’t contain world Linux.

$ grep UNIX example.txt
UNIX operating system
UNIX and Linux operating system

Now to exclude all lines which contains Linux we will apply another grep command in this output with option -v to exclude matching word as shown in below grep command :

$ grep UNIX example.txt | grep -v Linux
UNIX operating system

 2 : How to count occurrence of a word in a file using grep command

Below example of command will print how many times word “Error” has appeared in logfile.txt.

$ grep -c "Error" logfile.txt

If we apply this grep command on our example file to find how many lines contains word e.g. UNIX has occurred in the file :

$ grep -c UNIX example.txt
2

 3 : printing lines before and after of matching word using grep

 Below example of grep command in UNIX will print 6 lines around matching line of word “successful” in logfile.txt

$ grep --context=6 successful logfile.txt

Show additional six lines after matching very useful to see what is around and to print whole message if it splits around multiple lines. You can also use command line option “C” instead of “–context” for example

$ grep -C 2 'hello' *

Prints two lines of context around each matching line.

 4. How to search pattern using egrep and regular expression

stands for extended grep and it is more powerful than grep command in Unix and allows more regular exception like you can use “|” option to search for either Error or Exception by executing just one command.

$ egrep 'Error|Exception' logfile.txt

 5 : How to do case insensitive searching using grep in Linux

If you want to do case insensitive search than use -i option from grep command in UNIX. grep -i command will find occurrence of both Error, error and ERROR and quite useful to display any sort of Error from log file.

$ grep -i Error logfile

 6 : How to search patterns in gzip files using zgrep command

zgrep is another great version of grep command in Unix which is used to perform same operation as grep does but with .gz files. Many a times we gzip the old file to reduce size and later wants to look or find something on those file. zgrep is your man for those days. Below command will print all files which have “Error” on them.

$ zgrep -i Error *.gz

 7 : How to search whole word in a file using grep command

You can use grep -w command in UNIX to find whole word instead of just pattern, as shown in following example. This example will only print lines from logfile.txt which contains full word ERROR.

$ grep -w ERROR logfile.txt

Above grep command in UNIX searches only for instances of ‘ERROR’ that are entire words; it does not match `SysERROR’.
For more control, use `\<‘ and `\>’ to match the start and end of words. For example:

$ grep 'ERROR>' *

Searches only for words ending in ‘ERROR’, so it matches the word `SysERROR’.

 8 : UNIX command to display files names which contains given word

Another useful grep command line option is “grep -l” which display only the file names which matches the given pattern. Below command will only display file names which have ERROR?

$ grep -l ERROR *.log

grep -l ‘main’ *.java will list the names of all Java files in the current directory whose contents mention `main’.

9:Write a command to print the line numbers along with the line that has the word “july”?

grep -n july filename
$ grep -n ERROR log file.

10 : How to do recursive search in a directory using grep in UNIX

If you want to do recursive search using grep command in Unix there are two options either use “-R” command line option or increase directory one by one as shown below.

$ grep -R store *

This command will search for directory or file with name store in current directory and it’s all sub-directory.

grep command in UNIX can show matching patter in color which is quite useful to highlight the matching section , to see matching pattern in color use below command.

$ grep Exception today.log --color

You can also create alias grep=’grep –color’ in your bash_profile file to avoid typing –color every time.

There are three version of grep command in UNIX `grep, fgrep, egrep’. `fgrep’ stands for Fixed `grep’, `egrep’ Extended `grep’

Read more

11. Write a command to print the lines that has the the pattern “july” in all the files in a particular directory?

grep july *

This will print all the lines in all files that contain the word “july” along with the file name. If any of the files contain words like “JULY” or “July”, the above command would not print those lines.

12. Write a command to print the lines that has the word “july” in all the files in a directory and also suppress the file name in the output.

grep -h july *

13. When you use a single file as input to the grep command to search for a pattern, it won’t print the filename in the output. Now write a grep command to print the file name in the output without using the ‘-H’ option.

grep pattern file name /dev/null

The /dev/null or null device is special file that discards the data written to it. So, the /dev/null is always an empty file.
Another way to print the file name is using the ‘-H’ option. The grep command for this is

grep -H pattern filename

14. Write a command to print the file names in a directory that does not contain the word “july”?

grep -L july *

The ‘-L’ option makes the grep command to print the file names that do not contain the specified pattern.

15. Write a command to print the lines that starts with the word “start”?

grep '^start' filename

The ‘^’ symbol specifies the grep command to search for the pattern at the start of the line.

16. Running the last executed grep command

This saves a lot of time if you are executing the same command again and again.

!grep

This displays the last executed grep command and also prints the result set of the command on the terminal.

17. Search for a string in a file

This is the basic usage of grep command. It searches for the given string in the specified file.

grep "Error" logfile.txt

This searches for the string “Error” in the log file and prints all the lines that has the word “Error”.

18. Searching for a string in multiple files.

grep "string" file1 file2
grep "string" file_pattern

This is also the basic usage of the grep command. You can manually specify the list of files you want to search or you can specify a file pattern (use regular expressions) to search for.

19. Specifying the search string as a regular expression pattern.

grep "^[0-9].*" file.txt

This will search for the lines which starts with a number.

20. Displaying the lines before the match.

Some times, if you are searching for an error in a log file; it is always good to know the lines around the error lines to know the cause of the error.

grep -B 2 "Error" file.txt

This will prints the matched lines along with the two lines before the matched lines.

21. Displaying the lines after the match.

grep -A 3 "Error" file.txt

This will display the matched lines along with the three lines after the matched lines.

 22. Inverting the pattern match

You can display the lines that are not matched with the specified search sting pattern using the -v option.

grep -v "string" file.txt

23. Displaying the non-empty lines

You can remove the blank lines using the grep command.

grep -v "^$" file.txt

24. Displaying only the matched pattern.

By default, grep displays the entire line which has the matched string. We can make the grep to display only the matched string by using the -o option.

grep -o "string" file.txt

25. Displaying the position of the matched string in the line

The -b option allows the grep command to display the character position of the matched string in a file.

grep -o -b "string" file.txt

26. Matching the lines that start with a string

The ^ regular expression pattern specifies the start of a line. This can be used in grep to match the lines which start with the given string or pattern.

grep "^start" file.txt

27. Matching the lines that end with a string

The $ regular expression pattern specifies the end of a line. This can be used in grep to match the lines which end with the given string or pattern.

grep "end$" file.txt

28.How do you rename the files in a directory with _new as suffix?

ls -lrt|grep '^-'| awk '{print "mv "$9" "$9".new"}' | sh

 

29. Write command to list all the links from a directory?

ls -lrt | grep "^l"

30. How do you list the hidden files in current directory?

ls -a | grep '^\.'

31.isplay all the files in current directory sorted by size?

ls -l | grep '^-' | awk '{print $5,$9}' |sort -n|awk '{print $2}'

 32.How to find Exception in log files available in current directory and how to find number of occurrence?

$ grep ‘Exception’ log1.txt | wc -l

33.Search for a sample string in particular files ?

grep .Debug. *.confHere grep uses the string .Debug. to search in all files with extension..conf. under current directory.

34. How to remove blank lines in a file ?

grep -v ‘^$’ filename > new_filename

35. How to display the processes that were run by your user name ?

ps -aef | grep <user_name>

36.Delete blank lines in a file ?

cat sample.txt | grep -v ‘^$’ > new_sample.txt

FIND

 

37. How to run the last executed find command?

!find

This will execute the last find command. It also displays the last find command executed along with the result on the terminal.

38. How to find for a file using name?

find -name “sum.java”
./bkp/sum.java
./sum.java

This will find all the files with name “sum.java” in the current directory and sub-directories.

39. How to find for files using name and ignoring case?

findiname “sum.java”
./SUM.java
./bkp/sum.java
./sum.java

This will find all the files with name “sum.java” while ignoring the case in the current directory and sub-directories.

40. How to find for a file in the current directory only?

find -maxdepth 1 -name “sum.java”
./sum.java

This will find for the file “sum.java” in the current directory only

41. How to find for files containing a specific word in its name?

findname “*java*”
./SUM.java
./bkp/sum.java
./sum.java
./multiply.java

It displayed all the files which have the word “java” in the filename

42. How to find for files in a specific directory?

find /etc -name “*java*”

This will look for the files in the /etc directory with “java” in the filename

43. How to find the files whose name are not “sum.java“?

findnot -name “sum.java”
.
./SUM.java
./bkp
./multiply.java

This is like inverting the match. It prints all the files except the given file “sum.java”.

44. How to limit the file searches to specific directories?

find -name “sum.java”
./tmp/sum.java
./bkp/var/tmp/files/sum.java
./bkp/var/tmp/sum.java
./bkp/var/sum.java
./bkp/sum.java
./sum.java

You can see here the find command displayed all the files with name “sum.java” in the current directory and sub-directories.

a. How to print the files in the current directory and one level down to the current directory?

find -maxdepth 2 -name “sum.java”
./tmp/sum.java
./bkp/sum.java
./sum.java

b. How to print the files in the current directory and two levels down to the current directory?

find -maxdepth 3 -name “sum.java”
./tmp/sum.java
./bkp/var/sum.java
./bkp/sum.java
./sum.java

c. How to print the files in the subdirectories between level 1 and 4?

find -mindepth 2 -maxdepth 5 -name “sum.java”
./tmp/sum.java
./bkp/var/tmp/files/sum.java
./bkp/var/tmp/sum.java
./bkp/var/sum.java
./bkp/sum.java

45. How to find the empty files in a directory?

find . –maxdepth 1 -empty
./empty_file

46. How to find the largest file in the current directory and sub directories

find . -type f -exec ls -s {} \; | sort -n -r | head -1

The find command “find . -type f -exec ls -s {} \;” will list all the files along with the size of the file. Then the sort command will sort the files based on the size. The head command will pick only the first line from the output of sort.

47. How to find the smallest file in the current directory and sub directories

find . -type f -exec ls -s {} \; | sort -n -r | tail -1

Another method using find is

find . -type f -exec ls -s {} \; | sort -n | head -1

48. How to find files based on the file type?

a. Finding socket files

find . -type s

b. Finding directories

find . -type d

c. Finding hidden directories

find -type d -name “.*”

d. Finding regular files

find . -type f

e. Finding hidden files

find . -type f -name “.*”

49. How to find files based on the size?

a. Finding files whose size is exactly 10M

find . -size 10M

b. Finding files larger than 10M size

find . -size +10M

c. Finding files smaller than 10M size

find . –size -10M

50. How to find the files which are modified after the modification of a give file.

findnewer “sum.java”

This will display all the files which are modified after the file “sum.java”

51. Display the files which are accessed after the modification of a give file.

findanewer “sum.java”

52. Display the files which are changed after the modification of a give file.

findcnewer “sum.java”

53. How to find the files based on the file permissions?

find . -perm 777

This will display the files which have read, write, and execute permissions. To know the permissions of files and directories use the command “ls -l”.

find –perm option is used to find files based upon permissions.

You can used find –perm 444 to get all files which allows read to owner, group and others.

javin@testenv1:~/java $ find . -perm 644
./.vimrc
./OnlineStockTrading.java

find out all the executable files, you can also modify it to find all the read only files or files having write permission etc by changing permissions e.g. to find all read only files in current directory :

find . –perm 555  Here “.” or period denotes current directory. You can replace it with any directory you want.

54. Write a command to list the files that are accessed 5 days ago in the current directory?
find -atime 5 -type f

55. Write a command to list the files whose status is changed 5 days ago in the current directory?
find -ctime 5 -type f

56. Write a command to list the files that were modified 5 days ago in the current directory?
find –mtime 5 -type f

57.How to find files which has been modified less than one day, minute or hour  in Unix:

find –mtime is used to search files based upon modification time.

find –atime which denote last accessed time of file

find –ctime denotes last changed time. + sign is used to search for greater than, – sign is used to search for less than and without sign is used for exact.

For example find –mtime -1 will search all files which has been modified

javin@testenv1 ~/java : find . -mtime 1  (find all the files modified exact 1 day)

javin@testenv1 ~/java : find . -mtime -1 (find all the files modified less than 1 day)
.
./StockTrading.java

javin@testenv1 ~/java : find . -mtime +1 (find all the files modified more than 1 day)
./.vimrc
./OnlineStockTrading.java
./StockTrading.java~

58. Find the files which are modified within 30 minutes.

find . -mmin -30

59. Find the files which are modified within 1 day.

find . -mtime -1

60. How to find the files which are modified 30 minutes back

find . -not -mmin -30

61. How to find the files which are modified 1 day back.

find . -not -mtime -1

62.Print the files which are accessed within 1 hour.

find . -amin -60

63. Print the files which are accessed within 1 day.

find . -atime -1

64. Display the files which are changed within 2 hours.

find . -cmin -120

65. Display the files which are changed within 2 days.

find . -ctime -2

66. How to find the files which are created between two files.

find . -cnewer f1 -and ! -cnewer f2

67. How to find the permissions of the files which contain the name “java“?

find -name “*java*”|xargs ls -l

Alternate method is

find -name “*java*” -exec ls -l {} \;

68. Find the files which have the name “java” in it and then display only the files which have “class” word in them?

find -name “*java*” -exec grep -H class {} \;

69. How to remove files which contain the name “java”.

find -name “*java*” -exec rm -r {} \;

70. Write a command to search for the file ‘test’ in the current directory?
find -name test -type f

71. Write a command to search for the file ‘temp’ in ‘/usr’ directory?
find /usr -name temp -type f

72. Write a command to search for the files in the current directory which are not owned by any user in the /etc/passwd file?
find . -nouser -type f

73. Write a command to search for the files in ‘/usr’ directory that start with ‘te’?
find /usr -name ‘te*’ -type f

74. Write a command to search for the files that start with ‘te’ in the current directory and then display the contents of the file?
find . –name ‘te*’ -type f -exec cat {} \;

75. Write a command to list the files in ‘/usr’ directory that start with ‘ch’ and then display the number of lines in each file?
find /usr -name ‘ch*’ -type f -exec wc -l {} \;

76.How to delete temporary files using find command in Unix?

In order to delete files you can use either –delete option of find command or use xargs in combination. Its better to create house keeping script for such task which can perform cleanup on periodic basis.

find . -name “*.tmp” -print | xargs rm –f

considering use of -print0 to avoid problems with white space in the path when piping to xargs (use with the xargs -0 option)

77.How to find all text file which contains word Exception using find command in Unix ?

find . –name “*.java” –print | xargs grep “MemoryCache,

this will search all java files starting from current directory for word “MemoryCache”.

we can also leave -print option in all cases because its default for UNIX find command You can further sort result of find command using Sort command in unix.

find . –name “*.txt” –print | xargs grep “Exception”

78. Write a command to search for zero byte size files in the current directory?
find –size 0 -type f

79. Finding files only in current directory not searching on sub directories:

–type option can be used to specifiy search for only file, link or directory and maxdepth specifies how deep find has to search.

find . -maxdepth 1 -type f -newer first_file

Another way of doing it is below:

find . -type f -cmin 15 -prune
Means type file, last modified 15 minutes ago, only look at the current directory. (No sub-directories)

80 .How to find files based on size in Unix and Linux

find . -size +1000c -exec ls -l {} \;

find . -size +10000c -size -50000c -print
This find example lists all files that are greater than 10,000 bytes, but less than 50,000 bytes:

81.How to find files some days older and above certain size

find command will find which are more than 10 days old and size greater than 50K.

find . -mtime +10 -size +50000c -exec ls -l {} \;

You can use awk in combination of find to print a formatted output e.g. next command will find all of the symbolic links in your home directory, and print the files your symbolic links points to:

find . –type l -print | xargs ls -ld | awk ‘{print $10}’

“.” says starts from current directory and include all sub directory and  “-type l” says list all links.
Tip: 
$* :    $* is one of the special bash parameter which is used to expands positional parameters from position one.

82.How to use find command on file names with space in

find -print0 and xargs -0 on find examples,

we don’t specify any expression after find command the default option is -print which prints the name of each found files followed by \n or newline.since we mostly pipe output of find command to xargs -print could cause problem if file name itself contain new line or any form of white space.

To resolve this issue instead of -print use -print0.

Difference between find -print and find -print0 is, print0 display file name on the stdout followed by a “NUL” character and then you can use xargs -0command to process file names with null character.

UNIX find command example with file name having space in them:

javin@testenv1:~/test find . -name “*equity*” -print
./cash equity trading ./equity~
You see here “cash equity trading” has space in there name

javin@testenv1:~/test find . -name “*equity*” -print | xargs ls -l
ls: cannot access ./cash: No such file or directory
ls: cannot access equity: No such file or directory
ls: cannot access trading: No such file or directory
-r–r–r– 1 stock_trading cash_domain trading 0 Jul 15 11:42 ./equity~
Now if we pass this to xargs, xargs treat them as three separate files.

javin@testenv1:~/test find . -name “*equity*” -print0 | xargs ls

xargs: WARNING: a NUL character occurred in the input.  It cannot be passed through in the argument list.  Did you mean to use the –null option?

ls: cannot access ./cash: No such file or directory
ls: cannot access equity: No such file or directory
ls: cannot access trading: No such file or directory
Now to solve this we have used find command with -print0 which appends NUL character on file name but without xargs -0, xargs will not able to handle those inputs.

javin@testenv1:~/test find . -name “*equity*” -print0 | xargs -0 ls -l
-rw-r–r– 1 stock_trading cash_domain trading 0 Jul 21 09:54 ./cash equity trading
-r–r–r– 1 stock_trading cash_domain trading 0 Jul 15 11:42 ./equity~

you can see with find -print0| xargs -0 it looks good

In conclusion always use find -print0 along with xargs -0 if you see slightest possibilities of file names containing space in UNIX or Linux.

 

  • find –print and find is same as –print is a default option of find command.
  • find –print0 should be used to avoid any issue with white space in file name or path while forwarding output to xargs, also use xargs -0 along with find –print0.
  • find has an option called –delete which can be used in place of  -exec rm {} \;

83.find all files in current and subdirectories which contains ‘log’ name?

$ find . -name ‘log’

84.Write a command to find all of the files which have been accessed within the last 30 days.

find / -type f -atime -30 > December.files

This command will find all the files under root, which is ‘/’, with file type is file. ‘-atime -30′ will give all the files accessed less than 30 days ago. And the output will put into a file call December.files.

. Write a command to search for the file ‘map’ in the current directory?

find -name map -type f

85. Write a command to list the files in ‘/usr’ directory that start with ‘ch’ and then display the number of lines in each file?

wc -l /usr/ch*

Another way is

find /usr -name 'ch*' -type f -exec wc -l {} \;

86. Write a command to display all the files recursively with path under current directory?

find . -depth -print

87. Display zero byte size files in the current directory?

find -size 0 -type f

CUT

Unix Cut Command Example

We will see the usage of cut command by considering the below text file as an example

> cat file.txt
unix or linux os
is unix good os
is linux good os

88.Write a unix/linux cut command to print characters by position?

The cut command can be used to print characters in a line by specifying the position of the characters. To print the characters in a line, use the -c option in cut command

cut -c4 file.txt
x
u
l

The above cut command prints the fourth character in each line of the file. You can print more than one character at a time by specifying the character positions in a comma separated list as shown in the below example

cut -c4,6 file.txt
xo
ui
ln

This command prints the fourth and sixth character in each line.

89.Write a unix/linux cut command to print characters by range?

You can print a range of characters in a line by specifying the start and end position of the characters.

cut -c4-7 file.txt
x or
unix
linu

The above cut command prints the characters from fourth position to the seventh position in each line. To print the first six characters in a line, omit the start position and specify only the end position.

cut -c-6 file.txt
unix o
is uni
is lin

To print the characters from tenth position to the end, specify only the start position and omit the end position.

cut -c10- file.txt
inux os
ood os
good os

If you omit the start and end positions, then the cut command prints the entire line.

cut -c- file.txt

90.Write a unix/linux cut command to print the fields using the delimiter?

You can use the cut command just as awk command to extract the fields in a file using a delimiter. The -d option in cut command can be used to specify the delimiter and -f option is used to specify the field position.

cut -d' ' -f2 file.txt
or
unix
linux

This command prints the second field in each line by treating the space as delimiter. You can print more than one field by specifying the position of the fields in a comma delimited list.

cut -d' ' -f2,3 file.txt
or linux
unix good
linux good

The above command prints the second and third field in each line.

Note: If the delimiter you specified is not exists in the line, then the cut command prints the entire line. To suppress these lines use the -s option in cut command.

91. Write a unix/linux cut command to display range of fields?

You can print a range of fields by specifying the start and end position.

cut -d' ' -f1-3 file.txt

The above command prints the first, second and third fields. To print the first three fields, you can ignore the start position and specify only the end position.

cut -d' ' -f-3 file.txt

To print the fields from second fields to last field, you can omit the last field position.

cut -d' ' -f2- file.txt

92. Write a unix/linux cut command to display the first field from /etc/passwd file?

The /etc/passwd is a delimited file and the delimiter is a colon (:). The cut command to display the first field in /etc/passwd file is

cut -d':' -f1 /etc/passwd

. The input file contains the below text

> cat filenames.txt
logfile.dat
sum.pl
add_int.sh

Using the cut command extract the portion after the dot.

First reverse the text in each line and then apply the command on it.

rev filenames.txt | cut -d'.' -f1

104. How do you make an existing file empty?

cat /dev/null >  filename

105. How to get the nth word of a line in Unix?

cut –f<n> -d' '

 

SORT

The options are:

-b : Ignores leading spaces in each line
-d : Uses dictionary sort order. Conisders only spaces and alphanumeric characters in sorting
-f : Uses case insensitive sorting.
-M : Sorts based on months. Considers only first 3 letters as month. Eg: JAN, FEB
-n : Uses numeric sorting
-R : Sorts the input file randomly.
-r : Reverse order sorting
-k : Sorts file based on the data in the specified field positions.
-u : Suppresses duplicate lines
-t : input field separator

Sort Command Examples:

Before practicing the examples create the below two files in your unix system:

> cat order.txt
Unix distributed 05 server
Linux virtual 3 server
Unix distributed 05 server
Distributed processing 6 system

> cat delim_sort.txt
Mayday|4
Janmon|1
Declast|12

106. Sorting lines of text

The default sort command uses alphabetical order (ASCII order) to sort the file. It treats each line as a string and then sorts the lines.

> sort order.txt
Distributed processing 6 system
Linux virtual 3 server
Unix distributed 05 server
Unix distributed 05 server

107. Sorting based on the field positions.

You can specify the field postions using the -k option of sort command. The sort command uses the space or tab as the default delimiter. To sort based on the data in the second field, run the below command:

> sort -k2 order.txt
Unix distributed 05 server
Unix distributed 05 server
Distributed processing 6 system
Linux virtual 3 server

You can also pecify more than field with k option as a comma separated list. The below command uses the second and fourth fields to sort the data.

> sort -k2,4 order.txt

108. Numeric sorting

Instead of the default alphabetical sorting order, you can make the sort command to sort in numeric order using the -n option. This is shown below:

> sort -nk3 order.txt
Linux virtual 3 server
Unix distributed 05 server
Unix distributed 05 server
Distributed processing 6 system

109. Sort in reverse order

By default, the sort command sorts the data in ascending order. You can change this to descending order using the -r option.

> sort -nrk3 order.txt
Distributed processing 6 system
Unix distributed 05 server
Unix distributed 05 server
Linux virtual 3 server

110. Suppressing duplicates or Print only unique values

You can produce only unique values in the output using the – u option of the sort command.

> sort -u order.txt
Distributed processing 6 system
Linux virtual 3 server
Unix distributed 05 server

Another way is piping the output of sort command to uniq command.

> sort order.txt | uniq

111. Delimited file input

In the second, third and fourth examples we have sorted the data based on the field positions. Here the fields are separted by space or tab character. What if the fields are specifed by any other character? In such cases, we have to specify the input delimiter with the -t option. An example is shown below:

> sort -t'|' -nrk2 delim_sort.txt
Declast|12
Mayday|4
Janmon|1

112. Sorting on months.

We can sort the data in the monthwise using the -M option of the sort command. This is shown below:

> sort -M delim_sort.txt
Janmon|1
Mayday|4
Declast|12

ps-ef

 

 

 

TAR

The options of tar command are:

c : creates a tar file.
v : verbose. Displays the files information.
f : Specify the tar file name.
r : updates the tar file with new files.
x : Extracts files from the archive (tar file).
t : view contents of tar file.
z : Specify the tar command to create a tar file using gzip in unix.
j : uses bzip2 to create the tar file.

Tar Command Examples:

113. Creating a tar file

Let see a sample example by archiving all the files in my current directory. The ls -l command displays the files and directories in the current directory.

> ls -l 
drwxr-xr-x 2 user group 4096 Aug  8 03:23 debian
-rw-r--r-- 1 user group  174 Aug  2 23:39 file
-rw-r--r-- 1 user group    0 Aug  8 03:22 linux_server.bat
-rw-r--r-- 1 user group   76 Aug  2 02:21 test.sh
-rw-r--r-- 1 user group    0 Aug  8 03:22 unix_distro

We see how to tar all these files using the -c option with the tar command. This is shown below:

> tar -cvf archive.tar *
debian/
file
linux_server.bat
test.sh
unix_distro

> ls
archive.tar  debian  file  linux_server.bat  test.sh  unix_distro

Observe the output of ls command and see the archive.tar file is created.

114. Printing the contents of tar file

We have created the tar file and we dont know whether it contains the actual files or not. To view the contents of the tar file use the -t option as

> tar -tvf archive.tar
drwxr-xr-x user/group   0 2012-08-08 03:23:07 debian/
-rw-r--r-- user/group 174 2012-08-02 23:39:51 file
-rw-r--r-- user/group   0 2012-08-08 03:22:19 linux_server.bat
-rw-r--r-- user/group  76 2012-08-02 02:21:32 test.sh
-rw-r--r-- user/group   0 2012-08-08 03:22:09 unix_distro

115. Updating the tar file with new contents.

You can add new files to the existing archive (tar) file using the -r option.

>touch red-hat-linux.dat

>tar -rvf archive.tar red-hat-linux.dat
red-hat-linux.dat

>tar -tvf archive.tar
drwxr-xr-x pcenter/pcenter   0 2012-08-08 03:23:07 debian/
-rw-r--r-- pcenter/pcenter 174 2012-08-02 23:39:51 file
-rw-r--r-- pcenter/pcenter   0 2012-08-08 03:22:19 linux_server.bat
-rw-r--r-- pcenter/pcenter  76 2012-08-02 02:21:32 test.sh
-rw-r--r-- pcenter/pcenter   0 2012-08-08 03:22:09 unix_distro
-rw-r--r-- pcenter/pcenter   0 2012-08-08 04:00:00 red-hat-linux.dat

Here the touch command creates a new file. The first tar command adds the new file to the existing archive file. The second command displays the contents of the tar file.

116. Extracting the contents of tar file

In the first example, we have created the archive file. Now we will see how to extract the set of files from the archive. To extract the contents of the tar file use the -x option.

> tar -xvf archive.tar
debian/
file
linux_server.bat
test.sh
unix_distro

117. Creating compressed tar file

So far we have created a uncompressed tar file in the above examples. We can create a compressed tar file using the gzip or bzip2.

Compressing files using gzip

> tar -zcvf new_tar_file.tar.gz *

Compressing files using bzip2

> tar -jcvf new_tar_file.tar.bz2 *

To extract or to view the files in a compressed tar file use the appropriate compression option (z or j).

To view files in a gzip compressed tar file
> tar -ztvf new_tar_file.tar.gz

To extract files from a gip compressed tar file
> tar -zxvf new_tar_file.tar.gz

To view files in a bzip2 compressed tar file
> tar -jtvf new_tar_file.tar.bz2

To extract files from a bzip2 compressed tar file
> tar -jxvf new_tar_file.tar.bz2

118. Creating tar file with specified list of files

You can specify a list of files to be included in the newly created tar file.

> tar -cvf unix_files.tar unix_server.bat unix_system.dat

Here the tar command creates the unix_files.tar file which contains only the files unix_server.bat and unix_system.dat

119. Extracting specific files from the tar

You can extract a specific file or a set of files from the archived file.

To extract a specifi file

> tar -xvf unix_files.tar unix_server.bat

To extract all files that start with name unix

> tar -xvf unix_files.tar --wildcards "unix*"

120. Extracting files from multiple archive files.

To extract the files from multiple archive files use the -M option with each -f option. This is shown below:

> tar -xv -Mf archive.tar -Mf unix_files.tar

you see creating tar file with gzip is very easy just use “-z” option and it will crate a gzip tar.tgz or tar.gz extension

121.Calculating size of tar file in UNIX

Some time its useful to know the size of tar file before creating it and you can get it by using unix tar command as shown in below example:

stock_trader@system:~/test tar -cf – * | wc -c

20480

Size shown here is in KB and you can also calculate size for compressed tar file by using “z” for gzip and “j” for bzip2

 

LS

122. Write a unix/linux ls command to display the hidden files and directories?

To display the hidden files and directories in the current directory use the -a option of the ls command.

> ls -a
.  ..  documents  .hidden_file  sum.pl

Hidden files are the one whose name starts with dot (.). The las -a displays the current directory (.) and parent directory (..) also. If you want to exclude the current directory, parent directory, then use -A option.

> ls -A
documents  .hidden_file  sum.pl

.List the hidden files in current directory ?

ls -a1 | grep “^\.”

. How to find hidden files in current directory?

$ ls -lrta

123. Write a unix/linux ls command to classify the files with special characters

The -F option to ls command classifies the files. It marks the

  • Directories with trailing slash (/)
  • Executable files with trailing asterisk (*)
  • FIFOs with trailing vertical bar (|)
  • Symbolic links with trailing at the rate sign (@)
  • Regular files with nothing
> ls -F
documents/  sum.pl link@

124. Write a unix/linux ls command to print each file in a separate line?

The -1 option to the ls command specifies that each file should be displayed on a separate line

> ls -1
documents
sum.pl

125. Write a unix/linux ls command to display the inode number of file?

Use -i option to the ls command to print the inode number of a file.

> ls -i1
10584066 documents
3482450 sum.pl

126. Write a unix/linux ls command to display complete information about the files?

The -l option provides lots of information about the file type, owner, group, permissions, file size, last modification date.

> ls -l
total 16
drwxr-xr-x 2 matt db 4096 Jan 30 23:08 documents
-rw-r--r-- 1 matt db   49 Jan 31 01:17 sum.pl
  • The first character indicates the type of the file. – for normal file, d for directory, l for link file and s for socket file
  • The next 9 characters in the first field represent the permissions. Each 3 characters refers the read (r), write (w), execute (x) permissions on owner, group and others. – means no permission.
  • The second field indicates the number of links to that file.
  • The third field indicates the owner name.
  • The fourth field indicates the group name.
  • The fifth field represents the file size in bytes.
  • The sixth field represents the last modification date and time of the file.
  • And finally the seventh field is the name of the file.

127. Write a unix/linux ls command to sort the files by their modification time?

The -t option allows the ls command to sort the files in descending order based on the modification time.

> ls -t1
sum.pl
documents

128. Write a unix/linux ls command to sort the files in ascending order of modification time?

The -r option reverses the order of the files displayed. Combine the -t and -r options to sort the files in ascending order.

> ls -rt1
documents
sum.pl

129. Write a unix/linux ls command to print the files recursively?

So far the ls command prints the files in the current directory. Use the -R option to recursively print the files in the sub-directories also.

> ls -R
.:
documents  sum.pl

./documents:
file.txt

130. Write a unix/linux ls command to print the files in a specific directory?

You can pass a directory to the ls command as an argument to print for the files in it.

> ls /usr/local/bin

131. Write a unix/linux ls command to display files in columns?

The -x option specifies the ls command to display the files in columns.

> ls -x

132.List the files in current directory sorted by size ?

ls -l | grep ^- | sort -nr

133.Display the files in the directory by file size ?

ls .ltr | sort .nr .k 5

 134. How to display zero byte size files?

ls -l | grep '^-' | awk '/^-/ {if ($5 !=0 ) print $9 }'

 

135. Write a command to find the sum of bytes (size of file) of all files in a directory.

ls -l | grep '^-'| awk 'BEGIN {sum=0} {sum = sum + $5} END {print sum}'

136. Write a command to display the files in the directory by file size?

ls -l | grep '^-' |sort -nr -k 5

137. Write a command to find the number of files in a directory.

ls -l|grep '^-'|wc -l

COPY

138. Write a unix/linux cp command to copy file in to a directory?

The basic usage of cp command is to copy a file from the current directory to another directory.

cp sum.pl tmp/

The cp command copies the file sum.pl into the tmp directory. The cp command does not remove the source file. It just copies the file into a new location. If a file with the same name as the source exists in the destination location, then by default the cp command overwrites that new file

139. Write a unix/linux cp to prompt for user before overwriting a file ( Interactive cp command)?

The -i option to the cp command provides the ability to prompt for a user input whether to overwrite the destination file or not.

> cp sum.pl tmp/
cp: overwrite `tmp/sum.pl'?

If you enter y, then the cp command overwrites the destination file, otherwise the cp command does not copy the file.

140. Write a unix/linux cp command to copy multiple files in to a new directory?

You can specify multiple files as the source and can copy to the new location.

cp log.dat bad.dat tmp/

The cp command copies the log.dat, bad.dat files in the current directory to the tmp directory.

141. Write a unix/linux cp command to do a Regular expression copy?

You can copy a set of files by specifying a regular expression pattern.

cp *.dat tmp/

Here the cp command copies all the files which has “dat” as suffix to the destination directory.

142. Write a unix/linux cp command to copy a file in to the current directory?

You can copy a file from a different directory to the current directory.

cp /usr/local/bin/multiply.sh .

Here the cp command copies the multiply.sh file in the /usr/local/bin directory the current directory. The dot (.) indicates the current directory.

143. Write a unix/linux cp command to copy all the files in a directory?

The cp command can be used to copy all the files in directory to another directory.

cp docs/* tmp/

This command copies all the files in the docs directory to the tmp directory.

144. Write a unix/linux cp command to copy files from multiple directories?

You can copy the files from different directories into a new location.

cp docs/* scripts/* tmp/

The command copies the files from docs and script directories to the destination directory tmp.

145. Write a unix/linux cp command to Copy a directory.

You can recursively copy a complete directory and its sub directory to another location using the cp command

cp -r docs tmp/

This copies the complete directory docs into the new directory tmp

146. Write a unix/linux cp command to Forcibly copy a file with -f option?

You can force the cp command to copy an existing destination file even it cannot be opened.

cp -f force_file.txt /var/tmp/

CD

TOUCH,CAT,MOVE,FTP file from dektop get put

 

cd [directory]
Here directory is the name of the directory where you wish to go.

 

147. Write a unix/linux cd command to change to home directory?

Just simply type cd command on the unix terminal and then press the enter key. This will change your directory to home directory.

> pwd
/usr/local/bin

Now i am in the /usr/local/bin directory. After typing the cd command and unix window, you will go to your home directory.

> cd
> pwd
/home/matt

Here pwd command displays the present working directory.

148. Write a unix/linux cd command to go back to one directory?

The cd .. changes the directory to its parent directory by going back one level. The space between the cd and .. is must.

> pwd
/var/tmp
> cd ..
> pwd
/var

149. Write a unix/linux cd command to go back to two directories?

The cd ../../ takes you back to two directories. You can extend this cd command to go back to n number of directories.

> pwd
/usr/local/bin
> cd ../../
> pwd
/usr

150. Write a unix/linux cd command to change the directory using the absolute path?

In case of changing directory using absolute path you have to specify the full directory path. Absolute path directories always start with a slash (/). An example is changing your directory to /usr/bin from your home directory.

> cd /usr/bin

151. Write a unix/linux cd command to change the directory using the relative path?

In relative path, you have to specify the directory path relative to your current directory. For example, you are in /var/tmp directory and you want to go to /var/lib directory, then you can use the relative path.

> pwd
/var/tmp
> cd ../lib
> pwd
/var/lib

Here the cd ../lib, first takes you to the parent directory which is /var and then changes the directory to the lib.

152. Write a unix/linux cd command to change back to previous directory.

As an example, i am in the directory /home/matt/documents and i changed to a new directory /home/matt/backup. Now i want to go back to my previous directory /home/matt/documents. In this case, you can use the cd – command to go back to the previous directory.

> pwd
/home/matt/documents
> cd /home/matt/backup
>pwd
/home/matt/backup
> cd -
> pwd
/home/matt/documents

 

153. Write a unix/linux command to remove a directory?

The rmdir command deletes only the empty directories. If a directory contains files or sub directories, then the rmdir command fails.

rmdir docs/
rmdir: docs/: Directory not empty

Here the docs directory is not empty, that is why the rmdir command failed to remove the directory. To remove the docs directory first we have to make the directory empty and then delete the directory.

rm doc/*
rmdir docs/

154. Write a unix/linux command to remove the directory and its parent directories?

As mentioned earlier the -p option allows the rmdir command to delete the directory and also its parent directories.

rmdir -p docs/entertainment/movies/

This rmdir command removes the docs directory completely. If you don’t use the -p option, then it only deletes the movies directory.

155. Write a unix/linux command to remove directories using pattern matching?

You can specify the directory names using the regular expressions and can delete them.

rm doc*

This rm command deletes the directories like doc, documents, doc_1 etc.

Now we will see the rm command in unix.

Unix rm command syntax

The syntax of rm command is

rm [options] [directory|file]

The rm command options are

f : Removes all files in a directory without prompting the user.
i : Interactive: prompts the user for confirmation before deleting a file.
R or r : Recursively remove directories and sub directories.

The rm command can be used to delete both the files and directories. The rm command also deletes the non-empty directories.

Unix rm command examples

156. Write a unix/linux command to remove a file?

This is the basic feature of rm command. To remove a file, logfile.dat, in the current directory use the below rm command

rm logfile.dat

157. Write a unix/linux command to remove all the files in a directory?

use the * regular pattern as the file list in rm command for deleting all the files in the current directory.

rm *

158. Write a unix/linux command to delete empty directory?

The rm command can also be used to delete the empty directory. The command for this is

rm docs/

If the directory is non-empty, then the above command fails to remove the directories.

159. Write a unix/linux command to delete directories recursively (delete non empty directories)?

As mentioned earlier, the -r option can be used to remove the directories and sub directories.

rm -r docs

MOVE

 

160.How to rename a file or directory in unix (or linux) and how to move a file or directory from the current directory to another directory?

Unix provides a simple mv (move) command which can be used to rename or move files and directories. The syntax of mv command is

mv [options] oldname newname

The options of mv command are

f : Do not prompt before overwriting a file. 
i : Prompts for the user input before overwriting a file.

If the newname already exists, then the mv command overwrites that file. Let see some examples on how to use mv command.

161. Write a unix/linux command to rename a file?

Renaming a file is one of the basic features of the mv command. To rename a file from “log.dat” to “bad.dat”, use the below mv command

> mv log.dat bad.dat

Note that if the “bad.dat” file already exists, then its contents will be overwritten by “log.dat”. To avoid this use the -i option, which prompts you before overwriting the file.

mv -i log.dat bad.dat
mv: overwrite `bad.dat'?

162. Write a unix/linux command to rename a directory?

Just as renaming a file, you can use the mv command to rename a directory. To rename the directory from docs to documents, run the below command

mv docs/ documents/

If the documents directory already exists, then the docs directory will be moved in to the documents directory.

163. Write a unix/linux command to move a file into another directory?

The mv command can also be used to move the file from one directory to another directory. The below command moves the sum.pl file in the current directory to /var/tmp directory.

mv sum.pl /var/tmp/

If the sum.pl file already exists in the /var/tmp directory, then the contents of that file will be overwritten.

164. Write a unix/linux command to move a directory in to another directory?

Just as moving a file, you can move a directory into another directory. The below mv command moves the documents directory into the tmp directory

mv documents /tmp/

165. Write a unix/linux command to move all the files in the current directory to another directory?

You can use the regular expression pattern * to move all the files from one directory to another directory.

mv * /var/tmp/

The above command moves all the files and directories in the current directory to the /var/tmp/ directory.

. mv *

What happens if you simply type mv * and then press enter?

It depends on the files you have in the directory. The * expands to all the files and directories. Three scenarios are possible.

  • If the current directory has only files, then the contents of all the files (except one file) will be written in to the one file. The one file is the last file which depends on the pattern *.
  • If the current directory contains only directories, then all the directories (except one directory) will be moved to another directory.
  • If the current directory contains both files and directories, then it depends on the expansion of the *. If the pattern * gives the last one as directory then all the files will be moved to that directory. Otherwise the mv command will fail.

Some Tips:

  • Try to avoid mv *
  • Avoid moving large number of files.

 

 XARGS

Xargs command in unix or linux operating system is used to pass the output of one command as an argument to another command. Some of the unix or linux commands like ls and find produces a long list of filenames. We want to do some operation on this list of file names like searching for a pattern, removing and renaming files etc. The xargs command provide this capability by taking the huge list of arguments as input , divides the list into small chunks and then passes them as arguments to other unix commands.

171. Renaming files with xargs

We have to first list the files to be renamed either by using the ls or find command and then pipe the output to xargs command to rename the files. First list the files which end with “.log” using the ls command.

ls *.log
oracle.log  storage.log
> ls *.log | xargs -i mv {} {}_bkp
> ls *_bkp
oracle.log_bkp  storage.log_bkp

You can see how the log files are renamed with backup (bkp) suffix. Here the option i” tells the xargs command to replace the {} with the each file returned by the ls command.

172. Searching for a pattern

We can combine the grep command with xargs to search for a pattern in a list of files returned by another unix command (ls or find). Let’s list out all the bash files in the current directory with the find command in unix.

find . -name "*.bash"
./sql_server.bash
./mysql_backup.bash
./oracle_backup.bash

Now we grep for the “echo” statements from the list of files returned by the find command with the help of xargs. The command is shown below:

find . -name "*.bash" |xargs grep "echo"

If you don’t use xargs and piped the output of find command to grep command directly, then the grep command treats each file returned by the find command as a line of string and searches for the word “echo” in that line rather in that file.

173. Removing files using xargs

We can remove the temporary files in a directory using the rm command along with the xargs command. This is shown below:

ls "*.tmp" | xargs rm

This removes all the files with “.tmp” suffix.

174. Converting Multi-line output to Single line output.

If you run the ls -1 command, it will list each file on a separate line. This is shown below:

ls -1
oracle.txt
online_backup.dat
mysql_storage.bat

We can convert this multi-line output to single line output using the xargs command. This is shown below:

ls -1 | xargs
oracle.txt online_backup.dat mysql_storage.bat

175. Handling spaces in file names

By default the xargs command treats the space as a delimiter and sends each item as an argument to the unix command. If the file name contains a space (example: “oracle storage”), then each item will be treated as a separate file and will be passed as an argument. This will cause an issue. Let see how to handle the spaces in file names with an example.

Creating a file which contains space in the name
> touch "oracle storage"

> ls oracle\ storage | xargs grep "log"
grep: oracle: No such file or directory
grep: storage: No such file or directory

You can see that grep command is treating oracle as separate file and storage as separate file. This is because of xargs treats space as delimiter. To avoid this kind of errors use the -i option with braces as shown in below:

> ls oracle\ storage | xargs -i grep "log" {}

If you want to know what command the xargs is executing use the -t option with xargs. This will print the command on the terminal before executing it.

176. Passing subset of arguments

We can pass only a subset of arguments from a long list of arguments using the -n option with xargs command. This is shown in below.

> ls -1
backup
mysql
network
online
oracle
storage
wireless

> ls -1 | xargs -n 3 echo
backup mysql network
online oracle storage
wireless

You can see from the above output that 3 arguments are passed at a time to the echo statement.

Important Notes on Xargs Command:

. Xargs directly cannot handle files which contain new lines or spaces in their names. To handle this kind of files use the -i option with xargs command. Another way to handle these characters is to treat the new line or spaces as null characters using th -0 option with xargs. However this requires that the input to xargs should also use the null as separator. An example is shown below

find . -print0 | xargs -0 rm

The print0 in find command makes the newline or space characters as null separator.

. By default the xargs uses the end of the file string as “_”. If this string appears in input string, then xargs command stops reading the input and rest of the input is ignored. You can change the end of file string by using the “-eof” option.

. To know more about xargs command, run the xargs –help on the unix or linux terminal.

 

 AWK

 

AWK

. How to print the squares of numbers from 1 to 10 using awk command

awk 'BEGIN { for(i=1;i<=10;i++) {print "square of",i,"is",i*i;}}'

.What does the following command line produce? Explain each aspect of this line.

$ (date ; ps -ef | awk ‘{print $1}’ | sort | uniq | wc -l ) >> Activity.log

A: First let’s dissect the line: The date gives the date and time as the first command of the line, this is followed by the a list of all running processes in long form with UIDs listed first, this is the ps -ef. These are fed into the awk which filters out all but the UIDs; these UIDs are piped into sort for no discernible reason and then onto uniq (now we see the reason for the sort – uniq only works on sorted data – if the list is A, B, A, then A, B, A will be the output of uniq, but if it’s A, A, B then A, B is the output) which produces only one copy of each UID.

These UIDs are fed into wc -l which counts the lines – in this case the number of distinct UIDs running processes on the system. Finally the results of these two commands, the date and the wc -l, are appended to the file “Activity.log”. Now to answer the question as to what this command line produces. This writes the date and time into the file Activity.log together with the number of distinct users who have processes running on the system at that time. If the file already exists, then these items are appended to the file, otherwise the file is created.

. How to run awk command specified in a file?

awk -f filename

. In the text file, some lines are delimited by colon and some are delimited by space. Write a command to print the third field of each line.

awk '{ if( $0 ~ /:/ ) { FS=":"; } else { FS =" "; } print $3 }' filename

. Write a command to print the line number before each line?

awk '{print NR, $0}' filename

. Write a command to print the second and third line of a file without using NR.

awk 'BEGIN {RS="";FS="\n"} {print $2,$3}' filename

. Write a command to find the total number of lines in a file?

wc -l filename

Other ways to print the total number of lines are

awk 'BEGIN {sum=0} {sum=sum+1} END {print sum}' filename
awk 'END{print NR}' filename

. How to display the fields in a text file in reverse order?

awk 'BEGIN {ORS=""} { for(i=NF;i>0;i--) print $i," "; print "\n"}' filename

. Write a command to convert a string to Initcap.

echo apple | awk '{print toupper(substr($1,1,1)) tolower(substr($1,2))}'

SED

 

>cat file.txt
unix is great os. unix is opensource. unix is free os.
learn operating system.
unixlinux which one you choose.

186. Replacing or substituting string

Sed command is mostly used to replace the text in a file. The below simple sed command replaces the word “unix” with “linux” in the file.

>sed 's/unix/linux/' file.txt
linux is great os. unix is opensource. unix is free os.
learn operating system.
linuxlinux which one you choose.

Here the “s” specifies the substitution operation. The “/” are delimiters. The “unix” is the search pattern and the “linux” is the replacement string.

By default, the sed command replaces the first occurrence of the pattern in each line and it won’t replace the second, third…occurrence in the line.

187. Replacing the nth occurrence of a pattern in a line.

Use the /1, /2 etc flags to replace the first, second occurrence of a pattern in a line. The below command replaces the second occurrence of the word “unix” with “linux” in a line.

>sed 's/unix/linux/2' file.txt
unix is great os. linux is opensource. unix is free os.
learn operating system.
unixlinux which one you choose.

188. Replacing all the occurrence of the pattern in a line.

The substitute flag /g (global replacement) specifies the sed command to replace all the occurrences of the string in the line.

>sed 's/unix/linux/g' file.txt
linux is great os. linux is opensource. linux is free os.
learn operating system.
linuxlinux which one you choose.

189. Replacing from nth occurrence to all occurrences in a line.

Use the combination of /1, /2 etc and /g to replace all the patterns from the nth occurrence of a pattern in a line. The following sed command replaces the third, fourth, fifth… “unix” word with “linux” word in a line.

>sed 's/unix/linux/3g' file.txt
unix is great os. unix is opensource. linux is free os.
learn operating system.
unixlinux which one you choose.

190. Changing the slash (/) delimiter

You can use any delimiter other than the slash. As an example if you want to change the web url to another url as

>sed 's/http:\/\//www/' file.txt

In this case the url consists the delimiter character which we used. In that case you have to escape the slash with backslash character, otherwise the substitution won’t work.

Using too many backslashes makes the sed command look awkward. In this case we can change the delimiter to another character as shown in the below example.

>sed 's_http://_www_' file.txt
>sed 's|http://|www|' file.txt

191. Using & as the matched string

There might be cases where you want to search for the pattern and replace that pattern by adding some extra characters to it. In such cases & comes in handy. The & represents the matched string.

>sed 's/unix/{&}/' file.txt
{unix} is great os. unix is opensource. unix is free os.
learn operating system.
{unix}linux which one you choose.

>sed 's/unix/{&&}/' file.txt
{unixunix} is great os. unix is opensource. unix is free os.
learn operating system.
{unixunix}linux which one you choose.

192. Using \1,\2 and so on to \9

The first pair of parenthesis specified in the pattern represents the \1, the second represents the \2 and so on. The \1,\2 can be used in the replacement string to make changes to the source string. As an example, if you want to replace the word “unix” in a line with twice as the word like “unixunix” use the sed command as below.

>sed 's/\(unix\)/\1\1/' file.txt
unixunix is great os. unix is opensource. unix is free os.
learn operating system.
unixunixlinux which one you choose.

The parenthesis needs to be escaped with the backslash character. Another example is if you want to switch the words “unixlinux” as “linuxunix”, the sed command is

>sed 's/\(unix\)\(linux\)/\2\1/' file.txt
unix is great os. unix is opensource. unix is free os.
learn operating system.
linuxunix which one you choose.

Another example is switching the first three characters in a line

>sed 's/^\(.\)\(.\)\(.\)/\3\2\1/' file.txt
inux is great os. unix is opensource. unix is free os.
aelrn operating system.
inuxlinux which one you choose.

193. Duplicating the replaced line with /p flag

The /p print flag prints the replaced line twice on the terminal. If a line does not have the search pattern and is not replaced, then the /p prints that line only once.

>sed 's/unix/linux/p' file.txt
linux is great os. unix is opensource. unix is free os.
linux is great os. unix is opensource. unix is free os.
learn operating system.
linuxlinux which one you choose.
linuxlinux which one you choose.

194. Printing only the replaced lines

Use the -n option along with the /p print flag to display only the replaced lines. Here the -n option suppresses the duplicate rows generated by the /p flag and prints the replaced lines only one time.

>sed -n 's/unix/linux/p' file.txt
linux is great os. unix is opensource. unix is free os.
linuxlinux which one you choose.

If you use -n alone without /p, then the sed does not print anything.

195. Running multiple sed commands.

You can run multiple sed commands by piping the output of one sed command as input to another sed command.

>sed 's/unix/linux/' file.txt| sed 's/os/system/'
linux is great system. unix is opensource. unix is free os.
learn operating system.
linuxlinux which one you chosysteme.

Sed provides -e option to run multiple sed commands in a single sed command. The above output can be achieved in a single sed command as shown below.

>sed -e 's/unix/linux/' -e 's/os/system/' file.txt
linux is great system. unix is opensource. unix is free os.
learn operating system.
linuxlinux which one you chosysteme.

196. Replacing string on a specific line number.

You can restrict the sed command to replace the string on a specific line number. An example is

>sed '3 s/unix/linux/' file.txt
unix is great os. unix is opensource. unix is free os.
learn operating system.
linuxlinux which one you choose.

The above sed command replaces the string only on the third line.

197. Replacing string on a range of lines.

You can specify a range of line numbers to the sed command for replacing a string.

>sed '1,3 s/unix/linux/' file.txt
linux is great os. unix is opensource. unix is free os.
learn operating system.
linuxlinux which one you choose.

Here the sed command replaces the lines with range from 1 to 3. Another example is

>sed '2,$ s/unix/linux/' file.txt
linux is great os. unix is opensource. unix is free os.
learn operating system.
linuxlinux which one you choose.

Here $ indicates the last line in the file. So the sed command replaces the text from second line to last line in the file.

198. Replace on a lines which matches a pattern.

You can specify a pattern to the sed command to match in a line. If the pattern match occurs, then only the sed command looks for the string to be replaced and if it finds, then the sed command replaces the string.

>sed '/linux/ s/unix/centos/' file.txt
unix is great os. unix is opensource. unix is free os.
learn operating system.
centoslinux which one you choose.

Here the sed command first looks for the lines which has the pattern “linux” and then replaces the word “unix” with “centos”.

199. Deleting lines.

You can delete the lines a file by specifying the line number or a range or numbers.

>sed '2 d' file.txt
>sed '5,$ d' file.txt

200. Duplicating lines

You can make the sed command to print each line of a file two times.

>sed 'p' file.txt

201. Sed as grep command

You can make sed command to work as similar to grep command.

>grep 'unix' file.txt
>sed -n '/unix/ p' file.txt

Here the sed command looks for the pattern “unix” in each line of a file and prints those lines that has the pattern.

You can also make the sed command to work as grep -v, just by using the reversing the sed with NOT (!).

>grep -v 'unix' file.txt
>sed -n '/unix/ !p' file.txt

The ! here inverts the pattern match.

202. Add a line after a match.

The sed command can add a new line after a pattern match is found. The “a” command to sed tells it to add a new line after a match is found.

>sed '/unix/ a "Add a new line"' file.txt
unix is great os. unix is opensource. unix is free os.
"Add a new line"
learn operating system.
unixlinux which one you choose.
"Add a new line"

203. Add a line before a match

The sed command can add a new line before a pattern match is found. The “i” command to sed tells it to add a new line before a match is found.

>sed '/unix/ i "Add a new line"' file.txt
"Add a new line"
unix is great os. unix is opensource. unix is free os.
learn operating system.
"Add a new line"
unixlinux which one you choose.

204. Change a line

The sed command can be used to replace an entire line with a new line. The “c” command to sed tells it to change the line.

>sed '/unix/ c "Change line"' file.txt
"Change line"
learn operating system.
"Change line"

205. Transform like tr command

The sed command can be used to convert the lower case letters to upper case letters by using the transform “y” option.

>sed 'y/ul/UL/' file.txt
Unix is great os. Unix is opensoUrce. Unix is free os.
Learn operating system.
UnixLinUx which one yoU choose.

Here the sed command transforms the alphabets “ul” into their uppercase format “UL”

Sed Command to Delete Lines: Sed command can be used to delete or remove specific lines which matches a given pattern or in a particular position in a file. Here we will see how to delete lines using sed command with various examples.

The following file contains a sample data which is used as input file in all the examples:

> cat file
linux
unix
fedora
debian
ubuntu

Sed Command to Delete Lines – Based on Position in File

In the following examples, the sed command removes the lines in file that are in a particular position in a file.

206. Delete first line or header line

The d option in sed command is used to delete a line. The syntax for deleting a line is:

> sed 'Nd' file

Here N indicates Nth line in a file. In the following example, the sed command removes the first line in a file.

> sed '1d' file
unix
fedora
debian
ubuntu

207. Delete last line or footer line or trailer line

The following sed command is used to remove the footer line in a file. The $ indicates the last line of a file.

> sed '$d' file
linux
unix
fedora
debian

208. Delete particular line

This is similar to the first example. The below sed command removes the second line in a file.

> sed '2d' file
linux
fedora
debian
ubuntu

209. Delete range of lines

The sed command can be used to delete a range of lines. The syntax is shown below:

> sed 'm,nd' file

Here m and n are min and max line numbers. The sed command removes the lines from m to n in the file. The following sed command deletes the lines ranging from 2 to 4:

> sed '2,4d' file
linux
ubuntu

210. Delete lines other than the first line or header line

Use the negation (!) operator with d option in sed command. The following sed command removes all the lines except the header line.

> sed '1!d' file
linux

211. Delete lines other than last line or footer line

> sed '$!d' file
ubuntu

212. Delete lines other than the specified range

> sed '2,4!d' file
unix
fedora
debian

Here the sed command removes lines other than 2nd, 3rd and 4th.

213. Delete first and last line

You can specify the list of lines you want to remove in sed command with semicolon as a delimiter.

> sed '1d;$d' file
unix
fedora
debian

214. Delete empty lines or blank lines

> sed '/^$/d' file

The ^$ indicates sed command to delete empty lines. However, this sed do not remove the lines that contain spaces.

whitspaces escape special character names

Sed Command to Delete Lines – Based on Pattern Match

In the following examples, the sed command deletes the lines in file which match the given pattern.

215. Delete lines that begin with specified character

> sed '/^u/d' file
linux
fedora
debian

^ is to specify the starting of the line. Above sed command removes all the lines that start with character ‘u’.

216. Delete lines that end with specified character

> sed '/x$/d' file
fedora
debian
ubuntu

$ is to indicate the end of the line. The above command deletes all the lines that end with character ‘x’.

217. Delete lines which are in upper case or capital letters

> sed '/^[A-Z]*$/d' file

218. Delete lines that contain a pattern

> sed '/debian/d' file
linux
unix
fedora
ubuntu

219. Delete lines starting from a pattern till the last line

> sed '/fedora/,$d' file
linux
unix

Here the sed command removes the line that matches the pattern fedora and also deletes all the lines to the end of the file which appear next to this matching line.

220. Delete last line only if it contains the pattern

> sed '${/ubuntu/d;}' file
linux
unix
fedora
debian

Here $ indicates the last line. If you want to delete Nth line only if it contains a pattern, then in place of $ place the line number.

Note: In all the above examples, the sed command prints the contents of the file on the unix or linux terminal by removing the lines. However the sed command does not remove the lines from the source file. To Remove the lines from the source file itself, use the -i option with sed command.

> sed -i '1d' file

If you dont wish to delete the lines from the original source file you can redirect the output of the sed command to another file.

sed '1d' file > newfile

 

221. Write a command to replace the word “bad” with “good” in file?
sed s/bad/good/ < filename
 
222.Write a command to replace the word “bad” with “good” globally in a file?
sed s/bad/good/g < filename
 
223.Write a command to replace the character ‘/’ with ‘,’ in a file?
sed ‘s/\//,/’ < filename
sed ‘s|/|,|’ < filename 224.Write a command to replace the word “apple” with “(apple)” in a file?
sed s/apple/(&)/ < filename 225.Write a command to switch the two consecutive words “apple” and “mango” in a file?
sed ‘s/\(apple\) \(mango\)/\2 \1/’ < filename226.Write a command to replace the second occurrence of the word “bat” with “ball” in a file?
sed ‘s/bat/ball/2’ < filename227. Write a command to remove all the occurrences of the word “jhon” except the first one in a line with in the entire file?
sed ‘s/jhon//2g’ < filename228. Write a command to remove the first number on line 5 in file?
sed ‘5 s/[0-9][0-9]*//’ < filename229. Write a command to remove the first number on all lines that start with “@”?
sed ‘\,^@, s/[0-9][0-9]*//’ < filename

230. Write a command to replace the word “gum” with “drum” in the first 100 lines of a file?
sed ‘1,00 s/gum/drum/’ < filename

231. write a command to replace the word “lite” with “light” from 100th line to last line in a file?
sed ‘100,$ s/lite/light/’ < filename

232. Write a command to remove the first 10 lines from a file?
sed ‘1,10 d’ < filename

233. Write a command to duplicate each line in a file?
sed ‘p’ < filename

234. Write a command to duplicate empty lines in a file?
sed ‘/^$/ p’ < filename

235. Write a sed command to print the lines that do not contain the word “run”?
sed -n ‘/run/!p’ < filename

 SED

How to remove the first 10 lines from a file?

sed '1,10 d' < filename

318. Write a command to duplicate each line in a file?

sed 'p' < filename

327. How to replace the word “Gun” with “Pen” in the first 100 lines of a file?

sed '1,00 s/Gun/Pen/' < filename
334. How to replace the second occurrence of the word “bat” with “ball” in a file?
sed 's/bat/ball/2' < filename

335. How to remove all the occurrences of the word “jhon” except the first one in a line with in the entire file?

sed 's/jhon//2g' < filename

336. How to replace the word “lite” with “light” from 100th line to last line in a file?

sed '100,$ s/lite/light/' < filename

348. Write a command to replace the word “bad” with “good” in file?

sed s/bad/good/ < filename

349. Write a command to replace the word “bad” with “good” globally in a file?

sed s/bad/good/g < filename

350. Write a command to replace the word “apple” with “(apple)” in a file?

sed s/apple/(&)/ < filename

351. Write a command to switch the two consecutive words “apple” and “mango” in a file?

sed 's/\(apple\) \(mango\)/\2 \1/' < filename

280. How to remove the header from a file?

sed -i '1 d' filename

281. How to remove the footer from a file?

sed -i '$ d' filename

282. Write a command to find the length of a line in a file?

The below command can be used to get a line from a file.

sed –n '<n> p' filename

We will see how to find the length of 10th line in a file

sed -n '10 p' filename|wc -c

292. How to display the first 20 lines of a file? By default, the head command displays the first 10 lines from a file. If we change the option of head, then we can display as many lines as we want.

head -20 filename

An alternative solution is using the sed command

sed '21,$ d' filename

The d option here deletes the lines from 21 to the end of the file

293. Write a command to print the last line of a file?

The tail command can be used to display the last lines from a file.

tail -1 filename

Alternative solutions are:

sed -n '$ p' filename
awk 'END{print $0}' filename

286. How to replace the n-th line in a file with a new line in Unix?

sed -i'' '10 d' filename      # d stands for delete
sed -i'' '10 i new inserted line' filename    # i stands for insert

301. How do you remove the first number on 10th line in file?

sed '10 s/[0-9][0-9]*//' < filename

307. Write a command to remove the first number on all lines that start with “@”?

sed '\,^@, s/[0-9][0-9]*//' < filename

312. How to duplicate empty lines in a file?

sed '/^$/ p' < filename

 

340. How to replace the character ‘/’ with ‘,’ in a file?

sed 's/\//,/' < filename
sed 's|/|,|' < filename

 

.Display the last newly appending lines of a file during appendingdata to the same file by some processes ?

tail .f Debug.logHere tail shows the newly appended data into Debug.log by some processes/user.

Explain iostat, vmstat and netstat?

    • Iostat: reports on terminal, disk and tape I/O activity.
  • Vmstat: reports on virtual memory statistics for processes, disk, tape and CPU activity.
  • Netstat: reports on the contents of network data structures.

. How to find out the usage of the CPU by the processes?

The top utility can be used to display the CPU usage by the proce

. Write a command to remove the prefix of the string ending with ‘/’.

The basename utility deletes any prefix ending in /. The usage is mentioned below:

basename /usr/local/bin/file

This will display only file

.Display the Disk Usage of file sizes under each directory in currentDirectory ?

du -k * | sort .nr (or) du .k . | sort -nr

.Change to a directory, which is having very long name ?

cd CDMA_3X_GEN*Here original directory name is . .CDMA_3X_GENERATION_DATA..

.Set the Display automatically for the current new user ?

export DISPLAY=`eval ‘who am i | cut -d”(” -f2 | cut -d”)” -f1′`Here in above command, see single quote, double quote, grave ascent is used. Observe carefully.

.Display the processes, which are running under yourusername ?

ps .aef | grep MaheshvjHere, Maheshvj is the username.

.List some Hot Keys for bash shell ?

– Ctrl+l . Clears the Screen. Ctrl+r . Does a search in previously given commands in shell. Ctrl+u – Clears the typing before the hotkey. Ctrl+a . Places cursor at the beginning of the command at shell. Ctrl+e . Places cursor at the end of the command at shell. Ctrl+d . Kills the shell. Ctrl+z . Places the currently running process into background.

.How to save man pages to a file ?

man <command> | col .b > <output-file>Example : man top | col .b > top_help.txt

.How to know the date & time for . when script is executed ?

Add the following script line in shell script.eval echo “Script is executed at `date`” >> timeinfo.infHere, .timeinfo.inf. contains date & time details ie., when script is executed and history related to execution.

.How do you find out drive statistics ?

iostat -E

.Display disk usage in Kilobytes ?

du -k

.Display top ten largest files/directories

 – du -sk * | sort -nr | head

.How much space is used for users in kilobytes ?

quot -af

.How to create null file ?

cat /dev/null > filename1

.Access common commands quicker ?

ps -ef | grep -i $@

Display the page size of memory ?

pagesize -a

.Display Ethernet Address arp table ?

arp -a

.Display the no.of active established connections to localhost ?

netstat -a | grep EST

259.Display the state of interfaces used for TCP/IP traffice ?

netstat -i

.Display the parent/child tree of a process ?

ptree <pid> Example: ptree 1267

.Show the working directory of a process ?

pwdx <pid> Example: pwdx 1267

.Display the processes current open files ?

pfiles <pid> Example: pfiles 1267

.Display the inter-process communication facility status ?

ipcs

.Display the top most process utilizing most CPU ?

top .b 1

.Alternative for top command ?

prstat -a

.What is the most graceful way to get to run level single user mode?

A: The most graceful way is to use the command init s.
If you want to shut everything down before going to single user mode then do init 0 first and from the ok prompt do a boot -s.

 .How to find current running processes in unix server?

$ ps -ef

and if we want to find specific process we can use ‘grep’ with pipe
$ ps -ef | grep -i ‘application’

 .How to find process which is taking maximum memory in server?
$ top
top command tell us about cpu usage , process id and other details. below is output of top command

 

.How do you access command line arguments from within a shell script?

Arguments passed from the command line to a shell script can be accessed within the shell script by using a $ (dollar sign) immediately followed with the argument’s numeric position on the command line.

 .How to tails last 200 lines of any log fine?

$ tail -200f filename.txt

 .How to find remaining disk space in unix\linux server?
$ df -kl
df -kl
Filesystem   1024-blocks      Used Available Capacity  iused    ifree %iused  Mounted on
/dev/disk0s2   244277768 153679844  90341924    63% 38483959 22585481   63%   /
 .How to make any script file executable?
    $chmod 755 *.sh

. How to kill process in unix server?
$ kill -9 #pid
these #pid can be found using ps -ef command.
 

. How to display the 10th line of a file?

head -10 filename | tail -1

. How to reverse a string in unix?

echo "java" | rev

. How to get the last word from a line in Unix file?

echo "unix is good" | rev | cut -f1 -d' ' | rev

. How to check if the last command was successful in Unix?

echo $?

. How will you find which operating system your system is running on in UNIX?

uname -a

. Create a read-only file in your home directory?

touch file; chmod 400 file

. How do you see command line history in UNIX?

The ‘history’ command can be used to get the list of commands that we are executed.

. Write a command to convert a string from lower case to upper case?

echo "apple" | tr [a-z] [A-Z]

. Write a command to redirect the output of date command to multiple files?

The tee command writes the output to multiple files and also displays the output on the terminal.

date | tee -a file1 file2 file3

. How to create an alias for the complex command and remove the alias?

The alias utility is used to create the alias for a command. The below command creates alias for ps -aef command.

alias pg='ps -aef'

If you use pg, it will work the same way as ps -aef.

To remove the alias simply use the unalias command as

unalias pg

. Write a command to display today’s date in the format of ‘yyyy-mm-dd’?

The date command can be used to display today’s date with time

date '+%Y-%m-%d'

. List out some of the Hot Keys available in bash shell?

  • Ctrl+l – Clears the Screen.
  • Ctrl+r – Does a search in previously given commands in shell.
  • Ctrl+u – Clears the typing before the hotkey.
  • Ctrl+a – Places cursor at the beginning of the command at shell.
  • Ctrl+e – Places cursor at the end of the command at shell.
  • Ctrl+d – Kills the shell.
  • Ctrl+z – Places the currently running process into background.

. What is the difference between join -v and join -a?

join -v : outputs only matched lines between two files. join -a : In addition to the matched lines, this will output unmatched lines also.

 How do you display the calendar for the month march in the year 1985?

The cal command can be used to display the current month calendar. You can pass the month and year as arguments to display the required year, month combination calendar.

cal 03 1985

This will display the calendar for the March month and year 1985.

. How do you write the contents of 3 files into a single file?

cat file1 file2 file3 > file

 

. Write a command to display your name 100 times. The Yes utility can be used to repeatedly output a line with the specified string or ‘y’.

yes <your_name> | head -100