GREP
1 : How to ignore some words while doing search using grep in UNIX
grep Exception logfile.txt | grep -v ERROR
This grep command example will search for word “Exception” in logfile.txt and print them but since we have piped out of first grep command to second grep command which will exclude all lines which match world “ERROR”.
$ cat example.txt UNIX operating system UNIX and Linux operating system Linux operation system
Now we want to search all lines in file example.txt which contains word UNIX but same time doesn’t contain world Linux.
$ grep UNIX example.txt UNIX operating system UNIX and Linux operating system
Now to exclude all lines which contains Linux we will apply another grep command in this output with option -v to exclude matching word as shown in below grep command :
$ grep UNIX example.txt | grep -v Linux
UNIX operating system
2 : How to count occurrence of a word in a file using grep command
Below example of command will print how many times word “Error” has appeared in logfile.txt.
$ grep -c "Error" logfile.txt
If we apply this grep command on our example file to find how many lines contains word e.g. UNIX has occurred in the file :
$ grep -c UNIX example.txt 2
3 : printing lines before and after of matching word using grep
Below example of grep command in UNIX will print 6 lines around matching line of word “successful” in logfile.txt
$ grep --context=6 successful logfile.txt
Show additional six lines after matching very useful to see what is around and to print whole message if it splits around multiple lines. You can also use command line option “C” instead of “–context” for example
$ grep -C 2 'hello' *
Prints two lines of context around each matching line.
4. How to search pattern using egrep and regular expression
stands for extended grep and it is more powerful than grep command in Unix and allows more regular exception like you can use “|” option to search for either Error or Exception by executing just one command.
$ egrep 'Error|Exception' logfile.txt
5 : How to do case insensitive searching using grep in Linux
If you want to do case insensitive search than use -i option from grep command in UNIX. grep -i command will find occurrence of both Error, error and ERROR and quite useful to display any sort of Error from log file.
$ grep -i Error logfile
6 : How to search patterns in gzip files using zgrep command
zgrep is another great version of grep command in Unix which is used to perform same operation as grep does but with .gz files. Many a times we gzip the old file to reduce size and later wants to look or find something on those file. zgrep is your man for those days. Below command will print all files which have “Error” on them.
$ zgrep -i Error *.gz
7 : How to search whole word in a file using grep command
You can use grep -w command in UNIX to find whole word instead of just pattern, as shown in following example. This example will only print lines from logfile.txt which contains full word ERROR.
$ grep -w ERROR logfile.txt
Above grep command in UNIX searches only for instances of ‘ERROR’ that are entire words; it does not match `SysERROR’.
For more control, use `\<‘ and `\>’ to match the start and end of words. For example:
$ grep 'ERROR>' *
Searches only for words ending in ‘ERROR’, so it matches the word `SysERROR’.
8 : UNIX command to display files names which contains given word
Another useful grep command line option is “grep -l” which display only the file names which matches the given pattern. Below command will only display file names which have ERROR?
$ grep -l ERROR *.log
grep -l ‘main’ *.java will list the names of all Java files in the current directory whose contents mention `main’.
9:Write a command to print the line numbers along with the line that has the word “july”?
grep -n july filename
$ grep -n ERROR log file.
10 : How to do recursive search in a directory using grep in UNIX
If you want to do recursive search using grep command in Unix there are two options either use “-R” command line option or increase directory one by one as shown below.
$ grep -R store *
This command will search for directory or file with name store in current directory and it’s all sub-directory.
grep command in UNIX can show matching patter in color which is quite useful to highlight the matching section , to see matching pattern in color use below command.
$ grep Exception today.log --color
You can also create alias grep=’grep –color’ in your bash_profile file to avoid typing –color every time.
There are three version of grep command in UNIX `grep, fgrep, egrep’. `fgrep’ stands for Fixed `grep’, `egrep’ Extended `grep’
Read more
11. Write a command to print the lines that has the the pattern “july” in all the files in a particular directory?
grep july *
This will print all the lines in all files that contain the word “july” along with the file name. If any of the files contain words like “JULY” or “July”, the above command would not print those lines.
12. Write a command to print the lines that has the word “july” in all the files in a directory and also suppress the file name in the output.
grep -h july *
13. When you use a single file as input to the grep command to search for a pattern, it won’t print the filename in the output. Now write a grep command to print the file name in the output without using the ‘-H’ option.
grep pattern file name /dev/null
The /dev/null or null device is special file that discards the data written to it. So, the /dev/null is always an empty file.
Another way to print the file name is using the ‘-H’ option. The grep command for this is
grep -H pattern filename
14. Write a command to print the file names in a directory that does not contain the word “july”?
grep -L july *
The ‘-L’ option makes the grep command to print the file names that do not contain the specified pattern.
15. Write a command to print the lines that starts with the word “start”?
grep '^start' filename
The ‘^’ symbol specifies the grep command to search for the pattern at the start of the line.
16. Running the last executed grep command
This saves a lot of time if you are executing the same command again and again.
!grep
This displays the last executed grep command and also prints the result set of the command on the terminal.
17. Search for a string in a file
This is the basic usage of grep command. It searches for the given string in the specified file.
grep "Error" logfile.txt
This searches for the string “Error” in the log file and prints all the lines that has the word “Error”.
18. Searching for a string in multiple files.
grep "string" file1 file2 grep "string" file_pattern
This is also the basic usage of the grep command. You can manually specify the list of files you want to search or you can specify a file pattern (use regular expressions) to search for.
19. Specifying the search string as a regular expression pattern.
grep "^[0-9].*" file.txt
This will search for the lines which starts with a number.
20. Displaying the lines before the match.
Some times, if you are searching for an error in a log file; it is always good to know the lines around the error lines to know the cause of the error.
grep -B 2 "Error" file.txt
This will prints the matched lines along with the two lines before the matched lines.
21. Displaying the lines after the match.
grep -A 3 "Error" file.txt
This will display the matched lines along with the three lines after the matched lines.
22. Inverting the pattern match
You can display the lines that are not matched with the specified search sting pattern using the -v option.
grep -v "string" file.txt
23. Displaying the non-empty lines
You can remove the blank lines using the grep command.
grep -v "^$" file.txt
24. Displaying only the matched pattern.
By default, grep displays the entire line which has the matched string. We can make the grep to display only the matched string by using the -o option.
grep -o "string" file.txt
25. Displaying the position of the matched string in the line
The -b option allows the grep command to display the character position of the matched string in a file.
grep -o -b "string" file.txt
26. Matching the lines that start with a string
The ^ regular expression pattern specifies the start of a line. This can be used in grep to match the lines which start with the given string or pattern.
grep "^start" file.txt
27. Matching the lines that end with a string
The $ regular expression pattern specifies the end of a line. This can be used in grep to match the lines which end with the given string or pattern.
grep "end$" file.txt
28.How do you rename the files in a directory with _new as suffix?
ls -lrt|grep '^-'| awk '{print "mv "$9" "$9".new"}' | sh
29. Write command to list all the links from a directory?
ls -lrt | grep "^l"
30. How do you list the hidden files in current directory?
ls -a | grep '^\.'
31.isplay all the files in current directory sorted by size?
ls -l | grep '^-' | awk '{print $5,$9}' |sort -n|awk '{print $2}'
32.How to find Exception in log files available in current directory and how to find number of occurrence?
$ grep ‘Exception’ log1.txt | wc -l
33.Search for a sample string in particular files ?
– grep .Debug. *.confHere grep uses the string .Debug. to search in all files with extension..conf. under current directory.
34. How to remove blank lines in a file ?
grep -v ‘^$’ filename > new_filename
35. How to display the processes that were run by your user name ?
ps -aef | grep <user_name>
36.Delete blank lines in a file ?
– cat sample.txt | grep -v ‘^$’ > new_sample.txt
FIND
37. How to run the last executed find command?
!find
This will execute the last find command. It also displays the last find command executed along with the result on the terminal.
38. How to find for a file using name?
find -name “sum.java”
./bkp/sum.java
./sum.java
This will find all the files with name “sum.java” in the current directory and sub-directories.
39. How to find for files using name and ignoring case?
find –iname “sum.java”
./SUM.java
./bkp/sum.java
./sum.java
This will find all the files with name “sum.java” while ignoring the case in the current directory and sub-directories.
40. How to find for a file in the current directory only?
find -maxdepth 1 -name “sum.java”
./sum.java
This will find for the file “sum.java” in the current directory only
41. How to find for files containing a specific word in its name?
find –name “*java*”
./SUM.java
./bkp/sum.java
./sum.java
./multiply.java
It displayed all the files which have the word “java” in the filename
42. How to find for files in a specific directory?
find /etc -name “*java*”
This will look for the files in the /etc directory with “java” in the filename
43. How to find the files whose name are not “sum.java“?
find –not -name “sum.java”
.
./SUM.java
./bkp
./multiply.java
This is like inverting the match. It prints all the files except the given file “sum.java”.
44. How to limit the file searches to specific directories?
find -name “sum.java”
./tmp/sum.java
./bkp/var/tmp/files/sum.java
./bkp/var/tmp/sum.java
./bkp/var/sum.java
./bkp/sum.java
./sum.java
You can see here the find command displayed all the files with name “sum.java” in the current directory and sub-directories.
a. How to print the files in the current directory and one level down to the current directory?
find -maxdepth 2 -name “sum.java”
./tmp/sum.java
./bkp/sum.java
./sum.java
b. How to print the files in the current directory and two levels down to the current directory?
find -maxdepth 3 -name “sum.java”
./tmp/sum.java
./bkp/var/sum.java
./bkp/sum.java
./sum.java
c. How to print the files in the subdirectories between level 1 and 4?
find -mindepth 2 -maxdepth 5 -name “sum.java”
./tmp/sum.java
./bkp/var/tmp/files/sum.java
./bkp/var/tmp/sum.java
./bkp/var/sum.java
./bkp/sum.java
45. How to find the empty files in a directory?
find . –maxdepth 1 -empty
./empty_file
46. How to find the largest file in the current directory and sub directories
find . -type f -exec ls -s {} \; | sort -n -r | head -1
The find command “find . -type f -exec ls -s {} \;” will list all the files along with the size of the file. Then the sort command will sort the files based on the size. The head command will pick only the first line from the output of sort.
47. How to find the smallest file in the current directory and sub directories
find . -type f -exec ls -s {} \; | sort -n -r | tail -1
Another method using find is
find . -type f -exec ls -s {} \; | sort -n | head -1
48. How to find files based on the file type?
a. Finding socket files
find . -type s
b. Finding directories
find . -type d
c. Finding hidden directories
find -type d -name “.*”
d. Finding regular files
find . -type f
e. Finding hidden files
find . -type f -name “.*”
49. How to find files based on the size?
a. Finding files whose size is exactly 10M
find . -size 10M
b. Finding files larger than 10M size
find . -size +10M
c. Finding files smaller than 10M size
find . –size -10M
50. How to find the files which are modified after the modification of a give file.
find –newer “sum.java”
This will display all the files which are modified after the file “sum.java”
51. Display the files which are accessed after the modification of a give file.
find –anewer “sum.java”
52. Display the files which are changed after the modification of a give file.
find –cnewer “sum.java”
53. How to find the files based on the file permissions?
find . -perm 777
This will display the files which have read, write, and execute permissions. To know the permissions of files and directories use the command “ls -l”.
find –perm option is used to find files based upon permissions.
You can used find –perm 444 to get all files which allows read to owner, group and others.
javin@testenv1:~/java $ find . -perm 644
./.vimrc
./OnlineStockTrading.java
find out all the executable files, you can also modify it to find all the read only files or files having write permission etc by changing permissions e.g. to find all read only files in current directory :
find . –perm 555 Here “.” or period denotes current directory. You can replace it with any directory you want.
54. Write a command to list the files that are accessed 5 days ago in the current directory?
find -atime 5 -type f
55. Write a command to list the files whose status is changed 5 days ago in the current directory?
find -ctime 5 -type f
56. Write a command to list the files that were modified 5 days ago in the current directory?
find –mtime 5 -type f
57.How to find files which has been modified less than one day, minute or hour in Unix:
find –mtime is used to search files based upon modification time.
find –atime which denote last accessed time of file
find –ctime denotes last changed time. + sign is used to search for greater than, – sign is used to search for less than and without sign is used for exact.
For example find –mtime -1 will search all files which has been modified
javin@testenv1 ~/java : find . -mtime 1 (find all the files modified exact 1 day)
javin@testenv1 ~/java : find . -mtime -1 (find all the files modified less than 1 day)
.
./StockTrading.java
javin@testenv1 ~/java : find . -mtime +1 (find all the files modified more than 1 day)
./.vimrc
./OnlineStockTrading.java
./StockTrading.java~
58. Find the files which are modified within 30 minutes.
find . -mmin -30
59. Find the files which are modified within 1 day.
find . -mtime -1
60. How to find the files which are modified 30 minutes back
find . -not -mmin -30
61. How to find the files which are modified 1 day back.
find . -not -mtime -1
62.Print the files which are accessed within 1 hour.
find . -amin -60
63. Print the files which are accessed within 1 day.
find . -atime -1
64. Display the files which are changed within 2 hours.
find . -cmin -120
65. Display the files which are changed within 2 days.
find . -ctime -2
66. How to find the files which are created between two files.
find . -cnewer f1 -and ! -cnewer f2
67. How to find the permissions of the files which contain the name “java“?
find -name “*java*”|xargs ls -l
Alternate method is
find -name “*java*” -exec ls -l {} \;
68. Find the files which have the name “java” in it and then display only the files which have “class” word in them?
find -name “*java*” -exec grep -H class {} \;
69. How to remove files which contain the name “java”.
find -name “*java*” -exec rm -r {} \;
70. Write a command to search for the file ‘test’ in the current directory?
find -name test -type f
71. Write a command to search for the file ‘temp’ in ‘/usr’ directory?
find /usr -name temp -type f
72. Write a command to search for the files in the current directory which are not owned by any user in the /etc/passwd file?
find . -nouser -type f
73. Write a command to search for the files in ‘/usr’ directory that start with ‘te’?
find /usr -name ‘te*’ -type f
74. Write a command to search for the files that start with ‘te’ in the current directory and then display the contents of the file?
find . –name ‘te*’ -type f -exec cat {} \;
75. Write a command to list the files in ‘/usr’ directory that start with ‘ch’ and then display the number of lines in each file?
find /usr -name ‘ch*’ -type f -exec wc -l {} \;
76.How to delete temporary files using find command in Unix?
In order to delete files you can use either –delete option of find command or use xargs in combination. Its better to create house keeping script for such task which can perform cleanup on periodic basis.
find . -name “*.tmp” -print | xargs rm –f
considering use of -print0 to avoid problems with white space in the path when piping to xargs (use with the xargs -0 option)
77.How to find all text file which contains word Exception using find command in Unix ?
find . –name “*.java” –print | xargs grep “MemoryCache”,
this will search all java files starting from current directory for word “MemoryCache”.
we can also leave -print option in all cases because its default for UNIX find command You can further sort result of find command using Sort command in unix.
find . –name “*.txt” –print | xargs grep “Exception”
78. Write a command to search for zero byte size files in the current directory?
find –size 0 -type f
79. Finding files only in current directory not searching on sub directories:
–type option can be used to specifiy search for only file, link or directory and maxdepth specifies how deep find has to search.
find . -maxdepth 1 -type f -newer first_file
Another way of doing it is below:
find . -type f -cmin 15 -prune
Means type file, last modified 15 minutes ago, only look at the current directory. (No sub-directories)
80 .How to find files based on size in Unix and Linux
find . -size +1000c -exec ls -l {} \;
find . -size +10000c -size -50000c -print
This find example lists all files that are greater than 10,000 bytes, but less than 50,000 bytes:
81.How to find files some days older and above certain size
find command will find which are more than 10 days old and size greater than 50K.
find . -mtime +10 -size +50000c -exec ls -l {} \;
You can use “awk“ in combination of find to print a formatted output e.g. next command will find all of the symbolic links in your home directory, and print the files your symbolic links points to:
find . –type l -print | xargs ls -ld | awk ‘{print $10}’
“.” says starts from current directory and include all sub directory and “-type l” says list all links.
Tip:
$* : $* is one of the special bash parameter which is used to expands positional parameters from position one.
82.How to use find command on file names with space in
find -print0 and xargs -0 on find examples,
we don’t specify any expression after find command the default option is -print which prints the name of each found files followed by \n or newline.since we mostly pipe output of find command to xargs -print could cause problem if file name itself contain new line or any form of white space.
To resolve this issue instead of -print use -print0.
Difference between find -print and find -print0 is, print0 display file name on the stdout followed by a “NUL” character and then you can use xargs -0command to process file names with null character.
UNIX find command example with file name having space in them:
javin@testenv1:~/test find . -name “*equity*” -print
./cash equity trading ./equity~
You see here “cash equity trading” has space in there name
javin@testenv1:~/test find . -name “*equity*” -print | xargs ls -l
ls: cannot access ./cash: No such file or directory
ls: cannot access equity: No such file or directory
ls: cannot access trading: No such file or directory
-r–r–r– 1 stock_trading cash_domain trading 0 Jul 15 11:42 ./equity~
Now if we pass this to xargs, xargs treat them as three separate files.
javin@testenv1:~/test find . -name “*equity*” -print0 | xargs ls
xargs: WARNING: a NUL character occurred in the input. It cannot be passed through in the argument list. Did you mean to use the –null option?
ls: cannot access ./cash: No such file or directory
ls: cannot access equity: No such file or directory
ls: cannot access trading: No such file or directory
Now to solve this we have used find command with -print0 which appends NUL character on file name but without xargs -0, xargs will not able to handle those inputs.
javin@testenv1:~/test find . -name “*equity*” -print0 | xargs -0 ls -l
-rw-r–r– 1 stock_trading cash_domain trading 0 Jul 21 09:54 ./cash equity trading
-r–r–r– 1 stock_trading cash_domain trading 0 Jul 15 11:42 ./equity~
you can see with find -print0| xargs -0 it looks good
In conclusion always use find -print0 along with xargs -0 if you see slightest possibilities of file names containing space in UNIX or Linux.
- find –print and find is same as –print is a default option of find command.
- find –print0 should be used to avoid any issue with white space in file name or path while forwarding output to xargs, also use xargs -0 along with find –print0.
- find has an option called –delete which can be used in place of -exec rm {} \;
83.find all files in current and subdirectories which contains ‘log’ name?
$ find . -name ‘log’
84.Write a command to find all of the files which have been accessed within the last 30 days.
find / -type f -atime -30 > December.files
This command will find all the files under root, which is ‘/’, with file type is file. ‘-atime -30′ will give all the files accessed less than 30 days ago. And the output will put into a file call December.files.
. Write a command to search for the file ‘map’ in the current directory?
find -name map -type f
85. Write a command to list the files in ‘/usr’ directory that start with ‘ch’ and then display the number of lines in each file?
wc -l /usr/ch*
Another way is
find /usr -name 'ch*' -type f -exec wc -l {} \;
86. Write a command to display all the files recursively with path under current directory?
find . -depth -print
87. Display zero byte size files in the current directory?
find -size 0 -type f
CUT
Unix Cut Command Example
We will see the usage of cut command by considering the below text file as an example
> cat file.txt unix or linux os is unix good os is linux good os
88.Write a unix/linux cut command to print characters by position?
The cut command can be used to print characters in a line by specifying the position of the characters. To print the characters in a line, use the -c option in cut command
cut -c4 file.txt x u l
The above cut command prints the fourth character in each line of the file. You can print more than one character at a time by specifying the character positions in a comma separated list as shown in the below example
cut -c4,6 file.txt xo ui ln
This command prints the fourth and sixth character in each line.
89.Write a unix/linux cut command to print characters by range?
You can print a range of characters in a line by specifying the start and end position of the characters.
cut -c4-7 file.txt x or unix linu
The above cut command prints the characters from fourth position to the seventh position in each line. To print the first six characters in a line, omit the start position and specify only the end position.
cut -c-6 file.txt unix o is uni is lin
To print the characters from tenth position to the end, specify only the start position and omit the end position.
cut -c10- file.txt inux os ood os good os
If you omit the start and end positions, then the cut command prints the entire line.
cut -c- file.txt
90.Write a unix/linux cut command to print the fields using the delimiter?
You can use the cut command just as awk command to extract the fields in a file using a delimiter. The -d option in cut command can be used to specify the delimiter and -f option is used to specify the field position.
cut -d' ' -f2 file.txt or unix linux
This command prints the second field in each line by treating the space as delimiter. You can print more than one field by specifying the position of the fields in a comma delimited list.
cut -d' ' -f2,3 file.txt or linux unix good linux good
The above command prints the second and third field in each line.
Note: If the delimiter you specified is not exists in the line, then the cut command prints the entire line. To suppress these lines use the -s option in cut command.
91. Write a unix/linux cut command to display range of fields?
You can print a range of fields by specifying the start and end position.
cut -d' ' -f1-3 file.txt
The above command prints the first, second and third fields. To print the first three fields, you can ignore the start position and specify only the end position.
cut -d' ' -f-3 file.txt
To print the fields from second fields to last field, you can omit the last field position.
cut -d' ' -f2- file.txt
92. Write a unix/linux cut command to display the first field from /etc/passwd file?
The /etc/passwd is a delimited file and the delimiter is a colon (:). The cut command to display the first field in /etc/passwd file is
cut -d':' -f1 /etc/passwd
. The input file contains the below text
> cat filenames.txt logfile.dat sum.pl add_int.sh
Using the cut command extract the portion after the dot.
First reverse the text in each line and then apply the command on it.
rev filenames.txt | cut -d'.' -f1
104. How do you make an existing file empty?
cat /dev/null > filename
105. How to get the nth word of a line in Unix?
cut –f<n> -d' '
SORT
The options are:
-b : Ignores leading spaces in each line -d : Uses dictionary sort order. Conisders only spaces and alphanumeric characters in sorting -f : Uses case insensitive sorting. -M : Sorts based on months. Considers only first 3 letters as month. Eg: JAN, FEB -n : Uses numeric sorting -R : Sorts the input file randomly. -r : Reverse order sorting -k : Sorts file based on the data in the specified field positions. -u : Suppresses duplicate lines -t : input field separator
Sort Command Examples:
Before practicing the examples create the below two files in your unix system:
> cat order.txt Unix distributed 05 server Linux virtual 3 server Unix distributed 05 server Distributed processing 6 system > cat delim_sort.txt Mayday|4 Janmon|1 Declast|12
106. Sorting lines of text
The default sort command uses alphabetical order (ASCII order) to sort the file. It treats each line as a string and then sorts the lines.
> sort order.txt Distributed processing 6 system Linux virtual 3 server Unix distributed 05 server Unix distributed 05 server
107. Sorting based on the field positions.
You can specify the field postions using the -k option of sort command. The sort command uses the space or tab as the default delimiter. To sort based on the data in the second field, run the below command:
> sort -k2 order.txt Unix distributed 05 server Unix distributed 05 server Distributed processing 6 system Linux virtual 3 server
You can also pecify more than field with k option as a comma separated list. The below command uses the second and fourth fields to sort the data.
> sort -k2,4 order.txt
108. Numeric sorting
Instead of the default alphabetical sorting order, you can make the sort command to sort in numeric order using the -n option. This is shown below:
> sort -nk3 order.txt Linux virtual 3 server Unix distributed 05 server Unix distributed 05 server Distributed processing 6 system
109. Sort in reverse order
By default, the sort command sorts the data in ascending order. You can change this to descending order using the -r option.
> sort -nrk3 order.txt Distributed processing 6 system Unix distributed 05 server Unix distributed 05 server Linux virtual 3 server
110. Suppressing duplicates or Print only unique values
You can produce only unique values in the output using the – u option of the sort command.
> sort -u order.txt Distributed processing 6 system Linux virtual 3 server Unix distributed 05 server
Another way is piping the output of sort command to uniq command.
> sort order.txt | uniq
111. Delimited file input
In the second, third and fourth examples we have sorted the data based on the field positions. Here the fields are separted by space or tab character. What if the fields are specifed by any other character? In such cases, we have to specify the input delimiter with the -t option. An example is shown below:
> sort -t'|' -nrk2 delim_sort.txt Declast|12 Mayday|4 Janmon|1
112. Sorting on months.
We can sort the data in the monthwise using the -M option of the sort command. This is shown below:
> sort -M delim_sort.txt Janmon|1 Mayday|4 Declast|12
ps-ef
TAR
The options of tar command are:
c : creates a tar file. v : verbose. Displays the files information. f : Specify the tar file name. r : updates the tar file with new files. x : Extracts files from the archive (tar file). t : view contents of tar file. z : Specify the tar command to create a tar file using gzip in unix. j : uses bzip2 to create the tar file.
Tar Command Examples:
113. Creating a tar file
Let see a sample example by archiving all the files in my current directory. The ls -l command displays the files and directories in the current directory.
> ls -l drwxr-xr-x 2 user group 4096 Aug 8 03:23 debian -rw-r--r-- 1 user group 174 Aug 2 23:39 file -rw-r--r-- 1 user group 0 Aug 8 03:22 linux_server.bat -rw-r--r-- 1 user group 76 Aug 2 02:21 test.sh -rw-r--r-- 1 user group 0 Aug 8 03:22 unix_distro
We see how to tar all these files using the -c option with the tar command. This is shown below:
> tar -cvf archive.tar * debian/ file linux_server.bat test.sh unix_distro > ls archive.tar debian file linux_server.bat test.sh unix_distro
Observe the output of ls command and see the archive.tar file is created.
114. Printing the contents of tar file
We have created the tar file and we dont know whether it contains the actual files or not. To view the contents of the tar file use the -t option as
> tar -tvf archive.tar drwxr-xr-x user/group 0 2012-08-08 03:23:07 debian/ -rw-r--r-- user/group 174 2012-08-02 23:39:51 file -rw-r--r-- user/group 0 2012-08-08 03:22:19 linux_server.bat -rw-r--r-- user/group 76 2012-08-02 02:21:32 test.sh -rw-r--r-- user/group 0 2012-08-08 03:22:09 unix_distro
115. Updating the tar file with new contents.
You can add new files to the existing archive (tar) file using the -r option.
>touch red-hat-linux.dat >tar -rvf archive.tar red-hat-linux.dat red-hat-linux.dat >tar -tvf archive.tar drwxr-xr-x pcenter/pcenter 0 2012-08-08 03:23:07 debian/ -rw-r--r-- pcenter/pcenter 174 2012-08-02 23:39:51 file -rw-r--r-- pcenter/pcenter 0 2012-08-08 03:22:19 linux_server.bat -rw-r--r-- pcenter/pcenter 76 2012-08-02 02:21:32 test.sh -rw-r--r-- pcenter/pcenter 0 2012-08-08 03:22:09 unix_distro -rw-r--r-- pcenter/pcenter 0 2012-08-08 04:00:00 red-hat-linux.dat
Here the touch command creates a new file. The first tar command adds the new file to the existing archive file. The second command displays the contents of the tar file.
116. Extracting the contents of tar file
In the first example, we have created the archive file. Now we will see how to extract the set of files from the archive. To extract the contents of the tar file use the -x option.
> tar -xvf archive.tar debian/ file linux_server.bat test.sh unix_distro
117. Creating compressed tar file
So far we have created a uncompressed tar file in the above examples. We can create a compressed tar file using the gzip or bzip2.
Compressing files using gzip > tar -zcvf new_tar_file.tar.gz * Compressing files using bzip2 > tar -jcvf new_tar_file.tar.bz2 *
To extract or to view the files in a compressed tar file use the appropriate compression option (z or j).
To view files in a gzip compressed tar file > tar -ztvf new_tar_file.tar.gz To extract files from a gip compressed tar file > tar -zxvf new_tar_file.tar.gz To view files in a bzip2 compressed tar file > tar -jtvf new_tar_file.tar.bz2 To extract files from a bzip2 compressed tar file > tar -jxvf new_tar_file.tar.bz2
118. Creating tar file with specified list of files
You can specify a list of files to be included in the newly created tar file.
> tar -cvf unix_files.tar unix_server.bat unix_system.dat
Here the tar command creates the unix_files.tar file which contains only the files unix_server.bat and unix_system.dat
119. Extracting specific files from the tar
You can extract a specific file or a set of files from the archived file.
To extract a specifi file > tar -xvf unix_files.tar unix_server.bat To extract all files that start with name unix > tar -xvf unix_files.tar --wildcards "unix*"
120. Extracting files from multiple archive files.
To extract the files from multiple archive files use the -M option with each -f option. This is shown below:
> tar -xv -Mf archive.tar -Mf unix_files.tar
you see creating tar file with gzip is very easy just use “-z” option and it will crate a gzip tar.tgz or tar.gz extension
121.Calculating size of tar file in UNIX
Some time its useful to know the size of tar file before creating it and you can get it by using unix tar command as shown in below example:
stock_trader@system:~/test tar -cf – * | wc -c
20480
Size shown here is in KB and you can also calculate size for compressed tar file by using “z” for gzip and “j” for bzip2
LS
122. Write a unix/linux ls command to display the hidden files and directories?
To display the hidden files and directories in the current directory use the -a option of the ls command.
> ls -a . .. documents .hidden_file sum.pl
Hidden files are the one whose name starts with dot (.). The las -a displays the current directory (.) and parent directory (..) also. If you want to exclude the current directory, parent directory, then use -A option.
> ls -A documents .hidden_file sum.pl
.List the hidden files in current directory ?
– ls -a1 | grep “^\.”
. How to find hidden files in current directory?
$ ls -lrta
123. Write a unix/linux ls command to classify the files with special characters
The -F option to ls command classifies the files. It marks the
- Directories with trailing slash (/)
- Executable files with trailing asterisk (*)
- FIFOs with trailing vertical bar (|)
- Symbolic links with trailing at the rate sign (@)
- Regular files with nothing
> ls -F documents/ sum.pl link@
124. Write a unix/linux ls command to print each file in a separate line?
The -1 option to the ls command specifies that each file should be displayed on a separate line
> ls -1 documents sum.pl
125. Write a unix/linux ls command to display the inode number of file?
Use -i option to the ls command to print the inode number of a file.
> ls -i1 10584066 documents 3482450 sum.pl
126. Write a unix/linux ls command to display complete information about the files?
The -l option provides lots of information about the file type, owner, group, permissions, file size, last modification date.
> ls -l total 16 drwxr-xr-x 2 matt db 4096 Jan 30 23:08 documents -rw-r--r-- 1 matt db 49 Jan 31 01:17 sum.pl
- The first character indicates the type of the file. – for normal file, d for directory, l for link file and s for socket file
- The next 9 characters in the first field represent the permissions. Each 3 characters refers the read (r), write (w), execute (x) permissions on owner, group and others. – means no permission.
- The second field indicates the number of links to that file.
- The third field indicates the owner name.
- The fourth field indicates the group name.
- The fifth field represents the file size in bytes.
- The sixth field represents the last modification date and time of the file.
- And finally the seventh field is the name of the file.
127. Write a unix/linux ls command to sort the files by their modification time?
The -t option allows the ls command to sort the files in descending order based on the modification time.
> ls -t1 sum.pl documents
128. Write a unix/linux ls command to sort the files in ascending order of modification time?
The -r option reverses the order of the files displayed. Combine the -t and -r options to sort the files in ascending order.
> ls -rt1 documents sum.pl
129. Write a unix/linux ls command to print the files recursively?
So far the ls command prints the files in the current directory. Use the -R option to recursively print the files in the sub-directories also.
> ls -R .: documents sum.pl ./documents: file.txt
130. Write a unix/linux ls command to print the files in a specific directory?
You can pass a directory to the ls command as an argument to print for the files in it.
> ls /usr/local/bin
131. Write a unix/linux ls command to display files in columns?
The -x option specifies the ls command to display the files in columns.
> ls -x
132.List the files in current directory sorted by size ?
– ls -l | grep ^- | sort -nr
133.Display the files in the directory by file size ?
– ls .ltr | sort .nr .k 5
134. How to display zero byte size files?
ls -l | grep '^-' | awk '/^-/ {if ($5 !=0 ) print $9 }'
135. Write a command to find the sum of bytes (size of file) of all files in a directory.
ls -l | grep '^-'| awk 'BEGIN {sum=0} {sum = sum + $5} END {print sum}'
136. Write a command to display the files in the directory by file size?
ls -l | grep '^-' |sort -nr -k 5
137. Write a command to find the number of files in a directory.
ls -l|grep '^-'|wc -l
COPY
138. Write a unix/linux cp command to copy file in to a directory?
The basic usage of cp command is to copy a file from the current directory to another directory.
cp sum.pl tmp/
The cp command copies the file sum.pl into the tmp directory. The cp command does not remove the source file. It just copies the file into a new location. If a file with the same name as the source exists in the destination location, then by default the cp command overwrites that new file
139. Write a unix/linux cp to prompt for user before overwriting a file ( Interactive cp command)?
The -i option to the cp command provides the ability to prompt for a user input whether to overwrite the destination file or not.
> cp sum.pl tmp/ cp: overwrite `tmp/sum.pl'?
If you enter y, then the cp command overwrites the destination file, otherwise the cp command does not copy the file.
140. Write a unix/linux cp command to copy multiple files in to a new directory?
You can specify multiple files as the source and can copy to the new location.
cp log.dat bad.dat tmp/
The cp command copies the log.dat, bad.dat files in the current directory to the tmp directory.
141. Write a unix/linux cp command to do a Regular expression copy?
You can copy a set of files by specifying a regular expression pattern.
cp *.dat tmp/
Here the cp command copies all the files which has “dat” as suffix to the destination directory.
142. Write a unix/linux cp command to copy a file in to the current directory?
You can copy a file from a different directory to the current directory.
cp /usr/local/bin/multiply.sh .
Here the cp command copies the multiply.sh file in the /usr/local/bin directory the current directory. The dot (.) indicates the current directory.
143. Write a unix/linux cp command to copy all the files in a directory?
The cp command can be used to copy all the files in directory to another directory.
cp docs/* tmp/
This command copies all the files in the docs directory to the tmp directory.
144. Write a unix/linux cp command to copy files from multiple directories?
You can copy the files from different directories into a new location.
cp docs/* scripts/* tmp/
The command copies the files from docs and script directories to the destination directory tmp.
145. Write a unix/linux cp command to Copy a directory.
You can recursively copy a complete directory and its sub directory to another location using the cp command
cp -r docs tmp/
This copies the complete directory docs into the new directory tmp
146. Write a unix/linux cp command to Forcibly copy a file with -f option?
You can force the cp command to copy an existing destination file even it cannot be opened.
cp -f force_file.txt /var/tmp/
CD
cd [directory] Here directory is the name of the directory where you wish to go.
147. Write a unix/linux cd command to change to home directory?
Just simply type cd command on the unix terminal and then press the enter key. This will change your directory to home directory.
> pwd /usr/local/bin
Now i am in the /usr/local/bin directory. After typing the cd command and unix window, you will go to your home directory.
> cd > pwd /home/matt
Here pwd command displays the present working directory.
148. Write a unix/linux cd command to go back to one directory?
The cd .. changes the directory to its parent directory by going back one level. The space between the cd and .. is must.
> pwd /var/tmp > cd .. > pwd /var
149. Write a unix/linux cd command to go back to two directories?
The cd ../../ takes you back to two directories. You can extend this cd command to go back to n number of directories.
> pwd /usr/local/bin > cd ../../ > pwd /usr
150. Write a unix/linux cd command to change the directory using the absolute path?
In case of changing directory using absolute path you have to specify the full directory path. Absolute path directories always start with a slash (/). An example is changing your directory to /usr/bin from your home directory.
> cd /usr/bin
151. Write a unix/linux cd command to change the directory using the relative path?
In relative path, you have to specify the directory path relative to your current directory. For example, you are in /var/tmp directory and you want to go to /var/lib directory, then you can use the relative path.
> pwd /var/tmp > cd ../lib > pwd /var/lib
Here the cd ../lib, first takes you to the parent directory which is /var and then changes the directory to the lib.
152. Write a unix/linux cd command to change back to previous directory.
As an example, i am in the directory /home/matt/documents and i changed to a new directory /home/matt/backup. Now i want to go back to my previous directory /home/matt/documents. In this case, you can use the cd – command to go back to the previous directory.
> pwd /home/matt/documents > cd /home/matt/backup >pwd /home/matt/backup > cd - > pwd /home/matt/documents
153. Write a unix/linux command to remove a directory?
The rmdir command deletes only the empty directories. If a directory contains files or sub directories, then the rmdir command fails.
rmdir docs/ rmdir: docs/: Directory not empty
Here the docs directory is not empty, that is why the rmdir command failed to remove the directory. To remove the docs directory first we have to make the directory empty and then delete the directory.
rm doc/* rmdir docs/
154. Write a unix/linux command to remove the directory and its parent directories?
As mentioned earlier the -p option allows the rmdir command to delete the directory and also its parent directories.
rmdir -p docs/entertainment/movies/
This rmdir command removes the docs directory completely. If you don’t use the -p option, then it only deletes the movies directory.
155. Write a unix/linux command to remove directories using pattern matching?
You can specify the directory names using the regular expressions and can delete them.
rm doc*
This rm command deletes the directories like doc, documents, doc_1 etc.
Now we will see the rm command in unix.
Unix rm command syntax
The syntax of rm command is
rm [options] [directory|file]
The rm command options are
f : Removes all files in a directory without prompting the user. i : Interactive: prompts the user for confirmation before deleting a file. R or r : Recursively remove directories and sub directories.
The rm command can be used to delete both the files and directories. The rm command also deletes the non-empty directories.
Unix rm command examples
156. Write a unix/linux command to remove a file?
This is the basic feature of rm command. To remove a file, logfile.dat, in the current directory use the below rm command
rm logfile.dat
157. Write a unix/linux command to remove all the files in a directory?
use the * regular pattern as the file list in rm command for deleting all the files in the current directory.
rm *
158. Write a unix/linux command to delete empty directory?
The rm command can also be used to delete the empty directory. The command for this is
rm docs/
If the directory is non-empty, then the above command fails to remove the directories.
159. Write a unix/linux command to delete directories recursively (delete non empty directories)?
As mentioned earlier, the -r option can be used to remove the directories and sub directories.
rm -r docs
MOVE
160.How to rename a file or directory in unix (or linux) and how to move a file or directory from the current directory to another directory?
Unix provides a simple mv (move) command which can be used to rename or move files and directories. The syntax of mv command is
mv [options] oldname newname
The options of mv command are
f : Do not prompt before overwriting a file. i : Prompts for the user input before overwriting a file.
If the newname already exists, then the mv command overwrites that file. Let see some examples on how to use mv command.
161. Write a unix/linux command to rename a file?
Renaming a file is one of the basic features of the mv command. To rename a file from “log.dat” to “bad.dat”, use the below mv command
> mv log.dat bad.dat
Note that if the “bad.dat” file already exists, then its contents will be overwritten by “log.dat”. To avoid this use the -i option, which prompts you before overwriting the file.
mv -i log.dat bad.dat mv: overwrite `bad.dat'?
162. Write a unix/linux command to rename a directory?
Just as renaming a file, you can use the mv command to rename a directory. To rename the directory from docs to documents, run the below command
mv docs/ documents/
If the documents directory already exists, then the docs directory will be moved in to the documents directory.
163. Write a unix/linux command to move a file into another directory?
The mv command can also be used to move the file from one directory to another directory. The below command moves the sum.pl file in the current directory to /var/tmp directory.
mv sum.pl /var/tmp/
If the sum.pl file already exists in the /var/tmp directory, then the contents of that file will be overwritten.
164. Write a unix/linux command to move a directory in to another directory?
Just as moving a file, you can move a directory into another directory. The below mv command moves the documents directory into the tmp directory
mv documents /tmp/
165. Write a unix/linux command to move all the files in the current directory to another directory?
You can use the regular expression pattern * to move all the files from one directory to another directory.
mv * /var/tmp/
The above command moves all the files and directories in the current directory to the /var/tmp/ directory.
. mv *
What happens if you simply type mv * and then press enter?
It depends on the files you have in the directory. The * expands to all the files and directories. Three scenarios are possible.
- If the current directory has only files, then the contents of all the files (except one file) will be written in to the one file. The one file is the last file which depends on the pattern *.
- If the current directory contains only directories, then all the directories (except one directory) will be moved to another directory.
- If the current directory contains both files and directories, then it depends on the expansion of the *. If the pattern * gives the last one as directory then all the files will be moved to that directory. Otherwise the mv command will fail.
Some Tips:
- Try to avoid mv *
- Avoid moving large number of files.
XARGS
Xargs command in unix or linux operating system is used to pass the output of one command as an argument to another command. Some of the unix or linux commands like ls and find produces a long list of filenames. We want to do some operation on this list of file names like searching for a pattern, removing and renaming files etc. The xargs command provide this capability by taking the huge list of arguments as input , divides the list into small chunks and then passes them as arguments to other unix commands.
171. Renaming files with xargs
We have to first list the files to be renamed either by using the ls or find command and then pipe the output to xargs command to rename the files. First list the files which end with “.log” using the ls command.
ls *.log oracle.log storage.log
> ls *.log | xargs -i mv {} {}_bkp > ls *_bkp oracle.log_bkp storage.log_bkp
You can see how the log files are renamed with backup (bkp) suffix. Here the option “i” tells the xargs command to replace the {} with the each file returned by the ls command.
172. Searching for a pattern
We can combine the grep command with xargs to search for a pattern in a list of files returned by another unix command (ls or find). Let’s list out all the bash files in the current directory with the find command in unix.
find . -name "*.bash" ./sql_server.bash ./mysql_backup.bash ./oracle_backup.bash
Now we grep for the “echo” statements from the list of files returned by the find command with the help of xargs. The command is shown below:
find . -name "*.bash" |xargs grep "echo"
If you don’t use xargs and piped the output of find command to grep command directly, then the grep command treats each file returned by the find command as a line of string and searches for the word “echo” in that line rather in that file.
173. Removing files using xargs
We can remove the temporary files in a directory using the rm command along with the xargs command. This is shown below:
ls "*.tmp" | xargs rm
This removes all the files with “.tmp” suffix.
174. Converting Multi-line output to Single line output.
If you run the ls -1 command, it will list each file on a separate line. This is shown below:
ls -1 oracle.txt online_backup.dat mysql_storage.bat
We can convert this multi-line output to single line output using the xargs command. This is shown below:
ls -1 | xargs oracle.txt online_backup.dat mysql_storage.bat
175. Handling spaces in file names
By default the xargs command treats the space as a delimiter and sends each item as an argument to the unix command. If the file name contains a space (example: “oracle storage”), then each item will be treated as a separate file and will be passed as an argument. This will cause an issue. Let see how to handle the spaces in file names with an example.
Creating a file which contains space in the name > touch "oracle storage" > ls oracle\ storage | xargs grep "log" grep: oracle: No such file or directory grep: storage: No such file or directory
You can see that grep command is treating oracle as separate file and storage as separate file. This is because of xargs treats space as delimiter. To avoid this kind of errors use the -i option with braces as shown in below:
> ls oracle\ storage | xargs -i grep "log" {}
If you want to know what command the xargs is executing use the -t option with xargs. This will print the command on the terminal before executing it.
176. Passing subset of arguments
We can pass only a subset of arguments from a long list of arguments using the -n option with xargs command. This is shown in below.
> ls -1 backup mysql network online oracle storage wireless > ls -1 | xargs -n 3 echo backup mysql network online oracle storage wireless
You can see from the above output that 3 arguments are passed at a time to the echo statement.
Important Notes on Xargs Command:
. Xargs directly cannot handle files which contain new lines or spaces in their names. To handle this kind of files use the -i option with xargs command. Another way to handle these characters is to treat the new line or spaces as null characters using th -0 option with xargs. However this requires that the input to xargs should also use the null as separator. An example is shown below
find . -print0 | xargs -0 rm
The print0 in find command makes the newline or space characters as null separator.
. By default the xargs uses the end of the file string as “_”. If this string appears in input string, then xargs command stops reading the input and rest of the input is ignored. You can change the end of file string by using the “-eof” option.
. To know more about xargs command, run the xargs –help on the unix or linux terminal.
AWK
189. Inserting a new line after every 2 lines
We will see how to implement this using the awk command with an example.
The input “file.txt” contains the below data:
1 A 2 B 3 C 4 D 5 E 6 F
Let say, we want to insert the new line “9 Z” after every two lines in the input file. The required output data after inserting a new line looks as
1 A 2 B 9 Z 3 C 4 D 9 Z 5 E 6 F 9 Z
The awk command for getting this output is
awk '{ if(NR%2 == 0) { print $0"\n9 Z"; } else { print $0 } }' file.txt
190. Replace the Nth occurrence of a pattern
The input file contains the data.
AAA 1 BBB 2 CCC 3 AAA 4 AAA 5 BBB 6 CCC 7 AAA 8 BBB 9 AAA 0
Now we want to replace the fourth occurrence of the first filed “AAA” with “ZZZ” in the file.
The required output is:
AAA 1 BBB 2 CCC 3 AAA 4 AAA 5 BBB 6 CCC 7 ZZZ 8 BBB 9 AAA 0
The awk command for getting this output is
awk 'BEGIN {count=0} { if($1 == "AAA") { count++ } if(count == 4) { sub("AAA","ZZZ",$1) } } { print $0 }' file.txt
191. Find the sum of even and odd lines separately
The input file data:
A 10 B 39 C 22 D 44 E 75 F 89 G 67
You have to get the second field and then find the sum the even and odd lines.
The required output is
174, 172
The awk command for producing this output is
awk '{ if(NR%2 == 1) { sum_e = sum_e + $2 } else { sum_o = sum_o + $2 } } END { print sum_e,sum_o }' file.txt
192. Fibonacci series using awk command
Now we will produce the Fibonacci series using the awk command.
awk ' BEGIN{ for(i=0;i<=10;i++) { if (i <=1 ) { x=0; y=1; print i; } else { z=x+y; print z; x=y; y=z; } } }'
The output is
0 1 1 2 3 5 8 13 21 34 55
193. Remove leading zeros from a file using the awk command. The input file contains the below data.
0012345 05678 01010 00001
After removing the leading zeros, the output should contain the below data.
12345 5678 1010 1
The awk command for this is.
awk '{print $1 + 0}' file.txt awk '{printf "%d\n",$0}' file.txt
awk -f filename
awk ‘BEGIN { for(i=1;i<=10;i++) {print “square of”,i,”is”,i*i;}}’
ls -l | awk ‘BEGIN {sum=0} {sum = sum + $5} END {print sum}’
awk ‘{print NR, $0}’ filename
awk ‘BEGIN {RS=””;FS=”\n”} {print $2,$3}’ filename
ls -l | awk ‘/^-/ {if ($5 !=0 ) print $9 }’
201. Write a command to rename the files in a directory with “_new” as postfix?
ls -F | awk ‘{print “mv “$1” “$1″.new”}’ | sh
202. Write a command to print the fields in a text file in reverse order?
awk ‘BEGIN {ORS=””} { for(i=NF;i>0;i–) print $i,” “; print “\n”}’ filename
203. Write a command to find the total number of lines in a file without using NR
awk ‘BEGIN {sum=0} {sum=sum+1} END {print sum}’ filename
Another way to print the number of lines is by using the NR. The command is
awk ‘END{print NR}’ filename
AWK
. How to print the squares of numbers from 1 to 10 using awk command
awk 'BEGIN { for(i=1;i<=10;i++) {print "square of",i,"is",i*i;}}'
.What does the following command line produce? Explain each aspect of this line.
$ (date ; ps -ef | awk ‘{print $1}’ | sort | uniq | wc -l ) >> Activity.log
A: First let’s dissect the line: The date gives the date and time as the first command of the line, this is followed by the a list of all running processes in long form with UIDs listed first, this is the ps -ef. These are fed into the awk which filters out all but the UIDs; these UIDs are piped into sort for no discernible reason and then onto uniq (now we see the reason for the sort – uniq only works on sorted data – if the list is A, B, A, then A, B, A will be the output of uniq, but if it’s A, A, B then A, B is the output) which produces only one copy of each UID.
These UIDs are fed into wc -l which counts the lines – in this case the number of distinct UIDs running processes on the system. Finally the results of these two commands, the date and the wc -l, are appended to the file “Activity.log”. Now to answer the question as to what this command line produces. This writes the date and time into the file Activity.log together with the number of distinct users who have processes running on the system at that time. If the file already exists, then these items are appended to the file, otherwise the file is created.
. How to run awk command specified in a file?
awk -f filename
. In the text file, some lines are delimited by colon and some are delimited by space. Write a command to print the third field of each line.
awk '{ if( $0 ~ /:/ ) { FS=":"; } else { FS =" "; } print $3 }' filename
. Write a command to print the line number before each line?
awk '{print NR, $0}' filename
. Write a command to print the second and third line of a file without using NR.
awk 'BEGIN {RS="";FS="\n"} {print $2,$3}' filename
. Write a command to find the total number of lines in a file?
wc -l filename
Other ways to print the total number of lines are
awk 'BEGIN {sum=0} {sum=sum+1} END {print sum}' filename awk 'END{print NR}' filename
. How to display the fields in a text file in reverse order?
awk 'BEGIN {ORS=""} { for(i=NF;i>0;i--) print $i," "; print "\n"}' filename
. Write a command to convert a string to Initcap.
echo apple | awk '{print toupper(substr($1,1,1)) tolower(substr($1,2))}'
SED
>cat file.txt unix is great os. unix is opensource. unix is free os. learn operating system. unixlinux which one you choose.
186. Replacing or substituting string
Sed command is mostly used to replace the text in a file. The below simple sed command replaces the word “unix” with “linux” in the file.
>sed 's/unix/linux/' file.txt linux is great os. unix is opensource. unix is free os. learn operating system. linuxlinux which one you choose.
Here the “s” specifies the substitution operation. The “/” are delimiters. The “unix” is the search pattern and the “linux” is the replacement string.
By default, the sed command replaces the first occurrence of the pattern in each line and it won’t replace the second, third…occurrence in the line.
187. Replacing the nth occurrence of a pattern in a line.
Use the /1, /2 etc flags to replace the first, second occurrence of a pattern in a line. The below command replaces the second occurrence of the word “unix” with “linux” in a line.
>sed 's/unix/linux/2' file.txt unix is great os. linux is opensource. unix is free os. learn operating system. unixlinux which one you choose.
188. Replacing all the occurrence of the pattern in a line.
The substitute flag /g (global replacement) specifies the sed command to replace all the occurrences of the string in the line.
>sed 's/unix/linux/g' file.txt linux is great os. linux is opensource. linux is free os. learn operating system. linuxlinux which one you choose.
189. Replacing from nth occurrence to all occurrences in a line.
Use the combination of /1, /2 etc and /g to replace all the patterns from the nth occurrence of a pattern in a line. The following sed command replaces the third, fourth, fifth… “unix” word with “linux” word in a line.
>sed 's/unix/linux/3g' file.txt unix is great os. unix is opensource. linux is free os. learn operating system. unixlinux which one you choose.
190. Changing the slash (/) delimiter
You can use any delimiter other than the slash. As an example if you want to change the web url to another url as
>sed 's/http:\/\//www/' file.txt
In this case the url consists the delimiter character which we used. In that case you have to escape the slash with backslash character, otherwise the substitution won’t work.
Using too many backslashes makes the sed command look awkward. In this case we can change the delimiter to another character as shown in the below example.
>sed 's_http://_www_' file.txt >sed 's|http://|www|' file.txt
191. Using & as the matched string
There might be cases where you want to search for the pattern and replace that pattern by adding some extra characters to it. In such cases & comes in handy. The & represents the matched string.
>sed 's/unix/{&}/' file.txt {unix} is great os. unix is opensource. unix is free os. learn operating system. {unix}linux which one you choose. >sed 's/unix/{&&}/' file.txt {unixunix} is great os. unix is opensource. unix is free os. learn operating system. {unixunix}linux which one you choose.
192. Using \1,\2 and so on to \9
The first pair of parenthesis specified in the pattern represents the \1, the second represents the \2 and so on. The \1,\2 can be used in the replacement string to make changes to the source string. As an example, if you want to replace the word “unix” in a line with twice as the word like “unixunix” use the sed command as below.
>sed 's/\(unix\)/\1\1/' file.txt unixunix is great os. unix is opensource. unix is free os. learn operating system. unixunixlinux which one you choose.
The parenthesis needs to be escaped with the backslash character. Another example is if you want to switch the words “unixlinux” as “linuxunix”, the sed command is
>sed 's/\(unix\)\(linux\)/\2\1/' file.txt unix is great os. unix is opensource. unix is free os. learn operating system. linuxunix which one you choose.
Another example is switching the first three characters in a line
>sed 's/^\(.\)\(.\)\(.\)/\3\2\1/' file.txt inux is great os. unix is opensource. unix is free os. aelrn operating system. inuxlinux which one you choose.
193. Duplicating the replaced line with /p flag
The /p print flag prints the replaced line twice on the terminal. If a line does not have the search pattern and is not replaced, then the /p prints that line only once.
>sed 's/unix/linux/p' file.txt linux is great os. unix is opensource. unix is free os. linux is great os. unix is opensource. unix is free os. learn operating system. linuxlinux which one you choose. linuxlinux which one you choose.
194. Printing only the replaced lines
Use the -n option along with the /p print flag to display only the replaced lines. Here the -n option suppresses the duplicate rows generated by the /p flag and prints the replaced lines only one time.
>sed -n 's/unix/linux/p' file.txt linux is great os. unix is opensource. unix is free os. linuxlinux which one you choose.
If you use -n alone without /p, then the sed does not print anything.
195. Running multiple sed commands.
You can run multiple sed commands by piping the output of one sed command as input to another sed command.
>sed 's/unix/linux/' file.txt| sed 's/os/system/' linux is great system. unix is opensource. unix is free os. learn operating system. linuxlinux which one you chosysteme.
Sed provides -e option to run multiple sed commands in a single sed command. The above output can be achieved in a single sed command as shown below.
>sed -e 's/unix/linux/' -e 's/os/system/' file.txt linux is great system. unix is opensource. unix is free os. learn operating system. linuxlinux which one you chosysteme.
196. Replacing string on a specific line number.
You can restrict the sed command to replace the string on a specific line number. An example is
>sed '3 s/unix/linux/' file.txt unix is great os. unix is opensource. unix is free os. learn operating system. linuxlinux which one you choose.
The above sed command replaces the string only on the third line.
197. Replacing string on a range of lines.
You can specify a range of line numbers to the sed command for replacing a string.
>sed '1,3 s/unix/linux/' file.txt linux is great os. unix is opensource. unix is free os. learn operating system. linuxlinux which one you choose.
Here the sed command replaces the lines with range from 1 to 3. Another example is
>sed '2,$ s/unix/linux/' file.txt linux is great os. unix is opensource. unix is free os. learn operating system. linuxlinux which one you choose.
Here $ indicates the last line in the file. So the sed command replaces the text from second line to last line in the file.
198. Replace on a lines which matches a pattern.
You can specify a pattern to the sed command to match in a line. If the pattern match occurs, then only the sed command looks for the string to be replaced and if it finds, then the sed command replaces the string.
>sed '/linux/ s/unix/centos/' file.txt unix is great os. unix is opensource. unix is free os. learn operating system. centoslinux which one you choose.
Here the sed command first looks for the lines which has the pattern “linux” and then replaces the word “unix” with “centos”.
199. Deleting lines.
You can delete the lines a file by specifying the line number or a range or numbers.
>sed '2 d' file.txt >sed '5,$ d' file.txt
200. Duplicating lines
You can make the sed command to print each line of a file two times.
>sed 'p' file.txt
201. Sed as grep command
You can make sed command to work as similar to grep command.
>grep 'unix' file.txt >sed -n '/unix/ p' file.txt
Here the sed command looks for the pattern “unix” in each line of a file and prints those lines that has the pattern.
You can also make the sed command to work as grep -v, just by using the reversing the sed with NOT (!).
>grep -v 'unix' file.txt >sed -n '/unix/ !p' file.txt
The ! here inverts the pattern match.
202. Add a line after a match.
The sed command can add a new line after a pattern match is found. The “a” command to sed tells it to add a new line after a match is found.
>sed '/unix/ a "Add a new line"' file.txt unix is great os. unix is opensource. unix is free os. "Add a new line" learn operating system. unixlinux which one you choose. "Add a new line"
203. Add a line before a match
The sed command can add a new line before a pattern match is found. The “i” command to sed tells it to add a new line before a match is found.
>sed '/unix/ i "Add a new line"' file.txt "Add a new line" unix is great os. unix is opensource. unix is free os. learn operating system. "Add a new line" unixlinux which one you choose.
204. Change a line
The sed command can be used to replace an entire line with a new line. The “c” command to sed tells it to change the line.
>sed '/unix/ c "Change line"' file.txt "Change line" learn operating system. "Change line"
205. Transform like tr command
The sed command can be used to convert the lower case letters to upper case letters by using the transform “y” option.
>sed 'y/ul/UL/' file.txt Unix is great os. Unix is opensoUrce. Unix is free os. Learn operating system. UnixLinUx which one yoU choose.
Here the sed command transforms the alphabets “ul” into their uppercase format “UL”
Sed Command to Delete Lines: Sed command can be used to delete or remove specific lines which matches a given pattern or in a particular position in a file. Here we will see how to delete lines using sed command with various examples.
The following file contains a sample data which is used as input file in all the examples:
> cat file linux unix fedora debian ubuntu
Sed Command to Delete Lines – Based on Position in File
In the following examples, the sed command removes the lines in file that are in a particular position in a file.
206. Delete first line or header line
The d option in sed command is used to delete a line. The syntax for deleting a line is:
> sed 'Nd' file
Here N indicates Nth line in a file. In the following example, the sed command removes the first line in a file.
> sed '1d' file unix fedora debian ubuntu
207. Delete last line or footer line or trailer line
The following sed command is used to remove the footer line in a file. The $ indicates the last line of a file.
> sed '$d' file linux unix fedora debian
208. Delete particular line
This is similar to the first example. The below sed command removes the second line in a file.
> sed '2d' file linux fedora debian ubuntu
209. Delete range of lines
The sed command can be used to delete a range of lines. The syntax is shown below:
> sed 'm,nd' file
Here m and n are min and max line numbers. The sed command removes the lines from m to n in the file. The following sed command deletes the lines ranging from 2 to 4:
> sed '2,4d' file linux ubuntu
210. Delete lines other than the first line or header line
Use the negation (!) operator with d option in sed command. The following sed command removes all the lines except the header line.
> sed '1!d' file linux
211. Delete lines other than last line or footer line
> sed '$!d' file ubuntu
212. Delete lines other than the specified range
> sed '2,4!d' file unix fedora debian
Here the sed command removes lines other than 2nd, 3rd and 4th.
213. Delete first and last line
You can specify the list of lines you want to remove in sed command with semicolon as a delimiter.
> sed '1d;$d' file unix fedora debian
214. Delete empty lines or blank lines
> sed '/^$/d' file
The ^$ indicates sed command to delete empty lines. However, this sed do not remove the lines that contain spaces.
whitspaces escape special character names
Sed Command to Delete Lines – Based on Pattern Match
In the following examples, the sed command deletes the lines in file which match the given pattern.
215. Delete lines that begin with specified character
> sed '/^u/d' file linux fedora debian
^ is to specify the starting of the line. Above sed command removes all the lines that start with character ‘u’.
216. Delete lines that end with specified character
> sed '/x$/d' file fedora debian ubuntu
$ is to indicate the end of the line. The above command deletes all the lines that end with character ‘x’.
217. Delete lines which are in upper case or capital letters
> sed '/^[A-Z]*$/d' file
218. Delete lines that contain a pattern
> sed '/debian/d' file linux unix fedora ubuntu
219. Delete lines starting from a pattern till the last line
> sed '/fedora/,$d' file linux unix
Here the sed command removes the line that matches the pattern fedora and also deletes all the lines to the end of the file which appear next to this matching line.
220. Delete last line only if it contains the pattern
> sed '${/ubuntu/d;}' file linux unix fedora debian
Here $ indicates the last line. If you want to delete Nth line only if it contains a pattern, then in place of $ place the line number.
Note: In all the above examples, the sed command prints the contents of the file on the unix or linux terminal by removing the lines. However the sed command does not remove the lines from the source file. To Remove the lines from the source file itself, use the -i option with sed command.
> sed -i '1d' file
If you dont wish to delete the lines from the original source file you can redirect the output of the sed command to another file.
sed '1d' file > newfile
sed s/bad/good/ < filename
sed s/bad/good/g < filename
sed ‘s/\//,/’ < filename
sed ‘s|/|,|’ < filename 224.Write a command to replace the word “apple” with “(apple)” in a file?
sed s/apple/(&)/ < filename 225.Write a command to switch the two consecutive words “apple” and “mango” in a file?
sed ‘s/\(apple\) \(mango\)/\2 \1/’ < filename226.Write a command to replace the second occurrence of the word “bat” with “ball” in a file?
sed ‘s/bat/ball/2’ < filename227. Write a command to remove all the occurrences of the word “jhon” except the first one in a line with in the entire file?
sed ‘s/jhon//2g’ < filename228. Write a command to remove the first number on line 5 in file?
sed ‘5 s/[0-9][0-9]*//’ < filename229. Write a command to remove the first number on all lines that start with “@”?
sed ‘\,^@, s/[0-9][0-9]*//’ < filename
230. Write a command to replace the word “gum” with “drum” in the first 100 lines of a file?
sed ‘1,00 s/gum/drum/’ < filename
231. write a command to replace the word “lite” with “light” from 100th line to last line in a file?
sed ‘100,$ s/lite/light/’ < filename
232. Write a command to remove the first 10 lines from a file?
sed ‘1,10 d’ < filename
233. Write a command to duplicate each line in a file?
sed ‘p’ < filename
234. Write a command to duplicate empty lines in a file?
sed ‘/^$/ p’ < filename
235. Write a sed command to print the lines that do not contain the word “run”?
sed -n ‘/run/!p’ < filename
SED
How to remove the first 10 lines from a file?
sed '1,10 d' < filename
318. Write a command to duplicate each line in a file?
sed 'p' < filename
327. How to replace the word “Gun” with “Pen” in the first 100 lines of a file?
sed '1,00 s/Gun/Pen/' < filename
sed 's/bat/ball/2' < filename
335. How to remove all the occurrences of the word “jhon” except the first one in a line with in the entire file?
sed 's/jhon//2g' < filename
336. How to replace the word “lite” with “light” from 100th line to last line in a file?
sed '100,$ s/lite/light/' < filename
348. Write a command to replace the word “bad” with “good” in file?
sed s/bad/good/ < filename
349. Write a command to replace the word “bad” with “good” globally in a file?
sed s/bad/good/g < filename
350. Write a command to replace the word “apple” with “(apple)” in a file?
sed s/apple/(&)/ < filename
351. Write a command to switch the two consecutive words “apple” and “mango” in a file?
sed 's/\(apple\) \(mango\)/\2 \1/' < filename
280. How to remove the header from a file?
sed -i '1 d' filename
281. How to remove the footer from a file?
sed -i '$ d' filename
282. Write a command to find the length of a line in a file?
The below command can be used to get a line from a file.
sed –n '<n> p' filename
We will see how to find the length of 10th line in a file
sed -n '10 p' filename|wc -c
292. How to display the first 20 lines of a file? By default, the head command displays the first 10 lines from a file. If we change the option of head, then we can display as many lines as we want.
head -20 filename
An alternative solution is using the sed command
sed '21,$ d' filename
The d option here deletes the lines from 21 to the end of the file
293. Write a command to print the last line of a file?
The tail command can be used to display the last lines from a file.
tail -1 filename
Alternative solutions are:
sed -n '$ p' filename awk 'END{print $0}' filename
286. How to replace the n-th line in a file with a new line in Unix?
sed -i'' '10 d' filename # d stands for delete sed -i'' '10 i new inserted line' filename # i stands for insert
301. How do you remove the first number on 10th line in file?
sed '10 s/[0-9][0-9]*//' < filename
307. Write a command to remove the first number on all lines that start with “@”?
sed '\,^@, s/[0-9][0-9]*//' < filename
312. How to duplicate empty lines in a file?
sed '/^$/ p' < filename
340. How to replace the character ‘/’ with ‘,’ in a file?
sed 's/\//,/' < filename sed 's|/|,|' < filename
.Display the last newly appending lines of a file during appendingdata to the same file by some processes ?
– tail .f Debug.logHere tail shows the newly appended data into Debug.log by some processes/user.
Explain iostat, vmstat and netstat?
-
- Iostat: reports on terminal, disk and tape I/O activity.
- Vmstat: reports on virtual memory statistics for processes, disk, tape and CPU activity.
- Netstat: reports on the contents of network data structures.
. How to find out the usage of the CPU by the processes?
The top utility can be used to display the CPU usage by the proce
. Write a command to remove the prefix of the string ending with ‘/’.
The basename utility deletes any prefix ending in /. The usage is mentioned below:
basename /usr/local/bin/file
This will display only file
.Display the Disk Usage of file sizes under each directory in currentDirectory ?
– du -k * | sort .nr (or) du .k . | sort -nr
.Change to a directory, which is having very long name ?
– cd CDMA_3X_GEN*Here original directory name is . .CDMA_3X_GENERATION_DATA..
.Set the Display automatically for the current new user ?
– export DISPLAY=`eval ‘who am i | cut -d”(” -f2 | cut -d”)” -f1′`Here in above command, see single quote, double quote, grave ascent is used. Observe carefully.
.Display the processes, which are running under yourusername ?
– ps .aef | grep MaheshvjHere, Maheshvj is the username.
.List some Hot Keys for bash shell ?
– Ctrl+l . Clears the Screen. Ctrl+r . Does a search in previously given commands in shell. Ctrl+u – Clears the typing before the hotkey. Ctrl+a . Places cursor at the beginning of the command at shell. Ctrl+e . Places cursor at the end of the command at shell. Ctrl+d . Kills the shell. Ctrl+z . Places the currently running process into background.
.How to save man pages to a file ?
– man <command> | col .b > <output-file>Example : man top | col .b > top_help.txt
.How to know the date & time for . when script is executed ?
– Add the following script line in shell script.eval echo “Script is executed at `date`” >> timeinfo.infHere, .timeinfo.inf. contains date & time details ie., when script is executed and history related to execution.
.How do you find out drive statistics ?
– iostat -E
.Display disk usage in Kilobytes ?
– du -k
.Display top ten largest files/directories
– du -sk * | sort -nr | head
.How much space is used for users in kilobytes ?
– quot -af
.How to create null file ?
– cat /dev/null > filename1
.Access common commands quicker ?
– ps -ef | grep -i $@
Display the page size of memory ?
– pagesize -a
.Display Ethernet Address arp table ?
– arp -a
.Display the no.of active established connections to localhost ?
– netstat -a | grep EST
259.Display the state of interfaces used for TCP/IP traffice ?
– netstat -i
.Display the parent/child tree of a process ?
– ptree <pid> Example: ptree 1267
.Show the working directory of a process ?
– pwdx <pid> Example: pwdx 1267
.Display the processes current open files ?
– pfiles <pid> Example: pfiles 1267
.Display the inter-process communication facility status ?
– ipcs
.Display the top most process utilizing most CPU ?
– top .b 1
.Alternative for top command ?
– prstat -a
.What is the most graceful way to get to run level single user mode?
A: The most graceful way is to use the command init s.
If you want to shut everything down before going to single user mode then do init 0 first and from the ok prompt do a boot -s.
.How to find current running processes in unix server?
$ ps -ef
and if we want to find specific process we can use ‘grep’ with pipe
$ ps -ef | grep -i ‘application’
.How to find process which is taking maximum memory in server?
$ top
top command tell us about cpu usage , process id and other details. below is output of top command
.How do you access command line arguments from within a shell script?
.How to tails last 200 lines of any log fine?
$ tail -200f filename.txt
. How to kill process in unix server?
$ kill -9 #pid
these #pid can be found using ps -ef command.
. How to display the 10th line of a file?
head -10 filename | tail -1
. How to reverse a string in unix?
echo "java" | rev
. How to get the last word from a line in Unix file?
echo "unix is good" | rev | cut -f1 -d' ' | rev
. How to check if the last command was successful in Unix?
echo $?
. How will you find which operating system your system is running on in UNIX?
uname -a
. Create a read-only file in your home directory?
touch file; chmod 400 file
. How do you see command line history in UNIX?
The ‘history’ command can be used to get the list of commands that we are executed.
. Write a command to convert a string from lower case to upper case?
echo "apple" | tr [a-z] [A-Z]
. Write a command to redirect the output of date command to multiple files?
The tee command writes the output to multiple files and also displays the output on the terminal.
date | tee -a file1 file2 file3
. How to create an alias for the complex command and remove the alias?
The alias utility is used to create the alias for a command. The below command creates alias for ps -aef command.
alias pg='ps -aef'
If you use pg, it will work the same way as ps -aef.
To remove the alias simply use the unalias command as
unalias pg
. Write a command to display today’s date in the format of ‘yyyy-mm-dd’?
The date command can be used to display today’s date with time
date '+%Y-%m-%d'
. List out some of the Hot Keys available in bash shell?
- Ctrl+l – Clears the Screen.
- Ctrl+r – Does a search in previously given commands in shell.
- Ctrl+u – Clears the typing before the hotkey.
- Ctrl+a – Places cursor at the beginning of the command at shell.
- Ctrl+e – Places cursor at the end of the command at shell.
- Ctrl+d – Kills the shell.
- Ctrl+z – Places the currently running process into background.
. What is the difference between join -v and join -a?
join -v : outputs only matched lines between two files. join -a : In addition to the matched lines, this will output unmatched lines also.
How do you display the calendar for the month march in the year 1985?
The cal command can be used to display the current month calendar. You can pass the month and year as arguments to display the required year, month combination calendar.
cal 03 1985
This will display the calendar for the March month and year 1985.
. How do you write the contents of 3 files into a single file?
cat file1 file2 file3 > file
. Write a command to display your name 100 times. The Yes utility can be used to repeatedly output a line with the specified string or ‘y’.
yes <your_name> | head -100
RM