Command Line -- Lesson 2

In this lesson, I learned how to use the command line to redirect standard input and standard output.

• Redirection reroutes standard input, standard output, and standard error.

• -- The common redirection commands are:

  • > redirects standard output of a command to a file, overwriting previous content.
  • >> redirects standard output of a command to a file, appending new content to old content.
  • < redirects standard input to a command.
  • | redirects standard output of a command to another command.

• -- A number of other commands are powerful when combined with redirection commands:

  • sort: sorts lines of text alphabetically.
  • uniq: filters duplicate, adjacent lines of text.
  • grep: searches for a text pattern and outputs it.
  • sed : searches for a text pattern, modifies it, and outputs it.

About standard input, standard output, and standard error:

1. standard input, abbreviated as stdin, is information inputted into the terminal through the keyboard or input device.
2. standard output, abbreviated as stdout, is the information outputted after a process is run.
3. standard error, abbreviated as stderr, is an error message outputted by a failed process.

1. > command 

$ echo "Hello" > hello.txt

The > command redirects the standard output to a file. Here, "Hello" is entered as the standard input. The standard output "Hello" is redirected by> to the file hello.txt.

2. cat command 

$ cat hello.txt

The cat command outputs the contents of a file to the terminal. When you type cat hello.txt, the contents of hello.txt are displayed.

$ cat oceans.txt > continents.txt

> takes the standard output of the command on the left, and redirects it to the file on the right. Here the standard output of cat oceans.txt is redirected to continents.txt.

Note that > overwrites all original content in continents.txt. When you view the output data by typing cat on continents.txt, you will see only the contents of oceans.txt.

3.>> command 

$ cat glaciers.txt >> rivers.txt

>> takes the standard output of the command on the left and appends (adds) it to the file on the right. You can view the output data of the file with cat and the filename.

Here, the the output data of rivers.txt will contain the original contents ofrivers.txt with the content of glaciers.txt appended to it.

4. < command 

$ cat < lakes.txt

< takes the standard input from the file on the right and inputs it into the program on the left. Here, lakes.txt is the standard input for the catcommand. The standard output appears in the terminal.

5. | command 

$ cat volcanoes.txt | wc

| is a "pipe". The | takes the standard output of the command on the left, and pipes it as standard input to the command on the right. You can think of this as "command to command" redirection.

Here the output of cat volcanoes.txt is the standard input of wc. in turn, the wc command outputs the number of lines, words, and characters involcanoes.txt, respectively.

$ cat volcanoes.txt | wc | cat > islands.txt

Multiple |s can be chained together. Here the standard output ofcat volcanoes.txt is "piped" to the wc command. The standard output ofwc is then "piped" to cat. Finally, the standard output of cat is redirected toislands.txt.

6. sort command 

$ sort lakes.txt

sort takes the standard input and orders it alphabetically for the standard output. Here, the lakes in sort lakes.txt are listed in alphabetical order.

$ cat lakes.txt | sort > sorted-lakes.txt

Here, the command takes the standard output from cat lakes.txt and "pipes" it to sort. The standard output of sort is redirected to sorted-lakes.txt.

7. uniq command 

$ uniq deserts.txt

uniq stands for "unique" and filters out adjacent, duplicate lines in a file. Hereuniq deserts.txt filters out duplicates of "Sahara Desert", because the duplicate of 'Sahara Desert' directly follows the previous instance. The "Kalahari Desert" duplicates are not adjacent, and thus remain.

8. grep command 

$ grep Mount mountains.txt

grep stands for "global regular expression print". It searches files for lines that match a pattern and returns the results. It is also case sensitive. Here, grepsearches for "Mount" in mountains.txt.

$ grep -i Mount mountains.txt

grep -i enables the command to be case insensitive. Here, grep searches for capital or lowercase strings that match Mount in mountains.txt.

The above commands are a great way to get started with grep. If you are familiar with regular expressions, you can use regular expressions to search for patterns in files.

$ grep -R Arctic /home/ccuser/workspace/geography

grep -R searches all files in a directory and outputsfilenames and lines containing matched results. -R stands for "recursive". Here grep -R searches the /home/ccuser/workspace/geography directory for the string "Arctic" and outputs filenames and lines with matched results.

$ grep -Rl Arctic /home/ccuser/workspace/geography

grep -Rl searches all files in a directory and outputs only filenames with matched results. -R stands for "recursive" and l stands for "files with matches". Here grep -Rl searches the/home/ccuser/workspace/geography directory for the string "Arctic" and outputs filenames with matched results.

9. sed command 

$ sed 's/snow/rain/' forests.txt

sed stands for "stream editor". It accepts standard input and modifies it based on an expression, before displaying it as output data. It is similar to "find and replace".

Let's look at the expression 's/snow/rain/':

• s: stands for "substitution". it is always used when using sed for substitution.
• snow: the search string, the text to find.
• rain: the replacement string, the text to add in place.

In this case, sed searches forests.txt for the word "snow" and replaces it with "rain." Importantly, the above command will only replace the first instance of "snow" on a line.

$ sed 's/snow/rain/g' forests.txt

The above command uses the g expression, meaning "global". Here sedsearches forests.txt for the word "snow" and replaces it with "rain", globally. All instances of "snow" on a line will be turned to "rain".