Manage input/output

Manage input/output#

This page deals with common tools that improves the terminal experience. They don’t have a specific purpose, can be used for many purposes and are related with transfering and processing information with LinuxTerminal.

Stdout#

Consider typical Linux shell patterns for processing standard output:

  • <command> > <file>: redirects the output of the command to the file. The contents of the file is replaced.

  • <command> >> <file>: redirects the output of the command to the file. Appends the <command> stdout to the file.

  • tee <file>: Saves the stdin of the to <file> and prints it to the it’s stdout.

The following cell redirects the output of echo "to file" to the file and prints the content of the file.

echo "to file" > /tmp/my_file.txt
cat /tmp/my_file.txt

to file

However, if the same operation performed one more time the contents of the file is replaced:

echo "new information" > /tmp/my_file.txt
cat /tmp/my_file.txt

new information

Using the >> changes this behavior.

echo "new information 2" >> /tmp/my_file.txt
cat /tmp/my_file.txt

new information
new information 2

New content is added below the previous content.

Note that none of the previous commands do not print anything to the stdout. If you still need to see redirected content to a file, you have to use <command> | tee <file> pattern. The following cell shows how it works.

echo "to file" | tee /tmp/some_file.txt

to file

The content displayed by echo is still saved to the file.

cat /tmp/some_file.txt

to file

Grep#

Global Regular Expression Print (grep) is a very popular Linux utility for searching within texts. It allows you to find sections of text that match specified patterns and is extremely useful in many cases. Check official manual on gnu.org.

The most common use of grep is to find patterns in the output of another command. The following cell creates a file with some text that will be used as an example.

cat << EOF > /tmp/play_with_grep
This text is used
to show how the grep
utility works
EOF

The following cell applies the cat command to the file we created earlier - so its contents will be shown in the standard output. Using the syntax <command> | grep <pattern> applies grep <pattern> to the output of <command>.

cat /tmp/play_with_grep | grep text

This text is used

The result is a line in which the specified pattern appears.

The following cell similarly shows the line where the word grep appears.

cat /tmp/play_with_grep | grep grep

to show how the grep

Heredoc (<< delimiting_identifier)#

Using the << symbol followed by a delimiter identifier, you can define a multiline string (here document) that will be passed as input to the chosen command. The block of text should be terminated by the same delimiter identifier.

In the following cell, the cat command is passed a multiline expression, which is then printed in the output — this is exactly what the cat command does.

cat << EOF
hello
my name 
is fedor
EOF

hello
my name 
is fedor

To understand better what exactly it does. Anoter example with other delimiting identifier and other command applied to the result. Here we are using grep to find line that contains FIND ME. Begining and ending of the document is defined by SSS combination of the symbols.

grep "FIND ME" << SSS
this is some line
it's great FIND ME that
something strange in this line 
SSS

it's great FIND ME that

Another useful application of heredocs is passing programs to bash inline. The following cell passes a program to bash directly from stdin:

bash << EOF
for i in {1..5}
do
    echo "Line \$i"
done
EOF

Line 1
Line 2
Line 3
Line 4
Line 5

Clipboard#

There are special utility to manipulate with clipboard in linux xclip. Check decription using man xclip or on the official github page.

To to paste result as a “ctrl+v” combination use <command> | xclip -sel clip.

The following cell uses xclip to save the result of the uptime command.

uptime | xclip

xclip -o retrieves the content you’ve previously saved.

xclip -o

 17:29:43 up  1:41,  1 user,  load average: 1.53, 1.17, 1.07

Background run#

Use the symbol & after a command to run it in the background. A really useful fact is that it saves the PID of the process to the ! variable.

The following cell runs the some Python code in the background.

python3 -c "
from time import sleep
for i in range(10): print(i);sleep(5);" &

[1] 69331

The number in the output is the PID of the process. However, it is also saved to the “!” variable.

echo $!

69331

The process killing is displayed in the following cell.

kill $!

Stdout arguments#

You can interprete the output of one command as arguments for other command. There are a few ways to do this:

  • Substitution the output using the syntax $(<code>).

  • The xargs command which is designed to add the stdout to the arguments of the specified command and allows to overcome some of the limitations of substitution approach.

Consider a few examples. The following cell saves to the values “hello1”, “hello2”, “hello3”. And subpresses it

values=$(for ((i=1; i<=3; i++)); do echo "hello$i"; done;)
echo $values

hello1 hello2 hello3

It’s really common use case with the echo command. Considers the following less common case: the following cell creates files named hello7...hello10 by substituting the output of the cycle for the touch command.

rm -rf /tmp/my_args && mkdir /tmp/my_args && cd /tmp/my_args
touch $(for ((i=7; i<=10; i++)); do echo "hello$i"; done;)
ls

hello10  hello7  hello8  hello9

As each program processes the input differently, there is no issue with substituting named arguments:

args="-m cowsay -t hello"
python3 $args

  _____
| hello |
  =====
     \
      \
        ^__^
        (oo)\_______
        (__)\       )\/\
            ||----w |
            ||     ||

The following cell uses xargs to pass arguments to Python.

args="-m cowsay -t xargs"
echo $args | xargs python3 

  _____
| xargs |
  =====
     \
      \
        ^__^
        (oo)\_______
        (__)\       )\/\
            ||----w |
            ||     ||
Contents