Getting Tools to Fit the Job. Using Components, Pipes, Redirection, etc

By Jim Jackson


In this, any reference to a SHELL is assumed to be either the Bourne shell
or one of its descendants, e.g. BASH.

Program Input and Output

The Conventional Model:

                     -------------       stdout
                     |           | ----- OUTPUT ----->
    ---- INPUT ----> |  Program  |
         stdin       |           | ----- ERRORS ----->

                     -------------       stderr

Each program when executed has 3 files preset for it:

  • One input file, STDIN
  • and 2 seperate outputs files, STDOUT and STDERR.

STDIN is usually the keyboard
STDOUT is usually the console, terminal or shell window
STDERR is usually the console, terminal or shell window

Remember: In Unix/Linux “everything is a file” –
even the keyboard/screen

The program can of course completely ignore these and get its inputs,
and set its outputs to anything it wants. But most of the standard Unix/Linux
command line tools follow a convention in the use of these preset files.

  • If there is no alternative input file(s) specified, then the program takes
    its input from STDIN
  • It directs its normal output to STDOUT
  • It directs any error messages etc to STDERR

OK Why is this useful? …


… Because when executing programs, the SHELL can change
what “file” is connected to STDIN, STDOUT and


Redirecting STDOUT

Use the ‘>‘ or ‘>>‘ symbols to
change where the normal output goes. ‘>>‘ causes the
output to be appended to any previous contents of the file.

  e.g.  % ls -l /bin > /tmp/bin_list
        % ls -l /usr/bin >> /tmp/bin_list

Redirecting STDIN

Use the ‘<‘ symbol to change where the normal input
comes from

  e.g.  % wc -l < /tmp/etc_list

mmmmm……. compare with “wc -l /tmp/etc_list”, it’s only a
small change. A less forced e.g.

        % /usr/sbin/sendmail -t -oem -oi < /tmp/mailfile

Redirecting STDERR

Use ‘2>‘ to change where the error output goes,
2 because STDERR is file 2.
To append use ‘2>>

  e.g.  % ls -l Z* 2> /dev/null

Redirecting ALL output to same place

Use the construct ‘> outputfile 2>&1‘. That is,
redirect STDOUT to file and redirect file 2
(STDERR) to file 1 (&1)

  e.g.       % ( ./configure ; make ) > /tmp/make.output 2>&1
  or better  % ( ./configure && make ) > /tmp/make.output 2>&1

Joining programs together – PIPES

The shell can use a PIPE (see man pipe) to
feed the output of one program into the input of another.

Joining STDOUT of one command with STDIN of a second command

Use the ‘|‘ symbol between the 2 commands.

  e.g.  % ls -l /etc | less

Joining STDERR and STDOUT of one command to STDIN of a second command

Use the construct ‘2>&1 |

  e.g.       % ( ./configure ; make) 2>&1 | less
  or better  % ( ./configure && make) 2>&1 | less

Because programs can be so joined together, the UNIX convention of making
utilities do essentially one thing, but do it well, has developed. More complex
actions can be produced by joining commands together with pipes.

Some of the main CLI utilities

some programs that generate useful info

  • ls list directory contents
  • ps report process status
  • file determine file type
  • du estimate file space usage
  • dmesg print or control the kernel ring buffer


  • mail send and receive mail
  • sendmail an electronic mail transport agent
    even is
    sendmail is not installed, you might find that a program called sendmail is
  • man an interface to the on-line reference manuals
    also info , if you must.

Some of the main CLI “filters”

These all work on files, which can be specified on the command line.
If no filesnames are specified then they take their input from


  • cat concatenate files and print on STDOUT
  • tac , rev see man pages
  • less less is more, but better
  • head output the first part of files
  • tail output the last part of files
  • tee read from STDIN and write to STDOUT and files
  • wc print the number of bytes, words, and lines in files
  • grep print lines matching a pattern
  • sort sort lines of text files
  • uniq remove duplicate lines from a sorted file
  • cut remove sections from each line of files
  • sed a Stream EDitor

Example pipelines

  show the mozilla processes running...

       % ps wuax | grep mozill

  show the number of files in the _Docs_ directory...

       % ls Docs | wc -l

  mail yourself a list of files in your archive directory...

       % ls -l archive | mail -s "List of Current Archive files" jj

  browse the kernel messages from the syslog file...

       % grep kernel /var/log/messages | less

  watch for tcpwrapper connect messages in syslog file and save

       % tail -f /var/log/syslog | grep "connect from"

Comments are closed.