Pipeline

A Bash pipeline is an inter-process communication (IPC) mechanism that chains multiple commands together by connecting the standard output (stdout) of one process directly to the standard input (stdin) of the next process using anonymous pipes.

Syntax

# Standard pipeline (stdout to stdin)
command1 | command2 | command3


# Standard error and standard output pipeline (stdout + stderr to stdin)
command1 |& command2

Technical Mechanics

File Descriptor Mapping

• The standard pipe operator (|) connects File Descriptor 1 (FD 1, stdout) of the preceding command to File Descriptor 0 (FD 0, stdin) of the succeeding command.
• The |& operator is a Bash shorthand introduced in version 4.0. It connects both FD 1 (stdout) and FD 2 (stderr) of the preceding command to FD 0 (stdin) of the succeeding command. It is functionally equivalent to command1 2>&1 | command2.

Execution Environment (Subshells) By default, Bash executes every command within a pipeline in its own separate subshell environment. Consequently, variable assignments, state changes, or directory changes (cd) made by any command within the pipeline will not persist in the parent shell once the pipeline terminates.

Note: In Bash 4.2+, enabling shopt -s lastpipe allows the final command of a pipeline to execute in the current shell environment, provided job control is disabled.

Concurrency and I/O Blocking Commands in a pipeline are invoked concurrently, not sequentially. The operating system kernel allocates a bounded memory buffer for each anonymous pipe.

• The writing process will block (suspend execution) if the pipe buffer is full.
• The reading process will block if the pipe buffer is empty.
• The pipeline terminates when all constituent processes have terminated.

Exit Status By default, the exit status ($?) of an entire pipeline is strictly the exit status of the last (rightmost) command in the chain. The exit statuses of preceding commands are discarded.

To alter this behavior, Bash provides the pipefail option:

set -o pipefail

When pipefail is enabled, the pipeline’s exit status becomes the exit status of the rightmost command to exit with a non-zero status. If all commands in the pipeline exit successfully (status 0), the pipeline returns 0.

Array of Exit Codes Bash retains the individual exit statuses of all commands executed in the most recent pipeline within the internal array variable PIPESTATUS.

command1 | command2 | command3
echo "${PIPESTATUS[@]}" # Outputs an array of three exit codes, e.g., "0 1 0"

Master Bash with Deep Grasping Methodology!

Learn More