How to make Bash scripts faster?

Prev: Built-in commands, Up: Index, Next: Compounds and functions

Command substitution

When writing a script, we often need to compute some value and store it in a variable for further processing. Some examples are: evaluation of an arithmetic expression, replacing a substring in a string or generating a sequence of numbers. In Bourne shell, such effects were achieved using command substitution.

For example, to save the current date in ISO 8601 one would type:

TODAY=`date -Idate`

In Bash, there’s an alternative syntax for command substitution: $(command). It’s easier in use, and it allows nesting substitutions in a natural way, but otherwise there’s no difference between those two forms.

For reasons described in the previous part, such substitutions were quite heavy as every such substitution resulted in a new subprocess.

Expansions

Bash, following Korn shell, introduced a number of advanced expansions — simple substitutions performed by the shell itself, eliminating the need of creating subprocesses. Throughout the rest of this part, we’ll go through a series of common command substitutions along with corresponding Bash expansions. Of course, we’ll also measure and compare their execution times.

Base name

Extracting the last segment of the path was historically performed using a command called basename:

time for ((i=0; i<10000; i++)); do
    NAME=`basename ${path}`
done

real    0m15,059s
user    0m10,274s
sys     0m5,372s

A similar effect can be achieved using a prefix-removing expansion:

for ((i=0; i<10000; i++)); do
    NAME="${path##*/}"
done

real    0m0,037s
user    0m0,037s
sys     0m0,000s

Directory name

In a similar way we can extract the directory part:

for ((i=0; i<10000; i++)); do
    DIR=`dirname ${path}`
done

real    0m16,149s
user    0m10,810s
sys     0m5,907s

But this time with a suffix-removing expansion:

for ((i=0; i<10000; i++)); do
    DIR="${path%/*}"
done

real    0m0,029s
user    0m0,029s
sys     0m0,000s

String replacement

Replacing something with something else in a string is another very common use case:

OLD="Mary had a little lamb"
time for ((i=0; i<10000; i++)); do
    NEW=`echo "${OLD}" | sed 's/lamb/llama/'`
done

real    0m27,099s
user    0m23,851s
sys     0m9,255s

And, of course, there’s also an expansion for that:

OLD="Mary had a little lamb"
for ((i=0; i<10000; i++)); do
    NEW="${OLD/lamb/llama}"
done

real    0m0,047s
user    0m0,047s
sys     0m0,000s

Listing files

If you would like to get a list of all files in the current directory, you could simply use ls:

for ((i=0; i<10000; i++)); do
    FILES=`ls`
done

real    0m18,732s
user    0m11,992s
sys     0m7,455s

Or you could use path expansion (a.k.a. globbing):

for ((i=0; i<10000; i++)); do
    FILES="*"
done

real    0m0,016s
user    0m0,016s
sys     0m0,000s

Arithmetic expressions

Doing simple math used to be a matter of calling expr:

for ((i=0; i<10000; i++)); do
    VAL=`expr 2 + 2 \* 2`
done

real    0m15,990s
user    0m10,522s
sys     0m6,101s

In Bash, there’s a syntax for that:

for ((i=0; i<10000; i++)); do
    VAL=$((expr 2 + 2 \* 2))
done

real    0m0,022s
user    0m0,022s
sys     0m0,000s

Regular expressions

expr can also be used to check if something matches a regular expression:

for ((i=0; i<10000; i++)); do
    expr abba : 'a\(.*\)a'
done

real    0m13,297s
user    0m8,698s
sys     0m5,191s

In Bash, double square brackets [[ do the job (with the =~ operator):

for ((i=0; i<10000; i++)); do
    [[ abba =~ a(.*)a ]]
done

real    0m0,058s
user    0m0,058s
sys     0m0,000s

In the Bash manpage you’ll find much more than those few examples. What’s particularly interesting in the results is that all expansions are executed entirely in the user space. This allows us avoid mode switching, which happens every time we make a system call.