Writing safer scripts

set -euo pipefail

By default bash carries on to the next line when a command fails. The input copy fails but the calculation starts anyway; the cd fails and rm runs in the wrong directory. This line at the top of a script prevents much of that.

#!/bin/bash
set -euo pipefail

Without pipefail, a pipeline's exit status is that of its last command. When grep fails because of a wrong file name, tail still succeeds, and so does the pipeline as a whole.

$ e=$(grep "FINAL SINGLE POINT ENERGY" watr.out | tail -n 1); echo "status: $?"
grep: watr.out: No such file or directory
status: 0
$ set -o pipefail
$ e=$(grep "FINAL SINGLE POINT ENERGY" watr.out | tail -n 1); echo "status: $?"
grep: watr.out: No such file or directory
status: 2

Where set -e applies and where it does not

set -e is convenient, but its rules are not simple. Newcomers usually see a script stop without a word in one of these three cases.

First, grep returns 1 when it finds nothing. That is a normal answer ("no match"), not an error, but -e treats it as a failure.

#!/bin/bash
set -e
n=$(grep -c "ERROR" water.out)      # stops here if there is no ERROR
echo "errors: $n"

$ bash e1.sh
$ echo $?
1

It exits with no message at all. Where finding nothing is acceptable, add || true: n=$(grep -c "ERROR" water.out || true) prints errors: 0.

Second, ((i++)) returns 1 when i is 0. (( )) fails when its value is 0, and post-increment yields the old value, 0. Write counters as i=$((i + 1)).

#!/bin/bash
set -e
i=0
((i++))            # stops here
echo "i = $i"

Third, -u considers $1 undefined when the script is run without arguments.

$ cat e3.sh
#!/bin/bash
set -u
mol=$1
echo "mol = $mol"
$ bash e3.sh
e3.sh: line 3: $1: unbound variable

Where an argument may be missing, give it a default with ${1:-}. It expands to an empty string without an error, and you can then test it with [[ -z $mol ]] and print a usage message.

In the other direction, some places ignore -e: the condition of if and while, and the left side of && and ||. When a function is called as an if condition, -e is off for the entire function body. Treat set -e as a safety net and check important commands explicitly, for example with || exit 1.

Quoting and cd

Quote every variable as "$f" and every command substitution as "$(...)". Quoting always is quicker than memorising where it is safe not to. The exceptions are roughly the inside of [[ ]] and $(( )).

A failed cd leaves you where you were. If the next line is rm -f *.tmp, it deletes files in the wrong directory. Always follow cd with || exit 1.

cd "$scratch" || exit 1
rm -f ./*.tmp

Check that a program exists with command -v. It prints the path and returns 0 if found, 1 otherwise. ORCA has to be called by its full path for parallel runs, so the path you get here is exactly what you need.

if ! orca_bin=$(command -v orca); then
    echo "error: orca not found in PATH" >&2
    exit 1
fi

Example: running ORCA in a scratch directory

This script runs each calculation on the node's local disk and copies only the output back. It pulls together everything above. mktemp -d creates a temporary directory with a unique name. trap '...' EXIT runs the given command when the script ends, whether it finishes normally or stops on an error; here it removes the temporary directory.

#!/bin/bash
# run ORCA on each input in a temporary directory and copy the output back
set -euo pipefail

dry_run=0
if [[ ${1:-} == "-n" ]]; then
    dry_run=1
    shift
fi

log() {
    printf '[%s] %s\n' "$(date '+%F %T')" "$*"
}

if ! orca_bin=$(command -v orca); then
    echo "error: orca not found in PATH" >&2
    exit 1
fi

scratch=$(mktemp -d "${TMPDIR:-/tmp}/orca.XXXXXX")
trap 'rm -rf "$scratch"' EXIT
start_dir=$PWD

for inp in "$@"; do
    dir=$(dirname "$inp")
    name=$(basename "$inp" .inp)

    if (( dry_run )); then
        log "would run $inp"
        continue
    fi

    log "start $name"
    cp "$inp" "$scratch/"
    cd "$scratch" || exit 1
    status=0
    "$orca_bin" "$name.inp" > "$name.out" || status=$?
    cp "$name.out" "$start_dir/$dir/"
    rm -f "$scratch"/*
    cd "$start_dir"

    if grep -q "ORCA TERMINATED NORMALLY" "$dir/$name.out"; then
        log "done $name"
    else
        log "FAILED $name (exit status $status)"
    fi
done

$ ./run_orca.sh -n */*.inp
[2026-09-29 00:15:54] would run benzene/benzene.inp
[2026-09-29 00:15:54] would run water/water.inp
$ ./run_orca.sh */*.inp
[2026-09-29 00:15:54] start benzene
[2026-09-29 00:15:54] FAILED benzene (exit status 1)
[2026-09-29 00:15:54] start water
[2026-09-29 00:15:54] done water
$ ls -d /tmp/orca.*
ls: cannot access '/tmp/orca.*': No such file or directory

A few points worth noting:

• With -n the script only prints what it would do. Run scripts that touch hundreds of files this way first.
• "$orca_bin" ... || status=$? records ORCA's exit status without letting set -e stop the script. One failed calculation does not abandon the rest.
• log stamps every line with the time in date '+%F %T' format. Kept in the job's output file, it tells you later how long each calculation took.
• The final ls finds nothing because the trap removed the temporary directory.

When a job hits its time limit, the scheduler first sends SIGTERM. bash exits on that signal and the EXIT trap still runs. A SIGKILL, however, ends the process without running any trap. Handling scratch space in PBS and Slurm scripts is covered in chapter 07.

ShellCheck

ShellCheck is a static analyser that catches common shell script mistakes. Paste a script into the website, or install it with apt install shellcheck or conda install -c conda-forge shellcheck. Here is a script that collects several mistakes from this chapter.

#!/bin/bash
for f in $(ls */*.out); do
    cd $(dirname $f)
    grep "ORCA TERMINATED NORMALLY" $(basename $f)
    cd ..
done

$ shellcheck naive.sh

In naive.sh line 2:
for f in $(ls */*.out); do
         ^-----------^ SC2045 (error): Iterating over ls output is fragile. Use globs.
              ^-- SC2035 (info): Use ./*glob* or -- *glob* so names with dashes won't become options.


In naive.sh line 3:
    cd $(dirname $f)
    ^--------------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
       ^-----------^ SC2046 (warning): Quote this to prevent word splitting.
                 ^-- SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean:
    cd $(dirname "$f") || exit


In naive.sh line 4:
    grep "ORCA TERMINATED NORMALLY" $(basename $f)
                                    ^------------^ SC2046 (warning): Quote this to prevent word splitting.
                                               ^-- SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean:
    grep "ORCA TERMINATED NORMALLY" $(basename "$f")


In naive.sh line 5:
    cd ..
    ^---^ SC2103 (info): Use a ( subshell ) to avoid having to cd back.

For more information:
  https://www.shellcheck.net/wiki/SC2045 -- Iterating over ls output is fragi...
  https://www.shellcheck.net/wiki/SC2046 -- Quote this to prevent word splitt...
  https://www.shellcheck.net/wiki/SC2164 -- Use 'cd ... || exit' or 'cd ... |...

Searching for an SC number brings up a page explaining it. With every suggestion applied, the script becomes the one below. The parentheses ( ) start a subshell, so a cd inside them does not change the loop's working directory.

#!/bin/bash
for f in */*.out; do
    (
        cd "$(dirname "$f")" || exit 1
        grep -H "ORCA TERMINATED NORMALLY" "$(basename "$f")"
    )
done

Windows line endings (CRLF)

Scripts written in a Windows editor end each line with \r\n (CRLF). Linux treats only \n as a line break, so the \r sticks to the end of commands and paths. On the shebang line it produces this error:

$ ./crlf.sh
bash: ./crlf.sh: /bin/bash^M: bad interpreter: No such file or directory
$ bash crlf2.sh
crlf2.sh: line 2: cd: $'jobs\r': No such file or directory
crlf2.sh: line 3: $'ls\r': command not found

^M and \r are the same character. Confirm with file and fix with sed or dos2unix.

$ file crlf.sh
crlf.sh: Bourne-Again shell script, ASCII text executable, with CRLF line terminators
$ sed -i 's/\r$//' crlf.sh
$ file crlf.sh
crlf.sh: Bourne-Again shell script, ASCII text executable

dos2unix crlf.sh does the same, but it is not always installed on clusters, whereas sed is everywhere. If you edit in VS Code on Windows, click CRLF in the status bar at the bottom right, switch it to LF and save. Input files and lists copied over from Windows cause the same problem.