Variables and expansion
| Syntax | Meaning |
|---|
mol=water | set a variable; no spaces around = |
"$mol", "${mol}_opt.inp" | read it; braces when letters follow |
'$mol' | single quotes keep text literal |
n=$(grep -c "^H" water.xyz) | command substitution: output into a variable |
$(( nprocs * 2 )) | integer arithmetic; decimals need awk or bc |
export OMP_NUM_THREADS=8 | environment variable, visible to child programs |
${f%.xyz} | remove .xyz from the end |
${f##*/} / ${f%/*} | file name only / directory only |
${f#*_} / ${f%%_*} | remove up to the first _ / from the first _ on |
${f/.xyz/.inp} | replace the first match; ${f//a/b} replaces all |
${n:-4} | 4 if unset or empty |
${#mol} | string length |
Tests
| Test | True when |
|---|
[[ -f $f ]], -d, -e | file / directory / anything exists |
[[ -s $f ]] | file exists and is not empty |
[[ -z $s ]] / [[ -n $s ]] | string is empty / not empty |
[[ $a == $b ]], != | string comparison |
[[ $f == *.xyz ]] | pattern match (leave the right side unquoted) |
[[ $n -lt 8 ]] | integer comparison: -eq -ne -lt -le -gt -ge |
(( n > 8 )) | integer comparison, arithmetic syntax |
!, &&, || | not, and, or inside [[ ]] |
Conditionals and loops
| Form | Notes |
|---|
if cmd; then ...; elif ...; else ...; fi | true when the command's exit status is 0 |
A && B / A || B | B if A succeeds / if A fails |
for f in *.xyz; do ...; done | loop over files; never $(ls) |
shopt -s nullglob | no match gives an empty list |
for i in {1..10}, {01..10} | number range; no variables inside |
for (( i=0; i<n; i++ )) | C-style, for ranges that use variables |
while read -r a b; do ...; done < list.txt | file line by line; piping into it loses variable changes |
done < <(find . -name '*.out') | command output line by line (process substitution) |
case $f in *.xyz) ... ;; *) ... ;; esac | branch on patterns |
continue / break | skip to the next item / leave the loop |
Functions and arguments
| Syntax | Meaning |
|---|
name() { ...; } | define a function; call it as name arg1 arg2 |
local x=$1 | variable visible only inside the function |
$0, $1 ... ${10} | script name, arguments |
$# | number of arguments |
"$@" / "$*" | arguments as separate words / as one string |
shift, shift 2 | drop leading arguments |
return 1 | function exit status (0–255) |
e=$(orca_energy water.out) | get a value back through output |
while getopts "m:n:h" opt | parse options; value in $OPTARG, then shift $((OPTIND - 1)) |
source lib.sh, . lib.sh | run another file in the current shell |
Redirection and pipes
| Syntax | Meaning |
|---|
cmd > out / cmd >> out | standard output to a file, overwrite / append |
cmd 2> err | standard error only |
cmd > log 2>&1, cmd &> log | both into one file |
cmd 2>/dev/null | discard error messages |
echo "msg" >&2 | print to standard error |
cmd < file | file as standard input |
a | b | output of a into b |
cmd | tee log | show on screen and write to a file |
cat > f.inp <<EOF ... EOF | here-document; variables and $( ) are expanded |
<<'EOF' | no expansion, text kept literally |
<<-EOF | strips leading tabs (not spaces) |
cmd <<< "$var" | here-string: one string as input |
Safety
| Syntax | Meaning |
|---|
set -euo pipefail | stop on failure, error on unset variables, catch failures mid-pipeline |
grep ... || true | allow "no match" under -e |
i=$((i + 1)) | instead of ((i++)), which trips -e when i is 0 |
${1:-} | missing argument as empty string under -u |
cd "$d" || exit 1 | stop if the directory change fails |
command -v orca | check a program exists and get its path |
tmp=$(mktemp -d) | temporary directory with a unique name |
trap 'rm -rf "$tmp"' EXIT | clean up when the script ends |
date '+%F %T' | timestamp like 2026-09-29 00:15:54 |
bash -n script.sh | syntax check without running |
bash -x script.sh | print each command as it runs |
shellcheck script.sh | find common mistakes |
sed -i 's/\r$//' script.sh | remove Windows line endings (CRLF) |
grep · sed · awk
Only the forms used most often on ORCA outputs. Chapter 05 and chapter 06 explain them in detail.
| Command | What it does |
|---|
grep -n "FINAL SINGLE POINT ENERGY" water.out | matching lines with line numbers |
grep -c "FINAL SINGLE POINT ENERGY" water.out | number of matching lines (e.g. optimisation steps) |
grep -A 10 "^VIBRATIONAL FREQUENCIES" water.out | the match and 10 lines after it |
grep -B 2 "TERMINATED NORMALLY" water.out | the match and 2 lines before it |
grep -l "TERMINATED NORMALLY" */*.out | names of files that match; -L for files that do not |
grep -r --include='*.out' -l "TERMINATED NORMALLY" jobs | search a directory tree |
grep -q "..." f | exit status only, no output (for if) |
sed -n '10,20p' water.out | lines 10 to 20 only |
sed -n '/CARTESIAN COORDINATES (ANGSTROEM)/,+4p' water.out | from the pattern plus 4 lines |
sed -i 's/nprocs 4/nprocs 8/' */*.inp | replace in place; -i.bak keeps the original |
sed '/^#/d' f | delete lines starting with # |
awk '{print $5}' | fifth whitespace-separated column |
awk '{print $NF}' | last column (NF is the number of fields) |
awk 'NR > 2 {print $1}' water.xyz | first column from line 3 on (NR is the line number) |
awk -F, '{print $2}' data.csv | second comma-separated column |
awk '/FINAL SINGLE POINT ENERGY/ {e=$5} END {print e}' water.out | value from the last matching line |
awk '{s += $2} END {print s / NR}' f | mean of the second column |
awk -v x="$e" 'BEGIN {printf "%.2f\n", x * 627.509}' | decimal arithmetic on a shell variable (Hartree to kcal/mol) |
$ grep -n "FINAL SINGLE POINT ENERGY" water.out | tail -n 1
3124:FINAL SINGLE POINT ENERGY -76.426369457144
$ awk '/FINAL SINGLE POINT ENERGY/ {e=$5} END {print e}' water.out
-76.426369457144
$ grep -L "ORCA TERMINATED NORMALLY" jobs/*/*.out
jobs/methane_opt/methane_opt.out
PBS and Slurm
The PBS column is for PBS Pro. Queue names, memory units and defaults differ between clusters, so keep your site's documentation at hand.
Full job scripts are in chapter 07 and arrays in chapter 08.
| Task | PBS Pro | Slurm |
|---|
| Submit | qsub job.pbs | sbatch job.sh |
| List my jobs | qstat -u $USER | squeue -u $USER |
| Job details | qstat -f <jobid> | scontrol show job <jobid> |
| Finished jobs | qstat -x <jobid> | sacct -j <jobid> |
| Cancel | qdel <jobid> | scancel <jobid> |
| Interactive job | qsub -I -l select=1:ncpus=4 | srun --pty -c 4 bash |
| Job name | #PBS -N opt | #SBATCH --job-name=opt (-J opt) |
| Cores and memory | #PBS -l select=1:ncpus=8:mem=16gb | #SBATCH --ntasks=1
#SBATCH --cpus-per-task=8
#SBATCH --mem=16G |
8 MPI processes (ORCA %pal) | #PBS -l select=1:ncpus=8:mpiprocs=8 | #SBATCH --ntasks=8 |
| Time limit | #PBS -l walltime=24:00:00 | #SBATCH --time=24:00:00 |
| Queue / partition | #PBS -q normal | #SBATCH --partition=normal |
| Output file | #PBS -j oe (merge stderr into stdout) | #SBATCH --output=%x-%j.out |
| Job array | #PBS -J 1-10 | #SBATCH --array=1-10 |
| Array index | $PBS_ARRAY_INDEX | $SLURM_ARRAY_TASK_ID |
| Submission directory | $PBS_O_WORKDIR | $SLURM_SUBMIT_DIR |
| Job ID | $PBS_JOBID | $SLURM_JOB_ID |
| Cores granted | $NCPUS | $SLURM_CPUS_PER_TASK, $SLURM_NTASKS |
Same letter, different meaning
PBS -N is the job name, but Slurm -N is the number of nodes. PBS -J is an array, but Slurm -J is the job name.
When porting scripts, Slurm's long options (--job-name, --nodes, --array) avoid the confusion.
PBS jobs start in your home directory, so the script needs cd "$PBS_O_WORKDIR" near the top.
Slurm jobs start in the submission directory. To test a job script locally, cd "${PBS_O_WORKDIR:-$PWD}" gives it a fallback.