The ORCA examples are ORCA 6 outputs from the ORCA guide. opt.out is a
B3LYP-D4/def2-TZVP optimisation + frequency run on water. The QE and LAMMPS examples are also real outputs, produced while
writing those guides. grep, sed and awk syntax is covered in the previous chapter.
ORCA: the final energy
ORCA writes a FINAL SINGLE POINT ENERGY line every time it computes an energy. An optimisation prints one per cycle,
so you want the last one.
$ grep 'FINAL SINGLE POINT ENERGY' opt.out
FINAL SINGLE POINT ENERGY -76.426286364586
FINAL SINGLE POINT ENERGY -76.426353431114
FINAL SINGLE POINT ENERGY -76.426364583348
FINAL SINGLE POINT ENERGY -76.426366351564
FINAL SINGLE POINT ENERGY -76.426369457144
$ grep 'FINAL SINGLE POINT ENERGY' opt.out | tail -1 | awk '{print $5}'
-76.426369457144
awk alone works too: overwrite a variable on every matching line and print it in END, and the last value is what remains.
If no line matches you get an empty line, so check for an empty result in scripts.
$ awk '/FINAL SINGLE POINT ENERGY/ {e = $5} END {print e}' opt.out
-76.426369457144ORCA: energy per optimisation cycle
To plot the convergence, save cycle number and energy as two columns. gnuplot and matplotlib read this directly.
$ grep 'FINAL SINGLE POINT ENERGY' opt.out | awk '{printf "%d %.10f\n", NR, $5}' > scf.dat
$ cat scf.dat
1 -76.4262863646
2 -76.4263534311
3 -76.4263645833
4 -76.4263663516
5 -76.4263694571
The five lines are the four optimisation cycles and the final evaluation at the converged geometry. A third column with the energy
relative to the last value, in kcal/mol, shows how much was left to converge. Collect all values in an array and compute in END.
$ awk '/FINAL SINGLE POINT ENERGY/ {n++; e[n] = $5}
END {for (i = 1; i <= n; i++) printf "%d %.10f %8.4f\n", i, e[i], (e[i] - e[n]) * 627.5095}' opt.out
1 -76.4262863646 0.0521
2 -76.4263534311 0.0101
3 -76.4263645833 0.0031
4 -76.4263663516 0.0019
5 -76.4263694571 0.0000ORCA: normal termination and convergence
Before using a number, check that the job ran to the end. A finished ORCA output ends with
ORCA TERMINATED NORMALLY, and a converged optimisation prints THE OPTIMIZATION HAS CONVERGED.
05-killed is a copy of an output cut off partway, to stand in for a job killed at the walltime limit.
$ grep -c 'ORCA TERMINATED NORMALLY' opt.out
1
$ grep -c 'ORCA TERMINATED NORMALLY' jobs/05-killed/water.out
0
$ grep -q 'THE OPTIMIZATION HAS CONVERGED' opt.out && echo converged || echo 'NOT converged'
converged
grep -q prints nothing and only sets the exit status: 0 if found, 1 if not. That makes it usable directly in an if.
ORCA: imaginary frequencies and free energy
The frequencies sit between VIBRATIONAL FREQUENCIES and NORMAL MODES. An awk range pattern
/start/,/end/ cuts out that block, and the second column holds the values.
$ awk '/VIBRATIONAL FREQUENCIES/,/NORMAL MODES/' opt.out | awk '/cm\*\*-1/ {print $2}'
0.00
0.00
0.00
0.00
0.00
0.00
1615.68
3780.78
3885.42
The first six zeros are translations and rotations. ORCA prints imaginary frequencies as negative numbers followed by
***imaginary mode***. The command below prints and counts the negative lines inside the block. ts.out is a copy
in which mode 6 was edited to a negative value, as in the ORCA guide example, to show the format.
$ awk '/VIBRATIONAL FREQUENCIES/ {f = 1} /NORMAL MODES/ {f = 0}
f && /cm\*\*-1/ && $2 < 0 {n++; print}
END {print n + 0, "imaginary"}' opt.out
0 imaginary
$ awk '/VIBRATIONAL FREQUENCIES/ {f = 1} /NORMAL MODES/ {f = 0}
f && /cm\*\*-1/ && $2 < 0 {n++; print}
END {print n + 0, "imaginary"}' ts.out
6: -456.78 cm**-1 ***imaginary mode***
1 imaginary
n + 0 makes a variable that was never incremented print as 0 instead of an empty string.
Thermochemistry results are single labelled lines, so grep and awk get them directly.
$ grep 'Final Gibbs free energy' opt.out
Final Gibbs free energy ... -76.42286885 Eh
$ awk '/Final Gibbs free energy/ {print $6}' opt.out
-76.42286885ORCA: last geometry to xyz
A CARTESIAN COORDINATES (ANGSTROEM) block is printed every cycle. Skip the dashed line after the header and collect
coordinates until the next blank line. Each new block resets the counter, so only the last block is left at the end.
awk '/CARTESIAN COORDINATES \(ANGSTROEM\)/ {getline; n = 0; blk = 1; next}
blk && NF == 0 {blk = 0}
blk {line[++n] = $0}
END {print n; print "final geometry"; for (i = 1; i <= n; i++) print line[i]}' opt.out > final.xyz$ cat final.xyz
3
final geometry
O 0.000000 0.000000 0.111489
H 0.000000 0.765189 -0.473160
H 0.000000 -0.765189 -0.473160
Parentheses in the regex are escaped as \(. getline reads (and so skips) the next line.
ORCA also saves the optimised geometry as inputname.xyz; this recipe is for when only the output file is left.
Many jobs into one CSV table
Suppose each calculation has its own directory with an output inside. Skip unfinished jobs, collect the energies of the rest and add energies relative to the lowest one (1 Eh = 627.5095 kcal/mol).
#!/bin/bash
# collect.sh: final energy of every job directory into a CSV
set -euo pipefail
echo "name,energy_Eh" > energies.csv
for out in jobs/*/*.out; do
name=$(basename "$(dirname "$out")")
if ! grep -q 'ORCA TERMINATED NORMALLY' "$out"; then
echo "skip $name: not finished" >&2
continue
fi
e=$(awk '/FINAL SINGLE POINT ENERGY/ {e = $5} END {print e}' "$out")
echo "$name,$e" >> energies.csv
done
awk -F, 'NR > 1 {name[NR] = $1; e[NR] = $2; if (NR == 2 || $2 < min) min = $2}
END {print "name,energy_Eh,rel_kcal"
for (i = 2; i <= NR; i++) printf "%s,%.8f,%.2f\n", name[i], e[i], (e[i] - min) * 627.5095}' \
energies.csv > table.csv$ bash collect.sh
skip 05-killed: not finished
$ column -t -s, table.csv
name energy_Eh rel_kcal
01-single-point -76.32127441 65.95
02-opt-freq -76.42636946 0.00
03-dlpno-ccsdt -76.33195934 59.24
04-solvation-smd -76.33297155 58.61These four jobs are the same water molecule at different levels of theory, so the relative energies mean nothing chemically. The one meaningful comparison is gas phase (01) versus SMD water (04) at the same B3LYP/def2-SVP level: −7.34 kcal/mol. In practice you run this script over directories of isomers or conformers computed with one method.
Files created on or copied through Windows can have CRLF (\r\n) line endings. The last field then carries a
\r, invisible characters end up in your CSV, and $ anchors stop matching. If cat -A file shows
^M at the line ends, fix it with tr -d '\r' < old > new or dos2unix.
Quantum ESPRESSO
pw.x prints total energy at every SCF iteration and marks only the converged value with a leading
!, so searching for ^! skips the intermediate values. Below is the silicon SCF example output from the
QE guide. Energies are in Ry; 1 Ry = 13.605693 eV.
$ grep '^!' si.scf.out
! total energy = -93.45273690 Ry
$ grep '^!' si.scf.out | tail -1 | awk '{printf "%.8f Ry = %.6f eV\n", $5, $5 * 13.605693}'
-93.45273690 Ry = -1271.489248 eV
$ grep -c 'convergence has been achieved' si.scf.out
1
$ grep -c 'convergence NOT achieved' si.scf.out
0
$ grep 'JOB DONE' si.scf.out
JOB DONE.
A vc-relax runs a new SCF at every ionic step, so there are several ! lines. The pressure is the last field of the P= lines.
$ grep '^!' si.vcrelax.out | awk '{printf "%d %.8f\n", NR, $5}'
1 -93.44443849
2 -93.45157833
3 -93.45321788
4 -93.45325068
5 -93.45325084
6 -93.45327391
$ grep 'P=' si.vcrelax.out | awk '{print $NF}'
108.71
43.06
5.66
0.39
0.00
-0.03
$ grep -E 'bfgs converged|Final enthalpy' si.vcrelax.out
bfgs converged in 5 scf cycles and 4 bfgs steps
Final enthalpy = -93.4532508364 Ry
The sixth value is an extra SCF run on the final cell after BFGS finishes. A new cell means a new plane-wave basis, so the value shifts
slightly. Report Final enthalpy as the result of the relaxation; a large gap between the two suggests the cutoff is too low.
LAMMPS thermo table
In log.lammps, thermo output sits between a header line starting with Step and a Loop time line.
Each run command adds another such block. This is the Cu slab log from the LAMMPS guide,
with a 10000-step equilibration and a 5000-step production run.
$ awk '$1 == "Step" {on = 1; next} /^Loop time/ {on = 0} on' log.lammps | head -4
0 253.8184 -5749.9163 -5695.3556 -5778.9907
500 253.05769 -5694.4003 -5640.0031 -9546.1221
1000 267.19235 -5695.7562 -5638.3206 1729.171
1500 253.1248 -5692.6357 -5638.2241 -9774.9528
The header is indented, so /^Step/ would miss it. Comparing the first field with $1 == "Step" works regardless
of leading spaces. Keeping the header once as a # comment gives a file that numpy's loadtxt and gnuplot read as is.
$ awk '$1 == "Step" {on = 1; if (!hdr++) print "#", $0; next} /^Loop time/ {on = 0} on' log.lammps > thermo.dat
$ head -3 thermo.dat
# Step Temp PotEng TotEng Press
0 253.8184 -5749.9163 -5695.3556 -5778.9907
500 253.05769 -5694.4003 -5640.0031 -9546.1221
The two blocks are concatenated and the step count restarts at 0 (this input calls reset_timestep 0 before production).
To keep only the production run, count headers and switch on only in the second block.
$ awk '$1 == "Step" {run++; on = (run == 2); next} /^Loop time/ {on = 0} on' log.lammps | head -3
0 247.4071 -5695.3154 -5642.1329 -4114.9433
500 254.87546 -5696.5557 -5641.7678 -4072.5059
1000 242.9701 -5696.8721 -5644.6433 -4897.1177 xtb
xtb prints the same energy in two summary boxes. Below are xtb 6.7.1 outputs of a GFN2-xTB single point on water
(xtb_sp.out) and an optimisation of methanol (xtb_opt.out).
The TOTAL ENERGY line appears only once, at the end, so in an optimisation output it is the final value.
$ grep -E ':: total energy|TOTAL ENERGY' xtb_sp.out
:: total energy -5.070222286727 Eh ::
| TOTAL ENERGY -5.070222286727 Eh |
$ awk '/TOTAL ENERGY/ {print $4}' xtb_opt.out
-8.226118421023
$ grep 'GEOMETRY OPTIMIZATION CONVERGED' xtb_opt.out
*** GEOMETRY OPTIMIZATION CONVERGED AFTER 5 ITERATIONS ***
xtb writes normal termination of xtb to standard error, not standard output. With only xtb mol.xyz > sp.out
that line never reaches the file; save both streams with > sp.out 2>&1 if you want to check for it.
When to switch to Python
grep and awk are right for one or two numbers. For data spread over several blocks (normal-mode vectors, atomic charges, orbital
energies), or sections whose format changes between program versions, a parser library breaks less often.
cclib reads ORCA, Gaussian and other quantum chemistry outputs into numpy arrays, and
ASE's ase.io.read handles most structure formats, including QE outputs and LAMMPS dumps.