Processes and environment

Seeing processes: ps, top, htop

Each running program is a process. Every process has a number, its PID, which you use to inspect or stop it. ps lists the processes started from the current terminal.

$ ps
    PID TTY          TIME CMD
 454500 pts/3    00:00:00 bash
 454505 pts/3    00:00:00 sleep
 454506 pts/3    00:00:00 ps

ps -u $USER shows all of your processes and ps aux everything on the machine. To find one by name, pipe it into grep.

$ ps aux | grep orca

top shows processes live, sorted by CPU use; q quits. htop is an easier-to-read version with colour and per-core bars, but it may not be installed (sudo apt install htop).

top - 00:13:21 up 11:37,  1 user,  load average: 9.72, 9.24, 5.89
Tasks: 392 total,   2 running, 389 sleeping,   0 stopped,   1 zombie
%Cpu(s): 23.6 us,  2.2 sy,  0.0 ni, 73.0 id,  1.1 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :  30962.3 total,   2806.1 free,  12789.0 used,  16344.9 buff/cache
MiB Swap:   8192.0 total,   8190.8 free,      1.2 used.  18173.4 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND

A parallel ORCA run shows processes such as orca_scf_mpi, one per core. If a job is running but %CPU stays near 100%, the parallel settings have not taken effect.

Stopping: Ctrl+C and kill

Ctrl+C interrupts the command running in the foreground. For a program running in another terminal, or one that has frozen, find its PID and use kill.

$ kill 454505          # ask it to terminate (SIGTERM)
$ kill -9 454505       # force it (SIGKILL)

Plain kill asks the program to clean up and exit, which is usually enough. kill -9 cannot be refused and may leave temporary files behind. Try kill first and use -9 only if it does not work. You cannot kill other users' processes. Jobs submitted through the scheduler are cancelled with qdel <job id> (PBS), not kill.

Running in the background

Ending a command with & runs it in the background and gives you the prompt back immediately. A command already running in the foreground can be paused with Ctrl+Z and then sent to the background with bg. The example uses sleep (wait for a number of seconds) in place of a long calculation.

$ sleep 600 &
[1] 453276
$ sleep 900 &
[2] 453277
$ jobs
[1]-  Running                    sleep 600 &
[2]+  Running                    sleep 900 &
$ kill %1
$ jobs
[1]-  Terminated                 sleep 600
[2]+  Running                    sleep 900 &

Surviving a closed terminal: nohup

A program started with just & can die when you close the terminal or the ssh connection drops. Prefixing it with nohup keeps it running. Send its output to a file.

$ cd ~/proj/h2o-opt
$ nohup orca water.inp > water.out 2>&1 &

Without a redirect, nohup prints nohup: ignoring input and appending output to 'nohup.out' and writes to nohup.out. This is fine for short runs in WSL or on a personal workstation. On a cluster, use the scheduler, and do long interactive work inside tmux.

Environment variables

The shell keeps settings in named variables, and $ retrieves a value. Variables that are passed on to child processes (programs the shell starts) are called environment variables.

$ echo $HOME
/home/kim
$ echo $USER
kim
$ name=water
$ echo $name
water
$ bash -c 'echo child sees: [$name]'
child sees: []
$ export name
$ bash -c 'echo child sees: [$name]'
child sees: [water]

No spaces around =: name = water tries to run a command called name. A plain variable exists only in the current shell; export makes it visible to programs the shell runs. Anything ORCA or xtb reads from the environment has to be exported. env or printenv prints all environment variables. Variables and quoting in scripts are covered in detail in the shell guide.

PATH and which

When you type ls, the shell searches the directories listed in PATH, in order, for an executable called ls. command not found means it was in none of them.

$ echo $PATH | tr ':' '\n'
/usr/local/sbin
/usr/local/bin
/usr/sbin
/usr/bin
/sbin
/bin
/snap/bin
$ which grep
/usr/bin/grep
$ command -v cd
cd

In WSL the list continues with a long run of Windows paths starting with /mnt/c/..., so that Windows programs can be called from WSL. which and command -v tell you which program will actually run. command -v also finds shell built-ins like cd, so prefer it in scripts. If you install a program in your home directory, put its directory at the front of PATH.

export PATH=$HOME/orca:$PATH

Leaving out the existing $PATH, as in export PATH=$HOME/orca, means the shell can no longer find even ls. Opening a new terminal restores it.

.bashrc and .bash_profile

Exported variables disappear when the terminal closes. Settings you want every time go into files that bash reads at startup.

Keeping settings in two places gets confusing, so the usual approach is to put everything in ~/.bashrc and have ~/.bash_profile read it. Ubuntu's default ~/.profile already does this. On a cluster, check that ~/.bash_profile contains these lines.

# ~/.bash_profile
if [ -f ~/.bashrc ]; then
    source ~/.bashrc
fi

After editing ~/.bashrc, open a new terminal or run source ~/.bashrc. source runs a file's contents in the current shell. zsh on macOS reads ~/.zshrc instead.

Aliases

An alias gives a long, frequently typed command a short name. Put aliases in ~/.bashrc so they are always available.

# ~/.bashrc
alias ll='ls -lh'
alias rm='rm -i'
alias e='grep "FINAL SINGLE POINT ENERGY"'

With these, e water.out expands to grep "FINAL SINGLE POINT ENERGY" water.out. Aliases work only in interactive shells; they are not expanded inside scripts.

Environment variables for calculations

Calculation programs often take settings such as core count or memory from environment variables.

$ export OMP_NUM_THREADS=4
$ echo $OMP_NUM_THREADS
4
$ ulimit -s
8192
$ ulimit -s unlimited
$ ulimit -s
unlimited

If OMP_NUM_THREADS is unset, a program tries to use every core on the machine. On a laptop that stalls everything else; on a cluster it uses more cores than you requested. In job scripts, match it to the number of cores you asked for. With too small a stack, xtb segfaults on larger molecules for no obvious reason. The specific settings are in the xtb guide.