Files and directories

The directory tree

Linux has no C: or D: drives. Every file hangs off a single tree that starts at /, the root. Folders are usually called directories.

/ and ~ are easy to mix up. / is the root of the whole system, and ordinary users cannot create files there. ~ is your home. Clusters usually give you a larger working area (often called scratch) besides your home; its path and quota differ between clusters, so check your site's documentation (chapter 09).

Absolute and relative paths

An absolute path starts with / and spells out the full location. It means the same thing wherever you use it. A relative path is interpreted from the directory you are currently in.

From inside ~/proj/h2o-opt, water.inp, ./water.inp and ~/proj/h2o-opt/water.inp all name the same file. In job scripts you often cannot be sure where the script will run, so absolute paths are safer there.

Moving around and looking

cd (change directory) moves you; pwd tells you where you are.

$ cd ~/proj
$ pwd
/home/kim/proj
$ ls
h2o-opt  h2o-sp
$ cd h2o-opt
$ cd ../h2o-sp
$ pwd
/home/kim/proj/h2o-sp
$ cd
$ pwd
/home/kim
$ cd -
/home/kim/proj/h2o-sp

cd on its own takes you home, and cd - goes back to the previous directory.

ls options

ls -l gives one file per line with permissions, size and modification time. Options can be combined, as in -la.

$ ls -la ~/proj/h2o-sp
total 60
drwxr-xr-x 2 kim kim  4096 Sep 28 18:12 .
drwxr-xr-x 4 kim kim  4096 Sep 28 18:12 ..
-rw-r--r-- 1 kim kim   186 Sep 28 18:12 water.inp
-rw-r--r-- 1 kim kim 48678 Sep 28 18:12 water.out

Hidden files

Files whose names start with . are not shown by ls. Most configuration files are hidden this way.

$ cd
$ ls
proj
$ ls -a
.  ..  .bash_history  .bashrc  .cache  .profile  .ssh  proj

.bashrc comes back in chapter 05 and .ssh in chapter 06.

Tab completion

Type the first few letters of a name and press Tab; the shell fills in the rest. If there are several candidates, press it again to list them. Typing cd ~/proj/h2 and pressing Tab twice shows h2o-opt/ and h2o-sp/. It saves typing and catches mistakes: if a path will not complete, it probably does not exist.

Create, copy, move, delete

$ cd ~/proj
$ mkdir ch4-conf/conf01
mkdir: cannot create directory ‘ch4-conf/conf01’: No such file or directory
$ mkdir -p ch4-conf/conf01 ch4-conf/conf02
$ cp h2o-sp h2o-sp-copy
cp: -r not specified; omitting directory 'h2o-sp'
$ cp -r h2o-sp h2o-sp-copy
$ mv h2o-sp-copy h2o-sp-old
$ rmdir h2o-sp-old
rmdir: failed to remove 'h2o-sp-old': Directory not empty
$ rm -r h2o-sp-old

A common pattern for rerunning a calculation with a small change is to cp -r the old directory and edit only the input. The old outputs come along too, so delete the stale .out and .gbw files in the new directory before starting.

Wildcards

Special characters in a file name are expanded by the shell into the list of matching names. The shell does this before the command runs, so wildcards work with ls, cp, rm, grep and everything else.

$ ls
conf01.out  conf01.xyz  conf02.out  conf02.xyz  conf03.xyz  conf10.xyz
$ ls *.xyz
conf01.xyz  conf02.xyz  conf03.xyz  conf10.xyz
$ ls conf0?.xyz
conf01.xyz  conf02.xyz  conf03.xyz
$ ls conf0[12].*
conf01.out  conf01.xyz  conf02.out  conf02.xyz
$ ls *.log
ls: cannot access '*.log': No such file or directory

conf0?.xyz does not match conf10.xyz. When nothing matches, bash passes the pattern through unchanged, so ls goes looking for a file literally named *.log and fails.

Naming files

The shell splits arguments on spaces, so a name with a space in it is read as two names. Below, rm did not try to delete my water.xyz; it tried to delete my and water.xyz, and removed the wrong file.

$ ls
my water.xyz  water.xyz
$ rm my water.xyz
rm: cannot remove 'my': No such file or directory
$ ls
my water.xyz

Quote names that contain spaces ("my water.xyz") or put a backslash before the space (my\ water.xyz); tab completion adds the backslash for you. Better still, avoid spaces. Letters, digits, -, _ and . are safe in every program. Non-ASCII characters, parentheses and & in names also cause trouble in calculation programs and scripts.

Organising calculation directories

ORCA, xtb and QE all write plenty of intermediate files into the current directory. Giving each calculation its own directory keeps them from overwriting each other and makes it clear later which input produced which output. One way to lay it out:

proj
├── ch4-conf
│   ├── conf01
│   │   └── ch4.inp
│   └── conf02
│       └── ch4.inp
├── h2o-freq
│   └── water.inp
├── h2o-opt
│   ├── run.pbs
│   ├── water.inp
│   └── water.xyz
├── notes
│   └── log.md
├── scripts
│   └── get-energy.sh
└── structures
    ├── methane.xyz
    └── water.xyz

9 directories, 10 files

This view comes from tree proj (sudo apt install tree). Putting the molecule and the type of calculation in the directory name, and naming consistently, makes it easy to handle everything at once later with a wildcard such as */water.out. Turning that kind of repetition into scripts is the subject of the shell scripting guide.