Guide to Unix/Explanations/Filesystems and Swap

Wikipedia has related information at File system

A common feature in Unix-like filesystems is that all files appear in one file hierarchy. The filesystem storing the base of the hierarchy is referred to as the root filesystem. /, the directory containing all other directories, is itself is called the root directory. Other filesystems are mounted on directories under / which make these filesystems look like directories in the root filesystem. For example, a CD-ROM containing files might be mounted at /mnt/cdrom. In this example /mnt/cdrom is called a mountpoint. Access to devices is supplied by another filesystem mounted at /dev. Here files representing physical, and virtual devices can be accessed. Filesystems themselves are represented as files in /dev such as /dev/cdrom and can be mounted to directories in the root filesystem such as /mnt/cdrom. Each disk or storage medium may contain one or more filesystems, each of which contains files which can contain data. Some disks are also used for swap, which supplies a temporary storage space for data in memory, when memory is full. Though "swap" resides on a disk, it is not actually a filesystem.

To do:

List of Filesystem Types

The choice of Unix-like system influences the choice of filesystem. There are two kinds of filesystems:

Filesystems with full Unix support can hold all types of Unix files, including normal files, directories, named pipes, symbolic links, and device nodes. They also support the entire Unix user and group permission model. This makes these filesystems good for mounting on /, and also /home, /tmp, /usr, /var, and other such filesystems if those are separate partitions.
Non-Unix filesystems were intended for non-Unix operating systems or to exchange files between different filesystems. Sometimes, the permission on every file on the filesystems must be the same, and only normal files and directories are allowed.

Filesystems with full Unix support

Unix File System (ufs/ffs) is the filesystem of *BSD and several commercial Unix variants. UFS is the on-disk layout, while FFS is some kernel optimizations for UFS in *BSD. There are several incompatible extensions to UFS, including the UFS2 of FreeBSD, NetBSD, and Mac OS X. Therefore, UFS is good for the root filesystem, but not good for trading files between different Unix-like operating systems that implement UFS incompatibly.
Linux Second Extended Filesystem (ext2, ext3 or ext4) was inspired by ufs/ffs, and is similar. ext2 is the main filesystem implementation for Linux, and can be used as the root filesystem. Unlike ufs/ffs, ext2 always has the same on-disk layout, and can be shared between different Linux systems, and other systems that understand ext2. Note that ext3 is really the same as ext2, but with journaling enabled and supports Hashed B trees which speed up lookups in larger directorys. Linux also provides other filesystems that you might use instead of ext2.
reiserfs is an alternative to ext2 on Linux. Reiserfs is a fast journaled filesystem written by Hans Reiser.
jfs is a high performance journaled filesystem written by IBM originally for IBM AIX, and then ported to Linux by the same.
xfs is a high performance journaled filesystem written by SGI. Originally written for SGI's Irix operating system this fs has been ported to Linux using a shim layer converting Linux VFS and locking semantics to Irix semantics.
HFS+ (hfsplus) is the main filesystem implementation for Mac OS X, and can be used as the root filesystem. Like ext2, HFS+ always has the same on-disk layout, and can be shared between different Mac systems. HFS+ supports both Unix and Mac file attributes. HFS+ has binary trees, which make it faster than ffs and ext2 sometimes.

Foreign Filesystems

iso9660 is a common filesystem for CD-ROMs. Most Unix-like systems can read it.
UDF is a common filesystem for DVDs. Some newer Unix-like systems can read and write it.
msdos (FAT) is the filesystem from MS-DOS, FreeDOS, ReactOS, and Microsoft Windows. Normally you can have normal files and directories, but not special files, symbolic links, nor Unix file permissions. Because nearly every Unix-like system (including Linux, *BSD, and Mac OS X) and DOS/Windows understand it, the msdos filesystem is often used to trade files between computers or on computers which boot multiple operating systems.
ntfs is a newer filesystem from Microsoft Windows NT. Some Linux kernels can read them.

Disk Partitioning

If the disk is not partitioned, it can be used for one filesystem, or entirely as swap space.

On Linux, IDE/ATA disks are called /dev/hda, /dev/hdb, /dev/hdc...
On Linux, SCSI disks are called /dev/sda, /dev/sdb, /dev/sdc...
On *BSD, IDE/ATA disks are called /dev/wd0c, /dev/wd1c, /dev/wd2c...
- On FreeBSD, IDE/ATA disks are called /dev/ad0c, /dev/ad1c, /dev/ad2c...
On *BSD, SCSI disks are called /dev/sd0c, /dev/sd1c, /dev/sd2c...
On *BSD, compact disks (CDs) are called /dev/cd0c, /dev/cd1c...
On Mac OS X, disks are called /dev/disk0, /dev/disk1, /dev/disk2...
NetBSD/OpenBSD disk images are /dev/svnd0c, /dev/svnd1c, /dev/svnd2c...

Partioning the disk allows for multiple filesystems and swap spaces.

On Linux, hda is partitioned into /dev/hda1, /dev/hda2, /dev/hda3, /dev/hda4...
On *BSD, wd0 is partitioned into /dev/wd0a, /dev/wd0b, /dev/wd0c, /dev/wd0d...
On Mac OS X, disk0 is partitioned into /dev/disk0s1, /dev/disk0s2, /dev/disk0s3...

In addition, *BSD has some extra rules:

Partition 'c' (/dev/wd0c) must be the entire disk.
If a *BSD operating system boots on the disk, then partition 'a' must be root (/) and 'b' must be swap. (If there is no OS on the disk, then 'a' can be used for any filesystem).

The chapter Partitioning Disks describes the formats in more detail and how to edit the partition maps.

Mounting Filesystems and Activating Swap

The commands one uses are:

mount to mount filesystems
umount to unmount filesystems
swapctl, swapon, swapoff

Simple mounting

Suppose /dev/hdb is a device (for example, a CD drive) and /mnt/cdrom is the place to mount. Then the mount command is:

mount /dev/hdb /mnt/cdrom

There are more mount options so look for mount somewhere in Guide to UNIX/Commands.

Mounting with fstab

The /etc/fstab file helps you mount by specifying a default mount point.

Create an /etc/fstab entry for /dev/hdb.
Run this command

mount /dev/hdb

Unmounting

It is called umount, not unmount...

umount /dev/hdb

Mounts by Users

The user mounts are mounting and unmounting of filesystems by users other than the superuser. Unfortunately, the different Unix variants have different approaches to this task. (An alternative is to configure "sudo" to allow users to become root when mounting and unmounting.) Here are some ways that root can permit user mounts:

Linux allows the user to perform mounts permitted through the /etc/fstab file. Each line with the mount option "user" describes a permitted mount. For example, mount /dev/cdrom by a non-root user will mount /dev/cdrom using the mount point and options shows in /etc/fstab, but only if "user" is listed as a mount option.
Many popular BSD-family operating systems allow the user to mount any device which the user can already read and write. The user can pick any mount point which the user owns. A sysctl value must be changed by the root user in order to do this on most of them.
- In OpenBSD, kern.usermount must be set to 1.
- In NetBSD, vfs.generic.usermount must be set to 1.
- In FreeBSD, PcBSD, and DragonflyBSD, vfs.usermount must be set to 1.

Union Mounts

Normally, filesystems are mounted on empty directories. If the directory was not empty, its files are hidden until the filesystem is unmounted. The union mount allows these files to show through. Each existing file from the directory only shows if there is no file in the same place on the mounted filesystem. All new files are made on the mounted filesystem. Note that union mounts are strange and might not work well on your system.

Some operating systems provide a mount(8) option -o union for this. In this example, a non-partitioned SCSI disk sd0 will be mounted on /etc (which is not normally a mount point). Any existing /etc files that we have that are not also on sd0 will still be available.

# mount -o union /dev/sd0c /etc

Disk Images

A filesystem can be stored on a file on another filesystem. Such files are called disk images and they have several applications:

Use dd to copy data from a device node to a normal file to preserve the content of a small disk, for example a floppy disk.
Some programs let you make iso9660 disk images before burning them to CD. One might want to mount your disk image read-only to examine it before the burn. One can also move such images to a computer with a CD burner, or offer the image for download so others may burn it.
If you have no free space for new partitions on your disk, but some partition has free space, then you can create a disk image if you need a new filesystem. Sometimes this helps in the creation of encrypted filesystems.

This guide only describes raw disk images, which contain only the filesystem. There are other formats (such as the Mac OS X NDIF format) but these must be converted or handled specially.

Mounting Disk Images on NetBSD and OpenBSD

Normally, NetBSD and OpenBSD only allow mounting of filesystems stored on devices. Thus five steps are involved:

Associate the disk image with a svnd device.
Mount the svnd device.
Use the disk image.
Unmount the svnd device.
Disassociate the disk image from the device.

Association and dissociation are handled by the vnconfig(8) command. In this example, /the/disk/image will be associated with /dev/svnd0c. This particular disk image has a disklabel with a filesystem on partition 'a', which we mount on /mnt. If there was no disklabel, we would instead mount partition 'c'.

# vnconfig -c /dev/svnd0c /the/disk/image
# mount /dev/svnd0a /mnt
...
# umount /dev/svnd0a
# vnconfig -u /dev/svnd0c

Mounting Disk Images on FreeBSD 5

In FreeBSD 5, one uses mdconfig(8) with a vnode backed storage.

Mounting Disk Images on Linux

The mount(8) option -o loop can be used to mount disk images instead of devices.

# mount -o loop /the/disk/image /mnt
...
# umount /mnt

Some Linux kernels can mount a cloop (compressed loop), which is a compressed disk image. Such an image must be mounted read-only. One first prepares the image and then compresses it how? Then one mounts it how? The Knoppix livecd uses a cloop to fit more programs to the CD.

Creating Disk Images

Creating a new, empty disk image involves two steps before you mount it and start using it:

Use dd(1) to make a new file of zeros with the correct size.
Format the disk image with partitions and filesystems.

Here is an example. We use dd and an unlimited supply of zeros, /dev/zero, to create a 1440 kilobyte disk image (1440 blocks of 1024 bytes). Then we use mkfs.ext3(8) to create an unpartitioned ext3 filesystem. Note that mkfs.ext3 does no mounting, so it cares not if it formats a device or a disk image. (FIXME is this correct?)

$ dd bs=1024 count=1440 if=/dev/zero of=example
$ mkfs.ext3 example

The *BSD newfs(8) and newfs_msdos(8) utilities insist on formatting devices only. This is solved with vnconfig(8). If we wanted to partition the disk image, we would also have made a disklabel. One can ignore the imaginary disk geometry.

$ dd bs=1024 count=1440 if=/dev/zero of=example
$ sudo vnconfig -c /dev/svnd0c example
$ sudo newfs /dev/rsvnd0c
...
$ sudo vnconfig -u /dev/svnd0c

In fact, on *BSD it is a good idea to always have a disklabel, especially if when creating an ffs filesystem, so that the filesystem will be correctly listed in the label, and the letter will be different from 'c'.

$ dd bs=1024 count=1440 if=/dev/zero of=example
$ sudo vnconfig -c /dev/svnd0c example
$ sudo disklabel -E svnd0

Initial label editor (enter '?' for help at any prompt)
> a
partition: [a]
offset: [0]
size: [2880]
FS type: [4.2BSD]
> p
device: /dev/rsvnd0c
type: SCSI
disk: vnd device
label: fictitious
bytes/sector: 512
sectors/track: 100
tracks/cylinder: 1
sectors/cylinder: 100
cylinders: 28
total sectors: 2880
free sectors: 0
rpm: 3600

16 partitions:
#             size        offset  fstype [fsize bsize  cpg]
  a:          2880             0  4.2BSD   2048 16384   32 # Cyl     0 -    28*
  c:          2880             0  unused      0     0      # Cyl     0 -    28*
> q
Write new label?: [y]
$ sudo newfs /dev/rsvnd0a
...
$ sudo vnconfig -u /dev/svnd0c

Encrypted Filesystems

Encrypted Filesystems on OpenBSD

Because OpenBSD has integrated crypto, one would expect that there is a way to have an encrypted filesystem. OpenBSD only provides Blowfish encryption for disk images. This is possible with the -k flag to vnconfig(8), used with -c.

In this example, we will mount the encrypted disk image called /secrets, which contains a single partion letter 'a':

# vnconfig -ck /dev/svnd0c /secrets
Encryption key:
# mount /dev/svnd0a
...
# umount /dev/svnd0a
# vnconfig -u /dev/svnd0c

Note that if you entered the wrong encryption key, vnconfig will still succeed. The mount will fail because the disklabel and filesystem header are not readable. In this case, you must vnconfig -u and then vnconfig -ck with the correct key. Make sure to use the correct key when initially formatting a new disk image.

To have an entire encrypted disk partition, create a normal unencrypted disk image, mount it, then fill the partition with one large file to serve as the disk image.

Encrypted Filesystems on Linux

Some Linux kernel configurations let you use losetup to encrypt disk images. Think of lo(opback) setup, not of lose tup.

References

Manual pages for FreeBSD and several Unix-like systems: http://www.FreeBSD.org/cgi/man.cgi
Little UFS2 FAQ http://sixshooter.v6.thrupoint.net/jeroen/faq.html