LPI Linux Certification/LPIC1 Exam 101/Linux Filesystems

Section Overview

1.104.1 Create partitions and filesystems
1.104.2 Maintain the integrity of filesystems
1.104.3 Control mounting and unmounting filesystems
1.104.4 Managing disk quota
1.104.5 Use file permissions to control access to files
1.104.6 Manage file ownership
1.104.7 Create and change hard and symbolic links
1.104.8 Find system files and place files in the correct location

Linux filesystems
Create partitions and filesystems
Maintain the integrity of filesystems
Control mounting and unmounting filesystems
Managing disk quota
Use file permissions to control access to files
Manage file ownership
Create and change hard and symbolic links

Create partitions and filesystems

Overview

Description: Candidates should be able to configure disk partitions and then create filesystems on media such as hard disks. This objective includes using various mkfs commands to set up partitions to various filesystems, including ext2, ext3, reiserfs, vfat, and xfs.

Key files terms and utilities include:
fdisk mkfs

Partitions

Media can be divided into partitions. Partitions are usually created at installation time but can also be created with the fdisk program or other utilities. This will divide the media in partitions and different filesystems can be built and different operating systems can be installed.

IDE is recognized as follows:

Primary Master:

/dev/hda : Full disk

/dev/hda1: First partition

/dev/hda2: Second partition

Primary Slave: /dev/hdb
Secondary Master: /dev/hdc
Secondary Slave: /dev/hdd

SCSI is recognized as follows:

ID1:

/dev/sda: Full disk

/dev/sda1: First partition

ID2: /dev/sdb

USB and FireWire disks are recognized as SCSI disks.

Once it is partitioned it is possible to build a filesystem on every partition.

Filesystems

Filesystems exist to allow you to store, retrieve and manipulate data on a media. Filesystems maintain an internal data structure (meta-data) that keeps all you data organized and accessible. The structure of this meta-data gives the characteristic of the filesystem. A filesystem is accessed by a driver through the organized meta-data structure. When Linux boots it reads in /etc/fstab all the filesystems that needs to be mounted and checks if they are in an usable state.

When a power failure occurs Linux won't be able to unmout the filesystem properly and some data in the cache won't be synchronized on the media. The meta-data may be corrupted.

Once you reboot the system, it will detect this and do a fsck to the full meta-data structure for a consistency check. This can take a very long time. Few minutes to few hours in proportion of the media size. Journaling a filesystem is adding a new data structure called a journal. This journal is on-disk and before each modification to the meta-data is made by the driver it is first written into the journal. Before each meta-data modification the journal maintains a log of the next operation.

Now, when a power failure occurs there is only the need to check the journal. A journaling filesystem recovery is very fast. It just needs to go through the logs and fix the latest operation. Journaling filesystem recovery can take only few seconds.

On a cluster systems, journaling allows to quickly recover a shared partition of a node that went down.

Linux filesystems

ext2: Old very stable Linux filesystem. Efficient for files larger than ~2-3K.
ext3: Journaling extension for ext2. Possible to move a filesystem back and forth between ext2 and ext3.
Reiserfs: Journaling filesystem. 8-15 time faster than ext2 when manipulating small files.
XFS: A powerful journaling filesystem with possible quota and ACL
Msdos: MS-Windows FAT filesystem type. (mainly used for floppy)
Vfat: MS-Windows FAT filesystem type. (mainly used for large hdd partitions)
NTFS (ReadOnly but loop files): MS-Windows journaling filesystem
SMBFS: A filesystem to mount windows or samba sharing from Linux
NFS: Network File System

...

Filesystem tree

A Linux file system has one top directory called root (/) where all sub directories of the entire system is stored. The sub directories can be another partition, a remote directory, or a remote partition accessible through the network with NFS protocol.

Create filesystems

To create a file system on a partition, use mkfs.

mkfs [options] -t [fstype] device [blocksize]

Common options:

-t: fstype: File system type.

-c : Check the device for bad blocks before building the filesystem.

The full partition will be erased and organized to the type of filesystem requested. There is no undo command. The fstype possible are: msdos, ext2, ext3, reiserfs,minix,xfs

The blocksize allows to customize the block size for your filesystem.

Examples:

mkfs -t msdos /dev/fd0
mkfs -t reiserfs /dev/hdd1  4096

Create extended filesystems

To create an extended (ext2, ext3) filesystem on an partition, use mke2fs.

mke2fs [options] device [blocksize]

Common options:

-b: Specify the block sizefstype: File system type.

-c : Check the device for bad blocks before building the filesystem.

-j: Create the filesystem with an ext3 journal.

-L: Set the volume label for the filesystem.

With mke2fs it is possible to store the super-block the journal information on another device. Examples:

mkefs -b 2048 /dev/fd0 -L floppy
mkfs -V
mke2fs 1.26 (3-Feb-2002) Using EXT2FS Library version 1.263

Monitoring disk usage

To print the disk usage, use du.

du [options] [files...]

Common options:

-a: All files not just directories

-b: Print size in bytes

-c: Total

-h: Human readable format. (1K, 20M,...)

Examples:

$ du -ch Documents
112k    Documents/Cours/LPI101
4.0k    Documents/Cours/LPI102
4.0k    Documents/Cours/LPI201
4.0k    Documents/Cours/LPI202
124k    total

du -sk ~ # Sums up your total disk usage in kilobytes

du -ak ~ | sort -n more # Display every file and its disk space in numerical order.

Filesystem disk space

A filesystem is composed of a meta-data structure plus a list of blocks. To print the filesystem disk space usage, use df.

df [options] [files...]

Common options:

-a: All included filesystems with 0 blocks.

-t: Limit listing to a filesystem type.

-h: Human readable format. (1K, 20M,...)

Examples:

$ df -t reiserfs -h
F           1k-blocks      Used Available Use% Mounted on
/dev/hda3             28771528   3121536  25649992  11% /

$ df -t ext2 -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda1              15M  3.8M   10M  27% /boot

Exercises

Exercises results.

Maintain the integrity of filesystems

Overview

Description: Candidates should be able to verify the integrity of filesystems, monitor free space and inodes, and repair simple filesystem problems. This objective includes the commands required to maintain a standard filesystem, as well as the extra data associated with a journaling filesystem.

Key files terms and utilities include:
du df fsck e2fsck mke2fs debugfs dumpe2fs tune2fs

Checking filesystems

To check filesystems consistency, use fsck.

fsck [options] -t [fstype] device [fsck-options]

Common options:

-A: Go through the /etc/fstab file and try to check all file systems. Typically used at boot time from a script.

-t fslist: Specify the type of file system to be checked. With -A, only filesystems that match fslist are checked

-C: Display completion/progression bar.

Common fsck-options:

-a: Automatically repair.

-r: Interactively repair.

Examples:

fsck -t msdos /dev/fd0 -a
fsck -t reiserfs /dev/hda2 -r

Checking extended filesystems

To check extended filesystems consistency, use e2fsck.

e2fsck [options] device

Common options:

-b: Use an alternate super block filename.

-c: This option makes badblocks program to run and marks all the bad blocks.

-f: Force checking even if the filesystem seems clean.

-a or -p: Automatically repair.

-y: non-interactive mode

Examples:

e2fsck -ay /dev/fd0
e2fsck -f /dev/hda2

Debugging extended filesystems

The debugfs program is an interactive file system debugger. It can be used to examine and change the state of an ext2 file system.

debugfs device

Common commands:

Help: Give an help on all the command available.

Stats: Give a statistics of the media.

Stat file: Give a statistic of the file on the media.

logdump: Dump the journal's media into a file

Example:

stat haut.gif
Inode: 14   Type: regular    Mode:  0644   Flags: 0x0   Generation: 67558
User:     0   Group:     0   Size: 3786
File ACL: 0    Directory ACL: 0
Links: 1   Blockcount: 8
Fragment:  Address: 0    Number: 0    Size: 0
ctime: 0x3ddf3840 -- Sat Nov 23 09:11:44 2002
atime: 0x3ddf3840 -- Sat Nov 23 09:11:44 2002
mtime: 0x3ddf3840 -- Sat Nov 23 09:11:44 2002
BLOCKS:
(0-3):55-58
TOTAL: 4

Dumping extended filesystems info

To print the super block and blocks group information of an extended filesystem, use dumpe2fs.

dumpe2fs [options] device

Common options:

-b: print the bad blocks of the filesystem.

-h: Display only the superblock information.

Example:

dumpe2fs -h /dev/fd0
dumpe2fs 1.26 (3-Feb-2002)
Filesystem volume name:   floppy
Last mounted on:          <not available>
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              184
Block count:              1440
Reserved block count:     72
Free blocks:              1258
Free inodes:              168
First block:              1
Block size:               1024
First inode:              11
Inode size:               128
...

Tuning extended filesystems

To tune an extended filesystem, use tune2fs.

tune2fs [options] device

Common options:

-i#: Interval between filesystem checks [d|m|w].

-l: List the contents of the filesystem superblock.

-L: Set the volume label of the filesystem.

Examples:

tune2fs -L floppy /dev/fd0
tune2fs -l /dev/fd0
(Same output as dumpe2fs -h /dev/fd0)

tune2fs 1.26 (3-Feb-2002)
Filesystem volume name:   floppy
Block count:              1440
Reserved block count:     72
Free blocks:              1258
Free inodes:              168
First block:              1
Block size:               1024
First inode:              11
Inode size:               128
...

Exercises

Exercises results

Build an ext2 file system, with a block size of 2048 bytes, on a floppy.
Change the label of the floppy to BACKUP.
Try to add a journal on the floppy media.
Use debugfs to validate your floppy file system information, and check when it was last accessed.
Use watch to monitor the size when you copy a big file.
Create a shell script to list all files on the floppy bigger than 100 Kb.
Display file system usage for all MSDOS file systems.
Which directory MUST exist in / to qualify this OS as Linux?
What is the file system usage of /proc?

.

Attach and detach filesystems

Control mounting and unmounting filesystems

Overview

Description: Candidates should be able to configure the mounting of a filesystem. This objective includes the ability to manually mount and unmount filesystems, configure filesystem mounting on bootup, and configure user mountable removable filesystems such as tape drives, floppies, and CDs.

Key files terms and utilities include:
/etc/fstab
mount umount

Attach a filesystem

The mount command serves to attach the file system found on some device to the big file tree.

mount [options]
mount [options] [-t vfstype] [-o options] device dir

If the device or directory is listed in /etc/fstab you can use the following:

mount [options] [-o options [,...]] device | dir

Normally only root has the privilege to mount devices unless it is specified in the /etc/fstab file. Examples:

# Print all the mounted filesystems (/etc/mtab).
mount

# Mount devices or dirs listed in /etc/fstab.
mount -a

# Mount /dev/hdc partition in read only mode without updating /etc/mtab.
mount -n -o ro /dev/hdc /mnt

# Allow a user to mount the CDROM if the following line is in /etc/fstab:
# /dev/cdrom /media/cdrom iso9660 ro,user,noauto,unhide
mount /media/cdrom 
mount /dev/cdrom

# Sync in realtime
mount -o sync /dev/sdb1 /mnt/usb

Detach a filesystem

To detach a filesystem from the file tree, use umount.

umount [options]
umount [options] [-o options [,...]] device | dir

A busy filesystem cannot be unmounted.

Open files
Working directory of a process.

Examples:

umount -a # Unmount devices or dirs listed in /etc/fstab.
umount /mnt # Unmount the filesystem attached to /mnt.
umount /media/cdrom  # Allow a user to unmount the CDROM if the following line is in /etc/fstab:
/dev/cdrom  /media/cdrom  iso9660  ro,user,noauto,unhide

File system information

The file /etc/fstab contains all the file systems and related information that will be used when doing a mount -a. (Boot time)

The file /etc/mtab is maintained by the kernel and keeps track on what is or isn't mounted. The /etc/fstab format is:

#Device     Mount point    Fs type  Options             1 2
/dev/hda3   /              reiserfs defaults            1 2
/dev/hda1   /boot          ext2     defaults            1 2
/dev/cdrom  /media/cdrom   auto     ro,noauto,user,exec 0 0
usbdevfs    /proc/bus/usb  usbdevfs noauto              0 0
/dev/hda2   swap           swap     pri=42              0 0

Common options:

ro: read only

noauto: Don't mount automatically

exec: Can execute binary on the filesystem

suid: Allow to setuser bit

user: Allow a user to mount/unmount it

unhide: hidden file visible

async: All operations will be done asynchronously

default: rw, suid, dev, exec, auto, nouser, and async

Exercises

Exercises results

Create a line in /etc/fstab that allows any user to access the floppy disk. Check that you can mount the floppy and can create a file with touch.
Do the following manipulation:
- Create an ext2 file system on the floppy.
- Mount the floppy.
- Copy all the files /etc/*.conf into the floppy.
- Unmount it. What's happening?
- Mount it back and check that all the files are there.
- Issue the following command:
- Tar cvf /dev/fd0 /etc/*.conf
- Try to mount it back. What's happening?
- Use tar to view the contents of the floppy..

Managing disk quota

Overview

Description: Candidates should be able to manage disk quotas for users. This objective includes setting up a disk quota for a filesystem, editing, checking, and generating user quota reports.

Key files terms and utilities include:
quota edquota repquota quotaon

Managing quotas for filesystems
Printing quota reports
Managing disk quota

Quotas

On a system, root can manage the usage of disk space per user and per filsystems. The two limits that can be setup are: The soft limit (soft =) specifies the maximum amount of disk usage a quota user is allowed to have. The hard limit (hard =) specifies the absolute limit on the disk usage a quota user can't go beyond it. There is also the possibility to setup a grace period that will enforce the soft limit only after an amount of time specified.

Setting up quotas for users

1) The keyword usrquota or/and grpquota must be added in file /etc/fstab for the partition interested.

/dev/fd0  /home/yann/mnt auto    rw,noauto,user,usrquota 0 0
/dev/hda5 /home     ext2    defaults,usrquota,grpquota 1 2

2) Add in each root filesystems the file user.quota or/and group.quota.

touch /mnt/aquota.user
touch /home/aquota.user
touch /home/aquota.group 
chmod 600  /mnt/aquota.user 
chmod 600  /home/aquota.user 
chmod 600  /home/aquota.group

Only root can do the quota administration and once the empty files have been created some disk quota can be set such as:

Soft limitation on number of files and inodes.
Hard limitation on number of files and inodes if the grace time is set.

Setting up quotas for users

3) Check the setting

quotacheck -v mnt
quotacheck: Scanning /dev/fd0 [/home/yann/mnt] done
quotacheck: Checked 6 directories and 1 files

4) Enable quota on the disk

quotaon -av
/dev/fd0 [/home/yann/mnt]: user quotas turned on

5) Customize the disk quota limits:

$ edquota -u yann
Disk quotas for user yann (uid 500):
Filesystem    locks       soft       hard     inodes     soft     hard
/dev/fd0       15          0          0          4        0        0

$ edquota -g yann

$ edquota -t
Grace period before enforcing soft limits for users:
Time units may be: days, hours, minutes, or seconds
Filesystem             Block grace period     Inode grace period
/dev/fd0                      7days                  7days

List quotas

To list quotas for a user or group, use quota.

quota [options] [user|group]

Common options:

-u:Default, print user quotas.

-g:Print group quotas for the group of which the user is a member.

-q: Print a more terse message, containing only information on filesystems where usage is over quota.

Example:

quota -u yann

Display a quota report

To display a quota report, use repquota.

repquota [options] [user|group]

Common options:

-a: Report on all filesystems indicated in /etc/mtab to be read-write with quotas.

-g: Report for group.

Example:

$ repquota /dev/fd0
*** Report for user quotas on device /dev/fd0
Block grace time: 7days; Inode grace time: 7days
                Block limits                   File limits
User            used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root   --       8       0       0              2     0     0
yann   --      15       0       0              4     0     0

Exercises

Exercises results

Setup a soft limitation for any users that have their home directory in /home to 500M.
Change the grace time to 0.
Log as the user and check if the limitation works..

File Permissions and Security

Overview

Unix File Security File and Directory Permissions Default Permissions Changing File Permissions Changing File Owner and Group More Privileges Unix file security The biggest security problem may be you. You own any directory or files you created. You are responsible for accessibility of your files. You decide who can access which files and directories.

In your home directory you will be able to grant different levels of permission to yourself, users in your group and the all other users.

File and Directory Permissions

The permission of a file or of a directory can be viewed with ls -l.

File permissions

Examples of file permissions:

ls -l readme
-rwxrw---- 1 toto users 14 Jul 5 10:00 readme

This means read,write, and execution permissions for user toto, read and write permissions for members of group users. No permissions for others. (0760)

ls -l /etc/hosts
-rw-r--r-- 1 root root 14 Jul 5 10:00 /etc/hosts

This means read and write permissions for user root, read permissions for members of group root and all others. (0644)

Examples of directory permissions:

ls -ld /bin
drwxr-xr-x 2 root root 4096 Jul 5 10:00 /bin

This means read,write, and execution permissions for user root, read and execution permissions for members of group root and others. (0755)

ls -l /home/toto
drwxr-xr-x 10 toto  users 4096 Jul 5 1:00 /home/toto

This means read, write, and execution permissions for user toto, read and execution permissions for members of group users and others. (0755)

Default permissions

The default permissions when creating a file are 0666 and when creating a directory are 0777. Most of the systems overwrite this at boot time with the program umask. Generally the mask value is 022. It means the write for group and other will be blocked. To check or change the mask value, do:

umask 
umask 066

Examples for file:

default: rw- rw- rw- (0666)
umask: 0 2 2 (0022) Block
result: rw- r-- r-- (0644)

Examples for directory:

default: rwx rwx rwx (0777)
umask: 0 2 2 (0022) Block
result: rwx r-x r-x (0755)

Changing file permissions

To change permissions on a file or directory, use chmod. To overwrite the existing permissions, do:

chmod 0755 /tmp #rwx for user, rx for group and others

To change add or cancel some permissions without overwriting all the existing permissions, do:

chmod u+w readme  # Add write permission for user
chmod +r readme  # Add read permission for everybody
chmod -r readme  # Remove read permission for everybody
chmod u+x,g=r readme  # Add execution for user and set read for group
chmod u=rwx,go=rx readme  # Set read write and execution for user, read and execution for group and others

To change in recursive mode, use the -R option.

chmod -R +x /sbin/*

Exercises

Exercises results.

Manage file ownership

Overview

Description: Candidates should be able to control user and group ownership of files. This objective includes the ability to change the user and group owner of a file as well as the default group owner for new files.

Key files terms and utilities include:
chmod chown chgrp

Changing file owner and group

To change the owner of a file or directory, use chown.

chown yann mon_fichier.txt

To change the group of a file or directory, use chgrp.

chgrp dialout caller

The programs gpasswd and yast2 allow you to administrate groups.

gpasswd [-A user,...] [-M user,...] group

-A: Add users with group administrator privileges.

-M: Add members in group.

Group administrators can add or delete members of the group

gpasswd -d toto users
gpasswd -a toto users

Group administrators can set or remove the password for the group.

gpasswd users
gpasswd -r users

More privileges

It is possible to give more privileges to a user when it executes a particular script or program by setting the uid or gid bit of the file.

If the bit is set, the process will inherit the permissions of the owner of the file not the permissions of the user. To set the effective uid or gid, use chmod.

chmod 2640 [file] # (2) gid is inheritable for group.
chmod 4640 [file] # (4) uid is inheritable for user.

Example of such program is /bin/passwd.

The sticky bit can also be set and can make the program text segment resident in RAM. chmod 1640 [file] (1) The file program stays in RAM.

Exercises

Exercises results

1) Write the command line by using letters with chmod to set the following permissions:

rwxrwxr-x :
rwxr--r-- :
r--r----- :
rwxr-xr-x :
rwxr-xr-x :
r-x--x--x :
-w-r----x :
-----xrwx :

2) Write the command line by using octal numbers with chmod to set the following permissions:

rwxrwxrwx :
--x--x--x :
r---w---x :
-w------- :
rw-r----- :
rwx--x--x :

3) With the following umask values what would be the files and directories creation permissions?

umask = 0027
File permissions:
Directory permissions:

umask = 0011
File permissions:
Directory permissions:

umask = 0541
File permissions:
Directory permissions:

umask = 0777
File permissions:
Directory permissions:

4) Create two user accounts

Logging in id: tst1, group users, with bash shell, home directory /home/tst1
Logging in id: tst2, group public, with bash shell, home directory /home/tst2
For the two accounts set a password.

Logging in as tst1 and copy /bin/ls into tst1 home directory as myls. Change the owner of myls to tst1 and the permissions to 0710. What does this permission value mean?

Logging in as tst2 and try to use /home/tst1/myls to list your current directory. Does it work ?

Create in /etc/group and /etc/gshadow a new group labo with tst1 and tst2. Change the owner group of myls to labo.

Try again from tst2 account to execute /home/tst1/myls to list your current directory. Does it work?.

Hard and symbolic links

Overview

Description: Candidates should be able to create and manage hard and symbolic links to a file. This objective includes the ability to create and identify links, copy files through links, and use linked files to support system administration tasks.

Key files terms and utilities include:
ln

Links

Use link when: You want to create a pathname to a file. Set a shorter or fixed pathname to a file.

To link one file to another, use ln:

ln [options] filename linkname
ln [options] filename linkdirectory

Common options:

-f force: clobber existing link

-s symbolic link

The default links are hard links (ln without an option). On Windows they're called shortcuts. A hard link can only be created to an existing file on the same physical device, after creation no visible association can be displayed between a link name and a file name. A symbolic link can be created on a file that doesn’t exist yet, the association between the link name and the file name can be viewed with the ls command.

Linking to a file

The symbolic and hard link can be displayed with ls -l. Symbolic link are indicated with an arrow: link_name->real_filename.

$ ls -l /dev/midi
lrwxrwxrwx   1   root   root        6    Jul 4 21:50   /dev/midi -> midi00

Hard links are indicated with the number of links counter (3-1=2 in this case).

$ ls -l readme
-rwxrwxrwx   3   yann   users       677  Jul 4 21:50   readme

When removing a link name, use rm. Only the link will be removed not the linked file.

Exercises

Exercises results

Create a directory etc and bin in your home directory.
Copy all the files in recursive mode from /etc to your etc directory and do the same for /bin to bin.
In your local etc directory rename all files *.conf by *.conf.bak
Create in your home directory a symbolic link call dir that points to your local bin/ls. Check if dir do execute ls.
Remove the dir link. Is bin/ls still there?.

Find system files and place files in the correct location

Overview

Exercices

Exercises results