1.1. Introduction

When creating tiny-sized and embedded Linux systems, every byte of the storage device (floppy, flash disk, etc.) is very important, so compression is used everywhere possible. Also, compressed file systems are frequently needed for archiving purposes. For huge public archives, as well as for personal media archives, this is essential.

SquashFS brings all this to a new level. It is a read-only file system that lets you compress whole file systems or single directories, write them to other devices/partitions or to ordinary files, and then mount them directly (if a device) or using a loopback device (if it is a file). The modular, compact system design of SquashFS is bliss. For archiving purposes, SquashFS gives you a lot more flexibility and performance speed than a tarball archive.

SquashFS is distributed as a Linux kernel source patch (which enables SquashFS read support in your kernel), the mksquashfs tool, which creates squashed file systems (in a file or on a block device) and the unsquashfs tool, which extracts multiple files from an existing squashed file system.

The latest SquashFS release tree is 3.x, the former one was 2.x. This document describes both these releases with proper notes given. For example, if some feature or parameter is different in these release trees, it will be written as follows: new value (3.x) or old value (2.x)

1.2. Overview of SquashFS

• Data, inodes and directories are compressed

• SquashFS stores full uid/gids (32 bits), and file creation time

• Files up to 2^64 bytes are supported; file systems can be up to 2^64 bytes

• Inode and directory data are highly compacted, and packed on byte boundaries; each compressed inode is on average 8 bytes in length (the exact length varies on file type, i.e. regular file, directory, symbolic link, and block/character device inodes have different sizes)

• SquashFS can use block sizes up to up to 64 Kb (2.x) and 1Mb (3.x). The default size is 128Kb (3.x), which achieves greater compression ratios than the normal 4K block size

• By the 2.x release it was introduced the concept of fragment blocks: an ability to join multiple files smaller than block size into a single block, achieving greater compression ratios

• File duplicates are detected and removed

• Both big and little endian architectures are supported; SquashFS can mount file systems created on different byte-order machines

1.3. Making it clear

Now let's make sure any further discussions will be clearer for you to understand. The procedure of getting SquashFS working basically consists of the following steps:

1. Patching and recompiling the target Linux kernel to enable SquashFS support

2. Compiling the mksquashfs and the unsquashfs tools

3. Creating a compressed file system with mksquashfs

4. Testing: mounting a squashed file system to a temporary location

5. Modifying the /etc/fstab or startup scripts of your target Linux system to mount the new squashed file system when needed

2.1. Acquiring SquashFS

The SquashFS home site is located at http://squashfs.sourceforge.net/ - it contains news for the latest release and it's changelog, as well as general information about SquashFS. You can grab the latest version at the SquashFS project page at SourceForge.

Squashfs is also avaliable with LZMA compression at http://www.squashfs-lzma.org/

2.2. Preparing a SquashFS-capable kernel

In order to read SquashFS, you need it supported in your kernel - just as if it was a reiserfs or ext3 file system. You have to make sure there is an appropriate patch for your kernel version. It should be located in kernel-patches/linux-2.x.y subdirectory of the SquashFS source tree. Also, remember that in most cases you will need a clean (original) Linux kernel source from kernel.org. If your kernel source is from a distro vendor, it may be already pre-patched with custom vendor patches, and patching with a SquashFS patch will almost surely not work, as SquashFS patches are made against original Linux kernels. However some distributions make avaliable SquashFS-kernel-modules and SquashFS-tools packages from the repositories. If you consider to use these packages from your distro, you don't need to patch your kernel or to build SquashFS tools from source. In such case, make sure to get the proper module for your kernel from the distro repositories. Please note that doing so you will have an easy installation step, but no control on kernel configuration parameters if you mind to use SquashFS for particular purposes (eg. for embedded systems).

	bash# cd /usr/src/squashfs
	bash# cp linux-2.x.y/squashfs-patch /usr/src/linux

	bash# cd /usr/src/linux

	bash# patch -p1 < squashfs-patch

	bash# make distclean
	bash# make mrproper

	bash# make menuconfig

	bash# make

	bash# make menuconfig

	bash# make dep
	bash# make bzImage
	bash# make modules

	bash# cp arch/i386/boot/bzImage /boot/bzImage-sqsh

	bash# make modules_install

	bash# cat /proc/filesystems

	bash# insmod squashfs
	bash# cat /proc/filesystems

2.3. Compiling the SquashFS tools

Now you need to compile mksquashfs - the tool for creating squashed file systems and the unsquashfs which extracts files from an existing squashed file system.

	bash# cd /usr/src/squashfs/squashfs-tools

Compile and install the tools:

	bash# make
	bash# cp mksquashfs /usr/sbin
	bash# cp unsquashfs /usr/sbin

If everything went fine, typing mksquashfs or unsquashfs at the shell prompt should print the "usage" message.

2.4. Installing SquashFS on Debian

If you use Debian (or onother Linux distribution) you may want to get SquashFS module and tools from the distro repositories. With Debian you have to install the proper kernel module and the tools with the following commands:

(Assuming your architecture is x86)

	bash# apt-get install squashfs-modules-2.6-486  squashfs-tools

Now load the squashfs module for the Linux kernel and if it was correctly loaded you should find it in the relative list

	bash# modprobe squashfs
	
	bash# lsmod|grep squash
	squashfs               39620  0

	

Then you have to add the squashfs module to /etc/modules if you need it loaded at boot time.

	bash# echo squashfs >> /etc/modules

Note that at the time of writing, the Debian packages (Etch. 4.0 r2) relates to squashfs 3.1 release. Some latest options and features of 3.2 release may be not supported. Look at the next section for details.

3.1. Using mksquashfs

mksquashfs is the tool for creating new squashed file systems, and for appending new data to existing squashed file systems. The general command-line format for mksquashfs is:

	bash# mksquashfs source1 source2 ... destination [options]

• source1, source2, etc.: files and directories to be added to the resulting file system, given with relative and/or absolute paths

• destination: a regular file (filesystem image file), or a block device (such as /dev/fd0 or /dev/hda3) where you want to have your squashed file system

Notes for default mksquashfs behavior:

• When the new files are added to the new file system or appended to an existing one, mksquashfs will automatically rename files with duplicate names: if two or more files named text will appear in the same resulting directory, the second file will be renamed to text_1, third one to text_2 and so on.

• Duplicate files will be removed, so there will be only one physical instance (By the SquashFS 2.x, you can disable the detection/removal of the duplicates with the -no-duplicates option).

• If destination has a pre-existing SquashFS file system on it, by default, the new source items will be appended to the existing root directory. Examine the options table below to force mksquashfs to overwrite the whole destination and/or change the way new source items are added.

• If a single source file or directory is given, it becomes the root in a newly created file system. If two or more source files and/or directories are given, they will all become sub-items in the root of the new file system.

• The resulting filesystem will be padded to a multiple of 4 Kb: this is required for filesystems to be used on block devices. If you are very sure you don't ned this, use the -nopad option to disable this operation.

See the next section for more details about all possible options.

3.2. Command-line options

All possible options for mksquashfs are shown in the table below.

In most cases, you should leave all compression/block options by default, as they allow mksquashfs to achieve the best possible compression ratios.

3.3. Using unsquashfs

unsquashfs is the tool for extracting data from squashed file systems. The general command-line format for unsquashfs is:

unsquashfs [options] target [files/directories to extract]

• target is the squashed file system to extract.

Notes for unsquashfs behavior:

• By not specifying any destination path, unsquashfs extracts the compressed file system in the ./squashfs-root directory.

• The tool does not extract a squashed file system on already exsisting directory unless the -f option is specified.

• You can specify on the command line, a multiple number of files/directories to extract and the items to be extracted can be also be given in a file with -e [file] option.

All possible options for unsquashfs are shown in the table below.

Note that by 3.x release you can extract 1.x and 2.x squashed file system too.

4.1. Basic steps

In order to create a squashed file system out of a single directory (say, /some/dir), and output it to a regular file (thus, producing a file system image), you need to say only one magic phrase:

	bash# mksquashfs /some/dir dir.sqsh

mksquashfs will perform the squashing and print the resulting number of inodes and size of data written, as well as the average compression ratio. Now you have your /some/dir directory image in the dir.sqsh file. You can now use the mount command to mount it using a loopback device:

	bash# mkdir /mnt/dir
	bash# mount dir.sqsh /mnt/dir -t squashfs -o loop

To check if you have what's expected:

	bash# ls /mnt/dir

If you want to output the file system directly into a device (say, your floppy at /dev/fd0):

	bash# mksquashfs /some/dir /dev/fd0

Then just mount the device:

	bash# mount /dev/fd0 /mnt/floppy -t squashfs

And check if it's okay:

	bash# ls /mnt/floppy

4.2. Squashing file systems

Operations described here correspond to most cases where a read-only compressed file system can be used, whether you want it to be on a block device or in a file. This could be anything from large FTP/HTTP-served archives that don't change often, to having a squashed /usr partition and anything alike with these.

	bash# mksquashfs /var/arch /var/arch.sqsh
	bash# mkdir /mnt/tmp
	bash# mount /var/arch.sqsh /mnt/tmp -t squashfs -o loop
	bash# ls /mnt/tmp

	/var/arch.sqsh	/var/arch	squashfs	ro,defaults	0 0

	bash# umount /mnt/tmp
	bash# mount /var/arch

	bash# ls /var/arch

	/dev/hda7	/var/arch	reiserfs	defaults	0 0

	bash# mksquashfs /var/arch /var/arch.sqsh
	bash# mkdir /mnt/tmp
	bash# mount /var/arch.sqsh /mnt/tmp -t squashfs -o loop
	bash# ls /mnt/tmp

	bash# umount /dev/hda7
	bash# dd if=/var/arch.sqsh of=/dev/hda6

	/dev/hda6	/var/arch	squashfs	ro,defaults	0 0

	bash# mount /var/arch
	bash# ls /var/arch

	bash# rm /var/arch.sqsh

4.3. Creating tiny/embedded systems

By saying "tiny/embedded", I mean Linux systems that are being built for booting from floppy disks, IDE/USB flash disks, iso9660 CD-ROMs, small-sized hard drives and the like. Whether you want to have your whole root file system on a single media (a single partition, a single floppy), or have a modular system (several floppies or disk partitions), the procedure is almost identical. Creating such Linux systems themselves is out of scope of this HOWTO - there are dedicated HOWTOs and guides for this (like the Bootdisk HOWTO and Linux From Scratch - visit www.tldp.org to retrieve these documents).

	bash# cd /home/user
	bash# mksquashfs floppylinux root.sqsh -e usr
	bash# mksquashfs floppylinux/usr usr.sqsh

	bash# mount /mnt/floppy
	bash# cp root.sqsh /mnt/floppy/boot

	bash# dd if=usr.sqsh of=/dev/fd0

4.4. Making it writeble

As mentioned, another interesting use for SquashFS is with Unionfs filesystem, which provides copy-on-write semantics for the read-only file systems, enahancing the possibilities. (For unionfs you can look at http://www.filesystems.org/project-unionfs.html)

Just to make an example, you may want to make your /home/user squashed, to compress and backup your files without losing the possibility to apply changes or writing new files.

Create the ro.fs squashed file system and the rw.fs dir.

bash# mksquashfs /home/user1 ro.fs
bash# mkdir /home/rw.fs

Mount the squashed ro.fs file system using the loopback device

bash# mount -t squashfs ro.fs /mnt -o loop

mount the unionfs file system, that makes /mnt and /home/rw.fs apparently merged under /home/user1 location.

bash# cd /home
bash# mount -t unionfs -o dirs=rw.fs=rw:/mnt=ro unionfs user1

As you can see, now you can create new files in /home/user1.

bash# cd /home/user1
bash# touch file1
bash# ls

umount the unionfs and the squashfs file systems and list the content of /home/user1 and /home/rw.fs dir

bash# cd ..
bash# umount /home/user1
bash# umount /mnt

bash# ls /home/user1
bash# ls /home/rw.fs 

You can see that the new file1 was created in /home/rw.fs

When you want to add the new created files to the stable and compressed squashed file system, you have to add them to the exsisting one.

bash# mksquashfs /home/rw.fs /home/ro.fs

Now, to mount your squashed user home directory at system startup, you can do as follow:

Make squashfs and unionfs modules loaded at boot time.

bash# echo squashfs >> /etc/modules
bash# echo unionfs >> /etc/modules

Change the owner of the writeble branch to match user1.

chown user1 /home/rw.fs

Add these lines to /etc/fstab file to mount squashfs and unionfs at boot time.

...
/home/ro.fs  /mnt squashfs loop 0 0
unionfs /home/user1 unionfs dirs=/home/rw.fs=rw:/mnt=ro 0 0