Transforming your Installation Manually into a Live DVD/CD

This HOWTO is about making a live CD/DVD from the main system on your hard drive. This might be desired if you have customized your system and want to have it on CD. It can be useful also if you want to create a recovery CD from scratch, as you can make a minimal system using debootstrap and transform it into a live CD.

The live CD is usually created with a filesystem called squashfs. Squashfs is read only compressed filesystem that allow us to squeeze our system into a single CD. Note that your system has to be about 2GB (this might need some trial an error) to produce a compressed image that fits on the CD. Ofcourse compression incurs performance penalties. If you are using DVD and your installation is less than 4GB you don't have to use compression, instead you can use ordinary unix (e.g ext2) filesystem as pointed out at the end of this guide.

What you need:

• A working Debian or Ubuntu machine with internet access.

• CD/DVD Writer.

• Enough space in your harddirve. At least two times as much space as your installation size.

Outline of the steps:

A. Prepare Our work environment.

B. Copy the Source system to the target directory.

C. Chroot into the new system and make some modifications.

D. Prepare The CD directory tree.

E. Build the CD/DVD

Appendix 1. Using a non compressed filesystem.

Appendix 2. Adapting this guide to Debian.

Appendix 3. Building the live media form scratch using Debootstrap.

Conventions used in this HOWTO:

• Text highlighted in Magenta is meant to be replaced by the user's custom value.

• Commands performed within a chroot will be in Blue.

• Optional arguments or steps will be highlighted in Gray.

• I will use gedit as my default text editor. Replace gedit with your favorite text editor.

A. Preparing the environment

1. Set some variables

The WORK Directory is where our temporary files and mount point will reside.

The CD is the location of the CD tree.

FORMAT is the filesystem type. We you are going to use a compressed squashfs

FS_DIR is the location of the actual filesystem image within the cd tree.

Replace only the values highlighted in Magenta.

< People opting for ext2 look at Appendix.1 for modification of this step >

UPDATE: Using ext2 is not necessary as you can use squashfs without compression. See appendix 1 for more details

2. Create the CD and the WORK directory structure:

3. Install some packages on the source system:

qemu is optional. It is only needed for testing the cd before burning it. It can be substituted with any other virtualization software like virtualbox.

linux-ubuntu-modules-$(uname -r) is only needed for Ubuntu Gutsy and later. If using an Ubuntu version prior to Gutsy omit this package as it is part of the main kerenl package.

< People opting for ext2 look at Appendix.1 for additional step to done right here >.

UPDATE: Using ext2 is not necessary as you can use squashfs without compression. See appendix 1 for more details

B. Copy your installation into the new filesystem.

Note: rsync is used instead of cp to take advantage of the --one-file-system and the --exclude options.

If you have a separate boot partition you will have to copy it using the following command:

(Optional) Copy settings in your home dir:

If you want to preseve your user account settings which are stored in your home directory, you can copy them to ${WORK}/rootfs/etc/skel/.

But first we have to define what files we want to copy. For example I am using xcfe4 as my DE, and it stores all it settings in a directory called .config in my home directory, so I am going to add .config to the variable $CONFIG:

C. Chroot into the new system and modify it:

1. Chroot into the copied system after mounting proc and dev:

Now, Copy the CONFIG files using the following command:

 

N.B: All commands in Blue are done within a chroot.

Now you are within chroot environment, type the following command:

2. Install Packages Essential for live CD:

casper contain the live scirpts.

discover1 & xresprobe are used for autodetectin hardware at startup.

3. (Optional) If you want your live cd to have an installer, install the Ubuntu installer:

(Optional Step)Install any packages you want to be in the CD. Some of the following packages are useful in emergency situations:

sudo apt-get install gparted ms-sys testdisk wipe partimage xfsprogs reiserfsprogs jfsutils ntfs-3g ntfsprogs dosfstools mtools[/code]

gparted: patitioning tool. It is automatically installed as a dependecy of ubiquity.

ms-sys: writing a Microsoft compatible boot record (MBR).

testdisk: Partition scanner and disk recovery tool.

wipe: Secure file deletion.

partimage: backup partitions into a compressed image file (like norton ghost).

xfsprogs reiserfsprogs jfsutils: Tools for handling different filesystems.

mtools: Tools for manipulating MSDOS files

4. Update the initramfs:

First update modules.dep:

 

As already metioned above, the initramfs is reponsible for much of the preparation required at the boot time of the CD/DVD. The updated initramfs now contain the live scirpts installed with casper.

5. Remove non system users

Non-system users are users created by you that have user id more than 999.

6. Delete these files.

These files are not needed in the CD/DVD. some of them are could interfer with the CD/DVD boot process. (e.g. shadow and gdm.conf-custom can interfere with autologin).

7. Clean the chroot environment form unnecessary files:

8. If you are using GDM recreate it's config file:

Sometimes a customized /etc/gdm/gdm.conf can interfere with the live CD/DVD autologin.

9. Create some files in /var/log:

Most of these files are log files that have been cleaned in step 7. We created an empty files in their place to prevent the system from complaining at boot.

10. Exit chroot

D. Prepare The CD directory tree:

1. Copy the kernel, the updated initrd and memtest prepared in the chroot:

2. Generate manifest:

Note: This step is only needed if you installed the Ubuntu installer ubiquity. This step generates two files (filesystem.manifest & filesystem.manifest-desktop).

3. Unmount bind mounted dirs:

These two files are used by the ubiquity installer when installing to harddisk. These two files are just lists of packages. Ubiquity compares these two files and removes packages unique to filesystem.manifest. This way when installing to harddisk, packages like casper which is only useful in a live CD/DVD are removed. These packages that will be removed at install are defined in the variable $REMOVE

4. Convert the directory tree into a squashfs:

Note: Make sure the resulting file size can fit into your live media.

< People opting for ext2 look at Appendix.1 for modification of this step >

UPDATE: Using ext2 is not necessary as you can use squashfs without compression. See appendix 1 for more details

5. Make Grub the bootloader of the CD

Copy grub file:

Make the menu.lst

Copy the following text into it and save it.

6. Calculate MD5

E. Build the CD/DVD

1. Make the ISO file

2. Test the CD

Test using qemu emulator

Or use any other virtualization program you like.

3. (Optional) Clean our workspace

Final Notes:

Note: If you are using ext2 instead of squashfs you may want to skip this step. Keeping these files in place will be handy whenever you update your CD.

UPDATE: Using ext2 is not necessary as you can use squashfs without compression. See appendix 1 for more details

• You can add a splash image to the grub menu.

 

• If you are using a custom kernel make sure it has support for the following:
  1. Support of loopback device.
  2. Support for the filesystem format you are using (e.g. squashfs or ext2).
  3. Support for unionfs.
  4. Support for initramfs.

 

• There are some extra options I put in the grub menu. The ones that I have not tried are highlighted in Red below:
  1. Start linux form RAM. This option is only possible if your ram is larger than data on the live media. This option can be useful if you are building a minimal command line rescue disc as it would enhance performance to start it from RAM.
  2. Start in presistent mode. To learn about it more look here.
  3. Start Linux in Text Mode. This will not start X. The user will be autologged into a virtual terminal (the kind of terminal you get when you press Alt+Ctrl+F1). Note that this option will not work in all Ubuntu versions prior to Gutsy.

Appendix 1. Building the CD/DVD using a non compressed filesystem (e.g. ext2)

Update: You can use squashfs without compression by adding the -noI -noD -noF switches to mksquashfs. This is easier than using a unix filesystem. It also has the advantage that you do not have to predict the size of the system in advance, which is needed for ext2 in order to create a virtual filesystem with enough space ( with squashfs you do not need to create a virtual filesystem, you just use mksquashfs to create the squashfs form a directory tree ). I have not tried this option yet. But I imagine it would make using a unix filesystem totally unnecessary.

Instead of using squashfs for the live DVD filesystem you can use an ordinary unix filesystem like ext2.

Advantages of using a unix filesystem:

- The CD/DVD will be faster than a CD/DVD with a compressed fileystem.

- Easier update of the live media provided you have not cleaned your workspace. An ordinary filesystem can be remounted and modified, then you can recreate the ISO image. Squashfs on the other hand, is a read only filesystem so it not as easy to update.

Disadvantages of using a unix filesystem:

- You lose the advantages of filesystem compression. So you will end up with large image.

- You need a DVD unless you are building a minimal system that can fit within a CD without compression (e.g. a non graphical rescue disk).

- Your system size cannot exceed 4GB as that is the maximum filesize supported by ISO9660. Of course this also applies when using squahsfs. But the differece is that a 4GB squashfs can contain roughly 12GB of data within it.

Note: If you are opting for the non compressed filesystem, it makes sesne to go for ext2 instead of ext3. It does not make sense to user a journaling filesystem with it's overhead in a read only media

To use ext2 instead of squashf follow the same steps above with the following modifications:

• In Step A.1 you have to define FORMAT=ext2:
  Code:

  ---

  export FORMAT=ext2

  ---

  Or replace ext2 with the filesystem of your choice (e.g. xfs or jfs)

NOTE: The only filesystem I have tried is ext2. I have not tried any other filesystem and I do not know if they will work.

• Step A.3 there is no need to install squashfs-tools, so we replace the command with:
  Code:

  ---

  sudo apt-get install mkisofs grub qemu

  ---

• After doing the A.3 step (Before going into step B) you have to preform these extra commands:

  We need to make a virtual filesystem and then format it:

  Code:

  ---

  sudo dd if=/dev/zero of=${CD}/${FS_DIR}/filesystem.${FORMAT} bs=1024 count=4000000

  ---

  Note: This file size should not exceed 4 Gb because that is the maximum filesize ISO9660 can handle.

  Note: The size of the virtual filesystem is the bs (block size) multiplied by the count. In the above example it is 1024 * 4000000 = 4096000000 (4 GB)

  Note: You want to make the size of the virtual filesystem more than the space taken up by your hard drive installation to allow the filesystem structures and extra packages that will be installed later.

  Format the filesystem:

  Code:

  ---

  sudo mkfs.${FORMAT} -m 0 -b 1024 ${CD}/${FS_DIR}/filesystem.${FORMAT}

  ---

  You may notice a warning prompt, saying that filesystem.${FORMAT} is not a block device, would you like to proceed? Say yes.

  Note: The options in Magenta are specific to ext2 filesystem. Omit them if you opt for other format like xfs.

  Mount the filesystem:

  Code:

  ---

  sudo mount -o loop -t $FORMAT ${CD}/${FS_DIR}/filesystem.${FORMAT} ${WORK}/rootfs

  ---

 

• Replace Step D.4 with this:

  Unmount the loopback device you created:

  Code:

  ---

  sudo umount ${WORK}/rootfs/

  ---

Appendix 2. Adapting this guide to Debian

This guide can be modified to apply to Debian systems as well. There are number of differeces between Ubuntu and Debian that we must take into account:

1. As of Debian lenny, casper is deprecated and replaced with live-initramfs. live-initramfs is a fork of casper and it has the same options, with one difference in the CD directory tree sturcute. This can be solved by setting the variable FS_DIR=live instead of FS_DIR=casper.
2. Sqaushfs modules and unionfs modules are in two separate packages.
3. Ubiquity installer is not present in the Debain repositories. Ubiquity is only needed if you intend to install Linux from the live CD/DVD to the harddisk. I have not tried using ubiquity on Debian so I am not sure if it will work. To install it on Debian you have to add Ubuntu main repository to your sources.list.

So in light of the points mentioned above we have to make the following modifications to adapt the guide to Debian:

• In step A.1 replace FS_DIR=casper with
  Code:

  ---

  FS_DIR=live

  ---

• Replace the command in Step A.3 with:
  Code:

  ---

  sudo apt-get install mkisofs grub squashfs-tools squashfs-modules-$(uname -r) qemu

  ---

• Replace the command in Step C.2 with:
  Code:

  ---

  apt-get install live-initramfs unionfs-modules-$(uname -r) discover1 xresprobe

  ---

• Skip step C.3 and D.2 if you do not intend to try ubiquity on Debian.

• In Step D.5 Replace every occurence of BOOT=casper and boot=casper in menu.lst with BOOT=live and boot=live respectively

Appendix 3. Building the live media form Debootstrap installed system.

Instead of using your current installation to be the basis of you live CD, you can install a custom system into any directory in your system using debootstrap, and then use that as the basis of your CD. The modifications you have to make are

• Before doing the steps in this howto, install your custom system using debootstrap.

• Step A.3 has to be done after chrooting into the debootstrap installed system. And to be able to access the internet you have to copy your system's resolv.conf. Also we replace $(uname -r) with the kernel version of the debootstrap system. So the commands in A.3 should be replaced with:
  Code:

  ---

  sudo cp -pv /etc/resolv.conf ~/debootstrap/*

  ---

  Code:

  ---

  sudo LANG= chroot ~/debootstrap/* /bin/bash

  ---

  Code:

  ---

  apt-get update && apt-get install mkisofs grub squashfs-tools linux-ubuntu-modules-$(uname -r) qemu

  ---

  Code:

  ---

  exit

  ---

  Where version is the version of the kernel of your debootstrap installed system.

Replace ~/debootstrap/* with the path of the debootstrap installed system.[/list]

• In Step B Replace "/" with the directory you installed your new system in. So if you installed your new system in ~/debootstrap the command would be:
  Code:

  ---

  sudo rsync -av --one-file-system --exclude=/proc/* --exclude=/dev/* --exclude=/sys/* --exclude=/tmp/* --exclude=/home/* --exclude=/lost+found ~/debootstrap/* ${WORK}/rootfs

  ---

• Modify the two commands in step C.4 so they look lik this:
  Code:

  ---

  depmod -a version

  ---

  Code:

  ---

  update-initramfs -u -k version

  ---

  Where version is the version of the kernel of your debootstrap installed system.

 

• Replace $(uname -r) in Step D.1 with the kernel version of debootstarp system, so the commands should be:
  Code:

  ---

  sudo cp -vp ${WORK}/rootfs/boot/vmlinuz-version ${CD}/boot/vmlinuz \;

  ---

  Code:

  ---

  sudo cp -vp ${WORK}/rootfs/boot/initrd.img-version ${CD}/boot/initrd.gz \;

  ---

  Code:

  ---

  sudo cp -vp ${WORK}/rootfs/boot/memtest86+.bin ${CD}/boot

  ---

  Where version is the version of the kernel of your new system.

Edit (1): 20/02/08

1. Added Suggestions for packages useful in rescue CD.
2. Added ext2 tag as per RumorsOfWar suggestion.
3. Modified step D.5 as per RumorsOfWar suggestion.

Edit (2): 22/02/08

If you want a non compressed filesystem (in case of DVD), using a unix filesystem like ext2 is not recommended anymore as you can use squashfs without compression by adding the -noI -noD -noF switches to mksquashfs. The option for creating ext2 filesystem is still present in the guide but with notes to use an uncom