Transforming your Installation Manually into a Live DVD/CD
This HOWTO is about making a live CD/DVD from the main system on your hard drive. This might be desired if you have customized your system and want to have it on CD. It can be useful also if you want to create a recovery CD from scratch, as you can make a minimal system using debootstrap and transform it into a live CD.
The live CD is usually created with a filesystem called squashfs. Squashfs is read only compressed filesystem that allow us to squeeze our system into a single CD. Note that your system has to be about 2GB (this might need some trial an error) to produce a compressed image that fits on the CD. Ofcourse compression incurs performance penalties. If you are using DVD and your installation is less than 4GB you don't have to use compression, instead you can use ordinary unix (e.g ext2) filesystem as pointed out at the end of this guide.
Background on live CD/DVD Note: This section is a clarification of how live CD works. You don't have to read it. You can skip it if you want. A live CD/DVD is basically a normal linux installation just like an ordinary harddrive installation. However, simply copying the harddirve installation over to a CD/DVD is not enough to produce a working system. Why? because there are still minor differences between a live CD/DVD and on ordinary harddrive installation. So in addition to copying our harddirve installation to the CD/DVD we must address those differences as well. So what are these differences?
These considerations require special preparation at boot time, some of which must be performed even before mounting the actual filesystem. How can we do this? Linux introduced a mechanism that allow for such preparations at boot time before the actual root filesystem is mounted. It is called the initial root filesystem or initramfs. This mechanism is used also in mounting normal harddirve installations, as it adds flexibilty to the boot process. initramfs is virtual filesystem. It is a compressed cpio (cpio is an archive format similar to tar) archive that contains a minimal shell, kernel modules necessary for mounting the root filesystem and number of scripts that perform some tasks at boot time. The most important of these scripts is a script called init located at the root of the initramfs. How does initramfs work? The boot loader loads both the kernel and the initramfs into memory and starts the kernel. The kernel then unpacks the initramfs and mount it as initial root filesystem, and then looks for the init program within the initial filesystem, and once it finds it, it executes it and hand the boot process over to it. This init scirpt is responsible for finding the real root filesystem and mounting it. It is also responsible for any special preparations required at boot time. So any special operations required for booting the system from live media can be coded into the initramfs boot scripts. How is initramfs is created? We do not have to create initramfs manually (although it can be done). There are tools for creating and updating initramfs like the command update-initramfs. Moreover, these tools can include custom scripts into the initramfs if they are placed in a certain preset locations (/usr/share/initramfs/scripts). So all we have to do is dump our custom scripts (which will do all the required preparation for booting the live CD/DVD) into these preset locations, and then create a custom initramfs by running update-initramfs. We don't even have to write these scripts. Why? becuase there are packages that have scripts tailored for booting form live CDs. One of these packages is called casper (this is the package used in this howto). By installing casper into the system, it places the scripts in there proper locations (where they can be spotted by update-initrfamfs). The only thing we need to do after installing casper is running update-initramfs to create an initramfs suitable for live CD/DVD. The live CD/DVD structure: The directory tree of the live CD/DVD we are going to create is going to look like this: Code:
(CD ROOT)
|
What you need:
Outline of the steps:
A. Prepare Our work environment.
B. Copy the Source system to the target directory.
C. Chroot into the new system and make some modifications.
D. Prepare The CD directory tree.
E. Build the CD/DVD
Appendix 1. Using a non compressed filesystem.
Appendix 2. Adapting this guide to Debian.
Appendix 3. Building the live media form scratch using Debootstrap.
Conventions used in this HOWTO:
A. Preparing the environment
1. Set some variables
export WORK=~/work
export CD=~/cd
export FORMAT=squashfs
export FS_DIR=casper
The WORK Directory is where our temporary files and mount point will reside.
The CD is the location of the CD tree.
FORMAT is the filesystem type. We you are going to use a compressed squashfs
FS_DIR is the location of the actual filesystem image within the cd tree.
Replace only the values highlighted in Magenta.
< People opting for ext2 look at Appendix.1 for modification of this step >
UPDATE: Using ext2 is not necessary as you can use squashfs without compression. See appendix 1 for more details
2. Create the CD and the WORK directory structure:
sudo mkdir -p ${CD}/{${FS_DIR},boot/grub} ${WORK}/rootfs
3. Install some packages on the source system:
sudo apt-get update
sudo apt-get install mkisofs grub squashfs-tools linux-ubuntu-modules-$(uname -r) qemu
qemu is optional. It is only needed for testing the cd before burning it. It can be substituted with any other virtualization software like virtualbox.
linux-ubuntu-modules-$(uname -r) is only needed for Ubuntu Gutsy and later. If using an Ubuntu version prior to Gutsy omit this package as it is part of the main kerenl package.
< People opting for ext2 look at Appendix.1 for additional step to done right here >.
UPDATE: Using ext2 is not necessary as you can use squashfs without compression. See appendix 1 for more details
B. Copy your installation into the new filesystem.
sudo rsync -av --one-file-system --exclude=/proc/* --exclude=/dev/*\
--exclude=/sys/* --exclude=/tmp/* --exclude=/home/*\
--exclude=/lost+found / ${WORK}/rootfs
Note: rsync is used instead of cp to take advantage of the --one-file-system and the --exclude options.
If you have a separate boot partition you will have to copy it using the following command:
sudo cp -av /boot/* ${WORK}/rootfs/boot
(Optional) Copy settings in your home dir:
If you want to preseve your user account settings which are stored in your home directory, you can copy them to ${WORK}/rootfs/etc/skel/.
But first we have to define what files we want to copy. For example I am using xcfe4 as my DE, and it stores all it settings in a directory called .config in my home directory, so I am going to add .config to the variable $CONFIG:
C. Chroot into the new system and modify it:
1. Chroot into the copied system after mounting proc and dev:
CONFIG='.config .bashrc'
Now, Copy the CONFIG files using the following command:
cd ~ && for i in $CONFIG
do
sudo cp -rpv --parents $i ${WORK}/rootfs/etc/skel
done
sudo mount -o bind /dev/ ${WORK}/rootfs/dev
sudo mount -t proc proc ${WORK}/rootfs/proc
sudo chroot ${WORK}/rootfs /bin/bash
N.B: All commands in Blue are done within a chroot.
Now you are within chroot environment, type the following command:
LANG=
2. Install Packages Essential for live CD:
apt-get update
apt-get install casper discover1 xresprobe
casper contain the live scirpts.
discover1 & xresprobe are used for autodetectin hardware at startup.
3. (Optional) If you want your live cd to have an installer, install the Ubuntu installer:
apt-get install ubiquity
(Optional Step)Install any packages you want to be in the CD. Some of the following packages are useful in emergency situations:
sudo apt-get install gparted ms-sys testdisk wipe partimage xfsprogs reiserfsprogs jfsutils ntfs-3g ntfsprogs dosfstools mtools[/code]
gparted: patitioning tool. It is automatically installed as a dependecy of ubiquity.
ms-sys: writing a Microsoft compatible boot record (MBR).
testdisk: Partition scanner and disk recovery tool.
wipe: Secure file deletion.
partimage: backup partitions into a compressed image file (like norton ghost).
xfsprogs reiserfsprogs jfsutils: Tools for handling different filesystems.
mtools: Tools for manipulating MSDOS files
4. Update the initramfs:
First update modules.dep:
depmod -a $(uname -r)
update-initramfs -u -k $(uname -r)
As already metioned above, the initramfs is reponsible for much of the preparation required at the boot time of the CD/DVD. The updated initramfs now contain the live scirpts installed with casper.
5. Remove non system users
for i in `cat /etc/passwd | awk -F":" '{print $1}'`
do
uid=`cat /etc/passwd | grep "^${i}:" | awk -F":" '{print $3}'`
[ "$uid" -gt "999" -a "$uid" -ne "65534" ] && userdel --force ${i} 2>/dev/null
done
Non-system users are users created by you that have user id more than 999.
6. Delete these files.
for i in "/etc/hosts /etc/hostname /etc/resolv.conf /etc/timezone /etc/fstab /etc/mtab /etc/shadow /etc/shadow- /etc/gshadow /etc/gshadow- /etc/gdm/gdm-cdd.conf /etc/gdm/gdm.conf-custom /etc/X11/xorg.conf /boot/grub/menu.lst /boot/grub/device.map"
do
rm $i
done 2>/dev/null
These files are not needed in the CD/DVD. some of them are could interfer with the CD/DVD boot process. (e.g. shadow and gdm.conf-custom can interfere with autologin).
7. Clean the chroot environment form unnecessary files:
apt-get clean
find /var/run /var/log /var/mail /var/spool /var/lock /var/backups /var/tmp -type f -exec rm {} \;
rm -r /tmp/* /root/* 2>/dev/null
8. If you are using GDM recreate it's config file:
[ -f "/etc/gdm/factory-gdm.conf" ] && cp -f /etc/gdm/factory-gdm.conf /etc/gdm/gdm.conf 2>/dev/null
Sometimes a customized /etc/gdm/gdm.conf can interfere with the live CD/DVD autologin.
9. Create some files in /var/log:
for i in dpkg.log lastlog mail.log syslog auth.log daemon.log faillog lpr.log mail.warn user.log boot debug mail.err messages wtmp bootstrap.log dmesg kern.log mail.info
do
touch /var/log/${i}
done
Most of these files are log files that have been cleaned in step 7. We created an empty files in their place to prevent the system from complaining at boot.
10. Exit chroot
exit
D. Prepare The CD directory tree:
1. Copy the kernel, the updated initrd and memtest prepared in the chroot:
sudo cp -vp ${WORK}/rootfs/boot/vmlinuz-$(uname -r) ${CD}/boot/vmlinuz
sudo cp -vp ${WORK}/rootfs/boot/initrd.img-$(uname -r) ${CD}/boot/initrd.gz
sudo cp -vp ${WORK}/rootfs/boot/memtest86+.bin ${CD}/boot
2. Generate manifest:
Note: This step is only needed if you installed the Ubuntu installer ubiquity. This step generates two files (filesystem.manifest & filesystem.manifest-desktop).
3. Unmount bind mounted dirs:
sudo chroot ${WORK}/rootfs dpkg-query -W --showformat='${Package} ${Version}\n' | sudo tee ${CD}/${FS_DIR}/filesystem.manifest
sudo cp -v ${CD}/${FS_DIR}/filesystem.manifest{,-desktop}
REMOVE='ubiquity casper live-initramfs user-setup discover1 xresprobe os-prober libdebian-installer4'
for i in $REMOVE
do
sudo sed -i "/${i}/d" ${CD}/${FS_DIR}/filesystem.manifest-desktop
done
These two files are used by the ubiquity installer when installing to harddisk. These two files are just lists of packages. Ubiquity compares these two files and removes packages unique to filesystem.manifest. This way when installing to harddisk, packages like casper which is only useful in a live CD/DVD are removed. These packages that will be removed at install are defined in the variable $REMOVE
sudo umount ${WORK}/rootfs/dev ${WORK}/rootfs/proc
4. Convert the directory tree into a squashfs:
sudo mksquashfs ${WORK}/rootfs ${CD}/${FS_DIR}/filesystem.${FORMAT}
Note: Make sure the resulting file size can fit into your live media.
< People opting for ext2 look at Appendix.1 for modification of this step >
UPDATE: Using ext2 is not necessary as you can use squashfs without compression. See appendix 1 for more details
5. Make Grub the bootloader of the CD
Copy grub file:
sudo find /boot /usr/lib/grub/ -iname 'stage2_eltorito' -exec cp -v {} ${CD}/boot/grub \;
Make the menu.lst
sudo gedit ${CD}/boot/grub/menu.lst
Copy the following text into it and save it.
# By default, boot the first entry.
default 0
# Boot automatically after 30 secs.
timeout 30
color cyan/blue white/blue
title Start Linux in Graphical Mode
kernel /boot/vmlinuz BOOT=casper boot=casper nopersistent rw quiet splash
initrd /boot/initrd.gz
title Start Linux in Safe Graphical Mode
kernel /boot/vmlinuz BOOT=casper boot=casper xforcevesa rw quiet splash
initrd /boot/initrd.gz
title Start Linux in Text Mode
kernel /boot/vmlinuz BOOT=casper boot=casper nopersistent textonly rw quiet
initrd /boot/initrd.gz
title Start Presistent Live CD
kernel /boot/vmlinuz BOOT=casper boot=casper persistent rw quiet splash
initrd /boot/initrd.gz
title Start Linux Graphical Mode from RAM
kernel /boot/vmlinuz BOOT=casper boot=casper toram nopersistent rw quiet splash
initrd /boot/initrd.gz
title Memory Test
kernel /boot/memtest86+.bin
title Boot the First Hard Disk
root (hd0)
chainloader +1
6. Calculate MD5
cd $CD && find . -type f -print0 | xargs -0 sudo md5sum | sudo tee ${CD}/md5sum.txt
E. Build the CD/DVD
1. Make the ISO file
sudo mkisofs -b boot/grub/stage2_eltorito \
-no-emul-boot -boot-load-size 4 -boot-info-table \
-V "Custom Live CD" -cache-inodes -r -J -l \
-o ~/live-cd.iso $CD
2. Test the CD
Test using qemu emulator
qemu -cdrom ~\live-cd.iso -boot d
Or use any other virtualization program you like.
3. (Optional) Clean our workspace
Final Notes:
[ -d "$WORK" ] && rm -r $WORK $CD
Note: If you are using ext2 instead of squashfs you may want to skip this step. Keeping these files in place will be handy whenever you update your CD.
UPDATE: Using ext2 is not necessary as you can use squashfs without compression. See appendix 1 for more details
Appendix 1. Building the CD/DVD using a non compressed filesystem (e.g. ext2)
Update: You can use squashfs without compression by adding the -noI -noD -noF switches to mksquashfs. This is easier than using a unix filesystem. It also has the advantage that you do not have to predict the size of the system in advance, which is needed for ext2 in order to create a virtual filesystem with enough space ( with squashfs you do not need to create a virtual filesystem, you just use mksquashfs to create the squashfs form a directory tree ). I have not tried this option yet. But I imagine it would make using a unix filesystem totally unnecessary.
Instead of using squashfs for the live DVD filesystem you can use an ordinary unix filesystem like ext2.
Advantages of using a unix filesystem:
- The CD/DVD will be faster than a CD/DVD with a compressed fileystem.
- Easier update of the live media provided you have not cleaned your workspace. An ordinary filesystem can be remounted and modified, then you can recreate the ISO image. Squashfs on the other hand, is a read only filesystem so it not as easy to update.
Disadvantages of using a unix filesystem:
- You lose the advantages of filesystem compression. So you will end up with large image.
- You need a DVD unless you are building a minimal system that can fit within a CD without compression (e.g. a non graphical rescue disk).
- Your system size cannot exceed 4GB as that is the maximum filesize supported by ISO9660. Of course this also applies when using squahsfs. But the differece is that a 4GB squashfs can contain roughly 12GB of data within it.
Note: If you are opting for the non compressed filesystem, it makes sesne to go for ext2 instead of ext3. It does not make sense to user a journaling filesystem with it's overhead in a read only media
To use ext2 instead of squashf follow the same steps above with the following modifications:
export FORMAT=ext2
NOTE: The only filesystem I have tried is ext2. I have not tried any other filesystem and I do not know if they will work.
sudo apt-get install mkisofs grub qemu
We need to make a virtual filesystem and then format it:
sudo dd if=/dev/zero of=${CD}/${FS_DIR}/filesystem.${FORMAT} bs=1024 count=4000000
Note: The size of the virtual filesystem is the bs (block size) multiplied by the count. In the above example it is 1024 * 4000000 = 4096000000 (4 GB)
Note: You want to make the size of the virtual filesystem more than the space taken up by your hard drive installation to allow the filesystem structures and extra packages that will be installed later.
Format the filesystem:
sudo mkfs.${FORMAT} -m 0 -b 1024 ${CD}/${FS_DIR}/filesystem.${FORMAT}
Note: The options in Magenta are specific to ext2 filesystem. Omit them if you opt for other format like xfs.
Mount the filesystem:
sudo mount -o loop -t $FORMAT ${CD}/${FS_DIR}/filesystem.${FORMAT} ${WORK}/rootfs
Unmount the loopback device you created:
sudo umount ${WORK}/rootfs/
Appendix 2. Adapting this guide to Debian
This guide can be modified to apply to Debian systems as well. There are number of differeces between Ubuntu and Debian that we must take into account:
So in light of the points mentioned above we have to make the following modifications to adapt the guide to Debian:
FS_DIR=live
sudo apt-get install mkisofs grub squashfs-tools squashfs-modules-$(uname -r) qemu
apt-get install live-initramfs unionfs-modules-$(uname -r) discover1 xresprobe
Appendix 3. Building the live media form Debootstrap installed system.
Instead of using your current installation to be the basis of you live CD, you can install a custom system into any directory in your system using debootstrap, and then use that as the basis of your CD. The modifications you have to make are
sudo cp -pv /etc/resolv.conf ~/debootstrap/*
sudo LANG= chroot ~/debootstrap/* /bin/bash
apt-get update && apt-get install mkisofs grub squashfs-tools linux-ubuntu-modules-$(uname -r) qemu
exit
Replace ~/debootstrap/* with the path of the debootstrap installed system.[/list]
sudo rsync -av --one-file-system --exclude=/proc/* --exclude=/dev/* --exclude=/sys/* --exclude=/tmp/* --exclude=/home/* --exclude=/lost+found ~/debootstrap/* ${WORK}/rootfs
depmod -a version
update-initramfs -u -k version
sudo cp -vp ${WORK}/rootfs/boot/vmlinuz-version ${CD}/boot/vmlinuz \;
sudo cp -vp ${WORK}/rootfs/boot/initrd.img-version ${CD}/boot/initrd.gz \;
sudo cp -vp ${WORK}/rootfs/boot/memtest86+.bin ${CD}/boot
Edit (1): 20/02/08
Edit (2): 22/02/08
If you want a non compressed filesystem (in case of DVD), using a unix filesystem like ext2 is not recommended anymore as you can use squashfs without compression by adding the -noI -noD -noF switches to mksquashfs. The option for creating ext2 filesystem is still present in the guide but with notes to use an uncom