Cloning systems

Introduction

This page describes various procedures I’ve used for cloning systems.

Cloning a system across the network using a live CD

  1. On the new system boot from a live CD (e.g. Ubuntu)
  2. On the new system temporarily configure the intended destination IP parameters. E.g.: 
    ifconfig eth0 192.168.1.100
route add -net default 192.168.1.1
echo "nameserver <my-name-server-ip-addr>" > /etc/resolv.conf
  3. On the new system connect via SSH to an old system and examine the partition table. E.g.: 
    ssh <old-system-hostname>
fdisk -l /dev/sda
  4. On the new system set up the same partition table. E.g.: 
    fdisk /dev/sda
n     #  new
p     #  primary
1     #  partition #1
...
w     #  write and exit
  5. On the new system format the partitions. E.g.: 
    mkfs -t ext3 /dev/sda1
mkswap /dev/sda2
  6. On the new system mount the new partitions. E.g.: 
    mount /dev/sda1 /mnt
  7. On the new system rsync data across from the old system. E.g.: 
    rsync -ax <old-system-hostname>:/ /mnt/
  8. On the new system several files will need adjusting; this is most easily done in a chroot environment so run: 
    chroot /mnt
  9. On the new system consider adjusting the following files:
    • /etc/hosts (references to <old-system-hostname> should be replaced with <new-system-hostname> and <old-system-ip-addr> with <new-system-ip-addr>)
    • /etc/sysconfig/network (references to <old-system-hostname> should be replaced with <new-system-hostname>)
    • /etc/sysconfig/networking-scripts/ifcfg-eth0 (references to <old-system-ip-addr> should be replaced with <new-system-ip-addr>)
    • /etc/udev/rules.d/* (references to <old-system-mac-addr> should be replaced with <new-system-mac-addr>)
    • /etc/fstab (since partitions were not created with labels, replace all references to labels with device paths)
    • /boot/grub/menu.lst (since partitions were not created with labels, replace all references to labels with device paths)
  10. Exit the chroot environment: 
    exit
  11. On the new system install grub on the new system’s disk’s MBR and tell it to take second stage loader from the new system’s disk, which is currently not mounted as / but as /mnt (not the live CD’s root partition). E.g.: 
    grub-install --root-directory=/mnt /dev/sda
  12. On the new system the source of the rsync was a running system and is not clean, so unmount it and clean it. E.g.: 
    umount /mnt
fsck -f /dev/sda1

    (Despite this some distributions will still think the filesystem needs checking as they are booted.)

  13. Reboot and remove the live CD.

Cloning a system to a new virtual system

This procedure describes how I cloned a physical system to a Xen virtual system.

  1. The following placeholders are used in this procedure, so work out their values for your particular migration:
    • <vm-server-host> the physical system on which your virtual system will reside
    • <src-host> the system you want to clone
    • <dst-host> the new clone
    • <third-host> an existing virtual system on <vm-server-host>, which you can reboot as needed
  2. Log in to <vm-server-host>.
  3. Use virt-manager to create a 12G disk file and temporarily integrate it into <third-host>.
  4. Start <third-host> and log into it as root.
  5. Use fdisk to format the new second disk, which will appear as /dev/xvdb.alert Once an OS has been copied to this second disk then one might expect to have to run 
    grub-install ... /dev/xvdb

    but this is totally unnecessary! libvirt (?) calls /usr/lib/xen-default/bin/pygrub to examine that disk’s /boot/grub/menu.lst and the program on the MBR is never called!alert To confuse matters further pygrub uses a different logic to the real GRUB; pygrub will assume that /boot/grub/menu.lst can be found in:

    1. the active partition if there is one, otherwise
    2. the first partition

    Once a partition has been selected according to the above rules then the filesystem inside that partition will be examined if the partition is of type:

    1. GPT (0xee)
    2. Solaris OR Linux Swap (0x82)

    If, like me, you’re used to putting swap in front of the Linux root filesystem in order to reduce seek times, and you never bother marking a partition active, then you’re going to wind up seeing the error message

    Invalid disklabel magic

    If you do see this message then the either:

    1. mark the partition containing the Linux root filesystem active, or
    2. reorder your partitions so that the partition containing the Linux root filesystem is first.
  6. Make a filesystem and swap on /dev/xvdb; e.g. if you put the partition for the root filesystem first then: 
    mkfs -t ext3 /dev/xvdb1
mkswap /dev/xvdb2
  7. Mount that filesystem on /mnt: 
    mount /dev/xvdb1 /mnt
  8. Rsync the physical <src-host>’s root filesystem over: 
    rsync -ax --delete / <third-host>:/mnt/
  9. The initrd that was rsynced over is appropriate for the physical <src-host>’s hardware but not for the virtual version. I tried to generate the initrd under /mnt/boot from <third-host>’s root filesystem as follows, but it did not work: 
    <third-host># rm -f /mnt/boot/initrd.img-2.6.26-1-686-bigmem
<third-host># update-initramfs -k 2.6.26-1-686-bigmem -b /mnt/boot -c
update-initramfs: Generating /mnt/boot/initrd.img-2.6.26-1-686-bigmem
Cannot find /lib/modules/2.6.26-1-686-bigmem
update-initramfs: failed for /mnt/boot/initrd.img-2.6.26-1-686-bigmem
<third-host>#

    Then I tried after chrooting into the root filesystem on /dev/xvdb1 as follows, but that also did not work:

    <third-host># chroot /mnt
<third-host># update-initramfs -k 2.6.26-1-686-bigmem -c
update-initramfs: Generating /boot/initrd.img-2.6.26-1-686-bigmem
grep: /proc/modules: No such file or directory
grep: /proc/modules: No such file or directory
grep: /proc/modules: No such file or directory
grep: /proc/modules: No such file or directory
grep: /proc/modules: No such file or directory
grep: /proc/modules: No such file or directory
grep: /proc/modules: No such file or directory
grep: /proc/modules: No such file or directory
grep: /proc/modules: No such file or directory
<third-host># exit
<third-host>#

    Finally I copied the contents of <third-host>’s /proc/modules (which isn’t really a file, but just a view of the kernel) to a file in /proc on the second disk as follows, and that worked:

    <third-host># cat /proc/modules > /mnt/proc/modules
<third-host># chroot /mnt
<third-host># rm -f /boot/initrd.img-2.6.26-1-686-bigmem
<third-host># update-initramfs -k 2.6.26-1-686-bigmem -c
update-initramfs: Generating /boot/initrd.img-2.6.26-1-686-bigmem
<third-host># exit
<third-host># rm /mnt/proc/modules
<third-host>#
  10. Quite a few files on the second disk cannot be used as they are; here is what I needed to edit (all paths are relative to /mnt; don’t edit the ones outside /mnt by mistake; you can chroot into /mnt if you like):
    1. /etc/fstab, /boot/grub/menu.lst, /etc/exports: deleted references to non-existent partitions, changed root and swap devices to /dev/xvda1 and /dev/xvda2 (remember what is currently xvdb will become xvdb once it is is removed and attached to a diskless virtual system)
    2. /etc/ssh/sshd_config to allow root to log in over the network, set: 
      PermitRootLogin yes
PasswordAuthentication yes

      (Remember to revert these later.)

    3. /mnt/boot/grub/menu.lst: 
      title           <dst-host>
root            (hd0,0)
kernel          /boot/vmlinuz-2.6.26-1-686-bigmem root=/dev/xvda1 ro
initrd          /boot/initrd.img-2.6.26-1-686-bigmem
    4. /etc/inittab: add the line: 
      co:2345:respawn:/sbin/getty hvc0 9600 linux

      (This will make ‘virsh console <domain>‘ work.)

    5. /etc/hostname, /etc/hosts, /etc/network/interfaces: to assign a new hostname (<dst-host>) and IP address and not use bridging
    6. /etc/resolv.conf to avoid using self as a nameserver yet (DNS services will probably be moved; also I want to be sure I’m using one single DNS source everywhere to avoid confusion)
    7. Consider disabling some services in /etc/init.d (e.g. by renaming files there to give then .off suffixes). I had to move: 
      cd /etc/init.d
for i in transmission* nis gdm uml-utilities noip2 dhcp3-server \
    dovecot postfix mdadm bind9 kvm vboxdrv postgrey mailgraph; do
mv $i $i.off
done
    8. Consider disabling cronjobs in root’s crontab, users’ crontabs as well as anything under /etc/cron*.
  11. On the virtual system server, shutdown <third-host> (without that NIC names will be in the XML file and will conflict at load time) and dump <third-host>’s VM configuration with: 
    virsh shutdown <third-host>
sleep 10
virsh dumpxml <third-host> > <dst-host>.xml
  12. Edit that XML file and change:
    1. hostname
    2. UUID (just bump the least significant digit)
    3. path to the disk image (and move the image to a sensible name)
    4. NIC MAC address
  13. Create a barebones virtual system by load the XML file with: 
    virsh create <dst-host>.xml

Cloning root filesystem on software RAID device

This procedure has not worked successfully yet! It is a work in progress! (04/01/2009)

This procedure assumes that /dev/sda and /dev/sdb are identically layed with all filesystems and swap using RAID1.

  1. Unmount non-essential non-RAIDed filesystems (e.g. /dev/sdc1 where I do snapshotting)
  2. Remove the entries for these filesystems in /etc/fstab (/dev/sdc1)
  3. Shutdown
  4. Physically disconnect any disks which the removal of entries for the above filesystems facilitates (e.g. sdc)
  5. Physically disconnect the second half the old RAIDed disk pair (sdb)
  6. Startup.
  7. Verify bootable and that /proc/mdstat shows degraded state)
  8. Shutdown.
  9. Physically remove all disconnected disks (sdc and sdb)
  10. Insert new disk to be first half of the new RAIDed disk pair (sdb)
  11. Insert new disk to be second half of the new RAIDed disk pair (sdc)
  12. Startup.
  13. Verify new disks seen in dmesg and fdisk.
  14. Make identical partitions on both disks, configure RAID, start the merging, run mkfs and mkswap.
  15. Save the new MD layout with: 
    {
    echo -e "DEVICE partitions\nMAILADDR root"
    mdadm --examine --scan
} > /etc/mdadm/mdadm.conf
  16. Mount the filesystem on the new RAIDed pair which will become the new / on /mnt.
  17. Mount other filesystems on the new RAIDed pair on the appropriate directory under /mnt.
  18. Rsync data across. E.g.: 
    rsync -ax / /mnt/
rsync -ax /vol1/ /mnt/vol1/
  19. Stop and then disable (by moving /etc/init.d/XXX to /etc/init.d/XXX.off) the following services in the following order: mldonkey, azureus, apache2, cron, i2p.
  20. Do second pass rsyncs.
  21. Edit /mnt/etc/fstab and update the devices (e.g. /dev/md0 –> /dev/md4, /dev/md1 –> /dev/md5, etc).
  22. Reboot using the MBR of the first half of the old RAIDed pair (sda) The new RAID devices could not be seen although the old ones could! Does the initrd need to be updated?
  23. Verify /proc/mdstat shows all devices seen and in sync.
  24. Use grub --batch to write new MBRs on the new RAIDed pair. E.g.: 
    grub --batch
root (hd1,1)
setup (hd1)
root (hd2,1)
setup (hd2)
  25. Note the grub commands needed to boot.
  26. Shutdown.
  27. Physically disconnect the first half of the old RAIDed pair (sda)
  28. Note that at this point what was sdb will become sda and what was sdc will become sdb.
  29. Startup (manual entry of boot commands will be needed because grub.conf still says md1 and we’re now using md5)
  30. Verify /proc/mdstat shows all new devices seen and in sync.
  31. Update /boot/grub/grub.conf.
  32. Run ‘grub --batch‘ again.
  33. Reboot.
  34. Check /proc/mdstat, df, mount, etc.
  35. Shutdown.
  36. Physically remove the first half of the old RAIDed pair.
  37. Shuffle SATA disk connectors up (so that the last port becomes free; this shuffle will not affect device names).
  38. Insert and attach any disks (e.g. sdc)
  39. Make necessary filesystems.
  40. Add the entries for these filesystems in /etc/fstab (/dev/sdc1)
  41. Run: 
    mount -a
  42. Reboot.

See also