p2v

P2V can refer to: read more at WikiPedia

  • logo_virtualized_vmware_s

    My idea is to be able to run my production server (STRATO- OpenSuse) hosting www.waltercedric.com&160; in a VMWARE instance. So I can test and make change to the operating first there before bringing down my host for several hours ;-)

    I would like to duplicate the whole LINUX system (lets call it PROD for now), including data and make 2 VMWARE instance.. why two instances you may ask?

    1. One will be called DEV, and be very instable by definition, it is always a copy of the TEST instance, and will be use to try dirty system changes, that may never be deployed to TEST or PROD
    2. The instance named TEST will be by definition identical to the main server (or PROD). All changes made there has to be documented in small release notes so they can be applied to PROD.

    I am in the process of documenting everything, and I will&160; provide You the set of scripts to make the whole things works. (mainly a bash file that use RSYNC to keep your new server TEST up to date with the PROD), First step is to copy the server on the fly and luckily this is the purpose of this article!

    Solution 1: cloning with VMware vCenter Converter (if no raid, easiest)

    You’ll see later why but VMWARE vCenter Converter is not able to copy software raid /boot devices (what a shame)

    Install VMware vCenter Converter Standalone

    This software is free, all you have to do is register and download the software HERE

    You can install it like any other windows software, just follow the steps

    &160;vmware.vcenter.converter

    &160;vmware.vcenter.converter.1

    Note the default port or change them if you already have another services running on your PC (XAMPP, TeamCity, Bamboo etc…)

    Https Service port: 443
    Remote Converter clients connect to this port on the server and submit conversion tasks.

    HTTP Service port: 80
    Remote machines connect to the server through this port to download the converter client.

    Agent service Port: 9089
    Remote converter agents connect to this port to convert this machine.

    &160;vmware.vcenter.converter.2

    Authorize root access through SSH

    Even if this is considered highly unsecure, VMware vCenter Converter will require it. But temporary having a ssh root access open to the outside world is ok for the time taken to replicate through SSH all disks.

    # vi /etc/ssh/sshd_config

    and set

    PermitRootLogin yes

    Restart the SSH daemon by issuing the following command

    # /etc/init.d/sshd restart

    &160;

    Tell the converter to start replication of remote server

    vmware.vcenter.converter.3

    bang!

    checking that /boot is mounted on my server

    # mount –l
    ..
    /dev/md0 on /boot type ext2 (rw,acl,user_xattr) [/boot]
    ..

    The device /boot is mounted, I did Google for it and found that:

    Converter Standalone 4.0 does not recognize source volumes that reside on Linux Software RAID configurations

    During cloning of powered-on Linux machines, Converter Standalone does not recognize source volumes that are part of a Software RAID configuration (also referred to as multiple disk, or md, configurations).
    (If your question has been resolved please mark the answers as "Helpful" or "Correct".)

    That's because the /boot is on Software RAID which is not supported by Linux P2V: 58">http://www.vmware.com/support/converter/doc/releasenotes_conv40.html58

    I have a RAID1 array

    RAID 1 (mirrored settings/disks) duplicates data across every disk in the array, providing full redundancy. Two (or more) disks each store exactly the same data, at the same time, and at all times. Data is not lost as long as one disk survives. Total capacity of the array equals the capacity of the smallest disk in the array. At any given instant, the contents of each disk in the array are identical to that of every other disk in the array.

    Solution 2: cloning with VMware vCenter Converter (workaround for raid)

    If you still want to clone your server with VMware vCenter Converter and&160;

    • if /boot is the only important file system on Software RAID (/dev/md*), which is not my case :-()
    • You can plug an external non RAID USB disk into the server

    you could workaround this by:

    1. Back up the /boot files to /tmp (or some place with space),
    2. unmount /boot,
    3. Create a new volume (partition or LVM logical volume) on some available storage (might need to plug in a USB drive),
    4. Formatting it with ext3/ext4
    5. Mount it at /boot
    6. Copy all the /boot files over to there,
    7. Make backups of /etc/fstab and /boot/grub/menu.lst
    8. Change fstab and menu.lst to point to the new /boot file system.

    That should allow Converter to work. And you can revert this all very easily by restoring the backups of fstab and menu.lst, unmounting /boot, remounting the original /boot (/dev/md0).

    Solution 3: cloning with VMware vCenter Converter (awful workaround raid)

    Another solution would be to break the Software RAID mirroring, just temporary to make a snapshot, but that would also require me to boot the server –> not acceptable for me!

    Solution 4 cloning with clonesys

    clonesys is a Shell script that can be used to get an image of running Linux boxes.
    The image can then be burned on CD/DVD. This CD/DVD is bootable and can be used to restore the system as well as to install new similar boxes.
    clonesys *IS NOT* a backup tool: it should not be used to backup users data, it is more similar to other tools such as MkCdRec, Mondo Rescue, Timo's Rescue CD, etc...

    1. Get the latest version (wget) and unpack
    2. Configure config.ini and backup.ini
    3. Start ./clonesys.sh and wait
    4. ISO disc will be available in /tmp/clonesys/iso

    Create a new VMWARE machine and boot on the CD to create your new machine.

    Solution 5 cloning live system with Mondo

    Mondo is reliable. It backs up your GNU/Linux server or workstation to tape, CD-R, CD-RW, DVD-R[W], DVD+R[W], NFS or hard disk partition. In the event of catastrophic data loss, you will be able to restore all of your data [or as much as you want], from bare metal if necessary. Mondo is in use by Lockheed-Martin, Nortel Networks, Siemens, HP, IBM, NASA's JPL, the US Dept of Agriculture, dozens of smaller companies, and tens of thousands of users.

    Mondo&160;is comprehensive. Mondosupports LVM 1/2, RAID, ext2, ext3, ext4, JFS, XFS, ReiserFS, VFAT, and can support additional filesystems easily: just e-mail the mailing list with your request. It supports software raid as well as most hardware raid controllers. It supports adjustments in disk geometry, including migration from non-RAID to RAID. Mondoruns on all major Linux distributions (RedHat, RHEL, SuSE, SLES, Mandriva, Debian, Gentoo) and is getting better all the time. You may even use it to backup non-Linux partitions, such as NTFS.

    Mondois free! It has been published under the GPL v2 (GNU Public License), partly to expose it to thousands of potential beta-testers but mostly as a contribution to the Linux community.

    Its main advantages are in my case

    • It wont require any reboot,
    • It can create CD (.ISO with ISOLINUX) that can be later booted in VMWARE.

    If you want to come from a RAID 0,1,5 to a non RAID host setup, this article may help you.

    Solution 6: manual RSYNC or the “you’re on your own”

    Fallback to manual RSYNC of the server…quite tricky. This is what what VMware vCenter is doing in background when it works :-(

    1. Boot with Knoppix in mode 2 (terminal&160; only)
    2. Create partitions with fdisk (or the utility of your choice).
    3. Make file systems, run mkswap, etc.
    4. Mount partitions appropriately.
    5. Run something like: rsync -zav -e ssh oldhost:/ /mnt (FYI, I almost always botch the syntax the first try)
    6. Make changes as appropriate for new system (/etc/fstab, /etc/sysconfig/network and /etc/sysconfig/network-scripts/ifcfg-*, /etc/hosts, and so on).
    7. Run grub to install the boot loader
    8. Reboot. If it boots, do a quick check to make sure all the partitions are started, swap has been turned on, etc.

    I’ve made it once but was blocked at step 9 (because of grub 1 / OpenSuSE 9.3)

    &160;

    If you know any other method that dont require the server to be rebooted (no solutions with dd please as it make corrupted disk on running server)

  •  

    LOGO_virtualized_vmware_S

    You'll need a lot of patience...Since there is no VMWARE Converter for Linux...

    My objective is to virtualizes my Internet server running SUSE in a VMWARE to ease the migration to a more powerful and up to date server.

     

    I am using RSYNC since:

    • I have no access to the machine, So I can't stop the server and make a binary images of the disk as the server is in a STRATO data center in Germany (Berlin)
    • I don't like operations down time.

    I HAVE TO virtualize my server because:

    • The operating system SuSE is too old and is no more security patches available.
    • I want to have a local reference in VMWARE of my Internet server.

    This tutorial can be see as an add-on of this one : http://www.linuxjournal.com/article/9942 (very good by the way) I just complete the missing steps or document some issues I've encountered..

    On your server, run

    # fdisk -l

    Disk /dev/sda: 164.6 GB, 164696555520 bytes
    255 heads, 63 sectors/track, 20023 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes

       Device Boot      Start         End      Blocks   Id  System
    /dev/sda1   *           1           7       56196   fd  Linux raid autodetect
    /dev/sda2               8         138     1052257+  82  Linux swap / Solaris
    /dev/sda3             139       20023   159726262+  fd  Linux raid autodetect

    Disk /dev/sdb: 164.6 GB, 164696555520 bytes
    255 heads, 63 sectors/track, 20023 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes

       Device Boot      Start         End      Blocks   Id  System
    /dev/sdb1   *           1           7       56196   fd  Linux raid autodetect
    /dev/sdb2               8         138     1052257+  82  Linux swap / Solaris
    /dev/sdb3             139       20023   159726262+  fd  Linux raid autodetect

    Disk /dev/md0: 57 MB, 57475072 bytes
    2 heads, 4 sectors/track, 14032 cylinders
    Units = cylinders of 8 * 512 = 4096 bytes

    Disk /dev/md0 doesn't contain a valid partition table

    Disk /dev/md1: 163.5 GB, 163559571456 bytes
    2 heads, 4 sectors/track, 39931536 cylinders
    Units = cylinders of 8 * 512 = 4096 bytes

    Disk /dev/md1 doesn't contain a valid partition table

    This is how partitions look like, sda1 is the root partition, sda2 is a swap partition and sda3 is my home.  The server use a RAID0 with 2 * 160GB disk.

    You'll have to recreate them in the virtual machine. Nothing force me to have a RAID0 also in my VMWARE. I will only create a normal disk layout (sda1,sda2, sda3). Save this output in a text file for further references. By reading the fstab, I will  be able to look at the mount point name later. On your server, run:

    # cat /etc/fstab
    /dev/md1        /       ext3    acl,user_xattr,usrquota 1       1
    /dev/md0            /boot                ext2       acl,user_xattr        1 2
    /dev/sda2            swap                 swap       pri=42                0 0
    /dev/sdb2            swap                 swap       pri=42                0 0

    devpts               /dev/pts             devpts     mode=0620,gid=5       0 0
    proc                 /proc                proc       defaults              0 0
    sysfs                /sys                 sysfs      noauto                0 0

    Download VMWARE Server, you'll have to register and will receive a limited in time version.

    Create a new Virtual machine matching as close as possible the distribution you are using on your production server. Boot the virtual machine with a Knoppix CD, type knoppix 2 at boot time to jump into console mode of Knoppix. Use cfdisk to create the same partition layout, this tool is very easy to use (use bottom menu). Use the same file system, partitions can be set bigger or smaller depending on your intentions with the virtual machine (backup or replacement of a productive machine). My Old server was using GRUB (and not GRUB2), GRUB do not accept to install a boot loader on partition with 256-byte inodes. Unfortunately this is exactly what cfdisk  create as default!

    The quickest and easiest option (when you have control of file system creation) is to format any new Ext3 file systems with 128-byte inodes. This command tells you your e2fsprogs version:

    # mke2fs -V
    mke2fs 1.40.8 (13-Mar-2008)
    Using EXT2FS Library version 1.40.8

    This is the usual command for creating new Ext3 file systems with e2fsprogs:

    # mkfs.ext3 /dev/sda1

    The newer versions that default to 256-byte inodes will emit this warning, and create your new file system anyway:

    Warning: 256-byte inodes not usable on older systems

    To force the creation of 128-byte inodes partitions:

    # mkfs.ext3 -I 128 /dev/sda1

    You can check your work with tune2fs, if you read 128 it is OK to continue

    # tune2fs -l /dev/sda1 | grep -i ‘inode size
    Inode size: 128

    Create new file system

    According to the # cat /etc/fstab

    # sudo mkfs -t ext3 /dev/sda1
    # sudo mkswap /dev/sda2
    # sudo mkfs -t ext2 /dev/sda3

    creating fylsystem

    RSYNC your data

    The example in article was not using a SSH connection with secure keys. If you use public private key pair (recommended), just copy your public, private key in Knoppix root home

    # su
    # cd ~/.ssh

    I have my RSA key on a USB stick, Knoppix mount it thanks to UDEV automatically

    # cp /mnt/usbstick/id_rsa* .

    Try to connect to your host

    # ssh -l root yourHostOrIP

    You'll be prompted for your pass phrase, enter it, if everything work...its time to sync the first partition /mnt/sda1

    # rsync -avx --numeric-ids --progress yourHostOrIP:/ /mnt/sda1/

    You'll be prompted for your pass phrase, enter it, if everything work just wait, when finished do the same for the second partition /mnt/sda3

    # rsync -avx --numeric-ids --progress yourHostOrIP:/home/ /mnt/sda3/

    That was easy.. the most difficult is to make this virtual machine boot, and that mean play with GRUB...

    TO BE CONTINUED... next part this week