XenServer boot from iSCSI

UPDATE: XenServer boot from iSCSI (at least with iBFT) is just plain awful. Don't bother.  Multipath does not work. Too many hacks to try to get things to work. It is not supported, anyway, and it will probably break every time you do an upgrade.  Plus, the NICs that you use for iSCSI boot will be unusable for any other purpose.  This is aspect of XenServer is very immature and not robust.

1. Credits (Special thanks for pointers from:)
1. https://www.krystalmods.com/index.php?title=xenserver-6-supports-iscsi-boot-undocumented-feature&more=1&c=1&tb=1&pb=1
2. http://serverfault.com/questions/598773/install-xenserver-on-iscsi-target
3. http://serverfault.com/questions/431864/boot-from-iscsi-how-does-it-work
2. Notes/Warnings
1. This was originally writtenf or XenServer 6.2.
2. The NIC that is used for iSCSI boot will not be available for use for any other purposes (admin network, regular storage network, VM network, etc.)
3. This is with Intel I350 Gbit NICs, as on a Supermicro X9DRT motherboard, which uses the ibft module.
4. Once booted, you're in a Busybox environment... some commands will be limited or may not work the way in which you are accustomed on a full Linux system. Consider the information here for more troubleshooting steps to confirm that iSCSI works (LUNs are reachable, access is granted, etc.)
5. As a best practice, use all lower-case named target and initiator names (some firmwares may silently convert case).
6. WARNING: If you do not have your LUNs properly masked, do not specify the installation target correctly, and so forth, you may destroy your data. Know what you are doing, and use at your own risk.
3. Connect up the Intel NICs
4. Enable the Option ROM in the BIOS and add to the boot order; configure the iSCSI boot-enabled NICs in the NIC ROM set-up (perhaps ctrl-D when prompted after POST and prior to OS boot); You'll want to configure networking, initiator and target name; consider following these SAN best practices
5. Configure the iscsi target to allow access from this initiator
6. Boot off the CD and enter the command shell:
1. Insert the XenServer 6.2 CD and power on the computer.  At the "boot: " prompt, type "shell" and press enter.
7. Prepare iscsi:
1. echo "InitiatorName=iqn.2014-01.local:hostname" > /etc/iscsi/initiatorname.iscsi (replace the initiator name and/or hostname as appropriate)
2. echo "node.session.initial_logon_retry_max = 60" >> /etc/iscsi/iscsid.conf
3. modprobe iscsi_ibft
4. modprobe scsi_transport_iscsi
5. modprobe iscsi_tcp
6. iscsid -c /etc/iscsi/iscsid.conf -i /etc/iscsi/initiatorname.iscsi -f &
8. set up multipath (if you have multiple paths)
1. modprobe dm-multipath
9. Start the installer
1. /opt/xensource/installer/init --use_ibft
2. (use --mpath if you have set up multipath)
3. (use --device_mapper_multipath=true)
4. (use --network_device=eth0 , or the correct iSCSI network device, if needed)

Multipath doesn't seem to "just work"... at one point, I had to do this, because if I don't, then installer runs without multipath, but when the init does switchroot, the OS hangs and complains that "a device that was previously mounted without multipath is now mounted with multipath."

1. Go to http://debaan.blogspot.com/2014/10/iscsi-stuff-in-busybox-on-linux.html and follow those steps to get logged in to both LUNs and get them seen with multipath.
2. Start the installer as before.  Wait until the last step, when it says that it is ready to reboot.
3. Alt+F2 to get to a command shell (again).
4. Set up a chroot:
1. mount -o bind /dev /tmp/root/dev
2. mount -o bind /sys /tmp/root/sys
3. mount -o bind /proc /tmp/root/proc
4. chroot /tmp/root /bin/bash
5. cd /boot
6. re-run the
   iscsiadm -m discovery -t sendtargets -p 10.10.10.20 commands to make the chroot environment aware of the targets
7. mkinitrd --with=dm-multipath --with=iscsi_ibft --with=scsi_transport_iscsi -f /boot/initrd-2.6.32.43-0.4.1.xs1.8.0.835.170778xen 2.6.32.43-0.4.1.xs1.8.0.835.170778xen
8.