Test Procedure with KVM guests + iPXE
=====================================
- 2 guests: iSCSI target/server and iSCSI initiator/client.
- 1 bridge for iSCSI traffic (virbr-iscsi, new), static ip.
- 1 bridge for internet access (virbr0, exists), dhcp ip.
Host:
----
Configure the iSCSI bridge and QEMU access in the host:
$ sudo ip link add dev virbr-iscsi type bridge
$ sudo ip link set dev virbr-iscsi up
$ echo 'allow virbr-iscsi' | sudo tee -a /etc/qemu/bridge.conf
$ sudo chmod +s /usr/lib/qemu/qemu-bridge-helper
iSCSI target:
------------
This guest serves an iSCSI target with one LUN
in iSCSI NIC with IP 10.0.0.1 for IP 10.0.0.2.
Install/boot this guest:
$ qemu-img create -f qcow2 guest-iscsi-target.qcow2 16g
$ qemu-system-x86_64 \
-nodefaults \
-enable-kvm \
-smp 2 -m 4096 \
-serial stdio \
-vga virtio \
-display vnc=0.0.0.0:2 \
-netdev bridge,id=bridge-world,br=virbr0 \
-netdev bridge,id=bridge-iscsi,br=virbr-iscsi \
-device
virtio-net-pci,netdev=bridge-world,id=nic-world,mac=52:54:00:00:00:11 \
-device
virtio-net-pci,netdev=bridge-iscsi,id=nic-iscsi,mac=52:54:00:00:00:22 \
-drive file=guest-iscsi-target.qcow2,if=virtio \
-drive file=$RELEASE-server-amd64.iso,media=cdrom,read-only,if=scsi \
-boot once=d
Configure iSCSI NIC:
$ cat <<EOF | sudo tee /etc/netplan/ens5.yaml && sudo netplan apply
network:
version: 2
ethernets:
ens5:
addresses: [10.0.0.1/24]
EOF
# ip addr list ens5
3: ens5: <...>
link/ether 52:54:00:00:00:22 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.1/24 brd 10.0.0.255 scope global ens4
...
Configure iSCSI target/lun:
# apt-get install -y tgt
# mkdir /var/lib/iscsi
# dd if=/dev/zero of=/var/lib/iscsi/disk bs=1 count=0 seek=8G
# tgtadm --lld iscsi --op new --mode target --tid 1 -T
iqn.2019-03.com.example:target1
# tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1 -b
/var/lib/iscsi/disk
# tgtadm --lld iscsi --op bind --mode target --tid 1 -I 10.0.0.2
# tgt-admin --dump >/etc/tgt/conf.d/target1.conf
iSCSI initiator:
---------------
This guest first boots iPXE to configure iBFT,
and then boots/chainloads to debian-installer.
The netboot initrd does not contain all patched udebs,
so download and install disk-detect and partman-iscsi
from the PPA during the install.
$ wget http://boot.ipxe.org/ipxe.lkrn
$ wget
http://ppa.launchpad.net/mfo/lp1817321v3/ubuntu/dists/$RELEASE/main/installer-amd64/current/images/netboot/ubuntu-installer/amd64/{linux,initrd.gz}
$ python3 -m http.server &
Serving HTTP on 0.0.0.0 port 8000 ...
$ qemu-system-x86_64 \
-nodefaults \
-enable-kvm \
-smp 2 -m 4096 \
-serial stdio \
-vga virtio \
-display vnc=0.0.0.0:1 \
-netdev bridge,id=bridge-world,br=virbr0 \
-netdev bridge,id=bridge-iscsi,br=virbr-iscsi \
-device
virtio-net-pci,netdev=bridge-world,id=nic-world,mac=52:54:00:00:00:01 \
-device
virtio-net-pci,netdev=bridge-iscsi,id=nic-iscsi,mac=52:54:00:00:00:02 \
-kernel ipxe.lkrn
Connect to VNC for iPXE shell:
$ vncviewer <address>:1
iPXE <...>
<wait for NIC address detection>
Press Ctrl-B for iPXE command line.
^B
iPXE>
Configure iSCSI NIC:
iPXE> ifopen net1
iPXE> set net1/ip 10.0.0.2
iPXE> set net1/netmask 255.255.255.0
Configure iBFT: (iSCSI portal 10.0.0.1, LUN 1 on target
iqn.<...>:target1)
iPXE> sanhook iscsi:10.0.0.1:::1:iqn.2019-03.com.example:target1
Registered SAN device 0x80
Boot the installer
(add option 'disk-detect/ibft/enable=true' for installer to detect iBFT iSCSI
disks
and option 'partman-iscsi/iscsi_auto=true' to set the system to boot from
iBFT):
iPXE> ifopen net0
iPXE> kernel http://192.168.122.1:8000/linux initrd=initrd.gz
disk-detect/ibft/enable=true partman-iscsi/iscsi_auto=true --- console=ttyS0
iPXE> initrd http://192.168.122.1:8000/initrd.gz
iPXE> boot
Back to serial console.
Proceed with the installer.
In 'Users and passwords' dialog, select 'Go back', and 'Execute a
shell', and 'Continue'.
Check kernel version and iscsi_ibft.ko module is present.
~ # uname -rv
5.0.0-8-generic #9-Ubuntu SMP Tue Mar 12 21:58:11 UTC 2019
~ # depmod -a
~ # modinfo --filename iscsi_ibft
/lib/modules/5.0.0-8-generic/kernel/drivers/firmware/iscsi_ibft.ko
~ # wget
http://ppa.launchpad.net/mfo/lp1817321v3/ubuntu/pool/main/h/hw-detect/disk-detect_1.117ubuntu7.$VERSION_amd64.udeb
~ # wget
http://ppa.launchpad.net/mfo/lp1817321v3/ubuntu/pool/main/p/partman-iscsi/partman-iscsi_40ubuntu4.$VERSION_all.udeb
~ # udpkg --unpack *.udeb
~ # debconf-get disk-detect/ibft/enable
true
~ # debconf-get partman-iscsi/iscsi_auto
true
(Use this if you need it; e.g., forgot kernel cmdline options)
~ # debconf-set disk-detect/ibft/enable true
Start another installer menu with the new debconf templates/question:
~ # debconf -o d-i /usr/bin/main-menu
Proceed with the installer.
In the 'Partition disks' dialog, the iSCSI LUN should be present:
SCSI3 (0,0,1) (sda) - 8.6 GB IET VIRTUAL-DISK
Proceed with the installer.
In the 'Finish the Installation' dialog, 'Go Back', 'Execute a shell',
'Continue'.
Check ISCSI_AUTO is set to true in the installation target's
/etc/iscsi/iscsi.initramfs.
~ # cat /target/etc/iscsi/iscsi.initramfs
ISCSI_AUTO=true
Resume the installation with 'exit' and finish it.
System reboots.
Back to VNC console, go again to iPXE shell:
$ vncviewer <address>:1
iPXE <...>
<wait for NIC address detection>
Press Ctrl-B for iPXE command line.
^B
iPXE>
Configure iSCSI NIC:
iPXE> ifopen net1
iPXE> set net1/ip 10.0.0.2
iPXE> set net1/netmask 255.255.255.0
Boot from iSCSI (iSCSI portal 10.0.0.1, LUN 1 on target
iqn.<...>:target1)
iPXE> sanboot iscsi:10.0.0.1:::1:iqn.2019-03.com.example:target1
Registered SAN device 0x80
Booting from SAN device 0x80
Back to serial console.
Watch the system boot:
...
Setting up software interface ens4
iscsistart: Logging into iqn.2019-03.com.example:target1 10.0.0.1:3260,1
iscsistart: version 2.0-874
iscsistart: Connection1:0 to [target: iqn.2019-03.com.example:target1,
portal: 10.0.0.1,3260] through [iface: default] is operational now
/scripts/local-top/iscsi: line 397: : Permission denied
/dev/sda1: recovering journal
/dev/sda1: clean, 60800/524288 files, 519693/2096640 blocks
...
ubuntu login: ubuntu
Password:
ubuntu@ubuntu:~$ mount | grep -w /
/dev/sda1 on / type ext4 (rw,relatime,errors=remount-ro) # plus
'data=ordered' in bionic
ubuntu@ubuntu:~$ ls -ld /sys/block/sda
lrwxrwxrwx 1 root root 0 Apr 3 21:38 /sys/block/sda ->
../devices/platform/host2/session1/target2:0:0/2:0:0:1/block/sda
ubuntu@ubuntu:~$ cat
/sys/devices/platform/host2/session1/iscsi_session/session1/targetname
iqn.2019-03.com.example:target1
syslog example
==============
$ sed -n '/iBFT found/p ; /iBFT detected/,/iBFT disk detection
finished/p' syslog
Apr 3 20:03:53 kernel: [ 0.006887] iBFT found at 0x9e520.
Apr 3 20:12:59 kernel: [ 546.682048] iBFT detected.
Apr 3 20:12:59 disk-detect: insmod
/lib/modules/5.0.0-8-generic/kernel/drivers/scsi/iscsi_boot_sysfs.ko
Apr 3 20:12:59 disk-detect: insmod
/lib/modules/5.0.0-8-generic/kernel/drivers/firmware/iscsi_ibft.ko
Apr 3 20:12:59 disk-detect: # BEGIN RECORD 2.0-874
Apr 3 20:12:59 disk-detect: iface.initiatorname =
iqn.2010-04.org.ipxe:00000000-0000-0000-0000-000000000000
Apr 3 20:12:59 disk-detect: iface.hwaddress = 52:54:00:00:00:02
Apr 3 20:12:59 disk-detect: iface.bootproto = STATIC
Apr 3 20:12:59 disk-detect: iface.ipaddress = 10.0.0.2
Apr 3 20:12:59 disk-detect: iface.subnet_mask = 255.255.255.0
Apr 3 20:12:59 disk-detect: iface.primary_dns = 192.168.122.1
Apr 3 20:12:59 disk-detect: iface.vlan_id = 0
Apr 3 20:12:59 disk-detect: iface.net_ifacename = ens4
Apr 3 20:12:59 disk-detect: node.name = iqn.2019-03.com.example:target1
Apr 3 20:12:59 disk-detect: node.conn[0].address = 10.0.0.1
Apr 3 20:12:59 disk-detect: node.conn[0].port = 3260
Apr 3 20:12:59 disk-detect: node.boot_lun = 01000000
Apr 3 20:12:59 disk-detect: # END RECORD
Apr 3 20:12:59 disk-detect: Setting up software interface ens4
Apr 3 20:12:59 disk-detect: iscsistart:
Apr 3 20:12:59 disk-detect: can not connect to iSCSI daemon (111)!
Apr 3 20:12:59 disk-detect:
Apr 3 20:12:59 disk-detect: iscsistart: version 2.0-874
Apr 3 20:12:59 kernel: [ 546.699057] Loading iSCSI transport class
v2.0-870.
Apr 3 20:13:00 kernel: [ 547.707438] iscsi: registered transport (tcp)
Apr 3 20:13:00 kernel: [ 547.709836] scsi host2: iSCSI Initiator over
TCP/IP
Apr 3 20:13:00 disk-detect: iscsistart: Connection1:0 to [target:
iqn.2019-03.com.example:target1, portal: 10.0.0.1,3260] through [iface:
default] is operational now
Apr 3 20:13:00 kernel: [ 547.716901] scsi 2:0:0:0: RAID
IET Controller 0001 PQ: 0 ANSI: 5
Apr 3 20:13:00 kernel: [ 547.719266] scsi 2:0:0:0: Attached scsi
generic sg0 type 12
Apr 3 20:13:00 kernel: [ 547.722572] scsi 2:0:0:1: Direct-Access
IET VIRTUAL-DISK 0001 PQ: 0 ANSI: 5
Apr 3 20:13:00 kernel: [ 547.724360] sd 2:0:0:1: Attached scsi
generic sg1 type 0
Apr 3 20:13:00 kernel: [ 547.724668] sd 2:0:0:1: Power-on or device
reset occurred
Apr 3 20:13:00 disk-detect: iscsistart: Logging into
iqn.2019-03.com.example:target1 10.0.0.1:3260,1
Apr 3 20:13:00 kernel: [ 547.727114] sd 2:0:0:1: [sda] 16777216
512-byte logical blocks: (8.59 GB/8.00 GiB)
Apr 3 20:13:00 kernel: [ 547.727117] sd 2:0:0:1: [sda] 4096-byte
physical blocks
Apr 3 20:13:00 kernel: [ 547.727463] sd 2:0:0:1: [sda] Write Protect
is off
Apr 3 20:13:00 kernel: [ 547.727466] sd 2:0:0:1: [sda] Mode Sense: 69
00 10 08
Apr 3 20:13:00 kernel: [ 547.728047] sd 2:0:0:1: [sda] Write cache:
enabled, read cache: enabled, supports DPO and FUA
Apr 3 20:13:00 kernel: [ 547.742488] sd 2:0:0:1: [sda] Attached SCSI
disk
Apr 3 20:13:00 disk-detect: iBFT disk detection finished.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1817321
Title:
installer does not support iSCSI iBFT
Status in debian-installer package in Ubuntu:
Confirmed
Status in hw-detect package in Ubuntu:
Confirmed
Status in linux package in Ubuntu:
Fix Released
Status in partman-iscsi package in Ubuntu:
Confirmed
Status in debian-installer source package in Bionic:
Confirmed
Status in hw-detect source package in Bionic:
Confirmed
Status in linux source package in Bionic:
Fix Released
Status in partman-iscsi source package in Bionic:
Confirmed
Status in debian-installer source package in Cosmic:
Confirmed
Status in hw-detect source package in Cosmic:
Confirmed
Status in linux source package in Cosmic:
Fix Released
Status in partman-iscsi source package in Cosmic:
Confirmed
Status in debian-installer source package in Disco:
Confirmed
Status in hw-detect source package in Disco:
Confirmed
Status in linux source package in Disco:
Fix Released
Status in partman-iscsi source package in Disco:
Confirmed
Status in debian-installer source package in Eoan:
Confirmed
Status in hw-detect source package in Eoan:
Confirmed
Status in linux source package in Eoan:
Fix Released
Status in partman-iscsi source package in Eoan:
Confirmed
Bug description:
[Impact]
* It's not possible to access iBFT (iSCSI Boot Firmware Table) information
(settings for network interface, initiator, and target) in the installer
because the 'iscsi_ibft' module is not present in udeb packages.
* Even if it was, the installer does not handle iBFT information at all,
thus any settings are ignored, and iSCSI-related configuration has to
be done manually or with workarounds.
* This impacts user-experience and automatic installation on systems and
deployments which actually do provide the iBFT feature and information,
but cannot use it practically.
* With proper iBFT support in the installer (kernel module in udeb package
and automatic iSCSI-related configuration) users will be able to rely on
iBFT to install/deploy Ubuntu on their servers and datacenters.
* These fixes add the 'iscsi_ibft' kernel module in the scsi-modules udeb,
and configure network/iSCSI according to iBFT information in disk-detect.
This is done in disk-detect so that the iSCSI LUNs are detected as disks
(useful in case of no other disks in the system so the installer doesn't
complain nor wait too long) and that any partman-related preseed options
are not required and may be still available for the user.
[Test Case]
* linux package / kernel module in udeb:
$ dpkg-deb -c scsi-modules_*.udeb | grep iscsi_ibft.ko
Check the module loads in the installer environment.
See comment with example for disco.
* d-i/hw-detect/partman-iscsi package:
See comments 11, 12, 13.
[Regression Potential]
* linux package: low, the kernel module is not loaded by default,
and only checks whether iBFT information is present in firmware,
then exposes that in sysfs in read-only mode.
* d-i/hw-detect/partman-iscsi:
- d-i: kernel version update to include iscsi_ibft module,
based on kernel released to -updates plus one week
monitoring bug reports -- it should be OK.
Tested on amd64/i386/arm64/ppc64el on QEMU, plus amd64
on baremetal -- see comment 11.
- hw-detect: low, the changes are enabled by a preseed option.
see comment 12.
- partman-iscsi: low, simple changes, plus one fix that has
been tested in detail, and falls back to
previous behavior if it fails.
see comment 13.
[Other Info]
* This has been verified both by the developer with a simple iSCSI
iBFT environment (2 VMs: iSCSI target & initiator with UEFI+iPXE)
and by an user with system/firmware that supports iBFT for iSCSI.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-installer/+bug/1817321/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : [email protected]
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
