Re: [Xen-devel] [PATCH OSSTEST v2 01/19] Introduce mg-pxe-loader-update

2015-06-19 Thread Ian Campbell
On Thu, 2015-06-18 at 18:42 +0100, Ian Jackson wrote:
 Ian Campbell writes ([PATCH OSSTEST v2 01/19] Introduce 
 mg-pxe-loader-update):
  The story for PXE booting via UEFI (at least on arm64) is not so
  straightforward as with pxelinux on x86. There seems to no good
  bootloader to launch via UEFI+pxe, in fact all I could find was grub
  (syslinux, and by extension pxelinux.efi, is x86 only).
  
  Add mg-pxe-loader-update modelled on mg-debian-installer-update which
  will download the necessary grub binaries and produce a grub image
  which can be used to pxe boot.
  
  grub lacks the convenient ability to search for config file based on
  (substrings of) the MAC or IP address. So we arrange for the grub.cfg
  in TftpGrubBase to chain load another config file from
  TftpTmpDir/'$net_default_mac' where $net_default_mac is a grub
  variable which is substituted at boot time.
  
  Actually using this requires that bootp/dhcp provide a next-file so
  UEFI knows what to boot (usually this would be pxelinux.0 which we
  can't use here). Locally we have configured this as $name/pxe.img, so
  we can use different loaders.
 ...
  +curl -s $pfile Packages.gz
  +
  +echo 2 collecting $grubdeb
  +pkgfile=`zcat Packages.gz | grep-dctrl -PX $grubdeb -nsFilename | sort 
  -n -r | head -n1`
  +rc=$?
  +set -e
  +if [ $rc -ne 0 ] || [ x$pkgfile = x ]; then fail $grubdeb package not 
  found; fi
  +curl -s $site/$pkgfile $grubdeb.deb
 
 This is indeed _very_ similar to parts of mg-debian-installer-update.
 Perhaps some of this should be made common ?  You may need to invent a
 new file.

I'll try and come up with mgi-debian (by analogy with cr-* using cri-*)
with some helper routines for fetch packages etc.

 I notice that mg-debian-installer-update has `fetch' which uses a
 Pragma: to try to avoid proxies.  Do we need the same here ?

I suppose we do, yes.

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v2 01/19] Introduce mg-pxe-loader-update

2015-06-18 Thread Ian Campbell
The story for PXE booting via UEFI (at least on arm64) is not so
straightforward as with pxelinux on x86. There seems to no good
bootloader to launch via UEFI+pxe, in fact all I could find was grub
(syslinux, and by extension pxelinux.efi, is x86 only).

Add mg-pxe-loader-update modelled on mg-debian-installer-update which
will download the necessary grub binaries and produce a grub image
which can be used to pxe boot.

grub lacks the convenient ability to search for config file based on
(substrings of) the MAC or IP address. So we arrange for the grub.cfg
in TftpGrubBase to chain load another config file from
TftpTmpDir/'$net_default_mac' where $net_default_mac is a grub
variable which is substituted at boot time.

Actually using this requires that bootp/dhcp provide a next-file so
UEFI knows what to boot (usually this would be pxelinux.0 which we
can't use here). Locally we have configured this as $name/pxe.img, so
we can use different loaders.

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
 Osstest.pm   |  3 ++
 README   |  4 ++-
 mg-pxe-loader-update | 89 
 production-config|  5 +++
 4 files changed, 100 insertions(+), 1 deletion(-)
 create mode 100755 mg-pxe-loader-update

diff --git a/Osstest.pm b/Osstest.pm
index e8bd77b..e668b3c 100644
--- a/Osstest.pm
+++ b/Osstest.pm
@@ -199,6 +199,9 @@ sub readglobalconfig () {
 $c{TftpDiBase} ||= $c{TftpPlayDir}debian-installer;
 $c{TftpDiVersion} ||= 'current';
 
+$c{TftpGrubBase} ||= $c{TftpPlayDir}grub;
+$c{TftpGrubVersion} ||= 'current';
+
 $c{WebspaceFile} ||= $ENV{'HOME'}/public_html/;
 $c{WebspaceUrl} ||= http://$myfqdn/~$whoami/;;
 $c{WebspaceCommon} ||= 'osstest/';
diff --git a/README b/README
index 44e2989..503d15d 100644
--- a/README
+++ b/README
@@ -206,7 +206,9 @@ To run osstest in standalone mode:
  netcat
  chiark-utils-bin
 
- - Optional: ipmitool
+ - Optional:
+ ipmitool -- for hosts which use IPMI for power control
+ grub-common -- for mg-pxe-loader-update
 
  - Write a config file
 ~/.xen-osstest/config
diff --git a/mg-pxe-loader-update b/mg-pxe-loader-update
new file mode 100755
index 000..b336fdb
--- /dev/null
+++ b/mg-pxe-loader-update
@@ -0,0 +1,89 @@
+#!/bin/bash
+# usage
+#   ./mg-pxe-loader-update jessie
+#
+# Requires grub-mkimage (Debian package: grub-common)
+
+# This is part of osstest, an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see http://www.gnu.org/licenses/.
+
+
+set -e
+
+. cri-getconfig
+
+suite=$1
+
+fail () { echo 2 $0: $1; exit 1; }
+
+site=http://ftp.debian.org/debian/
+sbase=$site/dists/$suite
+
+archs=amd64 i386 arm64
+
+dstroot=`getconfig TftpPath`/`getconfig TftpGrubBase`/
+date=`date +%Y-%m-%d`
+dst=$date
+
+grubpfx=`getconfig TftpGrubBase`/$date/grub
+grubcfg=`getconfig TftpTmpDir`/grub.cfg-'$net_default_mac'
+
+mkdir -p $dstroot
+cd $dstroot
+mkdir -p $dst
+cd $dst
+rm -rf grub
+mkdir grub
+
+cat grub/grub.cfg EOF
+set stage1=yes
+configfile $grubcfg
+EOF
+for arch in $archs ; do
+case $arch in
+   amd64) grubdeb=grub-efi-amd64-bin; platform=x86_64-efi;;
+   i386)  grubdeb=grub-efi-ia32-bin;  platform=i386-efi;;
+   arm64) grubdeb=grub-efi-arm64-bin; platform=arm64-efi;;
+   *) echo No grub on $arch 2; exit 0;;
+esac
+
+pfile=$sbase/main/binary-$arch/Packages.gz
+
+curl -s $pfile Packages.gz
+
+echo 2 collecting $grubdeb
+pkgfile=`zcat Packages.gz | grep-dctrl -PX $grubdeb -nsFilename | sort -n 
-r | head -n1`
+rc=$?
+set -e
+if [ $rc -ne 0 ] || [ x$pkgfile = x ]; then fail $grubdeb package not 
found; fi
+curl -s $site/$pkgfile $grubdeb.deb
+
+dpkg-deb -x $grubdeb.deb x
+
+mv x/usr/lib/grub/* grub/
+
+rm -rf x
+
+rm Packages.gz
+
+grub-mkimage -O $platform \
+-d ./grub/$platform \
+-o pxegrub-$arch.efi -p $grubpfx \
+search configfile normal efinet tftp net
+done
+
+echo $date
+echo 2 downloaded $dstroot/$date
diff --git a/production-config b/production-config
index 47c0c4c..46fedc9 100644
--- a/production-config
+++ b/production-config
@@ -85,12 +85,17 @@ TftpPxeTemplates %name%/pxelinux.cfg
 TftpPxeTemplatesReal pxelinux.cfg/%ipaddrhex%
 
 TftpPxeGroup osstest
+# Update with 

Re: [Xen-devel] [PATCH OSSTEST v2 01/19] Introduce mg-pxe-loader-update

2015-06-18 Thread Ian Jackson
Ian Campbell writes ([PATCH OSSTEST v2 01/19] Introduce mg-pxe-loader-update):
 The story for PXE booting via UEFI (at least on arm64) is not so
 straightforward as with pxelinux on x86. There seems to no good
 bootloader to launch via UEFI+pxe, in fact all I could find was grub
 (syslinux, and by extension pxelinux.efi, is x86 only).
 
 Add mg-pxe-loader-update modelled on mg-debian-installer-update which
 will download the necessary grub binaries and produce a grub image
 which can be used to pxe boot.
 
 grub lacks the convenient ability to search for config file based on
 (substrings of) the MAC or IP address. So we arrange for the grub.cfg
 in TftpGrubBase to chain load another config file from
 TftpTmpDir/'$net_default_mac' where $net_default_mac is a grub
 variable which is substituted at boot time.
 
 Actually using this requires that bootp/dhcp provide a next-file so
 UEFI knows what to boot (usually this would be pxelinux.0 which we
 can't use here). Locally we have configured this as $name/pxe.img, so
 we can use different loaders.
...
 +curl -s $pfile Packages.gz
 +
 +echo 2 collecting $grubdeb
 +pkgfile=`zcat Packages.gz | grep-dctrl -PX $grubdeb -nsFilename | sort 
 -n -r | head -n1`
 +rc=$?
 +set -e
 +if [ $rc -ne 0 ] || [ x$pkgfile = x ]; then fail $grubdeb package not 
 found; fi
 +curl -s $site/$pkgfile $grubdeb.deb

This is indeed _very_ similar to parts of mg-debian-installer-update.
Perhaps some of this should be made common ?  You may need to invent a
new file.

I notice that mg-debian-installer-update has `fetch' which uses a
Pragma: to try to avoid proxies.  Do we need the same here ?

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel