Russell, That's what I had been thinking.
# rpm -qa | grep -i xcat | sort conserver-xcat-8.1.16-10.x86_64 elilo-xcat-3.14-4.noarch ipmitool-xcat-1.8.11-3.x86_64 perl-xCAT-2.8.3-snap201311122316.noarch syslinux-xcat-3.86-2.noarch xCAT-2.8.3-snap201311122318.x86_64 xCAT-buildkit-2.8.3-snap201311122318.noarch xCAT-client-2.8.3-snap201311122316.noarch xCAT-genesis-base-x86_64-2.8-snap201308090229.noarch xCAT-genesis-scripts-x86_64-2.8.3-snap201311122318.noarch xCAT-server-2.8.3-snap201311122316.noarch If that is the case, I am troubled by the incorrect pxelinux.cfg configuration generated by 'mknb x86_64'. And this is to say nothing of having successful node discover, which still eludes me. On 01/21/2014 03:58 PM, Russell Jones wrote: > xNBA is a customized gpxe image that xCAT uses. > > NBFS is the older maintenance image that was used for if you set your > node to boot to shell, or booted a runimage script. NBFS is deprecated, > and Genesis replaced NBFS as the maintenance image for these tasks. > > In a standard 2.8 install, there should no longer be any nbk/nbfs RPMs > installed - Genesis replaced them. > > perl-xCAT-2.8.3-snap201311122316.noarch > xCAT-2.8.3-snap201311122318.x86_64 > xCAT-client-2.8.3-snap201311122316.noarch > xCAT-genesis-base-x86_64-2.8-snap201308090229.noarch > elilo-xcat-3.14-4.noarch > xCAT-server-2.8.3-snap201311122316.noarch > xCAT-genesis-scripts-x86_64-2.8.3-snap201311122318.noarch > ipmitool-xcat-1.8.11-3.x86_64 > conserver-xcat-8.1.16-10.x86_64 > xCAT-buildkit-2.8.3-snap201311122318.noarch > syslinux-xcat-3.86-2.noarch > > > > On 1/21/2014 2:38 PM, Josh Nielsen wrote: >> Hi Jonathan, >> >> Yes, I definitely think that would cause a problem. This is jogging my >> memory because I think that when the new Genesis boot loader was >> rolled out in the first version of xCAT that supported it that I faced >> a similar problem. I had assumed that only Genesis was needed but xNBA >> is still used an an intermediate image even if it is no longer the >> final image. I will check my yum repos as soon as I can - but by some >> unfortunate coincidence I just discovered that YUM is not working >> since our RHEL license expired three days ago (unbeknownst to me until >> 10 minutes ago). Do you have xCAT-genesis-x86_64 and elilo-xCAT? You >> may even have to pull xNBA images from an older install(?) and then >> run mknb to build the images. >> >> I remember downloading the tarred files with the RPM manually and >> creating a local repo for xCAT. Whenever I get YUM back I'll give you >> more specifics if I can. >> >> -Josh >> >> On Tue, Jan 21, 2014 at 1:54 PM, Jonathan Mills <[email protected]> wrote: >>> Josh, >>> >>> I don't doubt that you're on to something. But if this is the case, it >>> means my systems are missing some files, namely: >>> >>> /tftpboot/xcat/nbk.x86_64 >>> /tftpboot/xcat/nbfs.x86_64.gz >>> >>> Can you tell me what RPM installed those files on your system? They >>> don't exist on mine, and even a 'yum provides' doesn't find them. >>> >>> >>> On 01/21/2014 11:51 AM, Josh Nielsen wrote: >>>> Hi Jonathan, >>>> >>>> It is my understanding, from extensive debugging and notes that I have >>>> taken about the xCAT netbooting process in the past, that xCAT uses a >>>> two-stage image deployment method. It will first come up with a more >>>> "generic" boot image (normally xnba or sometimes yaboot) which - when it >>>> contacts the xCAT headnode (or the node handling DHCP requests) - the >>>> headnode will then recognize the current image on the client that is >>>> sending requests to DHCP for further boot instructions, and will tell >>>> the client to then load another image based on the subnet and image type >>>> it is currently using. For example my headnode's /etc/dhcpd.conf file >>>> has an entry that looks like this: >>>> >>>> hared-network eth0 { >>>> subnet 10.20.0.0 netmask 255.255.0.0 { >>>> max-lease-time 43200; >>>> min-lease-time 43200; >>>> default-lease-time 43200; >>>> next-server 10.20.0.1; >>>> option log-servers 10.20.0.1; >>>> option ntp-servers 10.20.0.1; >>>> option domain-name "xxxxxxxxx"; >>>> option domain-name-servers 10.20.0.1; >>>> if option user-class-identifier = "xNBA" and option >>>> client-architecture = 00:00 { #x86, xCAT Network Boot Agent >>>> always-broadcast on; >>>> filename = >>>> "http://10.20.0.1/tftpboot/xcat/xnba/nets/10.20.0.0_16";; >>>> } else if option user-class-identifier = "xNBA" and option >>>> client-architecture = 00:09 { #x86, xCAT Network Boot Agent >>>> filename = >>>> "http://10.20.0.1/tftpboot/xcat/xnba/nets/10.20.0.0_16.uefi";; >>>> } else if option client-architecture = 00:00 { #x86 >>>> filename "xcat/xnba.kpxe"; >>>> } else if option vendor-class-identifier = "Etherboot-5.4" { #x86 >>>> filename "xcat/xnba.kpxe"; >>>> } else if option client-architecture = 00:07 { #x86_64 uefi >>>> filename "xcat/xnba.efi"; >>>> } else if option client-architecture = 00:09 { #x86_64 uefi >>>> alternative id >>>> filename "xcat/xnba.efi"; >>>> } else if option client-architecture = 00:02 { #ia64 >>>> filename "elilo.efi"; >>>> } else if substring(filename,0,1) = null { #otherwise, provide >>>> yaboot if the client isn't specific >>>> filename "/yaboot"; >>>> } >>>> range dynamic-bootp 10.20.200.254 10.20.254.254; >>>> } # 10.20.0.0/255.255.0.0 <http://10.20.0.0/255.255.0.0> subnet_end >>>> >>>> So if it boots with the xNBA image it then directs it to the >>>> http://10.20.0.1/tftpboot/xcat/xnba/nets/10.20.0.0_16 which has the >>>> genesis boot instructions in it: >>>> >>>> #!gpxe >>>> imgfetch -n kernel >>>> http://${next-server}/tftpboot/xcat/genesis.kernel.x86_64 quiet >>>> xcatd=10.20.0.1:3001 <http://10.20.0.1:3001> BOOTIF=01-${netX/machyp} >>>> imgfetch -n nbfs http://${next-server}/tftpboot/xcat/genesis.fs.x86_64.gz >>>> imgload kernel >>>> imgexec kernel >>>> >>>> So first it boots with xnba (first stage of boot), it contacts the DHCP >>>> server which gives it a "next-server" option of itself (saying to the >>>> client: request the next image from me - the headnode - again), and then >>>> gives it a boot file with instructions for the next image, then it >>>> executes it and finally loads genesis. You will also notice that the >>>> very last options (if it matches nothing else) is yaboot, which is >>>> another generic image, which will in turn probably request the next >>>> image. Try watching your log for the tftp daemon messages to see what is >>>> being sent. >>>> >>>> I wonder if you are having problems at the first stage DHCP redirecting >>>> stage though. Check your options statements in /etc/dhcpd.conf to see >>>> where it is directing xNBA images. >>>> >>>> Regards, >>>> Josh Nielsen >>>> >>>> >>>> On Tue, Jan 21, 2014 at 10:26 AM, Jonathan Mills <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> Wang, >>>> >>>> Thank you for your response. I did some digging and here is what I >>>> found. >>>> >>>> cat /tftpboot/xcat/xnba/nets/10.100.0.0_24 >>>> #!gpxe >>>> imgfetch -n kernel >>>> http://${next-server}/tftpboot/xcat/genesis.kernel.x86_64 quiet >>>> xcatd=10.100.0.1:3001 <http://10.100.0.1:3001> >>>> BOOTIF=01-${netX/machyp} >>>> imgfetch -n nbfs >>>> http://${next-server}/tftpboot/xcat/genesis.fs.x86_64.lzma >>>> imgload kernel >>>> imgexec kernel >>>> >>>> >>>> >>>> cat /tftpboot/pxelinux.cfg/0A6400 >>>> DEFAULT xCAT >>>> LABEL xCAT >>>> KERNEL xcat/nbk.x86_64 >>>> APPEND initrd=xcat/nbfs.x86_64.gz quiet xcatd=10.100.0.1:3001 >>>> <http://10.100.0.1:3001> >>>> >>>> >>>> >>>> So, clearly, those things don't match up. That strikes me as an xCAT >>>> issue, but nevermind. I manually modified >>>> /tftpboot/pxelinux.cfg/0A6400 >>>> to make it look like: >>>> >>>> DEFAULT xCAT >>>> LABEL xCAT >>>> KERNEL xcat/genesis.kernel.x86_64 >>>> APPEND initrd=xcat/genesis.fs.x86_64.lzma quiet >>>> xcatd=10.100.0.1:3001 <http://10.100.0.1:3001> >>>> BOOTIF=eth0 >>>> >>>> >>>> (It is safe, in this case, to designate BOOTIF as 'eth0' -- with >>>> Cisco >>>> UCS hardware, and using vNICs, the first interface will always show >>>> up >>>> in Linux as eth0 -- at least, that is my experience). >>>> >>>> After this change, I was indeed able to PXE boot the first node, and >>>> I >>>> was hopeful that node discovery would then take place. However, this >>>> still did not occur. On console, I dug into the running genesis >>>> image >>>> on the first node, and I found that it had no ethernet interfaces >>>> whatsoever, because the genesis kernel has no driver support for >>>> Cisco >>>> UCS hardware. >>>> >>>> For example, this is the ethtool output of a Cisco UCS vNIC: >>>> >>>> [root@ncsu-hn nets]# ethtool -i eth0 >>>> driver: enic >>>> version: 2.1.1.39 >>>> firmware-version: 2.0(4b) >>>> bus-info: 0000:06:00.0 >>>> supports-statistics: yes >>>> supports-test: no >>>> supports-eeprom-access: no >>>> supports-register-dump: no >>>> supports-priv-flags: no >>>> >>>> >>>> You can see it requires the 'enic' kernel module, usually located at: >>>> /lib/modules/`uname -r`/kernel/drivers/net/enic/enic.ko >>>> >>>> This module isn't found within the genesis image, so the node PXE >>>> boots, >>>> and then can do no more. Node discovery fails. >>>> >>>> On 01/20/2014 09:19 PM, Xiao Peng Wang wrote: >>>> > xCAT is using genesis (an xCAT customized pxe tool) to function >>>> the >>>> > discovery process. The configuration for genesis is put in >>>> > /tftpboot/xcat/xnba/nets/ for a specific network. Could you check >>>> your >>>> > specific xnba configuration file for your deployment network has >>>> been >>>> > put in /tftpboot/xcat/xnba/nets/? >>>> > >>>> > The prerequisite for booting of genesis is to make the node has a >>>> > dynamic IP address. Did you configure the dynamic IP range for >>>> your >>>> > deployment network? Could you take a look of your syslog to see >>>> whether >>>> > the node has sent out dhcp request and what did your dhcp server >>>> replied >>>> > to them? >>>> > >>>> > Thanks >>>> > Best Regards >>>> > >>>> >>>> ---------------------------------------------------------------------- >>>> > Wang Xiaopeng (王晓朋) >>>> > IBM China System Technology Laboratory >>>> > Tel: 86-10-82453455 >>>> > Email: [email protected] <mailto:[email protected]> >>>> > Address: 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West >>>> Road, >>>> > Haidian District Beijing P.R.China 100193 >>>> > >>>> > Inactive hide details for Jonathan Mills ---2014/01/19 >>>> 06:24:02---I'm >>>> > running xCAT 2.8.3 and CentOS 6.4 atop of Cisco UCS-C harJonathan >>>> Mills >>>> > ---2014/01/19 06:24:02---I'm running xCAT 2.8.3 and CentOS 6.4 >>>> atop of >>>> > Cisco UCS-C hardware. I'm attempting to do a sequent >>>> > >>>> > From: Jonathan Mills <[email protected] >>>> <mailto:[email protected]>> >>>> > To: xCAT Users Mailing list <[email protected] >>>> <mailto:[email protected]>>, >>>> > Date: 2014/01/19 06:24 >>>> > Subject: [xcat-user] Frustrating time with sequential node >>>> discovery >>>> > >>>> > >>>> >>>> ------------------------------------------------------------------------ >>>> > >>>> > >>>> > >>>> > I'm running xCAT 2.8.3 and CentOS 6.4 atop of Cisco UCS-C >>>> hardware. I'm >>>> > attempting to do a sequential nodediscovery. I've pre-populated >>>> the >>>> > nodelist table with the nodenames, so I shouldn't need to do >>>> anything >>>> > more than >>>> > >>>> > nodediscoverystart noderange=node[1-15] >>>> > >>>> > However, none of the nodes ever gets discovered. >>>> > >>>> > Digging deeper, it seems that none of them ever successfully PXE >>>> boot at >>>> > all. They should be PXE booting off of the genesis netboot image >>>> and >>>> > speaking back to the xcatmaster, correct? >>>> > >>>> > When I run 'mknb x86_64', it populates /tftpboot/pxelinux.cfg with >>>> > entries to non-existent netboot images. Watch: >>>> > >>>> > [root@ncsu-hn ~]# rpm -qf /opt/xcat/sbin/mknb >>>> > xCAT-client-2.8.3-snap201311122316.noarch >>>> > [root@ncsu-hn ~]# mknb x86_64 >>>> > Creating genesis.fs.x86_64.lzma in /tftpboot/xcat >>>> > [root@ncsu-hn ~]# cd /tftpboot/pxelinux.cfg/ >>>> > [root@ncsu-hn pxelinux.cfg]# ls >>>> > 0A6400 0A6500 0A6600 7F 98300D 98300DE6 98300DE7 C0A86B >>>> > [root@ncsu-hn pxelinux.cfg]# cat * >>>> > DEFAULT xCAT >>>> > LABEL xCAT >>>> > KERNEL xcat/nbk.x86_64 >>>> > APPEND initrd=xcat/nbfs.x86_64.gz quiet xcatd=10.100.0.1:3001 >>>> <http://10.100.0.1:3001> >>>> > DEFAULT xCAT >>>> > LABEL xCAT >>>> > KERNEL xcat/nbk.x86_64 >>>> > APPEND initrd=xcat/nbfs.x86_64.gz quiet xcatd=10.101.0.1:3001 >>>> <http://10.101.0.1:3001> >>>> > DEFAULT xCAT >>>> > LABEL xCAT >>>> > KERNEL xcat/nbk.x86_64 >>>> > APPEND initrd=xcat/nbfs.x86_64.gz quiet xcatd=10.102.0.1:3001 >>>> <http://10.102.0.1:3001> >>>> > DEFAULT xCAT >>>> > LABEL xCAT >>>> > KERNEL xcat/nbk.x86_64 >>>> > APPEND initrd=xcat/nbfs.x86_64.gz quiet xcatd=127.0.0.1:3001 >>>> <http://127.0.0.1:3001> >>>> > DEFAULT xCAT >>>> > LABEL xCAT >>>> > KERNEL xcat/nbk.x86_64 >>>> > APPEND initrd=xcat/nbfs.x86_64.gz quiet xcatd=152.48.13.3:3001 >>>> <http://152.48.13.3:3001> >>>> > DEFAULT xCAT >>>> > LABEL xCAT >>>> > KERNEL xcat/nbk.x86_64 >>>> > APPEND initrd=xcat/nbfs.x86_64.gz quiet >>>> xcatd=152.48.13.230:3001 <http://152.48.13.230:3001> >>>> > DEFAULT xCAT >>>> > LABEL xCAT >>>> > KERNEL xcat/nbk.x86_64 >>>> > APPEND initrd=xcat/nbfs.x86_64.gz quiet >>>> xcatd=152.48.13.231:3001 <http://152.48.13.231:3001> >>>> > DEFAULT xCAT >>>> > LABEL xCAT >>>> > KERNEL xcat/nbk.x86_64 >>>> > APPEND initrd=xcat/nbfs.x86_64.gz quiet >>>> xcatd=192.168.107.10:3001 <http://192.168.107.10:3001> >>>> > [root@ncsu-hn pxelinux.cfg]# cd ../xcat/ >>>> > [root@ncsu-hn xcat]# ls -la >>>> > total 21528 >>>> > drwxr-xr-x 4 root root 4096 Jan 17 13:06 . >>>> > drwxr-xr-x. 7 root root 4096 Jan 18 22:02 .. >>>> > -rwxr-xr-x 1 root root 242929 Jan 15 2012 elilo-x64.efi >>>> > -rw-r--r-- 1 root root 17573621 Jan 18 22:03 >>>> genesis.fs.x86_64.lzma >>>> > -rwxr-xr-x 1 root root 3986608 Aug 9 06:29 >>>> genesis.kernel.x86_64 >>>> > drwxr-xr-x 3 root root 4096 Jan 17 13:06 osimage >>>> > drwxr-xr-x 3 root root 4096 Dec 23 07:42 xnba >>>> > -rw-r--r-- 1 root root 139200 Oct 28 16:16 xnba.efi >>>> > -rw-r--r-- 1 root root 74792 Oct 28 16:16 xnba.kpxe >>>> > >>>> > >>>> > >>>> > As you can see....it ought to be netbooting the genesis kernel, >>>> but >>>> > instead all my pxelinux.cfg/* files are instructing clients to >>>> boot the >>>> > non-existent "nbk.x86_64" image. >>>> > >>>> > Your advice is appreciated. >>>> > >>>> > -- >>>> > Jonathan Mills >>>> > Systems Administrator >>>> > Renaissance Computing Institute >>>> > UNC-Chapel Hill >>>> > >>>> > >>>> >>>> ------------------------------------------------------------------------------ >>>> > CenturyLink Cloud: The Leader in Enterprise Cloud Services. >>>> > Learn Why More Businesses Are Choosing CenturyLink Cloud For >>>> > Critical Workloads, Development Environments & Everything In >>>> Between. >>>> > Get a Quote or Start a Free Trial Today. >>>> > >>>> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk >>>> > _______________________________________________ >>>> > xCAT-user mailing list >>>> > [email protected] >>>> <mailto:[email protected]> >>>> > https://lists.sourceforge.net/lists/listinfo/xcat-user >>>> > >>>> > >>>> >>>> -- >>>> Jonathan Mills >>>> Systems Administrator >>>> Renaissance Computing Institute >>>> UNC-Chapel Hill >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> CenturyLink Cloud: The Leader in Enterprise Cloud Services. >>>> Learn Why More Businesses Are Choosing CenturyLink Cloud For >>>> Critical Workloads, Development Environments & Everything In Between. >>>> Get a Quote or Start a Free Trial Today. >>>> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk >>>> _______________________________________________ >>>> xCAT-user mailing list >>>> [email protected] >>>> <mailto:[email protected]> >>>> https://lists.sourceforge.net/lists/listinfo/xcat-user >>>> >>>> >>> -- >>> Jonathan Mills >>> Systems Administrator >>> Renaissance Computing Institute >>> UNC-Chapel Hill >>> >>> ------------------------------------------------------------------------------ >>> CenturyLink Cloud: The Leader in Enterprise Cloud Services. >>> Learn Why More Businesses Are Choosing CenturyLink Cloud For >>> Critical Workloads, Development Environments & Everything In Between. >>> Get a Quote or Start a Free Trial Today. >>> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> xCAT-user mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/xcat-user >> ------------------------------------------------------------------------------ >> CenturyLink Cloud: The Leader in Enterprise Cloud Services. >> Learn Why More Businesses Are Choosing CenturyLink Cloud For >> Critical Workloads, Development Environments & Everything In Between. >> Get a Quote or Start a Free Trial Today. >> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk >> _______________________________________________ >> xCAT-user mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/xcat-user > > > ------------------------------------------------------------------------------ > CenturyLink Cloud: The Leader in Enterprise Cloud Services. > Learn Why More Businesses Are Choosing CenturyLink Cloud For > Critical Workloads, Development Environments & Everything In Between. > Get a Quote or Start a Free Trial Today. > http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk > _______________________________________________ > xCAT-user mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/xcat-user > -- Jonathan Mills Systems Administrator Renaissance Computing Institute UNC-Chapel Hill ------------------------------------------------------------------------------ CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk _______________________________________________ xCAT-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/xcat-user
