Hello,
Still trying to separate provisionning (management/install) and public
(production/data) subnets with xCAT 2.15 x86_64 Centos 8 / stateless
nodes, I've stumbled about a conceptual questioning meaning I may not
quite understand the way it is supposed to work which comes down to the
2 following questions :
- how do you handle the fact that postscripts run very late in the boot
"sequence" (it's Systemd) when you rely on 'confignetwork'
to setup your public interface ?
- how do you unconfigure, if needed or wanted the provisioning interface
once the image is booted (I can think of reasons for doing so)
Here's what I've done :
Basically I followed steps linked by Cassandra (thanks again !) in a
previous thread which consist in (for a node named
maestro-300-prov.<provisionning domain> which I want clients to know as
maestro-300.<public-domain> for instance) :
- create 2 subnets with 2 domains : maestro.pasteur.fr and
maestroprov.pasteur.fr
- setup dhcpinterfaces (site) to serve both but with 'noboot' for the
public subnet facing one
- have my external DNS (that xCAT uses) handle the above domains
[additionnal difficulty for me is that maestroprov.pasteur.fr is a non
routed subnet]
- create xCAT maestro-300-prov node object with ip and name in
provisioning subnet : maestro-300-prov.maestroprov.pasteur.fr
- configure (in nics table) an additionnal interface of the node in the
public subnet which I aliased to the name I want in the public subnet :
maestro-300-prov-eth2.maestro.pasteur.fr aliased to
maestro-300.maestro.pasteur.fr
- makehosts + makedns maestro-300-prov *(see note at the end of this
message)
- PXE the compute node on the provisioning interface
- let the node be switch-based discovered
- let configurenetwork (-s) setup the above
I end up with things looking quite close to what I'd want :
- /etc/hosts on Management Node (same in DNS)
192.168.10.10 maestro-300-prov maestro-300-prov.maestroprov.pasteur.fr
192.168.153.48 maestro-300-prov-eth2 maestro-300.maestro.pasteur.fr
- nics on compute node :
eth0: 192.168.10.10/24
eth2: 192.168.153.48/20
[I have to play with nicextraparameters to have the correct resolv.conf]
**(see note at the end)
But now I'm facing the following problem :
1) my netboot image has an fstab configured to **nfs mount** remote
filesystems **ON THE PUBLIC NETWORK** and eth2 is configured way too
late in the systemd boot scheme.
So the node comes up without nfs mounts (mount -a afterwards works though)
How can I deal with this ? add a depenency to the xcat systemd unit
which run the postscsripts ?
2) suppose I want to forbid further traffic through eth0 once eth2 is up
: how would I do that ? (by the way if I do this it exclude one solution
from 1) consisting of mounting nfs filesystems through provisioning
subnet, which I can't anyway as it's not routed)
Notes :
* I find it strange for makehosts to be mandatory as one can expect to
handle everything in dns only but if makeshosts is not run prior makdns,
I end up with "Unable to find an IP for maestro-300-prov-eth2 in hosts
table or via system lookup (i.e. /etc/hosts)"
** : as for resolv.conf I end up with
search <public domain> <provisioning domain>
nameserver <public nameserver>
nameserver <public nameserver>
I've got the correct (public) nameservers because I added a
ipv4.dns-priority (IPV4_DNS_PRIORITY in ifcfg- format) nic
extraparameter to the ifcfg-xcat-eth2 profile but no matter what I do
(except restarting NetworkManger) I still got the merge of
maestroprov.pasteur.fr domain in 'search' which I guess is a merge from
the initial dhcp auto profile of eth0...
Can I do something about this ?
Finally I see that confignetwork -s just creates a NetworkMange profile
with higher autoconnect priority than the dhcp one but the latter is
still here, which is not I think what we'd want when "statically
configure the nic" or am I missing something ?
Here are the actual relevant values for what I just explained :
- site :
#key,value,comments,disable
"domain","maestroprov.pasteur.fr",,
"master","192.168.10.2",,
"nameservers","192.168.149.101,192.168.149.102",,
"dnsinterfaces","eth0",,
"dhcpinterfaces","eth0:noboot,eth1",,
"externaldns","1",,
- hosts :
#node,ip,hostnames,otherinterfaces,comments,disable
"maestro-300-prov","192.168.10.10",,,,
- noderes :
#node,servicenode,netboot,tftpserver,tftpdir,nfsserver,monserver,nfsdir,installnic,primarynic,discoverynics,cmdinterface,xcatmaster,current_osimage,next_osimage,nimserver,routenames,nameservers,proxydhcp,syslog,comments,disable
"maestro-300-prov",,"xnba","maestro-xcat.maestroprov.pasteur.fr",,"maestro-xcat.maestroprov.pasteur.fr",,,,,,,,,,,,"192.168.10.4",,,,
- nics :
#node,nicips,nichostnamesuffixes,nichostnameprefixes,nictypes,niccustomscripts,nicnetworks,nicaliases,nicextraparams,nicdevices,nicsadapter,comments,disable
"maestro-300-prov","eth2!192.168.153.48","eth2!-eth2",,"eth2!Ethernet",,"eth2!maestro_net","eth2!maestro-300.maestro.pasteur.fr","eth2!GATEWAY=192.168.144.1
DNS1=192.168.149.101 DNS2=192.168.149.102 DOMAIN=maestro.pasteur.fr
IPV4_DNS_PRIORITY=-1 ",,,,
- chain :
#node,currstate,currchain,chain,ondiscover,comments,disable
"maestro-300-prov","netboot
rhels8.0-x86_64-cpu","osimage=netboot-cpu-centos8.0","runcmd=bmcsetup,runimage=http://192.168.10.2/install/runimage/runimage.tgz,osimage=netboot-cpu-centos8.0";,,,
- command I run :
nodeadd maestro-300-prov
groups=compute,v5,standard,compute_ipmi,ipmi,slurm,maestro
switch.switch=<switch> switch.port=<port>
makehosts maestro-300-prov
makedns maestro-300-prov
- /etc/hosts on Management node :
192.168.10.10 maestro-300-prov maestro-300-prov.maestroprov.pasteur.fr
192.168.153.48 maestro-300-prov-eth2 maestro-300.maestro.pasteur.fr
- same in DNS
a) maestro.pasteur.fr zone :
maestro-300.maestro.pasteur.fr. 86400 IN CNAME
maestro-300-prov-eth2.maestro.pasteur.fr.
maestro-300-prov-eth2.maestro.pasteur.fr. 86400 IN A 192.168.153.48
b) maestroprov.pasteur.fr zone :
maestro-300-prov.maestroprov.pasteur.fr. 86400 IN A 192.168.10.10
Thanks for your help.
--
Thomas HUMMEL
_______________________________________________
xCAT-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/xcat-user
