So technically I haven't used xCAT for this either (I've been using confluent),
and I've not actually configured satellite myself (using the corporate ones),
but all I did was:
-Add the katello rpm provided by the satellite administrators to the
installation
-Run subscription-manager-register --org=.... --activationkey=base
-yum update from there
I know this was probably what was done first in this thread but wanted to
highlight for those not seeking the absolute optimization.
I can take it as an improvement in confluent to explictly have a suggested
procedure to accomplish this before any package is installed. Note that
confluent already supports full TLS for internal servers, and you can add 'CAs
to trust from the beginning' by putting their pem files in
/var/lib/confluent/public/site/tls/.
As to why do this instead of foreman it's ultimately a matter of preference.
Note that if you want foreman but want xCAT-style discovery, then confluent can
gather macs for foreman instead of internal if desired without interfering with
the DHCP infrastructure. If you set net.bootable=1, and pxe boot, discovery is
done without sending any dhcpoffers at all, and then you can:
noderun n1-n100 hammer host create --name {node} .... --mac {net.hwaddr} ....
For its own deployment it can use mac addresses, but it favors UUIDs that it
gathers, which is particularly nice for pxe booting over what will be a bond
from either mac. I think while xCAT can be difficult, confluent is easiest of
them all at managing the host binding (no mare 'makedhcp', it supports either
directly answering PXE discover packets, or not caring and letting a normal
dhcp server handle the ip while it handles the filename and next-server
withtout requiring anything from the dhcp server nor interfering with the dhcp
server).
Of course, this is an easier suggestion when 3.3 comes out in a couple of
weeks, since that has the 'confignet' to auto-conifgure multiple nics and bonds
as well as cloning and stateless (the confluent stateless is easier, updates
are normal without special stateless steps to apply new kernel or drivers, and
doesn't cost so much RAM by default, instead using a zram for writing and a
multipath https mount for on-demand access to the image). It doesn't support
xCAT OS images, so migration if you have high customization would be daunting,
but if using mostly stock or haven't gotten started yet, I think confluent as
of 3.3 will be an interesting choice.
________________________________
From: Kevin Keane <[email protected]>
Sent: Tuesday, November 2, 2021 11:11 PM
To: xCAT Users Mailing list <[email protected]>
Subject: [External] Re: [xcat-user] RedHat Satellite and xCAT stateful installs
I have made the almost exact same migration a few months ago, and am using the
exact same tools you are using (Satellte + Ansible). Also using RHEL 8.4. We
are using both throughout our data center, so using them introduces a lot of
consistency and manageability.
The only difference is that I'm not using xCAT at all any more. Satellite can
do the same thing as xCAT's stateful installation using PXE booting. Instead of
post scripts etc., I'm using Ansible. Note that PXE booting will work even if
your data center only uses static IP addresses, as ours does!
Most of what I write below applies regardless of whether you use Satellite
Server or xCAT to provision your system
I'm assuming here that your architecture is similar to ours, with the
management node also acting as a NAT gateway to the systems behind it.
I found that managing DHCP and DNS with Ansible is a lot easier than with xCAT.
Since usually, the management node is also a jump host, there are a few
considerations:
- Regardless of if you use xCAT or not, use the management node as an SSH jump
host from the satellite server, as well as from your Ansible controller.
- TFTP is not NAT friendly at all. Set up a TFTP helper on your NAT gateway
(i.e., management node)
To make the management node an SSH jump host for Satellite, put this stanza
into /usr/share/foreman-proxy/.ssh/config (assuming that your management node
is
mn.example.com<https://apc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmn.example.com%2F&data=04%7C01%7Cjjohnson2%40lenovo.com%7Ce342b154db9b4da8722108d99e7b7dce%7C5c7d0b28bdf8410caa934df372b16203%7C1%7C0%7C637715076553547704%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=BOhlQdw6%2Fr6OSk1tbBuyx%2F3i0nqzy39D40%2BxHMSbR80%3D&reserved=0>,
your nodes are all in a zone
mgmt.example.com<https://apc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmgmt.example.com%2F&data=04%7C01%7Cjjohnson2%40lenovo.com%7Ce342b154db9b4da8722108d99e7b7dce%7C5c7d0b28bdf8410caa934df372b16203%7C1%7C0%7C637715076553547704%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=e5k5SU2oTHyrF3TiStXx5qJVCNJuKicNbhgY1pkjG0g%3D&reserved=0>,
and you configured Satellite to use the svc-satellite user for Remote
Execution. Of course, pay attention to ownership and permissions (this file
needs to be owned by foreman-proxy:foreman-proxy, and should have 600
permissions).
---------------------------------------------
Host
mn.example.com<https://apc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmn.example.com%2F&data=04%7C01%7Cjjohnson2%40lenovo.com%7Ce342b154db9b4da8722108d99e7b7dce%7C5c7d0b28bdf8410caa934df372b16203%7C1%7C0%7C637715076553557705%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=KqZUqxK%2FdHX4DO0y8o5cL0%2BSnX8Spwr1IJPXzCxQh9o%3D&reserved=0>
User svc-satellite
Host
*.mgmt.example.com<https://apc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmgmt.example.com%2F&data=04%7C01%7Cjjohnson2%40lenovo.com%7Ce342b154db9b4da8722108d99e7b7dce%7C5c7d0b28bdf8410caa934df372b16203%7C1%7C0%7C637715076553567692%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=6f1Ta4Hw06Hn59Txfu40YrU3JwXqMwvyBHYu1e23%2Fds%3D&reserved=0>
ProxyJump [email protected]<mailto:[email protected]>
---------------------------------------------
Use a similar stanza on your Ansible controller, using whatever user name you
use for Ansible.
Overall, the process is:
Make sure that everything in Satellite Server is up to snuff. It is easy to get
it to "kinda" work, but there are a lot of T's to cross and I's to dot. That is
really more of a general gotcha with Satellite Server. In particular, make sure
to set up ALL the subnets involved, and use host groups to assign the
configurations for your nodes. Also set up locations, organizations, etc.
In the subnets, be sure to configure it for DHCP booting, and also make sure to
set the DNS name and IP address range correctly. Obviously, you will want to
specify your management node as DNS server. This is not plainly informational
stuff, but is actually used during PXE booting. A single typo here can cause a
lot of headaches (I remember having specified a default gateway with the .0
instead of the .1 address. PXE booting failed with a cryptic indecipherable
message, and then I spent three days with RedHat support before spotting that
error).
In host groups, configure all the stuff you would ordinarily configure at the
host level. Satellite Server will pick up those values when it auto-creates
your nodes during the PXE-Boot process. In particular, make sure to provide
whatever Satellite activation keys you need.
RedHat says that for nodes behind a NAT gateway, you need a separate capsule
server. I found that to be overkill and unnecessary. The TFTP helper and the
SSH jump host configuration do the job instead. THIS IS UNDOCUMENTED and won't
be supported by RedHat support.
- In Ansible, provide the MAC addresses, IP addresses and DNS names you want
for the nodes as a host variable to the management node.
- Set up the Management Node with the following:
DHCP. Generate /etc/dhcp/dhcpd.conf from the Ansible host variables.
DNS. Generate /etc/named.conf based on whatever zones you need, and, using
the Ansible host variables, generate /var/named/chroot/var/named/*.zone files
as appropriate.
TFTP helper in firewalld. Make sure you add the service "tftp" to the zone
facing your nodes, and the service "tftp-client" to the zone facing outward. I
had to do a bit of troubleshooting of this part using tcpdump.
SSH configuration. Make sure to allow agent forwarding. Also, make sure that
ProxyJump is allowed (it should be by default, but there are some ways you
could undermine it).
If you need it, I'll be happy to provide (via private email) templates we use
for the DHCP and DNS configuration.
Now boot your nodes, using PXE booting. If everything is 100% correct, the node
should pull the installation from Satellite Server in a completely unattended
install. Satellite registration happens automatically during this step.
After provisioning the nodes, bootstrap them into Ansible (you can also do that
with Kickstart files during the initial provisioning step).
Run your Ansible playbooks to finish the installation and configuration of
whatever you need (such as your scheduler, application software,
authentication, users etc.)
One more thing: do not use VLANs. There is a bug in Satellite Server
provisioning with VLANs that in some scenarios prevents it from working. You
can use VLANs after provisioning. Just not during. RedHat knows about the bug,
but hasn't fixed it yet.
Regards,
Kevin
_______________________________________________________________________
Kevin Keane | Systems Architect | University of San Diego ITS |
[email protected]<mailto:[email protected]>
Pronouns: he/him/his
Maher Hall, 162 |5998 Alcalá Park | San Diego, CA 92110-2492 | 619.260.6859 |
Text: 760-721-8339
REMEMBER! No one from IT at USD will ever ask to confirm or supply your
password.
These messages are an attempt to steal your username and password. Please do
not reply to, click the links within, or open the attachments of these
messages. Delete them!
On Tue, Nov 2, 2021 at 11:16 AM THomas HUMMEL
<[email protected]<mailto:[email protected]>> wrote:
Hello,
I'm in the process of migrating xCAT 2.16 CentOS 8.3 (x86_64) stateless
and stateful installs to RHEL 8.4 (x86_64) stateless and stateful
installs using RedHat Satellite 6.9 because of the CentOS new
positionning in RH lifecyle
RH Satellite will be used basically to provide repos to hosts, I'm not,
for now, planning to use any other features
This new paradigm implies two major changes:
# hosts have to be registered to access repos
As a matter of fact it is the registration process which configures
client SSL certs and the redhat.repo file
-> this one is not a problem as register is in my case performed by
ansible after node initial install and I'm using Simple Content Access
Note:
I managed though, as a proof of concept, to make it possible to register
either in %pre or %post kickstart section.
# xCAT stateful installs (kickstart)
I still want xCAT to manage DHCP,PXE,DNS...so stateful install as well
-> does anyone has any hint/experience about this one when using a RH
Satellite ?
As a matter of fact I did encounter issues with the points described below.
Note:
The standard method (point pkgdir to the content of an RHEL DVD iso) and
register later still works but I thought it would be nice to better
integrate with Satellite from the start, meaning using repos from inside
Satellite to perform the install, not installing with DVD repos then
register and point/use other repos (which are Satellite configured
content views). Or at least install from some repos inside Satellite,
register then point/use Content view repos
1) xCAT (pkgdir=) is meant to deal with ISO content accessible via some
fs mountpoint, my understanding is that it cannot point to some http URL
and that xCAT looks for kernel/initrd to be served by PXE in that same
location. Besides, xCAT sets up inst.repo= to this location as well
-> this could be manually worked around by commenting out %include
/tmp/repos and stating an url --url directive instead and by overwriting
inst.repo= in the iPXE xnba script file (/tftp/xcat/xnba/nodes/<node>)
This is not practical though as nodeset would override the latter
2) Satellite RH BaseOS and Appstream repos, stored in a "Content View"
require registration to be accessed
-> this could be worked around in 2 ways:
a) by registering in %pre ks section
b) by using (trying to use) what Satellite calls "kickstart repos"
So my first idea was to try to use RH BOOT.iso bootnet only (no rpms)
image with xCAT which would be pointed BaseOS/AppStream repos in a
Satellite Content View while registering in %pre
-> this failed because the inst.repo/url --url combination basically
made it mandatory to use DVD.iso in order for Anaconda either to start
or to have a non-interactive source install set up
My second idea was to try to use Satellite kickstart repos, which are
accessible by HTTP without SSL client certs, but again:
a) xCAT still have to have kernel/initrd to serve via iPXE: which one
would fit ?
b) those repos are just repos and thus does not match the layout of the
root of the DVD for instance (they're more like the inidivual BaseOS and
AppStream repos once you're cd inside them). Anaconda still have to be
found somewhere.
Besides, my understanding is that in order to natively (the Satellite
way) kickstart against kickstart repos, one have to somehow create a
host and use the Satellite generated boot media (using in particular
Satellite generated kernel/initrd)
My guess is that it would be feasible to generate such a boot media iso
inside Satellite, download it and point xCAT to its content with
pkgdir=, just as I did with DVD.iso
In short: what would be the best way to use xCAT and Satellite for a
stateful install ?
Thanks for you help
--
Thomas HUMMEL
_______________________________________________
xCAT-user mailing list
[email protected]<mailto:[email protected]>
https://lists.sourceforge.net/lists/listinfo/xcat-user<https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fxcat-user&data=04%7C01%7Cjjohnson2%40lenovo.com%7Ce342b154db9b4da8722108d99e7b7dce%7C5c7d0b28bdf8410caa934df372b16203%7C1%7C0%7C637715076553577687%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=gp8LZxffWTVLIE%2BVrPFEDXWDsPkhsqxD8Rhk5JbEctY%3D&reserved=0>
_______________________________________________
xCAT-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/xcat-user
