[Puppet Users] Re: How to handle predictable network interface names
On Saturday, 27 August 2016 18:51:09 UTC+1, Marc Haber wrote:
>
> On Fri, Aug 26, 2016 at 08:40:49AM -0700, Luke Bigum wrote:
> > My Dell XPS 13, 2016 model:
> >
> > /sys/class/net/docker0
> > /sys/class/net/enp0s20u1u3i5
> > E: ID_NET_NAME_MAC=enx9cebe824ebee
> > E: ID_NET_NAME_PATH=enp0s20u1u3i5
>
> What a name!
>
http://accessories.euro.dell.com/sna/productdetail.aspx?c=uk&l=en&s=bsd&cs=ukbsdt1&sku=452-bboo
> > For both the Dell R720 and R730, there's no NET_NAME stuff:
> >
> > [root@r720 ~]# udevadm info -q all -p /sys/class/net/p4p2
> > P: /devices/pci:40/:40:02.0/:42:00.1/net/p4p2
> > E: UDEV_LOG=3
> > E: DEVPATH=/devices/pci:40/:40:02.0/:42:00.1/net/p4p2
> > E: INTERFACE=p4p2
> > E: IFINDEX=7
> > E: SUBSYSTEM=net
>
> Maybe OS too old? The interface name "p4p2" also looks fishy.
>
Nope, CentOS 6 (which I guess is pretty old now).
> > Yes, we definitely don't define resources, and don't include component /
> > base level classes. I think we pulled it from an early Gary Larizza
> post,
> > along with "roles don't have parameters, if you need to configure a role
> > you've got two different roles".
>
> Yes, but dropping a supplementary file does not mean that a role has
> parameters. And also, it would be duplication if one had two distinct
> roles that would only differ in single setting?
>
Mmmm that's one of the two big questions in all our Puppet design
discussions here. Do I introduce some sort of boolean switch to drop a new
file,
or do I create a new role / profile. (The other big question is "Should
this value be hard coded in a profile or as a parameter in Hiera?")
If it is just one file difference, then I would be persuaded to allow a
boolean switch in the profile. If the file did functionality differences
inside a given location/environment, or or was necessary in this location
but not others, I'd be ok with the boolean being set in Hiera for those
locations/environments.
If on the other hand it the role needed to be used in two different ways
inside an environment (maybe the file was the difference between Master and
Slave) then I would
advocate two roles. In my mind that's quite clear that there are two
different "jobs to do" here, rather than something being done slightly
differently in a different location:
class role::master {
class { 'profile::something': master => true }
}
class role::slave {
class { 'profile::something': master => false }
}
The problem I've found with such boolean switches is once someone sees one
of them as a potential solution, they tend to explode into a mass of if
statements and
make things really complicated to read. I think it's because it's easy - I
can just solve my problem right now with an if statement here, rather than
spend a few hours
thinking about and refactoring all the related classes.
Getting a good balance between duplication (and testing) versus purist
design is difficult, and I don't think there will ever be a right answer.
-Luke
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/3ca61fb4-389d-4738-be23-d63b7a4b3d0d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: How to handle predictable network interface names
I went the other direction-- we deploy all of our servers (and most of our workstations) via cobbler, so all new RHEL 7 and CentOS 7 boxes have "net.ifnames=0" in the default profile, both pre and post install. I don't agree with upstream's complaint... udev-persistent rules, while a bit of a hack, work just fine, especially when the vast majority of workstations and servers I support have exactly one ethernet device and no wireless. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/6d34beb1-33cb-495f-bea4-91a56db0e564%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: How to handle predictable network interface names
On Fri, Aug 26, 2016 at 08:40:49AM -0700, Luke Bigum wrote: > My Dell XPS 13, 2016 model: > > /sys/class/net/docker0 > /sys/class/net/enp0s20u1u3i5 > E: ID_NET_NAME_MAC=enx9cebe824ebee > E: ID_NET_NAME_PATH=enp0s20u1u3i5 What a name! > For both the Dell R720 and R730, there's no NET_NAME stuff: > > [root@r720 ~]# udevadm info -q all -p /sys/class/net/p4p2 > P: /devices/pci:40/:40:02.0/:42:00.1/net/p4p2 > E: UDEV_LOG=3 > E: DEVPATH=/devices/pci:40/:40:02.0/:42:00.1/net/p4p2 > E: INTERFACE=p4p2 > E: IFINDEX=7 > E: SUBSYSTEM=net Maybe OS too old? The interface name "p4p2" also looks fishy. > > What I get from the abstraction above is being able to take our > > > profiles and re-use them in a completely different site on the other > > > side of the world, or in a staging / testing environment. So I don't > > > have the concept of "VLAN 123 in Production UK", I've just got "The > > > STORAGE network" which in Production UK happens to be vlan 123 > > > (buried low down in Hiera, and only specified once once), but in Dev > > > it's 456, and over there it doesn't exist so we'll give it the same > > > vlan tag as the CLIENT network, etc... The physical-ness of the > > > network is abstracted from the concepts our software relies on. > > > > Yes, that is a really nice concept with should have been considered > > here years ago. Alas, people didn't. > > To be fair we didn't design it this way from the start, it's only in the > last couple evolutions that abstraction appeared. Yes. I have never seen an installation doing it _this_ right in the first iteration. > > > > So you do create network interfaces in the profile and not in the > > > > role? > > > > > > > > > > We try to follow the design rule that "Roles only include Profiles". > > > > ... "and don't define their own resources", you mean? > > > > That's one of the aspects of the role-and-profiles approach that I > > have never seen spelled out explicitly, but still honored by nearly > > anybody, and I have not yet fully grokked the reasons for doing so. > > > > Yes, we definitely don't define resources, and don't include component / > base level classes. I think we pulled it from an early Gary Larizza post, > along with "roles don't have parameters, if you need to configure a role > you've got two different roles". Yes, but dropping a supplementary file does not mean that a role has parameters. And also, it would be duplication if one had two distinct roles that would only differ in single setting? > I've always understood the logic to be based on the layers of abstraction > the role/profile design is trying to achieve. I always envision this tree: > > Node (has one) -> > Role (has one or more) -> > Profile (has one or more) -> > Defined Type (has zero or more) -> > Resources > or Class (has zero or more) -> > Resources > or Resource That's the way it's in the books. What I'd like to see clarified (and explained why) is that this tree should have "and nothing else" spelled out in multiple places: Node (has one) -> Role (and nothing else!) Role (has one or more) -> Profile (and nothing else!) Profile (has one or more) -> Defined Type (has zero or more) -> Resources or Class (has zero or more) -> Resources or Resource Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421 -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/20160827175059.GB2471%40torres.zugschlus.de. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: How to handle predictable network interface names
On Friday, 26 August 2016 10:57:25 UTC+1, Marc Haber wrote: > > On Thu, Aug 25, 2016 at 08:08:13AM -0700, Luke Bigum wrote: > > On Thursday, 25 August 2016 13:21:24 UTC+1, Marc Haber wrote: > > > On Wed, Aug 24, 2016 at 08:36:49AM -0700, Luke Bigum wrote: > > > > Here we have very strict control over our hardware and what > interface > > > goes > > > > where. We keep CentOS 6's naming scheme on Dell hardware, so p2p1 is > PCI > > > > slot 2, Port 1, and don't try rename it. > > > > > > Isn't CentOS 6 still using eth0, 1, 2, 3? How do you handle different > > > hardware having different slot numbers, or PCI bridges shifting bus > > > numbers? > > > > > > > I find this depends on the manufacturer. I've never come across a Dell > > server newer than an R510 that *doesn't* give you PCI based names. I > just > > checked an R510 and it does. All of our ancient HP gear (7 years, older > > than the R510s which is old) give the ethX names. Also random SuperMicro > > hardware gives ethX. I don't really know what's missing for the kernel / > > udev to name them so, but for us it doesn't really matter. > > Can you run > $ for iface in /sys/class/net/*; do echo $iface; sudo udevadm info -q all > -p $iface | grep ID_NET_NAME; done > on some of your gear? I'd like to learn what different vendors deliver. > > My Dell XPS 13, 2016 model: /sys/class/net/docker0 /sys/class/net/enp0s20u1u3i5 E: ID_NET_NAME_MAC=enx9cebe824ebee E: ID_NET_NAME_PATH=enp0s20u1u3i5 /sys/class/net/lo /sys/class/net/virbr0 /sys/class/net/virbr0-nic /sys/class/net/virbr1 /sys/class/net/virbr1-nic /sys/class/net/virbr2 /sys/class/net/virbr2-nic /sys/class/net/wlp2s0 E: ID_NET_NAME=wlp2s0 E: ID_NET_NAME_MAC=wlxacd1b8c05607 E: ID_NET_NAME_PATH=wlp2s0 For both the Dell R720 and R730, there's no NET_NAME stuff: [root@r720 ~]# udevadm info -q all -p /sys/class/net/p4p2 P: /devices/pci:40/:40:02.0/:42:00.1/net/p4p2 E: UDEV_LOG=3 E: DEVPATH=/devices/pci:40/:40:02.0/:42:00.1/net/p4p2 E: INTERFACE=p4p2 E: IFINDEX=7 E: SUBSYSTEM=net And this is an FC430, which is a blade-like chassis with internal PCI switches: [root@FC430 ~]# udevadm info -q all -p /sys/class/net/p5p1/ P: /devices/pci:00/:00:03.0/:02:00.0/:03:01.0/:04:00.0/:05:0c.0/:08:00.0/net/p5p1 E: UDEV_LOG=3 E: DEVPATH=/devices/pci:00/:00:03.0/:02:00.0/:03:01.0/:04:00.0/:05:0c.0/:08:00.0/net/p5p1 E: INTERFACE=p5p1 E: IFINDEX=6 E: SUBSYSTEM=net > What I get from the abstraction above is being able to take our > > profiles and re-use them in a completely different site on the other > > side of the world, or in a staging / testing environment. So I don't > > have the concept of "VLAN 123 in Production UK", I've just got "The > > STORAGE network" which in Production UK happens to be vlan 123 > > (buried low down in Hiera, and only specified once once), but in Dev > > it's 456, and over there it doesn't exist so we'll give it the same > > vlan tag as the CLIENT network, etc... The physical-ness of the > > network is abstracted from the concepts our software relies on. > > Yes, that is a really nice concept with should have been considered > here years ago. Alas, people didn't. > To be fair we didn't design it this way from the start, it's only in the last couple evolutions that abstraction appeared. What we did have from the start though was the concept that the same network segment in different environments would have the same IP address segments, so the DATABASE network over here is 1.15.7.0, and over there it's 1.40.7.0. The third octet for the same network segment at different sites is the same (and hopefully the same VLAN tag on switches, but not mandatory). It's easy to abstract the numbers into names from there. However there's no reason why we couldn't use the same abstraction idea for vastly different or public IP ranges, it would just require more Hiera glue. > > > So you do create network interfaces in the profile and not in the > > > role? > > > > > > > We try to follow the design rule that "Roles only include Profiles". > > ... "and don't define their own resources", you mean? > > That's one of the aspects of the role-and-profiles approach that I > have never seen spelled out explicitly, but still honored by nearly > anybody, and I have not yet fully grokked the reasons for doing so. > Yes, we definitely don't define resources, and don't include component / base level classes. I think we pulled it from an early Gary Larizza post, along with "roles don't have parameters, if you need to configure a role you've got two different roles". All rules subject to be broken if someone is in a hurry/lazy (which is too often). I've always understood the logic to be based on the layers of abstraction the role/profile design is trying to achieve. I always envision this tree: Node (has one) -> Role (has one or more) -> Profile (has one or
[Puppet Users] Re: How to handle predictable network interface names
On Thu, Aug 25, 2016 at 08:08:13AM -0700, Luke Bigum wrote: > On Thursday, 25 August 2016 13:21:24 UTC+1, Marc Haber wrote: > > On Wed, Aug 24, 2016 at 08:36:49AM -0700, Luke Bigum wrote: > > > Here we have very strict control over our hardware and what interface > > goes > > > where. We keep CentOS 6's naming scheme on Dell hardware, so p2p1 is PCI > > > slot 2, Port 1, and don't try rename it. > > > > Isn't CentOS 6 still using eth0, 1, 2, 3? How do you handle different > > hardware having different slot numbers, or PCI bridges shifting bus > > numbers? > > > > I find this depends on the manufacturer. I've never come across a Dell > server newer than an R510 that *doesn't* give you PCI based names. I just > checked an R510 and it does. All of our ancient HP gear (7 years, older > than the R510s which is old) give the ethX names. Also random SuperMicro > hardware gives ethX. I don't really know what's missing for the kernel / > udev to name them so, but for us it doesn't really matter. Can you run $ for iface in /sys/class/net/*; do echo $iface; sudo udevadm info -q all -p $iface | grep ID_NET_NAME; done on some of your gear? I'd like to learn what different vendors deliver. My Thinkpad T450: /sys/class/net/br0 /sys/class/net/enx11121314 E: ID_NET_NAME=enp0s20u4 E: ID_NET_NAME_MAC=enx11121314 E: ID_NET_NAME_PATH=enp0s20u4 /sys/class/net/eth0 E: ID_NET_NAME=enp0s25 E: ID_NET_NAME_MAC=enx507b9d681b36 E: ID_NET_NAME_PATH=enp0s25 /sys/class/net/lo My APU at home (with a lot of bridges and VLANs): [2/501]mh@aida:~$ for iface in /sys/class/net/*; do echo $iface; sudo udevadm info -q all -p $iface | grep ID_NET_NAME; done /sys/class/net/br181 /sys/class/net/br182 /sys/class/net/br183 /sys/class/net/br184 /sys/class/net/br185 /sys/class/net/br186 /sys/class/net/br187 /sys/class/net/br188 /sys/class/net/br189 /sys/class/net/br191 /sys/class/net/br192 /sys/class/net/br281 /sys/class/net/br381 /sys/class/net/br382 /sys/class/net/br383 /sys/class/net/brenp1s0 /sys/class/net/brenp2s0 /sys/class/net/brenp3s0 /sys/class/net/enp1s0 E: ID_NET_NAME_MAC=enx000db9342afc E: ID_NET_NAME_PATH=enp1s0 /sys/class/net/enp2s0 E: ID_NET_NAME_MAC=enx000db9342afd E: ID_NET_NAME_PATH=enp2s0 /sys/class/net/enp3s0 E: ID_NET_NAME_MAC=enx000db9342afe E: ID_NET_NAME_PATH=enp3s0 /sys/class/net/int181 /sys/class/net/int182 /sys/class/net/int191 /sys/class/net/int192 /sys/class/net/lo /sys/class/net/per281 /sys/class/net/unt381 /sys/class/net/unt382 /sys/class/net/unt383 [3/502]mh@aida:~$ A KVM VM with a lot of interfaces: /sys/class/net/eth0 E: ID_NET_NAME=enx525400422c88 E: ID_NET_NAME_MAC=enx525400422c88 /sys/class/net/eth1 E: ID_NET_NAME=enx525400d22ad3 E: ID_NET_NAME_MAC=enx525400d22ad3 /sys/class/net/eth2 E: ID_NET_NAME=enx52540095dfa6 E: ID_NET_NAME_MAC=enx52540095dfa6 /sys/class/net/int181 /sys/class/net/int182 /sys/class/net/int183 /sys/class/net/int184 /sys/class/net/int185 /sys/class/net/int186 /sys/class/net/int187 /sys/class/net/int188 /sys/class/net/int189 /sys/class/net/int191 /sys/class/net/int192 /sys/class/net/lo /sys/class/net/per281 /sys/class/net/unt381 /sys/class/net/unt382 /sys/class/net/unt383 > > We have a 3rd party patch manager tool (patchmanager.com), LLDP on > > > our switches, and a Nagios check that tells me if an interface is not > > > plugged into the switch port it is supposed to be plugged into > > > (according to patchmanager). > > > > Nice ;-) Is the code for the Nagios stuff public? > > > > Unfortunately no Too bad ;-) > :-( Another one of those LMAX modules that's had years of > development but too much company specific stuff hard coded in it to > release. It's not a huge amount though, and I did just ask my Lead if > I could clean up our networking module and release it and he was more > than happy, I'm sure I could do the same for our nagios module. Watch > this space, but don't hold your breath. That would be really really nice. > > This works perfectly on Dell hardware because the PCI name mapping > > > works. > > > > And you don't have many different kinds of servers. > > We try keep as few as possible, but it's not that small a list: > > *** > [root@puppet ~]# mco facts productname > Report for fact: productname > > .found 1 times > KVM found 603 times > OptiPlex 7010found 1 times > OptiPlex 7020found 2 times > PowerEdge FC430 found 15 times > PowerEdge FC630 found 56 times > PowerEdge R220 found 1 times > PowerEdge R320 found 92 times > PowerEdge R330 found 1 times > PowerEdge R510 found 17 times > PowerEdge R520 found 66 times >
[Puppet Users] Re: How to handle predictable network interface names
On Thursday, 25 August 2016 13:21:24 UTC+1, Marc Haber wrote: > > On Wed, Aug 24, 2016 at 08:36:49AM -0700, Luke Bigum wrote: > > Here we have very strict control over our hardware and what interface > goes > > where. We keep CentOS 6's naming scheme on Dell hardware, so p2p1 is PCI > > slot 2, Port 1, and don't try rename it. > > Isn't CentOS 6 still using eth0, 1, 2, 3? How do you handle different > hardware having different slot numbers, or PCI bridges shifting bus > numbers? > I find this depends on the manufacturer. I've never come across a Dell server newer than an R510 that *doesn't* give you PCI based names. I just checked an R510 and it does. All of our ancient HP gear (7 years, older than the R510s which is old) give the ethX names. Also random SuperMicro hardware gives ethX. I don't really know what's missing for the kernel / udev to name them so, but for us it doesn't really matter. > We have a 3rd party patch manager tool (patchmanager.com), LLDP on > > our switches, and a Nagios check that tells me if an interface is not > > plugged into the switch port it is supposed to be plugged into > > (according to patchmanager). > > Nice ;-) Is the code for the Nagios stuff public? > Unfortunately no :-( Another one of those LMAX modules that's had years of development but too much company specific stuff hard coded in it to release. It's not a huge amount though, and I did just ask my Lead if I could clean up our networking module and release it and he was more than happy, I'm sure I could do the same for our nagios module. Watch this space, but don't hold your breath. > This works perfectly on Dell hardware because the PCI name mapping > > works. > > And you don't have many different kinds of servers. We try keep as few as possible, but it's not that small a list: *** [root@puppet ~]# mco facts productname Report for fact: productname .found 1 times KVM found 603 times OptiPlex 7010found 1 times OptiPlex 7020found 2 times PowerEdge FC430 found 15 times PowerEdge FC630 found 56 times PowerEdge R220 found 1 times PowerEdge R320 found 92 times PowerEdge R330 found 1 times PowerEdge R510 found 17 times PowerEdge R520 found 66 times PowerEdge R720 found 36 times PowerEdge R720xd found 30 times PowerEdge R730 found 7 times PowerEdge R730xd found 37 times Precision Tower 5810 found 10 times Precision WorkStation T5500 found 7 times ProLiant DL360 G6found 2 times ProLiant DL380 G5found 16 times ProLiant DL380 G6found 11 times To Be Filled By O.E.M. found 1 times X9SCL/X9SCM found 6 times * > > On really old HP gear it doesn't work, > > What does that mean? > > I meant that on our very old HP servers the PCI device name mapping doesn't come up, so you end up with eth0, eth1, etc. > > We still need some sort of "glue record" that says "this interface > should > > be up and have this IP". In our older designs this was managed entirely > in > > Hiera - so there's a giant multi-level hash that we run > create_resources() > > over to define every single network interface. You can imagine the > amount > > of Hiera data we have. > > That's what we're trying to avoid. Can you share example snippets? > Here is a snippet of the older style, in a Node's Hiera. It is what I'm trying to move away from, because if you want to create 20 of these machines you've got to copy this Hiera hash around 20 times over. Oh the number of typos... You can probably interpolate the defined types that this data has create_resources() run over, the key names are pretty Red Hat specific: *** networking::interfaces: bond1: bonding_opts: mode=802.3ad xmit_hash_policy=layer3+4 lacp_rate=slow miimon=100 enable: true onboot: 'yes' type: Bonding bond1.3: broadcast: 1.1.3.255 enable: true ipaddr: 1.1.3.7 netmask: 255.255.255.0 network: 1.1.3.0 onboot: 'yes' vlan: 'yes' p4p1: enable: true master: bond1 onboot: 'yes' slave: 'yes' type: Ethernet p4p2: enable: true master: bond1 onboot: 'yes' slave: 'yes' type: Ethernet networking::routes: bond1: device: bond1 routes: - 1.1.2.0/24
[Puppet Users] Re: How to handle predictable network interface names
On Wed, Aug 24, 2016 at 08:36:49AM -0700, Luke Bigum wrote:
> Here we have very strict control over our hardware and what interface goes
> where. We keep CentOS 6's naming scheme on Dell hardware, so p2p1 is PCI
> slot 2, Port 1, and don't try rename it.
Isn't CentOS 6 still using eth0, 1, 2, 3? How do you handle different
hardware having different slot numbers, or PCI bridges shifting bus
numbers?
> We have a 3rd party patch manager tool (patchmanager.com), LLDP on
> our switches, and a Nagios check that tells me if an interface is not
> plugged into the switch port it is supposed to be plugged into
> (according to patchmanager).
Nice ;-) Is the code for the Nagios stuff public?
> This works perfectly on Dell hardware because the PCI name mapping
> works.
And you don't have many different kinds of servers.
> On really old HP gear it doesn't work,
What does that mean?
> We still need some sort of "glue record" that says "this interface should
> be up and have this IP". In our older designs this was managed entirely in
> Hiera - so there's a giant multi-level hash that we run create_resources()
> over to define every single network interface. You can imagine the amount
> of Hiera data we have.
That's what we're trying to avoid. Can you share example snippets?
> In the newer designs which are a lot more of a role/profile approach
> I've been trying to conceptualise the networking based on our
> profiles. So if one of our servers is fulfilling function "database"
> there will be a Class[profile::database]. This Class might create a
> bonded interface for the "STORAGE" network and another interface for
> the "CLIENT" network.
That is interesting and a nice concept. But nothing one introduces
just to remedy an error report named "help, my interface names do not
fit any more".
So you do create network interfaces in the profile and not in the
role?
> Through various levels of Hiera I can define the STORAGE network as
> VLAN 100, because it might be a different vlan tag at a different
> location. Then at the Hiera node level (on each individual server) I
> will have something like:
>
> profile::database::bond_storage_slaves: [ 'p2p1', 'p2p2' ]
>
> That's the glue. At some point I need to tell Puppet that on this specific
> server, the storage network is a bond of p2p1 and p2p2.
So you need to know when writing this code what kind of hardware the
system is running on, probably down to firmware version and hardware
extras?
> I have bounced around the idea of removing this step and trusting the
> switch - ie: write a fact to do an LLDP query for the VLAN of the switch
> port each interface is connected to, that way you wouldn't need the glue,
> there'd be a fact called vlan_100_interfaces.
So the fact would be coming from the _server_ based on what lldpcli
show neighbors detail returns, which is supposed to include the VLAN
information? Would this work on 801.1q trunks as well?
> Two problems with this approach: we end up trusting the switch to be
> our source of truth (it may not be correct,
The switch uses its own source of truth which also influences which
network traffic gets sent down the link, so trusting the switch will
at least fail to the safe side and avoid accidentally putting full
trust on an Internet link.
> and, what if the switch port is down?).
One would have to fall back to a certain safety net then.
> Secondly the quality and consistency of LLDP information you get out
> of various manufacturers of networking hardware is very different, so
> relying on LLDP information to define your OS network config is a bit
> risky for me.
Is it really this bad? I do have experience with HP and Cisco, and
their LLDP/CDP information is usually fine.
> It's a different story for our VMs. Since they are Puppet defined we
> specify a MAC address and so we "know" which MAC will be attached to which
> VM bridge. We drop a MAC based udev rule into the guest to name them
> similarly, ie: eth100 is on br100.
How do you puppet define your MAC addresses? Which virtualization do
you use? Can i see a code snippet?
> That's what we do, but it's made easy by an almost homogeneous hardware
> platform and strict physical patch management.
Yes. The homogenous hardware platform is probably something that can
only be maintained for really large installations.
> When I read about your problem, it sounds like you are missing a "glue
> record" that describes your logical interfaces to your physical devices.
We're desperately trying to avoid having this in Hiera.
> If you were to follow something along the lines of our approach, you
> might have something like this:
>
> class profile::some_firewall(
> $external_interface_name = 'eth0',
> $internal_interface_name = 'eth1',
> $perimiter_interface_name = 'eth2'
> ) {
> firewall { '001_allow_internal':
> chain => 'INPUT',
> iniface => $internal_interface_name,
> action => 'accept',
> proto =>
[Puppet Users] Re: How to handle predictable network interface names
Here we have very strict control over our hardware and what interface goes
where. We keep CentOS 6's naming scheme on Dell hardware, so p2p1 is PCI
slot 2, Port 1, and don't try rename it. We have a 3rd party patch manager
tool (patchmanager.com), LLDP on our switches, and a Nagios check that
tells me if an interface is not plugged into the switch port it is supposed
to be plugged into (according to patchmanager). This works perfectly on
Dell hardware because the PCI name mapping works. On really old HP gear it
doesn't work, so we fall back on always assuming eth0 is the first onboard
port, etc. If the kernel scanned these devices in a different order we'd
get the same breakage you describe, but that's never happened on it's own,
it's only happened if an engineer has gone and added re-arranged cards.
We still need some sort of "glue record" that says "this interface should
be up and have this IP". In our older designs this was managed entirely in
Hiera - so there's a giant multi-level hash that we run create_resources()
over to define every single network interface. You can imagine the amount
of Hiera data we have. In the newer designs which are a lot more of a
role/profile approach I've been trying to conceptualise the networking
based on our profiles. So if one of our servers is fulfilling function
"database" there will be a Class[profile::database]. This Class might
create a bonded interface for the "STORAGE" network and another interface
for the "CLIENT" network. Through various levels of Hiera I can define the
STORAGE network as VLAN 100, because it might be a different vlan tag at a
different location. Then at the Hiera node level (on each individual
server) I will have something like:
profile::database::bond_storage_slaves: [ 'p2p1', 'p2p2' ]
That's the glue. At some point I need to tell Puppet that on this specific
server, the storage network is a bond of p2p1 and p2p2. If I took that
profile to a HP server, I'd be specifying a different set of interface
names. In some situations I even just put in one bond interface member,
which is useless, but in most situations I find less entropy is worth more
than having a slightly more efficient networking stack.
I have bounced around the idea of removing this step and trusting the
switch - ie: write a fact to do an LLDP query for the VLAN of the switch
port each interface is connected to, that way you wouldn't need the glue,
there'd be a fact called vlan_100_interfaces. Two problems with this
approach: we end up trusting the switch to be our source of truth (it may
not be correct, and, what if the switch port is down?). Secondly the
quality and consistency of LLDP information you get out of various
manufacturers of networking hardware is very different, so relying on LLDP
information to define your OS network config is a bit risky for me.
It's a different story for our VMs. Since they are Puppet defined we
specify a MAC address and so we "know" which MAC will be attached to which
VM bridge. We drop a MAC based udev rule into the guest to name them
similarly, ie: eth100 is on br100. I could technically use the same Puppet
code to write udev rules for my hardware, but the PCI based naming scheme
is fine so far.
That's what we do, but it's made easy by an almost homogeneous hardware
platform and strict physical patch management.
When I read about your problem, it sounds like you are missing a "glue
record" that describes your logical interfaces to your physical devices. If
you were to follow something along the lines of our approach, you might
have something like this:
class profile::some_firewall(
$external_interface_name = 'eth0',
$internal_interface_name = 'eth1',
$perimiter_interface_name = 'eth2'
) {
firewall { '001_allow_internal':
chain => 'INPUT',
iniface => $internal_interface_name,
action => 'accept',
proto => 'all',
}
firewall { '002_some_external_rule':
chain => 'INPUT',
iniface => $external_interface_name,
action => 'accept',
proto => 'tcp',
dport => '443',
}
}
That very simple firewall profile probably already works on your HP
hardware, and on your Dell hardware you'd need to override the 3 parameters
in Hiera:
profile::some_firewall::internal_interface_name: 'em1'
profile::some_firewall::external_interface_name: 'p3p1'
profile::some_firewall::perimiter_interface_name: 'p1p1'
Hope that helps,
-Luke
On Wednesday, 24 August 2016 14:55:38 UTC+1, Marc Haber wrote:
>
> Hi,
>
> I would like to discuss how to handle systemd's new feature of
> predictable network interface names. This is a rather hot topic in the
> team I'm currently working in, and I'd like to solicit your opinions
> about that.
>
> On systems with more than one interface, the canonical way to handle
> this issue in the past was "assume that eth0 is connected to network
> foo, eth1 is connected to network bar, and eth2 is connected to
> network baz" and to accept t
