Re: [Puppet Users] How to handle predictable network interface names

[Marc Haber]

On Thu, Aug 25, 2016 at 09:00:46AM -0700, Luke Bigum wrote:
> On Thursday, 25 August 2016 13:31:17 UTC+1, Marc Haber wrote:
> > So the "database" machine wouldn't have an entry in 
> > networking::interfaces at all, or could one define, for example, the 
> > management interface in networking::interfaces and the database 
> > interfaces in the machine-specific hiera tree? 
> >
> 
> That's technically possible with our module, yes, although I personally 
> don't want to mix the styles.

I understand.

> In my current profiles though I started with the management interfaces 
> inside the same software profiles. Turns out this was not the best idea as 
> they are not directly related, and what our roles should really look like 
> is this:
> 
> ***
> class role::database {
>   include profile::mandatory#Everything mandatory  on 
> EL6
>   include profile::authentication   #Authentication is not 
> mandatory
>   include profile::database  #The profile that does 
> most of the work for our software
>   class { 'profile::management':   #management network  
> definition and dependent services (sshd, etc)
>  type => 'database'   #but for a specific 
> type of machine
>   }
> }
> ***
> 
> So management would be separate. This would allow me to do smarter ordering 
> of Puppet classes for management services like SSH (and remove a little bit 
> more Hiera glue).

That looks smart, thanks.

Greetings
Marc

-- 
-
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany|  lose things."Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/20160826073850.GI2471%40torres.zugschlus.de.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] How to handle predictable network interface names

[Luke Bigum]

On Thursday, 25 August 2016 13:31:17 UTC+1, Marc Haber wrote:
>
> On Wed, Aug 24, 2016 at 09:03:16AM -0700, Luke Bigum wrote: 
> > The template will create udev rules from two sources. The first is 
> > @interfaces, which is the giant multi-level hash of network interfaces 
> that 
> > our old designs use. A VM might look like this in Hiera: 
> > 
> > networking::interfaces: 
> >   eth0: 
> > ipaddr: 1.1.1.1 
> > hwaddr: 52:54:00:11:22:33 
> > 
> > The second source of udev rules is also a Hash and also from Hiera, but 
> > rather than it be embedded in the giant hash of networking information, 
> it 
> > is there to compliment the newer role/profile approach where we don't 
> > specify MAC addresses. This is purely a cosmetic thing for VMs to make 
> our 
> > interface names look sensible. Here is a sanitised Hiera file for a VM 
> with 
> > the fictitious "database" profile: 
> > 
> > profile::database::subnet_INTERNAL_slaves: 
> >   - 'eth100' 
> > profile::database::subnet_CLIENT_slaves: 
> >   - 'eth214' 
> > networking::extra_udev_static_interface_names: 
> >   eth100: '52:54:00:11:22:33' 
> >   eth214: '52:54:00:44:55:66' 
>
> So the "database" machine wouldn't have an entry in 
> networking::interfaces at all, or could one define, for example, the 
> management interface in networking::interfaces and the database 
> interfaces in the machine-specific hiera tree? 
>

That's technically possible with our module, yes, although I personally 
don't want to mix the styles. It has to do with our Hiera hierarchy being 
mostly based on physical location and entire "instances" of our 
application, where what we're talking about here is functionality based. If 
we had a business rule where "every server in this data centre has 
management interface eth76" then yeah, that would match our Hiera hierarchy 
perfectly. We don't have those hard and fast rules though, we've got 
several management networks, with different levels of security applied, 
appropriate for different layers of our application. So our management 
networks are a function of defence in depth security design alongside our 
software, rather than a simple physical location or group of VMs. Since 
they're a function of design or "business logic", rather than location, our 
management networks are defined in profiles (on new systems) because it's 
only at the role/profile level do you know that a "database" server should 
have a certain type of management network.

In my current profiles though I started with the management interfaces 
inside the same software profiles. Turns out this was not the best idea as 
they are not directly related, and what our roles should really look like 
is this:

***
class role::database {
  include profile::mandatory#Everything mandatory 
on EL6
  include profile::authentication   #Authentication is not 
mandatory
  include profile::database  #The profile that does 
most of the work for our software
  class { 'profile::management':   #management network 
definition and dependent services (sshd, etc)
 type => 'database'   #but for a specific 
type of machine
  }
}
***

So management would be separate. This would allow me to do smarter ordering 
of Puppet classes for management services like SSH (and remove a little bit 
more Hiera glue).


Greetings 
> Marc 
>
> -- 
> - 
>
> Marc Haber | "I don't trust Computers. They | Mailadresse im 
> Header 
> Leimen, Germany|  lose things."Winona Ryder | Fon: *49 6224 
> 1600402 
> Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 
> 1600421 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/651d7a2d-8eac-49d6-aaac-8e03937dc7c4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] How to handle predictable network interface names

[Rob Nelson]

We are provisioning everything with vmxnet3 drivers, only weird old OVAs
from vendors have non-vmxnet3, and we don't manage those with puppet. We
are cloning from template and getting the same results on different
vSphere/vCenter versions and instances. Not sure why they would come out
different since the "slot" ends up being the same on the virtual hardware.


Rob Nelson
rnels...@gmail.com

On Thu, Aug 25, 2016 at 8:30 AM, Marc Haber 
wrote:

> Hi Rob,
>
> On Wed, Aug 24, 2016 at 10:30:20AM -0400, Rob Nelson wrote:
> > We use VMware's vSphere, which still results in "random" but predictable
> > interface names - eth0 is now eno16780032, eth1 is now eno3359296, etc.
>
> In my experience, the numbers are rather random, different on each VM.
>
> > We've stuck with that because while it's somewhat painful (eth# is
> soo
> > much easier to comprehend), it's far less painful to memorize that than
> to
> > maintain some udev rules that may need tweaked across time.
>
> having net.ifnames=0 looks easier than working with those eight-digit
> numbers.
>
> >  However, if we were on bare metal, it might be worth disabling the
> >  rules to get the older style back. That's probably still less optimal
> >  than customized names, but it's well documented at least. For
> >  example, http://carminebufano.com/?p=108 or
> >  http://amolkg.blogspot.in/2015/05/centos-7-change-
> network-interface-name.html
> >  - though there are multiple ways to do it even then.
>
> Yes, that's what I'd see as a last moment manoeuvre. My gut feeling
> says "bad idea, don't do that". I don't have any evidence-based
> arguments for that though.
>
> Greetings
> Marc
>
> --
> 
> -
> Marc Haber | "I don't trust Computers. They | Mailadresse im Header
> Leimen, Germany|  lose things."Winona Ryder | Fon: *49 6224 1600402
> Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/puppet-users/20160825123035.GZ2471%40torres.zugschlus.de.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAC76iT9xdN_CQZO8h9jvQZKMtsqKG8%3DfrJy_evxb-%2B_8_32hLQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] How to handle predictable network interface names

[Marc Haber]

On Wed, Aug 24, 2016 at 09:03:16AM -0700, Luke Bigum wrote:
> The template will create udev rules from two sources. The first is 
> @interfaces, which is the giant multi-level hash of network interfaces that 
> our old designs use. A VM might look like this in Hiera:
> 
> networking::interfaces:
>   eth0:
> ipaddr: 1.1.1.1
> hwaddr: 52:54:00:11:22:33
> 
> The second source of udev rules is also a Hash and also from Hiera, but 
> rather than it be embedded in the giant hash of networking information, it 
> is there to compliment the newer role/profile approach where we don't 
> specify MAC addresses. This is purely a cosmetic thing for VMs to make our 
> interface names look sensible. Here is a sanitised Hiera file for a VM with 
> the fictitious "database" profile:
> 
> profile::database::subnet_INTERNAL_slaves:
>   - 'eth100'
> profile::database::subnet_CLIENT_slaves:
>   - 'eth214'
> networking::extra_udev_static_interface_names:
>   eth100: '52:54:00:11:22:33'
>   eth214: '52:54:00:44:55:66'

So the "database" machine wouldn't have an entry in
networking::interfaces at all, or could one define, for example, the
management interface in networking::interfaces and the database
interfaces in the machine-specific hiera tree?

Greetings
Marc

-- 
-
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany|  lose things."Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/20160825123111.GA2471%40torres.zugschlus.de.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] How to handle predictable network interface names

[Marc Haber]

Hi Rob,

On Wed, Aug 24, 2016 at 10:30:20AM -0400, Rob Nelson wrote:
> We use VMware's vSphere, which still results in "random" but predictable
> interface names - eth0 is now eno16780032, eth1 is now eno3359296, etc.

In my experience, the numbers are rather random, different on each VM.

> We've stuck with that because while it's somewhat painful (eth# is soo
> much easier to comprehend), it's far less painful to memorize that than to
> maintain some udev rules that may need tweaked across time.

having net.ifnames=0 looks easier than working with those eight-digit
numbers.

>  However, if we were on bare metal, it might be worth disabling the
>  rules to get the older style back. That's probably still less optimal
>  than customized names, but it's well documented at least. For
>  example, http://carminebufano.com/?p=108 or
>  http://amolkg.blogspot.in/2015/05/centos-7-change-network-interface-name.html
>  - though there are multiple ways to do it even then.

Yes, that's what I'd see as a last moment manoeuvre. My gut feeling
says "bad idea, don't do that". I don't have any evidence-based
arguments for that though.

Greetings
Marc

-- 
-
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany|  lose things."Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/20160825123035.GZ2471%40torres.zugschlus.de.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] How to handle predictable network interface names

[Marc Haber]

On Wed, Aug 24, 2016 at 09:20:56AM -0700, Luke Bigum wrote:
> Now that I think about it, I might be able to post a sanitised version of 
> the module online with most of the internal stuff stripped out. It might 
> prove useful for educating our own staff in the concepts, as well as other 
> people. It's not a 5 minute job though so if/when it's done, I'll write a 
> new Group post instead of continuing to hijack this one :-)

Thanks in advance!

Greetings
Marc

-- 
-
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany|  lose things."Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/20160825122621.GY2471%40torres.zugschlus.de.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] How to handle predictable network interface names

[Rob Nelson]

Just tell them you wanted to make sure you were satisfying the external pen
testing requirements of PCI ;)

On Wednesday, August 24, 2016, Luke Bigum  wrote:

>  if I gave out the module the Security team would throttle me for
> releasing what is part of a map of internal network architecture ;-)
>


-- 

Rob Nelson
rnels...@gmail.com

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAC76iT-WXMQ64j5KYBo%2BCU7AAh%2BQYv3pUJ9i0%2B0JJ0PfhSZGRA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] How to handle predictable network interface names

[Luke Bigum]

Now that I think about it, I might be able to post a sanitised version of 
the module online with most of the internal stuff stripped out. It might 
prove useful for educating our own staff in the concepts, as well as other 
people. It's not a 5 minute job though so if/when it's done, I'll write a 
new Group post instead of continuing to hijack this one :-)

On Wednesday, 24 August 2016 17:05:47 UTC+1, LinuxDan wrote:
>
> It is a starting point.
> Many thanks for sharing what you can.
>
> Dan White | d_e_...@icloud.com 
> 
> “Sometimes I think the surest sign that intelligent life exists elsewhere in 
> the universe is that none of it has tried to contact us.”  (Bill Waterson: 
> Calvin & Hobbes)
>
>
> On Aug 24, 2016, at 12:03 PM, Luke Bigum  
> wrote:
>
> No, not really :-( It's a very "internal" module that I forked from 
> someone's Google Summer of Code project over 5 years ago (way before 
> voxpupuli/puppet-network). You know all those Hiera keys about vlan tags I 
> mentioned? The defaults are in this module and are the default VLAN 
> interfaces for all of our networks. if I gave out the module the Security 
> team would throttle me for releasing what is part of a map of internal 
> network architecture ;-)
>
> I can however, just post the bit that does the UDEV rules...
>
> *
> $ cat ../templates/etc/udev/rules.d/70-persistent-net.rules.erb
> # Managed by Puppet
>
> <% if @interfaces.is_a?(Hash) -%>
> <%   @interfaces.sort.each do |key,val| -%>
> <% if val['hwaddr'] -%>
> SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="<%= 
> val['hwaddr'] -%>", ATTR{type}=="1", KERNEL=="eth*", NAME="<%= key -%>"
> <% end # if val['hwaddr'] -%>
> <%   end # @interfaces.sort.each -%>
> <% end -%>
> <% if @extra_udev_static_interface_names.is_a?(Hash) -%>
> <%   @extra_udev_static_interface_names.sort.each do |interface,hwaddr| -%>
> SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="<%= 
> hwaddr.downcase -%>", ATTR{type}=="1", KERNEL=="eth*", NAME="<%= interface 
> -%>"
> <%   end -%>
> <% end -%>
> *
>
> The template will create udev rules from two sources. The first is 
> @interfaces, which is the giant multi-level hash of network interfaces that 
> our old designs use. A VM might look like this in Hiera:
>
> networking::interfaces:
>   eth0:
> ipaddr: 1.1.1.1
> hwaddr: 52:54:00:11:22:33
>
> The second source of udev rules is also a Hash and also from Hiera, but 
> rather than it be embedded in the giant hash of networking information, it 
> is there to compliment the newer role/profile approach where we don't 
> specify MAC addresses. This is purely a cosmetic thing for VMs to make our 
> interface names look sensible. Here is a sanitised Hiera file for a VM with 
> the fictitious "database" profile:
>
> profile::database::subnet_INTERNAL_slaves:
>   - 'eth100'
> profile::database::subnet_CLIENT_slaves:
>   - 'eth214'
> networking::extra_udev_static_interface_names:
>   eth100: '52:54:00:11:22:33'
>   eth214: '52:54:00:44:55:66'
>
>
>
>
>
> On Wednesday, 24 August 2016 16:41:28 UTC+1, LinuxDan wrote:
>>
>> Very nice, Luke.
>>
>> Does the code that lets you custom-name your interfaces live in github or 
>> puppet-forge anywhere ?
>>
>> If not, would you be willing to share ?  I can bring brownies and/or beer 
>> to the collaboration :)
>>
>> Dan White | d_e_...@icloud.com
>> 
>> “Sometimes I think the surest sign that intelligent life exists elsewhere in 
>> the universe is that none of it has tried to contact us.”  (Bill Waterson: 
>> Calvin & Hobbes)
>>
>>
>> On Aug 24, 2016, at 11:36 AM, Luke Bigum  wrote:
>>
>> Here we have very strict control over our hardware and what interface 
>> goes where. We keep CentOS 6's naming scheme on Dell hardware, so p2p1 is 
>> PCI slot 2, Port 1, and don't try rename it. We have a 3rd party patch 
>> manager tool (patchmanager.com), LLDP on our switches, and a Nagios 
>> check that tells me if an interface is not plugged into the switch port it 
>> is supposed to be plugged into (according to patchmanager). This works 
>> perfectly on Dell hardware because the PCI name mapping works. On really 
>> old HP gear it doesn't work, so we fall back on always assuming eth0 is the 
>> first onboard port, etc. If the kernel scanned these devices in a different 
>> order we'd get the same breakage you describe, but that's never happened on 
>> it's own, it's only happened if an engineer has gone and added re-arranged 
>> cards.
>>
>> We still need some sort of "glue record" that says "this interface should 
>> be up and have this IP". In our older designs this was managed entirely in 
>> Hiera - so there's a giant multi-level hash that we run create_resources() 
>> over to define every single network interface. You can imagine the amount 
>> of Hiera data we have. In 

Re: [Puppet Users] How to handle predictable network interface names

[Dan White]

It is a starting point.
Many thanks for sharing what you can.
Dan White | d_e_wh...@icloud.com

“Sometimes I think the surest sign that intelligent life exists elsewhere in the 
universe is that none of it has tried to contact us.”  (Bill Waterson: Calvin & 
Hobbes)

On Aug 24, 2016, at 12:03 PM, Luke Bigum  wrote:

No, not really :-( It's a very "internal" module that I forked from someone's 
Google Summer of Code project over 5 years ago (way before voxpupuli/puppet-network). You 
know all those Hiera keys about vlan tags I mentioned? The defaults are in this module 
and are the default VLAN interfaces for all of our networks. if I gave out the module the 
Security team would throttle me for releasing what is part of a map of internal network 
architecture ;-)

I can however, just post the bit that does the UDEV rules...

*
$ cat ../templates/etc/udev/rules.d/70-persistent-net.rules.erb
# Managed by Puppet

<% if @interfaces.is_a?(Hash) -%>
<%   @interfaces.sort.each do |key,val| -%>
<%     if val['hwaddr'] -%>
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="<%= val['hwaddr'] -%>", ATTR{type}=="1", 
KERNEL=="eth*", NAME="<%= key -%>"
<%     end # if val['hwaddr'] -%>
<%   end # @interfaces.sort.each -%>
<% end -%>
<% if @extra_udev_static_interface_names.is_a?(Hash) -%>
<%   @extra_udev_static_interface_names.sort.each do |interface,hwaddr| -%>
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="<%= hwaddr.downcase -%>", 
ATTR{type}=="1", KERNEL=="eth*", NAME="<%= interface -%>"
<%   end -%>
<% end -%>
*

The template will create udev rules from two sources. The first is @interfaces, 
which is the giant multi-level hash of network interfaces that our old designs 
use. A VM might look like this in Hiera:

networking::interfaces:
  eth0:
    ipaddr: 1.1.1.1
    hwaddr: 52:54:00:11:22:33

The second source of udev rules is also a Hash and also from Hiera, but rather than it be 
embedded in the giant hash of networking information, it is there to compliment the newer 
role/profile approach where we don't specify MAC addresses. This is purely a cosmetic 
thing for VMs to make our interface names look sensible. Here is a sanitised Hiera file 
for a VM with the fictitious "database" profile:

profile::database::subnet_INTERNAL_slaves:
  - 'eth100'
profile::database::subnet_CLIENT_slaves:
  - 'eth214'
networking::extra_udev_static_interface_names:
  eth100: '52:54:00:11:22:33'
  eth214: '52:54:00:44:55:66'





On Wednesday, 24 August 2016 16:41:28 UTC+1, LinuxDan wrote:
Very nice, Luke.

Does the code that lets you custom-name your interfaces live in github or 
puppet-forge anywhere ?

If not, would you be willing to share ?  I can bring brownies and/or beer to 
the collaboration :)
Dan White | d_e_...@icloud.com

“Sometimes I think the surest sign that intelligent life exists elsewhere in the 
universe is that none of it has tried to contact us.”  (Bill Waterson: Calvin & 
Hobbes)

On Aug 24, 2016, at 11:36 AM, Luke Bigum  wrote:

Here we have very strict control over our hardware and what interface goes 
where. We keep CentOS 6's naming scheme on Dell hardware, so p2p1 is PCI slot 
2, Port 1, and don't try rename it. We have a 3rd party patch manager tool 
(patchmanager.com), LLDP on our switches, and a Nagios check that tells me if 
an interface is not plugged into the switch port it is supposed to be plugged 
into (according to patchmanager). This works perfectly on Dell hardware because 
the PCI name mapping works. On really old HP gear it doesn't work, so we fall 
back on always assuming eth0 is the first onboard port, etc. If the kernel 
scanned these devices in a different order we'd get the same breakage you 
describe, but that's never happened on it's own, it's only happened if an 
engineer has gone and added re-arranged cards.

We still need some sort of "glue record" that says "this interface should be up and have this IP". In our 
older designs this was managed entirely in Hiera - so there's a giant multi-level hash that we run create_resources() over to 
define every single network interface. You can imagine the amount of Hiera data we have. In the newer designs which are a lot 
more of a role/profile approach I've been trying to conceptualise the networking based on our profiles. So if one of our servers 
is fulfilling function "database" there will be a Class[profile::database]. This Class might create a bonded interface 
for the "STORAGE" network and another interface for the "CLIENT" network. Through various levels of Hiera I 
can define the STORAGE network as VLAN 100, because it might be a different vlan tag at a different location. Then at the Hiera 
node level (on each individual server) I will have something like:

profile::database::bond_storage_slaves: [ 'p2p1', 'p2p2' ]

That's the 

Re: [Puppet Users] How to handle predictable network interface names

[Luke Bigum]

No, not really :-( It's a very "internal" module that I forked from 
someone's Google Summer of Code project over 5 years ago (way before 
voxpupuli/puppet-network). You know all those Hiera keys about vlan tags I 
mentioned? The defaults are in this module and are the default VLAN 
interfaces for all of our networks. if I gave out the module the Security 
team would throttle me for releasing what is part of a map of internal 
network architecture ;-)

I can however, just post the bit that does the UDEV rules...

*
$ cat ../templates/etc/udev/rules.d/70-persistent-net.rules.erb
# Managed by Puppet

<% if @interfaces.is_a?(Hash) -%>
<%   @interfaces.sort.each do |key,val| -%>
<% if val['hwaddr'] -%>
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="<%= 
val['hwaddr'] -%>", ATTR{type}=="1", KERNEL=="eth*", NAME="<%= key -%>"
<% end # if val['hwaddr'] -%>
<%   end # @interfaces.sort.each -%>
<% end -%>
<% if @extra_udev_static_interface_names.is_a?(Hash) -%>
<%   @extra_udev_static_interface_names.sort.each do |interface,hwaddr| -%>
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="<%= 
hwaddr.downcase -%>", ATTR{type}=="1", KERNEL=="eth*", NAME="<%= interface 
-%>"
<%   end -%>
<% end -%>
*

The template will create udev rules from two sources. The first is 
@interfaces, which is the giant multi-level hash of network interfaces that 
our old designs use. A VM might look like this in Hiera:

networking::interfaces:
  eth0:
ipaddr: 1.1.1.1
hwaddr: 52:54:00:11:22:33

The second source of udev rules is also a Hash and also from Hiera, but 
rather than it be embedded in the giant hash of networking information, it 
is there to compliment the newer role/profile approach where we don't 
specify MAC addresses. This is purely a cosmetic thing for VMs to make our 
interface names look sensible. Here is a sanitised Hiera file for a VM with 
the fictitious "database" profile:

profile::database::subnet_INTERNAL_slaves:
  - 'eth100'
profile::database::subnet_CLIENT_slaves:
  - 'eth214'
networking::extra_udev_static_interface_names:
  eth100: '52:54:00:11:22:33'
  eth214: '52:54:00:44:55:66'




On Wednesday, 24 August 2016 16:41:28 UTC+1, LinuxDan wrote:
>
> Very nice, Luke.
>
> Does the code that lets you custom-name your interfaces live in github or 
> puppet-forge anywhere ?
>
> If not, would you be willing to share ?  I can bring brownies and/or beer 
> to the collaboration :)
>
> Dan White | d_e_...@icloud.com 
> 
> “Sometimes I think the surest sign that intelligent life exists elsewhere in 
> the universe is that none of it has tried to contact us.”  (Bill Waterson: 
> Calvin & Hobbes)
>
>
> On Aug 24, 2016, at 11:36 AM, Luke Bigum  
> wrote:
>
> Here we have very strict control over our hardware and what interface goes 
> where. We keep CentOS 6's naming scheme on Dell hardware, so p2p1 is PCI 
> slot 2, Port 1, and don't try rename it. We have a 3rd party patch manager 
> tool (patchmanager.com), LLDP on our switches, and a Nagios check that 
> tells me if an interface is not plugged into the switch port it is supposed 
> to be plugged into (according to patchmanager). This works perfectly on 
> Dell hardware because the PCI name mapping works. On really old HP gear it 
> doesn't work, so we fall back on always assuming eth0 is the first onboard 
> port, etc. If the kernel scanned these devices in a different order we'd 
> get the same breakage you describe, but that's never happened on it's own, 
> it's only happened if an engineer has gone and added re-arranged cards.
>
> We still need some sort of "glue record" that says "this interface should 
> be up and have this IP". In our older designs this was managed entirely in 
> Hiera - so there's a giant multi-level hash that we run create_resources() 
> over to define every single network interface. You can imagine the amount 
> of Hiera data we have. In the newer designs which are a lot more of a 
> role/profile approach I've been trying to conceptualise the networking 
> based on our profiles. So if one of our servers is fulfilling function 
> "database" there will be a Class[profile::database]. This Class might 
> create a bonded interface for the "STORAGE" network and another interface 
> for the "CLIENT" network. Through various levels of Hiera I can define the 
> STORAGE network as VLAN 100, because it might be a different vlan tag at a 
> different location. Then at the Hiera node level (on each individual 
> server) I will have something like:
>
> profile::database::bond_storage_slaves: [ 'p2p1', 'p2p2' ]
>
> That's the glue. At some point I need to tell Puppet that on this specific 
> server, the storage network is a bond of p2p1 and p2p2. If I took that 
> profile to a HP server, I'd be specifying a different set of interface 
> names. In some situations I even just put in one bond interface member, 
> 

Re: [Puppet Users] How to handle predictable network interface names

[Dan White]

Very nice, Luke.

Does the code that lets you custom-name your interfaces live in github or 
puppet-forge anywhere ?

If not, would you be willing to share ?  I can bring brownies and/or beer to 
the collaboration :)
Dan White | d_e_wh...@icloud.com

“Sometimes I think the surest sign that intelligent life exists elsewhere in the 
universe is that none of it has tried to contact us.”  (Bill Waterson: Calvin & 
Hobbes)

On Aug 24, 2016, at 11:36 AM, Luke Bigum  wrote:

Here we have very strict control over our hardware and what interface goes 
where. We keep CentOS 6's naming scheme on Dell hardware, so p2p1 is PCI slot 
2, Port 1, and don't try rename it. We have a 3rd party patch manager tool 
(patchmanager.com), LLDP on our switches, and a Nagios check that tells me if 
an interface is not plugged into the switch port it is supposed to be plugged 
into (according to patchmanager). This works perfectly on Dell hardware because 
the PCI name mapping works. On really old HP gear it doesn't work, so we fall 
back on always assuming eth0 is the first onboard port, etc. If the kernel 
scanned these devices in a different order we'd get the same breakage you 
describe, but that's never happened on it's own, it's only happened if an 
engineer has gone and added re-arranged cards.

We still need some sort of "glue record" that says "this interface should be up and have this IP". In our 
older designs this was managed entirely in Hiera - so there's a giant multi-level hash that we run create_resources() over to 
define every single network interface. You can imagine the amount of Hiera data we have. In the newer designs which are a lot 
more of a role/profile approach I've been trying to conceptualise the networking based on our profiles. So if one of our servers 
is fulfilling function "database" there will be a Class[profile::database]. This Class might create a bonded interface 
for the "STORAGE" network and another interface for the "CLIENT" network. Through various levels of Hiera I 
can define the STORAGE network as VLAN 100, because it might be a different vlan tag at a different location. Then at the Hiera 
node level (on each individual server) I will have something like:

profile::database::bond_storage_slaves: [ 'p2p1', 'p2p2' ]

That's the glue. At some point I need to tell Puppet that on this specific 
server, the storage network is a bond of p2p1 and p2p2. If I took that profile 
to a HP server, I'd be specifying a different set of interface names. In some 
situations I even just put in one bond interface member, which is useless, but 
in most situations I find less entropy is worth more than having a slightly 
more efficient networking stack.

I have bounced around the idea of removing this step and trusting the switch - 
ie: write a fact to do an LLDP query for the VLAN of the switch port each 
interface is connected to, that way you wouldn't need the glue, there'd be a 
fact called vlan_100_interfaces. Two problems with this approach: we end up 
trusting the switch to be our source of truth (it may not be correct, and, what 
if the switch port is down?). Secondly the quality and consistency of LLDP 
information you get out of various manufacturers of networking hardware is very 
different, so relying on LLDP information to define your OS network config is a 
bit risky for me.

It's a different story for our VMs. Since they are Puppet defined we specify a MAC 
address and so we "know" which MAC will be attached to which VM bridge. We drop 
a MAC based udev rule into the guest to name them similarly, ie: eth100 is on br100. I 
could technically use the same Puppet code to write udev rules for my hardware, but the 
PCI based naming scheme is fine so far.

That's what we do, but it's made easy by an almost homogeneous hardware 
platform and strict physical patch management.

When I read about your problem, it sounds like you are missing a "glue record" 
that describes your logical interfaces to your physical devices. If you were to follow 
something along the lines of our approach, you might have something like this:

class profile::some_firewall(
  $external_interface_name = 'eth0',
  $internal_interface_name = 'eth1',
  $perimiter_interface_name = 'eth2'
) {
  firewall { '001_allow_internal':
    chain   => 'INPUT',
    iniface => $internal_interface_name,
    action  => 'accept',
    proto => 'all',
  }

  firewall { '002_some_external_rule':
    chain   => 'INPUT',
    iniface => $external_interface_name,
    action  => 'accept',
    proto => 'tcp',
    dport => '443',
  }
}

That very simple firewall profile probably already works on your HP hardware, 
and on your Dell hardware you'd need to override the 3 parameters in Hiera:

profile::some_firewall::internal_interface_name: 'em1'
profile::some_firewall::external_interface_name: 'p3p1'
profile::some_firewall::perimiter_interface_name: 'p1p1'

Hope that helps,

-Luke

On 

Re: [Puppet Users] How to handle predictable network interface names

[Rob Nelson]

Marc,

We use VMware's vSphere, which still results in "random" but predictable
interface names - eth0 is now eno16780032, eth1 is now eno3359296, etc.
We've stuck with that because while it's somewhat painful (eth# is soo
much easier to comprehend), it's far less painful to memorize that than to
maintain some udev rules that may need tweaked across time. However, if we
were on bare metal, it might be worth disabling the rules to get the older
style back. That's probably still less optimal than customized names, but
it's well documented at least. For example, http://carminebufano.com/?p=108
or
http://amolkg.blogspot.in/2015/05/centos-7-change-network-interface-name.html
- though there are multiple ways to do it even then.


Rob Nelson
rnels...@gmail.com

On Wed, Aug 24, 2016 at 9:55 AM, Marc Haber 
wrote:

> Hi,
>
> I would like to discuss how to handle systemd's new feature of
> predictable network interface names. This is a rather hot topic in the
> team I'm currently working in, and I'd like to solicit your opinions
> about that.
>
> On systems with more than one interface, the canonical way to handle
> this issue in the past was "assume that eth0 is connected to network
> foo, eth1 is connected to network bar, and eth2 is connected to
> network baz" and to accept that things fail horribly if the order in
> which network interfaces are detected changes.
>
> While upstream's focus is as usual on desktop machines where Ethernet,
> USB and WWAN interfaces come and go multiple times a day (see
> upstream's reasoning in
> https://www.freedesktop.org/wiki/Software/systemd/
> PredictableNetworkInterfaceNames/),
> this seldomly happens in our happy server environment, which reduces
> the breakage potential to disruptive kernel updates or vendor firmware
> changes peddling with the order in which network interfaces are
> enumerated.
>
> This happens rather seldomly in my experience.
>
> I would, however, like to stay with the new scheme since I see its
> charms.
>
> But, how would I handle this in a puppetized environment?
>
> Currently, the code that is already, for example for a firewall
> assumes that eth0 is the external interface, eth1 the internal one and
> eth2 the perimeter networks.
>
> In the new setup, all those interfaces can have different names
> depending on different hardware being used. That means that the same
> puppet code cannot be used on one firewall instance running on Dell
> hardware and a second one running on HP hardware because BIOS indices
> and/or PCI paths will vary. If I used the MAC scheme, things are even
> worse since interface names will be different even on different pieces
> of otherwise identical hardware.
>
> Many of my team members thinkt hat one should simply turn of
> predictable network interface names altgether and so that our old code
> continues to work. I think that this would be a bad idea, but don't
> have any logical arguments other than my gut feeling.
>
> Generating udev rules to fix the network names (and assign names like
> ext1, int1, per1) already in postinst of the OS does not work since we
> don't know how the machine is going to be wired and even used.
>
> Any ideas? How do _you_ do this?
>
> Greetings
> Marc
>
> --
> 
> -
> Marc Haber | "I don't trust Computers. They | Mailadresse im Header
> Leimen, Germany|  lose things."Winona Ryder | Fon: *49 6224 1600402
> Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/puppet-users/20160824135527.GP2471%40torres.zugschlus.de.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAC76iT8YZCWeKDB%2BdPiTRxE80WTe2BrCL%3D1hJEFteZKJF-gHEw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] How to handle predictable network interface names

[Marc Haber]

Hi,

I would like to discuss how to handle systemd's new feature of
predictable network interface names. This is a rather hot topic in the
team I'm currently working in, and I'd like to solicit your opinions
about that.

On systems with more than one interface, the canonical way to handle
this issue in the past was "assume that eth0 is connected to network
foo, eth1 is connected to network bar, and eth2 is connected to
network baz" and to accept that things fail horribly if the order in
which network interfaces are detected changes.

While upstream's focus is as usual on desktop machines where Ethernet,
USB and WWAN interfaces come and go multiple times a day (see
upstream's reasoning in
https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/),
this seldomly happens in our happy server environment, which reduces
the breakage potential to disruptive kernel updates or vendor firmware
changes peddling with the order in which network interfaces are
enumerated.

This happens rather seldomly in my experience.

I would, however, like to stay with the new scheme since I see its
charms.

But, how would I handle this in a puppetized environment?

Currently, the code that is already, for example for a firewall
assumes that eth0 is the external interface, eth1 the internal one and
eth2 the perimeter networks.

In the new setup, all those interfaces can have different names
depending on different hardware being used. That means that the same
puppet code cannot be used on one firewall instance running on Dell
hardware and a second one running on HP hardware because BIOS indices
and/or PCI paths will vary. If I used the MAC scheme, things are even
worse since interface names will be different even on different pieces
of otherwise identical hardware.

Many of my team members thinkt hat one should simply turn of
predictable network interface names altgether and so that our old code
continues to work. I think that this would be a bad idea, but don't
have any logical arguments other than my gut feeling.

Generating udev rules to fix the network names (and assign names like
ext1, int1, per1) already in postinst of the OS does not work since we
don't know how the machine is going to be wired and even used.

Any ideas? How do _you_ do this?

Greetings
Marc

-- 
-
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany|  lose things."Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/20160824135527.GP2471%40torres.zugschlus.de.
For more options, visit https://groups.google.com/d/optout.