Re: [CentOS] Back to eth shuffling ...
When I was working on this last time (with the r8169 driver), someone on
this list provided the following script which is what fixed the issue at
the time by creating a new 70-persistent-net.rules file with the devices
enumerated in order. However, this no longer works now.
echo [KICKSTART] Binding eth interfaces to the expected MAC address in
UDEV
echo ## Created by Kickstart to keep network interfaces in an expected
order \
/etc/udev/rules.d/70-persistent-net.rules
echo \
/etc/udev/rules.d/70-persistent-net.rules
cd /sys/class/net/
for NETDEV in $(ls | /bin/grep eth | sort)
do
## Create a UDEV rule for each eth interface
echo ## ${NETDEV} interface \
/etc/udev/rules.d/70-persistent-net.rules
## We throw this one in here as it can contain some useful information
echo ## $(dmesg | /bin/grep ${NETDEV} | /bin/grep -i -v -e console
-e Command line | head -1) \
/etc/udev/rules.d/70-persistent-net.rules
echo -n SUBSYSTEM==\net\, ACTION==\add\, DRIVERS==\?*\, \
/etc/udev/rules.d/70-persistent-net.rules
echo -n ATTR{address}==\$(cat ${NETDEV}/address)\, \
/etc/udev/rules.d/70-persistent-net.rules
echo -n ATTR{dev_id}==\0x0\, ATTR{type}==\1\, KERNEL==\eth?*\,
\
/etc/udev/rules.d/70-persistent-net.rules
echo -e NAME=\${NETDEV}\\n \
/etc/udev/rules.d/70-persistent-net.rules
## Make a log of the devices present during install
echo -e ${NETDEV} $(cat ${NETDEV}/address)\n /root/ksnet-devices
## Also remove the HWADDR line from all of the net config files
/bin/grep -v -e NAME -e HWADDR -e NM_CONTROLLED \
/etc/sysconfig/network-scripts/ifcfg-${NETDEV} | sed 's/\//g' \
/etc/sysconfig/network-scripts/ifcfg-${NETDEV}-tmp
echo NM_CONTROLLED=no
/etc/sysconfig/network-scripts/ifcfg-${NETDEV}-tmp
#/usr/bin/perl -p -i -e 's/dhcp/none/'
/etc/sysconfig/network-scripts/ifcfg-${NETDEV}-tmp
/bin/mv -f /etc/sysconfig/network-scripts/ifcfg-${NETDEV}-tmp \
/etc/sysconfig/network-scripts/ifcfg-${NETDEV}
done
After the machine boots and I look in /root/ksnet-devices, I see the MAC
addresses for the devices as:
Port1 - eth0
PCIe Card- eth1
Port2 - eth2
And yet, during the machine's POST (which can verify by the PXE boot up of
each device), it correctly enumerates the ethernet devices as:
Port1
Port2
PCIe card
So where and why is the order changed when the kernel kicks in and start
booting the machine? And how can I stop it, or change its behavior? I'm not
seeing any 'renaming' going on in dmesg like I sometimes find. So something
is causing this and I can't figure it out.
On Wed, May 13, 2015 at 5:17 PM, Kahlil Hodgson
kahlil.hodg...@dealmax.com.au wrote:
Have you tried having kickstart set up a more appropriate
/etc/udev/rules/70-persistent-net.rules?
This is normally written by /lib/udev/write_net_rules. You should be able
to modify the automatically generated one to match what you need.
K
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Back to eth shuffling ...
Actually, I know what the MAC is for the builtin Port1 and 2. Those are listed in the BIOS. But ultimately I don't want to rely on them as I want the same kickstart file to work for other machines, so hardcoding those in the kickstart file wouldn't quite work, unless I start writing multiple kickstart files, one per machine. Anyway, lspci reports this: 00:19.0 Ethernet controller: Intel Corporation 82566DM-2 Gigabit Network Connection (rev 02) 03:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06) 08:00.0 Ethernet controller: Intel Corporation 82573V Gigabit Ethernet Controller (Copper) (rev 03) Now when I look at the enumeration from dmesg, I get this: e1000e :00:19.0: eth0: (PCI Express:2.5GT/s:Width x1) 00:1e:68:58:00:4c -- this is Port1 e1000e :03:00.0: eth1: (PCI Express:2.5GT/s:Width x1) 00:15:17:d3:43:62 -- PCIe Card e1000e :08:00.0: eth2: (PCI Express:2.5GT/s:Width x1) 00:1e:68:58:00:4d -- this is Port2 So this is how it gets enumerated for some reason. But during the machine's POST, it always brings up the PXE boot order as Port1, Port2, and finally the PCIe card, which is the correct order that I want it in. It also only has one single expansion slot so it's not like I can try a different one to see if it makes a difference, unfortunately. On Thu, May 14, 2015 at 5:47 PM, Kahlil Hodgson kahlil.hodg...@dealmax.com.au wrote: On 15 May 2015 at 03:51, Ashley M. Kirchner ash...@pcraft.com wrote: After the machine boots and I look in /root/ksnet-devices, I see the MAC addresses for the devices as: Port1 - eth0 PCIe Card- eth1 Port2 - eth2 And yet, during the machine's POST (which can verify by the PXE boot up of each device), it correctly enumerates the ethernet devices as: Port1 Port2 PCIe card So where and why is the order changed when the kernel kicks in and start booting the machine? And how can I stop it, or change its behavior? I'm not seeing any 'renaming' going on in dmesg like I sometimes find. So something is causing this and I can't figure it out. Yeah, my understanding is that the kernel does not really trust all the information that the BIOS tells it and probes for the network devices itself. I'm thinking, since you are using kickstart and PXE boot, you're not going to know the HWADDR for your NICs before hand, but you probably do have a predictable pci bus layout. So grab pciutils package and run: lspci | grep Ethernet I get something like 00:0a.0 Ethernet controller: NVIDIA Corporation MCP67 Ethernet (rev a2) 01:06.0 Ethernet controller: Intel Corporation 82541PI Gigabit Ethernet Controller (rev 05) 01:07.0 Ethernet controller: Intel Corporation 82541PI Gigabit Ethernet Controller (rev 05) The first number is the bus id which is going to be consistent across machines with the same mainboard and nic layout. So I can make a persistent-net.rules file like: ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:00:0a.0, NAME=eth0 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:01:06.0, NAME=eth1 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:01:07.0, NAME=eth2 Hope this helps. K ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Back to eth shuffling ...
another identical machine will have the same bus ids. that's why this works. Kahlil (Kal) Hodgson GPG: C9A02289 Head of Technology (m) +61 (0) 4 2573 0382 DealMax Pty LtdGitHub: @tartansandal Suite 1416 401 Docklands Drive Docklands VIC 3008 Australia All parts should go together without forcing. You must remember that the parts you are reassembling were disassembled by you. Therefore, if you can't get them together again, there must be a reason. By all means, do not use a hammer. -- IBM maintenance manual, 1925 On 15 May 2015 at 11:02, Ashley M. Kirchner ash...@pcraft.com wrote: Right, I understand that part. However I believe I'm now in the realm of making this specific to this machine as I have no guarantee that another identical machine will pop up with those same bus IDs. Maybe for the internal ports, but I don't know if the same will happen for the PCIe bus. Would that be correct? On Thu, May 14, 2015 at 6:21 PM, Kahlil Hodgson kahlil.hodg...@dealmax.com.au wrote: So a 70-persistent-net.rules like # onboard port 1 - eth0 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:00:19.0, NAME=eth0 # PCIe card - eth2 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:03:00.0, NAME=eth2 # onboard port 2 - eth1 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:08:00.0, NAME=eth1 will do what you want. Note: I've just changed the ID and NAME values to match what you have and what you want. Kahlil (Kal) Hodgson GPG: C9A02289 Head of Technology (m) +61 (0) 4 2573 0382 DealMax Pty LtdGitHub: @tartansandal Suite 1416 401 Docklands Drive Docklands VIC 3008 Australia All parts should go together without forcing. You must remember that the parts you are reassembling were disassembled by you. Therefore, if you can't get them together again, there must be a reason. By all means, do not use a hammer. -- IBM maintenance manual, 1925 On 15 May 2015 at 10:12, Ashley M. Kirchner ash...@pcraft.com wrote: Actually, I know what the MAC is for the builtin Port1 and 2. Those are listed in the BIOS. But ultimately I don't want to rely on them as I want the same kickstart file to work for other machines, so hardcoding those in the kickstart file wouldn't quite work, unless I start writing multiple kickstart files, one per machine. Anyway, lspci reports this: 00:19.0 Ethernet controller: Intel Corporation 82566DM-2 Gigabit Network Connection (rev 02) 03:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06) 08:00.0 Ethernet controller: Intel Corporation 82573V Gigabit Ethernet Controller (Copper) (rev 03) Now when I look at the enumeration from dmesg, I get this: e1000e :00:19.0: eth0: (PCI Express:2.5GT/s:Width x1) 00:1e:68:58:00:4c -- this is Port1 e1000e :03:00.0: eth1: (PCI Express:2.5GT/s:Width x1) 00:15:17:d3:43:62 -- PCIe Card e1000e :08:00.0: eth2: (PCI Express:2.5GT/s:Width x1) 00:1e:68:58:00:4d -- this is Port2 So this is how it gets enumerated for some reason. But during the machine's POST, it always brings up the PXE boot order as Port1, Port2, and finally the PCIe card, which is the correct order that I want it in. It also only has one single expansion slot so it's not like I can try a different one to see if it makes a difference, unfortunately. On Thu, May 14, 2015 at 5:47 PM, Kahlil Hodgson kahlil.hodg...@dealmax.com.au wrote: On 15 May 2015 at 03:51, Ashley M. Kirchner ash...@pcraft.com wrote: After the machine boots and I look in /root/ksnet-devices, I see the MAC addresses for the devices as: Port1 - eth0 PCIe Card- eth1 Port2 - eth2 And yet, during the machine's POST (which can verify by the PXE boot up of each device), it correctly enumerates the ethernet devices as: Port1 Port2 PCIe card So where and why is the order changed when the kernel kicks in and start booting the machine? And how can I stop it, or change its behavior? I'm not seeing any 'renaming' going on in dmesg like I sometimes find. So something is causing this and I can't figure it out. Yeah, my understanding is that the kernel does not really trust all the information that the BIOS tells it and probes for the network devices itself. I'm thinking, since you are using kickstart and PXE boot, you're not going to know the HWADDR for your NICs before hand, but you probably do have a predictable pci bus layout. So grab pciutils package and run: lspci | grep Ethernet I get something like 00:0a.0 Ethernet controller: NVIDIA Corporation MCP67 Ethernet (rev a2)
Re: [CentOS] Back to eth shuffling ...
apologies. just realised I was top posting again. damn this email client :-( ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Back to eth shuffling ...
On 15 May 2015 at 03:51, Ashley M. Kirchner ash...@pcraft.com wrote: After the machine boots and I look in /root/ksnet-devices, I see the MAC addresses for the devices as: Port1 - eth0 PCIe Card- eth1 Port2 - eth2 And yet, during the machine's POST (which can verify by the PXE boot up of each device), it correctly enumerates the ethernet devices as: Port1 Port2 PCIe card So where and why is the order changed when the kernel kicks in and start booting the machine? And how can I stop it, or change its behavior? I'm not seeing any 'renaming' going on in dmesg like I sometimes find. So something is causing this and I can't figure it out. Yeah, my understanding is that the kernel does not really trust all the information that the BIOS tells it and probes for the network devices itself. I'm thinking, since you are using kickstart and PXE boot, you're not going to know the HWADDR for your NICs before hand, but you probably do have a predictable pci bus layout. So grab pciutils package and run: lspci | grep Ethernet I get something like 00:0a.0 Ethernet controller: NVIDIA Corporation MCP67 Ethernet (rev a2) 01:06.0 Ethernet controller: Intel Corporation 82541PI Gigabit Ethernet Controller (rev 05) 01:07.0 Ethernet controller: Intel Corporation 82541PI Gigabit Ethernet Controller (rev 05) The first number is the bus id which is going to be consistent across machines with the same mainboard and nic layout. So I can make a persistent-net.rules file like: ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:00:0a.0, NAME=eth0 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:01:06.0, NAME=eth1 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:01:07.0, NAME=eth2 Hope this helps. K ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Back to eth shuffling ...
So a 70-persistent-net.rules like # onboard port 1 - eth0 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:00:19.0, NAME=eth0 # PCIe card - eth2 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:03:00.0, NAME=eth2 # onboard port 2 - eth1 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:08:00.0, NAME=eth1 will do what you want. Note: I've just changed the ID and NAME values to match what you have and what you want. Kahlil (Kal) Hodgson GPG: C9A02289 Head of Technology (m) +61 (0) 4 2573 0382 DealMax Pty LtdGitHub: @tartansandal Suite 1416 401 Docklands Drive Docklands VIC 3008 Australia All parts should go together without forcing. You must remember that the parts you are reassembling were disassembled by you. Therefore, if you can't get them together again, there must be a reason. By all means, do not use a hammer. -- IBM maintenance manual, 1925 On 15 May 2015 at 10:12, Ashley M. Kirchner ash...@pcraft.com wrote: Actually, I know what the MAC is for the builtin Port1 and 2. Those are listed in the BIOS. But ultimately I don't want to rely on them as I want the same kickstart file to work for other machines, so hardcoding those in the kickstart file wouldn't quite work, unless I start writing multiple kickstart files, one per machine. Anyway, lspci reports this: 00:19.0 Ethernet controller: Intel Corporation 82566DM-2 Gigabit Network Connection (rev 02) 03:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06) 08:00.0 Ethernet controller: Intel Corporation 82573V Gigabit Ethernet Controller (Copper) (rev 03) Now when I look at the enumeration from dmesg, I get this: e1000e :00:19.0: eth0: (PCI Express:2.5GT/s:Width x1) 00:1e:68:58:00:4c -- this is Port1 e1000e :03:00.0: eth1: (PCI Express:2.5GT/s:Width x1) 00:15:17:d3:43:62 -- PCIe Card e1000e :08:00.0: eth2: (PCI Express:2.5GT/s:Width x1) 00:1e:68:58:00:4d -- this is Port2 So this is how it gets enumerated for some reason. But during the machine's POST, it always brings up the PXE boot order as Port1, Port2, and finally the PCIe card, which is the correct order that I want it in. It also only has one single expansion slot so it's not like I can try a different one to see if it makes a difference, unfortunately. On Thu, May 14, 2015 at 5:47 PM, Kahlil Hodgson kahlil.hodg...@dealmax.com.au wrote: On 15 May 2015 at 03:51, Ashley M. Kirchner ash...@pcraft.com wrote: After the machine boots and I look in /root/ksnet-devices, I see the MAC addresses for the devices as: Port1 - eth0 PCIe Card- eth1 Port2 - eth2 And yet, during the machine's POST (which can verify by the PXE boot up of each device), it correctly enumerates the ethernet devices as: Port1 Port2 PCIe card So where and why is the order changed when the kernel kicks in and start booting the machine? And how can I stop it, or change its behavior? I'm not seeing any 'renaming' going on in dmesg like I sometimes find. So something is causing this and I can't figure it out. Yeah, my understanding is that the kernel does not really trust all the information that the BIOS tells it and probes for the network devices itself. I'm thinking, since you are using kickstart and PXE boot, you're not going to know the HWADDR for your NICs before hand, but you probably do have a predictable pci bus layout. So grab pciutils package and run: lspci | grep Ethernet I get something like 00:0a.0 Ethernet controller: NVIDIA Corporation MCP67 Ethernet (rev a2) 01:06.0 Ethernet controller: Intel Corporation 82541PI Gigabit Ethernet Controller (rev 05) 01:07.0 Ethernet controller: Intel Corporation 82541PI Gigabit Ethernet Controller (rev 05) The first number is the bus id which is going to be consistent across machines with the same mainboard and nic layout. So I can make a persistent-net.rules file like: ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:00:0a.0, NAME=eth0 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:01:06.0, NAME=eth1 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:01:07.0, NAME=eth2 Hope this helps. K ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Back to eth shuffling ...
Right, I understand that part. However I believe I'm now in the realm of making this specific to this machine as I have no guarantee that another identical machine will pop up with those same bus IDs. Maybe for the internal ports, but I don't know if the same will happen for the PCIe bus. Would that be correct? On Thu, May 14, 2015 at 6:21 PM, Kahlil Hodgson kahlil.hodg...@dealmax.com.au wrote: So a 70-persistent-net.rules like # onboard port 1 - eth0 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:00:19.0, NAME=eth0 # PCIe card - eth2 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:03:00.0, NAME=eth2 # onboard port 2 - eth1 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:08:00.0, NAME=eth1 will do what you want. Note: I've just changed the ID and NAME values to match what you have and what you want. Kahlil (Kal) Hodgson GPG: C9A02289 Head of Technology (m) +61 (0) 4 2573 0382 DealMax Pty LtdGitHub: @tartansandal Suite 1416 401 Docklands Drive Docklands VIC 3008 Australia All parts should go together without forcing. You must remember that the parts you are reassembling were disassembled by you. Therefore, if you can't get them together again, there must be a reason. By all means, do not use a hammer. -- IBM maintenance manual, 1925 On 15 May 2015 at 10:12, Ashley M. Kirchner ash...@pcraft.com wrote: Actually, I know what the MAC is for the builtin Port1 and 2. Those are listed in the BIOS. But ultimately I don't want to rely on them as I want the same kickstart file to work for other machines, so hardcoding those in the kickstart file wouldn't quite work, unless I start writing multiple kickstart files, one per machine. Anyway, lspci reports this: 00:19.0 Ethernet controller: Intel Corporation 82566DM-2 Gigabit Network Connection (rev 02) 03:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06) 08:00.0 Ethernet controller: Intel Corporation 82573V Gigabit Ethernet Controller (Copper) (rev 03) Now when I look at the enumeration from dmesg, I get this: e1000e :00:19.0: eth0: (PCI Express:2.5GT/s:Width x1) 00:1e:68:58:00:4c -- this is Port1 e1000e :03:00.0: eth1: (PCI Express:2.5GT/s:Width x1) 00:15:17:d3:43:62 -- PCIe Card e1000e :08:00.0: eth2: (PCI Express:2.5GT/s:Width x1) 00:1e:68:58:00:4d -- this is Port2 So this is how it gets enumerated for some reason. But during the machine's POST, it always brings up the PXE boot order as Port1, Port2, and finally the PCIe card, which is the correct order that I want it in. It also only has one single expansion slot so it's not like I can try a different one to see if it makes a difference, unfortunately. On Thu, May 14, 2015 at 5:47 PM, Kahlil Hodgson kahlil.hodg...@dealmax.com.au wrote: On 15 May 2015 at 03:51, Ashley M. Kirchner ash...@pcraft.com wrote: After the machine boots and I look in /root/ksnet-devices, I see the MAC addresses for the devices as: Port1 - eth0 PCIe Card- eth1 Port2 - eth2 And yet, during the machine's POST (which can verify by the PXE boot up of each device), it correctly enumerates the ethernet devices as: Port1 Port2 PCIe card So where and why is the order changed when the kernel kicks in and start booting the machine? And how can I stop it, or change its behavior? I'm not seeing any 'renaming' going on in dmesg like I sometimes find. So something is causing this and I can't figure it out. Yeah, my understanding is that the kernel does not really trust all the information that the BIOS tells it and probes for the network devices itself. I'm thinking, since you are using kickstart and PXE boot, you're not going to know the HWADDR for your NICs before hand, but you probably do have a predictable pci bus layout. So grab pciutils package and run: lspci | grep Ethernet I get something like 00:0a.0 Ethernet controller: NVIDIA Corporation MCP67 Ethernet (rev a2) 01:06.0 Ethernet controller: Intel Corporation 82541PI Gigabit Ethernet Controller (rev 05) 01:07.0 Ethernet controller: Intel Corporation 82541PI Gigabit Ethernet Controller (rev 05) The first number is the bus id which is going to be consistent across machines with the same mainboard and nic layout. So I can make a persistent-net.rules file like: ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:00:0a.0, NAME=eth0 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:01:06.0, NAME=eth1 ACTION==add, SUBSYSTEM==net, BUS==pci, ID==:01:07.0, NAME=eth2 Hope this helps. K ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] Back to eth shuffling ...
So I'm back to this problem. A quick run down of what the original problem was: I have a machine that I'm configuring to use kickstart to setup. It has two builtin ethernet ports (labeled ports 1 and 2) and I'm adding a third one on its PCIe bus. Originally I was using an r8169 clone a default kickstart always put it as eth0 with the builtin ones as eth1 and eth2 respectively. After some fiddling with blacklisting the driver during the kickstart process, I was able to get the machine to boot up with the r8169 as eth3 (!!!) while the other two were correctly identified as eth0 and eth1 (this is what I want except the PCIe card should've been eth2 but it never came up that way) However I ran into other issues with the r8169 driver and ultimately replaced it for an IntelPRO, which happens to use the same e1000e module as the builtin ones. Great. Except ... Now when I kickstart the machine, I get this: builtin port1 - eth0 PCIe card - eth1 builtin port2 - eth2 What I want is: builtin port1 - eth0 builtin port2 - eth1 PCIe card - eth2 I should also point out here that when PXE kicks in on this machine, the machine correctly enumerates the ethernet devices as port1, port2 and then the PCIe card. So somewhere in the kickstart process things change. So I'm back to trying to figure out how to achieve this. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Back to eth shuffling ...
Have you tried having kickstart set up a more appropriate /etc/udev/rules/70-persistent-net.rules? This is normally written by /lib/udev/write_net_rules. You should be able to modify the automatically generated one to match what you need. K ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
