[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[MAAS Lander]

** Changed in: maas/1.9
   Status: In Progress => Fix Committed

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Blake Rouse]

** Changed in: maas
   Status: Triaged => In Progress

** Also affects: maas/1.9
   Importance: Undecided
   Status: New

** Changed in: maas/1.9
   Status: New => In Progress

** Changed in: maas/1.9
Milestone: None => 1.9.3

** Changed in: maas/1.9
   Importance: Undecided => High

** Changed in: maas
   Importance: Critical => High

** Changed in: maas
 Assignee: (unassigned) => Blake Rouse (blake-rouse)

** Changed in: maas/1.9
 Assignee: (unassigned) => Trent Lloyd (lathiat)

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Launchpad Bug Tracker]

** Branch linked: lp:~blake-rouse/maas/fix-1521618

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[james beedy]

Has a fix for this been implemented?

This still seems to exist in MAAS-2.0 beta6+bzr5017

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Launchpad Bug Tracker]

** Branch linked: lp:~lathiat/maas/1.9-lp1521618-dhcp-incorrect-router

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Trent Lloyd]

Full patch and explanation here:
https://code.launchpad.net/~lathiat/maas/1.9-lp1521618-dhcp-incorrect-router

With multiple subnets, PXE DHCP clients would receive the router of the first
subnet in the configuration file regardless of which subnet they actually
received a lease from.

This happens because match sections such as "class" (used here to give PXE
clients a smaller lease time) and "host" (not currently used, but is the
example given in the below e-mail) are not actually scoped within the subnet{}
declaration as you would expect. They are actually defined in the global scope
for all subnets, but also inherit options from the subnet{} declaration they
were defined in such as "option routers".

This strange behavior is explained in the following email:
https://lists.isc.org/pipermail/dhcp-users/2011-September/014001.html

We fix this by moving the conditional declarations out into the global scope,
instead of repeating them for each subnet. This way they do not inherit any
options from the subnet scope.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Paolo de Rosa]

We hit this bug also with DT and we noticed that moving  conditional options 
(those below) in the global section solve the problem, so we worked around 
changing dhcp.conf template.
   
if option arch = 00:0E {
 filename "pxelinux.0";
  option path-prefix "ppc64el/";
   } elsif option arch = 00:07 {
  filename "bootx64.efi";
   } elsif option arch = 00:0B {
  filename "grubaa64.efi";
   } elsif option arch = 00:0C {
  filename "bootppc64.bin";
   } else {
  filename "pxelinux.0";
}
class "PXE" {
   match if substring (option vendor-class-identifier, 0, 3) = "PXE";
default-lease-time 30;
max-lease-time 30;
}

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Blake Rouse]

Okay I have played around with this even more and its just not going to
work without fixing isc-dhcp itself. Here is an example configuration,
where the gateway is set in one shared-network and not in the other.
When the machine boots it gets the gateway from the other shared-
network.

http://paste.ubuntu.com/15273844/

Now if you re-order the configuration and place the one without the
gateway first it works as expected. But this will not work as the
ordering can never correct as described in my previous comment.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Blake Rouse]

Okay so what needs to be done to fix this issue is to order the subnets
in the generated configuration where the ones without gateways come
first and the ones with gateways follow. This causes isc-dhcp to work
correctly. This might be an issue with shared-network because the order
of the subnets can only go so far before the order of shared-network
breaks it. That is to say if there are 2 shared networks each with 2
subnets and one of each have a gateway defined then one of the 4 will
have an issue as the order in that case cannot be enforced.

Note for next-server:
You also need to set next-server on each subnet to make sure that it can 
communicate back to the rack controller on that subnet. If not the PXE client 
will select the IP from where it recieved the DHCP response which can be a 
different subnet.

I think it might be best to say you can only provide DHCP to subnets
that a rack controller has an IP address in. Without it the machines
will fail to PXE boot. Or we don't enforce it which is legal and might
be a network configuration that the administrator wants, but might be
surprising to other users of MAAS.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Blake Rouse]

** Changed in: maas
   Importance: Medium => Critical

** Changed in: maas
Milestone: None => 2.0.0

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Mike Pontillo]

As an aside, Zoltan, I'm curious what symptoms you see due to this
issue. Since the gateway is off-link, it should not be reachable from
the provisioning network. What errors occur in your environment due to
this bug?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Mike Pontillo]

Ah, disregard my previous question. When I re-read this bug I missed the
part where you said "So instead of picking up the /16 subnet correctly
for the 10.6.239.3 IP, it picks up the /24 from the network where it
gets it's default gateway from."

** Changed in: isc-dhcp (Ubuntu)
   Status: New => Confirmed

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Zoltan Arnold Nagy]

In the meantime I had to change the environment to get on with the
project so I don't have this setup anymore, but I could try to reproduce
it on a VMware environment later this week if you guys couldn't.

This is on trusty running isc-dhcp-server 4.2.4-7ubuntu12.3, with maas-
dhcp 1.9.0~rc2+bzr4509-0ubuntu1~trusty1

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Zoltan Arnold Nagy]

Mike, the symptom was that the PXE boot just hang as can bee seen from
the packet capture. No idea why actually, but the first thing fishy was
the wrong DHCP reply :)

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[LaMont Jones]

I'm having trouble reproducing this...

What version of isc-dhcp-server is running here?

Does it change anything if you (manually) reorder the 2 subnets in
dhcpd.conf and restart the dhcp server?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Mike Pontillo]

Ah, thanks for all the information. (I didn't realize you were pasting
the kernel parameter.)

After researching this problem, I think this definitely looks like a bug
in isc-dhcp. (The other possibility is that MAAS is configuring dhcpd
incorrectly in this situation, but so far it looks like our
configuration is correct, but dhcpd is interpreting it incorrectly.)

I noticed that on Trusty we are using 4.2.4-7ubuntu12.3, while on Xenial
we are using 4.3.1-5ubuntu4. 4.3.1-5ubuntu4. The latest "Extended
support" version from ISC seems to be 4.3.3.[1]

To move forward, we'll need to further triage this to see if the bug
occurs on other versions of dhcpd.

Meanwhile, I think you should be able to work around this issue by
changing your hosts' network configuration after commissioning. You can
configure MAAS to disable the boot interface upon deployment, so that
your provisioning network will only be used for the initial PXE boot.

[1]: https://www.isc.org/downloads/

** Changed in: maas
   Status: Incomplete => Triaged

** Changed in: maas
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Zoltan Arnold Nagy]

Adding the packet capture. The DHCP reply clearly has the wrong subnet;
after that I see it starts transferring the file over tftp so not sure
why it gets stuck.

** Attachment added: "failed_maas_pxe.pcap"
   
https://bugs.launchpad.net/maas/+bug/1521618/+attachment/4529431/+files/failed_maas_pxe.pcap

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Zoltan Arnold Nagy]

Attaching the screenshot from the console of the server.

** Attachment added: "pxe_boot_screenshot.jpg"
   
https://bugs.launchpad.net/maas/+bug/1521618/+attachment/4529430/+files/pxe_boot_screenshot.jpg

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Zoltan Arnold Nagy]

Mike, I think the syntax of that line is the following:

ip=

(from the kernel's documentation)

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[LaMont Jones]

Can we also get the output of the following on the cluster controller:
ip addr list; ip route list

thanks

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Zoltan Arnold Nagy]

I will be able to do the tcpdumps a bit later; in the meantime, here are
the requested outputs:

root@maas:~# ip addr list; ip route list
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group 
default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
   valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
   valid_lft forever preferred_lft forever
2: eth0:  mtu 1500 qdisc mq state UP group 
default qlen 1000
link/ether 00:50:56:bf:29:c4 brd ff:ff:ff:ff:ff:ff
inet 9.4.113.251/24 brd 9.4.113.255 scope global eth0
   valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:febf:29c4/64 scope link
   valid_lft forever preferred_lft forever
3: eth1:  mtu 1500 qdisc mq state UP group 
default qlen 1000
link/ether 00:50:56:bf:a4:48 brd ff:ff:ff:ff:ff:ff
inet 10.6.250.250/16 brd 10.6.255.255 scope global eth1
   valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:febf:a448/64 scope link
   valid_lft forever preferred_lft forever
4: eth2:  mtu 1500 qdisc mq state UP group 
default qlen 1000
link/ether 00:50:56:bf:f1:71 brd ff:ff:ff:ff:ff:ff
inet 10.7.250.250/16 brd 10.7.255.255 scope global eth2
   valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:febf:f171/64 scope link
   valid_lft forever preferred_lft forever
default via 9.4.113.254 dev eth0
9.4.113.0/24 dev eth0  proto kernel  scope link  src 9.4.113.251
10.6.0.0/16 dev eth1  proto kernel  scope link  src 10.6.250.250
10.7.0.0/16 dev eth2  proto kernel  scope link  src 10.7.250.250
root@maas:~#

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Mike Pontillo]

Thanks for taking the time to report a bug in MAAS.

Can you give us more information on this line in your bug report:

ip=10.6.239.3:10.6.250.250:9.4.113.254:255.255.255.0

Is this from a packet capture? (if so, on which interface?) I'm trying
to figure out what this means. In it you have:

10.6.239.3 <-- seems to be an IP address within "range dynamic-bootp 10.6.239.0 
10.6.239.239"
10.6.250.250 <-- seems to be another IP address within your /16
9.4.113.254 <-- router
255.255.255.0 <-- /24

It's interesting that dhcpd would behave in this way; it seems to be
picking up the "option routers" from a subnet it didn't select. It looks
like this may be a dhcpd bug...

** Also affects: isc-dhcp (Ubuntu)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1521618] Re: wrong subnet in DHCP answer when multiple networks are present

[Mike Pontillo]

Would it be possible for you to capture the DHCP packets (for example,
using Wireshark) and attach them to the bug? Thanks!

** Changed in: maas
   Status: New => Incomplete

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1521618

Title:
  wrong subnet in DHCP answer when multiple networks are present

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1521618/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs