Send kea-dev mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.isc.org/mailman/listinfo/kea-dev
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of kea-dev digest..."
Today's Topics:
1. Re: DHCP Hackaton summary (Tomek Mrugalski)
2. Re: DHCP Hackaton summary (Tomek Mrugalski)
3. Re: DHCP Hackaton summary (Marcin Siodelski)
4. Re: DHCP Hackaton summary (Francis Dupont)
5. Re: DHCP Hackaton summary (Thomas Markwalder)
6. Re: DHCP Hackaton summary (Thomas Markwalder)
7. Re: Possibility of disabling raw socket use in Kea ? ->
works! (Chaigneau, Nicolas)
----------------------------------------------------------------------
Message: 1
Date: Wed, 24 Sep 2014 14:16:08 +0200
From: Tomek Mrugalski <[email protected]>
To: Marcin Siodelski <[email protected]>, "[email protected]"
<[email protected]>
Subject: Re: DHCP Hackaton summary
Message-ID: <[email protected]>
Content-Type: text/plain; charset=windows-1252
On 24.09.2014 13:35, Marcin Siodelski wrote:
> So, I'd really like to see the host reservations in a separate table
> (see attachment for the proposed database layout). The only reservation
> that may remain in the Hosts table is an IPv4 reservation (or NULL).
> Technically, it could also be moved to the reservations table but since
> IPv4 address is only 4 bytes long it should be rather stored in a column
> of a different type than the IPv6 address or prefix. So, for performance
> reasons it doesn't make much sense to store IPV4 address as a text in
> the same column as IPv6 address.
>
> I also point out that the entry in the table with reservations doesn't
> make sense on its own (Identifying Relationship) so there should
> probably be triggers on the table with hosts, which removes the
> reservations from the reservations table before the DELETE operation on
> the Hosts table. This way, removal of the host entry would trigger
> removal of all reservations.
>
> In addition to the fields that Tomek listed for the options table, I
> added one called "persistent" which carries a boolean value and
> identifies an option as a one that should always be returned to the
> client or the one that is returned only if requested (using ORO or PRL).
Yup, persistent options are useful.
> I am working on documenting the design for host reservation and I am
> planning to use the attached (or similar) database layout for this
> design, unless I hear objections.
You asked, so here it is :)
I think I heard only once that someone asked for reserving more than one
address or prefix. Even though the protocol allows that, great majority
of users are reserving just one address and/or prefix. So I think that
particular use case should be as simple to use as possible. So how about
tweaking your proposal with a little twist: keep the single address
field in the Hosts table and allow additional ones to be stored in
ipv6_reservations if needed. I *think*, but I'm not sure, that with that
approach we still can enforce address uniqueness. Maybe we can do a view
from two columns from two tables (options.ip6_address and
ipv6_reservations.address) and check/enforce uniqueness there?
On a related note, depending on the address uniqueness is perhaps not
the best strategy. More than one user talked to us and said about the
need to store multiple instances of the same subnet (e.g. you have 10
hotspots and each of them is handling the same 10.0.0.0/8 subnet).
Leases would need to not be unique. What would be unique is (address,
subnet_id) tuple.
This is not a requirement we thought about when we were creating MySQL
or Postgres database layouts. But it is something we will have to
support one day. My only point wrt host reservation is to not make
matters worse here.
Tomek
------------------------------
Message: 2
Date: Wed, 24 Sep 2014 14:27:47 +0200
From: Tomek Mrugalski <[email protected]>
To: [email protected]
Subject: Re: DHCP Hackaton summary
Message-ID: <[email protected]>
Content-Type: text/plain; charset=windows-1252
On 22.09.2014 15:50, Tomek Mrugalski wrote:
> Here's the DHCP Hackaton summary. These were the discussions that I
> participated in. Couple times several things were happening in parallel,
> so I'll let others to fill in the blanks.
Oh, I forgot about two interesting topics that were discussed.
First, Dominik mentioned an interesting feature that was recently added
to Linux kernel. It is apparently possible for multiple processes to
bind to the same address/port. It works similar to REUSE_ADDR, but only
one process gets the packet, not all of them. If this is indeed working
as expected, we could start doing multi-process experiments much sooner
that we initially thought.
Another discussion was about raw sockets in DHCPv4. Our implementation
is working, but raw sockets are cumbersome to use. There are complains
raised already. One user was displeased that iptables are not working.
Indeed, they can't as raw sockets essentially bypass the kernel stack.
This is a limitation (or rather a property) of raw sockets and not
something we could fix in the code.
Also, two possible alternatives were mentioned: libraries that offer
more or less a user-land IP stack. The first one is called DPDK and is
backed by Intel. The other one is called netmap. Both are used for high
performance networking. Netmap webpage claims ability to handle wire
speed of a 10G NIC. That's almost 15 millions of packets per seconds. Yikes!
It is unclear how big the dependency would be. If we ever decide to add
support for one (or both) of those libraries, it will have to be an
optional dependency.
Links:
http://dpdk.org/
http://info.iet.unipi.it/~luigi/netmap/
On a related note, we may want to add support for plain UDP sockets in
DHCPv4. That would work great for users who receive only relayed traffic
and want to use iptables. In fact, the code is already there, we just
need a knob to switch between raw sockets and UDP.
Tomek
------------------------------
Message: 3
Date: Wed, 24 Sep 2014 14:48:22 +0200
From: Marcin Siodelski <[email protected]>
To: Tomek Mrugalski <[email protected]>, "[email protected]"
<[email protected]>
Subject: Re: [kea-dev] DHCP Hackaton summary
Message-ID: <[email protected]>
Content-Type: text/plain; charset=windows-1252
On 24/09/14 14:16, Tomek Mrugalski wrote:
> On 24.09.2014 13:35, Marcin Siodelski wrote:
>> So, I'd really like to see the host reservations in a separate table
>> (see attachment for the proposed database layout). The only reservation
>> that may remain in the Hosts table is an IPv4 reservation (or NULL).
>> Technically, it could also be moved to the reservations table but since
>> IPv4 address is only 4 bytes long it should be rather stored in a column
>> of a different type than the IPv6 address or prefix. So, for performance
>> reasons it doesn't make much sense to store IPV4 address as a text in
>> the same column as IPv6 address.
>>
>> I also point out that the entry in the table with reservations doesn't
>> make sense on its own (Identifying Relationship) so there should
>> probably be triggers on the table with hosts, which removes the
>> reservations from the reservations table before the DELETE operation on
>> the Hosts table. This way, removal of the host entry would trigger
>> removal of all reservations.
>>
>> In addition to the fields that Tomek listed for the options table, I
>> added one called "persistent" which carries a boolean value and
>> identifies an option as a one that should always be returned to the
>> client or the one that is returned only if requested (using ORO or PRL).
> Yup, persistent options are useful.
>
>> I am working on documenting the design for host reservation and I am
>> planning to use the attached (or similar) database layout for this
>> design, unless I hear objections.
> You asked, so here it is :)
>
So rapid response indicates that requests for design review should
always end with the clause "... unless I hear objections". :-)
> I think I heard only once that someone asked for reserving more than one
> address or prefix. Even though the protocol allows that, great majority
> of users are reserving just one address and/or prefix. So I think that
> particular use case should be as simple to use as possible. So how about
> tweaking your proposal with a little twist: keep the single address
> field in the Hosts table and allow additional ones to be stored in
> ipv6_reservations if needed. I *think*, but I'm not sure, that with that
> approach we still can enforce address uniqueness. Maybe we can do a view
> from two columns from two tables (options.ip6_address and
> ipv6_reservations.address) and check/enforce uniqueness there?
I don't think that this tweak makes it any easier. First, you need to
duplicate the fields in two tables: IPv6 address and IPv6 prefix in one
table and another table for additional addresses/prefixes. So, the
database becomes more complex. It makes it more complicated to perform
statistical/aggregated queries because you'll need to look at two
columns in two tables. If you just had one place where you keep the V6
reservation the query to retrieve all reservations for a host is as
simple as:
SELECT h.ipv4_address, r.address
FROM hosts AS h LEFT JOIN ipv6_reservations AS r
ON h.host_id = r.host_id
WHERE h.dhcp_identifier = "0x0102030405";
Each returned entry represents one address that you stuff into the
IA_NA/IA_PD sent to a client so you just have to iterate over the
returned rows.
Whereas if you had two fields where you keep addresses and prefixes your
application layer would need to combine the results and take one address
from one table and the rest from another.
Moreover, if you put addresses into secondary table only if you have
them, it doesn't mean that you don't have to query for them the way I
showed above. The only difference is that the NULL result indicates that
there is only one address in the Host table.
So, I am not sure that this tweak really makes things easier but I am
sure it puts additional constraints and creates redundancy.
Also, how do you actually know that admins don't configure their
networks to provide more than one address to a single client? Even if
this is the case most of the time, RFC accounts for this case, and
implementation should not put constraints on this.
The stateful issues draft in DHCWG is all about providing means to
obtain multiple bindings in a single transaction not only for single
address + prefix but in general. I don't think that we should go against
this with Kea.
>
> On a related note, depending on the address uniqueness is perhaps not
> the best strategy. More than one user talked to us and said about the
> need to store multiple instances of the same subnet (e.g. you have 10
> hotspots and each of them is handling the same 10.0.0.0/8 subnet).
> Leases would need to not be unique. What would be unique is (address,
> subnet_id) tuple.
I heard about this case earlier. But, it is an example from the IPv4
world which doesn't really count for IPv6. Does it?
But anyway, it indicates that the Hosts table needs to store the
information about the subnet_id that the V4 client belongs to.
>
> This is not a requirement we thought about when we were creating MySQL
> or Postgres database layouts. But it is something we will have to
> support one day. My only point wrt host reservation is to not make
> matters worse here.
I agree with this, so maybe the database design should account for
unique tuples of addresses/subnet id (assuming that a single client
belongs to a single subnet id).
Marcin
------------------------------
Message: 4
Date: Wed, 24 Sep 2014 13:36:42 +0000
From: Francis Dupont <[email protected]>
To: Tomek Mrugalski <[email protected]>
Cc: [email protected]
Subject: Re: [kea-dev] DHCP Hackaton summary
Message-ID: <[email protected]>
Tomek Mrugalski writes:
> First, Dominik mentioned an interesting feature that was recently added
> to Linux kernel. It is apparently possible for multiple processes to
> bind to the same address/port. It works similar to REUSE_ADDR, but only
> one process gets the packet, not all of them. If this is indeed working
> as expected, we could start doing multi-process experiments much sooner
> that we initially thought.
=> we had a discussion about this for bind9. BTW I don't believe it will
really help for KEA because the bottleneck is not the packet processing
but the lease database (or worse, file :-) update. Anyway it is still
useful for stateless exchange and it can be simulated with a load
sharing feature...
> Another discussion was about raw sockets in DHCPv4. Our implementation
> is working, but raw sockets are cumbersome to use. There are complains
> raised already. One user was displeased that iptables are not working.
> Indeed, they can't as raw sockets essentially bypass the kernel stack.
=> iptables are before the kernel stack so this reason is not enough
(but there are other reasons so I agree to not reject the request).
Regards
Francis Dupont <[email protected]>
------------------------------
Message: 5
Date: Wed, 24 Sep 2014 10:05:52 -0400
From: Thomas Markwalder <[email protected]>
To: [email protected]
Subject: Re: [kea-dev] DHCP Hackaton summary
Message-ID: <[email protected]>
Content-Type: text/plain; charset=ISO-8859-1
On 9/24/14, 7:47 AM, Marcin Siodelski wrote:
> I was thinking about organizing the data in a database a bit more and I
> feel like the reserved IPv6 addresses/prefixes should not be stored as a
> list but they should rather be stored in a separate table and reference
> the Host table (1:n) - just like we decided for options. Storing the
> reservations as a list in a single field will have numerous pitfalls
> listed here in no particular order:
>
> - No easy way to search a reservation for a particular IP address or
> prefix unless you use SQL functions searching by substring
> - No way to use the indexes per-address because they would point to a
> list of addresses. Note that indexes play important role for improving
> performance of queries.
> - No way to enforce uniqueness of the addresses in the database
> - Number of addresses is constrained by the number of bytes allocated
> for a string or a blob holding a list
> - Complex notation of the prefix, as it needs to be encoded with its length.
> - Can't delete a single reservation without fetching all addresses
> - It is hard to count reserved resources (for monitoring/statistical
> purposes)
> - It is hard to query for all reserved addresses
> - It complicates life of implementors of future tools to manage
> reservations as these tools would have to handle complex data structures
> within a particular data field
>
> So, I'd really like to see the host reservations in a separate table
> as depicted here
> http://kea.isc.org/attachment/wiki/HostReservationDesign/kea_host_reservation_db.png.
> The only reservation that may remain in the Hosts table is an IPv4
> reservation (or NULL).Technically, it could also be moved to the
> reservations table but since IPv4 address is only 4 bytes long it should
> be rather stored in a column of a different type than the IPv6 address
> or prefix. So, for performance reasons it doesn't make much sense to
> store IPV4 address as a text in the same column as IPv6 address.
>
> I also point out that the entry in the table with reservations doesn't
> make sense on its own (Identifying Relationship) so there should
> probably be triggers on the table with hosts, which removes the
> reservations from the reservations table before the DELETE operation on
> the Hosts table. This way, removal of the host entry would trigger
> removal of all reservations.
Marcin is correct. The ipv6 values should be in their own table. You do
not need a trigger to delete reservations when a host is deleted, rather
use a cascading delete, standard SQL construct:
http://www.mysqltutorial.org/mysql-on-delete-cascade/
Thomas
------------------------------
Message: 6
Date: Wed, 24 Sep 2014 10:13:31 -0400
From: Thomas Markwalder <[email protected]>
To: [email protected]
Subject: Re: [kea-dev] DHCP Hackaton summary
Message-ID: <[email protected]>
Content-Type: text/plain; charset=ISO-8859-1
On 9/24/14, 8:16 AM, Tomek Mrugalski wrote:
> On 24.09.2014 13:35, Marcin Siodelski wrote:
>> So, I'd really like to see the host reservations in a separate table
>> (see attachment for the proposed database layout). The only reservation
>> that may remain in the Hosts table is an IPv4 reservation (or NULL).
>> Technically, it could also be moved to the reservations table but since
>> IPv4 address is only 4 bytes long it should be rather stored in a column
>> of a different type than the IPv6 address or prefix. So, for performance
>> reasons it doesn't make much sense to store IPV4 address as a text in
>> the same column as IPv6 address.
>>
>> I also point out that the entry in the table with reservations doesn't
>> make sense on its own (Identifying Relationship) so there should
>> probably be triggers on the table with hosts, which removes the
>> reservations from the reservations table before the DELETE operation on
>> the Hosts table. This way, removal of the host entry would trigger
>> removal of all reservations.
>>
>> In addition to the fields that Tomek listed for the options table, I
>> added one called "persistent" which carries a boolean value and
>> identifies an option as a one that should always be returned to the
>> client or the one that is returned only if requested (using ORO or PRL).
> Yup, persistent options are useful.
>
>> I am working on documenting the design for host reservation and I am
>> planning to use the attached (or similar) database layout for this
>> design, unless I hear objections.
> You asked, so here it is :)
>
> I think I heard only once that someone asked for reserving more than one
> address or prefix. Even though the protocol allows that, great majority
> of users are reserving just one address and/or prefix. So I think that
> particular use case should be as simple to use as possible. So how about
> tweaking your proposal with a little twist: keep the single address
> field in the Hosts table and allow additional ones to be stored in
> ipv6_reservations if needed. I *think*, but I'm not sure, that with that
> approach we still can enforce address uniqueness. Maybe we can do a view
> from two columns from two tables (options.ip6_address and
> ipv6_reservations.address) and check/enforce uniqueness there?
Sorry Tomek, but that's an ugly hack based on "premature optimization".
It is cleaner and simpler to go with the second table. It is a 1-to-M
relationship, you should build it that way from the start. That way all
logic assumes more than one, so having only 1 isn't an exceptional case.
Thomas
------------------------------
Message: 7
Date: Wed, 24 Sep 2014 15:47:32 +0000
From: "Chaigneau, Nicolas" <[email protected]>
To: Tomek Mrugalski <[email protected]>, Marcin Siodelski
<[email protected]>, "[email protected]" <[email protected]>
Subject: Re: [kea-dev] Possibility of disabling raw socket use in Kea
? -> works!
Message-ID:
<ab94b0b675bdf14189cd5a861db36c8414197...@de-cm-mbx26.corp.capgemini.com>
Content-Type: text/plain; charset="us-ascii"
Hello,
I've rebuilt Kea with the code change you described.
In file:
src/bin/dhcp4/dhcp4_srv.h
The following modification was applied to Dhcpv4Srv constructor:
// const bool direct_response_desired = true);
const bool direct_response_desired = false);
And I'm quite happy with the results, it works perfectly :)
I ran the following test cases:
1) Started Kea on an interface with a single IP address (no iptables filtering
set up initially)
- Sent a unicast DHCP request from a relay to Kea
-> Request is correctly received and handled by Kea. Client gets a response.
- Applied a simple filtering rule through iptables:
iptables -I INPUT -p udp --dport 67:68 --sport 67:68 -j DROP
- Sent a unicast DHCP request from a relay to Kea
-> iptables drops the packet. Kea does not receive anything. Client does not
get a response.
2) Started Kea on an interface with two IP addresses (no iptables filtering is
set up initially)
- Noticed the following warning log:
2014-09-24 17:18:51.134 WARN [kea-dhcp4.dhcpsrv/10731]
DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: Binding socket to an interface
is not supported on this OS; therefore only one socket listening to broadcast
traffic can be opened. Sockets will not be opened on remaining interfaces
(which could be improved to be more informative, when (if) disabling raw socket
becomes configurable. But that's a minor detail.)
- Sent a unicast DHCP request from a relay to Kea (using either IP address)
-> Request is correctly received and handled by Kea. Client gets a response.
- Applied iptables filtering, as previously
- Sent a unicast DHCP request from a relay to Kea (using either IP address)
-> iptables drops the packet. Kea does not receive anything. Client does not
get a response.
Regards,
Nicolas.
> On 22/09/14 12:13, Marcin Siodelski wrote:
> > The code in Kea is prepared to switch between the use of raw sockets
> > and regular datagram sockets. But, currently the selection is
> > hardcoded and there is no configuration parameter to control this
> > selection by the administrator. We're now going through some
> > refactoring of the configuration code, so once this is done we can easily
> > implement the switch.
> For the time being, you can do an experiment. It requires a minor change to
> the source code.
>
> The specific socket handling objects are called PktFilterLPF (raw
> sockets) PktFilterInet(udp sockets). They are initialized in
> IfaceMgr::setMatchingPacketFilter in iface_mgr_linux.cc in src/lib/dhcp
> directory. That is called from Dhcpv4Srv constructor, which is in turn
> controlled by direct_response_desired flag. Its default value is set to true.
> If you edit it (line 91 in src/lib/dhcp/dhcp4_srv.h) to false and recompile,
> it is possible that the code will be able to successfully use UDP sockets.
>
> Disclaimer: It's a quick hack. I haven't done any experiments with this, so
> it may break down. We can't apply it, as most people are interested in direct
> traffic, so a proper switch (either compile time or run time) is needed.
> That's something that we can't do immediately.
>
> Tomek
>
>
This message contains information that may be privileged or confidential and is
the property of the Capgemini Group. It is intended only for the person to whom
it is addressed. If you are not the intended recipient, you are not authorized
to read, print, retain, copy, disseminate, distribute, or use this message or
any part thereof. If you receive this message in error, please notify the
sender immediately and delete all copies of this message.
------------------------------
_______________________________________________
kea-dev mailing list
[email protected]
https://lists.isc.org/mailman/listinfo/kea-dev
End of kea-dev Digest, Vol 6, Issue 18
**************************************
