Send kea-dev mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.isc.org/mailman/listinfo/kea-dev
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of kea-dev digest..."
Today's Topics:
1. Re: Client classification design (Tomek Mrugalski)
2. Re: Client classification design (Tomek Mrugalski)
3. Re: Client classification design (Tomek Mrugalski)
4. Kea manuals in HTML online (Jeremy C. Reed)
5. Re: Client classification design (Tomek Mrugalski)
----------------------------------------------------------------------
Message: 1
Date: Thu, 15 Oct 2015 18:56:29 +0200
From: Tomek Mrugalski <[email protected]>
To: [email protected]
Subject: Re: [kea-dev] Client classification design
Message-ID: <[email protected]>
Content-Type: text/plain; charset=windows-1252
Thanks a lot for your comments. See my responses inline.
On 14.10.2015 13:39, Marcin Siodelski wrote:
> On 13.10.2015 17:31, Tomek Mrugalski wrote:
>> Folks,
>> I took a first stab at the client classification design. See the
>> document here: http://kea.isc.org/wiki/ClientClassificationDesign
>>
>> For the reference, we have the requirements more or less described here:
>> http://kea.isc.org/wiki/ClientClassificationRequirements.
>>
>> We had some discussions about the syntax, but I did think about it a bit
>> more and came up with three alternative proposals. Please comment on
>> which one seems to be the best in your opinion (or reject all of them
>> and propose a fourth one). I tried to be objective and not push for any
>> specific one.
>>
>> Keep in mind that we're working under very tight schedule here and
>> client classification is a broad and complex topic. There's a lot we
>> could do, but due to the time constraints, we need to be very selective
>> for phase 1 (aka Kea 1.0).
>>
>> Some parts of the design will likely be moved to the requirements page,
>> but I want to keep it in the new doc for now, so you could review it
>> more easily as one chunk.
>>
>> It would be great if you could provide your comments by end of this week
>> (Oct. 16th), so we could start an implementation the next week.
>>
>> Thoughts? Comments? Tomatoes?
>>
> First of all, let me say that it is nice and comprehensive document. It
> does lack some of the details but given the amount of time you had to
> write it up, and the time constraints for this work overall, you've done
> pretty good job. The code snippets are valuable addition.
Thanks.
> It would be good to extend this document with some text describing what
> bits and pieces go where in the code structure. For example, are you
> going to create a separate library for client classification or put it
> into the libdhcpsrv. I feel the client classification is going to be a
> large feature and it actually deserves its own place in the code. The
> libdhcpsrv is already bloated. Also, the future implementation of the
> client and the relay may benefit from this.
Good idea. Added.
> It would be nice to have a list of all tasks to be performed to
> implement the limited client classification for 1.0. These tasks would
> be converted to actual tickets once the design is approved. For now it
> would serve to the reviewers as an indicator what exactly has to be done
> to meet our 1.0 goals.
Added a list of tasks. It's depressingly long. Currently has 16 tickets.
All of them are necessary for *basic* client classification to be usable.
> Apart from implementing new classes, what are the changes needed to
> Option class and classes derived from it? What are the requirements for
> new Option classes to be implemented in the future?
>
> I presume that expressions will be converted to RPNs in the server
> configuration time. But then, where would those RPNs be held until the
> next configuration or server shutdown? Is this CfgMgr, SrvConfig class,
> Subnet class?
There's a new section that explains how the lib will be plugged into
existing code.
> It would be helpful to see some performance considerations here. For
> example: how are we going to mitigate string conversions? Is there any
> plan for mitigating the conversion of the whole option to string if only
> specific field is of interest?
Added, but it's likely not what you'd like to see. The bottom line is
that we can do some tweaks, but if you need high performance and lots of
classes, our recommendation will be to write dedicated hooks. (it's
possible and quite easy to do classification in hooks, even now).
> Example Use Cases -> Use case 1
> From what I have seen the following condition:
> match if substring (option vendor-class-identifier, 0, 9) = "PXEClient";
> is a typical one to check if the client is a PXE client (apart from
> other use scenarios for it). So, I think this should be implemented in
> phase1.
Updated to be phase 1.
> Design Assumptions -> General requirements
>
> Some of the listed assumptions are not candidates for moving into the
> requirements document, e.g. G.1, G.4, G.6 as they are vague and
> unmeasurable. However, this is not to say that they are wrong as design
> assumptions.
Reorganized the whole thing. Former requirements page is now called
ClientClassificationDiscussion. I wrote the requirements page from
scratch. Most of the general requirements are there, except those you
mentioned. They stayed in the design page.
> Design Assumptions -> Expressions:
>
> Extracting option values from the vendor-class-identifier (60) must be
> supported. In particular you should be able to do this:
>
> match if substring (option vendor-class-identifier, 0, 9) = "PXEClient";
Added.
> Design Assumptions -> Functionality
>
> There is more to this:
> F.3 It MUST be possible to override the options defined for a given
> subnet, with the options defined for a given class.
Due to text reorg, I couldn't find F.3 anymore. Anyway, I reworded
existing requirements and added an new one (F.3) that says explicitly
that if there's a generic option (for all clients) and there's another
option specific for a class, the server must use value specified for the
class, not the generic one.
> Note that F.3. is different than F.1. because F.1. says "extra" options.
> As for E.6 - yes I think it is required.
Ok, it's now phase 1.
> Configuration
> I am leaning towards proposal 2.
Acknowledged. There's one significant issue with proposal 2, though.
Currently the code execution order is as follows
1. classify packet
2. call selectSubnet (and use class information)
And that order must stay if we want to keep the ability to allow or
reject clients into subnet based on classes.
We could flip the order and do:
1. selectSubnet
2. classify packet based on the per subnet definition
It would be slightly better performance wise (need to evaluate only
those classes that are actually used in this subnet), but we would lose
the ability to select subnet based on class.
As a reminder, we need that ability to handle cable modems. Cable modems
should go to subnet X and all devices behind it should go to subnet Y.
It's essential for X and Y to not overlap, as cable users are not
supposed to be able to fiddle with their modem's configuration interface.
> Parsing
>
> I think we really need to clarify on the need for substrings being
> extracted from the vendor-class-identifier options. Currently the design
> is based on the assumption that the only required operator is equality
> operator. It is explicitly said in this section.
It's now clarified that vendor-class-identifier and substring are required.
> Evaluation
>
> I am slightly confused with the code examples. I realize that you're
> describing two proposals for retrieving the result of the evaluation,
> i.e. one is to store the result in the value_, another one return it.
> The evaluateToBool() example seems to be using the latter approach
> because it assigns the result of evaluate() to a value. However, the
> evaluareToBool() later obtains the value using the toString() method.
> bool evaluateToBool(const Expression& expr, Pkt6Ptr& pkt) {
> TokenPtr prv1;
> TokenPtr prv2;
> TokenPtr prv3;
>
> /// Let's evaluate all tokens in their RPN order
> for (Expression::iterator current = expr.begin(); current !=
> exp.end(); ++current) {
>
> string value = *current->evaluate(pkt, prv1, prv2, prv3);
>
> /// Let's remember up to three last tokens
> prv3 = prv2;
> prv2 = prv1;
> prv1 = *current;
> }
> return (value == "true");
> }
>
> In order to determine whether it is better to store the evaluation
> result internally in the Token or return it, it would be good to have
> code snippet for both cases and see how the code looks like. However,
> from the given examples I tend to think that the Token should be
> stateless and the result of evaluation should be returned. This is your
> proposal 2.
Ok, updated the snipped. It's still not correct, as the parameters
passed to evaluate() would be token values, not tokens, but I don't
think it's that important. The point I was trying to make here is that
there will be a stack that the evaluation code will walk over,
evaluating tokens one at a time. Once the last token is evaluated, its
value determines the value of the whole expression.
Tomek
------------------------------
Message: 2
Date: Thu, 15 Oct 2015 19:06:14 +0200
From: Tomek Mrugalski <[email protected]>
To: Shawn Routhier <[email protected]>
Cc: [email protected]
Subject: Re: [kea-dev] Client classification design
Message-ID: <[email protected]>
Content-Type: text/plain; charset=utf-8
On 15.10.2015 05:09, Shawn Routhier wrote:
> I echo Thomas?s question about nailing down what the requirements are.
>
> Some general thoughts:
>
> 1) Are we going to want to use the expression evaluation code for uses other
> than classification?
> For example to construct an outgoing option which I don?t believe we allow
> yet. This would not
> be a feature for the near future but it may suggest if we want the expression
> code easily split from
> the classification code.
I surely hope it won't ever get to that. It would be recreating all the
annoyances of ISC DHCP *on top* of new hooks related annoyances that are
unique to Kea. But just in case someone some time in the future would
think it's a good idea, I updated the design. It now clearly states that
the expressions code should be a stand-alone library with minimal
dependencies. More realistic use case would be a client or maybe a
feature rich relay?
> 2) Do we have or need a way to specify which relay options to use in the case
> that the packet
> has traversed multiple relays to get to the server? (I thought I remembered
> seeing a discussion
> about adding some features in this area some time ago but was unable to find
> anything in the
> manual.) I think we can probably defer any general work till later but may
> need to specify
> (in the documentation) what happens in 1.0
Good point. Added E.11 and E.12 requirements.
> 3) If I understand it correctly I think G.7 may lead to problems. It sounds
> like the intention is to
> have multiple classes all named ?FOO? and a class is selected based on which
> subnet the client
> is in. I think that will end up confusing users and causing configuration
> errors.
See my response to Marcin's mail. This is something we need to sort out
as currently classification is used to select a subnet, so we need to do
1. classification
2. subnet selection
which is incompatible with the requirement to define classes on a per
subnet basis.
> 4) It would be quite useful to have some method to help the admin debug these
> things. In ISC DHCP
> the standard way is to insert a statement that will log something into the
> class definition
> or use of class in a subnet. This provides a way for the admin to see if
> they wrote the
> matching statements correctly for the given clients.
This will be taken case with a dedicated logger in the expressions
library. Added G.9 requirement to explicitly state that.
> If we go with proposal 1 it would seem simple enough to have the class
> definitions be global
> while the use of the classes would be only per subnet in phase 1. It would
> also seem that we
> would then be able to extend the global class definition to add option data
> in phase 2.
>
> To me it seems like it will be easier to expand something like proposal 1
> into phase 2 rather
> than expanding proposal 2. It also may be a bit easier for people to update
> their configuration
> files .
Ack. So it's now a tie:
Proposal 1: 1
Proposal 2: 1
> While there is a possibility of mis-spelled names in proposal 1 this is
> similar to what is done
> in ISC DHCP and people don?t seem to have a large problem with it. Overall I
> think going
> with something where the class definitions are easily split from the options
> is a good idea.
>
> I?d also suggest that we be extremely clear in the documentation for
> whichever we decide.
> In ISC DHCP the classes are global but can be defined within a subnet which
> leads to odd
> behavior and generally not what the admin expected. This is another reason
> why I would
> suggest the class definitions be global and only able to be parsed at the
> proper level.
I'm ok with that approach. Actually, having this insight from ISC DHCP
world is a strong indicator that this is indeed what we want to do.
> 3A) Do we have either requirements or goals for the arrangement of classes?
> That is
> the total number that a user may define? the number per-subnet? Do we
> expect that
> there may be a large number of classes but only a limited number of them
> active for
> any subnet? This may affect our decision about defining classes at the
> subnet level.
Each expression has to be evaluated for each incoming packet. Also, a
packet can belong to more than one class. That brings several
implications. First, we need to evaluate each expression for every
packet, so it doesn't matter in which order we do it. Each class defined
will incur performance penalty, even if it never matches.
> My assumption is that most admins will have a smallish number of classes and
> they will
> probably be the same classes across most subnets. Though if ISPs use classes
> to
> provide per customer specific information they may end up with a lot of small
> classes.
I added a requirement G.10. Number of allowed classes is not limited.
However, after a certain arbitrary chosen number of classes, the code
would print out a warning that large number of classes will degrade
performance.
> 4) I agree with Marcin I think E.6 is probably required. At least I would
> put this as the very first
> thing on the not required but really want to have list.
It's now phase 1.
> 5) I also agree with Marcin that F.1 either needs to be broadened or F.3
> needs to be added that
> a class can override a subnet. The documentation should be clear on which
> option will be used
> if a given client could get that option from multiple places. From the
> current description
> it appears that while a client can be a member of many classes it will only
> get options from
> one class at a time (I think we can only have one class per subnet
> currently?). Eventually
> if we allow options at a global level for classes we might want to specify
> options are chosen
> (or simply state that it is undefined and the admin shouldn?t define an
> option in multiple
> classes such that a client could get any of them).
Please let me know if the requirements F.1-F.3 are sufficient or should
I tweak it more? Also, proposing a text would be very welcome.
> 6) I?m not sure if the idea is to limit the parsing to a stack of 3 tokens or
> that is simply for
> example purposes. There are expressions that may take more, though I?m not
> sure
> if we will eventually want them or not. The example I?m thinking of is the
> binary-to-ascii
> expression. This takes 4 arguments (base, width, separator and data to
> format). I don?t think
> we will need such a thing in phase 1 but we probably want to write the code
> to allow
> for expansion in the future and allowing for arbitrary numbers of tokens
> might be useful.
That was the intention for phase 1. But since we added substring
operator, that's no longer true and we have to do full parsing. The good
news about it is that we will handle arbitrary number of tokens. That
bad news about it that we'll need a full flex/Bison grammar to do that.
Tomek
------------------------------
Message: 3
Date: Thu, 15 Oct 2015 19:13:38 +0200
From: Tomek Mrugalski <[email protected]>
To: [email protected]
Subject: Re: [kea-dev] Client classification design
Message-ID: <[email protected]>
Content-Type: text/plain; charset=windows-1252
Thanks a lot for all your comments. The documents are now updated. Here
are the major changes:
- Former Requirements document (that was lacking proper requirements
definition) has been renamed to ClientClassificationDiscussion
- New Requirements document written
- Classes are now global only. Trying to define classes on per subnet
basis was a mistake. Classification is (and should be) done before
the subnet is selected.
- substring operator is now MUST for phase 1
- added explicit requirement for the evaluation code to be a
stand-alone library with minimal set of dependencies.
- Added a number of new expressions for phase 2 (E.7 - E.13)
- case 1 (substring(option vendor-class-identifier, 0, 3) = "APC") is
now phase 1
- added a list of expected tickets
- updated E.1 requirement to cover vendor-class-identifier(60)
- clarified examples
- Added G.8 (code must be located in separate lib)
- Added G.9 (must use its own dedicated logger)
- Added G.10 (number of classes must not be limited.
Two (related) things that remain unsolved are:
- which configuration syntax to use (we have a tie, one vote for
proposal 1 and one for proposal 2)
- what order to use
Currently we do:
1. classify packet
2. select subnet (we use class information here)
We need 2. to for cable networks (separate modems and other devices into
distinct subnets). If we stick with it, we won't be able to define per
subnet classes (or at least not use them to select subnets). This would
also force us to use proposal 1.
The list of expected tickets just for phase 1 is whopping 16. Even
ridiculously underestimating the tasks as 2 days/task (good luck with
implementing Bison grammar in two days), this gives us 32 days (or more
than 6 weeks) of engineering time. I leave it up to you to decide
whether it's realistic to shove this into 1.0.
Tomek
------------------------------
Message: 4
Date: Thu, 15 Oct 2015 12:16:10 -0500 (CDT)
From: "Jeremy C. Reed" <[email protected]>
To: [email protected]
Subject: [kea-dev] Kea manuals in HTML online
Message-ID: <[email protected]>
Content-Type: TEXT/PLAIN; charset=US-ASCII
I generated the HTML versions of the Kea manuals from recent source. I
used groff, but later can use the original docbook xml with styles to
generate HTML for nicer webpages.
The uploaded man pages are at
http://kea.isc.org/docs/
We will make sure these are updated on every release.
Jeremy C. Reed
ISC
------------------------------
Message: 5
Date: Thu, 15 Oct 2015 19:34:09 +0200
From: Tomek Mrugalski <[email protected]>
To: [email protected]
Subject: Re: [kea-dev] Client classification design
Message-ID: <[email protected]>
Content-Type: text/plain; charset=utf-8
On 15.10.2015 19:06, Tomek Mrugalski wrote:
> On 15.10.2015 05:09, Shawn Routhier wrote:
>> 4) It would be quite useful to have some method to help the admin debug
>> these things. In ISC DHCP
>> the standard way is to insert a statement that will log something into the
>> class definition
>> or use of class in a subnet. This provides a way for the admin to see if
>> they wrote the
>> matching statements correctly for the given clients.
> This will be taken case with a dedicated logger in the expressions
> library. Added G.9 requirement to explicitly state that.
>
>> If we go with proposal 1 it would seem simple enough to have the class
>> definitions be global
>> while the use of the classes would be only per subnet in phase 1. It would
>> also seem that we
>> would then be able to extend the global class definition to add option data
>> in phase 2.
>>
>> To me it seems like it will be easier to expand something like proposal 1
>> into phase 2 rather
>> than expanding proposal 2. It also may be a bit easier for people to update
>> their configuration
>> files .
> Ack. So it's now a tie:
> Proposal 1: 1
> Proposal 2: 1
Oops, I forgot about Francis's vote. But we still have a tie, as he
prefers proposal 3 :)
Tomek
------------------------------
_______________________________________________
kea-dev mailing list
[email protected]
https://lists.isc.org/mailman/listinfo/kea-dev
End of kea-dev Digest, Vol 19, Issue 5
**************************************
