+1 on work to support OVAL authoring. Greg Elin
On Mon, Sep 14, 2015 at 3:17 AM, Jan Cerny <[email protected]> wrote: > Hello, > > ----- Original Message ----- > > From: "Jan Lieskovsky" <[email protected]> > > To: "Jan Cerny" <[email protected]> > > Cc: "SCAP Security Guide" <[email protected]> > > Sent: Friday, September 11, 2015 11:39:40 AM > > Subject: Re: A tool for support of SCAP content development > > > > > > Hello Jan, > > > > ----- Original Message ----- > > > From: "Jan Cerny" <[email protected]> > > > To: "SCAP Security Guide" <[email protected]> > > > Sent: Thursday, June 4, 2015 9:15:42 AM > > > Subject: A tool for support of SCAP content development > > > > > > Hello, > > > > > > Nowadays, there is almost no support for OVAL content development. > > > > Agree in this point. > > > > > The developers have to edit the XML files manually, there is no > > > analysis tool, no debugger or any other tool that could make > > > the content development faster and easier. > > > I have started working on a new utility that will be able to > > > debug OVAL checks, show step-by-step how they are evaluated, > > > browse collected objects and system characteristics, communicate > > > with probes, etc. > > > > Two questions: > > * is this new tool intended to become part of the openscap-* package, > > or rather the plan being it to be a separate tool?, > > It was decided to be as part of the openscap. > Official instructions approved by university says "implement this in the > OpenSCAP project > by form of editing existing source code." > For university instructions, see Description in Czech on this page: > > https://diplomky.redhat.com/topic/show/335/a-tool-for-development-of-oval-definitions-within-openscap-project > I should definitely put the link on the wiki. > > > * above you speak about debugger -- is the plan to add also profiler? > > It is not in my plan now, but thank your very much for your idea! > > > > > > I'm working on design of the utility now. I'm considering many > > > alternatives, so I would like to ask you a few questions. > > > * Can you describe most common problems that you have to face > > > when you create the OVAL content? > > > > Absence of ability to debug a particular OVAL check. Right now > > it's possible to use 'oscap oval eval / oscap oval collect' > > for particular rule to determine what got actually collected on > > the system. This approach is working for simple rules. But it does > > not scale properly (if the underlying OVAL object consist of another > > OVAL variable, that variable using some OVAL functions on other > > variables and so on, it's hard to determine which OVAL object failed > > to be collected at the beginning -- which OVAL object is responsible > > for the whole rule failure). > > > > Next step is XCCDF evaluation - while the corresponding OVAL check might > > work properly, it's possible (due to combination of "check", > > "check_existence", > > "entity_check" attributes, order and / or logical operations between the > > different OVAL criterions) the final XCCDF rules is still failing even > > in the case "some" OVAL objects have been collected on the system. > > > > Right now oscap provides 'oscap xccdf eval .. --oval-results' combination > > alllowing to inspect, why the rule as a whole did not pass (e.g. wrong > > "check" / "check_existence" attributes etc.). > > > > But it would be good if the new tool would support XCCDF debugging too > > (something like 'oscap xccdf eval ... --oval-results' is allowing now, > > but enhanced). > > I am working now on enhancing the debug infos and introduce "--verbose" > option > in oscap tool. I think it is a good first step. > > > > > > > > * How do you usually solve these problems? > > > > See above. By separating the rule intended to be debugged. Then running: > > * oscap oval eval --id > > * oscap oval collect --id > > * oscap xccdf eval .. --oval-results > > > > combination of commands (depending on the level [e.g. if OVAL itself, > > or at XCCDF level] the OVAL check is failing). > > > > > * Can you tell me an example when did you run into issues with OVAL? > > > > Basically any a bit more complex OVAL check needs to be designed in > > stages (from top to bottom). Consider OVAL criterion requiring presence > > just some OVAL object, this object to consist of multiple stages, e.g.: > > > > * you first collect something on the system into object, save the > > subexpression of it into <local_variable>, > > * apply couple of functions to that variable, finally create new final > > object from the last variable. This final object is then used in the > > OVAL test. > > > > Right now it needs to be done from bottom to the top (at each level, at > > each <local_variable> you mostly need to verify it returns appropriate > > results). Other issue is OVAL standard reporting functionality. AFAICT > > the standard requires the rule to return 'error' also in the case the > > particular OVAL rule wasn't able to obtain child results (results for > > the child objects compositing the final object). Having seen this 'error' > > result for the first time, it's not clear if: > > * OVAL check truly has a syntax error, > > * or OVAL check is correct, but the engine wasn't able to determine value > > for some of the OVAL variables above in the evaluation tree. > > > > Another issue / RFE in 'oscap oval eval / collect' functionality is the > > '--id' parameter. IIRC as of right now it supports specification of > > an ID just for OVAL objects (therefore having OVAL variable manipulating > > some previously collected OVAL object via some OVAL function, it's > > impossible to see the retrieved values of that variable directly. > > > > You need to see the values of the original OVAL object, then create > > another OVAL object from that OVAL variable using the OVAL function, > > to be able to see the true values, that OVAL variable is holding). > > > > Not to speak to abstract. Consider the following: > > > > <ind:textfilecontent54_object id="my_object" ..> > > <ind:filepath>..</ind:filepath> > > <ind:pattern ..>..</ind:pattern> > > <ind:instance ..>..</ind:instance> > > </ind:textfilecontent54_object> > > > > <local_variable id="my_variable> > > <some_oval_function_here, e.g. "split", "concat", whatever> > > <object_component object_ref="my_object" /> > > .. > > </some_oval_function_here> > > </local_variable> > > > > Right now it's not possible to perform the following: > > > > oscap oval collect --id "my_variable" --syschar /tmp/out > > > > because "my_variable" doesn't constitute an OVAL object (therefore > > to bypass it, it's necessary to define new artificial OVAL object > > from that "my_variable" local variable, and try to collect it). > > > > Therefore it would be very handy if 'collect' could be enhanced > > to support variables too. > > This is a nice use-case. > > > > > > * Imagine an ideal tool for an OVAL developer. What should it be > > > able to do? What would be its features? > > > > Maybe above being a sketch. From the above it's not clear if the > > new tool should be just debugger, or profiler too. If the debugger, > > basically it would be able to: > > * report values of particular OVAL objects, variables at any time > > of OVAL evaluation, > > * report the branch OVAL check has been evaluated (like OVAL check > > went through criterion #1, the rest of the criteria weren't taken > > into account, since criterion #1 already succeeded), > > I am going to achieve this by collecting all the possible information > during evaluating the content and processing those information to have > a meaningful output. > > > * + the requests as formulated above. > > > > > If we would talk also about profiling capabilities - it would be > > able to report things like CPU usage in %, virtual memory usage, > > and or count of system files processed when evaluating particular rule. > > I would consider this, it can be useful feature to avoid bugs with > excessive memory consumption caused by badly written content. > > > > > This would be handy to optimize current form in many rules. > > > > > I would appreciate any suggestions, remarks or other inputs. > > > Thank you very much for your reply. > > > > Should you need any other clarification wrt to some of the points > > above, just ask. > > Thank you for your input. > > Jan Černý > > > > > Thank you && Regards, Jan > > -- > > Jan iankko Lieskovsky / Red Hat Security Technologies Team > > > > > > > > Regards > > > > > > Jan Černý > > > Security Technologies | Red Hat, Inc. > > > -- > > > SCAP Security Guide mailing list > > > [email protected] > > > https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide > > > https://github.com/OpenSCAP/scap-security-guide/ > > > -- > SCAP Security Guide mailing list > [email protected] > https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide > https://github.com/OpenSCAP/scap-security-guide/ >
-- SCAP Security Guide mailing list [email protected] https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide https://github.com/OpenSCAP/scap-security-guide/
