----- Original Message ----- > From: "Jan Lieskovsky" <[email protected]> > To: "SCAP Security Guide" <[email protected]> > Cc: "open-scap-list" <[email protected]> > Sent: Wednesday, July 27, 2016 9:34:04 AM > Subject: Re: Latest OpenSCAP changes to speed up SSG builds > > ----- Original Message ----- > > From: "Gabe Alford" <[email protected]> > > To: "SCAP Security Guide" <[email protected]> > > Cc: "open-scap-list" <[email protected]> > > Sent: Wednesday, July 27, 2016 3:00:34 PM > > Subject: Re: Latest OpenSCAP changes to speed up SSG builds > > > > > > On Tue, Jul 26, 2016 at 3:24 PM, Martin Preisler < [email protected] > > > wrote: > > > > > > I found pretty bad inefficiencies in some of our XSLTs. > > > > Check out > > https://github.com/OpenSCAP/openscap/commit/a65bf27dec4a93e2b87cec8cbcd80bec4fd2328a > > or > > https://github.com/OpenSCAP/openscap/commit/1dd5573b2c964b00af57215cadb7f13b1938bac6 > > > > Long story short I was able to cut SSG build time on my machine from > > 2m21.061s to 1m08.771s. In other words my machine now needs roughly half > > the time to build SSG. I was timing `make -j 4`. > > > > It will take a while for these changes to make it into releases > > and into distributions but I am writing this email because the > > savings are significant and perhaps you want to enjoy them sooner. > > SCAP Security Guide contributors are doing a lot of content builds. > > > > This will only work if you !!! have OPENSCAP 1.2.x !!! > > > > If you want to speed up the build, replace > > > > /usr/share/openscap/xsl/xccdf_1.1_to_1.2.xsl with > > https://raw.githubusercontent.com/OpenSCAP/openscap/maint-1.2/xsl/xccdf_1.1_to_1.2.xsl > > > > /usr/share/openscap/xsl/xccdf-share.xsl with > > https://raw.githubusercontent.com/OpenSCAP/openscap/maint-1.2/xsl/xccdf-share.xsl > > > > and > > > > /usr/share/openscap/xsl/xccdf-guide-impl.xsl with > > https://github.com/OpenSCAP/openscap/blob/maint-1.2/xsl/xccdf-guide-impl.xsl > > > > > > Perhaps it would be worth it to deploy this on our Jenkins slaves > > even before it hits releases? What do you think? > > > > +1 > > Not like I would want to be the blocker for speedups / SSG Jenkins > optimizations, but how would you like to deploy these changes? > > Two scenarios come to mind: > * just replace the affected XSLT transformations in recent RHEL openscap > builds, > * make a separate openscap RPM (applied on Jenkins slaves outside of RHEL > release cycle process) > > While the advantage might be more quicker completion of Jenkins SSG build > jobs, there are two disadvantages / concerns related with this: > * if modified oscap returns different result than latest oscap available > in RHEL - which report should be considered the right one? Which one is > valid? > * the concern with maybe even more importance is we wouldn't test on default > RHEL / Fedora systems like we do now. The environment would be modified a > bit > (and it might or might not return same results). So to ensure we "will > continue working" on default RHEL / Fedora - should we create another > Jenkins > CI jobs for default RHEL / Fedora? > > Maybe we could use "speed optimized" openscap on > 'scap-security-guide-pull-requests' > Jenkins CI job, and 'default one' on 'scap-security-guide' Jenkins CI job > (the latter > one being which actually tells if we don't regress across PRs).
I think that would overcomplicate things. I can relate to your concerns. The only reason why we are even considering this is because the speed-up is so dramatic. What do you think about me generating RHEL 6 and 7 datastreams and guides and diff-ing them? If the results are the same across the board that would take care of my concerns. Do you agree? -- Martin Preisler Identity Management and Platform Security | Red Hat, Inc. -- SCAP Security Guide mailing list [email protected] https://lists.fedorahosted.org/admin/lists/[email protected] https://github.com/OpenSCAP/scap-security-guide/
