From: "Jan Lieskovsky" <jlieskov(a)redhat.com>
To: "SCAP Security Guide" <scap-security-guide(a)lists.fedorahosted.org>
Cc: "open-scap-list" <open-scap-list(a)redhat.com>
Sent: Wednesday, July 27, 2016 9:34:04 AM
Subject: Re: Latest OpenSCAP changes to speed up SSG builds
----- Original Message -----
> From: "Gabe Alford" <redhatrises(a)gmail.com>
> To: "SCAP Security Guide"
<scap-security-guide(a)lists.fedorahosted.org>
> Cc: "open-scap-list" <open-scap-list(a)redhat.com>
> Sent: Wednesday, July 27, 2016 3:00:34 PM
> Subject: Re: Latest OpenSCAP changes to speed up SSG builds
>
>
> On Tue, Jul 26, 2016 at 3:24 PM, Martin Preisler < mpreisle(a)redhat.com >
> wrote:
>
>
> I found pretty bad inefficiencies in some of our XSLTs.
>
> Check out
>
https://github.com/OpenSCAP/openscap/commit/a65bf27dec4a93e2b87cec8cbcd80...
> or
>
https://github.com/OpenSCAP/openscap/commit/1dd5573b2c964b00af57215cadb7f...
>
> Long story short I was able to cut SSG build time on my machine from
> 2m21.061s to 1m08.771s. In other words my machine now needs roughly half
> the time to build SSG. I was timing `make -j 4`.
>
> It will take a while for these changes to make it into releases
> and into distributions but I am writing this email because the
> savings are significant and perhaps you want to enjoy them sooner.
> SCAP Security Guide contributors are doing a lot of content builds.
>
> This will only work if you !!! have OPENSCAP 1.2.x !!!
>
> If you want to speed up the build, replace
>
> /usr/share/openscap/xsl/xccdf_1.1_to_1.2.xsl with
>
https://raw.githubusercontent.com/OpenSCAP/openscap/maint-1.2/xsl/xccdf_1...
>
> /usr/share/openscap/xsl/xccdf-share.xsl with
>
https://raw.githubusercontent.com/OpenSCAP/openscap/maint-1.2/xsl/xccdf-s...
>
> and
>
> /usr/share/openscap/xsl/xccdf-guide-impl.xsl with
>
https://github.com/OpenSCAP/openscap/blob/maint-1.2/xsl/xccdf-guide-impl.xsl
>
>
> Perhaps it would be worth it to deploy this on our Jenkins slaves
> even before it hits releases? What do you think?
>
> +1
Not like I would want to be the blocker for speedups / SSG Jenkins
optimizations, but how would you like to deploy these changes?
Two scenarios come to mind:
* just replace the affected XSLT transformations in recent RHEL openscap
builds,
* make a separate openscap RPM (applied on Jenkins slaves outside of RHEL
release cycle process)
While the advantage might be more quicker completion of Jenkins SSG build
jobs, there are two disadvantages / concerns related with this:
* if modified oscap returns different result than latest oscap available
in RHEL - which report should be considered the right one? Which one is
valid?
* the concern with maybe even more importance is we wouldn't test on default
RHEL / Fedora systems like we do now. The environment would be modified a
bit
(and it might or might not return same results). So to ensure we "will
continue working" on default RHEL / Fedora - should we create another
Jenkins
CI jobs for default RHEL / Fedora?
Maybe we could use "speed optimized" openscap on
'scap-security-guide-pull-requests'
Jenkins CI job, and 'default one' on 'scap-security-guide' Jenkins CI
job
(the latter
one being which actually tells if we don't regress across PRs).
I think that would overcomplicate things. I can relate to your concerns.
The only reason why we are even considering this is because the speed-up
is so dramatic.
What do you think about me generating RHEL 6 and 7 datastreams and guides
and diff-ing them? If the results are the same across the board that would
take care of my concerns. Do you agree?
--
Martin Preisler
Identity Management and Platform Security | Red Hat, Inc.