Hello folks,

jFYI I have recently configured the Jenkins 'scap-security-guide-pull-requests' CI job to be always running in parallel on all three of the following Jenkins slaves: * el6, * el7, and * fedora23

Why?

* OVAL-5.11 / OVAL-5.11.1 is not an official SCAP-1.2 standard => we still need to produce valid OVAL-5.10 files (IOW "make validate" target still needs to pass with openscap-1.0.x),

* OVAL-5.11.1 might become a SCAP-1.3 part => we need to produce valid OVAL-5.11 files (IOW "make validate" target need to pass with openscap-1.2.x),

* The "ShellCheck" test (remediation scripts bash syntax verification) is performed only on Fedora systems. In the past it happened "make validate" target stopped to PASS due the ShellCheck executable syntax change / expectations.

Therefore the only way how to satisfy all three of the above requirements and not to introduce regressions in the future SSG content, we need to test PRs on all three of the aforementioned systems (in the past we experienced PRs that succeeded only e.g. with "OVAL-5.11", but due to random nature how Jenkins schedules the tests on the slaves, this PRs were improperly reported as passing, even when they didn't met the condition the "make validate" target to pass also with OVAL-5.10. This kind of PRs is making e.g. "scap-security-guide-nightly-oval510-zip" to fail: https://jenkins.open-scap.org/job/scap-security-guide-nightly-oval510-zip/)

Therefore to prevent these regressions in the future, from now on every "scap-security-guide-pull-requests" CI job will be tested on all three of the above systems.

Example PR where this happened already is: https://github.com/OpenSCAP/scap-security-guide/pull/1048

The particular testing jobs then being: * https://jenkins.open-scap.org/job/scap-security-guide-pull-requests/394/ (el6) * https://jenkins.open-scap.org/job/scap-security-guide-pull-requests/395/ (el7) * https://jenkins.open-scap.org/job/scap-security-guide-pull-requests/396/ (fedora23)

Right now "All checks have passed" section is not able to report results of all three tests though (and possibly fail of at least one of those three jobs has failed). I have filed: https://github.com/OpenSCAP/jenkins/issues/11

This to be possible to inspect in the future.

Thank you && Regards, Jan. -- Jan iankko Lieskovsky / Red Hat Security Technologies Team