Hello,
I have run make validate on RHEL7 content from current SSG upstream. Make validate fails because there are many invalid references to non-existent OVAL definitions.
Moreover, I have discovered that "validate" target in SSG Makefile is commented out for RHEL7, OpenStack and RHEVM3 content. It's commented out since August 2014. I think this is very unpleasant because we haven't validated the RHEL content for a long time.
I have filed an issue on github: https://github.com/OpenSCAP/scap-security-guide/issues/635
Regards
Jan Černý Security Technologies | Red Hat, Inc.
Hello Jan,
thank you for your report.
(Replied in your ticket, but replying also here).
----- Original Message -----
From: "Jan Cerny" jcerny@redhat.com To: "SCAP Security Guide" scap-security-guide@lists.fedorahosted.org Sent: Monday, August 3, 2015 4:23:06 PM Subject: "Make validate" fails
Hello,
I have run make validate on RHEL7 content from current SSG upstream. Make validate fails because there are many invalid references to non-existent OVAL definitions.
While not tracked via SSG ticket yet, this has been known issue.
Moreover, I have discovered that "validate" target in SSG Makefile is commented out for RHEL7, OpenStack and RHEVM3 content. It's commented out since August 2014. I think this is very unpleasant because we haven't validated the RHEL content for a long time.
Just to clarify - it's not the case the RHEL/7 content would be failing for couple of rules, and instead of fixing it, we would comment the ```make validate``` target out. In fact, the RHEL/7 content has never ever been in such a state (since it's creation) we could switch the ```make validate``` target on (there has been a lot of failing rules from the start). Actually if you would checkout the repository content for RHEL/7 in the state it was in August 2014, and compare with the current state, you would notice that since that time there are less rules failing that it were previously. We are progressing against the goal (```make validate``` target for RHEL/7 content to start passing too -- in that moment we could switch it on), but right now we are not that far yet. -- Help / contributions appreciated here.
If someone is searching a way how to contribute to SSG, and wouldn't want to participate in creating new XCCDF rules / OVAL checks / remediations, this concrete issue is another place where help would be appreciated.
For the case of 'RHEVM3' and 'Openstack' products the situation is even worse. If you would have a look at the actual content - rules present in these products, you would find out, there are not rules present at all (IOW the whole 'RHEVM3' and 'Openstack' folders are just 'templated' directory copies created when new product is created).
For now, since benchmarks for these two products do not contain valuable SCAP (XCCDF && OVAL content), the ```make validate``` target is disabled for them for now. And that also being the reason why we are not packaging these benchmarks when making RPM from upstream code or in RPMs distributed in downstream releases.
I have filed an issue on github: https://github.com/OpenSCAP/scap-security-guide/issues/635
Thanks! (For now I have set 0.1.25 target release for fixing that one, but take this just as a tentative estimation -- subject of change for now).
Regards, Jan. -- Jan iankko Lieskovsky / Red Hat Security Technologies Team
Regards
Jan Černý Security Technologies | Red Hat, Inc.
-- SCAP Security Guide mailing list scap-security-guide@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide https://github.com/OpenSCAP/scap-security-guide/
Hello Jan,
thank you very much for your detailed reply. I would like to add that fixing this issue and running make validate for RHEL7 content every time could help us with continuous integration and avoid regressions.
Regards
Jan Černý Security Technologies | Red Hat, Inc.
----- Original Message -----
From: "Jan Lieskovsky" jlieskov@redhat.com To: "SCAP Security Guide" scap-security-guide@lists.fedorahosted.org Sent: Monday, August 3, 2015 5:55:41 PM Subject: Re: "Make validate" fails
Hello Jan,
thank you for your report.
(Replied in your ticket, but replying also here).
----- Original Message -----
From: "Jan Cerny" jcerny@redhat.com To: "SCAP Security Guide" scap-security-guide@lists.fedorahosted.org Sent: Monday, August 3, 2015 4:23:06 PM Subject: "Make validate" fails
Hello,
I have run make validate on RHEL7 content from current SSG upstream. Make validate fails because there are many invalid references to non-existent OVAL definitions.
While not tracked via SSG ticket yet, this has been known issue.
Moreover, I have discovered that "validate" target in SSG Makefile is commented out for RHEL7, OpenStack and RHEVM3 content. It's commented out since August 2014. I think this is very unpleasant because we haven't validated the RHEL content for a long time.
Just to clarify - it's not the case the RHEL/7 content would be failing for couple of rules, and instead of fixing it, we would comment the ```make validate``` target out. In fact, the RHEL/7 content has never ever been in such a state (since it's creation) we could switch the ```make validate``` target on (there has been a lot of failing rules from the start). Actually if you would checkout the repository content for RHEL/7 in the state it was in August 2014, and compare with the current state, you would notice that since that time there are less rules failing that it were previously. We are progressing against the goal (```make validate``` target for RHEL/7 content to start passing too -- in that moment we could switch it on), but right now we are not that far yet. -- Help / contributions appreciated here.
If someone is searching a way how to contribute to SSG, and wouldn't want to participate in creating new XCCDF rules / OVAL checks / remediations, this concrete issue is another place where help would be appreciated.
For the case of 'RHEVM3' and 'Openstack' products the situation is even worse. If you would have a look at the actual content - rules present in these products, you would find out, there are not rules present at all (IOW the whole 'RHEVM3' and 'Openstack' folders are just 'templated' directory copies created when new product is created).
For now, since benchmarks for these two products do not contain valuable SCAP (XCCDF && OVAL content), the ```make validate``` target is disabled for them for now. And that also being the reason why we are not packaging these benchmarks when making RPM from upstream code or in RPMs distributed in downstream releases.
I have filed an issue on github: https://github.com/OpenSCAP/scap-security-guide/issues/635
Thanks! (For now I have set 0.1.25 target release for fixing that one, but take this just as a tentative estimation -- subject of change for now).
Regards, Jan.
Jan iankko Lieskovsky / Red Hat Security Technologies Team
Regards
Jan Černý Security Technologies | Red Hat, Inc.
-- SCAP Security Guide mailing list scap-security-guide@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide https://github.com/OpenSCAP/scap-security-guide/
-- SCAP Security Guide mailing list scap-security-guide@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide https://github.com/OpenSCAP/scap-security-guide/
scap-security-guide@lists.fedorahosted.org
-
Jan Cerny -
Jan Lieskovsky