In Cockpit we run thousands of integration tests per day against pull
requests and git master. Each test brings up up Cockpit in a full
operating system VM, and hammers on it in some way. Without these tests
it's impossible to validate that Cockpit works.
Last week, the server doing this testing work broke down.
We've been working over the last week to fix that. But we've done more,
instead of just putting this on another server, we've worked to make
these integration tests run in a distributed manner.
As a developer using the new distributed tests is a bit different, and
the documentation has been updated:
https://github.com/cockpit-project/cockpit/blob/master/test/README
Other Operating Systems
-----------------------
Previously we were limited to running the tests on Fedora 22 and RHEL
7.1 for each pull request. One of the reasons was that the server was
simply maxed out much of the time.
By distributing the integration tests, we can open the floor to testing
on further operating systems, and having multiple cases of the tests
running in parallel for each pull request.
Obviously this is not the only factor in running integration tests
against say Debian ... but it was certainly a big blocker in the past.
If dedicating time toward Cockpit's integration tests running on another
OS is something that interests you, don't hesitate to post here on this
list or in the #cockpit channel on FreeNode.
Cheers,
Stef