Hi,
I have changed the way that the test suite needs to be operated. Right
now, it requires quite a bit of manual interaction and inside knowledge,
but as we gain experience, we will probably automate more of this, and
make it more fool-proof.
Here is how I (intend) to do it:
$ export TEST_DATA=$HOME/cockpit-tarballs/
# Get the magic base tarballs. Do this only once, they wont change
# without a loud announcement here.
$ mkdir $TEST_DATA/base
$ curl
https://dl.dropboxusercontent.com/s/10b3q7ualqxew1d/fedora-18-x86_64.tar.gz \
$TEST_DATA/base/fedora-18-x86_64.tar.gz
$ curl
https://dl.dropboxusercontent.com/s/10b3q7ualqxew1d/fedora-20-x86_64.tar.gz \
$TEST_DATA/base/fedora-20-x86_64.tar.gz
# Create the tarballs for the actual test runs. Do this whenever the
# OS repository or cockpit-deps have changed enough. Once a day or
# so. This is slow and a bit unreliable since it runs VERIFY to test
# the new tarballs.
$ cd .../cockpit
$ git checkout master
$ TEST_OS=fedora-18 ./PREPARE
$ mv ./test/fedora-18.tar.gz $TEST_DATA/
$ TEST_OS=fedora-20 ./PREPARE
$ mv ./test/fedora-20.tar.gz $TEST_DATA/
# Run the actual tests. This shouldn't go to the network much, only
# mock still runs yum.
$ cd .../cockpit
$ git checkout wip/whatever
$ TEST_OS=fedora-18 ./VERIFY
$ TEST_OS=fedora-20 ./VERIFY
So the biggest change is that we are now isolated from changes in the OS
and in cockpit-deps when running tests (but not while building). VERIFY
should also be a bit faster, but not dramatically so.
The idea is that the PREPARE step is done centrally, and we all download
fresh tarballs whenever necessary. That comes soon, I hope, but for now
you all need to run PREPARE yourself.