Re: upgradepath: FAIL; 0 OK, 1 FAILED for GMT-4.5.5-2.fc13
by Kamil Paral
----- "James Laska" <jlaska(a)redhat.com> wrote:
> Basically, I think the upgradepath test below is correctly
> highlighting
> a failure, but the text associated with the error is wrong. The test
> is
> triggered because of a bodhi request for GMT-4.5.5-2.fc13 into
> dist-f13-updates-testing. The test fails giving the following
> reason ...
>
> > [FAIL] dist-f14
> > Latest package: GMT-4.5.3-3.fc14
> > Failed condition: Requested package <= Latest package
Well... not really :)
There are two ways how to read that error message. The first one (mine,
in the time of writing) reads:
"Failed condition: *expected state*"
The second one (yours) reads:
"Failed condition: *current state*"
The logic is reversed, I am trying to state which condition is broken
(and state it how it should look when it is correct), you are trying
to read out what's currently wrong with it.
I knew this could cause some confusion, but I didn't want to spend more
time rewriting application logic. But now I'm convinced we should
clarify this more. I'll post a patch where I will change the symbolic
equation ('>') to a sentence ('foo must be greater or equal then bar').
That should do it.
13 years, 6 months
Virtualization support
by Scott M Ferguson
Hey all,
Joza was kinda enough to let me have a go at part of the
virtualization support: enabling 2-way communication between a
virt-host and virt-guests and I'm starting to dig into it. Per his
suggestion I'm working on a generic api that can be tuned based on the
needs of the project and I wanted to bring it up to the list. A few
initial thoughts from Joza regarding communication were:
Host->Guest - this needs to be able to select a certain virt guest
1) Is a test actually running?
2) Disconnect autotest-enabled eth
3) run a command
Guest->Host
1) Test finished
2) Test crashed
3) Test running too long
4) Destructive test finished, revert me to 'safe' snapshot
He also noted a management interface on the server might be
interesting and James noted that it will need to play nice with
autotest.
I know this is a long-term goal, but I'd love to hear everyone's
thoughts when time permits.
Best,
Scott
smferguson(a)gmail.com
PS - congrats on F14 going out last week!
13 years, 6 months
upgradepath: FAIL; 0 OK, 1 FAILED for GMT-4.5.5-2.fc13
by James Laska
<top post>
Greetings all,
I mentioned this to Kamil on a IRC a few days ago, and promised I'd send
an example to the list when I saw it again.
Basically, I think the upgradepath test below is correctly highlighting
a failure, but the text associated with the error is wrong. The test is
triggered because of a bodhi request for GMT-4.5.5-2.fc13 into
dist-f13-updates-testing. The test fails giving the following
reason ...
> [FAIL] dist-f14
> Latest package: GMT-4.5.3-3.fc14
> Failed condition: Requested package <= Latest package
Again, I think the test is correctly failing, but the stated 'failed
condition' is incorrect. I've been running a few tests to compare, and
I think the following patch will make the error message more accurate.
http://fpaste.org/zAKm/
Thanks,
James
-------- Forwarded Message --------
> From: autoqa(a)fedoraproject.org
> Reply-to: test(a)lists.fedoraproject.org
> To: autoqa-results(a)lists.fedorahosted.org
> Subject: upgradepath: FAIL; 0 OK, 1 FAILED for GMT-4.5.5-2.fc13
> Date: Mon, 8 Nov 2010 17:56:00 +0000 (UTC)
>
> Stored logs available at <http://autoqa.fedoraproject.org/results/24626-autotest/qa03.c.fedoraproje...>
>
> ----------------------------------------------------------------------
> ========================================
> GMT-4.5.5-2.fc13 into dist-f13-updates-testing
> ========================================
> [ OK ] dist-f10
> Latest package: GMT-4.3.1-2.fc10
> [ OK ] dist-f10-updates
> Latest package: GMT-4.4.0-1.fc10
> [ OK ] dist-f11
> Latest package: GMT-4.4.0-2.fc11
> [ OK ] dist-f11-updates
> Latest package: GMT-4.5.0-4.fc11
> [ OK ] dist-f12
> Latest package: GMT-4.5.1-1.fc12
> [ OK ] dist-f12-updates
> Latest package: GMT-4.5.3-3.fc12
> [ OK ] dist-f13
> Latest package: GMT-4.5.2-1.fc13
> [ OK ] dist-f13-updates
> Latest package: GMT-4.5.3-3.fc13
> [ OK ] dist-f13-updates-testing
> No such package: GMT
> [FAIL] dist-f14
> Latest package: GMT-4.5.3-3.fc14
> Failed condition: Requested package <= Latest package
> [ OK ] dist-f14-updates
> No such package: GMT
> [ OK ] dist-f15
> Latest package: GMT-4.5.5-2.fc15
> SUMMARY: 0 OK, 1 FAILED
>
>
> _______________________________________________
> autoqa-results mailing list
> autoqa-results(a)lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/autoqa-results
13 years, 6 months
The next update autoqa
by James Laska
Hi gang,
With Fedora 14 mostly behind us, at least our test execution efforts, I
was hoping to plan out the next version of autoqa. Given we've been
primarily focused on producing a high quality release, there aren't a
large number of changes in master since v0.4.2-1. However, there were
some significant changes, including:
6670ee2 2010-10-14 allow tests to distinguish CRASHED and ABORTED (kparal(a)redhat.com)
a5ce337 2010-10-13 Reviewed. (mkrizek(a)redhat.com)
821f162 2010-10-11 Added reviewed initscript tests. (jskladan(a)redhat.com)
e0f77a1 2010-10-06 Include traceback in email output (kparal(a)redhat.com)
82baf5f 2010-10-06 Removed individual XYZ_failed methods in test.py (jskladan(a)redhat.com)
78cacd5 2010-10-05 guard setup() with ExceptionCatcher (kparal(a)redhat.com)
8e4e894 2010-10-05 allow self.outputs and self.highlights to be lists of strings (kparal(a)redhat.com)
03ca1e8 2010-10-04 Automatically log test output (kparal(a)redhat.com)
c6e5d3a 2010-10-01 Correct missing config_loader import (jlaska(a)redhat.com)
a2bd1b1 2010-10-01 Tell post-tree-compose that composes are enabled for F14 (jlaska(a)redhat.com)
885e0e1 2010-09-30 tidy up upgradepath test code (kparal(a)redhat.com)
993e394 2010-09-29 upgradepath doesn't support -testing repos yet, disable scheduling (kparal(a)redhat.com)
Additionally, I believe there are some upcoming changes we'll want to
include. I'm guessing we're targeting the following changes ...
1. First release of depcheck
* Add mash support to depcheck
(https://fedorahosted.org/autoqa/ticket/201)
* Add depcheck to post-bodhi-update hook
(https://fedorahosted.org/autoqa/ticket/204)
* Make depcheck submit karma/comments to bodhi
(https://fedorahosted.org/autoqa/ticket/205)
2. ResultsDB support
* Enable results collection (ticket?)
3. Anything else?
Should we cut a new version (autoqa-0.4.3-1) now including all changes
in master. Should we wait for the remaining changes listed above? What
is a good timeframe for the release?
Thanks,
James
13 years, 6 months
2010-11-03 - AutoQA and Seneca College discussion
by James Laska
Hey gang,
Kamil and I met with Chris Tyler (Seneca College) about the possibility
of getting students involved in the AutoQA project. We met over the
phone and discussed AutoQA background, what the students have been
tasked with and some ideas to get started. The minutes are recorded
below.
Thanks,
James
= AutoQA Startup Meeting =
Phone call 2010-11-03 (will be recorded)
Please use mediawiki markup.
== Attendees ==
* James Laska (jlaska) - Fedora QA
* Jaewoo Park(jwpark2)
* Chris Tyler (ctyler)
* Kamil Paral (kparal) - Fedora QA
* Eric Shum (escom)
* Hoc Tran (hvtran)
== Introductions ==
== Background to AutoQA ==
Discussed general design and goals for the AutoQA project
(http://fedoraproject.org/wiki/AutoQA).
Question - is autoqa integrated with koji/bodhi?
Answer - not integrated, but monitors for koji/bodhi events. autoqa is
running in hardware inside Fedora infrastructure, but not yet packaged
for fedora due to missing Java dependencies
Question - is autoqa polling to look for work?
Answer - yes, right now the "hooks" monitor for test events (such as
koji or bodhi builds)
== What needs to be done ==
* Future test cases are recorded in TRAC at
https://fedorahosted.org/autoqa/milestone/Future%20test%20cases
* Simple web front-end for our ResultDB
* Finishing some of half-baked tests: Package Sanity Test, Rpmlint
whitelisting
* Student projects broken into 3 phases
** bite-sized initial project (0.1) (Nov 12)
** cleanup-refinement (0.2)
** New test or extending features (0.3)
* Sample bite-sized tests to consider
** Creating autoqa wrapper for man-page checker
(https://fedorahosted.org/autoqa/ticket/197)
** Font-lint (https://fedorahosted.org/autoqa/ticket/92)
** Automate fedora package acceptance
(https://fedorahosted.org/autoqa/ticket/218)
** Test for un-owned directories
(https://fedorahosted.org/autoqa/ticket/31)
** ABI validator (https://fedorahosted.org/autoqa/ticket/190)
* Larger possibilities
** systemd tests (similar to
http://git.fedorahosted.org/git/?p=autoqa.git;a=tree;f=tests/initscripts;)
== Rough plan for getting it done ==
* don't be afraid to ask any question on autoqa-devel ML (join that
list!)
** https://fedorahosted.org/mailman/listinfo/autoqa-devel
* contacts: jlaska, kparal, wwoods, jskladan on #fedora-qa IRC channel
* read https://fedoraproject.org/wiki/AutoQA
* ctyler to meet with students to discuss divvy-up of first-step tests
13 years, 6 months