autoqa-devel November 2010

autoqa-devel@lists.fedorahosted.org

8 participants
85 discussions

Re: upgradepath: FAIL; 0 OK, 1 FAILED for GMT-4.5.5-2.fc13
by Kamil Paral 09 Nov '10

09 Nov '10

----- "James Laska" <jlaska(a)redhat.com> wrote: > Basically, I think the upgradepath test below is correctly > highlighting > a failure, but the text associated with the error is wrong. The test > is > triggered because of a bodhi request for GMT-4.5.5-2.fc13 into > dist-f13-updates-testing. The test fails giving the following > reason ... > > > [FAIL] dist-f14 > > Latest package: GMT-4.5.3-3.fc14 > > Failed condition: Requested package <= Latest package Well... not really :) There are two ways how to read that error message. The first one (mine, in the time of writing) reads: "Failed condition: *expected state*" The second one (yours) reads: "Failed condition: *current state*" The logic is reversed, I am trying to state which condition is broken (and state it how it should look when it is correct), you are trying to read out what's currently wrong with it. I knew this could cause some confusion, but I didn't want to spend more time rewriting application logic. But now I'm convinced we should clarify this more. I'll post a patch where I will change the symbolic equation ('>') to a sentence ('foo must be greater or equal then bar'). That should do it.

2 3

Virtualization support
by Scott M Ferguson 09 Nov '10

09 Nov '10

Hey all, Joza was kinda enough to let me have a go at part of the virtualization support: enabling 2-way communication between a virt-host and virt-guests and I'm starting to dig into it. Per his suggestion I'm working on a generic api that can be tuned based on the needs of the project and I wanted to bring it up to the list. A few initial thoughts from Joza regarding communication were: Host->Guest - this needs to be able to select a certain virt guest 1) Is a test actually running? 2) Disconnect autotest-enabled eth 3) run a command Guest->Host 1) Test finished 2) Test crashed 3) Test running too long 4) Destructive test finished, revert me to 'safe' snapshot He also noted a management interface on the server might be interesting and James noted that it will need to play nice with autotest. I know this is a long-term goal, but I'd love to hear everyone's thoughts when time permits. Best, Scott smferguson(a)gmail.com PS - congrats on F14 going out last week!

2 1

upgradepath: FAIL; 0 OK, 1 FAILED for GMT-4.5.5-2.fc13
by James Laska 08 Nov '10

08 Nov '10

<top post> Greetings all, I mentioned this to Kamil on a IRC a few days ago, and promised I'd send an example to the list when I saw it again. Basically, I think the upgradepath test below is correctly highlighting a failure, but the text associated with the error is wrong. The test is triggered because of a bodhi request for GMT-4.5.5-2.fc13 into dist-f13-updates-testing. The test fails giving the following reason ... > [FAIL] dist-f14 > Latest package: GMT-4.5.3-3.fc14 > Failed condition: Requested package <= Latest package Again, I think the test is correctly failing, but the stated 'failed condition' is incorrect. I've been running a few tests to compare, and I think the following patch will make the error message more accurate. http://fpaste.org/zAKm/ Thanks, James -------- Forwarded Message -------- > From: autoqa(a)fedoraproject.org > Reply-to: test(a)lists.fedoraproject.org > To: autoqa-results(a)lists.fedorahosted.org > Subject: upgradepath: FAIL; 0 OK, 1 FAILED for GMT-4.5.5-2.fc13 > Date: Mon, 8 Nov 2010 17:56:00 +0000 (UTC) > > Stored logs available at <http://autoqa.fedoraproject.org/results/24626-autotest/qa03.c.fedoraproject…> > > ---------------------------------------------------------------------- > ======================================== > GMT-4.5.5-2.fc13 into dist-f13-updates-testing > ======================================== > [ OK ] dist-f10 > Latest package: GMT-4.3.1-2.fc10 > [ OK ] dist-f10-updates > Latest package: GMT-4.4.0-1.fc10 > [ OK ] dist-f11 > Latest package: GMT-4.4.0-2.fc11 > [ OK ] dist-f11-updates > Latest package: GMT-4.5.0-4.fc11 > [ OK ] dist-f12 > Latest package: GMT-4.5.1-1.fc12 > [ OK ] dist-f12-updates > Latest package: GMT-4.5.3-3.fc12 > [ OK ] dist-f13 > Latest package: GMT-4.5.2-1.fc13 > [ OK ] dist-f13-updates > Latest package: GMT-4.5.3-3.fc13 > [ OK ] dist-f13-updates-testing > No such package: GMT > [FAIL] dist-f14 > Latest package: GMT-4.5.3-3.fc14 > Failed condition: Requested package <= Latest package > [ OK ] dist-f14-updates > No such package: GMT > [ OK ] dist-f15 > Latest package: GMT-4.5.5-2.fc15 > SUMMARY: 0 OK, 1 FAILED > > > _______________________________________________ > autoqa-results mailing list > autoqa-results(a)lists.fedorahosted.org > https://fedorahosted.org/mailman/listinfo/autoqa-results

1 0

The next update autoqa
by James Laska 08 Nov '10

08 Nov '10

Hi gang, With Fedora 14 mostly behind us, at least our test execution efforts, I was hoping to plan out the next version of autoqa. Given we've been primarily focused on producing a high quality release, there aren't a large number of changes in master since v0.4.2-1. However, there were some significant changes, including: 6670ee2 2010-10-14 allow tests to distinguish CRASHED and ABORTED (kparal(a)redhat.com) a5ce337 2010-10-13 Reviewed. (mkrizek(a)redhat.com) 821f162 2010-10-11 Added reviewed initscript tests. (jskladan(a)redhat.com) e0f77a1 2010-10-06 Include traceback in email output (kparal(a)redhat.com) 82baf5f 2010-10-06 Removed individual XYZ_failed methods in test.py (jskladan(a)redhat.com) 78cacd5 2010-10-05 guard setup() with ExceptionCatcher (kparal(a)redhat.com) 8e4e894 2010-10-05 allow self.outputs and self.highlights to be lists of strings (kparal(a)redhat.com) 03ca1e8 2010-10-04 Automatically log test output (kparal(a)redhat.com) c6e5d3a 2010-10-01 Correct missing config_loader import (jlaska(a)redhat.com) a2bd1b1 2010-10-01 Tell post-tree-compose that composes are enabled for F14 (jlaska(a)redhat.com) 885e0e1 2010-09-30 tidy up upgradepath test code (kparal(a)redhat.com) 993e394 2010-09-29 upgradepath doesn't support -testing repos yet, disable scheduling (kparal(a)redhat.com) Additionally, I believe there are some upcoming changes we'll want to include. I'm guessing we're targeting the following changes ... 1. First release of depcheck * Add mash support to depcheck (https://fedorahosted.org/autoqa/ticket/201) * Add depcheck to post-bodhi-update hook (https://fedorahosted.org/autoqa/ticket/204) * Make depcheck submit karma/comments to bodhi (https://fedorahosted.org/autoqa/ticket/205) 2. ResultsDB support * Enable results collection (ticket?) 3. Anything else? Should we cut a new version (autoqa-0.4.3-1) now including all changes in master. Should we wait for the remaining changes listed above? What is a good timeframe for the release? Thanks, James

2 1

2010-11-03 - AutoQA and Seneca College discussion
by James Laska 03 Nov '10

03 Nov '10

Hey gang, Kamil and I met with Chris Tyler (Seneca College) about the possibility of getting students involved in the AutoQA project. We met over the phone and discussed AutoQA background, what the students have been tasked with and some ideas to get started. The minutes are recorded below. Thanks, James = AutoQA Startup Meeting = Phone call 2010-11-03 (will be recorded) Please use mediawiki markup. == Attendees == * James Laska (jlaska) - Fedora QA * Jaewoo Park(jwpark2) * Chris Tyler (ctyler) * Kamil Paral (kparal) - Fedora QA * Eric Shum (escom) * Hoc Tran (hvtran) == Introductions == == Background to AutoQA == Discussed general design and goals for the AutoQA project (http://fedoraproject.org/wiki/AutoQA) Question - is autoqa integrated with koji/bodhi? Answer - not integrated, but monitors for koji/bodhi events. autoqa is running in hardware inside Fedora infrastructure, but not yet packaged for fedora due to missing Java dependencies Question - is autoqa polling to look for work? Answer - yes, right now the "hooks" monitor for test events (such as koji or bodhi builds) == What needs to be done == * Future test cases are recorded in TRAC at https://fedorahosted.org/autoqa/milestone/Future%20test%20cases * Simple web front-end for our ResultDB * Finishing some of half-baked tests: Package Sanity Test, Rpmlint whitelisting * Student projects broken into 3 phases ** bite-sized initial project (0.1) (Nov 12) ** cleanup-refinement (0.2) ** New test or extending features (0.3) * Sample bite-sized tests to consider ** Creating autoqa wrapper for man-page checker (https://fedorahosted.org/autoqa/ticket/197) ** Font-lint (https://fedorahosted.org/autoqa/ticket/92) ** Automate fedora package acceptance (https://fedorahosted.org/autoqa/ticket/218) ** Test for un-owned directories (https://fedorahosted.org/autoqa/ticket/31) ** ABI validator (https://fedorahosted.org/autoqa/ticket/190) * Larger possibilities ** systemd tests (similar to http://git.fedorahosted.org/git/?p=autoqa.git;a=tree;f=tests/initscripts;) == Rough plan for getting it done == * don't be afraid to ask any question on autoqa-devel ML (join that list!) ** https://fedorahosted.org/mailman/listinfo/autoqa-devel * contacts: jlaska, kparal, wwoods, jskladan on #fedora-qa IRC channel * read https://fedoraproject.org/wiki/AutoQA * ctyler to meet with students to discuss divvy-up of first-step tests

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

autoqa-devel November 2010