autoqa-devel August 2011

autoqa-devel@lists.fedorahosted.org

8 participants
75 discussions

[AutoQA] #371: stop analog daemon after every installation
by fedora-badges 12 Aug '11

12 Aug '11

#371: stop analog daemon after every installation ----------------------+----------------------------------------------------- Reporter: hongqing | Owner: Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: core | Keywords: ----------------------+----------------------------------------------------- rsyslog has to be restarted to get logs from guest every time, from f15, although rsyslog is restarted , but the daemon rsyslogd_6080.pid is still there, this should be killed as rsyslog stops, or the logs will not be transferred to the host. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/371> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

AutoQA Staging Related Proposals
by Tim Flink 11 Aug '11

11 Aug '11

I put together some draft proposals on how I'd like to see us use the new staging environment. As a whole, the two proposals will require a decent amount of time and effort but I've tried to break them up into smaller chunks so that we can start seeing the benefits sooner rather than later. This all makes sense in my head and I think that I've put enough detail into the proposals for them to make sense to everyone else but I'm sure that I forgot some details. Please let me know what parts of this aren't clear and I will add detail and/or clarify. Other comments and/or suggestions are also very welcome. Thanks, Tim AutoQA in a box: - https://fedoraproject.org/wiki/User:Tflink/Sandbox:AutoQA_in_a_box - The idea here is to automate the creation and updating of the AutoQA environment. This would make it easier to setup dev environments and simplify the management of the production infrastructure. AutoQA Staging: - https://fedoraproject.org/wiki/User:Tflink/Sandbox:AutoQA_staging_environme… - This is the longer of the two proposals. I think that it will be good for us in the long run and ties together some of the things that we've talked about for testing AutoQA.

2 2

Error: Can't evaluate test 'initscripts': Distro name not found in glibc-2.14.90-4.1
by James Laska 11 Aug '11

11 Aug '11

Greetings gang, I discovered the following failure while examining cron output for in production. I haven't dug into the issue at all yet, but it seems to be caused by package NVR's that do not contain a %{dist} tag at the end. In this particular case, the package was 'glibc' [1]. I suspect this is not a systemmic problem, but perhaps a conditional check around the get_distro() call? Perhaps something like the following ... diff --git a/tests/initscripts/control.autoqa b/tests/initscripts/control.autoqa index c380c81..d088db1 100644 --- a/tests/initscripts/control.autoqa +++ b/tests/initscripts/control.autoqa @@ -9,8 +9,13 @@ labels = ['virt'] # must be present (like 'fc13'). If proper label is not found the test will not execute if autoqa_args.has_key('nvrs'): from autoqa.util import get_distro - distro = get_distro(autoqa_args['nvrs'][0]) - labels.append(distro) + try: + distro = get_distro(autoqa_args['nvrs'][0]) + except IndexError: + # Optionally emit a warning that no %{dist} string was found? + pass + else: + labels.append(distro) # we want to run initscripts just for post-koji-build if event in ['post-koji-build']: Thanks, James [1] http://koji.fedoraproject.org/koji/packageinfo?packageID=57 ----- Forwarded message from Cron Daemon <root(a)autoqa01.qa.fedoraproject.org> ----- Date: Tue, 9 Aug 2011 13:40:25 +0000 (UTC) From: Cron Daemon <root(a)autoqa01.qa.fedoraproject.org> To: autotest(a)autoqa01.qa.fedoraproject.org Subject: Cron <autotest@autoqa01> /usr/share/autoqa/watchers/koji-bodhi/watcher.py Created job: 'post-koji-build:rpmguard.noarch (id 155001)' Created job: 'post-koji-build:rpmlint.noarch (id 155002)' Created job: 'post-koji-build:rpmguard.noarch (id 155003)' Created job: 'post-koji-build:rpmlint.noarch (id 155004)' File "/usr/bin/autoqa", line 185, in resolve_tests test_vars[test] = eval_test_vars(test, default_test_vars) File "/usr/bin/autoqa", line 147, in eval_test_vars exec cfile in vars File "/usr/share/autotest/client/site_tests/depcheck/control.autoqa", line 12, in <module> distro = get_distro(autoqa_args['nvrs'][0]) File "/usr/lib/python2.6/site-packages/autoqa/util.py", line 253, in get_distro raise ValueError('Distro name not found in %s' % rpmstr) File "/usr/bin/autoqa", line 185, in resolve_tests test_vars[test] = eval_test_vars(test, default_test_vars) File "/usr/bin/autoqa", line 147, in eval_test_vars exec cfile in vars File "/usr/share/autotest/client/site_tests/initscripts/control.autoqa", line 12, in <module> distro = get_distro(autoqa_args['nvrs'][0]) File "/usr/lib/python2.6/site-packages/autoqa/util.py", line 253, in get_distro raise ValueError('Distro name not found in %s' % rpmstr) Error: Can't evaluate test 'depcheck': Distro name not found in glibc-2.14.90-4.1 Error: Can't evaluate test 'initscripts': Distro name not found in glibc-2.14.90-4.1 ----- End forwarded message -----

2 6

[AutoQA] #313: ResultsDB: Script to parse the mailing-list and fill the resultsDB
by fedora-badges 10 Aug '11

10 Aug '11

#313: ResultsDB: Script to parse the mailing-list and fill the resultsDB ----------------------+----------------------------------------------------- Reporter: vhumpa | Owner: vhumpa Type: task | Status: new Priority: major | Milestone: Resultdb Component: resultdb | Keywords: ----------------------+----------------------------------------------------- To continue development and AutoQA/ResultDB integration we need to have ResultsDB filled with relevant data for testing. Later on we might need to put already existing result data into ResultDB when it becomes operational. All the past data sits in the mailing list archives, so let's create a script that will parse the stored mbox file and input the data into ResultDB using it's input API. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/313> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 6

upgradepath bugfix, candidate for 0.6.1
by Kamil Paral 10 Aug '11

10 Aug '11

Hello, I have fixed a bug in upgradepath that demonstrated itself by crashing in certain conditions: commit 62aa5c02e89a7e4e6d442651e927b2f9ac4d9ce2 (HEAD, refs/remotes/origin/master, refs/remotes/origin/HEAD, refs/heads/master) Author: Kamil Páral <kparal(a)redhat.com> Date: Tue Aug 9 14:31:34 2011 +0200 upgradepath: fix crash This fixes a crash when there was no build of the tested package in the stable repository of an older release. You can see it in action here: https://fedorahosted.org/pipermail/autoqa-results/2011-August/181303.html I think it is desirable to push this into production asap. If no other important bugs are found, I'd like to tag, build & deploy 0.6.1 today or tomorrow. Just a heads-up.

2 3

[PATCH] control.autoqa: check for exceptions when using distro_tag()
by Kamil Paral 10 Aug '11

10 Aug '11

--- doc/control.autoqa.template | 7 +++++-- tests/depcheck/control.autoqa | 7 +++++-- tests/initscripts/control.autoqa | 24 +++++++++++------------- 3 files changed, 21 insertions(+), 17 deletions(-) diff --git a/doc/control.autoqa.template b/doc/control.autoqa.template index 435ca40..178ac1d 100644 --- a/doc/control.autoqa.template +++ b/doc/control.autoqa.template @@ -40,5 +40,8 @@ if event in ['post-koji-build']: # correct distribution must be present (like 'fc13') if autoqa_args.has_key('nvrs'): from autoqa.util import get_distro - distro = get_distro(autoqa_args['nvrs'][0]) - labels.append(distro) + try: + distro = get_distro(autoqa_args['nvrs'][0]) + labels.append(distro) + except ValueError: # no distro tag found + execute = False diff --git a/tests/depcheck/control.autoqa b/tests/depcheck/control.autoqa index bdd6041..c3236e4 100644 --- a/tests/depcheck/control.autoqa +++ b/tests/depcheck/control.autoqa @@ -9,5 +9,8 @@ if event in ['post-bodhi-update-batch']: # must be present (like 'fc13') If proper label is not found the test will not execute if autoqa_args.has_key('nvrs'): from autoqa.util import get_distro - distro = get_distro(autoqa_args['nvrs'][0]) - labels.append(distro) + try: + distro = get_distro(autoqa_args['nvrs'][0]) + labels.append(distro) + except ValueError: + execute = False diff --git a/tests/initscripts/control.autoqa b/tests/initscripts/control.autoqa index 51ea825..b7ec70b 100644 --- a/tests/initscripts/control.autoqa +++ b/tests/initscripts/control.autoqa @@ -5,23 +5,11 @@ # virtual machines as a precaution labels = ['virt'] -# because we install the package, the autotest label of the correct distribution -# must be present (like 'fc13'). If proper label is not found the test will not execute -if autoqa_args.has_key('nvrs'): - from autoqa.util import get_distro - try: - distro = get_distro(autoqa_args['nvrs'][0]) - except IndexError: - # FIXME - Optionally emit a warning that no %{dist} string was found? - pass - else: - labels.append(distro) - # we want to run initscripts just for post-koji-build if event in ['post-koji-build']: execute = True -# we also don't want to run this test for packages we currently don't have +# we don't want to run this test for packages we currently don't have # covered or for packages that contain no init scripts at all # let's just look into tests/ directory if we have test available and decide if autoqa_args.has_key('name'): @@ -29,3 +17,13 @@ if autoqa_args.has_key('name'): if autoqa_args['name'] not in os.listdir('tests'): # not supported execute = False + +# because we install the package, the autotest label of the correct distribution +# must be present (like 'fc13'). If proper label is not found the test will not execute +if autoqa_args.has_key('nvrs'): + from autoqa.util import get_distro + try: + distro = get_distro(autoqa_args['nvrs'][0]) + labels.append(distro) + except ValueError: + execute = False -- 1.7.6

1 0

[AutoQA] #372: bad URLs in bodhi comments (extra blank character)
by fedora-badges 09 Aug '11

09 Aug '11

#372: bad URLs in bodhi comments (extra blank character) --------------------+------------------------------------------------------- Reporter: till | Owner: Type: defect | Status: new Priority: major | Milestone: Component: core | Keywords: --------------------+------------------------------------------------------- URLs in bodhi comments contain an extra blank character. For example in this update: https://admin.fedoraproject.org/updates/youtube-dl-2011.08.04-1.fc16 The first comment from AutoQA is: {{{ AutoQA: depcheck test PASSED on x86_64. Result log: http://autoqa.fedoraproject.org/results/150013-autotest/qa01.qa.fedoraproje… /youtube-dl-2011.08.0.html (results are informative only) }}} The URL included in the comment should be http://autoqa.fedoraproject.org/results/150013-autotest/qa01.qa.fedoraproje… /youtube-dl-2011.08.0.html and not http://autoqa.fedoraproject.org/results/150013-autotest/qa01.qa.fedoraproje… youtube-dl-2011.08.0.html -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/372> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #366: remove previous run logs
by fedora-badges 05 Aug '11

05 Aug '11

#366: remove previous run logs ----------------------+----------------------------------------------------- Reporter: hongqing | Owner: Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: autotest | Keywords: ----------------------+----------------------------------------------------- the installation steps are read from anaconda.log, if the previous run anaconda.log exists, the installation test steps will be read from the old one, so the logs should be cleaned before the next run. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/366> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #352: Determine AutoQA Functional Self Test Cases
by fedora-badges 05 Aug '11

05 Aug '11

#352: Determine AutoQA Functional Self Test Cases --------------------+------------------------------------------------------- Reporter: tflink | Owner: Type: task | Status: new Priority: major | Milestone: Self Test Component: core | Keywords: --------------------+------------------------------------------------------- In order to start forming a better self test suite for AutoQA, we need to start cataloging situations that will make for good functional regression tests. Instead of focusing on how to get them all run, the idea is to describe the situations that would make for good tests. Formatting and categorizing the tests will be left for another ticket. Examples of jobs that have caused problems in the past or continue to cause strange problems would be helpful. An extremely simplistic example for depcheck might look like: === Package Descriptions === || '''Package''' || '''Depends On''' || || foo-1.0-1 || bar-1.0-1 || || foo-2.0-1 || bar-2.0-1 || === Repo Contents === || '''Name''' || '''Packages''' || || stable || foo-1.0-1, bar-1.0-1 || || pending || foo-2.0-1 || === Expected Results === Failure because foo-2.0-1 depends on bar-2.0-1 which is not available. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/352> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

AutoQA 0.6 released
by Kamil Paral 05 Aug '11

05 Aug '11

Please read: https://kparal.wordpress.com/2011/08/05/autoqa-0-6-released/ Kamil

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

autoqa-devel August 2011