autoqa-devel August 2011

autoqa-devel@lists.fedorahosted.org

8 participants
75 discussions

split test into small tests
by Hongqing Yang 30 Aug '11

30 Aug '11

Greetings, Now I am trying to integrate mediakit_sanity test into AutoQA, mediakit sanity test includes tests of media size, checksum, repoclosure, conflicts. I find it is trouble to deal with the test results, since it is impossible to specify which test is FAILED, we can only give a general result to all tests self.detail.result=FAILED even some tests passed. So do you think it it OK that I split the test into small tests? apparently, there will be many test in the tests directory. Or do you have any suggestion to solve this? Thanks. Hongqing

2 1

[AutoQA] #370: bad test result
by fedora-badges 29 Aug '11

29 Aug '11

#370: bad test result --------------------+------------------------------------------------------- Reporter: till | Owner: Type: defect | Status: new Priority: major | Milestone: Future tasks Component: core | Keywords: --------------------+------------------------------------------------------- This test result is bad: http://autoqa.fedoraproject.org/results/150039-autotest/qa02.qa.fedoraproje… /youtube-dl-2011.08.0.html It was for this update: https://admin.fedoraproject.org/updates/youtube-dl-2011.08.04-1.fc14 But I created a Fedora 15 and 16 update with the same version at the same time: https://admin.fedoraproject.org/updates/youtube-dl-2011.08.04-1.fc16 -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/370> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 9

question about AutoQA test integration
by Hongqing Yang 29 Aug '11

29 Aug '11

Hi, all I try to integrate mediakit_sanity into AutoQA, but every time I run it, it gives the following error. I also tried to run the exist test, it also gives the same error as below: Error: Can't evaluate test '__init__.py': [Errno 20] Not a directory: '/usr/share/autotest/client/site_tests/__init__.py/control.autoqa' File "/usr/bin/autoqa", line 185, in resolve_tests test_vars[test] = eval_test_vars(test, default_test_vars) File "/usr/bin/autoqa", line 139, in eval_test_vars cfile = open(os.path.join(conf['testdir'], test, 'control.autoqa')) Do you have any idea about this? __init__.py is an empty file. Thanks, Hongqing YANG

2 6

[AutoQA] #378: Avoid spamming Bodhi's 'Latest Comments' section
by fedora-badges 29 Aug '11

29 Aug '11

#378: Avoid spamming Bodhi's 'Latest Comments' section -------------------+-------------------------------------------------------- Reporter: kalev | Owner: Type: task | Status: new Priority: major | Milestone: Future tasks Component: core | Keywords: -------------------+-------------------------------------------------------- For some time, Bodhi no longer sends emails for each of the added AutoQA comments. However, the AutoQA comments still appear in Bodhi's 'Latest Comments' section (https://admin.fedoraproject.org/updates/comments , also visible on the front page). It would be nice if only human comments appeared in there; currently AutoQA comment flood pretty much makes the section useless. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/378> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #282: Update minimum python version requirement to 2.6
by fedora-badges 17 Aug '11

17 Aug '11

#282: Update minimum python version requirement to 2.6 --------------------+------------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: major | Milestone: 0.5.0 Component: core | Keywords: --------------------+------------------------------------------------------- Supporting RHEL-5 (python-2.4) isn't fun. The packaging and python support is so dramatically different. I'd like to upgrade our autoqa/autotest server to use RHEL-6. I'd consider using a stable Fedora release, but as tflink pointed out, I'm even less excited about updating our infrastructure every 6 months. This ticket is intended to track: 1. Removal of the epel-5 packages and repositories 2. Any code updates/simplification (e.g. use of finally) -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/282> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 6

[AutoQA] #374: Improve docs in autoqa package
by fedora-badges 17 Aug '11

17 Aug '11

#374: Improve docs in autoqa package -------------------------+-------------------------------------------------- Reporter: kparal | Owner: mkrizek Type: enhancement | Status: new Priority: trivial | Milestone: Finger Food Component: packaging | Keywords: -------------------------+-------------------------------------------------- Currently we have these files marked as documentation: {{{ /usr/share/doc/autoqa-0.6.0 /usr/share/doc/autoqa-0.6.0/LICENSE /usr/share/doc/autoqa-0.6.0/README /usr/share/doc/autoqa-0.6.0/TODO /usr/share/doc/autoqa-0.6.0/autoqa.cron }}} Improvements: * delete (completely delete, not just unmark) TODO file, no longer used/necessary * mark NEWS file as documentation * optional: mark all files inside autoqa/doc/ (currently only test templates) also as documentation - I'm not fully sure of this, but I think it's a good move. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/374> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

[AutoQA] #376: failed QA because wrong checking of nvr
by fedora-badges 17 Aug '11

17 Aug '11

#376: failed QA because wrong checking of nvr ----------------------+----------------------------------------------------- Reporter: mmaslano | Owner: Type: defect | Status: new Priority: major | Milestone: Future tasks Component: core | Keywords: ----------------------+----------------------------------------------------- <snip> Update FAILED: perl-CPAN-Meta-YAML-0.003-7.fc16 Build FAILED: perl-CPAN-Meta-YAML-0.003-7.fc16 Ignored: perl-CPAN-Meta-YAML-0.003-7.fc16 Detailed Results Package matching perl-CPAN-Meta-YAML-0.003-7.fc16.noarch already installed. Checking for update. </snip> perl-CPAN-Meta-YAML is shipped twice. Once in perl and once in separated package. The one from perl has nvr: perl-CPAN-Meta- YAML-0:0.003-166.fc16.noarch It was already installed and autoqa doesn't count with this option. It's only valid for Perl packages. It wouldn't be problem, I can ignore it, but error message is confusing: Package matching perl-CPAN-Meta-YAML-0.003-7.fc16.noarch already installed. Checking for update. http://autoqa.fedoraproject.org/results/167954-autotest/10.5.124.164/depche… /perl-CPAN-Meta-YAML-.html -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/376> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

third-party tests *now* ?
by Kamil Paral 16 Aug '11

16 Aug '11

There is this discussion in test list [1] where Steve Grubb posted some security testing scripts. He asked these scripts to be included in AutoQA and executed regularly. I would like to discuss it with the whole team because I believe we don't have a clear idea what our current stance is. I'd like to propose that we currently don't accept any third-party tests. It is tempting and I'd love to have more tests inside AutoQA. But we don't have the manpower to maintain them. Every single extra test case drains power from implementing the AutoQA infrastructure. We already have some test cases we (I) don't understand. And it would get worse. I believe we need to implement proper AutoQA infrastructure and then allow people to run and maintain their tests, without our babysitting. Different approach will consume our resources. What do you think? PS: If you believe some of the tests should be included now, I expect a volunteer to step up and take care of the integration work. Don't be shy :) [1] http://lists.fedoraproject.org/pipermail/test/2011-August/101624.html

4 4

[AutoQA] #377: Security tests by Steve Grubb
by fedora-badges 16 Aug '11

16 Aug '11

#377: Security tests by Steve Grubb --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: minor | Milestone: Future test cases Component: tests | Keywords: --------------------+------------------------------------------------------- There are some security tests by Steve Grubb that would be ideal to include into AutoQA. Read: http://lists.fedoraproject.org/pipermail/test/2011-August/101624.html -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/377> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 0

[AutoQA] #375: EPEL Support?
by fedora-badges 12 Aug '11

12 Aug '11

#375: EPEL Support? ---------------------+------------------------------------------------------ Reporter: lmacken | Owner: Type: task | Status: new Priority: major | Milestone: Future tasks Component: core | Keywords: ---------------------+------------------------------------------------------ What would it take to support EPEL in AutoQA? -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/375> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

autoqa-devel August 2011