split test into small tests
by Hongqing Yang
Greetings,
Now I am trying to integrate mediakit_sanity test into AutoQA, mediakit sanity test includes
tests of media size, checksum, repoclosure, conflicts. I find it is trouble to deal with the
test results, since it is impossible to specify which test is FAILED, we can only give a general
result to all tests self.detail.result=FAILED even some tests passed.
So do you think it it OK that I split the test into small tests? apparently, there will be many
test in the tests directory. Or do you have any suggestion to solve this? Thanks.
Hongqing
12 years, 8 months
question about AutoQA test integration
by Hongqing Yang
Hi, all
I try to integrate mediakit_sanity into AutoQA, but every time I run it, it gives the following error.
I also tried to run the exist test, it also gives the same error as below:
Error: Can't evaluate test '__init__.py': [Errno 20] Not a directory: '/usr/share/autotest/client/site_tests/__init__.py/control.autoqa'
File "/usr/bin/autoqa", line 185, in resolve_tests
test_vars[test] = eval_test_vars(test, default_test_vars)
File "/usr/bin/autoqa", line 139, in eval_test_vars
cfile = open(os.path.join(conf['testdir'], test, 'control.autoqa'))
Do you have any idea about this? __init__.py is an empty file.
Thanks,
Hongqing YANG
12 years, 8 months
[AutoQA] #282: Update minimum python version requirement to 2.6
by fedora-badges
#282: Update minimum python version requirement to 2.6
--------------------+-------------------------------------------------------
Reporter: jlaska | Owner:
Type: task | Status: new
Priority: major | Milestone: 0.5.0
Component: core | Keywords:
--------------------+-------------------------------------------------------
Supporting RHEL-5 (python-2.4) isn't fun. The packaging and python
support is so dramatically different. I'd like to upgrade our
autoqa/autotest server to use RHEL-6. I'd consider using a stable Fedora
release, but as tflink pointed out, I'm even less excited about updating
our infrastructure every 6 months.
This ticket is intended to track:
1. Removal of the epel-5 packages and repositories
2. Any code updates/simplification (e.g. use of finally)
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/282>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
12 years, 9 months
[AutoQA] #374: Improve docs in autoqa package
by fedora-badges
#374: Improve docs in autoqa package
-------------------------+--------------------------------------------------
Reporter: kparal | Owner: mkrizek
Type: enhancement | Status: new
Priority: trivial | Milestone: Finger Food
Component: packaging | Keywords:
-------------------------+--------------------------------------------------
Currently we have these files marked as documentation:
{{{
/usr/share/doc/autoqa-0.6.0
/usr/share/doc/autoqa-0.6.0/LICENSE
/usr/share/doc/autoqa-0.6.0/README
/usr/share/doc/autoqa-0.6.0/TODO
/usr/share/doc/autoqa-0.6.0/autoqa.cron
}}}
Improvements:
* delete (completely delete, not just unmark) TODO file, no longer
used/necessary
* mark NEWS file as documentation
* optional: mark all files inside autoqa/doc/ (currently only test
templates) also as documentation - I'm not fully sure of this, but I think
it's a good move.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/374>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
12 years, 9 months
[AutoQA] #376: failed QA because wrong checking of nvr
by fedora-badges
#376: failed QA because wrong checking of nvr
----------------------+-----------------------------------------------------
Reporter: mmaslano | Owner:
Type: defect | Status: new
Priority: major | Milestone: Future tasks
Component: core | Keywords:
----------------------+-----------------------------------------------------
<snip>
Update FAILED: perl-CPAN-Meta-YAML-0.003-7.fc16
Build FAILED: perl-CPAN-Meta-YAML-0.003-7.fc16
Ignored:
perl-CPAN-Meta-YAML-0.003-7.fc16
Detailed Results
Package matching perl-CPAN-Meta-YAML-0.003-7.fc16.noarch already
installed. Checking for update.
</snip>
perl-CPAN-Meta-YAML is shipped twice. Once in perl and once in separated
package. The one from perl has nvr: perl-CPAN-Meta-
YAML-0:0.003-166.fc16.noarch
It was already installed and autoqa doesn't count with this option. It's
only valid for Perl packages. It wouldn't be problem, I can ignore it, but
error message is confusing:
Package matching perl-CPAN-Meta-YAML-0.003-7.fc16.noarch already
installed. Checking for update.
http://autoqa.fedoraproject.org/results/167954-autotest/10.5.124.164/depc...
/perl-CPAN-Meta-YAML-.html
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/376>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
12 years, 9 months
third-party tests *now* ?
by Kamil Paral
There is this discussion in test list [1] where Steve Grubb posted some security testing scripts. He asked these scripts to be included in AutoQA and executed regularly. I would like to discuss it with the whole team because I believe we don't have a clear idea what our current stance is.
I'd like to propose that we currently don't accept any third-party tests. It is tempting and I'd love to have more tests inside AutoQA. But we don't have the manpower to maintain them. Every single extra test case drains power from implementing the AutoQA infrastructure. We already have some test cases we (I) don't understand. And it would get worse.
I believe we need to implement proper AutoQA infrastructure and then allow people to run and maintain their tests, without our babysitting. Different approach will consume our resources.
What do you think?
PS: If you believe some of the tests should be included now, I expect a volunteer to step up and take care of the integration work. Don't be shy :)
[1] http://lists.fedoraproject.org/pipermail/test/2011-August/101624.html
12 years, 9 months
[AutoQA] #375: EPEL Support?
by fedora-badges
#375: EPEL Support?
---------------------+------------------------------------------------------
Reporter: lmacken | Owner:
Type: task | Status: new
Priority: major | Milestone: Future tasks
Component: core | Keywords:
---------------------+------------------------------------------------------
What would it take to support EPEL in AutoQA?
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/375>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
12 years, 9 months
