autoqa-devel February 2012

autoqa-devel@lists.fedorahosted.org

9 participants
44 discussions

[AutoQA] #251: Move autotest applications into /usr/*bin
by fedora-badges 26 Sep '12

26 Sep '12

#251: Move autotest applications into /usr/*bin -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: minor | Milestone: Packaging, Review, & Deployment Component: packaging | Keywords: -----------------------+---------------------------------------------------- The autotest package should supply binaries in /usr/bin/ ... so we can rely on PATH and not have to call autotest with the full-path (/usr/share/autotest/client/bin/autotest) when scheduling jobs in '/usr/bin/autoqa'. Alternatively, the autotest package could drop a script that updates PATH appropriately into /etc/profile.d/autotest. But yuck. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/251> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #191: Submit autotest for Fedora package approval
by fedora-badges 26 Sep '12

26 Sep '12

#191: Submit autotest for Fedora package approval -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: major | Milestone: Packaging and Review Component: packaging | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- * https://bugzilla.redhat.com/show_bug.cgi?id=548522 Currently stalled, pending packaging of the gwt package. This includes packaging all bundled .jar files included with gwt. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/191> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #157: conflicts: requested arch not respected
by fedora-badges 06 Aug '12

06 Aug '12

#157: conflicts: requested arch not respected --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: defect | Status: new Priority: major | Milestone: Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- This is a bug in the potential_conflict.py script in tests/conflicts/. Usage suggests using --arch options for specifying requested architecture to be tested. But if you look at the attached log, the first and second output match (that's correct), but the third doesn't match, even though it should. Everything was tested on x86_64 machine. Also from the usage line it is not clear whether --arch is related only to particular yum repository selection (replacing $basearch keyword), or whether it also relates to packages built for one architecture but being in different repo (e.g. i686 packages are very often in x86_64 repo). I have looked into the source code of the script and it seems that the --arch option is not used at all, just printed out in the usage help?? Seth, you're listed as script author. Any comments on that? Thanks. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/157> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 7

[AutoQA] #270: Package and submit for review - compat-Django-1.0.4
by fedora-badges 02 Aug '12

02 Aug '12

#270: Package and submit for review - compat-Django-1.0.4 -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan Component: packaging | Keywords: -----------------------+---------------------------------------------------- Currently packaged so that it can coexist with the current Django package (see http://repos.fedorapeople.org/repos/fedora- qa/autoqa/fedora-14-testing/). I need to port existing security patches linked from http://www.djangoproject.com/weblog/2010/dec/22/security/ Once patched, submit for review ... and enjoy. Eventually autotest will be updated to use Django-1.2 ... but packaging Django-1.0.4 removes one more packaging obstacle to having autotest as an official Fedora package. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/270> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #329: implement RPM checksum verification
by fedora-badges 30 Jul '12

30 Jul '12

#329: implement RPM checksum verification -------------------------+-------------------------------------------------- Reporter: kparal | Owner: Type: enhancement | Status: new Priority: major | Milestone: Finger Food Component: core | Keywords: -------------------------+-------------------------------------------------- Currently this method is in autoqa.util module: {{{ 261 def valid_rpm(rpm_file): 262 '''Check that RPM file is valid. 263 Note: This currently only extracts RPM header and presumes the file is 264 OK if that succeeds. Full RPM hashsum check would be nice to have 265 implemented too (FIXME). 266 @param rpm_file RPM file path 267 @return True if file seems OK or False if there is some problem (file 268 can't be read, RPM header is corrupt) 269 ''' ... }}} We would like to have full RPM checksum verification, not just reading the header and saying that it's OK. Find out how to do it and change the method accordingly. It would be nice to have a method argument to enable full checksum verification/fall back to original header verification, because it can be performance intensive operation and it may not be suitable for all our use cases). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/329> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #396: bodhi_util: Could not connect to bodhi!
by fedora-badges 02 Jul '12

02 Jul '12

#396: bodhi_util: Could not connect to bodhi! ---------------------+------------------------ Reporter: kparal | Owner: Type: defect | Status: new Priority: major | Milestone: Hot issues Component: core | Keywords: Blocked By: | Blocking: ---------------------+------------------------ There seems to be some problem when trying to submit certain results to Bodhi: {{{ 12/22 07:10:13 INFO | test:0478| Submitting results (log, email, bodhi) for: depcheck: PASSED; 66 PASSED for kde- l10n-4.7.4-1.fc16,PyKDE4-4.7.4-1.fc16,blinken-4.7.4-1.fc16,cantor-4.7.4-1.fc16,gwenview-4.7.4-1.fc16,kalgebra-4.7.4-1.fc16,kalzium-4.7.4-1.fc16,kamera-4.7.4-1.fc16,kanagram-4.7.4-1.fc16,kate-4.7.4-1.fc16,kbruch-4.7.4-1.fc16,kcolorchooser-4.7.4-1.fc16 ,kde- wallpapers-4.7.4-1.fc16,kdeaccessibility-4.7.4-1.fc16,kdeadmin-4.7.4-1.fc16,kdeartwork-4.7.4-1.fc16,kdebase-4.7.4-2.fc16 ,kdebase-runtime-4.7.4-2.fc16,kdebase- workspace-4.7.4-5.fc16,kdeedu-4.7.4-1.fc16,kdegames-4.7.4-1.fc16,kdegraphics-4.7.4-1.fc16 ,kdegraphics-strigi-analyzer-4.7.4-1.fc16,kdegraphics- thumbnailers-4.7.4-1.fc16,kdelibs-4.7.4-1.fc16,kdemultimedia-4.7.4-2.fc16,kdenetwork-4.7.4-1.fc16,kdepim-4.7.4-1.fc16 ,kdepim-runtime-4.7.4-1.fc16,kdepimlibs-4.7.4-2.fc16,kdeplasma- addons-4.7.4-1.fc16,kdesdk-4.7.4-1.fc16,kdetoys-4.7.4-1.fc16,kdeutils-4.7.4-1.fc16,kgamma-4.7.4-1.fc16,kgeography-4.7.4-1.fc16,khangman-4.7.4-1.fc16,kig-4.7.4-1.fc16,kiten-4.7.4-1.fc16,klettres-4.7.4-1.fc16,kmplot-4.7.4-1.fc16,kolourpaint-4.7.4-1.fc16,konsole-4.7.4-1.fc16 ,kross- interpreters-4.7.4-1.fc16,kruler-4.7.4-1.fc16,ksaneplugin-4.7.4-1.fc16,ksnapshot-4.7.4-1.fc16,kstars-4.7.4-1.fc16,ktouch-4.7.4-1.fc16,kturtle-4.7.4-1.fc16,kwordquiz-4.7.4-1.fc16,libkdcraw-4.7.4-1.fc16,libkdeedu-4.7.4-1.fc16,libkexiv2-4.7.4-1.fc16,libkipi-4.7.4-1.fc16,libksane-4.7.4-1.fc16,marble-4.7.4-1.fc16,okular-4.7.4-1.fc16 ,oxygen-icon- theme-4.7.4-1.fc16,parley-4.7.4-1.fc16,rocs-4.7.4-1.fc16,smokegen-4.7.4-1.fc16,smokekde-4.7.4-1.fc16,smokeqt-4.7.4-1.fc16,step-4.7.4-1.fc16,svgpart-4.7.4-1.fc16 12/22 07:10:13 INFO | test:0361| Log created: /usr/share/autotest/results/default/depcheck/results/kde- l10n-4.7.4-1.fc1.html 12/22 07:10:13 INFO | test:0307| Email not sent: "depcheck: PASSED; 66 PASSED for kde- l10n-4.7.4-1.fc16,PyKDE4-4.7.4-1.fc16,blinken-4.7.4-1.fc16,cantor-4.7.4-1.fc16,gwenview-4.7.4-1.fc16,kalgebra-4.7.4-1.fc16,kalzium-4.7.4-1.fc16,kamera-4.7.4-1.fc16,kanagram-4.7.4-1.fc16,kate-4.7.4-1.fc16,kbruch-4.7.4-1.fc16,kcolorchooser-4.7.4-1.fc16 ,kde- wallpapers-4.7.4-1.fc16,kdeaccessibility-4.7.4-1.fc16,kdeadmin-4.7.4-1.fc16,kdeartwork-4.7.4-1.fc16,kdebase-4.7.4-2.fc16 ,kdebase-runtime-4.7.4-2.fc16,kdebase- workspace-4.7.4-5.fc16,kdeedu-4.7.4-1.fc16,kdegames-4.7.4-1.fc16,kdegraphics-4.7.4-1.fc16 ,kdegraphics-strigi-analyzer-4.7.4-1.fc16,kdegraphics- thumbnailers-4.7.4-1.fc16,kdelibs-4.7.4-1.fc16,kdemultimedia-4.7.4-2.fc16,kdenetwork-4.7.4-1.fc16,kdepim-4.7.4-1.fc16 ,kdepim-runtime-4.7.4-1.fc16,kdepimlibs-4.7.4-2.fc16,kdeplasma- addons-4.7.4-1.fc16,kdesdk-4.7.4-1.fc16,kdetoys-4.7.4-1.fc16,kdeutils-4.7.4-1.fc16,kgamma-4.7.4-1.fc16,kgeography-4.7.4-1.fc16,khangman-4.7.4-1.fc16,kig-4.7.4-1.fc16,kiten-4.7.4-1.fc16,klettres-4.7.4-1.fc16,kmplot-4.7.4-1.fc16,kolourpaint-4.7.4-1.fc16,konsole-4.7.4-1.fc16 ,kross- interpreters-4.7.4-1.fc16,kruler-4.7.4-1.fc16,ksaneplugin-4.7.4-1.fc16,ksnapshot-4.7.4-1.fc16,kstars-4.7.4-1.fc16,ktouch-4.7.4-1.fc16,kturtle-4.7.4-1.fc16,kwordquiz-4.7.4-1.fc16,libkdcraw-4.7.4-1.fc16,libkdeedu-4.7.4-1.fc16,libkexiv2-4.7.4-1.fc16,libkipi-4.7.4-1.fc16,libksane-4.7.4-1.fc16,marble-4.7.4-1.fc16,okular-4.7.4-1.fc16 ,oxygen-icon- theme-4.7.4-1.fc16,parley-4.7.4-1.fc16,rocs-4.7.4-1.fc16,smokegen-4.7.4-1.fc16,smokekde-4.7.4-1.fc16,smokeqt-4.7.4-1.fc16,step-4.7.4-1.fc16,svgpart-4.7.4-1.fc16" 12/22 07:10:17 ERROR|bodhi_util:0278| An error occured: ServerError(http://aqd/bodhi/list, 500, Internal Server Error) 12/22 07:10:17 ERROR|bodhi_util:0279| Could not connect to bodhi! 12/22 07:10:17 ERROR|bodhi_util:0279| Could not post a comment to bodhi }}} This is running on my development machine when using mock_fedorainfra tool. I haven't spotted it yet on either production or staging machine. That indicates that this could be a problem inside the mock_fedorainfra tool. I have seen this bug happening several times, always only for updates consisting of several builds. Investigate whether this is really a problem in mock_fedorainfra and then act on accordingly (fix the problem/report in upstream). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/396> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 5

Playing around with Sikuli
by Tim Flink 05 Jun '12

05 Jun '12

I've been wanting to try out Sikuli (http://www.sikuli.org) for a while now and finally got around to making it build and work on Fedora. The basic concept is to create GUI test cases based on images instead of absolute (x,y) coordinates (like selenium), scripted key sequences (openqa) or using an accessability layer (like dogtail). I've put together a tarball with source and build scripts if anyone else wants to try it out: http://tflink.fedorapeople.org/tools/sikuli/README_build-sikuli.txt http://tflink.fedorapeople.org/tools/sikuli/fedora-sikuli-20120220.tar.gz The distribution method is a really dirty hack but I've been able to build on both F15 and F16 after installing a huge number of -devel dependencies and building all of Sikuli's direct dependencies from source. Xpresser (https://wiki.ubuntu.com/Xpresser) might be another option - it's a re-implementation of the concepts from Sikuli. It seems to be missing some of Sikuli's features but would likely be quite a bit easier to build and integrate. One of the things I have in mind is to attempt automating at least some of the installation test cases because a lot of them pass most of the time and they can be a bit mind-numbing to do over and over again. I'm of the opinion that our human resources could be put to better use than monkey-button-pushing their way through simple test cases. A non-trivial amount of work would be needed before we could fully utilize this (fixing the code so that it is package-able, better integration with VNC etc.) but I'm wondering if it could be a good candidate for automating some of the installation test cases. I've written two test cases for Sikuli thus far: a simple DVD graphical install and graphical firstboot. I had to workaround some timing issues due to occasional lag in the VM's VNC interface but they seem to be working well with F17 Alpha RC2 for now. F17 DVD Basic Graphical Install: http://tflink.fedorapeople.org/tools/sikuli/f17-basicGraphicalDVD_sikuli.ta… F17 Graphical Firstboot: http://tflink.fedorapeople.org/tools/sikuli/f17-graphicalFirstboot_sikuli.t… - NOTE: This only works some of the time due to rhbz#750527 which was closed with no bug comment by the developers. I'm going to bug them about this but will likely update the test case to work with unpredictable order of the firstboot screens. To run those test cases, extract them somewhere and point the Sikuli IDE at the extracted directories. I'm going to be running my basic graphical install testcase for different F17 releases to get an initial feel for how fragile the test cases are and how much effort would be required to maintain these kinds of test cases over a release. Anyhow, long email over. Any thoughts on whether this might be worth investigating farther? Tim

3 4

[AutoQA] #206: Update bodhi to enforce Package Update Acceptance policy
by fedora-badges 26 Mar '12

26 Mar '12

#206: Update bodhi to enforce Package Update Acceptance policy ----------------------------+----------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- In order to enforce the Package Update Acceptance Test Plan, Bodhi will need be modified to reject the push of any package that fails acceptance testing. The acceptance tests may need to send status/data to bodhi in order for it to make decisions about policy (see e.g. ticket #205). Later bodhi might just get data from resultdb (see that milestone for details). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/206> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 7

[AutoQA] #252: Provide all autotest client combinations that we need
by fedora-badges 16 Mar '12

16 Mar '12

#252: Provide all autotest client combinations that we need ------------------------+--------------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: 0.5.0 Component: production | Keywords: ------------------------+--------------------------------------------------- Currently our production server at http://autoqa.fedoraproject.org/ has 6 autotest clients assigned, all bare metals with Fedora 13 installed, with mixed architectures. But we need more. According to https://fedoraproject.org/wiki/Managing_autotest_labels we have two architecture labels, two fedora release labels (the two currently supported Fedora releases, I would omit Rawhide for now) and two virt labels (virtual machine or bare metal). We need machines for every combination of those labels (that is 8 IICC - if I count correctly). And we would like to see a bit more 64bit machines, because we use them for noarch tests. Currently the only test that really requires all those combinations is initscripts test. But there will be more in the future. It is quite some farm, but it seems we'll have to build it. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/252> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #391: re-enable autoqa-results ML
by fedora-badges 16 Mar '12

16 Mar '12

#391: re-enable autoqa-results ML ------------------------+--------------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: 0.8.0 Component: production | Keywords: ------------------------+--------------------------------------------------- Read these threads: * https://fedorahosted.org/pipermail/autoqa- devel/2011-September/002894.html * https://fedorahosted.org/pipermail/autoqa- devel/2011-October/002898.html Find and implement fix so that we can easily inspect the test results again. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/391> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 9

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

autoqa-devel February 2012