autoqa-devel May 2012

autoqa-devel@lists.fedorahosted.org

4 participants
18 discussions

[AutoQA] #396: bodhi_util: Could not connect to bodhi!
by fedora-badges 02 Jul '12

02 Jul '12

#396: bodhi_util: Could not connect to bodhi! ---------------------+------------------------ Reporter: kparal | Owner: Type: defect | Status: new Priority: major | Milestone: Hot issues Component: core | Keywords: Blocked By: | Blocking: ---------------------+------------------------ There seems to be some problem when trying to submit certain results to Bodhi: {{{ 12/22 07:10:13 INFO | test:0478| Submitting results (log, email, bodhi) for: depcheck: PASSED; 66 PASSED for kde- l10n-4.7.4-1.fc16,PyKDE4-4.7.4-1.fc16,blinken-4.7.4-1.fc16,cantor-4.7.4-1.fc16,gwenview-4.7.4-1.fc16,kalgebra-4.7.4-1.fc16,kalzium-4.7.4-1.fc16,kamera-4.7.4-1.fc16,kanagram-4.7.4-1.fc16,kate-4.7.4-1.fc16,kbruch-4.7.4-1.fc16,kcolorchooser-4.7.4-1.fc16 ,kde- wallpapers-4.7.4-1.fc16,kdeaccessibility-4.7.4-1.fc16,kdeadmin-4.7.4-1.fc16,kdeartwork-4.7.4-1.fc16,kdebase-4.7.4-2.fc16 ,kdebase-runtime-4.7.4-2.fc16,kdebase- workspace-4.7.4-5.fc16,kdeedu-4.7.4-1.fc16,kdegames-4.7.4-1.fc16,kdegraphics-4.7.4-1.fc16 ,kdegraphics-strigi-analyzer-4.7.4-1.fc16,kdegraphics- thumbnailers-4.7.4-1.fc16,kdelibs-4.7.4-1.fc16,kdemultimedia-4.7.4-2.fc16,kdenetwork-4.7.4-1.fc16,kdepim-4.7.4-1.fc16 ,kdepim-runtime-4.7.4-1.fc16,kdepimlibs-4.7.4-2.fc16,kdeplasma- addons-4.7.4-1.fc16,kdesdk-4.7.4-1.fc16,kdetoys-4.7.4-1.fc16,kdeutils-4.7.4-1.fc16,kgamma-4.7.4-1.fc16,kgeography-4.7.4-1.fc16,khangman-4.7.4-1.fc16,kig-4.7.4-1.fc16,kiten-4.7.4-1.fc16,klettres-4.7.4-1.fc16,kmplot-4.7.4-1.fc16,kolourpaint-4.7.4-1.fc16,konsole-4.7.4-1.fc16 ,kross- interpreters-4.7.4-1.fc16,kruler-4.7.4-1.fc16,ksaneplugin-4.7.4-1.fc16,ksnapshot-4.7.4-1.fc16,kstars-4.7.4-1.fc16,ktouch-4.7.4-1.fc16,kturtle-4.7.4-1.fc16,kwordquiz-4.7.4-1.fc16,libkdcraw-4.7.4-1.fc16,libkdeedu-4.7.4-1.fc16,libkexiv2-4.7.4-1.fc16,libkipi-4.7.4-1.fc16,libksane-4.7.4-1.fc16,marble-4.7.4-1.fc16,okular-4.7.4-1.fc16 ,oxygen-icon- theme-4.7.4-1.fc16,parley-4.7.4-1.fc16,rocs-4.7.4-1.fc16,smokegen-4.7.4-1.fc16,smokekde-4.7.4-1.fc16,smokeqt-4.7.4-1.fc16,step-4.7.4-1.fc16,svgpart-4.7.4-1.fc16 12/22 07:10:13 INFO | test:0361| Log created: /usr/share/autotest/results/default/depcheck/results/kde- l10n-4.7.4-1.fc1.html 12/22 07:10:13 INFO | test:0307| Email not sent: "depcheck: PASSED; 66 PASSED for kde- l10n-4.7.4-1.fc16,PyKDE4-4.7.4-1.fc16,blinken-4.7.4-1.fc16,cantor-4.7.4-1.fc16,gwenview-4.7.4-1.fc16,kalgebra-4.7.4-1.fc16,kalzium-4.7.4-1.fc16,kamera-4.7.4-1.fc16,kanagram-4.7.4-1.fc16,kate-4.7.4-1.fc16,kbruch-4.7.4-1.fc16,kcolorchooser-4.7.4-1.fc16 ,kde- wallpapers-4.7.4-1.fc16,kdeaccessibility-4.7.4-1.fc16,kdeadmin-4.7.4-1.fc16,kdeartwork-4.7.4-1.fc16,kdebase-4.7.4-2.fc16 ,kdebase-runtime-4.7.4-2.fc16,kdebase- workspace-4.7.4-5.fc16,kdeedu-4.7.4-1.fc16,kdegames-4.7.4-1.fc16,kdegraphics-4.7.4-1.fc16 ,kdegraphics-strigi-analyzer-4.7.4-1.fc16,kdegraphics- thumbnailers-4.7.4-1.fc16,kdelibs-4.7.4-1.fc16,kdemultimedia-4.7.4-2.fc16,kdenetwork-4.7.4-1.fc16,kdepim-4.7.4-1.fc16 ,kdepim-runtime-4.7.4-1.fc16,kdepimlibs-4.7.4-2.fc16,kdeplasma- addons-4.7.4-1.fc16,kdesdk-4.7.4-1.fc16,kdetoys-4.7.4-1.fc16,kdeutils-4.7.4-1.fc16,kgamma-4.7.4-1.fc16,kgeography-4.7.4-1.fc16,khangman-4.7.4-1.fc16,kig-4.7.4-1.fc16,kiten-4.7.4-1.fc16,klettres-4.7.4-1.fc16,kmplot-4.7.4-1.fc16,kolourpaint-4.7.4-1.fc16,konsole-4.7.4-1.fc16 ,kross- interpreters-4.7.4-1.fc16,kruler-4.7.4-1.fc16,ksaneplugin-4.7.4-1.fc16,ksnapshot-4.7.4-1.fc16,kstars-4.7.4-1.fc16,ktouch-4.7.4-1.fc16,kturtle-4.7.4-1.fc16,kwordquiz-4.7.4-1.fc16,libkdcraw-4.7.4-1.fc16,libkdeedu-4.7.4-1.fc16,libkexiv2-4.7.4-1.fc16,libkipi-4.7.4-1.fc16,libksane-4.7.4-1.fc16,marble-4.7.4-1.fc16,okular-4.7.4-1.fc16 ,oxygen-icon- theme-4.7.4-1.fc16,parley-4.7.4-1.fc16,rocs-4.7.4-1.fc16,smokegen-4.7.4-1.fc16,smokekde-4.7.4-1.fc16,smokeqt-4.7.4-1.fc16,step-4.7.4-1.fc16,svgpart-4.7.4-1.fc16" 12/22 07:10:17 ERROR|bodhi_util:0278| An error occured: ServerError(http://aqd/bodhi/list, 500, Internal Server Error) 12/22 07:10:17 ERROR|bodhi_util:0279| Could not connect to bodhi! 12/22 07:10:17 ERROR|bodhi_util:0279| Could not post a comment to bodhi }}} This is running on my development machine when using mock_fedorainfra tool. I haven't spotted it yet on either production or staging machine. That indicates that this could be a problem inside the mock_fedorainfra tool. I have seen this bug happening several times, always only for updates consisting of several builds. Investigate whether this is really a problem in mock_fedorainfra and then act on accordingly (fix the problem/report in upstream). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/396> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 5

Playing around with Sikuli
by Tim Flink 05 Jun '12

05 Jun '12

I've been wanting to try out Sikuli (http://www.sikuli.org) for a while now and finally got around to making it build and work on Fedora. The basic concept is to create GUI test cases based on images instead of absolute (x,y) coordinates (like selenium), scripted key sequences (openqa) or using an accessability layer (like dogtail). I've put together a tarball with source and build scripts if anyone else wants to try it out: http://tflink.fedorapeople.org/tools/sikuli/README_build-sikuli.txt http://tflink.fedorapeople.org/tools/sikuli/fedora-sikuli-20120220.tar.gz The distribution method is a really dirty hack but I've been able to build on both F15 and F16 after installing a huge number of -devel dependencies and building all of Sikuli's direct dependencies from source. Xpresser (https://wiki.ubuntu.com/Xpresser) might be another option - it's a re-implementation of the concepts from Sikuli. It seems to be missing some of Sikuli's features but would likely be quite a bit easier to build and integrate. One of the things I have in mind is to attempt automating at least some of the installation test cases because a lot of them pass most of the time and they can be a bit mind-numbing to do over and over again. I'm of the opinion that our human resources could be put to better use than monkey-button-pushing their way through simple test cases. A non-trivial amount of work would be needed before we could fully utilize this (fixing the code so that it is package-able, better integration with VNC etc.) but I'm wondering if it could be a good candidate for automating some of the installation test cases. I've written two test cases for Sikuli thus far: a simple DVD graphical install and graphical firstboot. I had to workaround some timing issues due to occasional lag in the VM's VNC interface but they seem to be working well with F17 Alpha RC2 for now. F17 DVD Basic Graphical Install: http://tflink.fedorapeople.org/tools/sikuli/f17-basicGraphicalDVD_sikuli.ta… F17 Graphical Firstboot: http://tflink.fedorapeople.org/tools/sikuli/f17-graphicalFirstboot_sikuli.t… - NOTE: This only works some of the time due to rhbz#750527 which was closed with no bug comment by the developers. I'm going to bug them about this but will likely update the test case to work with unpredictable order of the firstboot screens. To run those test cases, extract them somewhere and point the Sikuli IDE at the extracted directories. I'm going to be running my basic graphical install testcase for different F17 releases to get an initial feel for how fragile the test cases are and how much effort would be required to maintain these kinds of test cases over a release. Anyhow, long email over. Any thoughts on whether this might be worth investigating farther? Tim

3 4

[AutoQA] #422: Update repoinfo.conf for F17 release in staging and production
by fedora-badges 30 May '12

30 May '12

#422: Update repoinfo.conf for F17 release in staging and production ---------------------+------------------------ Reporter: tflink | Owner: Type: task | Status: new Priority: major | Milestone: Hot issues Component: core | Keywords: Blocked By: | Blocking: ---------------------+------------------------ Now that F17 has been released, there are some minor changes to the repo locations and repoinfo.conf needs to be updated on staging and production. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/422> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #420: Some builds are tested many times over
by fedora-badges 29 May '12

29 May '12

#420: Some builds are tested many times over ---------------------+------------------------ Reporter: kparal | Owner: Type: defect | Status: new Priority: major | Milestone: Hot issues Component: tests | Keywords: Blocked By: | Blocking: ---------------------+------------------------ mkasik asked me why he has received 19 test reports from rpmlint and rpmguard about poppler-0.18.4-3.fc18 in the last few days. It seems we really tested the same build 19 times: http://autoqa.fedoraproject.org/resultsdb/frontend/search?type=Envr&terms=p… I looked into autoqa.log on production server, the koji-bodhi watcher really detected it as new so many times. And it seems a lot of other builds as well. What might be the cause? I have looked into /var/cache/autoqa/watch-koji- builds/koji_history_prevtimes.cache and there is no timestamp for f18 repository, which I find weird. But maybe I just don't understand the file properly. Anyone feels like taking this? -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/420> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 9

autoqa-stage results now kept for one month
by Kamil Paral 24 May '12

24 May '12

We run into inode issues on autoqa-stage server. I have changed the maximum time to keep old results from 2 months to 1 month. It should be enough for testing purposes.

1 0

[AutoQA] #415: results not being reported to bodhi (Expectation Error)
by fedora-badges 11 May '12

11 May '12

#415: results not being reported to bodhi (Expectation Error) -----------------------+------------------------ Reporter: tflink | Owner: Type: defect | Status: new Priority: critical | Milestone: Hot issues Component: core | Keywords: Blocked By: | Blocking: -----------------------+------------------------ I was just pinged regarding the lack of a depcheck run for a bodhi update - https://admin.fedoraproject.org/updates/abattis-cantarell- fonts-0.0.8-1.fc17 After looking into the logs: - http://autoqa.fedoraproject.org/results/315536-autotest/virt01.qa/debug/cli… we're getting errors when attempting to make the bodhi comment to report the results: {{{ 03/29 14:16:37 ERROR|bodhi_util:0249| An error occured: ServerError(https://admin.fedoraproject.org/updates/list, 417, Expectation Failed) 03/29 14:16:37 ERROR|bodhi_util:0250| Could not connect to bodhi! 03/29 14:16:37 ERROR|bodhi_util:0250| Could not post a comment to bodhi }}} I'm not sure what's going on here but will investigate and update here as I learn more. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/415> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 6

autoqa clients update
by Kamil Paral 11 May '12

11 May '12

I have updated all test clients associated with the staging server and also the server itself. That should resolve this ticket: https://fedorahosted.org/autoqa/ticket/415 . If you see any problems (any, not just related to that ticket), let me know. If everything looks OK, I'll update the production infrastructure tomorrow as well.

1 1

autoqa-results usage questions
by Tim Flink 02 May '12

02 May '12

I guess that the autoqa-results list is still causing some problems for fedora-infra. I'm honestly not sure what the problem is with our current usage of the list - I didn't really get a clear answer to that, just that they want to understand more about our usage in case there is a better solution than what we're currently using. The old archives are going to be deleted but I've downloaded a compressed copy of the old archives in case we ever decide to parse them. I've been asked for the following information: - What do we do with the list - What do we plan to do with the list With the recent changes that Kamil made, the only thing that I can think of is "notification of failed or questionable test runs". Is there anything that I'm missing? Tim

2 4

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

autoqa-devel May 2012