[AutoQA] #251: Move autotest applications into /usr/*bin
by fedora-badges
#251: Move autotest applications into /usr/*bin
-----------------------+----------------------------------------------------
Reporter: jlaska | Owner:
Type: task | Status: new
Priority: minor | Milestone: Packaging, Review, & Deployment
Component: packaging | Keywords:
-----------------------+----------------------------------------------------
The autotest package should supply binaries in /usr/bin/ ... so we can
rely on PATH and not have to call autotest with the full-path
(/usr/share/autotest/client/bin/autotest) when scheduling jobs in
'/usr/bin/autoqa'.
Alternatively, the autotest package could drop a script that updates PATH
appropriately into /etc/profile.d/autotest. But yuck.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/251>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
11 years, 7 months
[AutoQA] #414: rats_install: run twice, once for updates, once for updates+updates-testing
by fedora-badges
#414: rats_install: run twice, once for updates, once for updates+updates-testing
--------------------------+-------------------------------
Reporter: kparal | Owner:
Type: enhancement | Status: new
Priority: minor | Milestone: Nice to have soon
Component: tests | Keywords:
Blocked By: | Blocking:
--------------------------+-------------------------------
We decided it would be beneficial to run rats_install twice. Once for
stable updates only, once for main+updates, once for main+updates+updates-
testing. Currently we're running just the second option. The discussion is
here:
https://fedoraproject.org/wiki/QA/Meetings/20120319#t15:55:42
Change rats_install to execute twice and report both variants. The easiest
way is probably to modify 'control' file to execute the job twice and
modify some input argument to distinguish the two cases. Also we need to
change the summary accordingly so that we can easily distinguish that in
our ResultsDB view.
Marking "Nice to have soon" if we can do it before F17 Final, otherwise
it'll probably go into "Future tasks".
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/414>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
11 years, 8 months
[AutoQA] #157: conflicts: requested arch not respected
by fedora-badges
#157: conflicts: requested arch not respected
--------------------+-------------------------------------------------------
Reporter: kparal | Owner:
Type: defect | Status: new
Priority: major | Milestone:
Component: tests | Version: 1.0
Keywords: |
--------------------+-------------------------------------------------------
This is a bug in the potential_conflict.py script in tests/conflicts/.
Usage suggests using --arch options for specifying requested architecture
to be tested. But if you look at the attached log, the first and second
output match (that's correct), but the third doesn't match, even though it
should. Everything was tested on x86_64 machine.
Also from the usage line it is not clear whether --arch is related only to
particular yum repository selection (replacing $basearch keyword), or
whether it also relates to packages built for one architecture but being
in different repo (e.g. i686 packages are very often in x86_64 repo).
I have looked into the source code of the script and it seems that the
--arch option is not used at all, just printed out in the usage help??
Seth, you're listed as script author. Any comments on that? Thanks.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/157>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
11 years, 8 months
[AutoQA] #329: implement RPM checksum verification
by fedora-badges
#329: implement RPM checksum verification
-------------------------+--------------------------------------------------
Reporter: kparal | Owner:
Type: enhancement | Status: new
Priority: major | Milestone: Finger Food
Component: core | Keywords:
-------------------------+--------------------------------------------------
Currently this method is in autoqa.util module:
{{{
261 def valid_rpm(rpm_file):
262 '''Check that RPM file is valid.
263 Note: This currently only extracts RPM header and presumes the
file is
264 OK if that succeeds. Full RPM hashsum check would be nice to
have
265 implemented too (FIXME).
266 @param rpm_file RPM file path
267 @return True if file seems OK or False if there is some
problem (file
268 can't be read, RPM header is corrupt)
269 '''
...
}}}
We would like to have full RPM checksum verification, not just reading the
header and saying that it's OK. Find out how to do it and change the
method accordingly. It would be nice to have a method argument to enable
full checksum verification/fall back to original header verification,
because it can be performance intensive operation and it may not be
suitable for all our use cases).
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/329>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
11 years, 9 months
[AutoQA] #396: bodhi_util: Could not connect to bodhi!
by fedora-badges
#396: bodhi_util: Could not connect to bodhi!
---------------------+------------------------
Reporter: kparal | Owner:
Type: defect | Status: new
Priority: major | Milestone: Hot issues
Component: core | Keywords:
Blocked By: | Blocking:
---------------------+------------------------
There seems to be some problem when trying to submit certain results to
Bodhi:
{{{
12/22 07:10:13 INFO | test:0478| Submitting results (log, email,
bodhi) for: depcheck: PASSED; 66 PASSED for kde-
l10n-4.7.4-1.fc16,PyKDE4-4.7.4-1.fc16,blinken-4.7.4-1.fc16,cantor-4.7.4-1.fc16,gwenview-4.7.4-1.fc16,kalgebra-4.7.4-1.fc16,kalzium-4.7.4-1.fc16,kamera-4.7.4-1.fc16,kanagram-4.7.4-1.fc16,kate-4.7.4-1.fc16,kbruch-4.7.4-1.fc16,kcolorchooser-4.7.4-1.fc16
,kde-
wallpapers-4.7.4-1.fc16,kdeaccessibility-4.7.4-1.fc16,kdeadmin-4.7.4-1.fc16,kdeartwork-4.7.4-1.fc16,kdebase-4.7.4-2.fc16
,kdebase-runtime-4.7.4-2.fc16,kdebase-
workspace-4.7.4-5.fc16,kdeedu-4.7.4-1.fc16,kdegames-4.7.4-1.fc16,kdegraphics-4.7.4-1.fc16
,kdegraphics-strigi-analyzer-4.7.4-1.fc16,kdegraphics-
thumbnailers-4.7.4-1.fc16,kdelibs-4.7.4-1.fc16,kdemultimedia-4.7.4-2.fc16,kdenetwork-4.7.4-1.fc16,kdepim-4.7.4-1.fc16
,kdepim-runtime-4.7.4-1.fc16,kdepimlibs-4.7.4-2.fc16,kdeplasma-
addons-4.7.4-1.fc16,kdesdk-4.7.4-1.fc16,kdetoys-4.7.4-1.fc16,kdeutils-4.7.4-1.fc16,kgamma-4.7.4-1.fc16,kgeography-4.7.4-1.fc16,khangman-4.7.4-1.fc16,kig-4.7.4-1.fc16,kiten-4.7.4-1.fc16,klettres-4.7.4-1.fc16,kmplot-4.7.4-1.fc16,kolourpaint-4.7.4-1.fc16,konsole-4.7.4-1.fc16
,kross-
interpreters-4.7.4-1.fc16,kruler-4.7.4-1.fc16,ksaneplugin-4.7.4-1.fc16,ksnapshot-4.7.4-1.fc16,kstars-4.7.4-1.fc16,ktouch-4.7.4-1.fc16,kturtle-4.7.4-1.fc16,kwordquiz-4.7.4-1.fc16,libkdcraw-4.7.4-1.fc16,libkdeedu-4.7.4-1.fc16,libkexiv2-4.7.4-1.fc16,libkipi-4.7.4-1.fc16,libksane-4.7.4-1.fc16,marble-4.7.4-1.fc16,okular-4.7.4-1.fc16
,oxygen-icon-
theme-4.7.4-1.fc16,parley-4.7.4-1.fc16,rocs-4.7.4-1.fc16,smokegen-4.7.4-1.fc16,smokekde-4.7.4-1.fc16,smokeqt-4.7.4-1.fc16,step-4.7.4-1.fc16,svgpart-4.7.4-1.fc16
12/22 07:10:13 INFO | test:0361| Log created:
/usr/share/autotest/results/default/depcheck/results/kde-
l10n-4.7.4-1.fc1.html
12/22 07:10:13 INFO | test:0307| Email not sent: "depcheck: PASSED;
66 PASSED for kde-
l10n-4.7.4-1.fc16,PyKDE4-4.7.4-1.fc16,blinken-4.7.4-1.fc16,cantor-4.7.4-1.fc16,gwenview-4.7.4-1.fc16,kalgebra-4.7.4-1.fc16,kalzium-4.7.4-1.fc16,kamera-4.7.4-1.fc16,kanagram-4.7.4-1.fc16,kate-4.7.4-1.fc16,kbruch-4.7.4-1.fc16,kcolorchooser-4.7.4-1.fc16
,kde-
wallpapers-4.7.4-1.fc16,kdeaccessibility-4.7.4-1.fc16,kdeadmin-4.7.4-1.fc16,kdeartwork-4.7.4-1.fc16,kdebase-4.7.4-2.fc16
,kdebase-runtime-4.7.4-2.fc16,kdebase-
workspace-4.7.4-5.fc16,kdeedu-4.7.4-1.fc16,kdegames-4.7.4-1.fc16,kdegraphics-4.7.4-1.fc16
,kdegraphics-strigi-analyzer-4.7.4-1.fc16,kdegraphics-
thumbnailers-4.7.4-1.fc16,kdelibs-4.7.4-1.fc16,kdemultimedia-4.7.4-2.fc16,kdenetwork-4.7.4-1.fc16,kdepim-4.7.4-1.fc16
,kdepim-runtime-4.7.4-1.fc16,kdepimlibs-4.7.4-2.fc16,kdeplasma-
addons-4.7.4-1.fc16,kdesdk-4.7.4-1.fc16,kdetoys-4.7.4-1.fc16,kdeutils-4.7.4-1.fc16,kgamma-4.7.4-1.fc16,kgeography-4.7.4-1.fc16,khangman-4.7.4-1.fc16,kig-4.7.4-1.fc16,kiten-4.7.4-1.fc16,klettres-4.7.4-1.fc16,kmplot-4.7.4-1.fc16,kolourpaint-4.7.4-1.fc16,konsole-4.7.4-1.fc16
,kross-
interpreters-4.7.4-1.fc16,kruler-4.7.4-1.fc16,ksaneplugin-4.7.4-1.fc16,ksnapshot-4.7.4-1.fc16,kstars-4.7.4-1.fc16,ktouch-4.7.4-1.fc16,kturtle-4.7.4-1.fc16,kwordquiz-4.7.4-1.fc16,libkdcraw-4.7.4-1.fc16,libkdeedu-4.7.4-1.fc16,libkexiv2-4.7.4-1.fc16,libkipi-4.7.4-1.fc16,libksane-4.7.4-1.fc16,marble-4.7.4-1.fc16,okular-4.7.4-1.fc16
,oxygen-icon-
theme-4.7.4-1.fc16,parley-4.7.4-1.fc16,rocs-4.7.4-1.fc16,smokegen-4.7.4-1.fc16,smokekde-4.7.4-1.fc16,smokeqt-4.7.4-1.fc16,step-4.7.4-1.fc16,svgpart-4.7.4-1.fc16"
12/22 07:10:17 ERROR|bodhi_util:0278| An error occured:
ServerError(http://aqd/bodhi/list, 500, Internal Server Error)
12/22 07:10:17 ERROR|bodhi_util:0279| Could not connect to bodhi!
12/22 07:10:17 ERROR|bodhi_util:0279| Could not post a comment to bodhi
}}}
This is running on my development machine when using mock_fedorainfra
tool. I haven't spotted it yet on either production or staging machine.
That indicates that this could be a problem inside the mock_fedorainfra
tool. I have seen this bug happening several times, always only for
updates consisting of several builds.
Investigate whether this is really a problem in mock_fedorainfra and then
act on accordingly (fix the problem/report in upstream).
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/396>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
11 years, 10 months
Playing around with Sikuli
by Tim Flink
I've been wanting to try out Sikuli (http://www.sikuli.org) for a while
now and finally got around to making it build and work on Fedora.
The basic concept is to create GUI test cases based on images instead of
absolute (x,y) coordinates (like selenium), scripted key sequences
(openqa) or using an accessability layer (like dogtail).
I've put together a tarball with source and build scripts if anyone
else wants to try it out:
http://tflink.fedorapeople.org/tools/sikuli/README_build-sikuli.txt
http://tflink.fedorapeople.org/tools/sikuli/fedora-sikuli-20120220.tar.gz
The distribution method is a really dirty hack but I've been able to
build on both F15 and F16 after installing a huge number of -devel
dependencies and building all of Sikuli's direct dependencies from
source. Xpresser (https://wiki.ubuntu.com/Xpresser) might be another
option - it's a re-implementation of the concepts from Sikuli. It seems
to be missing some of Sikuli's features but would likely be quite a bit
easier to build and integrate.
One of the things I have in mind is to attempt automating at least some
of the installation test cases because a lot of them pass most of the
time and they can be a bit mind-numbing to do over and over again. I'm
of the opinion that our human resources could be put to better use than
monkey-button-pushing their way through simple test cases.
A non-trivial amount of work would be needed before we could fully
utilize this (fixing the code so that it is package-able, better
integration with VNC etc.) but I'm wondering if it could be a good
candidate for automating some of the installation test cases.
I've written two test cases for Sikuli thus far: a simple DVD graphical
install and graphical firstboot. I had to workaround some timing issues
due to occasional lag in the VM's VNC interface but they seem to be
working well with F17 Alpha RC2 for now.
F17 DVD Basic Graphical Install:
http://tflink.fedorapeople.org/tools/sikuli/f17-basicGraphicalDVD_sikuli....
F17 Graphical Firstboot:
http://tflink.fedorapeople.org/tools/sikuli/f17-graphicalFirstboot_sikuli...
- NOTE: This only works some of the time due to rhbz#750527 which was
closed with no bug comment by the developers. I'm going to bug them
about this but will likely update the test case to work with
unpredictable order of the firstboot screens.
To run those test cases, extract them somewhere and point the Sikuli
IDE at the extracted directories.
I'm going to be running my basic graphical install testcase for
different F17 releases to get an initial feel for how fragile the test
cases are and how much effort would be required to maintain these kinds
of test cases over a release.
Anyhow, long email over. Any thoughts on whether this might be worth
investigating farther?
Tim
11 years, 10 months