I've been wanting to try out Sikuli (http://www.sikuli.org) for a while now and finally got around to making it build and work on Fedora.
The basic concept is to create GUI test cases based on images instead of absolute (x,y) coordinates (like selenium), scripted key sequences (openqa) or using an accessability layer (like dogtail).
I've put together a tarball with source and build scripts if anyone else wants to try it out:
http://tflink.fedorapeople.org/tools/sikuli/README_build-sikuli.txt http://tflink.fedorapeople.org/tools/sikuli/fedora-sikuli-20120220.tar.gz
The distribution method is a really dirty hack but I've been able to build on both F15 and F16 after installing a huge number of -devel dependencies and building all of Sikuli's direct dependencies from source. Xpresser (https://wiki.ubuntu.com/Xpresser) might be another option - it's a re-implementation of the concepts from Sikuli. It seems to be missing some of Sikuli's features but would likely be quite a bit easier to build and integrate.
One of the things I have in mind is to attempt automating at least some of the installation test cases because a lot of them pass most of the time and they can be a bit mind-numbing to do over and over again. I'm of the opinion that our human resources could be put to better use than monkey-button-pushing their way through simple test cases.
A non-trivial amount of work would be needed before we could fully utilize this (fixing the code so that it is package-able, better integration with VNC etc.) but I'm wondering if it could be a good candidate for automating some of the installation test cases.
I've written two test cases for Sikuli thus far: a simple DVD graphical install and graphical firstboot. I had to workaround some timing issues due to occasional lag in the VM's VNC interface but they seem to be working well with F17 Alpha RC2 for now.
F17 DVD Basic Graphical Install: http://tflink.fedorapeople.org/tools/sikuli/f17-basicGraphicalDVD_sikuli.tar...
F17 Graphical Firstboot: http://tflink.fedorapeople.org/tools/sikuli/f17-graphicalFirstboot_sikuli.ta... - NOTE: This only works some of the time due to rhbz#750527 which was closed with no bug comment by the developers. I'm going to bug them about this but will likely update the test case to work with unpredictable order of the firstboot screens.
To run those test cases, extract them somewhere and point the Sikuli IDE at the extracted directories.
I'm going to be running my basic graphical install testcase for different F17 releases to get an initial feel for how fragile the test cases are and how much effort would be required to maintain these kinds of test cases over a release.
Anyhow, long email over. Any thoughts on whether this might be worth investigating farther?
Tim
Hi Tim,
I know that folks at Spice QE have been playing around with Sikuli. I can imagine you'd like to use this for automating anaconda tests especially in virtual environments in which spice folks have been operating too. If you'd like some tips etc., contacting Lubos Kocman might be helpful.
-- Vita Humpa Desktop QE
----- Original Message -----
I've been wanting to try out Sikuli (http://www.sikuli.org) for a while now and finally got around to making it build and work on Fedora.
The basic concept is to create GUI test cases based on images instead of absolute (x,y) coordinates (like selenium), scripted key sequences (openqa) or using an accessability layer (like dogtail).
I've put together a tarball with source and build scripts if anyone else wants to try it out:
http://tflink.fedorapeople.org/tools/sikuli/README_build-sikuli.txt http://tflink.fedorapeople.org/tools/sikuli/fedora-sikuli-20120220.tar.gz
The distribution method is a really dirty hack but I've been able to build on both F15 and F16 after installing a huge number of -devel dependencies and building all of Sikuli's direct dependencies from source. Xpresser (https://wiki.ubuntu.com/Xpresser) might be another option - it's a re-implementation of the concepts from Sikuli. It seems to be missing some of Sikuli's features but would likely be quite a bit easier to build and integrate.
One of the things I have in mind is to attempt automating at least some of the installation test cases because a lot of them pass most of the time and they can be a bit mind-numbing to do over and over again. I'm of the opinion that our human resources could be put to better use than monkey-button-pushing their way through simple test cases.
A non-trivial amount of work would be needed before we could fully utilize this (fixing the code so that it is package-able, better integration with VNC etc.) but I'm wondering if it could be a good candidate for automating some of the installation test cases.
I've written two test cases for Sikuli thus far: a simple DVD graphical install and graphical firstboot. I had to workaround some timing issues due to occasional lag in the VM's VNC interface but they seem to be working well with F17 Alpha RC2 for now.
F17 DVD Basic Graphical Install: http://tflink.fedorapeople.org/tools/sikuli/f17-basicGraphicalDVD_sikuli.tar...
F17 Graphical Firstboot: http://tflink.fedorapeople.org/tools/sikuli/f17-graphicalFirstboot_sikuli.ta...
- NOTE: This only works some of the time due to rhbz#750527 which
was closed with no bug comment by the developers. I'm going to bug them about this but will likely update the test case to work with unpredictable order of the firstboot screens.
To run those test cases, extract them somewhere and point the Sikuli IDE at the extracted directories.
I'm going to be running my basic graphical install testcase for different F17 releases to get an initial feel for how fragile the test cases are and how much effort would be required to maintain these kinds of test cases over a release.
Anyhow, long email over. Any thoughts on whether this might be worth investigating farther?
Tim
autoqa-devel mailing list autoqa-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/autoqa-devel
On Tue, 21 Feb 2012 05:00:53 -0500 (EST) Vitezslav Humpa vhumpa@redhat.com wrote:
Hi Tim,
I know that folks at Spice QE have been playing around with Sikuli. I can imagine you'd like to use this for automating anaconda tests especially in virtual environments in which spice folks have been operating too. If you'd like some tips etc., contacting Lubos Kocman might be helpful.
Good to know - adding the ability to connect to a VM over VNC or Spice was one of the things that I wanted to look into.
Thanks for the tip, I'll make sure to contact Lubos.
Tim
Thank you for the package and instructions.
After everything builds I am left with two files in the folder where you need to run the shell script from. (forms-1.2.1.jar & sikuli-ide.sh) When I run the "sh sikuli-ide.sh", I get this error: "Unable to access jarfile ./sikuli-ide.jar"
Any ideas why? (I am running Fedora 15 (64bit))
Thanks!
On Tue, 29 May 2012 21:44:45 +0000 (UTC) Ryan rhnrksn@gmail.com wrote:
Thank you for the package and instructions.
After everything builds I am left with two files in the folder where you need to run the shell script from. (forms-1.2.1.jar & sikuli-ide.sh) When I run the "sh sikuli-ide.sh", I get this error: "Unable to access jarfile ./sikuli-ide.jar"
Any ideas why? (I am running Fedora 15 (64bit))
Off the top of my head, no but I don't think that I tried building it on F15.
Do the jar files exist? Were there any errors when building?
Tim
autoqa-devel@lists.fedorahosted.org
-
Ryan -
Tim Flink -
Vitezslav Humpa