autoqa-devel November 2009

autoqa-devel@lists.fedorahosted.org

4 participants
12 discussions

TRAC tickets bcc'd to the mailing list?
by James Laska 05 Jan '10

05 Jan '10

Hey folks, Would people be interested in having trac ticket updates automatically bcc'd to the mailing list? I thought it might increase the visibility the milestones people are working on. But wasn't sure if people would consider it SPAM. Thanks, James

2 2

[PATCH] provide --help and --dry-run for all watchers
by Kamil Paral 02 Dec '09

02 Dec '09

All the watchers have standardized --help and --dry-run now. This will ease the development of your own tests. Koji watcher was modified not to save any information in dry run (in compliance with other watchers). --- hooks/post-koji-build/watch-koji-builds.py | 30 ++++++++++++++++++--------- hooks/post-repo-update/watch-repos.py | 10 +++++++- hooks/post-tree-compose/watch-composes.py | 10 +++++++- 3 files changed, 36 insertions(+), 14 deletions(-) diff --git a/hooks/post-koji-build/watch-koji-builds.py b/hooks/post-koji-build/watch-koji-builds.py index 6a27aee..19b6a60 100755 --- a/hooks/post-koji-build/watch-koji-builds.py +++ b/hooks/post-koji-build/watch-koji-builds.py @@ -16,6 +16,7 @@ import time import pickle import xmlrpclib import subprocess +import optparse from autoqa import koji_utils # Directory where we cache info between runs @@ -60,7 +61,7 @@ def save_list(pkglist): pickle.dump(pkglist, out) out.close() -def new_tagged_builds_since(session, taglist, prevtime): +def new_builds_since(session, taglist, prevtime): untagged_builds = [] tagged_builds = {} for tag in taglist: @@ -78,19 +79,25 @@ def new_tagged_builds_since(session, taglist, prevtime): untagged_builds.append(nvr) #print "untagged: %s" % " ".join(untagged_builds) #print "tagged: %s" % " ".join(tagged_builds) - save_list(untagged_builds) - return tagged_builds + return (tagged_builds, untagged_builds) if __name__ == '__main__': - # TODO: optparse for verbose/prevtime/dryrun - verbose = ('--verbose' in sys.argv) - dryrun = ('--dryrun' in sys.argv) + # Set up the option parser + # TODO: optparse for prevtime + parser = optparse.OptionParser(description='Script to watch a set of koji \ +tags for new builds and kick off tests when new builds/packages are found.') + parser.add_option('--dry-run', action='store_true', dest='dryrun', + help='Do not actually execute commands, just show what would be done') + parser.add_option('--verbose', action='store_true', + help='Print extra information') + (opts, args) = parser.parse_args() + # load prevtime from some saved timestamp prevtime = get_prevtime() if not prevtime: print "No previous run - checking builds in the past 3 hours" prevtime = time.time() - 10800 - if verbose: + if opts.verbose: print "Looking up builds since %s" % time.ctime(prevtime) # Set up the koji connection kojiopts = {} # Possible items: user, password, debug_xmlrpc, debug.. @@ -99,8 +106,11 @@ if __name__ == '__main__': tagged_builds = [] try: session.ensure_connection() - new_builds = new_tagged_builds_since(session, taglist, prevtime) - for tag, builds in new_builds.items(): + (tagged_builds, untagged_builds) = new_builds_since(session, + taglist, prevtime) + if not opts.dryrun: + save_list(untagged_builds) + for tag, builds in tagged_builds.items(): for b in builds: # Get a list of all package arches in this build arches = [r['arch'] for r in session.listRPMs(b['build_id'])] @@ -116,7 +126,7 @@ if __name__ == '__main__': for arch in testarches: harnesscall += ['--arch', arch] harnesscall.append(b['nvr']) - if dryrun: + if opts.dryrun: print " ".join(harnesscall) continue subprocess.call(harnesscall) diff --git a/hooks/post-repo-update/watch-repos.py b/hooks/post-repo-update/watch-repos.py index 7f96264..3de5083 100755 --- a/hooks/post-repo-update/watch-repos.py +++ b/hooks/post-repo-update/watch-repos.py @@ -13,8 +13,14 @@ import os import sys import subprocess from autoqa.repoinfo import repoinfo +import optparse -dryrun = ('--dryrun' in sys.argv) +# Set up the option parser +parser = optparse.OptionParser(description='A utility to watch a set of repos \ +for changes and to kick off tests if the repo changes.') +parser.add_option('--dry-run', action='store_true', dest='dryrun', + help='Do not actually execute commands, just show what would be done') +(opts, args) = parser.parse_args() # Set the default arch to our placeholder, '%a' repoinfo.setarch('%%a') # two %% because of ConfigParser interpolation @@ -69,7 +75,7 @@ for reponame, arches in testable.items(): harnesscall += ['--parent', repoinfo.get(preponame,'url')] harnesscall.append(repo['url']) - if dryrun: + if opts.dryrun: print ' '.join(harnesscall) continue diff --git a/hooks/post-tree-compose/watch-composes.py b/hooks/post-tree-compose/watch-composes.py index 868cd71..6fc3632 100755 --- a/hooks/post-tree-compose/watch-composes.py +++ b/hooks/post-tree-compose/watch-composes.py @@ -13,8 +13,14 @@ import os import sys import subprocess from autoqa.repoinfo import repoinfo +import optparse -dryrun = ('--dryrun' in sys.argv) +# Set up the option parser +parser = optparse.OptionParser(description='A utility to watch a set of \ +compose trees for changes and to kick off tests if the compose changes.') +parser.add_option('--dry-run', action='store_true', dest='dryrun', + help='Do not actually execute commands, just show what would be done') +(opts, args) = parser.parse_args() # Set the default arch to our placeholder, '%a' repoinfo.setarch('%%a') # two %% because of ConfigParser interpolation @@ -70,7 +76,7 @@ for reponame, arches in testable.items(): for arch in arches: harnesscall += ['--arch', arch] harnesscall.append(repo['url']) - if dryrun: + if opts.dryrun: print ' '.join(harnesscall) continue -- 1.6.5.2

2 1

New branch - israwhidebroken
by James Laska 25 Nov '09

25 Nov '09

Greetings, Per https://fedorahosted.org/autoqa/ticket/91, I have created a new branch 'israwhidebroken' [1]. These changes add a new sub-package: autoqa-israwhidebroken which provides the TurboGears app that rats_sanity and rats_install publish their test results to [2]. There are a few documentation changes still to come to outline how to prepare the a sqlite database, but overall this should be ready for review. Thanks to Toshio for assistance cleaning up the WSGI files. I believe the next steps are to ... * Review/comment on branch * Merge changes into autoqa-0.3 (hopefully early next week) Thanks, James [1] http://git.fedorahosted.org/git/?p=autoqa.git;a=shortlog;h=refs/heads/israw… [2] http://jlaska.fedorapeople.org/irb.png

1 0

Re: [PATCH] keep control files on dry run
by Kamil Paral 25 Nov '09

25 Nov '09

Oh, oh, still learning. I didn't separate the headers from body text by one empty line, so first half of my message is missing. Recreating that part from memory: Hello, this patch implies --keep-control-file when --dry-run is used. The --dry-run itself is great, but you usually see in the output only lines like these: (continue reading quoted part) ----- "Kamil Páral" <kparal(a)redhat.com> wrote: > /usr/share/autotest/client/bin/autotest --verbose -t > post-repo-update:repoclosure.x86_64 /tmp/autoqa-control.O1kKrU > > To really inspect what will be run (which is the reason why you run > dry) you > still need to inspect the control file. Therefore I think the > --dry-run should > quite conveniently also imply the --keep-control-file (which is a > little long > option to have to type it everytime). > > What do you think, should I commit it or not? > _______________________________________________ > autoqa-devel mailing list > autoqa-devel(a)lists.fedorahosted.org > https://fedorahosted.org/mailman/listinfo/autoqa-devel

1 0

[PATCH] keep control files on dry run
by Kamil Paral 25 Nov '09

25 Nov '09

/usr/share/autotest/client/bin/autotest --verbose -t post-repo-update:repoclosure.x86_64 /tmp/autoqa-control.O1kKrU To really inspect what will be run (which is the reason why you run dry) you still need to inspect the control file. Therefore I think the --dry-run should quite conveniently also imply the --keep-control-file (which is a little long option to have to type it everytime). What do you think, should I commit it or not?

1 1

AutoQA wiki front page
by Kamil Paral 19 Nov '09

19 Nov '09

I have made some changes to the AutoQA wiki page [1]. The idea is to have some kind of guidepost, where you easily find part relevant for you and continue to more detailed description at linked page. I have tried to split our users to two parts - end users (using AutoQA website to see results or receive our notifications) and developers (which contribute to tests and AutoQA itself). I don't know if I have separated them (and described them) in an understandable way. If you have some improvements, please don't hesitate to adjust the page. What we want to see included on the front page, what we don't want to see there? All suggestions welcome! Thanks, Kamil [1] https://fedoraproject.org/wiki/AutoQA

2 5

Re: [PATCH] Fix conflicting parser options on post-koji-build
by Kamil Paral 14 Nov '09

14 Nov '09

----- "Will Woods" <wwoods(a)redhat.com> wrote: > > This was already fixed in the wwoods-autotest branch by removing the > -t, Ah, I have forgotten to look at that. > but using -g is a good idea, so I added that. Thanks! > I know you have big changes in your branch, but if you could merge into master soon, it would be great. I need to work on the koji watcher and rpmguard. Thanks.

2 1

Proposal: let's make test development easier
by Kamil Paral 13 Nov '09

13 Nov '09

I have been thinking about ticket #52: https://fedorahosted.org/autoqa/ticket/52 The main idea behind it is: "I have the test script written, let me see if it actually works in AutoQA." I don't know if my patch is the best approach available, but it served my needs. Imagine a situation: You have a working test. You have written a control file and a test wrapper. Now, how do you know it works? You know your test works, but it is hard to ensure that your test will get correct command-line parameters, because you have no way to try it. Forwarding (and transformation) of parameters works in this way: watcher -> autoqa -> -> hook -> control file -> wrapper -> -> your test. Oof, that's a long way where many mistakes can happen. The only current way is to install autotest-server, which is painful. I would like to lower this entrance barrier by avoiding that. I have found that many watcher scripts have hidden undocumented option --dryrun. Then they only print command they would have run. That's great. Still there is a long way from this command to actual test execution command. Also, there are some use cases in which you can't really separate your test from the watcher. For example the repoclosure test - it can work perfect when you test it manually, but it depends on some repository metadata that the _watcher_ downloads. Then you simply don't know if all the data is downloaded ok, in correct formats, correct locations, etc. And if you don't want to hack and slash the watcher source code, then the only method is to run the whole thing. Because Will is developing some more libraries for tests, the code can soon became even more intertwined and there can be even bigger desire for running the whole thing together. The last use case is when contributing to autoqa directly and wanting to see "something" running. Just to have some idea how it works. Which takes me to a list of possible improvements: 1. Standardize "--dry-run" option for the all watchers, document it (let them have --help). It helps and it is easy. 2a. Provide "--dry-run" also for the autoqa itself, so it wouldn't actually run the tests, it would just print all the commands that would be run for the particular hook (so printing lines like "repoclosure --foo=bar ..." etc). This is hard, because autotest is assembling the execution commands from our provided control files/wrappers. I haven't found any --dry-run support in autotest. Therefore I think this can't be done. But if it could be, it would be cool. 2b. Allow people to run single hook/test without the need of autotest-server. That can be most easily done by directly calling autotest-client. This is just for development/demonstrative purposes. Just to see if everything is working ok and if the new test can be deployed on the "real" autoqa instance. This can be done as config option (my patch), command-line option, whatever. Suggestions, comments, remarks, corrections, anything... welcome.

2 1

[PATCH] Fix conflicting parser options on post-koji-build
by Kamil Paral 13 Nov '09

13 Nov '09

This renames -t/--tag option of post-koji-build watcher to -g/--tag option, because -t already exists on autoqa as -t/--test. Therefore it fixes crash that would occur when running eg.: $ autoqa post-koji-build -l --- hooks/post-koji-build/hook.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/hooks/post-koji-build/hook.py b/hooks/post-koji-build/hook.py index 347dc77..68f3e5a 100644 --- a/hooks/post-koji-build/hook.py +++ b/hooks/post-koji-build/hook.py @@ -15,7 +15,7 @@ def extend_parser(parser): '''Extend the given OptionParser object with settings for this hook.''' parser.set_usage('%%prog %s [options] PACKAGENVR' % name) group = optparse.OptionGroup(parser, '%s options' % name) - group.add_option('-t', '--tag', default='', + group.add_option('-g', '--tag', default='', help='Koji tag that has just been applied to this new build') group.add_option('-n', '--name', default='', help='The name of the package, by itself') -- 1.6.2.5

2 2

koji watcher taglist
by Kamil Paral 11 Nov '09

11 Nov '09

Hello, currently the watched taglist for koji watcher looks like this: taglist = set(('dist-f10-updates-candidate', 'dist-f11-updates-candidate', 'dist-f12-updates-candidate', )) I'm new to Koji, but I have discussed these things with Daniel Mach and I think the current koji watcher implementation isn't right. The problem is that the dist-fxx-updates-candidate tag is valid for only a while. After that it is changed to dist-fxx-updates tag (build goes directly to stable updates) or it is changed to dist-fxx-updates-testing (build goes to testing updates). If we have not detected that new build in the meantime, we have lost that information and won't know about it anymore. One solution is to check all new builds for dist-fxx-updates-candidate, dist-fxx-updates-testing and dist-fxx-updates tags. Another (better) solution is to check for all new builds and filter for only those packages, which have dist-fxx-updates-candidate tag in *their history* (not necessarily as current tag, which does current implementation). To see the concept have a look at: $ koji list-tag-history --build wormux-0.8.3-1.fc11 There is certainly some API to achieve that. Will, if you want, I can try to provide a patch for that. Daniel also suggested a few alternative mechanisms for new build notification. If our tests would be quick, we could modify koji tasks (example here: http://koji.fedoraproject.org/koji/taskinfo?taskID=1789344 ) and add a Descendant Task to test the package. If our tests would take longer (which I assume in the longer run) we could modify the koji tasks and add at least a notification to our autoqa server that a new build has been done. This way we wouldn't have to query periodically but we would be notified immediately. That was just a remark about alternative notification method. The important issue was the first one.

2 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

autoqa-devel November 2009