autoqa-devel February 2011

autoqa-devel@lists.fedorahosted.org

5 participants
65 discussions

[AutoQA] #267: Remove 'frontends' (israwhidebroken) directory from the master branch in git
by fedora-badges 30 Mar '11

30 Mar '11

#267: Remove 'frontends' (israwhidebroken) directory from the master branch in git ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: trivial | Milestone: Finger Food Component: core | Keywords: ----------------------+----------------------------------------------------- Since we're not building/packaging israwhidebroken any more, I'd suggest removing the 'frontends' directory alltogether - it will stay in the git history, if we ever decide we need it, and will not mess up our grep searches any more :) -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/267> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

4 8

[AutoQA] #262: Rethink the watcher/event/test architecture
by fedora-badges 29 Mar '11

29 Mar '11

#262: Rethink the watcher/event/test architecture --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: 0.5.0 Component: core | Keywords: --------------------+------------------------------------------------------- We've hit the limits of our architecture. The one watcher-one event style is no longer sufficient. We need to think about our needs and change that architecture, so that we are more flexible with watchers, events, target architectures, batch scheduling, etc. I have mentioned some bits at: https://fedorahosted.org/pipermail/autoqa-devel/2011-January/001522.html Let's refactor the architecture and get rid of many hacks that are currently necessary. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/262> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #245: Add support for test re-scheduling
by fedora-badges 29 Mar '11

29 Mar '11

#245: Add support for test re-scheduling --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: 0.5.0 Component: core | Keywords: --------------------+------------------------------------------------------- After we start sending results to Bodhi/ResultDB there will immediately emerge a need for test re-scheduling. There may be different reasons for that: 1. Something went wrong in the test execution, we need to run it again. 2. The environment has changed and test needs to be executed once again. For example the upgradepath test for some update failed because of invalid package versions in higher Fedora releases. That problem has been corrected and we need now a new execution of the test. The important thing is that the maintainer should be able to trigger the re-scheduling, not just us. Find a way how to properly reschedule tests (we will probably utilize ResultDB for that) and add that functionality. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/245> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #213: remove collection_name from repoinfo.conf
by fedora-badges 29 Mar '11

29 Mar '11

#213: remove collection_name from repoinfo.conf ----------------------------+----------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: minor | Milestone: Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- Once ticket #212 is complete we have no real need for collection_name anymore; remove it from repoinfo.conf. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/213> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #212: fix check_opt_in to not use collection_name
by fedora-badges 29 Mar '11

29 Mar '11

#212: fix check_opt_in to not use collection_name -----------------------+---------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: minor | Milestone: Component: docs/wiki | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- collection_name was used to line up with the dist-cvs branch layout; now that we have switched to dist-git/fedpkg the branch names match the repo names used in repoinfo, so we have no need for collection_name. This will require modifying the existing autoqa-optin file structure to match the git layout. We can leave symlinks for the old ('F-14') distro names but use the new ones ('f14') as the canonical ones. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/212> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #249: upgradepath: support check-whole-distro mode
by fedora-badges 29 Mar '11

29 Mar '11

#249: upgradepath: support check-whole-distro mode -------------------------+-------------------------------------------------- Reporter: kparal | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: tests | Keywords: -------------------------+-------------------------------------------------- This is a suggestion proposed by [https://fedoraproject.org/wiki/User:Spot spot] at [https://fedoraproject.org/wiki/Fedora_14_QA_Retrospective#Wishlist Fedora 14 QA Retrospective]: ''Upgradepath - Need to schedule upgradepath test runs against the entire distro, not just for each bodhi build. Also, adjust release criteria accordingly'' We could add mode that allows this. This script could then be used for ensuring all the packages in the whole distribution have upgrade path criteria met. This could be useful to check for example right before a new Fedora is going to be released. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/249> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #243: doc: How to run AutoQA tests
by fedora-badges 29 Mar '11

29 Mar '11

#243: doc: How to run AutoQA tests ---------------------------+------------------------------------------------ Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Component: documentation | Keywords: ---------------------------+------------------------------------------------ We currently have "How to run AutoQA tests" section at [[BR]] https://fedoraproject.org/wiki/Writing_AutoQA_Tests#How_to_run_AutoQA_tests Because the page tends to grow, it is no longer optimal to have it together. It should probably make sense to have separate page for "Writing AutoQA Tests" and separate page for "Running AutoQA Tests". Maybe the new page could be somehow merged with [[BR]] https://fedoraproject.org/wiki/Verifying_AutoQA_tests -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/243> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #247: upgradepath: possible broken-updates scenario for new packages
by fedora-badges 29 Mar '11

29 Mar '11

#247: upgradepath: possible broken-updates scenario for new packages --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: defect | Status: new Priority: major | Milestone: Component: tests | Keywords: --------------------+------------------------------------------------------- Here is a scenario that will create broken upgrade path and our current tests will not catch it: 1. Package foo is available for F12 (foo-1.0-1.fc12), but not for F13. 2. foo-1.0-1.fc13 is sent to f13-updates-pending as a new package update. upgradepath passes. 3. foo-2.0-1.fc12 is sent to f12-updates-pending. Since there is still no foo in F13 (AutoQA passed but it has not been pushed by RelEng team), upgradepath passes. 4. foo-2.0-1.fc12 is pushed to f12-updates. 5. foo-1.0-1.fc13 is pushed to f13-updates. 6. Broken upgrade path. What can we do about it? We can break the scenario between steps 4 and 5. But we can't probably do that just by watching -pending tags. This requires cooperation from the updates manager (i.e. Bodhi). Bodhi must know that if some foo build enters f12-updates, and there is also another foo build in f13-updates- pending, it must notify us (over Fedora Message Bus) and ask us to retest the latter build. We can also break the scenario between steps 2 and 3. When we run upgradepath on foo in f12-updates-pending, we can take into account not just f13-updates, but also f13-updates-pending. But this is quite controversial. Some builds in -updates-pending may be denied by RelEng team, but we could have failed some other builds because of them. We will probably solve this ticket only when we have advanced infrastructure ready (ResultDB, Fedora Message Bus). Without it this seems pretty hard to be solved properly. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/247> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #266: Organize the 'autoqa' script into functions
by fedora-badges 29 Mar '11

29 Mar '11

#266: Organize the 'autoqa' script into functions -------------------------+-------------------------------------------------- Reporter: jskladan | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Finger Food Component: core | Keywords: -------------------------+-------------------------------------------------- It would be great to take the autoqa script, identify the 'functionally standalone' parts (e.g. parsing command line options), and encapsulate these into separate functions. This would definitely make the script not even more readable, but also understandable. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/266> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 6

[AutoQA] #265: Create function to download RPMs from koji for given NVR
by fedora-badges 29 Mar '11

29 Mar '11

#265: Create function to download RPMs from koji for given NVR -------------------------+-------------------------------------------------- Reporter: jskladan | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Bite Food Component: core | Keywords: -------------------------+-------------------------------------------------- Through most (if not all) the tests, we use a set of commands like this: #install packages from koji koji = autoqa.koji_utils.SimpleKojiClientSession() pkgurls = koji.nvr_to_urls(envr, arches = os.uname()[-1]) rpms = [] print "Saving RPMs to %s" % self.rpmdir #download packages for p in pkgurls: # fetch package to rpmdir print "Grabbing %s" % p localfile = os.path.join(self.rpmdir, os.path.basename(p)) autoqa.util.grabber.urlgrab(p, localfile) rpms.append(localfile) Which takes NVR and downloads all the relevant RPM files. I'd like to have this enclosed in a method in the autoqa lib, which takes NVR (or a list of NVRs), downloads packages to the tmpdir, and returns list os.path.realpath() of the respective RPM files. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/265> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 5

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

autoqa-devel February 2011