autoqa-devel January 2010

autoqa-devel@lists.fedorahosted.org

5 participants
50 discussions

[AutoQA] #113: rpmguard: Behaviour when comparing identical packages?
by fedora-badges 26 Jan '10

26 Jan '10

#113: rpmguard: Behaviour when comparing identical packages? -------------------------+-------------------------------------------------- Reporter: kparal | Owner: kparal Type: enhancement | Status: new Priority: minor | Milestone: Component: tests | Version: 1.0 Keywords: rpmguard | -------------------------+-------------------------------------------------- Currently rpmguard compares packages even when they are identical. Is there a better approach? Should rpmguard for identical packages just skip comparison and print info? Should identity be judged just from NVRA or must the packages be really identical on file level? Is there some RPM checksum that could help us decide identity? -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/113> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #112: Update the page: QA:Posting Install Test Results
by fedora-badges 22 Jan '10

22 Jan '10

#112: Update the page: QA:Posting Install Test Results -------------------------+-------------------------------------------------- Reporter: rhe | Owner: rhe Type: enhancement | Status: new Priority: major | Milestone: Fedora 13 Component: docs/wiki | Version: 1.0 Keywords: | -------------------------+-------------------------------------------------- Hi all, I update the page [https://fedoraproject.org/wiki/QA:Posting_Install_Test_Results QA:Posting Install Test Results], as the original one was designed for F-12 pre- releases. The *Posting_Test_Results* section is replace by a link to the *key* part in current template. And Fedora_version in syntax is all changed to {{FedoraVersion|long|next(current)}}. Please refer to [https://fedoraproject.org/w/index.php?title=QA%3APosting_Install_Test_Resul… Difference between revisions] for details. Is it cool? :) Any comments/modifications welcomed. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/112> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

Re: [AutoQA] #88: bodhi_utils module for autoqa lib
by fedora-badges 21 Jan '10

21 Jan '10

#88: bodhi_utils module for autoqa lib ---------------------+------------------------------------------------------ Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: autoqa depcheck Component: tests | Version: 1.0 Resolution: | Keywords: ---------------------+------------------------------------------------------ Changes (by alexlan): * cc: alexlan (added) -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/88#comment:1> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 0

Re: rpmguard: comparing identical packages
by Kamil Paral 15 Jan '10

15 Jan '10

----- "Kamil Paral" <kparal(a)redhat.com> wrote: > > Generally > there is one use case where rpmguard compares the new package with > itself. It is when the updated package moves to official repositories > before the rpmguard test is performed. > As I have mentioned, it is currently possible for a package to "fly under our radar" by simply moving to official repositories (stable, updates, updates-testing) too soon. It caused by our current strategy to compare the package against its most recent version in official repositories. So how to tackle it? Solution #1: Leave it be I suppose this events will not happen too often. If you deem it appropriate, we can just let it be. Currently it's mainly proof of concept. In the future if we have a policy that no package can move from -updates-candidate unless it has been checked by AutoQA test, then the problem is automatically solved. Solution #2: Re-implement "search for latest" strategy I have been playing with koji command and it seems to be able to list all the history of builds with particular tag, not just the latest build. You can try it yourself: $ koji list-tag-history --package=PackageKit $ koji list-tagged dist-f12-updates PackageKit Therefore we can re-implement/add a new method to our library that would not just provide the latest build of a package, but all of them from the history. Then we would pick the one that suits us most. The API clearly offers this capabilities. There is also one question that is directly connected to this topic. Which tags do we want to search through when looking for previous builds? a) Currently we search through stable, -updates and -updates-testing. b) Would it make sense to search also through -updates-candidate? The differences would always show to the very latest build, doesn't have to be released. c) Or the other way round, would it be better to search through only stable and -updates (skip -updates-testing)? When more builds are in testing repo, the differences would be displayed always against the latest build in stable repos (stable or -updates). I would say solution #2 shouldn't be that hard, I would do it. And as for repositories searched, maybe c) would be better than a), but I don't know packagers' needs.

2 3

rpmguard: comparing identical packages
by Kamil Paral 15 Jan '10

15 Jan '10

----- "Will Woods" <wwoods(a)redhat.com> wrote: > No, because apparently PackageKit-0.5.6-1.fc12 is already public in > f12-updates. The test is supposed to check things that land in > dist-f12-updates-candidate (e.g. new builds) against whatever is > currently public. > > So somehow PackageKit-0.5.6-1.fc12 appeared in updates-candidate > despite > it already being in the updates repo? I wonder how that happened. > I have dug into that issue and it's little more complicated. It doesn't matter if James started the test manually or not. Generally there is one use case where rpmguard compares the new package with itself. It is when the updated package moves to official repositories before the rpmguard test is performed. Usually it goes like this: 1. Maintainer builds a new package, it's tagged as -updates-candidate. 2. AutoQA detects the updated package and using its library rpmguard looks for the most recent package in stable, -updates and -updates-testing. 3. Rpmguard compares new package from 1) with older package from 2). 4. Some time later a package from 1) is re-tagged as -updates or -updates-testing. The problem arises when step 4) is between steps 1) and 2), ie. when maintainer moves his new package from -updates-candidate to either -updates or -updates-testing too soon. If AutoQA watcher is run only after that, then obviously the package version from 1) and from 2) are equal. And rpmguard simply compares them. So that's the explanation of the situation. Of course I can modify rpmguard to simply skip checking of identical packages, but that does not solve the problem of that particular package not being checked against the older version. I will cover possible solutions in a future email, I have to play with it a little more. One more thing: > Previous package for dist-f12-updates was PackageKit-0.5.6-1.fc12 > Previous package for dist-f12-updates was PackageKit-0.5.6-1.fc12 > Previous package for dist-f12 was PackageKit-0.5.4-0.1.20091029git.fc12 This is not weird at all. It just means that there is no PackageKit in dist-f12-updates-testing, and according to tag inheritance [1] it found the most recent PackageKit in dist-f12-updates (which is obviously the same result as when checking the dist-f12-updates tag itself), hence the same line. If there hadn't been any PackageKit updates since F12 release, there would've been three identical lines mentioning dist-12 tag. [1] http://koji.fedoraproject.org/koji/taginfo?tagID=106

1 0

Re: rpmguard: 0 warnings for package PackageKit-0.5.6-1.fc12
by James Laska 14 Jan '10

14 Jan '10

On Thu, 2010-01-14 at 12:03 -0500, autoqa(a)fedoraproject.org wrote: > N: Comparing PackageKit-yum-0.5.6-1.fc12 and PackageKit-yum-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ... > N: ---- > N: Comparing PackageKit-docs-0.5.6-1.fc12 and PackageKit-docs-0.5.6-1.fc12 (archs: noarch) ... > N: ---- > N: Comparing PackageKit-backend-devel-0.5.6-1.fc12 and PackageKit-backend-devel-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ... > N: ---- > N: Comparing PackageKit-gtk-module-0.5.6-1.fc12 and PackageKit-gtk-module-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ... > N: ---- > N: Comparing PackageKit-cron-0.5.6-1.fc12 and PackageKit-cron-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ... > N: ---- > N: Comparing PackageKit-command-not-found-0.5.6-1.fc12 and PackageKit-command-not-found-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ... > N: ---- > N: Comparing PackageKit-gstreamer-plugin-0.5.6-1.fc12 and PackageKit-gstreamer-plugin-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ... > N: ---- > N: Comparing PackageKit-browser-plugin-0.5.6-1.fc12 and PackageKit-browser-plugin-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ... > N: ---- > N: Comparing PackageKit-0.5.6-1.fc12 and PackageKit-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ... > N: ---- > N: Comparing PackageKit-glib-devel-0.5.6-1.fc12 and PackageKit-glib-devel-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ... > N: ---- > N: Comparing PackageKit-qt-0.5.6-1.fc12 and PackageKit-qt-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ... > N: ---- > N: Comparing PackageKit-yum-plugin-0.5.6-1.fc12 and PackageKit-yum-plugin-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ... > N: ---- > N: Comparing PackageKit-debug-install-0.5.6-1.fc12 and PackageKit-debug-install-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ... > N: ---- > N: Comparing PackageKit-device-rebind-0.5.6-1.fc12 and PackageKit-device-rebind-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ... > N: ---- > N: Comparing PackageKit-glib-0.5.6-1.fc12 and PackageKit-glib-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ... > N: ---- > N: Comparing PackageKit-smart-0.5.6-1.fc12 and PackageKit-smart-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ... > N: ---- > N: Comparing PackageKit-qt-devel-0.5.6-1.fc12 and PackageKit-qt-devel-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ... > N: ---- Not sure what's going on, but trying to investigate why this test was comparing two packages of the same nvr. >>> import autoqa.koji_utils >>> koji = autoqa.koji_utils.SimpleKojiClientSession() >>> for pb in koji.list_previous_releases("PackageKit", "dist-f12-updates-candidate").values(): ... print "Previous package for %s was %s" % ( pb['tag_name'], pb['nvr'] ) ... Previous package for dist-f12-updates was PackageKit-0.5.6-1.fc12 Previous package for dist-f12-updates was PackageKit-0.5.6-1.fc12 Previous package for dist-f12 was PackageKit-0.5.4-0.1.20091029git.fc12 I'm not sure whether list_previous_releases() is in error for returning a duplicate entry or whether the rpmguard control file should handle duplicate entries and keep scanning for the previous package? Thanks, James

2 4

Re: architecture idea
by Kamil Paral 14 Jan '10

14 Jan '10

----- "James Laska" <jlaska(a)redhat.com> wrote: > > One challenge I see is designing a schema that can efficiently > accommodate current and future front-end reporting needs. I often > worry > if we try to build something too universal, will it become useless? > If > the cost (time+energy) of making something generic is too high, I'd > certainly be in favor of multiple front-ends and results servers. Do > we > have a good handle on what the specific schema needs each front-end > might have? Well this is certainly one of the harder parts. Unfortunately I don't really know much about databases and I have just basic knowledge about database design practices. But I believe this should not be rushed, because a lot of code would have to be re-implemented after such change. So I think it will take some week(s) of discussion. If we know all the test types (installer tests, repository tests, rpm package tests, etc), we should be able to create a schema generic enough to accommodate all of those. If some new test type appears later, then it may be quite difficult to put it into an existing schema (unless the schema is really very generic, but then it's harder to work with). It is also possible to have several databases and some daemon listening to all incoming results and classify them into relevant databases according to the result type. Do we have any database expert amongst us? Anyone? :) PS: On IRC James mentioned we can have a look at autotest DB schema, so I attach a link: http://autotest.kernel.org/wiki/TkoDatabase

1 0

Re: [AutoQA] #110: rpmlint, rpmguard: Send results to mailing list
by Kamil Paral 14 Jan '10

14 Jan '10

----- "James Laska" <jlaska(a)redhat.com> wrote: > > That one rpmguard test result [1] is from when I manually scheduled > the > job on the command-line using the 'atest' command. I also modified > all > site_tests/rpmguard/* files to remove the '' character in your name. > I > have a suspicion that autotest might not handle that well > (http://pastie.org/777826) > > Any thoughts? Short term solution is easy - don't include non-ascii characters in test scripts. Long term solution probably involves conversation with autotest developers, because we might need unicode characters for some reason in the future. > > Thanks, > James > > [1] > https://fedorahosted.org/pipermail/autoqa-results/2010-January/002944.html > > _______________________________________________ > autoqa-devel mailing list > autoqa-devel(a)lists.fedorahosted.org > https://fedorahosted.org/mailman/listinfo/autoqa-devel

2 2

architecture idea
by Kamil Paral 14 Jan '10

14 Jan '10

As we were discussing in the latest threads, we will probably need some results database server in the future so we can then access it by different front-ends or RPC calls. I tried to think about the whole concept of AutoQA and create a picture that would correspond to my ideas. Have a look at it at: http://kparal.fedorapeople.org/misc/autoqa/architecture_idea.png What are your opinions on this topic, which things need to be discussed, what do you disagree with? I believe we should discuss these issues properly so won't implement something that would need to be re-worked soon after. I see the database server as the biggest and nearest challenge now, because the current irb site can easily display installer tests results, but you can hardly use it for displaying package update results (rpmlint, rpmguard). When we want to do something more with it than just sending it to the mailing list, we will need the results server, I reckon. Will, is it possible to separate the current irb database from the irb front-end itself, so we could use it as a basis for the results database server? And then connect the current front-end to it, as well as other front-ends? Thanks for responses, Kamil

2 1

Re: [Fwd: rpmguard: 0 warnings for package xorg-x11-server-1.7.4-2.fc12]
by Kamil Paral 14 Jan '10

14 Jan '10

----- "Will Woods" <wwoods(a)redhat.com> wrote: > It would probably be better if it didn't send out email if there's > nothing bad to report... > > -w I can make that change for rpmlint and rpmguard, no problem. Other tests currently send PASS results, but it's true it's a little different category (and also the email volume is much lower). Does anyone have another opinion, or shall I commit the patch? I would just wait with rpmguard until its birth problems are solved, so we just know if it works or not easily.

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

autoqa-devel January 2010