Is there a wiki page that explains the linkage between RCs that are created for final testing and the associated "official spins"?
IOW do official spins need additional testing with the creation of each new RC?
John
On Thu, 2009-06-04 at 07:48 -0700, John Poelstra wrote:
Is there a wiki page that explains the linkage between RCs that are created for final testing and the associated "official spins"?
IOW do official spins need additional testing with the creation of each new RC?
Good question. Who creates the "official spins" (GNOME, KDE), do these come from release engineering, or from the spin sig?
The only intersection with live images currently in the test plan relates to installing from live media (anaconda --method=liveinst). This implies that you've managed to boot the live image, but doesn't specify which live image or other criteria around the live image environment. I think it's worth considering adding some verification around this post F11.
As far as which images will be tested, and when/who delivers them ... I could use clarification on that as well.
Thanks, James
On Thu, 2009-06-04 at 11:34 -0400, James Laska wrote:
Good question. Who creates the "official spins" (GNOME, KDE), do these come from release engineering, or from the spin sig?
I create them.
The only intersection with live images currently in the test plan relates to installing from live media (anaconda --method=liveinst). This implies that you've managed to boot the live image, but doesn't specify which live image or other criteria around the live image environment. I think it's worth considering adding some verification around this post F11.
Yes, live image testing has been a big hole in our test matrix. I've done some testing ad hoc but not to any great degree.
As far as which images will be tested, and when/who delivers them ... I could use clarification on that as well.
I created them when I created the release candidates.
On Thu, Jun 04, 2009 at 09:41:39 -0700, Jesse Keating jkeating@redhat.com wrote:
Yes, live image testing has been a big hole in our test matrix. I've done some testing ad hoc but not to any great degree.
I have been doing some testing of live images based on current rawhide fairly regularly. I have been using the games spin plus a few things from several other F11 repos. So it's not exactly what will be released, but I have caught some problems ahead of time by doing this.
For F12 I want to do a better job testing the games spin. But I ended up dealing with video driver and storage rewrite issues a lot and only got to do some smoke testing this time around.
On Thu, 4 Jun 2009 12:03:29 -0500 Bruno Wolff III bruno@wolff.to wrote:
On Thu, Jun 04, 2009 at 09:41:39 -0700, Jesse Keating jkeating@redhat.com wrote:
Yes, live image testing has been a big hole in our test matrix. I've done some testing ad hoc but not to any great degree.
I have been doing some testing of live images based on current rawhide fairly regularly. I have been using the games spin plus a few things from several other F11 repos. So it's not exactly what will be released, but I have caught some problems ahead of time by doing this.
For F12 I want to do a better job testing the games spin. But I ended up dealing with video driver and storage rewrite issues a lot and only got to do some smoke testing this time around.
I've been doing local spins and testing here for the Xfce spin.
More testers would surely be welcome.
I was out on vacation last week and have been swamped this week, so I haven't done any in a bit. I hope nothing last minute has cropped up. ;(
kevin
On 06/05/2009 12:31 AM, Kevin Fenzi wrote:
I was out on vacation last week and have been swamped this week, so I haven't done any in a bit. I hope nothing last minute has cropped up. ;(
Kevin,
Next time, please ask for help. I have been very busy as well but might have been able to test it. Others who use the Xfce spin would probably have participated as well.
Rahul
On Fri, 05 Jun 2009 00:44:31 +0530 Rahul Sundaram sundaram@fedoraproject.org wrote:
On 06/05/2009 12:31 AM, Kevin Fenzi wrote:
I was out on vacation last week and have been swamped this week, so I haven't done any in a bit. I hope nothing last minute has cropped up. ;(
Kevin,
Next time, please ask for help. I have been very busy as well but might have been able to test it. Others who use the Xfce spin would probably have participated as well.
I did ask on IRC for those folks who might be able to do so and also in the QA meeting open floor section.
I think for f12 it would be nice to have a more formalized process of testing the spins (ie, with checklists and tests and run at specific intervals). This would help us avoid any problems and remind people to do the testing.
Rahul
kevin
On Sat, Jun 06, 2009 at 00:26:40 -0600, Kevin Fenzi kevin@scrye.com wrote:
On Fri, 05 Jun 2009 00:44:31 +0530
I think for f12 it would be nice to have a more formalized process of testing the spins (ie, with checklists and tests and run at specific intervals). This would help us avoid any problems and remind people to do the testing.
We are supposed to be making test plans for the spins. I didn't get mine done for F11, but I am going to try to make some progress for F12.
There was also some guidelines for testing builds regularly made during the F11 cycle. Though how much testing was supposed to be done at these is a bit vague. I suspect a lot of it is just making sure the spin didn't get too big or that something critical didn't get broken so that you notice some fixing needs to be done before it's too late in the cycle.
I think the plan is to get people doing this this time around. Karnip's going to miss this biweekly meeting, so it probably won't get talked about in the spins group seriously for another two weeks.
On Sat, 6 Jun 2009 12:06:05 -0500 Bruno Wolff III bruno@wolff.to wrote:
On Sat, Jun 06, 2009 at 00:26:40 -0600, Kevin Fenzi kevin@scrye.com wrote:
On Fri, 05 Jun 2009 00:44:31 +0530
I think for f12 it would be nice to have a more formalized process of testing the spins (ie, with checklists and tests and run at specific intervals). This would help us avoid any problems and remind people to do the testing.
We are supposed to be making test plans for the spins. I didn't get mine done for F11, but I am going to try to make some progress for F12.
Well, there was:
https://fedoraproject.org/wiki/Spins_Test_Matrix
but I think I was the only one to use it. ;) I'm glad I did because I found and fixed a selinux issue.
There was also some guidelines for testing builds regularly made during the F11 cycle. Though how much testing was supposed to be done at these is a bit vague. I suspect a lot of it is just making sure the spin didn't get too big or that something critical didn't get broken so that you notice some fixing needs to be done before it's too late in the cycle.
Yeah, it would be good to do some test runs before any milestone where they would be composed...
I think the plan is to get people doing this this time around. Karnip's going to miss this biweekly meeting, so it probably won't get talked about in the spins group seriously for another two weeks.
Yep.
kevin
On Sat, Jun 06, 2009 at 12:29:40 -0600, Kevin Fenzi kevin@scrye.com wrote:
On Sat, 6 Jun 2009 12:06:05 -0500 Bruno Wolff III bruno@wolff.to wrote:
We are supposed to be making test plans for the spins. I didn't get mine done for F11, but I am going to try to make some progress for F12.
Well, there was:
That's the generic one. Each of the spin maintainers are supposed to be making their own test plans for things that are important in their spin. For example for the games spin, there should be a short test for each game specifically included, to make sure it is at least starts and is playable for a little while.
On Sat, Jun 6, 2009 at 8:39 PM, Bruno Wolff III wrote:
That's the generic one. Each of the spin maintainers are supposed to be making their own test plans for things that are important in their spin. For example for the games spin, there should be a short test for each game specifically included, to make sure it is at least starts and is playable for a little while.
In the case of FEL, the desktop environment and the extra software which needed for electronics are separate items. with %include fedora-livecd-kde.ks
Thereby, the core is being prepared and developed by KDE sig while, we(FEL) can concentrate on electronics only.
So the only testing needed is to test the extra software needed for FEL, normally I test each software through my own designs, but little by little I'm creating an automated testsuite that will qualify each new update and the design tools FEL provide. https://fedorahosted.org/fedora-electronic-lab/browser/testsuite
The testsuite is open and can be used by other contributors as well.
From a hierarchical point of view, this decrease the work load and let
spin maintainers concentrate on improving the spins both in terms in documentation and packaging.
I once proposed this idea to szdiallas (FedoraEducation) I think he opted some of FEL ground work already so that he can concentrate on the goals of his spin and less software headaches.
Chitlesh
On 06/06/2009 11:56 AM, Kevin Fenzi wrote:
On Fri, 05 Jun 2009 00:44:31 +0530 Rahul Sundaram wrote:
Next time, please ask for help. I have been very busy as well but might have been able to test it. Others who use the Xfce spin would probably have participated as well.
I did ask on IRC for those folks who might be able to do so and also in the QA meeting open floor section.
Those will be missed out by the majority. If it is important, it should be posted in the mailing list.
Rahul
James Laska wrote:
Good question. Who creates the "official spins" (GNOME, KDE), do these come from release engineering, or from the spin sig?
The kickstart file is maintained in the spin-kickstarts git repository by the spin maintainer(s) (for KDE, that's Sebastian Vahl from KDE SIG). At least for the KDE spin, the maintainer also occasionally makes snapshot images available for testing. The maintainer is responsible for having a working kickstart file in the repository at the time of the release.
When preparing any release (Alpha, Beta, Preview, RC, GA), Fedora's lead release engineer (Jesse Keating) builds the official images from the current kickstart file in the spin-kickstarts git repository.
Kevin Kofler
On Thu, 2009-06-04 at 20:54 +0200, Kevin Kofler wrote:
[snip]
When preparing any release (Alpha, Beta, Preview, RC, GA),
I realize this is being overly pedantic, but "GA" shouldn't be in the list above. You don't prepare a "GA" release. You prepare and test an RC release and if it passes the release criteria it is deemed a "GA."
Referring to "GA" as a separate spin/compose/whatnot gives people (mostly developers :-)) the sense there's another build coming, so changes can be targeted there. And trust me, there's nothing finding a blocking bug at 2am just to have the developer tell you, "Yep, was planning on getting that fixed in the GA spin."
- jkt
-
Bruno Wolff III -
Chitlesh GOORAH -
James Laska -
Jay Turner -
Jesse Keating -
John Poelstra -
Kevin Fenzi -
Kevin Kofler -
Rahul Sundaram