#35: Sync up install test priorities with the release criteria --------------------------+------------------------------------------------- Reporter: jlaska | Owner: jlaska Type: task | Status: assigned Priority: major | Milestone: Fedora 13 Component: Test Review | Version: Resolution: | Keywords: --------------------------+------------------------------------------------- Comment (by rhe):
Replying to [comment:9 jlaska]:
<skip>
You make a good point. I keep going back and forth on ... if the use
cases aren't release critical, should we continue testing them? Anyone have a suggested phrasing for tests we want to run, but they aren't release critical?
Wow, I keep thinking this for a looong time but still can't find a good solution. I think it depends on the goal for installation tests. We test it whether to meet the release criteria or to fully check the installation. In my view so far we design it for a full check, or we won't create the repo cases. Then I think we can have extra label(s) for our own. But it's really tough to think out one. ：）
Yes, PUD should be Alpha, I think.
Okay, thanks. Updated wiki.
traceback debug mode is Alpha, isn't it? From Alpha requirements: "The
installer must be able to report failures to Bugzilla, with appropriate information included"
Maybe. As I read it, the criteria captures the test
[https://fedoraproject.org/wiki/QA:Testcase_Anaconda_save_traceback_to_bugzil... QA:Testcase_Anaconda_save_traceback_to_bugzilla]. I feel that might need to be expanded or a new criteria is added.
How do folks feel about adding a criteria to the
[https://fedoraproject.org/wiki/Fedora_13_Beta_Release_Criteria Fedora_13_Beta_Release_Criteria] ... something like ... ''The installer must be able to copy failures logs to a remote system and offer a debug shell for investigating failures, and support copying failure information to a remote system.''
Agree.
updates.img working is probably Beta or Final, I'd think.
Agreed. We currently have
[https://fedoraproject.org/wiki/QA:Testcase_Anaconda_updates.img_via_URL QA:Testcase_Anaconda_updates.img_via_URL] listed as an Alpha priority case. I think this is correct since the use case offers the only mechanism to work around issues once the installer is released as Alpha. Unfortunately, I can't see a Alpha (or Beta) release criteria to link this back to. Any suggested updates to the criteria to accommodate this?
With regards to
[https://fedoraproject.org/wiki/QA/TestCases/PackageSetsMinimalPackageInstall QA/TestCases/PackageSetsMinimalPackageInstall]. Any concerns around obsoleting this test?
Sometimes when sth wrong happens during default package installation, minimal install is another choice. So I think we can just keep it.
-
fedora-badges