Does it have to be an approved fedora package for nitrate to be deployed on fedora infra?
by Chaobin Tang
Hi James -
This is chaobin, new DEV to nitrate. Here I have a problem regarding to introducing nitrate to fedora.
As the subject -
As how I understand it, making a software a package is meant to distribute. In our case, the purpose is to deploy nitrate for fedora to use, and the deployment often takes place in just one system. Is it going to be an overkill for us to make nitrate an approved package just in order to deploy it(because it entails a certain amount of work to do so)? Or there is a general unexceptional regulation that would require it to be an approved package, in situations also include deploying an application?
Just wanna understand the scenario and the possibility of a bypass.
Thanks,
Chaobin
13 years, 1 month
Re: Proposed must-have Features for Nitrate compared with Wiki
by He Rui
On Wed, 2011-03-09 at 14:43 -0500, James Laska wrote:
> On Wed, 2011-03-09 at 18:06 +0800, He Rui wrote:
> > Greetings,
> >
> > Now let's continue the discussion[1] about TCMS management. After the
> > feature comparison between wiki and nitrate, the features which are
> > supported in current wiki, but not in nitrate have been identified[2].
> > Among these, I priorized some of them as Must-have features, which were
> > summarized as below[3]:
> >
> > 1. History rollback -
> > User can view history versions and undo changes.
>
> Good, I do use this more often than I realize.
>
> > 2. History comparison -
> > Different versions can compare with each other.
> >
> > 3. Description part in test case -
> > Currently there's no 'Description' part for test case format on Nitrate,
> > should we add this part or use 'Setup' part instead?
>
> No, I don't think 'setup' is the right place since we also use 'setup'
> in our wiki test template. Does nitrate support a summary or any other
> field to describe the intent of the test in human readable terms?
Oh, it has "Notes" area, maybe we can use that.
>
> > 4. Grouping cases (by media) -
> > Need a way to separate the cases to different groups in one test run.
> > How can we achieve this in Nitrate?
>
> Probably different test runs of the same plan. One run would be for the
> DVD ISO, one for the netinst.iso etc... Perhaps this will need further
> exploration.
Yeah, use different test runs or sorting the cases by order.
> > 5. Documents in test result page(Run) -
> > There's only a little space called 'Note' for documentation on Nitrate
> > and it doesn't support syntax. should we ask for more spaces with syntax
> > supported for test runs like test days?
>
> Can you explain in more detail? Are you proposing we move the test day
> wiki content into nitrate, so it's a one-stop-shop for test information
> for that event?
I meant in nitrate, there's only a 'Notes' field to support a simple
summary of each test run. For a test day which has many contents[1] such
as "What to test", "Prerequisite for Test Day", "How to test" etc, it is
hard to put all of these into the 'Notes' field. This field doesn't
support TinyMCE or any other syntax editing.
[1] https://fedoraproject.org/wiki/QA/Test_Days/Template
>
> > 6. Moving test results -
> > Moving previous results from a test run to another. AFAIK, there's no
> > way to copy some of one test run results to another.
>
> Good catch. I think with nitrate, you can selectively include test
> cases when creating a new test run, right? Meaning, we wouldn't need to
> include *all* tests in the plan when creating a new test run?
Yeah. but then we need to make very sure that certain cases results
won't be changed in new run. The summary of the new run results is also
not intact. Anyway, this is a workaround.
> This brings up another question ... permissions. With the wiki, anyone
> has permission to add or remove tests from a test run. How will that be
> handled in nitrate?
Since it's internally use only, the permissions only have three: 1.
anonymous - read. 2. tester - read and write any case/run/plan. 3. Admin
- manage the tcms system.
>
> > 7. Result type -
> > Nitrate doesn't have 'warn' but has 'error' result status. Modify it to
> > 'warn'?
>
> What is the significance of 'error' in nitrate? Does that mean there
> was a test case error, not necessarily a software error?
It presents an environment error, the user guide suggesting the use of
result status is:
Idle - Default value. The Test Case has not been examined.
Passed - Test Case met all the expected results.
Failed - Test Case did not meet all the expected results, or produced an
unhandled exception.
Running - Test Case is in progress.
Paused - This status is used to denote a problem with the test case
itself that prevents the test from being completed.
Blocked - Test Case has a dependency that has failed.
Error - Test environment has problems that prevent Test Case execution.
But I guess such 'error' status will be seldom used.
> > 8. Multiple contributions for each case -
> > Support multiple test results for one case run.
>
> Agreed and good catch! I think it's very important to allow multiple
> testers to provide feedback on a single test given that the environment
> under test (hardware) can vary for tests.
Yeah, so far, nitrate supports the view of editing logs but only display
the result changed at last time.
> > 9. Authorities for pages -
> > Pages on wiki with different namespaces can have different permissions,
> > so hopefully pages on Nitrate can also have diff permissions.
>
> I was confused by this a little earlier as well. I don't understand the
> concept of pages within nitrate. Is this to replace the use of our wiki
> test plan document and validation pages (e.g.
> http://fedoraproject.org/wiki/QA:Installation_validation_testing)?
I meant for example if use certain namespace like 'Test_Results:' or
'Test_Day:', these test runs allow anonymous editing, while test runs
without such names don't. So nitrate should also provide a way to give
different permissions for test runs.
> > 10. Supporting anonymous user read-write access -
> > Anonymous have read-write access for specified test runs.
>
> +1
>
> > 11. Page protection -
> > Protect certain plans/cases
>
> Oh, I think I understand now. Is the analogy here that a valid FAS
> login would be required, like we currently have with our wiki test cases
> and test plans?
Yeah, it's similar to No.9, a valid login is required to change certain
test cases/plans.
> > 12. License the content -
> > License the contents the same with wiki
>
> Or at least listed in the list of "GOOD" licenses -
> https://fedoraproject.org/wiki/Licensing#Good_Licenses_3
>
> > 13. Upstream project community -
> > Monitoring it and actively discussing topics in nitrate-devel@, as well
> > as file/share bugs/requirements in bugzilla
>
> +1
>
> > 14. Test day page(run) creation -
> > In Nitrate each test run are built based on its test plan. Therefore,
> > test day plan and its cases are needed before creating a test day run.
> > Should we change our test day process(adding test plan and its cases) or
> > should we change Nitrate to support creating test runs without test
> > plan?
>
> Perhaps something we need to experiment/research with a pilot instance?
Okay, then in a pilot instance, I'll create a test day plan and add
cases to it first.
> > 15. Test cases priority -
> > Nitrate now has P1, P2, P3..., modify them to Alpha, Beta, and Final?
>
> Alpha/Beta/Final are appropriate for our two test plans; desktop and
> installation. I don't know if they apply for all test plans folks might
> contribute. But it seems like an easy future adjustment, so for our
> immediate needs, P3=Alpha, P2=Beta, P1=Final etc...
>
> In general, for all nitrate metadata fields (priority, status,
> resolution etc...), whomever administers the tool in Fedora should have
> control over re-defining the values to something appropriate for Fedora.
> Perhaps that might be a more generic way to phrase the requirement?
Ah, I see. Agree with it!
> > 16. Each case with different platforms -
> > Better support two platform results(i386 and x86_64) submitting for each
> > case run
>
> Sure, a test run can only be for a single architecture, right? So we'll
> have two test runs for a compose.
>
> 1. RC1 - Installation (i386)
> 2. RC1 - Installation (x86_64)
Yeah, more runs can achieve this as well as grouping cases etc, but I'm
afraid this way runs of each event will be too many for management.
> > Welcome to discuss the above features and provide feedback/suggestion.
> > If there's any important feature missed, or this reminds you the
> > advantages of other tcms tools etc, feel free to talk.
>
> I know you mention anonymous login support above, can you also list FAS
> authentication? Apologies if I missed it already.
Do you mean integration with FAS?
>
> Thank you for continuing to move forward with this initiative. I'll
> keep thinking of important features and reply here if I have anything
> worth sharing :)
>
> Thanks,
> James
Thanks!
Hurry
--
Contacts
Hurry
FAS Name: Rhe
Timezone: UTC+8
TEL: 86-010-62608141
IRC nick: rhe #fedora-qa #fedora-zh
13 years, 1 month
TCMS Product Version Nitrate 3.3 was released
by Victor Chen
*All,*
**
*TCMS product version Nitrate 3.3 has been updated to product server.
Location: _https://tcms.engineering.redhat.com/
_*
*Schedule for further release:
https://fedorahosted.org/nitrate/browser/design/Milestone/TCMS_schedule.pdf*
**
*Bug Fixed****:
---------------------------
*************# 681156 - [Test Plan]Can not expand all the test case in
test plan.
# 679677 - [Test Run]The button should be "cancel" in Property page.
# 672495 - Old test run shows updated case information but its text
version is unchanged.
# 680315 - [Reporting]Open a product will lead to the error page.
# 680321 - [Test Run]Click "View My Assigned Runs" will list all runs
# 627236 - s/2009/2010/ orequivalent of date in page footer
# 680322 - New: [spelling mistake]"Highligt" should be "Highlight"
# 680059 - [Test Run]The total number of test case run is NULL
# 676259 - [FEAT] Need to get a break out of manual vs auto in the tcms
reporting section
# 678643 - TestPlan.get_text - multiple failures
# 674754 - [xmlrpc] TestRun.create() fails when list of tags provided
# 676590 - In run execute page, 'expand all' generates tons of http requests
# 664025 - TCMS main check box to control test cases doesn't work
# 658372 - Cannot select "Product Version" when clone multiple test plans
# 667304 - Click "Build" label, it won't be sorted by build
# 654533 - [TCMS]Document Version in test plan on opera browser
# 672873 - xml export can't be parsed
# 664743 - [RFE] supply existing bugs when marking test-case-run as failed
# 672857 - Typo in error message when a test plan hasn't been
# 657474 - [TCMS3.2-2]List the runs which have not environment
# 649293 - Make the case run "notes" field visible in the run
# 643324 - Provide a bit more space for the test run notes
# 653815 - Unable to re-order test cases in test run
# 658475 - The bug can not be deleted inside the run
# 672622 - product version gets set to "unused" when editing a plan
# 661951 - Messed-up warning message pop up when clicking Add without
entering Bug ID
# 665945 - run export button dosn't work
# 667293 - The first product is the default product.
# 665934 - choose no plan to "Printalbe Copy"
# 654953 - [RFE] Report an expanded list of Test Cases by Tag
# 664467 - TCMS: cells overlapping when using long name for test case
summary
# 662944 - Resort case run is broken in Firefox
# 642644 - update nitrate.py to work with the latest xmlrpclib
# 578717 - [REF] Provide filter in test run
# 653812 - Filtering test case runs
# 534063 - [RFE] Allow sorting / filtering test cases while executing
the test run
# 660234 - Add links to IDLE, PASSED, WAIVED items in report table again
# 661579 - Incorrect bug counting method - Ugly code, Ugly bug***
<http://hurl.test.redhat.com/HEOpXX>*
------
Victor Chen,
Red Hat Asia Pacific
Phone: +86 10 6260 8158
E-mail: vchen(a)redhat.com
13 years, 1 month
Re: Proposed must-have Features for Nitrate compared with Wiki
by He Rui
On Thu, 2011-03-10 at 10:19 -0500, James Laska wrote:
> On Thu, 2011-03-10 at 18:01 +0800, He Rui wrote:
> > On Wed, 2011-03-09 at 14:43 -0500, James Laska wrote:
> > > On Wed, 2011-03-09 at 18:06 +0800, He Rui wrote:
> > > > Greetings,
> > > >
> > > > Now let's continue the discussion[1] about TCMS management. After the
> > > > feature comparison between wiki and nitrate, the features which are
> > > > supported in current wiki, but not in nitrate have been identified[2].
> > > > Among these, I priorized some of them as Must-have features, which were
> > > > summarized as below[3]:
> > > >
> > > > 1. History rollback -
> > > > User can view history versions and undo changes.
> > >
> > > Good, I do use this more often than I realize.
> > >
> > > > 2. History comparison -
> > > > Different versions can compare with each other.
> > > >
> > > > 3. Description part in test case -
> > > > Currently there's no 'Description' part for test case format on Nitrate,
> > > > should we add this part or use 'Setup' part instead?
> > >
> > > No, I don't think 'setup' is the right place since we also use 'setup'
> > > in our wiki test template. Does nitrate support a summary or any other
> > > field to describe the intent of the test in human readable terms?
> >
> > Oh, it has "Notes" area, maybe we can use that.
> >
> > >
> > > > 4. Grouping cases (by media) -
> > > > Need a way to separate the cases to different groups in one test run.
> > > > How can we achieve this in Nitrate?
> > >
> > > Probably different test runs of the same plan. One run would be for the
> > > DVD ISO, one for the netinst.iso etc... Perhaps this will need further
> > > exploration.
> >
> > Yeah, use different test runs or sorting the cases by order.
> >
> > > > 5. Documents in test result page(Run) -
> > > > There's only a little space called 'Note' for documentation on Nitrate
> > > > and it doesn't support syntax. should we ask for more spaces with syntax
> > > > supported for test runs like test days?
> > >
> > > Can you explain in more detail? Are you proposing we move the test day
> > > wiki content into nitrate, so it's a one-stop-shop for test information
> > > for that event?
> >
> > I meant in nitrate, there's only a 'Notes' field to support a simple
> > summary of each test run. For a test day which has many contents[1] such
> > as "What to test", "Prerequisite for Test Day", "How to test" etc, it is
> > hard to put all of these into the 'Notes' field. This field doesn't
> > support TinyMCE or any other syntax editing.
> >
> > [1] https://fedoraproject.org/wiki/QA/Test_Days/Template
>
> Oh I see. My initial thought is this might be more of a nice-to-have
> feature, and possibly out of scope for the initial effort. Either way,
> I would say not a MUSTHAVE requirement. To me, anything content related
> should be left to the wiki ... the wiki does an outstanding job in this
> regard. Anything related to test cases and test results, should be
> managed by nitrate (or something similar).
I see. Today I talked with TCMS members, they said they could support
TinyMCE syntax editing to the 'Notes' area. Therefore we can either
achieve this by a link to wiki or edit it in 'Notes' field.
> > > > 6. Moving test results -
> > > > Moving previous results from a test run to another. AFAIK, there's no
> > > > way to copy some of one test run results to another.
> > >
> > > Good catch. I think with nitrate, you can selectively include test
> > > cases when creating a new test run, right? Meaning, we wouldn't need to
> > > include *all* tests in the plan when creating a new test run?
> >
> > Yeah. but then we need to make very sure that certain cases results
> > won't be changed in new run. The summary of the new run results is also
> > not intact. Anyway, this is a workaround.
>
> Ah. Another question, with nitrate, when we create a test run, will the
> goal for that run be to reach 100% completion? How do previous results
> factor into that calculation? This seems like another area where
> nitrate is just different, and we may need to explore new ways to model
> this behavior?
>
> > > This brings up another question ... permissions. With the wiki, anyone
> > > has permission to add or remove tests from a test run. How will that be
> > > handled in nitrate?
> >
> > Since it's internally use only, the permissions only have three: 1.
> > anonymous - read. 2. tester - read and write any case/run/plan. 3. Admin
> > - manage the tcms system.
Correction. Testers have not only read and write any case/run/plan
permission but also site/management administrations. Admin has
Authentication permission besides what testers have.
> Should we add this to the musthave requirements list. Support for
> anonymous read/write *and* FAS integration?
Sure, I've written them down.
>
> > > > 7. Result type -
> > > > Nitrate doesn't have 'warn' but has 'error' result status. Modify it to
> > > > 'warn'?
> > >
> > > What is the significance of 'error' in nitrate? Does that mean there
> > > was a test case error, not necessarily a software error?
> >
> > It presents an environment error, the user guide suggesting the use of
> > result status is:
> >
> > Idle - Default value. The Test Case has not been examined.
> > Passed - Test Case met all the expected results.
> > Failed - Test Case did not meet all the expected results, or produced an
> > unhandled exception.
> > Running - Test Case is in progress.
> > Paused - This status is used to denote a problem with the test case
> > itself that prevents the test from being completed.
> > Blocked - Test Case has a dependency that has failed.
> > Error - Test environment has problems that prevent Test Case execution.
> >
> > But I guess such 'error' status will be seldom used.
>
> Yeah, probably. I can see the case for warn, but warn really just means
> the test passed, but bugs were discovered during execution of the test
> but did not impact the outcome of the test.
Yeah, then let's add another 'warn' status?
>
> > > > 8. Multiple contributions for each case -
> > > > Support multiple test results for one case run.
> > >
> > > Agreed and good catch! I think it's very important to allow multiple
> > > testers to provide feedback on a single test given that the environment
> > > under test (hardware) can vary for tests.
> >
> > Yeah, so far, nitrate supports the view of editing logs but only display
> > the result changed at last time.
> >
> > > > 9. Authorities for pages -
> > > > Pages on wiki with different namespaces can have different permissions,
> > > > so hopefully pages on Nitrate can also have diff permissions.
> > >
> > > I was confused by this a little earlier as well. I don't understand the
> > > concept of pages within nitrate. Is this to replace the use of our wiki
> > > test plan document and validation pages (e.g.
> > > http://fedoraproject.org/wiki/QA:Installation_validation_testing)?
> >
> > I meant for example if use certain namespace like 'Test_Results:' or
> > 'Test_Day:', these test runs allow anonymous editing, while test runs
> > without such names don't. So nitrate should also provide a way to give
> > different permissions for test runs.
>
> I see, thanks for the explanation.
>
> > > > 10. Supporting anonymous user read-write access -
> > > > Anonymous have read-write access for specified test runs.
> > >
> > > +1
> > >
> > > > 11. Page protection -
> > > > Protect certain plans/cases
> > >
> > > Oh, I think I understand now. Is the analogy here that a valid FAS
> > > login would be required, like we currently have with our wiki test cases
> > > and test plans?
> >
> > Yeah, it's similar to No.9, a valid login is required to change certain
> > test cases/plans.
> >
> > > > 12. License the content -
> > > > License the contents the same with wiki
> > >
> > > Or at least listed in the list of "GOOD" licenses -
> > > https://fedoraproject.org/wiki/Licensing#Good_Licenses_3
> > >
> > > > 13. Upstream project community -
> > > > Monitoring it and actively discussing topics in nitrate-devel@, as well
> > > > as file/share bugs/requirements in bugzilla
> > >
> > > +1
> > >
> > > > 14. Test day page(run) creation -
> > > > In Nitrate each test run are built based on its test plan. Therefore,
> > > > test day plan and its cases are needed before creating a test day run.
> > > > Should we change our test day process(adding test plan and its cases) or
> > > > should we change Nitrate to support creating test runs without test
> > > > plan?
> > >
> > > Perhaps something we need to experiment/research with a pilot instance?
> >
> > Okay, then in a pilot instance, I'll create a test day plan and add
> > cases to it first.
> >
> > > > 15. Test cases priority -
> > > > Nitrate now has P1, P2, P3..., modify them to Alpha, Beta, and Final?
> > >
> > > Alpha/Beta/Final are appropriate for our two test plans; desktop and
> > > installation. I don't know if they apply for all test plans folks might
> > > contribute. But it seems like an easy future adjustment, so for our
> > > immediate needs, P3=Alpha, P2=Beta, P1=Final etc...
> > >
> > > In general, for all nitrate metadata fields (priority, status,
> > > resolution etc...), whomever administers the tool in Fedora should have
> > > control over re-defining the values to something appropriate for Fedora.
> > > Perhaps that might be a more generic way to phrase the requirement?
> >
> > Ah, I see. Agree with it!
>
> Oh, how does nitrate designate the site admin and test run admins?
So far there are two user groups: admin and tester. Both of them have
the permission for site admin and test run admins.
> Are
> they single persons, or is it a user group (aka FAS group)? I think
> we'll want to ability to designate multiple (but few) site admins
> (SHOULDHAVE), and need the ability to have multiple admins for test runs
> (MUSTHAVE). If it only supports one user being listed as the
> administrator, we'll want to review what types of tasks that restricts
> (add tests to the run, new runs etc...).
I was told that they are working on multiple site admins and multiple
admins for different groups by trying to integrate with another system
which is not open source. The TCMS members informed the FAS system are
investigating it.
> > > > 16. Each case with different platforms -
> > > > Better support two platform results(i386 and x86_64) submitting for each
> > > > case run
> > >
> > > Sure, a test run can only be for a single architecture, right? So we'll
> > > have two test runs for a compose.
> > >
> > > 1. RC1 - Installation (i386)
> > > 2. RC1 - Installation (x86_64)
> >
> > Yeah, more runs can achieve this as well as grouping cases etc, but I'm
> > afraid this way runs of each event will be too many for management.
> >
> > > > Welcome to discuss the above features and provide feedback/suggestion.
> > > > If there's any important feature missed, or this reminds you the
> > > > advantages of other tcms tools etc, feel free to talk.
> > >
> > > I know you mention anonymous login support above, can you also list FAS
> > > authentication? Apologies if I missed it already.
> >
> > Do you mean integration with FAS?
>
> Yes please. Apologies if that was already listed, I may have missed it.
It's listed.
> Thanks,
> James
> --
Thanks,
Hurry
--
Contacts
Hurry
FAS Name: Rhe
Timezone: UTC+8
TEL: 86-010-62608141
IRC nick: rhe #fedora-qa #fedora-zh
13 years, 1 month
Proposed must-have Features for Nitrate compared with Wiki
by He Rui
Greetings,
Now let's continue the discussion[1] about TCMS management. After the
feature comparison between wiki and nitrate, the features which are
supported in current wiki, but not in nitrate have been identified[2].
Among these, I priorized some of them as Must-have features, which were
summarized as below[3]:
1. History rollback -
User can view history versions and undo changes.
2. History comparison -
Different versions can compare with each other.
3. Description part in test case -
Currently there's no 'Description' part for test case format on Nitrate,
should we add this part or use 'Setup' part instead?
4. Grouping cases (by media) -
Need a way to separate the cases to different groups in one test run.
How can we achieve this in Nitrate?
5. Documents in test result page(Run) -
There's only a little space called 'Note' for documentation on Nitrate
and it doesn't support syntax. should we ask for more spaces with syntax
supported for test runs like test days?
6. Moving test results -
Moving previous results from a test run to another. AFAIK, there's no
way to copy some of one test run results to another.
7. Result type -
Nitrate doesn't have 'warn' but has 'error' result status. Modify it to
'warn'?
8. Multiple contributions for each case -
Support multiple test results for one case run.
9. Authorities for pages -
Pages on wiki with different namespaces can have different permissions,
so hopefully pages on Nitrate can also have diff permissions.
10. Supporting anonymous user read-write access -
Anonymous have read-write access for specified test runs.
11. Page protection -
Protect certain plans/cases
12. License the content -
License the contents the same with wiki
13. Upstream project community -
Monitoring it and actively discussing topics in nitrate-devel@, as well
as file/share bugs/requirements in bugzilla
14. Test day page(run) creation -
In Nitrate each test run are built based on its test plan. Therefore,
test day plan and its cases are needed before creating a test day run.
Should we change our test day process(adding test plan and its cases) or
should we change Nitrate to support creating test runs without test
plan?
15. Test cases priority -
Nitrate now has P1, P2, P3..., modify them to Alpha, Beta, and Final?
16. Each case with different platforms -
Better support two platform results(i386 and x86_64) submitting for each
case run
Welcome to discuss the above features and provide feedback/suggestion.
If there's any important feature missed, or this reminds you the
advantages of other tcms tools etc, feel free to talk.
Thanks,
Hurry
[1] https://fedorahosted.org/fedora-qa/ticket/152
[2] https://fedoraproject.org/wiki/Tcms_Comparison
[3] https://fedoraproject.org/wiki/Tcms_feature_requirements
--
Contacts
Hurry
FAS Name: Rhe
Timezone: UTC+8
TEL: 86-010-62608141
IRC nick: rhe #fedora-qa #fedora-zh
13 years, 1 month
