Hi, folks. So, I drew up a rough draft of the Server release criteria for Beta and Final as I suggest they might be. We could kick it around at tomorrow's meeting if desired. Here we go:

BETA

=== Remote logging ===

It must be possible to forward system logs to a remote system using Server packages.

=== Firewall configuration ===

Release-blocking roles must be able to report their status in regard to the system firewall as described in the [[Server/Technical_Specification#Firewall|technical specification]].

=== Roles ===

Release-blocking roles and the supported role configuration interfaces must meet the core functional [[Server/Technical_Specification#Role_Definition_Requirements|Role Definition Requirements]] to the extent that supported roles can be successfully started, stopped, and queried.

=== Cockpit ===

??????

FINAL

=== Roles === Release-blocking roles and the supported role configuration interfaces must fully meet the [[Server/Technical_Specification#Role_Definition_Requirements|Role Definition Requirements]].

=== Cockpit ===

??????

You will note first of all the big ?????? for Cockpit. Cockpit's role really isn't terribly well defined in the tech spec or PRD, so it is hard to draft requirements. Is it primarily supposed to be an interface for controlling roles? If so, can we treat it that way in the criteria, and require things from it as regards its job in configuring roles?

I think the Role criteria described above would be okay, and at least unambitious enough not to cause huge problems, for F21, but I'm really not that happy with them for the long term.

It's obviously not practical to start hardcoding specific requirements for specific roles into the criteria. What I would like instead is to have a little more process on the Product side for Roles. I'd suggest that Roles themselves should have definitions of their expected functionality baked in at the point of creation / approval.

These could be split into sections to be required at Alpha, Beta and Final. You could, for instance, expect roles to basically start up and not explode at Alpha, expect them to be broadly 'feature complete' and testable at Beta, and expect them to fully meet their listed functional requirements at Final, just as a quick cut. Here's a very rough mock-up for the "Domain Controller" role as a guide:

Alpha -----

The Role must start and stop successfully. When the role is running, a client system must be able to enrol into the FreeIPA domain.

Beta ----

When the Role is running, it must be able to enrol and unenrol multiple clients. Client systems must be able to authenticate user accounts via Kerberos. The FreeIPA configuration web UI must be available and allow at least basic configuration of user accounts and permissions.

Final -----

Oh, I don't know, I'm running out of ideas, I don't do that much more with my FreeIPA system than is described in Beta. You know better than me! That's why you should be writing this! :)

Well, I sort of ran out of steam at the end there, but you kinda get the idea, right?

We can set things up appropriately so the release criteria can sensibly call out to them. I'd expect the criteria would just say something like 'Release-blocking roles must meet the functional requirements listed in their definition pages for the Beta milestone' or whatever, with a link to a sensible target from which you could easily find the Role definition pages.

I'd be very grateful for feedback on both the proposed criteria for F21, and the idea above. We do need to get the Beta criteria in place ASAP, Beta TC1 is due tomorrow (yes, I know, no rest for the wicked :/) I'll start drafting up test cases ASAP.