8:47 a.m.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
During this week's Fedora Server Working Group meeting[1], we started
planning out how we want the public D-BUS API to look. I volunteered
to take the discussion to the list so we could start hashing it out
with a wider audience. I'm CC-ing the cockpit-devel list since a lot
of this will affect that project, but I'd prefer to keep replies on
server(a)lists.fedoraproject.org please (easier to track on one list).
Some pre-conditions/goals (many of these are reminders from earlier
discussions):
1) We want to be able to support two initial clients of this API: the
Cockpit management console and a local command-line tool (I've been
tentatively calling it 'fedora-role-manage') A third, future option
will be to also support remote access to this API via the OpenLMI
Project.
2) We do not need to have a single generic API that will work for any
conceivable Role. We've identified in the past that the selection of
Featured roles will (for the conceivable future) remain a small,
manageable list. For this reason, it is acceptable for the
deployment and post-deployment configuration APIs to be entirely
different between two roles. However, some functionality such as
the backup and firewall queries should be stable between all roles.
3) The supported clients should be able to determine if a target system
supports the deployment of a particular role (for backwards-
compatibility and architecture differences), but the Role API should
*NOT* be attempting to dictate the UI experience. UIs should merely
check whether the role is available and then use a previously-
developed experience. We don't want to be reimplementing PAM or
debconf here.
4) The D-BUS API must be stable. If it is likely that the set of input
required to deploy or configure will change, this should be planned
for in the design.
5) As it is not possible to plan for all contingencies, the API must be
versioned such that the clients can determine whether they can
properly perform the requested actions.
Some initial thoughts on the matter (all are open to discussion):
Standard Objects:
/org/fedoraproject/server/ServerRoleManager:
* properties:
* all_roles: list of object references to Roles that can be
installed and deployed
* staging_roles: list of object references to Roles that have been
pre-loaded but not yet configured
* active_roles: list of object references to deployed Roles
* version: API version
* methods:
# These properties and methods should be available and identical on
# all Roles
/org/fedoraproject/server/role/$ROLE:
* properties:
* version: API version
* loaded: Bool: whether the packages are installed
* deployed: Bool: whether the Role has been deployed
* methods:
* PreloadRole: Install all necessary software packages for
default operation
* GetFirewallPorts: list of port/protocol pairs that the Role needs
open in the firewall
* PrepBackup: method to invoke any necessary pre-backup steps
(such as running a database dump to a file)
* GetBackupFiles: list of filesystem path objects that identifies
all data that should be copied by backup software
# Individual Roles should have unique implementations of installation
# and deployment
e.g. /org/fedoraproject/server/role/DomainController
* methods:
* DeployPrimaryDomainController: Set up the first domain controller in
the domain.
* DeployReplicaDomainController: Add a new replica domain controller
to the domain
* AddCertificateAuthority: Add a certificate authority to this
domain controller
* EnableDisableDNS: Enable or disable the DNS server on
this server
== Open questions ==
1) How do we handle configuring the Firewall? I think we want to have
a Firewall object (/org/fedoraproject/server/RoleFirewallManager)
available to query the firewall as a whole, but we may also want to be
able to view and apply firewall rules from the Role objects directly.
(Note: I'm not suggesting we need a complete firewall solution here.
This should be a wrapper that deals only with the Roles). Firewalld
already provides a more comprehensive firewall interface for the
general case.
I'm sure there are other questions we'll discover out there, but this
email is already running long for an "introduction" to the topic.
[1]
http://meetbot.fedoraproject.org/fedora-meeting-1/2014-03-18/fedora-meeti...
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlMpn+gACgkQeiVVYja6o6Ou5wCeJw5zELa1zwLj++EBf66PKMKh
z6EAn1MOkhPQhY4bj9afTy1of68/QhXR
=xz9N
-----END PGP SIGNATURE-----
5:10 p.m.
On Wed, 19 Mar 2014 09:47:20 -0400
Stephen Gallagher <sgallagh(a)redhat.com> wrote:
...snip...
4) The D-BUS API must be stable. If it is likely that the set of
input required to deploy or configure will change, this should be
planned for in the design.
5) As it is not possible to plan for all contingencies, the API must
be versioned such that the clients can determine whether they can
properly perform the requested actions.
I agree with these, but we may not want to '1.0' the api until we are
further along?
Some initial thoughts on the matter (all are open to discussion):
Standard Objects:
/org/fedoraproject/server/ServerRoleManager:
* properties:
* all_roles: list of object references to Roles that can be
installed and deployed
It gets this from the fedora-release-server? or ?
* staging_roles: list of object references to Roles that have been
pre-loaded but not yet configured
This it can get from installed packages or a file.
* active_roles: list of object references to deployed Roles
* version: API version
* methods:
Ditto.
...snip...
== Open questions ==
1) How do we handle configuring the Firewall? I think we want to have
a Firewall object (/org/fedoraproject/server/RoleFirewallManager)
available to query the firewall as a whole, but we may also want to be
able to view and apply firewall rules from the Role objects directly.
(Note: I'm not suggesting we need a complete firewall solution here.
This should be a wrapper that deals only with the Roles). Firewalld
already provides a more comprehensive firewall interface for the
general case.
Should we try and rope in the firewalld maintainer(s) here?
They may have some ideas on how to do this so it causes the least
amount of pain.
I'm sure there are other questions we'll discover out there,
but this
email is already running long for an "introduction" to the topic.
I think it's a good start! :)
kevin
8 a.m.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 03/19/2014 06:10 PM, Kevin Fenzi wrote:
On Wed, 19 Mar 2014 09:47:20 -0400 Stephen Gallagher
<sgallagh(a)redhat.com> wrote:
...snip...
> 4) The D-BUS API must be stable. If it is likely that the set of
> input required to deploy or configure will change, this should
> be planned for in the design.
>
> 5) As it is not possible to plan for all contingencies, the API
> must be versioned such that the clients can determine whether
> they can properly perform the requested actions.
I agree with these, but we may not want to '1.0' the api until we
are further along?
I'd like for clients to be backwards and forwards-compatible. So new
versions of clients should be able to fall back to older, deprecated
versions of the API and newer versions of the server should be able to
support any version of the API that is still meaningful (such as we've
added a new version of the API that gives more control, but the old
version could still produce a usable system compatible with older
releases).
...snip...
> == Open questions ==
>
> 1) How do we handle configuring the Firewall? I think we want to
> have a Firewall object
> (/org/fedoraproject/server/RoleFirewallManager) available to
> query the firewall as a whole, but we may also want to be able to
> view and apply firewall rules from the Role objects directly.
> (Note: I'm not suggesting we need a complete firewall solution
> here. This should be a wrapper that deals only with the Roles).
> Firewalld already provides a more comprehensive firewall
> interface for the general case.
Should we try and rope in the firewalld maintainer(s) here? They
may have some ideas on how to do this so it causes the least amount
of pain.
Yes, good idea. I'm CCing Thomas Woerner and will politely ask him to
review this thread.[1]
[1]
https://lists.fedoraproject.org/pipermail/server/2014-March/000999.html
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlMq5osACgkQeiVVYja6o6P6AwCeJV91PZDeY1mdRB4GlP+zLOVy
ztcAnjg4/7Ta8MibVwMJKEyWIbAu467I
=lgUF
-----END PGP SIGNATURE-----
4:04 p.m.
2014-03-20 14:00 GMT+01:00 Stephen Gallagher <sgallagh(a)redhat.com>:
> I agree with these, but we may not want to '1.0' the api
until we
> are further along?
I'd like for clients to be backwards and forwards-compatible. So new
versions of clients should be able to fall back to older, deprecated
versions of the API and newer versions of the server should be able to
support any version of the API that is still meaningful (such as we've
added a new version of the API that gives more control, but the old
version could still produce a usable system compatible with older
releases).
That requires us to know and specify in advance how would the server know
which version of the API the client is indenting to use. So, for the
record, do we plan to just add methods (e.g. DeployV2) to the existing
interfaces?
Mirek
5:05 p.m.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 03/24/2014 05:04 PM, Miloslav Trmač wrote:
2014-03-20 14:00 GMT+01:00 Stephen Gallagher <sgallagh(a)redhat.com
<mailto:sgallagh@redhat.com>>:
> I agree with these, but we may not want to '1.0' the api until
> we are further along?
I'd like for clients to be backwards and forwards-compatible. So
new versions of clients should be able to fall back to older,
deprecated versions of the API and newer versions of the server
should be able to support any version of the API that is still
meaningful (such as we've added a new version of the API that gives
more control, but the old version could still produce a usable
system compatible with older releases).
That requires us to know and specify in advance how would the
server know which version of the API the client is indenting to
use. So, for the record, do we plan to just add methods (e.g.
DeployV2) to the existing interfaces?
That's certainly the easiest (and most D-BUS-esque) way to do it. I'm
open to other suggestions, of course.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlMwrDkACgkQeiVVYja6o6N4jACdEapCpVlPeQhrdwsIo5tpcPQe
eeUAn20aLGm+unGVRm4gl+7wAbiFyBur
=sOHR
-----END PGP SIGNATURE-----
10 a.m.
Hello,
On 03/20/2014 02:00 PM, Stephen Gallagher wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 03/19/2014 06:10 PM, Kevin Fenzi wrote:
> On Wed, 19 Mar 2014 09:47:20 -0400 Stephen Gallagher
> <sgallagh(a)redhat.com> wrote:
>
> ...snip...
>
>> 4) The D-BUS API must be stable. If it is likely that the set of
>> input required to deploy or configure will change, this should
>> be planned for in the design.
>>
>> 5) As it is not possible to plan for all contingencies, the API
>> must be versioned such that the clients can determine whether
>> they can properly perform the requested actions.
>
> I agree with these, but we may not want to '1.0' the api until we
> are further along?
>
I'd like for clients to be backwards and forwards-compatible. So new
versions of clients should be able to fall back to older, deprecated
versions of the API and newer versions of the server should be able to
support any version of the API that is still meaningful (such as we've
added a new version of the API that gives more control, but the old
version could still produce a usable system compatible with older
releases).
> ...snip...
>
>> == Open questions ==
>>
>> 1) How do we handle configuring the Firewall? I think we want to
>> have a Firewall object
>> (/org/fedoraproject/server/RoleFirewallManager) available to
>> query the firewall as a whole, but we may also want to be able to
>> view and apply firewall rules from the Role objects directly.
>> (Note: I'm not suggesting we need a complete firewall solution
>> here. This should be a wrapper that deals only with the Roles).
>> Firewalld already provides a more comprehensive firewall
>> interface for the general case.
>
> Should we try and rope in the firewalld maintainer(s) here? They
> may have some ideas on how to do this so it causes the least amount
> of pain.
Yes, good idea. I'm CCing Thomas Woerner and will politely ask him to
review this thread.[1]
I have been on vacation and are in the office since today again.
What exactly do you want to be able to configure? Do you think of
something like firewall zones or more fine grained configuration options
like for example ports or direct rules?
[1]
https://lists.fedoraproject.org/pipermail/server/2014-March/000999.html
Regards,
Thomas
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlMq5osACgkQeiVVYja6o6P6AwCeJV91PZDeY1mdRB4GlP+zLOVy
ztcAnjg4/7Ta8MibVwMJKEyWIbAu467I
=lgUF
-----END PGP SIGNATURE-----
12:29 p.m.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 03/25/2014 11:00 AM, Thomas Woerner wrote:
Hello,
On 03/20/2014 02:00 PM, Stephen Gallagher wrote:
>>>> == Open questions ==
>>>>
>>>> 1) How do we handle configuring the Firewall? I think we
>>>> want to have a Firewall object
>>>> (/org/fedoraproject/server/RoleFirewallManager) available
>>>> to query the firewall as a whole, but we may also want to
>>>> be able to view and apply firewall rules from the Role
>>>> objects directly. (Note: I'm not suggesting we need a
>>>> complete firewall solution here. This should be a wrapper
>>>> that deals only with the Roles). Firewalld already provides
>>>> a more comprehensive firewall interface for the general
>>>> case.
>>>
>>> Should we try and rope in the firewalld maintainer(s) here?
>>> They may have some ideas on how to do this so it causes the
>>> least amount of pain.
Yes, good idea. I'm CCing Thomas Woerner and will politely ask him
to review this thread.[1]
> I have been on vacation and are in the office since today again.
> What exactly do you want to be able to configure? Do you think
> of something like firewall zones or more fine grained
> configuration options like for example ports or direct rules?
I'm not going to attempt to specify an implementation right now. I'd
rather state the problem space as I see it and have you recommend an
approach instead.
So to restate the problem:
One of the primary goals of the Fedora Server product is to be able to
package up some common IT infrastructure components and provide an
easy way to deploy them on a physical or virtual machine[1]. Some
representative examples of this are a Domain Controller (FreeIPA) and
a Database Server (PostgreSQL).
We are working on building a D-BUS API and listener on Fedora Server
systems whose duty it is to install the requisite packages and deploy
a fully-functional Server Role onto the system.
Part of being a fully-functional part of infrastructure is having
those services be accessible by the appropriate clients. The Role
itself will "know" what ports it needs open, and this will be possible
to query through the D-BUS API in some manner TBD. We need a way in
the D-BUS API for an administrator to grant access to those requested
ports on some or all network interfaces attached to the machine. We
also want this interface to have an association with the Role object
in the system, so that a client such as Cockpit can easily query a
Role for "What ports do you need and on which interfaces can that port
be reached?" Furthermore, we want there to be a mechanism to apply a
set of very simple changes.
For a real-world example:
I have decided to stand up a database server. While I'm setting it up,
managing permissions and filling it with data, I want to keep this
private, so I only want access to be allowed on the 'lo' interface. I
want this to be done atomically as part of the initial deployment, so
I don't have any race-conditions to worry about where my database
might have been visible externally.
Once that is prepared, my tests are all complete and I know it's safe
to expose the database to the external interfaces. I have two external
ethernet interfaces. One is to the public network and the other is a
private internal connection (a storage network or management plane,
etc.) At this time, I want to update the firewall so that only 'lo'
and the public network have access to the set of ports required by my
database server.
Also, in cases where a Role might require more than one port (such as
the Domain Controller) I might also want to only allow a subset of the
Role's ports access on a particular interface.
Finally, I want this ability to allow access to some or all required
ports on some or all available interfaces to be offered by the
Deployment phase as well (in the same atomic manner as my example above).
I hope that describes the problem space sufficiently well. My
colleagues in the Server SIG can correct me if I got any of the
details wrong.
[1] I'm intentionally leaving containers out of the discussion for the
moment.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlMxvRYACgkQeiVVYja6o6MKcQCghA/naqrGVinqvXQq4ECwvYuf
fQsAnjQoFQpJo0dHWQzs/2//aESw5sSu
=7jSs
-----END PGP SIGNATURE-----
5:54 p.m.
2014-03-25 18:29 GMT+01:00 Stephen Gallagher <sgallagh(a)redhat.com>:
We
also want this interface to have an association with the Role object
in the system, so that a client such as Cockpit can easily query a
Role for "What ports do you need and on which interfaces can that port
be reached?" Furthermore, we want there to be a mechanism to apply a
set of very simple changes.
<snip>
Also, in cases where a Role might require more than one port (such
as
the Domain Controller) I might also want to only allow a subset of the
Role's ports access on a particular interface.
I think this really should be "a pre-designed subset", not "arbitrary
subset"; once the user starts listing port numbers, the connection
with the role starts becoming tenuous. It would be reasonable for
some roles to provide consistent set of ports (e.g. "company-visible
public data" vs. "DHCP and PXE and tftp server to be restricted to a
specific interface"), but we shouldn't need arbitrary subsets that
don't make sense (say, enabling cups announcing itself over avai while
preventing access to the IPP service).
IPA might be a "worst-case" situation; if IPA can only live with
pre-designed subsets of ports, probably every thing can.
Mirek
12:10 p.m.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 03/25/2014 06:54 PM, Miloslav Trmač wrote:
2014-03-25 18:29 GMT+01:00 Stephen Gallagher
<sgallagh(a)redhat.com>:
> We also want this interface to have an association with the Role
> object in the system, so that a client such as Cockpit can easily
> query a Role for "What ports do you need and on which interfaces
> can that port be reached?" Furthermore, we want there to be a
> mechanism to apply a set of very simple changes.
<snip>
> Also, in cases where a Role might require more than one port
> (such as the Domain Controller) I might also want to only allow a
> subset of the Role's ports access on a particular interface.
I think this really should be "a pre-designed subset", not
"arbitrary subset"; once the user starts listing port numbers, the
connection with the role starts becoming tenuous. It would be
reasonable for some roles to provide consistent set of ports (e.g.
"company-visible public data" vs. "DHCP and PXE and tftp server to
be restricted to a specific interface"), but we shouldn't need
arbitrary subsets that don't make sense (say, enabling cups
announcing itself over avai while preventing access to the IPP
service).
IPA might be a "worst-case" situation; if IPA can only live with
pre-designed subsets of ports, probably every thing can.
Good point. I could get on board with that.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEUEARECAAYFAlMzChQACgkQeiVVYja6o6NIbQCXSQOz3oCRVHuFFfM1dOsv7Ljn
0QCghtkKW9/QqvzTs7btm+uruF6lPJU=
=i1ls
-----END PGP SIGNATURE-----
7:38 a.m.
Hello everyone.
On Wed, 19 Mar 2014 09:47:20 -0400
Stephen Gallagher <sgallagh(a)redhat.com> wrote:
1) We want to be able to support two initial clients of this API:
the
Cockpit management console and a local command-line tool (I've been
tentatively calling it 'fedora-role-manage') A third, future option
will be to also support remote access to this API via the OpenLMI
Project.
Why do we need to separate the local and remote access in the CLI tool? You
know where I'm heading to... There's the LMIShell: interactive Python
shell-like environment allowing one to treat the remote CIM objects as local
Python objects, then there is the LMI metacommand on top of it wrapping the
direct CIM manipulations in a human-readable form since knowing the
details of the API itself is something we simply don't want the users to
bother with.
Now replace "CIM" with "D-Bus"... And at the "metacommand"
level you may not
even spot a difference. When you write about discoverability, API
versioning, etc., I can see so many similarities with OpenLMI server side
that I really think we don't need a brand new CLI tool to expose this new API.
Regards,
--
Tomáš Smetana
Platform Engineering, Red Hat
7:55 a.m.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 03/20/2014 08:38 AM, Tomáš Smetana wrote:
Hello everyone.
On Wed, 19 Mar 2014 09:47:20 -0400 Stephen Gallagher
<sgallagh(a)redhat.com> wrote:
> 1) We want to be able to support two initial clients of this API:
> the Cockpit management console and a local command-line tool
> (I've been tentatively calling it 'fedora-role-manage') A third,
> future option will be to also support remote access to this API
> via the OpenLMI Project.
Why do we need to separate the local and remote access in the CLI
tool? You know where I'm heading to... There's the LMIShell:
interactive Python shell-like environment allowing one to treat the
remote CIM objects as local Python objects, then there is the LMI
metacommand on top of it wrapping the direct CIM manipulations in a
human-readable form since knowing the details of the API itself is
something we simply don't want the users to bother with.
Now replace "CIM" with "D-Bus"... And at the "metacommand"
level
you may not even spot a difference. When you write about
discoverability, API versioning, etc., I can see so many
similarities with OpenLMI server side that I really think we don't
need a brand new CLI tool to expose this new API.
If the OpenLMI project would like to volunteer to implement this
provider and metacommand during the Fedora 21 schedule, I'd be all for
it. I was mostly assuming that the timing of things would be difficult
to manage and that we might want to hack up a simple local command for
the first pass.
I definitely see 'lmi role deploy' as a better solution (particularly
since the same syntax can be used locally or remotely) in the long
term. The provisional schedule for Fedora 21 means that the Alpha
change deadline (when all major features need to work) is July 22nd.
Are you willing to commit to being feature-complete by that date?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlMq5V4ACgkQeiVVYja6o6O4tACZAdgMF1Ja85baUIJC9cOecJqp
74wAn0IprGG64OIqDlZLTqRUTRAmIvu2
=aWXU
-----END PGP SIGNATURE-----
3:33 a.m.
Dne Thu, 20 Mar 2014 08:55:58 -0400
Stephen Gallagher <sgallagh(a)redhat.com> napsal(a):
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 03/20/2014 08:38 AM, Tomáš Smetana wrote:
> Hello everyone.
>
> On Wed, 19 Mar 2014 09:47:20 -0400 Stephen Gallagher
> <sgallagh(a)redhat.com> wrote:
>
>> 1) We want to be able to support two initial clients of this API:
>> the Cockpit management console and a local command-line tool
>> (I've been tentatively calling it 'fedora-role-manage') A third,
>> future option will be to also support remote access to this API
>> via the OpenLMI Project.
<...>
If the OpenLMI project would like to volunteer to implement this
provider and metacommand during the Fedora 21 schedule, I'd be all for
it. I was mostly assuming that the timing of things would be difficult
to manage and that we might want to hack up a simple local command for
the first pass.
It is difficult to manage not impossible though and it was generally admitted
that it makes sense for OpenLMI to participate in this effort.
I definitely see 'lmi role deploy' as a better solution
(particularly
since the same syntax can be used locally or remotely) in the long
term. The provisional schedule for Fedora 21 means that the Alpha
change deadline (when all major features need to work) is July 22nd.
Are you willing to commit to being feature-complete by that date?
We need to have some discussion about technical issues. OpenLMI is now trying
to figure out how to do HW RAID management which probably means using
libstoragemgmt and this would lead to: no CIM in OpenLMI this time. So we are
already moving to "non-CIM solutions" for certain tasks and have to find out
how to do that anyway.
I should know better on Monday (when the more knowledgeable people would be done
with other work). I'll let you know. OpenLMI is definitely interested.
Thanks and regards,
--
Tomáš Smetana
10:18 a.m.
On Fri, 21 Mar 2014 09:33:21 +0100
Tomáš Smetana <tsmetana(a)redhat.com> wrote:
> If the OpenLMI project would like to volunteer to implement
this
> provider and metacommand during the Fedora 21 schedule, I'd be all for
> it. I was mostly assuming that the timing of things would be difficult
> to manage and that we might want to hack up a simple local command for
> the first pass.
<...>
I should know better on Monday (when the more knowledgeable people
would be
done with other work). I'll let you know. OpenLMI is definitely
interested.
OK. We will implement the Server role provider and the metacommand and I'm
quite confident about these two to be ready in F21. We will also take a look
at patching Pegasus for the socket activation which would obviously make
things look much better when no remote management is required.
Regards,
--
Tomáš Smetana
Platform Engineering, Red Hat
4:02 p.m.
2014-03-19 14:47 GMT+01:00 Stephen Gallagher <sgallagh(a)redhat.com>:
?
A "redeploy" is more work for the role, having to compare the settings with
current deployment and apply them, but it also ensure that that code path
works; with the individual configuration modification operations, the roles
could still provide a GetCurrentSettings method, but we would have two
completely separate paths (EnableDisableDNS vs. Deploy) to test, and the
risk of them diverging.
1) How do we handle configuring the Firewall? I think we want to have
> a Firewall object (/org/fedoraproject/server/RoleFirewallManager)
> available to query the firewall as a whole, but we may also want to be
> able to view and apply firewall rules from the Role objects directly.
> (Note: I'm not suggesting we need a complete firewall solution here.
> This should be a wrapper that deals only with the Roles). Firewalld
> already provides a more comprehensive firewall interface for the
> general case.
If we provide a limited firewall facade so that Fedora Server can work
without firewalld, IMHO it should literally be the simplest possible
"role.FirewallPreventsNonLocalhostAccess" value; not even listing the ports
involved. If we provide something closer to a full-featured D-Bus API
firewall, we might just as well require firewalld directly instead of
writing a firewalld competitor.
Mirek
2) We do not need to have a single generic API that will work for
any
conceivable Role. We've identified in the past that the selection of
Featured roles will (for the conceivable future) remain a small,
manageable list. For this reason, it is acceptable for the
deployment and post-deployment configuration APIs to be entirely
different between two roles. However, some functionality such as
the backup and firewall queries should be stable between all roles.
(I.e. the universal Role in the proposal doesn't even have a DeployRole
method.)
I don't think this is efficient--every "universal" client (and there will
be
at least one or perhaps two, fedora-role-manage, anaconda/initial-setup
kickstart processing) would have to build essentially the same abstraction
over these role differences.
Wouldn't it be cleaner to have generic
ValidateSettings(settings)
DeployRole(settings)
as the abstraction used by the "universal" clients (with "settings"
possibly having some universal settings, like our favorite
firewall_enabled_after_deploy, but mostly role-specific, only using a
standard format, perhaps JSON or some object graph with similar
capabilities)?
The kickstart processor would only decode JSON and pass it to the API
without understanding the contents; the Role-specific GUIs would have
specific knowledge of settings to set--or specific Roles could even provide
helper APIs, say AutodetectDefaultReplicaControllerSettings() to query the
network and give the GUI information about what the role thinks is
appropriate for the specific situation.)
/org/fedoraproject/server/ServerRoleManager:
* properties:
* all_roles
* staging_roles
* active_roles
Isn't this redundant with the per-role properties specifying the role's
state? (Even if it were redundant, it wouldn't be a huge deal--just a
little work that we might be able to avoid.)
/org/fedoraproject/server/role/$ROLE:
* properties:
* loaded: Bool: whether the packages are installed
* deployed: Bool: whether the Role has been deployed
(Nitpicking? Rather an enum to eliminate possibilities of paradoxes like
deployed & ~loaded.)
* methods:
* GetFirewallPorts: list of port/protocol pairs that the Role
needs
I'd much rather have FirewallRestriction enum (LocalhostOnly, Unrestricted,
Complex) or something like that, or, if we decided to depend on firewalld,
an object reference to a firewalld service. Roles will be asking for ICMP,
new IP protocols, and whatnot, and we don't need to start with that
complexity in the API and the clients.
e.g. /org/fedoraproject/server/role/DomainController
* methods:
* DeployPrimaryDomainController: Set up the first domain controller in
the domain.
* DeployReplicaDomainController: Add a new replica domain controller
to the domain
See above.
* AddCertificateAuthority: Add a certificate authority to
this
domain controller
* EnableDisableDNS: Enable or disable the DNS server on
this server
This would of course work. A philosophical question, though: would we
rather move to closer to the "cattle" model, i.e.
settings = role.GetCurrentSettings()
settings.enableDNS = False
role.Redeploy(settings)
4:18 p.m.
2014-03-24 22:02 GMT+01:00 Miloslav Trmač <mitr(a)volny.cz>:
... with the individual configuration modification operations, the
roles
could still provide a GetCurrentSettings method
BTW the roles *should* provide a GetCurrentSettings in any case, to make it
as easy as possible to go from an experimental manually-configured server
to a kickstart for mass deployment.
Mirek
12:14 p.m.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 03/24/2014 05:02 PM, Miloslav Trmač wrote:
2014-03-19 14:47 GMT+01:00 Stephen Gallagher <sgallagh(a)redhat.com
<mailto:sgallagh@redhat.com>>:
2) We do not need to have a single generic API that will work for
any conceivable Role. We've identified in the past that the
selection of Featured roles will (for the conceivable future)
remain a small, manageable list. For this reason, it is acceptable
for the deployment and post-deployment configuration APIs to be
entirely different between two roles. However, some functionality
such as the backup and firewall queries should be stable between
all roles.
(I.e. the universal Role in the proposal doesn't even have a
DeployRole method.)
I don't think this is efficient—every "universal" client (and there
will be at least one or perhaps two, fedora-role-manage,
anaconda/initial-setup kickstart processing) would have to build
essentially the same abstraction over these role differences.
Wouldn't it be cleaner to have generic ValidateSettings(settings)
DeployRole(settings) as the abstraction used by the "universal"
clients (with "settings" possibly having some universal settings,
like our favorite firewall_enabled_after_deploy, but mostly
role-specific, only using a standard format, perhaps JSON or some
object graph with similar capabilities)?
That's an idea. I'm not sure it's doing anything other than moving the
problem to the settings object instead of the method call, but *shrug*.
The kickstart processor would only decode JSON and pass it to the
API without understanding the contents; the Role-specific GUIs
would have specific knowledge of settings to set—or specific Roles
could even provide helper APIs, say
AutodetectDefaultReplicaControllerSettings() to query the network
and give the GUI information about what the role thinks is
appropriate for the specific situation.)
*nod*
/org/fedoraproject/server/ServerRoleManager: * properties: *
all_roles * staging_roles * active_roles
Isn't this redundant with the per-role properties specifying the
role's state? (Even if it were redundant, it wouldn't be a huge
deal—just a little work that we might be able to avoid.)
Yes, see Jan's response as well. Better to leave that to the D-BUS
object manager.
/org/fedoraproject/server/role/$ROLE: * properties: * loaded:
Bool: whether the packages are installed * deployed: Bool:
whether the Role has been deployed
(Nitpicking? Rather an enum to eliminate possibilities of
paradoxes like deployed & ~loaded.)
See also Jan's reply. We also need a "deploying" state, so maybe an
enum is the right idea here.
* methods: * GetFirewallPorts: list of port/protocol pairs that the
Role needs
I'd much rather have FirewallRestriction enum (LocalhostOnly,
Unrestricted, Complex) or something like that, or, if we decided
to depend on firewalld, an object reference to a firewalld service.
Roles will be asking for ICMP, new IP protocols, and whatnot, and
we don't need to start with that complexity in the API and the
clients.
Sorry, that was a little thin. Could you go into a bit more detail
here, please?
e.g. /org/fedoraproject/server/role/DomainController * methods: *
DeployPrimaryDomainController: Set up the first domain controller
in the domain. * DeployReplicaDomainController: Add a new replica
domain controller to the domain
See above.
* AddCertificateAuthority: Add a certificate authority to
this domain controller * EnableDisableDNS: Enable or
disable the DNS server on this server
This would of course work. A philosophical question, though: would
we rather move to closer to the "cattle" model, i.e.
settings = role.GetCurrentSettings() settings.enableDNS = False
role.Redeploy(settings)
?
A "redeploy" is more work for the role, having to compare the
settings with current deployment and apply them, but it also ensure
that that code path works; with the individual configuration
modification operations, the roles could still provide a
GetCurrentSettings method, but we would have two completely
separate paths (EnableDisableDNS vs. Deploy) to test, and the risk
of them diverging.
Hmm, but redeploy may not be possible (or idempotent) for all possible
setting changes. For example, if we deploy FreeIPA with a built-in
Certificate Authority, we can't redeploy without it, since the whole
system will have been predicated on it. (Not without having a
mechanism for redeploy being effectively a complete migration...
possibly between VMs or containers?)
1) How do we handle configuring the Firewall? I think we want to
have a Firewall object
(/org/fedoraproject/server/RoleFirewallManager) available to query
the firewall as a whole, but we may also want to be able to view
and apply firewall rules from the Role objects directly. (Note: I'm
not suggesting we need a complete firewall solution here. This
should be a wrapper that deals only with the Roles). Firewalld
already provides a more comprehensive firewall interface for the
general case.
If we provide a limited firewall facade so that Fedora Server can
work without firewalld, IMHO it should literally be the simplest
possible "role.FirewallPreventsNonLocalhostAccess" value; not even
listing the ports involved. If we provide something closer to a
full-featured D-Bus API firewall, we might just as well require
firewalld directly instead of writing a firewalld competitor.
I think my original statement failed to get my point across, but I
think I was pretty much saying the same thing you are. First of all,
we *did* agree to use firewalld as the proper solution. I just meant
that in terms of defining the Role-firewall interop, it should *not*
try to be a complete set of firewall rules. For that, we should allow
a Role to have a 'managed_firewall=False' setting[1] or something like
it, and that admins would be expected to handle the firewall
appropriately through firewalld/iptables. But if we *are* managing it,
it's probably acceptable for it to be essentially either blocked or
allowed on a whitelist of interfaces ('lo' being just a special case).
Anyone requiring more control than that should set
managed_firewall=False and do it themselves.
[1] The reason for this being so that UIs can note that this is under
manual control and also so that later Role upgrades don't attempt to
open or close ports.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlMxuWwACgkQeiVVYja6o6MuYwCgim8J9OVKmmZ+69ClZa762GL6
sS4AnRns9c9OsH/LmpW0PwckV5xKXOv9
=+K0s
-----END PGP SIGNATURE-----
6:17 p.m.
2014-03-25 18:14 GMT+01:00 Stephen Gallagher <sgallagh(a)redhat.com>:
On 03/24/2014 05:02 PM, Miloslav Trmač wrote:
> 2014-03-19 14:47 GMT+01:00 Stephen Gallagher <sgallagh(a)redhat.com
> Wouldn't it be cleaner to have generic ValidateSettings(settings)
> DeployRole(settings) as the abstraction used by the "universal"
> clients (with "settings" possibly having some universal settings,
> like our favorite firewall_enabled_after_deploy, but mostly
> role-specific, only using a standard format, perhaps JSON or some
> object graph with similar capabilities)?
That's an idea. I'm not sure it's doing anything other than moving the
problem to the settings object instead of the method call, but *shrug*.
Right, it would be even better to provide role-specific APIs (or a generic
API with schema enforcement) to change values within the settings object
:) in order to make typos in settings names obvious, and that point it is
really similar.
It does only one structurally significant thing: encode the "settings
format" (JSON/whatever) as a canonical way to describe/dump/persistently
store role deployment configuration, giving the API users precisely the
same facilities they have during installation.
> * methods: * GetFirewallPorts: list of port/protocol pairs that
the
> Role needs
>
>
> I'd much rather have FirewallRestriction enum (LocalhostOnly,
> Unrestricted, Complex) or something like that, or, if we decided
> to depend on firewalld, an object reference to a firewalld service.
> Roles will be asking for ICMP, new IP protocols, and whatnot, and
> we don't need to start with that complexity in the API and the
> clients.
Sorry, that was a little thin. Could you go into a bit more detail
here, please?
Stepping back a little, the role *implementation* and *external API* should
be different. The implementation obviously does need to actually contain
port numbers and the like somewhere, but I don't think we want them in the
role's public API.
The reason is that this introduces too much complexity into the API, at a
level that isn't interesting to many important users of the API. The
complexity is in all the things a role might need (TCP ports, UDP ports, IP
protocols, specific multicast addresses, perhaps even conntrack modules),
and the presumed callers of a "Role deploy API" just don't care, they want
the role deployed firewalled/not firewalled, and a service dashboard needs
to to know whether it has been firewalled. Receiving a list of
half-a-dozen different individual firewall concepts to check isn't all that
helpful, the public API should be able to give a simple yes/no answer
(well, a "yes/no/"it's complicated" answer, which are the
badly-explained
LocalhostOnly/Unrestricted/Complex values above).
Depending on firewalld, it seems simplest for roles to provide firewalld
services, and refer to callers to them. We might still want a simple
facade above, providing only the ability set LocalhostOnly/Unrestricted,
and to receive current status as one of the three values. That would be
appropriate for a simple dashboard, more detailed management would be done
on the firewalld service object directly using the firewalld API.
> * AddCertificateAuthority: Add a certificate authority to
> this domain controller * EnableDisableDNS: Enable or
> disable the DNS server on this server
>
> This would of course work. A philosophical question, though: would
> we rather move to closer to the "cattle" model, i.e.
>
> settings = role.GetCurrentSettings() settings.enableDNS = False
> role.Redeploy(settings)
> ?
>
> A "redeploy" is more work for the role, having to compare the
> settings with current deployment and apply them, but it also ensure
> that that code path works; with the individual configuration
> modification operations, the roles could still provide a
> GetCurrentSettings method, but we would have two completely
> separate paths (EnableDisableDNS vs. Deploy) to test, and the risk
> of them diverging.
>
Hmm, but redeploy may not be possible (or idempotent) for all possible
setting changes.
It's always possible to delete all data and start fresh :) An API to
redeploy that doesn't delete data needs the option to fail, sure--but any
functionality that can be done using a GetValue/SetValue-like API should be
equally possible to do using a Redeploy model; in the role-already-deployed
case, it's just a series of
if (settings.option_name != role.GetValue(option_name))
role.setValue(option_name, settings.option_name)
and
if (settings.immutable_option_name !=
role.GetValue(immutable_option_name)
fail;
snippets.
So semantically the two are sort of equivalent, this is primarily a matter
of how we want to encourage the users to work with the system: keep a pet
and modify it over time with the API, or have a settings file managed in a
VCS somewhere and use the API only to sync the VCS and the deployment?
> If we provide a limited firewall facade so that Fedora Server
can
> work without firewalld, IMHO it should literally be the simplest
> possible "role.FirewallPreventsNonLocalhostAccess" value; not even
> listing the ports involved. If we provide something closer to a
> full-featured D-Bus API firewall, we might just as well require
> firewalld directly instead of writing a firewalld competitor.
I think my original statement failed to get my point across, but I
think I was pretty much saying the same thing you are.
I think so too.
First of all,
we *did* agree to use firewalld as the proper solution. I just meant
that in terms of defining the Role-firewall interop, it should *not*
try to be a complete set of firewall rules. For that, we should allow
a Role to have a 'managed_firewall=False' setting[1] or something like
it, and that admins would be expected to handle the firewall
appropriately through firewalld/iptables. But if we *are* managing it,
it's probably acceptable for it to be essentially either blocked or
allowed on a whitelist of interfaces ('lo' being just a special case).
Anyone requiring more control than that should set
managed_firewall=False and do it themselves.
We could even have the managed_firewall=False implicit, as "service is not
assigned to the Public or Drop zones" (... which would leave even the
"allow on a whitelist of interfaces" case up to non-Role-API firewalld
calls; I have no idea whether this would be appropriate).
Mirek
2:57 a.m.
On 03/26/2014 12:17 AM, Miloslav Trmač wrote:
2014-03-25 18:14 GMT+01:00 Stephen Gallagher <sgallagh(a)redhat.com
<mailto:sgallagh@redhat.com>>:
> On 03/24/2014 05:02 PM, Miloslav Trmač wrote:
>> 2014-03-19 14:47 GMT+01:00 Stephen Gallagher <sgallagh(a)redhat.com
<mailto:sgallagh@redhat.com>
>> Wouldn't it be cleaner to have generic ValidateSettings(settings)
>> DeployRole(settings) as the abstraction used by the "universal"
>> clients (with "settings" possibly having some universal settings,
>> like our favorite firewall_enabled_after_deploy, but mostly
>> role-specific, only using a standard format, perhaps JSON or some
>> object graph with similar capabilities)?
>
> That's an idea. I'm not sure it's doing anything other than moving the
> problem to the settings object instead of the method call, but *shrug*.
Right, it would be even better to provide role-specific APIs (or a
generic API with schema enforcement) to change values within the
settings object :) in order to make typos in settings names obvious,
and that point it is really similar.
It does only one structurally significant thing: encode the "settings
format" (JSON/whatever) as a canonical way to describe/dump/persistently
store role deployment configuration, giving the API users precisely the
same facilities they have during installation.
What UDisks and storaged do, they have 'options' parameter of nearly all
methods, e.g. BlockDevice.Format(fstype, options).
The options parameter is just dictionary string -> variant, where
variant can be literally anything, int, string, array, dictionary of
arrays, etc. I think you could represent it as JSON or XML to users.
Supported keys and allowed values are usually documented somewhere and
adding new key is not considered as API change, while adding a new
method parameter is.
See
http://udisks.freedesktop.org/docs/latest/gdbus-org.freedesktop.UDisks2.B...
for example.
Also, UDisks has a 'Configuration' property for a block device, which
basically sets fstab / crypttab properties + it has methods to
manipulate it. I don't know how common is this approach.
See
http://udisks.freedesktop.org/docs/latest/gdbus-org.freedesktop.UDisks2.B...
Jan
1 p.m.
2014-03-26 8:57 GMT+01:00 Jan Safranek <jsafrane(a)redhat.com>:
On 03/26/2014 12:17 AM, Miloslav Trmač wrote:
> 2014-03-25 18:14 GMT+01:00 Stephen Gallagher <sgallagh(a)redhat.com
> <mailto:sgallagh@redhat.com>>:
>> On 03/24/2014 05:02 PM, Miloslav Trmač wrote:
>>> 2014-03-19 14:47 GMT+01:00 Stephen Gallagher <sgallagh(a)redhat.com
> <mailto:sgallagh@redhat.com>
>>> Wouldn't it be cleaner to have generic ValidateSettings(settings)
>>> DeployRole(settings) as the abstraction used by the "universal"
>>> clients (with "settings" possibly having some universal settings,
>>> like our favorite firewall_enabled_after_deploy, but mostly
>>> role-specific, only using a standard format, perhaps JSON or some
>>> object graph with similar capabilities)?
>>
>> That's an idea. I'm not sure it's doing anything other than moving
the
>> problem to the settings object instead of the method call, but *shrug*.
>
> Right, it would be even better to provide role-specific APIs (or a
> generic API with schema enforcement) to change values within the
> settings object :) in order to make typos in settings names obvious,
> and that point it is really similar.
>
> It does only one structurally significant thing: encode the "settings
> format" (JSON/whatever) as a canonical way to describe/dump/persistently
> store role deployment configuration, giving the API users precisely the
> same facilities they have during installation.
What UDisks and storaged do, they have 'options' parameter of nearly all
methods, e.g. BlockDevice.Format(fstype, options).
The options parameter is just dictionary string -> variant, where
variant can be literally anything, int, string, array, dictionary of
arrays, etc. I think you could represent it as JSON or XML to users.
Actually a flat dictionary might well be sufficient. I was just
mentioning JSON as a way to avoid inventing a plain text
representation.
(And, to somewhat contradict the gist of my comments, an explicit API
on a settings object, say SetDHCPEnabled(bool) is IMHO generally
preferable to passing a dictionary and relying only on non-automatable
key name conventions within a dictionary. But starting with the
requirement to have the generic API define a plaintext format, the
plaintext format is already a way to bypass any explicit API in favor
of name conventions, so... ... so you need to keep pushing back
against me complicating the API I think :) )
Also, UDisks has a 'Configuration' property for a block
device, which
basically sets fstab / crypttab properties + it has methods to
manipulate it. I don't know how common is this approach.
See
http://udisks.freedesktop.org/docs/latest/gdbus-org.freedesktop.UDisks2.B...
I intuitively really don't like this; that's freezing the existing
not-allways-that-well-designed configuration as a stable API, while at
the same time making the API volatile and changed by any upstream
update to the upstream file formats. Any caller willing to deal with
the upstream file formats could just as well use them directly.
But it's you who has actual experience with defining and implementing
system management APIs, my intuition shouldn't have that much weight.
Mirek
3676
days inactive
3684
days old
server@lists.fedoraproject.org
18 comments
6 participants
participants (6)
-
Jan Safranek -
Kevin Fenzi -
Miloslav Trmač -
Stephen Gallagher -
Thomas Woerner -
Tomáš Smetana