Design of mounting in OpenLMI-Storage
by Jan Safranek
Hi,
we should do some mounting in OpenLMI storage provider and I want to
discuss its design.
(sorry for HTML, is there any better way how to send such proposals?)
* SMASH and DASH do not cover mounting of filesystems.
* SMI-S has Filesystem Manipulation Profile, but it's closely tight to
filesystem management, for example:
o Filesystems are mounted using SNIA_CreateFileSystem method, i.e.
it's possible to create & mount a FS in one call, but it's weird
to call SNIA_CreateFileSystem to mount existing FS (but it's
possible).
o Filesystems can be mounted only once.
o Only local filesystems can be mounted.
CIM provides lot of classes to manage mounts:
* CIM_Mount - association between a FS and a directory, where it's
mounted.
* CIM_FileImportService - service, which mounts remote filesystems
(NFS, CIFS) + usual Setting & Capabilities classes
There is no service which mounts local filesystems!
So, looking at the CIM_FileImportService, we might get inspired by it
and create following classes:
* LMI_MountService
o CreateMount(FS, type, mountpoint, goal, permanent)
+ FS: reference to either Local or RemoteFileSystem or NULL.
+ type: uint16, enum (nfs, cifs, xfs, ext2/3/4, vfat, ...).
+ mountpoint: string (or reference to LMI_Directory?).
+ goal: reference to LMI_MountSettings or NULL to use default.
+ permanent: boolean, whether to store in fstab.
+ if FS is NULL, type can be used to create tmpfs, sysfs,
procfs, cgroups, ...
+ LMI_Mount is created with attached LMI_MountSetting.
o Remote file systems are represented by RemoteFileSystem, but how
to enumerate them when they are not mounted? We probably don't
want to scan the network for available NFS/CIFS shares. We might
add a method to create such RemoteFileSystem and mount it using
CreateMount(), but I propose following function:
o CreateRemoteMount(specifier, type, mountpoint, permanent)
+ specifier: string like 'foo:/home/jsafrane' for NFS or
'//foo/home/jsafrane' for CIFS.
+ type, mountpoint, goal, permanent: same as above.
+ Appropriate CIM_RemoteFileSystem gets created.
+ LMI_Mount is created with attached LMI_MountSetting.
o ModifyMount(mount, goal, permanent)
+ mount: reference to existing LMI_Mount.
+ goal: reference to new LMI_MountSettings of the mount.
+ permanent: whether to store this mount in fstab.
# if the mount was permanent and is not anymore, it gets
removed from fstab.
# if the mount was not permanent and is now, it gets added
to fstab.
# if the mount was permanent and still is, fstab is edited.
o DeleteMount(mount)
+ mount: reference to existing LMI_Mount.
o All these methods return Job (mounting can take some time, right?)
* LMI_Mount: inherited from CIM_Mount
o Association between a CIM_FileSystem (local or remote) and
CIM_Directory where it's mounted.
o Has property if this mount is permanent (in fstab).
o Has property if this mount is active (i.e. currently mounted).
o There can be LMI_Mount, which does not exist on the system, but
has entry in fstab!
+ e.g. when admin remounts a mount which is in fstab with
different options, there will be two LMI_Mount instances.
# One for the fstab mount (inactive, permanent).
# One for the actual mount (active, not permanent).
# OpenLMI can edit both.
# One of the mount instances disappears when fstab and
running system gets in sync.
+ /We must add new key to the association to //distinguish
multiple mounts of one fs to the same directory, just with
different options and permanent/non permanent flag./
+ /There can be only one fstab entry for (device, directory)
pair./
o If there is an entry in fstab but the filesystem (or block
device) is not available, no LMI_Mount is created!
+ /How to edit/delete such leftovers? Maybe the device was
just renamed./
* LMI_MountSetting
o Configuration of a mount.
o Has properties for selected mount options
+ ro
+ rw
+ Anything else? We may gradually extend this class and even
make subclasses for each FS, e.g. have LMI_Ext2MountSetting,
LMI_XFSMountSetting, LMI_NFSMountSetting, ...
o Has generic array of strings of 'additional options' for all
mount options, which are not covered by specific properties.
o Is associated with LMI_Mount to shows its configuration.
+ /LMI_Mount is association, i.e. we have association between
association and MountSetting, which is slightly confusing./
* LMI_MountCapabilities
o Capabilities of the MountService, as usual in SMI-S and LMI
storage model.
UML class diagram
This CIM API does not follow any DMTF or SMI-S standard, but follows the
idea of it - have a Service, Setting and Capabilities classes and can be
extended in the future.
Later we still may implement CIM_FileImportService if someone wants, but
it would be just a wrapper around our MountService.
In Fedora19 I think we can do only 'transient' mounts, no fstab editing,
Blivet doesn't support non-destructive fstab manipulation now. In
addition, Blivet does support mounting of local filesystems on block
devices for now (no cgroups, sysfs, procfs, tmpfs, ...), but even this
limited functionality is IMHO quite lot of work on OpenLMI side.
Any comments are welcome, especially the parts in /italics/ are quite
disturbing to me. And links to any model which already covers mounting
would be greatly appreciated.
Jan
11 years, 1 month
What is the best way to model a name/value pair in CIM?
by John Dennis
In the provider I'm writing lists of key/value pairs are needed in a few
different contexts (as a parameter to a method, as an instance property,
etc.).
In CIM what is the best way to to model a name/value pair? In CIM I
dont' think there is any mechanism by which the value can be of an
arbitrary type (right?). Therefore lets assume both the name and value
are strings.
I didn't find much in the existing core schema for modeling name/value.
There is the CIM_Configuration class but it's deprecated, replaced by
CIM_Setting. But CIM_Setting has semantics inappropriate for a general
name/value pair.
Is the best approach to define a new class that has just a name and
value property? (I can't imagine others haven't had the need for
name/value pairs, so shouldn't such a fundamental class be part of the
core schema? If so I haven't found it.) Such a class would not need a
key property [1] see below for a requested clarification.
If the best approach is to define a class with just a name and value
property then is the best way to to have a multiple name/values is to
define an array of references to the individual name/value instances?
There doesn't seem to be any need or advantage to introducing a
CIM_Collection for this purpose, or is there?
Are there examples of other CIM classes that use sets of name/value
pairs which would provide an example of best CIM practice for this?
[1] I've been a bit confused by the term "concrete" as used in the
context of CIM. One definition I've seen is a concrete class is any
non-abstract class. Another definition I saw was a concrete class is an
instantiated class instance. And yet a third definition I saw was a
concrete class has to have a one or more key properties. So two questions:
What is a concrete class?
Is defining a class without any key properties problematic if there is
no need to uniquely reference a specific class instance?
--
John Dennis <jdennis(a)redhat.com>
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/
11 years, 1 month
Storage events
by Jan Safranek
Hello all,
I've talked to David Lehman (Blivet devel) how OpenLMI can report
various storage events to applications.
Currently, Blivet does not consume any events from udev and it would be
rather difficult to implement it - Blivet is a library and receiving
events means it must have some sort of listening thread with a main loop.
1: http://pyudev.readthedocs.org/en/latest/
So, what events OpenLMI wants to report to applications (and what info
it wants) [source of the event]:
- device appeared/disappeared [udev]
- device was changed (*what* changed - e.g. size, fs on it, RAID level,
...) [udev]
- mount appeared/disappeared [/proc/mounts]
- mount changed (list of changed options) [/proc/mounts]
- RAID events
- man mdamd, look for "MONITOR MODE". There might be
netlink/inotify/dbus events for that.
- LVM events (snapshot/thin LV/VG is running out of space; with
configurable threshold) [none at the moment]
As written on [2], select or poll on /proc/mounts or
/proc/self/mountinfo can be used to get mounting events.
2:
http://karelzak.blogspot.cz/2011/12/monitor-list-of-currently-mounted.html
Question is, how Blivet and OpenLMI cooperate to generate these events.
Option 1: OpenLMI listens for udev and mount and mdadm events (in C, not
in Python). When an event occurs, it re-creates Blivet's device tree and
compares old and new trees manually on its own. Benefit of this solution
is that Blivet (Python) does not need to be in memory for the whole
time. This would require some wrapping between CIMOM and OpenLMI, but I
think that's doable.
Option 2: OpenLMI tells Blivet to wait for events. When event occurs,
Blivet calls OpenLMI callback. Add/remove events are simple, just tell
OpenLMI what appeared/disappeared. Changes are more complex, but IMHO it
might be enough to call the callback with old StorageDevice and new
StorageDevice and we'll check what's changed. This solution can be used
by more applications on top of Blivet, but it requires Blivet to run a
main loop in a thread and be in memory for the whole time.
Option 1.5: OpenLMI listens for the udev, mount and mdadm events (in C,
not in Python), but passes them in some 'raw' form to Blivet. Blivet
then check what's going on and calls OpenLMI callback as in Option 2. We
don't need to have Blivet in memory for the whole time, but we need it
to be possible to store Blivet state to a file and reload it when requested.
Of course, we expect incremental development, let's start with udev
first and gradually integrate all other sources of events.
I personally vote for option 2, not because it's more work for Blivet
and less for OpenLMI, but if Blivet is going to be generic storage
library, this is one of core functionality. If option 2 show too
resource-heavy, we might try to go for 1.5.
Jan
11 years, 1 month
Is enumerateInstances guaranteed to be called exactly once?
by John Dennis
I've been looking at the Konkret doc and the various OpenLMI providers
and there seems to be an an inherent assumption that enumerateInstances
will be called exactly once. There are no checks to see if an instance
already exists during the enumeration. I went to the CMPI spec for a
clarification, except it had nothing to say about this (at least as far
I could find).
My naive assumption for a function with the verb Enumerate in it's name
followed by the noun Instance says you're suppose to return a list of
existing Instances, not create them. Also you might be asked to
enumerate those same Instances more than once (i.e. give me a list of
what you've got).
But it seems like the actual semantics are "create your initial set of
default instances and tell me what they are" Hence it has more to do
with creation than enumeration. Is this correct or do I have a
misunderstanding? Is this actually spelled out someplace? If so can you
point me at that doc?
Thanks,
John
--
John Dennis <jdennis(a)redhat.com>
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/
11 years, 1 month
Konkret array counts
by John Dennis
Oddly there doesn't seem to be an interface (API) in Konkret to get the
size of an array. This seems like a very odd omission.
For instance, lets say you define a method that has an IN parameter
which is an array of strings, for example sake let's call this array
"Names", your Konkret generated method will have a parameter like this:
const KStringA *Names
To do anything with the Names array you have to know how many elements
it has, this proved difficult to figure out.
The definition of KStringA looks like this:
typedef struct _KStringA
{
CMPIUint32 exists;
CMPIUint32 null;
const CMPIArray* value;
char __padding1[16 - sizeof(CMPIArray*)];
CMPIUint32 count;
char __padding2[4];
}
KStringA;
Notice it has a count member. If you were thinking there is an accessor
defined as an inline or macro for the count you'd be wrong. If you were
thinking you could access Names->count and get a valid value you would
be wrong. Apparently it's never initialized.
I verified that when my method is invoked via YAWN it's passing an array
of strings, I can see it in the generated XML. I also called the
XXX_Args_Print() function that Konkret generated. It also shows the
expected array elements. I also called KStringA_Get(Names, 0) to
retrieve the first element, and lo and behold got the first element.
But nowhere can I find how many elements are in this array nor a way to
iterate it.
Apparently there is nothing in Konkret to give you this critical
information. I resorted to calling CMPI directly (which seems like an
API violation) by doing this:
count = CMGetArrayCount(OptionNames->value, NULL);
which works.
But this seems wrong, I shouldn't be reaching in to CMPI data structures
that Konkret is supposed to be hiding from me, *plus* what's the point
of the count member of the KStringA struct? BTW all KArray types have a
count field, by looking at the source code it appears none of the count
fields are initialized, unless you create the array yourself via of of
the K*A_Init() functions which calls KArray_Init().
Also, there doesn't seem to be any doc whatsoever in Konkret for working
with arrays. That also seems odd. Are arrays a second thought in Konkret?
Comments? Am I missing something?
John
--
John Dennis <jdennis(a)redhat.com>
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/
11 years, 2 months
Re: openlmi-devel Digest, Vol 6, Issue 1
by Russell Doty
On Mon, 2013-02-04 at 12:30 +0000,
openlmi-devel-request(a)lists.fedorahosted.org wrote:
> Send openlmi-devel mailing list submissions to
> openlmi-devel(a)lists.fedorahosted.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.fedorahosted.org/mailman/listinfo/openlmi-devel
> or, via email, send a message with subject or body 'help' to
> openlmi-devel-request(a)lists.fedorahosted.org
>
> You can reach the person managing the list at
> openlmi-devel-owner(a)lists.fedorahosted.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of openlmi-devel digest..."
>
>
> Today's Topics:
>
> 1. Re: Proposed F19 Feature: systemd features (Stephen Gallagher)
> 2. RFC: Blivet & mounting (Jan Safranek)
> 3. Re: RFC: Blivet & mounting (Klaus Kaempf)
> 4. Re: RFC: Blivet & mounting (Jan Safranek)
> 5. Re: RFC: Blivet & mounting (Stephen Gallagher)
> 6. Re: RFC: Blivet & mounting (Klaus Kaempf)
> 7. About the firewall provider (Tomáš Smetana)
> 8. Re: RFC: Blivet & mounting (Stephen C. Tweedie)
>
>
> ----------------------------------------------------------------------
>
> Date: Thu, 31 Jan 2013 15:57:40 +0100
> From: Jan Safranek <jsafrane(a)redhat.com>
> To: David Lehman <dlehman(a)redhat.com>
> Cc: openlmi-devel(a)lists.fedorahosted.org, Discussion of Development
> and Customization of the Red Hat Linux Installer
> <anaconda-devel-list(a)redhat.com>
> Subject: RFC: Blivet & mounting
> Message-ID: <510A8664.2070104(a)redhat.com>
> Content-Type: text/plain; charset=UTF-8
>
> Hello,
>
> OpenLMI project uses Blivet, former Anaconda storage library, to manage
> storage. Management of storage also means management of mounts, i.e.
> ability to mount/unmount/remount stuff, list existing mounts and
> add/delete/modify/list entries in /etc/fstab.
>
> The installer somehow manipulates fstab and mounts stuff and there is
> some code in Blivet for that, but I am not sure it can be easily
> extended for generic use case.
>
> Is generic mounting/unmounting capability that something Blivet should
> do, so it would be integrated there, or there should be separate library
> for this? In both cases, we (=OpenLMI) would /probably/ provide the code.
>
> Is there any other potential Blivet user who needs mounting? Can the
> installer benefit from more generic mounting?
>
> What OpenLMI needs:
> - parse /proc/mounts and provide list of mounts with their options
> - parse /etc/fstab and provide list of persistent mounts with their
> options (which can be different to /proc/mounts, someone might re-mount
> something with different options!)
> - re-mount a mount with different options
> - create new mount, both permanent (with fstab entry) and 'temporary'
> - unmount anything (incl. removing entry from fstab).
>
> BTW, for fstab manipulation I am thinking about augeas-python. It does
> not pull any big dependency except libxml2, still it's yet another
> package needed in installer ramdisk if generic mounting is in Blivet.
>
> Jan
I would strongly encourage putting all storage related code into Blivet.
This gives us a single source for storage manipulation capabilities, the
opportunity to solve integration issues in one place, and a better end
user experience.
If we fragment storage we will reduce the value and use of Blivet.
>
>
> ------------------------------
> Message: 7
> Date: Mon, 4 Feb 2013 13:12:53 +0100
> From: Tomáš Smetana <tsmetana(a)redhat.com>
> To: openlmi-devel(a)lists.fedorahosted.org
> Subject: About the firewall provider
> Message-ID: <20130204131253.324c102d(a)zaphod.usersys.redhat.com>
> Content-Type: text/plain; charset=UTF-8
>
> Hello,
> I think it might be useful to dump my ideas about the firewall provider
> here to see what the others think.
>
> I was looking at the DMTF standard models for the packet filtering: There is
> a FilterList class to aggregate FilterEntryBase classes (or their
> descendants). All the FilterEntryBase items are ANDed in the list (no notion
> of "chain"). Given there is no "management" class that would allow altering
> the filters and the lists we will need to create proprietary ones for this
We aren't looking at creating a proprietary interface - we are looking
at creating an open interface which is non-conformant with parts of the
DMTF/CIM specification. Perhaps we should refer to these things as
custom interfaces or Linux specific interfaces?
> purpose. Since we've decided to rely on the firewalld as the backend the
> management classes would probably need to copy the Firewalld API to some
> extent. I'm inclined to think this provider is one of those where the
Makes sense.
> standard model is simply unsuitable.
Sounds like it.
>
> The other problem is: our Trac says [0] (it's quite vague otherwise) the first
> version of the firewall provider is scheduled to Milestone 1. Looking at the
> firewalld development I'm almost sure whatever we create by then would have
> to be rewritten anyway. Do we want to move the ticket deadline?
Sounds like we have to.
>
> Regards,
11 years, 2 months
RFC: Blivet & mounting
by Jan Safranek
Hello,
OpenLMI project uses Blivet, former Anaconda storage library, to manage
storage. Management of storage also means management of mounts, i.e.
ability to mount/unmount/remount stuff, list existing mounts and
add/delete/modify/list entries in /etc/fstab.
The installer somehow manipulates fstab and mounts stuff and there is
some code in Blivet for that, but I am not sure it can be easily
extended for generic use case.
Is generic mounting/unmounting capability that something Blivet should
do, so it would be integrated there, or there should be separate library
for this? In both cases, we (=OpenLMI) would /probably/ provide the code.
Is there any other potential Blivet user who needs mounting? Can the
installer benefit from more generic mounting?
What OpenLMI needs:
- parse /proc/mounts and provide list of mounts with their options
- parse /etc/fstab and provide list of persistent mounts with their
options (which can be different to /proc/mounts, someone might re-mount
something with different options!)
- re-mount a mount with different options
- create new mount, both permanent (with fstab entry) and 'temporary'
- unmount anything (incl. removing entry from fstab).
BTW, for fstab manipulation I am thinking about augeas-python. It does
not pull any big dependency except libxml2, still it's yet another
package needed in installer ramdisk if generic mounting is in Blivet.
Jan
11 years, 2 months
About the firewall provider
by Tomáš Smetana
Hello,
I think it might be useful to dump my ideas about the firewall provider
here to see what the others think.
I was looking at the DMTF standard models for the packet filtering: There is
a FilterList class to aggregate FilterEntryBase classes (or their
descendants). All the FilterEntryBase items are ANDed in the list (no notion
of "chain"). Given there is no "management" class that would allow altering
the filters and the lists we will need to create proprietary ones for this
purpose. Since we've decided to rely on the firewalld as the backend the
management classes would probably need to copy the Firewalld API to some
extent. I'm inclined to think this provider is one of those where the
standard model is simply unsuitable.
The other problem is: our Trac says [0] (it's quite vague otherwise) the first
version of the firewall provider is scheduled to Milestone 1. Looking at the
firewalld development I'm almost sure whatever we create by then would have
to be rewritten anyway. Do we want to move the ticket deadline?
Regards,
--
Tomáš Smetana
Base OS Engineering, Red Hat
[0] https://fedorahosted.org/openlmi/ticket/23
11 years, 2 months