Greetings,

I was recently tasked with standardizing/documenting our Linux server build/configuration process and as part of this process began training some of our Windows/Novell administrators to be able to carry out some of these basic tasks. It was this second part of helping train people that caused me to examine and re-examine the processes I used for this standardization. My goal was simple in theory, I would use pxe+kickstart and enforce standard configurations and policies via a series of post scripts while attempting to keep readability, simplicity and supportability from a "non-unix guru" perspective as my guiding light (A wise man once said: Make it as simple as possible but no simpler). For those interested an example of what I came up with it can be obtained here: http://geeklords.org/~shane/kickstart The process turned out to be not quite as simple as the theory but very educational none the less. To start with I had a number of ways I could manipulate my fresh Linux install. 1) I could create a custom RPM that had all of the changes imbedded in config files and rpm scripts that merely overwrote the pre-existing ones. Problem being this approach hides the complexity of what all was being changed and why. 2) Use cfengine after the kickstart install. This of course has some advantages over the rpm method but it also hides the same complexity and adds some of its own.

Greetings, This message to the Fedora Development Community is an attempt to start a useful discussion on where Fedora/RHEL is today and where it is going in the future, on the subject of OS management. My apologies in advance for the long message. OS management is a very broad topic, in specific I will focus on the elements that I consider foundational weaknesses (compared to what it could be). This is not to imply these perceived "weaknesses" out weigh its strengths or that other OS's are overall superior in this regard. A bit of background to put my technical comments in perspective: I remember when I was first introduced to Unix (1992) and then later Linux (1993) compared to what I was used to at the time they combined a flexibility, functionality and a form of simplicity (in design) that when combined made all other operating systems seem hallow. For the most part I still feel this way today, but over the years having worked in enterprise environments where Linux is the up and coming challenger to entrenched non-unix platforms (VMS, Windows, zOS, etc...) I have come to better appreciate some of the elements involved in limiting Linux's adoption. I was recently tasked with standardizing/documenting our Linux server build/configuration process and as part of this process began training some of our Windows/Novell administrators to be able to carry out some of these basic tasks. It was this second part of helping train people that caused me to examine and re-examine the processes I used for this standardization. My goal was simple in theory, I would use pxe+kickstart and enforce standard configurations and policies via a series of post scripts while attempting to keep readability, simplicity and supportability from a "non-unix guru" perspective as my guiding light (A wise man once said: Make it as simple as possible but no simpler). For those interested an example of what I came up with it can be obtained here: http://geeklords.org/~shane/kickstart The process turned out to be not quite as simple as the theory but very educational none the less. To start with I had a number of ways I could manipulate my fresh Linux install. 1) I could create a custom RPM that had all of the changes imbedded in config files and rpm scripts that merely overwrote the pre-existing ones. Problem being this approach hides the complexity of what all was being changed and why. 2) Use cfengine after the kickstart install. This of course has some advantages over the rpm method but it also hides the same complexity and adds some of its own. 3) Document on paper all of the steps required and how to perform them from the console, taking advantage of the guis when available or command line when required. I felt that was not only a big waste of time but left far too much to human error. 4) Create a series of simple scripts using basic OS applications/tools (sed, echo, cp, mv, authtool, postconf etc...) to perform all of the required modifications, documenting what and why along the way. I choose method 4. My conclusion having completed this process is that the number of variables, complexity and dependencies that must be understood and taken into account when modifying configuration files is much higher than it needs to be. I think this should be clear when reading the example scripts mentioned above. Some things I learned: Sed is a wonderful tool, but it is highly limited by the fact its user must take into account whole file(s) for each expression, this is further complicated when one must consider the file may change over time. The complexity and readability of regular expression tools is much higher than should be required to change OS/application variables. Creating new files or appending to the end of existing files with "echo" only takes one so far. This also tends to have the cost of hurting readability as it is often the case you would prefer putting data somewhere else in the file (i.e. sed). The nature of flat configuration files where each application has its own format is such where recovery and/or applying changes only if they have not been already applied is too complex and hurts readability far to much to be attempted in a simple shell script. Many applications have their own command line driven configuration utilities. However each has their own oddities and complexities (authtool, gconftool-2, postconf etc...) Oddly enough out of all of the command line configuration tools I have used, I find gconftool-2 to be closest to The Right Way (TM), which is odd as it is by far one of the most complex syntax's to use (God help anyone who tries to make major edits to gnome xml files using sed) example: "gconftool-2 --direct --config-source xml:readwrite:/etc/gconf/gconf.xml.mandator y --type float --set /desktop/gnome/peripherals/mouse/motion_acceleration 1" Even a complex (yet powerful) tool can meet the golden rule of "Make it as simple as possible but no simpler" when compared against a bunch of tools that are less powerful (and perhaps less complex when viewed in isolation). It is my opinion that we as a community need to find a Better Way (tm) to manage configuration changes for Linux subsystems and applications. Everything should be a file, but that does not mean every configuration file needs to be formatted and managed differently. In closing, a change this fundamental must be designed by the best minds and must have a strong champion for it to have a chance to succeed. The Fedora community is likely the only Linux entity with the brain power and influence to address this issue. Cheers, Shane

Basically, the problem arises in trying to move the inertia of 50 upstream applications to move to a "new, better" format versus the pain of trying to consistently maintain such patches for 50 applications outside of upstream.

GUI users don't want to be hunting through text files anyway, they want nice settings windows and wizards. Anyone hand editing config files better know what's going on anyway; the current situation isn't too bad there.

gconf already provides a reasonable way to change settings from the command line and via GUI tools. What would you change?

No matter what you come up with though, it will be many years before you see wide spread adoption. If anything, you might consider a project to create a system-wide config editor that knows all the different formats etc and provides a consistent CLI/GUI interface.

Dnia 27-03-2006, pon o godzinie 13:08 -0800, Shane Stixrud napisaÿÿ(a): >> No matter what you come up with though, it will be many years before >> you see wide spread adoption. If anything, you might consider a >> project to create a system-wide config editor that knows all >> the different formats etc and provides a consistent CLI/GUI >> interface. > Projects already exist http://www.libelektra.org/Main_Page for example. No, this concentrates on creating a single API for many programs to store their configs. Parent meant something understanding currently existent configuration formats. Something like that exists, too: http://www.solucorp.qc.ca/linuxconf/

On Mon, 27 Mar 2006 13:33:17 -0800 (PST) Shane Stixrud <shane(a)geeklords.org&gt; wrote: > The above project has the concept of "backends" that can be made to > understand specific configuration formats. There is an example where > Xorg.conf can be imported and exported on demand. > > http://www.libelektra.org/Backends > If you think this package is going to be the long term answer, why not start by packaging it up for extras?

> This is not a gui issue, nor is it just an "end user issue". This > attitude of "anyone hand editing config files better know what's going on > anyways" becomes largely invalid when a standard methodology exists. I actually don't buy that. You don't change anything when you go to a standard config file format. Anyone editing at the file/gconf/ registry level had better know what the heck they're doing.

Everyone else wants a nice GUI that is logically consistent with the problem space they're interested in and provides wizards etc to explain the configuration process.

> Projects already exist http://www.libelektra.org/Main_Page for example. > The problem is not that code doesn't exist, the problem is one of getting But gconf and gconf-2 and others existed before the package you cite. Perhaps this one is the end all and be all of config backends. Dunno.

Well the line is somewhere around the area of those who can look at a config file and figure it out in a minute or two, and those who can't. Anyone who can't deal with today's config files, isn't going to be any better off with a nice gui version of that same process.

I would be insulted if I didn't know you were just making a generalized statement :). Who here hasn't spent more than 2 minutes trying to figure a missing period in a named conf file, or more than two minutes setting up ldap for the first time, or more than two minutes figuring out where to find a specific option and value for /etc/modules.conf etc... The problem is each time you touch something you haven't touched in the past one must spend significant time figuring out how to make the change even if they already know what they want to change. This is not true for all configuration files, but it is for many. The amount of neurons I have dedicated to configuration syntax and where the lists of values and their descriptions are stored is many more than I would like.

The "non-initiated" see this as complex and on this specific issue I agree, it is more complex than it needs to be.. I am just used to it and shrug it off. If for every "key": its default, possible values (on, off string, float etc..) is readily accessible and for every key there is a simple description much of this complexity is eliminated.

On Mon, 27 Mar 2006, sean wrote: > GUI users don't want to be hunting through text files anyway, they > want nice settings windows and wizards. Anyone hand editing config > files better know what's going on anyway; the current situation isn't too > bad there. This is not a gui issue, nor is it just an "end user issue". This attitude of "anyone hand editing config files better know what's going on anyways" becomes largely invalid when a standard methodology exists. > > gconf already provides a reasonable way to change settings from the > command line and via GUI tools. What would you change? Gconf shows the gnome people realized early on having a standard method for storing and modifying configuration data is important, to the gnome platform... We are not talking about JUST the gnome platform and my guess is gconf would not meet the needs of Fedora as a whole. > > No matter what you come up with though, it will be many years before > you see wide spread adoption. If anything, you might consider a > project to create a system-wide config editor that knows all > the different formats etc and provides a consistent CLI/GUI > interface. Projects already exist http://www.libelektra.org/Main_Page for example. The problem is not that code doesn't exist, the problem is one of getting everyone to: a) agree its desirable b) Agreeing/creating an implementation c) having a plan for getting where we want to be "many years" later. I don't see how a few people can make this happen, it is going to take some serious influence to make any real world progress. -- fedora-devel-list mailing list fedora-devel-list(a)redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list

I find it hard to believe no one else sees the current complexity of system/application configuration as undesirable?

Is this problem too big to be addressed? Is there too much historical precedence involved? Or is there a solid technical reason for not having a standardized configuration file format?

1) I could create a custom RPM that had all of the changes imbedded in config files and rpm scripts that merely overwrote the pre-existing ones. Problem being this approach hides the complexity of what all was being changed and why. 2) Use cfengine after the kickstart install. This of course has some advantages over the rpm method but it also hides the same complexity and adds some of its own. 3) Document on paper all of the steps required and how to perform them from the console, taking advantage of the guis when available or command line when required. I felt that was not only a big waste of time but left far too much to human error. 4) Create a series of simple scripts using basic OS applications/tools (sed, echo, cp, mv, authtool, postconf etc...) to perform all of the required modifications, documenting what and why along the way.

My conclusion having completed this process is that the number of variables, complexity and dependencies that must be understood and taken into account when modifying configuration files is much higher than it needs to be. I think this should be clear when reading the example scripts mentioned above.

It is my opinion that we as a community need to find a Better Way (tm) to manage configuration changes for Linux subsystems and applications.

Everything should be a file, but that does not mean every configuration file needs to be formatted and managed differently.

But what is the advantage of building an entire new config language around these wrappers? Couldn't they be manipulated just as easily with shell or python? Can the administrator make simple ad-hoc command line config file changes?

> I have been using another config mgmt system, puppet, > (http://reductivelabs.com/projects/puppet) I wrote up an example of how > a config mgmt tool could help in solving the kind of problem you are > looking at: http://people.redhat.com/dlutter/puppet-app.html Puppet looks like a very promising centralized change distribution system. As you mentioned this separates the "how" changes are applied from the "what" is actually changing. If you combine the change re-visioning concept mentioned above with a puppet like centralized distribution system that can detect client changes we now have something quite impressive.

In fact it resembles very much what I have with my Cisco equipment + management software.

A lighter-weight approach seems more promising: encapsulate most of the config-file specific knowledge in simple script wrappers that can be controlled by a declarative description of the configuration you want to achieve and the logical interdependencies between them. This is what puppet does, and why I find it very attractive.

Hummm, never heard about puppet. Seems interesting. The problem I see with this approach is the additional layer that manages the configuration files syntax. And again: ALL configuration files can be represented by an hierarchy of key/value pairs. They are different because they received a considerable amount of syntax fat to make them look nicer to your human eyes. And what puppet seems to do is try to work with this fat.

Hummm, never heard about puppet. Seems interesting. The problem I see with this approach is the additional layer that manages the configuration files syntax. And again: ALL configuration files can be represented by an hierarchy of key/value pairs. They are different because they received a considerable amount of syntax fat to make them look nicer to your human eyes. And what puppet seems to do is try to work with this fat. Elektra completely removes that fat and reduces the configuration parameters to what they really are: key/value pairs.