Johnston, Christopher (GAE) wrote:
Does cobbler have the ability to throttle the number of servers
being
built at a single time? I know I can put throttling into yum and into
apache to prevent X number of data to be transferred, but curious if
there has been some thought on doing this for cobbler.
I have a system today that is all home grown built where we sometimes
stage around 10-50 builds off at a time (we have A LOT of blades and
grid nodes). We are interested in looking at cobbler for the future and
this would be a feature that would be of interest.
It depends on how you are provisioning the machines and where you're
seeing load challenges, if any. I really haven't heard of any load
concerns so far that are cobbler specific as we are essentially relying
on Apache and TFTP server to do the file transfers -- cobblerd is there
to generate kickstart files, do some logging, and execute triggers. I
would be /very/ interested in load testing to see how to best configure
things in environments that do multiple reinstalls at the same time if
they are being stressed. Ultimately, most of that stress would go on
Apache, and it's easy enough to teach the kickstart generators how to
distribute load between different web servers (if needed) and to tell
the TFTP config to pick different cobbler servers (if needed). I'll get
to that in a bit...
The closest example I have at present is a Cobbler powered cluster that
was doing 300 machine reinstalls at once, and seemed to be doing ok. At
the time (about a year ago), cobblerd was being a bit slow, but we've
since replaced all of the CGI code with mod_python, and for large
installs, we also have seralizer_catalog in 1.1 (a storage backend) that
is much zippier than serializer_yaml (and it is backwards compatible)
So, about how to distribute things between multiple cobbler servers if
you want to do that ... For clean installs of bare metal, basically TFTP
is going to serve up systems when they are requested. If optimizations
are needed, we do have the "--server-override" flag which can point
certain hardware at different cobbler servers, and "cobbler replicate"
which can be used to construct a mirror of a cobbler server.
Server-override could be used so that installs of the "webserver"
profile hit "cobbler2.example.org" and another profile hits
"cobbler.example.org", and "cobbler replicate" could be used to mirror
the data between the two on a crontab.
Another piece of advice might be to use "cobbler import" with
"--available-as" as opposed to mirroring into /var, such that you can
leave your data on a SAN/NFS share, to be shared by the multiple cobbler
servers (or otherwise bind mount /var/www/cobbler on the same SAN/NFS
share). This solves the need to replicate data between multiple cobbler
servers -- you can just use replicate to replicate the metadata.
Another thing you can do is if you are using something like koan, you
could reinstall the machines in staggered batches. This really isn't
possible with just PXE, though in a PXE setup, you could control it by
making the profile/system assignments in cobbler and then cycling power
on a staggered basis.
I would love to collect user cases on the Wiki of ways people are
building cobbler out for really aggressive reinstall type situations.
That would be great to have.
I'd love to hear more details as this is an interesting problem space. I
think we've covered by any of the above, but if not, I'm sure we can
work out something more specific to your environment. Let me know if
that works!
--Michael
Christopher Johnston
Linux Engineering
(201) 499-4839
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
This message is intended only for the personal and confidential use of the designated
recipient(s) named above. If you are not the intended recipient of this message you are
hereby notified that any review, dissemination, distribution or copying of this message is
strictly prohibited. This communication is for information purposes only and should not
be regarded as an offer to sell or as a solicitation of an offer to buy any financial
product, an official confirmation of any transaction, or as an official statement of
Lehman Brothers. Email transmission cannot be guaranteed to be secure or error-free.
Therefore, we do not represent that this information is complete or accurate and it should
not be relied upon as such. All information is subject to change without notice.
--------
IRS Circular 230 Disclosure:
Please be advised that any discussion of U.S. tax matters contained within this
communication (including any attachments) is not intended or written to be used and cannot
be used for the purpose of (i) avoiding U.S. tax related penalties or (ii) promoting,
marketing or recommending to another party any transaction or matter addressed herein.
_______________________________________________
cobbler mailing list
cobbler(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/cobbler