[devel] leaner services, cobbler_api_rw becomes cobbler_api, less ports required to open up
by Michael DeHaan
As part of the ongoing performance work, the following changes are
happening on the devel branch.
-- Consolidation of XMLRPC ports
Previously cobbler operated two XMLRPC endpoints:
http://server/cobbler_api_rw --> maps to 127.0.0.1:25152
http://server/cobbler_api --> maps to 127.0.0.1:25150
In newer versions, both of these endpoints will continue to work but
they will map to the same port (25150)
Both read-write and read-only operations will be able to be done on the
same port.
This means cobblerd startup involves half as much CPU and IO.
In addition, James Laska and Bill Peck's work on the upgraded Anaconda
monitoring system means cobblerd will no longer listen for UDP syslog
information and log it -- if Anaconda monitoring is desired, and
enabled, it will log /much/ more. This also eliminates a extra fork()
in cobblerd.
Cobbler also published itself via avahi if enabled, which is a feature I
don't suspect too many people were using. Koan is still avahi aware,
though if you want to publish the presence of Cobbler, cobblerd will
currently /not/ do that for you. Since cobbler sets up a profiles.d
file now that tells koan where the cobbler server that installs it is,
that's a much better way to do discovery.
All in all, cobblerd will be down from 3-4 processes to just 1, and now
requires two less API handles. Actual performance implications TBD
but it will be nice to have.
This is all in progress and should be testable soon.
If you have an existing script, continue to use the Apache-proxied http
endpoints instead of direct communication to the HTTP ports and things
will work as normal. If you are using port numbers, replace any
instance of "25151" with "25150" in your code, and things will also
continue to work as normal. If your code referenced "cobbler_api" or
25151, nothing changes, that still works as before also.
--Michael
15 years, 4 months
reposync downloads wrong package version
by Stephan Huiser
I filed a ticket in Cobbler Trac:
https://fedorahosted.org/cobbler/ticket/351
* Problem:
Cobbler reposync downloads the wrong version of packages.
* Cobbler version:
cobbler-1.4.1-1.el5
* OS-version:
CentOS release 5.2 (Final)
* Repo:
cobbler repo report epel-4-x86_64
repo : epel-4-x86_64
arch : x86_64
breed : yum
comment :
created : Sat Jan 10 10:13:07 2009
createrepo_flags : -c cache
environment : {}
keep updated : True
mirror : ftp://ftp.surfnet.nl/pub/os/Linux/distr/fedora-epel/4/x86_64
mirror locally : True
modified : Mon Jan 19 18:06:39 2009
owners : ['admin']
priority : 99
rpm list : ['bash-completion', 'nrpe']
yum options : {}
* RPM-packages on repo-mirror (important: *el4* ):
* ftp://ftp.surfnet.nl/pub/os/Linux/distr/fedora-epel/4/x86_64/nrpe-2.12-3.el4.x86_64.rpm
* ftp://ftp.surfnet.nl/pub/os/Linux/distr/fedora-epel/4/x86_64/bash-completion-20060301-4.el4.noarch.rpm
* Output 'cobbler reposync --only=epel-4-x86_64' :
- creating: /var/www/cobbler/repo_mirror/epel-4-x86_64/.origin/epel-4-x86_64.repo
- /usr/bin/yumdownloader --resolve -c /var/www/cobbler/repo_mirror/epel-4-x86_64/.origin/epel-4-x86_64.repo --destdir=/var/www/cobbler/repo_mirror/epel-4-x86_64 bash-completion nrpe
centos-5.2-x86_64-updates 100% |=========================| 951 B 00:00
epel-5-x86_64 100% |=========================| 951 B 00:00
centos-5.2-x86_64-addons 100% |=========================| 951 B 00:00
tref 100% |=========================| 951 B 00:00
core-0 100% |=========================| 1.1 kB 00:00
epel-4-x86_64 100% |=========================| 1.1 kB 00:00
epel-testing-5-x86_64 100% |=========================| 951 B 00:00
centos-5.2-x86_64-extras 100% |=========================| 951 B 00:00
--> Running transaction check
---> Package bash-completion.noarch 0:20060301-4.el5 set to be updated
---> Package nrpe.x86_64 0:2.12-3.el5 set to be updated
--> Finished Dependency Resolution
bash-completion-20060301- 100% |=========================| 144 kB 00:00
nrpe-2.12-3.el5.x86_64.rp 100% |=========================| 32 kB 00:00
- createrepo -c cache /var/www/cobbler/repo_mirror/epel-4-x86_64
2/2 - nrpe-2.12-3.el5.x86_64.rpm
Saving Primary metadata
Saving file lists metadata
Saving other metadata
- creating: /var/www/cobbler/repo_mirror/epel-4-x86_64/config.repo
* It downloads the *el5* versions of 'bash-completion' and 'nrpe' !!!
How is this possible?
* If you need more info, please let me know.
15 years, 4 months
Question regarding buildiso
by James Cammarata
Hey all, I'm working on building a few new systems, but they're in a build
lab that doesn't have access to a cobbler server. I was hoping to use
buildiso to get around this, but apparently there isn't any option to
actually copy the distros/all required files to the ISO image too, correct?
If so, I think this would be a handy option, for those situations where you
need cobbler-on-a-dvd. Perhaps a --copy-files or --standalone option? It
would probably be wise to limit this to single distros on a DVD iso only,
since more than one would be huge and I'd imagine a lot of users would run
into space issues while trying to compile that into the ISO.
Thoughts?
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
15 years, 4 months
A collection of anaconda monitor (anamon) patches
by James Laska
Greetings,
The attached patches address several fixes to the anaconda monitoring (anamon) support in cobbler. Several features have also been added, including:
* Adding post_anamon snippet to standard kickstart templates (sample.ks and sample_end.ks)
* Adding a anamon.init used to provide post-install boot notification
Comments welcome.
Thanks,
James
15 years, 4 months
[PATCH] buildiso network support doesn't work if you use bonding
by Dave Hatton
cobbler buildiso assumed that eth0 was the only place to get network
information from. If you use bonding this isn't true. Here is the fix.
Also noticed gateway wasn't being set so fixed that as well.
Daveh
PS - can you push this into 1.4 as well as devel as I use this a lot?
>From e10986fe1fbeef40b30fd2b012ecd995952fcc6a Mon Sep 17 00:00:00 2001
From: Dave Hatton <dave(a)starground.it>
Date: Thu, 22 Jan 2009 17:41:15 +0000
Subject: [PATCH] Fix network configuration broken by addation of bonding support
---
cobbler/action_buildiso.py | 10 ++++++++--
1 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/cobbler/action_buildiso.py b/cobbler/action_buildiso.py
index a6fa735..8fe7470 100644
--- a/cobbler/action_buildiso.py
+++ b/cobbler/action_buildiso.py
@@ -236,12 +236,18 @@ class BuildIso:
# add network info to avoid DHCP only if it is available
+ if data.has_key("ip_address_bond0") and data["ip_address_bond0"] != "":
+ append_line = append_line + " ip=%s" % data["ip_address_bond0"]
if data.has_key("ip_address_eth0") and data["ip_address_eth0"] != "":
append_line = append_line + " ip=%s" % data["ip_address_eth0"]
+
+ if data.has_key("subnet_bond0") and data["subnet_bond0"] != "":
+ append_line = append_line + " netmask=%s" % data["subnet_bond0"]
if data.has_key("subnet_eth0") and data["subnet_eth0"] != "":
append_line = append_line + " netmask=%s" % data["subnet_eth0"]
- if data.has_key("gateway_eth0") and data["gateway_eth0"] != "":
- append_line = append_line + " gateway=%s\n" % data["gateway_eth0"]
+
+ if data.has_key("gateway") and data["gateway"] != "":
+ append_line = append_line + " gateway=%s\n" % data["gateway"]
length=len(append_line)
if length > 254:
--
1.5.5.6
15 years, 4 months
koan-1.4.1-1 - replace self without DHCP/PXE
by Tom Brown
Hi
Previously using koan you could replace-self without the need for
DHCP/PXE as the networking information that grub used was present in the
grub command line. Using latest versions of koan/cobbler i now see this
to be not true, eg
title kick1232627210
root (hd0,0)
kernel /vmlinuz ro root=LABEL=/ console=tty0 nopmtimer nohpet
console=tty0,9600n8 ksdevice=eth2 lang= text
ks=http://server/cblr/svc/op/ks/system/test-host syslog=server:25150
kssendmac
initrd /initrd.img
if the networking info is not in the command line i thought it could be
in a file on the filesystem but i see the URL rather than file://
any pointers?
thanks
15 years, 4 months
cobbler and spacewalk
by Thomas von Steiger
Hi,
Are there a way to load spacewalk via cobbler comands ?
Thomas
15 years, 5 months
autodiscovery questions
by Vreman, Peter
I want to add an autodiscovery feature. I have some questions how to start:
- What kind of initrd shall be used?
o Use livecd-tools to build the initrd+kernel from Fedora/RHEL/CentOS?
* Minimum size is 150MB
* Easy to maintain and regenerate
o Create a small busy-box initrd+kernel
* Minimum size is 20MB
* Harder to understand
* Requires more indepth linux knowledge
- Which API to use?
o HTTP services?
o XMLRPC read/write API? (listens currently only on localhost)
- Instead of creating something new extend koan with a new --discover function?
Regards,
Peter
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.
15 years, 5 months
[devel] cobblerd performance (this works!)
by Michael DeHaan
New simpler plan, for which I have a partial implementation already.
Have cobblerd keep a cache of all objects that it loads on startup.
When an edit is made in the API, the API will signal cobblerd to update
it's cache for just that object. The result is that the API only has to
reload that /one/ config file, not all of them.
The API will also signal deletes.
Old behavior with 10k systems:
-- slow cobblerd startup (60 seconds)
-- slow CLI startup (60 seconds)
-- fast inserts
-- very slow webapp and XMLRPC
New behavior
-- slow cobblerd startup
-- slow CLI startup
-- slightly slower inserts due to XMLRPC signaling, most users should
not notice (maybe 2x slower, they were fast enough before)
-- very fast webapp and XMLRPC as if it didn't matter how many systems
were in it (win!)
The result is that a full reload of all data in cobblerd is therefore
/never/ done and everything is zippy, if a bit of a minor RAM hog. It
also makes 'inserts' marginally slower, but I'm mainly concerned with
API load time.
I also note that there is no reason to have a seperate fork for cobblerd
(read-only) and cobblerd (read-write), as you should be able to use the
public methods against the RW XMLRPC instance just fine. This saves us
a process and will be step two. I suspect this might even cut our
startup time down some as they won't be trying to access the filesystem
at the same time (even though they are loading in parallel).
Step three is that with a very large number of systems, opening up a API
handle for the first time is still going to be expensive. The answer
here is probably do a cobbler shell that pays for the API penalty once
and then lets you send commands rapidly as you see fit. I think this
would be easy to do.
All of this can be done without changes to the serializers, and doesn't
require anything as intrusive as inserting a database. Some of this is
working now on my machine and work will be checked in to the devel
branch as it goes along.
I can also work on deferring the cobblerd startup penalty to after the
service is started so it can be backgrounded.
Bottom line: we have a /simple/ solution that fits requirements, and
I'm happy.
This is also tunable so we don't have to stop here.
Largely I expect folks with 10k+ systems to being using cobbler as a
service, not a command line app, so I intend to approach scaling
problems in the context of making sure cobblerd (as an example client)
and koan remain performant. The CLI is a lesser priority.
These are good problems to have as they mean more people are interested
in using cobbler in larger configs -- I would rather have this
challenges than not have them, and this is looking like we are moving in
a very good direction.
--Michael
15 years, 5 months
reposync error - 1.4.1
by Tom Brown
Hi
Getting the following error when trying to sync a repo into cobbler. Is
this something obvious?
thanks
# cobbler version
cobbler 1.4.1
# cobbler reposync
- creating:
/var/www/cobbler/repo_mirror/software-5-x86_64/.origin/software-5-x86_64.repo
- /usr/bin/reposync -l
--config=/var/www/cobbler/repo_mirror/software-5-x86_64/.origin/software-5-x86_64.repo
--repoid=software-5-x86_64 --download_path=/var/www/cobbler/repo_mirror
-a x86_64
usage:
Reposync is used to synchronize a remote yum repository to a local
directory using yum to retrieve the packages.
/usr/bin/reposync [options]
reposync: error: no such option: -l
Traceback (most recent call last):
File "/usr/lib/python2.3/site-packages/cobbler/action_reposync.py",
line 105, in run
self.sync(repo)
File "/usr/lib/python2.3/site-packages/cobbler/action_reposync.py",
line 136, in sync
return self.yum_sync(repo)
File "/usr/lib/python2.3/site-packages/cobbler/action_reposync.py",
line 346, in yum_sync
raise CX(_("cobbler reposync failed"))
CX: 'cobbler reposync failed'
- reposync failed, tries left: 0
reposync failed, retry limit reached, aborting
15 years, 5 months
