2:26 p.m.
I'm trying to get plague set up and I'm running into compatibility
problems that lead me to think I'm starting in the wrong place.
I picked FC7 and sqlite to start with for plague's home OS and a
non-DB-server DB option. The production system will be RHEL4/5 once
I'm comfortable I know how to get things running smoothly. I tried
briefly with RHEL4 a few months back and didn't get this far.
I started with a fresh install of FC7 (incl. python-2.5,
python-sqlite2) and plague 0.4.4.1-4.fc7 RPMs. After running into
some of the first issues I switched to CVS MAIN with similar results.
Here's what I've done so far:
* server/DBManager.py
- import sqlite3 as sqlite
- remove encoding="utf-8" from sqlite.connect
* server/main.py
* server/BuildMaster.py
- sqlite hack: move dbm connection into BuildMaster so
it can create one for each thread.
* common/XMLRPCServerProxy.py
- set self._use_datetime = 0
* common/AuthedXMLRPCServer.py
- pass allow_none=False and encoding=None to
SimpleXMLRPCServer.SimpleXMLRPCDispatcher's
* builder/Config.py
server/Config.py
- change BaseConfig.BaseConfig.ConfigError to
BaseConfig.ConfigError
I have the builder and server on the same host, w/o SSL yet. With the
0.4.4.1 RPMs and some of the changes above the server was able to talk
to the builder but I ran into the sqlite problem and switched to CVS
at that point.
CVS has the new Active/Passive builders but the default port configs
don't appear to be correct for having the builder and server on the
same machine, but I can't match up the config sections and field names
in the server and builder configs to point them at each other
correctly. At this point, the server's not seeing any builders. I
tried running 'plague-client list' anyway and get an exception "local
variable 'curs' referenced before assignment" which I'm pretty sure is
caused by my sqlite thread hack.
What OS+versions should I use to get plague up and running most
quickly?
Once I'm up and running in a known-working environment I'd have a
better idea of what changes are right for FC7 or RHEL4/5.
Thanks,
-- Ken
5:06 a.m.
Hi Ken!
Am 2007-02-25 21:26, Ken MacLeod schrieb:
[ ... ]
Here's what I've done so far:
* server/DBManager.py
- import sqlite3 as sqlite
- remove encoding="utf-8" from sqlite.connect
* server/main.py
* server/BuildMaster.py
- sqlite hack: move dbm connection into BuildMaster so
it can create one for each thread.
* common/XMLRPCServerProxy.py
- set self._use_datetime = 0
Why that?
* common/AuthedXMLRPCServer.py
- pass allow_none=False and encoding=None to
SimpleXMLRPCServer.SimpleXMLRPCDispatcher's
Why that?
* builder/Config.py
server/Config.py
- change BaseConfig.BaseConfig.ConfigError to
BaseConfig.ConfigError
Yes. I already submitted a patch to Dan for this.
I have the builder and server on the same host, w/o SSL yet. With
the
0.4.4.1 RPMs and some of the changes above the server was able to talk
to the builder but I ran into the sqlite problem and switched to CVS
at that point.
So you' running 0.5.0 now. The same as I do. Right?
CVS has the new Active/Passive builders but the default port configs
don't appear to be correct for having the builder and server on the
same machine, but I can't match up the config sections and field names
in the server and builder configs to point them at each other
correctly. At this point, the server's not seeing any builders. I
tried running 'plague-client list' anyway and get an exception "local
variable 'curs' referenced before assignment" which I'm pretty sure is
caused by my sqlite thread hack.
What OS+versions should I use to get plague up and running most
quickly?
I have plg 0.5.0. mysql 4.1.20 running on ES4. But I also have an
instance running on FC6.
Once I'm up and running in a known-working environment I'd
have a
better idea of what changes are right for FC7 or RHEL4/5.
Example config:
builder.cfg:
====================================
[General]
comm_type = active
builder_user = plague-builder
hostname = somehost.somecompany.com
server = somehost.somecompany.com
debug = True
max_jobs = 3
builder_cmd = /usr/bin/mock
[Directories]
target_configs_dir = /etc/plague/builder/targets
builder_work_dir = /tmp/builder_work
[SSL]
use_ssl = False
[Active]
xmlrpc_port = 8889
fileserver_port= 8890
server.cfg:
====================================
[Database]
engine = mysql
[Directories]
repo_dir = /repodir
mock_configs_dir = /etc/mock
server_work_dir = /rpmbuild
target_configs_dir = /etc/plague/server/targets
tmpdir = /tmp
[Active Builders]
xmlrpc_server_port = 8889
file_server_port = 8890
builder1 = 0 somehost2.somecompany.com
builder2 = 50 somehost.somecompany.com
[General]
traceback_server = no
hostname = somehost.somecompany.com
depsolve_jobs = no
Debug = True
allow_reenqueue = no
[SSL]
server_key_and_cert =
/etc/plague/server/certs/server/server_key_and_cert.pem
ca_cert = /etc/plague/server/certs/ca/my_ca_ca_cert.pem
[UI]
log_url = http://somehost.somecompany.com/buildlogs/
guest_allowed = no
port = 8887
use_ssl = yes
client_ca_cert = /etc/plague/server/certs/ca/client_ca_ca_cert.pem
[mysql Engine]
timeout = 3
host = localhost
user = root
password =
database = plague
[CVS]
use_cvs = yes
[Builders]
use_ssl = no
[Email]
success_emails =
email_from = buildsys(a)somecompany.com
admin_emails = of(a)redhat.at
If you need help creating the ssl certificates or anything else, just
mail me. :-) And don't use sqlite as database backend; It often runs
into locking problems; That's why I switched to mysql. However, you do
need to create the tables in the mysql database first. I have the
statements ready if you need...
Best,
Oliver
6:39 a.m.
On Mon, 2007-02-26 at 12:06 +0100, Oliver Falk wrote:
Hi Ken!
Am 2007-02-25 21:26, Ken MacLeod schrieb:
[ ... ]
> Here's what I've done so far:
>
> * server/DBManager.py
> - import sqlite3 as sqlite
> - remove encoding="utf-8" from sqlite.connect
Why these?
> * server/main.py
> * server/BuildMaster.py
> - sqlite hack: move dbm connection into BuildMaster so
> it can create one for each thread.
sqlite sucks for threading, it's best to just setup a postgresql or
mysql server.
> * common/XMLRPCServerProxy.py
> - set self._use_datetime = 0
Why that?
Probably due to Python 2.5.
> * common/AuthedXMLRPCServer.py
> - pass allow_none=False and encoding=None to
> SimpleXMLRPCServer.SimpleXMLRPCDispatcher's
Why that?
> * builder/Config.py
> server/Config.py
> - change BaseConfig.BaseConfig.ConfigError to
> BaseConfig.ConfigError
Yes. I already submitted a patch to Dan for this.
Hmm, I thought I'd committed that. Will need to look again.
> I have the builder and server on the same host, w/o SSL yet.
With the
> 0.4.4.1 RPMs and some of the changes above the server was able to talk
> to the builder but I ran into the sqlite problem and switched to CVS
> at that point.
So you' running 0.5.0 now. The same as I do. Right?
> CVS has the new Active/Passive builders but the default port configs
> don't appear to be correct for having the builder and server on the
> same machine, but I can't match up the config sections and field names
> in the server and builder configs to point them at each other
> correctly. At this point, the server's not seeing any builders. I
> tried running 'plague-client list' anyway and get an exception "local
> variable 'curs' referenced before assignment" which I'm pretty sure
is
> caused by my sqlite thread hack.
>
> What OS+versions should I use to get plague up and running most
> quickly?
I have plg 0.5.0. mysql 4.1.20 running on ES4. But I also have an
instance running on FC6.
> Once I'm up and running in a known-working environment I'd have a
> better idea of what changes are right for FC7 or RHEL4/5.
Example config:
builder.cfg:
====================================
[General]
comm_type = active
builder_user = plague-builder
hostname = somehost.somecompany.com
server = somehost.somecompany.com
debug = True
max_jobs = 3
builder_cmd = /usr/bin/mock
[Directories]
target_configs_dir = /etc/plague/builder/targets
builder_work_dir = /tmp/builder_work
[SSL]
use_ssl = False
[Active]
xmlrpc_port = 8889
fileserver_port= 8890
server.cfg:
====================================
[Database]
engine = mysql
[Directories]
repo_dir = /repodir
mock_configs_dir = /etc/mock
server_work_dir = /rpmbuild
target_configs_dir = /etc/plague/server/targets
tmpdir = /tmp
[Active Builders]
xmlrpc_server_port = 8889
file_server_port = 8890
builder1 = 0 somehost2.somecompany.com
builder2 = 50 somehost.somecompany.com
[General]
traceback_server = no
hostname = somehost.somecompany.com
depsolve_jobs = no
Debug = True
allow_reenqueue = no
[SSL]
server_key_and_cert =
/etc/plague/server/certs/server/server_key_and_cert.pem
ca_cert = /etc/plague/server/certs/ca/my_ca_ca_cert.pem
[UI]
log_url = http://somehost.somecompany.com/buildlogs/
guest_allowed = no
port = 8887
use_ssl = yes
client_ca_cert = /etc/plague/server/certs/ca/client_ca_ca_cert.pem
[mysql Engine]
timeout = 3
host = localhost
user = root
password =
database = plague
[CVS]
use_cvs = yes
[Builders]
use_ssl = no
[Email]
success_emails =
email_from = buildsys(a)somecompany.com
admin_emails = of(a)redhat.at
If you need help creating the ssl certificates or anything else, just
mail me. :-) And don't use sqlite as database backend; It often runs
into locking problems; That's why I switched to mysql. However, you do
need to create the tables in the mysql database first. I have the
statements ready if you need...
Yeah, using a real database is the way to go.
Dan
6:53 a.m.
Am 2007-02-26 13:39, Dan Williams schrieb:
On Mon, 2007-02-26 at 12:06 +0100, Oliver Falk wrote:
[ ... ]
Yeah, using a real database is the way to go.
Maybe someone should also have a look at the default ports and other
defaults...
BUT... What's about plague? Will it live or will it die? I'm thinking
about koji... :-)
Best,
Oliver
10:48 a.m.
On Mon, 2007-02-26 at 13:53 +0100, Oliver Falk wrote:
Am 2007-02-26 13:39, Dan Williams schrieb:
> On Mon, 2007-02-26 at 12:06 +0100, Oliver Falk wrote:
[ ... ]
> Yeah, using a real database is the way to go.
Maybe someone should also have a look at the default ports and other
defaults...
BUT... What's about plague? Will it live or will it die? I'm thinking
about koji... :-)
I'm not entirely sure. I had thought the course of action was (a)
opening up most of Brew, and (b) grafting in the bits of plague that
were useful.
Dan
2:21 a.m.
Am 2007-02-26 17:48, Dan Williams schrieb:
On Mon, 2007-02-26 at 13:53 +0100, Oliver Falk wrote:
> Am 2007-02-26 13:39, Dan Williams schrieb:
>> On Mon, 2007-02-26 at 12:06 +0100, Oliver Falk wrote:
> [ ... ]
>> Yeah, using a real database is the way to go.
> Maybe someone should also have a look at the default ports and other
> defaults...
>
> BUT... What's about plague? Will it live or will it die? I'm thinking
> about koji... :-)
I'm not entirely sure. I had thought the course of action was (a)
opening up most of Brew, and (b) grafting in the bits of plague that
were useful.
OK. So it's better to stick with plague until some working >release< of
koji is there - at least for me this is true :-) Let's see if it will be
possible to run it without all the Kerberos stuff...
-of
10:18 p.m.
Dan Williams <dcbw(a)redhat.com> writes:
On Mon, 2007-02-26 at 12:06 +0100, Oliver Falk wrote:
> Am 2007-02-25 21:26, Ken MacLeod schrieb:
> > Here's what I've done so far:
> >
> > * server/DBManager.py
> > - import sqlite3 as sqlite
> > - remove encoding="utf-8" from sqlite.connect
Why these?
I'm guessing Python 2.5 too. If RHEL4 has it different I should be
able to work out a patch that works for both.
> > * builder/Config.py
> > server/Config.py
> > - change BaseConfig.BaseConfig.ConfigError to
> > BaseConfig.ConfigError
>
> Yes. I already submitted a patch to Dan for this.
Hmm, I thought I'd committed that. Will need to look again.
I'm not seeing it in CVS MAIN.
Yeah, using a real database is the way to go.
Yup, that appears to be the case!
-- Ken
10:04 p.m.
Oliver Falk <oliver(a)linux-kernel.at> writes:
Am 2007-02-25 21:26, Ken MacLeod schrieb:
[ ... ]
> Here's what I've done so far:
> * common/XMLRPCServerProxy.py
> - set self._use_datetime = 0
Why that?
> * common/AuthedXMLRPCServer.py
> - pass allow_none=False and encoding=None to
> SimpleXMLRPCServer.SimpleXMLRPCDispatcher's
Why that?
Yes, as Dan mentioned, both due to Python 2.5.
So you' running 0.5.0 now. The same as I do. Right?
It's more accurate to say I'm running CVS MAIN, but yes ;-)
Example config:
builder.cfg:
====================================
[General]
comm_type = active
[Active]
xmlrpc_port = 8889
fileserver_port= 8890
server.cfg:
====================================
[Active Builders]
xmlrpc_server_port = 8889
file_server_port = 8890
[UI]
port = 8887
If you need help creating the ssl certificates or anything else,
just
mail me. :-) And don't use sqlite as database backend; It often runs
into locking problems; That's why I switched to mysql.
The above config (thanks!) and switching to MySQL has got the builder
and server talking to each other and queueing jobs. Not that this
changes anything, but I also backed out the sqlite changes that I
can't verify. So I still have the XMLRPC change I made and DBManager
change same as yours.
I don't remember if I created /tmp/builder_work or if the server or
builder created it, but on the job I just queued the builder gets an
error because /tmp/builder_work is owned by root and the builder can't
get access to it.
I killed the job (noted a typo in the message about not being able to
send mail, patch sent) and requeued. A couple minor glitches later...
Success!
However, you do need to create the tables in the mysql database
first. I have the statements ready if you need...
I didn't see that, the tables got created.
Going to test it out some more...
-- Ken
6275
days inactive
6279
days old
buildsys@lists.fedoraproject.org
7 comments
3 participants
participants (3)
-
Dan Williams -
Ken MacLeod -
Oliver Falk