4:28 p.m.
Hi folks,
Recently I wrote a Twisted-based chat bot that can interact with a Koji hub.
To do this, I wrote a "txkoji" client library:
https://github.com/ktdreyer/txkoji
(the bot code is separate, this is just the koji client library)
txkoji allows me to interact with a Koji Hub in an event-driven way,
without blocking the reactor for my program. This works well for
client programs that must be responsive to input from several sources
(like IRC and a message bus and many other inputs). I return the
XML-RPC responses as Munch (dict-like) objects because I ran across
Munch in a couple other Fedora infra projects and I like the syntax.
If Koji gets a REST API, I could convert this to use treq instead of
Twisted's XML-RPC client.
- Ken
8:37 p.m.
Neat!
Note that an authenticated koji session will probably not play nice with
async use because of the callnum data.
I've never used twisted. I presume its xmlrpc lib handles the nil extension
(otherwise you would almost certainly have seen lots of errors). I wonder
if it can handle an i8 tag, which kojihub will now emit for large integers
in some contexts.
Koji's faults can have many different fault codes, though yes 1000 is the
most likely. The full list in near the beginning of koji/__init__.py.
On Tue, Dec 12, 2017 at 5:28 PM, Ken Dreyer <ktdreyer(a)ktdreyer.com> wrote:
Hi folks,
Recently I wrote a Twisted-based chat bot that can interact with a Koji
hub.
To do this, I wrote a "txkoji" client library:
https://github.com/ktdreyer/txkoji
(the bot code is separate, this is just the koji client library)
txkoji allows me to interact with a Koji Hub in an event-driven way,
without blocking the reactor for my program. This works well for
client programs that must be responsive to input from several sources
(like IRC and a message bus and many other inputs). I return the
XML-RPC responses as Munch (dict-like) objects because I ran across
Munch in a couple other Fedora infra projects and I like the syntax.
If Koji gets a REST API, I could convert this to use treq instead of
Twisted's XML-RPC client.
- Ken
_______________________________________________
koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
12:30 p.m.
On Tue, Dec 12, 2017 at 7:37 PM, Michael McLean <mikem(a)redhat.com> wrote:
Neat!
Note that an authenticated koji session will probably not play nice with
async use because of the callnum data.
Thanks, I see the callnum code in Koji now. What is the purpose of the
callnum? Is it a security feature, or just a way to maintain write
operation order?
I've never used twisted. I presume its xmlrpc lib handles the nil
extension
(otherwise you would almost certainly have seen lots of errors). I wonder if
it can handle an i8 tag, which kojihub will now emit for large integers in
some contexts.
Koji's faults can have many different fault codes, though yes 1000 is the
most likely. The full list in near the beginning of koji/__init__.py.
Thanks Mike for all these pointers. I checked Twisted and it uses
xmlrpclib.loads() to parse the responses. I tested <nil/> and it
translates fine to None, and the Koji git log indicates Python's
xmlrpclib will read <i8> too. I can't find a live Koji instance where
we've overflowed the 32-bit int yet to verify, though!
- Ken
12:58 p.m.
On Wed, Dec 13, 2017 at 11:30 AM, Ken Dreyer <ktdreyer(a)ktdreyer.com> wrote:
I can't find a live Koji instance where
we've overflowed the 32-bit int yet to verify, though!
Now I'm confused. https://pagure.io/koji/issue/464 mentions the
webkitgtk4-debuginfo RPMs, but when I check one with getRPM(10230453),
the size is a string, not i8:
$ cat xmlrpc
<?xml version="1.0"?>
<methodCall>
<methodName>getRPM</methodName>
<params>
<param>
<value>
<int>10230453</int>
</value>
</param>
</params>
</methodCall>
$ curl -d @xmlrpc https://koji.fedoraproject.org/kojihub | tidy -q -xml -i -
...
<member>
<name>size</name>
<value><string>2753783486</string></value>
</member>
...
- Ken
1:55 p.m.
So, over the years we addressed the int overflow problem in different ways.
Initially, we had some targeted code that would convert integers over the
limit into strings. The was done only in a few places that were known to
return large ints.
The most recent case of overflow was a bit messier, and we went with a more
systematic approach. We also discovered the i8 tag, which was had not
previously known about (and may not have existed when we first implemented
the earlier solution).
When we started using the i8 tags, it was late in our release cycle, and we
didn't want to break
existing code that might rely on those values being strings. So, we opted
to leave those older parts alone for the time being.
If you want a quick test, the echo call will give you whatever data you
pass in.
$ koji --noauth call echo 9223372036854775807
[9223372036854775807]
On Wed, Dec 13, 2017 at 1:58 PM, Ken Dreyer <ktdreyer(a)ktdreyer.com> wrote:
On Wed, Dec 13, 2017 at 11:30 AM, Ken Dreyer
<ktdreyer(a)ktdreyer.com>
wrote:
I can't find a live Koji instance where
> we've overflowed the 32-bit int yet to verify, though!
Now I'm confused. https://pagure.io/koji/issue/464 mentions the
webkitgtk4-debuginfo RPMs, but when I check one with getRPM(10230453),
the size is a string, not i8:
$ cat xmlrpc
<?xml version="1.0"?>
<methodCall>
<methodName>getRPM</methodName>
<params>
<param>
<value>
<int>10230453</int>
</value>
</param>
</params>
</methodCall>
$ curl -d @xmlrpc https://koji.fedoraproject.org/kojihub | tidy -q -xml
-i -
...
<member>
<name>size</name>
<value><string>2753783486</string></value>
</member>
...
- Ken
_______________________________________________
koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
3:55 p.m.
On Wed, Dec 13, 2017 at 12:55 PM, Michael McLean <mikem(a)redhat.com> wrote:
If you want a quick test, the echo call will give you whatever data
you pass
in.
$ koji --noauth call echo 9223372036854775807
[9223372036854775807]
Thanks Mike!
That echo call definitely breaks my client (OverflowError, "int
exceeds XML-RPC limits" from xmlrpclib). I added custom Marshaller
overrides to deal with this.
- Ken
4:23 p.m.
Odds are it's the encoding part that's breaking. You might also check that
you can encode None.
On Wed, Dec 13, 2017 at 4:55 PM, Ken Dreyer <ktdreyer(a)ktdreyer.com> wrote:
On Wed, Dec 13, 2017 at 12:55 PM, Michael McLean
<mikem(a)redhat.com> wrote:
> If you want a quick test, the echo call will give you whatever data you
pass
> in.
>
> $ koji --noauth call echo 9223372036854775807
> [9223372036854775807]
Thanks Mike!
That echo call definitely breaks my client (OverflowError, "int
exceeds XML-RPC limits" from xmlrpclib). I added custom Marshaller
overrides to deal with this.
- Ken
_______________________________________________
koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
5:20 p.m.
On Wed, Dec 13, 2017 at 3:23 PM, Michael McLean <mikem(a)redhat.com> wrote:
Odds are it's the encoding part that's breaking. You might
also check that
you can encode None.
To clarify: with my latest commit to txkoji to add the custom
Marshaller, that echo works now!
It gave me some ideas for optimizing koji/xmlrpcplus.py ,
https://pagure.io/koji/pull-request/751
- Ken
2338
days inactive
2339
days old
koji-devel@lists.fedorahosted.org
7 comments
2 participants
participants (2)
-
Ken Dreyer -
Michael McLean