Re: [fedmsg] Proposal on a replay mechanism
On Tue, Jul 09, 2013 at 01:35:59PM +0200, Simon Chopin wrote:
Quoting Ralph Bean (2013-07-09 03:08:24)
> On Mon, Jul 08, 2013 at 03:39:56PM +0200, Simon Chopin wrote:
.. <snip> ..
> One problem I see is in the implementation details. How long is
an
> endpoint expected to hold on to its old messages before discarding
> them? Whereas currently, an application that gets a fedmsg hook added
> doesn't retain much extra state as a result, this replay-request
> proposal would require a book keeping mechanism added to every
> endpoint (in our case, every mod_wsgi/httpd process, others).
Not every endpoint. For instance, I don't expect your wiki endpoints to
provide such mechanism, as the systems depending on it are not critical
(unless I'm missing something). For those that are critical, well, I'd
say it is the price to pay.
Cool. If we can keep it heterogeneous like this, then Fedora
Infrastructure can keep our endpoints as they are. What do you think
about designing this as a persistence plugin system, so the Debian
crew could implement a schema that meets your needs, while Fedora can
do something else or nothing in this respect.
As for how long, IMO there is no single answer to the question. Each
service should have its own policy. In my initial implementation I plan
to store the messages using sqlalchemy, without time limit, but I'm open
to suggestions on how to handle this differently.
I understand. Indefinitely is an option. At first I was thinking
primarily of doing this in memory (instead of on disk in a database),
but with a sql db for each publisher, this can work.
Have you thought much yet about how the API should be changed if
anywhere?
fedmsg.publish will need to record outgoing messages. That seems
pretty simple; no API changes needed.
Should fedmsg.tail_messages automatically detect outages and handle
sending the replay request for the user? Or should the user be
responsible for detecting the outage and making the request
themselves? Same question goes for hub daemons and their consumers.
-Ralph
Attachments:
- attachment.sig (application/pgp-signature — 490 bytes)