On Tue, Jul 09, 2013 at 01:35:59PM +0200, Simon Chopin wrote:
Quoting Ralph Bean (2013-07-09 03:08:24)
On Mon, Jul 08, 2013 at 03:39:56PM +0200, Simon Chopin wrote:
.. <snip> ..
One problem I see is in the implementation details. How long is an endpoint expected to hold on to its old messages before discarding them? Whereas currently, an application that gets a fedmsg hook added doesn't retain much extra state as a result, this replay-request proposal would require a book keeping mechanism added to every endpoint (in our case, every mod_wsgi/httpd process, others).
Not every endpoint. For instance, I don't expect your wiki endpoints to provide such mechanism, as the systems depending on it are not critical (unless I'm missing something). For those that are critical, well, I'd say it is the price to pay.
Cool. If we can keep it heterogeneous like this, then Fedora Infrastructure can keep our endpoints as they are. What do you think about designing this as a persistence plugin system, so the Debian crew could implement a schema that meets your needs, while Fedora can do something else or nothing in this respect.
As for how long, IMO there is no single answer to the question. Each service should have its own policy. In my initial implementation I plan to store the messages using sqlalchemy, without time limit, but I'm open to suggestions on how to handle this differently.
I understand. Indefinitely is an option. At first I was thinking primarily of doing this in memory (instead of on disk in a database), but with a sql db for each publisher, this can work.
Have you thought much yet about how the API should be changed if anywhere?
fedmsg.publish will need to record outgoing messages. That seems pretty simple; no API changes needed.
Should fedmsg.tail_messages automatically detect outages and handle sending the replay request for the user? Or should the user be responsible for detecting the outage and making the request themselves? Same question goes for hub daemons and their consumers.
-Ralph