Hey folks!
Unless I'm mistaken, we currently don't have many useful technical metrics about Mailman and HyperKitty. I've started writing a simple script that would collect those and output something we can eventually graph. It currently uses the "key: value" format that I've seen a lot of monitoring tools use, but I can adapt that to whatever we need.
I'm currently getting the following values :
- mailinglist count in HyperKitty - email count in HyperKitty - email senders count in HyperKitty - user count in HyperKitty - async task queue size - number of async tasks run per minute - async tasks average exec time - size of the archiving queues in mailman and hyperkitty (should be zero most of the time) - Mailman's shunt queue (for messages it failed to handle, usually encoding problems)
Do you think there would be other values that we would need? My aim is to detect issues with the server usage (say, a spam attack) as well as anticipate growth. Another useful metric would be the number of emails sent by the system, but I guess we already have a Postfix plugin somewhere, so I'm trying to stay mailman & hyperkitty-specific here.
Currently the plugin takes between 5 and 10 seconds to collect the data. I don't know how precise we want our metrics but a run every five minutes seems sufficient to me. It also depends on how we can set that in collectd, which I don't know (yet!).
Thanks!
Aurélien
Le 9 août 2017 14:19:32 UTC+03:00, Aurelien Bompard abompard@fedoraproject.org a écrit :
Hey folks!
Unless I'm mistaken, we currently don't have many useful technical metrics
Hi Aurélien,
On a technical point of view I have no ideas, but would it be possible to get an extract of the number of messages per month per mailing list? I would like to see if this may help to have a hint about activity per local community. I do know some mailing lists are not hosted by fedora infrastructure.
I collected website views, appstream file translations, zanata informations and am working on extracting information from packages.
Maybe some information from mailing lists may help to give an overview. I'll show my data to commops group at flock (there is a metrics workshop), to see if we can make use of it.
Have a nice day,
2017-08-09 15:12 GMT+02:00 Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr
:
On a technical point of view I have no ideas, but would it be possible to get an extract of the number of messages per month per mailing list? I would like to see if this may help to have a hint about activity per local community. I do know some mailing lists are not hosted by fedora infrastructure.
Hmm, this looks like something that would probably be better handled by an API call. There is currently no practical way to get that information, but I've temporarily added two values to the HyperKitty API endpoint for mailing-lists: "recent_threads_count" and "recent_participants_count". Those are the values displayed in list summaries.
Example here: https://lists.fedoraproject.org/archives/api/list/fr-users@lists.fedoraproje... (the last two values)
Would that information be helpful to you? I can consider adding more to this API if necessary (as long as it doesn't result in expensive calls in the backend).
Thanks
Aurélien
Le 10 août 2017 18:29:07 UTC+03:00, Aurelien Bompard abompard@fedoraproject.org a écrit :
2017-08-09 15:12 GMT+02:00 Jean-Baptiste Holcroft <jean-baptiste@holcroft.fr
:
On a technical point of view I have no ideas, but would it be
possible to
get an extract of the number of messages per month per mailing list?
I
would like to see if this may help to have a hint about activity per
local
community. I do know some mailing lists are not hosted by fedora infrastructure.
Hmm, this looks like something that would probably be better handled by an API call. There is currently no practical way to get that information, but I've temporarily added two values to the HyperKitty API endpoint for mailing-lists: "recent_threads_count" and "recent_participants_count". Those are the values displayed in list summaries.
Example here: https://lists.fedoraproject.org/archives/api/list/fr-users@lists.fedoraproje... (the last two values)
Would that information be helpful to you? I can consider adding more to this API if necessary (as long as it doesn't result in expensive calls in the backend).
Thanks
Aurélien
Thanks, for flock it's good enough, I'll try to script it to see if this can be of any use once I have all mailing list data.
I'll come back to you if it leads to something.
Have a nice evening,
infrastructure@lists.fedoraproject.org