On Mon, Nov 9, 2020 at 3:05 PM Aoife Moloney <amoloney(a)redhat.com&gt; wrote: > > Hi everyone, > > I hope you enjoyed the F33 release party this weekend! Getting back to > the GitLab topic mail threads, this weeks topic from the GitLab AMA > session on September 10th is on Message Bus. As always, here are some > links to the resources I have been pulling content from as well: > * Questions and Answers hackmd link https://hackmd.io/RW8HahOeR7OJPON1dwuo3w > * Chat log from session > https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-10/ama_session... > * AMA Blog post > https://communityblog.fedoraproject.org/gitlab-ama-follow-up/#more-9346 > * Here is this email in hackmd if you wish to view it there: > https://hackmd.io/tfOqCXNEQtqsGNLAEfZ2zg?view Sorry for taking so long to respond, the past weeks have been quite busy.

(snip) > ## Topic: Message Bus > - Question: Fedora uses a message bus to integrate different parts of > its infrastructure. How should we onboard GitLab into this message > bus? > - Answer: Currently we would need to have a service that proxies > GitLab’s events to fedora-messaging something similar to > github2fedmsg. > There were some concerns raised about the order of events sent by > GitLab’s webhooks, these will need to be looked after during a Proof > of Concept stage. Do we know if such a proxy would even be theoretically possible for GitLab? IIRC, some doubts were raised during the AMA that getting a chronologically consistent stream of events out of GitLab would be ∈ [hard, impossible[. What would that mean for fedora? Do services relying on fedora-messaging events related to dist-git need them to be consistent / chronological? What would be the effect of those services not having a reliable stream of events from dist-git?

Fabio > - Question: How would git push over http work with GitLab? (assuming > gitlab does not have Fedora's password since they are stored in FAS) > - Answer: GitLab has a good number of authentication options and > integrations where the "best" solution usually depends on a team's > specific needs and use case. As such, the best way to know and meet > Fedora's needs and use cases is to have a conversation and discuss the > options with GitLab. How does git push over HTTP work with FAS right > now, and what git push (over HTTP) auth requirements/flow would you > like to have for your projects in GitLab? > > These are the only two questions and answers I could gather relating > to message bus from the AMA question sheet, however I know there was a > lot of discussion regarding this topic during the AMA session itself, > so if you would like to resume/kick off that conversation again, > please feel free to use this email to discuss. > > A personal note and for full transparency: the weeks seem to be > passing in the blink of an eye lately, I assume it's because I'm > busy(?) but it might be just the weird 2020 vibe the world is on > nowadays. I really don't know. Whatever the reason, there has been no > further discussion with GitLab since early October-ish, but we will be > returning to the conversation of how this migration could be > technically possible soon, so sincerely thank you all again for > engaging with us/me here as I found reading the discussion on > permission and access much easier to follow and have been taking notes > on your expectations to use that feedback in conversations with GitLab > when we pick the discussion back up. > > > > I hope you all had a good weekend and will talk to you all next week > when the topic of Namespace & Issue Tracking is sent! > > > Kindest regards, > Aoife > -- > Aoife Moloney > Product Owner > Community Platform Engineering Team > Red Hat EMEA > Communications House > Cork Road > Waterford > _______________________________________________ > devel-announce mailing list -- devel-announce(a)lists.fedoraproject.org > To unsubscribe send an email to devel-announce-leave(a)lists.fedoraproject.org > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedora... _______________________________________________ devel mailing list -- devel(a)lists.fedoraproject.org To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

On Mon, 2020-11-23 at 19:29 +0100, Clement Verna wrote: > > > On Wed, 18 Nov 2020 at 18:30, Fabio Valentini <decathorpe(a)gmail.com&gt; > wrote: > > On Mon, Nov 9, 2020 at 3:05 PM Aoife Moloney <amoloney(a)redhat.com&gt; > > wrote: > > > > > > Hi everyone, > > > > > > I hope you enjoyed the F33 release party this weekend! Getting > > back to > > > the GitLab topic mail threads, this weeks topic from the GitLab > > AMA > > > session on September 10th is on Message Bus. As always, here are > > some > > > links to the resources I have been pulling content from as well: > > > * Questions and Answers hackmd link > > https://hackmd.io/RW8HahOeR7OJPON1dwuo3w > > > * Chat log from session > > > > > https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-10/ama_session... > > > * AMA Blog post > > > > > https://communityblog.fedoraproject.org/gitlab-ama-follow-up/#more-9346 > > > * Here is this email in hackmd if you wish to view it there: > > > https://hackmd.io/tfOqCXNEQtqsGNLAEfZ2zg?view > > > > Sorry for taking so long to respond, the past weeks have been quite > > busy. > > > > Sorry for taking even longer to answer, you know the feeling :P > > > > > (snip) > > > > > ## Topic: Message Bus > > > - Question: Fedora uses a message bus to integrate different > > parts of > > > its infrastructure. How should we onboard GitLab into this > > message > > > bus? > > > - Answer: Currently we would need to have a service that > > proxies > > > GitLab’s events to fedora-messaging something similar to > > > github2fedmsg. > > > There were some concerns raised about the order of events sent by > > > GitLab’s webhooks, these will need to be looked after during a > > Proof > > > of Concept stage. > > > > Do we know if such a proxy would even be theoretically possible for > > GitLab? > > IIRC, some doubts were raised during the AMA that getting a > > chronologically consistent stream of events out of GitLab would be > > ∈ > > [hard, impossible[. > > What would that mean for fedora? Do services relying on > > fedora-messaging events related to dist-git need them to be > > consistent > > / chronological? > > What would be the effect of those services not having a reliable > > stream of events from dist-git? > > > > I personally don't have a good answer to these questions, and I don't > think we will be able to > have without actually doing a Proof of Concept and see how that would > work and scale. > > Regarding the order or messages, I believe that anything related to > CI testing might need to > have the chronological order of messages consistent. > I am not sure if there are any other use cases ? > If any service relies on chronological ordering for messaging, I've got bad news on that front. There's plenty of scenarios where messages can arrive out-of-order now, regardless of whether they were queued up sequentially (which, in a distributed system, with multiple nodes accepting messages for publication...). There's also no guarantees about messages only being delivered once. Never mind that GitLab's webhooks may well fire multiple times if a response is not received in a timely manner. I would highly recommend not creating message consumers that rely on any particular message ordering because they're not going to work properly, GitLab or not.

On Mon, 2020-11-23 at 15:11 -0500, Neal Gompa wrote: > > > > I would highly recommend not creating message consumers that rely on > > any particular message ordering because they're not going to work > > properly, GitLab or not. > > Too late, pretty much every consumer I'm aware of relies on having > chronological order or at least some way to sort them chronologically > for processing for messages. I don't think any consumer I've written does. They all just work on the basis "a thing happened; do some things relevant to the thing that happened".

-- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net _______________________________________________ devel mailing list -- devel(a)lists.fedoraproject.org To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org