On Fri, May 15, 2020 at 2:33 PM Aoife Moloney <amoloney(a)redhat.com> wrote:
Hi Pagure Community,
During a recent office hours on IRC, I was asked, actually reminded,
that the analysis of Pagure had yet to be sent. My apologies for this
and thank you Defolos for the reminder :) and for holding me
accountable! The below text was lifted from an internal google
document used by CPE management and above, when discussing what
gitforge option to pursue, and copied to this hackmd file for
convenience for readers who are using terminals, etc to consume.
I hope this email serves an information sharing purpose and even
possibly a roadmap of sorts of improvements that I have no doubt are
either being actively worked on or could be developed in Pagure, and
I look forward to the good conversations this may generate.
# GitForge Decision Considerations
Recently, the Community Platform Engineering (CPE) team who look after
both the Fedora and CentOS Communities ran an analysis on what their
future Gitforge needs would look like. As part of that, we analysed
Pagure and in our follow on discussions we promised to feed back into
the Pagure community the gaps as we saw them which played a part in
our decision making process. The text below can be ported to various
issues, just let us know the most appropriate location and we are
happy to offer our assistance in fleshing these out. Please note that
this is a requirements driven view of the capabilities of Pagure and
is not a slight at the work the community and indeed the CPE team have
put into this project over the years. We deeply appreciate the efforts
of all contributors and we hope that this mail is insightful and helps
to build a strong Pagure.
Thanks for this information. I've responded inline to the various
## Technical Recommendations:
A number of technical requirements that Pagure would need to consider
to meet some of our requirements are as follows:
### Additional Single Sign On (SSO) Integration
The CPE team work closely with the RHEL, CentOS and Fedora projects
and the ability to have a workflow within the gitforge means a need
for SSO integration with those systems is a key need.
What additional integration do you want? We already support OpenID
Connect, OpenID, and FAS-OpenID and propagating permissions from
OpenID attributes. Is there a particular feature you're missing?
### Reliability & Performance
Some projects that host Pagure and wish to operate on a high uptime
backed by a Service Level Agreement (SLA) -- where uptime is measured
in the 99.9% bracket -- are unable to do so without additional
considerations already present within the Pagure project. Currently
all CPE can offer are general Service Level Expectations (SLE). To
enable an SLA, it’s partly additional resourcing to help ensure uptime
but a huge investment is required in API monitoring to spot a problem
and try and resolve it prior to an issue taking the service off the
air. Pagure needs extensive work on API monitoring to improve the
ability to transition this from an towards a viable SLA model. Right
now there are 6 Nagios checks running on Pagure which the CPE team
have created. SLA levels are in the thousands of Nagios Checks (>10k
in many cases) to help us understand where improvements need to go and
what is causing issues. Pagure is resource heavy and numerous tickets
have noted the sluggishness of Pagure (multiple reported issues on PR
creation and searching). To solve issues like this, we need
observability and even then the route fix might be re architecture of
core components to leverage more redundancy and create a means for
resiliency whereby individual failures will not take down the entire
service. That deep level of architectural analysis would be highly
recommended and if CPE can help we are more than happy to offer our
I've noticed that some of the sluggishness actually comes from Apache
rather than Pagure itself. When I started working on porting Pagure to
run with Gunicorn + Nginx for someone who wanted that over Apache, I
discovered that configuration eliminates a large segment of the
performance issues. Admittedly, this is not tested at a large scale
(e.g. stg.pagure.io or pagure.io itself), but the early results look
As for additional instrumentation for performance improvements, do you
have a suggestion of a FOSS APM (application performance monitoring)
solution that works well with Flask (the framework that Pagure is
built on) that we could use in Fedora Infrastructure to help with
this? I suspect the answer to this question would actually be quite
useful for a wide range of Fedora applications.
### Team Collaboration Features
Pagure is lacking several key features in team collaboration:
inline images in PRs
a full suite of PR comments  features to allow richer reviews on PRs
the ability to collaborate on project planning capabilities (Kanban
boards) including linking of issues through to PRs.
The ability to infer stats from your codebase and insights for improvements.
The ability to migrate issues easily between projects.
The ability to rename or move projects between namespaces.
The ability to edit tags[1,2] on issues and the ability to cleanup
forks through the API.
The ability to allow for a better granularity of which events are
sent on the webhook to facilitate integration with other applications
collaboration of apps through webhooks. While a webhook functionality
exists, this is all or nothing approach, whereby every event triggers
the hook or none do.
In short, the day to day working of a team and collaboration within a
project is right now not optimal
I'm confused about your references. There don't seem to be any links
associated with them?
* Issues can already be linked to PRs the same way they are done in
other forges: either by using a full URL or just a short number
* The statistics actually just got some improvements with the 5.10.0
release to help get information here, but I'm not sure what insights
you'd like to have.
* Migrating issues across projects and renaming and moving projects
are definitely weaknesses currently. It's possible to do if you're an
instance admin, but it's better if it can be done by project/namespace
* Having granularity on webhooks would probably also require
revisiting how webhooks are setup and managed, but I think this is
definitely something that could use improvement. Do you have any
particular idea of how you'd like this to look?
### Accessibility and User Interface(UI)/User Experience (UX):
Pagure has multiple accessibility compliance issues. While some of
these issues are easy fixes, many obvious issues relate to style and
theming. The overall UX has a number of issues which need a modern
refresh to make the system more amenable to new users of a Git Forge
or those familiar with other forge UIs who might struggle with
concepts like merging and conflicts [1,2]. Discovery of projects or
issues is difficult  which hampers attempts at drive by
contributors to discover and partake. Attracting and lowering the
barrier for new users and particularly those who do not have a strong
background in the usage of git was a focal point of several
This is a little hard to respond to without the references, but I
think improving the UX is always something worth striving for. There's
been some discussions recently about improving this and also making it
### Workflow Integrations
Pagure has integrations with other standalone services to help
facilitate workflow integrations and general Continuous Integration
(CI)/ Continuous Deployment (CD) capabilities. While the integrations
are positive, it blooms the scope of responsibility, as addressing
Pagure means by extension addressing the individual components that
help to plug feature gaps within Pagure. This extends our SLA coverage
by an order of magnitude. Additional considerations here are the all
or nothing nature of the webhooks which makes integrations with other
tooling more problematic (e.g. automatic build triggers)
Having done this sort of thing before, I would say it depends on how
you approach everything. This really doesn't change much with an
integrated system (e.g. GitLab) because you still have the increased
scope of responsibility. You wind up managing the runners,
configuration of the CI apparatus, and quality of service of the whole
The major advantage of Pagure not having a CI system built in is that
you can establish an interface and scope contract of responsibility of
what the system provides and what you support. For example, your team
may own a Pagure instance and all the configuration to make that
service available, including activating Pagure CI integrations, but
you would ultimately not be responsible for the CI system that the
project owner ultimately connected their project to. If you happen to
own both systems (as is the case if you own both the Pagure system and
the Jenkins system), then this is no net change from how it would be
if you had GitLab.
There's also the whole thing of not having the guess what kind of
system people need to test their project, but that depends on what
types of projects you're hosting.
That being said, providing a better experience with CI/CD is
definitely on the table. I'm not personally sure how to do it yet, but
I'm exploring a few options.
Take care everyone and have a lovely weekend,
Have a good weekend yourself, too!
真実はいつも一つ！/ Always, there's only one truth!