Greetings.
I meant to send this out sooner, but with the Fedora 20 release things have been crazy. :) Now that we are nearing the end of 2013, I think it might be a nice time to think ahead to next year.
What would you love to see happen? What would you like to work on making happen? (even if it's not practical resource wise or logistically)
Here's a list of some of mine:
* 0 downtime upgrades. By this I mean we can update and reboot all our servers (probibly in some specific order with specific actions between) and not have to schedule any downtime or notify users anything is going on. This means at least that we have db replication/failover working and 2 of everything.
* Migration to ansible fully done, with all hosts moved over and rebuilt and working.
* Migration to RHEL7. :)
* Ticket queue down to a very small set. When I first started it was gigantic, then I closed/fixed/redirected a lot of the ticket and we started going down in number, but over the last year or so we have hovered between 140-150.
* Migration to hyperkitty/mailman3 complete with all lists moved.
* All hosts selinux enforcing. :)
* No "app" servers left. All applications split out to their own (at least pair) of instances.
* Logging from every app/server goes to a known place and we process them all looking for problems.
I'll probibly think of some more, will send them to this thread when I do. So how about you? Any big dreams for 2014? Hopefully we can make at least some of them real.
kevin
On Fri, Dec 20, 2013 at 07:43:50AM +0800, Christopher Meng wrote:
Is it possible to use gitlab to replace cgit?
I would say no. gitlab would really be more of a trac replacement. cgit is a lightweight web frontend for git repositories. Two different purposes.
-Toshio
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 12/19/2013 09:48 PM, Toshio Kuratomi wrote:
On Fri, Dec 20, 2013 at 07:43:50AM +0800, Christopher Meng wrote:
Is it possible to use gitlab to replace cgit?
I would say no. gitlab would really be more of a trac replacement. cgit is a lightweight web frontend for git repositories. Two different purposes.
Also, gitlab does not have the ability to retrieve git blobs by hash (currently), so it would make it impossible for us to use tools like ReviewBoard without keeping a local clone up to date.
Another request is to fedpkg, can it support resumable upload?
Sometimes even if I try 20 times to upload the sources, it still fails everytime due to the bad internet environment.
Thanks.
On Thu, Dec 19, 2013 at 02:16:31PM -0700, Kevin Fenzi wrote:
Greetings.
I meant to send this out sooner, but with the Fedora 20 release things have been crazy. :) Now that we are nearing the end of 2013, I think it might be a nice time to think ahead to next year.
What would you love to see happen? What would you like to work on making happen? (even if it's not practical resource wise or logistically)
Here's a list of some of mine:
0 downtime upgrades. By this I mean we can update and reboot all our servers (probibly in some specific order with specific actions between) and not have to schedule any downtime or notify users anything is going on. This means at least that we have db replication/failover working and 2 of everything.
Migration to ansible fully done, with all hosts moved over and rebuilt and working.
Migration to RHEL7. :)
Ticket queue down to a very small set. When I first started it was gigantic, then I closed/fixed/redirected a lot of the ticket and we started going down in number, but over the last year or so we have hovered between 140-150.
Migration to hyperkitty/mailman3 complete with all lists moved.
All hosts selinux enforcing. :)
No "app" servers left. All applications split out to their own (at least pair) of instances.
Logging from every app/server goes to a known place and we process them all looking for problems.
I'll probibly think of some more, will send them to this thread when I do. So how about you? Any big dreams for 2014? Hopefully we can make at least some of them real.
kevin
One more for the list:
* Increased automatic QA/continuous integration: - More tests posting back to bodhi. - Dependencies between packages evaluated (if a package is updated, its dependants should be evaluated against it). - A clear way for contributors to submit tests. - fedmsg messages all around.
On Wed, Jan 08, 2014 at 11:18:29AM -0500, Ralph Bean wrote:
One more for the list:
- Increased automatic QA/continuous integration:
- More tests posting back to bodhi.
- Dependencies between packages evaluated (if a package is updated, its dependants should be evaluated against it).
- A clear way for contributors to submit tests.
- fedmsg messages all around.
Yes! Have you talked to Tim Flink and the QA team about this?
On Wed, Jan 08, 2014 at 12:19:00PM -0500, Matthew Miller wrote:
On Wed, Jan 08, 2014 at 11:18:29AM -0500, Ralph Bean wrote:
One more for the list:
- Increased automatic QA/continuous integration:
- More tests posting back to bodhi.
- Dependencies between packages evaluated (if a package is updated, its dependants should be evaluated against it).
- A clear way for contributors to submit tests.
- fedmsg messages all around.
Yes! Have you talked to Tim Flink and the QA team about this?
Yes, briefly before the holidays. We'll be in more touch this year.
On Wed, 8 Jan 2014 11:18:29 -0500 Ralph Bean rbean@redhat.com wrote:
<snip>
One more for the list:
- Increased automatic QA/continuous integration:
- More tests posting back to bodhi.
Actually, one of my goals is to kill off the current method of posting to bodhi. In my mind, it's the wrong way to be doing things and causes problems on our end. I want to have a restful api for results querying and use that as an interface instead of posting things directly to bodhi.
I talked to Luke about this informally at either PyCon or Flock and IIRC, he liked the idea. It'll take some work to make all that happen and I'm not expecting it to come with the initial deployment of taskotron, but it's on my mind for the future.
- Dependencies between packages evaluated (if a package is updated, its dependants should be evaluated against it).
If you're talking about rpm dependencies, that should already be taken care of with depcheck. I've honestly not given much thought to any attempts to get into "do these packages actually work with eachother" beyond checking for ABI breakage.
- A clear way for contributors to submit tests.
- fedmsg messages all around.
These are both on our todo list. The fedmsg part is going to be an interesting conversation, though - I'm not quite sure how to handle reporting and notifications. Interestingly enough, starting that conversation on qa-devel@ (for now, eventually moving to devel@) is on my todo list today :)
Tim
On Thu, Dec 19, 2013 at 02:16:31PM -0700, Kevin Fenzi wrote:
Greetings.
I meant to send this out sooner, but with the Fedora 20 release things have been crazy. :) Now that we are nearing the end of 2013, I think it might be a nice time to think ahead to next year.
What would you love to see happen? What would you like to work on making happen? (even if it's not practical resource wise or logistically)
Here's a list of some of mine:
0 downtime upgrades. By this I mean we can update and reboot all our servers (probibly in some specific order with specific actions between) and not have to schedule any downtime or notify users anything is going on. This means at least that we have db replication/failover working and 2 of everything.
Migration to ansible fully done, with all hosts moved over and rebuilt and working.
Migration to RHEL7. :)
Ticket queue down to a very small set. When I first started it was gigantic, then I closed/fixed/redirected a lot of the ticket and we started going down in number, but over the last year or so we have hovered between 140-150.
Migration to hyperkitty/mailman3 complete with all lists moved.
All hosts selinux enforcing. :)
No "app" servers left. All applications split out to their own (at least pair) of instances.
Logging from every app/server goes to a known place and we process them all looking for problems.
Here is my list (in no particular order):
* FAS3 - We need to port FAS to a new(er) framework - Move the user auth completely out of FAS? - Unit-tests! - Integrate w/ announces -> new election, new deadline... * Develop a 0Auth server? * fedocal - A number of changes and enhancements are required and I started a little bit on this already :) * pkgdb2 - We are already close, but we need to push it to production and fix the bugs that will be encountered * Port our webapp to 2FA - Toshio started some work on this but we need to continue it * Unit-tests - This makes our product so much more robust that we really should keep writing them * Fedmsg-Notifications (FMN) - Already quite advanced, we need to push it to production and add new notification options. * Cnucnu web - Basis is there but it would be nice to add to the database more mapping of the project in more linux distribution than just Fedora * Review-server - Kick the package review-request out of bugzilla - Tight(er) integration with koji - Run fedora-review on each changes - Run scratch build on each changes - Provide a temporary git repo for each review that could then be merged into the official git repo after the review - This seems to interest the dev and stack group, see third point at: https://fedoraproject.org/wiki/User:Mmaslano/Draft:Env_and_Stack_PRD#Automat... - We may want to see with tflink and the QA if there are tools in addition to fedora-review that could be included * Elections - It would be nice to re-work the election application to build in some sort of plugin systems allowing different votes - Eventually it would be very nice to merge nuancier into election as a plugin => I have no real idea yet on how to do so, I just think it would be nice to have it :) * MirrorManager - With FAS it is probably the oldest application that we have that does not have a new version coming. We need to update it (new db scheme (?), new framework, unit-tests...) * Auto-rebuild - Using fedmsg or koji directly we are able to retrieve the date of the last successful build of any packages in our collections. This way we could trigger an automatic rebuild to check FTBFS of every package every X time (6 month?). * Auto-rebuild of broken deps - When we detect a broken deps we should try to rebuild all the dependency and propose the change to the packager if the build succeeded
Some of these are duable, other are more in the range of thinking out loudly but all in all, this is what I have in mind for 2014 at the moment :)
Cheers, Pierre
On Wed, 8 Jan 2014 19:10:43 +0100 Pierre-Yves Chibon pingou@pingoured.fr wrote:
Here is my list (in no particular order):
- FAS3
- We need to port FAS to a new(er) framework
- Move the user auth completely out of FAS?
How would that work?
...snip all good stuff...
- Review-server
- Kick the package review-request out of bugzilla
- Tight(er) integration with koji
- Run fedora-review on each changes
- Run scratch build on each changes
- Provide a temporary git repo for each review that could then be merged into the official git repo after the review
- This seems to interest the dev and stack group, see third point
at: https://fedoraproject.org/wiki/User:Mmaslano/Draft:Env_and_Stack_PRD#Automat...
- We may want to see with tflink and the QA if there are tools in
addition to fedora-review that could be included
Interesting idea. Note that we also need to keep a record of all the work done there. By that I mean I often go back and look at a package review to see if something was missed or addressed or noted. We would need to make sure people could go back and look at the history easily.
- Elections
- It would be nice to re-work the election application to build in
some sort of plugin systems allowing different votes
- Eventually it would be very nice to merge nuancier into election
as a plugin => I have no real idea yet on how to do so, I just think it would be nice to have it :)
- MirrorManager
- With FAS it is probably the oldest application that we have that
does not have a new version coming. We need to update it (new db scheme (?), new framework, unit-tests...)
- Auto-rebuild
- Using fedmsg or koji directly we are able to retrieve the date of
the last successful build of any packages in our collections. This way we could trigger an automatic rebuild to check FTBFS of every package every X time (6 month?).
This could tie in with the taskotron efforts I would think.
- Auto-rebuild of broken deps
- When we detect a broken deps we should try to rebuild all the
dependency and propose the change to the packager if the build succeeded
Not at all easy to do, but yeah...
Speaking of auto rebuilding...
* autorebuild infra instances In particular buildvm's would be nice to reap and rebuild automatically, but this could extend to other things. Just take out of production, rebuild and readd.
Some of these are duable, other are more in the range of thinking out loudly but all in all, this is what I have in mind for 2014 at the moment :)
Good stuff. ;)
kevin
On Wed, Jan 08, 2014 at 07:10:43PM +0100, Pierre-Yves Chibon wrote:
On Thu, Dec 19, 2013 at 02:16:31PM -0700, Kevin Fenzi wrote:
Greetings.
I meant to send this out sooner, but with the Fedora 20 release things have been crazy. :) Now that we are nearing the end of 2013, I think it might be a nice time to think ahead to next year.
What would you love to see happen? What would you like to work on making happen? (even if it's not practical resource wise or logistically)
Here's a list of some of mine:
0 downtime upgrades. By this I mean we can update and reboot all our servers (probibly in some specific order with specific actions between) and not have to schedule any downtime or notify users anything is going on. This means at least that we have db replication/failover working and 2 of everything.
Migration to ansible fully done, with all hosts moved over and rebuilt and working.
Migration to RHEL7. :)
Ticket queue down to a very small set. When I first started it was gigantic, then I closed/fixed/redirected a lot of the ticket and we started going down in number, but over the last year or so we have hovered between 140-150.
Migration to hyperkitty/mailman3 complete with all lists moved.
All hosts selinux enforcing. :)
No "app" servers left. All applications split out to their own (at least pair) of instances.
Logging from every app/server goes to a known place and we process them all looking for problems.
Here is my list (in no particular order):
- FAS3
- We need to port FAS to a new(er) framework
- Move the user auth completely out of FAS?
- Unit-tests!
- Integrate w/ announces -> new election, new deadline...
- Develop a 0Auth server?
- fedocal
- A number of changes and enhancements are required and I started a little bit on this already :)
- pkgdb2
- We are already close, but we need to push it to production and fix the bugs that will be encountered
- Port our webapp to 2FA
- Toshio started some work on this but we need to continue it
- Unit-tests
- This makes our product so much more robust that we really should keep writing them
- Fedmsg-Notifications (FMN)
- Already quite advanced, we need to push it to production and add new notification options.
- Cnucnu web
- Basis is there but it would be nice to add to the database more mapping of the project in more linux distribution than just Fedora
- Review-server
- Kick the package review-request out of bugzilla
- Tight(er) integration with koji
- Run fedora-review on each changes
- Run scratch build on each changes
- Provide a temporary git repo for each review that could then be merged into the official git repo after the review
- This seems to interest the dev and stack group, see third point at: https://fedoraproject.org/wiki/User:Mmaslano/Draft:Env_and_Stack_PRD#Automat...
- We may want to see with tflink and the QA if there are tools in addition to fedora-review that could be included
- Elections
=> I have no real idea yet on how to do so, I just think it would be nice to have it :)
- It would be nice to re-work the election application to build in some sort of plugin systems allowing different votes
- Eventually it would be very nice to merge nuancier into election as a plugin
- MirrorManager
- With FAS it is probably the oldest application that we have that does not have a new version coming. We need to update it (new db scheme (?), new framework, unit-tests...)
- Auto-rebuild
- Using fedmsg or koji directly we are able to retrieve the date of the last successful build of any packages in our collections. This way we could trigger an automatic rebuild to check FTBFS of every package every X time (6 month?).
- Auto-rebuild of broken deps
- When we detect a broken deps we should try to rebuild all the dependency and propose the change to the packager if the build succeeded
Along the lines of automatic rebuilding, we should also think about what it will take to support the vision of continuous integration and increased automated testing that people like mattdm, threebean, and tflink have been discussing.
Maybe this is a difficult proposition simply using our current lineup of koji servers -- uncertain. But would it be possible to spin up instances in the cloud to build what Matt calls "artifacts" (loose term, could be a spin, could be a set of packages TBD)? I don't have answers here, only questions really. But this seems like an area we should really be diving into.
infrastructure@lists.fedoraproject.org
-
Christopher Meng -
Kevin Fenzi -
Matthew Miller -
Paul W. Frields -
Pierre-Yves Chibon -
Ralph Bean -
Stephen Gallagher -
Tim Flink -
Toshio Kuratomi