On Wed, 11 Feb 2015 14:10:01 +0100 Pierre-Yves Chibon pingou@pingoured.fr wrote:
Hi everyone,
We currently have MirrorManager2 running on staging. It's apparently not 100% set-up since we get emails once in a while that one of the crons failed (iirc among other we need to finish configuring fedmsg).
Yeah, there's two crons that are failing due to lack of fedmsg. :(
Other that this, MirrorManager2 is currently in a decent shape I think. However, we really need to make sure nothing broke in the re-write and we want to make sure we won't break it in the future. To try to ensure that last part, I have try to write some tests for the UI but also for the backend part (all the different scripts). The pull-request is opened for review: https://github.com/fedora-infra/mirrormanager2/pull/14
Cool.
I have also been trying to capitalize on the knowledge we acquired during the FAD by starting to write down how mirrormanager works in the documentation: https://github.com/fedora-infra/mirrormanager2/compare/tests...doc (pull-request to be opened once the tests branch is merged) I would appreciate if those that were at the FAD could go through this branch/changes and adjust as needed. I have been thinking about asking Matt to do the review so that we can adjust and improve the documentation.
That is an excellent idea. ;) The docs all look good/accurate to me. ;)
Before we move MirrorManager2 to prod, here is what I think would be nice to do/have:
- Pickle validation
- Figure a way to validate a pickle after its creation and before
moving it to the mirrorlist boxes
If it would help that we have tracebacks from the mm2 mirrorlist for when a bad pkl is loaded (the mm1 ones just fail, but the mm2 one does provide a traceback):
Feb 9 10:11:42 mirrorlist-host1plus python2[25890]: Traceback (most recent call last): Feb 9 10:11:42 mirrorlist-host1plus python2[25890]: File "/usr/share/mirrormanager2/mirrorlist_server.py", line 877, in handle Feb 9 10:11:42 mirrorlist-host1plus python2[25890]: r = do_mirrorlist(d) Feb 9 10:11:42 mirrorlist-host1plus python2[25890]: File "/usr/share/mirrormanager2/mirrorlist_server.py", line 718, in do_mirrorlist Feb 9 10:11:42 mirrorlist-host1plus python2[25890]: allhosts, cache, file, pathIsDirectory=pathIsDirectory) Feb 9 10:11:42 mirrorlist-host1plus python2[25890]: File "/usr/share/mirrormanager2/mirrorlist_server.py", line 423, in append_path Feb 9 10:11:42 mirrorlist-host1plus python2[25890]: s = hcurl_cache[hcurl_id] Feb 9 10:11:42 mirrorlist-host1plus python2[25890]: KeyError: 7716
- Find out if we can improve our tests some more (to improve our confidence that we're ready)
- Engage the mirror mailing list and try to get them to react on the
coming changes
Good idea.
The pickle validation might also be an interesting idea to check if there is a difference between the pickle generated by prod and the one generated in stg.
Finally, at DevConf we have been speaking quite a bit with Dennis around updates and MirrorManager and here is some of the ideas we spoke about:
- Be able to run the UMDL script on only a part of the tree (ie: be able to say, we updated f21-updates and we only update this
part)
- Crawl the mirrors for only a part of the tree (This goes together with updating only part of the tree via UMDL)
- Consider if we should/could drop the content of the
host_category_dir table before running the crawler
- Mirror versioning:
- run UMDL, detect changes, increase master mirror's version by 1
- run the crawler, check for the changes, align that mirror's
version with the master mirror one
- be able to see the difference between two versions
- be able to crawl a mirror only for the difference between the
version it is at and the version the master mirror is at note: we might still want to run a full crawl once in a while (daily? bi-daily?)
This list of ideas is more a long term todo list, not something we would want to have working for pushing MirrorManager2 to prod.
Thoughts? Agreements? Disagreements?
Sounds good.
There's some minor changes to make in mirrorlist rpms, but once thats done we can look at replacing the other mirrorlists anytime.
For the other parts, I think we will need to just create them all in production and run them in parallel to the mm1 setup for a short time until we are ready to cut over.
kevin