Re: About MirrorManager2

On 02/12/2015 02:43 AM, Stephen John Smoogen wrote:

- Mirror versioning:
  - run UMDL, detect changes, increase master mirror's version by 1
  - run the crawler, check for the changes, align that mirror's
version with the
    master mirror one
  - be able to see the difference between two versions
  - be able to crawl a mirror only for the difference between the
version it is
    at and the version the master mirror is at
  note: we might still want to run a full crawl once in a while (daily?
  bi-daily?)

Those last couple of steps would be very useful in cutting down load from mirrors running rsync against enchillada but only needing a couple of packages in updates since the last time they did it.

In that general vein, an interesting project Robert Collins introduced me to last year is his lmirror work: https://pypi.python.org/pypi/lmirror

It's designed to take advantage of various HTTP/2 features and the knowledge that it's specifically being used to create read-only mirrors to make the network operations more efficient than is feasible with the more general purpose rsync.

Design details: https://bazaar.launchpad.net/~lmirror/lmirror/master/view/head:/doc/DESIGN.t...

Regards, Nick.