Mirror Restructuring
There are some issues with the way we do the mirrors at the moment, so we are rethinking on how we should do that. All thoughts about this can be found on this page.
1. Meeting in #centos-mirror on 2010-10-25
There was a meeting in #centos-mirror on the 25th of October where we talked about the two mirror frameworks which seem to be most likely to replace what we have right now:
Fedora's MirrorManager and MirrorBrain
We had Adrian Reber who told us a bit about MirrorManager and Peter Poeml who is the person behind MirrorBrain. During the meeting a short pro/con list was done, which also talks a bit about how mirroring is done at the moment.
1.1. Short introduction
- self built system
- doesn't scale anymore
- cruft
- GeoIP based (country and near countries)
- mirmon disables stale mirrors
- 10 mirrors per country/region, randomized
- Needs work
1.2. Alternatives
MirrorManager (Fedora)
- + Fedora/rpmfusion runs on it
- - scanning also takes a long time (i.e. 1 hour from DE to CN for rpmfusion)
- + mirrors can run a script that updates database status themselves after an rsync
- + admin tasks on mirror admins side (mostly)
- + mirrors can decide what they carry (release/arch) on directory basis
- -+ uses !CityDB and ASNs (!CityDB on a finer granularity than we have)
- + can use netblocks
- - supports private mirrors (but cannot download from tier 0/1)
- - not sure how netblock is enforced
- account system for mirror admins
- metalink support
MirrorBrain (Peter/Suse)
- - admin tasks are distro side (mostly)
- + checks are fast (scans for filelists, but md5 can be done - needs to be implemented, though)
+ GeoCity, ASN, netblocks
- + large setups
- + only one crawler needed.
- + cli for mirror maintenance
- + torrent integration seems to be easier
- + metalink integration is "perfect"
- - supports private mirrors (but also with restrictions)
- + partial mirrors are possible, even only with one file
1.3. IRC logs