Rating this MAJOR as it appears the mirroring infrastructure is flawed. Description of problem: See also: https://bugs.mageia.org/show_bug.cgi?id=13461 https://bugs.mageia.org/show_bug.cgi?id=17517 After experiencing bugs such as the above, I suspected a problem with "infrastructure" files such as MD5SUM -- this issue could apply to synthesis and other files. The essence of the problem is that it appears not all mirroring is done in a consistent and reproducible way. I suspect MD5SUM (other?) files are reverted (older date) rather than corrected with a newer date. Without a newer date, reverted files will NOT propagate through all mirrors (programs such wget will not mirror that more correct file because of the older timestamp). In the following 'll', the first file is from my local mirror proving that MD5SUM did have Jan 15 01:52 timestamp on a recent overnight mirroring; while mirror.nexcess.net from which it was obtained now has an older timestamp. $ cd /homebak/distros/5.32/media/tainted/updates/media_info $ ll M* -rw-r--r-- 1 pfortin pfortin 576 Jan 15 01:52 MD5SUM -rw-r--r-- 1 pfortin pfortin 576 Jan 14 20:52 MD5SUM-distro.ibiblio.org -rw-r--r-- 1 pfortin pfortin 576 Jan 18 09:50 MD5SUM-mageia.webconquest.com -rw-r--r-- 1 pfortin pfortin 576 Nov 7 15:10 MD5SUM-mirror.nexcess.net -rw-r--r-- 1 pfortin pfortin 576 Jan 14 20:52 MD5SUM-mirrors.kernel.org Comparing my local mirror to several remote mirrors, the mirror that I generally use is clearly different. There is definitely a time-bomb here since most of the timestamps for the same file are different on each mirror... $ diff MD5SUM MD5SUM-distro.ibiblio.org $ diff MD5SUM MD5SUM-mageia.webconquest.com $ diff MD5SUM MD5SUM-mirror.nexcess.net 1,10c1,10 < 3841ad803519e7f007852bc95536d7e7 hdlist.cz < b11d0ab98dc910f560c5718274ec5d8c synthesis.hdlist.cz < f95a18cfc18dab7666f848e656d3e4a7 info.xml.lzma < e078f813335c6fef6e61f5a8d3cdb165 files.xml.lzma < 12bd76d49f7fdaabd5e43d53476dbcba changelog.xml.lzma < 3841ad803519e7f007852bc95536d7e7 20160115-015223-hdlist.cz < b11d0ab98dc910f560c5718274ec5d8c 20160115-015223-synthesis.hdlist.cz < f95a18cfc18dab7666f848e656d3e4a7 20160115-015223-info.xml.lzma < e078f813335c6fef6e61f5a8d3cdb165 20160115-015223-files.xml.lzma < 12bd76d49f7fdaabd5e43d53476dbcba 20160115-015223-changelog.xml.lzma --- > 7eea60c25f7e080596557d09f732023d hdlist.cz > 2c3b00c3de75e4608c77fa86c4003050 synthesis.hdlist.cz > 2a560ab890e09cc0610fab254c23026b info.xml.lzma > f12324761a6422a51d57fb130a9b6e20 files.xml.lzma > 6a44a9bec2c2dc6ef8f0636376960136 changelog.xml.lzma > 7eea60c25f7e080596557d09f732023d 20151107-201016-hdlist.cz > 2c3b00c3de75e4608c77fa86c4003050 20151107-201016-synthesis.hdlist.cz > 2a560ab890e09cc0610fab254c23026b 20151107-201016-info.xml.lzma > f12324761a6422a51d57fb130a9b6e20 20151107-201016-files.xml.lzma > 6a44a9bec2c2dc6ef8f0636376960136 20151107-201016-changelog.xml.lzma $ diff MD5SUM MD5SUM-mirrors.kernel.org Is there a standard process for reliable mirroring? If so, is it always followed? If not, why not? Version-Release number of selected component (if applicable): How reproducible: randomly Steps to Reproduce: 1. install/update mgaN 2. random failures 3. Reproducible: Steps to Reproduce:
We do not have any influence on how the sysadmins of the mirrors decide to sync, even if we do give some suggestions on how to do it, see: ftp://mageia.webconquest.com/mirror.readme Our sysadmin team is aware that our current solution http://mirrors.mageia.org/status (only checking the mageia_timestamp of the mirrors and manually removing those from the mirrorlist that lag behind too much) does have flaws. As you can read in bug 17400, the accepted Mageia6 DNF feature https://wiki.mageia.org/en/Feature:Add_DNF_as_Alternate_Repository_Manager needs mirrorbrain, of which should solve part of the problem with broken mirrors, and DNF + mirrorbrain would solve even more. Our sysadmin team is currently updating our servers, after which they can focus on improving our infrastructure and adding new features. (I'm not sure whether I can close this bug as duplicate of 17400 or of 3166, will leave that to a sysadmin to decide)
CC: (none) => marja11
Haven't had issues in a while, so closing...
Resolution: (none) => WORKSFORMEStatus: NEW => RESOLVED