Bug 17524

Summary:	mirror files causing random install problems
Product:	Websites	Reporter:	Pierre Fortin <pf>
Component:	mirrors.mageia.org	Assignee:	Sysadmin Team <sysadmin-bugs>
Status:	RESOLVED WORKSFORME	QA Contact:
Severity:	major
Priority:	Normal	CC:	marja11, sysadmin-bugs
Version:	trunk
Target Milestone:	---
Hardware:	All
OS:	Linux
Whiteboard:
Source RPM:		CVE:
Status comment:

Description Pierre Fortin 2016-01-18 17:37:36 CET

Rating this MAJOR as it appears the mirroring infrastructure is flawed.

Description of problem:  See also:
https://bugs.mageia.org/show_bug.cgi?id=13461
https://bugs.mageia.org/show_bug.cgi?id=17517

After experiencing bugs such as the above, I suspected a problem with "infrastructure" files such as MD5SUM -- this issue could apply to synthesis and other files.  The essence of the problem is that it appears not all mirroring is done in a consistent and reproducible way.

I suspect MD5SUM (other?) files are reverted (older date) rather than corrected with a newer date.  Without a newer date, reverted files will NOT propagate through all mirrors (programs such wget will not mirror that more correct file because of the older timestamp).

In the following 'll', the first file is from my local mirror proving that MD5SUM did have Jan 15 01:52 timestamp on a recent overnight mirroring; while mirror.nexcess.net from which it was obtained now has an older timestamp.

$ cd /homebak/distros/5.32/media/tainted/updates/media_info
$ ll M*
-rw-r--r-- 1 pfortin pfortin 576 Jan 15 01:52 MD5SUM
-rw-r--r-- 1 pfortin pfortin 576 Jan 14 20:52 MD5SUM-distro.ibiblio.org
-rw-r--r-- 1 pfortin pfortin 576 Jan 18 09:50 MD5SUM-mageia.webconquest.com
-rw-r--r-- 1 pfortin pfortin 576 Nov  7 15:10 MD5SUM-mirror.nexcess.net
-rw-r--r-- 1 pfortin pfortin 576 Jan 14 20:52 MD5SUM-mirrors.kernel.org

Comparing my local mirror to several remote mirrors, the mirror that I generally use is clearly different.  There is definitely a time-bomb here since most of the timestamps for the same file are different on each mirror...

$ diff MD5SUM MD5SUM-distro.ibiblio.org
$ diff MD5SUM MD5SUM-mageia.webconquest.com
$ diff MD5SUM MD5SUM-mirror.nexcess.net 
1,10c1,10
< 3841ad803519e7f007852bc95536d7e7  hdlist.cz
< b11d0ab98dc910f560c5718274ec5d8c  synthesis.hdlist.cz
< f95a18cfc18dab7666f848e656d3e4a7  info.xml.lzma
< e078f813335c6fef6e61f5a8d3cdb165  files.xml.lzma
< 12bd76d49f7fdaabd5e43d53476dbcba  changelog.xml.lzma
< 3841ad803519e7f007852bc95536d7e7  20160115-015223-hdlist.cz
< b11d0ab98dc910f560c5718274ec5d8c  20160115-015223-synthesis.hdlist.cz
< f95a18cfc18dab7666f848e656d3e4a7  20160115-015223-info.xml.lzma
< e078f813335c6fef6e61f5a8d3cdb165  20160115-015223-files.xml.lzma
< 12bd76d49f7fdaabd5e43d53476dbcba  20160115-015223-changelog.xml.lzma
---
> 7eea60c25f7e080596557d09f732023d  hdlist.cz
> 2c3b00c3de75e4608c77fa86c4003050  synthesis.hdlist.cz
> 2a560ab890e09cc0610fab254c23026b  info.xml.lzma
> f12324761a6422a51d57fb130a9b6e20  files.xml.lzma
> 6a44a9bec2c2dc6ef8f0636376960136  changelog.xml.lzma
> 7eea60c25f7e080596557d09f732023d  20151107-201016-hdlist.cz
> 2c3b00c3de75e4608c77fa86c4003050  20151107-201016-synthesis.hdlist.cz
> 2a560ab890e09cc0610fab254c23026b  20151107-201016-info.xml.lzma
> f12324761a6422a51d57fb130a9b6e20  20151107-201016-files.xml.lzma
> 6a44a9bec2c2dc6ef8f0636376960136  20151107-201016-changelog.xml.lzma
$ diff MD5SUM MD5SUM-mirrors.kernel.org

Is there a standard process for reliable mirroring?
If so, is it always followed?
If not, why not?

Version-Release number of selected component (if applicable):


How reproducible: randomly


Steps to Reproduce:
1. install/update mgaN
2. random failures
3.


Reproducible: 

Steps to Reproduce:

Comment 1 Marja Van Waes 2016-02-06 16:04:58 CET

We do not have any influence on how the sysadmins of the mirrors decide to sync, even if we do give some suggestions on how to do it, see:
ftp://mageia.webconquest.com/mirror.readme

Our sysadmin team is aware that our current solution 
http://mirrors.mageia.org/status
(only checking the mageia_timestamp of the mirrors and manually removing those from the mirrorlist that lag behind too much) does have flaws.

As you can read in bug 17400, the accepted Mageia6 DNF feature
https://wiki.mageia.org/en/Feature:Add_DNF_as_Alternate_Repository_Manager
needs mirrorbrain, of which should solve part of the problem with broken mirrors, and DNF + mirrorbrain would solve even more.

Our sysadmin team is currently updating our servers, after which they can focus on improving our infrastructure and adding new features.

(I'm not sure whether I can close this bug as duplicate of 17400 or of 3166, will leave that to a sysadmin to decide)

CC: (none) => marja11

Comment 2 Pierre Fortin 2021-07-09 00:34:43 CEST

Haven't had issues in a while, so closing...

Resolution: (none) => WORKSFORME
Status: NEW => RESOLVED