Bug 1921

Summary:	HTTP/FTP packages distribution to mirrors
Product:	Infrastructure	Reporter:	Darrel Johnston <darreljohnston>
Component:	Others	Assignee:	Sysadmin Team <sysadmin-bugs>
Status:	RESOLVED FIXED	QA Contact:
Severity:	normal
Priority:	Normal	CC:	davidwhodgins, misc, sysadmin-bugs
Version:	unspecified
Target Milestone:	---
Hardware:	i586
OS:	Linux
Whiteboard:
Source RPM:		CVE:
Status comment:

Description Darrel Johnston 2011-06-26 06:46:57 CEST

Description of problem: I got this error doing a net install, too. I tried two mirrors before trying ibiblio. Ibiblio doesn't have the perl file, either. Correct me if I'm wrong, but most of the mirrors sync from ibiblio as a central distribution point. And if ibilio doesn't have the file, causing the net installation to fail, ....

I finally picked FTP source belnet.be after trying several other mirrors. Some failed with different error messages. Being that I'm in the U.S. and they're in Belgium, a simple net install takes 3 hours and fifteen minutes. The mirrors really should all be in sync for a net installation to succeed from any one of them. However, I don't know how Mageia distributes packages to the mirrors.


Version-Release number of selected component (if applicable):


How reproducible: Very


Steps to Reproduce:
1. Try ibiblio.org for a net installation. It will fail, missing the perl package.
2. Also posted this to Mageia forum.

Comment 1 Darrel Johnston 2011-06-26 06:48:22 CEST

My apologies. The file is compssUsers.pl.

Comment 2 Michael Scherer 2011-06-26 13:51:36 CEST

According to  http://mirrors.mageia.org/report , no one sync from ibiblio.

Until we correct this ( ie, try to contact ibiblio, which is not easy nor fast ), I would suggest to go on another us mirrors, or the one in brasil.

CC: (none) => misc

Comment 3 Michael Scherer 2011-06-26 13:54:47 CEST

And the file seems present : 
 http://distro.ibiblio.org/pub/linux/distributions/mageia/distrib/1/i586/media/media_info/

Comment 4 Dave Hodgins 2011-06-26 19:06:19 CEST

It shows in the directory listing, but if you click on it, or try to get
it with wget, it's not found ...

$ wget http://distro.ibiblio.org/pub/linux/distributions/mageia/distrib/1/i586/media/media_info/compssUsers.pl
--2011-06-26 13:04:22--  http://distro.ibiblio.org/pub/linux/distributions/mageia/distrib/1/i586/media/media_info/compssUsers.pl
Resolving distro.ibiblio.org (distro.ibiblio.org)... 152.19.134.43
Connecting to distro.ibiblio.org (distro.ibiblio.org)|152.19.134.43|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2011-06-26 13:04:22 ERROR 404: Not Found

CC: (none) => davidwhodgins

Comment 5 Michael Scherer 2011-06-26 20:14:04 CEST

But it work on ftp, try 
 curl ftp://distro.ibiblio.org/pub/linux/distributions/mageia/distrib/1/i586/media/media_info/compssUsers.pl

I guess the extensions trigger something on their server.

Comment 6 James Kerr 2011-06-26 22:27:18 CEST

Mageia mirrors page is currently showing http access as unavailable on ibiblio. (I assume that's what the red text means.) Is that page updated auto-magically?

Comment 7 Dave Hodgins 2011-06-26 23:01:12 CEST

How does urpmi pick which mirror to use?  Can it be changed to prefer a ftp
url over an http one, to avoid this type of problem?

Comment 8 Ahmad Samir 2011-06-26 23:43:21 CEST

@Darrel: kernel.org in the u.s. is a good mirror.

Comment 9 Darrel Johnston 2011-06-27 23:01:17 CEST

(In reply to comment #7)
> How does urpmi pick which mirror to use?  Can it be changed to prefer a ftp
> url over an http one, to avoid this type of problem?

Dave, 
 
From the net install live CD, it's a manual selection. You can choose either FTP 
or HTTP. Then you get a choice of mirrors. Again, that's a manual selection.

Comment 10 Darrel Johnston 2011-06-27 23:02:00 CEST

(In reply to comment #8)
> @Darrel: kernel.org in the u.s. is a good mirror.

Thanks, Ahmad. But, after belnet.be timed out several times, download time went 
up to 22 hours. I elected to abort and download the DVD. Total download time, 40 
minutes. Total installation time from DVD, about one hour.

Comment 11 Remco Rijnders 2011-08-27 11:26:27 CEST

Ran into the same issue just now using a net install (via http) of the mirror at mageia.webconquest.com. As that happens to be my own server, I'll see what I can learn about this.

Comment 12 Remco Rijnders 2011-08-27 11:57:18 CEST

Looks like the mirror servers having this problem all run lighttpd as webserver software. In most lighttpd.conf files is a line like this:

static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

Taking out the .pl part in that line and restarting the webserver makes this problem go away.

I think there is not much Mageia can do about this, other than perhaps informing (prospective) mirror operators of this issue.

Comment 13 Remco Rijnders 2011-08-27 12:38:05 CEST

The above change fixed the problem for mageia.webconquest.com. I've contacted the people at ibiblio and hope they will fix and revert, but if they don't, there is not much more we can do.

I've checked all the other http-mirrors currently listed and operational at http://mirrors.mageia.org/ and none of them have this issue.

Status: NEW => RESOLVED
Resolution: (none) => FIXED

Comment 14 Michael Scherer 2011-08-30 13:40:46 CEST

I guess we could focus our check specifically on this file, to see if the mirror work.

Comment 15 Remco Rijnders 2011-09-09 04:54:11 CEST

I received a response from the people at ibiblio, saying they fixed it... and they did :-)