Bug 11247

Summary: urpmi No longer installes from local archieve
Product: Mageia Reporter: Robert Courtright <rbcourt>
Component: RPM PackagesAssignee: Thierry Vignaud <thierry.vignaud>
Status: RESOLVED INVALID QA Contact:
Severity: major    
Priority: Normal CC: andr999, jani.valimaa, zen25000
Version: Cauldron   
Target Milestone: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Source RPM: urpmi-7.27.3-1.mga4.noarch.rpm CVE:
Status comment:
Attachments: urpmi.cfg file for Mageia 3
urpmi.cfg file for Mageia 4
This is using the gui to add the local directory

Description Robert Courtright 2013-09-17 20:13:53 CEST
Description of problem:urpmi No longer installes from local archieve


Version-Release number of selected component (if applicable):
urpmi-7.27.3-1.mga4.noarch.rpm

How reproducible:
Every time

Steps to Reproduce:
1. Create or mirror one of multiple mirrors from the internet
2. Set source to use local mirrors over nfs
3. do an update.

 At this point even through local Mirrors are up to date it still downloads them from the remote mirrors.

This is not how it worked in Mageia3 
I do a lot on installs family, friends, It's faster to update from a local drive rather than on line.


Reproducible: 

Steps to Reproduce:
Comment 1 Barry Jackson 2013-09-17 20:33:29 CEST
So you still have remote media enabled?
Why?

CC: (none) => zen25000

Comment 2 Robert Courtright 2013-09-18 01:34:18 CEST
Because the mirror I may choose to sync up to may be more out of date then the remote one.

The bulk of them come from my local one. As in the case of Mageia 3 there are over 500 updates. I would rather download 5 or 6 insted of over 500.
Comment 3 Jani Välimaa 2013-09-18 15:38:21 CEST
Are these local repos before other/remote repos. IIRC urpmi uses first available media to dl pkgs.

You can use 'drakrpm-edit-media' to change/check the order.

CC: (none) => jani.valimaa

Comment 4 Robert Courtright 2013-09-18 20:26:54 CEST
after using drakrpm-edit-media this is what I found.

urpmi.addmedia doesn't even add the repository.
drakrpm-edit-media adds it but wont use it.
Comment 5 Barry Jackson 2013-09-18 20:41:49 CEST
This may be confusing the issue:

https://bugs.mageia.org/show_bug.cgi?id=7677

In practice to work around this bug, I add and remove media using drakrpm-edit-media, however to use local additional media I only use urpmi from command line.
Comment 6 Robert Courtright 2013-09-19 22:24:32 CEST
Thats what I do and have been doing it that way for years Since the Mandrake days. It's just that in Mageia 4 it don't work as it always has.
Comment 7 Barry Jackson 2013-09-19 23:46:51 CEST
I think a more detailed explanation of exactly what you are doing to reproduce this is required. Plus:-

1. How the result differs from that in Mga3.
2. Attach cat /etc/urpmi/urpmi.cfg
3. Attach "urpmi --debug foo" output for a simple case that can be reproduced
4. Anything else that you feel will help 

"It don't work" is not really useful in a bug report ;)
Comment 8 Robert Courtright 2013-09-20 05:32:52 CEST
Created attachment 4363 [details]
urpmi.cfg file for Mageia 3
Comment 9 Robert Courtright 2013-09-20 05:33:25 CEST
Created attachment 4364 [details]
urpmi.cfg file for Mageia 4
Comment 10 Robert Courtright 2013-09-20 05:54:59 CEST
Created attachment 4365 [details]
This is using the gui to add the local directory
Comment 11 Robert Courtright 2013-09-20 06:05:03 CEST
In Mga3 I run urpmi.addmedia --distrib /Mirrors/Mageia/3-1/x86_64
In Mga4 I run urpmi.addmedia --distrib /Mirrors/Mageia/c-1/x86_64
I get no error on ethier of these


In Mga3 It uses the local directory than the remote ftp sites
In Mga4 It only uses the remote sites

From the 3 files I sent you can see the the difference between the 2 no-gui ones
and the one labeled gui

It makes no difference the 2 Mga4 both act the same.

When you asked Attach "urpmi --debug foo" what does the foo represent?

I sync my 2 copies of each Mageia rev from different ftp sites:

Mga3 is rsync -avr rsync://ftp.fi.muni.cz/pub/linux/mageia/distrib/3/ /Mirrors/Mageia/3-1/

and Mga4 is 

rsync -avr rsync://ftp.jaist.ac.jp/pub/Linux/Mageia/distrib/cauldron/ /Mirrors/Mageia/c-1/
These are synced up every 6 hours.
Barry Jackson 2013-09-20 10:09:27 CEST

Attachment 4363 mime type: application/octet-stream => text/plain

Barry Jackson 2013-09-20 10:24:17 CEST

Attachment 4364 mime type: application/octet-stream => text/plain

Barry Jackson 2013-09-20 10:30:44 CEST

Attachment 4365 mime type: application/octet-stream => text/plain

Comment 12 Barry Jackson 2013-09-20 15:21:59 CEST
(In reply to Robert  Courtright from comment #11)

So this is really two separate bugs:

Bug 1 - urpmi.addmedia fails to install local media, but drakrpm-edit-media (GUI) works.
(May be related to https://bugs.mageia.org/show_bug.cgi?id=10018)

Bug 2 - urpmi fails to use local media when remote media is also enabled
This should be split into a separate bug report IMO.

> In Mga3 I run urpmi.addmedia --distrib /Mirrors/Mageia/3-1/x86_64
> In Mga4 I run urpmi.addmedia --distrib /Mirrors/Mageia/c-1/x86_64
> I get no error on ethier of these
>

Yet it fails totally, so it may be useful to try:
urpmi.addmedia -v --interactive --distrib /Mirrors/Mageia/c-1/x86_64

> 
> In Mga3 It uses the local directory than the remote ftp sites
Again it may be useful to run urpmi --debug packagename to see why it selects the local.

> In Mga4 It only uses the remote sites

> 
> From the 3 files I sent you can see the the difference between the 2 no-gui
> ones
> and the one labeled gui

Yes no media is insalled from the cauldron command line.

> 
> It makes no difference the 2 Mga4 both act the same.

OK

> 
> When you asked Attach "urpmi --debug foo" what does the foo represent?

foo is a package name that you have in your local repo that you wish to install. The --debug output from urpmi may give a clue as to why it selects the remote media rather than local.

> 
> I sync my 2 copies of each Mageia rev from different ftp sites:
> 
> Mga3 is rsync -avr rsync://ftp.fi.muni.cz/pub/linux/mageia/distrib/3/
> /Mirrors/Mageia/3-1/
> 
> and Mga4 is 
> 
> rsync -avr rsync://ftp.jaist.ac.jp/pub/Linux/Mageia/distrib/cauldron/
> /Mirrors/Mageia/c-1/
> These are synced up every 6 hours.

Fine for 3 but a lot can change in 6 hours in cauldron, so updates would tend to be picked up from remote unless your mirror had just sync'd.
Comment 13 Robert Courtright 2013-09-20 15:49:55 CEST
I updated this morning and all of the rpms updated were installed from the ftp and not local.
I then check each rpm and my local had all of them.
Comment 14 Manuel Hiebel 2013-09-24 17:31:14 CEST
I don't understand why you use mirrorlist, c3sl.ufpr.br, muni.cz, and your local mirror. 
Use one mirror (like the brazilian which is updated several times per hours) or mirrorlist but not several one...

(this is slowering urpmi too)
Comment 15 Robert Courtright 2013-09-25 04:02:51 CEST
As stated earily when I installl a system for someone and then have to install over 500 updates your talking hours just for the downloads. From my local copy it's less than an hour. I've been using this method since Mandrake days. 
How is loading from a remote mirror faster than a local copy for Mageia 1, 2 or 3?
I found the bug in cauldron/Mageia 4 and reported it.
Comment 16 Manuel Hiebel 2013-09-25 10:38:57 CEST
this doesn't explain why you need 3 remote mirrors
Comment 17 Robert Courtright 2013-10-24 18:11:16 CEST
They don't all update at the same time. Most of the time I don't but If I have someone in a hurry and they have ethier don't an internet connection or very slow. I can get the most updates as quickly as possable. 

Last night I installed Mageia 4 alpha 3 on the test partion of my file server and found it had over 1500 updates. Since it wont update from my copy of the mirrors it took hours to update.
Comment 18 andré blais 2014-01-02 13:12:20 CET
a workaround could be to
1) initially update from your local sources, without connection to remote mirrors,
then
2) connect to remote mirrors and update again.

This will guarantee not using a remote source for updates existing on your local source.
Packages with local updates and more recent remote updates will be updated twice, but it could still be faster since the remote updates will be only considered on the second pass.

So even for mga3, this might be a good strategy.  (I do that myself on new installs.)

Also, as suggested by Manuel, it would be faster to only use one remote source, rather than 3.  Each remote source is queried.  It takes time to pass the info of hundreds (or thousands) of updates available.  Just choose a mirror that is frequently updated.

CC: (none) => andre999mga

Comment 19 Robert Courtright 2014-01-02 13:36:44 CET
This was the default for 3 it would always update from the local and then move on to the remots. I just tried it. After my local repros were up to date I tried to update my system again. With it pointing only to my local no updates. Point to the remote more than 50. This is really impacting my installs. It can take hours on a slow connection. Not to mention the rpm's that I have that can't be found on the mirrors. I have to install them all by hand.
Comment 20 andré blais 2014-01-02 14:49:12 CET
(In reply to Robert  Courtright from comment #19)
> This was the default for 3

I understood, and agree that it is the default.

> ... After my local repros were up to
> date I tried to update my system again. With it pointing only to my local no
> updates. Point to the remote more than 50.

OK that looks like a serious problem.  It definitely shouldn't try to install an update already present.
A (somewhat awkward) workaround for that would be to use rpmdrake to see the updates it says are available (in "show all updates" mode), note these packages (in a temporary file or on paper),
then switch to "show all packages" mode, and with the search function verify each package.
You might try using rpmdrake for updates, to see if maybe only urpmi is impacted.

> It can take hours on a slow connection.
Understood.

> ... Not to mention the rpm's
> that I have that can't be found on the mirrors. I have to install them all
> by hand.

For local rpms you just have to define and activate a local repo.  (Easiest via rpmdrake).  These rpms don't have to be anywhere else.  Essentially the same as locally stored updates.
Comment 21 Robert Courtright 2014-01-02 21:44:36 CET
> For local rpms you just have to define and activate a local repo.  (Easiest via > rpmdrake).  These rpms don't have to be anywhere else.  Essentially the same as > locally stored updates.

Thried it. I don't work either.
Comment 22 Robert Courtright 2014-02-02 23:01:50 CET
Just installed the final release 4.
Still has the problem. Now all 20 rpms that arn't available from Mageia have to installed 1 by 1.
Comment 23 Manuel Hiebel 2014-02-02 23:09:26 CET
urpmi ./yourfolder/*.rpm should works fine
Comment 24 Robert Courtright 2014-02-03 03:18:10 CET
Manuel:

 Please read my comment 15 and you should understand the problem better.
Comment 25 Manuel Hiebel 2014-02-03 16:39:58 CET
well let see that with maintainer, personnaly no I still not understand why you use three mirrors set.

Assignee: bugsquad => thierry.vignaud

Comment 26 Robert Courtright 2014-02-03 17:22:07 CET
1 for the previous release Udated daily at midnight
2 for the current release. Mirrors update at different times & if one is down at time of update. updates twice daily
and 1 for cauldron Udated daily at midnight
Comment 27 Thierry Vignaud 2014-02-03 17:55:41 CET
I do exactly that without any problem.
Your issue is that in mga4, you ordered local mirrors after distant ones.
urpmi picks packages from the first medium that provide them.

Status: NEW => RESOLVED
Resolution: (none) => INVALID

Comment 28 Robert Courtright 2014-02-03 18:14:21 CET
I allways set up my locals first then the remotes.
In Mageia 3 it made no difference it always used local first.

It's acting different in Mageia 4 than it did in 1, 2 or 3.
And wont use my copy of the mirrors to update.
Comment 29 Thierry Vignaud 2014-02-03 21:37:43 CET
The logic has always been this.
Just compare your mga3 & mga4 config files...
Comment 30 Robert Courtright 2014-02-09 19:00:58 CET
Ok just run the updates and tried again.
Were getting better. It installs from my local copy of a mirrors and then it downloads the same rpms from the remote Mirror and then nothing.