I wonder if that's something we can easily fix. Users might have all kind of setup in urpmi.cfg, with some or all of the Mageia media, sometimes with duplicates, etc.
How would we identify media that we want to remove, and those that we don't? Grepping for "distrib/$initialversion" in the URL might work, but it's a bit of a dirty hack.

CC: (none) => remi

Dirty or not, it'd be better to delete too few than too many.

Why not simply set them to ignored instead of deleting them? 

Or at least inform the user that this is going to happen beforehand and that he needs to re-add (all/third party) media back (or re-enable them, depending on the outcome of this bug)?

If any development efforts are going to be spent on this, let's get a proper fix

(In reply to David Walser from comment #0)
> When upgrading using the installer, the sources in urpmi.cfg are removed,
> which for the previous distro version media makes sense,

Actually it makes sense for most repos.
Repos that are release agnostic are very rare (as in google chrome example as they ship most libs statically linked and only dynamically link to libs that are sure to be everywhere such as glibc)

(In reply to Florian Hubold from comment #3)
> Why not simply set them to ignored instead of deleting them? 

B/c we already have a huge list of media in urpm-edit-sources and it will just become unreadable

And adding a warning would just make the installer harder to understand to most end users...

AFAIC this is not a bug and should just be closed...

And let's do that

Status: NEW => RESOLVED
Resolution: (none) => WONTFIX

Thierry i totally agree -as i said during the meeting-, 'though during last night meeting people voted for making a backup of at least urpmi.cfg before deleting
all the medias.
see http://meetbot.mageia.org/mageia-dev/2015/mageia-dev.2015-02-05-19.39.html

The idea of only deleting media that use the Mageia GPG key was brought up in the meeting and sounds like the right solution.

Making assumptions about whether third party media that people have added is release-dependent makes no sense.  If it is, let the admin/user figure that out.  Don't just blindly delete everything.

Resolution: WONTFIX => (none)
Status: RESOLVED => REOPENED

It's not.
It may break urpmi...
Eg we can imagine a mga5 package has new deps that are also provided by a package from the repo but has non explicit file conflicts, ...

I've seen urpmi looping forever when upgrading b/c both N & N+1 media were set up.

On the other hand, the upgrade could fail b/c a package could not be removed b/c a 3rd party package has outdated dependencies no more backed in new media...

>On the other hand, the upgrade could fail b/c a package could not be removed b/c 
> a 3rd party package has outdated dependencies no more backed in new media...
But that also with the media removed.
I think the solution of a urpmi.cfg backup could be a good compromise at least for the incoming mga5

I think the discussion is not about keeping third-party repos *enabled* during the upgrade, but to restore them after the upgrade.
I don't know how it works currently, but maybe urpmi can use a "clean" urpmi.cfg file during the upgrade, and restore the previous one after the install (after having replaced the old official repos by the new ones too).

well maybe it's better to remove our repo to disable third party ones and
add our new medias from mirrors. 
At the final checkup after installation the urpm-edit-media view could
be provided to let the user enabling the repos back the ones he wanted to.
We cannot know which could be enabled and which not. I have 3 private repos
and google-chrome one if you enable all of them you will enable mga4 private repos. But you cannot know that just from the name...

(In reply to Angelo Naselli from comment #12)
> At the final checkup after installation the urpm-edit-media view could
> be provided to let the user enabling the repos back the ones he wanted to.

That sounds good. So the procedure would be:
- Remove Mageia N official repos
- Disable all other repos
- Add Mageia N+1 official repos
- Upgrade...
- After the upgrade, bring up urpm-edit-media and let the user do their job.

>- After the upgrade, bring up urpm-edit-media and let the user do their job.
or adding a "tab" for that configuration too, since there is a page with all the configuration to be checked

Thanks Angelo, that makes a lot of sense, especially when there's something remotely similar that allows you to check core vs. nonfree vs. tainted before the installation, having something that allows you to do the same with your non-native media afterward would probably not be too confusing.

Created attachment 5870 [details]
patch that saves a backup of urpmi.cfg

Just a guess since i gave a quick look to the code... and i cannot do any tests until next Thursday. But any comments is welcome so that i won't do any useless tests :)

Sorry, Angelo, I'm giving up on trying to test your patch.

I've created many drakx-installing-stage2 tarballs, but no matter what I do, after every upgrade install ddebug.log keeps saying that version 16.54 was used.

First I replaced the drakx-installer-stage2 rpm that was in my local mirror, with a copy of the drakx-installer-stage2 rpm that I built locally from the new tarball containing your patch. However, I should probably have done the following:

Then I copied new mdkinst.sqfs from the new package into /my/mageia/mirror/distrib/cauldron/x86_64/install/stage2/ instead
However, the VERSION file in the same directory as where mdkinst.sqfs came from, still said "16.54", no matter which crazy versions the tarball and the package had that I had made, so I don't know which version I was really testing :-(

Anyway, on not one of my numerous test I found a /etc/urpmi/urpmi.cfg.bak

The only mention I see of urpmi.cfg, is 

* wrote config file [/mnt/etc/urpmi/urpmi.cfg]
(after * starting step `choosePackages')

CC: (none) => marja11

Created attachment 5871 [details]
stage2 failing with error about urpmi.cfg.bak

Maybe my mistake was that I hadn't git added and committed the changed files :-/

Anyway, I tried again, now after adding and committing.

Package building failed on some localisation stuff, but I found a nice mdkinst.sqfs and VERSION (with correct version) in BUILDROOT, which I copied to my local mirror.

After that, boot-nonfree.iso install failed when loading stage2:

String found where operator expected at /usr/bin/libDrakX/install/media.pm line 896, near ""$::prefix/etc/urpmi/urpmi.cfg" "$::prefix/etc/urpmi/urpmi.cfg.bak""
     (Missing operator before "$::prefix/etc/urpmi/urpmi.cfg.bak"?)

See attached screenshot

marja sorry i was wrong in the patch (not tested as said)
eval { cp_af "$::prefix/etc/urpmi/urpmi.cfg", "$::prefix/etc/urpmi/urpmi.cfg.bak" };

I forgot the "," between two parameters... maybe

(In reply to Angelo Naselli from comment #16)
> Created attachment 5870 [details]
> patch that saves a backup of urpmi.cfg
> 
> Just a guess since i gave a quick look to the code... and i cannot do any
> tests until next Thursday. But any comments is welcome so that i won't do
> any useless tests :)

Seems like a reasonable change (except the missing comma!)

I still think that in an ideal world, we should do the following:

1. Take original urpmi.cfg and update any official media to be MGA(N+1) (basically delete all media with our key and add back in official media again.
2. Write that to disk
3. Bind mount a temporary file over the top of the written file.
4. Write nicer config to that tmp file which *only* has official media and thus not any unknown sources defined.

After we're done, the bind mount would be umounted and the user's config will be revealed again.

Either that or just make installer urpmi use a different config.

All that said, this is a bit beyond my perl skills so I think your solution is a good one for now!

(In reply to Thierry Vignaud from comment #5)
> 
> (In reply to Florian Hubold from comment #3)
> > Why not simply set them to ignored instead of deleting them? 
> 
> And adding a warning would just make the installer harder to understand to
> most end users...

It could simply be added to release notes, which endusers are expected to read, and which are already shown in the installer IIRC.

So the solution from Angelo or as proposed by Colin would totally suffice, to only use official media during installer and restore the other repos afterwards.

on Comment #20 the question is, does that work? :)

Colin I think we can fix this bug this way and open a new one after a discussion
during a meeting when mga5 is out. 
The way to take is clear, the way to implement could be chosen later, but imo
very soon after mga5 is out. This kind of changes has to be tested early :D

Sorry, Angelo, with the additional comma stage2 now starts fine, but I still don't have a urpmi.cfg.bak after reboot

@Marja that means that the patch was not good, it was a guess as i said. I will try to study better this issue if colin has not fixed yet.

Note added to https://wiki.mageia.org/en/Mageia_5_Release_Notes#Upgrading_from_Mageia_4

Severity lowered to enhancement, for Mageia 6, as it's too late to be
making changes to the installer, that are not for release blocker bugs.

Severity: normal => enhancement
CC: (none) => davidwhodgins

What about allow to configure third party repositories in the installer, of course giving a warning of do on user risk.

BTW we are on Mageia 9 and i no think that the change is implemented but maybe i'm wrong

Hardware: i586 => All

I agree with comments 5 and 9. Upgrading is difficult enough to get right which
is part of the reason it requires such thorough testing prior to release.

People are warned to remove the third party repos and the packages installed
from them in
https://wiki.mageia.org/en/How_to_choose_the_right_Mageia_upgrade_method#Preparations

Depending on where it fails, a failed upgrade may leave the system un-bootable,
or it may be left in a state where it's not possible to start a desktop
environment.

It's too risky in my opinion, not to remove the third party repos.

(In reply to Dave Hodgins from comment #27)
> I agree with comments 5 and 9. Upgrading is difficult enough to get right
> which
> is part of the reason it requires such thorough testing prior to release.
> 
> People are warned to remove the third party repos and the packages installed
> from them in
> https://wiki.mageia.org/en/
> How_to_choose_the_right_Mageia_upgrade_method#Preparations
> 
> Depending on where it fails, a failed upgrade may leave the system
> un-bootable,
> or it may be left in a state where it's not possible to start a desktop
> environment.
> 
> It's too risky in my opinion, not to remove the third party repos.

Well i migrate mageia 8 to mageia 9 in times of b1 with the blogdrake repositories for mageia 9 with urpmi and not find any issue, but is a small repo and is not a representative testing

So closing this as OLD

Status: REOPENED => RESOLVED
Resolution: (none) => OLD

Changing to WONTFIX as it sounds like the bug is still valid.  I guess the moral is don't use the installer to upgrade.

Resolution: OLD => WONTFIX

(In reply to David Walser from comment #29)
> Changing to WONTFIX as it sounds like the bug is still valid.  I guess the
> moral is don't use the installer to upgrade.

At less if you like/need software from sources external to mageia and you like to have some adventure ;)

The installer is good for upgrades without external sources

(In reply to katnatek from comment #30)
> (In reply to David Walser from comment #29)
> > Changing to WONTFIX as it sounds like the bug is still valid.  I guess the
> > moral is don't use the installer to upgrade.
> 
> At less if you like/need software from sources external to mageia and you
> like to have some adventure ;)
> 
> The installer is good for upgrades without external sources

And the installer is also a good way of reduce downloads if you now how to add to urpmi sources for the upgrade, specially if you have to upgrade more than one system