Bug 16002 - Add default sources at the end of the installation even if network is missing
Summary: Add default sources at the end of the installation even if network is missing
Status: REOPENED
Alias: None
Product: Mageia
Classification: Unclassified
Component: Installer (show other bugs)
Version: Cauldron
Hardware: All Linux
: High enhancement
Target Milestone: Mageia 6
Assignee: Mageia tools maintainers
QA Contact: Samuel Verschelde
URL:
Whiteboard:
Keywords: USABILITY
: 8351 (view as bug list)
Depends on: 17400 20131
Blocks:
  Show dependency treegraph
 
Reported: 2015-05-21 16:16 CEST by David Walser
Modified: 2017-01-17 10:29 CET (History)
12 users (show)

See Also:
Source RPM:
CVE:
Status comment:


Attachments

Description David Walser 2015-05-21 16:16:31 CEST
+++ This bug was initially created as a clone of Bug #8819 +++

In Bug 8819, users were given the opportunity to add online media to their urpmi configuration at the end of the installation, but only if they had an active network connection which they were able to use through the installer at that time.

Ideally, online media should be available without any further interaction from the user after installations done through the Classical Installer from ISOs or local sources like HDD installations.

It is not necessary to add them for network installations (HTTP, FTP, NFS), as it can be assumed in those cases that the remote source is an active mirror that will have updates.

All other major distros make online repositories automatically available after installation without further interaction, and without requiring an active network connection during the installation.  This makes things much easier for new users, and also reduces the burden on those who support new Mageia users, as the lack of configured repositories is one of the most common issues.

Requiring a network connection to set this up in the installer is a problem, because the network may not be available, or if it's wireless, not possible to use from the installer.
Comment 1 Thierry Vignaud 2015-05-25 12:40:04 CEST
There's no patch...
Comment 2 Samuel Verschelde 2015-05-26 14:53:55 CEST
Note: comment #1 was a reaction to the presence of the PATCH keyword (consequence of the cloning of bug 8819), now removed.
Comment 3 Manuel Hiebel 2016-07-15 11:30:47 CEST
dnf doesn't do that ?
Comment 4 Thomas Backlund 2016-07-15 11:35:11 CEST
dnf ships with preconfigured mirror metadata...

The same can be done for urpmi as soon as mirrorbrain is up
(currently waiting maat to do a DC trip and add needed storage)
Comment 5 Neal Gompa 2016-07-28 12:23:13 CEST
The only thing we don't have for DNF/PackageKit repository configuration is getting the user selections of repositories from the installer set for those. Right now, they are completely separate... See bug 18889 for details on that.

The current default as set by the mageia-repos package is that the core release and updates repository are enabled by default for the primary architecture, and everything else is disabled by default.

Since we allow users to select repositories during installation, it's possible for the user to have a configuration that differs from this. Aside from that, DNF and PackageKit are ready to go from the get-go.
Comment 6 Samuel Verschelde 2016-09-20 22:47:35 CEST
We had set this enhancement as release blocker for Mageia 6. I now need a status update: is someone working on it, or planning to? Is it doable in time for Mageia 6? 

Depending on the answers we'll either keep it or have to shed some tears and postpone to Mageia 7.
Comment 7 Samuel Verschelde 2016-10-10 17:17:25 CEST
*** Bug 8351 has been marked as a duplicate of this bug. ***
Comment 8 Nicolas Lécureuil 2016-11-06 15:09:36 CET
i think we should focus on blocker issues and postpone this one for mga7.
Comment 9 David Walser 2016-11-06 15:12:24 CET
We have postponed this one long enough.  This is the number one issue that causes new users to reach out for support.  Every other major distro has this fixed.  There is no excuse for continuing to kick this can further down the road.
Comment 10 Neal Gompa 2016-11-06 15:22:18 CET
Then it needs to be a priority to set up MirrorBrain or MirrorManager2 for this. We need reliable mirror selection for the experience to be solid.

Even now, though DNF provides default sources using our current mirror list system so that it's ready to go right after installation, the lack of mirror verification causes it to fail randomly because DNF has been given a series of bad mirrors, though the frequency of failures will vary depending on the region of the world you're in.

If we want to provide an excellent experience, we *need* a proper mirror management tool. Either MirrorBrain or MirrorManager 2. If we're having issues with MirrorBrain, I can very easily make MirrorManager 2 available in Cauldron and backport it to infra_5 (that was the original plan, after all).
Comment 11 Samuel Verschelde 2016-11-08 11:34:12 CET
I think that Nicolas is right that it's too late for Mageia 6, although I would really have liked to have it done at last. We can deplore it but it's a fact, especially since it involves infrastructure changes. We want Mageia 6 released ASAP and that means fix the remaining bugs and release it. I had asked for a status update almost 50 days ago, now is too late. No more feature can be added at this stage. 

However, we're going to try harder to define the priorities for Mageia 7 right from the beginning of the development, by using the Milestone, priority and improved views of bug and enhancement reports  in bugzilla. Setting milestone and priority accordingly and Mageia 7 will be the one that sees this implemented.
Comment 12 David Walser 2016-11-08 13:20:22 CET
This is ridiculous.  This is the easiest bug to fix.  Even without the MirrorBrain stuff, we know what a urpmi.cfg with MirrorList sources looks like.  It would be trivial to install that by default in a new installation.  If we're not going to fix this now, we never will, so we might as well close this as won't fix and tell new users that ask about this to try another distribution.
Comment 13 Thomas Backlund 2016-11-08 13:50:55 CET
Mirrorbrain is still waiting for storage :/

pterjan and maat are going to DC iirc before end of this month to do the needed maintenance ...
Comment 14 Samuel Verschelde 2016-11-08 14:20:36 CET
(In reply to David Walser from comment #12)
> This is ridiculous.  This is the easiest bug to fix.  Even without the
> MirrorBrain stuff, we know what a urpmi.cfg with MirrorList sources looks
> like.  It would be trivial to install that by default in a new installation.
> If we're not going to fix this now, we never will, so we might as well close
> this as won't fix and tell new users that ask about this to try another
> distribution.

Pfffff
Comment 15 Stephen Butler 2016-11-09 05:38:49 CET
I think what you are trying to do is have the repositories configured after the installation is done for installing or updating packages. This is really easy to do. 

The first thing is deciding how to do this and adding more software takes more space. You need to get creative and have a little fun with this. 

When deciding you need to look for your options.
How you do this is ask questions as how can we accomplish this task is the key to finding options. While I make software work they way it needs to I found their is several ways that I can make it work. The real problem was what was the best option. 

What I understand is you want the install and update media setup for end user.
My options from being creative not in any order:

Option 1
include all mirrors on ISO divided by country then when they choose the country their in that set of mirrors gets installed.

Option 2
Specify primary repository URL for mirror list when the update program starts up or software installer current mirror list is downloaded from the primary repository URL and from that list is then installed as the repository media.

From looking at my options here is how it will work.

Using both options.

Setup urpmi.cfg etc.. for the separate countries ( all users select their country right.) and store country info. The end of the setup process you get stored country info and install urpmi.cfg for user configured country this will be setup for end user at user first boot up or first login. Then when the system is first logged in ask user if they would like to get updates. When the program installer or updater starts then get updated mirror list from a primary URL that matches their country will always be up. This can be fully automated just once or periodically. 

By doing this you can eliminate dead mirrors.
Comment 16 Thierry Vignaud 2017-01-16 10:37:16 CET
If the user refuses to apply the updates, we can offer to add the urpmi media anywany, witouth installing the updates.
But only if there's network of course, like for updates...
Comment 17 Thierry Vignaud 2017-01-16 10:41:57 CET
The best thing would be for urpmi to reuse libdnf in mga7
Comment 18 Samuel Verschelde 2017-01-16 10:50:09 CET
(In reply to Thierry Vignaud from comment #16)
> If the user refuses to apply the updates, we can offer to add the urpmi
> media anywany, witouth installing the updates.
> But only if there's network of course, like for updates...

To elaborate on that, it's not enough as of today to add mirrorlist in urpmi.cfg. It needs an active network connection.

(In reply to David Walser from comment #12)
> This is ridiculous.  This is the easiest bug to fix.  Even without the
> MirrorBrain stuff, we know what a urpmi.cfg with MirrorList sources looks
> like.  It would be trivial to install that by default in a new installation.
> If we're not going to fix this now, we never will, so we might as well close
> this as won't fix and tell new users that ask about this to try another
> distribution.

Based on the above, it appears that it's not that trivial. I'll have to face your disappointement, David, but I really think we'll have to wait for mirrorbrain or have urpmi use libdnf in Mageia 7. Sorry for that.
Comment 19 Samuel Verschelde 2017-01-16 10:51:40 CET
(In reply to Samuel Verschelde from comment #18)
> (In reply to Thierry Vignaud from comment #16)
> > If the user refuses to apply the updates, we can offer to add the urpmi
> > media anywany, witouth installing the updates.
> > But only if there's network of course, like for updates...
> 
> To elaborate on that, it's not enough as of today to add mirrorlist in
> urpmi.cfg. It needs an active network connection.
> 
> (In reply to David Walser from comment #12)
> > This is ridiculous.  This is the easiest bug to fix.  Even without the
> > MirrorBrain stuff, we know what a urpmi.cfg with MirrorList sources looks
> > like.  It would be trivial to install that by default in a new installation.
> > If we're not going to fix this now, we never will, so we might as well close
> > this as won't fix and tell new users that ask about this to try another
> > distribution.
> 
> Based on the above, it appears that it's not that trivial. I'll have to face
> your disappointement, David, but I really think we'll have to wait for
> mirrorbrain or have urpmi use libdnf in Mageia 7. Sorry for that.

Oops, I was wrong. It does work to add mirrorlist to urpmi.cfg if we do urpmi.update at first boot. Retargeting.
Comment 20 Nicolas Lécureuil 2017-01-16 10:52:21 CET
and can we disable DVD Sources when the online updates are enabled ?
Comment 21 Thierry Vignaud 2017-01-16 10:53:58 CET
Actually it'll work for a poweruser (a command line user that would run urpmi.update -a)
However for the generic user, drakrpm won't update the non update media and thus updates with new deps in eg core/release wouldn't be solvable.
Thus if the user refused the updates at end of install, we still have to rely on drakrpm to add media after...
Comment 22 Thierry Vignaud 2017-01-16 10:56:23 CET
(In reply to Nicolas Lécureuil from comment #20)
> and can we disable DVD Sources when the online updates are enabled ?

That's another think, that would be bug #19742. We could probably do that.
Comment 23 Samuel Verschelde 2017-01-16 10:59:35 CET
(In reply to Thierry Vignaud from comment #21)
> Actually it'll work for a poweruser (a command line user that would run
> urpmi.update -a)
> However for the generic user, drakrpm won't update the non update media and
> thus updates with new deps in eg core/release wouldn't be solvable.
> Thus if the user refused the updates at end of install, we still have to
> rely on drakrpm to add media after...

True, re-retargeting at Mageia 7.
Comment 24 David Walser 2017-01-16 15:00:39 CET
Still wrong here.  It is trivial.  For what gets added to urpmi.cfg when you set up mirrorlist sources, a network connection is *not* required, as it is static information.  A connection is only required if you are choosing an individual mirror, as that information needs to be downloaded.  Also, I believe our sysadmins still intend to have MirrorBrain implemented by the release of Mageia 6.  So, either way, there's still no excuse for kicking this can further down the road.
Comment 25 Thierry Vignaud 2017-01-16 15:14:27 CET
Seriously?
Please read comment #21 ...
Comment 26 David Walser 2017-01-16 15:16:51 CET
(In reply to Thierry Vignaud from comment #25)
> Seriously?
> Please read comment #21 ...

Seriously?  If that's true, that's also something that can be fixed.
Comment 27 Nicolas Lécureuil 2017-01-16 15:18:49 CET
can't force the use of api.mageia.org  ?  the conf file to add would be a template.

maybe i am wrong
Comment 28 David Walser 2017-01-16 15:20:49 CET
MIRRORLIST doesn't use api.mageia.org until you actually try to update the media information, but it doesn't need to contact it for what it puts in urpmi.cfg, as that's static.  Try it and look at urpmi.cfg.
Comment 29 David Walser 2017-01-16 15:25:25 CET
Guys, every major distro has this issue solved except for Mageia.  It causes headaches and confusion for a large number of new users and we see it all the time.  I don't see how anyone can in good conscious continue to recommend Mageia to new users when we have artificial stumbling blocks in place like this if we won't be there to hold their hand through working around them.

Please let's keep our eye on the ball and find *some* way to fix this.  Again, my understanding was that MirrorBrain was going to be done and make much of this discussion irrelevant.  Either way, this can be overcome.
Comment 30 Thierry Vignaud 2017-01-16 15:28:25 CET
Please read comment #21 ...
You've not tested your "trivial" fix.
I've.
It won't work as non update media (eg: core/release) will never be updated by GUI and thus deps won't be resolvable for pkgs from core/updates

And this cannot be a release blocker as this has always been like this.

If someone nacks the updates and thus the update media, that's his problem
Comment 31 David Walser 2017-01-16 15:29:43 CET
So let's be honest here.  You have no intention of fixing this.  You just keep making excuses.
Comment 32 Thierry Vignaud 2017-01-16 15:31:55 CET
No comment...
Comment 33 Samuel Verschelde 2017-01-16 15:43:50 CET
(In reply to David Walser from comment #31)
> So let's be honest here.  You have no intention of fixing this.  You just
> keep making excuses.

I did discuss with Thierry about this on the phone before we demoted the bug report and we genuinely thought about possible solutions. I think you owe someone an apology.

The solution is not trivial as we thought. It is not a blocker anymore, for the following reasons: not that high a severity (a missing feature, and actually only for those without working network during installation), and not trivial to fix.

It does qualify for high priority + Milestone Mageia 6 though (and I do plan to review that list too with the Mageia Tools maintainers), but we will not hold the release for this bug report, will we?

Or maybe you'll tell me too that I'm just making excuses?
Comment 34 David Walser 2017-01-16 15:47:44 CET
All I can say is this.  The whole reason that I filed this bug was to have this fixed for Mageia 6.  This issue has persisted for far too long.  I hope we still can/will fix it for Mageia 6.  I've done all I can to try to encourage that to happen, as I can't fix it myself.  Obviously I disagree that it's not trivial in theory, but we're going around in circles here.

If we're not going to fix it for Mageia 6, you can close this bug and track it for Mageia 7 in Neal's Bug 20131.

If you plan to still review this with the Mageia Tools group before the Mageia 6 release, then yes please keep this open for now.
Comment 35 Thomas Backlund 2017-01-16 22:03:22 CET
(In reply to Thierry Vignaud from comment #30)
> Please read comment #21 ...
> You've not tested your "trivial" fix.
> I've.
> It won't work as non update media (eg: core/release) will never be updated
> by GUI and thus deps won't be resolvable for pkgs from core/updates


What about using some simple lockfile, something like /etc/sysconfig/first_update


and when gui update runs first time it checks for lockfile, runs without "--update", it runs full media update, if that succeeds, nukes lockfile, and after that only runs with "--update" ...

Would that be doable...

As for my todo... I'm hoping to be able to do some infra work this weekend to fix up that end...

Note You need to log in before you can comment on or make changes to this bug.