Bug 1153 - Import mdkonline and adapt for easy upgrades / updates
Summary: Import mdkonline and adapt for easy upgrades / updates
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: Cauldron
Hardware: i586 Linux
Priority: release_blocker normal
Target Milestone: Mageia 1
Assignee: Anne Nicolas
QA Contact:
URL:
Whiteboard:
Keywords:
: 886 (view as bug list)
Depends on: 1432 1457 1458
Blocks:
  Show dependency treegraph
 
Reported: 2011-05-05 11:21 CEST by Anne Nicolas
Modified: 2014-05-08 18:04 CEST (History)
16 users (show)

See Also:
Source RPM:
CVE:
Status comment:


Attachments
choose Free / Powerpack (33.27 KB, image/png)
2011-05-23 22:42 CEST, Anne Nicolas
Details
untested patch to disable the powerpack dialog (724 bytes, patch)
2011-05-26 13:55 CEST, Michael Scherer
Details | Diff

Description Anne Nicolas 2011-05-05 11:21:25 CEST
This is a bug report to follow mdkonline import, cleaning and adapt to Mageia.

Thierry will do the cleaning part and adapt code for Mageia
Romain will manage server side.
Romain d'Alverny 2011-05-05 11:24:32 CEST

Target Milestone: --- => Mageia 1

Comment 1 Romain d'Alverny 2011-05-05 17:59:52 CEST
Reminding the scope of this tool:
 * checking, notifying of and listing available packages updates;
 * checking, notifying of available system upgrade;
 * triggering update or upgrade process on user request.

Uses:
 * releases API, publishing available system releases
 * mirrors API, publishing available mirrors

This was used at Mandriva before, we're changing a few bits:
 * mirrors API:
    - now at http://mirrors.mageia.org/api/{name}.{version}.i586.list (http://mirrors.mageia.org/api/mageia.cauldron.i586.list for cauldron, http://mirrors.mageia.org/api/mageia.1.i586.list for first stable release)
    - contents format & semantics don't change
 * releases API:
    - now at http://releases.mageia.org/api/a/{arch}?product={name}.{version} and should be called with the current system name and version as in: http://releases.mageia.org/api/a/i586?product=mageia.1
    - contents format & semantics don't change so far, but we could maybe simplify a bit (there are some constraints that we don't have anymore)
Comment 2 Ahmad Samir 2011-05-05 18:07:33 CEST
(FWIW, if possible, drop the mdk/mga from the name altogether and pick a generic name, this makes for easier adaptation by e.g. other distros (forks).. ).
Comment 3 Romain d'Alverny 2011-05-05 18:15:49 CEST
Yep. But finding a new name is somehow a curse too. The "online" suffix here is to be dropped as well, it's nowhere significant/related to the task of this module (checking, notifying, listing update/upgrade options and asking for action). At an extreme extent, we should maybe even see if splitting the software packages updates and system upgrade checks shouldn't be two separate modules.

Suggestions: softwareupdate, swupdate, checkupdate, grow, growth, newupdates, whatsupsys (like that one)
Comment 4 Dave Hodgins 2011-05-05 20:36:36 CEST
How about update-auto-checker?

CC: (none) => davidwhodgins

Comment 5 Anne Nicolas 2011-05-05 20:38:53 CEST
make it short please :)
Comment 6 Lucien XU 2011-05-06 22:26:57 CEST
systemupdater ?

CC: (none) => sfietkonstantin

Comment 7 John Balcaen 2011-05-07 02:37:44 CEST
the release api does not seems to work for x86_64 arch.

CC: (none) => balcaen.john

Comment 8 Anne Nicolas 2011-05-12 09:15:54 CEST
Any update on that ? Romain, Thierry ?
Comment 9 Romain d'Alverny 2011-05-12 10:56:39 CEST
(In reply to comment #7)
> the release api does not seems to work for x86_64 arch.

Fixed it seems.

(In reply to comment #8)
> Any update on that ? Romain, Thierry ?

It's ready server-side (modulo maybe we should add there existing Mandriva 2010.x releases info? I don't know, it depends on the use case).

No news client-side. Thierry?
Comment 10 Sander Lepik 2011-05-12 15:15:45 CEST
name suggestion: updateD/updated where last d stands for daemon

or longer: update-daemon, update-notifier

CC: (none) => sander.lepik

Comment 11 Thierry Vignaud 2011-05-13 15:58:01 CEST
(In reply to comment #9)
> No news client-side. Thierry?

Sorry, I need to git-svn-dcommit all my changes tonight.

A few remarks though:

For compatibility, it would have maybe been better to keep the
same type of URI as mdv so that only the vendor has to changed

Aka we do not include anymore the product type nor the ".list" suffix in the URI
eg:
"https://api.mandriva.com/distributions/$extra_path$type.$product_id->{arch}.list?product=$product_id->{product}"
vs:
"http://releases.mageia.org/api/a/$extra_path$product_id->{arch}?product=$product_id->{product}"

Dunno if we'll have several products (maybe some flash image to dump?).

Also we may consider using https like used to do back at mdv...
Comment 12 Thierry Vignaud 2011-05-13 20:26:27 CEST
I pushed initial client into Mageia
Comment 13 Anne Nicolas 2011-05-13 23:43:44 CEST
Could you please import it in soft/ also ?
Comment 14 D Morgan 2011-05-14 02:22:26 CEST
i just imported it in soft/

CC: (none) => dmorganec

Comment 15 Anne Nicolas 2011-05-14 09:35:50 CEST
just saw. The point is we cannot test it for now as we need to tag release repos as updates. Anybody to do this ?

CC: (none) => misc

Comment 16 James Kerr 2011-05-14 11:03:01 CEST
There seems to be a problem.

The red icon (and the notification) appear indicating that updates are available, and when I select "Install updates", the icon turns to orange and the pop-up says checking for updates. However, instead of launching mageiaupdate it simply returns to displaying the red icon.

"urpmi --auto-update --update" shows a list of packages available for update

How I tested:

Disabled online sources
Set up Beta1 iso as a source
Installed task-xfce-minimal
Re-enabled online sources, tagged as update sources and installed mgaonline.
Re-booted

There is also no GUI in drakconf for setting updates frequency. Is this a separate package?
Comment 17 James Kerr 2011-05-14 11:15:46 CEST
One additional point. Mageiaupdate does launch properly from within drakconf and displays the list of updates. (I have not installed these updates, so that I can test any newer version of mgaonline.)
Comment 18 Marek Laane 2011-05-14 11:18:57 CEST
One more thought: as it is quite visible tool, how are translations managed (I assume there will be some modifications? E.g. by now one can see there "Mageia Powerpack" and other nonsense phrases)? Will there be an exception to the freeze?
Comment 19 James Kerr 2011-05-14 12:06:35 CEST
Regarding the GUI: mgaapplet-config does run from a terminal, it just fails to appear in drakconf. (The references to powerpack and restricted have been removed in the English version.)
Comment 20 Marek Laane 2011-05-14 12:16:30 CEST
Well, at least POT (http://svnweb.mageia.org/soft/mgaonline/trunk/po/mdkonline.pot?revision=1310&view=markup) still includes such references and also source (http://svnweb.mageia.org/soft/mgaonline/trunk/mgaonline.pm?revision=1310&view=markup), though it seems POT is just taken over not regenerated, as seen by references to the source (mdkonline.pm versus mgaonline.pm).

CC: (none) => bald

Comment 21 James Kerr 2011-05-14 12:18:55 CEST
I think that I probably should have said that the references to powerpack and restricted to do not appear, because I'm obviously not running a powerpack version  :)
Comment 22 Anne Nicolas 2011-05-14 14:40:20 CEST
fixed typo and it's working ok now. Needs strong tests for upgrade from Mandriva. Needs to add Mandriva releases on server side
Comment 23 James Kerr 2011-05-14 15:56:50 CEST
Confirmed that it's working fine now. (There is still no link to mgaapplet-config in drakconf, but that's not critical.)

To test the distro-upgrade feature is it sufficient to install mgaonline on a Mandriva 2010.1 system?

Am I correct in understanding that we need to wait for server side changes before trying this test?
Comment 24 Ahmad Samir 2011-05-14 21:24:49 CEST
(Not a new package request any more).

(In reply to comment #23)
> Confirmed that it's working fine now. (There is still no link to
> mgaapplet-config in drakconf, but that's not critical.)
> 

Fixed in SVN.

Component: New RPM package request => RPM Packages

Comment 25 Thierry Vignaud 2011-05-15 11:20:44 CEST
BTW, if we want to support older mdv releases, we would need to support %mdkversion in our BS...
else mgaonline will require too new urpmi
Comment 26 Nicolas Vigier 2011-05-15 13:06:40 CEST
(In reply to comment #25)
> BTW, if we want to support older mdv releases, we would need to support
> %mdkversion in our BS...
> else mgaonline will require too new urpmi

Why ?

CC: (none) => boklm

Comment 27 Nicolas Vigier 2011-05-15 13:55:03 CEST
(In reply to comment #11)
>
> Also we may consider using https like used to do back at mdv...

https is now available on releases.mageia.org
Comment 28 Romain d'Alverny 2011-05-15 22:58:57 CEST
(In reply to comment #22)
> fixed typo and it's working ok now. Needs strong tests for upgrade from
> Mandriva. Needs to add Mandriva releases on server side

Done: https://releases.mageia.org/api/a/i586?product=mageia.1

(In reply to comment #27)
> (In reply to comment #11)
> >
> > Also we may consider using https like used to do back at mdv...
> 
> https is now available on releases.mageia.org

For the record, we need to know why it was published over HTTPS (ensuring the document was really published by a known certificate belonging to MDV? or else?).
Comment 29 Marek Laane 2011-05-16 02:16:23 CEST
Well, I'm very sorry to ask again somehow OT question but as I see there is at least 7 fuzzy strings now in translation file (I looked just at my language, Estonian http://svnweb.mageia.org/soft/mgaonline/trunk/po/et.po?revision=1333&view=markup ). So should we de-fuzzy them now (what means the freeze is extended temporarily at least for this package) and are these corrections included in the release then or would mgaonline appear just partially translated anyway? In any case it should be noticed in i18n list, too.
Comment 30 Thierry Vignaud 2011-05-16 14:27:01 CEST
(In reply to comment #28)
> For the record, we need to know why it was published over HTTPS (ensuring the
> document was really published by a known certificate belonging to MDV? or
> else?).

It was done by cfergeau on September 1 2009 (20 months ago):
"use https to grab mirrorlist from api.mandriva.com"
http://svn.mandriva.com/cgi-bin/viewvc.cgi/soft/mdkonline/trunk/mdkapplet?r1=258569&r2=259833&sortby=date

He also enabled the strict-certificate-check option btw (which is a nono as it's enabled by default in urpmi, probably just to be bullet proof against any API change) which makes sure curl/aria2/wget checks the CA.

It was then released by alefebvre on Sept 8 2009 in 2.77.5.

CC: (none) => cfergeau

Comment 31 Thierry Vignaud 2011-05-16 14:53:17 CEST
(In reply to comment #26)
> > BTW, if we want to support older mdv releases, we would need to support
> > %mdkversion in our BS...
> > else mgaonline will require too new urpmi
> 
> Why ?

MDV wanted to enable new features in older releases (extended support, detection of & upgrade to newer releases, ...).
Each of those older release had a different perl-URPM/urpmi (some did shared the same branch) (b/c pixel greatly improved urpmi by this time):
- 2008.1: urpmi-4.10.14 (4.10 branch)
- 2008.1: urpmi-5.19.x  (5.19 branch)
- 2009.0: urpmi-6.14.X  (6.14 branch)
- 2009.1: urpmi-6.25.x  (6.25 branch)
- 2010.0: urpmi-6.32    (HEAD of that time)

In order to do so, pixel & me had to fix some packages in order to fix some known issues and to have the same foundations for mdkonline:
- urpmi/gurpmi (for fixes)
- drakxtools (for fixes)
- rpmdrake (for gurpmi.addmedia & update API)
- libdrakx-net (for network connection detection)

See eg:
http://svn.mandriva.com/cgi-bin/viewvc.cgi/soft/rpm/urpmi/branches/4.10.14/NEWS?view=log
http://svn.mandriva.com/cgi-bin/viewvc.cgi/soft/rpm/urpmi/branches/5.19/NEWS?view=log
http://svn.mandriva.com/cgi-bin/viewvc.cgi/soft/rpm/urpmi/branches/6.14/NEWS?view=log

BTW, b/c of that mdkonline also:
- do not use some perl-5.10-icisms (thus have warnings with
  new perl, see eg bug #1273)
- checks the release for which drakx API to use
- disables some stuff on 2008.0 (missing GUIes APIs)

Thus, mdkonline require a new enough working urpmi/rpmdrake/drakxtools-backend/drakx-net in both Cauldron and each supported release.
See http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/mdkonline/current/SPECS/mdkonline.spec?revision=666399&view=markup , especially the lines 16 to 57

While we do not have support for %mdkverssion, we can only support upgrading from mdv2010.0+ (that is 2010.0, 2010.1 & 2010.2), aka nothing older than september 2009.

I would be happy to clean mdkonline for checks for 2008.0, warnings with new perl & the like but I'd rather be honest about the (im)possibility to support older mdv releases.
Maybe we just don't care about those, but I don't think that's my call.
Comment 32 Thierry Vignaud 2011-05-16 14:54:22 CEST
(In reply to comment #30)
> It was done by cfergeau on September 1 2009 (20 months ago):
> "use https to grab mirrorlist from api.mandriva.com"
> http://svn.mandriva.com/cgi-bin/viewvc.cgi/soft/mdkonline/trunk/mdkapplet?r1=258569&r2=259833&sortby=date

And I'm pretty sure all the people who could have ordered that are here: romain, anne (but Frederic)
Comment 33 Nicolas Vigier 2011-05-16 18:46:57 CEST
(In reply to comment #31)
> Thus, mdkonline require a new enough working
> urpmi/rpmdrake/drakxtools-backend/drakx-net in both Cauldron and each supported
> release.
> See
> http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/mdkonline/current/SPECS/mdkonline.spec?revision=666399&view=markup
> , especially the lines 16 to 57
> 
> While we do not have support for %mdkverssion, we can only support upgrading
> from mdv2010.0+ (that is 2010.0, 2010.1 & 2010.2), aka nothing older than
> september 2009.

Could we replace tests like :
%if %mdkversion < 200910 
 with something like :
%if %{?mdkversion}%{!?mdkversion:201020} < 200910

?
Comment 34 Romain d'Alverny 2011-05-16 19:01:29 CEST
(In reply to comment #31)
> I would be happy to clean mdkonline for checks for 2008.0, warnings with new
> perl & the like but I'd rather be honest about the (im)possibility to support
> older mdv releases.
> Maybe we just don't care about those, but I don't think that's my call.

Migration/upgrade path is only provided for 2010.1 and 2010.2. So you should be able to drop everything that is related to older releases (if that's secure enough).

(In reply to comment #32)
> And I'm pretty sure all the people who could have ordered that are here:
> romain, anne (but Frederic)

"Ordered" is a big word for that :-) Anyway. The only reason I can remember of is to be sure that a client doesn't accept as valid an answer that does not come from a certified, known server (hence the SSL certificate - but is it checked really? one problem at a time, so let's leave this for another bug)
Comment 35 Anne Nicolas 2011-05-16 22:08:34 CEST
Yep only 2010.1 and 2010.2 are supported for upgrade. So we can go on on this and clean sources. Thierry can you do it please ? Would be nice to have something working by friday so that we have time to test
Comment 36 Thierry Vignaud 2011-05-16 22:28:47 CEST
Well, why not wait after Mageia 1 release?
It's safer... WDYT?Simpsons.S22E21.avi
Comment 37 Anne Nicolas 2011-05-16 22:31:17 CEST
(In reply to comment #36)
> Well, why not wait after Mageia 1 release?
> It's safer... WDYT?Simpsons.S22E21.avi

why not indeed. Does it work in current state? I mean upgrade ?
Comment 38 Thierry Vignaud 2011-05-19 10:33:30 CEST
Romain: could you add support for --testing on the server side?
It means having not only 'http://mirrors.mageia.org/api/FOOBAR'
but also 'http://mirrors.mageia.org/api/testing-FOOBAR'

This enables to support "mgaapplet --testing" like in MDV.

To Nicolas Vigier:
Also, mirrors.mageia.org still doesn't support https whereas
the installer uses to rely on https
Comment 39 Nicolas Vigier 2011-05-19 12:19:25 CEST
mirrors.mageia.org is now available in https, with http redirected to https.
Comment 40 Romain d'Alverny 2011-05-19 14:33:47 CEST
(In reply to comment #38)

 * https://releases.mageia.org/api/a/testing-i586
 * https://releases.mageia.org/api/a/testing-x86_64 (coming in a few minutes)
Comment 41 Nicolas Vigier 2011-05-19 14:35:59 CEST
(In reply to comment #39)
> mirrors.mageia.org is now available in https, with http redirected to https.

I reverted the redirection to https, as https is not supported in stage1.

And having both http and https is not possible with our current config, so there is no https for now.
Comment 42 Michael Scherer 2011-05-19 15:00:01 CEST
I will take a look for http and https, that should just requires some change to puppet manifests.
Marcello Anni 2011-05-20 13:02:33 CEST

CC: (none) => marcello.anni

Comment 43 Thierry Vignaud 2011-05-20 15:24:52 CEST
Looks like it's working, I'll make drakx's stage2 (and thus rpmdrake)
switch back to https.
Comment 44 Nicolas Vigier 2011-05-20 15:26:14 CEST
(In reply to comment #43)
> Looks like it's working, I'll make drakx's stage2 (and thus rpmdrake)
> switch back to https.

mirrors.mageia.org is not working in https at the moment.
Comment 45 Michael Scherer 2011-05-20 17:44:36 CEST
I have a idea, what about doing a reverse proxy on https to http ( or the reverse ) ? This would be ok with the issue of FastCgiServer working only for 1 vhost ?

( reverse proxy like we did for forums on alamut ).
Comment 46 José Jorge 2011-05-21 09:12:50 CEST
I just tested :
- install mgaonline and lauch it
- a few after, updates avalaible notified, click on notify link -> nothing happens BUG!
- click on RED icon, it works like in Mandriva -> OK

So I just see a last bug with notify link in KDE at least not working.

CC: (none) => lists.jjorge

Comment 47 James Kerr 2011-05-21 11:57:36 CEST
(In reply to comment #37)
> Does it work in current state? I mean upgrade ?

It didn't for me. It failed - could not find a mirror in MIRRORLIST

Running mgaapplet-upgrade-helper --new_distro_version=1 in a terminal, I see

cached mirror list uses an old format, invalidating it
URPMI_ADDMEDIA_REASON reason=upgrade
getting mirror list from https://api.mandriva.com/mirrors/basic.1.i586.list?reason=upgrade
Error: mirror list not found

What I did was install mgaonline on an up-to-date mdv 2010.1 and re-boot.

Is there something more required?

It is obviously using the wrong url to get the mirrorlist.
Comment 48 James Kerr 2011-05-21 12:33:56 CEST
I see that there's a new version of mgaonline on it's way. I'll test it when it reaches the mirrors.
Comment 49 Thierry Vignaud 2011-05-21 13:39:27 CEST
(In reply to comment #47)
> It didn't for me. It failed - could not find a mirror in MIRRORLIST
> 
> Running mgaapplet-upgrade-helper --new_distro_version=1 in a terminal, I see
> 
> cached mirror list uses an old format, invalidating it
> URPMI_ADDMEDIA_REASON reason=upgrade
> getting mirror list from
> https://api.mandriva.com/mirrors/basic.1.i586.list?reason=upgrade
> Error: mirror list not found

Humm, that's because mdv is running its own urpmi where
urpm::mirrors::_mandriva_mirrorlist() hardcoded api.mdv.com URL.

We should have used a config file for that years ago
(but obviosly we failed to forsight a fork...)

We can either:
- offer a new urpmi with mgaonline for upgrading mdv to mga
- patch urpm/mirrors.pm in %post (dirty trick...)

WDYT?

CC: (none) => mageia

Comment 50 Michael Scherer 2011-05-21 16:51:08 CEST
no possibility to monkeypatch urpm::mirrors, or just a new module placed before in PERL5LIB ?

 I have more uglier idea, like touching to dns ( /etc/hosts ) to redirect for our server.
Comment 51 Thierry Vignaud 2011-05-21 18:26:19 CEST
I didn't dare suggest that last one :-) (needs for cleaning after...)

I think the first idea (live patching) might be the simpler:

%post
perl -pi -e '$_ = "http://mirrors.mageia.org/api/mageia.\$product_version.\$arch.list" if m!https://api.mandriva.com/mirrors/!' /usr/lib/perl5/vendor_perl/5.*/urpm/mirrors.pm
Comment 52 Olivier Blin 2011-05-21 20:08:19 CEST
Live patching of urpm::mirrors looks a bit cleaner.
You can get a look at any patch-oem.pl file, or at draklive-install to have an idea of how packages functions can be overriden.
Comment 53 Thierry Vignaud 2011-05-22 14:14:30 CEST
This is offtopic since this is not mgaonline which will use the patched module but other programs (urpmi*)
Comment 54 Thierry Vignaud 2011-05-23 13:22:31 CEST
Just push a new mgaonline, it'll do live patching
Comment 55 James Kerr 2011-05-23 15:12:19 CEST
The applet still fails of course, since Mageia 1 does not yet exist.

Running from a terminal, it does use the correct url to attempt to get the mirrorlist.

Using cauldron as the target new_distro_version the upgrade does begin. It is now downloading. I'll report back when it is completed.

For future reference, what is the correct designation of the new_distro_version to use when running the upgrade-helper from a terminal, "1" or "stable1"? (There's an empty directory on the mirrors named stable1.)

One minor point, since I have a powerpack installation, the applet began by inviting me to add the restricted repo's, presumably because ./MgaOnline/mgaonline is empty. If possible, it might be better to create ./MgaOnline/mgaonline with the line DO_NOT_ASK_FOR_RESTRICTED=true entered by default.
Comment 56 Anne Nicolas 2011-05-23 16:44:14 CEST
try mgaapplet --testing
Comment 57 James Kerr 2011-05-23 18:26:38 CEST
The upgrade completed and I re-booted into Mageia.

urpmi --auto-update reports that packages are up to date.

The limited testing that I've had time to do indicates that the distro-upgrade has been successful.
Comment 58 James Kerr 2011-05-23 21:42:23 CEST
(In reply to comment #56)
> try mgaapplet --testing

Thanks. I hadn't thought of that.

The system that I used for testing was a "throw-away" that I created for this purpose. I can create another if more testing is required - and I'll keep the --testing option in mind.
Comment 59 Anne Nicolas 2011-05-23 22:42:07 CEST
Created attachment 459 [details]
choose Free / Powerpack
Comment 60 Anne Nicolas 2011-05-23 23:44:25 CEST
Oups looks like my comment did not appear. On previous screenshot we should remove this step about Free vs Powerpack choice.
btw I still have that pb about MIRRORLIST
Comment 61 Barry Jackson 2011-05-24 18:00:36 CEST
I think this should have been added here rather than a separate bug :-
https://bugs.mageia.org/show_bug.cgi?id=1408

CC: (none) => zen25000

Comment 62 Marcello Anni 2011-05-24 19:05:11 CEST
*** Bug 886 has been marked as a duplicate of this bug. ***

CC: (none) => dglent

Comment 63 Anne Nicolas 2011-05-25 11:58:58 CEST
Guys, 2 days left. Can we check all this is working well now?
Comment 64 Michael Scherer 2011-05-26 10:53:05 CEST
I have a testvm for that, so let me summarize : 
- i install mgaonline rpm ( from where ? ) on mandriva, by rebuilding the srpm from cauldron
- I run  mgaapplet --testing 

and that's all ?
Comment 65 Anne Nicolas 2011-05-26 10:55:25 CEST
you should have a  about new mageia. Then just follow what is asked there
Comment 66 Michael Scherer 2011-05-26 11:22:45 CEST
I tested on a 2010.0, and all I have is : 
"this version is no longer maintened", followed by "here is what you can do to fix this : 
( ) do not ask me anymore"  "[next]"    and nothing happen once I click next.
Comment 67 Michael Scherer 2011-05-26 11:23:46 CEST
Restarting the applet seemed to solve the issue however, I know have the popup with upgrade choice.
Comment 68 Michael Scherer 2011-05-26 11:24:31 CEST
The link "more information about this release" should go to the release note, I think ( it goes to mageia.org right now )
Comment 69 Michael Scherer 2011-05-26 11:28:15 CEST
I still see the popup with free/powerpack, and it should indeed be removed, and I also see a error message "cannot find a mirror from the list mirrorlist". Mgaonline complain about some unknow option on the command line ( "nocheck" ). The upgrade didn't start at all.
Comment 70 Romain d'Alverny 2011-05-26 11:45:02 CEST
(In reply to comment #68)
> The link "more information about this release" should go to the release note, I
> think ( it goes to mageia.org right now )

Will go to the product release page, but still looking for the right URL for that.
Comment 71 Michael Scherer 2011-05-26 12:45:26 CEST
Also, choosing Powerpack lead to a question about subscription to mandriva. I think this is not what we want.
Comment 72 James Kerr 2011-05-26 12:58:09 CEST
Nothing has changed from what I reported in comment 55.

It first asks to add restricted repo's, presumably because I have a powerpack installation.

I do not see the dialogue offering to upgrade to powerpack perhaps because, since I refused the restricted repo's, the line DO_NOT_ASK_FOR_RESTRICTED=true has now been added to ./MgaOnline/mgaonline.

It looks for a mirror at the correct url, but can't find a mirror - is there one for it to find?

getting mirror list from http://mirrors.mageia.org/api/mageia.1.i586.list?reason=upgrade
retrieved   mageia.1.i586.list?reason=upgrade
found geolocalisation IE 53.33 -5.75 from timezone Europe/Dublin
Fatal: Could not find a mirror from mirrorlist $MIRRORLIST
Comment 73 Michael Scherer 2011-05-26 13:00:01 CEST
So, after more debugging, the issue is that update fail because mirrors do not give proper answer for the url http://mirrors.mageia.org/api/mageia.1.i586.list?various_options, and it doesn't answer because the distribution is not released. ( ie, there is no mageia 1 mirror to give )
Comment 74 Michael Scherer 2011-05-26 13:11:06 CEST
After forcing it to use a copy of a fake mirrors list, pointing to cauldron, this seems to work. However, it do not seems to be very fast, and the ui disappeared. I see aria2 doing some stuff in the background using ps.
Comment 75 Michael Scherer 2011-05-26 13:14:35 CEST
It failed, aria2 was not able to cope with rsync://distrib-coffee, it seems. I try again, hoping it work now.
Comment 76 Michael Scherer 2011-05-26 13:25:41 CEST
Ok, it seems that on 2010.0, gurpmi.addmedia --distrib --mirrorlist '$MIRRORLIST' is unable to get media.cfg, I tested with another mirror ( in cae the problem is on d-c side, and it change nothing ). I test with aria2 removed.
Comment 77 Michael Scherer 2011-05-26 13:27:30 CEST
After "urpme aria2", it work fine. The vm is being upgraded at the moment.
Michael Scherer 2011-05-26 13:39:23 CEST

Depends on: (none) => 1432

Comment 78 Michael Scherer 2011-05-26 13:47:18 CEST
Upgade failed, see new bug 1432. The upgrade continue with urpmi --auto-update fine so far.
Comment 79 Michael Scherer 2011-05-26 13:55:25 CEST
Created attachment 472 [details]
untested patch to disable the powerpack dialog

Here is a patch that should disable the dialog for powerpack. I didn't test however, just took a look at the code.
Comment 80 James Kerr 2011-05-26 14:52:44 CEST
I replaced mandriva-release-powerpack with mandria-release-free and I now see the offer to upgrade to powerpack. I haven't tested Michael's patch - patching perl files is beyond my (very limited) abilities.
Comment 81 Romain d'Alverny 2011-05-26 15:23:17 CEST
(In reply to comment #73)
> So, after more debugging, the issue is that update fail because mirrors do not
> give proper answer for the url
> http://mirrors.mageia.org/api/mageia.1.i586.list?various_options, and it

The right URL would be http://mirrors.mageia.org/api/1.i586.list?various_options
Comment 82 Michael Scherer 2011-05-26 15:37:51 CEST
The upgrade failed due to not enough disk space. It seems that the system was installed with 4g, and this is not enough for a upgrade ( which mean that there is some rpm bloating the installation ).
Comment 83 Sander Lepik 2011-05-26 15:42:15 CEST
(In reply to comment #82)
> The upgrade failed due to not enough disk space. It seems that the system was
> installed with 4g, and this is not enough for a upgrade ( which mean that there
> is some rpm bloating the installation ).

Was texlive-texmf installed? It takes ~600MB when installed. And it gets installed w/o no obvious reason. Tested yesterday with RC dvd, choosed LXDE desktop, texlive-texmf got installed and after reboot i was able to remove it. Nothing depends on it, but it takes 300MB on dvd and 600MB on disk, when you use internet it's almost 1GB wasted disk space during upgrade.
Comment 84 James Kerr 2011-05-26 15:49:22 CEST
I did manage to apply the patch to mgaapplet-upgrade-helper and I no longer see the dialogue offering the upgrade to powerpack.

If I switch back to powerpack release and reboot, then the bogus offer to add restricted repo's still appears. Perhaps mgaapplet-add-media-helper needs to be removed from mgaonline or disabled in some way.
Comment 85 Michael Scherer 2011-05-26 15:53:53 CEST
That's not texlive. I didn't investigate much, the keyboard of the vm seems messed up ( but this is not caused by the upgrade ), and without {} | and others, my script-fu is weak.
Comment 86 Thierry Vignaud 2011-05-27 13:48:36 CEST
Just push new mgaonline for those fixes
Comment 87 D Morgan 2011-05-27 14:55:47 CEST
the pb i have with mgaonline is that he tells me that the maintenance is over on my mandriva instead of telling me that mageia 1 is ready and that i can update the distribution.
Comment 88 Thierry Vignaud 2011-05-27 15:13:06 CEST
which should be fixed with new .27 release
Comment 89 Thomas Backlund 2011-05-27 23:52:26 CEST
(In reply to comment #81)
> (In reply to comment #73)
> > So, after more debugging, the issue is that update fail because mirrors do not
> > give proper answer for the url
> > http://mirrors.mageia.org/api/mageia.1.i586.list?various_options, and it
> 
> The right URL would be
> http://mirrors.mageia.org/api/1.i586.list?various_options

NO, this is wrong. this url is broken, it mixes both i586 and x86_64 in the same list and is missing version and arch info, so it completely breaks mgaonline and urpmi.

What's more, it introduces and url mismatch compared to urpmi/rpmdrake/stage1/stage2/ ...

and will be overwritten by rpm when urpmi gets installed/updated...

So I have reverted the url change in mgaonline-2.77.27-2.mga1

An other thing I noticed in mgaonline spec is that it builds with:
make PREFIX=$RPM_BUILD_ROOT MANDRIVA_VERSION=%{mandriva_release} install

and building that on mageia, %{mandriva_release} evalutes to 1.
I wonder if that messes up something...

Will check more tomorrow, when I test upgrades.

CC: (none) => tmb

Comment 90 James Kerr 2011-05-28 13:11:50 CEST
Testing the .27.2 release

1. On a powerpack system, the bogus invitation to add restricted repo's no longer appears.

2. On a Free system, I still get the dialogue inviting me to upgrade to powerpack. 

3. I am not sure if it's using the correct url to get the mirror list or not:

getting mirror list from http://mirrors.mageia.org/api/mageia.1.i586.list?reason=upgrade
retrieved   mageia.1.i586.list?reason=upgrade
found geolocalisation IE 53.33 -5.75 from timezone Europe/Dublin
Fatal: Could not find a mirror from mirrorlist $MIRRORLIST

I got identical results with .27.1.
Comment 91 Romain d'Alverny 2011-05-28 13:26:29 CEST
(In reply to comment #89)
> (In reply to comment #81)
> > The right URL would be
> > http://mirrors.mageia.org/api/1.i586.list?various_options
> 
> NO, this is wrong. this url is broken, it mixes both i586 and x86_64 in the
> same list and is missing version and arch info, so it completely breaks
> mgaonline and urpmi.

This URL is right - that's the format expected. What gets published by it is something else, and indeed, mixes 32/64. Not good.

> and building that on mageia, %{mandriva_release} evalutes to 1.
> I wonder if that messes up something...

Why? That's the version number.

(In reply to comment #90)
> 3. I am not sure if it's using the correct url to get the mirror list or not:
> 
> getting mirror list from
> http://mirrors.mageia.org/api/mageia.1.i586.list?reason=upgrade
> retrieved   mageia.1.i586.list?reason=upgrade
> found geolocalisation IE 53.33 -5.75 from timezone Europe/Dublin
> Fatal: Could not find a mirror from mirrorlist $MIRRORLIST

Sure, this URL does not work.

The URL format to query is: mirrors/api/{version}.{arch}.list?reason={reason} so it's, in our case, mirrors/api/1.i586.list?reason=upgrade or mirrors/api/1.x86_64.list?reason=update depending on the situation.
Comment 92 Thomas Backlund 2011-05-28 14:01:13 CEST
(In reply to comment #91)
> (In reply to comment #89)
> > (In reply to comment #81)
> > > The right URL would be
> > > http://mirrors.mageia.org/api/1.i586.list?various_options
> > 
> > NO, this is wrong. this url is broken, it mixes both i586 and x86_64 in the
> > same list and is missing version and arch info, so it completely breaks
> > mgaonline and urpmi.
> 
> This URL is right - that's the format expected. What gets published by it is
> something else, and indeed, mixes 32/64. Not good.
> 

But we are in _release_freeze_

and mirrors.mageia.org/api is part of the freeze so you cant change it at this point.

as already stated, stage1/2 and urpmi tools have 
http://mirrors.mageia.org/api/mageia.{version}.{arch}.list hardcoded, so mgaonline need to follow


> 
> (In reply to comment #90)
> > 3. I am not sure if it's using the correct url to get the mirror list or not:
> > 
> > getting mirror list from
> > http://mirrors.mageia.org/api/mageia.1.i586.list?reason=upgrade
> > retrieved   mageia.1.i586.list?reason=upgrade
> > found geolocalisation IE 53.33 -5.75 from timezone Europe/Dublin
> > Fatal: Could not find a mirror from mirrorlist $MIRRORLIST
> 
> Sure, this URL does not work.
> 


So fix it to respond properly then!

--
Thomas
Romain d'Alverny 2011-05-28 14:24:49 CEST

Depends on: (none) => 1458

Comment 93 Romain d'Alverny 2011-05-28 14:25:45 CEST
(In reply to comment #92)
> > This URL is right - that's the format expected. What gets published by it is
> > something else, and indeed, mixes 32/64. Not good.
> 
> But we are in _release_freeze_
> 
> and mirrors.mageia.org/api is part of the freeze so you cant change it at this
> point.

Services such as this one don't have the same life cycle as the shipped distribution so we can fix those at any time.

> as already stated, stage1/2 and urpmi tools have 
> http://mirrors.mageia.org/api/mageia.{version}.{arch}.list hardcoded, so
> mgaonline need to follow

And no one realized that these URL just don't work at all at this point? so we need more testing.

So we will fix that by symlinking/duplicating this server-side but this will need to be fixed in later releases. As for actually fixing it, this involves two things:
 * answering the /api/mageia.1.i586.list path (see bug 1458)
 * sorting i586 from x86_64 (see bug 1457)
Romain d'Alverny 2011-05-28 14:26:19 CEST

Depends on: (none) => 1457

Comment 94 Thomas Backlund 2011-05-28 15:22:18 CEST
api server is now fixed for

http://mirrors.mageia.org/api/mageia.1.x86_64.list
http://mirrors.mageia.org/api/mageia.1.i586.list

It currently only shows two servers and we have faked it so version 1 points to cauldron in the above url lists,


but it should now be possible to test mgaonline-2.77.27-2.mga1
Comment 95 James Kerr 2011-05-28 15:54:12 CEST
It now downloads the hdlists, but fails with:

"Unable to add medium, errors reported:

unable to import pubkey file of "Core Updates"
unable to import pubkey file of "Core Updates Debug"
unable to import pubkey file of "Nonfree Updates"
unable to import pubkey file of "Nonfree Updates Debug"
unable to import pubkey file of "Tainted Updates"
unable to import pubkey file of "Tainted Updates Debug"

and finally:

"Failure when adding medium"

This could be similar to the problem that was encountered on mdv, when an attempt was made to offer an automatic upgrade from cooker to stable. (IIRC pubkeys are not automatically imported when sources are changed.)


In case it's forgotten, the patch mentioned in comment 79 (or something equivalent) needs to be committed to SVN (or wherever these things are done) to prevent the display of the offer to upgrade to powerpack. The patch seemed to work when I applied it locally.
Comment 96 Thomas Backlund 2011-05-28 16:04:44 CEST
(In reply to comment #95)
> It now downloads the hdlists, but fails with:
> 
> "Unable to add medium, errors reported:
> 
> unable to import pubkey file of "Core Updates"
> unable to import pubkey file of "Core Updates Debug"
> unable to import pubkey file of "Nonfree Updates"
> unable to import pubkey file of "Nonfree Updates Debug"
> unable to import pubkey file of "Tainted Updates"
> unable to import pubkey file of "Tainted Updates Debug"
> 
> and finally:
> 
> "Failure when adding medium"
> 
> This could be similar to the problem that was encountered on mdv, when an
> attempt was made to offer an automatic upgrade from cooker to stable. (IIRC
> pubkeys are not automatically imported when sources are changed.)
> 


Nope, just checked...
the pubkey is actually missing on the mirrors for those medias...
I'll fix it


> In case it's forgotten, the patch mentioned in comment 79 (or something
> equivalent) needs to be committed to SVN (or wherever these things are done) to
> prevent the display of the offer to upgrade to powerpack. The patch seemed to
> work when I applied it locally.


I'll verify it here too, and apply it after if it works for me too..
Comment 97 James Kerr 2011-05-28 16:07:39 CEST
(In reply to comment #96)

> > 
> > This could be similar to the problem that was encountered on mdv, when an
> > attempt was made to offer an automatic upgrade from cooker to stable. (IIRC
> > pubkeys are not automatically imported when sources are changed.)
> > 
> 
> 
> Nope, just checked...
> the pubkey is actually missing on the mirrors for those medias...
> I'll fix it
> 
> 

I just realised that myself and was about to post a comment. You beat me to it.
Comment 98 Thomas Backlund 2011-05-28 20:22:08 CEST
pubkeys are fixed on primary mirrors a few hours ago, and should show up on mirrors as soon as they sync 

And patch in comment 79 works to silence stuff about powerpack

But then we hit a problem. mgaonline reads version from host, so it will suggest 2010.1 or 2010.2 to the mageia api server:

http://mirrors.mageia.org/api/mageia.2010.2.x86_64.list?reason=upgrade 

but we only have versions cauldron and 1 (obviously)

so either we add check for 2010.1 / 2010.2 in mgaonline and replace it with 1 before polling mageia api

Other way to fix is on mageia api server, with symlinks from *.2010.1/2.* to 1
Comment 99 Thomas Backlund 2011-05-28 20:50:03 CEST
Never mind... it works if it starts from mgaapplet.

it was only when started manually it got wrong version

I'll submit a fixed mgaonline in a little while
Comment 100 Thomas Backlund 2011-05-28 21:25:53 CEST
A fixed mgaonline-2.77.28-1.mga1 is now uploaded and should be available on mirrors soon.

It's also available here:
http://bcd.mageia.org/mgaonline-2.77.28-1.mga1.noarch.rpm
Comment 101 James Kerr 2011-05-29 00:32:48 CEST
Testing 28-1

I just let it run to completion. It appears to have worked fine - I have a usable Mageia system. It's too late in the day to do anything further. If I have time I'll do a bit more checking of this system, probably not before Monday.
Comment 102 Anne Nicolas 2011-05-29 19:16:49 CEST
Tested here on 2010.1 with KDE and GNOME installed. No problem, upgrade went smoothly
Comment 103 Thomas Backlund 2011-05-29 20:32:42 CEST
Closing as fixed as it's confirmed that both i586 and x86_64 upgrades work.

Status: NEW => RESOLVED
Resolution: (none) => FIXED

Nicolas Vigier 2014-05-08 18:04:25 CEST

CC: boklm => (none)


Note You need to log in before you can comment on or make changes to this bug.