Bug 9744 - Push mageia-prepare-upgrade & mgaonline to mga2 on mga3 release
Summary: Push mageia-prepare-upgrade & mgaonline to mga2 on mga3 release
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: Cauldron
Hardware: i586 Linux
Priority: release_blocker major
Target Milestone: ---
Assignee: Colin Guthrie
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 10053
Blocks: 6061 8016 8157
  Show dependency treegraph
 
Reported: 2013-04-15 18:52 CEST by Colin Guthrie
Modified: 2014-05-08 18:05 CEST (History)
6 users (show)

See Also:
Source RPM: mageia-prepare-update,mgaonline
CVE:
Status comment:


Attachments
Pseudo-perl patch to implement the logic required (4.26 KB, text/plain)
2013-04-25 22:33 CEST, Colin Guthrie
Details
A patch that seems to work (4.61 KB, text/plain)
2013-04-26 02:04 CEST, Colin Guthrie
Details
Patch as currently applied (1.37 KB, patch)
2013-05-09 23:45 CEST, Dave Hodgins
Details | Diff

Description Colin Guthrie 2013-04-15 18:52:26 CEST
This is not a bug. It's just a reminder mechanism to ensure we push this at the same time as (or shortly before) we release Mageia 3.

Reproducible: 

Steps to Reproduce:
Colin Guthrie 2013-04-15 18:52:50 CEST

Priority: Normal => release_blocker
Blocks: (none) => 8016

Comment 1 Colin Guthrie 2013-04-25 22:27:59 CEST
Changing the bug title to also track changes required in mgaonline on mga2.

Summary: Push mageia-prepare-upgrade to mga2 on mga3 release => Push mageia-prepare-upgrade & mgaonline to mga2 on mga3 release

Comment 2 Colin Guthrie 2013-04-25 22:31:25 CEST
mgaonline upgrades
==================

Problem:
The upgrade from MGA2 -> MGA3 requires the preparation of the filesystem for the usrmove.

As a result, the distribution upgrade options suggested by mgaonline will fail.

In order to counteract this, the following changes are proposed.

 1. Switch current API from http://releases.mageia.org/api/a/ to http://releases.mageia.org/api/b/
 2. Only announce MGA3 release via API b.
 3. Push update mgaonline to MGA2 that looks at API b.


In the updated mgaonline, the basic principle of operation is to do the following:
 1. Check the releases webservice API to see if the distro version 'needs_preparation'.
 2. If it does, check for a file /var/lib/mageia-prepare-upgrade/state.
    If it says 'ready' then we are ready and the user is not questioned further.
 3. If it does not exist or says anything else, the user is prompted to install the
    package 'mageia-prepare-upgrade'.
 4. After package installation, check the state file again as this may be
    all that is required. If the state file does not yet say 'ready', then
    display the text from the /usr/share/doc/mageia-prepare-upgrade/README.prepare
    file (todo: add i18n)
 5. Assuming that further action is required by the user (e.g. rebooting to
    convert the filesystem), then the upgrade helper will exit.
 6. After the user has taken the action required to do the preparation,
    the state file should contain 'ready'. The next time the distro upgrade
    dialog shows, the preparation checks will pass and the user can continue.

CC: (none) => thierry.vignaud

Comment 3 Colin Guthrie 2013-04-25 22:33:11 CEST
Created attachment 3814 [details]
Pseudo-perl patch to implement the logic required

This is a patch that does what I propose. It's untested and as I can't code in perl, I'd be shocked if it even half worked.

Theirry, can you review/make suggestions before I continue?
Comment 4 Colin Guthrie 2013-04-25 22:37:57 CEST
I've committed a 'b' API example to the web repository:
http://svnweb.mageia.org/web?view=revision&revision=2209
Comment 5 Colin Guthrie 2013-04-26 02:04:42 CEST
Created attachment 3815 [details]
A patch that seems to work

I actually ran this code and it pretty much worked!! Unbelievable... (for me at least!).

Here is a slightly more polished version. Seems to work tho' not fully tested it.

A small issue remains. If it reads data from a file (using cat_()) and displays it in a dialog the "OK" button in the dialog is the same size as the text (it contains newlines) which looks bad. If I use the default string in the code, even although it contains new lines, the "OK" button is a good size. I'm obviously missing something...

Attachment 3814 is obsolete: 0 => 1

Dimitrios Glentadakis 2013-04-26 09:49:36 CEST

CC: (none) => dglent

Comment 6 Colin Guthrie 2013-04-26 11:48:04 CEST
So I've now applied these patches to mga2's mgaonline and the whole process works as expected (amazingly!)

I'll push it to updates-testing on mga2 to get others to test.
Comment 7 Dave Hodgins 2013-05-09 20:50:54 CEST
With the current version, I have to use mgaapplet --testing to see the
upgrade notification.

After it asks if I want to install the upgrade package, and when I select
yes, the terminal output shows ...
retrieved   testing-x86_64?product=Default&version=2&mgaonline_version=2.77.33
i would install packages mageia-prepare-upgrade

and displays a pop-up with a link to the release notes. Selecting ok
at that point, actually starts the upgrade (which fails), without
installing the mageia-prepare-upgrade package.

At present, it must still be installed manually.

CC: (none) => davidwhodgins

Comment 8 Colin Guthrie 2013-05-09 21:27:24 CEST
@Dave, this is with TEST_DISTRO_UPGRADE=YES in /etc/sysconfig/mgaapplet and with the testing repo enabled?

It's odd that the --testing switch actually installs packages anyway as it would appear to be interpreted in the ensure_is_installed() code to prevent actual package installation... Seems the meaning isn't the same throughout all code here :(

Anyway, I didn't use mgaaplet --testing, I just used the above stuff and it works for me (of course mgaapplet from updates_testing needs to be installed first :))
Comment 9 Dave Hodgins 2013-05-09 23:45:52 CEST
Created attachment 3913 [details]
Patch as currently applied

Attached is the version of the patch I used, with the last Tainted changed
to updates.  Nonfree and Tainted are still not being kept enabled.
Manuel Hiebel 2013-05-10 23:02:59 CEST

Depends on: (none) => 10053

Manuel Hiebel 2013-05-16 01:24:54 CEST

Blocks: (none) => 8157

Colin Guthrie 2013-05-16 17:56:03 CEST

Blocks: (none) => 6061

Manuel Hiebel 2013-05-19 09:38:12 CEST

CC: (none) => qa-bugs, sysadmin-bugs
Source RPM: mageia-prepare-update => mageia-prepare-update,mgaonline

Comment 10 Manuel Hiebel 2013-05-19 09:39:35 CEST
missed the comment

should we not push packages as soon as possible ?
(x86_64 testing complete here)
Comment 11 Colin Guthrie 2013-05-19 11:33:32 CEST
Yes we should push this soon - at the same time as changes to http://svnweb.mageia.org/web/releases/api/ (only the 'b') API for new releases.
Comment 12 Colin Guthrie 2013-05-19 15:47:24 CEST
OK, so for clarity, the following updates should be pushed:

dracut-017-16.2.mga2.src.rpm
mageia-prepare-upgrade-2-2.mga2.src.rpm
mgaonline-2.77.33-1.8.mga2.src.rpm
urpmi-6.48.4-1.1.mga2.src.rpm 

These are all currently in updates_testing. There isn't perfect i18n due to some new strings, but probably better to get it out there and we can always update them again for that if translations are forthcoming.

Advisory Text
=============

In order to smoothly update to Mageia 3, you need to follow the update preparation procedure, which is to install the "mageia-prepare-upgrade" and reboot and select the "Mageia 3 Upgrade Preparation" boot entry.

The distribution upgrade system built into mgaonline will guide you through these steps.
Comment 13 Colin Guthrie 2013-05-19 17:05:34 CEST
OK, as Thierry wanted a new release of urpmi rather than the patch, here is the new list of srpms:

dracut-017-16.2.mga2.src.rpm
mageia-prepare-upgrade-2-2.mga2.src.rpm
mgaonline-2.77.33-1.9.mga2.src.rpm
urpmi-6.48.5-1.mga2.src.rpm 

(note as well as new urpmi, I bumped the mgaonline for the new urpmi version - not strictly needed but it's clearer).
Comment 14 claire robinson 2013-05-19 17:32:19 CEST
Thanks colin, testing a 64bit upgrade in vbox.
Comment 15 claire robinson 2013-05-19 17:43:43 CEST
To anybody testing, at the moment it is still necessary to add TEST_DISTRO_UPGRADE=YES in /etc/sysconfig/mgaapplet and leave Core Updates Testing enabled during the upgrade.

When these are pushed the API will be updated so this is no longer necessary.
Comment 16 Dave Hodgins 2013-05-19 18:59:59 CEST
Testing a 32 bit upgrade under vbox.
Comment 17 claire robinson 2013-05-19 20:09:25 CEST
Still seeing the pango/harfbuzz segfault at the end of package installation.

It pops up a bug reporting tool so we'd be flooded with bug reports if we release that way. Is there a way to fix this, or perhaps if not then suppress the report?
Comment 18 claire robinson 2013-05-19 20:09:56 CEST
bug 9882
claire robinson 2013-05-19 20:11:40 CEST

Depends on: (none) => 9882

Comment 19 claire robinson 2013-05-19 20:21:22 CEST
There is still the plymouthd fail on reboot also, unfortunately.
bug 10128
Comment 20 Nicolas Vigier 2013-05-19 23:33:46 CEST
(In reply to Colin Guthrie from comment #13)
> OK, as Thierry wanted a new release of urpmi rather than the patch, here is
> the new list of srpms:
> 
> dracut-017-16.2.mga2.src.rpm
> mageia-prepare-upgrade-2-2.mga2.src.rpm
> mgaonline-2.77.33-1.9.mga2.src.rpm
> urpmi-6.48.5-1.mga2.src.rpm 

Packages have been pushed to updates.

CC: (none) => boklm

Comment 21 Dave Hodgins 2013-05-20 00:11:35 CEST
I've updated https://wiki.mageia.org/en/Mageia_3_Errata#Upgrade_Issues
Comment 22 Colin Guthrie 2013-05-20 01:35:57 CEST
Awesome thanks Nicolas.

And thanks Dave for updating the Errata. I think it's better to just put the various remaining issues in the errata as I think we will always have problems like these when running a live environment. In the future, I firmly believe we should only support some kind of offline update (although something that is very guided via the mgaonline applet. This is one of my proposed features for Mageia 4: https://wiki.mageia.org/en/Feature:OfflineUpgrades
Thierry Vignaud 2013-05-20 07:44:32 CEST

Depends on: 9882 => (none)

Comment 23 claire robinson 2013-05-20 12:30:44 CEST
I don't think the packages were actually pushed, unless it's just my mirror is outdated, could you check Nicolas please.
Comment 24 Nicolas Vigier 2013-05-20 12:34:06 CEST
They were pushed yesterday, but I forgot to update hdlists. Sander noticed it this morning and this should now be fixed (as soon as mirrors are synced).
Comment 25 Manuel Hiebel 2013-05-20 13:35:24 CEST
still nothing 
from d-c
1369048801
Mon May 20 13:20:01 2013


|_ _| |  _ \  / ___|  | |    
 | |  | |_) | \___ \  | |    
 | |  |  __/   ___) | | |___ 
|___| |_|     |____/  |_____|

Institut Pierre Simon Laplace
http://www.ipsl.jussieu.fr/

Please have a look at http://distrib-coffee.ipsl.jussieu.fr/

Olivier Thauvin <olivier.thauvin@aero.jussieu.fr>
Your favorite sys admin.

2008/03/23: ftp ls -R is now disabled

receiving file list ... 
117193 files to consider
2/x86_64/VERSION
          49 100%   47.85kB/s    0:00:00 (xfer#1, to-check=117189/117193)
2/x86_64/media/core/updates/
2/x86_64/media/core/updates/dracut-017-16.2.mga2.noarch.rpm
      160957 100%  858.93kB/s    0:00:00 (xfer#2, to-check=97156/117193)
2/x86_64/media/core/updates/gurpmi-6.48.5-1.mga2.noarch.rpm
       21165 100%   98.89kB/s    0:00:00 (xfer#3, to-check=95923/117193)
2/x86_64/media/core/updates/mageia-prepare-upgrade-2-2.mga2.noarch.rpm
       11014 100%   48.23kB/s    0:00:00 (xfer#4, to-check=93279/117193)
2/x86_64/media/core/updates/mgaonline-2.77.33-1.9.mga2.noarch.rpm
      158797 100%  344.61kB/s    0:00:00 (xfer#5, to-check=93199/117193)
2/x86_64/media/core/updates/urpmi-6.48.5-1.mga2.noarch.rpm
      629977 100%  170.70kB/s    0:00:03 (xfer#6, to-check=91123/117193)
2/x86_64/media/core/updates/urpmi-ldap-6.48.5-1.mga2.noarch.rpm
       13449 100%   13.48kB/s    0:00:00 (xfer#7, to-check=91119/117193)
2/x86_64/media/core/updates/urpmi-parallel-ka-run-6.48.5-1.mga2.noarch.rpm
       12177 100%   11.77kB/s    0:00:01 (xfer#8, to-check=91115/117193)
2/x86_64/media/core/updates/urpmi-parallel-ssh-6.48.5-1.mga2.noarch.rpm
       12077 100%   11.37kB/s    0:00:01 (xfer#9, to-check=91111/117193)
2/x86_64/media/core/updates_testing/
2/x86_64/media/core/updates_testing/media_info/
2/x86_64/media/core/updates_testing/media_info/20130520-100053-changelog.xml.lzma -> changelog.xml.lzma
2/x86_64/media/core/updates_testing/media_info/20130520-100053-files.xml.lzma -> files.xml.lzma
2/x86_64/media/core/updates_testing/media_info/20130520-100053-hdlist.cz -> hdlist.cz
2/x86_64/media/core/updates_testing/media_info/20130520-100053-info.xml.lzma -> info.xml.lzma
2/x86_64/media/core/updates_testing/media_info/20130520-100053-synthesis.hdlist.cz -> synthesis.hdlist.cz
2/x86_64/media/core/updates_testing/media_info/MD5SUM
         576 100%    0.54kB/s    0:00:01 (xfer#10, to-check=90271/117193)
2/x86_64/media/core/updates_testing/media_info/changelog.xml.lzma
       50169 100%   41.95kB/s    0:00:01 (xfer#11, to-check=90270/117193)
2/x86_64/media/core/updates_testing/media_info/files.xml.lzma
      343261 100%  717.81kB/s    0:00:00 (xfer#12, to-check=90269/117193)
2/x86_64/media/core/updates_testing/media_info/hdlist.cz
     3063142 100%    8.00MB/s    0:00:00 (xfer#13, to-check=90268/117193)
2/x86_64/media/core/updates_testing/media_info/info.xml.lzma
       21941 100%   20.92MB/s    0:00:00 (xfer#14, to-check=90267/117193)
2/x86_64/media/core/updates_testing/media_info/synthesis.hdlist.cz
       44231 100%    3.24MB/s    0:00:00 (xfer#15, to-check=90265/117193)
2/x86_64/media/media_info/
2/x86_64/media/media_info/MD5SUM
        8064 100%  562.50kB/s    0:00:00 (xfer#16, to-check=90263/117193)
2/x86_64/media/media_info/rpmsrate
       26151 100%    1.78MB/s    0:00:00 (xfer#17, to-check=90135/117193)
deleting 2/x86_64/media/core/updates_testing/mgaonline-2.77.33-1.7.mga2.noarch.rpm
deleting 2/x86_64/media/core/updates_testing/mageia-prepare-upgrade-2-0.10.mga2.noarch.rpm
deleting 2/x86_64/media/core/updates_testing/dracut-017-16.1.2.mga2.noarch.rpm
deleting 2/x86_64/media/core/updates_testing/media_info/20130517-195832-synthesis.hdlist.cz
deleting 2/x86_64/media/core/updates_testing/media_info/20130517-195832-info.xml.lzma
deleting 2/x86_64/media/core/updates_testing/media_info/20130517-195832-hdlist.cz
deleting 2/x86_64/media/core/updates_testing/media_info/20130517-195832-files.xml.lzma
deleting 2/x86_64/media/core/updates_testing/media_info/20130517-195832-changelog.xml.lzma
                    
sent 18190 bytes  received 6229161 bytes  277660.04 bytes/sec
total size is 170059559535  speedup is 27221.07

Heure de fin du sync: 13:29:37
Comment 26 Manuel Hiebel 2013-05-20 14:45:42 CEST
fixed thanks, so we close this bug of not ? :)
Comment 27 Malo Deniélou 2013-05-20 19:46:18 CEST
Closing

Status: NEW => RESOLVED
CC: (none) => pierre-malo.denielou
Resolution: (none) => FIXED

Nicolas Vigier 2014-05-08 18:05:06 CEST

CC: boklm => (none)


Note You need to log in before you can comment on or make changes to this bug.