Bug 9819 - Attempt to upgrade from M2 -> M3 cauldron fails
Summary: Attempt to upgrade from M2 -> M3 cauldron fails
Status: RESOLVED OLD
Alias: None
Product: Mageia
Classification: Unclassified
Component: Installer (show other bugs)
Version: 3
Hardware: All Linux
Priority: release_blocker normal
Target Milestone: ---
Assignee: Mageia Bug Squad
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 8016
  Show dependency treegraph
 
Reported: 2013-04-21 21:56 CEST by William Kenney
Modified: 2013-08-27 15:30 CEST (History)
10 users (show)

See Also:
Source RPM:
CVE:
Status comment:


Attachments
Attempt to upgrade from M2 -> M3 cauldron fails (806.56 KB, application/zip)
2013-04-21 21:57 CEST, William Kenney
Details
Attempt to upgrade from M2 -> M3 cauldron fails, 23apr13 (631.36 KB, application/zip)
2013-04-23 18:22 CEST, William Kenney
Details
Attempt to upgrade from M2 -> M3 cauldron fails, 26apr13 (596.50 KB, application/zip)
2013-04-27 03:47 CEST, William Kenney
Details
Attempt to upgrade from M2 -> M3 cauldron fails, 2May13 (597.04 KB, application/zip)
2013-05-03 04:38 CEST, William Kenney
Details
Attempt to upgrade from M2 -> M3 cauldron fails, 7May13 (605.09 KB, application/zip)
2013-05-08 04:48 CEST, William Kenney
Details

Description William Kenney 2013-04-21 21:56:54 CEST
Today I attempted upgrading a 64-bit M2 install using an
up-to-date local cauldron repo. The M2 install was created
using an up to date local repo, a completely blank drive and
the M2 boot.iso CD ( 64-bit, 1 Jun 12 ). The install of M2
went normally and successfully. I then put some bookmarks
in Firefox and rearranged the desktop slightly. I then went
through the update process and a few files were updated.
I set the boot process to automatically boot to the user
workspace. No other apps, features and/or functions were
installed. I then did a successful reboot back to workspace.
I then installed:

/core/updates_testing/mageia-prepare-upgrade-2-0.3.mga2.noarch.rpm

I then booted the M3 64-Bit Cauldron boot.iso ( 19 Apr 13 )
installed on a USB drive and rebooted the system. I instructed
the installer to upgrade the already installed M2. About halfway
into the install 50+ errors were reported.

Using a Puppy Linux 5.5 Live-CD I was able to capture
/var/log/syslog and the /root/drakx directory. Those
files are contained in a ZIP file attached to this bug.
I had to pull the .log1 files as it created a ZIP file
larger then 1MB.

I went no further then generating this data. This is
all done on real hardware and can be duplicated. But
do note it takes me more then 2-hours++ to go through
this process.

Reproducible: 

Steps to Reproduce:
Comment 1 William Kenney 2013-04-21 21:57:54 CEST
Created attachment 3783 [details]
Attempt to upgrade from M2 -> M3 cauldron fails

See attachment
Comment 2 claire robinson 2013-04-21 23:52:22 CEST
* 58 installation transactions failed
* rpmdrake < 5.36 conflicts with urpmi-7.24-1.mga3.noarch
perl-CPANPLUS < 0.912.0-2 conflicts with perl-2:5.16.3-1.mga3.x86_64
perl-base >= 2:5.16.2 is needed by perl-MDK-Common-1.2.29-2.mga3.noarch
usermode-consoleonly < 1:1.110 conflicts with systemd-195-20.mga3.x86_64
ConsoleKit >= 0.2.1 is needed by (installed) lib64polkit2-0.9-8.mga1.x86_64
systemd >= 195 is needed by resolvconf-1.68-4.mga3.noarch
libdb-5.3.so()(64bit) is needed by iproute2-3.7.0-2.mga3.x86_64
systemd >= 195 is needed by pam-1.1.6-5.mga3.x86_64

CC: (none) => thierry.vignaud

Comment 3 Thomas Backlund 2013-04-22 02:14:08 CEST
(In reply to William Kenney from comment #0)

> I then installed:
> 
> /core/updates_testing/mageia-prepare-upgrade-2-0.3.mga2.noarch.rpm
> 
> I then booted the M3 64-Bit Cauldron boot.iso ( 19 Apr 13 )
> installed on a USB drive and rebooted the system. I instructed
> the installer to upgrade the already installed M2. About halfway
> into the install 50+ errors were reported.


Here you are mixing two types of installation.

the "mageia-prepare-upgrade" package is only meant for use if you intend to do a live upgrade using urpmi.

booting from a boot(-nonfree).iso has the needed code in stage2 to do the conversion, so no need for the above "prepare" package.

CC: (none) => tmb

Comment 4 William Kenney 2013-04-22 02:29:58 CEST
Thank you for that input. This was a followup test to see
if it made a difference. It did not. Previously I submitted:

https://bugs.mageia.org/show_bug.cgi?id=9774

which is duplicate of:

https://bugs.mageia.org/show_bug.cgi?id=9341

I'll continue to run this upgrade test every couple days or
so until everything clears. I rsync with mirrors.kernel.org
at least 1x every day ( 04:05AM California time ). Next test
will probably be Tuesday morning CA time. Two of my test
platforms have removable/replaceable modular HD systems so
I can exchange scratch drives at any time. Feel free to
make suggestions or requests.
Comment 5 Thierry Vignaud 2013-04-22 18:06:50 CEST
(In reply to William Kenney from comment #0)

You can speed up install by booting the installer with "linux tune-rpm justdb" instead of the default "linux".
This will skip actually installing files and will not use fsync.

I failed to locally reproduce with (/mageia being my local mirror):
(but then I've all the packages available whereas DVD has only a subset; I should try with downloaded isos as repositories)

PKGS="basesystem task-gnome"

mkdir $ROOT; makedev $ROOT/dev
urpmi.addmedia --urpmi-root $ROOT --distrib /mageia/stable/x86_64
# save it for easy redo:
tar cfz $ROOT{.tgz,}

urpmi --urpmi-root $ROOT --justdb --auto --no-verify-rpm --tune-rpm=all --replacefiles $PKGS 2>&1|tee LOG.mga2
urpmi.removemedia --urpmi-root $ROOT -a
# save it for easy redo:
tar cfz $ROOT{2.tgz,}

urpmi.addmedia --urpmi-root $ROOT --distrib /mageia/unstable/x86_64
# save it for easy redo:
tar cfz $ROOT{3.tgz,}
urpmi --urpmi-root $ROOT --justdb --auto-select --no-verify-rpm --tune-rpm=all --replacefiles 2>&1|tee LOG.mga3


I tried to play with --split-length= & --split-level in order to make it even faster but we've stuff to fix in packages with big transactions
Comment 6 David Walser 2013-04-22 23:12:30 CEST
He said he used boot.iso, not the DVD, so all packages should be available, and these kinds of errors should not happen.  That being said, I've done three upgrade tests using boot.iso recently and can't reproduce any package upgrade problems either.
Comment 7 William Kenney 2013-04-23 00:14:03 CEST
(In reply to David Walser from comment #6)

> He said he used boot.iso, not the DVD, so all packages should be available,
> and these kinds of errors should not happen.  That being said, I've done
> three upgrade tests using boot.iso recently and can't reproduce any package
> upgrade problems either.

I am going to rerun the test again tomorrow, 23 Apr 2013 California time.
Tell me what to do different and I will.
Comment 8 William Kenney 2013-04-23 18:22:06 CEST
Today I again attempted upgrading a 64-bit M2 install using an
up-to-date local cauldron repo. The M2 install was created
using an up to date local repo, a completely blank drive and
the M2 boot.iso CD ( 64-bit, 1 Jun 12 ). The install of M2
went normally and successfully. I then put some bookmarks
in Firefox and rearranged the desktop slightly. I then went
through the update process and a few files were updated.
I set the boot process to automatically boot to the user
workspace. No other apps, features and/or functions were
installed. I then did a successful reboot back to workspace.
All of this is on an i7 platform that runs M2 just fine.
I did not install:

/core/updates_testing/mageia-prepare-upgrade-2-0.3.mga2.noarch.rpm

I then booted the M3 64-Bit Cauldron boot.iso ( 22 Apr 13 )
installed on a USB drive and rebooted the system. I instructed
the installer to upgrade the already installed M2. About halfway
into the install 70+ errors were reported.

Using a Puppy Linux 5.5 Live-CD I was able to capture
/var/log/syslog and the /root/drakx directory. Those
files are contained in a ZIP file attached to this bug.
I had to pull the .log1 files as it created a ZIP file
larger then 1MB.

I went no further then generating this data. This is
all done on real hardware and can be duplicated. But
do note it takes me more then 2-hours++ to go through
this process.

Please feel free to make any suggestions here as to if
I am doing something wrong.
Comment 9 William Kenney 2013-04-23 18:22:52 CEST
Created attachment 3794 [details]
Attempt to upgrade from M2 -> M3 cauldron fails, 23apr13
Comment 10 Thierry Vignaud 2013-04-23 21:15:16 CEST
Humm, perl-Archive-Tar & perl-CPANPLUS were not in the proper transaction
Thierry Vignaud 2013-04-23 21:22:29 CEST

CC: (none) => jquelin

Comment 11 William Kenney 2013-04-27 03:46:27 CEST
Today I again attempted upgrading a 64-bit M2 install using an
up-to-date local cauldron repo. The M2 install was created
using an up to date local repo, a completely blank drive and
the M2 boot.iso CD ( 64-bit, 1 Jun 12 ). The install of M2
went normally and successfully. I then put some bookmarks
in Firefox and rearranged the desktop slightly. I then went
through the update process and a few files were updated.
I set the boot process to automatically boot to the user
workspace. No other apps, features and/or functions were
installed. I then did a successful reboot back to workspace.
All of this is on an i7 platform that runs M2 just fine.
I did not install:

/core/updates_testing/mageia-prepare-upgrade-2-0.3.mga2.noarch.rpm

I then booted the M3 64-Bit Cauldron boot.iso ( 24 Apr 13 )
installed on a USB drive and rebooted the system. I instructed
the installer to upgrade the already installed M2. About halfway
into the install 70+ errors were reported.

Using a Puppy Linux 5.5 Live-CD I was able to capture
/var/log/syslog and the /root/drakx directory. Those
files are contained in a ZIP file attached to this bug.
I had to pull the .log1 files as it created a ZIP file
larger then 1MB.

I went no further then generating this data. This is
all done on real hardware and can be duplicated. But
do note it takes me more then 2-hours++ to go through
this process.

Please feel free to make any suggestions here as to if
I am doing something wrong.
Comment 12 William Kenney 2013-04-27 03:47:11 CEST
Created attachment 3825 [details]
Attempt to upgrade from M2 -> M3 cauldron fails, 26apr13
Comment 13 bozonius 2013-05-03 00:02:54 CEST
eh, shouldn't this be a release blocker?  Failure to upgrade seems like a serious problem, if we want to keep the M2 user base with us?

CC: (none) => bozonius

Comment 14 Manuel Hiebel 2013-05-03 00:18:03 CEST
well you have also a lot of mix between i586 and x86_64 pkgs ...
Comment 15 bozonius 2013-05-03 01:43:08 CEST
Will the user care?  Or will the user throw their arms up in frustration and move on to a different distro?  I don't really understand the point to the remark.
Comment 16 William Kenney 2013-05-03 01:59:43 CEST
FWIW a basic m2 64-bit install includes the 32-bit core repo.
Could there be a problem here in the upgrade?
Comment 17 William Kenney 2013-05-03 04:36:52 CEST
Today I again attempted upgrading a 64-bit M2 install using an
up-to-date local cauldron repo. The M2 install was created
using an up to date local repo, a completely blank drive and
the M2 boot.iso CD ( 64-bit, 1 Jun 12 ). The install of M2
went normally and successfully. I then put some bookmarks
in Firefox and rearranged the desktop slightly. I then went
through the update process and a few files were updated.
I set the boot process to automatically boot to the user
workspace. No other apps, features and/or functions were
installed. I then did a successful reboot back to workspace.
All of this is on an i7 platform that runs M2 just fine.
I did not install:

/core/updates_testing/mageia-prepare-upgrade-2-0.3.mga2.noarch.rpm

I then booted the M3 64-Bit Cauldron boot.iso ( 1 May 13 )
installed on a USB drive and rebooted the system. I instructed
the installer to upgrade the just installed M2. About halfway
into the install a large number of errors were reported failed.

Using a Puppy Linux 5.5 Live-CD I was able to capture
/var/log/syslog and the /root/drakx directory. Those
files are contained in a ZIP file attached to this bug.
I had to pull the .log1 files as it created a ZIP file
larger then 1MB.

I went no further then generating this data. This is
all done on real hardware and can be duplicated. But
do note it takes me more then 2-hours++ to go through
this process.

Again, please feel free to make any suggestions here
as to if I am doing something wrong.
Comment 18 William Kenney 2013-05-03 04:38:03 CEST
Created attachment 3879 [details]
Attempt to upgrade from M2 -> M3 cauldron fails, 2May13
claire robinson 2013-05-03 11:41:53 CEST

Priority: Normal => release_blocker
Blocks: (none) => 8016

claire robinson 2013-05-03 11:42:01 CEST

CC: (none) => eeeemail

Comment 19 David Walser 2013-05-03 17:51:23 CEST
"
/usr/lib64/gdk-pixbuf-2.0/bin/gdk-pixbuf-query-loaders: symbol lookup error: /usr/lib64/gdk-pixbuf-2.0/bin/gdk-pixbuf-query-loaders: undefined symbol: g_type_ensure
"

Looks like %{_lib}gdk_pixbuf2.0_0 needs Requires: %{_lib}glib2.0_0 >= 2.34.0:
http://opensuse.14.x6.nabble.com/Tumbleweed-gdk-pixbuf-query-loaders-undefined-symbol-g-type-ensure-td4982420.html

"
/usr/bin/gtk-query-immodules-3.0-64: symbol lookup error: /lib64/libwayland-cursor.so.0: undefined symbol: wl_shm_pool_interface
"

That program is in the gtk+3.0 package, which doesn't even require the wayland libraries, even though ldd shows it's linked to them.  Maybe it doesn't need to because it requires them recursively through one of its other requires.  If so, possibly that might need a versioned requires on libwayland-cursor0.

"
Cannot load module /usr/lib64/gtk-2.0/2.10.0/immodules/im-ibus.so: GModule (/usr/lib64/gtk-2.0/2.10.0/immodules/im-ibus.so) initialization check failed: GLib version too old (micro mismatch)
/usr/lib64/gtk-2.0/2.10.0/immodules/im-ibus.so does not export GTK+ IM module API: GModule (/usr/lib64/gtk-2.0/2.10.0/immodules/im-ibus.so) initialization check failed: GLib version too old (micro mismatch)
"

Not sure what that's about.  Maybe ibus-gtk also needs a versioned libglib2.0_0 requires.

Not sure how important any of the above scriplet errors are though.

As far as all the "Installation failed" messages, I am not seeing that in my install.log files for my upgrade tests.  What error pops up in the installer GUI once it finishes trying to install packages (not sure any of the logs show the final error)?  It looks like urpmi is having some real problems with transaction grouping in your install.
Comment 20 David Walser 2013-05-03 17:53:20 CEST
Worth noting that all of my tests are i586, so maybe this is an x86_64-specific issue.

Hardware: i586 => x86_64

Comment 21 William Kenney 2013-05-03 18:09:52 CEST
(In reply to David Walser from comment #20)
> Worth noting that all of my tests are i586, so maybe this is an
> x86_64-specific issue.

I've encountered the same errors on my i586 platform, it's just
a little more cumbersome to run the tests there. Older system.
Also I have to burn the 32-bit boot.iso CD as against using a
USB drive on my 64-bit test system.
Thierry Vignaud 2013-05-03 22:51:21 CEST

Hardware: x86_64 => All

Comment 22 Jürgen Preuà 2013-05-08 02:01:16 CEST
I can confirm behavior of interrupted upgrade M2 -> M3. Testing object was a Dell D430 laptop with living M2 installation. Latest patches from official sources were installed. Installation device is DVD with RC-image.

Majority of failed installation packages were dependend from perl-5.16* and perl-base*. After manual forcing installation of the two packages at stopping interrupt (Ctrl-Alt-F2; "rpm -r /mnt --dbpath=/mnt/var/lib/rpm perl-5.16.3-1.mga3*.rpm perl-base-5.16.3-1.mga3*.rpm --nodeps --force") the next installation loop was successfully. Some ruby dependend component seems to have failed in missing lib64ruby.

Watching a clean full installation (i586) of M3 on another device (HP NC620 Laptop) showed an early (forced?) installation of the perl packages in question and went clean till the end.

After upgrading little Dell was a further X11 error (possibly a separate bug) in showstopper quality. The X-display showed an error banner after successfully booting because of missing ressources in filesystem before exiting. Root cause is missing lib64fam package. Automatic dependency is not given, but should be. After manually installing missing package the X-desktop is starting ok.

Can somebody prove my observations?
Thanks in advance for the QA team.
With regards, Juergen

CC: (none) => preugen

Comment 23 William Kenney 2013-05-08 02:54:27 CEST
I have a successful 64-bit M2 -> M3 upgrade here using the
Classical Installer DVD.

Today I again attempted upgrading a 64-bit M2 install but
used the DVD 64-bit Classical Install. The M2 install was
created using an up to date local repo, a completely blank
drive and the M2 boot.iso CD ( 64-bit, 1 Jun 12 ). The
install of M2 went normally and successfully. I then put 
some bookmarks in Firefox and rearranged the desktop slightly.
I then went through the update process and a few files were
updated. I set the boot process to automatically boot to the
user workspace. No other apps, features and/or functions were
installed. I then did a successful reboot back to workspace.
All of this is on an i7 platform that runs M2 just fine.
I did not install:

/core/updates_testing/mageia-prepare-upgrade-2-0.3.mga2.noarch.rpm

I then booted the M3 64-Bit Classical DVD installer ( 24 Apr 13 )
9a4f80d6b03b5c88a2242cda8078f0b0  Mageia-3-RC-x86_64-DVD.iso  24 Apr 13
installed on a USB drive and rebooted the system. The install
upgrade process proceeded normally and the system booted back
to a M3 working desktop.

I then redirected the system media source from the Classical Installer
DVD to my local ( updated ) repo and executed an update. 328 files
were updated. I rebooted the system and it successfully booted back
to a working M3 desktop.

I will re-run the test using the 64-bit boot.iso shortly.
Comment 24 William Kenney 2013-05-08 04:47:03 CEST
Using exactly the same repo as used in successful upgrade used
in Comment 23.

I again attempted upgrading a 64-bit M2 install using an
up-to-date local cauldron repo. The M2 install was created
using an up to date local repo, a completely blank drive and
the M2 boot.iso CD ( 64-bit, 1 Jun 12 ). The install of M2
went normally and successfully. I then put some bookmarks
in Firefox and rearranged the desktop slightly. I then went
through the update process and a few files were updated.
I set the boot process to automatically boot to the user
workspace. No other apps, features and/or functions were
installed. I then did a successful reboot back to workspace.
All of this is on an i7 platform that runs M2 just fine.
I did not install:

/core/updates_testing/mageia-prepare-upgrade-2-0.3.mga2.noarch.rpm

I then booted the M3 64-Bit Cauldron boot.iso ( 1 May 13 )
installed on a USB drive and rebooted the system. I instructed
the installer to upgrade the just installed M2. About halfway
into the install a large number of errors were reported failed.

Using a Puppy Linux 5.5 Live-CD I was able to capture
/var/log/syslog and the /root/drakx directory. Those
files are contained in a ZIP file attached to this bug.
I had to pull the .log1 files as it created a ZIP file
larger then 1MB.

I went no further then generating this data. This is
all done on real hardware and can be duplicated. But
do note it takes me more then 2-hours++ to go through
this process.
Comment 25 William Kenney 2013-05-08 04:48:10 CEST
Created attachment 3901 [details]
Attempt to upgrade from M2 -> M3 cauldron fails, 7May13
Comment 26 Dave Hodgins 2013-05-08 05:31:22 CEST
I don't see any errors in those log files.

Only thing I see that's strange, is the repo using
ftp://xxx:xxx@192.168.1.2//distrib/3/x86_64

On my repo, updated within the last half hour
$ ll distrib
total 12
drwxr-xr-x 1 2000 2000 4096 May 11  2011 1/
drwxr-xr-x 1 2000 2000 4096 Jul 19  2012 2/
drwxr-xr-x 1 2000 2000 4096 Apr 25 10:48 cauldron/

The 3 directory doesn't exist yet.

CC: (none) => davidwhodgins

Comment 27 Jürgen Preuà 2013-05-08 22:05:04 CEST
(In reply to William Kenney from comment #24)
...
> I then booted the M3 64-Bit Cauldron boot.iso ( 1 May 13 )
> installed on a USB drive and rebooted the system. I instructed
> the installer to upgrade the just installed M2. About halfway
> into the install a large number of errors were reported failed.
...

Hallo William,

would you mind me proving next time in installation cycle the status of the two core perl packages (mentioned above)?

In /mnt/var/cache/urpmi/rpms you shall find uninstalled packages, which were not installable because of missing dependencies. There I didn't found the core perl packages. So it wasn't tried to install it until break.
If you manages to install in this state the core perl packages from installation media, system should be able to install most of the packages. You have to resume to graphical installation window, quit error message and try again to install. This time cached content of urpmi should be installed/emptied.

I'm interested to hear your measures.
Good luck, with regards Juergen
Comment 28 Morgan Leijström 2013-05-09 12:00:32 CEST
@ William, as per Comment 26, just a shot: what happens if you on your local repo duplicate the folder distrib/cauldron/ to distrib/3/ ?

CC: (none) => fri

Comment 29 William Kenney 2013-05-09 16:57:16 CEST
(In reply to Morgan Leijström from comment #28)

> @ William, as per Comment 26, just a shot: what happens if you on your local
> repo duplicate the folder distrib/cauldron/ to distrib/3/ ?

Good point. In fact the local cauldron repo is in fact distrib/3/.
Here's the script code that's run every day at 04:05AM:

rsync -aH --delete  rsync://mirrors.kernel.org/mirrors/mageia/distrib/cauldron/i586/      /home/mageia/distrib/3/i586

and


rsync -aH --delete  rsync://mirrors.kernel.org/mirrors/mageia/distrib/cauldron/x86_64/    /home/mageia/distrib/3/x86_64
Comment 30 Samuel Verschelde 2013-08-27 15:23:43 CEST
Mageia 3 has been released since, what must we do with this bug report? My proposal: let's close it since there are a lot of comments and it's hard to follow. If bugs are still valid, please open a new bug report, preferrably against cauldron.

Status: NEW => RESOLVED
CC: (none) => stormi
Resolution: (none) => OLD

Comment 31 David Walser 2013-08-27 15:30:21 CEST
Or against Mageia 3 if there are issues in individual packages that can be fixed with updates to help upgrading from Mageia 2 work better.

CC: (none) => luigiwalser


Note You need to log in before you can comment on or make changes to this bug.