Bug 15350 - Upgrade failed mga4 to mga5 when adding online media to DVD - 195 transactions failed
Summary: Upgrade failed mga4 to mga5 when adding online media to DVD - 195 transaction...
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: Cauldron
Hardware: x86_64 Linux
Priority: release_blocker normal
Target Milestone: ---
Assignee: Thierry Vignaud
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 14069 15013 15637
  Show dependency treegraph
 
Reported: 2015-02-24 15:38 CET by claire robinson
Modified: 2018-09-14 15:44 CEST (History)
8 users (show)

See Also:
Source RPM:
CVE:
Status comment:


Attachments
report.bug.xz (201.64 KB, application/x-xz)
2015-02-24 15:40 CET, claire robinson
Details
report.bug.xz (199.56 KB, application/x-xz)
2015-03-09 17:49 CET, claire robinson
Details
report.bug.xz with debug_urpmi (199.30 KB, application/x-xz)
2015-03-10 11:44 CET, claire robinson
Details
report.bug from Claire's VM: OK (192.55 KB, application/octet-stream)
2015-03-13 06:43 CET, Thierry Vignaud
Details
report.bug.xz from RC build 3 (196.78 KB, application/x-xz)
2015-03-17 10:39 CET, claire robinson
Details
patch to put on a USB key to attach to the VM (3.49 KB, application/octet-stream)
2015-03-17 12:53 CET, Thierry Vignaud
Details
report.bug.xz with patch used (hopefully) (197.98 KB, application/x-xz)
2015-03-17 15:29 CET, claire robinson
Details
(check) fix reporting problems (1.49 KB, patch)
2015-03-24 10:30 CET, Thierry Vignaud
Details | Diff
report obsoletse issues (772 bytes, patch)
2015-03-24 11:16 CET, Thierry Vignaud
Details | Diff
report29-03-15.bug.xz (201.01 KB, application/x-xz)
2015-03-30 11:20 CEST, claire robinson
Details
report01-04-15.bug.xz (199.24 KB, application/x-xz)
2015-04-02 12:54 CEST, claire robinson
Details
compressed bug command output (261.36 KB, application/octet-stream)
2015-04-07 17:47 CEST, Dave Hodgins
Details
Initial ananlyse of packages causing transaction failures (1023 bytes, text/plain)
2015-04-07 18:22 CEST, Dave Hodgins
Details
Patch which may help (944 bytes, patch)
2015-04-12 02:40 CEST, Pascal Terjan
Details | Diff
sort by ids (like in other place) when promoting b/c of eg: conflicts (549 bytes, patch)
2015-04-12 02:40 CEST, Thierry Vignaud
Details | Diff
more complete fix (1.40 KB, patch)
2015-04-12 02:56 CEST, Thierry Vignaud
Details | Diff
one more fix :-( (493 bytes, patch)
2015-04-12 05:14 CEST, Thierry Vignaud
Details | Diff

Description claire robinson 2015-02-24 15:38:50 CET
Attempted upgrade from mga4 to mga5 using 5b3 dvd 64 with online medias added.

Two script failures related to gnome-icon-theme (bug 14971)

195 transactions failed

report.bug.xz attached

Reproducible: 

Steps to Reproduce:
claire robinson 2015-02-24 15:39:33 CET

Priority: Normal => release_blocker
CC: (none) => ennael1, tmb
Blocks: (none) => 15013

Comment 1 claire robinson 2015-02-24 15:40:01 CET
Created attachment 5943 [details]
report.bug.xz
Comment 2 Thierry Vignaud 2015-02-27 21:24:36 CET
strange (10 vs 9 & util-linux disapeared from the transaction):

remove=0= update=10=lib64systemd0,lib64mount1,findutils,gawk,lib64uuid1,lib64cap-ng0,lib64blkid1,util-linux,lib64smartcols1,lib64sigsegv2

(...)

installing lib64sigsegv2-2.10-5.mga5.x86_64.rpm lib64blkid1-2.25.2-3.mga5.x86_64.rpm lib64cap-ng0-0.7.4-5.mga5.x86_64.rpm gawk-4.1.1-3.mga5.x86_64.rpm lib64uuid1-2.25.2-3.mga5.x86_64.rpm lib64smartcols1-2.25.2-3.mga5.x86_64.rpm findutils-4.5.14-3.mga5.x86_64.rpm lib64systemd0-217-8.mga5.x86_64.rpm lib64mount1-2.25.2-3.mga5.x86_64.rpm from /tmp/image/media/core
starting installing packages
created transaction for installing on /mnt (remove=0, install=0, upgrade=9)

Maybe either rpmlib's rpmtsCheck() or rpmtsOrder() fscked sg?

CC: (none) => thierry.vignaud

Comment 3 Anne Nicolas 2015-03-09 09:56:22 CET
Claire, is it still valid with RC iso ?
Comment 4 claire robinson 2015-03-09 15:54:56 CET
I'll test now, still have the mga4 snapshot.
Comment 5 claire robinson 2015-03-09 17:46:28 CET
204 failed, with local http frozen medias added during upgrade.

I'll add an updated log
Comment 6 claire robinson 2015-03-09 17:49:31 CET
Created attachment 6020 [details]
report.bug.xz
Comment 7 Thierry Vignaud 2015-03-09 21:15:24 CET
Can you try again?
But instead of typing '<ENTER>' at the CD prompt, can you type "linux debug_urpmi"?
Florian Hubold 2015-03-09 21:27:04 CET

CC: (none) => doktor5000

Comment 8 claire robinson 2015-03-09 21:41:56 CET
Will check this in the morning Thierry, thanks
Rémi Verschelde 2015-03-09 21:44:37 CET

CC: (none) => remi
Blocks: (none) => 14069

Comment 9 claire robinson 2015-03-10 11:44:53 CET
Created attachment 6030 [details]
report.bug.xz with debug_urpmi

The ISO doesn't pause to enter 'linux debug_urpmi' in the same way as boot.iso but entered debug_urpmi as a kernel option at the boot menu isolinux/gfboot, not sure of the terminology.
Comment 10 claire robinson 2015-03-10 11:52:14 CET
Should have said .. "223 installation transactions failed" this time.
Comment 11 Thierry Vignaud 2015-03-11 14:06:48 CET
I suspect that there's issue with deps loops and thus librpm breaks at random places thus we got broken transactions.

I guess you've a snapshot of a VM you keep trying update?
So that you can easily revert to pre-update state?

If yes, could you:
- start it,
- removee all urpmi media,
- add mga5 media,
- run the following commands:
urpmi --bug bug15350 --auto-select --auto -v --debug --deploops 2>&1|tee bug15350.txt
tar cfa bug15350{.tar.xz,}

- then:
  o attach the bug15350.txt log to this bug report
  o send me the bug15350.tar.xz file (it will be
    too big for bugzilla to attach)

Keywords: (none) => NEEDINFO

Comment 12 claire robinson 2015-03-11 14:12:36 CET
Yes, it is, thats right. I'll do that now.
Comment 13 claire robinson 2015-03-11 16:21:09 CET
Email emailed Thierry, thanks.

It's 4.8Mb but should be OK gmail to gmail.
Comment 14 Thierry Vignaud 2015-03-11 21:15:55 CET
Can you try again with --no-priority-upgrade?
That should be more similar to drakx.
If urpmi still doesn't fail, it may be b/c you installed some soft for which the updates are not on the DVD.
Comment 15 claire robinson 2015-03-11 23:39:24 CET
I added online medias during upgrade from the ISO too.

You want me to use this..?

urpmi --bug bug15350 --auto-select --auto -v --debug --deploops --no-priority-upgrade 2>&1|tee bug15350.txt
Comment 16 Thierry Vignaud 2015-03-12 06:40:27 CET
Could you run urpmi with only the media from the ISO and not the online media?
Comment 17 claire robinson 2015-03-12 09:23:36 CET
I'm not sure :\ I'll give it a go!
Comment 18 claire robinson 2015-03-12 11:12:29 CET
Email emailed again Thierry.
claire robinson 2015-03-12 11:31:16 CET

Keywords: NEEDINFO => (none)

Comment 19 claire robinson 2015-03-12 13:23:19 CET
There's been some email debugging so adding the current command used, for future reference..

# urpmi --bug bug15350-3 --auto-select --auto -v --debug --deploops --previous-priority-upgrade='rpm,perl-URPM,perl-MDV-Distribconf,urpmi,meta-task,glibc,aria2' 2>&1|tee bug15350-3.txt
Comment 20 Thierry Vignaud 2015-03-13 06:43:32 CET
Created attachment 6056 [details]
report.bug from Claire's VM: OK

I've imported Claire's VM.
I've tried an update with Mageia-5-rc-x86_64-DVD.iso.
It worked smoothly
Comment 21 Thierry Vignaud 2015-03-13 06:46:35 CET
The ISO I tested is:
-rw-r--r-- 1 tv users 3901751296 Meu   3 09:15 Mageia-5-rc-x86_64-DVD.iso


A difference being you added online media
Comment 22 claire robinson 2015-03-16 21:42:04 CET
Link to the VM

https://drive.google.com/file/d/0B6AjnXi2R6HcNk1GQkdGT3IwVnM/view?usp=sharing

user/pass = user/user & root/root
Comment 23 claire robinson 2015-03-17 10:39:33 CET
Created attachment 6072 [details]
report.bug.xz from RC build 3

Valid RC build 3 (16th March)

191 transactions failed
claire robinson 2015-03-17 10:39:56 CET

Hardware: i586 => x86_64

Comment 24 Thierry Vignaud 2015-03-17 12:46:50 CET
Does the bug only happen if you add online media?

Keywords: (none) => NEEDINFO

Comment 25 Thierry Vignaud 2015-03-17 12:53:32 CET
Created attachment 6073 [details]
patch to put on a USB key to attach to the VM

Could you add this patch file on a USB key that you'll attach to the VM prior to starting the stage2?

See:
https://wiki.mageia.org/en/Drakx-installer_tips_and_tricks#Using_a_patch_.26_regular_ISO

Then add the "deploops" option on the command line.

Let's see if we see more interesting stuff in report.bug
Comment 26 claire robinson 2015-03-17 15:05:44 CET
I'll have to do some tests to confirm it's only when online media and iso media are being used together.

At the moment I've dropped patch.pl onto a USB stick which I atteched to the VM before starting it. I added "patch deploops" to the kernel options in isolinux/gfxboot.

Added online medias and it's currently upgrading. I'll attach report.bug again when it's done.

It seemed to do something different at the end of stage 1. There was an extra message, which I've forgotten of course.
Comment 27 claire robinson 2015-03-17 15:29:58 CET
Created attachment 6074 [details]
report.bug.xz with patch used (hopefully)

194 failed this time
Comment 28 claire robinson 2015-03-17 16:15:53 CET
Upgrade completes using only ISO media, all except a script error for ibus-gtk3 which I'll create a bug for now.
Thierry Vignaud 2015-03-17 17:41:04 CET

Summary: Upgrade failed mga4 to mga5 - 195 transactions failed => Upgrade failed mga4 to mga5 when adding online media to DVD - 195 transactions failed

Comment 29 Mageia Robot 2015-03-17 21:42:06 CET
commit cf9265e5981c5fe24c9e654935600d2348e2ba82
Author: Thierry Vignaud <thierry.vignaud@...>
Date:   Tue Mar 17 21:39:40 2015 +0100

    add 'deploops' option for debugging (mga#15350)
---
 Commit Link:
   http://gitweb.mageia.org/software/drakx/commit/?id=cf9265e5981c5fe24c9e654935600d2348e2ba82
Comment 30 Chris Denice 2015-03-18 11:38:40 CET
Sorry for the naive question, but is that just not the online media being Cauldron and therefore continuously updated during install?

I posted on the mailing list last year similar issues with network install during beta releases of mga4. Because our mirrors are not frozen beta snapshot, the install fails after a while with a random number of missed transactions due to a few packages being actually changed during the install. On the bright side, this does not happen anymore when the mirror are really frozen.

CC: (none) => dirteat

Comment 31 Rémi Verschelde 2015-03-18 11:54:19 CET
That's a relevant remark, but IMO that's not the issue here. MrsB has done tons of failed upgrade tests, and cauldron is in hard freeze currently with very few packages being pushed, and most of them are not relevant for upgrades (e.g. games). So it's unlikely that Claire hit a mirror issue so many times.
Comment 32 Anne Nicolas 2015-03-23 22:38:28 CET
Thierry, need some more information about this one ?
Comment 33 Thierry Vignaud 2015-03-24 10:16:06 CET
It would be nice to try again with latest ISO which include the right support for "deploops" option (previous patch was incomplete)
Comment 34 Thierry Vignaud 2015-03-24 10:30:00 CET
Created attachment 6129 [details]
(check) fix reporting problems

Though I suspect many issues were hidden from urpmi since at least 2003 :-(, ie calling check() actually makes urpmi ignore potential issues.

It could be nice if either you or Thomas could:
- locally build a new perl-URPM with that patch
- then rebuild stage2 with that new perl-URPM installed
- then redo a test ISO with that new stage2
(could be done by temporary rename install/stage2/mdkinst.sqfs in your build tree and putting the new patched one there)

So that Claire (or someone else) could try with it, aka a DVD install with adding online media.
Comment 35 Anne Nicolas 2015-03-24 10:56:39 CET
ok I'll do that
Comment 36 Thierry Vignaud 2015-03-24 11:16:21 CET
Created attachment 6130 [details]
report obsoletse issues

also, we would ignore obsolete issues reported by rpmlib since rpm-4.9.0
Comment 37 Anne Nicolas 2015-03-24 11:18:19 CET
ok adding also that patch
Comment 38 claire robinson 2015-03-30 10:36:17 CEST
Testing vbox upgrade now with latest RC ISO with deploops option. I'll attach a log when it's finished/failed.

ISO:  Mageia-5-RC-x86_64-DVD.iso
DATE: Sun Mar 29 15:06:35 CEST 2015
MD5:  OK     SHA1: OK

I saw this problem on another mga4 install yesterday on real hardware so I don't think the particular install matters so much.
Comment 39 claire robinson 2015-03-30 11:20:23 CEST
Created attachment 6158 [details]
report29-03-15.bug.xz
claire robinson 2015-03-30 11:25:25 CEST

Keywords: NEEDINFO => (none)

Comment 40 claire robinson 2015-04-02 12:54:39 CEST
Created attachment 6174 [details]
report01-04-15.bug.xz

Updated deploops upgrade log for new ISOs.
Comment 41 Anne Nicolas 2015-04-05 23:19:54 CEST
Thierry any input on that one?
Marja Van Waes 2015-04-06 22:39:20 CEST

Blocks: (none) => 15637

Comment 42 Dave Hodgins 2015-04-06 22:41:53 CEST
I just completed a test install, using the latest RC x86_64 dvd, with online
media added, selecting everything except individual packages.

No transaction errors.

Closing the bug. Please reopen, if other selections do result in transaction
errors.

Status: NEW => RESOLVED
CC: (none) => davidwhodgins
Resolution: (none) => FIXED

Comment 43 Dave Hodgins 2015-04-06 22:48:10 CEST
Oops. Tested a clean install. Reopening, until I can test an
upgrade.

Status: RESOLVED => REOPENED
Resolution: FIXED => (none)

Comment 44 Dave Hodgins 2015-04-07 06:27:35 CEST
Starting with a Mageia 4 x86_64 install, done with the
boot-nonfree.iso, selecting all package groups except
individual packages results in 266 transactions failed,
starting with perl-base, apparently due to a version
specific require in the Mageia 4 version of the perl
package.

In Mageia 4 ...
$ rpm -q --requires perl|grep base
perl-base = 2:5.18.1-3.2.mga4
perl-base >= 2:5.18.1

I expect that if perl-base and perl are installed in
the same transaction, that the problem would not show
up, which may explain why it doesn't always happen.

I'll dig through the logs tomorrow, to try and figure
out which other packages are triggering the problem.

Just realized it didn't save the logs. I'll have to
redo the upgrade (about 4 hours, for over 3000 packages),
and use the bug command to get the logs.
Comment 45 Thierry Vignaud 2015-04-07 09:40:54 CEST
You can boot the DVD by typing "linux tune-rpm" instead of just <enter> (which implies "linux").
It'll make librpm skips costly fsync() calls making installation faster...

Status: REOPENED => NEW

Comment 46 claire robinson 2015-04-07 10:18:53 CEST
This is still valid, current logs in comment 40.
Comment 47 Dave Hodgins 2015-04-07 17:47:23 CEST
Created attachment 6212 [details]
compressed bug command output

The failures start at line 36619
Comment 48 Dave Hodgins 2015-04-07 18:22:48 CEST
Created attachment 6213 [details]
Initial ananlyse of packages causing transaction failures
Comment 49 Dave Hodgins 2015-04-07 20:47:39 CEST
After discussing with David Walser on irc, instead of playing
whack-a-mole trying to fix the version specific requires,
urpmi should be fixed, to ensure packages with version specific
requires are installed in the same transaction as the required
package, or that the required package is installed in an earlier
transaction.

Assignee: bugsquad => thierry.vignaud

Comment 50 Thomas Backlund 2015-04-07 20:57:34 CEST
That's already the case. but if there are broken deps that creates dependency loops, there is nothing urpmi can do as rpm will break the loop arbitrary in order to try and get the install done instead of getting stuck in an endless loop.
Comment 51 Dave Hodgins 2015-04-08 01:53:35 CEST
As per comment 48 /tmp/image/media/core/perl-base-5.20.1-8.mga5.x86_64.rpm
is in the second transaction (line 36579), and fails as perl is not included
in that transaction, and perl has version specific requires on perl-base.

The perl package, /tmp/image/media/core/perl-5.20.1-8.mga5.x86_64.rpm isn't
attempted until a later transaction at line 36681.

Either we need a way to force urpmi to keep required/requiring packages in
the same transaction, or we need an automated way to locate and replace all
version specific requires with >= requires, in both Mageia 4 and 5.

As per comment 17, this install attempt has shown at least 17 rpm packages
where packages with version specific requires failed, so fixing the requires
in Mageia 4 and 5, will affect an unknown number of packages.
Comment 52 Thierry Vignaud 2015-04-11 19:58:12 CEST
That's a bogus explanation since it works fine with only the DVD's media...
Comment 53 Pascal Terjan 2015-04-11 21:43:26 CEST
"Fixing" the requires will break things. They are valid and needed.

There is another problem with some packages, yet to determine.

This is the transaction urpmi is creating, containing everything needed:

remove=0= update=90=perl-XML-Parser,gcc-cpp,perl-Curses,lib64gio2.0_0,perl-BSD-Resource,harddrake-ui,glib2.0-common,perl-Term-ReadKey,userdrake,lib64ncursesw5,frozen-bubble,perl-File-FnMatch,perl-Data-Dump,microcode,gtk+2.0,perl-Digest-SHA1,perl,perl-Apache-Test,lib64gtk+-x11-2.0_0,locales-en,perl-Compress-Bzip2,perl-SVN,lib64rpm3,gcc,libgcc1,perl-Glib,drakxtools-curses,perl-String-CRC32,subversion,samba-client,perl-Glib-Object-Introspection,vim-common,perl-URPM,perl-Crypt-SSLeay,lib64glib2.0_0,perl-WWW-Curl,lib64ncursesw-devel,rpm-build,lib64glib2.0-devel,perl-Data-Flow,perl-Cairo,perl-Filesys-Df,perl-HTML-Parser,glibc,rpm,glib-gettextize,lib64ncurses5,run-parts,perl-SDL,perl-Term-ReadLine-Gnu,perl-Linux-Pid,perl-DBD-mysql,perl-Gtk2,locales,bash,perl-Pango,perl-Gnome2-Vte,apache-mod_perl,perl-Net-DBus,lib64gtk+2.0_0,perl-Cairo-GObject,perl-File-Sync,perl-List-MoreUtils,transfugdrake,libstdc++6,libstdc++-devel,perl-Locale-gettext,perl-devel,lib64SDL_gfx15,drakxtools-backend,perl-DBI,drakxtools,lib64rpmbuild3,harddrake,perl-Archive-Zip,perl-base,lib64svn0,vim-enhanced,perl-Gtk2-WebKit,perl-Net-SSLeay,lib64ncurses-devel,perl-XML-LibXML,dash-static,filesystem,setup,gcc-c++,glibc-devel,python-rpm,lib64lua5.2,ncurses

But then forseome reason (probably a loop) it gets split and the first transaction only contains 74 packages instead of 90:

retrieving rpm files from medium "Core Release2"...
retrieved   subversion-1.8.13-1.mga5.x86_64.rpm harddrake-16.77-1.mga5.x86_64.rpm harddrake-ui-16.77-1.mga5.x86_64.rpm perl-SDL-2.544.0-5.mga5.x86_64.rpm perl-Archive-Zip-1.380.0-3.mga5.noarch.rpm lib64svn0-1.8.13-1.mga5.x86_64.rpm perl-Compress-Bzip2-2.180.0-5.mga5.x86_64.rpm perl-SVN-1.8.13-1.mga5.x86_64.rpm frozen-bubble-2.212.0-24.mga5.x86_64.rpm drakxtools-backend-16.77-1.mga5.x86_64.rpm drakxtools-16.77-1.mga5.x86_64.rpm drakxtools-curses-16.77-1.mga5.x86_64.rpm
...retrieving done
installing /tmp/image/media/core/perl-URPM-5.05-1.mga5.x86_64.rpm
/tmp/image/media/core/vim-common-7.4.430-7.mga5.x86_64.rpm
/tmp/image/media/core/perl-Glib-Object-Introspection-0.25.0-2.mga5.x86_64.rpm
/mnt/var/cache/urpmi/rpms/subversion-1.8.13-1.mga5.x86_64.rpm
/tmp/image/media/core/samba-client-3.6.25-1.mga5.x86_64.rpm
/tmp/image/media/core/perl-String-CRC32-1.500.0-5.mga5.x86_64.rpm
/mnt/var/cache/urpmi/rpms/drakxtools-curses-16.77-1.mga5.x86_64.rpm
/tmp/image/media/core/libgcc1-4.9.2-4.mga5.x86_64.rpm
/tmp/image/media/core/perl-Glib-1.305.0-5.mga5.x86_64.rpm
/tmp/image/media/core/lib64rpm3-4.12.0.1-20.mga5.x86_64.rpm
/tmp/image/media/core/gcc-4.9.2-4.mga5.x86_64.rpm
/mnt/var/cache/urpmi/rpms/perl-SVN-1.8.13-1.mga5.x86_64.rpm
/tmp/image/media/core/perl-Gtk2-1.249.500-2.mga5.x86_64.rpm
/tmp/image/media/core/perl-Filesys-Df-0.920.0-13.mga5.x86_64.rpm
/tmp/image/media/core/perl-Data-Flow-1.20.0-5.mga5.noarch.rpm
/tmp/image/media/core/lib64glib2.0-devel-2.42.1-1.mga5.x86_64.rpm
/tmp/image/media/core/lib64ncursesw-devel-5.9-21.mga5.x86_64.rpm
/tmp/image/media/core/perl-WWW-Curl-4.170.0-6.mga5.x86_64.rpm
/tmp/image/media/core/lib64glib2.0_0-2.42.1-1.mga5.x86_64.rpm
/tmp/image/media/core/perl-Term-ReadKey-2.320.0-4.mga5.x86_64.rpm
/tmp/image/media/core/perl-List-MoreUtils-0.330.0-12.mga5.x86_64.rpm
/tmp/image/media/core/glib2.0-common-2.42.1-1.mga5.x86_64.rpm
/mnt/var/cache/urpmi/rpms/harddrake-ui-16.77-1.mga5.x86_64.rpm
/tmp/image/media/core/perl-BSD-Resource-1.290.700-6.mga5.x86_64.rpm
/tmp/image/media/core/gcc-cpp-4.9.2-4.mga5.x86_64.rpm
/tmp/image/media/core/perl-XML-Parser-2.410.0-9.mga5.x86_64.rpm
/tmp/image/media/core/python-rpm-4.12.0.1-20.mga5.x86_64.rpm
/mnt/var/cache/urpmi/rpms/perl-Compress-Bzip2-2.180.0-5.mga5.x86_64.rpm
/tmp/image/media/core/lib64gtk+-x11-2.0_0-2.24.26-2.mga5.x86_64.rpm
/tmp/image/media/core/perl-Apache-Test-1.380.0-6.mga5.noarch.rpm
/tmp/image/media/core/perl-Digest-SHA1-2.130.0-15.mga5.x86_64.rpm
/tmp/image/media/core/gtk+2.0-2.24.26-2.mga5.x86_64.rpm
/mnt/var/cache/urpmi/rpms/frozen-bubble-2.212.0-24.mga5.x86_64.rpm
/tmp/image/media/core/perl-Data-Dump-1.220.0-5.mga5.noarch.rpm
/tmp/image/media/core/lib64ncursesw5-5.9-21.mga5.x86_64.rpm
/tmp/image/media/core/userdrake-2.8-1.mga5.x86_64.rpm
/mnt/var/cache/urpmi/rpms/lib64svn0-1.8.13-1.mga5.x86_64.rpm
/tmp/image/media/core/perl-base-5.20.1-8.mga5.x86_64.rpm
/mnt/var/cache/urpmi/rpms/perl-Archive-Zip-1.380.0-3.mga5.noarch.rpm
/mnt/var/cache/urpmi/rpms/harddrake-16.77-1.mga5.x86_64.rpm
/tmp/image/media/core/lib64rpmbuild3-4.12.0.1-20.mga5.x86_64.rpm
/mnt/var/cache/urpmi/rpms/drakxtools-16.77-1.mga5.x86_64.rpm
/tmp/image/media/core/perl-DBI-1.631.0-6.mga5.x86_64.rpm
/mnt/var/cache/urpmi/rpms/drakxtools-backend-16.77-1.mga5.x86_64.rpm
/tmp/image/media/core/lib64SDL_gfx15-2.0.25-4.mga5.x86_64.rpm
/tmp/image/media/core/perl-XML-LibXML-2.11.600-7.mga5.x86_64.rpm
/tmp/image/media/nonfree/microcode-0.20140913-4.mga5.nonfree.noarch.rpm
/tmp/image/media/core/ncurses-5.9-21.mga5.x86_64.rpm
/tmp/image/media/core/lib64lua5.2-5.2.3-6.mga5.x86_64.rpm
/tmp/image/media/core/glibc-devel-2.20-16.mga5.x86_64.rpm
/tmp/image/media/core/gcc-c++-4.9.2-4.mga5.x86_64.rpm
/tmp/image/media/core/setup-2.7.21-4.mga5.noarch.rpm
/tmp/image/media/core/filesystem-2.1.9-25.mga5.x86_64.rpm
/tmp/image/media/core/dash-static-0.5.7-6.mga5.x86_64.rpm
/tmp/image/media/core/lib64ncurses-devel-5.9-21.mga5.x86_64.rpm
/tmp/image/media/core/perl-Gtk2-WebKit-0.90.0-11.mga5.x86_64.rpm
/mnt/var/cache/urpmi/rpms/perl-SDL-2.544.0-5.mga5.x86_64.rpm
/tmp/image/media/core/run-parts-4.4-4.mga5.x86_64.rpm
/tmp/image/media/core/lib64ncurses5-5.9-21.mga5.x86_64.rpm
/tmp/image/media/core/glib-gettextize-2.42.1-1.mga5.x86_64.rpm
/tmp/image/media/core/locales-en-2.20-4.mga5.x86_64.rpm
/tmp/image/media/core/rpm-4.12.0.1-20.mga5.x86_64.rpm
/tmp/image/media/core/glibc-2.20-16.mga5.x86_64.rpm
/tmp/image/media/core/perl-HTML-Parser-3.710.0-5.mga5.x86_64.rpm
/tmp/image/media/core/libstdc++6-4.9.2-4.mga5.x86_64.rpm
/tmp/image/media/core/libstdc++-devel-4.9.2-4.mga5.x86_64.rpm
/tmp/image/media/core/transfugdrake-1.9.7-1.mga5.noarch.rpm
/tmp/image/media/core/apache-mod_perl-2.0.7-17.20130221svn1448245.mga5.x86_64.rpm
/tmp/image/media/core/lib64gtk+2.0_0-2.24.26-2.mga5.x86_64.rpm
/tmp/image/media/core/perl-Gnome2-Vte-0.110.0-5.mga5.x86_64.rpm
/tmp/image/media/core/perl-Pango-1.226-5.mga5.x86_64.rpm
/tmp/image/media/core/bash-4.3-33.1.mga5.x86_64.rpm
/tmp/image/media/core/locales-2.20-4.mga5.x86_64.rpm
/tmp/image/media/core/perl-DBD-mysql-4.28.0-3.mga5.x86_64.rpm
starting installing packages
created transaction for installing on /mnt (remove=0, install=0, upgrade=74)
Installation failed:
	lib64gio2.0_0 = 2.42.1 is needed by lib64glib2.0-devel-2.42.1-1.mga5.x86_64
	lib64glib2.0_0 = 2.38.2 is needed by (installed) lib64gio2.0_0-2.38.2-4.mga4.x86_64
	perl-base = 2:5.18.1-3.2.mga4 is needed by (installed) perl-2:5.18.1-3.2.mga4.x86_64
	perl-base = 2:5.18.1 is needed by (installed) vim-enhanced-7.4.052-3.mga4.x86_64
	perlapi-5.18.1 is needed by (installed) perl-File-Sync-0.110.0-6.mga4.x86_64
	perlapi-5.18.1 is needed by (installed) perl-File-FnMatch-0.20.0-10.mga4.x86_64
	perlapi-5.18.1 is needed by (installed) perl-Locale-gettext-1.50.0-10.mga4.x86_64
	perlapi-5.18.1 is needed by (installed) perl-Cairo-1.104.0-2.mga4.x86_64
	perlapi-5.18.1 is needed by (installed) perl-Net-SSLeay-1.550.0-2.mga4.x86_64
	perlapi-5.18.1 is needed by (installed) perl-Curses-1.280.0-11.mga4.x86_64
	perlapi-5.18.1 is needed by (installed) perl-Linux-Pid-0.40.0-4.mga4.x86_64
	perlapi-5.18.1 is needed by (installed) perl-Cairo-GObject-1.1.0-5.mga4.x86_64
	perlapi-5.18.1 is needed by (installed) perl-Crypt-SSLeay-0.640.0-4.mga4.x86_64
	perlapi-5.18.1 is needed by (installed) perl-Net-DBus-1.0.0-5.mga4.x86_64
	perlapi-5.18.1 is needed by (installed) perl-Term-ReadLine-Gnu-1.200.0-10.mga4.x86_64
	rpm = 1:4.11.1-9.1.mga4 is needed by (installed) rpm-build-1:4.11.1-9.1.mga4.x86_64

List of packages urpmi wanted in the first transaction which were excluded:

< lib64gio2.0_0
< perl
< perl-Cairo
< perl-Cairo-GObject
< perl-Crypt-SSLeay
< perl-Curses
< perl-File-FnMatch
< perl-File-Sync
< perl-Linux-Pid
< perl-Locale-gettext
< perl-Net-DBus
< perl-Net-SSLeay
< perl-Term-ReadLine-Gnu
< perl-devel
< rpm-build
< vim-enhanced
Comment 54 Pascal Terjan 2015-04-11 21:51:38 CEST
strange thing, in setSelectedFromCompssList, lib64gio2.0_0 is listed twice:

... lib64gimp2.0_0 lib64gio2.0_0 lib64gio2.0_0 lib64girepository-gir2.0 ...

Also in "rpms sorted by dependencies":

dash-static
glibc+setup+filesystem+bash+lib64ncurses5+libgcc1+libstdc++6+ncurses+run-parts
lib64ffi6
lib64pcre1
lib64glib2.0_0
lib64expat1
lib64dbnss5.3
lib64zlib1
lib64crack2
lib64audit1
lib64pam0
lib64gio2.0_0
lib64nspr4
lib64mozjs185_1.0
lib64gio2.0_0
lib64lzma5

CC: (none) => pterjan

Comment 55 Pascal Terjan 2015-04-11 22:06:21 CEST
Duplicates in rpms sorted by dependencies:

$ sort deps | uniq -d
kernel-firmware-nonfree
lib64gio2.0_0
locales-en
microcode
perl-Cairo
perl-File-FnMatch
perl-List-MoreUtils
perl-Locale-gettext
python-rpm
radeon-firmware
Comment 56 Pascal Terjan 2015-04-11 22:17:11 CEST
Some are very strange, for example kernel-firmware-nonfree does not require anything and is not required by anything.
Comment 57 Pascal Terjan 2015-04-11 22:51:23 CEST
When I run it here:

installed lib64gio2.0_0-2.38.2-4.mga4.x86_64 is conflicting because of unsatisfied lib64glib2.0_0[== 2.38.2] lib64glib2.0_0[*][== 2.38.2]
promoting lib64gio2.0_0-2.42.1-1.mga5.x86_64 because of conflict above

In the report.bug:

* installed lib64gio2.0_0-2.38.2-4.mga4.x86_64 is conflicting because of unsatisfied lib64glib2.0_0[*][== 2.38.2] lib64glib2.0_0[== 2.38.2]
* promoting lib64gio2.0_0-2.42.1-1.mga5.x86_64 lib64gio2.0_0-2.42.1-1.mga5.x86_64 because of conflict above

I am not using the install media, only cauldron.
Downloading an iso to try again with it + cauldron.
Comment 58 Pascal Terjan 2015-04-11 23:02:50 CEST
OK that line is unrelated, it is indeed twice there when in 2 media, but still only once in the list sorted by dependencies:

installed lib64gio2.0_0-2.38.2-4.mga4.x86_64 is conflicting because of unsatisfied lib64glib2.0_0[== 2.38.2] lib64glib2.0_0[*][== 2.38.2]
promoting lib64gio2.0_0-2.42.1-1.mga5.x86_64 lib64gio2.0_0-2.42.1-1.mga5.x86_64 because of conflict above
Comment 59 Pascal Terjan 2015-04-11 23:42:20 CEST
other duplicate I had not noticed as it's on a single line, perl:

perl-base+perl+perl+perl-parent+perl-Apache-Test

which gives me more like it:

e2fsprogs
gtk
lib64gtk
perl
perl-Gtk2
perl-Net-DBus
perl-URPM
perl-XML-LibXML
plymouth
sound-scripts
util-linux
Comment 60 Thierry Vignaud 2015-04-11 23:48:28 CEST
similar issue in my run:

13 vs 14 and thus boom

* scheduled sets of transactions:
remove=0= update=14=libgcc1,microcode,lib64ncurses5,libstdc++6,bash,setup,locales,glibc,lib64ncursesw5,ncurses,filesystem,dash-static,run-parts,locales-en


installing setup-2.7.21-4.mga5.noarch.rpm bash-4.3-33.1.mga5.x86_64.rpm lib64ncursesw5-5.9-21.mga5.x86_64.rpm glibc-2.20-16.mga5.x86_64.rpm locales-2.20-4.mga5.x86_64.rpm libgcc1-4.9.2-4.mga5.x86_64.rpm libstdc++6-4.9.2-4.mga5.x86_64.rpm lib64ncurses5-5.9-21.mga5.x86_64.rpm locales-en-2.20-4.mga5.x86_64.rpm run-parts-4.4-4.mga5.x86_64.rpm dash-static-0.5.7-6.mga5.x86_64.rpm filesystem-2.1.9-25.mga5.x86_64.rpm ncurses-5.9-21.mga5.x86_64.rpm from /tmp/image/media/core
starting installing packages
created transaction for installing on /mnt (remove=0, install=0, upgrade=13)
Installation failed:
        microcode < 0.20140913-2 conflicts with glibc-6:2.20-16.mga5.x86_64
Comment 61 Pascal Terjan 2015-04-11 23:56:43 CEST
Interesting that only one package was excluded, microcode while it doesn't require anything or is required by anything.
Comment 62 Pascal Terjan 2015-04-11 23:57:45 CEST
But as the error points out, it has a conflict:

microcode:glibc[< 6:2.20-11]
Comment 63 Pascal Terjan 2015-04-11 23:58:56 CEST
Other side of the conflict:

glibc:microcode[< 0.20140913-2]
Comment 64 Pascal Terjan 2015-04-12 00:19:29 CEST
I had missed comment #2 pointing out another package missing from a transaction (util-linux).

It was also part of the packages listed twice in the "sorted by dependencies" list:

e2fsprogs
gtk
kernel-firmware-nonfree
lib64gio2.0_0
lib64gtk
locales-en
microcode
perl
perl-Cairo
perl-File-FnMatch
perl-Gtk2
perl-List-MoreUtils
perl-Locale-gettext
perl-Net-DBus
perl-URPM
perl-XML-LibXML
plymouth
python-rpm
radeon-firmware
sound-scripts
util-linux

Could it be that urpmi moves it to another transaction when it sees it the second time, and it ends up not put in the first one?
Comment 65 Pascal Terjan 2015-04-12 00:21:15 CEST
In this example lib64gio2.0_0 is in 2 transactions, and gets dropped from the first one:

remove=0= update=90=perl-XML-Parser,gcc-cpp,perl-Curses,lib64gio2.0_0,perl-BSD-Resource,harddrake-ui,glib2.0-common,perl-Term-ReadKey,userdrake,lib64ncursesw5,frozen-bubble,perl-File-FnMatch,perl-Data-Dump,microcode,gtk+2.0,perl-Digest-SHA1,perl,perl-Apache-Test,lib64gtk+-x11-2.0_0,locales-en,perl-Compress-Bzip2,perl-SVN,lib64rpm3,gcc,libgcc1,perl-Glib,drakxtools-curses,perl-String-CRC32,subversion,samba-client,perl-Glib-Object-Introspection,vim-common,perl-URPM,perl-Crypt-SSLeay,lib64glib2.0_0,perl-WWW-Curl,lib64ncursesw-devel,rpm-build,lib64glib2.0-devel,perl-Data-Flow,perl-Cairo,perl-Filesys-Df,perl-HTML-Parser,glibc,rpm,glib-gettextize,lib64ncurses5,run-parts,perl-SDL,perl-Term-ReadLine-Gnu,perl-Linux-Pid,perl-DBD-mysql,perl-Gtk2,locales,bash,perl-Pango,perl-Gnome2-Vte,apache-mod_perl,perl-Net-DBus,lib64gtk+2.0_0,perl-Cairo-GObject,perl-File-Sync,perl-List-MoreUtils,transfugdrake,libstdc++6,libstdc++-devel,perl-Locale-gettext,perl-devel,lib64SDL_gfx15,drakxtools-backend,perl-DBI,drakxtools,lib64rpmbuild3,harddrake,perl-Archive-Zip,perl-base,lib64svn0,vim-enhanced,perl-Gtk2-WebKit,perl-Net-SSLeay,lib64ncurses-devel,perl-XML-LibXML,dash-static,filesystem,setup,gcc-c++,glibc-devel,python-rpm,lib64lua5.2,ncurses

remove=0= update=9=lib64pam0,lib64lzma-devel,lib64lzma5,lib64nspr4,lib64gdbm4,lib64gio2.0_0,lib64mozjs185_1.0,lib64db5.3,xz
Comment 66 Thierry Vignaud 2015-04-12 00:31:54 CEST
so there would be 2 issues:
- duplicate packages
- removing package from 1st transaction when seen a 2nd tile
Comment 67 Pascal Terjan 2015-04-12 00:41:25 CEST
Yes, I think so.

So far looking through the code and reading the log I would believe they have to have different ids, so probably the same package gets selected from both media.

I couldn't find where it would get dropped, still reading.
Comment 68 Thierry Vignaud 2015-04-12 01:35:34 CEST
in urpm::main_loop::run(), in my case, microcode is part of the first $set, but it's not present anymore in $transaction_blists returned by urpm::install::prepare_transaction()

It's likely that the duplicate packages cause issue in urpm::get_pkgs::selected2local_and_blists()
Comment 69 Pascal Terjan 2015-04-12 02:40:31 CEST
Created attachment 6239 [details]
Patch which may help

This patch should help avoid duplicates
Comment 70 Thierry Vignaud 2015-04-12 02:40:39 CEST
Created attachment 6240 [details]
sort by ids (like in other place) when promoting b/c of eg: conflicts
Comment 71 Thierry Vignaud 2015-04-12 02:52:13 CEST
Comment on attachment 6213 [details]
Initial ananlyse of packages causing transaction failures

sort {$a <=> $b}

Attachment 6213 is obsolete: 0 => 1

Thierry Vignaud 2015-04-12 02:53:32 CEST

Attachment 6158 is obsolete: 0 => 1

Thierry Vignaud 2015-04-12 02:54:09 CEST

Attachment 6074 is obsolete: 0 => 1

Thierry Vignaud 2015-04-12 02:54:10 CEST

Attachment 6073 is obsolete: 0 => 1

Thierry Vignaud 2015-04-12 02:54:11 CEST

Attachment 6072 is obsolete: 0 => 1

Thierry Vignaud 2015-04-12 02:54:14 CEST

Attachment 5943 is obsolete: 0 => 1

Thierry Vignaud 2015-04-12 02:54:21 CEST

Attachment 6030 is obsolete: 0 => 1

Thierry Vignaud 2015-04-12 02:54:23 CEST

Attachment 6020 is obsolete: 0 => 1

Comment 72 Thierry Vignaud 2015-04-12 02:56:15 CEST
Created attachment 6241 [details]
more complete fix

Pascal, I didn't saw your patch, I'll check if it's OK too.

Attachment 6240 is obsolete: 0 => 1

Comment 73 Pascal Terjan 2015-04-12 03:01:52 CEST
Mine may or may not be useful, yours look more interesting for fixing this bug :)
Comment 74 Thierry Vignaud 2015-04-12 03:19:48 CEST
Sadly there's still a failure b/c of shorewall around 3/4 of the upgrade :-(
Comment 75 Thierry Vignaud 2015-04-12 03:27:10 CEST
I think we'll need both patches.
Waiting for a full upgrade.
If you want faster testing, you can use drakx-in-chroot on the mounted ISO:
tools/drakx-in-chroot $PWD/iso/x86_64 /T2 --useless_thing_accepted --justdb --flang fr --keyboard fr --tune-rpm --debug_urpmi --lang fr --stage2-update any.pm /usr/lib/libDrakX/install/any.pm --stage2-update main_loop.pm /usr/lib/perl5/vendor_perl/5.20.1/urpm/main_loop.pm --stage2-update install.pm /usr/lib/perl5/vendor_perl/5.20.1/urpm/install.pm --stage2-update Resolve.pm /usr/lib/perl5/vendor_perl/5.20.1/x86_64-linux-thread-multi/URPM/Resolve.pm --stage2-update steps_interactive.pm /usr/lib/libDrakX/install/steps_interactive.pm

I've local patches for forcing my local mirror w/o typing its URL in ask_mirror() and for spiking media screen

If that's OK, I'll still have to run urpmi testsuite
Comment 76 Thierry Vignaud 2015-04-12 05:14:04 CEST
Created attachment 6242 [details]
one more fix :-(
Comment 77 Pascal Terjan 2015-04-12 18:58:39 CEST
(In reply to Thierry Vignaud from comment #76)
> Created attachment 6242 [details]
> one more fix :-(

I was looking into why this happen with installer but not with urpmi --auto-select and ended up in that place, so this one may be the most important.
Comment 78 Thierry Vignaud 2015-04-12 19:57:49 CEST
Well with all of those, upgrade completed last night.
Comment 79 Thierry Vignaud 2015-04-12 20:54:39 CEST
Comment on attachment 6239 [details]
Patch which may help

Sorry your patch doesn't help.
But we need both my URPM patch & my drakx one.
Neither is enough

Attachment 6239 is obsolete: 0 => 1

Comment 80 Mageia Robot 2015-04-12 21:26:18 CEST
commit fb1aa950f8a70e05c123a48bcab369465037cb0f
Author: Thierry Vignaud <thierry.vignaud@...>
Date:   Sun Apr 12 20:55:47 2015 +0200

    fix upgrade when package's in several media
    
    sort by ids (like in other places) when finding providing package, thus
    fixing upgrade when the same package is offered in several media
    (mga#15350)
---
 Commit Link:
   http://gitweb.mageia.org/software/drakx/commit/?id=fb1aa950f8a70e05c123a48bcab369465037cb0f
Comment 81 Mageia Robot 2015-04-12 22:04:39 CEST
commit ffb2d910aa08ff5feb4337454af10ecc03bfd0c3
Author: Thierry Vignaud <thierry.vignaud@...>
Date:   Sun Apr 12 02:39:09 2015 +0200

    fix upgrade when package's in several media
    
    sort by ids (like in other places) when promoting (b/c of eg:
    conflicts), thus fixing upgrade when the same package is offered in
    several media (mga#15350)
    
    see commit 53e38beae4d9638f41b219521b802a2969d68909 for similar issue
---
 Commit Link:
   http://gitweb.mageia.org/software/rpm/perl-URPM/commit/?id=ffb2d910aa08ff5feb4337454af10ecc03bfd0c3
Comment 82 Thierry Vignaud 2015-04-12 23:00:09 CEST
Closing

Status: NEW => RESOLVED
Resolution: (none) => FIXED

Comment 83 claire robinson 2015-04-13 09:41:28 CEST
Great job tracking it down!
Comment 84 claire robinson 2015-04-16 18:09:52 CEST
Confirmed fixed with latest isos (april 15th)
Comment 85 Mageia Robot 2016-10-14 12:20:08 CEST
commit e134398e5535ddf0e4fbda067eeffd7581a9b339
Author: Thierry Vignaud <thierry.vignaud@...>
Date:   Tue Mar 24 05:08:14 2015 -0400

    (check) fix reporting problems
    
    for at least 12 years, since swiching rpm to 4.2
    (see commit 60031191b7012fdfe8e1af6bd43ff9b36b0c5825)
    $trans->check() failed to actually report issues
    
    rationale: rpmtsCheck() only actually return !0 if it fails to open
    rpmdb...
    
    in order to check if any problem was found by rpmtsCheck(), one must call
    retrieving the problem set with rpmtsProblems()
    
    rpmtsCheck() success only means that the resolution was successfully
    attempted for all packages in the set, which isn't that usefull...
    
    this might help mga#15350...
---
 Commit Link:
   http://gitweb.mageia.org/software/rpm/perl-URPM/commit/?id=e134398e5535ddf0e4fbda067eeffd7581a9b339
Comment 86 Thierry Vignaud 2018-09-14 15:44:08 CEST
oh robot works again?
I've 1 one commit in urpmi & one in drakx and voila, no more issue...

Note You need to log in before you can comment on or make changes to this bug.