Bug 23037 - installer and mga5->mga6 upgrade failures after Grand Update (URPM incorrectly selects obsolete lib64kf5ksieveui_5 package; subsequent dependencies cause many KDE package conflicts)
Summary: installer and mga5->mga6 upgrade failures after Grand Update (URPM incorrectl...
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: 6
Hardware: All Linux
Priority: release_blocker critical
Target Milestone: ---
Assignee: KDE maintainers
QA Contact:
URL:
Whiteboard:
Keywords: PATCH
Depends on:
Blocks: 21340
  Show dependency treegraph
 
Reported: 2018-05-15 18:06 CEST by William Kenney
Modified: 2018-06-24 22:40 CEST (History)
8 users (show)

See Also:
Source RPM: task-obsolete, kdeplasma-addons, plasma-desktop, perl-URPM
CVE:
Status comment:


Attachments
netinstall of M6 after Grand Update fails (632.49 KB, image/png)
2018-05-15 18:07 CEST, William Kenney
Details
report.bug from failed installed (395.07 KB, application/gzip)
2018-05-15 21:50 CEST, Martin Whitaker
Details
report.bug from failed install (398.38 KB, application/gzip)
2018-05-18 01:17 CEST, Martin Whitaker
Details
report.bug from failed install with kde-l10n-* removed from rpmsrate (388.16 KB, application/gzip)
2018-05-18 09:48 CEST, Martin Whitaker
Details
netinstall errors 1 (173.10 KB, image/png)
2018-05-25 15:30 CEST, William Kenney
Details
netinstall errors 2 (181.36 KB, image/png)
2018-05-25 15:30 CEST, William Kenney
Details
Installer log after rpmsrate and bug 23065 have been fixed (416.38 KB, application/x-xz)
2018-06-02 23:08 CEST, Martin Whitaker
Details
patch to perl-URPM that fixes this bug (needs improvement) (824 bytes, text/plain)
2018-06-03 13:49 CEST, Martin Whitaker
Details
Log file from upgrade test (64.34 KB, application/x-xz)
2018-06-12 20:40 CEST, Martin Whitaker
Details
remove doble obsolete on kde l10n pkgs (5.74 KB, patch)
2018-06-13 16:28 CEST, Thierry Vignaud
Details | Diff
update logs diff (87.59 KB, application/x-xz)
2018-06-13 16:35 CEST, Thierry Vignaud
Details
fix finding better pkg (mga#23037) (1.56 KB, patch)
2018-06-14 00:21 CEST, Thierry Vignaud
Details | Diff
update logs diff (73.02 KB, application/x-xz)
2018-06-14 00:52 CEST, Thierry Vignaud
Details
Latest upgrade conflicts (298.62 KB, image/png)
2018-06-19 21:16 CEST, Martin Whitaker
Details

Description William Kenney 2018-05-15 18:06:31 CEST
Description of problem:

On real hardware, M6, Plasma, 64-bit

Start by using:
Mageia-6-x86_64-DVD.iso 7/15/17
md5sum: 55e20da532496124e6e720896fdf9fe4

Install to empty drive, Plasma, the install updates correctly
using my local repo mirrored to http://mirrors.kernel.org/
hdlist.cz 5/12/18
c76232e257a237dfbe5bd122cd47fdf9  hdlist.cz
49ee2cee4a7d56663a409a5fff5d8712  synthesis.hdlist.cz
752db2ec3b9b72ccf95a4693e576e359  info.xml.lzma
b92da09530f934cfad94d72033f79248  files.xml.lzma
49d27b1435ca67b1bcfe96c6bb9d3bcb  changelog.xml.lzma
c76232e257a237dfbe5bd122cd47fdf9  20180512-075049-hdlist.cz
49ee2cee4a7d56663a409a5fff5d8712  20180512-075049-synthesis.hdlist.cz
752db2ec3b9b72ccf95a4693e576e359  20180512-075049-info.xml.lzma
b92da09530f934cfad94d72033f79248  20180512-075049-files.xml.lzma
49d27b1435ca67b1bcfe96c6bb9d3bcb  20180512-075049-changelog.xml.lzma

Next use:
Mageia-6-netinstall-x86_64.iso 7/6/17
md5sum: 7b3a11f29a8225ac119221bdbf3bccfb

Very soon after install starts hundreds of packages are reported
as failed and install stops. Photo capture of just a few is attached.
Comment 1 William Kenney 2018-05-15 18:07:23 CEST
Created attachment 10154 [details]
netinstall of M6 after Grand Update fails
Comment 2 William Kenney 2018-05-15 18:41:37 CEST
Same problem occurs in a Vbox client.
Comment 3 William Kenney 2018-05-15 19:08:02 CEST
In another new Vbox client change the repo from my
local repo mirrored to mirrors.kernel.org to going
directly to mirrors.kernel.org I get the same thing.
Comment 4 Martin Whitaker 2018-05-15 21:50:47 CEST
Created attachment 10155 [details]
report.bug from failed installed

Reproduced in VirtualBox. The report generated by 'bug' is attached.

I suspect this is related to bug 23016 - there are a number of 32-bit packages being selected and installed, and the eventual failure is due to a conflict between the 32-bit and 64-bit packages.

CC: (none) => mageia

Comment 5 William Kenney 2018-05-15 22:15:38 CEST
We await the outcome of Bug 23016
Comment 6 Marja Van Waes 2018-05-17 09:23:37 CEST
(In reply to Martin Whitaker from comment #4)

> I suspect this is related to bug 23016 - there are a number of 32-bit
> packages being selected and installed, and the eventual failure is due to a
> conflict between the 32-bit and 64-bit packages.

(In reply to William Kenney from comment #5)
> We await the outcome of Bug 23016

Thanks for the tests :-)

That bug is fixed. I this one still valid?

CC: (none) => marja11

Marja Van Waes 2018-05-17 09:24:30 CEST

Source RPM: Mageia-6-x86_64-DVD.iso => (none)

Comment 7 William Kenney 2018-05-17 19:08:57 CEST
In a Vbox client, M6, Plasma, 64-bit

Using:
Mageia-6-netinstall-x86_64.iso 7/6/17
md5sum: 7b3a11f29a8225ac119221bdbf3bccfb

Install continues to fail for a large number of i586 packages.
Comment 8 Nicolas Lécureuil 2018-05-17 21:22:06 CEST
can you please provide some logs ? so we can fix the conflicts

CC: (none) => mageia

Comment 9 William Kenney 2018-05-18 01:01:31 CEST
(In reply to Nicolas Lécureuil from comment #8)

> can you please provide some logs ? so we can fix the conflicts

I've got to think about how to do that.
Comment 10 Martin Whitaker 2018-05-18 01:17:25 CEST
Created attachment 10164 [details]
report.bug from failed install

(In reply to Nicolas Lécureuil from comment #8)
> can you please provide some logs ? so we can fix the conflicts

Updated log, as per comment 4, but using latest updates.

Attachment 10155 is obsolete: 0 => 1

Comment 11 William Kenney 2018-05-18 01:27:35 CEST
Your the best Martin, thanks

1572 of them?
Comment 12 Nicolas Lécureuil 2018-05-18 08:12:46 CEST
* file /usr/bin/libwacom-list-local-devices from install of libwacom2-1:0.24-1.mga6.i586 conflicts with file from package lib64wacom2-1:0.24-1.mga6.x86_64

Installation failed:
	file /usr/bin/libwacom-list-local-devices from install of libwacom2-1:0.24-1.mga6.i586 conflicts with file from package lib64wacom2-1:0.24-1.mga6.x86_64


in the installer, something requires kde-l10n-* ?
Comment 13 Martin Whitaker 2018-05-18 09:48:45 CEST
Created attachment 10166 [details]
report.bug from failed install with kde-l10n-* removed from rpmsrate

(In reply to Nicolas Lécureuil from comment #12)
> in the installer, something requires kde-l10n-* ?

CAT_PLASMA5 in rpmsrate. But that doesn't seem to be the problem :-(

Attachment 10164 is obsolete: 0 => 1

Comment 14 Nicolas Lécureuil 2018-05-18 10:29:00 CEST
please  remove it. we do not have kde-l10n-* anymore.


Installation failed:
	file /usr/bin/libwacom-list-local-devices from install of libwacom2-1:0.24-1.mga6.i586 conflicts with file from package lib64wacom2-1:0.24-1.mga6.x86_64

this needs to be fixed too.

can someone tell a i586 or x86_64 only install ?
Comment 15 Martin Whitaker 2018-05-18 19:35:27 CEST
(In reply to Nicolas Lécureuil from comment #14)
> Installation failed:
> 	file /usr/bin/libwacom-list-local-devices from install of
> libwacom2-1:0.24-1.mga6.i586 conflicts with file from package
> lib64wacom2-1:0.24-1.mga6.x86_64
> 
> this needs to be fixed too.

I agree, but it's been like that for a long time...

A lot of 32-bit packages are being installed unnecessarily. The two culprits are:

* requested libpulseaudio0, lib64alsa-plugins-pulseaudio recommended by lib64pulseaudio0-10.0-1.1.mga6.x86_64
* selecting libpulseaudio0-10.0-1.1.mga6.i586

and

* requested kontact-handbook, kalarm, kaddressbook, korganizer, kmail, knotes recommended by kontact-17.12.2-1.mga6.x86_64
* selecting kmail-17.12.2-1.mga6.i586

In both cases, a 32-bit package is being erroneously selected instead of a 64-bit package. I guess this is a bug in the 64-bit package Provides.

> can someone tell a i586 or x86_64 only install ?

If you deselect the 32-bit repositories in the installer Media Selection screen, the 64-bit install completes without error.
Comment 16 Martin Whitaker 2018-05-18 20:16:16 CEST
For libpulseaudio0, it does appear to be deliberate. From the spec file:

%ifarch x86_64
# (cg) Suggest the 32 bit library on 64 bits to ensure compatibility
#      with (typically closed source) 32 bit apps.
Recommends: lib%{name}%{major}
%endif
Comment 17 Martin Whitaker 2018-05-18 20:35:33 CEST
This looks to be the explanation for kmail:

# urpmi --test --auto kmail-17.12.2-1.mga6
A requested package cannot be installed:
lib64kf5ksieveui_5-16.12.3-1.mga6.x86_64 (due to unsatisfied libksieve[== 2:16.12.3])

Without --auto:

# urpmi --test kmail-17.12.2-1.mga6.i586
In order to satisfy the 'libKF5Codecs.so.5' dependency, one of the following packages is needed:
 1- libkcodecs5-5.32.0-1.mga6.i586: KDE Frameworks 5 Tier 1 addon with string manipulation methods (to install)
 2- libkf5codecs5-5.42.0-1.1.mga6.i586: KDE Frameworks 5 Tier 1 addon with string manipulation methods (to install)
What is your choice? (1-2) 
In order to satisfy the 'libKF5KSieveUi.so.5' dependency, one of the following packages is needed:
 1- libkf5ksieveui_5-16.12.3-1.mga6.i586: This lib manages sieve support (to install)
 2- libkf5ksieveui5-17.12.2-5.mga6.i586: This lib manages sieve support (to install)
What is your choice? (1-2) 
A requested package cannot be installed:
libkf5ksieveui_5-16.12.3-1.mga6.i586 (due to unsatisfied libksieve[== 2:16.12.3])

Missing obsoletes?
Martin Whitaker 2018-05-20 09:43:19 CEST

See Also: (none) => https://bugs.mageia.org/show_bug.cgi?id=23065

Comment 18 Martin Whitaker 2018-05-20 09:51:24 CEST
I have created a new bug (bug 23065) for the libwacom2 conflict, as that doesn't stop this bug being fixed.

I have updated rpmsrate for you:

meta-task-6-2.1.mga6.src.rpm
meta-task-6-2.1.mga6.noarch.rpm

(also in cauldron).

That just leaves the libKF5 package dependencies (comment 17) to fix.

Component: Installer => RPM Packages

Comment 19 Nicolas Lécureuil 2018-05-20 10:41:38 CEST
#--------------------------------------------------------------------

%define kf5ksieve_major 5
%define libkf5ksieve %mklibname kf5ksieve %{kf5ksieve_major}

%package -n %libkf5ksieve
Summary:      This lib manages sieve support
Group:        System/Libraries
Requires:     %name = %epoch:%version

Obsoletes:    %{_lib}kf5ksieve_5 < 2:17.12.0
Provides:     %{_lib}kf5ksieve_5 = %epoch:%version-%release

%description -n %libkf5ksieve
This lib manages sieve support.

%files -n %libkf5ksieve
%_kf5_libdir/libKF5KSieve.so.%{kf5ksieve_major}*

#--------------------------------------------------------------------


I don't see what is wrong.
If someone sees  accept advices :)
Comment 20 Martin Whitaker 2018-05-20 13:27:19 CEST
(In reply to Nicolas Lécureuil from comment #19)
> I don't see what is wrong.
> If someone sees  accept advices :)

I don't know if this is the problem, but

# urpmq --provides lib64kf5ksieveui5
lib64kf5ksieveui5[== 2:17.12.2-5.mga6]
lib64kf5ksieveui5(x86-64)[== 2:17.12.2-5.mga6]
lib64kf5ksieveui_5[== 2:17.12.2-5.mga6]
libKF5KSieveUi.so.5()(64bit)
# urpmq --provides lib64kf5ksieveui_5
lib64kf5ksieveui_5[== 2:16.12.3-1.mga6]
lib64kf5ksieveui_5(x86-64)[== 2:16.12.3-1.mga6]
libKF5KSieveUi.so.5()(64bit)

Does lib64kf5ksieveui5 also need to provide lib64kf5ksieveui_5(x86-64) ?
Comment 21 William Kenney 2018-05-20 14:19:25 CEST
Further testing here indicates that using netinstall, removing the
32-bit repos, results in a clean install but I'm beginning to
find things that don't work. Like if you log out of user space
you can't log back in. More testing today to try and define
this.
Comment 22 William Kenney 2018-05-20 17:33:12 CEST
So if I do a netinstall with the 32-bit repos disabled the install
will complete correctly and boot to a working desktop. If I immediately
enable an auto boot to desktop then reboot the system it gets hung up
and never gets to the desktop. Is there not 32-bit apps that really
need to be installed?
Comment 23 Frank Griffin 2018-05-20 17:51:57 CEST
See also bug#22630

CC: (none) => ftg

Comment 24 William Kenney 2018-05-20 17:53:49 CEST
Ya, there's a lot of things going on here that we don't fully understand yet.
Comment 25 Martin Whitaker 2018-05-20 18:27:42 CEST
(In reply to William Kenney from comment #24)
> Ya, there's a lot of things going on here that we don't fully understand yet.

Please remember the golden rule: a new bug report for each bug. We do now have a reasonable idea of what's going on with this bug; let's not get sidetracked.

(not saying don't keep testing and reporting problems - just keep them separate. If a report contains multiple bugs, it's too easy for one to get overlooked)
Comment 26 William Kenney 2018-05-20 18:54:12 CEST
(In reply to Martin Whitaker from comment #25)

> Please remember the golden rule: a new bug report for each bug. We do now
> have a reasonable idea of what's going on with this bug; let's not get
> sidetracked.
> 
> (not saying don't keep testing and reporting problems - just keep them
> separate. If a report contains multiple bugs, it's too easy for one to get
> overlooked)

Yes, I'm struggling this morning getting a clean from the ground up install of M6 then updating from anything. I'm trying to get a clean new install to test the new Vbox and nvidia bugs. I'll probably back off for today and wait for the netinstall process work its way through.
Comment 27 William Kenney 2018-05-20 18:54:46 CEST
Thanks for the help Martin
Comment 28 William Kenney 2018-05-25 15:29:20 CEST
Sorry, I'm still seeing lots of errors on netinstall.
2 screen shots attached
Comment 29 William Kenney 2018-05-25 15:30:22 CEST
Created attachment 10189 [details]
netinstall errors 1
Comment 30 William Kenney 2018-05-25 15:30:48 CEST
Created attachment 10190 [details]
netinstall errors 2
Comment 31 Martin Whitaker 2018-05-25 21:40:03 CEST
Yes, libkf5ksieveui5 still needs to be fixed to properly obsolete libkf5ksieveui_5.

@Nicolas, did you see my suggestion in comment 20?
Comment 32 Dave Hodgins 2018-05-29 02:13:49 CEST
Still a problem, when upgrading i586 from m5 to m6.

Some requested packages cannot be installed:
libkf5ksieveui_5-16.12.3-1.mga6.i586 (due to unsatisfied libksieve[== 2:16.12.3])
Continue installation anyway?

CC: (none) => davidwhodgins

Dave Hodgins 2018-05-29 02:15:32 CEST

Blocks: (none) => 21340

Comment 33 Martin Whitaker 2018-06-02 23:08:57 CEST
Created attachment 10218 [details]
Installer log after rpmsrate and bug 23065 have been fixed

Now that the fixes to rpmsrate and bug 23065 have been pushed, the install completes without reporting an error. However, checking the installer log shows that the failure to properly obsolete libkf5ksieveui_5 is still preventing kmail from being installed and is still causing unnecessary i586 packages to be installed. Checking the installed systems shows 265 i586 packages are installed.

This still needs to be fixed.
Comment 34 William Kenney 2018-06-02 23:19:31 CEST
Getting there. Yipppeeee!!!!!
Comment 35 Martin Whitaker 2018-06-03 13:49:50 CEST
Created attachment 10221 [details]
patch to perl-URPM that fixes this bug (needs improvement)

I think this is a limitation/bug in perl-URPM. When it selects packages, it doesn't check whether any of the candidates are obsolete. The attached patch is a quick hack to prove this. When applied to the installer, the net install goes cleanly, with kmail installed and no unnecessary i586 packages installed.
Comment 36 Martin Whitaker 2018-06-03 13:51:15 CEST
@Thierry, WDYT? Is there a better way of fixing this?

Keywords: (none) => PATCH
CC: (none) => thierry.vignaud

Comment 37 Ben McMonagle 2018-06-04 08:16:00 CEST
(In reply to Dave Hodgins from comment #32)
> Still a problem, when upgrading i586 from m5 to m6.
> 
> Some requested packages cannot be installed:
> libkf5ksieveui_5-16.12.3-1.mga6.i586 (due to unsatisfied libksieve[==
> 2:16.12.3])
> Continue installation anyway?

same here today for i586 Mga5 upgrade

CC: (none) => westel

Comment 38 William Kenney 2018-06-04 15:50:17 CEST
Today I executed a netinstall of M7 and it did not exhibit this issue.
Comment 39 Martin Whitaker 2018-06-04 22:16:01 CEST
(In reply to ben mcmonagle from comment #37)
> (In reply to Dave Hodgins from comment #32)
> > Still a problem, when upgrading i586 from m5 to m6.
> > 
> > Some requested packages cannot be installed:
> > libkf5ksieveui_5-16.12.3-1.mga6.i586 (due to unsatisfied libksieve[==
> > 2:16.12.3])
> > Continue installation anyway?
> 
> same here today for i586 Mga5 upgrade

I've just tested and confirmed that the attached patch will fix this problem on upgrades too. Because perl-URPM is in the priority update list, it shouldn't be necessary to backport the fix to mga5.
Comment 40 Martin Whitaker 2018-06-07 10:02:33 CEST
[moving discussion from bug 21431]

(In reply to Morgan Leijström from bug 21431 comment #4)
> Hmm... do that message say it thinks it need libksieve[== 2:16.12.3] for
> lib64kf5managesieve5-17.12.2, if so it looks like a version dependency error
> to me.

No, it's because when you do an upgrade, both

lib64kf5ksieveui_5-16.12.3-1.mga6.x86_64

and

lib64kf5ksieveui5-17.12.2-5.mga6.x86_64

are available to be installed. Notice the subtle name change. This cause urpmi to treat them as different packages which both satisfy the same requirement. If you run urpmi without the --auto option, it asks you to choose which one you want. With the --auto option, it picks the first one in the list, which happens to be lib64kf5ksieveui_5-16.12.3-1.mga6.x86_64.

> But why is it OK in a installed mga6 system then...?

Because when you are updating a system with lib64kf5ksieveui_5 already installed, urpmi checks for obsoletes, and finds that lib64kf5ksieveui5 is meant to replace lib64kf5ksieveui_5. But it doesn't perform a cross-check for obsoletes when choosing between available packages. That's what my patch aims to fix.
Comment 41 Morgan Leijström 2018-06-07 10:41:20 CEST
Ah, thanks for the clarity :)

CC: (none) => fri

Comment 42 Martin Whitaker 2018-06-09 12:24:40 CEST
Setting to release blocker as this is blocking the re-enabling of mgaapplet upgrades and should also block any 6.1 ISO release.

Assigning to the RPM stack maintainers as that's where I think the fault lies (see comment 35).

Assignee: bugsquad => rpmstack
Source RPM: (none) => perl-URPM-5.12-1.mga6.src.rpm
Priority: Normal => release_blocker
Summary: netinstall fails after Grand Update => installer and mga5->mga6 upgrade failures after Grand Update (URPM incorrectly selects obsolete lib64kf5ksieveui_5 package; subsequent dependencies cause many KDE package conflicts)

Comment 43 Thierry Vignaud 2018-06-11 16:51:02 CEST
(In reply to Martin Whitaker from comment #35)
> I think this is a limitation/bug in perl-URPM. When it selects packages, it
> doesn't check whether any of the candidates are obsolete. The attached patch
> is a quick hack to prove this. When applied to the installer, the net
> install goes cleanly, with kmail installed and no unnecessary i586 packages
> installed.

(In reply to Martin Whitaker from comment #40)
err, I'd like more details, b/c from your logs:

* packageCallbackChoices: default choice ('lib64kf5ksieveui_5') from lib64kf5ksieveui_5,lib64kf5ksieveui5 for libKF5KSieveUi.so.5()(64bit)
* replacing libKF5KSieveUi.so.5()(64bit) with lib64kf5ksieveui_5
* selecting lib64kf5ksieveui_5-16.12.3-1.mga6.x86_64
(...)
* unselecting lib64kf5ksieveui_5-16.12.3-1.mga6.x86_64
(...)
* chosen lib64kf5ksieveui5-17.12.2-5.mga6.x86_64 for libKF5KSieveUi.so.5()(64bit)
* selecting lib64kf5ksieveui5-17.12.2-5.mga6.x86_64

Same for:

* unselecting libkf5ksieveui_5-16.12.3-1.mga6.i586
(...)
* chosen libkf5ksieveui5-17.12.2-5.mga6.i586 for libKF5KSieveUi.so.5
* selecting libkf5ksieveui5-17.12.2-5.mga6.i586

AFAIC the right package is selected.
Comment 44 Martin Whitaker 2018-06-11 20:08:57 CEST
(In reply to Thierry Vignaud from comment #43)
> (In reply to Martin Whitaker from comment #40)
> err, I'd like more details, b/c from your logs:
> 
> * packageCallbackChoices: default choice ('lib64kf5ksieveui_5') from
> lib64kf5ksieveui_5,lib64kf5ksieveui5 for libKF5KSieveUi.so.5()(64bit)
> * replacing libKF5KSieveUi.so.5()(64bit) with lib64kf5ksieveui_5
> * selecting lib64kf5ksieveui_5-16.12.3-1.mga6.x86_64
> (...)
> * unselecting lib64kf5ksieveui_5-16.12.3-1.mga6.x86_64
> (...)
> * chosen lib64kf5ksieveui5-17.12.2-5.mga6.x86_64 for
> libKF5KSieveUi.so.5()(64bit)
> * selecting lib64kf5ksieveui5-17.12.2-5.mga6.x86_64
> 
> Same for:
> 
> * unselecting libkf5ksieveui_5-16.12.3-1.mga6.i586
> (...)
> * chosen libkf5ksieveui5-17.12.2-5.mga6.i586 for libKF5KSieveUi.so.5
> * selecting libkf5ksieveui5-17.12.2-5.mga6.i586
> 
> AFAIC the right package is selected.

Yes, in the installer the wrong choice is eventually corrected (because the installer runs multiple selection passes). But by then the damage is done - kmail has been permanently deselected, and ~230 unnecessary i586 packages have been selected and get installed. OK, so this is no longer a fatal error (thanks to the fix for bug 23065), but it's still wrong.

The situation is more serious when running an online upgrade. There, only one selection pass is made, so the wrong choice is never corrected. This ends with the user being told that the upgrade cannot be completed due to package conflicts. Depending on the order that the dependencies were resolved, the number of conflicts listed can be in the hundreds. No doubt this could be corrected by running further update, but I don't think this is something the users of the upgrade applet should be expected to know or to do.
Comment 45 Thierry Vignaud 2018-06-12 17:03:10 CEST
I tried a live upgrade.
It installs task-plasma w/ & w/o your patch.
Your patch does install more pkgs (eg: kmail).
But it doesn't reduce the number of i586 pkgs to install/update (20 in both cases).

mkdir R
urpmi.addmedia --urpmi-root R ftp://ftp-stud.hs-esslingen.de/pub/Mirrors/Mageia/distrib/5/x86_64 --distrib
urpmi --urpmi-root R --justdb basesystem-minimal
urpmi --urpmi-root R --justdb basesystem
urpmi --urpmi-root R --justdb task-x11
urpmi --urpmi-root R --justdb task-kde4
# for easier reproduction:
tar cfz R{.tgz,}

urpmi.removemedia -a --urpmi-root R
urpmi.addmedia --urpmi-root R  /mageia/unstable/x86_64/ --distrib
tar cfz R{2.tgz,}

# stopping at summary, before installing pkgs:
urpmi --urpmi-root R --auto-select --justdb 2>&1|tee -a LOG.4update

cd /usr/lib/perl5/vendor_perl/5.26.1/x86_64-linux-thread-multi/
patch -p1 < /tmp/urpm-dont-select-obsolete-packages.patch
cd -

# stopping at summary, before installing pkgs:
urpmi --urpmi-root R --auto-select --justdb 2>&1|tee -a LOG.4update2
Comment 46 Thierry Vignaud 2018-06-12 17:12:25 CEST
but it consistently makes the urpmi testsuite to fail (t/superuser--obsolete-and-conflict.t):
Test Summary Report
-------------------
t/superuser--obsolete-and-conflict.t (Wstat: 1024 Tests: 26 Failed: 4)
  Failed tests:  9-10, 15-16
  Non-zero exit status: 4
Files=39, Tests=3248, 184 wallclock secs ( 0.78 usr  0.07 sys + 158.24 cusr 23.86 csys = 182.95 CPU)
Result: FAIL
Failed 1/39 test programs. 4/3248 subtests failed.

Reverting the URPM patch makes test the testsuite to pass again
Comment 47 Martin Whitaker 2018-06-12 20:40:59 CEST
Created attachment 10237 [details]
Log file from upgrade test

(In reply to Thierry Vignaud from comment #45)
> I tried a live upgrade.
> It installs task-plasma w/ & w/o your patch.
> Your patch does install more pkgs (eg: kmail).
> But it doesn't reduce the number of i586 pkgs to install/update (20 in both
> cases).

As said in my last comment, the unnecessary i586 packages come when running the installer:

% curl 'https://bugs.mageia.org/attachment.cgi?id=10218' > ddebug.log.xz
% xzcat ddebug.log.xz | grep 'scheduling update of' | grep i586 | wc
    265    2120   38595

About 10% of those 265 are pulled in by pulse-audio. The rest come from the installer trying to select the i586 version of kmail, having unselected the x86_64 version due to the lib64kf5sieveui conflicts.

For the upgrade failure, try:

urpmi.addmedia --urpmi-root R --distrib ftp://ftp.mirrorservice.org/pub/mageia/distrib/5/x86_64
urpmi --urpmi-root R --justdb --auto basesystem-minimal
urpmi --urpmi-root R --justdb --auto basesystem
urpmi --urpmi-root R --justdb --auto task-x11
urpmi --urpmi-root R --justdb --auto task-kde4
tar cfz R.tgz R
urpmi.removemedia --urpmi-root R -a
urpmi.addmedia --urpmi-root R --distrib ftp://ftp.mirrorservice.org/pub/mageia/distrib/6/x86_64
urpmi --urpmi-root R --justdb --auto-select --auto 2>&1| tee upgrade.log

Checking the end of the log, I find:

Installation failed:
...
While some packages may have been installed, there were failures.
Some requested packages cannot be installed:
...

Log file attached.
Comment 48 Martin Whitaker 2018-06-12 20:42:15 CEST
(In reply to Thierry Vignaud from comment #46)
> but it consistently makes the urpmi testsuite to fail
> (t/superuser--obsolete-and-conflict.t):
> Test Summary Report
> -------------------
> t/superuser--obsolete-and-conflict.t (Wstat: 1024 Tests: 26 Failed: 4)
>   Failed tests:  9-10, 15-16
>   Non-zero exit status: 4
> Files=39, Tests=3248, 184 wallclock secs ( 0.78 usr  0.07 sys + 158.24 cusr
> 23.86 csys = 182.95 CPU)
> Result: FAIL
> Failed 1/39 test programs. 4/3248 subtests failed.
> 
> Reverting the URPM patch makes test the testsuite to pass again

I did clearly label the patch as "needs improvement" ;-)
Comment 49 Dave Hodgins 2018-06-13 02:34:55 CEST
Just to be clear on the impact this bug is having.

Until it's solved, the mgaapplet upgrading from Mageia 5 to 6 will not be turned on. Limited support of security updates for Mageia 5 will continue. Production
of Mageia 6.1 iso images will not start. Mageia 7 scheduling will not start, let
alone any iso production or testing.

Severity: normal => critical

Comment 50 William Kenney 2018-06-13 03:51:28 CEST
(In reply to Dave Hodgins from comment #49)

> Just to be clear on the impact this bug is having......

:-(
Comment 51 Thierry Vignaud 2018-06-13 16:28:13 CEST
Created attachment 10239 [details]
remove doble obsolete on kde l10n pkgs

I just noticed that task-obsolete was wrongly obsoleting some pkgs already obsoleted by other pkgs.
1) that violates task-obsolete goal
2) URPM doesn't like it when several pkgs obsolete the same target

With that simple packaging fix, the update went quite a lot more smoothly for me
Thierry Vignaud 2018-06-13 16:28:22 CEST

Source RPM: perl-URPM-5.12-1.mga6.src.rpm => task-obsolete, perl-URPM-5.12-1.mga6.src.rpm

Comment 52 Thierry Vignaud 2018-06-13 16:32:15 CEST
I didn't check but there might be some other bogus kde4 obsoletes there.
Comment 53 Thierry Vignaud 2018-06-13 16:35:36 CEST
Created attachment 10240 [details]
update logs diff

I used:

# saying no just before actually installing:
urpmi --urpmi-root R --auto-select -vv --debug  --justdb  2>&1|tee -a LOG.4update3

echo '/task-obsolete-6-128.mga6/'>>R/etc/urpmi/skip.list
urpmi.addmedia --urpmi-root R OBSOLETE ~tv/mga/pkgs/6/task-obsolete/RPMS/noarch
# saying no just before actually installing:
urpmi --urpmi-root R --auto-select -vv --debug  --justdb  2>&1|tee -a LOG.4update4
Comment 54 Thierry Vignaud 2018-06-13 16:43:53 CEST
(In reply to Thierry Vignaud from comment #52)
> I didn't check but there might be some other bogus kde4 obsoletes there.

Using the following command & filtering its output, on the pkg set installed by the task pkgs in comment#45, we may want to fix or at least check those:

# egrep 'preferr*ing.*obsolet' LOG.4update4d|cut -f 3-9 -d\ |sort 
kmahjongglib4-14.12.3-2.1.mga6.noarch obsoleting kmahjongglib-4.14.3-1.mga5.noarch over kmahjongglib-17.12.2-1.mga6.noarch
libkdegames4-14.12.3-3.mga6.x86_64 obsoleting libkdegames-common-4.14.3-1.mga5.noarch over libkdegames-common-17.12.2-2.mga6.noarch
libkdegames4-14.12.3-3.mga6.x86_64 obsoleting libkdegames-corebindings-4.14.3-1.mga5.x86_64 over libkdegames-corebindings-17.12.2-2.mga6.x86_64
plasma-pa-5.12.2-1.mga6.x86_64 obsoleting kmix-4.14.3-1.mga5.x86_64 over kmix-17.12.2-1.mga6.x86_64
plasma-workspace-wallpapers-5.12.2-1.mga6.noarch obsoleting kde-wallpapers-4.14.3-1.mga5.noarch over kde-wallpapers-15.08.3-1.mga6.noarch

eg:
1) older versions of libkdegames4 & kmahjongglib4 should probably be obsoleted
2) the proper kmix replacement should be choosen between plasma-pa & kmix
3) same for kde-wallpapers

Assignee: rpmstack => kde

Comment 55 Thierry Vignaud 2018-06-13 16:49:10 CEST
After this, it's not perfect, but it's in a quite better shape.
The only remaining issue I saw is that on 50% of the update tentatives, it schedules to update to kdeplasma-addons-5.8.7-1.mga instead of kdeplasma-addons-5.12.2-1.mga
But that would likely be solved on next urpmi --auto-select.
Comment 56 Thierry Vignaud 2018-06-13 17:42:51 CEST
Actually this one should be solved by removing the obsoletes from either kdeplasma-addons or plasma-desktop:

# fgrep multiple LOG.4update4f5|cut -f 4-9 -d\ |sort 
(kdeartwork-15.08.3-3.mga6.x86_64 task-obsolete-6-128.2.mga7.noarch) obsoleting kdeartwork4-kscreensaver-4.14.3-1.mga5.x86_64
(kdeplasma-addons-5.12.2-1.mga6.x86_64 plasma-desktop-5.12.2-1.mga6.x86_64) obsoleting plasma-applet-frame-4.14.3-4.mga5.x86_64
(kdeplasma-addons-5.12.2-1.mga6.x86_64 plasma-desktop-5.12.2-1.mga6.x86_64) obsoleting plasma-applet-icontasks-4.14.3-4.mga5.x86_64
(kdeplasma-addons-5.12.2-1.mga6.x86_64 plasma-desktop-5.12.2-1.mga6.x86_64) obsoleting plasma-applet-showdashboard-4.14.3-4.mga5.x86_64
(kdeplasma-addons-5.12.2-1.mga6.x86_64 plasma-desktop-5.12.2-1.mga6.x86_64) obsoleting plasma-applet-showdesktop-4.14.3-4.mga5.x86_64
(kdeplasma-addons-5.12.2-1.mga6.x86_64 plasma-desktop-5.12.2-1.mga6.x86_64) obsoleting plasma-applet-weather-4.14.3-4.mga5.x86_64
(kdeplasma-addons-5.12.2-1.mga6.x86_64 plasma-desktop-5.12.2-1.mga6.x86_64) obsoleting plasma-applet-weatherstation-4.14.3-4.mga5.x86_64
(kdeplasma-addons-5.12.2-1.mga6.x86_64 plasma-desktop-5.12.2-1.mga6.x86_64) obsoleting plasma-runner-contacts-4.14.3-4.mga5.x86_64
(kdeplasma-addons-5.12.2-1.mga6.x86_64 plasma-desktop-5.12.2-1.mga6.x86_64) obsoleting plasma-runner-datetime-4.14.3-4.mga5.x86_64
(kdeplasma-addons-5.12.2-1.mga6.x86_64 plasma-desktop-5.12.2-1.mga6.x86_64) obsoleting plasma-runner-events-4.14.3-4.mga5.x86_64
(task-obsolete-6-128.2.mga7.noarch x11-driver-input-6.0.0-5.mga6.x86_64) obsoleting x11-driver-input-aiptek-1.4.1-14.mga5.x86_64

Source RPM: task-obsolete, perl-URPM-5.12-1.mga6.src.rpm => task-obsolete, kdeplasma-addons, plasma-desktop

Comment 57 Thierry Vignaud 2018-06-13 17:47:32 CEST
To be completed by the plasma pkgs (at least those mentioned in comment #56, though it would be best to also fix those listed in comment#54)

Advisory
========
The task-obsolete has been fixed in order not to obsolete some packages already obsoleted by the Plasma stack, thus fixing upgrade (mga#23037)

List of pkgs:
==============
task-obsolete-6-128.3.mga7.src.rpm
task-obsolete-6-128.3.mga7.noarch.rpm
Thierry Vignaud 2018-06-13 17:48:46 CEST

Attachment 10221 is obsolete: 0 => 1

Comment 58 Thierry Vignaud 2018-06-13 17:51:25 CEST
@martin, @neoclust: if you want, I can share my 13M R.txz tarball of my R/ chroot so that you can further test updates with the new task-obsolete and your future fixed Plasma pkgs.
Comment 59 Dave Hodgins 2018-06-14 00:11:29 CEST
For the advisory, the srpm/rpm names are mga6, not mga7.

For qa testing, we will just be testing that the updates all install cleanly
(assuming there will be additional kde updates) in Mageia 6 installs, so we can
get them moved from updates testing to updates, and will then close this bug
report.

At that point testing of upgrading using mgaapplet --testing will resume.
Comment 60 Thierry Vignaud 2018-06-14 00:21:06 CEST
Created attachment 10243 [details]
fix finding better pkg (mga#23037)

actually, we may well need an URPM patch for some cases.
My task-obsolete fixes were used on top of martin's patch.
So we do need an URPM fix.
With that fix, urpmi testsuite passes.
Thierry Vignaud 2018-06-14 00:23:48 CEST

Source RPM: task-obsolete, kdeplasma-addons, plasma-desktop => task-obsolete, kdeplasma-addons, plasma-desktop, perl-URPM

Comment 61 Thierry Vignaud 2018-06-14 00:36:48 CEST
Update:

Advisory
========
The task-obsolete package has been fixed in order not to obsolete some packages already obsoleted by the Plasma stack, thus fixing upgrade (mga#23037)

The perl-URPM package has been fixed in order to make sure to pick the right packages to upgrade when updating from the KDE4 stack to the Plasma stack (mga#23037).

List of pkgs:
==============
task-obsolete-6-128.3.mga6.src.rpm
task-obsolete-6-128.3.mga6.noarch.rpm
perl-URPM-5.12.1-1.mga6.src.rpm
perl-URPM-5.12.1-1.mga6.i586.rpm
perl-URPM-5.12.1-1.mga6.x86_64.rpm
perl-URPM-5.12.1-1.mga6.armv5tl.rpm
perl-URPM-5.12.1-1.mga6.armv7hl.rpm
perl-URPM-debuginfo-5.12.1-1.mga6.i586.rpm
perl-URPM-debuginfo-5.12.1-1.mga6.x86_64.rpm
perl-URPM-debuginfo-5.12.1-1.mga6.armv5tl.rpm
perl-URPM-debuginfo-5.12.1-1.mga6.armv7hl.rpm
Comment 62 Thierry Vignaud 2018-06-14 00:52:24 CEST
Created attachment 10244 [details]
update logs diff

The URPM patch improves the package selection.
But we would still need a fixed kdeplasma-addons in order to select the right kdeplasma-addons update on all runs
Comment 63 Dave Hodgins 2018-06-14 19:53:03 CEST
Nicolas, any eta on the plasma changes?
Comment 64 Nicolas Lécureuil 2018-06-14 21:43:54 CEST
(In reply to Thierry Vignaud from comment #58)
> @martin, @neoclust: if you want, I can share my 13M R.txz tarball of my R/
> chroot so that you can further test updates with the new task-obsolete and
> your future fixed Plasma pkgs.

thanks for your proposal i think this can help me to understand.
Comment 65 Ben McMonagle 2018-06-19 02:06:40 CEST
upgrading Mga5 i586 to  Mga6 i586 (with kde4 DE):

"some requested packages cannot be installed: 
task-obsolete-6-128.3.mga6.noarch (due to conflicts with task-obsolete-6-128.mga6.noarch.)
continue installation anyway? " 

(see also bug 23182)
Comment 66 Dave Hodgins 2018-06-19 15:15:53 CEST
Latest upgrade test had no kde related conflicts. Just the task-obsolete
conflict which is now in bug 23202, for the Mageia 5 version of perl-URPM.

Closing as fixed.

Status: NEW => RESOLVED
Resolution: (none) => FIXED

Comment 67 Martin Whitaker 2018-06-19 21:16:34 CEST
Created attachment 10246 [details]
Latest upgrade conflicts

(In reply to Dave Hodgins from comment #66)
> Latest upgrade test had no kde related conflicts. Just the task-obsolete
> conflict which is now in bug 23202, for the Mageia 5 version of perl-URPM.
> 
> Closing as fixed.

Not for me. First try gave just the task-obsolete conflicts described above. A second try, with an identical starting point, gave a load of conflicts. This may be a side effect of not being able to select the latest version of task-obsolete, so I won't reopen this bug just yet. But it does indicate we can't ignore bug 23202.
Comment 68 William Kenney 2018-06-19 21:25:14 CEST
I executed a Vbox netinstall today, M6 x86_64 Plasma, and all went just fine.
Martin Whitaker 2018-06-24 22:34:52 CEST

See Also: (none) => https://bugs.mageia.org/show_bug.cgi?id=23222

Martin Whitaker 2018-06-24 22:40:44 CEST

See Also: (none) => https://bugs.mageia.org/show_bug.cgi?id=23223


Note You need to log in before you can comment on or make changes to this bug.