Bug 17300

Summary: urpmi --auto-select (or --auto-update) doesn't install several packages b/c of kipi packages
Product: Mageia Reporter: Thomas Spuhler <thomas>
Component: RPM PackagesAssignee: Nicolas Lécureuil <mageia>
Status: RESOLVED DUPLICATE QA Contact:
Severity: normal    
Priority: Normal CC: bittwister2, ftg, marja11, thierry.vignaud, zen25000
Version: Cauldron   
Target Milestone: ---   
Hardware: All   
OS: Linux   
See Also: https://bugs.mageia.org/show_bug.cgi?id=17266
Whiteboard:
Source RPM: libkipi, kipi-plugins CVE:
Status comment:

Description Thomas Spuhler 2015-12-06 22:04:32 CET
Description of problem:
urpmi --auto-select (or --auto-update) doesn't install several packages, but many uf them install when doing urpmi "specific pacakge"

Version-Release number of selected component (if applicable):
This has been around at least since mga4

How reproducible:
occasionally.

Steps to Reproduce:
1. do urpmi --auto-update
2. get a list of up to 50 rpms
3. say  "Y"

See details:
https://pastebin.mozilla.org/8853980


Reproducible: 

Steps to Reproduce:
Thomas Spuhler 2015-12-06 22:05:47 CET

Assignee: bugsquad => thierry.vignaud

Comment 1 Rémi Verschelde 2015-12-06 22:08:54 CET
I'm not sure what the issue is apart from an obvious packaging issue in kipi-plugins?
Comment 2 Thomas Spuhler 2015-12-06 22:12:34 CET
maybe, I didn't express myself correctly. I am not complaining about the kipi-plugins. I told already on the dev list.

The problem is that a lot of packages, actaully all of the none kipi install when doing it explicitly.
# urpmi libquadmath0
Marking libquadmath0 as manually installed, it won't be auto-orphaned
writing /var/lib/rpm/installed-through-deps.list
To satisfy dependencies, the following packages are going to be installed:
  Package                        Version      Release       Arch    
(medium "Core Release (distrib1)")
  gcc                            5.3.0        2.mga6        x86_64  
  gcc-c++                        5.3.0        2.mga6        x86_64  
  gcc-cpp                        5.3.0        2.mga6        x86_64  
  gcc-gfortran                   5.3.0        2.mga6        x86_64  
  libgfortran3                   5.3.0        2.mga6        x86_64  
  libgomp-devel                  5.3.0        2.mga6        x86_64  
  libgomp1                       5.3.0        2.mga6        x86_64  
  libquadmath-devel              5.3.0        2.mga6        x86_64  
  libquadmath0                   5.3.0        2.mga6        x86_64  
  libstdc++-devel                5.3.0        2.mga6        x86_64  
  libstdc++6                     5.3.0        2.mga6        x86_64  
55KB of disk space will be freed.
35MB of packages will be retrieved.
Proceed with the installation of the 11 packages? (Y/n) 


installing libquadmath0-5.3.0-2.mga6.x86_64.rpm gcc-c++-5.3.0-2.mga6.x86_64.rpm libgomp-devel-5.3.0-2.mga6.x86_64.rpm gcc-gfortran-5.3.0-2.mga6.x86_64.rpm libstdc++6-5.3.0-2.mga6.x86_64.rpm libgfortran3-5.3.0-2.mga6.x86_64.rpm libquadmath-devel-5.3.0-2.mga6.x86_64.rpm libstdc++-devel-5.3.0-2.mga6.x86_64.rpm libgomp1-5.3.0-2.mga6.x86_64.rpm gcc-cpp-5.3.0-2.mga6.x86_64.rpm gcc-5.3.0-2.mga6.x86_64.rpm from /var/cache/urpmi/rpms
Preparing...                     #########################################################################################################
     1/11: libstdc++6            #########################################################################################################
     2/11: libstdc++-devel       #########################################################################################################
     3/11: gcc-cpp               #########################################################################################################
     4/11: gcc                   #########################################################################################################
     5/11: libquadmath-devel     #########################################################################################################
     6/11: libgomp1              #########################################################################################################
     7/11: libquadmath0          #########################################################################################################
     8/11: libgomp-devel         #########################################################################################################
     9/11: gcc-c++               #########################################################################################################
    10/11: libgfortran3          #########################################################################################################
    11/11: gcc-gfortran          #########################################################################################################
     1/11: removing gcc-gfortran-5.2.1-0.20151124.1.mga6.x86_64
                                 #########################################################################################################
     2/11: removing gcc-c++-5.2.1-0.20151124.1.mga6.x86_64
                                 #########################################################################################################
     3/11: removing libgomp-devel-5.2.1-0.20151124.1.mga6.x86_64
                                 #########################################################################################################
     4/11: removing libgfortran3-5.2.1-0.20151124.1.mga6.x86_64
                                 #########################################################################################################
     5/11: removing libquadmath0-5.2.1-0.20151124.1.mga6.x86_64
                                 #########################################################################################################
     6/11: removing libgomp1-5.2.1-0.20151124.1.mga6.x86_64
                                 #########################################################################################################
     7/11: removing libquadmath-devel-5.2.1-0.20151124.1.mga6.x86_64
                                 #########################################################################################################
     8/11: removing gcc-5.2.1-0.20151124.1.mga6.x86_64
                                 #########################################################################################################
     9/11: removing libstdc++-devel-5.2.1-0.20151124.1.mga6.x86_64
                                 #########################################################################################################
    10/11: removing libstdc++6-5.2.1-0.20151124.1.mga6.x86_64
                                 #########################################################################################################
    11/11: removing gcc-cpp-5.2.1-0.20151124.1.mga6.x86_64
                                 ############################################

after they installed all other none kipi installed. Why not the first time?
Comment 3 Marja Van Waes 2015-12-07 00:11:47 CET
as a workaround: if you put kipi-plugins (and/or anything else that won't update) in the skip list, then urpmi --auto-update works fine.

for those who never used it: adding e.g.
/^kipi-plugins/
/^mono/
in /etc/urpmi/skip.list will cause all packages of which the names start with 'mono' or 'kipi-plugins' to be omitted when the system  is updated

CC: (none) => marja11

Comment 4 Thierry Vignaud 2015-12-07 08:59:36 CET
That's not an urpmi bug, but a packaging issue.
Urpmi computes the packages that need to be updated then split them in transactions, sorting them according to their dependencies.
Then if b/c of some packaging bug, a package cannot be installed (eg: file conflict that is not explicited by a Conflicts: tag), then the transaction will fail.
Then if next transaction contains a package that expect a deps to be fulfilled by a package in the previous transaction, there'll be unsatisfied deps and thus this transaction will fail too.
That's a cascade effect due to a packaging bug.
There's nothing that urpmi can do.

You can run urpmi with "-vv --debug" option, note every packaging issue and then fix it.

Source RPM: urpmi => kipi-plugins

Comment 5 Thomas Spuhler 2015-12-07 20:59:57 CET
Thanks Thierry. So you are saying, if in an update some packages cannot be installed because of a packaging bug, also the good packages like gcc will not be installed?
Comment 6 Bit Twister 2015-12-08 01:50:36 CET
(In reply to Marja van Waes from comment #3)
> as a workaround: if you put kipi-plugins (and/or anything else that won't
> update) in the skip list, then urpmi --auto-update works fine.

Downside is no easy way to know when the package/dependencies are fixed without removing the /etc/urpmi/skip.list items.

CC: (none) => bittwister2

Thierry Vignaud 2015-12-08 10:50:18 CET

Assignee: thierry.vignaud => bugsquad

Thierry Vignaud 2015-12-08 10:52:47 CET

CC: (none) => thierry.vignaud
Assignee: bugsquad => mageia
Summary: urpmi --auto-select (or --auto-update) doesn't install several packages => urpmi --auto-select (or --auto-update) doesn't install several packages b/c of kipi packages
Source RPM: kipi-plugins => libkipi, kipi-plugins

Comment 7 Barry Jackson 2015-12-08 11:52:12 CET
In this situation I find it easier to use the mcc GUI and deselect only the conflicting packages while installing the others.

When the packages are fixed you have nothing special to do.

I agree with Bit Twister, plus it's easy to forget what you have in skip.list.

CC: (none) => zen25000

Comment 8 Bit Twister 2015-12-08 13:02:45 CET
(In reply to Barry Jackson from comment #7)
> In this situation I find it easier to use the mcc GUI and deselect only the
> conflicting packages while installing the others.

Sounds good except when it doesn't run, see bug 16626
or you have a large bunch you have to keep deselecting just to get the next package installed. 

It would go a whole lot faster if you could un-select all then select the ones you can install. See bug 14351
Comment 9 Frank Griffin 2015-12-08 14:45:46 CET
This has been discussed before with no resolution.

IIRC the "transactional" behavior (which isn't transactional at all, but pretty much just random batching with needed deps added in) is embedded in rpm, not urpmi.  It is rpm that isn't smart enough to realize that just because one package in a "transaction" fails, other unrelated packages may have no problem and it should still press on.

You can sort of work around this as follows:

1) Run urpmi --auto-update
2) If urpmi says it needs to remove things you don't think should be removed (e. g. mirror lag), then run urpmi --auto-update --keep
3) If a failure is still blocking unrelated packages, run urpmi --auto-update --keep --split-level=1 --split-length=1

(3) will force urpmi to submit "transactions" of a single package to rpm.  I can't swear that urpmi will continue if one of these fails, but that's what it usually looks like.  Do this until the proposed list of packages urpmi wants to install includes only packages that have previously failed.

In the previous discussion, the proposed solution was to have urpmi do the transition from (2) to (3) automatically, or by using a new switch.

The real solution, of course, would be to add a behavior to rpm that causes it to continue to plod through transactions in a batch regardless of failure.  

I think rpm doesn't do this because it expects batches of packages given to it to contain only related packages, and assumes that it one fails, the others will too.  Urpmi, on the other hand, batches up packages for performance reasons, in order to minimize the number of calls to rpm, and doesn't limit batches to interrelated packages.

CC: (none) => ftg

Comment 10 Thomas Spuhler 2015-12-08 15:55:58 CET
Frank, thanks for explaining this. I think I (we) can live with this as most packaging errors should be resolved once the distro is ready to be released. 
I can close this bug if you like me to.
Comment 11 Frank Griffin 2015-12-08 15:59:52 CET
That explanation was my understanding from the last discussion.  That doesn't mean it's true :-)  Sometimes my brain fills in missing gene sequences with frog DNA.  Thierry can confirm one way or the other.
Comment 12 Marja Van Waes 2015-12-08 19:14:10 CET
Thx for the --split-level=1 --split-length=1 hint, Frank, I didn't know about those options.

Since this bug got re-assigned to the kipi-plugins packaging issues, this report has become a duplicate.

Neoclust will fix it today

*** This bug has been marked as a duplicate of bug 17266 ***

Status: NEW => RESOLVED
See Also: (none) => https://bugs.mageia.org/show_bug.cgi?id=17266
Resolution: (none) => DUPLICATE