Description of problem: Plasma etc, update on real hardware, 64-bit with some 32-bit apps installed. Update attempted all at once. In my previous attempts, I deselected all packages presented from Updates Testing, and laboriously went through the list, selecting the packages I wanted and OKing the dependencies that come up. This time, thinking it would be more efficient, I left packages selected as they were when Mageia Update came up, and de-selected the packages I didn't want. The resulting list to be updated contained 488 packages. Somewhere around 270 packages left, Mageia Update failed with a message that three Updates had failed. Unfortunately, I did not have the presence of mind to copy the message before proceeding, but I remember that one of them was that kpat-handbook needed kpat, but I KNOW that kpat had been selected. Another, as I recall, was about a kdegames library. I don't remember the third. Thinking that there was some kind of Internet or mirror problem, I tried again, again using the same selection method, and had the same result within seconds. A third trial after updating the repositories was the same. Finally, I went back to selecting the packages one by one. This time, the update was successful. A point of information: This install was created with the Classical iso. I have noticed before now that if kpat is installed from the iso, kpat-handbook is not installed. But, if I install it from the repositories after the install, kpat-handbook is a dependency. I don't remember how I installed it on this machine - that was many months ago. It could have been either way, but I suspect I installed it from the iso. I will attempt to re-create this install in a VirtualBox guest so I can make a pre-update snapshot, but it will take a day or so by the time I install the extra apps that are on it. One thing that will be missing from the guest is VirtualBox, but I don't think that will make a difference.
Depends on: (none) => 22656
CC: (none) => marja11Assignee: bugsquad => kde
I did create a vbox guest and tried to update. My plan was to simulate as closely as I could what most users would do when getting an update like this. On the first pass, I scanned through the list presented by Mageia Update, deselecting the packages I didn't want, and then tried the update, just over 500 packages. The updates proceeded for a while, then failed with the messages in the attached file conflict1.txt. I then tried again, same procedure, trying to be extra careful not to deselect the wrong packages. Just over 300 packages. Same result, with the messages in conflict2.txt. Another trial, just under 300 packages, same result. Messages in conflict3.txt. Another trial, this time deselecting all packages, then laboriously going through the list one by one, selecting the appropriate packages and approving dependencies as I went. Another failure, with a message nearly identical to conflict1. At that point, I gave up.
Created attachment 10034 [details] Errors arfter first attempt
Created attachment 10035 [details] Errors after second attempt
Created attachment 10036 [details] Errors after third attempt.
Thomas, try uninstalling all the packages shown by urpmq --not-available|grep mga6|sort|less These are packages that were available in updates testing, but have been removed.
CC: (none) => davidwhodgins
Should have added, run it first with --test to see what will be removed. If packages that obviously shouldn't be removed will be, report that instead of removing those packages.
Unfortunately, I gave up on my guest at a point where less than half of the updated packages had been installed, so I doubted it would even boot again, and eliminated the entire guest. Admittedly, as much from frustration as anything else. I will need to start from scratch, with a guest that, while fully updated, has nothing from updates testing installed on it. It seems strange though that a couple of hours later a 32-bit update from the same mirror, done in several passes of no more than 120 packages at a time, went without a hitch. I get that the situation is still fluid. Perhaps kernel.org had had time to sync again. Or perhaps something else is going on. We'll see.
I had neglected to export the above guest before attempting the update (forgetful TJ), so I had to set up another. Other than what came from the iso, I installed three games(kpat, kmines, kreversei) and the 32-bit version of Google Earth. Apps that had tainted versions were updated to those versions. When attempting the update, I left everything selected and deselected packages relating to the kernel, Image Magick, vlc, the pending sddm fix, libuser/userdrake, libraw, and a few others that were not listed in the various Plasma/QT/kf/etc. bugs. Also any dependencies of those packages. I started the process. It downloaded for a while, then installed for a while then repeated as I tried to watch. Eventually, there was a long session of downloading, with a couple of breaks to verify signatures. Upon starting to install, it stopped with this message: 3 installation transactions failed There was a problem during the installation: file /usr/share/locale/en_GB/LC_MESSAGES/libkdegames5.mo from install of libkdegames-common-1:17.12.2-1.mga6.noarch conflicts with file from package kde-l10n-en_GB-16.12.3-2.mga6.noarch libAppStreamQt.so.2()(64bit) is needed by plasma-workspace-5.12.2-1.mga6.x86_64 libkerfuffle.so.17()(64bit) is needed by ark-17.12.2-1.mga6.x86_64 libkioarchive.so.5()(64bit) is needed by kio-extras-17.12.2-4.mga6.x86_64 libmolletnetwork5.so.17()(64bit) is needed by kio-extras-17.12.2-4.mga6.x86_64 kirigami is needed by systemsettings-5.12.2-1.mga6.x86_64 kamera is needed by plasma-desktop-5.12.2-1.mga6.x86_64 libAppStreamQt.so.2()(64bit) is needed by plasma-desktop-5.12.2-1.mga6.x86_64 kpat >= 1:17.12.2-1.mga6 is needed by kpat-handbook-1:17.12.2-1.mga6.noarch kmines >= 1:17.12.2-1.mga6 is needed by kmines-handbook-1:17.12.2-1.mga6.noarch I stopped there. I remember seeing the kpat and kmines handbooks download, but not the games themselves. I didn't note anything about the others. I did not do as David asked in Comment 5, because these notices have been rather consistent in their content, and the packages involved, had they been changed recently, should have showed up on the mirror long ago. My WAG, from the perspective of someone unfamiliar with its inner workings, is that the Mageia Update download cache filled up before the missing files were downloaded and started installing without them, resulting in the error.
OK, ran David's command, and there was nothing on the list. I saw this kind of situation before, recently, after creating a new Mageia 6 install and attempting to update it. The update failed with a problem with kernel-firmware-nonfree and related dependencies, similar to the notices I'm seeing here. If I went back to the updates and selected the packages with the problem, and the dependencies, and updated just those, the rest of the update proceeded normally. The problem was later fixed with a new firmware update. I'm not sure how but I believe the packages had to be rebuilt.
One more attempt to finish the update. This time I selected only the packages involved in the error message detailed in Comment 8, and installed them. That went smoothly, no problems. Then I went back to the list, again deselecting those packages which I do not believe are involved in the update, and those associated with the vlc update. The packages installed cleanly, and I rebooted to a working Plasma 5.12.2 desktop. This time I saved the pristine Plasma 5.8 guest, so I can do another test when needed.
(In reply to Thomas Andrews from comment #8) > > My WAG, from the perspective of someone unfamiliar with its inner workings, > is that the Mageia Update download cache filled up before the missing files > were downloaded and started installing without them, resulting in the error. Isn't the default transaction size for urpmi (and thus for MageiaUpdate) 50 packages nowadays? That still isn't enough for huge updates like this. Is there anything against patching urpmi to use "--split-length 0" by default, before the Plasma packages are pushed to core/updates, so that all packages will be downloaded and installed in one transaction, ?
Assignee: kde => mageiatoolsCC: (none) => kde, thierry.vignaudSource RPM: (none) => urpmi
Yeah, I have seen more big transaction updates fail since we switched to 50 packages transactions... mostly when there are more problematic deps... When I tried to build the virtualbox package in testing with a lot of hard deps and conflicts to try an avoid the QT stuff in testing I saw a lot of failed rounds until I downgraded to --split-length 10 So I wonder if the fact that we add a lot of packages at the same time makes us overflow or simply feeds too much into rpm solver so it breaks deps to get itself out of the loop and goes on from there... @thomas: If you change /etc/urpmi/urpmi.cfg from: { } to: { split-length: 8 } does it work better then ? If it seems smaller transactions are safer for this big update we could push new urpmi with: http://gitweb.mageia.org/software/rpm/urpmi/commit/?id=287518ab333db89b5cf3f1821c4043f8e9f57cc2 reverted, as that is a priority package and will be installed before the rest gets installed... Yes, it will make full install a bit slower, but I prefer stability over speed...
CC: (none) => tmb
(In reply to Thomas Backlund from comment #12) > Yeah, > > I have seen more big transaction updates fail since we switched to 50 > packages transactions... mostly when there are more problematic deps... > > When I tried to build the virtualbox package in testing with a lot of hard > deps and conflicts to try an avoid the QT stuff in testing I saw a lot of > failed rounds until I downgraded to --split-length 10 > > So I wonder if the fact that we add a lot of packages at the same time makes > us overflow or simply feeds too much into rpm solver so it breaks deps to > get itself out of the loop and goes on from there... > > > @thomas: > > If you change /etc/urpmi/urpmi.cfg from: > > { > } > > to: > > { > split-length: 8 > } > > > does it work better then ? > I will give it a try, but I think it best to wait until tomorrow. My Internet service has been out most of the day, only coming back on about 15 minutes ago. I want to be sure it stays on before trying this again. Oh, and a suggestion: To avoid potential confusion, we should probably use my TJ nickname. I have no problem using my real name, but there are just too many Thomases around here. :) TJ
Don't know if it makes any difference, but the Mageia guest is allowed 3GB of RAM out of a total of 8 on the host, and the virtual hard drive has 20GB, dynamically allocated, with an actual size of 6.20 GB after it has been updated. However, I've also seen approximately the same error on real hardware, with 8GB of RAM and 14GB available in the / partition, less than half full. TJ
Better this time, MUCH better - though not perfect. This time, I included the digicam update, as well as vlc. 520 packages in all. Things proceeded smoothly until almost the end, when it stopped with this message: 1 installation transactions failed There was a problem during the installation: file /usr/share/locale/en_GB/LC_MESSAGES/libkdegames5.mo from install of libkdegames-common-1:17.12.2-1.mga6.noarch conflicts with file from package kde-l10n-en_GB-16.12.3-2.mga6.noarch That message has appeared in all the other failures, too. Another pass, this time 13 packages, finished without incident, and I was able to reboot into a working desktop. This has the feel of a separate issue. There are so MANY packages, from so MANY lists, from so MANY bugs, with some packages that don't seem to be listed anywhere but REALLY look like they ought to be a part of the update that it is easy to select or deselect something that shouldn't be. That wouldn't be a problem when the updates are pushed, because all of the packages needed, and none of the ones not needed would be presented to the user for updating. At least, that's the way it should be. In any case, needing to make two passes to complete an update this massive isn't bad, not bad at all. I think our users could live with it, though we should try to warn them somehow that multiple passes may be necessary. I have one set of real hardware that I haven't used in about a month where I can try this. Core 2 Duo (one of the slower ones), 4GB, Geforce 210 graphics card, Atheros wifi. It has both 64-bit and 32-bit Plasma on it. I'll give it a shot this afternoon.
file /usr/share/locale/en_GB/LC_MESSAGES/libkdegames5.mo from install of libkdegames-common-1:17.12.2-1.mga6.noarch conflicts with file from package kde-l10n-en_GB-16.12.3-2.mga6.noarch should be fied in the next rpms. Please test
CC: (none) => mageia
OK, holding off on the real hardware until I see that it's hit my mirror. Not there yet...
(In reply to Thomas Backlund from comment #12) > > Yes, it will make full install a bit slower, but I prefer stability over > speed... I didn't do any timing of it, but I didn't notice a significant difference in the speed of the update before and after the urpmi adjustment. The only noticeable slowdowns that I saw were, as always, due to my relatively slow ISP. Just my opinion, but I'd say a "faster" install that pops errors and needs multiple passes is actually slower by the end than a "slower" install that gets the job done in one pass.
Complete success on real hardware with the 64-bit Plasma update. Smooth as can be. One thing different from the VVM trial - I updated the desktop kernel to 4.14.25 while I was waiting for the mirror. So, that tells us that the update should work, even if the new kernel is pushed first. The 32-bit update is running now...
32-bit Plasma update was a success, too. Just as smooth as the 64-bit update went. Doing the VM update once more, just to be absolutely sure...
And the VM update went perfectly, too. As far as I'm concerned, this bug is fixed. I could mark it as such, but I'd say you guys need to decide exactly what you'll be doing with urpmi first. So, for now, I'm leaving it open. Once you decide, feel free to mark it as fixed.
(In reply to Thomas Andrews from comment #21) > And the VM update went perfectly, too. > > As far as I'm concerned, this bug is fixed. > You don't explicitly mention you patched /etc/urpmi/urpmi.cfg with the "split-length: 8" adjustment that tmb suggested, for your VM update and for the 64-bit and for the 32-bit update. You did adjust it for all those last updates tests, right?
seems fixed now that the conflicts are fixed. Please open a new bugreport if you encounter some conflicts
Resolution: (none) => FIXEDStatus: NEW => RESOLVED
(In reply to Marja van Waes from comment #22) > (In reply to Thomas Andrews from comment #21) > > And the VM update went perfectly, too. > > > > As far as I'm concerned, this bug is fixed. > > > > > You don't explicitly mention you patched /etc/urpmi/urpmi.cfg with the > "split-length: 8" adjustment that tmb suggested, for your VM update and for > the 64-bit and for the 32-bit update. You did adjust it for all those last > updates tests, right? Sorry about that. Yes, I did make the patch on all of the last three trials. And, I haven't done any new trials without the patch. I see a report on one of the bugs that someone did an uneventful and successful all-in-one update, and from his description, without the patch. And, early on I did a successful trial in a VM without it, on a basic system that didn't need several of the packages that most real-world updates would require. My guess is that may be due to a lucky mix of packages that didn't trigger the buffer overflow speculated by tmb, but I don't know enough to be anywhere near sure of it.
In VM's, both 64 and 32, I installed all the updates in a single transaction. I did NOT patch urpmi and all of the updates went smoothly. https://bugs.mageia.org/show_bug.cgi?id=22656#c8 https://bugs.mageia.org/show_bug.cgi?id=22656#c11 I do have my own local mirror, but I don't know if that would make any difference.
CC: (none) => jim