On first boot of a live system without doing anything before, i executed our tool remove-unused-packages to get rid of a half thousand language packages, to make updating quicker. But it quit because wrong owner of XDG_RUNTIME_DIR and rpm database is now broken :( To me, it raises too many questions to understand, i.e why did it finish 366 uninstalls before reacting to that?? Steps to Reproduce: 1. Create a persistent USB stick from Mageia-8-beta1-Live-Xfce-x86_64.iso 2. Boot it, open xfce terminal app, and there 3. Attempt removing unused locales, per https://wiki.mageia.org/en/Removing_packages#Removing_unused_localisation_and_hardware_support $ su # remove-unused-packages And in there i deselected to remove hardware support and let it go. Result: After operating a while it interrupts, complaining "XDG_RUNTIME_DIR (/run/user/1000) is not owned by us": ... tar bort paket mageia-doc-mcc-es-7.3-2.mga8.noarch 365/505: tar bort mageia-doc-mcc-es-7.3-2.mga8.noarch ############################################# tar bort paket hunspell-yi-1.1-12.mga8.noarch 366/505: tar bort hunspell-yi-1.1-12.mga8.noarch ############################################# XDG_RUNTIME_DIR (/run/user/1000) is not owned by us (uid 0), but by uid 1000! (This could e.g. happen if you try to connect to a non-root PulseAudio as a root user, over the native protocol. Don't do that.) [root@localhost live]# I found rpm database is broken, tried to repair: [root@localhost live]# LC_ALL=C rpm --rebuilddb error: failed to replace old database with new database! error: replace files in /var/lib/rpm with files from /var/lib/rpmrebuilddb.4881 to recover (And I tried that file replacing: still no joy, but that is not important for me) I have earlier used drakrpm GUI to remove localisations, selecting manually, before i knew about our tool and that finished without error. But I did not remove so many in one pass.
I guess you Martin can see where the problem is? I am about to try repeating using mga7 xfce live.
Assignee: bugsquad => mageiatoolsCC: (none) => mageia
Please try again after using "su -" instead of just su.
CC: (none) => davidwhodgins
My basic guess would be that the problem was triggered by using su instead of "su -" meaning that all environment variables from the current user remain defined, and root ends up using all the stuff from the user (and possibly breaking other things by having root owned files in user directory). Still it's not great that somehow pulseaudio kills things running a command as root with a non root user environment. The strange thing is what would be using pulseaudio while urpme is uninstalling packages? If it was a %preun script, I would expect it to only kill the script.
Yes, the error message you report is because you used 'su' not 'su -'. But it is otherwise harmless. I have just tried to reproduce the problem, and it completed without error for me, but gave me the same error message at the end (as well as several times on the way through. Check the system log for other errors.
I think the reference to pulseaudio is just an example of what might trigger that error message.
Thank you for the very quick replies :) Yes getting shot in the back is no fun. I just verified that there is not the same problem with mga7.1 64bit xfce live. But it may just be coincidence. No occurrence of that message (at least in the terminal) but maybe I am just lucky. Yes i should learn to always use su - - and we should also write it in all relevant user instructions... Now i am loading that stick again with the 8b1 iso to try with su -. Any new iso on the horizon? Hardware i am testign on: Very fast USB stick, but limited by USB2 port; on a Thinkpad T400.
Yes, the " -" after "su" did the trick :) OK, so how to best avoid other users fall in same hole as me?
Summary: Mageia Live 8b1 persistent: rpm database get irrepairably broken by removing localisations => Mageia Live 8b1 persistent: rpm database get irrepairably broken by simple mistake while removing localisations
Using "su --login" (which can be abbreviated as just "su -" is described in the man page for su. It's something all linux users have to learn, usually the hard way as they won't think to look for the information until they encounter the bad effects it can have. For example, using just "su", and then running any program that updates files in ~/.config will change ownership of those files to root:root. When the user tries running the same program under their own id they will fail since it can no longer update the files in ~/.config You're the first person I'm aware of that hadn't already learned the impact of using "su" before trying to use the remove-unused-packages script. I frequently explain it in various linux related usenet newsgroups and other places, and instruct them to read the su man page. I'm closing the bug as invalid since it's a linux education problem rather then a Mageia specific problem. Feel free to continue the discussion here, even though the bug has been closed if you have an idea of where it can be better documented.
Status: NEW => RESOLVEDResolution: (none) => INVALID
Lesson learned. Any suggestion about where to put such information in our wiki or other docs? "su --login" is nowhere to be found currently. ( And searching for "su -" in our wiki result in a database query error :/ ) ( For this specific case i have suggested: Bug 27372 - MCC: Link to our tool for removing unused localisations and translations, remove-unused-packages )
I've created https://wiki.mageia.org/en/Never_use_%22su%22! and asked the docteam to pretty it up etc.
Great! :) I think the page should mention sudo, and some words when and when not sudo can be used instead of "su -". Could you, who know much better than me, add some words? Use link to [[Configuring_sudo|Configuring sudo]]. Announced on forum: https://forums.mageia.org/en/viewtopic.php?f=41&t=13769 I edited https://wiki.mageia.org/en/Removing_packages#Removing_unused_localisation_and_hardware_support to reference your page.
Correct headline ;)
Summary: Mageia Live 8b1 persistent: rpm database get irrepairably broken by simple mistake while removing localisations => Mageia Live 8b1 persistent: rpm database get irrepairably broken by forgetting dash after su while removing localisations
(In reply to Morgan Leijström from comment #12) > Correct headline ;) As said in comment 4, using 'su' is responsible for the XDG_RUNTIME_DIR message, but I'm not convinced it caused the rpm database error - it didn't for me.
OK. Details are too convoluted for me. Should we reopen and dig more?
Only if you can reproduce the fault. If you can, capture the system log as well as the full terminal output.
OK then. In a few days i will try.
Resolution: INVALID => (none)Status: RESOLVED => REOPENED
Wiki page moved to https://wiki.mageia.org/en/Never_use_just_su
Caught a crash now on first attempt. Logs of journal and terminal output follows as attachments. And, this time I _did_ use "su -" ! And no error visible nor noted in the log during the process. Journal tells process urpmi runaway. And again rpm fail to rebuild database. Strangely, it now quit a hundred packages earlier in the process. This happens to be a few seconds after screensaver kicked in. During the process i reported in comment #0, I used the keyboard/pointer so there screensaver did not kick in, so it is probably not that problem. This is the same machine and same stick. To make the install it juvenile, I first output the stick in another system and renamed the folder memory/ in the persistent partition. (so I can easily revert to my system in use.) During boot I selected the same options as before, for Swedish localisation. But how come it crashed this time and not last time?? Both times I used "su -". So Remi is right, it is some other problem. I wonder if it may be a hardware problem. I bought this old laptop together with four other T400 machines, sold one but had to exchange it because sometimes it hangs completely. But i find it strange it is only during removal of many packages this happened. I could update 1000+ packages and install lxde, and have used it for surfing an evening without any visible problem. I can not see a pattern. It often amuse me that system tells me how exiting things are ("### Program is exiting ###") :^)
Summary: Mageia Live 8b1 persistent: rpm database get irrepairably broken by forgetting dash after su while removing localisations => Mageia Live 8b1 persistent: urpmi runaway while removing localisations - rpm database get irrepairably broken
Created attachment 11985 [details] File from running # LC_ALL=C remove-unused-packages 2>&1 | tee 1_rmu.txt
Created attachment 11986 [details] Relevant part of journal
This means the command took longer than the given timeout. The code does not set one: run_program::rooted($o_prefix, 'urpme', '--auto', if_($hardware, @unused_hardware_packages), if_($locales, @unselected_locales), ); So it is using a default of 10 minutes.
CC: (none) => pterjan
See bug 44
Yes you are correct: it got killed after exactly 10 minutes :) The variable results from testing could come from variable write times to the USB stick. Do that timeout also affect urpmi commands given by user, i.e urpmi --auto-up --auto ? I used the GUI to update another install on same stick and machine, and that ran for very long without problems. Also note, this is on a rather fast USB stick. (but on USB2 port)
Priority: Normal => HighSummary: Mageia Live 8b1 persistent: urpmi runaway while removing localisations - rpm database get irrepairably broken => Mageia Live 8b1 persistent: urpmi killed by timeout while removing localisations - rpm database get irrepairably brokenSource RPM: (none) => drakxtools-curses
The timeout is only applied to commands run from inside one of the drakx tools, so no, it won't affect urpm* commands you run from the command line. We had a similar problem when I added the option to install updates in the Live installer (see thread starting at https://ml.mageia.org/l/arc/qa-discuss/2020-07/msg00002.html). I fixed that by disabling the timeout for that case.
Fixed in git, will be fixed in next release of drakxtools.
Source RPM: drakxtools-curses => drakxtools-18.34Assignee: mageiatools => mageia
Great :) BTW, I tested Mageia 7.1 i586 xfce live persistent: There it did not get interrupted. (I tried to interrupt after 9 minutes using ctrl-C but it did not stop.) It took 80 minutes to remove unused localisations, 501 packages. And that was not on a very slow USB, writing the iso took 50 seconds. Hmm that makes me wonder why so slow and what can be done about it... If buffering could be increased, or flushing/syncing relaxed? At least for us enthusiast with some CLI command?
I don't know why some people see such poor performance with persistence (you aren't the first to report this). What make/model of USB stick are you using? If you install a system onto that USB stick, is it similarly slow when you are running that system? On my 6 year old laptop with a SanDisk Ultra USB 3.0 memory stick, when plugged into a USB 3.0 port it took 2.5 minutes to remove the 520 unused packages, and when plugged into a USB 2.0 port it took 3 minutes.
Whoa - yes very different speed with another USB drive!! Now i tried the exact same .iso put on another stick with persistence. It finishes removing unused locales, 501 packages in just exactly one minute. !! That is *80 times* faster !! It is still USB2 port on the same one core i586 1,86GHz CPU Thinkpad T43. What info can I provide? For starter, maybe i should test mga8b1 xfce 32 bit and we work from there? journal excerpts follow, first part of lines deleted for readability The stick where persistence is s l o w : usb 5-2: new high-speed USB device number 5 using xhci_hcd usb 5-2: New USB device found, idVendor=1b1c, idProduct=1a90, bcdDevice=11.00 usb 5-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3 usb 5-2: Product: Voyager GT usb 5-2: Manufacturer: Corsair usb 5-2: SerialNumber: A164012700015819 usb-storage 5-2:1.0: USB Mass Storage device detected scsi host10: usb-storage 5-2:1.0 scsi 10:0:0:0: Direct-Access Corsair VoyagerGT 1100 PQ: 0 ANSI: 0 CCS sd 10:0:0:0: [sdc] 31719424 512-byte logical blocks: (16.2 GB/15.1 GiB) sd 10:0:0:0: [sdc] Write Protect is off sd 10:0:0:0: [sdc] Mode Sense: 43 00 00 00 sd 10:0:0:0: [sdc] No Caching mode page found sd 10:0:0:0: [sdc] Assuming drive cache: write through sdc: sdc1 sdc2 sdc3 sd 10:0:0:0: [sdc] Attached SCSI removable disk The stick where persistence is lightning fast: usb 6-1: new SuperSpeed Gen 1 USB device number 4 using xhci_hcd usb 6-1: New USB device found, idVendor=1b1c, idProduct=1a0e, bcdDevice= 1.00 usb 6-1: New USB device strings: Mfr=2, Product=3, SerialNumber=1 usb 6-1: Product: Voyager GTX usb 6-1: Manufacturer: Corsair usb 6-1: SerialNumber: 511190606267A0D6018B scsi host10: uas scsi 10:0:0:0: Direct-Access Corsair Voyager GTX 0 PQ: 0 ANSI: 6 sd 10:0:0:0: [sdc] Spinning up disk... .ready sd 10:0:0:0: [sdc] 250069680 512-byte logical blocks: (128 GB/119 GiB) sd 10:0:0:0: [sdc] Write Protect is off sd 10:0:0:0: [sdc] Mode Sense: 43 00 00 00 sd 10:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA sd 10:0:0:0: [sdc] Optimal transfer size 33553920 bytes sdc: sdc1 sdc2 sdc3 sd 10:0:0:0: [sdc] Attached SCSI disk
Feel free to open a separate bug for the speed issue if you start working on that.
From https://usb.userbenchmark.com/Corsair-Flash-Voyager-GT-USB-30-16GB/Rating/1272 https://usb.userbenchmark.com/SpeedTest/13691/Corsair-Voyager-GTX Corsair Voyager GT 16GB scores 9.74/0.37/0.55 MB/s on random 4K read/write/mixed Corsair Voyager GTX 128GB scores 17/19.2/7.45 MB/s on random 4K read/write/mixed I think that explains the difference! My SanDisk Ultra USB 3.0 scores 5.32/1.45/2.51 MB/s, so is between your two.
Oh, that performance of the slow stick on random writes is really ugly yes... But I still think that is only part of explanation: Averages number difference factor is 52 ( 19,2/0,37 ). If i had logged a factor 52 in my tests, i would also have had to change CPU, memory, buses and USB port to 52 times faster devices. But I logged even greater difference - a factor 80 - with everything else but the stick unchanged. Could it be that package management happens to be a worse test case than that linked tests, and some USB are perform extremely poor then... Or something regarding how kernel handles USB memories is not optimal for some? To make it less random, can we buffer more, or aggregate writes? Would a flash optimised file system perform better for persistence? How much data is written when i *remove* 512 packages?
It's likely most of the writes when deleting packages are less than 4K, so yes it will be a worse test case. And if CPU/memory/USB ports can keep up with the faster device, they don't matter. You can try experimenting with different file systems for the persistent partition (just reformat it, and make sure to give it the volume label 'mgalive-persist'). I would guess a non-journaled file system would do better. But really, for what you are trying to do, you want to be using a USB stick with better performance.
I think you are correct. I will try to write something in wiki about how to select USB stick for persistence use, especially for tasks like updating. The userbenchmark site is a good information source, thanks.
Fix released in drakxtools 18.35.
Resolution: (none) => FIXEDStatus: REOPENED => RESOLVED
Confirming fixed :)