Created attachment 6329 [details] urpme kernel transaction excerpt from journalctl -b-1 log Mailing list thread: https://ml.mageia.org/l/arc/dev/2015-04/msg00403.html https://ml.mageia.org/l/arc/dev/2015-04/msg00403.html Statistics: remove kernel-desktop-3.18.3 on 2.0GHz Sempron 2800+ host m7ncd with 46 partitions total, 4 partitions unformatted: 8 minutes 35 seconds remove kernel-desktop-3.18.3 on 2.6GHz P4 host gx27b with 30 partitions total, 3 partitions unformatted: 3 minutes 33 seconds remove kernel-desktop-3.19.0 on 2.6GHz P4 host gx27b with 46 partitions total, 21 partitions unformatted: 10 minutes 01 seconds (attachment is from this transaction) install kernel-desktop-3.19.4 on 2.6GHz P4 host gx27b with 46 partitions total, 21 partitions unformatted: 10 minutes 23 seconds Same systems booted to Tumbleweed (host gx27b: remove 3.19.0 18 seconds; install 3.19.4 65 seconds) and Rawhide install and remove kernels in far less time, about like installing kernel-desktop-3.14.32 on mga4, which took barely over a minute on 2.6GHz P4 host gx27b with 46 partitions total, 21 partitions unformatted.
@Colin: not using the blkid cache does hurt when there's quite a lot of partitions http://gitweb.mageia.org/software/drakx/commit/perl-install?id=ecd8db27c8ffb76ebf799e0de0dcf64c3ce256fe
CC: (none) => mageia, thierry.vignaud
Maybe it hurts, but the upstream maintainers are advising not to use it. I guess as per the quoted conversation it should be migrated to either use the udev db or lsblk.
No improvement as of 3.19.8 kernels.
Summary: kernel install/remove process taking unreasonably long => kernel install/remove process taking unreasonably long when there are lots of partitions
CC: (none) => tmb
Whiteboard: (none) => MGA5TOO FOR_ERRATA
Also, do you use grub2? if yes, do you've os-prober installed?
(In reply to Thierry Vignaud from comment #4) > Also, do you use grub2? Not as master bootloader, and not on any Mageia installations.
Still a problem as of kernel-desktop-4.6.0-1.mga6, ~5 minutes to install on 2.8GHz i686 host gx280 with 31 partitions.
Whiteboard: MGA5TOO FOR_ERRATA => MGA5TOO
Assignee: bugsquad => kernelSource RPM: drakxtools-backend => kernel
This looks the same terminal - not original - problem as Bug 16010 (now closed); and Bug 18538. If somebody Bugzilla responsible agrees that 18538 & this bug 15752 are effectively duplicates, I suggest closing one of them appropriately: probably this one. Looking back at 16010 re the long time to run os-prober, hence do installations & kernel updates, I have a significant addition (which I shall post also on 18538): Early in that closed bug Barry asked about a certain partition 10; comments 5 & 12. Real EFI/GPT hardware: It happens that I have recently removed Korora [ex Fedora] which was on partition 10, because of its very intrusive nature in NVRAM & the ESP, probably due to it doing 'secure boot'. At the same time I *increased* the number of partitions to 16. I have noticed since that os-prober (at least; yet to try a kernel upgrade or new installation) happens much faster, at least for Mageia 5, yet to try 6. From up-to-date Mageia 5: # time os-prober /dev/sda11:Mageia 6 (6):Mageia:linux /dev/sda13:Mageia 5 (5):Mageia1:linux /dev/sda14:LMDE MATE Edition (1):LinuxMint:linux /dev/sda16:Mageia 5 (5):Mageia2:linux /dev/sda2@/EFI/Microsoft/Boot/bootmgfw.efi:Windows Boot Manager:Windows:efi /dev/sda8:Manjaro Linux (16.10):ManjaroLinux:linux /dev/sda9:openSUSE 12.3 (x86_64):SUSE:linux real 0m25.038s which suggests that the presence of Korora/Fedora *did* matter, rather than the number of partitions.
CC: (none) => lewyssmith
This bug is probably distinguishable from the others in that in no case is grub2 or os-prober installed on any of my Mageia, Fedora, Debian or openSUSE installations. I do have a few *buntu installations I allow to install and keep grub2, but os-prober is absent from all, and none UEFI/GPT capable have it enabled.
I jumped the gun; apologies. Two bugs indeed. Your remark suggests that there is probably another reason (i.e. not os-prober, itself perhaps a victim) for the 10m delays some of us see - which we may have wrongly attributed.
Still very much a problem with current 6RC installing or rebuilding initrd for 4.9.25 using 64-bit host fi965 with /dev/sda34, more than 4 minutes.
kernel-desktop-4.4.16-1.mga5 took 4 minutes 20 seconds to remove in preparation to upgrade 64-bit host gx62b with 34 sda partitions to mga6rc.
Looks like anything that calls 'bootloader-conf' causes these symptoms. I tried removing KDE in order to install TDE prior to upgrading MGA5 to MGA6. The delay waiting for bootloader-conf to complete running 'urpmi trinity-tdebase trinity-tdm trinity-ksnapshot trinity-konsole trinity-mix' was truly noxious for the context. Why bootloader-conf had any reason to be called in order to add a new DE is inexplicable.
Not limited to 32-bit. Didn't get any better that I can tell in 6 final.
Hardware: i586 => All
The background issue is os-prober doing bad things sometimes IMHO
Source RPM: kernel => kernel, os-prober
None of my installations have grub2 or os-prober installed.
On host big41 I just did urpmi upgrade mga6 on /dev/sda32 to mga7b2: > inxi -GxxSm System: Host: big41 Kernel: 5.0.2-desktop-1.mga7 x86_64 bits: 64 compiler: gcc v: 8.3.1 Desktop: Trinity wm: Twin dm: startx Distro: Mageia 7 mga7 Memory: RAM: total: 3.73 GiB used: 252.7 MiB (6.6%)... CPU: Topology: Dual Core model: Intel Core2 Duo E7600 bits: 64 type: MCP arch: Penryn rev: A L1 cache: 64 KiB L2 cache: 3072 KiB flags: lm nx pae sse sse2 sse3 sse4_1 ssse3 vmx bogomips: 12235 Speed: 2762 MHz min/max: 1603/3066 MHz Core speeds (MHz): 1: 1917 2: 1840 Graphics: Device-1: Intel 4 Series Integrated Graphics vendor: Biostar Microtech Intl Corp driver: i915 v: kernel bus ID: 00:02.0 chip ID: 8086:2e32 Display: tty server: Mageia X.org 1.20.4 driver: modesetting alternate: fbdev,intel,vesa resolution: 1920x1200~60Hz OpenGL: renderer: Mesa DRI Intel G41 v: 2.1 Mesa 19.0.0 direct render: Yes mga4 remains on /dev/sda40, last partition. I followed up the upgrade with: # time urpme kernel-desktop-4.9.56-1.mga6 ... 5m28.118s real. # time urpme kernel-desktop-4.14.30-3.mga6 ... 5m47.075s real. # time urpme kernel-desktop-4.14.69-1.mga6 ... 5m28.912s real. Before the upgrade I forgot to use time, but when I removed three other kernels, each seemed to take closer to 3 minutes than to 5. :(
(In reply to Felix Miata from comment #16) > Before the upgrade I forgot to use time, but when I removed three other > kernels, each seemed to take closer to 3 minutes than to 5. :( Heheh, yup. My guess, the solution will be a grub fix/patch to umount commands. I wrote my own /etc/grub.d/ script to generate menu entries by label and wondered why it was taking so long to run. What I found was caused by systemd taking its time returning from the umount command for each partition I had mounted to checked for /boot. :( Solution was to add --lazy to umount command.
CC: (none) => bittwister2
Why would there be any mounting or unmounting going on during kernel installation or removal on an installation with neither os-prober nor grub2* nor syslinux* nor pxelinux installed, as all my mga6+ installations are?
From the attached log file, most of the time is being taken between these two messages: Apr 22 18:04:00 gx27b.ij.net bootloader-config[1772]: found a dos partition table on /dev/sda at sector 0 Apr 22 18:12:51 gx27b.ij.net bootloader-config[1772]: guess_geometry_from_partition_table sda: 30401/255/63 Felix, could you confirm that is still the case. If so, the first message comes from partition_table::read_primary(), called at the start of partition_table::read(), and the second message comes from partition_table::dos::guess_geometry_from_partition_table(), called (indirectly) at the end of partition_table::read(). So it would seem to be the libDrakX partition table read subroutine that is slow.
CC: (none) => mageia
The script bootloader-config is crap. I was not aware that it is actually used in Mageia tools. cf. https://bugs.mageia.org/show_bug.cgi?id=18534
CC: (none) => bequimao.de
Created attachment 10945 [details] Script to test partition table read speed @Felix, to confirm my analysis, please try running this Perl script. You'll need to run it as root, because it reads the disk partition table. If it shows the same slow behaviour, try installing the perl-Devel-Trace package and run it again by perl -d:Trace read_pt.pl This will produce a lot of output, but should pause at the point in the code that is causing the problem. Please capture a few lines of output when it pauses.
Tried with host gx62b Mageia 6 last updated in September: # cat b15752.sh #!/bin/perl use lib qw(/usr/lib/libDrakX); use partition_table; print "Detect devices\n"; my @drives = detect_devices::hds(); foreach my $hd (@drives) { $hd->{file} = devices::make($hd->{device}); } print "Get geometries\n"; my @drives = partition_table::raw::get_geometries(@drives); foreach my $hd (@drives) { my $name = $hd->{file}; print "Read partition table on $name\n"; partition_table::read($hd); } # sh b15752.sh b15752.sh: line 2: syntax error near unexpected token `(' b15752.sh: line 2: `use lib qw(/usr/lib/libDrakX);'
It's a Perl script, not a shell script. Try perl b15752.sh
It took only a few seconds to run: # perl b15752.sh Detect devices Get geometries HDIO_GETGEO on /dev/sda succeeded: heads=255 sectors=63 cylinders=60801 start=0 id2hd: 0xdf5ee108=>sda id2edd: 0xdf5ee108=>/sys/firmware/edd/int13_dev80 geometry_from_edd sda 0xdf5ee108: 60801/255/63 Read partition table on /dev/sda running: blkid -o udev -p /dev/sda running: blkid -o udev -p /dev/sda found a dos partition table on /dev/sda at sector 0 guess_geometry_from_partition_table sda: 60801/255/63 More than three minutes later, the disk was still churning, with top showing a lot of systemd-udevd PIDs.
Created attachment 10946 [details] journalctl | grep bootloader-config from mga6 host gx62b Highlights: Sep 10 03:01:22 gx62b bootloader-config[9979]: fs::get::device2part: unknown device <</dev/sda19>> Sep 10 03:01:22 gx62b bootloader-config[9979]: writing grub config to /boot/grub/menu.lst Sep 10 03:02:57 gx62b bootloader-config[11668]: HDIO_GETGEO on /dev/sda succeeded: heads=255 sectors=63 cylinders=60801 start=0 Sep 10 03:02:57 gx62b bootloader-config[11668]: id2hd: 0xdf5ee108=>sda Sep 10 03:03:03 gx62b bootloader-config[11668]: guess_geometry_from_partition_table sda: 60801/255/63 Sep 10 03:03:03 gx62b bootloader-config[11668]: running: udevadm settle Sep 10 03:05:03 gx62b bootloader-config[11668]: running: blkid -o udev -p /dev/sda1 Sep 10 03:05:03 gx62b bootloader-config[11668]: blkid gave: vfat 303F-1AFF P01WINDOS Sep 10 03:05:11 gx62b bootloader-config[11668]: fs::get::device2part: unknown device <</dev/sda19>> Sep 10 03:05:11 gx62b bootloader-config[11668]: writing grub config to /boot/grub/menu.lst Sep 10 03:09:02 gx62b bootloader-config[13600]: HDIO_GETGEO on /dev/sda succeeded: heads=255 sectors=63 cylinders=60801 start=0 Sep 10 03:09:03 gx62b bootloader-config[13600]: id2hd: 0xdf5ee108=>sda Sep 10 03:09:13 gx62b bootloader-config[13600]: guess_geometry_from_partition_table sda: 60801/255/63 Sep 10 03:09:13 gx62b bootloader-config[13600]: running: udevadm settle Sep 10 03:11:13 gx62b bootloader-config[13600]: running: blkid -o udev -p /dev/sda1 Sep 10 03:11:13 gx62b bootloader-config[13600]: blkid gave: vfat 303F-1AFF P01WINDOS Sep 10 03:11:22 gx62b bootloader-config[13600]: adding /boot/vmlinuz-4.14.65-desktop-1.mga6 Sep 10 03:11:22 gx62b bootloader-config[13600]: running: mkinitrd -f /boot/initrd-4.14.65-desktop-1.mga6.img 4.14.65-desktop-1.mga6 Sep 10 03:12:54 gx62b bootloader-config[13600]: running: /usr/share/bootsplash/scripts/make-boot-splash /boot/initrd-4.14.65-desktop-1.mga6.img 1024 Sep 10 03:13:01 gx62b bootloader-config[13600]: current labels: Mageia 6 defkernel Mageia 6 prvkernel linux 4.14.25 & 4.14.65 are the only currently installed kernels.
OK, that latest extract from the journal is quite different from the original mga5 one. And we can see that there is a 2 minute delay after each call to 'udevadm settle', which ties in with what you observe in comment 24.
CC: lewyssmith => (none)
See Also: (none) => https://bugs.mageia.org/show_bug.cgi?id=24681
As explained in comment #8, this bug cannot have anything to do with os-prober. This is from a 31 partition installation on 64bit mga6 host g5eas with only one installed kernel: # time urpmi kernel-desktop-latest To satisfy dependencies, the following packages are going to be installed: Package Version Release Arch (medium "Core Backports") kernel-desktop-4.18.20-1.mga6 1 1.mga6 x86_64 kernel-desktop-latest 4.18.20 1.mga6 x86_64 63MB of additional disk space will be used. 57MB of packages will be retrieved. Proceed with the installation of the 2 packages? (Y/n) y http://mirrors.us.kernel.org/mageia/distrib/6/x86_64/media/core/backports/kernel-desktop-4.18.20-1.mga6-1-1.mga6.x86_64.rpm 0% of 0 completed, ETA = --:--:--, speed = 0 0% of 57.4M completed, ETA = 0:06:49, speed = 160k ... 100% of 57.4M completed, ETA = --:--:--, speed = 1472k http://mirrors.us.kernel.org/mageia/distrib/6/x86_64/media/core/backports/kernel-desktop-latest-4.18.20-1.mga6.x86_64.rpm 0% of 0 completed, ETA = --:--:--, speed = 0 100% of 46574 completed, ETA = --:--:--, speed = 494k installing kernel-desktop-latest-4.18.20-1.mga6.x86_64.rpm kernel-desktop-4.18.20-1.mga6-1-1.mga6.x86_64.rpm from /var/cache/urpmi/rpms Preparing... ############################################# 1/2: kernel-desktop-4.18.20-1.mga6 ############################################# 2/2: kernel-desktop-latest ############################################# Creating: target|kernel|dracut args|basicmodules You should restart your computer for kernel-desktop-4.18.20-1.mga6 real 2m56.737s user 0m54.768s sys 0m44.495s #
Source RPM: kernel, os-prober => kernel,udev
The root cause of this bug is that the libDrakX partition table read subroutine uses the libparted ped_disk_probe() function to query the partition table type. The ped_disk_probe() function opens the raw disk device for read/write access (even though it only wants to read). That causes the kernel to generate a change event for each partition on the device, which triggers udev actions. A MBR/DOS extended partition table is a linked list of entries. Currently the libDrakX partition table read subroutine is calling the ped_disk_probe() function when it reads each entry. This means that the number of kernel events generated is proportional to the square of the number of partitions on the device. We can fix this by caching the value returned by ped_disk_probe().
Status: NEW => ASSIGNEDSource RPM: kernel,udev => drakxtoolsAssignee: kernel => mageia
Please test drakxtools 18.13-1 on your mga7 system when it reaches the mirrors.
Looks like you did well! # rpm -qa | egrep 'kernel|drakxtools' drakxtools-backend-18.13-1.mga7 drakxtools-curses-18.13-1.mga7 kernel-firmware-nonfree-20190314-1.mga7.nonfree kernel-desktop-latest-5.0.2-1.mga7 kernel-firmware-20190114-1.mga7 kernel-desktop-4.14.104-2.mga6-1-1.mga6 kernel-desktop-5.0.2-1.mga7-1-1.mga7 kernel-userspace-headers-5.0.2-1.mga7 # blkid | tail -n1 /dev/sda41: LABEL="mga4-h8s" UUID="52d99d42... # time urpme kernel-desktop-4.14.104-2.mga6 removing kernel-desktop-4.14.104-2.mga6-1-1.mga6.x86_64 removing package kernel-desktop-4.14.104-2.mga6-1-1.mga6.x86_64 1/1: removing kernel-desktop-4.14.104-2.mga6-1-1.mga6.x86_64 #################... real 0m30.208s user 0m2.416s sys 0m0.770s # inxi -S System: Host: big41 Kernel: 5.0.2-desktop-1.mga7 x86_64 bits: 64 Desktop: Trinity Distro: Mageia 7 mga7 # time urpmi kernel-desktop-latest To satisfy dependencies, the following packages are going to be installed: Package Version Release Arch (medium "Core Release") kernel-desktop-5.0.9-1.mga7 1 1.mga7 x86_64 kernel-desktop-latest 5.0.9 1.mga7 x86_64 65MB of additional disk space will be used. 59MB of packages will be retrieved. Proceed with the installation of the 2 packages? (Y/n) y http://mirror.math.princeton.edu/pub/mageia/distrib/7/x86_64/media/core/release/kernel-desktop-5.0.9-1.mga7-1-1.mga7.x86_64.rpm http://mirror.math.princeton.edu/pub/mageia/distrib/7/x86_64/media/core/release/kernel-desktop-latest-5.0.9-1.mga7.x86_64.rpm installing kernel-desktop-latest-5.0.9-1.mga7.x86_64.rpm kernel-desktop-5.0.9-1.mga7-1-1.mga7.x86_64.rpm from /var/cache/urpmi/rpms Preparing... ########################... 1/2: kernel-desktop-5.0.9-1.mga7 ########################... 2/2: kernel-desktop-latest ########################... 1/1: removing kernel-desktop-latest-5.0.2-1.mga7.x86_64 ########################... Creating: target|kernel|dracut args|basicmodules You should restart your computer for glibc, kernel-desktop-5.0.9-1.mga7, systemd real 1m50.080s user 0m14.137s sys 0m4.296s # uname -a Linux gx280 4.14.106-desktop-1.mga6 #1 SMP Thu Mar 14 19:13:32 UTC 2019 i686 i686 i686 GNU/Linux ## fdisk -l tail /dev/sda34 976752063 976768064 16002 7.8M 11 Hidden FAT12 # grep RETT /etc/os-release PRETTY_NAME="Mageia 7" # rpm -qa | grep nel-desk kernel-desktop-4.14.106-1.mga6-1-1.mga6 kernel-desktop-4.14.30-3.mga6-1-1.mga6 kernel-desktop-4.14.70-2.mga6-1-1.mga6 kernel-desktop-latest-4.14.106-1.mga6 kernel-desktop-4.14.10-1.mga6-1-1.mga6 kernel-desktop-4.14.44-2.mga6-1-1.mga6 # rpm -qa | grep kxtools-back drakxtools-backend-18.13-1.mga7 [root@gx280 ~]# time urpme kernel-desktop-4.14.30-3.mga6 removing kernel-desktop-4.14.30-3.mga6-1-1.mga6.i586 removing package kernel-desktop-4.14.30-3.mga6-1-1.mga6.i586 1/1: removing kernel-desktop-4.14.30-3.mga6-1-1.mga6.i586 ###########################... writing /var/lib/rpm/installed-through-deps.list real 0m41.981s user 0m11.606s sys 0m1.934s # time urpme kernel-desktop-4.14.10-1.mga6 removing kernel-desktop-4.14.10-1.mga6-1-1.mga6.i586 removing package kernel-desktop-4.14.10-1.mga6-1-1.mga6.i586 1/1: removing kernel-desktop-4.14.10-1.mga6-1-1.mga6.i586 ###########################... real 0m39.472s user 0m9.102s sys 0m1.600s # time urpme kernel-desktop-4.14.44-2.mga6 removing kernel-desktop-4.14.44-2.mga6-1-1.mga6.i586 removing package kernel-desktop-4.14.44-2.mga6-1-1.mga6.i586 1/1: removing kernel-desktop-4.14.44-2.mga6-1-1.mga6.i586 ###########################... writing /var/lib/rpm/installed-through-deps.list real 0m37.221s user 0m8.736s sys 0m1.758s # # time urpme kernel-desktop-4.14.70-2.mga6 removing kernel-desktop-4.14.70-2.mga6-1-1.mga6.i586 removing package kernel-desktop-4.14.70-2.mga6-1-1.mga6.i586 1/1: removing kernel-desktop-4.14.70-2.mga6-1-1.mga6.i586 ###########################... writing /var/lib/rpm/installed-through-deps.list real 0m35.255s user 0m7.424s sys 0m1.674s # time urpmi kernel-desktop-latest # time urpmi kernel-desktop-latest To satisfy dependencies, the following packages are going to be installed: Package Version Release Arch (medium "Core Release") kernel-desktop-5.0.9-1.mga7 1 1.mga7 i586 kernel-desktop-latest 5.0.9 1.mga7 i586 59MB of additional disk space will be used. 53MB of packages will be retrieved. Proceed with the installation of the 2 packages? (Y/n) y http://mirror.math.princeton.edu/pub/mageia/distrib/7/i586/media/core/release/kernel-desktop-5.0.9-1.mga7-1-1.mga7.i586.rpm http://mirror.math.princeton.edu/pub/mageia/distrib/7/i586/media/core/release/kernel-desktop-latest-5.0.9-1.mga7.i586.rpm installing kernel-desktop-latest-5.0.9-1.mga7.i586.rpm kernel-desktop-5.0.9-1.mga7-1-1.mga7.i586.rpm from /var/cache/urpmi/rpms Preparing... #################... 1/2: kernel-desktop-5.0.9-1.mga7 #################... 2/2: kernel-desktop-latest #################... 1/1: removing kernel-desktop-latest-4.14.106-1.mga6.i586 #################... Creating: target|kernel|dracut args|basicmodules remove-boot-splash: Format of /boot/initrd-5.0.9-desktop-1.mga7.img not recognized You should restart your computer for dbus, glibc, kernel-desktop-5.0.9-1.mga7, systemd real 3m3.160s user 0m56.462s sys 0m17.606s
Good, setting this to fixed then.
Resolution: (none) => FIXEDStatus: ASSIGNED => RESOLVED