s/drakconf/dracut, of course.

Hi, I'm the guy testimoning in the forum.
It may be worth to go directly to the "trou de mémoire" explaination post.https://www.mageialinux-online.org/forum/topic-29195-3+m7-vers-m8-erreurs-udevadm-cryptsetup_2-0.php#m290349

If needed, I can provide more information, such as the pieces in /root/.MgaOnline/gurpmi_upgrade_to_8_Adw27DYs.log

CC: (none) => antonin.roussel

You already provide on the forum an interesting output :
> rpm -qa --last
> [...]
> lib64cryptsetup12-2.3.4-2.mga8.x86_64         dim. 01 août 2021 00:08:55
> cryptsetup-2.3.4-2.mga8.x86_64                dim. 01 août 2021 00:08:55
> [...]
> kernel-desktop-latest-5.10.52-1.mga8.x86_64   ven. 30 juil. 2021 13:21:38
> [...]
which proves that the kernel were updates before the required library.

The forum topic is long (3 pages) and varied; the link given in comment 2: #A1# #A2# #A3# gives other problems, but not the one this bug is about.

The post 01/08/2021 à 14h51 questions the claim "Le système a bien installé le noyau construit l'initrd puis mis à jour la librairie cryptsetup" [= this bug] but concludes "il y a de fortes chances que l' initrd ait été construit avec une cryptsetup différente de celle du noyau" = "there is a strong chance that initrd was built with a different cryptsetup to that of the kernel".

Unsure about the 'critical' status: it is unclear whether the user is stuck with an unworkable system. So much went wrong...

Assigning to the MageiaTools team re the Upgrade failures.

CC: (none) => lewyssmith
Assignee: bugsquad => mageiatools

Reducing the severity since the old mga7 kernel should still be bootable.
Please increase the severity again if it isn't bootable.

Workaround is to boot the mga7 kernel and rerun ...
dracut -f /boot/initrd-5.10.55-server-1.mga8.img 5.10.55-server-1.mga8
manually after the upgrade, replacing the kernel version/flavor as appropriate.

CC: (none) => davidwhodgins
Severity: critical => normal

This isn't an errata item. The code that determines the order to install
updates should ensure that if cryptsetup is going to be updated, dracut
doesn't run until after cryptsetup has been updated.

Yes there is a workaround (assuming m7 is still bootable, which may not be
true if /usr is encrypted, since cryptsetup from m8 has been installed), but
even if it is bootable, that just lowers the severity, not the need for the
bug to be fixed.

Thinking about it a bit more, I should say it isn't just an errata item.

For the errata, please include that it if using cryptsetup in mga7, then
after the upgrade has completed, but before rebooting, run the above dracut
command. That can be done safely at that time whether the cryptsetup update
was installed before or after the dracut command was run during the upgrade.

Yes "for errata" does not imply "wontfix" :)

What I don't understand is when this problem occur.  Some short sentence for part of this errata item.

In English forum: https://forums.mageia.org/en/viewtopic.php?f=8&t=14268

For people using cryptsetup after upgrading from Mageia 7 to 8 finishes
installing the updates, please run ...
dracut -f /boot/initrd-5.10.55-server-1.mga8.img 5.10.55-server-1.mga8
before rebooting, replacing the kernel version/flavor as appropriate based
on the mga8 kernel that has been installed. This is to ensure the initrd
contains the correct version of the cryptsetup packages which may not be
the case if the cryptsetup package was installed after the upgrade ran
the dracut command. See bug 29316 for details.

About rebooting in mga7:
My booting choice for this system:
- 5.10.52-desktop-1.mga8 OK (but not mga7)
- 5.10.46-desktop-1.mga7 KO: cryptsetup error, ???
- 5.10.45-desktop-2.mga7 OK: emergency mode, with frozen keyboard... => Power button to halt.

Emergency is coming with that display:  (leading [   2.246038], [2.**] removed)
systemd[1]: Failed to look up module alias 'autofs4': Function not implemented
systemd[1]: Failed to mount NFSD configuration filesystem.
systemd[1]: Failed to start Cet Up Additional Binary Formats.
systemd[1]: Failed to start Load Legacy module configuration.
You are in emergency mode. After logging in, type "journalctl -xb" to view
system logs, "systemctl reboot" to reboot, "systemctl default" or "exit"
to boot into default mode.
Donnez le mot de passe du superutilisateur pour la maintenance
(ou appuyez sur Ctrl et D pour continuer) : _


My thought: Is this "no more keyboard" problem coming from removed mga7 packages?

Thank you Dave, added almost verbatim:
https://wiki.mageia.org/en/Mageia_8_Errata#Cryptsetup

And thanks Jybz for good explanation and hunting :)

Many users will not read this until after they hit the problem...
(especially as we have not posted it until now)

Per comment 11, booting mga7 give no useable terminal.
Can we find ans describe a workaround for that, or is only option to boot live and chroot per comment?

Keeping "for errata" until we can describe how to get out of it.
i.e simple explanation how to use chroot
https://wiki.mageia.org/en/Chroot is unfortunately intended for another use case.

Can the situation be repaired by running classic install ISO with updates repos enabled (and tainted etc if they have been used by the target)?

Status comment: Workaround in #c5 => (none)

I think that for our particular case, to many mga7 packages were removed for trying to get out of this situation and thus could lead to this un-bootable mga7 kernels state, which, I believe, in fresh update should normally boot.

I hope that we agree, workarounds are not solutions for the next release.

On my side, I've an encrypted root partition and this error didn't occurred. Am I lucky?

Antonin, can you first confirm that one of your partition is encrypted, then, can you describe clearly the steps to mount, chroot and regenerating the initrd with dracut ?

OK thanks. The errata currently ends with:

" If you already rebooted and failed, try booting the mga7 kernel still installed to maintenance mode and issue said dracut command. "

Neither mine nor my wifes system experienced this problem upgrading from mga7.  Unfortunately i right now dont remember the method i used to upgrade them.
Both systems use LVM on an encrypted partition, separate /boot.

As this hits seldom, and the non working maintenance mode may be due to experiments, i deem the errata is OK for now.  And errata links here. (Of course if we can improve it, please do :) )

Of course, we should aim to track down and fix the cause.

Keywords: FOR_ERRATA8 => IN_ERRATA8

#c13 Yes sure :
I have got none encrypted partition.

Following two ways to regenerate initrd : from a rescue system, or from Mageia rescue.

*****
* 1 * In the forum we used a parallel system with mga8 to mount and chroot
*****
Once in "rescue mga8", we need to chroot with / /boot /boot/EFI /usr /usr/bin  (for dracut executable)
Depending on partion table it can be a one line mount, or a bit more.

In my case steps are :
# choose your chroot mount point
mkdir /mnt/chroot

# mount /                                 (which includes /boot)
mount /dev/sda2 /mnt/chroot

# mount /boot/EFI
mount /dev/sda1 /mnt/chroot/boot/EFI

# mount /usr                              (which includes /usr/bin)
mount /dev/sda5 /mnt/chroot/usr

# chroot
chroot /mnt/chroot

# regenerate initrd
cd /boot
dracut -f initrd-5.10.52-desktop-1.mga8.img 5.10.52-desktop-1.mga8

# In case of success, you should be able to reboot directly on chrooted system.


*****
* 2 * From the net-install iso and it's Rescue mode, Mageia Linux Rescue Disk
*****
# Choose desired action
  1> Mount your partitions under /mnt

  2> Got to console
cd /boot
dracut -f initrd-5.10.52-desktop-1.mga8.img 5.10.52-desktop-1.mga8

NB: As I tested it right now, already existing initrd-5.10.52-desktop-1.mga8.img file lead dracut to suggest me to use --force

(In reply to Jybz from comment #13)

> On my side, I've an encrypted root partition and this error didn't occurred.
> Am I lucky?

FWIW, from what I understood from the forums report, which the affected person also posted in the english forum, the root cause was that the upgrade to mga8 was interrupted by a lack of disk space. That was followed by a reboot, which ended up with a broken initrd and due to that a broken udevadm (and hence also some devices not working if udev cannot create all device nodes).

Only protection or fix for that would probably be to ensure that for all packages in a transaction of priority packages the required diskspace for download AND installation is available beforehand. That won't catch corner cases where the diskspace is filled up in the meantime by something else, but it should help against the issue reported here.

CC: (none) => doktor5000

Oops!
Thanks, you seem to be correct.
Did not read that thorough until now myself.
Removing the errata item.

User seem to have hit into two problems we already warn for: lock screen and disk space.

@Jybz, could there be a potential problem with packages sequence anyway in some case, but we have just not seen that hit yet on a noninterrupted upgrade?

Keywords: IN_ERRATA8 => (none)

Thank you Dave & Morgan for your work on this, and Florian for his insight.

Running out of disc space is rather basic, and this case was not helped by having separate partitions for several main directories (without, I think, LVM to help).

I think when I did my own large upgrade I used a command option to provide an additional directory in a large space for the downloads.

CC: lewyssmith => (none)

Closing as invalid.

The bug can always be reopened if it's reported again, but without a space
problem preventing an updated initrd from being written.

Status: NEW => RESOLVED
Resolution: (none) => INVALID