Bug 16128 - initrd missing for newest kernel after upgrading with classical iso when using grub2, leading to kernel panic when booting
Summary: initrd missing for newest kernel after upgrading with classical iso when usin...
Status: RESOLVED OLD
Alias: None
Product: Mageia
Classification: Unclassified
Component: Installer (show other bugs)
Version: Cauldron
Hardware: All Linux
Priority: Normal normal
Target Milestone: ---
Assignee: Mageia Bug Squad
QA Contact:
URL:
Whiteboard:
Keywords: NEEDINFO
Depends on:
Blocks: 15637
  Show dependency treegraph
 
Reported: 2015-06-16 13:48 CEST by Marja Van Waes
Modified: 2016-06-26 19:07 CEST (History)
8 users (show)

See Also:
Source RPM:
CVE:
Status comment:


Attachments
add more debug to logs (1002 bytes, patch)
2015-06-16 14:00 CEST, Thierry Vignaud
Details | Diff
Live OEM patch (721 bytes, text/plain)
2015-06-16 14:06 CEST, Thierry Vignaud
Details
no new kernel, so no missing initrd, but anyway report.bug.xz with patch-oem.pl (34.81 KB, application/x-xz)
2015-06-16 20:42 CEST, Marja Van Waes
Details
Live patch (761 bytes, text/plain)
2015-06-17 12:46 CEST, Thierry Vignaud
Details
report.bug.xz with patch (133.40 KB, application/x-xz)
2015-06-18 12:15 CEST, Marja Van Waes
Details

Description Marja Van Waes 2015-06-16 13:48:53 CEST
+++ This bug was initially created as a clone of Bug #11827 +++

Like in that bug, after upgrade newest initrd was missing, resulting in a kernel panic when booting with the newest kernel. 
This time initrd-3.19.8-desktop-3.mga5.img was missing after upgrading with last night's (round 9) 32bits Classical iso

Tv checked newest report.bug.xz (see attachment 6744 [details] ) and found it's a different issue than the solved bug 11827, so cloning the bug report.
Comment 1 Thierry Vignaud 2015-06-16 14:00:05 CEST
The issue is that when we checked, we didn't found the vmlinuz images under /boot (really /mnt/boot).
Though update-grub2 found them later???

* dropping bootloader entry Mageia 5 (5) (op /dev/sdb5) since /boot/vmlinuz-desktop586 doesn't exist
* dropping bootloader entry Mageia (op /dev/sdb5) since /boot/vmlinuz-desktop586 doesn't exist
* dropping bootloader entry Mageia, with Linux desktop586 (op /dev/sdb5) since /boot/vmlinuz-desktop586 doesn't exist
* dropping bootloader entry Mageia, with Linux desktop586 (recovery mode) (op /dev/sdb5) since /boot/vmlinuz-desktop586 doesn't exist
* dropping bootloader entry Mageia, with Linux 3.19.8-desktop586-2.mga5 (op /dev/sdb5) since /boot/vmlinuz-3.19.8-desktop586-2.mga5 doesn't exist
* perImageAppend is now root=UUID=dabecb2d-e2bb-455c-9369-1c9de26a0f03 ro   splash quiet noiswmd resume=UUID=16e408de-6aea-4e19-b3a4-5522522c058a

What's strange is the existing config speaks about /dev/sdb5 whereas you were installing on /dev/sda7 !!!!!

Keywords: (none) => NEEDINFO
Source RPM: grub2 => (none)

Comment 2 Thierry Vignaud 2015-06-16 14:00:59 CEST
Created attachment 6745 [details]
add more debug to logs

Can you try with this patch?
Comment 3 Thierry Vignaud 2015-06-16 14:06:31 CEST
Created attachment 6746 [details]
Live OEM patch

This one also log a listing of /mnt/boot
Thierry Vignaud 2015-06-16 14:07:10 CEST

Attachment 6746 is patch: 1 => 0

Comment 4 Thierry Vignaud 2015-06-16 14:07:52 CEST
You can also add the "ls -l" line to first patch (the one for patching mdkinst.sqfs)
Comment 5 Marja Van Waes 2015-06-16 14:19:46 CEST
(In reply to Thierry Vignaud from comment #1)


> 
> What's strange is the existing config speaks about /dev/sdb5 whereas you
> were installing on /dev/sda7 !!!!!

I installed to (or rather: upgraded) sda7. There is a cauldron on sda5, too. However, I cannot imagine ever having used that old laptop to install to an external HD or to a USB-key and certainly not recently. Going by the above, sdb5 dates from very recently :-( (3.19.8-desktop586-2.mga5 is a very recent kernel)

I intend to use the patch later today
claire robinson 2015-06-16 15:06:35 CEST

CC: eeeemail => (none)

Comment 6 Marja Van Waes 2015-06-16 20:42:12 CEST
Created attachment 6747 [details]
no new kernel, so no missing initrd, but anyway report.bug.xz with patch-oem.pl

(In reply to Thierry Vignaud from comment #3)
> Created attachment 6746 [details]
> Live OEM patch
> 
> This one also log a listing of /mnt/boot

Tried with boot.iso and that patch.

I didn't manage to install the newest kernel, simply removing the newest kernel and then doing an upgrade install is apparently not enough.
Despite lines like 
* found 2 packages to install: kernel-desktop-latest-3.19.8-3.mga5.i586, kernel-desktop-3.19.8-3.mga5-1-1.mga5.i586
* install::pkgs::install the following: kernel-desktop-3.19.8-3.mga5 kernel-desktop-latest
and, iinm, no messages about the kernel being deselected, after reboot there is no kernel 3.19.8-3, so no needed initrd for it, either.

attaching report.bug.xz anyway, in case there is anything useful in it.

This time the other install was correctly seen on sda5 instead of sdb5
Comment 7 Thierry Vignaud 2015-06-17 06:35:11 CEST
You must downgrade kernel-desktop-latest
Also patch wasn't applied.
You must put it as patch.pl on a USB key.
Comment 8 Marja Van Waes 2015-06-17 10:27:35 CEST
(In reply to Thierry Vignaud from comment #7)
> You must downgrade kernel-desktop-latest

thx, will do when I try again

> Also patch wasn't applied.
> You must put it as patch.pl on a USB key.

that's what I did on my first try:
* put the patch as patch.pl in the top directory of a USB key
* start the traditional iso
* hit Esc in the boot menu
* type "linux patch"

However, I thought it wasn't applied because I saw lines in the ddebug.log about installer looking for *patch-oem.pl* in a different place (/i586 and /i586/install, iinm). :-/

That's why I decided to do a boot.iso install after renaming the patch and putting it one directory up from stage2 :-(
Comment 9 Marja Van Waes 2015-06-17 10:28:22 CEST
btw, in case that matters: the traditional iso was on a different USB key, so not on a DVD
Comment 10 Marja Van Waes 2015-06-17 11:53:55 CEST
More precisely, when I tried with patch.pl in the top directory of the USB key, I got:

* getFile install/patch-oem.pl on cdrom://i586
* change_phys_medium cdrom://i586 for file install/patch-oem.pl
* Cannot open /tmp/media/i586/install/patch-oem.pl: No such file or directory

after renaming it and putting it one level up from mdkinst.sqfs, I got:

* getFile install/patch-oem.pl on disk://sdc1/mageia/distrib/cauldron/i586

How could I have seen that patch.pl from the USB-key was applied the first time I tried?
Comment 11 Marja Van Waes 2015-06-17 11:58:48 CEST
and that disk://sdc1/mageia/distrib/cauldron/i586/install.patch-oem.pl was _not_ applied the second time? There was no error following the "getFile" line, but no message about it being applied, either.
Comment 12 Thierry Vignaud 2015-06-17 12:41:47 CEST
See http://gitweb.mageia.org/software/drakx/tree/perl-install/install/install2.pm#n661

Installer always check whether there's an OEM patch or not.
Then _IF_ the "defcfg=foobar" option was given, it'll try to load the given "foobar" name.
Then _IF_ the "patch" keyword was applied

If it succeeds in any of those, it logs so:
- "successfully read oem patch"
- "successfully read default configuration: $cfg"
- "successfully read patch"

For patch, it'll check for both patch & patch.pl:
http://gitweb.mageia.org/software/drakx/tree/perl-install/install/any.pm#n1086

If there's an error, it doesn't log so (which we should do in mga6)

Anyway, as you can use a custom mdkinst.sqfs, you might just patch it instead of using a live patch.
See attachment #6745 [details]
You can also add the following line in bootloader.pm
    log::l("found kernels: ", join(', ', @kernels));
after the call to :
    my @kernels = get_kernels_and_labels();

Which BTW may be an issue with the live patch :-(
Comment 13 Thierry Vignaud 2015-06-17 12:46:58 CEST
Created attachment 6748 [details]
Live patch

fixed Live patch

Attachment 6746 is obsolete: 0 => 1

Marja Van Waes 2015-06-17 15:05:04 CEST

Attachment 6748 mime type: application/octet-stream => text/plain

Comment 14 Marja Van Waes 2015-06-17 18:43:50 CEST
(In reply to Thierry Vignaud from comment #12)


> 
> Anyway, as you can use a custom mdkinst.sqfs, you might just patch it
> instead of using a live patch.
> See attachment #6745 [details]

Trying to build the current unpatched svn version of drakx-installer-stage2 on this 32bit system gives:
FATAL: no match for */LC_MESSAGES/draksnapshot.mo

draksnapshot-0.20.3-11.mga5 is installed, but there is not one 
/usr/share/locale/*/LC_MESSAGES/draksnapshot.mo

locale shows "nl_NL" for everything

Maybe this system is just too borked from iso-testing and not selecting the exact same (set of) locale(s) when upgrading? 

If it's borked, then some different borkedness could be the cause of this bug or "bug".

When I find time, I'll do a fresh stable Mageia 4.1 install on this partition and then use the Live patch from comment 13 when upgrading.

Setting to unconfirmed for now

Status: NEW => UNCONFIRMED
Ever confirmed: 1 => 0

Comment 15 Thierry Vignaud 2015-06-17 18:51:57 CEST
Don't bother rebuilding the package.
it's way much faster to directly patch mdkinst.sqfs:


cp install/stage2/mdkinst.sqfs{,.bak}
unsquashfs install/stage2/mdkinst.sqfs
# do your changes into squashfs-root/usr/lib/libDrakX
# eg: 
# cd squashfs-root/usr/lib/libDrakX; patch -p0 < /where/it/is/patch.diff
rm -f install/stage2/mdkinst.sqfs
mksquashfs squashfs-root/ install/stage2/mdkinst.sqfs
Comment 16 Marja Van Waes 2015-06-18 12:15:47 CEST
Created attachment 6755 [details]
report.bug.xz with patch

(In reply to Thierry Vignaud from comment #15)
> Don't bother rebuilding the package.
> it's way much faster to directly patch mdkinst.sqfs:
> 

Thanks, added it to the drakx tips and tricks :-)

Btw, if a patch doesn't apply cleanly, because a file you patched is not exactly the same as the one I have, do you then want me to say that?

This time, I didn't have a comment (a line starting with #), which can't be a problem. Apart from that, I also had different line numbers.

Anyway, I'm confident I correctly added the new lines from the patch in attachment 6745 [details]

I had first done a fresh Mageia 4 install with updates / on sda7 and /home on (encrypted) sda8 

Then a fresh dual iso cauldron/pre-final install to sda5

Then done an upgrade install of sda7 with patched mdkinst.sqfs

After reboot, new initrd is missing again

Attachment 6747 is obsolete: 0 => 1

Marja Van Waes 2015-06-18 12:16:22 CEST

Keywords: NEEDINFO => (none)
Status: UNCONFIRMED => NEW
Ever confirmed: 0 => 1

Comment 17 Marja Van Waes 2015-06-18 12:20:21 CEST
oops, now i see one more new line in the first chunk than I remember having seen last night, when I patched mdkinst

Don't look at report.bug.xz yet, let me first check I really added that!
Comment 18 Marja Van Waes 2015-06-18 12:24:39 CEST
diffing bootloader.pm and bootloader.pm.orig shows I did add that line, too

phew :-)
Comment 19 Thierry Vignaud 2015-06-18 14:27:40 CEST
(In reply to Marja van Waes from comment #16)
For the mksquashfs dance, I though the wiki was enough but it looks like not

We strip the comments & POD doc when building stage2 (space saving)
So the patches do not always apply cleanly indeed...
Comment 20 Mageia Robot 2016-01-02 15:01:23 CET
commit 0566e7e6b45f1b5672e24e345a76ad5d0de4c689
Author: Thierry Vignaud <thierry.vignaud@...>
Date:   Tue Jun 16 08:02:01 2015 -0400

    log found kernels (mga#16128)
---
 Commit Link:
   http://gitweb.mageia.org/software/drakx/commit/?id=0566e7e6b45f1b5672e24e345a76ad5d0de4c689
Comment 21 Mageia Robot 2016-01-02 15:01:43 CET
commit eb4af36767e31e789a40be1471aa761b23b87c7a
Author: Thierry Vignaud <thierry.vignaud@...>
Date:   Tue Jun 16 08:02:01 2015 -0400

    log found kernels (mga#16128)
---
 Commit Link:
   http://gitweb.mageia.org/software/drakx/commit/?id=eb4af36767e31e789a40be1471aa761b23b87c7a
Comment 22 Thierry Vignaud 2016-06-17 11:58:23 CEST
Is it still a valid issue?

Keywords: (none) => NEEDINFO

Comment 23 Marja Van Waes 2016-06-24 20:56:38 CEST
(In reply to Thierry Vignaud from comment #22)
> Is it still a valid issue?

There are too few hours in a day, I hope to find time before we start testing 6sta2.

@ QA-team

I'm not against anyone else upgrade-testing a grub2 system ;-)
Comment 24 Barry Jackson 2016-06-25 01:02:34 CEST
@ Marja
Hi, I can maybe do some upgrade testing but I don't have access to the iso's (only net installs) as it seems the mailing list gets cleared each cycle and I can never find it anyway.
Surely all packagers should automatically get the iso download info without asking so they/we can test our packages etc.
Comment 25 Charles Edwards 2016-06-25 04:30:09 CEST
(In reply to Barry Jackson from comment #24)
> @ Marja
> Hi, I can maybe do some upgrade testing but I don't have access to the iso's
> (only net installs) as it seems the mailing list gets cleared each cycle and
> I can never find it anyway.
> Surely all packagers should automatically get the iso download info without
> asking so they/we can test our packages etc.


If no one has contacted you drop a note to the QA list
qa-discuss@ml.mageia.org

Claire or David H should be able to help you get access to the Stage1 ISOs

CC: (none) => cae

Comment 26 claire robinson 2016-06-25 12:21:40 CEST
When the server is ready Barry I hope we can automate much of the basic stuff, with regular daily/weekly iso builds and OpenQA testing of various types of installation. Packagers could then add scenarios to be tested themselves. 
OpenQA is ready to go and iso builds are easily automated but the server is currently in need of a DC visit.

CC: (none) => eeeemail, vzawalin1

Comment 27 Barry Jackson 2016-06-25 14:16:36 CEST
Sounds good ;)

@marja thanks for the mail :)
Comment 28 Barry Jackson 2016-06-25 20:16:38 CEST
I just did a Mga5 -> Mga6 net-install upgrade (UEFI) using current Cauldron and there was no issue regarding this bug.

Using 6sta1 USB stick failed miserably due to conflicts and deps issues so I didn't get as far as a re-boot.
Comment 29 Thierry Vignaud 2016-06-26 19:07:28 CEST
Closing then?

Status: NEW => RESOLVED
Resolution: (none) => OLD


Note You need to log in before you can comment on or make changes to this bug.