Bug 4541 - kernel panic - not syncing: VFS: unable to mount root fs
Summary: kernel panic - not syncing: VFS: unable to mount root fs
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: Cauldron
Hardware: i586 Linux
Priority: Normal normal
Target Milestone: ---
Assignee: Thomas Backlund
QA Contact:
URL:
Whiteboard:
Keywords:
: 4535 4545 4616 (view as bug list)
Depends on:
Blocks: 2120 3342
  Show dependency treegraph
 
Reported: 2012-02-16 13:41 CET by luca pedrielli
Modified: 2012-05-02 21:29 CEST (History)
10 users (show)

See Also:
Source RPM: kernel
CVE:
Status comment:


Attachments
kernel panic on boot (47.20 KB, image/png)
2012-02-16 13:42 CET, luca pedrielli
Details
lsinitrd and dracut output (11.00 KB, text/plain)
2012-02-16 18:56 CET, luca pedrielli
Details
lsinitrd and dracut outout (11.00 KB, text/plain)
2012-02-16 18:59 CET, luca pedrielli
Details
lsinitrd for initrd-3.2.6-desktop-3.mga2.img (45.32 KB, text/plain)
2012-02-18 00:14 CET, tu kozaki
Details
lsinitrd for initrd-3.2.6-desktop-2.mga2.img (10.29 KB, text/plain)
2012-02-18 00:19 CET, tu kozaki
Details

Description luca pedrielli 2012-02-16 13:41:29 CET
Description of problem: 
Unable to boot with the last kernel version


Version-Release number of selected component (if applicable):
3.2.6-desktop586-2

How reproducible:
always
Comment 1 luca pedrielli 2012-02-16 13:42:44 CET
Created attachment 1570 [details]
kernel panic on boot
Comment 2 Manuel Hiebel 2012-02-16 14:11:51 CET
try the kernel before the current one, an update the OS, it should work, if not maybe I have misread the thread in the dev ml.

Source RPM: (none) => kernel

Comment 3 Manuel Hiebel 2012-02-16 16:48:35 CET
*** Bug 4545 has been marked as a duplicate of this bug. ***

CC: (none) => hc

Comment 4 Manuel Hiebel 2012-02-16 16:49:25 CET
Thomas, the bug was fixed ?

Assignee: bugsquad => tmb

Comment 5 Wolfgang Bornath 2012-02-16 18:20:21 CET
Bug confirmed.
 - booted with the previous kernel
 - did an 'urpmi --auto-update' with mirror sync status as of 17:00 Paris time
 - rebooted with new kernel -> same kernel panic as before

Bug not fixed.

CC: (none) => molch.b

Comment 6 Colin Guthrie 2012-02-16 18:31:57 CET
I suspect this is an initrd issue rather than a kernel issue.

See the thread on Cauldron list about New Dracut and looking for testers. Please provide the output of lsinitrd for the latest kernel initrd (lsinitrd /boot/initrd-3.2.6-desktop-2.mga2.img)

Also it would be useful to regenerate the initrd for the latest kernel (boot into the old kernel and do: dracut -f /boot/initrd-3.2.6-desktop-2.mga2.img 3.2.6-desktop-2.mga2) and post the log output.

CC: (none) => mageia

Comment 7 Wolfgang Bornath 2012-02-16 18:48:54 CET
Rebuilding initrd with dracut solved the problem. Thx!

1. Do you still need the output of lsinitrd *before* regenerating initrd?
2. Do you still need the output of the dracut command?

It's a bit tricky because networking does not work on the Cauldron machine (another bug = connection going on and off).
Comment 8 luca pedrielli 2012-02-16 18:56:34 CET
Created attachment 1576 [details]
lsinitrd and dracut output
Comment 9 luca pedrielli 2012-02-16 18:57:25 CET
Rebuilding initrd with dracut doesn't solve the problem for me :-(
Comment 10 luca pedrielli 2012-02-16 18:59:05 CET
Created attachment 1577 [details]
lsinitrd and dracut outout
Colin Guthrie 2012-02-16 19:52:16 CET

Attachment 1576 mime type: application/octet-stream => text/plain

Comment 11 Colin Guthrie 2012-02-16 19:56:12 CET
@luca Yeah this is the same problem reported to me on the ML and IRC from a few folk, but they they cannot reproduce now which is a problem :s

Can I ask, have you updated your dracut to 016 (recently submitted) yet? If not, can you try and update and regenerate? 

The strange thing about your output is that the log clearly shows:

I: *** Including module: base ***

Which should include lots of things in the "bin" folder... but the lsinitrd only shows the two plymouth binaries being included in the end result :s

I'm quietly hoping that the newer dracut (016) fixes the problem, so if you could let me know about that first that would be great and if it's still an issue, I will look further.
Comment 12 Henrik Christiansen 2012-02-16 20:11:36 CET
Just to confirm that rebuilding make it bootable again.

dracut -f /boot/initrd-3.2.6-desktop-2.mga2.img 3.2.6-desktop-2.mga2
I: *** Including module: dash ***
I: *** Including module: i18n ***
I: *** Including module: rpmversion ***
I: *** Including module: plymouth ***
I: *** Including module: kernel-modules ***
I: *** Including module: resume ***
I: *** Including module: rootfs-block ***
I: *** Including module: terminfo ***
I: *** Including module: udev-rules ***
I: Skipping udev rule: 50-udev.rules
I: Skipping udev rule: 95-late.rules
I: Skipping udev rule: 50-firmware.rules
I: *** Including module: usrmount ***
I: *** Including module: base ***
I: *** Including module: fs-lib ***
I: *** Including module: shutdown ***
I: Skipping program kexec as it cannot be found and is flagged to be optional
I: *** Including modules done ***
I: Wrote /boot/initrd-3.2.6-desktop-2.mga2.img:
I: -rw-r--r-- 1 root root 5174586 Feb 16 20:00 /boot/initrd-3.2.6-desktop-2.mga2.img
Comment 13 luca pedrielli 2012-02-16 21:06:06 CET
kernel 3.2.6-desktop586-3.mga2 and dracut-016-1.mga2 solve the problem

thanks.
Comment 14 Colin Guthrie 2012-02-16 21:07:32 CET
Cool, well I think we can probably put this down to a problem with dracut that is now resolved... Sorry for the hassle everyone.

Status: NEW => RESOLVED
Resolution: (none) => FIXED

Comment 15 tu kozaki 2012-02-18 00:12:51 CET
Hi,

Have the same problem both with 3.2.6-desktop-2.mga2 & 3.2.6-desktop-3.mga2.

Here are initrd listing, with only the first two allowing me to boot:
-rw-r--r-- 1 root root 7,0M févr.  8 01:16 /boot/initrd-3.2.5-desktop-1.mga2.img
-rw-r--r-- 1 root root 7,1M févr. 12 19:12 /boot/initrd-3.2.6-desktop-0.rc1.1.mga2.img
-rw-r--r-- 1 root root 2,3M févr. 15 10:49 /boot/initrd-3.2.6-desktop-2.mga2.img
-rw-r--r-- 1 root root 6,2M févr. 17 10:59 /boot/initrd-3.2.6-desktop-3.mga2.img

3.2.5-desktop-1.mga2 & 3.2.6-desktop-0.rc1.1.mga2 boot and run fine.
- 3.2.6-desktop-2.mga2 couldn't see my RAID0 arrays 
- 3.2.6-desktop-3.mga2 dropped me in rescue shell

fdisk -l /dev/sd{a,b}
/dev/sda1   *        2048      206848      102400+  83  Linux
/dev/sda2          207360   160836479    80314560   fd  RAID Linux autodétecté
/dev/sdb1   *        2048      206848      102400+  83  Linux
/dev/sdb2          207360   160836479    80314560   fd  RAID Linux autodétecté

Will attach lsinitrd for both initrd-3.2.6-desktop-{2,3}.mga2
Should I regenerate the initrd of latest kernel? (in case I understood it wrong and 3.2.6-desktop-3.mga2 corrected the initrd bug)

CC: (none) => tukozaki

Comment 16 tu kozaki 2012-02-18 00:14:24 CET
Created attachment 1586 [details]
lsinitrd for initrd-3.2.6-desktop-3.mga2.img

# lsinitrd /boot/initrd-3.2.6-desktop-3.mga2.img
Comment 17 tu kozaki 2012-02-18 00:19:05 CET
Created attachment 1587 [details]
lsinitrd for initrd-3.2.6-desktop-2.mga2.img
Colin Guthrie 2012-02-18 00:29:01 CET

Attachment 1586 mime type: application/octet-stream => text/plain

Colin Guthrie 2012-02-18 00:29:07 CET

Attachment 1587 mime type: application/octet-stream => text/plain

Comment 18 Colin Guthrie 2012-02-18 00:35:43 CET
@tu kozaki. It seems the initrd generated for 3.2.6-desktop-2.mga2 was totally busted (it's really small and doesn't contain anything other than plymouth in the /bin" folder (you can check with lsinitrd /boot/initrd-3.2.6-desktop-2.mga2.img | grep " bin/" - or just by looking :))

But the one generated for 3.2.6-desktop-3.mga2 looks a bit better. It has some other stuff in /bin, but also likely does not include the necessary stuff to properly include the raid support in the initrd.

I'm debugging this issue just now. I can give you a hack that will fix it up with a bit of manual intervention.

 1. Find out the UUID of your raid (mdadm --details /dev/md0 | grep UUID - assuming your raid is md0)
 2. Edit the file: /usr/lib/dracut/modules.d/90mdraid/module-setup.sh and on line 43 put: echo " rd.dm.uuid=MYUUID " >> "${initdir}/etc/cmdline.d/90dmraid.conf" (where MYUUID is the UUID found in step 1)
 3. Regenerate the initrd with: dracut -f /boot/initrd-3.2.6-desktop-3.mga2.img 3.2.6-desktop-3.mga2

That should result in a bootable system again.



To explain, the general issue is really two part. One is that it seems to not work for us generally - dunno why - but also that it certainly will not work for upgrades from mga1 if done via urpmi on a running system. This is because dracut is using udev metadata to get information about RAID and LVM systems. Sadly this just isn't available when booting with mga1's mkinitrd generated ramdisk. So we need to find another way to extract this info. Likely we'll use mdadm directly. I'm tracking this specific change in bug #4562
Comment 19 Frank Griffin 2012-02-20 15:37:31 CET
This bug is marked RESOLVED FIXED, but according to comment 18 the latest kernel in the repositories (-3) still requires manual intervention to work.  Which is it ?

CC: (none) => ftg

Comment 20 tu kozaki 2012-02-20 16:52:43 CET
(In reply to comment #18)
@Colin thank you.
First, this can wait the time you're working on it (people, see bug #4562).
Got a doubt on step 2:
Where exactly should your hack be added in /usr/lib/dracut/modules.d/90mdraid/module-setup.sh please? Here are lines 41 to 44:

41. [[ $hostonly ]] || [[ $mount_needs ]] && {
42.        for_each_host_dev_fs check_mdraid
43.        [[ -f "${initdir}/etc/cmdline.d/90mdraid.conf" ]] || return 1
44.    }

Here your hack line with my Raid UUID in it:
        echo " rd.dm.uuid=bf6034b3:4e856ec6:d788b34a:57f9e8bd " >> "${initdir}/etc/cmdline.d/90dmraid.conf"


@Frank, sorry I didn't change Status in comment #15; at least here it's not yet FIXED with latest kernel in repositories.

Status: RESOLVED => UNCONFIRMED
Resolution: FIXED => (none)
Ever confirmed: 1 => 0

Comment 21 Wolfgang Bornath 2012-02-20 19:03:49 CET
Same here - just did an update of an older Alpha2 to latest status (today 17:00 Paris time) - Update went without problems, booting started kernel 3.2.6-desktop-3.mga - resulting in the described kernel panic.
Comment 22 Pierre Bonneau 2012-02-21 09:30:49 CET
Same issue yesterday (20 feb 2012) with the last kernel release. i got the root UID problem.

I booted with the previous kernel, run an update with no result.
i will try the dracut command this evening.

CC: (none) => pmithrandir

Comment 23 Wolfgang Bornath 2012-02-21 10:00:05 CET
Ok, as with the previous kernel so with the current kernel. Used the dracut command (comment #6) to regenerate initrd, boot success.
Comment 24 Colin Guthrie 2012-02-21 10:32:01 CET
@wobo I was going to check to see if the newer dracut had been installed when the kernel stuff was run, but looking at the kernel specs, it does require(pre) dracut >= 016 so that really cannot happen.

I'm struggling to see how this can be the case during upgrade, and then work every time when run manually... 

Can you check the mkinitrd syslink to make sure it's OK?
[colin@jimmy ~]$ ll /sbin/mkinitrd
lrwxrwxrwx 1 root root 26 May 14  2010 /sbin/mkinitrd -> /etc/alternatives/mkinitrd*
[colin@jimmy ~]$ ll /etc/alternatives/mkinitrd
lrwxrwxrwx 1 root root 24 Feb 16 19:04 /etc/alternatives/mkinitrd -> /usr/bin/mkinitrd-dracut*

Perhaps something has gone wrong (again) with the mkinitrd symlink and thus everything done when going via the upgrades fails but running dracut manually works.


For the avoidance of doubt, there are two bugs intermingling here... one that breaks the generation of the initrd full stop and another that breaks only for some lvm+raid setups.

Please keep this bug ONLY for the former case.
Comment 25 Wolfgang Bornath 2012-02-21 10:52:08 CET
Links are ok here:

$ ll /sbin/mkinitrd
lrwxrwxrwx 1 root root 26 Dez 17 15:57 /sbin/mkinitrd ->
/etc/alternatives/mkinitrd*
$ ll /etc/alternatives/mkinitrd
lrwxrwxrwx 1 root root 24 Feb 20 18:34 /etc/alternatives/mkinitrd ->
/usr/bin/mkinitrd-dracut*

I don't use lvm nor raid, so it seems as if the generation of the initrd sucks in general.
Comment 26 Wolfgang Bornath 2012-02-21 10:53:42 CET
Sorry, forgot: dracut is 016-1.mga2
Comment 27 Colin Guthrie 2012-02-21 11:03:41 CET
(In reply to comment #25)
> so it seems as if the generation of the initrd sucks in general.

But that's the problem, it doesn't. No one has been able to cause the problem on-demand. It's only *ever* happened during kernel install. If it sucked in general that would be good, because then it could be found and fixed. This current situation is the most frustrating.

Do you still have the broken image you generated? Can you chuck it up on a URL for me?
Comment 28 Pierre Bonneau 2012-02-21 11:22:36 CET
I don't know for others, but both time dracut disn't do the job for me were after my holydays.

For exemple, in an update with 7+ days of updates to do. (and KDE updates both time, so a lot of packages)

Is it possible to reproduce it from alpha 1 installation ?
Comment 29 Wolfgang Bornath 2012-02-21 11:35:03 CET
(In reply to comment #27)
> (In reply to comment #25)
> > so it seems as if the generation of the initrd sucks in general.
> 
> But that's the problem, it doesn't. No one has been able to cause the problem
> on-demand. It's only *ever* happened during kernel install. If it sucked in
> general that would be good, because then it could be found and fixed. This
> current situation is the most frustrating.

Ok, misunderstanding.

> Do you still have the broken image you generated? Can you chuck it up on a URL
> for me?

Unfortunately the dracut command seems to overwrite the old image - at 3.1.4 and following I have some .old images, may be they were generated by a script which copies the old one to .old before regenerating.

If you need it I could boot with the previous kernel, remove the new kernel (which should also remove the image). Then re-install the new kernel which would generate a broken image - but all this only after 2nd breakfast! :)
Comment 30 Colin Guthrie 2012-02-21 12:18:34 CET
(In reply to comment #29)
> If you need it I could boot with the previous kernel, remove the new kernel
> (which should also remove the image). Then re-install the new kernel which
> would generate a broken image - but all this only after 2nd breakfast! :)

Oh, if you could that would be really great. I suspect however that this problem will be shy and refuse to show itself when you do this... fingers crossed tho'... :)
Comment 31 Frank Griffin 2012-02-21 13:21:37 CET
Col, I have put off updating and still have my systems at 3.2.2.  As there is a workaround now, I'd be willing to upgrade to generate dignostic data.  Is there anything you'd like to pre-set, such as changes to scripts or conf files to make the install more verbose ?
Comment 32 Wolfgang Bornath 2012-02-21 13:28:40 CET
(In reply to comment #30)
> (In reply to comment #29)
> > If you need it I could boot with the previous kernel, remove the new kernel
> > (which should also remove the image). Then re-install the new kernel which
> > would generate a broken image - but all this only after 2nd breakfast! :)
> 
> Oh, if you could that would be really great. I suspect however that this
> problem will be shy and refuse to show itself when you do this... fingers
> crossed tho'... :)

Important notice:
Do not fill your stomach wit too much food - it may jeopardize your brain!

As I was de-installing kernels I thought I could as well remove older kernels (4) which are not needed any more. Unfortunately I also removed 3.2.6-2mga :( As it is no longer available in the repo I could not re-install it, and I can't uninstall the current kernel because it is the only one I have.

Although it's not my fault that I was born dumb, still I apologize!
Comment 33 Colin Guthrie 2012-02-21 13:50:02 CET
@wobo Ha! Don't worry about it. I'll see if I can reproduce the same problem. I might try and locate an alphaX install ISO too and see if I can reproduce the upgrate problem.

@Frank Cool. I guess the things I'd be interested in are:
 1. What version of dracut is currently installed before upgrade.
 2. When the kernel is installed, is the newer dracut actually fully installed and available (just to rule out any urpmi ordering problems and ensure that it's not a potentially buggy dracut 015 that was installed when the initrd is generated)
 3. When the kernel is installing what commands are actually being run (the mkinird command and the dracut command should both be available in the ps output)
 4. Is the initrd generated after install buggy (lsinitrd /boot/$kernel.img | grep " bin/" ; if only two plymouth binaries listed there then yes, it's buggy).
 5. Does running "dracut -f /boot/$kernel.img $kernel" produce a working initrd (same test as above)?
 6. Are there any log messages from the dracut install in syslog etc.?
 7. If there was an error in 4, does uninstalling then reinstalling it again also result in the same broken initrd?


If possible can you keep backups of all the initrds just for future refefence (tho' I'm not sure its needed, but all the same).

Cheers!
Comment 34 Frank Griffin 2012-02-21 14:08:00 CET
>  2. When the kernel is installed, is the newer dracut actually fully installed
> and available (just to rule out any urpmi ordering problems and ensure that
> it's not a potentially buggy dracut 015 that was installed when the initrd is
> generated)
>  3. When the kernel is installing what commands are actually being run (the
> mkinird command and the dracut command should both be available in the ps
> output)

Do I assume correctly that capturing the stdout/stderr from urpmi --auto-update covers this ?

>  4. Is the initrd generated after install buggy (lsinitrd /boot/$kernel.img |
> grep " bin/" ; if only two plymouth binaries listed there then yes, it's
> buggy).
>  5. Does running "dracut -f /boot/$kernel.img $kernel" produce a working initrd

Do you want this verified through an attempted boot or is the output from (4) definitive ?
Comment 35 Colin Guthrie 2012-02-21 14:56:03 CET
(In reply to comment #34)
> Do I assume correctly that capturing the stdout/stderr from urpmi --auto-update
> covers this ?

I think so yeah. 

> >  5. Does running "dracut -f /boot/$kernel.img $kernel" produce a working initrd
> 
> Do you want this verified through an attempted boot or is the output from (4)
> definitive ?

Just the output from 4 is enough. If you get a very small amount of output (i.e. two plymouth binaries) I'll guarantee that it won't boot. If it's nicely populated, then it should boot (save for any unrelated problems that could crop up!)
Comment 36 Frank Griffin 2012-02-21 16:19:47 CET
Well, that didn't quite go as expected.  

The starting version of dracut was dracut-014-16.mga2.  I ran urpmi --auto-update --keep with stdout and stderr redirected to a urpmi.out file in a /home/xxx/tmp directory, and opened a terminal to tail -f it while it ran.

Right after the kernel installed, tail reported a slew of "file truncated" messages, as though someone had started rewriting the file.

However, I managed to scrape the interesting parts from the terminal window:

****************************************************************************
  335/438: dracut                #######################################
  336/438: kipi-plugins-timeadjust
                                 #######################################
  337/438: printer-utils         #######################################
  338/438: shorewall-doc         #######################################
ldconfig: /usr/lib64/libjpeg.so.62 is not a symbolic link

----------------------------------------------------------------------
More information on package dracut-016-1.mga2.noarch
dracut is the default mkinitrd replacement in mageia

If you really want to use old mkinitrd instead of dracut run
update-alternatives --set mkinitrd /sbin/mkinitrd-mkinitrd

NOTE!
We only support the use of mkinitrd with initscripts.
If you use systemd (the default setup) you _must_ use dracut.

----------------------------------------------------------------------


installing eog-3.3.90-1.mga2.x86_64.rpm ethumb-0.1.1.64327-0.r68120.1.mga2.x86_64.rpm kernel-desktop-3.2.6-3.mga2-1-1.mga2.x86_64.rpm lib64gtkmm2.4_1-2.24.2-4.mga2.x86_64.rpm rpmlint-mageia-policy-0.2.14-3.mga2.noarch.rpm kernel-desktop-devel-latest-3.2.6-3.mga2.x86_64.rpm lib64png12_0-1.2.47-1.mga2.x86_64.rpm gdl-i18n-3.3.90-1.mga2.x86_64.rpm from /mnt/cauldron/x86_64/media/core/release
Preparing...                     #######################################
  339/438: gdl-i18n              #######################################
  340/438: lib64png12_0          #######################################
  341/438: kernel-desktop-devel-latest
                                 #######################################
  342/438: rpmlint-mageia-policy #######################################
  343/438: lib64gtkmm2.4_1       #######################################
  344/438: kernel-desktop-3.2.6-3.mga2
                                 #######################################
WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future rel
I: *** Including module: dash ***
I: *** Including module: i18n ***
I: *** Including module: rpmversion ***
I: *** Including module: plymouth ***
I: *** Including module: kernel-modules **
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
E: WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will be ignored in a future release.
I: *** Including module: resume ***
I: *** Including module: rootfs-block ***
I: *** Including module: terminfo ***
I: *** Including module: udev-rules ***
I: Skipping udev rule: 50-udev.rules
I: Skipping udev rule: 95-late.rules
I: Skipping udev rule: 50-firmware.rules
I: *** Including module: usrmount ***
I: *** Including module: base ***
I: *** Including module: fs-lib ***
I: *** Including module: shutdown ***
I: Skipping program kexec as it cannot be found and is flagged to be optional
I: *** Including modules done ***
I: Wrote /boot/initrd-3.2.6-desktop-3.mga2.im
I: -rw-r--r-- 1 root root 5767487 Feb 21 09:33 /boot/initrd-3.2.6-desktop-3.mga2.img
defaulting background resolution to 1024x768
  345/438: ethumb                #######################################
  346/438: eog                   #######################################
ldconfig: /usr/lib64/libjpeg.so.62 is not a symbolic link

tail: urpmi.out: file truncated
*******************************************************************************

which seems to indicate that dracut 016 did install before kernel.

Here's an interesting thing: in the urpmi output, the 'ls' during the kernel install says the initrd length is 5767487, but if I ls it after urpmi has completed, the length is 5770348.

But, it appears to have worked:

*********************************************************************[root@ftglap boot]# ls -l init*
-rw-r--r-- 1 root root 6380590 Jan 10 10:35 initrd-3.2.0-desktop-1.mga2.img
-rw-r--r-- 1 root root 6415149 Jan 18 10:05 initrd-3.2.1-desktop-1.mga2.img
-rw-r--r-- 1 root root 6551077 Jan 21 11:06 initrd-3.2.1-desktop-2.mga2.img
-rw-r--r-- 1 root root 6550064 Jan 27 17:45 initrd-3.2.2-desktop-1.mga2.img
-rw-r--r-- 1 root root 6585958 Feb  7 19:07 initrd-3.2.5-desktop-1.mga2.img
-rw-r--r-- 1 root root 6594046 Feb 13 08:12 initrd-3.2.6-desktop-0.rc1.1.mga2.img
-rw-r--r-- 1 root root 5770348 Feb 21 09:33 initrd-3.2.6-desktop-3.mga2.img
lrwxrwxrwx 1 root root      31 Feb 21 09:33 initrd-desktop.img -> initrd-3.2.6-desktop-3.mga2.img
lrwxrwxrwx 1 root root      31 Feb 21 09:33 initrd.img -> initrd-3.2.6-desktop-3.mga2.img
-rw-r--r-- 1 root root 5792701 Jan  9 08:06 initrd.img.old
[root@ftglap boot]# lsinitrd initrd.img | grep " bin/"
-rwxr-xr-x   1 root     root        55992 Aug 27 05:58 bin/rm
-rwxr-xr-x   1 root     root        10480 Jan 25 02:56 bin/flock
-rwxr-xr-x   1 root     root        47736 Aug 27 05:58 bin/mkdir
-rwxr-xr-x   1 root     root        64088 Aug 27 05:58 bin/stty
-rwsr-xr-x   1 root     root        70152 Jan 25 02:56 bin/mount
-rwxr-xr-x   1 root     root        18752 Jan 25 02:56 bin/dmesg
-rwxr-xr-x   1 root     root        93472 Sep  8 17:22 bin/dash
-rwxr-xr-x   1 root     root       134568 Nov 18 14:28 bin/grep
-rwsr-xr-x   1 root     root        52544 Jan 25 02:56 bin/umount
-rwxr-xr-x   1 root     root       188096 Feb 16 08:04 bin/systemctl
-rwxr-xr-x   1 root     root        27064 Aug 27 05:58 bin/basename
-rwxr-xr-x   1 root     root        47832 Aug 27 05:58 bin/cat
-rwxr-xr-x   1 root     root        39752 Jul 21  2011 bin/setfont
-rwxr-xr-x   1 root     root       127936 Jan 11  2011 bin/sed
-rwxr-xr-x   1 root     root        27096 Aug 27 05:58 bin/sleep
-rwxr-xr-x   1 root     root       121976 Aug 27 05:58 bin/cp
-rwxr-xr-x   1 root     root        31224 Aug 27 05:58 bin/mknod
lrwxrwxrwx   1 root     root            4 Feb 21 09:32 bin/sh -> dash
-rwxr-xr-x   1 root     root       105880 Aug 27 05:58 bin/ls
-rwxr-xr-x   1 root     root       113720 Aug 27 05:58 bin/mv
-rwxr-xr-x   1 root     root        10408 Jul 21  2011 bin/kbd_mode
-rwxr-xr-x   1 root     root        27096 Aug 27 05:58 bin/uname
-rwxr-xr-x   1 root     root        47768 Aug 27 05:58 bin/ln
-rwxr-xr-x   1 root     root       111400 Jul 21  2011 bin/loadkeys
-rwxr-xr-x   1 root     root        35360 Dec 19 04:11 bin/plymouth
-rwxr-xr-x   1 root     root        77384 Dec 19 04:11 bin/plymouthd

*************************************************************************

and the system reboots fine.
Comment 37 Colin Guthrie 2012-02-21 16:26:54 CET
Many thanks Frank. It's interesting that the file size differs but I suspect that is related to some plymouthy script doing something immediately after (e.g. to append a splash screen or something - I think that's how things used to be done in the past). That's why the time didn't change (as it was completed in the same minute). I'll see if I can look into that.

Sadly, no sign of the elusive bug, but the tests were very helpful to confirm a lot of what I was thinking.
Comment 38 Frank Griffin 2012-02-21 16:36:12 CET
I've got one other system to upgrade; we'll see how that goes.

From the comments put out by the dracut install, I wonder if this could somehow be related to affected folks running sysvinit instead of systemd ?
Comment 39 Colin Guthrie 2012-02-21 16:45:03 CET
(In reply to comment #38)
> From the comments put out by the dracut install, I wonder if this could somehow
> be related to affected folks running sysvinit instead of systemd ?

Hmmm, interesting idea. Perhaps, tho' I can't think why that would be a problem off the top of my head :s I can see how booting in a non-dracut based intitrd to do the install could result in raid or lvm modules not being included, but I just cannot see how the whole of bin/ could be nuked... :s
Comment 40 Frank Griffin 2012-02-21 16:58:43 CET
Well, I was thinking that if they were still using real mkinitrd because of sysvinit...

Here's another thought.  Since, in the error scenario, the only things in bin/ are plymouth modules, and given your theory above about plymouth modifying the initrd after it's been installed, is it possible that it was the plymouth install that somehow did the dirty deed after dracut had built a good initrd ?  Maybe the difference in my case is that a newer plymouth package came along since the other posters got the error, and that fixed the error ?
Comment 41 Thomas Backlund 2012-02-21 17:42:28 CET
Well, we had a plymouth bug truncating initrd, but that was fixed by Anssi in plymouth-0.8.4-0.20111214.3.mga2 on 2012-12-19.

@all that this bug hits... do you have mkinitrd installed?

I'm thinking since current mkinitrd does not support .xz compressed kernels, andything calling mkinitrd will probably screw up initrd
Comment 42 Colin Guthrie 2012-02-21 17:53:01 CET
(In reply to comment #41)
> Well, we had a plymouth bug truncating initrd, but that was fixed by Anssi in
> plymouth-0.8.4-0.20111214.3.mga2 on 2012-12-19.

I doubt even Anssi can fix bugs in the future :D This was actually fixed in December, so I don't think it can be to blame here.... shouldn't rule it out tho' (perhaps it's a similar, but different issue now? - don't know how a truncated initrd would look to lsinitrd... perhaps the bin files are included later and it would happen like this... most of them are copied as part of the 99base module which is quite late on...), so maybe this is the place to look afterall...

> @all that this bug hits... do you have mkinitrd installed?
> 
> I'm thinking since current mkinitrd does not support .xz compressed kernels,
> andything calling mkinitrd will probably screw up initrd

I was thinking of that too. I tried installing mkinitrd just to see what it looks like and it's bin/ folder was quite well populated even if it didn't include any modules, so I don't think that's the route cause... (tho' again, I wouldn't rule it out).
Comment 43 Wolfgang Bornath 2012-02-21 18:04:46 CET
(In reply to comment #41)
> 
> @all that this bug hits... do you have mkinitrd installed?

Sorry, no mkinitrd present here.
Comment 44 Thomas Backlund 2012-02-21 21:31:53 CET
Ok, so another idea... as dracut seems to work when called directly, maybe we have a bug in DrakX...

So for those with the problem...

move the now working initrd to safety:

mv /boot/initrd-3.2.6-desktop-3.mga2.img /boot/initrd-3.2.6-desktop-3.mga2.img.saved

Then call the routine that kernel does when it gets installed:

/sbin/installkernel 3.2.6-desktop-3.mga2


Then check the contents and size of the new /boot/initrd-3.2.6-desktop-3.mga2.img
Comment 45 Filipe Saraiva 2012-02-21 22:40:39 CET
*** Bug 4616 has been marked as a duplicate of this bug. ***

CC: (none) => filip.saraiva

Comment 46 Filipe Saraiva 2012-02-21 22:52:03 CET
I have this same problem with kernel 3.2.6-server-3.mga2 32bits.
The solution in Comment 12 don't works for me.
Comment 47 Colin Guthrie 2012-02-21 23:02:05 CET
@Filipe, are you sure you have exactly the same issue? i.e. when you generate the initrd, it does not populate the "bin" folder therein with the needed binaries? Can you check with 'lsinitrd /boot/initrd-$KERNEL.img | grep " bin/"' and confirm that it only lists the two plymouth binaries? If it does, is this true also with your regenerated initrd?
Comment 48 Filipe Saraiva 2012-02-21 23:09:02 CET
Hi Colin, the outputs of the commands:

=
# dracut -f /boot/initrd-3.2.6-server-3.mga2.img
I: *** Including module: dash ***
I: *** Including module: i18n ***
I: *** Including module: rpmversion ***
I: *** Including module: plymouth ***
I: *** Including module: kernel-modules ***
I: *** Including module: resume ***
I: *** Including module: rootfs-block ***
I: *** Including module: terminfo ***
I: *** Including module: udev-rules ***
I: Skipping udev rule: 50-udev.rules
I: Skipping udev rule: 95-late.rules
I: Skipping udev rule: 50-firmware.rules
I: *** Including module: usrmount ***
I: *** Including module: base ***
I: *** Including module: fs-lib ***
I: *** Including module: shutdown ***
I: Skipping program kexec as it cannot be found and is flagged to be optional
I: *** Including modules done ***
I: Wrote /boot/initrd-3.2.6-server-3.mga2.img:
I: -rw-r--r-- 1 root root 5785027 Fev 21 20:03 /boot/initrd-3.2.6-server-3.mga2.img

=
# lsinitrd /boot/initrd-3.2.6-server-3.mga2.img | grep " bin/"
-rwxr-xr-x   1 root     root       124824 Aug 27 06:51 bin/cp
-rwxr-xr-x   1 root     root        26200 Aug 27 06:51 bin/uname
-rwxr-xr-x   1 root     root        17924 Jan 25 05:37 bin/dmesg
-rwsr-xr-x   1 root     root        68184 Jan 25 05:37 bin/mount
-rwxr-xr-x   1 root     root        38632 Jul 21  2011 bin/setfont
-rwxr-xr-x   1 root     root        63128 Aug 27 06:51 bin/stty
-rwsr-xr-x   1 root     root        47116 Jan 25 05:37 bin/umount
-rwxr-xr-x   1 root     root        46744 Aug 27 06:51 bin/mkdir
-rwxr-xr-x   1 root     root        30296 Aug 27 06:51 bin/mknod
-rwxr-xr-x   1 root     root        34424 Dec 19 07:12 bin/plymouth
-rwxr-xr-x   1 root     root        59064 Aug 27 06:51 bin/rm
-rwxr-xr-x   1 root     root       112728 Aug 27 06:51 bin/ls
-rwxr-xr-x   1 root     root         9692 Jan 25 05:37 bin/flock
-rwxr-xr-x   1 root     root       116632 Aug 27 06:51 bin/mv
-rwxr-xr-x   1 root     root        46776 Aug 27 06:51 bin/ln
-rwxr-xr-x   1 root     root        26204 Aug 27 06:51 bin/basename
-rwxr-xr-x   1 root     root        46808 Aug 27 06:51 bin/cat
-rwxr-xr-x   1 root     root        71832 Dec 19 07:12 bin/plymouthd
-rwxr-xr-x   1 root     root         9648 Jul 21  2011 bin/kbd_mode
-rwxr-xr-x   1 root     root       129124 Nov 18 17:38 bin/grep
-rwxr-xr-x   1 root     root        26200 Aug 27 06:51 bin/sleep
-rwxr-xr-x   1 root     root        86920 Jul 21  2011 bin/loadkeys
-rwxr-xr-x   1 root     root        83972 Sep  8 18:12 bin/dash
lrwxrwxrwx   1 root     root            4 Feb 21 20:03 bin/sh -> dash
-rwxr-xr-x   1 root     root        56728 Jan 11  2011 bin/sed
Comment 49 Frank Griffin 2012-02-21 23:21:07 CET
Second system upgrade - same results as the first, fully populated initrd.
Comment 50 Colin Guthrie 2012-02-22 00:04:02 CET
(In reply to comment #48)
> # dracut -f /boot/initrd-3.2.6-server-3.mga2.img

Be very careful with that command... you should pass the kernel name too as a final argument otherwise it will use the current kernel and thus you end up with an initrd that mismatches the kernel actually used (i.e. named 3.2.6-server-3.mga2, but actually for whatever uname -r says)

You should likely run this command instead:

# dracut -f /boot/initrd-3.2.6-server-3.mga2.img 3.2.6-server-3.mga2


> # lsinitrd /boot/initrd-3.2.6-server-3.mga2.img | grep " bin/"
> -rwxr-xr-x   1 root     root       124824 Aug 27 06:51 bin/cp
...snip...

This looks like a pretty valid initrd and not the problem reported here.. So perhaps if you include the kernel name in the command line, comment 12 solution does work for you after all? Either way, it's something different going on with the regenerated initramfs rather than this bug.
Comment 51 Filipe Saraiva 2012-02-22 00:24:01 CET
Hi Colin, now is ok! :)

I ran the command that you recommended in Comment 50.
Thanks for you help.
Comment 52 Dave Hodgins 2012-02-22 01:04:35 CET
(In reply to comment #31)
> Col, I have put off updating and still have my systems at 3.2.2.  As there is a
> workaround now, I'd be willing to upgrade to generate dignostic data.  Is there
> anything you'd like to pre-set, such as changes to scripts or conf files to
> make the install more verbose ?

Frank, is /boot on a separate filesystem?  What's the output of df.

I'm just wondering if the compress utility is running out of space,
and dracut is failing to identify that.

CC: (none) => davidwhodgins

Comment 53 Frank Griffin 2012-02-22 02:04:28 CET
Nope, /boot is on /.  But, as I'm not seeing the problem on two separate  systems, maybe my situation is not germane.
Comment 54 Pierre Bonneau 2012-02-22 09:25:03 CET
I'm on a different /boot system.

I got the initial issue with the root filesystem not detected.

yesterday evening, I run the commant dracut -f /boot/initrd**3-2.6.3*** blabla-3-2.6.3.mga2

(the one you give uin previous comments).

That resolved the root uid issue, but I got immediatly another error that put me in a dracut command line.

I didn't got a lot of time, so I uninstalled both 3.2.6 (rc1 and 3 release) to come back to the previous kernel.

It works now.

I'm going to run the updates this evening, so I will tell you if I see anything wrong.
If you want me to test it (in case I reproduce the initial bugs) please provide me the command line. I will try to update in commande line and to copy what you need here if I get the problem again...
Comment 55 Wolfgang Bornath 2012-02-22 11:14:36 CET
(In reply to comment #44)
> 
> Then check the contents and size of the new
> /boot/initrd-3.2.6-desktop-3.mga2.img

I did as you suggested. Result:
 - system boots ok with the new initrd
 - the new initrd size is 5425223, the old has 5430179

How can I compare the contents?
Comment 56 Colin Guthrie 2012-03-07 15:54:16 CET
*** Bug 4535 has been marked as a duplicate of this bug. ***

CC: (none) => tavvva

Comment 57 Wolfgang Bornath 2012-03-27 19:21:42 CEST
Ok, today I revived an older Cauldron (as of early February, kernel 3.2.2-desktop-1.mga2) which was running fine. I did all updates as up to today (including kernel  3.3.0-desktop-2.mga2.), rebooted and experienced the same kernel panic!

Booted the old kernel, looked at dracut.log and saw that again initrd was broken (size 611). I did the dracut dance (aka recreating initrd for 3.3.0-desktop-2) and everything was fine, the new initrd had the right size now. Rebooting 3.3.0-desktop-2 succeeded.

Just to confirm that the issue is still there. :(
Comment 58 Colin Guthrie 2012-03-27 20:38:12 CEST
(In reply to comment #57)
> Just to confirm that the issue is still there. :(

Damn, I was quietly hoping this issue had just been factored away :(

I've still only witnessed it once on my VMs (and even now I'm beginning to doubt it even happened :s). But there is definitely something lurking somewhere :s
Colin Guthrie 2012-03-27 20:39:15 CEST

Blocks: (none) => 3342, 2120

Comment 59 Colin Guthrie 2012-04-25 12:45:06 CEST
Has anyone come across this recently? I've not seen it any of my tests and we did push a new bootsplash that fixed a couple of possibly related issues not that long ago. I'm hoping this has disappeared now. Anyone confirm?
Comment 60 luca pedrielli 2012-04-25 16:17:55 CEST
Yes, i think this has disappeared now.
Comment 61 Anne Nicolas 2012-05-02 21:29:31 CEST
closing then feel free to reopen if needed

Status: UNCONFIRMED => RESOLVED
CC: (none) => ennael1
Resolution: (none) => FIXED


Note You need to log in before you can comment on or make changes to this bug.