Bug 30206 - HDD Live-boot via grub2 fails when loopback loop is a Live.ISO
Summary: HDD Live-boot via grub2 fails when loopback loop is a Live.ISO
Status: NEW
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: Cauldron
Hardware: All Linux
Priority: Normal enhancement
Target Milestone: ---
Assignee: Mageia tools maintainers
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-03-25 01:16 CET by Ben McMonagle
Modified: 2022-03-29 23:09 CEST (History)
1 user (show)

See Also:
Source RPM: dracut-051-4.mga8.src.rpm
CVE:
Status comment:


Attachments
/dev/disk/by-label/ warning (3.89 KB, image/png)
2022-03-25 01:18 CET, Ben McMonagle
Details
dracut :/# prompt (boot failure) (14.93 KB, image/png)
2022-03-25 01:19 CET, Ben McMonagle
Details
scanning for all btrfs devices (1.17 KB, image/png)
2022-03-25 01:20 CET, Ben McMonagle
Details
successful live boot (193.82 KB, text/plain)
2022-03-25 01:23 CET, Ben McMonagle
Details
contents of btrfs partition (133 bytes, text/plain)
2022-03-25 01:26 CET, Ben McMonagle
Details
contents of btrfs partition (133 bytes, text/plain)
2022-03-25 01:29 CET, Ben McMonagle
Details
.png 1-32 of /rdsosreport.txt (866.96 KB, application/x-xz)
2022-03-26 07:27 CET, Ben McMonagle
Details
.png 33-61 of /rdsosreport.txt (548.76 KB, application/x-xz)
2022-03-26 07:28 CET, Ben McMonagle
Details

Description Ben McMonagle 2022-03-25 01:16:05 CET
Description of problem:
When Live-booting a Mageia Live.iso from HDD the boot fails and shortly presents a dracut:/# prompt

However, if the directory "loopbacks" is extracted and and is in the same partition as the original .iso, the boot proceeds correctly.

I created a btrfs partition with the same label as the downloaded .iso (Mageia-8-Live-Plasma-x86_64).
A btrfs partition seems to be required, as without it, there is a ~180sec wait while dracut scans for all btrfs devices.

If the btrfs partiton is not correctly labelled, there is a warning that /dev/disk/by-label/Mageia-8-Live-Plasma-x86_64  does not exist, and the boot fails and presents a dracut:/# prompt.

test #1:

using the following /etc/grub.d/40_custom menu entry:
menuentry 'liveboot' {

         insmod gzio
         insmod part_msdos
         insmod btrfs
         insmod ext2
         insmod xfs
         insmod loopback
         insmod iso9660
         insmod udf
set iso="/Mageia-8-Live-Plasma-x86_64.iso"
if loopback loop (hd1,1)$iso 
then
   if search --no-floppy --set=root -l "Mageia-8-Live-Plasma-x86_64"
   then configfile /boot/grub2/grub.cfg
   else search --no-floppy --set=root -f "/boot/grub2/vmlinuz"
   fi
else search --no-floppy --set=root -f "/boot/grub2"
fi
}

1/ the boot firstly presents the expected USB/DVD Grub2 entry select screen and, should the extracted "loopbacks" directory be in the "loopback loop" partition, the boot will proceed correctly

2/the boot firstly presents the expected USB/DVD Grub2 entry select screen and, should the extracted "loopbacks" directory not be in the "loopback loop" partition, as it cannot mount the root filesystem, the boot will fail. 

this implies that as the grub.cfg file is read and acted upon, that the "set iso" and "loopback loop" entries are correct

test #2:

using the following /etc/grub.d/40_custom menu entry:

menuentry "liveboot" {

         insmod gzio
         insmod part_msdos
         insmod btrfs
         insmod ext2
         insmod xfs
         insmod loopback
         insmod iso9660
         insmod udf
 
    set iso="(hd1,1)/Mageia-8-Live-Plasma-x86_64.iso"
    loopback loop $iso
    linux (loop)/boot/vmlinuz root=mgalive:LABEL=Mageia-8-Live-Plasma-x86_64 noiswmd vga=788
    initrd (loop)/boot/initrd.img
}


1/ the boot proceeds and, should the extracted "loopbacks" directory be in the "loopback loop" partition, the boot will proceed correctly

2/the boot quickly fails as it cannot mount the root filesystem and presents the dracut:/# prompt.

its likely i am doing something wrong.
text editor is kwrite.
trials have been in a Oracle Virtualbox and with real hardware.

should --hint-baremetal be an option? 






Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.
Comment 1 Ben McMonagle 2022-03-25 01:18:23 CET
Created attachment 13196 [details]
/dev/disk/by-label/ warning
Comment 2 Ben McMonagle 2022-03-25 01:19:26 CET
Created attachment 13197 [details]
dracut :/# prompt (boot failure)
Comment 3 Ben McMonagle 2022-03-25 01:20:16 CET
Created attachment 13198 [details]
scanning for all btrfs devices
Ben McMonagle 2022-03-25 01:21:05 CET

Attachment 13198 description: scanning.... => scanning for all btrfs devices

Comment 4 Ben McMonagle 2022-03-25 01:23:14 CET
Created attachment 13199 [details]
successful live boot

this requires that "loopbacks" as a directory is in the same partition as the live.iso
Comment 5 Ben McMonagle 2022-03-25 01:26:59 CET
Created attachment 13200 [details]
contents of btrfs partition
Comment 6 Ben McMonagle 2022-03-25 01:29:32 CET
Created attachment 13201 [details]
contents of btrfs partition

Attachment 13200 is obsolete: 0 => 1

Comment 7 Ben McMonagle 2022-03-26 07:26:30 CET
i added rd.debug to the kernel line, but was unable to export the run/initramfs/rdsosreport.txt.

I was however able to screen shot the report (x61 .png) by using 
"sed -n 'n,np' run/initramfs/rdsosreport.txt" and save multiple .png images

created 2 .tar.xz files from the resultant .png files
Comment 8 Ben McMonagle 2022-03-26 07:27:44 CET
Created attachment 13202 [details]
.png 1-32 of /rdsosreport.txt
Comment 9 Ben McMonagle 2022-03-26 07:28:48 CET
Created attachment 13203 [details]
.png 33-61 of /rdsosreport.txt
Comment 10 Ben McMonagle 2022-03-26 07:44:47 CET
(In reply to Ben McMonagle from comment #0)

> 
> 2/the boot quickly fails as it cannot mount the root filesystem and presents
> the dracut:/# prompt.

should be:

2/ should the extracted "loopbacks" directory not be in the "loopback loop" partition, as it cannot mount the root filesystem, the boot quickly fails and presents the dracut:/# prompt.
Comment 11 Martin Whitaker 2022-03-26 11:48:26 CET
This doesn't work because the GRUB loopback command only sets up a loopback device for use by GRUB itself. Once control is passed to the Linux kernel, that loopback has gone.

The root=mgalive:LABEL=Mageia-8-Live-Plasma-x86_64 tells the dracut mgalive module that the compressed root filesystem can be found in the loopbacks directory of a disk/partition labelled Mageia-8-Live-Plasma-x86_64. You have labelled your btrfs partition with that name, so that is where dracut will look. It does not know that it needs to mount the ISO file as a loopback device and then look inside that.

You can avoid the labelling (and hence use any filesytem) by using UUIDs instead. I did this:

1. Create and format an ext4 partition. Note its UUID, e.g 8224e525-e126-4676-a7e1-8b63db0a8718.
2. Copy the contents of a Live ISO to that partition.
3. Add this entry to the main system /boot/grub/grub.cfg

menuentry 'Mageia Live' {
	insmod gzio
	insmod part_msdos
	insmod ext2
        search --no-floppy --fs-uuid --set=root 8224e525-e126-4676-a7e1-8b63db0a8718
	linux	/boot/vmlinuz root=mgalive:UUID="8224e525-e126-4676-a7e1-8b63db0a8718" audit=0 rd.luks=0 rd.lvm=0 rd.md=0 rd.dm=0 vga=788
	initrd	/boot/initrd.img
}

CC: (none) => mageia

Comment 12 Ben McMonagle 2022-03-26 21:07:04 CET
(In reply to Martin Whitaker from comment #11)

> 2. Copy the contents of a Live ISO to that partition.

thanks Martin.
in the two examples I gave, this works too.

however, if the .iso is copied intact (ie not extracted), it does not work.
grub complains that: /boot/vmlinuz not found
Comment 13 Ben McMonagle 2022-03-26 21:21:12 CET
eg:

# blkid
/dev/sda1: UUID="3ffa5781-9270-4ea9-b601-9483ee0eed33" BLOCK_SIZE="4096" TYPE="ext4"
/dev/sda5: UUID="47fce1f8-5f6f-436c-9f26-d7cedcf96d29" TYPE="swap"
/dev/sda6: UUID="835ca37a-f68a-4725-8c5f-9a2dcc0e8e3b" BLOCK_SIZE="4096" TYPE="ext4"
/dev/sda7: LABEL="Mageia-8-Live-Plasma-x86_64" UUID="57bbedb9-b802-47ea-99bd-80b5e837bc7f" UUID_SUB="5cc6519f-0abc-4c92-9b67-48c78bc17949" BLOCK_SIZE="4096" TYPE="btrfs"
/dev/sdb1: LABEL="liveboot" UUID="1afb52f8-daed-4c8d-9e73-3efb00ad2661" BLOCK_SIZE="4096" TYPE="ext4"

/dev/sdb1 is the one I am using, so this becomes: 


menuentry 'Mageia Live' {
    insmod gzio
    insmod part_msdos
    insmod ext2
    search --no-floppy --fs-uuid --set=root 1afb52f8-daed-4c8d-9e73-3efb00ad2661
    linux /boot/vmlinuz root=mgalive:UUID="1afb52f8-daed-4c8d-9e73-3efb00ad2661" audit=0 rd.luks=0 rd.lvm=0 rd.md=0 rd.dm=0 vga=788
    initrd /boot/initrd.img
}

this is with a Mageia-8-Live-Plasma-x86_64.iso, the only file in the partition.
Comment 14 Ben McMonagle 2022-03-26 21:27:14 CET
I used this:

https://www.linuxbabe.com/desktop-linux/boot-from-iso-files-using-grub2-boot-loader

and this as primers:

https://www.gnu.org/software/grub/manual/grub/grub.html

it seems other distros are able to boot from their  Live.iso  when on an internal partition, but Mageia seems 'difficult' :)
Comment 15 Martin Whitaker 2022-03-26 23:40:53 CET
Yes, you have to use the GRUB loopback command to allow GRUB to boot a Linux kernel that's inside a ISO file. But as per my previous comment, the dracut mgalive module can't then mount the compressed root filesystem. So your only option is to extract the contents of the ISO file.

My example was merely to show that you weren't restricted to using btrfs.
Comment 16 Ben McMonagle 2022-03-27 00:10:50 CET
(In reply to Martin Whitaker from comment #15)
> But as per my previous comment, the dracut
> mgalive module can't then mount the compressed root filesystem.

so, is it possible to alter the dracut mgalive module to allow this?

I'm going to ask a stupid question:
you mean that it cannot mount the compressed root file system when "in the .iso", but is able to when it is "outside the .iso"?

I am not the only user attempting this, there is a thread here: https://forums.mageia.org/en/viewtopic.php?f=7&t=14519
Comment 17 Martin Whitaker 2022-03-27 10:50:32 CEST
(In reply to Ben McMonagle from comment #16)
> (In reply to Martin Whitaker from comment #15)
> > But as per my previous comment, the dracut
> > mgalive module can't then mount the compressed root filesystem.
> 
> so, is it possible to alter the dracut mgalive module to allow this?

It's possible. Take a look at /usr/lib/dracut/modules.d/90mgalive/mgalive-root.sh (on any Mageia system)  - it's just a bash script - and have a go.

> I'm going to ask a stupid question:
> you mean that it cannot mount the compressed root file system when "in the
> .iso", but is able to when it is "outside the .iso"?

It can't mount the compressed filesystem - the .sqfs file - when that file is contained within an ISO image file, because the .sqfs file isn't directly accessible via a mount point. mgalive-root.sh needs to be instructed to do a loopback mount of the ISO image file so it then has access to the .sqfs file.

The above applies regardless of whether the Linux kernel was booted via the GRUB loopback method or directly from a disk partition.
Comment 18 Lewis Smith 2022-03-27 21:53:29 CEST
Thank you very much Martin for all your wisdom.

(In reply to Martin Whitaker from comment #17)
> (In reply to Ben McMonagle from comment #16)
> > (In reply to Martin Whitaker from comment #15)
> > > But as per my previous comment, the dracut
> > > mgalive module can't then mount the compressed root filesystem.
> > 
> > so, is it possible to alter the dracut mgalive module to allow this?
> 
> It's possible. Take a look at
> /usr/lib/dracut/modules.d/90mgalive/mgalive-root.sh (on any Mageia system) 
> - it's just a bash script - and have a go.
Given the "It's possible", and you having identified the package concerned, this looks OK to assign to the tools people.

Assignee: bugsquad => mageiatools
Source RPM: (none) => dracut-051-4.mga8.src.rpm

Comment 19 Martin Whitaker 2022-03-28 09:43:28 CEST
Setting severity to enhancement, as this is a new feature request.

Severity: normal => enhancement

Comment 20 Ben McMonagle 2022-03-29 20:53:17 CEST
test environment could be 2 partitions, with one labelled "mga-persist".
would not require rebuild / modifying the actual .iso.

would it be a valid test?
Comment 21 Martin Whitaker 2022-03-29 23:09:08 CEST
You need to regenerate the initrd.img file to test any changes to the dracut modules, and you would need to boot the live system to do that.

I think the easier way is to build a minimal live ISO (as described in https://wiki.mageia.org/en/Draklive2#Example_3_:_Minimal_Live_ISO). Then modify the mgalive-root.sh file in the draklive2 working directory and rerun draklive2 with just the --boot and --master options (which doesn't take long).

Note You need to log in before you can comment on or make changes to this bug.