Bug 28371 - On a USB memory formatted with Ventoy, when you boot a netinstall image, you need to unplug and replug for sources in Ventoy's partition
Summary: On a USB memory formatted with Ventoy, when you boot a netinstall image, you ...
Status: RESOLVED WORKSFORME
Alias: None
Product: Mageia
Classification: Unclassified
Component: Installer (show other bugs)
Version: Cauldron
Hardware: All Linux
Priority: Normal normal
Target Milestone: Mageia 9
Assignee: Thomas Backlund
QA Contact:
URL:
Whiteboard:
Keywords: IN_ERRATA8, NEEDINFO
Depends on:
Blocks:
 
Reported: 2021-02-18 04:48 CET by katnatek
Modified: 2022-03-09 21:17 CET (History)
5 users (show)

See Also:
Source RPM: drakx-installer-binaries
CVE:
Status comment:


Attachments

Description katnatek 2021-02-18 04:48:39 CET
Description of problem:

Not sure if this is valid only for the nonfree flavour of the Network installer image

I create a bootable usb with ventoy - https://www.ventoy.net/en/index.html , i put the network rc1 i586 image and the classical rc1 dvd i586 image, boot the network installer image and try to select as source the dvd image but can't find it, my guess is the installer can't read an exfat ((i choose that type of format in the process)

Version-Release number of selected component (if applicable):

network installer rc1

How reproducible:


Steps to Reproduce:
1. Use ventoy to create a bootable usb
2. Choose exfat as type of the partition that will store the isos
3. Put both network and classical dvd isos (both have to be the same arch i.e. both i586 or both x86_64)
4. Boot from the usb
5. Choose the network image
6. Try to find the dvd image in the partitions of the usb
7. you can't

BTW you can use the dvd image and works well ;)
Comment 1 Lewis Smith 2021-02-18 09:16:43 CET
Thank you for the report. Ventoy is an interesting looking utility; the link is worth following. Several ISOs on one bootable USB... Can you please post the output of
 # fdisk -l /dev/sd...       [the inserted USB stick device]

> you can use the dvd image and works well
Do you mean that if you select the classical rc1 dvd i586 image directly from the Ventoy boot menu, that runs & installs OK?

It is asking a lot, but can you test the theory with a USB formatted FAT32? Our in-house program IsoDumper includes exFAT for the USB format, so maybe it should be supported by the installers. (Although we could theoretically define supported  formats for this case).

CC: (none) => lewyssmith, mageia

Comment 2 Aurelien Oudelet 2021-02-18 09:21:34 CET
Note also that exfat (from Samsung's implementation and GPL'ed) landed in Kernel mainline since 5.9 series.

CC: (none) => ouaurelien

Comment 3 Aurelien Oudelet 2021-02-18 09:27:34 CET
Oh sorry, s/5.9/5.7... in my last reply.
Comment 4 Thomas Backlund 2021-02-18 10:06:42 CET
netinstall isos are probably missing the exfat module... will check

Assignee: bugsquad => tmb
Source RPM: (none) => drakx-installer-images

Comment 5 Thomas Backlund 2021-02-18 10:34:26 CET
actually exfat is on the iso, so that bit is ok.

If I re-read the initial report correctly, this happends when you instead of trying to add online media try to add local media (on exfat) as source...
so that would make it stage1 loader issue...

and looking at that stage1 loader code, we have not added support for exfat.

that also means it wont happend for Mageia 8 as we are in full freeze at this point

Source RPM: drakx-installer-images => drakx-installer-binaries
Target Milestone: --- => Mageia 9

Comment 6 Thomas Backlund 2021-02-18 12:24:01 CET
and reading more code shows we maybe only missed code for loading the module before trying to mount it, so a test iso is coming soon
Comment 7 Thomas Backlund 2021-02-18 12:59:11 CET
try with the netinstall isos with todays date, for example already available here:
http://ftp.free.fr/mirrors/mageia.org/distrib/cauldron/x86_64/install/images/
Comment 8 Thierry Vignaud 2021-02-18 17:09:24 CET
I don't think so since you uploaded it to core/updates_testing.
So I don't think it has been unpacked into install/images/.

Though the iso file can been manually extracted from http://ftp.free.fr/mirrors/mageia.org/distrib/cauldron/x86_64/media/core/updates_testing/

CC: (none) => thierry.vignaud

Comment 9 Thomas Backlund 2021-02-18 17:12:45 CET
(In reply to Thierry Vignaud from comment #8)
> I don't think so since you uploaded it to core/updates_testing.
> So I don't think it has been unpacked into install/images/.
> 

it actually was unpacked (even if I did not intend to...)

 (seems our youri unpacker config only match rpm name, not target media)
Comment 10 Thomas Backlund 2021-02-18 17:14:15 CET
(In reply to Thomas Backlund from comment #9)
> (In reply to Thierry Vignaud from comment #8)
> > I don't think so since you uploaded it to core/updates_testing.
> > So I don't think it has been unpacked into install/images/.
> > 
> 
> it actually was unpacked (even if I did not intend to...)
> 
>  (seems our youri unpacker config only match rpm name, not target media)

and the reason that it does not happend in stable trees is that all unpacking is disabled there
Comment 11 Thomas Backlund 2021-02-18 17:17:27 CET
I think I should maybe add some "info file" to images tree so one can track exactly what d-i-i package is unpacked... something like stage2 adding version (even if that also could add rel so one knows exactly what is on the mirrors
Comment 12 Thierry Vignaud 2021-02-18 17:40:11 CET
Well that info is already in git :-)
http://gitweb.mageia.org/infrastructure/puppet/tree/modules/buildsystem/manifests/var/youri.pp#n144
Comment 13 Lewis Smith 2021-02-18 20:14:35 CET
Thanks for your intervention tmb, & tv.
I do not think we should get in a flap about this. We are quite entitled to say that the installation (ISO) source cannot yet be exFAT, but this will be made possible. There is no shortage of supported formats.

Yet another ERRATA ?
Comment 14 Thomas Backlund 2021-02-18 20:19:28 CET
not yet... as the "exfat fix" I added is only loading exfat module if it gets pointed at a exfat system I think we should be all good... 

if more code changes is needed that will wait for mga9
Comment 15 Thomas Backlund 2021-02-18 20:23:14 CET
and if considered not working I'll simply revert it before final release process and distrib tree branching
Comment 16 Lewis Smith 2021-02-18 20:56:11 CET
Is there any way of getting the reporter to try it in time (I guess enabling your updated NetInstall for him to download)? The best test.
Except he has not replied to my comment 1; nor when I just saw it, his Live ISOs grub bug 28374.
Comment 17 Lewis Smith 2021-02-18 21:10:07 CET
@katnatek
I apologise quickly! That other bug mentioned above is *not* yours.
Confused by several bugs arriving together from E Europe.
Comment 18 katnatek 2021-02-21 04:30:29 CET
(In reply to Lewis Smith from comment #16)
> Is there any way of getting the reporter to try it in time (I guess enabling
> your updated NetInstall for him to download)? The best test.
> Except he has not replied to my comment 1; nor when I just saw it, his Live
> ISOs grub bug 28374.

I try as quick as i can, sorry for not put attention, some issues in RL

Is this


Name	Last Modified	Size	Type
Mageia-8-netinstall-nonfree-i586.iso	2021-Feb-18 12:48:42

Fine ?
Comment 19 katnatek 2021-02-21 05:14:21 CET
(In reply to Lewis Smith from comment #1)
> Thank you for the report. Ventoy is an interesting looking utility; the link
> is worth following. Several ISOs on one bootable USB... Can you please post
> the output of
>  # fdisk -l /dev/sd...       [the inserted USB stick device]

LC_ALL=C fdisk -l /dev/sdb
Disk /dev/sdb: 14.54 GiB, 15610576896 bytes, 30489408 sectors
Disk model: DataTraveler 2.0
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xe1fad4eb

Device     Boot    Start      End  Sectors  Size Id Type
/dev/sdb1  *        2048 30423871 30421824 14.5G  7 HPFS/NTFS/exFAT
/dev/sdb2       30423872 30489407    65536   32M ef EFI (FAT-12/16/32)

> 
> > you can use the dvd image and works well
> Do you mean that if you select the classical rc1 dvd i586 image directly
> from the Ventoy boot menu, that runs & installs OK?
> 
Yes

Aslo i did try the image in my comment (comment#18) and not work first, it appears and error about not found modules list, i play some and could get the partition is read but rebooting i can't repeat the process

I can see listed a fat partition where ventoy stores the bootloader and the other partition but can't be readed
Comment 20 katnatek 2021-02-21 05:52:09 CET
(In reply to katnatek from comment #19)
> Aslo i did try the image in my comment (comment#18) and not work first, it
> appears and error about not found modules list, i play some and could get
> the partition is read but rebooting i can't repeat the process
> 
> I can see listed a fat partition where ventoy stores the bootloader and the
> other partition but can't be readed

Of course the fat partition can be readed, the partition that can't be readed is the partition where the isos are stored i.e the exfat one
Comment 21 Thomas Backlund 2021-02-21 11:53:10 CET
(In reply to katnatek from comment #18)

> 
> Is this
> 
> 
> Name	Last Modified	Size	Type
> Mageia-8-netinstall-nonfree-i586.iso	2021-Feb-18 12:48:42
> 
> Fine ?

yes
Comment 22 katnatek 2021-02-21 18:12:46 CET
(In reply to Thomas Backlund from comment #21)
> (In reply to katnatek from comment #18)
> 
> > 
> > Is this
> > 
> > 
> > Name	Last Modified	Size	Type
> > Mageia-8-netinstall-nonfree-i586.iso	2021-Feb-18 12:48:42
> > 
> > Fine ?
> 
> yes

As say in the comment#19 still not read the exfat , i try to find
how i get it to put a errata
Comment 23 katnatek 2021-02-21 18:45:04 CET
(In reply to Thomas Backlund from comment #21)

Ok, at less if the exfat partition is in a usb device, you need to unplug and replug the usb to get the partition recognized

I can't test for the moment for exfat in a hardrive
Comment 24 katnatek 2021-02-21 19:22:02 CET
Added to erratas

Keywords: (none) => IN_ERRATA8

Comment 25 Morgan Leijström 2021-02-21 20:10:37 CET
Thanks, I was wondering how to write it.

CC: (none) => fri

Lewis Smith 2021-02-21 21:52:42 CET

CC: lewyssmith => (none)

Comment 26 Thierry Vignaud 2022-01-16 23:38:56 CET
(In reply to Thomas Backlund from comment #6)
> and reading more code shows we maybe only missed code for loading the module
> before trying to mount it, so a test iso is coming soon

Nope.
We relied on /bin/mount for quite some time instead of the old internal mount() so that was just dead code that went away for mga9:
https://gitweb.mageia.org/software/drakx/commit/?id=fa7cece91765ed48101d7fd5f281e4f129e0c048

I'm pretty sure that exfat worked smoothly when I tried patched stage1 prior to pushing that commit.
Comment 27 katnatek 2022-01-17 18:56:15 CET
After some test with ventoy and mageia's xfce live images i start to think that maybe if the exfat partition is other usb device that the one were ventoy and network installer are, the process will work fine, sadly i only have one usb  memory and can't make the test
Comment 28 Dave Hodgins 2022-01-22 17:01:41 CET
No problems mounting or reading a file in the exfat partition.

Tested using
Mageia-Cauldron-netinstall-x86_64.iso              20-Jan-2022 20:29     71M
from kernel.org.
booted from usb drive shown as sda (internal drive is nvme).

After booting, used alt+ctrl+f2 to get to a command line.
mkdir /b1
mount /dev/sdb1 /b1

That's with a second usb drive containing only one exfat partition in an msdos
style partition table.

CC: (none) => davidwhodgins

Comment 29 katnatek 2022-01-24 23:58:33 CET
(In reply to Dave Hodgins from comment #28)

This is the test i purpose

USB Memory "A" formatted with ventoy , having network installer iso
USB Memory "B" exfat formated, having the classical iso image (or perhaps equivalent tree of files)

With "A" and "B" plugged before power on:
Boot with USB A, in ventoy's menu select the network image iso
When reach to the stage of select the source for installation select "hard drive" see if you can find the classical iso in USB "B"


In more recent versions of ventoy you can reserve some space in the usb memory, i did not make time to make the test but i suppose is possible make the purposed test with just one memory reserving enough space (5GB ?), formatting the reserved space as exfat, put the classical iso in the reserved partition and the network image on the "ventoy" partition
Comment 30 Martin Whitaker 2022-01-26 09:21:29 CET
@Dave, you're in the stage 2 installer by then. What katnatek is trying to do is access an exfat partition during stage 1 (in order to load stage 2 from it).
Comment 31 Thierry Vignaud 2022-01-26 14:48:12 CET
Reading exfat worked when I tested latest changes from https://gitweb.mageia.org/software/drakx/log/mdk-stage1 (it showed listing of partition, complaining that there was no stage2 there but showing the files that did were there)

@katnatek: did you try with latest cauldron boot images?
eg the iso dated from January 20th such as found there:
https://distrib-coffee.ipsl.jussieu.fr/pub/linux/Mageia/distrib/cauldron/x86_64/install/images/
Either:
- Mageia-Cauldron-netinstall-nonfree-x86_64.iso
- Mageia-Cauldron-netinstall-x86_64.iso

If it's still failing, can you provide stage1 logs?
You can just manually mount a USB key & copy /tmp/stage1.log on that USB key and then attach it here.
(just use alt+f2 in order to get a shell for performing that)

Keywords: (none) => NEEDINFO

Comment 32 katnatek 2022-01-26 19:40:26 CET
(In reply to Thierry Vignaud from comment #31)
> @katnatek: did you try with latest cauldron boot images?
> eg the iso dated from January 20th such as found there:
> https://distrib-coffee.ipsl.jussieu.fr/pub/linux/Mageia/distrib/cauldron/
> x86_64/install/images/
> Either:
> - Mageia-Cauldron-netinstall-nonfree-x86_64.iso
> - Mageia-Cauldron-netinstall-x86_64.iso
> 
> If it's still failing, can you provide stage1 logs?
> You can just manually mount a USB key & copy /tmp/stage1.log on that USB key
> and then attach it here.
> (just use alt+f2 in order to get a shell for performing that)

I will do the possible to make the test and report results
Comment 33 katnatek 2022-01-27 19:06:40 CET
I made the test with netinstall-nonfree-i586 for mageia 8 and cualdron

I formatted my 16Gb memory with ventoy reserving 6Gb of space and copy the Mageia 8 i586 Classic installer iso (and both netinstall images) in the ventoy labelled partition and in the reserved partition (after formatting that as exfat) and the results are as i expect for both.

The installer can browse the reserved partition and detect the classic iso inside without having to unplug the memory, but for the ventoy labelled partition you can't see the classic image until unplug and replug the memory, i think this is due the way the boot/load process works.

By example i note that if you boot a live image from the ventoy menu you can't browse the ventoy labelled partition inside the live session.

I need rewrite the Errata, and maybe the title of this bug to reflect this behaviour.
Comment 34 katnatek 2022-03-09 21:17:43 CET
I update the Errata https://wiki.mageia.org/en/Mageia_8_Errata#USB_memory_formatted_with_ventoy

Also change the title and close this Bug as Works For Me

Summary: Network installer can't read exfat partition => On a USB memory formatted with Ventoy, when you boot a netinstall image, you need to unplug and replug for sources in Ventoy's partition
Status: NEW => RESOLVED
Resolution: (none) => WORKSFORME


Note You need to log in before you can comment on or make changes to this bug.