Bug 15302 - Wrong wiki instructions cause Classic M5b3 USB installation on real x64 EFI hardware to fail
Summary: Wrong wiki instructions cause Classic M5b3 USB installation on real x64 EFI h...
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: Installer (show other bugs)
Version: 4
Hardware: x86_64 Linux
Priority: release_blocker critical
Target Milestone: ---
Assignee: Documentation Team
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-02-16 11:46 CET by André DESMOTTES
Modified: 2015-02-25 12:37 CET (History)
4 users (show)

See Also:
Source RPM:
CVE:
Status comment:


Attachments
SC of gparted (48.46 KB, image/png)
2015-02-16 20:28 CET, André DESMOTTES
Details
SC of Partition/information (49.73 KB, image/png)
2015-02-16 20:29 CET, André DESMOTTES
Details

Description André DESMOTTES 2015-02-16 11:46:53 CET
Description of problem:
The USB stick is not detected and there is an error message "No partition found"

Version-Release number of selected component (if applicable):
Mageia-5-beta3-x86_64-DVD dated feb 12

How reproducible:
each time

Steps to Reproduce:
1.download Mageia-5-beta3-x86_64-DVD.iso and mount it with: 
mount -t iso9660 /run/media/samc/iso/Mageia-5-beta3-x86_64-DVD/Mageia-5-beta3-x86_64-DVD.iso /mnt/disk/

2.Prepare a USB stick: mkdosfs -F 32 -I -n Classical /dev/sdb

3.copy the ISO contents:cp -a /mnt/disk/* /run/media/samc/Classical/

4. plug the USB stick in the PC and boot. This PC has a GPT disk with already 2 OS installed in EFI mode (Ubuntu and Mageia5B3 Live) and a free partition created with Gparted. The PC has no DVD drive.

5. in the menu, choose Start Mageia 5 (Cauldron) Install 

6. The PC displays "Detecting USB Devices...", then "Detecting USB mass storage Devices...", then a message box "Please select the disk containing the copy..."
sda (ATA maxtor...)
sdb (Sandisk Cruzor...)
Choose sdb

7. error message "No partition found" 


Reproducible: 

Steps to Reproduce:
Comment 1 Marja Van Waes 2015-02-16 12:22:48 CET
Actually, I thought the iso had to be loopback mounted to be able to copy its contents, but I might be very wrong about that.

Does the USB-key indeed contain the contents of the iso, do you see directories and files on it?

CC: (none) => marja11

Comment 2 André DESMOTTES 2015-02-16 12:58:18 CET
In the USB key, I can see the folders:
dosutils, EFI, isolinux and x86_64

and the files:
autorun.inf and boot.catalog
Comment 3 Marja Van Waes 2015-02-16 17:05:54 CET
Just copied the contents of the classical DVD to a USB-stick, using your method.

The result is the same: I can see those directories and files on it, but I do also get the error about no partition being found when I try to boot from it.

Apart from that, Diskdrake does not see the USB-key, even if Dolphin had no problem at all seeing it.

GParted does see it, but gives a Warning:

plain floppy:device "/dev/sdc"
busy (Resource temporarily
unavailable):
Cannot initialize '::'
mlable: Cannot initialize drive

It shows "loop" as Partition table for the USB-key, should it really be that?
Comment 4 Marja Van Waes 2015-02-16 17:36:21 CET
I'm trying again, now: 

* FAT32 formatted the USB-key with Isodumper, and 
* copying the files with "cp -r", instead of "cp -a"

I also used "mount -o loop", but don't think that makes a difference, because I got the same "mount: /dev/loop0 is write-protected, mounting read-only" message as when I used "mount -t iso9660"

Before I started copying, gparted showed no errors on the USB-key, and the Partition table was shown as msdos, and it is still shown as msdos now that the files are being copied.

I'll report back tonight if this works better.

(if it does, then it is probably the "cp -a" that was wrong here)
Comment 5 Marja Van Waes 2015-02-16 18:33:27 CET
That method worked fine, so our wiki instructions seem to be wrong.

(Install went well, the only issue was that I had to remove the boot medium from /etc/fstab when I was finished, but that is a known bug.)

I assume it is the "cp -a" that was wrong, but am not 100% sure.

André, could you try doing everything exactly the same as you did before, but using "cp -r" instead of "cp -a"?

Assignee: bugsquad => doc-bugs
Summary: Classic M5b3 USB installation on real x64 EFI hardware failed => Wrong wiki instructions cause Classic M5b3 USB installation on real x64 EFI hardware to fail

Comment 6 André DESMOTTES 2015-02-16 18:58:33 CET
I retried the install with the commands:
mount -o loop /run/media/samc/iso/Mageia-5-beta3-x86_64-DVD/Mageia-5-beta3-x86_64-DVD.iso /mnt/disk/
and
cp -r /mnt/disk/* /run/media/samc/Classical/
(copy was already in progress when I read your last message)
The result is exactly the same "No partition found"
The mobo is a micro ATX Gigabyte H81M S2PV I bought especially to help writing the installer documentation with EFI and I can't use it ! :(
Comment 7 Marja Van Waes 2015-02-16 20:03:06 CET
what does GParted say about the Partition label of the USB key?
Comment 8 Marja Van Waes 2015-02-16 20:10:32 CET
s/label/table/ of course :-/
Comment 9 André DESMOTTES 2015-02-16 20:28:36 CET
Created attachment 5923 [details]
SC of gparted

Gparted see no label, whereas Dolphin sees it.
Comment 10 André DESMOTTES 2015-02-16 20:29:25 CET
Created attachment 5924 [details]
SC of Partition/information
Comment 11 Marja Van Waes 2015-02-16 20:47:47 CET
That doesn't look good.

Did you reformat the USB-key, before retrying the install?
It looks exactly like it looked here, when the partition table on the USB-key was "loop" instead of "msdos"
Comment 12 Marja Van Waes 2015-02-16 20:50:38 CET
oops, hadn't seen attachment 5923 [details] .... indeed, the partition table says "loop"
Comment 13 Marja Van Waes 2015-02-16 21:33:21 CET
Ady2 commented on IRC on this bug:

2015:02:16:21:12 < Ady2> https://bugs.mageia.org/show_bug.cgi?id=15302 I am not sure to which exact wiki page the reporter is referring to, but his original step ' mkdosfs -F 32 -I -n Classical /dev/sdb ' builds a "superfloppy", unpartitioned USB drive (there is no partition number in that "sdb").

I wasn't aware of that. So it was probably using isodumper to FAT32-format the USB stick that solved the problem for me.
Comment 14 papoteur 2015-02-16 22:18:37 CET
The source of the instruction is here:
https://wiki.mageia.org/en/Installation_Media#Removing_the_Live_ISO_Files_from_USB_Media
I am surprised that this command doesn't work. I used it before the format function of Isodumper (which is nice, isn't it? ;) )

CC: (none) => yves.brungard_mageia

Comment 15 Marja Van Waes 2015-02-16 22:37:46 CET
@ Yves

Well, if you used it for isodumper, then it does work: Every time I formatted using isodumper and then copied the files, the USB-stick was OK (apart from unrelated bugs that tmb already fixed)

But we still have to find out what is wrong with the full set of instructions in the description of this bug

I thought André had used the instructions here, but they are slightly different:
https://wiki.mageia.org/en/UEFI_how-to#Preparation_of_USB_stick
Comment 16 Marja Van Waes 2015-02-16 22:45:43 CET
(In reply to Marja van Waes from comment #13)
> Ady2 commented on IRC on this bug:
> 
> 2015:02:16:21:12 < Ady2> https://bugs.mageia.org/show_bug.cgi?id=15302 I am
> not sure to which exact wiki page the reporter is referring to, but his
> original step ' mkdosfs -F 32 -I -n Classical /dev/sdb ' builds a
> "superfloppy", unpartitioned USB drive (there is no partition number in that
> "sdb").

(In reply to papoteur from comment #14)
> The source of the instruction is here:
> https://wiki.mageia.org/en/
> Installation_Media#Removing_the_Live_ISO_Files_from_USB_Media
> I am surprised that this command doesn't work. I used it before the format
> function of Isodumper (which is nice, isn't it? ;) )

Well, Isodumper does work well: I just let it FAT32-partition another USB-key, and it shows a nice /dev/sdc1 partition
Comment 17 Marja Van Waes 2015-02-16 22:52:02 CET
(In reply to Marja van Waes from comment #16)
> (In reply to Marja van Waes from comment #13)
> > Ady2 commented on IRC on this bug:
> > 
> > 2015:02:16:21:12 < Ady2> https://bugs.mageia.org/show_bug.cgi?id=15302 I am
> > not sure to which exact wiki page the reporter is referring to, but his
> > original step ' mkdosfs -F 32 -I -n Classical /dev/sdb ' builds a
> > "superfloppy", unpartitioned USB drive (there is no partition number in that
> > "sdb").
> 
> (In reply to papoteur from comment #14)
> > The source of the instruction is here:
> > https://wiki.mageia.org/en/
> > Installation_Media#Removing_the_Live_ISO_Files_from_USB_Media
> > I am surprised that this command doesn't work. I used it before the format
> > function of Isodumper (which is nice, isn't it? ;) )
> 
> Well, Isodumper does work well: I just let it FAT32-partition another
> USB-key, and it shows a nice /dev/sdc1 partition

Isodumper must use a different command: running "mkdosfs -F 32 -I -n Classical /dev/sdb" does indeed give a loop partition file again, no partition on /dev/sdb, etc: the USB-key is unusable
Comment 18 Marja Van Waes 2015-02-16 22:59:49 CET
not loop partition file!
I mean loop partition _table_
Comment 19 André DESMOTTES 2015-02-16 23:13:52 CET
(In reply to Marja van Waes from comment #15)
> 
> I thought André had used the instructions here, but they are slightly
> different:
> https://wiki.mageia.org/en/UEFI_how-to#Preparation_of_USB_stick

Yes, indeed, don't know any other !
I tried several times on qa-discuss to know if this page was reliable, but I never got an answer :(
You are talking about isodumper, but I though it was a copy tool, like dd, irrelevant to "copy the contents".
Comment 20 Marja Van Waes 2015-02-17 09:03:32 CET
(In reply to André DESMOTTES from comment #19)

> I tried several times on qa-discuss to know if this page was reliable, but I
> never got an answer :(

ouch!

> You are talking about isodumper, but I though it was a copy tool, like dd,
> irrelevant to "copy the contents".

It is, and you cannot yet use it to copy iso-contents, but it does already have a nice option to format a USB-key in a format you select (it can give the desired label to the key at the same time)..... I use it all the time to FAT32 format my USB-keys.

Just select your USB-key with "Device to work on", click on "Format the device", select "Fat 32 (Windows)" and adjust the label, if desired, and click "Execute"
Comment 21 Marja Van Waes 2015-02-17 09:47:00 CET
It seems using the -I switch was wrong, it is not used in isodumper:
-------------------------------------------------------------------------
# Format partition according to the fstype specified
        if fstype == "fat32":
            os.system("mkdosfs -F 32 -n \"%s\" %s >/dev/null 2>&1" % (volume_label, partition.path))
-------------------------------------------------------------------------


And, according to the man page of mkdosfs:

 -I  It  is  typical  for  fixed  disk devices to be partitioned so, by default, you are not permitted to create a filesystem across the entire device.

So, IIUC,  -I seems good when going to dump an iso on a USB key, but not when going to copy the contents of an iso to it
Comment 22 André DESMOTTES 2015-02-17 10:56:38 CET
I re-did a try with:
-Formatting the USB stick with Isodumper, it imposed capital letters for the label.
- Copy with the command: cp -r /mnt/disk/* /run/media/samc/CLASSICAL. -a was verbose, -r is not.

And now it boots ! but in the partition screen, after choosing "Custom" I can see that /boot/EFI isn't automatically mounted. (not very important).
Thx a lot.
Comment 23 Marja Van Waes 2015-02-17 11:07:07 CET
(In reply to André DESMOTTES from comment #22)
> I re-did a try with:
> -Formatting the USB stick with Isodumper, it imposed capital letters for the
> label.


Had forgotten to say, that when I followed the method from the description, I had used "CLASSICAL" as label instead of "Classical"

So that (alone) is not enough to fix the problem.

> - Copy with the command: cp -r /mnt/disk/* /run/media/samc/CLASSICAL. -a was
> verbose, -r is not.

true, and I was wrong to think that "-a" caused the problem, so probably better to stick with "-a", or use "-rv"

> 
> And now it boots ! but in the partition screen, after choosing "Custom" I
> can see that /boot/EFI isn't automatically mounted. (not very important).

Yeah, I was wrong before (in another bug report) to think it gets automatically mounted, that is only true for updates.

Do we know enough now to update https://wiki.mageia.org/en/UEFI_how-to#Preparation_of_USB_stick or do we need to first test 
"mkdosfs -F 32 -n Classical /dev/sdb" ?

(I'll be away for some hours, but feel free to already update the wiki)
Comment 24 Marja Van Waes 2015-02-17 11:08:26 CET
s/updates/upgrade installs/
Comment 25 André DESMOTTES 2015-02-17 12:10:36 CET
> Do we know enough now to update
> https://wiki.mageia.org/en/UEFI_how-to#Preparation_of_USB_stick or do we
> need to first test 
> "mkdosfs -F 32 -n Classical /dev/sdb" ?
> 

[root@localhost ~]# mkdosfs -F 32 -n Classical /dev/sdb
mkfs.fat 3.0.27 (2014-11-12)
mkfs.fat: warning - lowercase labels might not work properly with DOS or Windows
mkdosfs: Device partition expected, not making filesystem on entire device '/dev/sdb' (use -I to override)
[root@localhost ~]# 

[root@localhost ~]# mkdosfs -F 32 -n CLASSICAL /dev/sdb
mkfs.fat 3.0.27 (2014-11-12)
mkdosfs: Device partition expected, not making filesystem on entire device '/dev/sdb' (use -I to override)
[root@localhost ~]# 

How does isodumper manage that without -I?
Comment 26 Marja Van Waes 2015-02-17 13:18:06 CET
(In reply to André DESMOTTES from comment #25)

> 
> How does isodumper manage that without -I?

I don't know.... :-(

Maybe it is better to replace the CLI command in the wiki with:

"Install the isodumper package and start it
Select your USB-key with "Device to work on"
Click on "Format the device"
Select "Fat 32 (Windows)" and adjust the label, if desired
click "Execute"

because, even if we don't know why it works: it does work well!
Comment 27 Marja Van Waes 2015-02-17 14:12:17 CET
(In reply to Marja van Waes from comment #26)
> (In reply to André DESMOTTES from comment #25)
> 
> > 
> > How does isodumper manage that without -I?
> 

It does some magic with parted first, including creating a msdos partition table, see line 70 and further, here
http://gitweb.mageia.org/software/isodumper/tree/lib/raw_format.py
Comment 28 André DESMOTTES 2015-02-17 16:23:14 CET
(In reply to Marja van Waes from comment #26)

> Maybe it is better to replace the CLI command in the wiki with:
> 
> "Install the isodumper package and start it
> Select your USB-key with "Device to work on"
> Click on "Format the device"
> Select "Fat 32 (Windows)" and adjust the label, if desired
> click "Execute"
> 
> because, even if we don't know why it works: it does work well!

Done
Comment 29 Marja Van Waes 2015-02-17 17:41:30 CET
(In reply to André DESMOTTES from comment #28)
> Done

Thanks, Yves, it looks very nice and is very clear :-)

Closing as fixed

Status: NEW => RESOLVED
Resolution: (none) => FIXED

Comment 30 Marja Van Waes 2015-02-17 17:42:22 CET
(In reply to Marja van Waes from comment #29)
> (In reply to André DESMOTTES from comment #28)
> > Done
> 
> Thanks, Yves, it looks very nice and is very clear :-)
> 
> Closing as fixed

s/Yves/André/

Sorry, don't know where my brain is atm :-[
Comment 31 Marja Van Waes 2015-02-23 22:07:56 CET
Reopening, because Mageia 4 users cannot use Isodumper to create a msdos file table and one FAT32 partition on the key

Our Mageia 4 instructions in the wiki are wrong, but I do not know which parted command needs to be added to make the instructions correct

Priority: Normal => release_blocker
Status: RESOLVED => REOPENED
Version: Cauldron => 4
Resolution: FIXED => (none)

Comment 32 Marja Van Waes 2015-02-23 22:13:45 CET
(In reply to Marja van Waes from comment #31)
> Reopening, because Mageia 4 users cannot use Isodumper to create a msdos
> file table and one FAT32 partition on the key
> 
> Our Mageia 4 instructions in the wiki are wrong, but I do not know (yet) which
> parted command needs to be added to make the instructions correct

CC'ing tmb, who will probably know parted better than me

@ tmb

the wiki instructions for Mageia 4 can make one end up with a fat32 formatted USB key, that is unusable, invisible for diskdrake, and shows an error with gparted.

it doesn't have a msdos partition table, but shows "loop"
it doesn't have a sdb1 (or sdc1, sdd1, etc) partition, either, but is only sdb, sdc, or sdd etc.
Comment 33 Marja Van Waes 2015-02-23 22:14:36 CET
now really CC'ing

@ tmb
please read comment 32

CC: (none) => tmb

Comment 34 Marja Van Waes 2015-02-24 00:26:34 CET
According to "info parted"

$ parted -s /dev/sdX -- mklabel msdos \
              mkpart primary fat32 64s 4MiB \
              mkpart primary fat32 4MiB -1s

(almost empty partition at the beginning, because of the advised 4MiB alignment for cheap flash drives)
("mklabel" defines the partition table, so here the msdos we need)

and then "info parted" suggests something like
$ mkfs.vfat /dev/sdX2

But let's first try the wiki instruction, appending "2" for the partition to it, instead of only defining the drive

#mkdosfs -F 32 -I -n MGALIVE /dev/sdf2

but now it is bed time
Comment 35 Marja Van Waes 2015-02-24 08:49:22 CET
(In reply to Marja van Waes from comment #34)
> According to "info parted"
> 
> $ parted -s /dev/sdX -- mklabel msdos \
>               mkpart primary fat32 64s 4MiB \
>               mkpart primary fat32 4MiB -1s
> 

Tried that:
[root@DenkBlok3Cauldron5a1 marja]# parted -s /dev/sdc -- mklabel msdos mkpart primary fat32 64s 4MiB mkpart primary fat32 4MiB -1s
Warning: The resulting partition is not properly aligned for best performance.
[root@DenkBlok3Cauldron5a1 marja]#

:-(

Why isn't it properly aligned, or is this warning only for the fake sdb1 partition?

Anyway:
[root@DenkBlok3Cauldron5a1 marja]# parted /dev/sdc -s print
Model:   (scsi)
Disk /dev/sdc: 7812MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start   End     Size    Type     File system  Flags
 1      32,8kB  4194kB  4162kB  primary               lba
 2      4194kB  7812MB  7808MB  primary               lba

[root@DenkBlok3Cauldron5a1 marja]#
Comment 36 Marja Van Waes 2015-02-24 08:57:01 CET
The next command seems completely OK:

[root@DenkBlok3Cauldron5a1 marja]# mkdosfs -F 32 -I -n CLASSIC /dev/sdc2
mkfs.fat 3.0.27 (2014-11-12)
[root@DenkBlok3Cauldron5a1 marja]# parted /dev/sdc -s print
Model:   (scsi)
Disk /dev/sdc: 7812MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start   End     Size    Type     File system  Flags
 1      32,8kB  4194kB  4162kB  primary               lba
 2      4194kB  7812MB  7808MB  primary  fat32        lba

[root@DenkBlok3Cauldron5a1 marja]#

Now going to write an iso to it, to see whether the fake sdc1 partition will keep the iso from booting. Will report back two hours from now.
Comment 37 Marja Van Waes 2015-02-24 10:49:39 CET
(In reply to Marja van Waes from comment #36)

> Now going to write an iso to it, to see whether the fake sdc1 partition will
> keep the iso from booting. Will report back two hours from now.

Well, for a correct test, I shouldn't have forgotten the "*" in
cp -a /mnt/Classical/* /run/media/marja/CLASSICAL/   :-(

Anyway, why aren't we telling users to use GParted. That's what I used before isodumper could do the trick:

1. Install gparted
2. insert the USB key you want to write the iso to
3. run dmesg to see whether it's sdb, sdc, sdd etc.
4. start gparted as root
5. select the correct device in the drop down list in the upper right of the gparted screen
6. Click on "Device" (in the menu bar) - Create Partition Table
7. Select new partition table type: msdos 
8. Click on "Apply"
9. Click on "Partition" - "New"
(In cauldron, the defaults are "Create as: Primary Partition" and "Align to: MiB", if that's different in Mga4, then we should tell the users to set it to those values)
10. For cheap flash drives: Change "Free space preceding (MiB):" to "4"
11. Change "File system:" to "fat32"
12. Insert "MGALIVE" after "Label:" for a Live iso
13. Click on "+ Add"
14. Click on the green "V" in the menu bar to apply the chosen operations
Comment 38 André DESMOTTES 2015-02-24 15:31:19 CET
I tried the Comment 37 from Marja. It works for an installation, but not for an update because sda1 isn't mounted on /boot/EFI. It was with MA5B3, I redo it with RC.
Comment 39 Martin Whitaker 2015-02-24 21:01:16 CET
Sorry I didn't see this before, as I could have saved you some time! (I did report this in bug 14438, but I guess it got lost in all the other issues). 

In case it helps, a command line alternative to Marja's solution is

  parted -s /dev/sdX mklabel msdos mkpart primary fat32 0% 100%
  mkfs.vfat -n MGALIVE /dev/sdX1

(replacing sdX with the appropriate device).

CC: (none) => mageia

Comment 40 Marja Van Waes 2015-02-24 22:38:29 CET

(In reply to André DESMOTTES from comment #38)
> I tried the Comment 37 from Marja. It works for an installation, but not for
> an update because sda1 isn't mounted on /boot/EFI. It was with MA5B3, I redo
> it with RC.

If you hit the same problem with RC, then please file a separate bug report. 

For upgrades, installer should see the EFI partition and mount it on /boot/EFI again, like it was before. How the USB-key was prepared and how the iso was written to it, can't make a difference. This sounds like a nasty bug.

Please also attach the compressed /root/drakx/report.bug(.xz) from the RC upgrade.



(In reply to Martin Whitaker from comment #39)
> Sorry I didn't see this before, as I could have saved you some time! (I did
> report this in bug 14438, but I guess it got lost in all the other issues).

Yeah, that report was already a bit messy. It is better to file separate issues in separate reports ;-)
 
> 
> In case it helps, a command line alternative to Marja's solution is
> 
>   parted -s /dev/sdX mklabel msdos mkpart primary fat32 0% 100%
>   mkfs.vfat -n MGALIVE /dev/sdX1
> 
> (replacing sdX with the appropriate device).

Thanks, Martin, I updated the UEFI how-to and replaced the wrong command with your commands.

I also added the Gparted method (I have the impression that keeping 4MiB free space before the partition makes my USB keys work faster, over twice as fast)

Closing again as fixed

Status: REOPENED => RESOLVED
Resolution: (none) => FIXED

Comment 41 Martin Whitaker 2015-02-25 00:24:11 CET
(In reply to Marja van Waes from comment #40)
> I also added the Gparted method (I have the impression that keeping 4MiB
> free space before the partition makes my USB keys work faster, over twice as
> fast)

You can do that with

  parted -s /dev/sdX mklabel msdos mkpart primary fat32 4MiB 100%

(doesn't help my USB keys, though :-( )
Comment 42 Marja Van Waes 2015-02-25 12:37:26 CET
(In reply to Martin Whitaker from comment #41)
> (In reply to Marja van Waes from comment #40)
> > I also added the Gparted method (I have the impression that keeping 4MiB
> > free space before the partition makes my USB keys work faster, over twice as
> > fast)
> 
> You can do that with
> 
>   parted -s /dev/sdX mklabel msdos mkpart primary fat32 4MiB 100%

Thanks!

> 
> (doesn't help my USB keys, though :-( )

Well, I bet none of your USB-keys ever took almost 90 minutes to have a traditional iso copied to it ;-)

I tend to be a bit too Dutch and buy the cheapest USB-keys around. You'll buy better ones, that won't need 4MiB free space.

And, even if I tested with the very same iso, I only tested with an identical USB key, same brand, type and batch as the one that took 1½ hour.... that only counts as "impression" not as proof ;-)

Note You need to log in before you can comment on or make changes to this bug.