This bug report replaces bug reports 15242 and 15408

Test environment validation:
Installation target 500 GB hard disk with gpt table but no defined partitions.

Objective:
Expect normal UEFI bootable system of M5RC on target disk.

Partitioner dialogue:
Lose all partitions? No
custom -> (normal level) -> Pick sde tab Auto-allocate -> checked sdb and sdc untouched -> Done.

Result:
Normal bootable installation with correct grub menu.

Created attachment 6006 [details]
Test case 1 screen shot 1

Requesting that selected partitioned disk be re-used for new installation.

Created attachment 6007 [details]
Partitioner selected  sdc (appearing blank but actually in use) selected fopartitioning

Created attachment 6008 [details]
Last chance before destroying sdc

Created attachment 6009 [details]
report.bug for test case 1

Attachments 6006 to 6009 relate to test case 1. Photographic images used because F2 inoperative due to aborting installation before file system created. 6009 is report.bug.

Test Case 1:
Test the behaviour of partitioner on fully partitioned installation candidate disk.

Expectation:
User will be warned/asked:
- Disk full. Please abort and remove partition(s) on this disk manually then try again
OR
- Disk Full. Can I repartition this disk before auto-allocating?

Outcome:
From custom partitioning select tab sdg. See attmt 6006.
Clicked Auto-Allocate. Nothing happened. Then checked sdc tab. See 6007. Auto-allocation happened there. I went back to the sdg tab and clicked on Done. Thankfully got warning shown in 6008. Aborted installation, produced the report.bug and powered off to avoid damaging sdc.

Bug description:
Using auto-allocate on a specified target which is fully partitioned, the partitioner finds a different disk to the one specified and allocates there, with no direct warning to the user. The partitioner should not go to any other disks, but restrict itself to the one specified and behave as per the expectation description above.

Created attachment 6010 [details]
TC2-1

Created attachment 6011 [details]
report.bug

Attachments 6010 and 6011 refer to test case 2.

Test case 2:
As test case 1 but added an additional gpt unpartitioned disk to observe which one partitioner will go to as per behaviour in test case 1.

Outcome.
Target installation disk was sdf. Additional 1TB empty disk provided as sdg.
in this case, partitioner chose to allocate on sdg and not sdc as it did in test case 1.  See 6010.  Report.bug in 6011

Bug descrition:
As in test case 1. Not known how partitioner chooses between other free disks to go to. Simply additional information to test case 1

Yes, auto-allocate uses free space anywhere, but only uses free space (which mean there is a valid partition table there and some empty space that can be used) so it doesn't cause any data loss.

Thank you for that clarification Pascal.  My interpretation of the UI was that if I selected a specific tab, only that disk would be involved.  The other issue I had was that sdc looked like a free disk but wasn't (being an Intel firmware raid member with its own partitioning scheme). As mentioned in the first entry above, gnome-disks seems the only partitioner that distinguishes between the two. When bug 11105 is fixed, I expect this will no longer be an issue.

I will therefore restate the bug (any difference between software behaviour and the relevant documentation) as follows.

Clarify the documentation wiki for the partitioner  with a footnote to state:

"auto-allocate uses free space anywhere, but only uses free space (which mean there is a valid partition table there and some empty space that can be used) so it doesn't cause any data loss".

There is actually an even more misleading behaviour, if you ask to wipe a disk, it will do it on current disk but then will call auto-alocate which may install onto a different one if there was space

That was (I think) what happened happened to me previously in testing around this area, though not for this particular bug report. That (I don't have screen shots etc to prove it) one prompted to me to first raise the whole issue. The fact that I fell over it shows that it can happen to someone although rarely.
Would there be much work (not now of course) to make this behaviour less misleading?

Comment 13 is consistent with an email from David Walser ' For the "Erase and user entire disk" option, it could be more clear and say "Erase and use all disks" if you have multiple disks available'
This statement should also be added to the wiki addendum proposed in comment 12 above.
.

(In reply to Vladimir Zawalinski from comment #6)
> Created attachment 6009 [details]
> report.bug for test case 1

There is something weird in the fdisk part, 

it jumps from sdc to sdd1 etc

Disk /dev/sdc: 931.5 GiB, 1000204886016 bytes, 1953525168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/sdd: 111.8 GiB, 120034123776 bytes, 234441648 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 5EBABB25-C258-41EA-80A7-C04B376093DB

Device       Start       End   Sectors   Size Type
/dev/sdd1     2048    616447    614400   300M Windows recovery environment
/dev/sdd2   616448    821247    204800   100M EFI System
/dev/sdd3   821248   1083391    262144   128M Microsoft reserved
/dev/sdd4  1083392 234440703 233357312 111.3G Microsoft basic data

So both any Device "/dev/sdc*" and "Disk /dev/sdd: " are missing.

I'm having a hard time following this, but as Pascal alluded to, the "Auto allocate" button is not specific to any disk, it applies to all of them.  That's why it's separated from the disks screen and is in the bottom-right corner.  The buttons that are specific to the disk you are viewing are in the disk frame in the top-right.

Many have tried in vain to reproduce the issue on non-EFI hardware.

For EFI / gpt:
Olivier Charles tried on real hw, and could not reproduce it
https://ml.mageia.org/l/arc/qa-discuss/2015-03/msg00102.html
James Kerr tried in a VM, and didn't hit it either.
https://ml.mageia.org/l/arc/doc-discuss/2015-03/msg00031.html


@ Vladimir

What is the output of "fdisk -l" in an installed system, is it then correct?
And when using a LiveCD or DVD in live mode?


(In reply to Pascal Terjan from comment #11)
> Yes, auto-allocate uses free space anywhere, but only uses free space (which
> mean there is a valid partition table there and some empty space that can be
> used) so it doesn't cause any data loss.

What is needed to rule out that Vladimir's problem stems from a corrupted partition table?

A thought from the sidelines.  I've reported bugs related to raid installation.  The assumption here is that despite the installer having a corrupted view of the raid disks, it can correctly ignore this and have a valid view of the target disk.  Is this assumption  valid, or does the faulty view of the raid disks cause this behaviour??
Tonyb

CC: (none) => tablackwell

Tony,

With classical  I find that installer initially has a corrupted view of the raid disks ("Message box popup 'The partition .on sdx.. is too .. for me.
Do you want  to lose all partitions?").  Answering no results in the partitioner treating these drives as normal and even partitioning them later (as single drives, not as an array)
if it sees fit. It retains a valid view of the other disks and installing on one of those generally works.

With live, diskdrake-install hits the first raid drive and aborts, so that it is not possible to install from live, but the iso happily boots up into a live
OS up to that point.

Vlad

Created attachment 6027 [details]
fdisk output

(In reply to Vladimir Zawalinski from comment #21)
> Created attachment 6027 [details]
> fdisk output

From the same system while running a LiveCD/DVD in Live mode?

Marja /22
Yes. Running efi-live CD through  VMware player (which is unaware of the md126_ raid array, results in a normal error-free install.  
The error on real hw occurs in a call to blkid query on  sdb with parameters I don't understand.

The fdisk -l output is in attachment 6027 [details] above.

Created attachment 6028 [details]
fdisk -l from M5B31 live

Created attachment 6029 [details]
fdisk -l from a non-mga linux on this hardware

effect not observed

(In reply to Vladimir Zawalinski from comment #25)
> Created attachment 6029 [details]
> fdisk -l from a non-mga linux on this hardware
> 
> effect not observed

Can you please check you attached the correct file, here?

> Can you please check you attached the correct file, here?

Even if it doesn't see a sdf, it has also shows sdd* partitions in sdc

Attachment 6028 [details] is the fdisk output  from MGA5Live. After the disk entry for sdc, five lines lower down is the similar entry for sdd. The list of sdd partitions follows after that.  There is anomalously no line feed before the line for Disk sdd, but ..

Disk /dev/sdd: 111.8 GiB, 120034123776 bytes, 234441648 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 5EBABB25-C258-41EA-80A7-C04B376093DB

Device       Start       End   Sectors   Size Type
/dev/sdd1     2048    616447    614400   300M Windows recovery environment
/dev/sdd2   616448    821247    204800   100M EFI System
/dev/sdd3   821248   1083391    262144   128M Microsoft reserved
/dev/sdd4  1083392 234440703 233357312 111.3G Microsoft basic data

The same is true of 6029.  Am I on the same page?

Ah, you're right, I missed "Disk /dev/sdd" in the attachments and in report.bug because there is no empty line before it :-[

So both Mageia and a non-Mageia distro don't see any partitions for sdc, nor a disk identifier, but only


Disk /dev/sdc: 931.5 GiB, 1000204886016 bytes, 1953525168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

So, same size etc. as sdb (which, btw, has Disklabel type: dos
Disk identifier: 0x00000000 ....)

which might be normal for a set of RAID

It seems that way. gparted falls over but recovers while disks shows one dos partition, while drakx installer complains "The partitioning system is too corrupt for me  ...."  I had to put a gpt table on them, and then define the raid on the m/b firmware. The raid array shows up as md126px in an installed os.

Not sure if I should open a new bug or keep going on this one. I'll stay here because this newly documented problem is fully consistent with the name of this bug report " MGA5 Installer targets wrong disk at partitioning step"

Identify a populated disk on which to install MGA5. In this case, a 500 GB external disk connected through eSATA, that has already been partitioned. Run installer through DVD (connected over usb2). Proceed to first screen of partitioner.
The target disk is sdd. Select this disk in the top right hand corner drop down list. Click "Erase and Use Entire Disk". Message "All data and partitions will be lost on  .. 500GB ..sdd". Sounds good. Click Next. Message appears saying "Error formatting *sdc2*.
Checking sdc2 showed that sdc (a disk that looks empty to the partitioner but is really a raid disk) has been partitioned, while the stipulated target sdd has been cleared but *otherwise untouched*.

I told the partitioner very explicitly and unambiguously to erase and use sdd. It chose to scribble all over sdc (destroying the raid array, but that is bug 11105)..  This is not a feature - it is a BUG.

Comment 13 above from Pascal probably pinpoints the source of this bug, since auto-allocate ought not be called directly in this context, which DOES specifically involve  one and only one particular disk.

I will add the relevant attachments and screen shots later today.
In compliance with Clare's directive to mark all partitioner bugs as release blockers, I will revert the status of this to release blocker.  Either the bug should be fixed before GA release, or the installer errata entry needs to be reinstated.

Is the bug caused by hardware raid or esata?

You need to provide a report.bug Vlad.

When the bug occurs, switch to tty2 while still in the installer, plug in a formatted usb stick and use the 'bug' command to drop report.bug onto the usb stick. You'll probably need to xz it before attaching it to the bug report.

Created attachment 6087 [details]
Report.bug for comment 32

Report.bug taken for issue in comment 32 when a 'blank' disk was partitioned instead of the one chosen.

Claire, report.bug as attachment 6087 [details]. I found nothing unusual on searching same for esata.

For raid, it seems that the array was there to start with but broken after the partitioning. Someone more skilled than I would need to determine involvement of raid with the bug and recommend any further diagnostics.

Here is the some of the partitioning wizard log from report.bug:

* partitionWizard calling solution Erase and use entire disk
* running: udevadm control --stop-exec-queue
* tell kernel add (sdc 1 2048 612353) force_reboot=1 rebootNeeded=
* tell kernel add (sdc 2 616448 105469953) force_reboot=1 rebootNeeded=
* tell kernel add (sdc 3 106088448 8386561) force_reboot=1 rebootNeeded=
* tell kernel add (sdc 4 114477056 1839048112) force_reboot=1 rebootNeeded=
* running: udevadm control --start-exec-queue
* tell kernel force_reboot (sdc), rebootNeeded=
error: /dev/sdd2: No such file or directory
* running: udevadm control --stop-exec-queue
* tell kernel del (sdd 1  ) force_reboot=1 rebootNeeded=e to a previous decision by the 
* tell kernel del (sdd 2  ) force_reboot=1 rebootNeeded=
* tell kernel del (sdd 3  ) force_reboot=1 rebootNeeded=
* tell kernel del (sdd 4  ) force_reboot=1 rebootNeeded=
* running: udevadm control --start-exec-queue
* tell kernel force_reboot (sdd), rebootNeeded=

This states fairly clearly that sdd should have its partitions deleted, as instructed.  Sdc seems to have been added resulting from an earlier conclusion by the wizard (in log but not shown here) that there was not enough space and also not taking into account that sufficient space would be made available by clearing sdd.

I believe this is the essence of the bug. I will add screen shots, but they need to be reduced in size first.

Re Comment 35.
Here are the 4 lines preceding the wizlog extract above.

>>wizlog>>Not enough free space to allocate new partitions: 0 < 1228800
>>wizlog>>There is no FAT partition to resize (or not enough space left)

* solutions found: Use existing partitions, Erase and use entire disk, Custom disk partitioning (all solutions found: Use existing partitions, Erase and use entire disk, Custom disk partitioning)
* solutions: 3
* HERE: Use existing partitions,Erase and use entire disk,Custom disk partitioning
* partitionWizard calling solution Erase and use entire disk

Then continues in Comment 35.

Created attachment 6088 [details]
Shows the available disks prior to choosing partitioning method

Clicking on specific disk list available for partitioning operations

Created attachment 6089 [details]
Choosing sdd

Choosing sdd from the list shown in previous attachment. The instruction is to take sdd as chosen, erase it and install Mageia 5 on this disk. I can't read this any other way.

Created attachment 6090 [details]
Warning that sdd will be erased

sdd .. as expected and intended

Created attachment 6091 [details]
sdc ?? - partitioner was told to erase AND USE sdd

Created attachment 6092 [details]
sdd - erased but not used .....

Created attachment 6093 [details]
sdc has been partitioned instead of sdd

There are two flavours to this problem:

(1) After erasing the nominated disk, the logic is clearly wrong, because it involves a call to 'auto-allocate' (see comment 13 above) which does not seem to know about the additional space on sdd, and does not know  that it should limit itself to one disk (comments 11, 13, 17). It can't anyway, since the decision to add sdc is taken before sdd is erased. This has to be cleaned up sooner or later. Other distros also have an analogous function to 'use free space' but give the user more control over what to do next.

(2) Whether sdc is a genuinely empty disk, or an Intel raid array member that looks like an empty disk is, in my opinion/assumption, irrelevant. Because in this case it is a raid disk, the damage caused is greater than it would be on an empty disk.  Resolution of bug 11105 should fix this, since sdc will, one hopes, be then correctly recognised for what it is.

Summarising then, hardware raid support is missing/broken currently, that's a known issue.

The side-effect of the known issue is deletion of part of the raid array, which is obviously very bad.

We should at least prevent this from occurring, show an error/force custom partitioning/etc, even if proper support for the hardware raid is to be added in a later release.

CC: (none) => thierry.vignaud, tmb

I have just tested this behaviour in VirtualBox, by creating one virtual disk of 8GB and another of 10GB (sda and sdb resp.), both initially unpartitioned.

First time, 'use free space' was offered and predictably that put the efi and / partitions on sda and the swap partition on sdb.

Second time, again starting from unpartitioned virtual disks, 'auto-allocate' was used, which partitioned the two disks exactly as above.

Third time, sda was left as it was from the previous attempt, but the swap partition was removed to make sdb free and unpartitioned. This time, 'erase and use entire disk' was offered for sda. I specified sda in the top right-hand corner of the frame and chose 'erase and use entire disk'. Partitioning outcome was as before - Efi and / on sda and swap on sdb.

Fourth time - deleted sdb. This time 'erase and use entire disk' erased the disk and partitioned the space there appropriately.

There is absolutely no suggestion of raid in any of this testing, before or after, nor does VBox cater for pre-existing firmware raid as far as I know. I hope this shows that 'erase and use entire disk' in fact uses other blank disks as well, contrary to the stated functional rule that "The buttons that are specific to the disk you are viewing are in the disk frame in the top-right". I was unable to get a report.bug for this due to problems with VBox recognising usb sticks on my setup.

Is this enough to revert the title of this bug to its original and more general form? This bug has nothing to do with the nature of the disk it attacks, just that it is free at the time.

Essence of this bug is that 'use-free space/auto-allocate' should not be called in its present form if the user chooses a particular disk and requests 'erase and use entire disk'.

There has been many bugfixes since DrakX v16.67 (we're currently at DrakX v16.75).
Could try again with boot.iso against an up to date mirror[1]

Can you attach report.bug with latest drakx?

[1] install/stage2/VERSION must contains "16.75", eg:
http://distrib-coffee.ipsl.jussieu.fr/pub/linux/Mageia//distrib/cauldron/x86_64/install/stage2/VERSION

Keywords: (none) => NEEDINFO

I am using version from cauldron/dorsync mga5rc_64b DVD. 
Sun Mar 29 15:06:35 CEST 2015
* second stage install running (DrakX v16.74)  from report.bug.
Not sure how or where to access  more recent drakx nor how to merge it into a test install?

Boot with "boot-nonfree.iso" from eg http://distrib-coffee.ipsl.jussieu.fr/pub/linux/Mageia//distrib/cauldron/x86_64/install/images/
When you see the error, please:
- plug a USB key
- go to tty2 (alt+ctrl+F2)
- run the "bug" command
- attach the report.bug file you will find on your USB key here.
  (but please compress it first)

quoting Vlad "Can probably be closed, as the destructive effects described there were not observed in these tests." - using last isos with last stage2

Closing now. Feel free to reopen  if needed

Status: NEW => RESOLVED
Resolution: (none) => FIXED