Bug 16055 - using auto-allocate, installer tries to put partition(s) on the USB-key which contains the iso
Summary: using auto-allocate, installer tries to put partition(s) on the USB-key which...
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: Installer (show other bugs)
Version: Cauldron
Hardware: All Linux
Priority: release_blocker critical
Target Milestone: ---
Assignee: Mageia Bug Squad
QA Contact:
URL:
Whiteboard: IN_ERRATA
Keywords:
: 15482 (view as bug list)
Depends on:
Blocks:
 
Reported: 2015-05-29 11:12 CEST by Marja Van Waes
Modified: 2017-02-25 15:16 CET (History)
10 users (show)

See Also:
Source RPM: drakx-installer-stage2 diskdrake
CVE:
Status comment:


Attachments
Journals for my last comment (439.70 KB, application/octet-stream)
2015-05-29 12:30 CEST, Vladimir Zawalinski
Details
Untested patch (4.54 KB, patch)
2015-06-02 00:20 CEST, Pascal Terjan
Details | Diff
sdb and sdc missing (96.07 KB, image/png)
2015-06-02 11:47 CEST, Marja Van Waes
Details
report.bug.xz from the install with pterjan's first patch (141.32 KB, application/x-xz)
2015-06-02 11:50 CEST, Marja Van Waes
Details
home was placed on the 2nd disk (97.90 KB, image/png)
2015-06-02 12:48 CEST, Marja Van Waes
Details
Updated patch (4.55 KB, patch)
2015-06-03 23:15 CEST, Pascal Terjan
Details | Diff
report.bug.xz of the last install, mentioned above (135.28 KB, application/x-xz)
2015-06-04 15:02 CEST, Marja Van Waes
Details

Description Marja Van Waes 2015-05-29 11:12:49 CEST
When using auto-allocate after wiping some partitions in custom mode, installer doesn't continue but returns to the partitioning step with a message that formatting sdb5 failed

report.bug.xz can be found in attachment 6661 [details] (which was uploaded for another bug report)


* formatting device sdb5 (type swap)
* running: mkswap /dev/sdb5
/dev/sdb5: Device or resource busy
* step "formatPartitions" took: 0:00:06
* step "formatPartitions" failed with error: swap formatting of sdb5 failed at /usr/lib/libDrakX/fs/format.pm line 191.

* error: swap formatting of sdb5 failed at /usr/lib/libDrakX/fs/format.pm line 191.


In "* fdisk before:", only sdb1 existed.

On the QA ml, vlad confirmed having hit this bug, too.
Marja Van Waes 2015-05-29 11:13:23 CEST

Summary: using auto-allocate, installer tries to put swap on the USB-key from which contains the iso => using auto-allocate, installer tries to put swap on the USB-key which contains the iso

Comment 1 Pascal Terjan 2015-05-29 11:22:45 CEST
Indeed:(

* partitionWizard calling solution Custom disk partitioning
* lang:en_US charset:C font:DejaVu Sans 12 consolefont:lat0-16
matchbox-wm: X error warning (0x201618): 140 (opcode: 138)
matchbox-wm: X error warning (0x201bed): 140 (opcode: 138)
* win parts: sda1
matchbox-wm: X error warning (0x2029c2): 140 (opcode: 138)
* partition::dos::write sda
* partition::dos::write sda
* partition::dos::write sda
* partition::dos::write sda
* partition::dos::write sda
* running: udevadm control --stop-exec-queue
* tell kernel del (sda 5  ) force_reboot= rebootNeeded=
* tell kernel del (sda 6  ) force_reboot= rebootNeeded=
* tell kernel del (sda 7  ) force_reboot= rebootNeeded=
* tell kernel del (sda 8  ) force_reboot= rebootNeeded=
* tell kernel del (sda 9  ) force_reboot= rebootNeeded=
* tell kernel del (sda 10  ) force_reboot= rebootNeeded=
* tell kernel add (sda 5 24905728 24521552) force_reboot= rebootNeeded=
* tell kernel add (sda 6 49430528 2914912) force_reboot= rebootNeeded=
* tell kernel add (sda 7 52348928 25791232) force_reboot= rebootNeeded=
* tell kernel add (sda 8 18841600 829520) force_reboot= rebootNeeded=
* tell kernel add (sda 9 19675136 5227504) force_reboot= rebootNeeded=
* tell kernel del (sda 8  ) force_reboot= rebootNeeded=
* tell kernel del (sda 9  ) force_reboot= rebootNeeded=
* tell kernel add (sda 8 19675136 5227504) force_reboot= rebootNeeded=
* tell kernel del (sda 8  ) force_reboot= rebootNeeded=
* tell kernel add (sda 5 12462080 12440560) force_reboot=1 rebootNeeded=
* tell kernel del (sda 5  ) force_reboot=1 rebootNeeded=
* tell kernel del (sda 6  ) force_reboot=1 rebootNeeded=
* tell kernel del (sda 7  ) force_reboot=1 rebootNeeded=
* tell kernel add (sda 6 24905728 24521552) force_reboot=1 rebootNeeded=
* tell kernel add (sda 7 49430528 2914912) force_reboot=1 rebootNeeded=
* tell kernel add (sda 8 52348928 25791232) force_reboot=1 rebootNeeded=
* running: udevadm control --start-exec-queue
* tell kernel force_reboot (sda), rebootNeeded=
* running: udevadm settle
matchbox-wm: X error warning (0x202bf7): 140 (opcode: 138)
* partition::dos::write sdb
* partition::dos::write sdb
* running: udevadm control --stop-exec-queue
* tell kernel add (sdb 5 2516992 5117952) force_reboot= rebootNeeded=
* running: udevadm control --start-exec-queue
* running: udevadm settle
* fdisk after

Disk /dev/sda: 37.3 GiB, 40007761920 bytes, 78140160 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xcccdcccd

Device     Boot    Start      End  Sectors  Size Id Type
/dev/sda1  *        2048 12458879 12456832    6G  b W95 FAT32
/dev/sda2       12462030 78140159 65678130 31.3G  5 Extended
/dev/sda5       12462080 24902639 12440560    6G 83 Linux
/dev/sda6       24905728 49427279 24521552 11.7G 83 Linux
/dev/sda7       49430528 52345439  2914912  1.4G 82 Linux swap / Solaris
/dev/sda8       52348928 78140159 25791232 12.3G 83 Linux

Disk /dev/sdb: 3.7 GiB, 3909091328 bytes, 7634944 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x7e272654

Device     Boot   Start     End Sectors  Size Id Type
/dev/sdb1  *          1 2514943 2514943  1.2G 17 Hidden HPFS/NTFS
/dev/sdb2       2516960 7634943 5117984  2.5G  5 Extended
/dev/sdb5       2516992 7634943 5117952  2.5G 82 Linux swap / Solaris

Disk /dev/loop0: 56.3 MiB, 59015168 bytes, 115264 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

* step "doPartitionDisks" took: 0:00:42
* step `doPartitionDisks' finished
Comment 2 Marja Van Waes 2015-05-29 12:03:38 CEST
also confirmed by lebarhon, on QA-discuss ml:

On 29/05/15 11:30, lebarhon wrote:> 
> Same thing here.
> DVD Classical x86_64 on UEFI system. I had 55GB free on sda, the only
> one disk. Auto allocate created a 55 GB /home partition on sda and /
> (6,2  GB), swap (1,2  GB) and a second ESP (4 MB  !) on sdb.
> I could get out of there with More/reload
> Lebarhon

Hardware: i586 => All

Marja Van Waes 2015-05-29 12:05:47 CEST

Summary: using auto-allocate, installer tries to put swap on the USB-key which contains the iso => using auto-allocate, installer tries to put partition(s) on the USB-key which contains the iso

Comment 3 Pascal Terjan 2015-05-29 12:17:55 CEST
I believe auto allocate uses all free space across disks, it should probably ignore some :(
Comment 4 Vladimir Zawalinski 2015-05-29 12:27:11 CEST
(In reply to Pascal Terjan from comment #3)
> I believe auto allocate uses all free space across disks, it should probably
> ignore some :(

I totally agree. Caused problems before by attacking raid disks.

CC: (none) => vzawalin1

Comment 5 Vladimir Zawalinski 2015-05-29 12:28:13 CEST
I hit this again, this time by first installing korora, changing usb keys and booting mga5 live-gnome then trying to install on the same slow disk that korora was put on.
I couldn't call 'bug' at that point, but here is the juicy bit from the journals:

May 29 19:07:22 localhost mageia-draklive-install.desktop[5277]: /dev/sdf3 is already mounted on /run/media/live/3329c2ea-25eb-4b41-ba4b-2955409d4af0
May 29 19:07:34 localhost draklive-install[5286]: partitioning wizard log:
                                                  >>wizlog>>There is no FAT partition to resize (or not enough space left)
May 29 19:07:34 localhost draklive-install[5286]: solutions found: Use existing partitions, Use free space, Erase and use entire disk, Custom disk partitioning (all solutions found: Use existing partitions, Use free space, Erase and use entire disk, Custom disk partitioning)
May 29 19:07:34 localhost draklive-install[5286]: solutions: 4
May 29 19:07:34 localhost draklive-install[5286]: HERE: Use existing partitions,Use free space,Erase and use entire disk,Custom disk partitioning
May 29 19:07:44 localhost draklive-install[5286]: partitionWizard calling solution Custom disk partitioning
May 29 19:07:44 localhost draklive-install[5286]: win parts: sda1,sdd4,mapper/isw_cijgfgdgbd_Volume1p1,mapper/isw_cijgfgdgbd_Volume1p4
May 29 19:07:51 localhost systemd-timesyncd[927]: interval/delta/delay/jitter/drift 512s/-0.001s/0.162s/0.009s/-14ppm
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: Error: Partition(s) 1 on /dev/sdf have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use.  As a result, the old partition(s) will remain in use.  You should reboot now before making further changes.
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: failed to del partition #1 on /dev/sdf at /usr/lib/libDrakX/partition_table/gpt.pm line 182.
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: diskdrake::hd_gtk::try_() called from /usr/lib/libDrakX/diskdrake/hd_gtk.pm:137
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: diskdrake::hd_gtk::try() called from /usr/lib/libDrakX/diskdrake/hd_gtk.pm:248
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: diskdrake::hd_gtk::__ANON__() called from /usr/lib/perl5/vendor_perl/5.20.1/Gtk3.pm:341
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: (eval)() called from /usr/lib/perl5/vendor_perl/5.20.1/Gtk3.pm:341
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: Gtk3::main() called from /usr/lib/perl5/vendor_perl/5.20.1/Gtk3.pm:294
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: Gtk3::__ANON__() called from /usr/lib/libDrakX/mygtk3.pm:1521
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: mygtk3::main() called from /usr/lib/libDrakX/ugtk3.pm:859
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: ugtk3::main() called from /usr/lib/libDrakX/diskdrake/hd_gtk.pm:131
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: diskdrake::hd_gtk::main() called from /usr/lib/libDrakX/fs/partitioning_wizard.pm:60
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: fs::partitioning_wizard::partition_with_diskdrake() called from /usr/lib/libDrakX/fs/partitioning_wizard.pm:274
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: fs::partitioning_wizard::__ANON__() called from /usr/lib/libDrakX/fs/partitioning_wizard.pm:613
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: (eval)() called from /usr/lib/libDrakX/fs/partitioning_wizard.pm:613
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: fs::partitioning_wizard::main() called from /usr/sbin/draklive-install:171
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: main::ask_partitions() called from /usr/sbin/draklive-install:162
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: (eval)() called from /usr/sbin/draklive-install:162
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: main::ask_partitions_loop() called from /usr/sbin/draklive-install:65
May 29 19:08:45 localhost mageia-draklive-install.desktop[5277]: main::install_live() called from /usr/sbin/draklive-install:42
May 29 19:08:45 localhost draklive-install[5286]: error: failed to del partition #1 on /dev/sdf at /usr/lib/libDrakX/partition_table/gpt.pm line 182.
May 29 19:09:11 localhost systemd[1]: Failed to reset devices.list on /system.slice: Invalid argument
May 29 19:09:18 localhost acpid[1952]: client 3425[0:0] has disconnected

Note that the installer did *not* show sdf3 as mounted.

As per my ml, when I powered off the computer and the target disk, rebooted into gnome-live, then powered on the target disk and finally started the installer, all proceeded and ended normally. Full journal attachment to follow.
Comment 6 Vladimir Zawalinski 2015-05-29 12:30:17 CEST
Created attachment 6664 [details]
Journals for my last comment
Comment 7 Thomas Backlund 2015-05-29 13:22:09 CEST
(In reply to Pascal Terjan from comment #3)
> I believe auto allocate uses all free space across disks, it should probably
> ignore some :(

Well, since installer should know wich drive we run from (/dev/sdX), why not simply ignore anything on that one during auto-allocation (that way those actually for some reason want to override it can do so with manual partitioning)
Comment 8 Thierry Vignaud 2015-05-29 13:26:45 CEST
(In reply to Pascal Terjan from comment #3)
> I believe auto allocate uses all free space across disks, it should probably
> ignore some :(

"use free space" option does too (see bug #15482)
Comment 9 Pascal Terjan 2015-05-29 13:38:31 CEST
Yes that's exactly the same code(In reply to Thierry Vignaud from comment #8)
> (In reply to Pascal Terjan from comment #3)
> > I believe auto allocate uses all free space across disks, it should probably
> > ignore some :(
> 
> "use free space" option does too (see bug #15482)

Yes that's exactly the same code
Comment 10 Thierry Vignaud 2015-05-29 13:42:46 CEST
*** Bug 15482 has been marked as a duplicate of this bug. ***

CC: (none) => zen25000

Marja Van Waes 2015-05-31 22:29:41 CEST

Whiteboard: (none) => IN_ERRATA

Comment 11 Pascal Terjan 2015-06-01 22:08:21 CEST
diskdrake has some logic to prefer the current disk (I believe the partitioning wizard does not):

    $all_hds_{hds} = [ sort { $a == $hd ? -1 : 1 } fs::get::hds($all_hds) ];
    eval { fsedit::auto_allocate(\%all_hds_, $suggestions) };

But this does not guarantee all the partitions will be in there.

This is used like this:

sub allocatePartitions {
    my ($all_hds, $to_add) = @_;

    my @to_add = @$to_add;

    foreach my $part_ (fs::get::holes($all_hds, 'non_readonly')) {
        my ($start, $size, $dev) = @$part_{"start", "size", "rootDevice"};
        my ($part, $suggested);
        while ($suggested = suggest_part($part = { start => $start, size => 0, maxsize => $size, rootDevice => $dev },
                                         $all_hds, \@to_add)) {
            my $hd = fs::get::part2hd($part, $all_hds);
            add($hd, $part, $all_hds, { primaryOrExtended => $part->{primaryOrExtended} });
            $size -= $part->{size} + $part->{start} - $start;
            $start = $part->{start} + $part->{size};
            @to_add = grep { $_ != $suggested } @to_add;
        }
    }
}

So it will try to place the partitions in the holes of this disk first (the loop is actually a bit strange as we loop through other holes and call suggest_part even if there is nothing left to add).

I would be in favour of restricting to the current disk, either at both calling points or to add an optional parameter and filter here.
Comment 12 Pascal Terjan 2015-06-01 23:59:35 CEST
Some notes:
- In text install there is no current disk from free_space so it will still be across disk. For wipe_disk we chose one so we can allocate on that one.
- Current disk can be either a real disk or some LVM (raid is commented and would break in kind2hd).
- This will be $o_target in partitionWizardSolutions (not defined in text mode)
Comment 13 Marja Van Waes 2015-06-02 00:14:22 CEST
@ pterjan

Thanks a lot for your work on fixing this bug!


(In reply to Pascal Terjan from comment #12)
> Some notes:
> - In text install there is no current disk from free_space so it will still
> be across disk.

Is it possible to exclude any USB-key to which the installer.iso or boot.iso was dumped?

> For wipe_disk we chose one so we can allocate on that one.
> - Current disk can be either a real disk or some LVM (raid is commented and
> would break in kind2hd).
> - This will be $o_target in partitionWizardSolutions (not defined in text
> mode)
Comment 14 Pascal Terjan 2015-06-02 00:20:15 CEST
Created attachment 6685 [details]
Untested patch
Comment 15 Pascal Terjan 2015-06-02 00:24:35 CEST
(In reply to Marja van Waes from comment #13)
> Is it possible to exclude any USB-key to which the installer.iso or boot.iso
> was dumped?

Not easily

If my patch is doing what I believe it is, the only remaining cases will be:
- "use free space" in text mode (wiping the disk will be fine as you get to select the disk).
- auto install without a partitioning defined
Comment 16 Dave Hodgins 2015-06-02 00:42:04 CEST
Note that the iso could be on the hard drive being used for the install, so just
blocking the drive with the iso, which could be an external drive connected via usb, would just cause other problems. I'm not sure how the problem should be
solved.
Comment 17 Dave Hodgins 2015-06-02 00:47:35 CEST
The only solution I can think of, is if the iso is on a drive being selected
for any of the file systems, is to have a new dialog asking the user if the
selections are ok, and let them exclude a drive, and then redo the auto
allocate without that drive.
Comment 18 Dave Hodgins 2015-06-02 00:49:07 CEST
Also, too late for Mageia 5. I think we should release with the current version.
Comment 19 Marja Van Waes 2015-06-02 00:57:57 CEST
(In reply to Pascal Terjan from comment #15)
> (In reply to Marja van Waes from comment #13)
> > Is it possible to exclude any USB-key to which the installer.iso or boot.iso
> > was dumped?
> 
> Not easily
> 
> If my patch is doing what I believe it is, the only remaining cases will be:
> - "use free space" in text mode (wiping the disk will be fine as you get to
> select the disk).
> - auto install without a partitioning defined

That's both corner cases, and those doing text installs will now how to workaround the issue. So a great improvement.

I'll test your patches tomorrow,  I tried just now, but am too tired

Do I (for a boot.iso install) only need a mdkinst.sqfs from a patched drakx-installer-stage2, or also need to have a patched drakxtools on my local mirror?
Comment 20 Marja Van Waes 2015-06-02 01:01:09 CEST
(In reply to Dave Hodgins from comment #18)
> Also, too late for Mageia 5. I think we should release with the current
> version.

thx for your earlier comments :-)

I assume here "current version" is the version with pterjan's patch ;-)
Comment 21 Pascal Terjan 2015-06-02 01:12:45 CEST
mdkinst.sqfs is enough thank you
I'll also test it tomorrow evening
Comment 22 Dave Hodgins 2015-06-02 02:14:26 CEST
(In reply to Marja van Waes from comment #20)
> (In reply to Dave Hodgins from comment #18)
> > Also, too late for Mageia 5. I think we should release with the current
> > version.
> thx for your earlier comments :-)

> I assume here "current version" is the version with pterjan's patch ;-)

As the fix has not been tested yet, I'm inclined to go ahead without it,
with warnings in the errata, release notes, and on the download page.
Comment 23 claire robinson 2015-06-02 09:32:16 CEST
This is the last remaining blocker AFAICT, let's fix it please.
Comment 24 Marja Van Waes 2015-06-02 10:21:19 CEST
about to test patched mdkinst.sqfs 

if it succeeds, then I'll upload it somewhere for others to test
Comment 25 Marja Van Waes 2015-06-02 11:16:47 CEST
mdksinst.sqfs (only 64bit) + matching VERSION can be found here:
http://waesvanm.home.xs4all.nl/Packaging/bug16055/x86_64/

The first test seemed OK, but I'm not sure the empty space on the USB key was really seen at all (maybe dumping a boot.iso on a USB key isn't the same atm, for remaining space, as dumping a traditional iso?)

The boot.iso USB key was seen as:
----------------------------------------------------------------
Disk /dev/sdc: 7.2 GiB, 7759462400 bytes, 15155200 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x26b0dccf

Device     Boot Start    End Sectors Size Id Type
/dev/sdc1  *        0 110591  110592  54M  0 Empty
/dev/sdc2         252   8443    8192   4M ef EFI (FAT-12/16/32)
-------------------------------------------------------------------

However, the traditional iso dumped to USB (from the install I filed this bug for) had
/dev/sdb1  *        1 2514943 2514943  1.2G 17 Hidden HPFS/NTFS

I'll test again with an empty USB key attached, to make sure installer finds some empty space on a different disk
Comment 26 Marja Van Waes 2015-06-02 11:47:26 CEST
Created attachment 6686 [details]
sdb and sdc missing

the patch seems to need a bit of patching.

in the diskdrake screen, not only the "boot.iso"-USB-key isn't seen, but the empty USB-key isn't shown, either, only sda and sdd (with stage2) are.
Comment 27 Pascal Terjan 2015-06-02 11:49:26 CEST
Hmm this is very strange, I don't think the patch touches that area.
Comment 28 Marja Van Waes 2015-06-02 11:50:27 CEST
Created attachment 6687 [details]
report.bug.xz from the install with pterjan's first patch

I'll do a third test, after first creating some empty space on the disk that contains stage2, then creating empty space on sda and then asking installer to auto-allocate
Comment 29 Marja Van Waes 2015-06-02 12:10:56 CEST
(In reply to Marja van Waes from comment #28)

> 
> I'll do a third test, after first creating some empty space on the disk that
> contains stage2, then creating empty space on sda and then asking installer
> to auto-allocate

That'll take a while, resizing a 2TB disk takes much longer than I'd expected.... I suppose diskdrake doesn't only look at the end of the disk when resizing, but at every block?
Comment 30 Marja Van Waes 2015-06-02 12:11:59 CEST
s/2TB disk/2TB partition/
Comment 31 Thierry Vignaud 2015-06-02 12:15:26 CEST
diskdrake does nothing (but for FAT). For NTFS or extX, the relevant program get called and the more blocks were allocated at end of partition, the more work it has to do in order to remap them in free holes at beginning of partition...
Comment 32 Marja Van Waes 2015-06-02 12:48:24 CEST
Created attachment 6688 [details]
home was placed on the 2nd disk

unfortunately, after using diskdrake to delete some partitions on sda and still seeing the sda partitions, letting diskdrake auto-allocate it puts /home on the free space of the diks that contains stage2.

My ddebug.log is useless, I stopped install right after the screenshot was made, but should have continued and only stopped after "fdisk after" was created.

I'll redo and get a good ddebug.log when asked
Comment 33 Marja Van Waes 2015-06-02 12:52:46 CEST
(In reply to Thierry Vignaud from comment #31)
> diskdrake does nothing (but for FAT). For NTFS or extX, the relevant program
> get called 

parted?

> and the more blocks were allocated at end of partition, the more
> work it has to do in order to remap them in free holes at beginning of
> partition...

thx for the explanation :-)
Comment 34 Marja Van Waes 2015-06-02 13:15:48 CEST
(In reply to Pascal Terjan from comment #27)
> Hmm this is very strange, I don't think the patch touches that area.

Let's hope I made a mistake.
Comment 35 Marja Van Waes 2015-06-03 09:41:27 CEST
(In reply to Marja van Waes from comment #2)
> also confirmed by lebarhon, on QA-discuss ml:
> 
> On 29/05/15 11:30, lebarhon wrote:> 
> > Same thing here.
> > DVD Classical x86_64 on UEFI system. I had 55GB free on sda, the only
> > one disk. Auto allocate created a 55 GB /home partition on sda and /
> > (6,2  GB), swap (1,2  GB) and a second ESP (4 MB  !) on sdb.
> > I could get out of there with More/reload
> > Lebarhon

So things have become worse for EFI-installs, too, since the change that made it possible for EFI-installs, to dump the iso on a USB-key instead of copying its contents to a FAT32 formatted USB-key.

I had considered proposing to have two sets of isos, the old type for legacy installs, and the new type for EFI, but lebarhon's comment shows that then the EFI-installs would still be bitten worse by this bug than before (when occasionally only the ESP could be placed on the installer USB key) :-(

@ tmb

There is no way, while keeping the possibility to _dump_ isos for EFI-installs to USB, to make the isos expand the room they need on the USB-keys they're dumped to, so that there's no free space left? /o\

CC'ing Martin, in case he can, and would like to, help.

pterjan didn't say that the patch - see attachment 6685 [details] - worked fine for him, so either he didn't find time to test (and it would be nice to have another tester), or it did indeed fail.

CC: (none) => mageia

Comment 36 Anne Nicolas 2015-06-03 18:08:42 CEST
Pascal, Thierry, Martin, any idea/proposal/comment ? This is the last bug that prevents us to finalize Mageia 5 release. It would be great to end all this :)
Comment 37 Pascal Terjan 2015-06-03 18:32:07 CEST
I couldn't test yesterday, I'll be home late tonight but will try a few test cases.
Comment 38 Martin Whitaker 2015-06-03 19:40:13 CEST
I'll take a look and see if I can help.
Comment 39 Anne Nicolas 2015-06-03 20:40:53 CEST
yes please. Any help is welcome
Comment 40 Martin Whitaker 2015-06-03 21:48:26 CEST
This line in Pascal's patch seems to be the problem:

+       next if $o_hd && ($o_hd->{device} || $_->{VG_name}) != $dev;

I think this is probably meant to be:

+       next if ($o_hd && $o_hd->{device} || $_->{VG_name}) != $dev;

but I've not yet fully understood the purpose of $_->{VG_name} (which is something to do with LVMs), so I may have got this wrong.

Making that change does fix the problem in the test case I've tried (auto allocate using the "server" option).
Comment 41 Pascal Terjan 2015-06-03 22:58:58 CEST
Parenthesis are correct but $_ should be $o_hd but this only breaks auto partitioning inside LVM.

If this is a real disk {device} will be set, if it's LVM it will be {VG_name}
Comment 42 Pascal Terjan 2015-06-03 23:05:01 CEST
Meaning of that line is:

If a specific disk is given ($o_hd is set) and it's device ($o_hd->{device} || $_->{VG_name}) is not the same as the one underlying current empty space ($dev) don't use this empty space (next).
Comment 43 Martin Whitaker 2015-06-03 23:05:51 CEST
Doesn't it need more parentheses?

+       next if $o_hd && (($o_hd->{device} || $o_hd->{VG_name}) != $dev);
Comment 44 Pascal Terjan 2015-06-03 23:13:06 CEST
I had initialy put them, then removed them, but you may be right :)
I'll add them to be safe and clear.
Comment 45 Pascal Terjan 2015-06-03 23:15:03 CEST
Created attachment 6697 [details]
Updated patch

Attachment 6685 is obsolete: 0 => 1

Comment 46 Martin Whitaker 2015-06-03 23:36:28 CEST
Actually not needed, but do make it more clear. What is needed though is ne in place of !=
Comment 47 Pascal Terjan 2015-06-03 23:47:28 CEST
Indeed :(
Comment 48 Pascal Terjan 2015-06-03 23:48:12 CEST
Tested current version of the patch: it doesn't work.
testing again with ne which should help.
Comment 49 Martin Whitaker 2015-06-04 00:02:56 CEST
(In reply to Pascal Terjan from comment #47)
> Indeed :(

It would help if it wasn't the opposite way round in shell scripts :-(

Works for me with ne.
Comment 50 Pascal Terjan 2015-06-04 00:57:07 CEST
Testing some additional cases, things lokk good so far.

Some empty space on first disk.
Selecting second disk and asking to delete all partitions -> without the patch the empty space on first disk was used, now it no longer is.

Some empty space on both disk,
Selecting second disk and asking to use free space -> without the patch the empty space on first disk was used, now it no longer is.

I haven't tried with some LVM and need to go to sleep. I'll commit current patch.
Comment 51 Mageia Robot 2015-06-04 01:00:31 CEST
commit 765b1e22ea39a0fb32000e77a6c7f0fe116cd9ed
Author: Pascal Terjan <pterjan@...>
Date:   Mon Jun 1 22:18:58 2015 +0000

    only use current disk when auto partitioning, wiping disk, using free space (mga#16055)
---
 Commit Link:
   http://gitweb.mageia.org/software/drakx/commit/?id=765b1e22ea39a0fb32000e77a6c7f0fe116cd9ed
Comment 52 Pascal Terjan 2015-06-04 01:01:52 CEST
Need to be tested:
- auto allocate in custom partitioning
- LVM
Comment 53 Martin Whitaker 2015-06-04 09:51:13 CEST
I notice when testing auto allocate in custom partitioning that it allocates a new swap partition even when there is an existing one, but when you go on to installation it uses both old and new partitions.

Surprisingly, my old installation also automatically detects and uses the new swap partition, even though it isn't listed in /etc/fstab.
Comment 54 Thierry Vignaud 2015-06-04 10:01:29 CEST
All your swaps belong to systemd :-)
Comment 55 Pascal Terjan 2015-06-04 10:03:12 CEST
Yes this is true, if you don't chose "use existing partitions" it will igonre all existing ones and create everything including a new swap.
Then all swap partitions available get used.
Comment 56 Marja Van Waes 2015-06-04 12:03:08 CEST
tested with
 * second stage install running (DrakX v16.103)

I had empty space on the disk that contained Stage2

Selected custom
wiped 3 partitions
clicked auto-allocate

installer put / and swap on the current disk

the empty space on the disk with Stage2 was not touched :-)
Comment 57 Marja Van Waes 2015-06-04 14:57:55 CEST
another test

* with today's official boot-nonfree.iso 
(to see whether diskdrake in installer would now show the empty space on the USB key it was dumped to, instead of when using a custom made boot.iso)

* also, an additional USB key with ±7GB of free space was added.

Again, diskdrake in installer refused to show the (empty space on the) USB-key with boot-nonfree.iso (which is absolutely fine with me, but I would never have created this bug if that had been the case with the classical iso that made me report this bug)
(Diskdrake in an installed system shows it)

+ This time, only / was created on the selected disk
+ Nothing was used of the free space on the other devices.

I'm wondering whether installer now starts off by considering to use all empty space accross disk, then sees that isn't allowed and therefore puts:

* / 
on the selected disk when 2 other disks with free space are shown in diskdrake

* / and swap 
on the same amount of free space  when 1 other disk with free space is shown

* / and /home and swap 
when no other disk with free space is shown

If the above assumption is true, then what will happen if there is 50GB of empty space on the selected disk, and a much smaller amount of free space on another disk? Will then only /home be created and install fail?
Comment 58 Marja Van Waes 2015-06-04 15:02:07 CEST
Created attachment 6699 [details]
report.bug.xz of the last install, mentioned above

attaching in case that is useful at all

Attachment 6687 is obsolete: 0 => 1

Comment 59 Pascal Terjan 2015-06-04 15:52:57 CEST
That's a good point, we probably want to count empty space only on the current disk :/
Comment 60 Marja Van Waes 2015-06-04 18:21:49 CEST
(In reply to Marja van Waes from comment #57)

> 
> If the above assumption is true, then what will happen if there is 50GB of
> empty space on the selected disk, and a much smaller amount of free space on
> another disk? Will then only /home be created and install fail?

I did _not_ manage to let that happen :-D

* I've tried to use auto-allocate on a disk with varying amounts of free space (± 48GB - ± 105GB), while another disk had ± 25 GB free space (sometimes combined with ± 14 GB free space on again another disk.

=> Every time / + /home + swap were created on that disk :-)

* In another test, using auto-allocate on that disk with ± 25 GB free space, when there was 48-105GB free space on another disk:
=> only / was created (so no swap, even if there was no swap on the selected disk)

(However, there was a swap partition on one of the other disks)
Comment 61 Pascal Terjan 2015-06-04 18:40:26 CEST
Partitions are created in order so it will first create /.
The main consequence would be to have only / or / and /home and no swap.
Comment 62 Marja Van Waes 2015-06-04 18:48:25 CEST
(In reply to Pascal Terjan from comment #61)
> Partitions are created in order so it will first create /.
> The main consequence would be to have only / or / and /home and no swap.

Good to know / will always be created :-)

(Btw, here, if not all three were created, it was / or / + swap, but never / + /home without swap)

To me, a chance that no swap is created with auto-allocate or use free windows space, isn't a release blocking issue, but YMMV
Comment 63 Martin Whitaker 2015-06-05 00:49:09 CEST
(In reply to Marja van Waes from comment #57)
> Again, diskdrake in installer refused to show the (empty space on the)
> USB-key with boot-nonfree.iso (which is absolutely fine with me, but I would
> never have created this bug if that had been the case with the classical iso
> that made me report this bug)

For me, diskdrake does show the empty space on the USB key. I copied boot-nonfree.iso using dd, and booted in UEFI mode. If I select the USB device in the custom partitioning screen, auto-allocate correctly creates partitions on that device (and ignores free space on my hard disk).

I've also tested what happens if you hibernate an installed system and then run the installer. I'm pleased to report that it then doesn't use the swap partition that contains the hibernated image.
Comment 64 Thierry Vignaud 2015-06-05 08:36:11 CEST
BTW I think Errata must be cleaned now...
Comment 65 Thierry Vignaud 2015-06-05 08:37:40 CEST
Closing

Status: NEW => RESOLVED
Resolution: (none) => FIXED

Comment 66 Marja Van Waes 2015-06-05 08:42:14 CEST
(In reply to Thierry Vignaud from comment #64)
> BTW I think Errata must be cleaned now...

done
Comment 67 Mageia Robot 2017-02-12 16:01:26 CET
commit 02ba0f311a04e3fe93985bb75b6e7e99c6315b73
Author: Martin Whitaker <mageia@...>
Date:   Sat Feb 4 23:46:14 2017 +0000

    Fix auto-allocation of BIOS boot partitions (mga#20161, mga#19888).
    
    This adds a specific subroutine, fsedit::auto_allocate_boot_bios_parts
    that detects if a BIOS boot partition is needed and allocates it if so.
    This allows us to relax the rules in fs::any::is_boot_bios_part_needed
    to allow the user to manually allocate the BIOS boot partition on a
    different device if they so wish.
    
    In the normal case that installation is confined to a single disk,
    this will allocate a single BIOS boot partition on that disk. In
    the rare case that installation is spread over multiple disks, it
    will allocate a BIOS boot partition on every disk. Given that the
    BIOS boot partitions are very small and that this is not a normal
    use case (see mga#16055), this seems an acceptable quirk - and does
    allow the user to then choose any disk when installing the boot
    loader.
---
 Commit Link:
   http://gitweb.mageia.org/software/drakx/commit/?id=02ba0f311a04e3fe93985bb75b6e7e99c6315b73

 Bug links:
   Mageia
      https://bugs.mageia.org/16055
      https://bugs.mageia.org/20161
      https://bugs.mageia.org/19888
Comment 68 Mageia Robot 2017-02-25 15:16:06 CET
commit daf8670c08e54ac64407f7f209aaf9b460abdcc6
Author: Martin Whitaker <mageia@...>
Date:   Sat Feb 4 23:46:14 2017 +0000

    Fix auto-allocation of BIOS boot partitions (mga#20161, mga#19888).
    
    This adds a specific subroutine, fsedit::auto_allocate_boot_bios_parts
    that detects if a BIOS boot partition is needed and allocates it if so.
    This allows us to relax the rules in fs::any::is_boot_bios_part_needed
    to allow the user to manually allocate the BIOS boot partition on a
    different device if they so wish.
    
    In the normal case that installation is confined to a single disk,
    this will allocate a single BIOS boot partition on that disk. In
    the rare case that installation is spread over multiple disks, it
    will allocate a BIOS boot partition on every disk. Given that the
    BIOS boot partitions are very small and that this is not a normal
    use case (see mga#16055), this seems an acceptable quirk - and does
    allow the user to then choose any disk when installing the boot
    loader.
---
 Commit Link:
   http://gitweb.mageia.org/software/drakx/commit/?id=daf8670c08e54ac64407f7f209aaf9b460abdcc6

 Bug links:
   Mageia
      https://bugs.mageia.org/16055
      https://bugs.mageia.org/20161
      https://bugs.mageia.org/19888

Note You need to log in before you can comment on or make changes to this bug.