Bug 27872 - Live Installation can't be completed on the system with low (512 Mb) RAM size
Summary: Live Installation can't be completed on the system with low (512 Mb) RAM size
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: Installer (show other bugs)
Version: 8
Hardware: All Linux
Priority: Normal normal
Target Milestone: Mageia 8
Assignee: Mageia tools maintainers
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-12-19 08:51 CET by Ihar Areshchankau
Modified: 2021-02-16 14:10 CET (History)
5 users (show)

See Also:
Source RPM: draklive-install-2.29-1.mga8
CVE:
Status comment:


Attachments
Patch for initrd.img/sbin/mgalive-root (556 bytes, text/plain)
2020-12-30 10:26 CET, Ihar Areshchankau
Details
Patched file /sbin/mgalive-root (6.13 KB, application/x-shellscript)
2020-12-31 08:42 CET, Ihar Areshchankau
Details

Description Ihar Areshchankau 2020-12-19 08:51:20 CET
Description of problem:

The installation can't be completed if it is started using the initial boot menu "Install Mageia Live" on the system that has low (512 Mb) RAM size (while the live system boots up and installation launched from the desktop shortcut "Install on Hard Disk" completes successfully). This installation completes successfully on the system that has 768 Mb RAM.

Version-Release number of selected component (if applicable):

Mageia-8-beta2-Live-Xfce-i586.iso

How reproducible:

Every time I attempt to install the Mageia8.

Steps to Reproduce:
1. Create a virtual machine (VM) and reduce its RAM size to 512 Mb.
2. Boot up the VM using Mageia-8-beta2-Live-Xfce-i586.iso DVD.
3. Choose "Install Mageia Live" from the bootloader menu.
4. Choose default options for language, timezone, etc.
5. Wait while the packages copy process will be terminated with messages:
a) "The package lightdm needs to be installed. Do you want to install it? [Ok]"
b) "No package named lightdm [Ok]"
c) "Could not install the lightdm package [Ok]"
Comment 1 Dave Hodgins 2020-12-19 11:28:26 CET
Confirmed. The package lightdm is being requested but is not present in the
xfce iso. In addition it later crashed as per bug 27873.

CC: (none) => davidwhodgins

Comment 2 Martin Whitaker 2020-12-19 13:54:39 CET
When persistence is not enabled, the live system uses a tmpfs filesystem to provide the writable overlay for the root filesystem. Even though draklive-install temporarily uses the swap partition on the installation target, it appears the tmpfs filesystem is still limited to half the physical RAM size. The tmpfs filesystem is running out of space when removing unwanted hardware support and locale packages, and I suspect this is corrupting the RPM database.

Skipping the removal of unwanted packages avoids the problem. You can remove the unwanted packages after installation by using the remove-unused-packages tool.

Alternatively, if using a USB stick, add a persistence partition, which is then used for the writable overlay instead of a tmpfs filesystem. You could also add a swap partition on the USB stick.

I don't know why erasing packages uses so much disk space. It appears to be temporary usage only - after that step completes the usage drops to a more reasonable level.

lightdm is certainly present on the ISO. Try 'rpm -q lightdm'.

CC: (none) => mageia

Comment 3 Dave Hodgins 2020-12-19 17:48:45 CET
Martin, read the Description again. It works in live mode, but failed when
selecting the installer from the boot menu.

I was wrong about it not being on the iso, but it is not being found when the
installer is running.

I've just tested again, this time unselecting the options to remove unneeded
hardware and localization support. It works. So presumably the package is
being removed by one of those two. I'll try again with the removal to see
if I can then find a log confirming that lightdm is being removed before
trying to install the bootloader.
Comment 4 Martin Whitaker 2020-12-19 18:10:14 CET
Dave, I read the description and followed the steps to reproduce the fault.

When you run the installer from the boot menu, you are still running the live system. The only difference is that it runs draklive-install using the matchbox window manager instead of using the main DE.

draklive-install does not (in general) install packages from RPM files. It simply copies the live root filesystem directly to the target root filesystem. The message about lightdm not being installed comes when it reconfigures the display manager (to disable auto-login). As lightdm is installed in the filesystem it's just copied, that indicates the problem is in the RPM database.

When running the install from the boot menu, use Ctrl-Alt-F3 to switch to a tty console, where you can log in as root. Whilst the installer is removing packages, switch to the console and use df to see what's happening to the overlay filesystem. You can also use journalctl to inspect the system log, where you should find some error messages.
Comment 5 Dave Hodgins 2020-12-19 19:32:33 CET
I understand now.

The tmpfs file system is limited to 50% of physical ram detected. That's
the file system that is running out of space when it's trying to remove the
unused packages, well before it starts trying to copy the live system to the
destination drive.

I think the best solution is to increase the ram specified in the
hardware requirements. I'll ask atellier to increase the ram to 768MB in the
hardware requirements at https://www.mageia.org/en/support/
Comment 6 Dave Hodgins 2020-12-19 19:43:03 CET
Assigning to atelier team. Please modify https://www.mageia.org/en/support/

Replace ...
Memory (RAM): 512MB minimum, 2GB recommended;

with ...

Memory (RAM): 768MB minimum for live iso installs, 512MB minimum for classical iso installs, 2GB recommended for either;

Assignee: bugsquad => atelier-bugs

Comment 7 Morgan Leijström 2020-12-19 20:34:34 CET
Per comment 0 & 2: Only when installing from a *the boot menu* of a Live *without persistence (nor swap)* it require 768 MB. For Live 512 should be OK both when installing from desktop icon, or booted up with persistence or swap.

(BTW impressive 512MB is enough for booted system plus installing)

So trying to boil it down:

Now:
"Memory (RAM): 512MB minimum, 2GB recommended;"

Correct but long:
"Memory (RAM): 512MB minimum, 2GB recommended; (Exception: installing from the boot menu of a Live ISO need 768MB unless it have persistence or swap partition, or you skip the step to remove unneeded packages. Installing from booted Live is OK with 512MB)"

Suggestion:
"Memory (RAM): 512MB minimum, 2GB recommended; (Installing from Live ISO boot menu need 768MB)

CC: (none) => fri

Comment 8 Morgan Leijström 2020-12-19 20:40:56 CET
Ah wait now i see the misplaced https://bugs.mageia.org/show_bug.cgi?id=27864#c20
"  If you run draklive-install from the command line in Xfce, you'll see it's running out of space there too. I think it's just chance whether or not it breaks the install in a visible way. "

And persistence is utilised also when running install from the boot menu?

So basically we can boil it down to that Live need 768 MB unless persistence is enabled?


Suggestion:
"Memory (RAM): 512MB minimum, 2GB recommended, (768MB Installing from Live ISO without persistence)"
Comment 9 Dave Hodgins 2020-12-19 21:18:58 CET
I think it's safer not to complicate it, and stick with

Memory (RAM): 768MB minimum for live iso installs, 512MB minimum for classical iso installs, 2GB recommended for either;

I doubt there are many real systems in use with that little ram now, so it's
really only for virtual system settings.
Comment 10 Ihar Areshchankau 2020-12-30 10:26:25 CET
Created attachment 12165 [details]
Patch for initrd.img/sbin/mgalive-root
Comment 11 Ihar Areshchankau 2020-12-30 10:30:14 CET
As Dave said, the trouble is in 50% default limit of physical RAM for tmpfs. But this limit can be changed using option "size=" when the filesystem mount (https://www.kernel.org/doc/html/latest/filesystems/tmpfs.html).

The root filesystem is mounted by the script MAGEIA.ISO/boot/initrd.img/sbin/mgalive-root. All that is needed is that use option "size=384M" for low-RAM computers. This can be done using the code:
...
    TRAM=`grep -E 'MemTotal:\s+[0-9]+\skB' /proc/meminfo | grep -oE '[0-9]+' -`
    if [ -z "`grep -o '\binstall\b' /proc/cmdline`" -o -z "$TRAM" -o $TRAM -ge 786432 ]; then
        mount -n -t tmpfs -o mode=755 none /live/overlay
    else
        mount -n -t tmpfs -o mode=755,size=384M none /live/overlay
    fi
    unset TRAM
...
I tested this solution on some virtual PCs and it seems to be working. The file mgalive-root.diff is attached for the script patching.
Comment 12 Ihar Areshchankau 2020-12-31 08:26:35 CET
Comment on attachment 12165 [details]
Patch for initrd.img/sbin/mgalive-root

140c140,148
<     mount -n -t tmpfs -o mode=755 none /live/overlay
---
>     TRAM=`grep -E 'MemTotal:\s+[0-9]+\skB' /proc/meminfo | grep -oE '[0-9]+' -`
>     if [ -z "`grep -o '\binstall\b' /proc/cmdline`" -o -z "$TRAM" -o $TRAM -ge 786432 ]; then
>         mount -n -t tmpfs -o mode=755 none /live/overlay
>     else
>         # Bug 27872 solution (Out of memory on low-RAM (<768 Mb) computers
>         # wheh installing from the boot menu of a Live image).
>         mount -n -t tmpfs -o mode=755,size=384M none /live/overlay
>     fi
>     unset TRAM
Comment 13 Ihar Areshchankau 2020-12-31 08:36:48 CET
Comment on attachment 12165 [details]
Patch for initrd.img/sbin/mgalive-root

This patch is "reversed" (mistakenly) and must be applied to the file sbin/mgalive-root using -R option in the patch command:

patch -R mgalive-root mgalive-root.diff

Sorry for the inconvenience.
Filip Komar 2020-12-31 08:37:57 CET

Assignee: atelier-bugs => bugsquad
CC: (none) => filip.komar

Comment 14 Ihar Areshchankau 2020-12-31 08:42:20 CET
Created attachment 12169 [details]
Patched file /sbin/mgalive-root
Comment 15 Lewis Smith 2021-01-11 19:39:39 CET
@Martin @Dave
What do you think of the proposed solution? If not good for any reason, I second Dave's comment 6 suggestion for memory recommendations. The idea of installing any modern Linux in 512Mb impresses me; 768Mb is modest if rare.

CC: (none) => lewyssmith

Comment 16 Lewis Smith 2021-01-11 19:42:29 CET
This bug is close to being a duplicate of bug 24606.

See Also: (none) => https://bugs.mageia.org/show_bug.cgi?id=24606

Comment 17 Dave Hodgins 2021-01-11 20:39:43 CET
I'd like to see testing running live mode with the limit before approving it.
Given how close we are to the RC iso images being built, I'd prefer to hold off
testing such a change until Mageia 9 starts iso testing.
Comment 18 Martin Whitaker 2021-01-12 01:02:55 CET
Allowing the tmpfs to grow to 384MB leaves just 128MB for the system memory. When swap is enabled, it may work, albeit slowly; the problem comes when you finish installation and swap is disabled again, at which point the system probably crashes. I would prefer to advise users to use the classical installer on systems with little RAM.
Comment 19 Ihar Areshchankau 2021-01-12 09:06:31 CET
@Martin
The system installed using the patched installer will be work correctly because the installer creates the swap partition by default and the overlay technology does not used more.

@All
In the current situation the installer crashes on low-RAM PCs with non-informative message and strange behaviour. The user can be confused: is the iso-image corrupted or the PC damaged? IMHO this behaviour must be patched anyway, maybe by a restriction for installation start on such PCs. The proposed patch solves (I hope) this problem for 512Mb PC but it does not for 384Mb and 256Mb.
Comment 20 Morgan Leijström 2021-01-12 09:21:44 CET
I noted tmb message that at least for him another RC for internal may be spun now.

If all else OK i think we should do so with this change, and there is time to revert with yet another RC if problem ( I think )

The reason i think to include this is for the non-informative message and strange behaviour failing if not.  I agree a system with so little RAM will need a user to know to use very slim DE and apps, but it would be elegant if the installer do not get in the way for users that would like to try.
Comment 21 Lewis Smith 2021-01-15 10:41:13 CET
For small RAM (< 768Mb) users, it seems we should either change the release documentation (minimum system requirements, possibly Release Note) as comment 15. The Classic ISO is our mainstay; the Live Xfce ISO, used Live, can serve as a "Does it work on my box?" test before installing the Classic.

OR if somebody competent agrees with Ihar's change comment 13, comment 14 (I am confused by the two), justified in comment 19, and we can do it in time, so much the better. I am wary of "The proposed patch solves (I hope) this problem for 512Mb PC but it does not for 384Mb and 256Mb".

Not for me to judge. If such systems still exist, I suppose they could work as a non-GUI system; but would even the Classic installer still function?
Comment 22 Morgan Leijström 2021-02-05 17:22:20 CET
Ping.
Also see Bug 24606

Target Milestone: --- => Mageia 9

Comment 23 Lewis Smith 2021-02-06 12:44:04 CET
We certainly seem to need to elaborate the published minimum system memory requirements. A global figure is not satisfactory. It is in our interest to:
- avoid over-ambitious installation attempts that will founder
- encourage small system customers with relevant advice.

The difference between Live & Classic ISOs, and Live-run v Live-install, seem to matter in tight situations. Is the absolute minimum (which could be for a non-GUI end system) what the Classic ISO Installer needs to run?

Passing on to the Installer people.

CC: lewyssmith => (none)
Assignee: bugsquad => mageiatools

Comment 24 Martin Whitaker 2021-02-14 22:08:45 CET
Investigating why removing unwanted packages consumes so much space in the RAM overlay filesystem, I found two main contributors:

1. /var/lib/rpm/Packages (~80MB)
2. /usr/share/locales/locale-archive (~175MB)

Both these files get modified when the packages are removed, and hence are copied into RAM. locale-archive never shrinks in size, even though locales are being removed.

As said in comment 2, the solution is to remove unwanted packages from the installed system after installation is complete. I can modify draklive-install to do that (it already does that when persistence is enabled), although maybe it's a bit late to change it for Mageia 8.
Comment 25 Thomas Backlund 2021-02-14 23:05:58 CET
I think the possible downside of removing packages after install is extra time it takes to copy all extra data to disk and then remove the packages.

now faster systems and ssds wont maybe notice... but what of slower spinning rust systems ?

have you timed the difference ?

of course install is "one-time" so it might be acceptable, but...
Comment 26 Martin Whitaker 2021-02-14 23:13:57 CET
It is indeed a lot slower - which is why it is currently only done for persistent systems. My proposed fix is to only do so on systems with < 750MB of RAM. Better slow than failing to install!
Comment 27 Thomas Backlund 2021-02-14 23:40:19 CET
Reading all this once more, I agree with comment 26.

lets do it for final isos.

as the code for removing after install is already tested on persistent systems, it should be safe even this late in the release process

maybe add a something in release notes or errata that the install will be slow(er) on systems with < 750 MB of ram
Comment 28 Thierry Vignaud 2021-02-16 12:27:35 CET
Closing per http://gitweb.mageia.org/software/draklive-install/commit/?id=e4686b6185488e1e71fb9bbe2e25b3321b8e471b as it's fixed in draklive-install-2.30-1.mga8

Resolution: (none) => FIXED
Target Milestone: Mageia 9 => Mageia 8
CC: (none) => thierry.vignaud
Summary: Installation can't be completed on the system with low (512 Mb) RAM size => Live Installation can't be completed on the system with low (512 Mb) RAM size
Source RPM: (none) => draklive-install-2.29-1.mga8
Status: NEW => RESOLVED

Comment 29 Morgan Leijström 2021-02-16 14:10:24 CET
I dont think this need to be noted in errata.
It would be if it did instead crash.
Now it is impressive that it works :)

We could note in release notes that all installers now can operate with as low as 512MB RAM, but i think that is better noted where minimal requirements generally is stated.

Which brings us to bug 24606 about what to write generally *there*

Note You need to log in before you can comment on or make changes to this bug.