Created attachment 2 [details]
Screenshot of install under VirtualBox log

I was able to workaround the problem by booting from an iso image of Knoppix,
which uses CONFIG_HZ_300=y, mounting the mageia install, and using chroot
to enter the magiea system.

I then ran the mkinitrd, and it completed in about 3 minutes.  I was then
able to reboot the mageia install, and use the upgrade option, to complete
the install.

The system is now working, but very, very slow.  I'm going to try and
compile a custom kernel woth CONFIG_HZ_100=y, to confirm that that will
fix the problem.

So the problem is not with mkinitrd itself.  The problem is excessive cpu
usage, combined with the time limit imposed by /usr/lib/libDrakX/run_program.pm

Regards, Dave Hodgins

Took 10 days (220 hours of cpu time) to compile a custom kernel.

The only change between my custom kernel, and the desktop kernel, is changing
the HZ from 1000 to 100.

Booting to run level 1 failsafe, from pressing b in grub to the bash prompt ...

custom   1m 40s
desktop  4m 10s

Booting to run level 3, from selecting the kernel, to the login prompt

custom   2  minutes
desktop  13 minutes

LXDE works fine with the custom kernel, but locks up with the desktop kernel.

So having a kernel using 100HZ available is important, and should be the default
used by the installation iso.  It's fine to install a 1000HZ kernel on systems that
are new enough, but the installer has to be able to run in older systems.

I'll now start trying to figure out why the server kernels are failing to run here.

CC: (none) => davidwhodgins

@ David

Sorry for responding so very, very late!

Was the problem still there the last time you installed Cauldron (rc 1?). And what about Mageia 1 official?

You only have the problem when installing, but never when upgrading your kernel, is that correct?

You're very different from me, It is only now that I have a new laptop that I would consider installing anything under virtual box.
 
No one confirmed this bug, that makes it very hard, even if the problem persists, to assign it to a maintainer.

What we can do, however, (if it is still a problem, of course) is turn it into an enhancement request. Then it doesn't need to be reproduced by someone else.

CC: (none) => m.van.waes
Source RPM: (none) => drakx-installer

I haven't tried a cauldron install under VirtualBox.  The problem is
still there with Mageia 1 Official.

I've been running xppro under VirtualBox since Early 2008.  Even with
my slow cpu and 2GB of ram (I give vb 768MB), I've found very little
difference when running xppro under VirtualBox compared to running it
on native hardware, provided the vdi file is a static size, not
dynamic, and the host is mostly idle.

I've had no problems with multiple other linux systems under VB, such
as Knoppix, which do not use the CONFIG_HZ=1000 kernel option.

As to no-one confirming the bug, just take a look at the VirtualBox
Help, Section 12.4.1. Linux guests may cause a high CPU load,
which states "Some Linux distributions, for example Fedora, ship a Linux kernel configured for a timer frequency of 1000Hz. We recommend to recompile the guest kernel and to select a timer frequency of 100Hz.".

As per comment 1, using a kernel with 1000hz, it takes 6 hours to install up
to the point where mkintrd fails, since it exceeds 10 minutes.

With the 1000hz timer, it takes 12 minutes to run mkinitrd.
With the  300hz timer (Knoppix), it takes 3 minutes.
Take a look at comment 3 for the differences between 1000hz and 100hz.

I'm not sure how many people still use 6 year old hardware, like mine
and would be willing to spend the 6+ hours it takes to get to the point
where the mkinitrd fails.

Mageia needs to have a kernel suitable for use in VirtualBox on older
hardware.  It should use 100hz, no PAE (since VirtualBox does not
enable it by default), and be used by the installer, as well as the
installed system.

Since it would be used by the installer, it should be based on the 586
kernel.  Since there are so few 586 systems still in use, I think that
kernel should be changed to use the 100hz timer.

(In reply to comment #5)


>  I think that
> kernel should be changed to use the 100hz timer.

And then no one is going to report a bug: "Timer frequency is only 100Hz" ??
I'm sure some people will start complaining.

It would be nice though, to be able to choose between frequencies. 

The kernel.org website is down for maintenance at this moment. Maybe someone is already working on it?

Well, some people do think about having variable kernel timer frequency:
http://kerneltrap.org/node/6532

I think we could maybe drop down the desktop586 kernel to 250Hz as a tradeoff between 100/1000, like we used to have before switching all desktop kernels to 1000Hz.

Would that be ok ?

An other option could be a kernel parameter, as the virtualbox faq states:

"Linux kernels shipped with Red Hat Enterprise Linux (RHEL) as of release 4.7 and 5.1 as well as kernels of related Linux distributions (for instance CentOS and Oracle Enterprise Linux) support a kernel parameter divider=N. Hence, such kernels support a lower timer frequency without recompilation. We suggest to add the kernel parameter divider=10 to select a guest kernel timer frequency of 100Hz."

(In reply to comment #9)
> An other option could be a kernel parameter, as the virtualbox faq states:
> 
> "Linux kernels shipped with Red Hat Enterprise Linux (RHEL) as of release 4.7
> and 5.1 as well as kernels of related Linux distributions (for instance CentOS
> and Oracle Enterprise Linux) support a kernel parameter divider=N. Hence, such
> kernels support a lower timer frequency without recompilation. We suggest to
> add the kernel parameter divider=10 to select a guest kernel timer frequency of
> 100Hz."

I prefer that option, because, IIUR, it won't lead to other users complaining ;)

Do you and tv have to work on this together, to make that work in installer? If so, can I assign this bug to you?

Whiteboard: NEEDINFO => (none)

Interesting results.

Using divider=10, the install of just console and config packages took only 40
minutes to get to the point of running mkinitrd, instead of almost 6 hours.

The mkinitrd failed on first run, as it still exceeded 10 minutes.  I checked
syslog on the host, and it turned out mgaapplet had run during the mkinitrd,
slowing it down.

On second run, the mkinitrd took about 8 minutes, but the install completed.

On reboot (with divider=10), the system was still quite slow.

I installed the kernel-server-lastest, during which the mkinitrd took
7 min, 50 seconds.

After rebooting, to use the server kernel (without the divide, since it
already uses 100 HZ, I reran the identical mkinitrd.  It took
4 min, 2 seconds.

So while using the divider=10 does allow the install to complete, it looks
like it's still not as good as using 100 HZ.

Also, using the server kernel, the mouse etc, are much more responsive.

Just for comparison, the same mkinitrd command on the host system takes 47
seconds.

(In reply to comment #11)
> Interesting results.
> 
> Using divider=10, the install of just console and config packages took only 40
> minutes to get to the point of running mkinitrd, instead of almost 6 hours.
> 

Thx! So the option is in Mageia kernel too (When I read "as well as kernels of related distributions" in comment 10, I didn't guess we were so much related to RHEL)

Since the divider=10 option is good enough complete install, I'll assign this bug to tv


> 
> So while using the divider=10 does allow the install to complete, it looks
> like it's still not as good as using 100 HZ.
> 

@ tmb, of course you're still welcome to react!

Assignee: bugsquad => thierry.vignaud

I'm waking up:

If this divider=10 option is available in installer, this isn't a bug. Or is the option impossible to find if you do not know it is there?

I just checked with Mageia 1 DVD.

The first screen I see when booting the DVD, with a lot of colored words, disappears so quickly that I don't have the slightest idea what it says.

After that one, in the next screen, for kernel I could choose between:
Default, Safe settings, No ACPI and No local ACPI.

@ tv Is it possible to add the divider=10 option there?

It's only taken 7 months for me to find out this parameter exists.
grep divider /usr/src/linux-2.6.38.8-6.mga/Documentation/kernel-parameters.txt
doesn't show anything.

It's an undocumented kernel parameter that only partially fixes the problem.

Using a 100HZ kernel cuts the cpu usage in half over using the undocumented
divider option.

Please use a kernel that uses 100HZ for the installer.  Preferably one that
doesn't require PAE, and if installing under VB, make that the default for
the installed kernel.

I disagree: this will only work for the couple users who'll search about.

Others will just discard Mga as the slow distro.

I think the best solution would be for the kernel to detect the VB environment and switch to 100Hz.
eg: looking for VBOXBIOS in DSDT

Summary: mkinitrd fails under VirtualBox => slow OS under VirtualBox due to HZ=1000 (was mkinitrd fails in VB)

Maybe simpler would be to detect it in syslinux and automatically add the paremeter

Source RPM: drakx-installer => mageia-gfxboot-theme / drakx-installer-images

Note from comment 11,
using the divider=10, mkinitrd took 7 min, 50 seconds
using the 100 HZ, mkinitrd took 4 min, 2 seconds.

While the divider does help, it is not the same as a 100 HZ kernel.

It was enough of a help to get the install to finish on second mkinitrd try,
however there is noticeable sluggishness with the system running with the
divider=10, compared to using a kernel with 100HZ.

Erwan, would that be possible?

CC: (none) => erwanaliasr1, thierry.vignaud

Erwan: see comment #18

Syslinux is able to detect the cpu flag meaning we run under a virtualized setup.
In such configuration, that would also apply for kvm or vmware but I do think this is a good idea. Do you agree on that ?

I can have a look at it once the migration of the syslinux 4 will be completed.

I agree.
And I offer you this bug :-)

Assignee: thierry.vignaud => erwanaliasr1

Have anyone actually verified that this happends on kvm or vmware ?

Erwan, any news?

Target set to mageia 3, as installer changes are needed.

Target Milestone: --- => Mageia 3
CC: (none) => guillomovitch

Hi,

This bug was filed against cauldron, but we do not have cauldron at the moment.

Please report whether this bug is still valid for Mageia 2.

Thanks :)

Cheers,
marja

Keywords: (none) => NEEDINFO

Thomas, is there a way to dynamically change this parameter from userspace?
Or can kernel be patched to change divider at runtime, on bootstrapping, in order to reduce it when detecting vbox?

@ tmb

Can you answer tv's question in comment 28?

(In reply to comment #28)
> Thomas, is there a way to dynamically change this parameter from userspace?
> Or can kernel be patched to change divider at runtime, on bootstrapping, in
> order to reduce it when detecting vbox?

Please look at the bottom of this mail to see whether you're the assignee of this  bug, if you don't already know whether you are.


If you're the assignee:

We'd like to know for sure whether this bug was assigned correctly. Please change status to ASSIGNED if it is, or put OK on the whiteboard instead.

If you don't have a clue and don't see a way to find out, then please put NEEDHELP on the whiteboard.

Please assign back to Bug Squad or to the correct person to solve this bug if we were wrong to assign it to you, and explain why.

Thanks :)

**************************** 

@ the reporter and persons in the cc of this bug:

If you have any new information that wasn't given before (like this bug being valid for another version of Mageia, too, or it being solved) please tell us.

@ the reporter of this bug

If you didn't reply yet to a request for more information, please do so within two weeks from now.

Thanks all :-D

During testing of Bug 6694, it appears qemu is affected too.

Adding the divider=10 does help, but not as much as switching to the
server kernel.

swecarp found this with 3alpha1 in virtualbox

Looks to be the same problem

http://i.imgur.com/WKNWt.png

Bumping priority so that we fixes it for mga3.
We could either:

- use a HZ=100 kernel for installer (one more flavor :-( since
  server flavor won't work everywhere)

- patch kernel so that it dynamically adjust HZ at boot time
  when detecting vbox

- detect in syslinux if we're under vbox and add the "divider"
  parameter (see comment #17 &  #18)

CC: (none) => ennael1
Priority: Normal => release_blocker
Severity: normal => critical

we could also always use "divider=10" in our syslinux/isolinux config.
It won't be copied to the installed system, which is both:
- nice: it won't affect systems not running under vbox
- sad: it won't fix booting installed systems under vbox

We will take some time before alpha2 to clean syslinux integration in Mageia so that we are able to use syslinux detection.

Note that this only happens on CPUs w/o VT-x or AMD-V (which includes new low
end CPUs)

Summary: slow OS under VirtualBox due to HZ=1000 (was mkinitrd fails in VB) => slow OS under VirtualBox on CPU w/o HW support for virt, due to HZ=1000 (was mkinitrd fails in VB)

is this still valid now that we use dracut ?

CC: (none) => nicolas.lecureuil

Yes, that's totally unrelated.

Removing the release blocker tag, as there is a partial
workaround, and there's been two releases with it set.

Priority: release_blocker => High

Removing "on CPU w/o HW support for virt" from the subject.

While it's less of a problem on cpus with hw support for virt,
it's still a problem.

Summary: slow OS under VirtualBox on CPU w/o HW support for virt, due to HZ=1000 (was mkinitrd fails in VB) => slow OS under VirtualBox, due to HZ=1000 (was mkinitrd fails in VB)

Btw, during testing of the xen update, I can confirm this bug affects
hardware with virt support, and isn't limited to virtualbox.

Using full hvm emulation under xen, it's much slower than virtualbox,
and on my new qaud core, still couldn't run the mkinitrd in under 10
minutes, without the divider=10 option.

Hi all,

this problem is still present in Mageia 4 Beta 2 KDE4-i586-LiveCD with VirtualBox running on top of Mageia Cauldron. The installer won't exit. After adding "divider=10" to the boot commands, it finishes the installation fine.

Regards,

-- Shlomi Fish

CC: (none) => shlomif

Note, that I suggested trying the divider=10 option, due to journalctl error
messages, that indicate to me, too many interrupts were being generated.
Nov 25 18:01:10 localhost rtkit-daemon[2533]: The canary thread is apparently starving. Taking action.
Nov 25 18:01:10 localhost rtkit-daemon[2533]: Demoting known real-time threads.

Shlomi, can you add what the host cpu is?

(In reply to Dave Hodgins from comment #43)
> Note, that I suggested trying the divider=10 option, due to journalctl error
> messages, that indicate to me, too many interrupts were being generated.
> Nov 25 18:01:10 localhost rtkit-daemon[2533]: The canary thread is
> apparently starving. Taking action.
> Nov 25 18:01:10 localhost rtkit-daemon[2533]: Demoting known real-time
> threads.
> 
> Shlomi, can you add what the host cpu is?

Yes, it's a «Intel(R) Core(TM) i3-2100 CPU @ 3.10GHz» (from «cat /proc/cpuinfo»). The machine's SPECs are:

[QUOTE]

    An Intel Core i3 CPU (x86-64).
    8 GB of RAM.
    Intel Corporation Sandy Bridge Integrated Graphics Controller (rev 09)
    A 2 TB hard-disk.
    A 21×´ Wide LCD Screen by LG.
    Intel Corporation Cougar Point High Definition Audio Controller.
    Intel Corporation 82579V Gigabit Network Connection.

[/QUOTE]

On my 2005 single core celeron system, now running Mageia 4, even with the
divider=10 option, dracut cannot run in a vb guest, in under 10 minutes.
The time limit has to be increased, if we still want to support running
under vb, on older systems.

@David or anyone: please advise if this applies to 5beta1 (because 4 installer won't be fixed), Thanks

(In reply to Dick Gevers from comment #46)
> @David or anyone: please advise if this applies to 5beta1 (because 4
> installer won't be fixed), Thanks

Yes, it does, I've seen people discussing it on #mageia-qa last week.
both Dave Hodgins and tmb advised others to use or try using "divider=10", they would have known if that was no longer needed.

Whiteboard: 4alpha1 => 5beta1

@Anne, Erwan: Do you think we can get this fixed for mga5?

CC: (none) => remi

I think it's time we advertize testers to use virt-manager instead of VirtualBox then...

Is this still an issue with VirtualBox 5.1 series ?

My old i586 system died, so the impact is minor on my current system, but
there is still a noticeable difference testing an install under vb with or
without the divider=10 option.

Could someone summarize this issue and possible solutions?

Basically network can be really slow with kernels build with high (aka normal) HZ values, such as in most distros.
See https://www.google.fr/search?q=divider%3D10+site:virtualbox.org

A workaround is to add the "divider=10" kernel parameter to the boot loader config.
A solution is to switch from VirtualBox to virt-manager (virtio for everything anyway :-) )

There were some improvements with VBox 5.1.x so we would like some VBox users to confirm whether it's better with VBox 5.1

See also comment #33 about possible way to auto add the "divider=10" parameter

(In reply to Thierry Vignaud from comment #53)
> There were some improvements with VBox 5.1.x so we would like some VBox
> users to confirm whether it's better with VBox 5.1

Adding NEEDINFO keyword for that.

Keywords: (none) => NEEDINFO

It would be good to add a note about this in the Errata.

Good night / morning QA Team

I am using MGA5 KDE 32 bits DVDISO installed in vbox 5.1.32
I am using MGA6 32 Live disk installed in the same vbox

Without modifyng kernel parameters at boot, the post install works very well in both versions
Installing kernel updates does not take more than expected, even the new kernels does not boot because issues that QA Team are working (not related to mkinitrd)

CC: (none) => neoser10

While systems slow enough to be impacted by the 10 minute timeout for dracut,
are now very rare, the problem still applies.

Running any cpu intensive task, such as installing a kernel update takes
much longer when the currently running kernel is the desktop kernel, instead
of the server kernel, in the vb guest. This is particularly noticeable for
me when testing kernel updates, where I install 5 kernels in one urpmi
transaction for i586 vb guests, 4 for x86_64 guests.

The simplest solution is to use the server kernel in the installers, and as
the suggested kernel for any install in a virtual system, as the server
kernel uses HZ=100.

Workarounds such as auto adding the divider=10 kernel option would also
work, though not as well.

Adding Martin to the cc list.

CC: (none) => mageia

(In reply to Dave Hodgins from comment #58)
> The simplest solution is to use the server kernel in the installers, and as
> the suggested kernel for any install in a virtual system, as the server
> kernel uses HZ=100.

Correct me if I'm wrong, but I believe the 32-bit server kernel won't run on i586 class machines (which is why we use the desktop586 kernel on the installer and Live ISOs). Also, proprietary kernel modules won't get built during installation if the installer kernel doesn't match the installed kernel, which then slows down the first boot (and for the broadcom-wl driver, may require an additional reboot),  so using the most commonly installed kernel in the installer is a good thing.

If we changed the suggested kernel for installs in a virtual system, I think I'd want to add an option in the GUI for the user to override the choice. That might be useful anyway...

Ah. You're correct that the server kernel will not run on an i586 cpu, though
it works in vb i586 installs with PAE/NX enabled. It also worked on the celeron
i686 cpu I was using when this started, though that system died in 2016.

I think the best approach would be:

Change the desktop586 kernel config to use HZ=100 instead of HZ=1000.
That would fix the problem for the i586 installer.

In the x86_64 installer's kernel options, add the divider=10 kernel option.

In virtual guest installs suggest using the server kernel, with the option
to override.

No, lets not blow this out of proportion...

This is a limited problem... not everyone have this issue... for example I havent been able to reproduce this on any of my systems... the oldest host hw is from 2008 and I dont see vbox slowdowns in it... 

So instead of papering it over... it needs to be found out _what_ kind of hw /configs gets this issue before screwing default setup for lots of users not needing this...

It can be as old as 2008 but still has VT-x or AMD-V which hides the issue.
For me, I did saw the difference when I switched to a vmx/svm aware box.
Last times I checked I had the impression that the divider option still made a small difference event with VT-x.
But I usually uses virt-manager which is faster for my usage.

Yeah, but that's also the point :)

AMD CPUs have supported AMD-V since ~mid 2006, so basically from:
- Athlon 64 ("Orleans"), Athlon 64 X2 ("Windsor")Athlon 64 FX ("Windsor")
- Turion 64 X2, Phenom, Opteron, ..
- Sempron (huron, sargas)

Intel VT-x showed up at end of 2005

Of course Intel hw got crippled by marketing so "cheaper cpus" got VT-x disabled so they could sell more expensive cpus... and no real logic between what cpus had VT-x and wich ones dont...


So the first thing for people hitting this issue is to check bios for possible option to enable vmx/svm...


But yeah, I guess we could set HZ_100 on desktop586 kernels as that is only there to support low end hw anyway...

(In reply to Thomas Backlund from comment #63)

> Of course Intel hw got crippled by marketing so "cheaper cpus" got VT-x
> disabled so they could sell more expensive cpus... and no real logic between
> what cpus had VT-x and wich ones dont...


And not only cheaper ones ... there were "market segmentation" fun too...

For example, there were some *single* core 1.6GHz Celerons that got VT-X

And iirc on the othger some revisions of Intel *enthusiast* platform processors got it disabled too because "enthusisasts only care about game speed" and dont want virtualization...

(In reply to Thomas Backlund from comment #63)
> But yeah, I guess we could set HZ_100 on desktop586 kernels as that is only
> there to support low end hw anyway...

Although it is the kernel used on the 32-bit Live ISOs, so this would affect anyone running a 32-bit Live system.

(In reply to Martin Whitaker from comment #65)
> (In reply to Thomas Backlund from comment #63)
> > But yeah, I guess we could set HZ_100 on desktop586 kernels as that is only
> > there to support low end hw anyway...
> 
> Although it is the kernel used on the 32-bit Live ISOs, so this would affect
> anyone running a 32-bit Live system.

Yeah, that's also a thing to consider...

Reviewing this issue because I am having a maybe related issue with MGA6 in vbox fully updated I found that my machine does not have virtualization instructions as listed in ark.intel.com for the e5200 cpu

https://ark.intel.com/products/37212/Intel-Pentium-Processor-E5200-2M-Cache-2_50-GHz-800-MHz-FSB

My MGA6 vm, with the 4.9.35 kernel, runs fast, with the 4.14.20 is slow to run in plasma desktop enviroment and to start if is runs with runlevel 3

The 4.9.56 kernel boots fast from 5 or 3 runlevel
Other 4.14 kernels did not boot because meditation guru... a little problem with vboz software emulator, because the "meditation guru problem" i can not test these kernels

Attached the journald for both mentioned kernels (Sorry to erase the other boots, but the file may be is unreadable without this edition)

4.9.56
https://drive.google.com/open?id=1TI6b6vQp2dRfiGlncnG8k58Sn0ohTomC
4.14.20
https://drive.google.com/open?id=1p6qYdJtwlbfplP5tix3cbRF4R5XUEG8h

Slowdowns with the newer kernels are expected due to the Meltdown/Spectre
mitigations.

Using the (no recomended kernel options) to disable the meltdown, spectre issues
My virtual machine works very fast

May be the spectre and meltdown patches slowsdown the VM without hardware virtualization support, but i can not reproduce the original issue here described

Hi, what about this old bug?
Almost 2+ years since last comment. Also, seems it affect turtle 32 bits CPU.
Can we consider users have upgraded their system to new and most powerful hardware?

Closing this 2020-09-01.

Keywords: NEEDINFO => (none)
Severity: critical => major
CC: (none) => ouaurelien
Hardware: All => i586

While it only noticeably impacts slower  systems, it's still true that
adding divider=10 to the kernel options that use 1000HZ reduces cpu usage
of virtualbox. Since very few people will try to run vb on slow systems,
closing this bug as wont fix.

Status: NEW => RESOLVED
Resolution: (none) => WONTFIX