Bug 19680

Summary: Stage2 hang and crash if Russian language is selected in installer ("* running: loadkeys ru * loading keymap ru" flood then "exec of loadkeys failed: Cannot allocate memory")
Product: Mageia Reporter: Oleg Bosis <olegbosis>
Component: InstallerAssignee: Mageia tools maintainers <mageiatools>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Normal CC: hazard157, loginov_alex, marja11, olegbosis, pablo, thierry.vignaud, ugal12v
Version: Cauldron   
Target Milestone: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Source RPM: drakx-installer-stage2, drakx-kbd-mouse-x11 CVE:
Status comment: fixed in drakx-kbd-mouse-x11, fix is included in drakx-installer-stage2-17.73-3.mga6
Attachments: report.bug from vm
report.bug from my physical PC
boot.log from installed system with the 'cannot open file ru.uni' lines

Description Oleg Bosis 2016-10-30 09:20:47 CET
Description of problem:
I've tried to install current cauldron in a virtualbox vm and was not able to do it.

I've tried to use both Mageia-Cauldron-netinstall-x86_64.iso and Mageia-Cauldron-netinstall-nonfree-x86_64.iso with no luck.

Installer starts normally, proceeds to language selection and after I do select Russian it begin to pause for several minutes on every next step (even on a license agreement) and then crashes and reboots vm before disk partitioning stage.

I do not speak many languages (sorry :) ) but I've also tested English and Ukrainian languages and they seem to work normally (I could also try Belorussian language but it is not included in netinstall image and installer uses English). Only Russian causes the problem.

The host is my Asus laptop with current Mageia 5 x86_64. Virtualbox 5.1.8 from updates-testing.

Version-Release number of selected component (if applicable):


How reproducible:
always

Steps to Reproduce:
1. Start installing using x86_64 netinstall iso image
2. Choose Russian when prompted for language to use
3.
Comment 1 Marja Van Waes 2016-10-30 09:59:01 CET
(In reply to Oleg Bosis from comment #0)

> 
> Steps to Reproduce:
> 1. Start installing using x86_64 netinstall iso image
> 2. Choose Russian when prompted for language to use
> 3.

when prompted.... so that's at the beginning of stage2.

It's long ago that I did a VBox install, I don't know whether you can switch to tty2 from the guest, and whether inserting a USB key and then typing:
   bug
will put report.bug on the key.

If you can do that before the crash, but after starting to see the pauses, that would be great.

If that's impossible, then it would be great to get the logs that are in /tmp, like /tmp/ddebug.log and /tmp/stage1.log 
Maybe they're even still there after the crash, so that the ddebug.log might contain more information?

Please attach the log files or report.bug to this bug report (if there's an avalanche of errors, you might need to compress with xz or so first)

CC: (none) => marja11
Assignee: bugsquad => mageiatools
Source RPM: (none) => drakx-installer-stage2

Marja Van Waes 2016-10-30 09:59:12 CET

Keywords: (none) => NEEDINFO

Comment 2 Oleg Bosis 2016-10-30 10:27:03 CET
Created attachment 8608 [details]
report.bug from vm

Attached report.bug from vm made just after language selection

CC: (none) => olegbosis

Comment 3 Oleg Bosis 2016-10-30 10:32:00 CET
Additional info: I've switched to vt2 before disk partitioning stage and the console is flooded with 

* running: loadkeys ru
* loading keymap ru

string pairs. And this strings seem to be printed endlessly...
Comment 4 Marja Van Waes 2016-10-30 11:07:30 CET
(In reply to Oleg Bosis from comment #3)
> Additional info: I've switched to vt2 before disk partitioning stage and the
> console is flooded with 
> 
> * running: loadkeys ru
> * loading keymap ru
> 
> string pairs. And this strings seem to be printed endlessly...

It ends with

* exec of loadkeys failed: Cannot allocate memory

after that there are many repeated lines of:

* running: setxkbmap -option 
* running: setxkbmap ru -variant  -model pc105 -option compose:rwin -compat

which ends with

* lang:ru charset:koi8-u font:DejaVu Sans 12 consolefont:UniCyr_8x16
* lang:ru charset:koi8-u font:font-family: DejaVu Sans; font-size: 12pt; consolefont:UniCyr_8x16

Keywords: NEEDINFO => (none)

Comment 5 Oleg Bosis 2016-10-30 11:17:37 CET
(In reply to Marja van Waes from comment #4)
> (In reply to Oleg Bosis from comment #3)
> > Additional info: I've switched to vt2 before disk partitioning stage and the
> > console is flooded with 
> > 
> > * running: loadkeys ru
> > * loading keymap ru
> > 
> > string pairs. And this strings seem to be printed endlessly...
> 
> It ends with
> 
> * exec of loadkeys failed: Cannot allocate memory
> 
> after that there are many repeated lines of:
> 
> * running: setxkbmap -option 
> * running: setxkbmap ru -variant  -model pc105 -option compose:rwin -compat
> 
> which ends with
> 
> * lang:ru charset:koi8-u font:DejaVu Sans 12 consolefont:UniCyr_8x16
> * lang:ru charset:koi8-u font:font-family: DejaVu Sans; font-size: 12pt;
> consolefont:UniCyr_8x16

You mean the content of the report.bug? It was created before license agreement and keyboard layout stages and it contains just a very small amount of loadkeys.
The flood occurs later just before the disk partitioning stage and probably causes vm reboot (probably just some memory gets exhausted by these "loadkeys").
Comment 6 Marja Van Waes 2016-10-30 11:24:25 CET
(In reply to Oleg Bosis from comment #5)

> 
> You mean the content of the report.bug? 

Yes, that's what I meant

> It was created before license
> agreement and keyboard layout stages and it contains just a very small
> amount of loadkeys.
> The flood occurs later just before the disk partitioning stage and probably
> causes vm reboot (probably just some memory gets exhausted by these
> "loadkeys").

Thanks for telling us!

Summary: Netinstall images hang and crash if Russian language is selected in installer in virtualbox vm => Netinstall images hang and crash if Russian language is selected in installer in virtualbox vm ("* running: loadkeys ru * loading keymap ru" flood)

Comment 7 Yuri Galitsky 2017-03-07 14:30:31 CET
This bug is valid for Mageia-6-sta2-x86_64-DVD.iso image. I've tried to install it in a virtualbox and my physical PC both. After I do select 'Russian' and press 'Next' button installer freezes.

CC: (none) => ugal12v

Comment 8 Yuri Galitsky 2017-03-07 14:32:31 CET
Created attachment 9035 [details]
report.bug from my physical PC
Thierry Vignaud 2017-03-07 15:17:36 CET

CC: (none) => thierry.vignaud
Summary: Netinstall images hang and crash if Russian language is selected in installer in virtualbox vm ("* running: loadkeys ru * loading keymap ru" flood) => Netinstall images hang and crash if Russian language is selected in installer ("* running: loadkeys ru * loading keymap ru" flood then "exec of loadkeys failed: Cannot allocate memory")

Marja Van Waes 2017-03-07 15:56:22 CET

Priority: Normal => release_blocker

Marja Van Waes 2017-03-07 15:58:25 CET

Summary: Netinstall images hang and crash if Russian language is selected in installer ("* running: loadkeys ru * loading keymap ru" flood then "exec of loadkeys failed: Cannot allocate memory") => Stage2 hang and crash if Russian language is selected in installer ("* running: loadkeys ru * loading keymap ru" flood then "exec of loadkeys failed: Cannot allocate memory")

Comment 9 Thierry Vignaud 2017-03-07 16:41:51 CET
It can be reproduced by running "loadkeys ru" on tty2 in installer
Comment 10 Thierry Vignaud 2017-03-07 16:51:58 CET
Reverting s/ru4/ru/ fixes it:
http://gitweb.mageia.org/software/drakx-kbd-mouse-x11/commit/lib/keyboard.pm?id=44a643f73fbb9a1d83fea952326f26fedc407842

Priority: release_blocker => Normal
CC: (none) => pablo
Source RPM: drakx-installer-stage2 => drakx-installer-stage2, drakx-kbd-mouse-x11

Comment 11 Mageia Robot 2017-03-07 20:27:29 CET
commit e2990244437375a022f4da40f633d150de63de8c
Author: Thierry Vignaud <thierry.vignaud@...>
Date:   Tue Mar 7 17:15:58 2017 +0100

    Revert "switch from legacy (KOI8-R) 'ru4' keyboard to 'ru'"
    
    This reverts commit 44a643f73fbb9a1d83fea952326f26fedc407842.
    
    This causes a crash in installer (mga#19680)
---
 Commit Link:
   http://gitweb.mageia.org/software/drakx-kbd-mouse-x11/commit/?id=e2990244437375a022f4da40f633d150de63de8c
Comment 12 Marja Van Waes 2017-03-08 09:07:56 CET
Added to the errata
https://wiki.mageia.org/en/Mageia_6_Errata#Russian_install_impossible

But not adding the IN_ERRATA6 keyword, because Thierry's commit fixes it for 6RC.
Netinstalls will be fixed as soon as drakx-installer-stage2 is rebuilt with new drakx-kbd-mouse-x11

@ Thierry

Do we need feedback from Pablo before rebuilding stage2?

@ Yuri and Oleg

Could both of you please consider becoming QA iso testers? 
https://wiki.mageia.org/en/QA_ISO_testers

Even if this bug will be fixed in 6RC, we don't want to end up missing another awful bug that doesn't affect our current iso testers, but does affect many in the rest of the world!

Mageia 4.1 had to be released because we had missed a bug that made it impossible to install Mageia for half of the Russian users. We don't want that to happen again.

Btw, do the 6sta2 Live isos work?

@ AlexL

I thought you had access to the QA isos, but fail to see you in the iso testers list. Is my memory wrong?

Status comment: (none) => fixed in drakx-kbd-mouse-x11, fix will end up in stage2 when it's rebuilt.
CC: (none) => loginov_alex

Comment 13 Marja Van Waes 2017-03-08 10:33:46 CET
Net install should work when drakx-installer-stage2-17.73-3.mga6 (pushed ± an hour ago) has arrived on your mirror

Status comment: fixed in drakx-kbd-mouse-x11, fix will end up in stage2 when it's rebuilt. => fixed in drakx-kbd-mouse-x11, fix is included in drakx-installer-stage2-17.73-3.mga6

Comment 14 Thierry Vignaud 2017-03-08 11:04:28 CET
Yes the bug is fixed.
Though I requested Pablo's advice for what should we do if we want to switch to non legacy encoding.
Pablo is the Jedi master regarding l10n/i18n :-)

Status: NEW => RESOLVED
Resolution: (none) => FIXED

Comment 15 Yuri Galitsky 2017-03-08 16:02:02 CET
> Btw, do the 6sta2 Live isos work?

I tested the Mageia-6-sta2-LiveDVD-Plasma-x86_64-DVD image. The Russian localization installs well from both the boot menu and the live system. 
When the installed system starts to boot, two lines 'cannot open file ru.uni' appears on the console. I don't know if it's important or not. The installed system boots and operates seems normally. I attached the log file with that lines.
Comment 16 Yuri Galitsky 2017-03-08 16:04:06 CET
Created attachment 9044 [details]
boot.log from installed system with the 'cannot open file ru.uni' lines
Comment 17 Marja Van Waes 2017-03-08 16:51:09 CET
(In reply to Yuri Galitsky from comment #15)
> > Btw, do the 6sta2 Live isos work?
> 
> I tested the Mageia-6-sta2-LiveDVD-Plasma-x86_64-DVD image. The Russian
> localization installs well from both the boot menu and the live system. 
> When the installed system starts to boot, two lines 'cannot open file
> ru.uni' appears on the console. I don't know if it's important or not. The
> installed system boots and operates seems normally. I attached the log file
> with that lines.

Thanks for testing :-)

The error means it cannot open /usr/lib/kbd/keymaps/i386/qwerty/ru.map.gz which is the Russian UTF-8 keymap. You'll probably notice that when you switch to a VT with e.g. ctrl+alt+F3, that you can't type Russian.

Becoming root and then typing "loadkeys ru4" should give you the old Russian keymap in your VT, which does work.
Comment 18 Thierry Vignaud 2017-03-08 17:09:45 CET
That's another issue that should be in another bug report :-)
I've tried switching Russian to UTF-8 but I obviously failed (as can be seen in this bug report existence :-))
The terminal setting us in keyboarddrake's keyboard.pm :

http://gitweb.mageia.org/software/drakx-kbd-mouse-x11/tree/lib/keyboard.pm#n127
http://gitweb.mageia.org/software/drakx-kbd-mouse-x11/tree/lib/keyboard.pm#n275
Comment 19 Marja Van Waes 2017-03-08 17:17:50 CET
/me is confused, but is aware that she jumped to conclusions too fast
Comment 20 Thierry Vignaud 2017-03-08 17:26:00 CET
Now that I think about it, the issue is that it didn't find a kbmap file in /usr/share/keymaps.
For installer, those came from http://gitweb.mageia.org/software/drakx/tree/perl-install/install/share/keymaps.tar.bz2
Underdocumented Pixel's generator is at http://gitweb.mageia.org/software/drakx/tree/perl-install/install/share/kmap2bkmap

So what happens is that loadkeys is implemented in perl in installer, calls loadkeys() from install::commands, which calls keyboard::setup_install() which if the bkmap file is missing, calls the external loadkey program instead of _builtin_loadkeys().
And the external loadkeys enters commands.pm and we got a nice old fork bomb written in perl :-(

http://gitweb.mageia.org/software/drakx/tree/perl-install/install/commands.pm#n422
http://gitweb.mageia.org/software/drakx-kbd-mouse-x11/tree/lib/keyboard.pm#n560

So the issue is that if we wants to switch away from "ru4", we must also regenerate install/share/keymaps.tar.bz2 with the new mapping...
Comment 21 Marja Van Waes 2017-03-09 12:11:30 CET
*** Bug 20430 has been marked as a duplicate of this bug. ***

CC: (none) => hazard157