Bug 1148

Summary: finish-install takes too long or may hang at first reboot
Product: Mageia Reporter: jcl darkc <jcldc13>
Component: RPM PackagesAssignee: Olivier Blin <mageia>
Status: ASSIGNED --- QA Contact:
Severity: normal    
Priority: Normal CC: davidwhodgins, ennael1, jcldc13, pmithrandir, thierry.vignaud, tmb
Version: CauldronKeywords: PATCH
Target Milestone: ---   
Hardware: i586   
OS: Linux   
Whiteboard:
Source RPM: drakxtools CVE:
Status comment:
Attachments: (untested) example of better heuristik

Description jcl darkc 2011-05-05 09:41:27 CEST
Description of problem:

 After the first reboot which follow the LIVE installation, the script /usr/sbin/finish-install is called and can take a very long time to complete OR may also hang. The user will never get access to his/her session and will have to hard reboot.

 This situation happens if the new user account created after the first reboot already exist on the disk. When you install several Linux distribution on the same machine, usually you have a partition /home/user shared between all the Linux distributions, because you want to have the same user name between all the distribution. Month after month, or even year after year, your /home/user get bigger with tons of files. Then the script /usr/sbin/finish-install at some point call the subroutine "ask_users", which run a big recursive grep command to rename some "/home/live" files to "/home/user" files. This grep can take a very long time if the users's home is very big... Moreover it can also totally hang, because the command grep hangs when it's run on named pipe file. A program like kaffeine creates such named pipe file (see .kde4/share/apps/kaffeine/dvbpipe.m2t). If a user as such kind of file in his HOME, then the script will hang and the user will never get access to his session after the first reboot. 
BTW to find named pipe file in your home, run the command: 
find ${HOME} -type p

for me it gives:
/home/jcl/.kaxtv1.ts
/home/jcl/.kaxtv.ts
/home/jcl/.kde4/share/apps/kaffeine/dvbpipe.m2t

then if I run the command
grep dummy /home/jcl/.kde4/share/apps/kaffeine/dvbpipe.m2t

grep command never returns...

 you can also simulate this behavior by creating file named pipe:
mkfifo myfifo
grep dummy myfifo

grep command never returns...

 SO, I think it's necessary to re-think the "grep command" used in /usr/sbin/finish-install perl script, because first  it can take too long or hang at first reboot.

PS: this bad behavior exist since almost all the last 6 Mandriva distributions and I finally decided to post a bug report for mageia... shame on me.

Version-Release number of selected component (if applicable):
Mageia Beta 2 live 32 bits

How reproducible:


Steps to Reproduce:
1. install mageia
2. reboot 
3. at first boot create a user which already exist on the disk
jcl darkc 2011-05-05 16:01:47 CEST

CC: (none) => jcldc13

Anne Nicolas 2011-05-05 16:03:29 CEST

CC: (none) => ennael1
Assignee: bugsquad => mageia

Comment 1 Ahmad Samir 2011-05-05 18:03:45 CEST
@jcl: you don't need to add yourself in CC in bug reports you open, because you'll get emails of changes in the report anyway.
Ahmad Samir 2011-05-05 18:04:09 CEST

Source RPM: drakxtools-13.49-1.mga1.src.rpm => drakx-finish-install

Comment 2 jcl darkc 2011-05-07 20:50:01 CEST
 I installed mageia beta2 on a new machine using LIVE method. At the first reboot, I rebooted in safe mode instead of normal mode. Then I edited the file /usr/sbin/finish-install and I modified the line 173:

I replaced :

-------------------------------------------------------------------------------
run_program::run(qq(grep -rl $old_home $new_home/.??* | while read f; do perl -pi -e 's,$old_home,$new_home,g' "\$f"; done));

with

#run_program::run(qq(grep -rl $old_home $new_home/.??* | while read f; do perl -pi -e 's,$old_home,$new_home,g' "\$f"; done));
-------------------------------------------------------------------------------

 which, actually, simply deactivate this line.

 Then I rebooted in normal mode and installation has completed very quickly and flawlessly without running this "grep" command inside the /usr/sbin/finish-install script. 

 So the question is : Do we still have any good reasons to keep this "grep" command in that script ?
 
PS : when I rebooted in normal mode I choosed a username which was already existing on the /home file system, which means that if the grep command would have been present in the finish-install script, the completion of the installation would have taken ages or never finished....
Comment 3 jcl darkc 2011-05-19 09:27:38 CEST
 I am wondering if I am the only guy who encountered that bug or what ? So far I did not get any answer. Is it postponed for the next release ?
Comment 4 Dave Hodgins 2011-05-19 10:07:37 CEST
My guess is that most people do not share /home, so the problem doesn't
show up.

/home should not be shared across versions/distributions, as the
formats used in the configuration files will vary.  (I learned that
the hard way!)

I give each distro it's own /home, but replace directories such as
~/Documents with a symlink to a directory within a separately mounted
"data" filesystem, so that data can be shared.  Sharing config files
is not a good idea.

Some very specific "configuration" files can be shared, when you know
the formats are the same, but you have to handle that case by case.

While I have signed on as a member of the QA team, I'm just a regular user.
My recommendation is to avoid sharing /home.  It will definitely cause
problems in the future, if you do share it.

In my opinion, jcl darkc, this bug should be closed as invalid, as it is
being caused by actions that should not be taken.

CC: (none) => davidwhodgins

Comment 5 jcl darkc 2011-05-19 11:10:39 CEST
Hi david,

 I do not agree with your opinion. People may want to use a shared /home directory for every distributions or OS upgrade.

If we follow your recommandation, where do you specify users's home ?

Imagine that at the first installation, you specified the home location for the user "jcl" here :

==> /home/jcl

so what do you do for the next upgrade/install ?
*) /home2/jcl ? (so you create a new partition or you create a symlink ?)
*) /home/jcl2   (so you change the username name ?)

So it means a user intervention to do this and some average linux administration skill, which is not the case for most of the people. To not have a shared
home directory looks like a regression and does not look like a really professional operating system...

The problem here, is that there is a **bad implementation** in the script /usr/sbin/finish-install, that I mentionned in my bug report (#1148), which needs to be fixed.

 It would be far better to re-think that script which will let the user every possibilities for her/his home directory (create a new one or use shared home).

If Mageia team think you are right, and I will be disapointed, it will be necessary to put some information about this issue in the ERRATA document...
Comment 6 Olivier Blin 2011-05-19 22:52:18 CEST
This will be fixed in next drakxtools release (using grep -D skip)

Status: NEW => ASSIGNED

Comment 7 jcl darkc 2011-05-19 23:16:23 CEST
Good evening,
 
Well, good point for you Olivier,  "grep -D skip" will fix the bug which hang the first boot process if there is a FIFO file in the user's home.

But I am still questionning of the utility of this grep command which can take a very long time to complete if the user's home is big (thousand of files) ? Could you give me some explanations about the purpose of this grep command, is it really necessary ?

 According to my experience, every time I install a new mandriva/mageia distribution on my system (with a shared user's home directory), I have to kill this "grep" command (which hang in my case bc of the fifo) and afterward my system still works perfectly. So it looks like this "grep" call is useless... Could we not simply remove it ?

 Anyway thanks for giving me some hope, I don't feel lonely anymore :)
Comment 8 Olivier Blin 2011-05-19 23:34:33 CEST
After live install, the user is asked to choose a user name, but it is not actually creating a new user, it is just renaming the "live" user, and keeping the home from live system (using usermod), so that documents and settings creating during live session are not lost.

This grep | sed command is used to modify the config files so that the settings reflect the new home path (instead of /home/live)

We could skip all this if the target home already exists.
But where to put the files and settings creating during live session?
Comment 9 Olivier Blin 2011-05-20 00:23:06 CEST
The hang is fixed in drakxtools-13.55-1.mga1

Status: ASSIGNED => RESOLVED
Resolution: (none) => FIXED

Comment 10 jcl darkc 2011-05-20 11:22:22 CEST
(In reply to comment #8)
> After live install, the user is asked to choose a user name, but it is not
> actually creating a new user, it is just renaming the "live" user, and keeping
> the home from live system (using usermod), so that documents and settings
> creating during live session are not lost.
> 
> This grep | sed command is used to modify the config files so that the settings
> reflect the new home path (instead of /home/live)
> 
> We could skip all this if the target home already exists.
> But where to put the files and settings creating during live session?

 At the first reboot, if the user choose to use his old (or previous) user's home, then ** YES ** skip the recursive grep command which is non sense in this case. And all settings created during the live session can be lost because the user WANTS his old session back, so no big deal !

 I looked carefully at the command grep used from /usr/sbin/finish-install script. Here it is:

grep -D skip -rl $old_home $new_home/.??* | while read f; do perl -pi -e 's,$old_home,$new_home,g' "\$f"; done

 Let'say I choosed "jcl" username (which is already exist, for my case on my disk, and which is /home/jcl from the previous install), then :

variable $new_home="/home/jcl"
variable $old_home="/home/live" (which comes from the live session)

then the real command is :

grep -D skip -rl /home/live /home/jcl/.??*   | while read f; do perl -pi -e 's,/home/live,/home/jcl,g' $f;done

which can be run directly, by everybody, from a terminal.

So, this command find recursively ALL the hidden files starting from /home/jcl like:
.login
.config/gtk-2.0/*
.kde4/*
/home/jcl/.cache/chromium/Default/Cache/*  !!!!!!!!!
.mozilla/firefox/2zzh7jne.default/Cache/ * !!!!!!!!!
..
..
etc

 then from all these files try to find if the string "/home/live" exist and if it exists change all string "/home/live" by "/home/jcl" !!! 

This is 1) non sense and useless from an existing user's home, and 2) it can take a very long time to complete. So please, I would advice to modify the script to skip this command in case if user's home already exist. And in such case, live settings can be lost with no dammage.

  Thanks in advance.
Comment 11 jcl darkc 2011-05-20 12:33:12 CEST
 
 Hi, I don't think that this bug is resolved see my last post https://bugs.mageia.org/show_bug.cgi?id=1148#c10

Status: RESOLVED => REOPENED
Resolution: FIXED => (none)

Comment 12 Manuel Hiebel 2011-10-02 00:36:09 CEST
This bug is still valid I think ?

CC: (none) => thierry.vignaud
Source RPM: drakx-finish-install => drakxtools

Comment 13 Thierry Vignaud 2011-10-04 12:08:43 CEST
I think there are several issues there:

- not checking if a user has actually been created
   any::ask_user_and_root(..)  OR RETURN

- assuming the renamed user (aka the newly created/asked one)
  is the first one whereas it will actually be the last

So we should:
- either do nothing if no user has actually been created
- use the last user as the renamed one
- or use some better heuristic (such as empty or nearly empty homedir)
Comment 14 Thierry Vignaud 2011-10-04 12:10:00 CEST
Created attachment 891 [details]
(untested) example of better heuristik
Comment 15 Thierry Vignaud 2011-10-04 12:10:43 CEST
Not critical though

Status: REOPENED => ASSIGNED
Severity: critical => normal

Comment 16 Thierry Vignaud 2011-11-17 12:32:43 CET
*** Bug 1599 has been marked as a duplicate of this bug. ***

CC: (none) => pmithrandir

Comment 17 Pierre Bonneau 2011-11-17 13:55:06 CET
Thanks for the duplicate entry, I didn't found this issue.

For me it's critical because you can arrive at a point where your system freeze during installation, just by using the same username.(very common)

A guess, is it possible to reduce the number of the files in the grep ? All jpg, mp4, mp3, avi files are examined in your command I think... and it's definitly not usefull to do a grep on such files. 

A question, do you replace the files from the home directory(/home/jcl) by the files from /home/live directory during installation ? It would make a lot of sence to reset the environment and to have all users with the same initial config. I don't say to remove files, but to copy and replace during the installation process. 
It would be easier for the developper to works on people with the same env, and a way for the user to reset his config, without loosing all the home partition.
(It was my first idea when I reinstall my mageia, reset my broken KDE4 plasmoid config easily, an installation process takes 20-25 minutes when looking for a bug takes hours...)
Thierry Vignaud 2012-04-18 17:07:57 CEST

Keywords: (none) => PATCH
Summary: /usr/sbin/finish-install takes too long or may hang at first reboot => finish-install takes too long or may hang at first reboot

Comment 18 Marja Van Waes 2012-05-26 13:02:02 CEST
Hi,

This bug was filed against cauldron, but we do not have cauldron at the moment.

Please report whether this bug is still valid for Mageia 2.

Thanks :)

Cheers,
marja

Keywords: (none) => NEEDINFO

Comment 19 Manuel Hiebel 2013-02-21 14:47:53 CET
*** Bug 9098 has been marked as a duplicate of this bug. ***

CC: (none) => joao.m.santos.silva

Manuel Hiebel 2013-02-21 14:48:09 CET

Keywords: NEEDINFO => (none)

Comment 20 Dick Gevers 2014-11-15 06:51:41 CET
@all: still valid to be kept or closed as old? thanks
João Silva 2014-11-15 11:55:06 CET

CC: joao.m.santos.silva => (none)

Thierry Vignaud 2015-02-11 09:17:52 CET

CC: (none) => tmb

Thierry Vignaud 2015-06-02 15:06:00 CEST

Component: Installer => RPM Packages