Bug 4475 - 2_a3 and other cauldron: shutdown hangs
Summary: 2_a3 and other cauldron: shutdown hangs
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: Cauldron
Hardware: All Linux
Priority: Normal major
Target Milestone: ---
Assignee: D Morgan
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 2120
  Show dependency treegraph
 
Reported: 2012-02-11 08:48 CET by Marja Van Waes
Modified: 2012-03-13 23:46 CET (History)
11 users (show)

See Also:
Source RPM: systemd-40-2.mga2.src.rpm
CVE:
Status comment:


Attachments
blkid | sort -V > blkid.txt (1.23 KB, text/plain)
2012-02-13 11:16 CET, Bit Twister
Details
fstab.txt (1.79 KB, text/plain)
2012-02-13 11:16 CET, Bit Twister
Details

Description Marja Van Waes 2012-02-11 08:48:49 CET
All 3 of my cauldrons hang at shutdown since I updated them thursday and friday. I don't use them regurlarly, last updates before that were past weekend for an 2_a3, longer ago for the others (2_a3 and an updated-to-cauldron 1 or 2_a1)

All 3 of them gave the same behaviour: shutdown hung after messages about ipv6 and ipv4, (something with shorewall?) but last night when I looked again to be able to quote the exact messages, the behaviour had changed again: shutdown gave no messages at all, just a black screen with a white dash in the left upper corner.

Using the magic keys works fine, the screen responds to every combination and it is possible to shutdown nicely with them.

Two persons confirmed the problem, but I forgot to ask whether they're using 32 or 64 bits,
Comment 1 Kamil Rytarowski 2012-02-11 09:13:51 CET
I can confirm it.
Cauldron, Alpha3 with the latest updates, x86_64 here.

It's stopping on the "System halted" and I have to make hard reset.

CC: (none) => n54

Marja Van Waes 2012-02-11 09:15:39 CET

Hardware: i586 => All

Comment 2 Bit Twister 2012-02-11 10:45:01 CET
After last round of updates, Ctrl+Delete cleans screen, shows no activity, takes minutes to finally display
Sending SIGTERM to remaining processes

This is a runlevel 3 default boot with splash=silent removed from boot line.

title cauldron_2_64_alpha3
kernel (hd0,8)/boot/vmlinuz BOOT_IMAGE=cauldron_2_64_alpha3 root=LABEL=cauldron pci=use_crs hpet=force vga=0x0324
initrd (hd0,8)/boot/initrd.img

Source RPM: (none) => systemd-40-2.mga2.src.rpm
CC: (none) => junk_no_spam
Blocks: (none) => 2120

Comment 3 Marja Van Waes 2012-02-11 11:01:05 CET
Haven't found time to check which updates I got, but I thought systemd couldn't be among them, because Colin was on holiday.

Because DMorgan wasn't on holiday and because an update of another package might have triggered an existing systemd bug to become visible and because systemd maintainers will assign back when their package isn't the culprit, I'll assign to them anyway :)

@ D Morgan 
@ Colin

Please assign back (and explain) if it was wrong to assign to you. Please set status to ASSIGNED if it was correct

CC: (none) => mageia
Assignee: bugsquad => dmorganec

Comment 4 Gerardo Bueno 2012-02-11 14:34:25 CET
I have the same bug as Bit Twister comment.

My cauldron alpha 3 shows no activity for a few minutes and later It starts to stop and kill processes.

I have cauldron 32 bits.

CC: (none) => gejobj

Comment 5 Marja Van Waes 2012-02-11 14:48:49 CET
I was wrong to think systemd wasn't among the updates, it was
Comment 6 Tico Perez 2012-02-13 00:39:18 CET
I have been suffering this for a while too, but it doesn't happen always.

CC: (none) => entidad_universal

Comment 7 Colin Guthrie 2012-02-13 10:18:58 CET
I *think* this could be due to certain storage daemons (i.e. LVM etc.) these daemons should be excluded from a killing spree issued by systemd, but cannot be if the root fs is used amoung them.

Lennart has talked about this recently and introduced a special scheme that would let special storage daemons started from the initrd indicate to systemd that they should not be killed and it is actually the job of the initrd (when pivoted back in during shutdown) to stop these applications.

This is a bit of a guess, but it seems like it could be related.

Can you describe what kind of file systems you have (if anyone suffering from the problem only has e.g. pure ext3 then it's likely not this issue).
Comment 8 Bit Twister 2012-02-13 11:15:00 CET
(In reply to comment #7)

 Lots of theory snipped, but why problem showed up sometime after systemd.36 or greater.  :(

> Can you describe what kind of file systems you have (if anyone suffering from
> the problem only has e.g. pure ext3 then it's likely not this issue).

How about the default ext4, which does not address the problem shows up after version 36. :-(
Comment 9 Bit Twister 2012-02-13 11:16:02 CET
Created attachment 1539 [details]
blkid | sort -V > blkid.txt
Comment 10 Bit Twister 2012-02-13 11:16:44 CET
Created attachment 1540 [details]
fstab.txt
Comment 11 Marja Van Waes 2012-02-13 17:58:42 CET
probably all 3 systems ext4, I'll check.

when i wait patiently, the systems do shutdown by themselves

sometimes I see messages, when I do, the system *always* hangs after:

Stopped ip6tables Firewall for IPv6
Stopped iptables Firewall for IPv4


The first message when shutdown continues, is something with "speech" in it, IIRC, I'll go check now
Comment 12 Marja Van Waes 2012-02-13 18:22:21 CET
the 2 Mga2a3 systems have ext4, still need to check a third cauldron.

Shutting down 6 times didn't make those messages visible this, but I'm very sure now it was something about speech dispatcher that appeared as first message after the long hang
Comment 13 Bit Twister 2012-02-13 19:16:59 CET
(In reply to comment #12)
>> Shutting down 6 times didn't make those messages visible this, but I'm very
> sure now it was something about speech dispatcher that appeared as first
> message after the long hang

If that were true, you would think a
  systemctl stop speech-dispatcherd.service
  systemctl disable speech-dispatcherd.service
would improve your future shutdown time.
So far I have numerous services disabled and still have the problem.
Comment 14 Marja Van Waes 2012-02-13 19:45:05 CET
@ Bill

Thx, I'll try that. :)

Just checked my 3rd system, both / and /home are ext3 (according to diskdrake on another partition)
Comment 15 Marja Van Waes 2012-02-13 20:56:56 CET
(In reply to comment #13)

> 
> If that were true, you would think a
>   systemctl stop speech-dispatcherd.service
>   systemctl disable speech-dispatcherd.service
> would improve your future shutdown time.
> So far I have numerous services disabled and still have the problem.

I really expected it to help a great deal, because it took 1 1/2 minutes to stop that service, but it doesn't seem to make much difference (rebooting took 4 1/2 minutes, just like before, maybe it went 10 seconds faster now)
Comment 16 Dave Hodgins 2012-02-14 02:14:16 CET
(In reply to comment #15)
> (In reply to comment #13)
> 
> > 
> > If that were true, you would think a
> >   systemctl stop speech-dispatcherd.service
> >   systemctl disable speech-dispatcherd.service
> > would improve your future shutdown time.
> > So far I have numerous services disabled and still have the problem.
> 
> I really expected it to help a great deal, because it took 1 1/2 minutes to
> stop that service, but it doesn't seem to make much difference (rebooting took
> 4 1/2 minutes, just like before, maybe it went 10 seconds faster now)

Can you check /etc/sysconfig/network-scripts/ifdown.d/vnstat_ip-down
and ensure the line
service vnstat force-reload
is commented out?

CC: (none) => davidwhodgins

Comment 17 Bit Twister 2012-02-14 03:09:31 CET
(In reply to comment #16)

> Can you check /etc/sysconfig/network-scripts/ifdown.d/vnstat_ip-down
> and ensure the line
> service vnstat force-reload
> is commented out?

Better than commented out on my system.
# cat /etc/sysconfig/network-scripts/ifdown.d/vnstat_ip-down
cat: /etc/sysconfig/network-scripts/ifdown.d/vnstat_ip-down: No such file or directory

[root@wb2 ~]# cd /etc/sysconfig/network-scripts/ifdown.d/
[root@wb2 ifdown.d]# dir
total 12
drwxr-xr-x 2 root root 4096 Feb  1 15:47 .
drwxr-xr-x 8 root root 4096 Feb  1 16:20 ..
-rwxr-xr-x 1 root root  224 Jan 23 13:21 vpn

FYI: clean alpha3-x86_64-DVD install + updates.
Comment 18 Marja Van Waes 2012-02-14 10:24:16 CET
same here, no such file (clean alpha3 i586 DVD install + updates) and ls -al gives the same results as above
Comment 19 Frédéric "LpSolit" Buclin 2012-02-14 13:15:57 CET
Pasting what I wrote in bug 4004 comment 4:

Booting Cauldron is terribly slow. And shutting down Cauldron is even slower. It takes ages before it decides to call SIGTERM on all remaining processes (which makes me wonder if it really asked processes to stop!).


Increasing the severity as this bug is a good candidate to throw the machine through the window. ;)

Severity: normal => major
CC: (none) => LpSolit

Comment 20 Marja Van Waes 2012-02-15 15:22:25 CET
(In reply to comment #19)
> Pasting what I wrote in bug 4004 comment 4:
> 
> Booting Cauldron is terribly slow. And shutting down Cauldron is even slower.

I agree booting is very slow, too. I wasn't really aware it was that bad (seeing text scrolling over the screen makes the time seem to be shorter) until I started clocking it. Booting into KDE until everything is started and an application responds when clicking on it, takes 2 1/2 minutes
Comment 21 dennis drown 2012-02-18 14:13:31 CET
i too had this problem until this morning 02/18/12

i installed "systemd-sysvinit"  through urpmi,  just trying something i saw on a mandriva forum post

when i exited and did a restart/shutdown the system shutdown/restarted  in like 5 seconds

hope this helps someone, it works for me with the latest up to date cauldron

CC: (none) => dadrown1

Comment 22 Frédéric "LpSolit" Buclin 2012-02-18 14:56:10 CET
(In reply to comment #21)
> i installed "systemd-sysvinit"  through urpmi

I tried that right now, and I see no improvement at all, neither when booting nor shutting down. :(
Comment 23 dennis drown 2012-02-18 15:44:13 CET
ok my mistake i installed sysvinit and it looks like systemd-sysvinit was removed

after that shutdown did not hang

Feb 18 06:00:11 localhost urpmi: called with: sysvinit
Feb 18 06:00:15 localhost urpmi: transaction on / (remove=1, install=0, upgrade=1)
Feb 18 06:00:16 localhost perl: [RPM] sysvinit-2.87-13.mga2.x86_64 installed
Feb 18 06:00:17 localhost perl: [RPM] systemd-sysvinit-40-3.mga2.x86_64 removed
Comment 24 Colin Guthrie 2012-02-18 15:54:00 CET
(In reply to comment #23)
> ok my mistake i installed sysvinit and it looks like systemd-sysvinit was
> removed
> 
> after that shutdown did not hang

Well clearly if you use a completely different init system, a bug/misconfiguration the other init system will not be present any more... It's kinda like saying, "I don't get the bug on windows" :)

At present I'm not 100% sure which unit it is that is holding things up at shutdown. My current suspect is udisks, but I've just not had time to look into this properly to see if that was just a red herring.
William Kenney 2012-02-21 16:57:23 CET

CC: (none) => wilcal.int

Comment 25 Morgan Leijström 2012-02-21 22:31:50 CET
On my sons old Thinkpad T40 I installed cauldron in the beginning of this month.  Not much used but updated now and then, and it have never been able to do a full shutdown. (And because of that also hibernate and suspend are broken.) "everything" else seem to work, and i was pleasantly surprised KDE4 can be really nice on this machine :)

When i select to shut down from within KDE, it ends in login screen.
Shutting down from there screen goes black in a few seconds, and it is unresponsive.  (Have not waited more than a couple minutes, will try longer...)

And yes it is pretty slow booting too.

Filesystem are all ext4 on LVM (except /boot) and it mounts two ntfs.

Are there some interesting log files we can read next boot about what happened during shutdown?

CC: (none) => fri

Comment 26 Morgan Leijström 2012-02-24 00:12:28 CET
For some reason i need not wait very long now; after a minute of blackness there comes "Sending SIGTERM to remaining processes" and it shuts down OK.
Comment 27 Rémi Verschelde 2012-02-24 09:38:47 CET
Experience the same issue as Morgan. On shutdown I get a black screen for about 1 min minute, and then SIGTERM and SIGKILL arrive.

I don't know if I had 4 minutes of black screen like Marja did before... if that's the case I would have probably thought something was broken and halted the system manually (using the familiar Power button).

CC: (none) => remi

Comment 28 Colin Guthrie 2012-02-24 12:00:56 CET
Seems to be better for me with systemd 43. It's making it's way slowly through the build system now (man page changes seem to require more stuff now).

When it's there can people test and report their experience?
Comment 29 Morgan Leijström 2012-02-24 13:02:04 CET
Sure :)  Bump here when it is in repo.
Comment 30 Marja Van Waes 2012-02-24 13:10:08 CET
it has just been uploaded http://pkgsubmit.mageia.org/
Comment 31 Marja Van Waes 2012-02-24 13:30:52 CET
No hang at all :)

Immediately after installing systemd43, rebooting from the konsole, the shutdown phase only took 10, maybe 12 seconds. Booting still took long, but this bug isn't about booting.

When using the shutdown button in KDE, shutdown took only 26 seconds even though I had the detour to KDM inlog screen where I had to choose shutdown again
Comment 32 Morgan Leijström 2012-02-24 14:29:01 CET
Now it was alreday @ mirror ftp.acc.umu.se

Great - that fix works, thank you Colin :)

@Marja: change to GDM, see bug 4605
Comment 33 Colin Guthrie 2012-02-24 14:33:47 CET
OK, so two (well three including me) confirmations of things working better now with newer systemd, so lets close this bug.

It can always be reopened if someone finds a problem.

We do still need to fix bug 4605 tho'. I'll try and take a look at some point but not sure when.

Status: NEW => RESOLVED
Resolution: (none) => FIXED

Comment 34 Frédéric "LpSolit" Buclin 2012-02-24 14:59:53 CET
I don't see the improvement you are talking about with systemd 43. It still very slow here (using Cauldron in a VirtualBox VM).
Comment 35 Bit Twister 2012-02-24 15:06:40 CET
(In reply to comment #34)
> I don't see the improvement you are talking about with systemd 43. It still
> very slow here (using Cauldron in a VirtualBox VM).

I tested on 64 bit host with 64 bit guest and on another system.
It really boots/shutdown really fast.

Problem in VB guest is /etc/resolv.conf has no nameserver line so any dns lookups fail.  :-(

Workaround   
echo nameserver 208.67.222.222 >> /etc/resolv.conf
Comment 36 Frédéric "LpSolit" Buclin 2012-02-24 15:41:00 CET
(In reply to comment #35)
> Problem in VB guest is /etc/resolv.conf has no nameserver line so any dns
> lookups fail.  :-(

The guest's resolv.conf file already has the same content as my host file, so this doesn't seem to be the problem.
Comment 37 Bit Twister 2012-02-24 16:16:53 CET
(In reply to comment #36)

> The guest's resolv.conf file already has the same content as my host file, so
> this doesn't seem to be the problem.


Those files should not look alike. :(

resolv.conf should have a nameserver ip.addy.here line.  Example:
nameserver 208.67.222.222

/etc/hosts example:
127.0.0.1 localhost
127.0.0.1 localhost.localdomain
192.168.1.100 wb2.home.test wb2
192.168.1.131 wm81.home.test wm81
Comment 38 Colin Guthrie 2012-02-24 16:23:14 CET
(In reply to comment #37)
> (In reply to comment #36)
> 
> > The guest's resolv.conf file already has the same content as my host file, so
> > this doesn't seem to be the problem.
> 
> 
> Those files should not look alike. :(

I think he means the VM host's resolv.conf, not his /etc/hosts file.
Comment 39 Frédéric "LpSolit" Buclin 2012-02-24 17:10:49 CET
(In reply to comment #38)
> I think he means the VM host's resolv.conf, not his /etc/hosts file.

Right. :) I know /etc/hosts and /etc/resolv.conf must be different. :)
Comment 40 Frédéric "LpSolit" Buclin 2012-03-13 23:46:42 CET
(In reply to comment #34)
> I don't see the improvement you are talking about with systemd 43. It still
> very slow here (using Cauldron in a VirtualBox VM).

FYI, I found the culprit: Oracle Express! Configuring it to not load its DB on boot highly improved both the boot time (20-30 seconds faster) and the shutdown time (from 1-2 minutes down to ~10 seconds).

Note You need to log in before you can comment on or make changes to this bug.