Bug 8647

Summary: systemd is not starting LSB initscripts automatically
Product: Mageia Reporter: Frank Griffin <ftg>
Component: RPM PackagesAssignee: Mageia Bug Squad <bugsquad>
Status: RESOLVED WONTFIX QA Contact:
Severity: normal    
Priority: Normal CC: mageia
Version: Cauldron   
Target Milestone: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Source RPM: systemd CVE:
Status comment:
Attachments: ftgpub1 initscript

Description Frank Griffin 2013-01-10 21:17:31 CET
I have three pre-systemd LSB initscripts.  I'll use the one called "ftgpub1" here.  /etc/rc.d has a symlink from "ftgpub1" to "/home/ftg/bin/ftgpub1", which contains:

#!/bin/sh

#
#  This is an initscript for the ftgpub1 Virtual Machine.
#


# Comments to support chkconfig on Mandriva Linux
# chkconfig: - 11 90
# description: A public virtual server

# Comments to support LSB init script conventions
### BEGIN INIT INFO
# Provides: ftgpub1
# Required-Start: $local_fs $network $remote_fs
# Should-Start: ypbind nscd ldap ntpd
# Required-Stop: $local_fs $network $remote_fs
# Default-Start:  3 4 5
# Default-Stop: 0 1 6
# Short-Description: start and stop ftgpub1
# Description: A public virtual server           
### END INIT INFO

plus other stuff.

I issue:
     
[root@ftgme2 init.d]# systemctl enable ftgpub1.service
ftgpub1.service is not a native service, redirecting to /sbin/chkconfig.
Executing /sbin/chkconfig ftgpub1 on

and reboot, but ftgpub1 does not start, and there is no reference to it in syslog or journalctl.

Checking the status:

[root@ftgme2 init.d]# systemctl status ftgpub1.service
ftgpub1.service - LSB: start and stop ftgpub1
          Loaded: loaded (/etc/rc.d/init.d/ftgpub1)
          Active: inactive (dead)
          CGroup: name=systemd:/system/ftgpub1.service

However:

[root@ftgme2 init.d]# systemctl start ftgpub1.service
[root@ftgme2 init.d]# 

and the server starts perfectly.

Is there some additional package needed to get systemd to support chkconfig ?
Comment 1 Frank Griffin 2013-01-10 21:20:53 CET
Follow-up:

After starting it manually:

[root@ftgme2 init.d]# systemctl status ftgpub1
ftgpub1.service - LSB: start and stop ftgpub1
          Loaded: loaded (/etc/rc.d/init.d/ftgpub1)
          Active: active (exited) since Thu, 2013-01-10 15:14:36 EST; 4min 28s ago
         Process: 24513 ExecStart=/etc/rc.d/init.d/ftgpub1 start (code=exited, status=0/SUCCESS)
          CGroup: name=systemd:/system/ftgpub1.service

Jan 10 15:14:36 ftgme2 su[24522]: (to ftg) root on none
Jan 10 15:14:36 ftgme2 ftgpub1[24513]: [35B blob data]
Jan 10 15:14:36 ftgme2 systemd[1]: Started LSB: start and stop ftgpub1.
Manuel Hiebel 2013-01-11 23:17:35 CET

CC: (none) => mageia

Comment 2 Colin Guthrie 2013-01-12 13:51:54 CET
Does systemctl enable command (and by extension the chkconfig command it executes create a symlink in /etc/rc.d/rc{3,4,5}.d/ appropriately?

You may need to adjust your script to ensure it has level information on the chkconfig line. e.g. change:

# Comments to support chkconfig on Mandriva Linux
# chkconfig: - 11 90
# description: A public virtual server

To:

# Comments to support chkconfig on Mandriva Linux
# chkconfig: 345 11 90
# description: A public virtual server


The LSB headers are needed for real ordering information but the chkconfig line is still (AFAIK) still needed for executing chkconfig without any explicit level information (i.e. I don't think chkconfig looks at the Default-Start LSB directive if it has it's own chkconfig: line (could be wrong tho')

Status: NEW => ASSIGNED

Comment 3 Frank Griffin 2013-01-14 21:15:50 CET
(In reply to comment #2)
> Does systemctl enable command (and by extension the chkconfig command it
> executes create a symlink in /etc/rc.d/rc{3,4,5}.d/ appropriately?

Yep.

> 
> You may need to adjust your script to ensure it has level information on the
> chkconfig line. e.g. change:
> 
> # Comments to support chkconfig on Mandriva Linux
> # chkconfig: - 11 90
> # description: A public virtual server
> 
> To:
> 
> # Comments to support chkconfig on Mandriva Linux
> # chkconfig: 345 11 90
> # description: A public virtual server
> 

No effect.
Comment 4 Colin Guthrie 2013-01-15 10:52:49 CET
Ahh OK, interesting.

The systemctl output doesn't show enabled vs. disabled for sysvinit scripts.

If it is enabled (i.e. has a symlink in /etc/rc[345].d/S??ftgpub1) then it should be started by systemd on boot.

Can you attach the initscript. I really don't see any reason why it's not started.
Comment 5 Frank Griffin 2013-01-15 13:29:02 CET
Created attachment 3366 [details]
ftgpub1 initscript

I had changed the startup 11 to 51 thinking that it might be starting too early, but that had no effect either.
Comment 6 Colin Guthrie 2013-01-27 16:15:26 CET
Note: the $network-up bit in the initscript is wrong.

$'s indicate approximately the same as targets in an LSB world. That's why you see things like $network or $local_fs, $remote_fs and $named (equiv to network.target, local-fs.target, remote-fs.target and nss-lookup.target in systemd land).

You should simply change that requirement to $network. If you have network-up.service enabled then $network will not be considered "started" until network-up.service is complete. You could also just call it "network-up" without the $, but you should also be explicit and say $network. Also saying $remote_fs is required seems a little too hard on things (although these "requires" in LSB are actually just translated to ordering info in systemd, the "requirement" bit is not actually used - as sysvinit consumers of this info didn't do this).

Anyway, that shouldn't affect things (as it translates to "After=network-up.target" which should basically be a noop).

FWIW, dropping that attachement into my /etc/init.d/ folder, running "systemctl daemon-reload; systemctl enable ftgpub1.service" did indeed create the /etc/rc[345].d/S??ftgpub1 as expected.

Can you confirm that those symlinks appear/disappear as you enabled/disable the service?
Comment 7 Frank Griffin 2013-03-07 13:44:19 CET
(In reply to Colin Guthrie from comment #6)
> Note: the $network-up bit in the initscript is wrong.

Fixed.

> 
> FWIW, dropping that attachement into my /etc/init.d/ folder, running
> "systemctl daemon-reload; systemctl enable ftgpub1.service" did indeed
> create the /etc/rc[345].d/S??ftgpub1 as expected.
> 
> Can you confirm that those symlinks appear/disappear as you enabled/disable
> the service?

Sorry for the delay.  I was traveling for a few weeks, and lost track of this.  Yes, I can confirm that they appear/disappear as I enable/disable the services.
Comment 8 Frank Griffin 2013-03-12 15:20:39 CET
There has to be something here that needs doing other than supplying the LSB initscript and doing systemctl enable, because the virtualbox script gets run perfectly, while mine doesn't.

journalctl -a -b has absolutely no mention of my script/service - no failures or attempts to run it.  I have exactly the same ownership and permissions as the virtualbox script.

I thought, even though virtualbox is an S30 and I'm an S51, maybe the vboxdrv driver hadn't been loaded by the time mine started, so I added vboxdrv (provided by virtualbox) to the Required-Start, but no joy there either.

This isn't just happening with VM scripts, either.  I have two others, which run derby and jboss, which have the same problem.

My current testing is in a repeated fresh install for a firewall system which is supposed to start a VM to run a DNS server.  This is more of a bare-bones system than my desktop installs, i. e. the MGA package groups plus a handful of server packages like VB, bind, wireshark...  It's also current cauldron on a day-by-day basis.
Comment 9 Frank Griffin 2013-03-12 16:15:26 CET
Another interesting thing is that if I manually start the service and then reboot, systemctl status shows that systemd knew enough to *stop* the service during shutdown, e. g. 

[root@ftgme2 ftg]# systemctl status ftgpub1
ftgpub1.service - LSB: start and stop ftgpub1
          Loaded: loaded (/etc/rc.d/init.d/ftgpub1)
          Active: inactive (dead)
          CGroup: name=systemd:/system/ftgpub1.service

Feb 27 14:53:06 ftgme2 systemd[1]: Started LSB: start and stop ftgpub1.
Mar 07 09:34:33 ftgme2 ftgpub1[21532]: Shutting down ftgpub1: [  OK  ]
Mar 07 09:38:43 ftgme2 su[27158]: (to ftg) root on none
Mar 07 09:38:43 ftgme2 ftgpub1[27152]: [35B blob data]
Mar 07 09:38:43 ftgme2 systemd[1]: Started LSB: start and stop ftgpub1.
Mar 08 09:22:05 ftgme2 systemd[1]: Stopping LSB: start and stop ftgpub1...
Mar 08 09:22:05 ftgme2 ftgpub1[2436]: Shutting down ftgpub1: kill: usage: k...c]
Mar 08 09:22:05 ftgme2 ftgpub1[2436]: [  OK  ]
Mar 08 09:22:06 ftgme2 systemd[1]: Stopped LSB: start and stop ftgpub1.
Comment 10 Colin Guthrie 2013-03-12 16:47:37 CET
Hmmm, I'm wondering if this is actually a product of using daemon --user.... Perhaps this is spawning a new login session which escapes the cgroup of systemd which then makes thing *appear* as if it's failed. In actual fact things have worked but the process then escapes it's service cgroup and is then lost by systemd. That said, I'm not sure that's a good theory as I'm sure you'd notice this and I'm also sure you'd expect to see something more about it not being able to start it.

I guess you'll just need to boot with some appropriate kernel command line to get some debug out of this.

systemd.log_level=debug

This should hopefully put things into journal that you can use to work out if the service is even enumerated.

Interestingly, I see this:

Mar 12 09:18:56 jimmy systemd[1]: access() failed on /etc/rc.d/rc1.d/K90ftgpub1: Permission denied
Mar 12 09:18:56 jimmy systemd[1]: access() failed on /etc/rc.d/rc2.d/S51ftgpub1: Permission denied
Mar 12 09:18:56 jimmy systemd[1]: access() failed on /etc/rc.d/rc3.d/S51ftgpub1: Permission denied
Mar 12 09:18:56 jimmy systemd[1]: access() failed on /etc/rc.d/rc4.d/S51ftgpub1: Permission denied
Mar 12 09:18:56 jimmy systemd[1]: access() failed on /etc/rc.d/rc5.d/S51ftgpub1: Permission denied
Mar 12 09:18:56 jimmy systemd[1]: access() failed on /etc/rc.d/rc0.d/K90ftgpub1: Permission denied
Mar 12 09:18:56 jimmy systemd[1]: access() failed on /etc/rc.d/rc6.d/K90ftgpub1: Permission denied

Although I'm not 100% sure why... Perhaps I had dangling symlinks? I've since run systemctl enable on it so they all exist... Hmm, perhaps it's simply because it's not executable? Perhaps that could be my problem.... chmod'ed it now and will check on next reboot.


Also note that if LSB headers are provided the numerical ordering stuff (30 vs 51) is totally irrelevant. It's only used for ordering when no LSB dependency data is supplied (and even then it can easily lead to deadlocks as the numerical ordering stuff is so hideously broken).
Comment 11 Frank Griffin 2013-03-12 17:44:46 CET
Turned on debug, but my service name doesn't appear in the resulting log (although a lot of other debug stuff does).

I may just give up and write a service file.  Question - what do I put on [Unit] After to indicate that the service requires something provided by a legacy initscript, such as vboxdrv provided by virtualbox ?  "vboxdrv.target" ?  "virtualbox.service" ?  Or something else ?
Comment 12 Colin Guthrie 2013-03-12 18:04:46 CET
I'll give you it's an odd one!!

Is vboxdrv a service or initscript?

AFACT there is /etc/init.d/virtualbox, so yes, "After=virtualbox.service" and "Requires=virtualbox.service" (the two are separate). You should likely also put After=network.target (you can combine the two: "After=network.target virtualbox.service") too.

You can also use the User=ftg to set the user properly.

If you want to do your little while loop it should also work, but you'll need to put it in ExecStartPre= and call it via /usr/bin/sh -c 'xxxxxxxxxxx'

HTHs
Comment 13 Frank Griffin 2013-03-12 18:57:30 CET
The following works like a charm:

[Unit]
Description=VirtualBox VM for FTGSRV01
After=syslog.target network.target nss-lookup.target virtualbox.service
Requires=virtualbox.service

[Service]
Type=simple
User=ftg
ExecStart=/usr/bin/VBoxHeadless -s FTGSRV01
ExecStop=/bin/kill -INT $MAINPID
TimeoutSec=30
Restart=on-failure
LimitNOFILE=4096

[Install]
WantedBy=multi-user.target

The "while" business is copied from elsewhere, and is only needed for one of the other examples (jboss).

Technically, I suppose this bug is still open, but given how much simpler the service file is, I'm just going to convert all of my stuff to that.

Thanks for the help !
Comment 14 Colin Guthrie 2013-03-13 10:15:08 CET
No problem.

FWIW, the same service file you attached here:

[root@jimmy ~]# systemctl status ftgpub1.service 
ftgpub1.service - LSB: start and stop ftgpub1
	  Loaded: loaded (/etc/rc.d/init.d/ftgpub1)
	  Active: active (exited) since Tue, 2013-03-12 20:33:42 GMT; 12h ago
	 Process: 2545 ExecStart=/etc/rc.d/init.d/ftgpub1 start (code=exited, status=0/SUCCESS)
	  CGroup: name=systemd:/system/ftgpub1.service

Mar 12 20:33:42 jimmy systemd[1]: Starting LSB: start and stop ftgpub1...
Mar 12 20:33:42 jimmy ftgpub1[2545]: Starting ftgpub1: su: user ftg does no...st
Mar 12 20:33:42 jimmy ftgpub1[2545]: [17B blob data]
Mar 12 20:33:42 jimmy systemd[1]: ftgpub1.service changed start -> exited
Mar 12 20:33:42 jimmy systemd[1]: Started LSB: start and stop ftgpub1.


Starts fine on my machine (that was on boot yesterday evening).

Obviously it fails as I don't have the user in question, but it is at least trying to start.

I suspect there is something really trivial wrong - perhaps a name clash or some weird conflict that arises from some other service (e.g. maybe two sysvinit scripts both have the "Provides: ftgpub1" line but on my machine they don't), but trying to find that "ahhhhh" moment is likely more trouble than it's worth, especially, as you say, when the native unit is simpler and works fine :)

FWIW, you can drop the After=syslog.target bit these days (at least on cauldron) as it's no longer needed.

Think I'll close this as I don't think either of us will be able (or that interested) in doing much more sleuthing!!)

All the best.

Status: ASSIGNED => RESOLVED
Resolution: (none) => WONTFIX