Bug 9579

After recent updates this weekend httpd was running OK as it has for months on my server.
I updated two days ago and re-booted OK.
To-night while testing I re-started httpd - only it didn't start again.
I ran --auto-update just to be sure that I didn't have a partial update, but no change.
journal reports:-

Mar 31 21:48:52 zmhost systemd[1]: Starting The Apache HTTP Server...
Mar 31 21:48:53 zmhost kernel: httpd[2946]: segfault at 1c ip 00007fa8034f2246 sp 00007fff2ed6be80 error 4 in libapr-1.so.0.4
Mar 31 21:48:53 zmhost systemd[1]: httpd.service: main process exited, code=killed, status=11/SEGV
Mar 31 21:48:54 zmhost kernel: httpd[2948]: segfault at 1c ip 00007f0e4a318246 sp 00007fff85590810 error 4 in libapr-1.so.0.4
Mar 31 21:48:54 zmhost systemd[1]: httpd.service: control process exited, code=killed status=11
Mar 31 21:48:54 zmhost systemd[1]: Failed to start The Apache HTTP Server.
Mar 31 21:48:54 zmhost systemd[1]: Unit httpd.service entered failed state

I have sshd access to the machine so I can give any requested info.

Priority: Normal => release_blocker
Hardware: i586 => x86_64
Severity: normal => critical

Apache packages installed:-

apache-mod_php-5.4.13-2.mga3
apache-2.4.4-4.mga3
apache-mod_perl-2.0.7-12.20130221svn1448242.mga3
apache-mod_ssl-2.4.4-4.mga3
apache-mod_fcgid-2.3.7-2.mga3

libapr packages installed:-

lib64apr-util1_0-1.5.1-3.mga3
lib64apr1_0-1.4.6-4.mga3

Assignee: bugsquad => guillomovitch
Source RPM: (none) => apache-2.4.4-4.mga3

here if I remove apache-mod_perl, it's work.
So I think that the problem is on apache-mode_perl not on apache

CC: (none) => bertauxx

I removed apache-mod_perl and no change here

[root@zmhost baz]# systemctl disable httpd.service
rm '/etc/systemd/system/multi-user.target.wants/httpd.service'
[root@zmhost baz]# systemctl enable httpd.service
ln -s '/usr/lib/systemd/system/httpd.service' '/etc/systemd/system/multi-user.target.wants/httpd.service'
[root@zmhost baz]# systemctl start httpd.service
Job for httpd.service failed. See 'systemctl status httpd.service' and 'journalctl -n' for details.
[root@zmhost baz]# systemctl status httpd.service
httpd.service - The Apache HTTP Server
          Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled)
          Active: failed (Result: signal) since Sun, 2013-03-31 22:47:26 BST; 6s ago
         Process: 3284 ExecStop=/usr/sbin/httpd $OPTIONS -k graceful-stop (code=killed, signal=SEGV)
         Process: 3282 ExecStart=/usr/sbin/httpd $OPTIONS -DFOREGROUND (code=killed, signal=SEGV)
          CGroup: name=systemd:/system/httpd.service

Mar 31 22:47:25 zmhost systemd[1]: Starting The Apache HTTP Server...
Mar 31 22:47:25 zmhost systemd[1]: httpd.service: main process exited, code=killed, status=11/SEGV
Mar 31 22:47:26 zmhost systemd[1]: Failed to start The Apache HTTP Server.
Mar 31 22:47:26 zmhost systemd[1]: Unit httpd.service entered failed state
[root@zmhost baz]# journalctl -n
-- Logs begin at Sat, 2013-01-19 11:30:02 GMT, end at Sat, 2013-01-19 11:30:02 GMT. --
Mar 31 22:44:40 zmhost systemd[1]: Unit httpd.service entered failed state
Mar 31 22:47:08 zmhost systemd[1]: Reloading.
Mar 31 22:47:19 zmhost systemd[1]: Reloading.
Mar 31 22:47:25 zmhost systemd[1]: Starting The Apache HTTP Server...
Mar 31 22:47:25 zmhost kernel: httpd[3282]: segfault at 1c ip 00007fdb800b3246 sp 00007fff79608f90 error 4 in libapr-1.so.0.4
Mar 31 22:47:25 zmhost systemd[1]: httpd.service: main process exited, code=killed, status=11/SEGV
Mar 31 22:47:26 zmhost kernel: httpd[3284]: segfault at 1c ip 00007fa7629f0246 sp 00007fff8d5016c0 error 4 in libapr-1.so.0.4
Mar 31 22:47:26 zmhost systemd[1]: httpd.service: control process exited, code=killed status=11
Mar 31 22:47:26 zmhost systemd[1]: Failed to start The Apache HTTP Server.
Mar 31 22:47:26 zmhost systemd[1]: Unit httpd.service entered failed state

strange I retested with reinstall apache-mod_perl, apache don't start.
Removed mod_perl apache start.

(In reply to Xavier BERTAUX from comment #5)
> strange I retested with reinstall apache-mod_perl, apache don't start.
> Removed mod_perl apache start.

Can you confirm that you started with this problem after last days updates in cauldron and that you are seeing the same errors?

Created attachment 3678 [details]
backtrace

Backtrace requested by colin

Created attachment 3679 [details]
Potential fix

So the problem seems to be in apache/server/util.c where it calls apr_sockaddr_info_get() and receives APR_SUCCESS, but apparently gets a NULL sockaddr.

Looking in the apr code, and tracing apr_sockaddr_info_get() through it's calls, it appears it's possible to get this routing to return APR_SUCCESS without ever setting a value to the sockaddr.

The the attached patch certainly solves one potential problem, but not sure it's the one here...

CC: (none) => mageia

Created attachment 3680 [details]
backtrace after patched apr

OK, so ultimately this was caused by the hostname not resolving.

To avoid the problem one should either:
 1. Install nss-myhostname (recommended)
 2. Add the hostname to /etc/hosts
 3. Add a "search ..." line to resolv.conf if a fully qualified hostname already exists in /etc/hosts
 4. Set some magic setting in apache to avoid this error?


In the mean time, when the above is NOT done, apache should NOT crash. In order to solve that I wrote two patches to apr to correct return values being APR_SUCCESS when they actually failed to look things up. I will work to get them upstreamed. Further defence in the actual crashing function is advised also, this just corrects the return value which avoids triggering the code in apache.

More checks are likely desirable in apr to sanity check the input supplied to functions like apr_getnameinfo() which does not check if the sockaddr argument is non-null before dereferencing it.

Status: NEW => RESOLVED
Resolution: (none) => FIXED