Bug 16634

Summary:	rsyslog starts, runs, does not log into /var/log/(etc)
Product:	Mageia	Reporter:	jim l <jiml>
Component:	RPM Packages	Assignee:	Chris Denice <eatdirt>
Status:	RESOLVED WONTFIX	QA Contact:
Severity:	normal
Priority:	Normal	CC:	doktor5000, eatdirt, mageia, mageia, marja11, warrendiogenese
Version:	5
Target Milestone:	---
Hardware:	x86_64
OS:	Linux
See Also:	https://bugs.mageia.org/show_bug.cgi?id=16263 https://bugs.mageia.org/show_bug.cgi?id=20712
Whiteboard:
Source RPM:	rsyslog-8.4.2-6.mga5.src.rpm	CVE:
Status comment:

Description jim l 2015-08-23 20:39:21 CEST

Description of problem:  In Mageia 5, on boot, rsyslog apparently starts and runs correctly.  However, there are no new entries in the various /var/log files, notably including messages and auth.log.

This system contains one modification from stock, which conceivably is relevant; the file /lib/dracut/modules.d/30convertfs/convertfs.sh creates the symlinks /var/run -> ../run and /var/lock -> ../run/lock .  I have changed this to create the symlinks /var/run -> /run and /var/lock -> /run/lock because my /var is symlinked from an unencrypted / to a directory on an encrypted filesystem located on another partition, and the indicated relative paths break my system rather badly.  I submit they should be absolute paths anyway; this gives the desired behavior.

Nonetheless, rsyslog is not working until I take the following actions:

1.  systemctl stop rsyslog.service
2.  systemctl stop syslog.socket
3.  systemctl start syslog.socket
4.  systemctl start rsyslog.service

If I do systemctl status rsyslog and systemctl status syslog.socket, systemd tells me both are running even before I restart them; there is no apparent indication of a problem other than no entries in the log files.

I considered the possibility that volumes were getting mounted too late, hence the symlink to /var was not valid until too late, but the contents of /var/log/dmesg start with:
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 3.19.8-desktop-3.mga5 (iurt@ecosse.mageia.org) (gcc version 4.9.2 (GCC) ) #1 SMP Sat Jun 13 17:05:48 UTC 2015

which indicates that /var is up at a time that should be well ahead of rsyslog being started.

How reproducible:

On every boot

Steps to Reproduce:
1.boot system
2.check logs in /var/log
3.see nothing new in messages, auth.log, and others written by rsyslogd


Reproducible: 

Steps to Reproduce:

Marja Van Waes 2015-08-24 14:41:23 CEST

CC: sysadmin-bugs => mageia, marja11, warrendiogenese
Component: Release (media or process) => RPM Packages

Comment 1 Florian Hubold 2015-09-08 10:18:05 CEST

@Chris: IIRC you added some of the glue for forwarding journal messages to rsyslog, would be nice if you could take a look. Some more details also in the relevant forums thread: https://forums.mageia.org/en/viewtopic.php?f=7&t=10144
Thanks in advance.

CC: (none) => dirteat, doktor5000

Comment 2 Chris Denice 2015-09-14 02:16:03 CEST

yes sure, just wait a bit, I am off till the end of the month.

Cheers.

Comment 3 Florian Hubold 2015-09-17 22:41:15 CEST

No haste :)

FWIW, after Jim explicitly added ForwardToSyslog=yes (wasn't present before) it worked again. But this whole thing seems pretty flaky (see also e.g. https://bugs.mageia.org/show_bug.cgi?id=16263 ) and it should also be clearly documented or enforced if e.g. rsyslog-journald is required for forwarding or not.

See Also: (none) => https://bugs.mageia.org/show_bug.cgi?id=16263
Assignee: bugsquad => dirteat
Severity: major => normal

Comment 4 Chris Denice 2015-09-24 18:35:27 CEST

Ok, I have checked.
There is indeed a bug in the *upgrade* of the rsyslog package that affects only the transition from mga4 to mga5.

As Jim found by himself, the option ForwardToSyslog should be now set to yet in /etc/systemd/journald.conf to allow rsyslog (and other syslog) grabbing the logs.

This is taken care automatically if you *install* rsyslog; and that option is reverted to default no when you uninstall rsyslog.

So, for those hitting this bug, the simplest fix is simply to uninstall and reinstall rsyslog.

NB: syslog-ng is affected in the same way. Uninstalling and reinstalling it will fix the issue.

Comment 5 Chris Denice 2015-09-24 18:39:28 CEST

*** Bug 16263 has been marked as a duplicate of this bug. ***

CC: (none) => mageia

Comment 6 Florian Hubold 2015-09-27 01:39:06 CEST

Thanks for the feedback :)

(In reply to Florian Hubold from comment #3)
> and it should also be
> clearly documented or enforced if e.g. rsyslog-journald is required for
> forwarding or not.

Chris, can you verify if that package is required nowadays for forwarding journal messages to rsyslog? Colin's comment https://bugs.mageia.org/show_bug.cgi?id=16263#c7 seems to indicate this would be the case since systemd >= 216 but I'd like to get a definitive statement about this. If yes, wouldn't it make sense to simply add a Requires on rsyslog-journald for rsyslog?

Related question, if rsyslog-journald is not mandatory for forwarding, what _is_ the actual purpose of that package? I'd be curious to know that one.

Comment 7 Chris Denice 2015-09-27 10:14:14 CEST

No rsyslog-journald is another package.

It allows rsyslog to directly grab logs from systemd (so it should work without even specifying forwarding=yes). I tested it for mga4 and it suffered the issues mentioned in the description:

Description :
Provides the ability to import structured log messages from systemd
journal to rsyslog (and conversely). Note that this module reads the
journal database, what is considered a relativly performance-intense
operation. As such, the performance of a configuration utilizing this
module may be notably slower. Some versions of systemd journal have
problems with database corruption, which leads to the journal to
return the same data endlessly in a thight loop. It is strongly
recommended to use this plugin only if there is hard need to do so.

I packaged it hoping that this issue will be fix at some point; but the normal method to use rsyslog is the one already implemented in the rsyslog package, namely with forwarding=yes in systemd.

The only bugs with the packages rsyslog and syslog-ng is that the %post script is not executed during distro upgrade and I don't understand why. If someone has an idea, please post it!

Comment 8 Chris Denice 2015-11-22 15:22:53 CET

Since this bug affects only the transition mga4->mga5, no one added any info on how this should happen and I have no clue how it happened knowing that the package works as expected, I am closing it as WONTFIX.

Of course, if someone has an idea, please reopen it and post some comments here, as it is likely that this will happen again from mga5->mga6. I am inclined to think that the installer switches off the post scriplet, is that possible?

Cheers.

Status: NEW => RESOLVED
Resolution: (none) => WONTFIX

Bruno Cornec 2017-04-21 12:33:02 CEST

See Also: (none) => https://bugs.mageia.org/show_bug.cgi?id=20712