Bug 32451 - systemd: timesyncd does not sync hwclock
Summary: systemd: timesyncd does not sync hwclock
Status: NEW
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: 9
Hardware: All Linux
Priority: Normal minor
Target Milestone: ---
Assignee: Mageia Bug Squad
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-10-27 10:39 CEST by Marc Krämer
Modified: 2024-02-07 20:57 CET (History)
4 users (show)

See Also:
Source RPM: systemd / initrd
CVE:
Status comment:


Attachments

Description Marc Krämer 2023-10-27 10:39:48 CEST
I've changed the fstab some days ago and rebooted the system a few times.

But on mounting an fstab entry, I still get the warning:
mount: (hint) your fstab has been modified, but systemd still uses
       the old version; use 'systemctl daemon-reload' to reload.

My guess is, the fstab in initrd (?) differs from the one I put /etc/fstab


$ ls -lad /etc/fstab 
-rw-rw-r-- 1 root root 932 Okt 17 09:24 /etc/fstab

$ uptime 
 10:39:10 up 50 min,  1 user,  load average: 1,44, 1,70, 1,86
Comment 1 Jens Persson 2023-10-27 21:27:50 CEST
(In reply to Marc Krämer from comment #0)
> I've changed the fstab some days ago and rebooted the system a few times.
> 
> But on mounting an fstab entry, I still get the warning:
> mount: (hint) your fstab has been modified, but systemd still uses
>        the old version; use 'systemctl daemon-reload' to reload.
> 
> My guess is, the fstab in initrd (?) differs from the one I put /etc/fstab
> 
> 
> $ ls -lad /etc/fstab 
> -rw-rw-r-- 1 root root 932 Okt 17 09:24 /etc/fstab
> 
> $ uptime 
>  10:39:10 up 50 min,  1 user,  load average: 1,44, 1,70, 1,86

Never seen this problem before. Could possibly be a problem with the clock.

https://askubuntu.com/questions/1467529/fstab-and-systemd

CC: (none) => xerxes2

Marja Van Waes 2023-10-29 14:04:10 CET

CC: (none) => basesystem, marja11

Comment 2 Marc Krämer 2023-10-31 16:02:21 CET
I guess I found some reason, but I don't know if this is a configuration issue, or sth I am missing.
I use systemd-timesyncd. But as I can see hwclock is not synced on shutdown/timesync. Did I forget a setting, or is this intentional?

hwclock -r
2018-08-18 05:57:01.528216+02:00

So it is clear fstab was changed, after the first sync to ntp (because the time moved forward for a bunch of time).
Comment 3 Lewis Smith 2023-11-03 07:57:39 CET
Thank you Jens for your comment. That reference looks very relevant.

Many users adjust their /etc/fstab, and if something like this report happened in consequence, we would surely have seen it before. /etc/fstab being a fundamental system reference, getting it wrong can lead to failure to boot; so I doubt that another version exists elsewhere.

Marc, if you conclude that it is not a Mageia bug, please close this.
Comment 4 Marc Krämer 2023-11-03 12:39:11 CET
@Lewis: the problem seems, that there is no service (anymore?) which syncs software clock to hardware clock.
On reboot the hardwareclock has a wrong time, and is corrected after the first contact to the timeserver. This can lead to strange errors as this one.

So my question is, am I missing to enable a (non obvious service), or is this an issue not (yet) adressed.
Comment 5 Jens Persson 2023-11-03 15:58:38 CET
(In reply to Marc Krämer from comment #2)
> I guess I found some reason, but I don't know if this is a configuration
> issue, or sth I am missing.
> I use systemd-timesyncd. But as I can see hwclock is not synced on
> shutdown/timesync. Did I forget a setting, or is this intentional?
> 
> hwclock -r
> 2018-08-18 05:57:01.528216+02:00
> 
> So it is clear fstab was changed, after the first sync to ntp (because the
> time moved forward for a bunch of time).

I don't think systemd-timesyncd (timedatectl) updates the hardwareclock though. So yeah it seems to be intentional.
Comment 6 Marc Krämer 2023-11-06 14:17:30 CET
do we have/use any service that syncs the clock on shutdown?

Am I missing sth? but the hardware time will differ from actual time very soon.

At boot time we only have the hardware time, and if this is very different from the actual time (in my case 5 years) you will never get boot logs as they get discarded for beeing too old. And you can run in this error I described here - and maybe others.

Do we need to integrate this into some shutdown service?
Comment 7 Marc Krämer 2023-11-06 14:18:22 CET
do we have/use any service that syncs the clock on shutdown?

Am I missing sth? but the hardware time will differ from actual time very soon.

At boot time we only have the hardware time, and if this is very different from the actual time (in my case 5 years) you will never get boot logs as they get discarded for beeing too old. And you can run in this error I described here - and maybe others.

Do we need to integrate this into some shutdown service?
Comment 8 Dave Hodgins 2023-11-07 18:12:04 CET
Doing it automatically is problematic. Not all hardware has a real time clock.
My rpi 4b does not, and the system has to be usable on a device that doesn't
have internet access, or at least doesn't have trusted internet access.

Options for setting the rtc can be set in ntp, chrony, or systemd-timesyncd,
but all of them require configuration first.

It can also be done manually using the hwclock command.

Doing it automatically can cause problems for systems that have windows
installed too.

It has to be left to the system administrator to decide how/when to sync
the rtc.

CC: (none) => davidwhodgins

Comment 9 Dave Hodgins 2023-11-07 18:19:08 CET
On my rpi 4b, I've modified several services to wait until chrony has synced
the time, to avoid problems caused by the time changing ...
$ grep -r chrony *
crond.service:After=auditd.service nss-user-lookup.target systemd-user-sessions.service time-sync.target ypbind.service autofs.service chrony-wait.service

dovecot.service:After=local-fs.target network-online.target chrony-wait.service

gdm.service:After=rc-local.service plymouth-start.service systemd-user-sessions.service chrony-wait.service

lightdm.service:After=systemd-user-sessions.service getty@tty1.service plymouth-quit.service chrony-wait.service

lxdm.service:After=systemd-user-sessions.service getty@tty1.service plymouth-quit.service livesys-late.service chrony-wait.service

xdm.service:After=systemd-user-sessions.service getty@tty1.service plymouth-quit.service chrony-wait.service

Without those changes, the desktop clock shows the wrong time, as do other
applications that start at login.
Comment 10 Marc Krämer 2023-11-07 19:40:53 CET
@Dave: that looks wrong. There are several systems to adjust the time at runtime. 
most of the services should sync to the correct time. For desktops I've seen, they sync after the 1 minute timeout was hit.

If the time is synced to hardware regulary (ntp-client had such an option), the difference of the hardware clock to ntp should not be too big, so there is no need to adapt all those services.
Comment 11 Dave Hodgins 2023-11-07 20:50:55 CET
Comment 9 is for my rpi 4b. It does not have a real time clock, so it is not
possible to sync one. It's possible to add a rtc as an external card
https://maker.pro/raspberry-pi/tutorial/how-to-add-an-rtc-module-to-raspberry-pi

If the desktop starts before the time has been synced from the internet, then
the clock doesn't properly handle the date/time changing.

I adjusted the startup sequence to ensure login and crond do not start before
the date/time has been obtained from the internet. As I was using it for
testing, I altered all of the dm startup services, as I would switch between
them when testing.

I just rebooted the rpi 4b.
$ journalctl -b --no-h|grep stepped
Nov 07 13:45:18 chronyd[740]: System clock was stepped by 5164306.164489 seconds
Comment 12 Marc Krämer 2023-11-07 23:29:53 CET
Ok, sorry - I just used the rpi1 and added an external rtc to it.


But still we're talking about desktop/server in general. On these systems we have rtc's, and we use them. But they become useless if the time is not close to the real time. In my case it differed 5 years (I guess the battery change 1 year ago) - that is not close.

So I propose to sync the time on shutdown/reboot of the system to the hardware clock. Or add a timer to do the sync regulary. I'm not sure, what other distros do about this.
Comment 13 Dave Hodgins 2023-11-08 00:55:32 CET
I wouldn't want my VirtualBox guests trying to update the rtc.
Comment 14 Marc Krämer 2023-11-08 09:01:44 CET
The guest should not be able to set hwclock or at least only the the virtual clock.
And if done via service or sysconfig-setting it could be deactivated....
Comment 15 katnatek 2023-11-08 19:50:53 CET
I recently see a message that recommends a slight different command, if you run the command the warning goes
Comment 16 katnatek 2023-11-11 00:00:54 CET
(In reply to katnatek from comment #15)
> I recently see a message that recommends a slight different command, if you
> run the command the warning goes

forget that, systemctl daemon-reload is the command recommended, I use it and the warning is no longer show
Comment 17 Marc Krämer 2023-11-11 01:45:43 CET
true, except on the next boot.
Marc Krämer 2023-12-13 13:47:15 CET

Summary: systemd: changed fstab not reloaded => systemd: timesyncd does not sync hwclock

Comment 18 Dave Hodgins 2023-12-13 20:43:57 CET
From "man systemd-timesyncd.service" it's setup to work with systems
where there is no real time clock.

Instead of updating the hardware clock, it modifies the timestamp of a file.
"stat /var/lib/systemd/timesync/clock|grep ^Modify" should show the current
timestamp, to be used on reboot. It's only updated roughly once per minute
by default.

The timedatectl command can be used to set timesyncd to update the RTC.

"timedatectl show" will show whether it's currently set to update the RTC
or not. It defaults to not updating the RTC.
Comment 19 Jens Persson 2023-12-13 22:07:33 CET
I investigated this a while back in this forum thread. https://forums.mageia.org/en/viewtopic.php?f=8&t=15133

Tldr, it looks like it is the kernel that updates the RTC. There is a setting that the kernel keeps tabs on and if NTP is enabled it updates the RTC. Support for this must be enabled in the kernel though.
Comment 20 Jens Persson 2023-12-13 22:16:20 CET
Just checked and that setting is enabled in the Cauldron kernel.

https://svnweb.mageia.org/packages/cauldron/kernel/current/SOURCES/defconfig-x86_64-desktop?revision=2013732&view=markup#l8515
Comment 21 Lewis Smith 2024-02-07 20:57:02 CET
A late but obvious comment:
(In reply to Dave Hodgins from comment #8)
> Doing it automatically is problematic. Not all hardware has a real time
> clock.
> My rpi 4b does not, and the system has to be usable on a device that doesn't
> have internet access, or at least doesn't have trusted internet access.
No h/w clock + no Internet NTP access possible & enabled = chaos: no means of establishing date/time.

The only way to push this is to pass it to 'basesystem', but it looks insoluble.

Note You need to log in before you can comment on or make changes to this bug.