Bug 10834

I just tried with pm-hibernate and only / was mounted readonly at restore time this time:

[bruno@localhost TCP]$ sudo mount | grep ' / '
/dev/sda1 on / type ext4 (ro,noatime,data=ordered)
[bruno@localhost TCP]$ sudo mount | grep ' /usr '
/dev/sda6 on /usr type ext4 (rw,noatime,data=ordered)

So I re-did a test with pm-suspend to be sure, and this time only / was remounted ro. However, this is not done immediately after restoration, as while pm-suspend was still not giving the hand back to prompt, I passed the same command as above and first / was rw, then after the command finished, then it was ro.

Then I looked for which command could do that and found a script /usr/lib64/pm-utils/power.d/journal-commit which calls mount.

This script reads /proc/mounts to find the status of the FS IIUC.
On my machine I have a named process running, with bind mounts read-only:

[bruno@localhost TCP]$ cat /proc/mounts 
rootfs / rootfs rw 0 0
proc /proc proc rw,relatime 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
devtmpfs /dev devtmpfs rw,nosuid,size=1971648k,nr_inodes=492912,mode=755 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
tmpfs /run tmpfs rw,nosuid,nodev,mode=755 0 0
/dev/sda1 / ext4 rw,noatime,data=ordered 0 0
/dev/sda6 /usr ext4 rw,noatime,data=ordered 0 0
securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /sys/fs/cgroup tmpfs rw,nosuid,nodev,noexec,mode=755 0 0
cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0
cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpuacct,cpu 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
cgroup /sys/fs/cgroup/net_cls cgroup rw,nosuid,nodev,noexec,relatime,net_cls 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=300,minproto=5,maxproto=5,direct 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0
mqueue /dev/mqueue mqueue rw,relatime 0 0
debugfs /sys/kernel/debug debugfs rw,relatime 0 0
/dev/sda9 /tmp ext4 rw,noatime,data=ordered 0 0
/dev/sda8 /boot ext4 rw,noatime,stripe=4,data=ordered 0 0
/dev/sda7 /var ext4 rw,noatime,data=ordered 0 0
/dev/sda10 /home ext4 rw,noatime,data=ordered 0 0
/dev/sda1 /var/lib/named/etc/named.conf ext4 ro,relatime,data=ordered 0 0
/dev/sda1 /var/lib/named/etc/rndc.key ext4 ro,relatime,data=ordered 0 0
/dev/sda1 /var/lib/named/etc/named.iscdlv.key ext4 ro,relatime,data=ordered 0 0
/dev/sda1 /var/lib/named/etc/named.root.key ext4 ro,relatime,data=ordered 0 0
/dev/sda6 /var/lib/named/usr/lib64/bind ext4 ro,relatime,data=ordered 0 0
/dev/sda6 /var/lib/named/usr/lib64/openssl ext4 ro,relatime,data=ordered 0 0
/dev/sda7 /var/lib/named/var/named ext4 rw,noatime,data=ordered 0 0
fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0
gvfsd-fuse /run/user/2000/gvfs fuse.gvfsd-fuse rw,nosuid,nodev,relatime,user_id=2000,group_id=2000 0 0

So /dev/sda1 appears with both the rw and ro flag then and I think this is what is causing the issue. So this commend should not read blindly /proc/mounts, but just a subset of it.

Source RPM: not sure => usermode-consoleonly-1.111-3.mga3, pm-utils-1.4.1-6.mga3

Here is the proposed patch. Not nice, but works for me so hopefully usefull for others as well:

In the script /usr/lib*/pm-utils/power.d/journal-commit change the handle_filesystems function with this one:

handle_filesystems()
{
        # $1 = new journal commit time in seconds
        while read DEV MOUNT FSTYPE REST;
        do
                echo $MOUNT | grep -q "/var/lib/named"
                if [ $? -eq 0 ]; then
                        continue
                fi
                command_exists "handle_${FSTYPE}" || continue
                printf "Setting journal commit time for %s to %d..." \
                    "$MOUNT" "$1"
                "handle_${FSTYPE}" $MOUNT $1 && echo Done. || echo Failed.
        done < /proc/mounts
}

It may be useful to also catch other bind mounted dirs (ntp ?)

Out of curiosity, does the same problem occur with "systemctl suspend" instead of pm-suspend?

I suspect not as the pm-utils hooks should not be run.

All of the GUIs now use upower hooks and systemd to suspend and thus I suspect most users are simply not using pm-utils (which I originally  proposed dropping completely).

If you do not notice any issues when NOT using pm-utils to suspend (if you have any manual quirks you want to do with the systemd approach you can use the /usr/lib/systemd/system-sleep/ folder see the notify-upower.sh file that is part of the upower pkg or see "man systemd-sleep", but really the general idea is to do away with the need for such scripts anyway).

So longer term, I'd rather just completely nuke pm-utils rather than fix it if it's genuinely no longer needed.

CC: (none) => mageia

No problem (for once ;-) when using systemctl suspend|hibernate|hybrid-sleep

Thanks for pointing that, but it now lead to new questions:

1/ Why are we providing a non working package ? I still think that in mga3 it should be fixed as it's provided.

2/ If systemd is the way forward for that also (grumph !) then we should provide compatibility aspects no ? providing pm-suspend being a redirection to systemctl suspend e.g. My concern with systemd is the full lack of compatibility provided for old sysadmin. Makes transition so painful. E.g. would be easy to provide a chkconfig command again, which will hive more time to people to learn systemd itself once they understand it's behind the scene.

3/ Where can a Mageia user find that type of info about that change without following all posts on the dev ML ? I read the release notes, without seeing anything relevant, but I may have missed something.

4/ I'm still concerned by the place taken by systemd in the distro. A lot needs to be learned on it, and the replacement of all small one function tool by an htdra like that create real concerns on my side. The KISS principle is far away, and the registry approach really near :-( What I'd really like is to see systemd available in multiple packages, so that I can keep older tools if I want instead of migrating. Here I'd have like to have systemd-sleep as a separate pakage, and a conflict with pm-utils around these 2. Or are we forcing everybody to move to systemd on all of its features (that was an point they made clear on their site, that we could keep other processes such as cron e.g. separated).

Thanks to your help I've found solutions to most of my systemd issues up to now, but each time it's taking too much time to resolve for me (you have to look at journalctl and the status and launch the daemon most of the time when an error occurs before being able to solve), however I just found that command completionso that will be useful. I still think Mageia should be the easiest possible to manipulate from the keyboard, even if we are a very good distro for newcomers with the graphical envs.

And thanks Colin for your positive defense of systemd which is much more interesting than the author's selfish attitude.

(In reply to Bruno Cornec from comment #4)
> No problem (for once ;-) when using systemctl suspend|hibernate|hybrid-sleep

Great!

> Thanks for pointing that, but it now lead to new questions:
> 
> 1/ Why are we providing a non working package ? I still think that in mga3
> it should be fixed as it's provided.

Well, I wanted to drop it completely but it was met with a few complaints regarding the quirks it contains. I was quite sure (after consultation with various upstreams) that the vast majority of such quirks were no longer needed - in particular the graphics card quirks. Times have moved on but these legacy things are still left to bitrot in pm-utils. As the upower backend changed to systemd and no major complains of suspend/resume causing graphical problems flowed in I was more convinced of this, but also it meant that pm-utils didn't really get much in the way of QA as most people were testing suspend/resume through their GUI rather than via commands. I guess this problem then slipped through the testing net.

I'll certainly be in favour of deprecating pm-utils completely for mga4. When this is done, it should be possible to provide a bunch of compatibility scripts that echo a warning and redirect to the systemctl command for convenience.
 
> 2/ If systemd is the way forward for that also (grumph !) then we should
> provide compatibility aspects no ? providing pm-suspend being a redirection
> to systemctl suspend e.g. My concern with systemd is the full lack of
> compatibility provided for old sysadmin. Makes transition so painful.

Yeah, I fully agree, that if/when pm-utils as a package goes completely we should make the commands therein into scripts that redirect. I still carry an alias in some systems for a command called nsu (network su) which I replaced probably 8 years ago with sudo!

> E.g.
> would be easy to provide a chkconfig command again, which will hive more
> time to people to learn systemd itself once they understand it's behind the
> scene.

chkconfig is trickier sadly as it's still needed as a native application because systemd still supports LSB initscripts. It's the chkconfig command that is used to enable and disable them, even if systemctl can redirect happily when calling "systemctl enable mylsb.service" and similar commands.

[root@jimmy ~]$ systemctl enable mylsb.service
mylsb.service is not a native service, redirecting to /sbin/chkconfig.
Executing /sbin/chkconfig mylsb on
[root@jimmy ~]$ systemctl disable mylsb.service
mylsb.service is not a native service, redirecting to /sbin/chkconfig.
Executing /sbin/chkconfig mylsb off


It should be possible to patch chkconfig to be "systemd aware" and do a similar redirect the opposite way, but we have to be very careful not to create an accidental loop of one calling the other calling the first again ad infinitum!

I'll certainly look into this.

> 3/ Where can a Mageia user find that type of info about that change without
> following all posts on the dev ML ? I read the release notes, without seeing
> anything relevant, but I may have missed something.

Yes, there is likely a degree of info that is lacking in terms of the documentation which is certainly partly my fault.

That said, if people are system administrators I would expect them to read some of the upstream documentation which is really very good. The "Systemd For Administrators" series are an excellent introduction.

For regular users, they are most likely using drakservices.

> 4/ I'm still concerned by the place taken by systemd in the distro. A lot
> needs to be learned on it, and the replacement of all small one function
> tool by an htdra like that create real concerns on my side.

I'm still a little confused by statements like this last one. I agree that things need to be learned if you are a real sysadmin but the statement "replacement of all small one function tools by a hydra like [systemd]" really doesn't ring true to me.

> The KISS principle is far away, and the registry approach really near :-( 

Again, I really disagree with this statement. I could provide any number of counter arguments that the previous system was a myriad of boiler plate code that differed subtly in confusing ways and contained different ways of working around the same problems, and with inittab supporting automatic restarts but, bizarrely other, separately distributed, helper tools performing the same role for "real" daemons.

Now the admin does not need to read and understand a several pages of scripts to understand how a service starts. Most are incredibly simple and can be read by anyone without a deep understanding of shell. The commands used to enable and disable and show status also capture exit status on failures (something that was previously lost completely) and capture all stderr/out by default to the logging system and show that too alongside the status. It becomes much simpler to *see* what is going on without *knowing* already how everything works.

> What I'd
> really like is to see systemd available in multiple packages, so that I can
> keep older tools if I want instead of migrating. Here I'd have like to have
> systemd-sleep as a separate pakage, and a conflict with pm-utils around
> these 2.

Some people think this is the "ideal world". I totally disagree and think this is pretty much the nightmare scenario. Such a setup just increases the test matrix for our QA team. We have to test all such combinations a user can create rather than the one setup that can get more testing and more concentrated bugfixing. This bug report that has occurred because QA testing didn't highlight manual usage of pm-utils commands is testament to the fact that such user "choice" comes at a massive cost in terms of support.

At FOSDEM this year, I sat in the audience, exasperated at a Gentoo talk where some developers behind some ill fated udev fork talked about how they wanted to support a certain disk layout and boot sequence scenario that just didn't make sense to support (the same disk layout could be supported in different way which bypassed all of the rather complex and intricate)problems that the alternative suggested). When I pushed them on the reasons, they said "choice", and "because we can". I had to remind them that "you can never please all of the people all of the time" and others that by providing this choice they increase complexity (and complicated edge cases) massively with a test-matrix explosion. The various kernel and core-os people present were also suitably exasperated!

> Or are we forcing everybody to move to systemd on all of its
> features (that was an point they made clear on their site, that we could
> keep other processes such as cron e.g. separated).

cron will never go away. It's a useful component and systemd does not, and will not, cover all it's capabilities. It's not a *replacement* for cron, but some of the functionality that systemd provides with regards to timers *can* replace some of the basic uses of cron as shipped in many of our packages (namely basic housekeeping jobs). In this case, as systemd is required anyway for boot, it makes sense to use timer units and not require another package to be installed and running, thus keeping the install and memory footprint down. If the user has a need for cron for any other purpose, or a package genuinely needs the enhanced functionality of cron itself, then they should install/require it.

> Thanks to your help I've found solutions to most of my systemd issues up to
> now, but each time it's taking too much time to resolve for me (you have to
> look at journalctl and the status and launch the daemon most of the time
> when an error occurs before being able to solve)

I've only occasionally found the need to run things manually personally. Most of the time the systemctl status' output contains sufficient information from the services stderr to tell me what is wrong.

I do however think it can (and should) be made better.

e.g. I'd like to see the journal output outputted automatically to stderr when a systemctl start command fails.

Various things are now logged better in newer systemds too, for example any Condition* directives in a unit that fail will be shown clearly in the status output.

> however I just found that
> command completionso that will be useful. I still think Mageia should be the
> easiest possible to manipulate from the keyboard, even if we are a very good
> distro for newcomers with the graphical envs.

I agree fully here :)

> And thanks Colin for your positive defense of systemd which is much more
> interesting than the author's selfish attitude.

Well, in slight defence of Lennart, his responses these days are much more reasoned and less blunt than those of a few years ago. He's still obviously very biased towards the "systemd way" but usually his arguments are valid, even if still delivered in a rather blunt way on occasion! But contrast this to the documentation written by him which is usually well thought out and delivered nicely.

But anyway, I hope this response is a mostly positive one. And I agree that pm-utils can still be make to work better in mga3 for this particular issue so will look to push out an update in due course.

I'm closing as using the systemd sleep indeed fixes stuff, and is called by all other tools.

Status: NEW => RESOLVED
Resolution: (none) => WONTFIX