Bug 31447

Summary: upgrading package filesystem fails with "error: unpacking of archive failed on file /proc: cpio: chown failed - Directory not empty"
Product: Mageia Reporter: PC LX <mageia>
Component: RPM PackagesAssignee: All Packagers <pkg-bugs>
Status: REOPENED --- QA Contact:
Severity: normal    
Priority: Normal CC: davidwhodgins
Version: Cauldron   
Target Milestone: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Source RPM: filesystem-2.1.9-36.mga9.src.rpm CVE:
Status comment:

Description PC LX 2023-01-23 18:54:16 CET
Description of problem:

While upgrading a Mageia 8 (in a container) to Mageia 9/cauldron, using the urpmi CLI tools, resulted in the following error when upgrading the package "filesystem".

""""""""""""""""""""""""""""""""
# LANGUAGE=C ionice -c 3 nice -n 19 time urpmi --split-length 0 --auto-update
medium "Core Release" is up-to-date
medium "Core Updates" is up-to-date
To satisfy dependencies, the following package is going to be installed:
  Package                        Version      Release       Arch    
(medium "Core Release")
  filesystem                     2.1.9        36.mga9       x86_64  
0B of additional disk space will be used.
16KB of packages will be retrieved.
Proceed with the installation of one package? (Y/n) 


    https://mirror.math.princeton.edu/pub/mageia/distrib/9/x86_64/media/core/release/filesystem-2.1.9-36.mga9.x86_64.rpm
installing filesystem-2.1.9-36.mga9.x86_64.rpm from /var/cache/urpmi/rpms      
Preparing...                     #############################################
      1/1: filesystem            #############################################
error: unpacking of archive failed on file /proc: cpio: chown failed - Directory not empty
ERROR: 'unpack' failed for filesystem-2.1.9-36.mga9.x86_64
error: filesystem-2.1.9-36.mga9.x86_64: install failed
error: filesystem-2.1.9-34.mga8.x86_64: erase skipped
""""""""""""""""""""""""""""""""



Version-Release number of selected component (if applicable):

filesystem-2.1.9-36.mga9.x86_64



How reproducible:

Always.



Steps to Reproduce:
1. Start with a minimal Mageia 8 system, running inside a systemd-nspwan container.
2. Remove the Mageia 8 repositories with "urpmi.removemedia -a"
3. Add the Mageia 9/cauldron repositories with "urpmi.addmedia --distrib https://mirror.math.princeton.edu/pub/mageia/distrib/9/x86_64/"
4. Upgrade the system with "urpmi --auto-update --auto".
5. Of the 236 packages, only upgrading the package "filesystem" failed with the "unpack failed" error.
Comment 1 Lewis Smith 2023-01-23 19:52:20 CET
Thank you for the report.
'filesystem' has no one maintainer, so assigning this globally.

Assignee: bugsquad => pkg-bugs

Comment 2 David Walser 2023-01-23 22:37:34 CET
Did you do something weird on your system like change /proc to a symlink?
Comment 3 Dave Hodgins 2023-01-24 04:24:05 CET
Testing an x86_64 upgrade using urpmi with the princeton mirror
  11/2287: filesystem            #############################################

filesystem-2.1.9-36.mga9 installed cleanly here.

Closing as invalid. Please reopen if you can figure out what triggered the
error on that system.

Status: NEW => RESOLVED
CC: (none) => davidwhodgins
Resolution: (none) => INVALID

Comment 4 PC LX 2023-01-24 16:06:12 CET
(In reply to David Walser from comment #2)
> Did you do something weird on your system like change /proc to a symlink?

The only "weird" thing is running in a container.

It is a normal directory and a mount point for the proc VFS.
The permissions are read-only (dr-xr-xr-x) and the owner is nobody:nogroup.
The permissions and owner are are the same in all containers.
These were not set by me. I think it is systemd-nspawn that sets it like that.
Adding options uid=0,gid=0 to proc line in fstab did not change the owner:group at all.

For reference, the host's /proc has the usual root:root as owner:group.

I'm reopening this bug but if running in a container is not supported, please feel free to close, maybe as wontfix.

##### VIEW FROM THE CONTAINER #####

# stat /proc/
  File: /proc/
  Size: 0               Blocks: 0          IO Block: 1024   directory
Device: 0,86    Inode: 1           Links: 518
Access: (0555/dr-xr-xr-x)  Uid: (65534/  nobody)   Gid: (65534/ nogroup)
Access: 2023-01-24 14:38:26.823020612 +0000
Modify: 2023-01-24 14:38:26.823020612 +0000
Change: 2023-01-24 14:38:26.823020612 +0000
 Birth: -
# ls -la /
total 16
drwxr-xr-x   1 root   root     102 Jan 24 14:38 ./
drwxr-xr-x   1 root   root     102 Jan 24 14:38 ../
lrwxrwxrwx   1 root   root       7 Jul 31  2020 bin -> usr/bin/
drwxr-xr-x   7 root   root     440 Jan 24 14:42 dev/
drwxr-xr-x   1 root   root    2530 Jan 24 14:38 etc/
drwxr-xr-x   1 root   root      10 Sep  6 18:17 home/
lrwxrwxrwx   1 root   root       7 Jul 31  2020 lib -> usr/lib/
lrwxrwxrwx   1 root   root       9 Jul 31  2020 lib64 -> usr/lib64/
dr-xr-xr-x 515 nobody nogroup    0 Jan 24 14:42 proc/
drwxr-x---   1 root   root     286 Jan 24 14:21 root/
drwxr-xr-x  15 root   root     400 Jan 24 14:42 run/
lrwxrwxrwx   1 root   root       8 Jul 31  2020 sbin -> usr/sbin/
drwxr-xr-x   1 root   root       0 Jan 24 14:38 srv/
dr-xr-xr-x   9 root   root     180 Jan 24 14:42 sys/
drwxrwxrwt   9 root   root     180 Jan 24 14:42 tmp/
drwxr-xr-x   1 root   root      84 Jan 23 17:41 usr/
drwxr-xr-x   1 root   root      76 Jan 24 14:37 var/

##### VIEW FROM THE HOST #####

# stat /var/lib/machines/jupiter-co-mageia-9-x86-64/proc
  Ficheiro: /var/lib/machines/jupiter-co-mageia-9-x86-64/proc
  Tamanho: 0            Blocos: 0          IO Bloco: 4096   pasta
Dispositivo: 41h/65d    Inode: 17888       Ligações: 1
Accesso: (0755/drwxr-xr-x)  Uid: (132055040/ UNKNOWN)   Gid: (132055040/ UNKNOWN)
Accesso: 2020-07-31 14:14:59.000000000 +0100
Modificado: 2020-07-31 14:14:59.000000000 +0100
Alterado: 2022-10-18 16:20:35.863797519 +0100
 Nascimento: 2022-10-18 16:13:49.524770033 +0100
# ls -la /var/lib/machines/jupiter-co-mageia-9-x86-64
total 16
drwxr-xr-x 1 132055040 132055040  102 jan 24 14:38 ./
drwx------ 1 root      root       408 jan 24 14:39 ../
lrwxrwxrwx 1 132055040 132055040    7 jul 31  2020 bin -> usr/bin/
drwxr-xr-x 1 132055040 132055040    0 nov 21 12:35 dev/
drwxr-xr-x 1 132055040 132055040 2530 jan 24 14:38 etc/
drwxr-xr-x 1 132055040 132055040   10 set  6 18:17 home/
lrwxrwxrwx 1 132055040 132055040    7 jul 31  2020 lib -> usr/lib/
lrwxrwxrwx 1 132055040 132055040    9 jul 31  2020 lib64 -> usr/lib64/
drwxr-xr-x 1 132055040 132055040    0 jul 31  2020 proc/
drwxr-x--- 1 132055040 132055040  286 jan 24 14:21 root/
drwxr-xr-x 1 132055040 132055040  120 out 18 08:44 run/
lrwxrwxrwx 1 132055040 132055040    8 jul 31  2020 sbin -> usr/sbin/
drwxr-xr-x 1 132055040 132055040    0 jan 24 14:38 srv/
drwxr-xr-x 1 132055040 132055040    0 jul 31  2020 sys/
drwxrwxrwt 1 132055040 132055040    0 jul 31  2020 tmp/
drwxr-xr-x 1 132055040 132055040   84 jan 23 17:41 usr/
drwxr-xr-x 1 132055040 132055040   76 jan 24 14:37 var/
$ ls -la /
total 16
drwxr-xr-x   1 root root  144 jan 24 14:37 ./
drwxr-xr-x   1 root root  144 jan 24 14:37 ../
lrwxrwxrwx   1 root root    7 jul 31  2020 bin -> usr/bin/
drwxr-xr-x   1 root root  754 jan 15 23:01 boot/
drwxr-xr-x  22 root root 4700 jan 24 13:55 dev/
drwxr-xr-x   1 root root 5248 jan 24 12:59 etc/
drwxr-xr-x   1 root root   10 jul 27 10:53 home/
drwxr-xr-x   1 root root    0 jul 31  2020 initrd/
lrwxrwxrwx   1 root root    7 jul 31  2020 lib -> usr/lib/
lrwxrwxrwx   1 root root    9 jul 31  2020 lib64 -> usr/lib64/
drwxr-xr-x   1 root root   32 ago  1 10:17 media/
drwxr-xr-x   1 root root    6 dez  8 18:02 mnt/
drwxr-xr-x   1 root root  314 jan 18 09:02 opt/
dr-xr-xr-x 492 root root    0 jan 24 09:41 proc/
drwx------   1 root root  812 jan 24 14:46 root/
drwxr-xr-x  40 root root 1160 jan 24 14:01 run/
lrwxrwxrwx   1 root root    8 jul 31  2020 sbin -> usr/sbin/
drwxr-xr-x   1 root root   26 out 30 14:21 srv/
dr-xr-xr-x  13 root root    0 jan 24 09:41 sys/
drwxrwxrwt  20 root root  480 jan 24 14:52 tmp/
drwxr-xr-x   1 root root  106 jun 24  2022 usr/
drwxr-xr-x   1 root root  160 nov  1 09:41 var/

Resolution: INVALID => (none)
Status: RESOLVED => REOPENED

Comment 5 Dave Hodgins 2023-01-24 22:46:11 CET
Please test to see if that's a regression from Mageia 8.
Comment 6 PC LX 2023-01-24 23:59:52 CET
The Mageia 8 container and all the other containers in this system (host is Mageia 8) have the same owner and permissions in /proc.
I tried to manually change the ownership of /proc in the containers, as the container's root, and it always results in the following error message.

"""""""""""""""""""""""""""""""""""
# chown root:root /proc
chown: changing ownership of '/proc': Operation not permitted
"""""""""""""""""""""""""""""""""""

The error message says chown failed so not being allowed to change the ownership may be the cause of the issue.

I'm not certain how to work around this without have a special case for /proc when in a container.
I'm not even certain if it is a good idea for the update to change permissions on existing files/directories, like it is trying to do here for /proc, be it in a container or not.
Comment 7 Dave Hodgins 2023-01-25 02:00:15 CET
This does not look like a regression. In the spec file for filesystem
the /proc directory is created with %defattr(0755,root,root)
In Mageia 8 and 9 installs that are not running ...
$ ls -ld /d2/proc
drwxr-xr-x 2 root root 4096 Jan 19 16:26 /d2/proc/

Once the system is started, /proc is used as a mountpoint
$ mount|grep ^proc
proc on /proc type proc (rw,relatime)

And the permissions change accordingly.
$ ls -ld /proc
dr-xr-xr-x 389 root root 0 Jan 19 17:26 /proc/

Note that the underlying directory used as a mount point is not changed.

I suspect the problem is that urpmi should be used to install the system
using a --root, and then systemd-nspawn system started. A chroot is not
a suitable environment for running the installer.

Have you ever done a linux install in a chroot or have any references to
indicate it should work?
Comment 8 Dave Hodgins 2023-01-25 02:04:07 CET
See https://wiki.mageia.org/en/Chroot for how to install in a chroot, which
can then be run using chroot or systemd-nspawn.
Comment 9 PC LX 2023-01-25 12:26:12 CET
This does not look like a regression.(In reply to Dave Hodgins from comment #7)
> This does not look like a regression.

I haven't tried upgrading from Mageia 7 running inside a container to Mageia 8 but I suspect it would have the same issue.

> In the spec file for filesystem
> the /proc directory is created with %defattr(0755,root,root)
> In Mageia 8 and 9 installs that are not running ...
> $ ls -ld /d2/proc
> drwxr-xr-x 2 root root 4096 Jan 19 16:26 /d2/proc/
> 
> Once the system is started, /proc is used as a mountpoint
> $ mount|grep ^proc
> proc on /proc type proc (rw,relatime)
> 
> And the permissions change accordingly.
> $ ls -ld /proc
> dr-xr-xr-x 389 root root 0 Jan 19 17:26 /proc/
> 
> Note that the underlying directory used as a mount point is not changed.

That conforms to what I see on my system also.

> I suspect the problem is that urpmi should be used to install the system
> using a --root, and then systemd-nspawn system started. A chroot is not
> a suitable environment for running the installer.

From my experience, running urpmi inside a container works correctly. I have been updating a Mageia 8 system running inside a container for a good while without issues. 

> Have you ever done a linux install in a chroot or have any references to
> indicate it should work?

The issue is not from a fresh install but an upgrade from an existing Mageia 8 system inside a sytemd-nspawn container to Mageia 9. Other than the package "filesystem", all other packages upgraded without issues.

This issue seems to be that cpio tries to change ownership of /proc to root:root but when inside a container that is not permitted.
Comment 10 David Walser 2023-01-25 16:52:24 CET
This is probably more fallout from Bug 8665.  We should probably just revert that change in rpm.
Comment 11 Dave Hodgins 2023-01-25 21:18:14 CET
In this case it isn't about actually changing the directory permissions as
the permissions are the same in both Mageia 8 and 9 for the /proc directory.

This one is about trying to set the permissions on the directory when it's
already been used as a mount point. The mount point has different permissions
than the underlying directory has, and the directory is not empty.

The pretrans scriptlet in the filesystem package does not reference /proc.
The only references to /proc are in the spec file in the lines
mkdir -p opt proc root sys tmp
%files
95 	%defattr(0755,root,root)
/proc

Once mounted, /proc has permissions 0555,root,root

None of the references to /proc have changed between m8 and m9.

Strange that it only causes an error when used in a systemd-nspawn container,
and not when upgrading a normal install.

I did notice that the man page for systemd-nspawn has ...
"systemd-nspawn limits access to various kernel interfaces in the container
to read-only, such as /sys, /proc/sys or /sys/fs/selinux. The host's network
interfaces and the system clock may not be changed from within the container.
Device nodes may not be created. The host system cannot be rebooted and
kernel modules may not be loaded from within the container."

Those limitations do not apply if using chroot instead of systemd-nspawn.

The answer may be that when using systemd-spawn a line with "filesystem"
must be added to /etc/urpmi/skip.list in the container.
Comment 12 PC LX 2023-01-26 10:44:29 CET
(In reply to Dave Hodgins from comment #11)
> The answer may be that when using systemd-spawn a line with "filesystem"
> must be added to /etc/urpmi/skip.list in the container.

That will prevent packages with dependencies on filesystem[>= 2.1.9-36] from being installed.
The only case in the repository seems to be the package "plocate".
Trying to install the package plocate with the package "filesystem" in "/etc/urpmi/skip.list" will fail.


# for PACK in $(urpmq --whatrequires filesystem | uniq) ; do urpmq --requires "$PACK" | grep filesystem ; done | sort -u
filesystem
filesystem[*]
filesystem[*][>= 2.1.9-18]
filesystem[*][>= 2.1.9]
filesystem[>= 2.1.9-18]
filesystem[>= 2.1.9-36]
filesystem[>= 2]


# for PACK in $(urpmq --whatrequires filesystem | uniq) ; do [ "$(urpmq --requires "$PACK" | grep 'filesystem\[>= 2.1.9-36\]')" != "" ] && echo "$PACK" ; done | sort -u
plocate


# cat /etc/urpmi/skip.list
# Here you can specify the packages that won't be upgraded automatically
# for example, to exclude all apache packages :
# /^apache/

/^filesystem-/


# urpmi plocate
A requested package cannot be installed:
plocate-1.1.17-1.mga9.x86_64 (due to unsatisfied filesystem[>= 2.1.9-36])
Continue installation anyway? (Y/n) 
While some packages may have been installed, there were failures.
A requested package cannot be installed:
plocate-1.1.17-1.mga9.x86_64 (due to unsatisfied filesystem[>= 2.1.9-36])
Continue installation anyway?