Bug 29096 - Docker and systemd - failed to stop Failed with result 'timeout'
Summary: Docker and systemd - failed to stop Failed with result 'timeout'
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: 8
Hardware: x86_64 Linux
Priority: Normal normal
Target Milestone: ---
Assignee: Bruno Cornec
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-06-09 08:32 CEST by david Cossé
Modified: 2023-06-26 11:47 CEST (History)
4 users (show)

See Also:
Source RPM: docker-20.10.9-3.mga8.src.rpm
CVE:
Status comment:


Attachments
shutting down process (185.60 KB, image/jpeg)
2022-01-19 15:51 CET, david Cossé
Details

Description david Cossé 2021-06-09 08:32:22 CEST
Description of problem:
systemctl stop docker.service take a uge time to stop

Version-Release number of selected component (if applicable):
docker-20.10.5-1.mga8.x86_64.rpm

How reproducible:
systemctl start docker.service
systemctl stop docker.service

Steps to Reproduce:
juin 09 08:28:48 davidcosse-NC-V3-571G-53236 systemd[1]: Stopping Docker Application Container Engine...
juin 09 08:28:48 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:48.560951719+02:00" level=error msg="Failed to get event" error="rpc error: code = Unavailable desc = transport is closing" module=libcontainerd namespace=plugins.moby
juin 09 08:28:48 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:48.561006881+02:00" level=info msg="Waiting for containerd to be ready to restart event processing" module=libcontainerd namespace=plugins.moby
juin 09 08:28:48 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:48.561073442+02:00" level=info msg="blockingPicker: the picked transport is not ready, loop back to repick" module=grpc
juin 09 08:28:48 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:48.561082104+02:00" level=info msg="Processing signal 'terminated'"
juin 09 08:28:48 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:48.561154580+02:00" level=error msg="Failed to get event" error="rpc error: code = Unavailable desc = transport is closing" module=libcontainerd namespace=moby
juin 09 08:28:48 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:48.561203815+02:00" level=info msg="Waiting for containerd to be ready to restart event processing" module=libcontainerd namespace=moby
juin 09 08:28:48 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:48.561260042+02:00" level=info msg="blockingPicker: the picked transport is not ready, loop back to repick" module=grpc
juin 09 08:28:49 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:49.561549538+02:00" level=warning msg="grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}. Err :connection error: desc = \"transport: Error while dialing dial unix:///var/run/docker/containerd/containerd.sock: timeout\". Reconnecting..." module=grpc
juin 09 08:28:49 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:49.561569972+02:00" level=warning msg="grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}. Err :connection error: desc = \"transport: Error while dialing dial unix:///var/run/docker/containerd/containerd.sock: timeout\". Reconnecting..." module=grpc
juin 09 08:28:49 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:49.561643084+02:00" level=warning msg="grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}. Err :connection error: desc = \"transport: error while dialing: dial unix:///var/run/docker/containerd/containerd.sock: timeout\". Reconnecting..." module=grpc
juin 09 08:28:51 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:51.561906224+02:00" level=info msg="Container failed to stop after sending signal 2 to the process, force killing"
juin 09 08:28:52 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:52.242237324+02:00" level=warning msg="grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}. Err :connection error: desc = \"transport: Error while dialing dial unix:///var/run/docker/containerd/containerd.sock: timeout\". Reconnecting..." module=grpc
juin 09 08:28:52 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:52.242394792+02:00" level=error msg="failed to shut down container" container=f4f0ea22f72f8bf8c407dc418b6045daa0fef9ba12a938ef2229db36a751a923 error="Failed to stop container f4f0ea22f72f8bf8c407dc418b6045daa0fef9ba12a938ef2229db36a751a923 with error: Cannot kill container f4f0ea22f72f8bf8c407dc418b6045daa0fef9ba12a938ef2229db36a751a923: connection error: desc = \"transport: Error while dialing dial unix:///var/run/docker/containerd/containerd.sock: timeout\": unavailable"
juin 09 08:28:52 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:52.245104076+02:00" level=info msg="Daemon shutdown complete"
juin 09 08:28:52 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:52.245188060+02:00" level=warning msg="Error while testing if containerd API is ready" error="rpc error: code = Canceled desc = latest balancer error: connection error: desc = \"transport: Error while dialing dial unix:///var/run/docker/containerd/containerd.sock: timeout\""
juin 09 08:28:52 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:52.245250001+02:00" level=info msg="stopping event stream following graceful shutdown" error="context canceled" module=libcontainerd namespace=plugins.moby
juin 09 08:28:52 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:52.245305965+02:00" level=warning msg="Error while testing if containerd API is ready" error="rpc error: code = Canceled desc = grpc: the client connection is closing"
juin 09 08:28:52 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:52.245340236+02:00" level=info msg="stopping event stream following graceful shutdown" error="context canceled" module=libcontainerd namespace=moby
juin 09 08:28:52 davidcosse-NC-V3-571G-53236 dockerd[188666]: time="2021-06-09T08:28:52.245389607+02:00" level=info msg="stopping healthcheck following graceful shutdown" module=libcontainerd
juin 09 08:30:22 davidcosse-NC-V3-571G-53236 systemd[1]: docker.service: State 'final-sigterm' timed out. Killing.
juin 09 08:30:22 davidcosse-NC-V3-571G-53236 systemd[1]: docker.service: Killing process 188888 (containerd-shim) with signal SIGKILL.
juin 09 08:30:22 davidcosse-NC-V3-571G-53236 systemd[1]: docker.service: Killing process 188894 (containerd-shim) with signal SIGKILL.
juin 09 08:30:22 davidcosse-NC-V3-571G-53236 systemd[1]: docker.service: Killing process 188895 (n/a) with signal SIGKILL.
juin 09 08:30:22 davidcosse-NC-V3-571G-53236 systemd[1]: docker.service: Killing process 188897 (n/a) with signal SIGKILL.
juin 09 08:30:22 davidcosse-NC-V3-571G-53236 systemd[1]: docker.service: Killing process 189232 (n/a) with signal SIGKILL.
juin 09 08:30:22 davidcosse-NC-V3-571G-53236 systemd[1]: docker.service: Failed with result 'timeout'.
juin 09 08:30:22 davidcosse-NC-V3-571G-53236 systemd[1]: Stopped Docker Application Container Engine.
juin 09 08:30:22 davidcosse-NC-V3-571G-53236 systemd[1]: docker.service: Consumed 1.362s CPU time.
david Cossé 2021-06-09 08:32:59 CEST

Hardware: All => x86_64
CC: (none) => saveurlinux

Comment 1 Lewis Smith 2021-06-09 20:56:23 CEST
http://mirror.math.princeton.edu/pub/mageia/distrib/8/x86_64/media/core/updates_testing/docker-20.10.5-1.mga8.x86_64.rpm

You seem to be reporting this on docker from updates_testing; is this deliberate; say are you testing it? The current issued version is docker-19.03.15-1.mga8.
Did you have the same problem with that?

CC: (none) => lewyssmith
Status: NEW => NEEDINFO
Source RPM: http://mirror.math.princeton.edu/pub/mageia/distrib/8/x86_64/media/core/updates_testing/docker-20.10.5-1.mga8.x86_64.rpm => (none)

Lewis Smith 2021-06-09 20:57:20 CEST

Source RPM: (none) => docker-19.03.15-1.mga8.src.rpm

Comment 2 david Cossé 2021-06-09 22:45:11 CEST
(In reply to Lewis Smith from comment #1)
> http://mirror.math.princeton.edu/pub/mageia/distrib/8/x86_64/media/core/
> updates_testing/docker-20.10.5-1.mga8.x86_64.rpm
> 
> You seem to be reporting this on docker from updates_testing; is this
> deliberate; say are you testing it? The current issued version is
> docker-19.03.15-1.mga8.
> Did you have the same problem with that?

I upgrade to 20.10.5-1 because of https://bugs.mageia.org/show_bug.cgi?id=27251
but as I can remember I also had the problem.
Comment 3 Lewis Smith 2021-06-10 21:25:44 CEST
Thank you for this clarification.

Assigning the bug to Bruno, our Docker man.

Assignee: bugsquad => bruno
Status: NEEDINFO => NEW
CC: lewyssmith => (none)

Comment 4 Bruno Cornec 2021-10-16 02:19:48 CEST
Do you have a containerd process running ? What is your docker-containerd package version ?

Just pushed updates tonight to updates_testing for new docker and docker-containerd. Maybe you can try these ?

Status: NEW => NEEDINFO

Comment 5 david Cossé 2022-01-19 09:44:22 CET
(In reply to Bruno Cornec from comment #4)
> Do you have a containerd process running ? What is your docker-containerd
> package version ?
> 
> Just pushed updates tonight to updates_testing for new docker and
> docker-containerd. Maybe you can try these ?

Sorry for the late answering :(
I've got:
docker-20.10.9-3.mga8.x86_64
docker-compose-1.26.2-1.mga8.noarch
docker-containerd-1.5.8-1.mga8.x86_64

with this packages the bug remain present

I observe that manually stopping the container works.
the process that works is:
docker-compose down
Stopping agrosyst-latest-db ... done
Removing agrosyst-latest-db ... done
Removing network agrosyst_internal
sudo systemctl stop docker

The process that doesn't work is if I don't do the docker-compose down first.

As I don't do it before shuting down the OS, I've got the problem

Maybe it can help ? this is my docker compose
version: '3.8'

services:
  postgresql-agrosyst-latest:
    image: "postgres:12.1"
    shm_size: 2g
    volumes:
     - ./dump:/var/lib/postgresql/dump
     - ./data/postgres:/var/lib/postgresql/data
     - ./dump/PERFORMANCE_MASS:/var/lib/postgresql/exports
    environment:
     - POSTGRES_USER=agrosystuserprod
     - POSTGRES_PASSWORD=ecophyto2018
     - POSTGRES_DB=inra-agrosyst-latest
    container_name: agrosyst-latest-db
    restart: always
    networks:
     - internal
networks:
  internal:
    driver: bridge

This is the result of docker inspect:

[
    {
        "Id": "6d652d9730a3ee328362cc9f9b939574fed2cf753e9dafe3f5815e2c446a053c",
        "Created": "2022-01-19T08:24:57.812986465Z",
        "Path": "docker-entrypoint.sh",
        "Args": [
            "postgres"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 6449,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2022-01-19T08:24:58.039905822Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:cf879a45faaacd2806705321f157c4c77682c7599589fed65d80f19bb61615a6",
        "ResolvConfPath": "/var/cache/docker/containers/6d652d9730a3ee328362cc9f9b939574fed2cf753e9dafe3f5815e2c446a053c/resolv.conf",
        "HostnamePath": "/var/cache/docker/containers/6d652d9730a3ee328362cc9f9b939574fed2cf753e9dafe3f5815e2c446a053c/hostname",
        "HostsPath": "/var/cache/docker/containers/6d652d9730a3ee328362cc9f9b939574fed2cf753e9dafe3f5815e2c446a053c/hosts",
        "LogPath": "/var/cache/docker/containers/6d652d9730a3ee328362cc9f9b939574fed2cf753e9dafe3f5815e2c446a053c/6d652d9730a3ee328362cc9f9b939574fed2cf753e9dafe3f5815e2c446a053c-json.log",
        "Name": "/agrosyst-latest-db",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/home/davidcosse/docker/bd/agrosyst/dump:/var/lib/postgresql/dump:rw",
                "/home/davidcosse/docker/bd/agrosyst/dump/PERFORMANCE_MASS:/var/lib/postgresql/exports:rw",
                "/home/davidcosse/docker/bd/agrosyst/data/postgres:/var/lib/postgresql/data:rw"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "agrosyst_internal",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "always",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": [],
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "private",
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "shareable",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 2147483648,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": null,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/cache/docker/overlay2/9031b2d6cc1ab8d5e6b916672a6969dc5cc987c7f9e08af63b6ef9653b61329f-init/diff:/var/cache/docker/overlay2/f1e598a2d39cb176744e8c9b3ed8549066c7d88a303f4c8e571e6f53eccd3d13/diff:/var/cache/docker/overlay2/879af835698bed92ceeb20af7bb5715afa71854a4d05b782bbe584692b20266e/diff:/var/cache/docker/overlay2/9c84e37860b2febacfdc2cea0d0d4e56697f9522d8f8a633c291ecb157e08d7f/diff:/var/cache/docker/overlay2/41307b0a52a3c738c56bfde8047438c89c67d4d73a58e2743cf46e27b3f3faee/diff:/var/cache/docker/overlay2/4427a681edd493f2a1b611c55ded11691bcb2360c9ac04f757e08a9b473cf52a/diff:/var/cache/docker/overlay2/05de4ed5695866ac659f5db52dcf47eaada2a07f3a1b41886fb4e9a78d980be9/diff:/var/cache/docker/overlay2/1b88718917f7c97e456ce1600eab2f290e35ce30386c2fd07cf6ca780f342a2c/diff:/var/cache/docker/overlay2/45ba0c9a59ec1fb4064e35b0ba3cd5a5c312e8e1e5a2f0a7bb6bf8733a036508/diff:/var/cache/docker/overlay2/a75bc2b6f832873d4a3405f26917be6bce623f228cb710c7dd51a533c7420610/diff:/var/cache/docker/overlay2/be373815d6ef6550ec841e1255a5971fc9767c2fbce2c643483faff2cc448a9c/diff:/var/cache/docker/overlay2/8e3f9763d833602e8ca7c7f9e969927a773039a9b59255e1de97b3074f49bc2e/diff:/var/cache/docker/overlay2/d302d4d0068b173de4b9f87247add0e1bdf7af42b4f88e3b0961eb0f1180019c/diff:/var/cache/docker/overlay2/12268b922355990b2a501ca395d7dfe22fcc835c482a795c603beeb4fd957100/diff:/var/cache/docker/overlay2/50b9dff4cd36dd2a440f81734e32d5cb43a14005d1d26188be14fc357843e1f2/diff",
                "MergedDir": "/var/cache/docker/overlay2/9031b2d6cc1ab8d5e6b916672a6969dc5cc987c7f9e08af63b6ef9653b61329f/merged",
                "UpperDir": "/var/cache/docker/overlay2/9031b2d6cc1ab8d5e6b916672a6969dc5cc987c7f9e08af63b6ef9653b61329f/diff",
                "WorkDir": "/var/cache/docker/overlay2/9031b2d6cc1ab8d5e6b916672a6969dc5cc987c7f9e08af63b6ef9653b61329f/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/home/davidcosse/docker/bd/agrosyst/dump",
                "Destination": "/var/lib/postgresql/dump",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/home/davidcosse/docker/bd/agrosyst/dump/PERFORMANCE_MASS",
                "Destination": "/var/lib/postgresql/exports",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/home/davidcosse/docker/bd/agrosyst/data/postgres",
                "Destination": "/var/lib/postgresql/data",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],
        "Config": {
            "Hostname": "6d652d9730a3",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "5432/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "POSTGRES_USER=agrosystuserprod",
                "POSTGRES_PASSWORD=ecophyto2018",
                "POSTGRES_DB=inra-agrosyst-latest",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/postgresql/12/bin",
                "GOSU_VERSION=1.11",
                "LANG=en_US.utf8",
                "PG_MAJOR=12",
                "PG_VERSION=12.1-1.pgdg100+1",
                "PGDATA=/var/lib/postgresql/data"
            ],
            "Cmd": [
                "postgres"
            ],
            "Image": "postgres:12.1",
            "Volumes": {
                "/var/lib/postgresql/data": {},
                "/var/lib/postgresql/dump": {},
                "/var/lib/postgresql/exports": {}
            },
            "WorkingDir": "",
            "Entrypoint": [
                "docker-entrypoint.sh"
            ],
            "OnBuild": null,
            "Labels": {
                "com.docker.compose.config-hash": "4d253a65bb99c6b13caead450be4a74868061d0b365f0e183d90cf4adac9f7b6",
                "com.docker.compose.container-number": "1",
                "com.docker.compose.oneoff": "False",
                "com.docker.compose.project": "agrosyst",
                "com.docker.compose.project.config_files": "docker-compose.yml",
                "com.docker.compose.project.working_dir": "/home/davidcosse/docker/bd/agrosyst",
                "com.docker.compose.service": "postgresql-agrosyst-latest",
                "com.docker.compose.version": "1.26.2"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "c622d989971722b2e2df7d33b878e5c505061bebe6846ac9b27a87fdeeca24bb",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "5432/tcp": null
            },
            "SandboxKey": "/var/run/docker/netns/c622d9899717",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "agrosyst_internal": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "6d652d9730a3",
                        "postgresql-agrosyst-latest"
                    ],
                    "NetworkID": "a490f07efb2a4b92b5473cb9525ed6cd6c6ac768613d88937e5da929f9c18f21",
                    "EndpointID": "f6a5f1ca5ce9a01f0a4f40436b219c9f510fef9fe98d29de5db5be94ba17e1ff",
                    "Gateway": "172.18.0.1",
                    "IPAddress": "172.18.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:12:00:02",
                    "DriverOpts": null
                }
            }
        }
    }
]
david Cossé 2022-01-19 15:05:42 CET

Source RPM: docker-19.03.15-1.mga8.src.rpm => docker-20.10.9-3.mga8.src.rpm

Comment 6 david Cossé 2022-01-19 15:51:30 CET
Created attachment 13092 [details]
shutting down process
Comment 7 Alex Kotov 2022-01-20 12:04:59 CET
Hello, David Cossé. I don't understand everything in English, but judging by the screenshot 13092 (shutting down process), you have problems with a long shutdown/reboot when the container is active... Recently, I made myself a flash drive of the OpenMeeting server for quick distribution of the installation and there was a docker-image "Kurento-Media-Server". 

Its receipt and launch were carried out by the command:
---
docker run -d --name kms -p 8888:8888 --mount type=bind,source=/opt/openmeetings/webapps/openmeetings/data,target=/opt/openmeetings/webapps/openmeetings/data kurento/kurento-media-server

Accordingly, the autorun of the container on reboot was performed by the command:
---
docker update --restart always kms

After "reboot" command, I also got exactly what you have in the screenshot 13092 (shutting down process).

However, after reading this:
https://stackoverflow.com/questions/30449313/how-do-i-make-a-docker-container-start-automatically-on-system-boot

...I decided to change THE WAY THE CONTAINER STARTS on boot the system and made an additional service:

/etc/systemd/system/kms-docker.service
---
[Unit]
Description=Kurento-Media-Server container
Requires=docker.service
After=docker.service

[Service]
Restart=always
ExecStart=/usr/bin/docker start -a kms
ExecStop=/usr/bin/docker stop -t 2 kms

[Install]
WantedBy=default.target
---
systemctl enable kms-docker.service; systemctl restart kms-docker.service

Now everything is being completed normally.

In other words: the problem was not that docker.service is not shutting down, but the container inside it is not shutting down. The reason is the launch of this container by means of (docker update --restart always bla-bla-bla). 


It's just that in the future it will probably also be useful for me to find out why docker behaves this way. On the other hand, from a logical point of view, docker should not be engaged in the completion of certain processes inside the container, although how exactly this whole structure works and what depends on what there, it is completely unclear to me.

p.s. If I have translated something badly again and this is not what we are talking about here, please treat it with understanding. :)

With respect,
Alex

CC: (none) => alex_q_2000

Comment 8 david Cossé 2022-01-21 07:12:36 CET
(In reply to Alex Kotov from comment #7)
> Hello, David Cossé. I don't understand everything in English, but judging by
> the screenshot 13092 (shutting down process), you have problems with a long
> shutdown/reboot when the container is active... Recently, I made myself a
> flash drive of the OpenMeeting server for quick distribution of the
> installation and there was a docker-image "Kurento-Media-Server". 
> 
> Its receipt and launch were carried out by the command:
> ---
> docker run -d --name kms -p 8888:8888 --mount
> type=bind,source=/opt/openmeetings/webapps/openmeetings/data,target=/opt/
> openmeetings/webapps/openmeetings/data kurento/kurento-media-server
> 
> Accordingly, the autorun of the container on reboot was performed by the
> command:
> ---
> docker update --restart always kms
> 
> After "reboot" command, I also got exactly what you have in the screenshot
> 13092 (shutting down process).
> 
> However, after reading this:
> https://stackoverflow.com/questions/30449313/how-do-i-make-a-docker-
> container-start-automatically-on-system-boot
> 
> ...I decided to change THE WAY THE CONTAINER STARTS on boot the system and
> made an additional service:
> 
> /etc/systemd/system/kms-docker.service
> ---
> [Unit]
> Description=Kurento-Media-Server container
> Requires=docker.service
> After=docker.service
> 
> [Service]
> Restart=always
> ExecStart=/usr/bin/docker start -a kms
> ExecStop=/usr/bin/docker stop -t 2 kms
> 
> [Install]
> WantedBy=default.target
> ---
> systemctl enable kms-docker.service; systemctl restart kms-docker.service
> 
> Now everything is being completed normally.
> 
> In other words: the problem was not that docker.service is not shutting
> down, but the container inside it is not shutting down. The reason is the
> launch of this container by means of (docker update --restart always
> bla-bla-bla). 
> 
> 
> It's just that in the future it will probably also be useful for me to find
> out why docker behaves this way. On the other hand, from a logical point of
> view, docker should not be engaged in the completion of certain processes
> inside the container, although how exactly this whole structure works and
> what depends on what there, it is completely unclear to me.
> 
> p.s. If I have translated something badly again and this is not what we are
> talking about here, please treat it with understanding. :)
> 
> With respect,
> Alex

Hello, thanks for this tweak, it works for me.
What i did:

I created a file: /etc/systemd/system/agrosyst-latest-db.service
---
[Unit]
Description=agrosyst-latest-db container
Requires=docker.service
After=docker.service

[Service]
Restart=always
ExecStart=/usr/bin/docker start -a agrosyst-latest-db
ExecStop=/usr/bin/docker stop -t 2 agrosyst-latest-db

[Install]
WantedBy=default.target
---
sudo systemctl enable agrosyst-latest-db.service; sudo systemctl restart agrosyst-latest-db.service
---
Comment 9 Alex Kotov 2022-01-21 15:46:55 CET
It's good, David Cossé.

...сontinuation...

In the unit /usr/lib/systemd/system/docker.service missing line "KillMode=process" in the section [Service]. I made this file for myself from two files: from Mageia and openSuse and it works. It is desirable to show this to Bruno...

/usr/lib/systemd/system/docker.service
---
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.com
After=network.target
Wants=docker-storage-setup.service

[Service]
EnvironmentFile=-/etc/sysconfig/docker
EnvironmentFile=-/etc/sysconfig/docker-storage
EnvironmentFile=-/etc/sysconfig/docker-network
Environment=GOTRACEBACK=crash

# While Docker has support for socket activation (-H fd://), this is not
# enabled by default because enabling socket activation means that on boot your
# containers won't start until someone tries to administer the Docker daemon.
Type=notify
ExecStart=/usr/sbin/dockerd \
          $OPTIONS \
          $DOCKER_STORAGE_OPTIONS \
          $DOCKER_NETWORK_OPTIONS \
          $INSECURE_REGISTRY

ExecReload=/bin/kill -s HUP $MAINPID

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=1048576
LimitNPROC=infinity
LimitCORE=infinity

# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this property.
TasksMax=infinity

# Set delegate yes so that systemd does not reset the cgroups of docker containers
# Only systemd 218 and above support this property.
Delegate=yes

# Kill only the docker process, not all processes in the cgroup.
KillMode=process

# Restart the docker process if it exits prematurely.
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s

ExecStartPre=/usr/sbin/docker-network-cleanup
ExecStopPost=/usr/sbin/docker-network-cleanup

[Install]
WantedBy=multi-user.target
Comment 10 Thomas Backlund 2022-01-21 16:30:01 CET
Never ever modify packaged files residing in /usr/lib/systemd as they will be overwritten on package update.

if you want to customize it, copy it to /etc/systemd/system/ and modify that
Comment 11 david Cossé 2022-01-21 18:31:15 CET
(In reply to Thomas Backlund from comment #10)
> Never ever modify packaged files residing in /usr/lib/systemd as they will
> be overwritten on package update.
> 
> if you want to customize it, copy it to /etc/systemd/system/ and modify that

thanks !
Comment 12 Alex Kotov 2022-01-21 19:29:03 CET
Hi, Thomas. Thanks for the mention. :)

Sincerely,
Alex
Comment 13 Alex Kotov 2022-01-30 13:02:20 CET
And docker does not report its version:
> docker -v
Docker version unknown-version, build unknown-commit

I came across this when I was making Docker Manager for myself:
https://github.com/AKotov-dev/docker-manager

p.s. It may be useful for someone to experiment with docker.
Comment 14 Alex Kotov 2022-02-25 07:47:06 CET
@Bruno Cornec

Hi, dear Bruno Cornec.
Taking this opportunity, if it doesn't burden you too much, could you explain to me how you can build a minimal base image (+urpmi/dnf)? I need this to create a Mageia-9-Cauldron container.

At the moment I am using VDIConverter for this purpose (see Release): https://github.com/AKotov-dev/vdiconverter

I install a minimal Mageia-9-Cauldron system on the VirtualBox VM via a network installer (only recommended packages, without X and without documentation), then remove everything superfluous from it (urpmi --clean; userdel -r user; urpme --auto --auto-orphans plymouth mandi iptables shorewall-core acpi acpid alsa-utils aoss dhcp-client fuse-common usbutils wireless-regdb) I turn off the VM and make TAR from VDI (RootFS). Then I import the resulting tarball into Docker via DockerManager (see above) and I get a well-functioning container.

Everything turned out to be very convenient except for the size of the RootFS tarball (TAR). No matter how hard I tried to remove everything superfluous from the VM, I ran into the size of RootFS = 712MB. If you get a ready-made mageia:latest image directly from the DockerHub repository, then it has a size of 313MB. How did the assembly of this Docker image take place?

I read about bootstrap (rpm-bootstrap), but nothing worked out for me. Or is it different? I also looked at your GitHUB: https://github.com/bcornec/mageia-docker but I didn't understand what the procedure is and where to start?

Sincerely,
Alex
Comment 15 Alex Kotov 2022-02-25 22:44:43 CET
@Bruno Cornec

Sorry to bother you, Bruno. The information from Bug 28756 turned out to be useful and I got the following.

Creating an docker-image 'mageia-cauldron:minimal' (size = 337MB)
---
su/password

mkdir -p ./rootfs/{dev,proc,sys}; cd ./rootfs

mount /dev ./dev -o bind; mount /proc ./proc -o bind; mount /sys ./sys -o bind

urpmi.addmedia --urpmi-root . --distrib http://ftp.fau.de/mageia/distrib/cauldron/x86_64/

urpmi --auto --no-recommends --urpmi-root . basesystem-minimal urpmi

chroot .
echo "nameserver 1.1.1.1" > /etc/resolv.conf
rm -rf /var/lib/rpm; rpm --rebuilddb; rpm --import /etc/pki/rpm-gpg/RPM*
urpmi.update -a
urpmi --auto --no-recommends filesystem curl zstd locales-en #(your locales)
urpmi --clean

exit

umount -l ./proc ./dev ./sys

tar -cvf /mageia-cauldron-minimal.tar ./

docker import /mageia-cauldron-minimal.tar mageia-cauldron:minimal

# Hooray! :)

Sincerely,
Alex
Comment 16 david Cossé 2022-09-14 09:30:45 CEST
After fresh install on an other PC I'm facing same problem.
I applied the upper change on /etc/systemd/system/docker.service
and it works now correctly.
This is the diff:
> 
> # While Docker has support for socket activation (-H fd://), this is not
> # enabled by default because enabling socket activation means that on boot your
> # containers won't start until someone tries to administer the Docker daemon.
> Type=notify
17a23,27
> 
> ExecReload=/bin/kill -s HUP $MAINPID
> 
> # Having non-zero Limit*s causes performance problems due to accounting overhead
> # in the kernel. We recommend using cgroups to do container-local accounting.
20a31,47
> 
> # Uncomment TasksMax if your systemd version supports it.
> # Only systemd 226 and above support this property.
> TasksMax=infinity
> 
> # Set delegate yes so that systemd does not reset the cgroups of docker containers
> # Only systemd 218 and above support this property.
> Delegate=yes
> 
> # Kill only the docker process, not all processes in the cgroup.
> KillMode=process
> 
> # Restart the docker process if it exits prematurely.
> Restart=on-failure
> StartLimitBurst=3
> StartLimitInterval=60s
>
Comment 17 Bruno Cornec 2022-12-30 02:43:10 CET
I updated the service file for cauldron. So will be in mga9 with docker 20.10.22

Status: NEEDINFO => ASSIGNED

Bruno Cornec 2022-12-30 02:43:22 CET

CC: (none) => bruno

Comment 18 Florian Hubold 2023-01-30 13:06:51 CET
(In reply to Thomas Backlund from comment #10)

> if you want to customize it, copy it to /etc/systemd/system/ and modify that

FWIW, no need for manually copying something, systemctl edit --full someservice will automatically concatenate the effective configuration and create a new override under /etc/systemd/system/ for someservice.

CC: (none) => doktor5000

Comment 19 Bruno Cornec 2023-06-26 11:47:00 CEST
I think I have integrated the modifications reported here. Feel free to reopen if it's still not working for you:

$ docker -v
Docker version 20.10.22, build baeda1f82a10204ec5708d5fbba130ad76cfee49

Resolution: (none) => FIXED
Status: ASSIGNED => RESOLVED


Note You need to log in before you can comment on or make changes to this bug.