Description of problem: I got warnings about disk space on my root partition. The cause was /var/log/journal with gigabytes of logs. All options in /etc/systemd/journald.conf are commented out so journalctl doesn't keep a certain maximum size. We should be smarter about this so that logs don't fill up the root partition. Version-Release number of selected component (if applicable): 217.11.1 How reproducible: Observed it on my own PC Steps to Reproduce: 1. Use pc normally, watch logs grow slowly and steadily 2. 3. Reproducible: Steps to Reproduce:
I thought systemd would only allow it to use a certain % of disk space on its partition. If it's really unbounded, that's a problem.
Assignee: bugsquad => mageia
The commented out options show the built in defaults - do not assume that this means there are no defaults!! Please see the various journalctl options to list the storage usage (not at a computer right now so cannot look up the commands) and the defaults. Note that when journals starts it logs a line showing the current calculations.
(In reply to Colin Guthrie from comment #2) > The commented out options show the built in defaults - do not assume that > this means there are no defaults!! Right, but whatever the defaults are, they allowed the journal to grow to > 1GB whereas the remaining space on my root partition was < 1GB.
So in order to investigate this any further, I'll need information about your system. What is your partition size, what is the free space etc. A listing of the files and folders inside /var/log/journal would also be useful (including their full path, dates and sizes). Also the output from "journalctl --disk-usage" (as root) and the output from "journalctl -b MESSAGE_ID=ec387f577b844b8fa948f33cad9a75e6" (also as root). As you'll already have read in journald.conf man page, the default is to use 10% of disk space for log data, but also to leave 15% free disk space. Journal files should then be rotated (and dropped) accordingly automatically. This kinda implies that the partition is ~10Gb in size (which seems kinda normal), but the fact they do not leave ~1.5Gb free is the troubling part. Of course depending on the actual size of the /var partition, the approximate sizes you give in your report could make this all work. If, however, the sizes really do not compute, I'm wondering if there are stale, non-journal files in the /var/log/journal tree, that are messing up the stats (hence why the --disk-usage output would be useful). This can happen if e.g. the machine-id has somehow been changed or you use nspawn with --link-journal=host mode. Anyway, it's very hard to tell for now without more info as the report assumes a problem but doesn't provide any size or metrics to verify it in any way. Once you provide this further information, we can look into whether there is a bigger problem. Cheers!
Yes, I should've documented it better. Unfortunately it's too late for that now, because I already deleted the files that were causing the problem. Anyway, here's some of the info you asked for (everything is looking pretty normal ATM): $ ls -lh /var/log/journal/1d86337a64994802ad13fa599557d420/ totaal 136M -rw-r----- 1 root systemd-journal 56M nov 6 14:01 system@000523ded7b5d907-f77e6791156d44db.journal~ -rw-r----- 1 root systemd-journal 24M nov 9 21:47 system.journal -rw-r-----+ 1 root systemd-journal 16M nov 9 21:46 user-1000.journal -rw-r-----+ 1 root systemd-journal 32M nov 9 21:10 user-1001.journal -rw-r-----+ 1 root systemd-journal 8,0M nov 9 20:57 user-65534.journal [reinout@linux ~]$ sudo journalctl --disk-usage Journals take up 136.1M on disk. [reinout@linux ~]$ sudo journalctl -b MESSAGE_ID=ec387f577b844b8fa948f33cad9a75e6 -- Logs begin at ma 2015-10-19 15:59:38 CEST, end at ma 2015-11-09 21:56:41 CET. -- nov 09 14:04:41 localhost systemd-journal[588]: Runtime journal is using 8.0M (max allowed 355.3M, trying to leave 532.9M free of 3.4G available â current limit 355.3M). nov 09 14:04:41 localhost systemd-journal[588]: Permanent journal is using 128.0M (max allowed 1.2G, trying to leave 1.0G free of 2.0G available â current limit 1.0G). I don't have a separate /var partition: [reinout@linux ~]$ df -h Bestandssysteem Grootte Gebruikt Besch Geb% Aangekoppeld op devtmpfs 3,5G 0 3,5G 0% /dev tmpfs 3,5G 824K 3,5G 1% /dev/shm tmpfs 3,5G 1,3M 3,5G 1% /run /dev/dm-0 13G 8,9G 2,1G 82% / tmpfs 3,5G 0 3,5G 0% /sys/fs/cgroup tmpfs 3,5G 92K 3,5G 1% /tmp /dev/mapper/crypt_sda6 110G 42G 67G 39% /home tmpfs 711M 16K 711M 1% /run/user/1001 tmpfs 711M 16K 711M 1% /run/user/1000 tmpfs 711M 12K 711M 1% /run/user/984
@ Reinout I assume this got fixed, because it is very long ago that I heard about journalctl logs eating the last bit of space on / Please reopen if I'm wrong and change the Version (near the top, at the left) to Mageia 6 or Cauldron, whichever you use.
Status: NEW => RESOLVEDResolution: (none) => FIXEDCC: (none) => marja11
CC: (none) => arkr17997