Bug 25888 - kdiff3 fails to compare huge directories: "Too many open files" error
Summary: kdiff3 fails to compare huge directories: "Too many open files" error
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: 7
Hardware: All Linux
Priority: Normal normal
Target Milestone: ---
Assignee: QA Team
QA Contact:
URL:
Whiteboard: MGA7-32-OK MGA7-64-OK
Keywords: advisory, validated_update
Depends on:
Blocks:
 
Reported: 2019-12-16 06:04 CET by diaphane d.
Modified: 2023-02-28 09:52 CET (History)
6 users (show)

See Also:
Source RPM: kdiff3-1.8.1-1.mga7.src.rpm
CVE:
Status comment:


Attachments

Description diaphane d. 2019-12-16 06:04:13 CET
Description of problem:
kdiff3 fails to compare huge directories. strace command shows "Too many open files" error.
It's probably related to https://bugs.kde.org/show_bug.cgi?id=408991

Version-Release number of selected component (if applicable):
kdiff3-1.8.1-1.mga7


How reproducible:

$ mkdir d1 d2
$ for ((i=0; i<1500; i++)); do echo az > d1/$i; echo qw > d2/$i; done
$ LC_ALL=C strace -e trace=openat kdiff3 d1/ d2/ |& tail -n +2590
openat(AT_FDCWD, "/tmp/d1/997", O_RDONLY|O_CLOEXEC) = -1 EMFILE (Too many open files)
openat(AT_FDCWD, "/tmp/d1/998", O_RDONLY|O_CLOEXEC) = -1 EMFILE (Too many open files)
openat(AT_FDCWD, "/tmp/d1/999", O_RDONLY|O_CLOEXEC) = -1 EMFILE (Too many open files)
openat(AT_FDCWD, "/usr/lib64/qt5/plugins/kf5/FrameworkIntegrationPlugin.so", O_RDONLY|O_CLOEXEC) = -1 EMFILE (Too many open files)
^C
Comment 1 David GEIGER 2019-12-16 06:46:00 CET
Should be fixed with kdiff3-1.8.1-1.1.mga7 in 7/Core/Updates_testing repo!

CC: (none) => geiger.david68210

Comment 2 diaphane d. 2019-12-16 15:36:40 CET
It works for me (paths with 12000+ subdirectories and 210000+ files).
Thanks.

$ cat /etc/mageia-release
Mageia release 7 (Official) for x86_64

$ rpm -q kdiff3
kdiff3-1.8.1-1.1.mga7
Comment 3 David GEIGER 2019-12-16 15:55:53 CET
Assigning to QA,


Advisory:
========================

Fix for "Too many open files" bug:

Sometimes when doing a kdiff3 between two folders, an error popup appear but this popup is empty and kdiff3 cannot be closed anymore (only solution is to kill it).

After a search, it appears the problem comes from the "fastFileComparison" function that opens lots of temporary files but never close them.
The fix has been successfully tested on the buggy directories.

When printing the "errors" variables of prepareListView, it shows 30 times the messages "Too many open files".


For reference:
https://phabricator.kde.org/D22294

========================

Packages in 7/core/updates_testing:
========================
kdiff3-1.8.1-1.1.mga7.i586.rpm
kdiff3-1.8.1-1.1.mga7.x86_64.rpm

Source RPM: 
========================
kdiff3-1.8.1-1.1.mga7.src.rpm

Assignee: bugsquad => qa-bugs

Comment 4 Brian Rockwell 2019-12-20 15:06:17 CET
Vbox - Plasma - 32bit

$ uname -a
Linux localhost 5.4.2-desktop-1.mga7 #1 SMP Thu Dec 5 18:22:56 UTC 2019 i686 i686 i386 GNU/Linux


The following package is going to be installed:
- kdiff3-1.8.1-1.1.mga7.i586

I pulled up some files and did a few compares.
I also compared directories.  

Nifty tool.

Works as designed.

CC: (none) => brtians1
Whiteboard: (none) => MGA7-32-OK

Comment 5 Brian Rockwell 2019-12-20 15:44:00 CET
Installed on 64 machine running Plasma

$ uname -a
Linux localhost 5.4.2-desktop-1.mga7 #1 SMP Thu Dec 5 17:40:00 UTC 2019 x86_64 x86_64 x86_64 GNU/Linu

Ran a compare of a directory between two devices that has a large number of files and directories.  kdiff did not fail.

It ran about 30 minutes and completed successfully.

Whiteboard: MGA7-32-OK => MGA7-32-OK MGA7-64-OK

Comment 6 Thomas Andrews 2019-12-21 19:45:48 CET
Validating. Advisory in Comment 3.

Keywords: (none) => validated_update
CC: (none) => andrewsfarm, sysadmin-bugs

Thomas Backlund 2019-12-24 12:49:02 CET

CC: (none) => tmb
Keywords: (none) => advisory

Comment 7 Mageia Robot 2019-12-24 13:25:41 CET
An update for this issue has been pushed to the Mageia Updates repository.

https://advisories.mageia.org/MGAA-2019-0243.html

Status: NEW => RESOLVED
Resolution: (none) => FIXED

Comment 8 jonas wade 2023-02-28 09:52:58 CET Comment hidden (spam)

CC: (none) => arunn3620


Note You need to log in before you can comment on or make changes to this bug.