Bug 25888

Summary: kdiff3 fails to compare huge directories: "Too many open files" error
Product: Mageia Reporter: diaphane d. <freedom.unix>
Component: RPM PackagesAssignee: QA Team <qa-bugs>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Normal CC: andrewsfarm, arunn3620, brtians1, geiger.david68210, sysadmin-bugs, tmb
Version: 7Keywords: advisory, validated_update
Target Milestone: ---   
Hardware: All   
OS: Linux   
Whiteboard: MGA7-32-OK MGA7-64-OK
Source RPM: kdiff3-1.8.1-1.mga7.src.rpm CVE:
Status comment:

Description diaphane d. 2019-12-16 06:04:13 CET 
Description of problem:
kdiff3 fails to compare huge directories. strace command shows "Too many open files" error.
It's probably related to https://bugs.kde.org/show_bug.cgi?id=408991

Version-Release number of selected component (if applicable):
kdiff3-1.8.1-1.mga7


How reproducible:

$ mkdir d1 d2
$ for ((i=0; i<1500; i++)); do echo az > d1/$i; echo qw > d2/$i; done
$ LC_ALL=C strace -e trace=openat kdiff3 d1/ d2/ |& tail -n +2590
openat(AT_FDCWD, "/tmp/d1/997", O_RDONLY|O_CLOEXEC) = -1 EMFILE (Too many open files)
openat(AT_FDCWD, "/tmp/d1/998", O_RDONLY|O_CLOEXEC) = -1 EMFILE (Too many open files)
openat(AT_FDCWD, "/tmp/d1/999", O_RDONLY|O_CLOEXEC) = -1 EMFILE (Too many open files)
openat(AT_FDCWD, "/usr/lib64/qt5/plugins/kf5/FrameworkIntegrationPlugin.so", O_RDONLY|O_CLOEXEC) = -1 EMFILE (Too many open files)
^C
Comment 1 David GEIGER 2019-12-16 06:46:00 CET 
Should be fixed with kdiff3-1.8.1-1.1.mga7 in 7/Core/Updates_testing repo!

CC: (none) => geiger.david68210

Comment 2 diaphane d. 2019-12-16 15:36:40 CET 
It works for me (paths with 12000+ subdirectories and 210000+ files).
Thanks.

$ cat /etc/mageia-release
Mageia release 7 (Official) for x86_64

$ rpm -q kdiff3
kdiff3-1.8.1-1.1.mga7
Comment 3 David GEIGER 2019-12-16 15:55:53 CET 
Assigning to QA,


Advisory:
========================

Fix for "Too many open files" bug:

Sometimes when doing a kdiff3 between two folders, an error popup appear but this popup is empty and kdiff3 cannot be closed anymore (only solution is to kill it).

After a search, it appears the problem comes from the "fastFileComparison" function that opens lots of temporary files but never close them.
The fix has been successfully tested on the buggy directories.

When printing the "errors" variables of prepareListView, it shows 30 times the messages "Too many open files".


For reference:
https://phabricator.kde.org/D22294

========================

Packages in 7/core/updates_testing:
========================
kdiff3-1.8.1-1.1.mga7.i586.rpm
kdiff3-1.8.1-1.1.mga7.x86_64.rpm

Source RPM: 
========================
kdiff3-1.8.1-1.1.mga7.src.rpm

Assignee: bugsquad => qa-bugs

Comment 4 Brian Rockwell 2019-12-20 15:06:17 CET 
Vbox - Plasma - 32bit

$ uname -a
Linux localhost 5.4.2-desktop-1.mga7 #1 SMP Thu Dec 5 18:22:56 UTC 2019 i686 i686 i386 GNU/Linux


The following package is going to be installed:
- kdiff3-1.8.1-1.1.mga7.i586

I pulled up some files and did a few compares.
I also compared directories.  

Nifty tool.

Works as designed.

CC: (none) => brtians1
Whiteboard: (none) => MGA7-32-OK

Comment 5 Brian Rockwell 2019-12-20 15:44:00 CET 
Installed on 64 machine running Plasma

$ uname -a
Linux localhost 5.4.2-desktop-1.mga7 #1 SMP Thu Dec 5 17:40:00 UTC 2019 x86_64 x86_64 x86_64 GNU/Linu

Ran a compare of a directory between two devices that has a large number of files and directories.  kdiff did not fail.

It ran about 30 minutes and completed successfully.

Whiteboard: MGA7-32-OK => MGA7-32-OK MGA7-64-OK

Comment 6 Thomas Andrews 2019-12-21 19:45:48 CET 
Validating. Advisory in Comment 3.

Keywords: (none) => validated_update
CC: (none) => andrewsfarm, sysadmin-bugs

Thomas Backlund 2019-12-24 12:49:02 CET

CC: (none) => tmb
Keywords: (none) => advisory

Comment 7 Mageia Robot 2019-12-24 13:25:41 CET 
An update for this issue has been pushed to the Mageia Updates repository.

https://advisories.mageia.org/MGAA-2019-0243.html

Status: NEW => RESOLVED
Resolution: (none) => FIXED

Comment 8 jonas wade 2023-02-28 09:52:58 CET Comment hidden (spam)

CC: (none) => arunn3620