Description of problem: When trying to do a recursive search of a pattern,known to exist in some files,in a directory the printed result is null.A command like "find searched_dir -type f -print | xargs grep -l 'search_pattern'" gives the expected result. Adding "-D" to the initial command "rgrep -r 'searched_pattern' searched_dir" doesn't show anything. The bug is located in the function "unix_is_dir" located in "src/rgrep.c" : this function always returns 0 because of a "return 0" as the first executable instruction. The attachment "jed-0.99.19-fix-bug-in-recurse.patch" is a patch that solves this problem Version-Release number of selected component (if applicable): jed-0.99.19-6.mga3 How reproducible: Easily reproductible Steps to Reproduce: 1.Starting from a directory containing sub-directories and files, start "rgep -r pattern dir" where "pattern" is a string known to exist in some files and "dir" is the current directory (any sub-dir) 2.The printed result is null as if the search pattern didn't exist 3.The expected result is given by "find dir -type f -print | xargs grep -l pattern" Reproducible: Steps to Reproduce:
Created attachment 5296 [details] Patch to fix "recursive search" bug of rgrep This patch is the result of "diff -Naur" between "rgrep.c~" (the old content) and "rgrep.c"(the fixed code)
Thanks for the report, and the patch. So, if I understand this correctly, rgrep is completely broken and has never worked in Mageia. Does rgrep provide some advantage over "grep -r" that makes it worth keeping this subpackage?
yes cf https://bugs.mageia.org/show_bug.cgi?id=10552
Keywords: (none) => PATCH, TriagedAssignee: bugsquad => shlomif
*** Bug 10552 has been marked as a duplicate of this bug. ***
CC: (none) => pablo
Patched packages uploaded for Mageia 3, Mageia 4, and Cauldron. Advisory: ---------------------------------------- The rgrep command, whose primary purpose is to search recursively, was unable to search recursively. This has been fixed. ---------------------------------------- Updated packages in core/updates_testing: ---------------------------------------- jed-0.99.19-6.1.mga3 jed-common-0.99.19-6.1.mga3 jed-xjed-0.99.19-6.1.mga3 rgrep-0.99.19-6.1.mga3 jed-0.99.19-7.1.mga4 jed-common-0.99.19-7.1.mga4 jed-xjed-0.99.19-7.1.mga4 rgrep-0.99.19-7.1.mga4 from SRPMS: jed-0.99.19-6.1.mga3.src.rpm jed-0.99.19-7.1.mga4.src.rpm
Version: 3 => 4Assignee: shlomif => qa-bugsWhiteboard: (none) => MGA3TOO
About to test this (rgrep, MGA4 64-bit), what is the relevence of Jed -which seems to be a text editor ?
CC: (none) => lewyssmith
The rgrep package comes from the jed SRPM.
Testing rgrep on MGA4 64-bit real hardware. Confirmed that as released rgrep does not recurse i.e. does not find strings in files in directories below the one specified to search. Updated to rgrep-0.99.19-7.1.mga4. Some improvement, but it does *not* work properly. It seems to descend one level only. Compare the outputs (grep is complicated because combining -c and -r causes it to output a count for every file searched even if the result is zero): $ grep -c -r Aspell /usr/share/* 2>/dev/null | grep -v ':0' /usr/share/abiword-3.0/strings/sc-IT.strings:1 ... /usr/share/abiword-3.0/strings/sk-SK.strings:1 /usr/share/doc/aspell/README:63 /usr/share/doc/hunspell/README:1 /usr/share/doc/hunspell/NEWS:1 /usr/share/doc/apache-commons-codec/aspell-mail.txt:1 /usr/share/doc/claws-mail/NEWS:1 /usr/share/doc/aspell-en/README:8 /usr/share/doc/mc/NEWS:1 /usr/share/doc/libreoffice-ure/LICENSE:2 /usr/share/doc/enchant/README:2 /usr/share/doc/autocorr-en/LICENSE:2 /usr/share/doc/gtkspell/ChangeLog:1 /usr/share/doc/vim-common/doc/develop.txt:1 /usr/share/doc/vim-common/doc/spell.txt:2 /usr/share/doc/libreoffice-opensymbol-fonts/LICENSE:2 /usr/share/doc/HTML/en/fundamentals/tasks.docbook:1 /usr/share/doc/HTML/en/kcontrol/spellchecking/index.docbook:1 /usr/share/doc/hunspell-en/README_en_GB.txt:1 /usr/share/help/C/gedit/gedit-spellcheck.page:1 /usr/share/hunspell/en_US.dic:1 /usr/share/hunspell/en_CA.dic:1 /usr/share/locale/en_GB/LC_MESSAGES/aspell.mo:6 /usr/share/locale/en_GB/LC_MESSAGES/kdelibs4.mo:2 /usr/share/man/whatis:3 /usr/share/mc/syntax/php.syntax:1 /usr/share/scribus/keysets/scribus14.xml:1 /usr/share/scribus/translations/scribus.pt.qm:1 ... /usr/share/scribus/translations/scribus.af.qm:1 /usr/share/scribus/dicts/README_en_GB.txt:1 /usr/share/scribus/dicts/README_en_EN.txt:1 and $ rgrep -c Aspell /usr/share/* /usr/share/hunspell/en_US.dic:1 /usr/share/hunspell/en_CA.dic:1 /usr/share/man/whatis:3 /usr/share/myspell/en_US.dic:1 /usr/share/myspell/en_CA.dic:1
Does it make any difference if you say "/usr/share" instead of "/usr/share/*" ?
(In reply to David Walser from comment #9) > Does it make any difference if you say "/usr/share" instead of > "/usr/share/*" ? Yes it makes a difference because of "*" that is expanded by Shell before sending the resulting list to rgrep. This list contains sub-dirs and files. For every sub-dir, "unix_is_dir" always returns 0 so that the sub-dir given as an argument will be considered an ordinary file => search pattern will obviously not be found. On the contrary, for every file, "unix_is_dir" returns 0 as expected and the search will take place; some result is printed if the pattern was found in that file. With "/usr/share" no expansion and this directory will be processed as if it were an ordinary file => no recurse & empty result even if the search pattern exists in some files under "/usr/share" or some sub-dir. Removing the initial "return 0" solves this unexpected behaviour.
In reply to Comment 9 & Comment 10: yes, I too wondered about the validity of /usr/share/* with regrep, but not soon enough. With the same grep O/P as in Comment 8 (plus a reference for future tests to reduce output):- $ grep -c -r Aspell /usr/share/* 2>/dev/null | grep -v ':0' | wc -l 128 $ rgrep -c Aspell /usr/share/ $ rgrep -c Aspell /usr/share $ rgrep Aspell /usr/share/ $ rgrep Aspell '/usr/share/' $ rgrep Aspell /usr/share $ rgrep Aspell '/usr/share' $ rgrep -c Aspell '/usr/share/' $ rgrep -c Aspell '/usr/share/*' $ rgrep -c Aspell '/usr/share' Nothing! I do not deny that I am doing something wrong. Tell me.
(In reply to Gilles Allard from comment #10) > (In reply to David Walser from comment #9) > > Does it make any difference if you say "/usr/share" instead of > > "/usr/share/*" ? > > Yes it makes a difference because of "*" that is expanded by Shell before > sending the resulting list to rgrep. > > This list contains sub-dirs and files. For every sub-dir, "unix_is_dir" > always returns 0 so that the sub-dir given as an argument will be considered > an ordinary file => search pattern will obviously not be found. > On the contrary, for every file, "unix_is_dir" returns 0 as expected and the > search will take place; some result is printed if the pattern was found in > that file. > > With "/usr/share" no expansion and this directory will be processed as if it > were an ordinary file => no recurse & empty result even if the search > pattern exists in some files under "/usr/share" or some sub-dir. > > Removing the initial "return 0" solves this unexpected behaviour. We already removed the initial return 0, your patch has been applied. Lewis was testing the update in Comment 8. I was curious if it makes a difference in the updated version. Lewis, I don't understand your answer in Comment 11.
CC: (none) => luigiwalser
David, in reply to Comment 12. What I meant was that none of the rgrep commands cited in my Comment 11 yielded any output at all. The commands are a console copy/paste. I tried quoting the directory pathname to exclude any shell influence. It made no difference. Put another way: it did not (for me) work at all. Anyone can try it looking for e.g. Aspell in /usr/share (which takes ages to traverse), and where in my case - as a control - grep -r found the string in 128 files.
Thanks Lewis, that cleared it up. Sorry Gilles, it appears that rgrep is more broken than just that spurious return. It appears every distro has the same jed version. You could look around to see if any of them have working patches for this.
CC: (none) => qa-bugsAssignee: qa-bugs => bugsquad
(In reply to David Walser from comment #14) > Thanks Lewis, that cleared it up. > > Sorry Gilles, it appears that rgrep is more broken than just that spurious > return. > > It appears every distro has the same jed version. You could look around to > see if any of them have working patches for this. The fact is that, for each example given by Lewis Smith in comment 11, the "-r" argument of rgrep is mandatory. Yes despite the name recursive is not the default behaviour of "rgrep". After adding a "-r" argument to these examples (or similar) a patched "rgrep" binary gives expected results. If "-r" switch is omitted "rgrep" considers its last argument as a file (if /usr/share) or processes only files in the list (if "/usr/share/*")
Thanks Gilles. Assigning back to QA. See Gilles' explanation.
CC: qa-bugs => (none)Assignee: bugsquad => qa-bugs
(In reply to David Walser from comment #16) > Thanks Gilles. Assigning back to QA. See Gilles' explanation. Trying to understand why rgrep (RecursiveGREP) is not recursive by default and, this given, why there is a "-N" argument (do not perform a recursive check) I had a look to the code (rgrep.c) and found that there is (may be) another bug. The recursive behaviour is under control of a static variable ("Do_Recursive") set to 0 by default; it is set to 1 by "-r" and to "-1" by "-N"; its value is tested by "Do_Recursive > 0" or "Do_Recursive >= 0" in main (lines 1051 & 1083) and by "Do_Recursive > 0" in function "grep_dir" (this function handles the recursive aspect of "rgrep"). I think this is not coherent with the other tests : "grep_dir" should contain "Do_Recursive >= 0" for rgrep to be recursive by default (then "-r" is useless !) Anyway this is a strange code (and without a single comment) !
(In reply to David Walser from comment #16) > Thanks Gilles. Assigning back to QA. See Gilles' explanation. The "-h" switch doesn't work too : the syntax of the strings ("HON_STR" & "HOFF_STR") surrounding the searched pattern do not meet the requirements of terminfo/termcap. But I think that this depend on the terminal type. Is "xterm" the default for Mageia ?
In Comment 11 I wrote:- > I do not deny that I am doing something wrong. Tell me. You did in Comment 15:- > The fact is that, for each example given by Lewis Smith in comment 11, the "-r" argument of rgrep is mandatory. Yes, I *had* read the rgrep Man page! But assumed wrongly that -r was sort of redundant. This is becoming farcical. I have repeated tests with grep & rgrep looking for Aspell in my /usr/share/ :- $ grep -l -r /usr/share/* 2>/dev/null | wc -l 128 $ rgrep -c -r Aspell /usr/share/ 2>/dev/null | wc -l 130 Pinning down the difference showed the 2 files found by rgrep but not grep:- lrwxrwxrwx 1 root root 21 Maw 16 19:43 /usr/share/myspell/en_CA.dic -> ../hunspell/en_CA.dic lrwxrwxrwx 1 root root 21 Maw 16 19:43 /usr/share/myspell/en_US.dic -> ../hunspell/en_US.dic which implies that rgrep counted linked files while grep did not. Adjusting grep accordingly:- $ grep -l -R Aspell /usr/share/* 2>/dev/null | wc -l 134 ! However, rgrep has its own link parameter:- $ rgrep -l -r -F Aspell /usr/share/ 2>/dev/null | wc -l 134 Enough! The rgrep update does what it says (remembering the necessity of -r). The advisory might well include this fact. In fact, rgrep shoots itself in the foot in its own DESCRIPTION [my emphasis]:- "rgrep, unlike grep(1) and egrep(1) rgrep *has the ability* to recursively descend directories. The traditional way of performing this kind of search on Unix systems utilizes the find(1) command in conjunction with grep(1). However, this results in very poor performance." which is simply no longer true. The only advantage I found for rgrep over grep was that there is much less 'noise' STDERR output. One could drop it if it were not included with Jed which presumably *is* useful.
Whiteboard: MGA3TOO => MGA3TOO MGA4-64-OK
We could drop the rgrep subpackage if jed doesn't need it. It's quite the silly command.
Should we proceed with this update and push the somewhat improved rgrep, or should another update be issued to remove rgrep altogether? I guess it would be better no to remove rgrep from stable distros, since people might be using it even though it's half broken.
CC: (none) => remi
It wouldn't hurt too much to kill it since it's totally broken, but yes, that's not something we would normally do. I guess this update at least makes it moderately useful. I still think it's a silly pointless command and we should get rid of it in Cauldron, but Manuel pointed out in Comment 3 that some might want to keep it. I guess you could ask on the dev ml if we can kill it.
Validating this. See the discussion in the QA meeting: http://meetbot.mageia.org/mageia-qa/2014/mageia-qa.2014-07-31-19.02.log.html#l-30 The advisory still needs to be uploaded. Please push this to core/updates for Mageia 3 and Mageia 4.
Keywords: PATCH, Triaged => validated_updateCC: (none) => sysadmin-bugs
Advisory uploaded.
Whiteboard: MGA3TOO MGA4-64-OK => MGA3TOO MGA4-64-OK advisory
Update pushed. http://advisories.mageia.org/MGAA-2014-0147.html
Status: NEW => RESOLVEDCC: (none) => mageiaResolution: (none) => FIXED