Bug 16372

Summary:	the 'file' command cannot be trusted
Product:	Mageia	Reporter:	John L. ten Wolde <johnltw>
Component:	RPM Packages	Assignee:	All Packagers <pkg-bugs>
Status:	RESOLVED OLD	QA Contact:
Severity:	normal
Priority:	Normal	CC:	davidwhodgins, johnltw, luigiwalser, marja11
Version:	5
Target Milestone:	---
Hardware:	All
OS:	Linux
Whiteboard:	only_in:MGA5
Source RPM:	file-5.19-10.mga5	CVE:
Status comment:

Description John L. ten Wolde 2015-07-13 19:11:58 CEST

The file command (version 5.19) that shipped with Mageia 5 is wonky.  For example, it mistakenly identifies over half of my LibreOffice *.odt documents as "Microsoft OOXML".

I suspect the issue at hand to be this:  http://bugs.gw.com/view.php?id=360

If so then upstream has already fixed it in version 5.20.

As hinted at in the link above, the currently borked version may lead to all sorts of unfortunate and unforeseen side-effects with respect to libmagic.  In my case it breaks file-validation in a custom script.


How reproducible:  Always.

Reproducible: 

Steps to Reproduce:

John L. ten Wolde 2015-07-13 19:12:10 CEST

CC: (none) => johnltw

Comment 1 David Walser 2015-07-30 17:30:34 CEST

The patch to fix that particular issue is this commit:
https://github.com/file/file/commit/bf3fea9e6c0b18e9a645d2a796a1e3ae29be5ee5

which is already in our file version in Mageia 5.

If you can locate the correct patch, I'll apply it.

CC: (none) => luigiwalser

Comment 2 Marja Van Waes 2016-10-20 21:27:08 CEST

@ John

Do you still have this problem with file-5.19-10.1.mga5 ?

Keywords: (none) => NEEDINFO
CC: (none) => marja11
Assignee: bugsquad => pkg-bugs

Comment 3 John L. ten Wolde 2016-10-21 01:24:20 CEST

Hi Marja. Yes, unfortunately the issue remains.

$ rpm -q file-5.19-10.1.mga5
file-5.19-10.1.mga5

$ file -v
file-5.19
magic file from /usr/share/misc/magic

$ file example.odt
example.odt: Microsoft OOXML

$ xxd example.odt | head -5
0000000: 504b 0304 1400 0008 0000 28a0 953e 5ec6 PK........(..>^.
0000010: 320c 2700 0000 2700 0000 0800 0000 6d69 2.'...'.......mi
0000020: 6d65 7479 7065 6170 706c 6963 6174 696f metypeapplicatio
0000030: 6e2f 766e 642e 6f61 7369 732e 6f70 656e n/vnd.oasis.open
0000040: 646f 6375 6d65 6e74 2e74 6578 7450 4b03 document.textPK.

The reason this was even an issue for me, and how I discovered it, was that a script I had written prior to Mageia updating to file 5.19 stopped working correctly. My script parses ODT files and used to use the file command to validate its input, but suddenly it started complaining the files I was feeding it were invalid. Until I noticed file's broken output, I thought the bug was with my script. I've since patched my script to grep the first few bytes of its input for the "vnd.oasis.opendocument.text" line you see above in the output of xxd and validate on that instead. While that's an effective workaround for my script, the file command continues to produce misleading output and sadly, can't be relied on to correctly identify ODT (or perhaps any ODF?) files.

I happen to have 16 ODT files in my root Documents directory at present. For exactly half of them, file correctly reports "OpenDocument Text" for the other half it says "Microsoft OOXML". There's nothing obvious in the nature of the files to suggest why file would be getting this wrong. If you check a dozen or so ODT files on your own machines, I can't imagine your results would be any different.

Thanks for checking in on the status of this bug, and I hope this more verbose response is useful in some way. If Mageia 6 ships with file 5.20 or higher, and that fixes this issue, I'll be sure to report it.

Comment 4 Marja Van Waes 2016-10-21 09:13:39 CEST

Hi John,

Thanks for the feedback :-)

I'm using Cauldron, with file-5.25-4.mga6, and can't reproduce the issue with the 9 .odt files I found: they were all correctly identified by file: 
       <filename>.odt: OpenDocument Text.

So Mageia 6 won't have the problem.

Keywords: NEEDINFO => (none)

Samuel Verschelde 2016-10-21 10:27:17 CEST

Keywords: (none) => NOT_IN_CAULDRON

Comment 5 John L. ten Wolde 2016-10-26 01:04:13 CEST

@Marja: Thank you, that's excellent news. When Mageia 6 hits general release, I'll happily put this bug to rest.

Cheers.

Comment 6 Dave Hodgins 2016-10-26 01:34:18 CEST

Note that the bug should remain open until either Mageia 5 is fixed, or it
reaches end of life.

CC: (none) => davidwhodgins

Samuel Verschelde 2016-10-26 21:58:31 CEST

Keywords: NOT_IN_CAULDRON => (none)
Whiteboard: (none) => only_in:MGA5

Comment 7 Marja Van Waes 2018-05-03 07:51:45 CEST

(In reply to Dave Hodgins from comment #6)
> Note that the bug should remain open until either Mageia 5 is fixed, or it
> reaches end of life.

Hi John,

We regret if this issue didn't get fixed in Mageia 5.

Mageia 5 has officially reached its End of Life on December 31st, 2017 https://blog.mageia.org/en/2017/11/07/mageia-5-eol-postponed/
It only continued to get important security updates since then, but non-security bugs have no chance of still getting fixed.

Closing as OLD.

Status: NEW => RESOLVED
Resolution: (none) => OLD