Description of problem: Update backport to version 5.5
Updated package uploaded for Mageia 6. Advisory: ======================== Updated rawtherapee package fixes bugs and enables new functionality: * Filter to remove striping artifacts caused by Phase Detection Auto Focus (PDAF) as seen in Sony cameras, and to remove banding artifacts caused by Nikon's too-aggressive in-camera PDAF correction. These are available for any camera which has a PDAF entry in camconst.json. * Ability to specify custom working color spaces through the workingspaces.json file. * Unbounded processing - allows you to decide whether out-of-gamut colors should get clipped. * Improved support for Canon mRaw format variants. * New Shadows/Highlights tool (replaced previous one). * Contrast threshold mask which divides an image into areas of high and low detail, allowing the effect of certain tools to be focused where it matters most and to mitigate the effect on areas where it would be undesirable, for example having the Sharpening tool affect only the in-focus subject without affecting the out-of-focus background. * Dual-demosaic algorithms, making use of the new contrast threshold mask, allowing one to use a combination of demosaicing algorithms where one is best for details and the other best for plain areas. * New color toning methods: - Grid, allowing you to separately tone the shadows and highlights using two points on a simple color grid. - Regions, allowing you to tone based on any number of masks. Supports functions from the American Society of Cinematographers Color Decision List (ASC CDL). * Resizable main histogram with scaling modes: Linear Log Log-Log * Support for Blackmagic and Canon Magic Lantern lj92 encoded files. * Allows you to specify how many border rows/columns to discard during demosaicing - those who shoot raw video at a standard resolution such as 1920x1080 will appreciate being able to preserve the dimensions. * New Soft Light tool which enhances contrast and saturation by emulating the effect of blending an image with a copy of itself in "soft light" blending mode in GIMP. ...and much more... Updated packages in core/backports_testing: ========================
Assignee: bugsquad => mrambo
Summary: Update backport to version 5.5 => Update rawtherapee backport to version 5.5
(finish the package list) Updated packages in core/backports_testing: ======================== rawtherapee-5.5-1.mga6 from rawtherapee-5.5-1.mga6.src.rpm Testing procedure https://bugs.mageia.org/show_bug.cgi?id=12693#c7
Assignee: mrambo => qa-bugs
MGA6-32 MATE on IBM Thinkpad R50e No installation issues Repeated tests as per bug 22125 Comment 3 $ rawtherapee (rawtherapee:17834): Gtk-WARNING **: Allocating size to gtkmm__GtkPaned 0xdd0b2c8 without calling gtk_widget_get_preferred_width/height(). How does the code know the size to allocate? Premature end of JPEG file JPEG datastream contains no image What I did was opening an the same ORF file, changing shadow lighting parameters and save as jpeg file. Despite the last sentence of the CLI feedback, the jpeg file was created and perfectly viewable in ristretto. Tried also the CLI: $ rawtherapee-cli -o p.tif -t -c P7212390.ORF RawTherapee, version 5.5, command line. Output is 16-bit integer. Processing: P7212390.ORF TIFFWriteDirectoryTagIfdIfd8Array: Attempt to write value larger than 0xFFFFFFFF in Classic TIFF file.. TIFFWriteDirectoryTagIfdIfd8Array: Attempt to write value larger than 0xFFFFFFFF in Classic TIFF file.. Error saving to: p.tif No tif file found, which is consistent. But this operation, on the same ORF file worked perfectly with the older version. I really don't know what to do with this update. I think withholding it is not a viable option, but I'm not really happy.
CC: (none) => herman.viaene
Working in parallel with Herman. rawtherapee does not seem to require any external libraries. This package really needs to be tested by somebody versed in image processing. QA tests shall cover basic functions only. The application can be run immersively from a gui or with the cli. The gui allows browsing files in whichever directory is specified at launch. e.g. for the current directory $ rawtherapee . Selected a Canon raw image and brightened it using one of the sliders at the right-hand side. Saved the file using the save icon at the bottom of the frame, which offers several formats for the output. Chose TIFF 16-bit, specified a name; saved immediately. The output file appeared in the ~/Pictures directory alongside a "sidecar" file containing the image profile parameters. There is a lot of information in that file. $ ll canon* -rw-r--r-- 1 lcl lcl 108278212 Jan 11 11:24 canon_1.tif -rw-r--r-- 1 lcl lcl 11446 Jan 11 11:24 canon_1.tif.out.pp3 $ cat canon_1.tif.out.pp3 | wc -l 623 The image could be displayed with ImageMagick. Tried the cli tool for image conversions. Convert RAW file to JPEG format. $ rawtherapee-cli -js3 -o kodak_1.jpg -c 'KODAK C603 C643 Format 420 CCDI0001.RAW' RawTherapee, version 5.5, command line. Output is 8-bit integer. Processing: KODAK C603 C643 Format 420 CCDI0001.RAW Cannot use camera white balance. $ rawtherapee-cli -n -o nikon_1.png -c RAW_NIKON_E5700_SRGB.NEF Processing: RAW_NIKON_E5700_SRGB.NEF $ rawtherapee-cli -b8 -t -o olympus_1.tif -c RAW_OLYMPUS_E420.ORF Output is 8-bit integer. Processing: RAW_OLYMPUS_E420.ORF All output images look fine with IM display. At this basic level rawtherapee works very well. Good for 64-bits.
CC: (none) => tarazed25
Whiteboard: (none) => MGA6-64-OK
Tried a 16-bit TIFF output to match Herman's command and it worked. $ rawtherapee-cli -t -o olympus_2.tif -c RAW_OLYMPUS_SP350.ORF Output is 16-bit integer. Processing: RAW_OLYMPUS_SP350.ORF $ display olympus_2.tif display: Incompatible type for "FileSource"; tag ignored. `TIFFFetchNormalTag' @ warning/tiff.c/TIFFWarnings/949. display: Incompatible type for "SceneType"; tag ignored. `TIFFFetchNormalTag' @ warning/tiff.c/TIFFWarnings/949. display: Wrong data type 3 for "GainControl"; tag ignored. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/949. Despite those warnings the displayed image looked fine.
From comment 3: > But this operation, on the same ORF file worked perfectly with the older > version. = a reversion. I agree with Herman's doubts, so withold the advisory in case... I only have 1 ginormous raw image to play with, so offer this devious suggestion (because bug attachments should not be huge, and I guess both Herman's & Len's examples are - unless you can post here URLs to get them): IF Herman can send to Len or myself his test raw image files (or post here the URLs to them), and the RawTherapee operations/commands used on them which did not work, we can try them at our end. Note that the unspecified image, first test, gave the same Premature end of JPEG file JPEG datastream contains no image as Herman's previous test - even though with a good output. Conversely, if Len could send Herman *his* test images (or post here the URLs to get them), and the corresponding operations/commands that worked for him, so Herman can see whether he gets the same success? All this is quickly done. RawTherapee is mature and basic software, it should not revert.
CC: (none) => lewyssmith
@Lewis, re comment 6. I had already thought about attachments but even after using xz one of them exceeded 2 MB. The others are probably similar. I don't exactly know how to make them available over the internet. I have not used my dropbox account for a while - forget how to open it. Have not had the time to investigate that. Later maybe.
Re comment 7. Managed to open Dropbox and have deposited 5 raw images there. Compression does very little good so we are stuck with anything up to 23MB. That is not a problem for me but it would be for people with lower bandwidth connections. More reading to do.
Files in https://drive.google.com/open?id=1hfwOEeP612L_181qmI5QDRTnFEBa3VlR Tell me when you've downloaded them, so I can delete the folder.
Re comment 9; Thanks Herman, done. Keep them until Lewis confirms. Mine are in DropBox - I have to send links to you by email for each image. Need some spare time for that. Later.
OK Herman. The four .ORF images you provided present no problems here. Converted all four to other image formats, PNG, JPEG, 8 and 16-bit TIFF. All displayed properly although display issued a warning for the 16-bit tif. display: Incompatible type for "FileSource"; tag ignored. `TIFFFetchNormalTag' @ warning/tiff.c/TIFFWarnings/949. display: Wrong data type 3 for "GainControl"; tag ignored. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/949. Despite that there was nothing wrong with the displayed image. So, this does look like a regression for 32-bit. Maybe the fix does not take account of differences in architectures. The "value larger than 0xFFFFFFFF" message means larger than -1 which is a classic sign of such an error; pushing a positive number beyond the 32-bit boundary switches it negative.
Correction, pushing a 32-bit integer past the 31-bit boundary switches it negative.
@ Len: yes, that looks like a good-oldfahioned overflow. Anyway, repeated the CLI commands with your files and got: $ rawtherapee-cli -t -o olympus_2.tif -c RAW_OLYMPUS_SP350.ORF RawTherapee, version 5.5, command line. Output is 16-bit integer. Processing: RAW_OLYMPUS_SP350.ORF TIFFWriteDirectoryTagIfdIfd8Array: Attempt to write value larger than 0xFFFFFFFF in Classic TIFF file.. TIFFWriteDirectoryTagIfdIfd8Array: Attempt to write value larger than 0xFFFFFFFF in Classic TIFF file.. Error saving to: olympus_2.tif and of course no tif file to be seen. $ rawtherapee-cli -js3 -o kodak_1.jpg -c KODAK_C603_C643_FORMAT422_CCDI0001.RAW RawTherapee, version 5.5, command line. Output is 8-bit integer. Processing: KODAK_C603_C643_FORMAT422_CCDI0001.RAW Cannot use camera white balance. $ display kodak_1.jpg jpg picture looks OK $ rawtherapee-cli -n -o nikon_1.png -c RAW_NIKON_E5700_SRGB.NEF RawTherapee, version 5.5, command line. Output is 8-bit integer. Processing: RAW_NIKON_E5700_SRGB.NEF $ display nikon_1.png Looks OK as well. So the tif conversion works OK with our two ORF files in 64-bit, but fail in 32-bit. Nice........
(In reply to Herman Viaene from comment #9) > Files in https://drive.google.com/open?id=1hfwOEeP612L_181qmI5QDRTnFEBa3VlR > Tell me when you've downloaded them, so I can delete the folder. Got them all (Len likewise), thanks a lot. Delete away. P.S. Thanks for persuing the problem you found. M6/x64: Cross-checking some of Herman's tests with his images. BEFORE update: rawtherapee-5.3-1.mga6 $ rawtherapee (rawtherapee:25132): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar [*n, but it started OK in the end] [but baffling to do basic operations like opening & saving] I selected a couple, fiddled randomely, and saved as JPEG (only 8-bit offered), with default (high) quality level - why the .jpg files are so big, perhaps. Premature end of JPEG file JPEG datastream contains no image Premature end of JPEG file JPEG datastream contains no image Standard errors! The output is however good. -rw-rw-r-- 1 lewis lewis 1922388 Ion 13 10:30 P7212389.jpg -rw-rw-r-- 1 lewis lewis 14127616 Ion 12 01:03 P7212389.ORF -rw-rw-r-- 1 lewis lewis 2678491 Ion 13 10:36 P7212391.jpg -rw-rw-r-- 1 lewis lewis 14127616 Ion 12 01:03 P7212391.ORF $ rawtherapee-cli -o x.tif -t -c P7212389.ORF RawTherapee, version 5.3, command line Processing: P7212389.ORF -rw-rw-r-- 1 lewis lewis 49934642 Ion 13 10:55 x.tif $ rawtherapee-cli -o y.tif -t -c P7212391.ORF RawTherapee, version 5.3, command line Processing: P7212391.ORF -rw-rw-r-- 1 lewis lewis 49934642 Ion 13 10:56 y.tif Both outputs good, but with console errors: $ display x.tif display: Incompatible type for "FileSource"; tag ignored. `TIFFFetchNormalTag' @ warning/tiff.c/TIFFWarnings/949. display: Wrong data type 3 for "GainControl"; tag ignored. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/949. -------------------------------------------------------------------- AFTER update: rawtherapee-5.5-1.mga6 [caught out by Backport] $ rawtherapee yielded a pop-up: "The default profile for raw photos could not be found or is not set Please check your profiles directory, it may be missing or damaged ${G]/Auro-Matched Curve - ISO Low will be used instead" OK loads the program; but with a new error msg as per Herman c3: (rawtherapee:24548): Gtk-WARNING **: Allocating size to gtkmm__GtkPaned 0x84b8940 without calling gtk_widget_get_preferred_width/height(). How does the code know the size to allocate? Fiddling P7212389.ORF and P7212391.ORF and saving as JPEG worked (with same errors) as before, also the command line TIF conversions. Second Len's OK. BUT... ======================= @ Mike Herman has definitely found a 32-bit problem. What to do? * This is posted as a Backport rather than Update. Does a jump from 5.3 to 5.5 warrant that? Can we change it to an Update? * Backports do not have advisories... * If the advisory in c1 is to go out somehow, we could add to it warning of possible problems on 32-bit systems. * Should we raise a bug?
Keywords: (none) => feedback
I had just pushed the new version in cauldron because it is the latest and we always put the latest in cauldron if we can. Now I'm wondering if cauldron 32 bit has the same problem. But anyway... My google-fu didn't turn up any known problem with rawtherapee 5.5 32 bit. I pushed a backport for mga6 because it adds a lot of nice things that I thought it would be nice to provide for those on mga6. At this point I don't really know what to do with this. We could just drop the backport IMO. Mageia 6 already has 5.3 backported and that is also a nice package. I guess we could just leave it at that. I don't know what the process looks like for that or whether I need to do something to make it happen. I didn't know backports didn't need any advisory or note. I just did it out of habit.
(In reply to Mike Rambo from comment #15) > Now I'm wondering if > cauldron 32 bit has the same problem. Something our M7 testing can (now should) specifically look for. > I pushed a backport for mga6 because it adds a lot of nice things that I > thought it would be nice to provide for those on mga6. Decent of you. > At this point I don't > really know what to do with this. We could just drop the backport IMO. > ... > I don't know what the process looks like for > that or whether I need to do something to make it happen. Best to ask on your side (packagers/devs) about these things. If it is decided to pull the backport, it would need removing from backports-testing; and this bug can be closed with an appropiate reason. > I didn't know backports didn't need any advisory or note. No worry. It *is* curious; I think they should. Herman & Len might have opinions about what to do.
Opened a bug upsteam. Appears Debian has run into the same thing. https://github.com/Beep6581/RawTherapee/issues/5141
(In reply to Mike Rambo from comment #17) > Opened a bug upsteam. Appears Debian has run into the same thing. Thanks Mike. So we are up with the big boys. I have had second thoughts about this backport: I think we *should* let it out (simply by validating it), even with this known bug - because of all the work done both to prepare and test it. It would be a shame to throw that away. The chances of a user falling foul of the fault are remote; and they can always, and easily, --downgrade the package if necessary. What do you others think? If you agree - please validate it! Already removed the feedback marker.
Keywords: feedback => (none)
fwiw - If/when upstream produces a fix for this I will package an update if it is in time for mga6. And since I expect cauldron suffers from the same problem, I hope they do. I agree that it should be safe to let it out. I know at least one other person has used the current backport aside from me, but I don't expect there are many, and even less on 32 bit. I don't think very many would be affected by this. But I'm a packager - I'll let someone in QA make that call.
Yes, agreeing with Mike and Lewis. Validating.
Keywords: (none) => validated_backport
Packages moved.
Resolution: (none) => FIXEDStatus: NEW => RESOLVEDCC: (none) => tmb
The rawtherapee folks have asked for a copy of the file which shows the problem. Could one of you make it available again?
https://drive.google.com/open?id=1OlmTW3i4qbk_KB_-9d5Z-lwIdU1VDxFz Please let me know when I can delete it again.
Herman, they have the photos and are working on the problem. You can take it down. Thanks.
Blocks: (none) => 29295