On Sun, May 1, 2016 at 10:16 AM, Ingo Liebhardt <ingo.liebha...@ziggo.nl> wrote:
> Hi all, > > I see that a lively discussion was started by my mail. > I’ll try to combine as many replies into one single e-mail as possible. > > 1. The trees and branches as test image: the colour artifacts in this > image are larger areas, not just one or two pixels (like in the black tie). > This makes it more difficult to treat them via noise reduction. That’s one > reason I chose this image for torture testing. > > 2. All the new test images from the X70: thanks a lot for these, they are > great for testing! I’m still busy with the bird at this time, but I’ll go > through them one by one as soon as I find some time. > > 3. The bird image: this is very interesting and revealing! I have a > prototypic implementation of my algorithm in matlab (slow and can handle > only very small images). I took your OOC JPEG, mosaiced, and demosaiced the > most critical part of the bird again. No problematic dots, > see ‚bird_matlab‘ in the usual dropbox location. Bird_ml_test shows a CPSNR > of 42.48 dB. Folks dealing with demosaicing might identify this as a > very respectable value for a difficult image area. So my conclusion is that > the problem is not with my algorithm per se, but with my openCL > implementation. I continue working on this topic for the time being. As it > appears to be only the implementation and not the algorithm per se it > _must_ be solvable. > > 4. My contrast curve and colors: I know it’s everything but ideal, but it > was a quick and dirty way to get an ‚infrastructure‘ for my implementation > by using dcraw. For the time being, this is also my best way of continuing > the tests, until I get something more evolved. > > 5. This brings me to the question of Dan: yes, it should be possible to > get my algorithm into darktable to enable a broader range of testing. This > is ultimately still my goal. I’m just not there yet, so please be a bit > patient, especially since this is just a little hobby of mine. I’m not a > programmer (at least not in the classical sense) by profession, and all I > know about demosaicing is self-taught. However, after re-checking the bird > image as described above, I’m quite confident things are moving into the > right direction. > > Cheers, > Ingo > > > > Ingo, I've added a 5th image to my set (the puppy) which shows the grid/speckle artifact much more strongly than the image #1 (the bird) does. As soon as I saw this puppy I knew he was going to be trouble for X-Trans... http://www.nevermindhim.com/fuji-xtrans-samples A bit of sharpening helps make the grid pop out. If you set DT's demosaic mode to VNG, you'll see a much stronger zipper artifact along the same lines that Markesteijn shows the speckles on. > > Am 29.04.2016 um 02:11 schrieb J. Liles <malnour...@gmail.com>: > > > > On Wed, Apr 27, 2016 at 5:14 PM, J. Liles <malnour...@gmail.com> wrote: > >> >> >> On Wed, Apr 27, 2016 at 3:36 PM, J. Liles <malnour...@gmail.com> wrote: >> >>> >>> >>> On Wed, Apr 27, 2016 at 1:53 PM, J. Liles <malnour...@gmail.com> wrote: >>> >>>> >>>> >>>> On Wed, Apr 27, 2016 at 11:56 AM, J. Liles <malnour...@gmail.com> >>>> wrote: >>>> >>>>> >>>>> >>>>> On Wed, Apr 27, 2016 at 11:49 AM, johannes hanika <hana...@gmail.com> >>>>> wrote: >>>>> >>>>>> [..] >>>>>> >>>>>> > You've lost the color of the sunset on the lettering on the clock >>>>>> face >>>>>> > though. >>>>>> >>>>>> oh, i bet that's the classic chrome film style. i just applied one >>>>>> from the list to get a fuji-like tonecurve, since those affect >>>>>> saturation of colours as well as the contrast a lot. >>>>>> >>>>>> > This is not the best test scene because it doesn't appear to >>>>>> actually >>>>>> > contain any vibrant colors. >>>>>> >>>>>> i think for a certain kind of artefact it's just fine.. these fine >>>>>> branches in front of a brighter sky already produce quite terrible >>>>>> colour fringes (if i turn of the denoising that is). >>>>>> >>>>>> > A better scene would have real green, magenta >>>>>> > and cyan objects, preferably with fine patterns. That would make it >>>>>> more >>>>>> > apparent when the processing is not producing color artifacts vs. >>>>>> just >>>>>> > smoothing all of the colors. >>>>>> >>>>>> the thing is, this sensor cannot capture colour information beyond a >>>>>> certain frequency, because the red/blue pixels are spaced so wide >>>>>> apart from each other. >>>>>> >>>>>> >>>>> True, but I'm inclined to use the Fuji camera generated JPGs as a >>>>> baseline for that: A minimum amount of luma and chroma resolution. With my >>>>> Bayer images, I can get much more detail out of them by processing the >>>>> RAWs >>>>> in Darktable comparted to what the camera JPGs contain. >>>>> >>>>> I should think that the same should be true of X-Trans, since >>>>> in-camera processing is presumably optimized for speed rather than >>>>> quality. >>>>> >>>>> >>>>> We always need to reference to the camera JPGs... If the RAW >>>>> processing is producing less color detail than the JPG, then there's still >>>>> room for improvement (and probably a lot of it). >>>>> >>>>> >>>> >>>> >>>> I've added a third image to the problem image set at: >>>> >>>> http://www.nevermindhim.com/fuji-xtrans-samples >>>> >>>> I wanted to include both a subject prone to demoasicing artifacts and >>>> subjects with fine color detail in the same image. >>>> >>>> If you play around with it in DT, I think you'll find that any setting >>>> that eliminates the color artifacts on the black tie will smear the color >>>> of the other ties, which does not happen in the camera JPG. >>>> >>>> >>> Replying to myself again... Actually with a bit more tweaking and a >>> different approach (which is actually similar to the approach that I use to >>> denoise high ISO Bayer images), I've been able to pretty closely match my >>> third problem image camera JPG in DT. The same style applied to Ingo's >>> backlit tree image also looks quite good. No discernable color artifacts in >>> the branches (even with saturation cranked to the maximum) without washing >>> out the color in other image areas. You'll notice in my sample #3 that even >>> the Fuji JPEG engine leaves a blob of a magenta artifact on the tie... I >>> didn't attempt to take it any fruther. >>> >>> Here's the style: >>> >>> >>> http://www.nevermindhim.com/files/fuji-xtrans-samples/X-Trans%20II%20Denoise%20200.dtstyle >>> >>> Even though it says 200, it seems to work pretty well on higher ISOs (up >>> to 3200), but those images could probably use a little luma denoising too. >>> >>> I can't post sample output for the backlit tree image because DT crashes >>> when exporting that one for some reason. >>> >>> >> I'm now less concerned about the color artifacts, as it seems that >> denoising can in fact clean them up without causing too much smearing (not >> more than the in-camera JPGs anyway). >> >> But what about the grid/speckles? >> >> I've put a gimp XCF here with two layers in it (warning: 100MB download): >> >> http://www.nevermindhim.com/files/fuji-xtrans-samples/bird.xcf >> >> By toggling the visibility of the top layer, perhaps you can see what I'm >> referring to. I have some other images which exhibit this also. Adding >> enough luminance NR to remove these speckles completely obliterates detail. >> >> >> >> > > > I've added 4th problem image to the set at: > > http://www.nevermindhim.com/fuji-xtrans-samples > > I think it does a slightly better job of including fine monochrome detail > and fine color detail in the same image. > > I've also updated my styles. I've gone back to using the equalizer, as the > styles using the bilateral chroma denoise were producing some color halo > effects. > > I've also stopped using the color smoothing option on the demosaic module, > as it turned out to be smearing fine color detail at any setting other than > off (in a way that I could avoid by doing all the color smoothing in the > equalizer module) > > For a good example of this, look at the guy's pink hair in image #2. The > camera JPG retains some detail there which I could not match with color > smoothing enabled. > > I'm not sure how much more sophsticated Fuji's algorithm could be, but I > suspect it is indeed doing something to reduce these color artifacts in the > demosaicing stage, because it's extremely difficult to remove them later > blurring the high frequency color detail. > > I wonder how many of the proprietary raw processors also produce the > grid/speckle artifacts... That seems to be a tricky one. It's very > 'digital' looking. > > > > ___________________________________________________________________________ > darktable developer mailing list to unsubscribe send a mail to > darktable-dev+unsubscr...@lists.darktable.org > > > ___________________________________________________________________________ darktable developer mailing list to unsubscribe send a mail to darktable-dev+unsubscr...@lists.darktable.org
