Hi all, For those who want to give it a try, I made some further improvements to the below-mentioned fork with the experimental approach to X-Trans demosaicking. In particular to the issue of colour bleeding found by J Liles, this should be much less now. There was also still some hue shift, which I think should be gone now. I finally managed to obtain the filters training them from multiple reference images of the McMaster (previously IMAX) reference image set.
As a general remark, this approach doesn’t magically solve all the issues, some further processing, e.g. bilateral filtering, might still be needed for difficult image contents. However, especially for images with high frequency in luma and for high ISO images, the starting point should be a quite bit better than the other approaches. You’ll see that e.g. oftentimes less bilateral filtering is needed to make the same image usable. For those of you who want to get an impression how subtle changes in the filters change the image, I included 4 alternative filter sets that can be used in lieu of the present filtercoeff.h (filtercoeff_11_4.h, broadest, filtercoeff_var_3.h, narrowest, and filtercoeff_11_3.h, filtercoeff_var_4.h in between). Thankful for further feedback. Cheers, Ingo > Am 07.02.2017 um 02:35 schrieb J. Liles <malnour...@gmail.com>: > > > > On Sun, Feb 5, 2017 at 2:35 AM, Ingo Liebhardt <ingo.liebha...@ziggo.nl > <mailto:ingo.liebha...@ziggo.nl>> wrote: > Hi all, > > Thanks again for the feedback, especially to J Liles, which helped me a lot > to find out some issues concerning the filter design. > In the meanwhile, I managed to get rid of the hue shift and of the textile > artifacts. > > A new commit from today is in the (still very experimental) fork at: > https://github.com/ILiebhardt/darktable.git > <https://github.com/ILiebhardt/darktable.git> > For trying, just compile, deactivate openCL (only C code thus far), and > choose ‚1 pass Markesteijn‘ as demosaicking method (doesn’t work for 3-pass, > and wouldn’t really yield advantages, either). > > Compared to the previous one, the following issues are addressed: > 1. Hue shift / lack of overall saturation > 2. Textile artefact > 3. For speed, I noticed that only six filters are required, exploiting > mathematical dependencies of the conversion matrix. It’s still very slow, but > speed optimization is for later… > > You will also see a commit from earlier in January with some first > improvements in the filters. > Using these earlier filters, it is possible to avoid aliasing in the original > problematic picture completely: > https://www.dropbox.com/s/jtgmvuhtuc8uqth/wide_filters.jpg?dl=0 > <https://www.dropbox.com/s/jtgmvuhtuc8uqth/wide_filters.jpg?dl=0> > > But this comes at the price of very pronounced colour bleeding. > > The latest set of filters, which are trained from a reference image, is a > much better compromise: > https://www.dropbox.com/s/ge98px15qm1zpxm/narrower_filters.jpg?dl=0 > <https://www.dropbox.com/s/ge98px15qm1zpxm/narrower_filters.jpg?dl=0> > Although it has a bit of moiré, it is tolerable, and the colour bleeding is > much less with this latest set of filters. > The latest filter set (the trained one) really gives the best compromise > between colour bleeding / false colour suppression. > It also manages really well my hardest test image, where high frequency > changes in luminance were causing severe problems in chroma: > https://www.dropbox.com/s/pgo2scu7o5243ga/L_best_shot.jpg?dl=0 > <https://www.dropbox.com/s/pgo2scu7o5243ga/L_best_shot.jpg?dl=0> > > To compare this to three pass Markesteijn with five times colour smoothing: > https://www.dropbox.com/s/wkv2xlyh9sgw69x/MS3pass.jpg?dl=0 > <https://www.dropbox.com/s/wkv2xlyh9sgw69x/MS3pass.jpg?dl=0> > > Finally, w.r.t. the high iso test image from J Liles, it looks much better > now, too. To note that the serial number of the dollar note are not deeply > green, but this is a price to pay with this CFA and this approach. > > Have fun trying. Looking forward to your feedback. If you’d like to try the > older (broader) filter set, you can go back to the my commit of early January > in the above mentioned git. > > Cheers, > Ingo > > > >> Am 28.12.2016 um 00:35 schrieb J. Liles <malnour...@gmail.com >> <mailto:malnour...@gmail.com>>: >> >> >> >> On Wed, Dec 21, 2016 at 11:27 AM, J. Liles <malnour...@gmail.com >> <mailto:malnour...@gmail.com>> wrote: >> >> >> On Wed, Dec 21, 2016 at 11:20 AM, J. Liles <malnour...@gmail.com >> <mailto:malnour...@gmail.com>> wrote: >> >> >> On Wed, Dec 21, 2016 at 11:17 AM, Ingo Liebhardt <ingo.liebha...@ziggo.nl >> <mailto:ingo.liebha...@ziggo.nl>> wrote: >> Ah, and by the way @J Liles: could you please explain me a bit more what you >> mean by ‚textile like artifact‘, I’d like to investigate that one a bit more >> in-depth. >> Thx >> >> >>> Am 21.12.2016 um 20:10 schrieb Ingo Liebhardt <ingo.liebha...@ziggo.nl >>> <mailto:ingo.liebha...@ziggo.nl>>: >>> >>> Hi all, >>> >>> Thanks a lot for the feedback, and no worries if it takes you a while >>> testing it. >>> As you see, I’m also progressing rather slowly on my side… >>> >>> It’s still a proof-of-concept and I have quite some items on my to do list, >>> most notably: >>> - the literature mentions training the filters based on reference images, >>> and I’m slowly working on this, hoping that it would further increase image >>> quality. So far, the filters are designed using the window design method. >>> - trying to find out where the hue shifts come from - I already noticed >>> them, too. >>> >>> Other things like performance improvements will be for later… >>> >>> I’ll let you know as soon as I make progress on the filters. >>> >>> Cheers, >>> Ingo >>> >>> >>> >>>> Am 21.12.2016 um 01:14 schrieb J. Liles <malnour...@gmail.com >>>> <mailto:malnour...@gmail.com>>: >>>> >>>> >>>> >>>> On Mon, Dec 19, 2016 at 7:40 PM, J. Liles <malnour...@gmail.com >>>> <mailto:malnour...@gmail.com>> wrote: >>>> >>>> >>>> On Mon, Dec 12, 2016 at 10:40 AM, Ingo Liebhardt <ingo.liebha...@ziggo.nl >>>> <mailto:ingo.liebha...@ziggo.nl>> wrote: >>>> Hi all, >>>> >>>> Maybe you still remember that I tried an alternative approach to X-Trans >>>> demosaicking (using guided filtering) in March / April this year… >>>> In the end, I was not satisfied, and I gave up on that approach. The >>>> problems were comparable to the Markesteijn algorithm, and the >>>> improvements marginal. >>>> >>>> After giving up on that approach, I was again browsing conference papers >>>> trying to get some inspiration. >>>> I came across the work of E. Dubois, which looked promising. >>>> It is promising, not so much when applied alone, but very much so when >>>> combined with a gradient based approach like Markesteijn. >>>> >>>> I like Jo’s xtrans fringes profile a lot, but the colors get somewhat >>>> muted, overall. >>>> >>>> Contrary to my first approach, this one finally seems to give reasonable >>>> results. >>>> I managed to get good output for the redline bug #10333. >>>> You can have a look here: dropbox link >>>> <https://www.dropbox.com/sh/un1y11uimbqxjjk/AAD3L-Rs9-ztwyBIm4rnCzK-a?dl=0> >>>> >>>> This is the output just with demosaic + base curve, nothing else. >>>> >>>> If you want to try some nasty X-Trans images yourself, I made a little >>>> proof-of-concept. >>>> This in form of a fork of darktable, which you can find here: >>>> https://github.com/ILiebhardt/darktable.git >>>> <https://github.com/ILiebhardt/darktable.git> >>>> For trying, just compile, deactivate openCL (only C code thus far), and >>>> choose ‚1 pass Markesteijn‘ as demosaicking method (doesn’t work for >>>> 3-pass, and wouldn’t really yield advantages, either). >>>> >>>> Have fun trying, and let me know if you think that this one’s worth >>>> pursuing further (only quick hack so far, and the used correlation filters >>>> are a slow, naive implementation O(m n p q)). >>>> >>>> If you’d like to read some basics concerning the idea, I made a mini-blog >>>> here: http://xtransdemosaicking.blogspot.nl >>>> <http://xtransdemosaicking.blogspot.nl/> >>>> >>>> Cheers, >>>> Ingo >>>> >>>> >>>> P.S.: concerning my previous approach, J Liles spotted single pixel >>>> artifacts. I found out that these are not related tot the demosaicking as >>>> such. X-Trans 2 and X-Trans 3 have hybrid AF, and the pixels used for >>>> phase detection show higher noise. These are all green pixels of a 4-group >>>> of pixels; never a red or blue, and never a solitary green. But solving >>>> this would be a whole different project... >>>> >>>> >>>> >>>> >>>> >>>> >>>> ___________________________________________________________________________ >>>> darktable developer mailing list to unsubscribe send a mail to >>>> darktable-dev+unsubscr...@lists.darktable.org >>>> <mailto:darktable-dev%2bunsubscr...@lists.darktable.org> >>>> >>>> Ingo, >>>> >>>> Great to hear you're still working on this! >>>> >>>> I haven't reviewed the code of the algorithm, but I did give it a try on a >>>> few images. >>>> >>>> Here's one in particular (lots of sharpening added to make the differences >>>> more obvious.) >>>> >>>> http://www.nevermindhim.com/liebhardt-test >>>> <http://www.nevermindhim.com/liebhardt-test> >>>> >>>> Direct image links: >>>> >>>> http://www.nevermindhim.com/files/liebhardt-test/6acffe60-09c5-11e6-93d7-178612e3e7eb_E1_VNG.png >>>> >>>> <http://www.nevermindhim.com/files/liebhardt-test/6acffe60-09c5-11e6-93d7-178612e3e7eb_E1_VNG.png> >>>> http://www.nevermindhim.com/files/liebhardt-test/6acffe60-09c5-11e6-93d7-178612e3e7eb_E1_Markesteijn.png >>>> >>>> <http://www.nevermindhim.com/files/liebhardt-test/6acffe60-09c5-11e6-93d7-178612e3e7eb_E1_Markesteijn.png> >>>> http://www.nevermindhim.com/files/liebhardt-test/6acffe60-09c5-11e6-93d7-178612e3e7eb_E1_Liebhardt.png >>>> >>>> <http://www.nevermindhim.com/files/liebhardt-test/6acffe60-09c5-11e6-93d7-178612e3e7eb_E1_Liebhardt.png> >>>> >>>> >>>> My first impressions are: >>>> >>>> 1) (obviously you know this) It's slow >>>> 2) It introduces a hue shift >>>> 3) It does a better job of controlling color noise than VNG or Markesteijn. >>>> 4) Artifacts are similar in structure to Markesteijn (maze-like) >>>> 5) There is an additional textile like artifact that Markesteijn doesn't >>>> exhibit. >>>> 6) It overshoots in interpolating across gradients, but not as much as VNG >>>> does. >>>> >>>> If you can get rid of the textile effect and, color cast, and speed it up, >>>> this looks like it would be an improvement over Markesteijn (with no color >>>> smoothing/noise reduction). It's already looking more "film like" >>>> >>>> >>>> >>>> Replying to myself here... >>>> >>>> Added another set of images to: >>>> >>>> http://www.nevermindhim.com/liebhardt-test >>>> <http://www.nevermindhim.com/liebhardt-test> >>>> >>>> (TEST IMAGE 2) >>>> >>>> This time correcting for the hue shift (with auto white balance). >>>> >>>> I wanted to illustrate how it deals with a high ISO (12800) image, with >>>> and without noise reduction and sharpening. >>>> >>>> As you can see, the result is a definite improvement, especially the noise >>>> reduced version. There may be a slight loss of sharpness, but for me it's >>>> worth it to get rid of those crusty false colors. >>>> >>>> However, whether or not even this is better than the SOOC JPEG (NR -4, >>>> Sharpness 0) is debatable. It seems like using the maximum NR in darktable >>>> is required to produce a similar result as the minimum NR in camera... >>>> >>> >>> >>> ___________________________________________________________________________ >>> darktable developer mailing list to unsubscribe send a mail to >>> darktable-dev+unsubscr...@lists.darktable.org >>> <mailto:darktable-dev+unsubscr...@lists.darktable.org> >> >> >> In TEST IMAGE 1, look at the blue TV screen behind the subject's head. You >> can see a textile/grid type effect that wasn't really there. This effect >> doesn't appear with VNG or Markesteijn. It looks like the your weightings >> might be off causing the X-Trans pattern to show through when interpolating >> solid colors. >> >> >> Just to add to this, other points of interest in this image for finding >> artifacts are the saturated purple lights in the upper right, and the edges >> of the TV screen and the subject's hair. Of particular interest is the >> serial number on the dollar bill. It should be dark green (the same color as >> the stamp/seal above). Too-aggressive chroma denoising may make it turn >> light gray/green like the rest of the bill. >> >> >> >> Continuing my habit of replying to myself, I was curious how your algorithm >> would handle the dreaded X-Trans II/III "purple flare/grid artifact" problem >> (which, AFAICT, remains an unsolved problem everywhere). The result is >> interesting. Your algorithm completely removes the purple color cast of the >> flare, resulting in, IMHO, a much more pleasing appearance. However, the >> grid aspect remains: >> >> http://www.nevermindhim.com/files/liebhardt-test/a75767d6-cc7c-11e6-95bd-739c86278d6a_E1_Liebhardt.png >> >> <http://www.nevermindhim.com/files/liebhardt-test/a75767d6-cc7c-11e6-95bd-739c86278d6a_E1_Liebhardt.png> >> > > > Ingo, > > The colors are looking better now and indeed the textile artifact appears to > be remedied (and that high ISO example is looking pretty good!). However I'm > seeing what I would consider to be excessive color bleeding/halos. > > Observe this image: > > http://www.nevermindhim.com/files/liebhardt-test/bf2e0f63ffa932e34701b47ed0a5a238_liebhardt2.png > > <http://www.nevermindhim.com/files/liebhardt-test/bf2e0f63ffa932e34701b47ed0a5a238_liebhardt2.png> > > It almost looks like chromatic aberration (but isn't). > > As a control, here's the same image processed with VNG + a bilateral filter > on the chroma channel (not enough to get rid of the moire on the tie > though...) > > http://www.nevermindhim.com/files/liebhardt-test/bf2e0f63ffa932e34701b47ed0a5a238_vngbilateral.png > > <http://www.nevermindhim.com/files/liebhardt-test/bf2e0f63ffa932e34701b47ed0a5a238_vngbilateral.png> > > The problem with the tree branch torture test is that the subject is too > monochrome to show these kinds of issues. > > > > ___________________________________________________________________________ > darktable developer mailing list to unsubscribe send a mail to > darktable-dev+unsubscr...@lists.darktable.org > <mailto:darktable-dev+unsubscr...@lists.darktable.org> ___________________________________________________________________________ darktable developer mailing list to unsubscribe send a mail to darktable-dev+unsubscr...@lists.darktable.org
