Le dimanche 17 juin 2018, Aurélien Pierre <rese...@aurelienpierre.com> a écrit :
> > > Le 13/06/2018 à 17:31, rawfiner a écrit : > > > > Le mercredi 13 juin 2018, Aurélien Pierre <rese...@aurelienpierre.com> a > écrit : > >> >> >>> On Thu, Jun 14, 2018 at 12:23 AM, Aurélien Pierre >>> <rese...@aurelienpierre.com> wrote: >>> > Hi, >>> > >>> > The problem of a 2-passes denoising method involving 2 differents >>> > algorithms, the later applied where the former failed, could be the >>> grain >>> > structure (the shape of the noise) would be different along the >>> picture, >>> > thus very unpleasing. >> >> >> I agree that the grain structure could be different. Indeed, the grain >> could be different, but my feeling (that may be wrong) is that it would be >> still better than just no further processing, that leaves some pixels >> unprocessed (they could form grain structures far from uniform if we are >> not lucky). >> If you think it is only due to a change of algorithm, I guess we could >> apply non local means again on pixels where a first pass failed, but with >> different parameters to be quite confident that the second pass will work. >> >> That sounds better to me… but practice will have the last word. >> > > Ok :-) > >> >> >>> > >>> > I thought maybe we could instead create some sort of total variation >>> > threshold on other denoising modules : >>> > >>> > compute the total variation of each channel of each pixel as the >>> divergence >>> > divided by the L1 norm of the gradient - we then obtain a "heatmap" of >>> the >>> > gradients over the picture (contours and noise) >>> > let the user define a total variation threshold and form a mask where >>> the >>> > weights above the threshold are the total variation and the weights >>> below >>> > the threshold are zeros (sort of a highpass filter actually) >>> > apply the bilateral filter according to this mask. >>> > >>> > This way, if the user wants to stack several denoising modules, he >>> could >>> > protect the already-cleaned areas from further denoising. >>> > >>> > What do you think ? >> >> >> That sounds interesting. >> This would maybe allow to keep some small variations/details that are not >> due to noise or not disturbing, while denoising the other parts. >> Also, it may be computationally interesting (depends on the complexity of >> the total variation computation, I don't know it), as it could reduce the >> number of pixels to process. >> I guess the user could use something like that also the other way?: to >> protect high detailed zones and apply the denoising on quite smoothed zones >> only, in order to be able to use stronger denoising on zones that are >> supposed to be background blur. >> >> >> The noise is high frequency, so the TV (total variation) threshold will >> have to be high pass only. The hypothesis behind the TV thresholding is >> noisy pixels should have abnormally higher gradients than true details, so >> you isolate them this way. Selecting noise in low frequencies areas would >> require in addition something like a guided filter, which I believe is what >> is used in the dehaze module. The complexity of the TV computation depends >> on the order of accuracy you expect. >> >> A classic approximation of the gradient is using a convolution product >> with Sobel or Prewitt operators (3×3 arrays, very efficient, fairly >> accurate for edges, probably less accurate for punctual noise). I have >> developped myself optimized methods using 2, 4, and 8 neighbouring pixels >> that give higher order accuracy, given the sparsity of the data, at the >> expense of computing cost : https://github.com/aurelienpie >> rre/Image-Cases-Studies/blob/947fd8d5c2e4c3384c80c1045d86f8c >> f89ddcc7e/lib/deconvolution.pyx#L342 (ignore the variable ut in the >> code, only u is relevant for us here). >> > Great, thanks for the explanations. > Looking at the code of the 8 neighbouring pixels, I wonder if we would > make sense to compute something like that on raw data considering only > neighbouring pixels of the same color? > > > the RAW data are even more sparse, so the gradient can't be computed this > way. One would have to tweak the Taylor theorem to find an expression of > gradient for sparse data. And that would be different for Bayer and X-Trans > patterns. It's a bit of a conundrum. > Ok, thank you for these explainations > > Also, when talking about the mask formed from the heat map, do you mean > that the "heat" would give for each pixel a weight to use between input and > output? (i.e. a mask that is not only ones and zeros, but that controls how > much input and output are used for each pixel) > If so, I think it is a good idea to explore! > > yes, exactly, think of it as an opacity mask where you remap the > user-input TV threshold and the lower values to 0, the max magnitude of TV > to 1, and all the values in between accordingly. > Ok that is really cool! It seems a good idea to try to use that! rawfiner > > > rawfiner > >> >> >> >>> > >>> > Aurélien. >>> > >>> > >>> > Le 13/06/2018 à 03:16, rawfiner a écrit : >>> > >>> > Hi, >>> > >>> > I don't have the feeling that increasing K is the best way to improve >>> noise >>> > reduction anymore. >>> > I will upload the raw next week (if I don't forget to), as I am not at >>> home >>> > this week. >>> > My feeling is that doing non local means on raw data gives much bigger >>> > improvement than that. >>> > I still have to work on it yet. >>> > I am currently testing some raw downsizing ideas to allow a fast >>> execution >>> > of the algorithm. >>> > >>> > Apart of that, I also think that to improve noise reduction such as the >>> > denoise profile in nlm mode and the denoise non local means, we could >>> do a 2 >>> > passes algorithm, with non local means applied first, and then a >>> bilateral >>> > filter (or median filter or something else) applied only on pixels >>> where non >>> > local means failed to find suitable patches (i.e. pixels where the sum >>> of >>> > weights was close to 0). >>> > The user would have a slider to adjust this setting. >>> > I think that it would make easier to have a "uniform" output (i.e. an >>> output >>> > where noise has been reduced quite uniformly) >>> > I have not tested this idea yet. >>> > >>> > Cheers, >>> > rawfiner >>> > >>> > Le lundi 11 juin 2018, johannes hanika <hana...@gmail.com> a écrit : >>> >> >>> >> hi, >>> >> >>> >> i was playing with noise reduction presets again and tried the large >>> >> neighbourhood search window. on my shots i could very rarely spot a >>> >> difference at all increasing K above 7, and even less so going above >>> >> 10. the image you posted earlier did show quite a substantial >>> >> improvement however. i was wondering whether you'd be able to share >>> >> the image so i can evaluate on it? maybe i just haven't found the >>> >> right test image yet, or maybe it's camera dependent? >>> >> >>> >> (and yes, automatic and adaptive would be better but if we can ship a >>> >> simple slider that can improve matters, maybe we should) >>> >> >>> >> cheers, >>> >> jo >>> >> >>> >> >>> >> >>> >> On Mon, Jan 29, 2018 at 2:05 AM, rawfiner <rawfi...@gmail.com> wrote: >>> >> > Hi >>> >> > >>> >> > Yes, the patch size is set to 1 from the GUI, so it is not a >>> bilateral >>> >> > filter, and I guess it corresponds to a patch window size of 3x3 in >>> the >>> >> > code. >>> >> > The runtime difference is near the expected quadratic slowdown: >>> >> > 1,460 secs (8,379 CPU) for 7 and 12,794 secs (85,972 CPU) for 25, >>> which >>> >> > means about 10.26x slowdown >>> >> > >>> >> > If you want to make your mind on it, I have pushed a branch here >>> that >>> >> > integrates the K parameter in the GUI: >>> >> > https://github.com/rawfiner/darktable.git >>> >> > The branch is denoise-profile-GUI-K >>> >> > >>> >> > I think that it may be worth to see if an automated approach for the >>> >> > choice >>> >> > of K may work, in order not to integrate the parameter in the GUI. >>> >> > I may try to implement the approach of Kervann and Boulanger (the >>> >> > reference >>> >> > from the darktable blog post) to see how it performs. >>> >> > >>> >> > cheers, >>> >> > rawfiner >>> >> > >>> >> > >>> >> > 2018-01-27 13:50 GMT+01:00 johannes hanika <hana...@gmail.com>: >>> >> >> >>> >> >> heya, >>> >> >> >>> >> >> thanks for the reference! interesting interpretation how the >>> blotches >>> >> >> form. not sure i'm entirely convinced by that argument. >>> >> >> your image does look convincing though. let me get this right.. you >>> >> >> ran with radius 1 which means patch window size 3x3? not 1x1 which >>> >> >> would be a bilateral filter effectively? >>> >> >> >>> >> >> also what was the run time difference? is it near the expected >>> >> >> quadratic slowdown from 7 (i.e. 15x15) to 25 (51x51) so about >>> 11.56x >>> >> >> slower with the large window size? (test with darktable -d perf) >>> >> >> >>> >> >> since nlmeans isn't the fastest thing, even with this coalesced >>> way of >>> >> >> implementing it, we should certainly keep an eye on this. >>> >> >> >>> >> >> that being said if we can often times get much better results we >>> >> >> should totally expose this in the gui, maybe with a big warning >>> that >>> >> >> it really severely impacts speed. >>> >> >> >>> >> >> cheers, >>> >> >> jo >>> >> >> >>> >> >> On Sat, Jan 27, 2018 at 7:34 AM, rawfiner <rawfi...@gmail.com> >>> wrote: >>> >> >> > Thank you for your answer >>> >> >> > I perfectly agree with the fact that the GUI should not become >>> >> >> > overcomplicated. >>> >> >> > >>> >> >> > As far as I understand, the pixels within a small zone may suffer >>> >> >> > from >>> >> >> > correlated noise, and there is a risk of noise to noise matching. >>> >> >> > That's why this paper suggest not to take pixels that are too >>> close >>> >> >> > to >>> >> >> > the >>> >> >> > zone we are correcting, but to take them a little farther (see >>> the >>> >> >> > caption >>> >> >> > of Figure 2 for a quick explaination): >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> > https://pdfs.semanticscholar.org/c458/71830cf535ebe6c2b7656f >>> 6a205033761fc0.pdf >>> >> >> > (in case you ask, unfortunately there is a patent associated with >>> >> >> > this >>> >> >> > approach, so we cannot implement it) >>> >> >> > >>> >> >> > Increasing the neighborhood parameter results in having >>> >> >> > proportionally >>> >> >> > less >>> >> >> > problem of correlation between surrounding pixels, and decreases >>> the >>> >> >> > size of >>> >> >> > the visible spots. >>> >> >> > See for example the two attached pictures: one with size 1, >>> force 1, >>> >> >> > and >>> >> >> > K 7 >>> >> >> > and the other with size 1, force 1, and K 25. >>> >> >> > >>> >> >> > I think that the best would probably be to adapt K >>> automatically, in >>> >> >> > order >>> >> >> > not to affect the GUI, and as we may have different levels of >>> noise >>> >> >> > in >>> >> >> > different parts of an image. >>> >> >> > In this post >>> >> >> > >>> >> >> > (https://www.darktable.org/2012/12/profiling-sensor-and-phot >>> on-noise/), >>> >> >> > this >>> >> >> > paper is cited: >>> >> >> > >>> >> >> > [4] charles kervrann and jerome boulanger: optimal spatial >>> adaptation >>> >> >> > for >>> >> >> > patch-based image denoising. ieee trans. image process. vol. 15, >>> no. >>> >> >> > 10, >>> >> >> > 2006 >>> >> >> > >>> >> >> > As far as I understand, it gives a way to choose an adaptated >>> window >>> >> >> > size >>> >> >> > for each pixel, but I don't see in the code anything related to >>> that >>> >> >> > >>> >> >> > Maybe is this paper related to the TODOs in the code ? >>> >> >> > >>> >> >> > Was it planned to implement such a variable window approach ? >>> >> >> > >>> >> >> > Or if it is already implemented, could you point me where ? >>> >> >> > >>> >> >> > Thank you >>> >> >> > >>> >> >> > rawfiner >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> > 2018-01-26 9:05 GMT+01:00 johannes hanika <hana...@gmail.com>: >>> >> >> >> >>> >> >> >> hi, >>> >> >> >> >>> >> >> >> if you want, absolutely do play around with K. in my tests it >>> did >>> >> >> >> not >>> >> >> >> lead to any better denoising. to my surprise a larger K often >>> led to >>> >> >> >> worse results (for some reason often the relevance of discovered >>> >> >> >> patches decreases with distance from the current point). that's >>> why >>> >> >> >> K >>> >> >> >> is not exposed in the gui, no need for another irrelevant and >>> >> >> >> cryptic >>> >> >> >> parameter. if you find a compelling case where this indeed >>> leads to >>> >> >> >> better denoising we could rethink that. >>> >> >> >> >>> >> >> >> in general NLM is a 0-th order denoising scheme, meaning the >>> prior >>> >> >> >> is >>> >> >> >> piecewise constant (you claim the pixels you find are trying to >>> >> >> >> express /the same/ mean, so you average them). if you let that >>> >> >> >> algorithm do what it would really like to, it'll create >>> unpleasant >>> >> >> >> blotches of constant areas. so for best results we need to tone >>> it >>> >> >> >> down one way or another. >>> >> >> >> >>> >> >> >> cheers, >>> >> >> >> jo >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> On Fri, Jan 26, 2018 at 7:36 AM, rawfiner <rawfi...@gmail.com> >>> >> >> >> wrote: >>> >> >> >> > Hi >>> >> >> >> > >>> >> >> >> > I am surprised to see that we cannot control the neighborhood >>> >> >> >> > parameter >>> >> >> >> > for >>> >> >> >> > the NLM algorithm (neither for the denoise non local mean, >>> nor for >>> >> >> >> > the >>> >> >> >> > denoise profiled) from the GUI. >>> >> >> >> > I see in the code (denoiseprofile.c) this TODO that I don't >>> >> >> >> > understand: >>> >> >> >> > "// >>> >> >> >> > TODO: fixed K to use adaptive size trading variance and bias!" >>> >> >> >> > And just some lines after that: "// TODO: adaptive K tests >>> here!" >>> >> >> >> > (K is the neighborhood parameter of the NLM algorithm). >>> >> >> >> > >>> >> >> >> > In practice, I think that being able to change the >>> neighborhood >>> >> >> >> > parameter >>> >> >> >> > allows to have a better noise reduction for one image. >>> >> >> >> > For example, choosing a bigger K allows to reduce the spotted >>> >> >> >> > aspect >>> >> >> >> > that >>> >> >> >> > one can get on high ISO images. >>> >> >> >> > >>> >> >> >> > Of course, increasing K increase computational time, but I >>> think >>> >> >> >> > we >>> >> >> >> > could >>> >> >> >> > find an acceptable range that would still be useful. >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > Is there any reason for not letting the user control the >>> >> >> >> > neighborhood >>> >> >> >> > parameter in the GUI ? >>> >> >> >> > Also, do you understand the TODOs ? >>> >> >> >> > I feel that we would probably get better denoising by fixing >>> >> >> >> > these, >>> >> >> >> > but >>> >> >> >> > I >>> >> >> >> > don't understand them. >>> >> >> >> > >>> >> >> >> > I can spend some time on these TODOs, or to add the K >>> parameter to >>> >> >> >> > the >>> >> >> >> > interface if you think it is worth it (I think so but it is >>> only >>> >> >> >> > my >>> >> >> >> > personal >>> >> >> >> > opinion), but I have to understand what the TODOs mean before >>> >> >> >> > >>> >> >> >> > Thank you for your help >>> >> >> >> > >>> >> >> >> > rawfiner >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > ____________________________________________________________ >>> _______________ >>> >> >> >> > darktable developer mailing list to unsubscribe send a mail to >>> >> >> >> > darktable-dev+unsubscr...@lists.darktable.org >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> ____________________________________________________________ >>> _______________ >>> >> >> >> darktable developer mailing list >>> >> >> >> to unsubscribe send a mail to >>> >> >> >> darktable-dev+unsubscr...@lists.darktable.org >>> >> >> >> >>> >> >> > >>> >> >> >>> >> >> >>> >> >> ____________________________________________________________ >>> _______________ >>> >> >> darktable developer mailing list >>> >> >> to unsubscribe send a mail to >>> >> >> darktable-dev+unsubscr...@lists.darktable.org >>> >> >> >>> >> > >>> > >>> > >>> > ____________________________________________________________ >>> _______________ >>> > darktable developer mailing list to unsubscribe send a mail to >>> > darktable-dev+unsubscr...@lists.darktable.org >>> > >>> > >>> > >>> > ____________________________________________________________ >>> _______________ >>> > darktable developer mailing list to unsubscribe send a mail to >>> > darktable-dev+unsubscr...@lists.darktable.org >>> ____________________________________________________________ >>> _______________ >>> darktable developer mailing list >>> to unsubscribe send a mail to darktable-dev+unsubscribe@list >>> s.darktable.org >>> >>> >> ___________________________________________________________________________ >> darktable developer mailing list to unsubscribe send a mail to >> darktable-dev+unsubscr...@lists.darktable.org >> >> >> >> ___________________________________________________________________________ >> darktable developer mailing list to unsubscribe send a mail to >> darktable-dev+unsubscr...@lists.darktable.org >> > > > ___________________________________________________________________________ > darktable developer mailing list to unsubscribe send a mail to > darktable-dev+unsubscr...@lists.darktable.org > > > ___________________________________________________________________________ darktable developer mailing list to unsubscribe send a mail to darktable-dev+unsubscr...@lists.darktable.org
