Re: [darktable-dev] Fwd: Feedback on Darktable for Windows

[Tobias Ellinghaus]

Am Sonntag, 28. Januar 2018, 07:10:18 CET schrieb Steve Strobel:

Hi,

I'll try to answer to as many points as I can.

> I recently installed Darktable on a Windows 10 machine and thought I should
> provide some feedback.  First of all, thanks for making it available on
> Windows!  I have tested only on the one Windows machine, so I don't know if
> some of the issues mentioned below are common to other Windows and/or Linux
> users.  Please take these things as feedback, not as complaints.

Feedback is always welcome. What version of darktable did you try?

[...]
> and there are still a few rough spots.  Please let me know if I should file
> an issue in Redmine for any of the following, or if listing the issues here
> is enough.

Redmine tickets are the way to go, most things written to the list will be 
forgotten soon.

> It took me quite a while to figure out how to import pictures from my phone.
[on Windows gphoto2 needs special USB drivers]
> given up before getting it to work.  If it is ever possible to make
> importing work without installing WinUSB drivers, it would reduce the
> learning curve a lot.

I fear there isn't much we can do about it. darktable doesn't talk to the 
camera directly but uses a library called gphoto2 for that.

> Once it found the phone, it started displaying a preview of the pictures.
> I selected a few and clicked "import", but nothing happened.  I later
> figured out that the phone had hundreds of pictures and I had to wait for
> the preview of all of them to be displayed before the import button would
> work.  The best way I found to know when it was ready was to keep dragging
> the thumb on the scrollbar to the bottom and checking to see whether it
> moved up on its own.  If it is necessary to wait, it would be nice if there
> was some kind of progress indicator, even if it is indeterminate
> (hourglass, something spinning) and if the import button remained visibly
> disabled until it was ready.

Please file a bug report for that. However, it seems hardly any one helping to 
develop darktable uses file import inside dt, so most of us know less about 
that than you.

> I am still a bit confused about what a "filmroll" is, especially when
> importing into folders with the EXIF date.  I suppose a single filmroll
> could incorporate pictures from multiple folders.  I can figure this out by
> reading the docs or experimenting, but I thought that reporting my first
> impression might be helpful.

"Filmroll" is just another word for "folder". The only difference being that 
not all images inside a folder have to be in the corresponding filmroll – you 
can just not import some of them. That's why we have that extra word to make 
it conceptually clear that we are talking about the imported files in a folder 
instead of all of them. In some places we do use "folder" when it refers to 
the place in the file system.

> It also wasn't obvious to me what a "jobcode" was, although I did
> eventually figure out that it could be incorporated into the folder names
> that the pictures are imported into.  I had a hard time remembering to set
> it before each import;  perhaps after clicking "import" it could prompt for
> the jobcode and provide a checkbox for "don't ask next time" for those that
> don't use one.  Or if the directory naming patterns don't refer to it (and
> it isn't used for anything else), it could just be skipped.

I am sure something like that could be done. Just don't expect anyone to rush 
for it as that's not the best maintained code in darktable and it seems no one 
feels responsible for it at the moment.

> When importing from the same source multiple times, it has to reload all of
> the pictures into the preview every time;  that was kind of painful when I
> was trying lots of things to get the directory naming figured out.  If that
> info could be cached, it would make it practical to import a subset of a
> large number of pictures with various jobcodes or into various filmrolls.
> Another thing that would help with that would be some kind of indication
> which pictures had already been transferred and/or a way to delete them
> from the phone (camera) after they have been successfully copied.

See above, some improvement could surely been made, but I don't see by whom. 
If you were on Linux I'd advice to use Rapid Photo Downloader to copy the 
files to your computer. Does anyone know an alternative for Windows?

[examples of bad UI in camera import]

See above.

> I found that the "base directory naming pattern" needed to end with a "/",
> or the subdirectory name would get merged into the base directory name.
> Perhaps DT should automatically append a "/" if one doesn't already exist.

Please file a bug report.

> It is unconventional in Windows to use "/" as a directory separator ("\" is
> more common).  Perhaps either one could be accepted (and perhaps internally
> converted to "/").  If there are multiple separators in a row ("//" or
> "///" or "\\"...), they 

Re: [darktable-dev] denoise profile non local means: neighborhood parameter

[rawfiner]

Hi

Yes, the patch size is set to 1 from the GUI, so it is not a bilateral
filter, and I guess it corresponds to a patch window size of 3x3 in the
code.
The runtime difference is near the expected quadratic slowdown:
1,460 secs (8,379 CPU) for 7 and 12,794 secs (85,972 CPU) for 25, which
means about 10.26x slowdown

If you want to make your mind on it, I have pushed a branch here that
integrates the K parameter in the GUI:
https://github.com/rawfiner/darktable.git
The branch is denoise-profile-GUI-K

I think that it may be worth to see if an automated approach for the choice
of K may work, in order not to integrate the parameter in the GUI.
I may try to implement the approach of Kervann and Boulanger (the reference
from the darktable blog post) to see how it performs.

cheers,
rawfiner


2018-01-27 13:50 GMT+01:00 johannes hanika :

> heya,
>
> thanks for the reference! interesting interpretation how the blotches
> form. not sure i'm entirely convinced by that argument.
> your image does look convincing though. let me get this right.. you
> ran with radius 1 which means patch window size 3x3? not 1x1 which
> would be a bilateral filter effectively?
>
> also what was the run time difference? is it near the expected
> quadratic slowdown from 7 (i.e. 15x15) to 25 (51x51) so about 11.56x
> slower with the large window size? (test with darktable -d perf)
>
> since nlmeans isn't the fastest thing, even with this coalesced way of
> implementing it, we should certainly keep an eye on this.
>
> that being said if we can often times get much better results we
> should totally expose this in the gui, maybe with a big warning that
> it really severely impacts speed.
>
> cheers,
>  jo
>
> On Sat, Jan 27, 2018 at 7:34 AM, rawfiner  wrote:
> > Thank you for your answer
> > I perfectly agree with the fact that the GUI should not become
> > overcomplicated.
> >
> > As far as I understand, the pixels within a small zone may suffer from
> > correlated noise, and there is a risk of noise to noise matching.
> > That's why this paper suggest not to take pixels that are too close to
> the
> > zone we are correcting, but to take them a little farther (see the
> caption
> > of Figure 2 for a quick explaination):
> >
> > https://pdfs.semanticscholar.org/c458/71830cf535ebe6c2b7656f6a205033
> 761fc0.pdf
> > (in case you ask, unfortunately there is a patent associated with this
> > approach, so we cannot implement it)
> >
> > Increasing the neighborhood parameter results in having proportionally
> less
> > problem of correlation between surrounding pixels, and decreases the
> size of
> > the visible spots.
> > See for example the two attached pictures: one with size 1, force 1, and
> K 7
> > and the other with size 1, force 1, and K 25.
> >
> > I think that the best would probably be to adapt K automatically, in
> order
> > not to affect the GUI, and as we may have different levels of noise in
> > different parts of an image.
> > In this post
> > (https://www.darktable.org/2012/12/profiling-sensor-and-photon-noise/),
> this
> > paper is cited:
> >
> > [4] charles kervrann and jerome boulanger: optimal spatial adaptation for
> > patch-based image denoising. ieee trans. image process. vol. 15, no. 10,
> > 2006
> >
> > As far as I understand, it gives a way to choose an adaptated window size
> > for each pixel, but I don't see in the code anything related to that
> >
> > Maybe is this paper related to the TODOs in the code ?
> >
> > Was it planned to implement such a variable window approach ?
> >
> > Or if it is already implemented, could you point me where ?
> >
> > Thank you
> >
> > rawfiner
> >
> >
> >
> >
> > 2018-01-26 9:05 GMT+01:00 johannes hanika :
> >>
> >> hi,
> >>
> >> if you want, absolutely do play around with K. in my tests it did not
> >> lead to any better denoising. to my surprise a larger K often led to
> >> worse results (for some reason often the relevance of discovered
> >> patches decreases with distance from the current point). that's why K
> >> is not exposed in the gui, no need for another irrelevant and cryptic
> >> parameter. if you find a compelling case where this indeed leads to
> >> better denoising we could rethink that.
> >>
> >> in general NLM is a 0-th order denoising scheme, meaning the prior is
> >> piecewise constant (you claim the pixels you find are trying to
> >> express /the same/ mean, so you average them). if you let that
> >> algorithm do what it would really like to, it'll create unpleasant
> >> blotches of constant areas. so for best results we need to tone it
> >> down one way or another.
> >>
> >> cheers,
> >>  jo
> >>
> >>
> >>
> >> On Fri, Jan 26, 2018 at 7:36 AM, rawfiner  wrote:
> >> > Hi
> >> >
> >> > I am surprised to see that we cannot control the neighborhood
> parameter
> >> > for
> >> > the NLM algorithm (neither for the denoise non local mean, nor for the
> >> > denoise profiled) from the GUI.
> >> >