This is great data, thanks. We're both seeing about a 2x slowdown for the single thread oiiotool case, so we're probably seeing the same effect. It seems to be greatly exacerbated in a highly threaded situation.
I'm digging into it on my end. If you test other version tags (such as 1.8.17), I'll be curious to hear the results. > On Sep 4, 2019, at 10:58 AM, Renaud Talon <[email protected]> wrote: > > > Is there anything characteristic about the DPX files? Just ordinary RGB? > > Anything unusual about them, especially high resolution, or other > > attributes? > No, those are just randomn basic RGB DPXs I was testing with. Same set of > images for each version of oiio tested with of course. I think all images are > HD or UHD, nothing out of the ordinary resolution wise either. > I sent you diretly a download link for 2 of the files I’m testing with. > > > > Or did you see orders of magnitude slowdown with every file you tried? > I did notice a perfomance hit (slow down) when converting a single DPX at a > time (no threading) > QThreadPool max thread count set to 1 (effectively forcing to process one DPX > a a time back to back. > On average over 2x slower than 1.8.5 or more. > The more concurrent thread I allow using QThreadPool’s setMaxThreadCount() > the slower each image > > (single threaded) > IMG NAME > 1.8.5 > 2.0.10 > imgA > 437 ms > 764 ms > imgB > 195 ms > 747 ms > imgC > 198 ms > 728 ms > imgD > 186 ms > 730 ms > imgE > 282 ms > 984 ms > imgF > 440 ms > 1035 ms > imgG > 408 ms > 998 ms > imgH > 432 ms > 1015 ms > imgI > 439 ms > 1001 ms > imgJ > 197 ms > 577 ms > imgK > 392 ms > 1017 ms > imgL > 439 ms > 1020 ms > imgM > 422 ms > 1013 ms > imgN > 421 ms > 1017 ms > imgO > 414 ms > 1059 ms > > (12 threads – oiio 2.0.10) > 1.8.5 > 2.0.10 > 2940 ms > 35969 ms > 1650 ms > 37071 ms > 3103 ms > 38580 ms > 2943 ms > 42238 ms > 2941 ms > 43270 ms > 3113 ms > 45908 ms > 2947 ms > 46273 ms > 2800 ms > 47580 ms > 3345 ms > 50409 ms > 3110 ms > 50969 ms > 3369 ms > 51239 ms > 3422 ms > 55233 ms > 1467 ms > 16947 ms > 1469 ms > 21685 ms > 1544 ms > 21025 ms > 1480 ms > 15997 ms > > The 12 threads test loads ALL the images above in a total of 700+ ms with > Michael’s patch applied to dpxinput.cpp. > > > > Are you Windows only or did you get a chance to see if a slowdown was > > apparent on Linux or OSX as well? > Yes I tested exclusively with Windows 10 since I do not have an OSX or Linux > dev environment to test with. > > > > How are you reading the files? I know that you saw a fix with implementing > > a DPXInput::read_image(), but are you indeed reading directly through an > > API call to ImageInput::read_image()? Or using an ImageBuf? > I am not using an ImgBuf to read the image, using ImageInput::read_image() > I sent you my test code so you can see all the detail. This is a little bit > rough around the edges since this was quickly written solely to troubleshoot > this issue. The OIIO reading code is located in the file called > “oiio_speed_test_thread.py” > I gave you access to a private GitHub repo which contains my test files. > > > > From C++ or from Python? > I tested from Python this time around. My first atempt using a previosuly > compiled version of oiio 2.0.x (of the first release, I forgot which one) I > had tested with both C++/Qt and Python3.6/PySide2. Both environments had the > same slow behavior. > > > > Do you see the slowdown if you are reading just one DPX file at a time, or > > do you only see it if you are reading a bunch of files simultaneously, as > > appeared in the demo video? > Please see quick benchmark results above. Yes it seems slower on a single > file but the slowdown seems to get exponentially worse the more threads > concurently read DPX files. > > > > Can you see if the slowdown is apparent in the following: > > oiiotool -threads 1 -runstats bigfile.dpx -o out.dpx > Yes, it’s seems a lot slower running oiiotool as-well. Please see output from > both versions below. > > Oiio 2.0.10’s output: > > Threads: 1 > oiiotool runtime statistics: > Total time: 1.80s > -o : 1.77 > input : 0.02 > unaccounted : 0.01 > Peak memory: 121.4 MB > Current memory: 88.3 MB > > OpenImageIO ImageCache statistics (shared) ver 2.0.10 > Options: max_memory_MB=4096.0 max_open_files=100 autotile=0 > autoscanline=0 automip=0 forcefloat=1 accept_untiled=1 > accept_unmipped=1 deduplicate=1 unassociatedalpha=0 > failure_retries=0 > Images : 1 unique > ImageInputs : 2 created, 1 current, 1 peak > Total pixel data size of all images referenced : 33.0 MB > Total actual file size of all images referenced : 33.0 MB > Pixel data read : 0 B > File I/O time : 0.0s > File open time only : 0.0s > Peak cache memory : 0 B > Image file statistics: > opens tiles MB read --redundant-- I/O time res > File > 1 2 0 0.0 0.0s 3200x1800x3.u16 > S:/episodic/csr/csr203/renders/comp/019_010/csr203_019_010_comp-main_002/csr203_019_010_comp-main_002.0079.dpx > UNTILED > > Tot: 2 0 0.0 ( 0 0.0) 0.0s > 1 not tiled, 1 not MIP-mapped > Broken or invalid files: 0 > > Oiio 1.8.5’s output: > Threads: 1 > oiiotool runtime statistics: > Total time: 0.45s > -o : 0.32 > input : 0.08 > unaccounted : 0.05 > Peak memory: 114.9 MB > Current memory: 81.8 MB > > OpenImageIO ImageCache statistics (000002E1BAF189C0) ver 1.8.5 > Options: max_memory_MB=4096.0 max_open_files=100 autotile=4096 > autoscanline=1 automip=0 forcefloat=1 accept_untiled=1 > accept_unmipped=1 read_before_insert=0 deduplicate=1 > unassociatedalpha=0 failure_retries=0 > Images : 1 unique > ImageInputs : 1 created, 1 current, 1 peak > Total pixel data size of all images referenced : 33.0 MB > Total actual file size of all images referenced : 33.0 MB > Pixel data read : 0 B > File I/O time : 0.0s > File open time only : 0.0s > Peak cache memory : 0 B > Image file statistics: > opens tiles MB read --redundant-- I/O time res > File > 1 1 0 0.0 0.0s 3200x1800x3.u16 > S:/episodic/csr/csr203/renders/comp/019_010/csr203_019_010_comp-main_002/csr203_019_010_comp-main_002.0079.dpx > UNTILED > > Tot: 1 0 0.0 ( 0 0.0) 0.0s > 1 not tiled, 1 not MIP-mapped > Broken or invalid files: 0 > > > > Do you see a big slowdown there for 1.8.5 vs 2.0 -- for a simple full > > read/write of a single large DPX file with one thread? What if you leave > > off the "-threads 1" argument? > I get the same results as above when I leave off the “threads 1” argument > > > I would tend to suspect that it's related to operating within the > > QtThreadPool, somehow interacting badly with OIIO's own thread pool. Except > > that in the video, PNG, EXR, and JPG actually speed up a bit between 1.8.5 > > and 2.0, so that feels more like a DPX specific issue. > Yes, I would assume the same issue would manifest itself with other formats > as-well. And you’re not using Qt for oiiotool.exe and it seems slower there > as-well. > > > I'm able to see a *small* slowdown in the oiiotool-based example, but not > > very dramatic (well within a factor of 2, but it's only a few seconds to > > fully read and write a 2k image) -- maybe that's the slowdown, or maybe > > not. You reported >10x degradation. > It’s about the slowdown I experience when only converting one image at a > time. The 10x degration occurs when reading multiple file at the same time > suing threading. > > > I know it's a lot to impose on you, but is it possible for you to build a > > few more versions of OIIO to see exactly where it broke? For starters, it > > would be helpful to test the last 1.8 release (1.8.17, I think) to narrow > > down if it truly happened for 2.0 and later, and not somewhere in the > > middle of 1.8.x. I'd also be curious to see if a relatively modern master > > exhibits the same problem. > I’m going to work on this right now. This might take a little longer > depending on what state my Visual Studio 2015 environment and third party > libs are in. I’ll report back to you on that as soon as I have something. > > I hope this helps, > Thanks again. > > From: Oiio-dev <[email protected]> On Behalf Of Larry > Gritz > Sent: Wednesday, September 4, 2019 12:00 AM > To: OpenImageIO developers <[email protected]> > Subject: Re: [Oiio-dev] OIIO 2.x extremely slow DPX load time > > And more: > > Can you see if the slowdown is apparent in the following: > > oiiotool -threads 1 -runstats bigfile.dpx -o out.dpx > > Do you see a big slowdown there for 1.8.5 vs 2.0 -- for a simple full > read/write of a single large DPX file with one thread? What if you leave off > the "-threads 1" argument? > > I would tend to suspect that it's related to operating within the > QtThreadPool, somehow interacting badly with OIIO's own thread pool. Except > that in the video, PNG, EXR, and JPG actually speed up a bit between 1.8.5 > and 2.0, so that feels more like a DPX specific issue. > > I'm able to see a *small* slowdown in the oiiotool-based example, but not > very dramatic (well within a factor of 2, but it's only a few seconds to > fully read and write a 2k image) -- maybe that's the slowdown, or maybe not. > You reported >10x degradation. > > I know it's a lot to impose on you, but is it possible for you to build a few > more versions of OIIO to see exactly where it broke? For starters, it would > be helpful to test the last 1.8 release (1.8.17, I think) to narrow down if > it truly happened for 2.0 and later, and not somewhere in the middle of > 1.8.x. I'd also be curious to see if a relatively modern master exhibits the > same problem. > > > > On Sep 3, 2019, at 11:23 PM, Larry Gritz <[email protected] > <mailto:[email protected]>> wrote: > > A few more questions: > > How are you reading the files? I know that you saw a fix with implementing a > DPXInput::read_image(), but are you indeed reading directly through an API > call to ImageInput::read_image()? Or using an ImageBuf? > > From C++ or from Python? > > Do you see the slowdown if you are reading just one DPX file at a time, or do > you only see it if you are reading a bunch of files simultaneously, as > appeared in the demo video? > > -- lg > > > > On Sep 3, 2019, at 11:09 PM, Larry Gritz <[email protected] > <mailto:[email protected]>> wrote: > > I still don't quite understand what would have caused the performance > degradation, so I'm going to compare the code to try to understand what > changed. > > Is there anything characteristic about the DPX files? Just ordinary RGB? > Anything unusual about them, especially high resolution, or other attributes? > Or did you see orders of magnitude slowdown with every file you tried? Are > you Windows only or did you get a chance to see if a slowdown was apparent on > Linux or OSX as well? Trying to understand what happened, as DPX is pretty > widely used in studios but nobody had previously noticed this. > > -- lg > > > > On Sep 3, 2019, at 10:37 PM, Renaud Talon <[email protected] > <mailto:[email protected]>> wrote: > > This use to work fine for me on 1.8.5 and previous. I honestly didn’t update > OIIO before the first(ish) release of OIIO 2.0 and that’s when I started > running into the performance issues with DPXs. > > At the time I was compiling with a certain set of thrid party libraries and > Visual Studio 2015. Since then I re-compiled all my dependencies for Visual > Studio 2017 and made a new attempt at compiling oiio 2.0 (in this case > 2.0.10) but I ran into the same performance issue. > > My initial test code was using the “read_scanline()” method instead of > “read_image()” and Michael’s patch only fixes the read_image() method of > course so the problem remains with “read_scanline()”. > > I did compare the dpx plugin code between 1.8.5 and 2.0.10 and noticed that > there were quite a few changes that were made but my C++ skills are not > advanced enough for me to figure out what could cause this behavior. > > I created a PR but this is more to help keep track of the issue than a > request to merge this code in as I don’t think this is a proper “long-term” > fix. > > Thanks again, > > Renaud > > From: Larry Gritz <mailto:[email protected]> > Sent: Tuesday, September 3, 2019 5:37 PM > To: OpenImageIO developers <mailto:[email protected]> > Subject: Re: [Oiio-dev] OIIO 2.x extremely slow DPX load time > > Definitely. Can you prepare a real PR with the patch that seems to work? > > I'll do some digging to try to figure out what broke. Did it used to be like > this and we broke something? > > > > > > On Sep 3, 2019, at 5:32 PM, Renaud Talon <[email protected] > <mailto:[email protected]>> wrote: > > Hi Michael, > > your patch absolutely works ! My benchmark goes from 60s+ to 700ms !!! > > I was getting corrupted images with my test code at first because I wasn't > passing the image spec.format to the "read_image()" method. (which worked > fine with OIIO 1.8.5 but is clearly needed to use the DPXInput::read_image > patch you provided) > > I changed : > pixels = srcFile.read_image() > > to: > pixels = srcFile.read_image(spec.format) > > and it worked like a charm. > > Larry, if you're reading this. Could we get Michael's patch or a similar fix > merged into OIIO code please ? > Thanks, > > Renaud > _______________________________________________ > Oiio-dev mailing list > [email protected] <mailto:[email protected]> > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> > > -- > Larry Gritz > [email protected] <mailto:[email protected]> > > > > > > > _______________________________________________ > Oiio-dev mailing list > [email protected] <mailto:[email protected]> > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> > > -- > Larry Gritz > [email protected] <mailto:[email protected]> > > > > > _______________________________________________ > Oiio-dev mailing list > [email protected] <mailto:[email protected]> > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> > > -- > Larry Gritz > [email protected] <mailto:[email protected]> > > > > > _______________________________________________ > Oiio-dev mailing list > [email protected] <mailto:[email protected]> > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> > > -- > Larry Gritz > [email protected] <mailto:[email protected]> > > > > > _______________________________________________ > Oiio-dev mailing list > [email protected] > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org -- Larry Gritz [email protected]
_______________________________________________ Oiio-dev mailing list [email protected] http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
