I played around with this a bit over the weekend and I have not yet been able
to reproduce the slow SHA-1 hash times on my end, but my advice was also to use
a more recent maketx because I know that there was a point not long ago where
we realized that the hashing was single threaded and fixed it.
Thiago, do you have any theories about why --threads 1 would make it faster?
-- lg
> On Jun 4, 2018, at 8:08 AM, Thiago Ize <[email protected]> wrote:
>
> Thanks for the repro! I was able to get similarly poor hashing times using
> the same older maketx that you reported in the other email list. Changing to
> "--threads 1" fixes the slowdown (hashing takes 5.8s and the total maketx
> time is 33s). Using a newer maketx (as supplied from at least Arnold 5.1.0)
> gives a big speedup in "--threads 1" mode, and hashing is now expectedly
> faster with "--threads 7" (the entire maketx run takes less than 13s on my
> laptop).
>
> So either switch to single threaded maketx, or better yet, upgrade your
> maketx to that supplied in a newer Arnold.
>
> On Mon, Jun 4, 2018 at 8:36 AM, Will Anielewicz <[email protected]
> <mailto:[email protected]>> wrote:
> This is a cross posting with 3d-pro. Larry Gritz suggested I post this here
> as well.
> We are using many openexr 8k textures for some CG production.
> Several of them can take many minutes to convert to .tx
> It seems to be correlated to image complexity.
>
> Example openexr image on googledrive:
> https://drive.google.com/open?id=118ucgUMlUjxuDg0z_kLoDpMEA8Cu6F7c
> <https://drive.google.com/open?id=118ucgUMlUjxuDg0z_kLoDpMEA8Cu6F7c>
>
>
> Here are some runstats
>
>
> Notice this stat in particular
>
> SHA-1 hash 4m 52.17s (492.6 MB)
>
>
> maketx -v --runstats --oiio --threads 7 --compression zip --format exr -d
> half testEXR_Mari_16bit_8K.exr
>
> I have a 16bit float exr test image which is taking 5 minutes on my
> workstation
>
> prep 0.00s (10.6 MB)
> Reading file: testEXR_Mari_16bit_8K.exr
> read "testEXR_Mari_16bit_8K.exr" 0.00s (10.7 MB)
> misc2 16.32s (274.7 MB)
> misc3 0.00s (274.7 MB)
> resize & data convert 0.00s (274.7 MB)
> SHA-1: 2F36DD66063B77836B264CEBEA56B1A9B5064FA8
> SHA-1 hash 4m 52.17s (492.6 MB)
> AverageColor: 0.531764,0.510344,0.514165,1
> misc4 0.00s (492.6 MB)
> Writing file: testEXR_Mari_16bit_8K.898d81fa.temp.tx
> Filter "box"
> Top level is 8192x8192
> Mipmapping...
> 4096x4096 (1018.4 MB)
> 2048x2048 (1018.4 MB)
> 1024x1024 (778.4 MB)
> 512x512 (778.4 MB)
> 256x256 (762.4 MB)
> 128x128 (762.4 MB)
> 64x64 (762.4 MB)
> 32x32 (762.4 MB)
> 16x16 (762.4 MB)
> 8x8 (762.4 MB)
> 4x4 (762.4 MB)
> 2x2 (762.4 MB)
> 1x1 (762.4 MB)
> Wrote file: testEXR_Mari_16bit_8K.898d81fa.temp.tx (762.4 MB)
> maketx run time (seconds): 333.31
> file read: 0.00
> file write: 10.36
> initial resize: 0.00
> hash: 292.17
> mip computation: 14.45
> color convert: 0.00
> unaccounted: 16.33 ( 0.00 16.32 0.00 0.00)
> maketx peak memory used: 1018.4 MB
>
> workstation specs
>
> CPU op-mode(s): 32-bit, 64-bit
> Byte Order: Little Endian
> CPU(s): 12
> On-line CPU(s) list: 0-11
> Thread(s) per core: 2
> Core(s) per socket: 6
> Socket(s): 1
> NUMA node(s): 1
> Vendor ID: GenuineIntel
> CPU family: 6
> Model: 63
> Model name: Intel(R) Core(TM) i7-5930K CPU @ 3.50GHz
>
>
> _______________________________________________
> Oiio-dev mailing list
> [email protected] <mailto:[email protected]>
> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org>
>
>
> _______________________________________________
> Oiio-dev mailing list
> [email protected]
> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
--
Larry Gritz
[email protected]
_______________________________________________
Oiio-dev mailing list
[email protected]
http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
