Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-27 Thread Sheet Spotter
I found a simple fix to the OpenJPEG v2 library that corrects the previous
crashes that occurred with the test harness.

The fix is attached to https://jira.secondlife.com/browse/SNOW-361

I also updated Robin's spreadsheet to reflect the improved performance.
OpenJPEG v2 is now marginally quicker than the OpenJPEG v1.3.0 that ships
with Viewer 2.

Previously the OpenJPEG v2 library was crashing when it skipped over packets
for higher discard levels. The library was skipping over too few bytes.

Earlier test results for OpenJPEG v2 were performing a full decode. The
Reduce feature (that performs partial decodes when the image was
incomplete) had to be disabled because of the crashes. Now that the crashes
are fixed, the updated test results for v2 reflect the same partial decodes
that OpenJPEG v1.3.0 was performing.

The fix will likely be submitted to the OpenJPEG project tomorrow night,
when I have more time.


Sheet Spotter

-Original Message-
From: Robin Cornelius [mailto:robin.cornel...@gmail.com] 
Sent: September 23, 2010 5:37 AM
To: Sheet Spotter
Cc: opensource-dev
Subject: Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

On Thu, Sep 23, 2010 at 3:40 AM, Sheet Spotter sheet.spot...@gmail.com
wrote:
 There may be another option for obtaining a common set of J2C files for
 comparison.

 Viewer 2 installs 784 JPEG-2000 files (*.j2c) in the local_assets folder
 under the main install folder (e.g., C:\Program
Files\SecondLifeViewer2).

 The j2k_metric test harness decoded all 784 files from the
local_assets
 folder without errors. The OpenJPEG 1.3.0 library was 3.7 times slower
than
 the KDU library that comes with Viewer 2.1.1.208043 (the current release
 version).

Hi (again) Sheet and everyone else,

I've started another sheet with a fresh set of results on another
system of mine, please feel free to add your own tests rows

https://spreadsheets.google.com/ccc?key=0AiSrUP47_VxIdEZ4NmlSY281UXFac0ZZTkV
jWGJtV1Ehl=enauthkey=CIGY9M4O

If you want to compare to KDU you need to compare to KDU on your
system don't compare to by KDU results or anyone elses, thats not a
fair comparision.

OJP 2.0 is not playing nice with the local_asset texture set needs some
work.

Robin

___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges


Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-26 Thread Sheet Spotter
I created a crude profiling tool to determine where to focus on improving
the performance of OpenJPEG v2.

The profiling tool records the number of function calls and combined
execution time for every function in the main source files (openjpeg.c,
j2k.c, dwt.c, tcd.c, t1.c, mqc.c, and raw.c).

Results from running the test harness on the 748 files from the
local_assets folder were recorded in a CSV (Comma-Separated Value) format.
The results were placed here:
http://pastebin.com/7K9PCFcB

The three columns in the file are:
1. Function name
2. Number of times the function was called
3. Combined execution time for the function. 
The order the rows appear in the file is not significant. (You can sort the
rows in any order you like after importing the file into a spreadsheet.)

Enabling the profiling tool definitely distorts the results. Overall
execution time for the test increased from 21 seconds to 129 seconds. 

The crude profiling tool was able to identify where most of the decoding
time is spent. Only a small number of routines are responsible for most of
the execution time. Hopefully this information can lead to improvements in
the performance of the OpenJPEG v2 library.


Sheet Spotter

-Original Message-
From: Robin Cornelius [mailto:robin.cornel...@gmail.com] 
Sent: September 23, 2010 5:37 AM
To: Sheet Spotter
Cc: opensource-dev
Subject: Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

On Thu, Sep 23, 2010 at 3:40 AM, Sheet Spotter sheet.spot...@gmail.com
wrote:
 There may be another option for obtaining a common set of J2C files for
 comparison.

 Viewer 2 installs 784 JPEG-2000 files (*.j2c) in the local_assets folder
 under the main install folder (e.g., C:\Program
Files\SecondLifeViewer2).

 The j2k_metric test harness decoded all 784 files from the
local_assets
 folder without errors. The OpenJPEG 1.3.0 library was 3.7 times slower
than
 the KDU library that comes with Viewer 2.1.1.208043 (the current release
 version).

Hi (again) Sheet and everyone else,

I've started another sheet with a fresh set of results on another
system of mine, please feel free to add your own tests rows

https://spreadsheets.google.com/ccc?key=0AiSrUP47_VxIdEZ4NmlSY281UXFac0ZZTkV
jWGJtV1Ehl=enauthkey=CIGY9M4O

If you want to compare to KDU you need to compare to KDU on your
system don't compare to by KDU results or anyone elses, thats not a
fair comparision.

OJP 2.0 is not playing nice with the local_asset texture set needs some
work.

Robin

___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges


Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-26 Thread Ponzu
Does the execution time *include* the time of sub-functions called, or does
it *exclude*?

On Sun, Sep 26, 2010 at 3:02 AM, Sheet Spotter sheet.spot...@gmail.comwrote:

 I created a crude profiling tool to determine where to focus on improving
 the performance of OpenJPEG v2.


___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-26 Thread Sheet Spotter
Execution time from the profile includes sub-functions.

 

For example, the tcd_t1_decode function calls the t1_decode_cblks function
multiple times. The execution time for tcd_t1_decode includes the time for
every call to t1_decode_cblks.

 

The t1_decode_cblk function was responsible for most of the execution time
(121 out of 128 seconds). Almost all of this time in this function is split
between three other functions t1_dec_sigpass (51 seconds), t1_dec_refpass
(36 seconds), and t1_dec_cInpass (35 seconds). Each of these functions make
millions of calls to only a few other functions.

 

The profiler was written very simply. 

 

The profiler consists of a single CProfiler class in C++. The constructor
records the start time using a high resolution timer. The destructor
calculates the elapsed time and updates a collection of counters. 

 

Declaring a variable of the CProfiler class is all that's required to
capture the execution time and call count. The constructor accepts a single
argument, which is used as a unique identifier. Results are grouped by this
identifier. The name of the function was used at the unique identifier. 

 

After running the simulation a static method is called to dump the
statistics. 

 

The most expensive part of the coding was adding CProfiler variables to the
start of each function. No attempt was made to subtract out the time spent
in sub-routines. No attempt was made to subtract out the impact of the
CProfiler class itself.

 

The CProfiler class will not be submitted to the OpenJPEG source code.
Comments in their Google group suggest they prefer to limit the source code
to C and are not interested in using C++ objects.

 

 

Sheet Spotter

 

  _  

From: lee.sai...@gmail.com [mailto:lee.sai...@gmail.com] On Behalf Of Ponzu
Sent: September 26, 2010 10:03 AM
To: Sheet Spotter
Cc: Robin Cornelius; opensource-dev
Subject: Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

 

Does the execution time *include* the time of sub-functions called, or does
it *exclude*?

On Sun, Sep 26, 2010 at 3:02 AM, Sheet Spotter sheet.spot...@gmail.com
wrote:

I created a crude profiling tool to determine where to focus on improving
the performance of OpenJPEG v2.

___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-23 Thread Robin Cornelius
On Thu, Sep 23, 2010 at 3:40 AM, Sheet Spotter sheet.spot...@gmail.com wrote:
 There may be another option for obtaining a common set of J2C files for
 comparison.

 Viewer 2 installs 784 JPEG-2000 files (*.j2c) in the local_assets folder
 under the main install folder (e.g., C:\Program Files\SecondLifeViewer2).


Doh!, good call. we could set the default path for the test to use
those +1 on that idea.

 The j2k_metric test harness decoded all 784 files from the local_assets
 folder without errors. The OpenJPEG 1.3.0 library was 3.7 times slower than
 the KDU library that comes with Viewer 2.1.1.208043 (the current release
 version).

That's significantly different to my latest results, but i need to
validate on that texture set. I'm expecting we will see massive
differences between relative performance from CPU to CPU. I'll put up
a google doc that any one can add to and we can all (if you want) add
our individual results against this set.


 I have been unsuccessful at implementing the additional functions to make
 the OpenJPEG v2 interface compatible with the earlier v1.3.0 interface. The
 test application aborted with buffer overrun errors. Oopsie!

Did you see https://jira.secondlife.com/browse/SNOW-361 ? i used that
code to interface to OJP 2, you do need to build openjpeg with a
compile flag to expose  two functions that have been deprecated ,
USE_OPJ_DEPRECATED is what is needed in openjpeg.

 There was one small surprise. Only two threads were in use for both the
 OpenJPEG v1.3.0 and KDU tests. I expected KDU to use all eight available
 threads.

I'm not overly concerned on threading for this reason, JPEG2000s often
are GB/TB images, Nasa etc use them for planet surveys and all kinds
of things, multi core support there is really valuable and will
clearly speed up the total process. In SL we are decoding lots of
small images so my argument is that KDU using all 4/8/what ever cores
is little real world advantage over 4/8/whatever threads decoding
separate images in parallel and for the end user the effects will be
very similar (assuming a similar single thread decode performance) and
infact depending on how KDU is written, it might even have a
performance penalty if it spawns threads for each decode as starting
threads has a cost, that cost is worth paying for single very large
images, but could be too much for multiple images as the thread start
costs adds up. I suspect the KDU api has an option to use/not use
multi core so may be LL have some control over this and can optimised?

Robin
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges


Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-23 Thread Robin Cornelius
On Thu, Sep 23, 2010 at 3:40 AM, Sheet Spotter sheet.spot...@gmail.com wrote:
 There may be another option for obtaining a common set of J2C files for
 comparison.

 Viewer 2 installs 784 JPEG-2000 files (*.j2c) in the local_assets folder
 under the main install folder (e.g., C:\Program Files\SecondLifeViewer2).

 The j2k_metric test harness decoded all 784 files from the local_assets
 folder without errors. The OpenJPEG 1.3.0 library was 3.7 times slower than
 the KDU library that comes with Viewer 2.1.1.208043 (the current release
 version).

Hi (again) Sheet and everyone else,

I've started another sheet with a fresh set of results on another
system of mine, please feel free to add your own tests rows

https://spreadsheets.google.com/ccc?key=0AiSrUP47_VxIdEZ4NmlSY281UXFac0ZZTkVjWGJtV1Ehl=enauthkey=CIGY9M4O

If you want to compare to KDU you need to compare to KDU on your
system don't compare to by KDU results or anyone elses, thats not a
fair comparision.

OJP 2.0 is not playing nice with the local_asset texture set needs some work.

Robin
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges


Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-22 Thread Ann Otoole
How many textures do you need? I bet I have a bunch that are perfectly OK for 
this purpose as well as for commercial use. And if you need more I can generate 
them using filters.

I.e.; this is not a block.





From: Robin Cornelius robin.cornel...@gmail.com
To: Sheet Spotter sheet.spot...@gmail.com
Cc: opensource-dev opensource-dev@lists.secondlife.com
Sent: Wed, September 22, 2010 3:55:20 AM
Subject: Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

On Wed, Sep 22, 2010 at 4:01 AM, Sheet Spotter sheet.spot...@gmail.com wrote:
 I would love to see Robin's test harness. I would also love to see the
 images that failed with v2.

For the record I cannot give you (or anyone else) the images that
failed, I was testing against a random selection of images pulled from
the texture cache (and yes i know how to get a complete image out the
cache) i wanted it to be a fair cross section of real SL textures not
contrived test cases. I can probably people on an individual basis
UUIDs of failed images but i'm afraid you will have to obtain the
images yourselves I can't just go around distributing SL textures that
are not mine.

Maybe we can make a collection of test textures that we (the
collective we) own, if an individual copyright holder has textures
they can donate we can build a collection of test textures that can be
used for metrics now and in the future.


 I started diving into the v2 branch of OpenJPEG a few days ago, after
 research JPEG 2000 for a week. Tackling the bugs from the failed images
 might be a good way to become more familiar with the source code.

I can get full back traces to the offending lines which will be some help.


 I posted a very minor patch correcting some simple build problems with the
 DLL version of OpenJPEG V2.
http://code.google.com/p/openjpeg/issues/detail?id=40

Yes i had to fix a few issues to get the dll to build, nothing major.

 Thank you for publishing your results, Robin. They are encouraging me to
 continue looking at OpenJPEG v2.

Thanks OJP V2 looks like its worth the effort, so far using it with
the viewer is failing badly. Encode now i've stopped it crashing is
failing causing  a bake loop of hell which consumes massive amounts
of CPU time and spams the LL servers with bakes, so bad all around.

As for the test harness, Yes let me tidy it slightly and I'll publish
it, its really simple but i'm lazily working around one issue with
error reporting currently. I also hopefully will have some new data
tonight myself, looks like my results are artificially slow, BUT there
remains a massive difference between KDU and openjpg, possibly worse
than the x4 data i obtained, maybe in the order of x16-x20 slower but
i need to rerun all the tests. This could also explain why SSE was
showing odd results.

Robin
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges



  ___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-22 Thread Robin Cornelius
On Wed, Sep 22, 2010 at 9:29 AM, Ann Otoole missannoto...@yahoo.com wrote:
 How many textures do you need? I bet I have a bunch that are perfectly OK
 for this purpose as well as for commercial use. And if you need more I can
 generate them using filters.

 I.e.; this is not a block.


Thanks,

I was running just under 300 for my tests, but we could do with a mix
of RGB and RGBA textures + some that have lossless encoding and also
some encoded on KDU and some on Openjpeg as there are differences. So
any you want to donate to the cause would be most welcome, i can
easily add a text file with a list of textures and copyright by who
etc so its clear who actually owns them. Also i can encode if
necessary but esp with the KDU ones i need to be 100% i've made them
the same as the viewer does.

We ideally want to get more that 300 and get a good test bed of say
1000 in total and then that will be very useful with a common set of
textures for everyone to compare results with as many things in common
as possible, this will then show differences on optimisations between
different CPUs, operating systems etc and make it very easy for
changes to openjpeg or any other decoder to be compared on mass.

Regards

Robin
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges


Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-22 Thread Tateru Nino


On 22/09/2010 5:55 PM, Robin Cornelius wrote:
 On Wed, Sep 22, 2010 at 4:01 AM, Sheet Spottersheet.spot...@gmail.com  
 wrote:
 I would love to see Robin's test harness. I would also love to see the
 images that failed with v2.
 For the record I cannot give you (or anyone else) the images that
 failed, I was testing against a random selection of images pulled from
 the texture cache (and yes i know how to get a complete image out the
 cache)
It's not hard to do. I've got this little HTTP texture-proxy thing that 
queries the local computers here to share local viewer caches and save 
on bandwidth. And yes, the contents aren't the sort of thing you want to 
pass around willy-nilly.

-- 
Tateru Nino
http://dwellonit.taterunino.net/

___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges


Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-22 Thread Lillian Yiyuan
Doing this out of a local cache would make it clearer what the time
for the library is. I suspect these numbers understate the kdu
advantage.


On Wed, Sep 22, 2010 at 7:30 AM, Tateru Nino tateru.n...@gmail.com wrote:


 On 22/09/2010 5:55 PM, Robin Cornelius wrote:
 On Wed, Sep 22, 2010 at 4:01 AM, Sheet Spottersheet.spot...@gmail.com  
 wrote:
 I would love to see Robin's test harness. I would also love to see the
 images that failed with v2.
 For the record I cannot give you (or anyone else) the images that
 failed, I was testing against a random selection of images pulled from
 the texture cache (and yes i know how to get a complete image out the
 cache)
 It's not hard to do. I've got this little HTTP texture-proxy thing that
 queries the local computers here to share local viewer caches and save
 on bandwidth. And yes, the contents aren't the sort of thing you want to
 pass around willy-nilly.

 --
 Tateru Nino
 http://dwellonit.taterunino.net/

 ___
 Policies and (un)subscribe information available here:
 http://wiki.secondlife.com/wiki/OpenSource-Dev
 Please read the policies before posting to keep unmoderated posting privileges

___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges


Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-22 Thread Trilo Byte
In theory, KDU 6.4 is supposed to be platform neutral - or at least that's one 
of the claims the mfr. is making about the new version.

On Sep 22, 2010, at 11:35 AM, Arrehn Oberlander wrote:

 Is it possible for you to release your test harness for these comparisons, 
 including the texture set?
 
 I noticed there isn't data for linux or mac platforms, which I am curious 
 about. My eyeball tells me KDU is more optimized for windows than other 
 platforms, but I'd like to test this empirically, as well as see how well my 
 hardware compares to these results using the same test.
 ___
 Policies and (un)subscribe information available here:
 http://wiki.secondlife.com/wiki/OpenSource-Dev
 Please read the policies before posting to keep unmoderated posting privileges

___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges


Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-22 Thread Gigs
On 09/22/2010 02:12 PM, Lillian Yiyuan wrote:
 Doing this out of a local cache would make it clearer what the time
 for the library is. I suspect these numbers understate the kdu
 advantage.


I'm pretty sure Robin's test harness is very minimal, if you are 
thinking that there's a bloated client attached to it, muddying the numbers.
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges


Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-22 Thread Sheet Spotter
There may be another option for obtaining a common set of J2C files for
comparison.

Viewer 2 installs 784 JPEG-2000 files (*.j2c) in the local_assets folder
under the main install folder (e.g., C:\Program Files\SecondLifeViewer2). 

The j2k_metric test harness decoded all 784 files from the local_assets
folder without errors. The OpenJPEG 1.3.0 library was 3.7 times slower than
the KDU library that comes with Viewer 2.1.1.208043 (the current release
version).

I have been unsuccessful at implementing the additional functions to make
the OpenJPEG v2 interface compatible with the earlier v1.3.0 interface. The
test application aborted with buffer overrun errors. Oopsie!

The tests were run under Windoze 7 on a Pentium i7-950 (3 GHz Hyperthreaded)
using a RelWithDebInfo build run from the command line. (Running the same
build from within VisualStudio 2005 was noticeably slower. 

There was one small surprise. Only two threads were in use for both the
OpenJPEG v1.3.0 and KDU tests. I expected KDU to use all eight available
threads.


Sheet Spotter

-Original Message-
From: opensource-dev-boun...@lists.secondlife.com
[mailto:opensource-dev-boun...@lists.secondlife.com] On Behalf Of Robin
Cornelius
Sent: September 22, 2010 2:00 PM
To: Arrehn Oberlander
Cc: opensource-dev
Subject: Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

On Wed, Sep 22, 2010 at 7:35 PM, Arrehn Oberlander arr...@gmail.com wrote:
 Is it possible for you to release your test harness for these comparisons,
 including the texture set?

I can't give out the texture set, hopefully as per a previous
discussion on this topic, we can assemble a good texture set that we
can all share and compare metrics.

the code can be found at
http://bitbucket.org/robincornelius/viewer-development-j2c-metrics

The code although based on 2.X should still work on 1.X (my original
tests were a 1.X build) but i wanted to clean this up a bit before
release and to be honest we are concerned with openjpeg/kdu not the
viewer for these tests, my test code itsself is painfully simple. Load
a texture into an LLImage, run decode time it. repeat and rince.

The code forms a seperate project j2k_metric which depends on
llcommon/llimage/llimagej2coj and llvfs (cmake rules included to make
it build out of the box. But to make it run you need to set the
working directory to be the sharedlibs folder (or copy dlls/so's
around) as i've not bothered to stage it yet (patches welcome)

The exe accepts one command line argument and that is the path to a
folder of textures, if no path is given it assumes a default path of
C:\Textures to make things a tiny bit easier on windows.

Hope this is enough to get people started

Robin
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting
privileges

___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges


Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-22 Thread Ambrosia
 There was one small surprise. Only two threads were in use for both the
 OpenJPEG v1.3.0 and KDU tests. I expected KDU to use all eight available
 threads.


Remember that LL is still shipping a downright ancient KDU 4.* based
llkdu.dll with the viewer.
If they successfully update to 6.4.* or later, you'll probably see all
cores used.
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges


[opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-21 Thread Robin Cornelius
Hi everyone

I have been doing some direct comparison between KDU and Openjpeg and
have some metrics to share with the rest of you (in case you have not
seen them already flying around on #opensl/ or AWG chat)

https://spreadsheets.google.com/ccc?key=0AiSrUP47_VxIdFY4Mi1VUkdnaXJsSVpldGRPZXoxc3chl=en#gid=0

The tests were done in a test harness using the LLImage class from the
viewer. 296 Typical SL textures which were complete were used, Each
texture was decoded to discard level 0 (fully) and the timing was only
done for the call to decode(). This was repeated 10 times to average
out some CPU load jitter and the results entered on to the
spreadsheet.  I did some variations on openjpeg 1.3 with builds on
VC2005 and 2008 and some different SSE and optimisation settings, I
also did a test with the Openjpeg V2 code from SVN, although I had to
remove 5 textures from the test to prevent this from crashing.

The SSE results are interesting, my CPU is an AMD Athlon II X4 64 bit
quad core but with 2005 and 2008 SSE2 slows down the decode
process,and 2008 also does a better optimisation job than 2005. I'm
still not 100% sure why the build of openjpeg supplied by imprudence
is better than my builds on 2005. I've also not done any tests on
linux, but the basic pattern of KDU being way faster than openjpeg 1.X
and 2.X being faster than 1.X i do not expect to change much, the only
thing that might change is ups and downs for various optimisations and
extra CPU instructions such as SSE behaving differently under gcc.

Openjpeg 2 is really looking like its made a massive improvement in
its speed, but currently its very unstable, as I said it crashes with
5 of my test textures (which appear to be perfectly normal RGBA
textures of common dimensions). Its also crashing with encode for
bakes and as far as i know it has issues/is not complete to handle
truncated streams either, all of which make usage in SL right this
moment not possible.

I think openjpeg 2 is where opensource effort should be directed as
its not _that_ far behind KDU for raw decode speed (compared to OJP
1.X) currently and yes I know the new KDU can use multiple cores, but
so what, we can run multiple decode threads, its nots if we are
decoding a GB image so that extra boost is probably not that
significant for the type of SL usage. So this brings us to the state
of Openjpeg 2.0, Now i know Dzontaz and Callum have done quite a bit
with Openjpeg of various versions in the past, if either of them are
watching do they have any insites to the current state of OJP 2.0 for
SL usage (or does anyone else), as very little has happened in
Openjpeg svn for a long while now I fear if we want this to work we
may have to fix it and supply the patches upstream ourselves. I
believe at one point Dzontas was using OJP 2 for download but OJP 1
for upload when testing it.

Regards

Robin
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges


Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-21 Thread Aidan Thornton
On 9/21/10, Robin Cornelius robin.cornel...@gmail.com wrote:
 I'm still not 100% sure why the build of openjpeg supplied by imprudence
 is better than my builds on 2005.

Imprudence is using the openjpeg SVN trunk version from a few months
ago, which is essentially a pre-release version of openjpeg 1.4 and is
slightly more optimised than 1.3.

You might want to try openjpeg from SVN yourself (the trunk version,
not the 2.0 branch). Make sure to try it with SSE enabled too - it has
some SSE-specific optimisations.
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges


Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-21 Thread Robin Cornelius
On Tue, Sep 21, 2010 at 7:44 PM, Aidan Thornton makos...@gmail.com wrote:
 On 9/21/10, Robin Cornelius robin.cornel...@gmail.com wrote:
 I'm still not 100% sure why the build of openjpeg supplied by imprudence
 is better than my builds on 2005.

 Imprudence is using the openjpeg SVN trunk version from a few months
 ago, which is essentially a pre-release version of openjpeg 1.4 and is
 slightly more optimised than 1.3.

I built off revision 635 from trunk and tried with no SSE/SSE1 and
SSE2 as per my results, SSE1 on 2008 gives the best effect, SSE2 is a
disaster all around on my setup.

Robin
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges


Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-21 Thread Lillian Yiyuan
Could you publish the test harness, I would like to try this on the
mac with a mactools build of v2

On Tue, Sep 21, 2010 at 2:49 PM, Robin Cornelius
robin.cornel...@gmail.com wrote:
 On Tue, Sep 21, 2010 at 7:44 PM, Aidan Thornton makos...@gmail.com wrote:
 On 9/21/10, Robin Cornelius robin.cornel...@gmail.com wrote:
 I'm still not 100% sure why the build of openjpeg supplied by imprudence
 is better than my builds on 2005.

 Imprudence is using the openjpeg SVN trunk version from a few months
 ago, which is essentially a pre-release version of openjpeg 1.4 and is
 slightly more optimised than 1.3.

 I built off revision 635 from trunk and tried with no SSE/SSE1 and
 SSE2 as per my results, SSE1 on 2008 gives the best effect, SSE2 is a
 disaster all around on my setup.

 Robin
 ___
 Policies and (un)subscribe information available here:
 http://wiki.secondlife.com/wiki/OpenSource-Dev
 Please read the policies before posting to keep unmoderated posting privileges

___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges


Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

2010-09-21 Thread Sheet Spotter
I would love to see Robin's test harness. I would also love to see the
images that failed with v2.

I started diving into the v2 branch of OpenJPEG a few days ago, after
research JPEG 2000 for a week. Tackling the bugs from the failed images
might be a good way to become more familiar with the source code.

I posted a very minor patch correcting some simple build problems with the
DLL version of OpenJPEG V2.
http://code.google.com/p/openjpeg/issues/detail?id=40

Does anyone have a sense of how actively the OpenJPEG source code is
maintained?

Thank you for publishing your results, Robin. They are encouraging me to
continue looking at OpenJPEG v2.


Sheet Spotter

-Original Message-
From: opensource-dev-boun...@lists.secondlife.com
[mailto:opensource-dev-boun...@lists.secondlife.com] On Behalf Of Lillian
Yiyuan
Sent: September 21, 2010 4:24 PM
Cc: opensource-dev
Subject: Re: [opensource-dev] Openjpeg/KDU the cold hard metrics

Could you publish the test harness, I would like to try this on the
mac with a mactools build of v2

On Tue, Sep 21, 2010 at 2:49 PM, Robin Cornelius
robin.cornel...@gmail.com wrote:
 On Tue, Sep 21, 2010 at 7:44 PM, Aidan Thornton makos...@gmail.com
wrote:
 On 9/21/10, Robin Cornelius robin.cornel...@gmail.com wrote:
 I'm still not 100% sure why the build of openjpeg supplied by imprudence
 is better than my builds on 2005.

 Imprudence is using the openjpeg SVN trunk version from a few months
 ago, which is essentially a pre-release version of openjpeg 1.4 and is
 slightly more optimised than 1.3.

 I built off revision 635 from trunk and tried with no SSE/SSE1 and
 SSE2 as per my results, SSE1 on 2008 gives the best effect, SSE2 is a
 disaster all around on my setup.

 Robin
 ___
 Policies and (un)subscribe information available here:
 http://wiki.secondlife.com/wiki/OpenSource-Dev
 Please read the policies before posting to keep unmoderated posting
privileges

___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting
privileges

___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges