Hi Rupert, Thanks for the feedback.
Matt On Wed, Jul 25, 2012 at 3:48 PM, Rupert Brooks <[email protected]> wrote: > Matt, > > I just upgraded master to current (which i think includes these changes), > rebuilt and reran on both systems. I cannot see a statisically significant > difference. To truly check we would have to run multiple times and > compare, but its safe to say if theres a difference, its subtle. However, i > did not particularly choose my filter chain to have the GetInput/Output > problem. I threw together something more or less similar to something i > already did, so this cant be considered conclusive. > > Rupert > > New results: > itk4, Visual 2005 > System: CAMD5C5PMN1 > Processor: Pentium III (0.18 micron) With 1 Or 2 MB On-Die L2 Cache > Serial #: > Cache: -1 > Clock: 2800 > Cores: 4 cpus x 1 Cores = 4 > OSName: Windows > Release: XP Professional > Version: Service Pack 3 (Build 2600) > Platform: x86 > Operating System is 32 bit > ITK Version: 4.3.0 > Virtual Memory: Total: 2047 Available: 2008 > Physical Memory: Total:3581 Available: 1370 > Probe Name: Count Min Mean > Stdev Max Total > FilterChain_1_threads 10 0.0799255 0.0812134 > 0.0856113 0.0824928 0.812134 > FilterChain_2_threads 10 0.0585442 0.101192 > 0.107718 0.1078 1.01192 > FilterChain_3_threads 10 0.0670929 0.107661 > 0.115547 0.124683 1.07661 > FilterChain_4_threads 10 0.0634041 0.109252 > 0.117602 0.125126 1.09252 > Image2D 10 0.0952263 0.0958263 > 0.101011 0.0965271 0.958263 > Image3D 10 0.195629 0.197874 > 0.208581 0.200047 1.97874 > MeanSquares_1_threads 10 0.979141 0.98356 > 1.03678 0.997391 9.8356 > MeanSquares_2_threads 10 0.685131 0.712635 > 0.751585 0.749096 7.12635 > MeanSquares_3_threads 10 0.525139 0.549089 > 0.579045 0.57336 5.49089 > MeanSquares_4_threads 10 0.439487 0.452317 > 0.476833 0.458878 4.52317 > > Linux, gcc 4.6.1 > ./itkbench > System: morrigan > Processor: Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz > Serial #: > Cache: 2048 > Clock: 2003 > Cores: 4 cpus x 4 Cores = 16 > OSName: Linux > Release: 3.0.0-21-generic > Version: #35-Ubuntu SMP Fri May 25 17:57:41 UTC 2012 > Platform: x86_64 > Operating System is 64 bit > ITK Version: 4.3.0 > Virtual Memory: Total: 20795 Available: 20695 > Physical Memory: Total:8001 Available: 3538 > Probe Name: Count Min Mean > Stdev Max Total > FilterChain_1_threads 10 0.0684249 0.0686138 > 0.0723256 0.0689371 0.686138 > FilterChain_2_threads 10 0.0367901 0.048942 > 0.0519886 0.0535438 0.48942 > FilterChain_3_threads 10 0.0256481 0.0262128 > 0.0276426 0.0280211 0.262128 > FilterChain_4_threads 10 0.02005 0.0246643 > 0.0269426 0.0390608 0.246643 > Image2D 10 0.026371 0.0265399 > 0.0279799 0.0279481 0.265399 > Image3D 10 0.0263801 0.0264313 > 0.0278611 0.0266159 0.264313 > MeanSquares_1_threads 10 0.683572 0.686879 > 0.724048 0.696691 6.86879 > MeanSquares_2_threads 10 0.613772 0.6994 > 0.739107 0.766093 6.994 > MeanSquares_3_threads 10 0.53469 0.565863 > 0.596594 0.579526 5.65863 > MeanSquares_4_threads 10 0.429782 0.463741 > 0.489788 0.50595 4.63741 > -------------------------------------------------------------- > Rupert Brooks > [email protected] > > > > > On Wed, Jul 25, 2012 at 10:47 AM, Matt McCormick > <[email protected]> wrote: >> >> Hi Rupert, >> >> Very interesting. I am curious if the recent GetInput/GetOutput patch >> helps with the quantities. >> >> Thanks, >> Matt >> >> On Wed, Jul 25, 2012 at 2:12 PM, Rupert Brooks <[email protected]> >> wrote: >> > Hi, >> > >> > I was commenting the other day that ITK4 seems to have lower performance >> > than ITK3. I put together a little benchmark to test this objectively. >> > Results for a couple of systems are below. The benchmark is neither >> > particularly sophisticated nor exact, but i have tried to make it >> > typical of >> > what i use ITK for. I have not tried it with Hans' patch as that got >> > abandoned. Will follow up separately on that. >> > >> > In the numbers below, you will see that performance seems to have >> > degraded >> > across the board. One exception is the 2D image test. In this test i >> > was >> > lazy, and i left the image axis aligned with unit spacing. I am >> > suspecting >> > a special case optimization in ITK4. At the moment, this blocks my >> > ability >> > to upgrade my projects to ITK4, as they are performance critical. Any >> > hints >> > and tips to get those milliseconds back will be much appreciated. >> > >> > The code is at https://github.com/rupertbrooks/itkbench, i'd be curious >> > if >> > others get the same results, and of course, criticism and improvements >> > are >> > welcome. >> > >> > Note that the output about what processor / cores is on the system is >> > bogus. >> > The windows system is a core i7 with 4 cores, the linux one is a Core2 >> > Quad. >> > I suspect a bug in itksys::SystemInformation, but i will follow up on >> > that >> > separately also. >> > >> > In all itk builds, ITK_USE_REVIEW is on. In the itk3 builds, >> > USE_OPTIMZIZED_REGISTRATION, and the new statistics framework are turned >> > on. >> > >> > Cheers, >> > Rupert >> > >> > Data follows..... >> > >> > itk3 Windows XP 32bit Visual Studio Pro 2005 Build RelWithDebInfo >> > System: CAMD5C5PMN1 >> > Processor: Pentium III (0.18 micron) With 1 Or 2 MB On-Die L2 Cache >> > Serial #: >> > Cache: -1 >> > Clock: 2800 >> > Cores: 4 cpus x 1 Cores = 4 >> > OSName: Windows >> > Release: XP Professional >> > Version: Service Pack 3 (Build 2600) >> > Platform: x86 >> > Operating System is 32 bit >> > ITK Version: 3.20.0 >> > Virtual Memory: Total: 2047 Available: 2021 >> > Physical Memory: Total:3581 Available: 1457 >> > Probe Name: Count Min Mean >> > Stdev Max Total >> > FilterChain_1_threads 10 0.0345688 0.0361805 >> > 0.0381523 0.0375099 0.361805 >> > FilterChain_2_threads 10 0.0257301 0.0265144 >> > 0.0279574 0.0282478 0.265144 >> > FilterChain_3_threads 10 0.022007 0.0232452 >> > 0.0245125 0.0240822 0.232452 >> > FilterChain_4_threads 10 0.0199661 0.0221756 >> > 0.0235205 0.0288773 0.221756 >> > Image2D 10 0.144909 0.14665 >> > 0.154586 0.148136 1.4665 >> > Image3D 10 0.19125 0.193378 >> > 0.203846 0.196495 1.93378 >> > MeanSquares_1_threads 10 0.612728 0.616159 >> > 0.649503 0.627789 6.16159 >> > MeanSquares_2_threads 10 0.359703 0.368051 >> > 0.388106 0.392696 3.68051 >> > MeanSquares_3_threads 10 0.268024 0.280672 >> > 0.296142 0.312557 2.80672 >> > MeanSquares_4_threads 10 0.199249 0.213209 >> > 0.224918 0.2243 2.13209 >> > >> > >> > itk4 Windows XP 32bit Visual Studio Pro 2005 Build RelWithDebInfo >> > >> > System: CAMD5C5PMN1 >> > Processor: Pentium III (0.18 micron) With 1 Or 2 MB On-Die L2 Cache >> > Serial #: >> > Cache: -1 >> > Clock: 2800 >> > Cores: 4 cpus x 1 Cores = 4 >> > OSName: Windows >> > Release: XP Professional >> > Version: Service Pack 3 (Build 2600) >> > Platform: x86 >> > Operating System is 32 bit >> > ITK Version: 4.3.0 >> > Virtual Memory: Total: 2047 Available: 2008 >> > Physical Memory: Total:3581 Available: 1574 >> > Probe Name: Count Min Mean >> > Stdev Max Total >> > FilterChain_1_threads 10 0.0797005 0.0808155 >> > 0.0851898 0.0820313 0.808155 >> > FilterChain_2_threads 10 0.0670509 0.0921925 >> > 0.0991914 0.109177 0.921925 >> > FilterChain_3_threads 10 0.0648499 0.113566 >> > 0.120975 0.125893 1.13566 >> > FilterChain_4_threads 10 0.0536919 0.104662 >> > 0.114558 0.132114 1.04662 >> > Image2D 10 0.0953979 0.0965282 >> > 0.101753 0.0978088 0.965282 >> > Image3D 10 0.196468 0.198288 >> > 0.209017 0.200562 1.98288 >> > MeanSquares_1_threads 10 0.982071 0.986089 >> > 1.03944 0.996445 9.86089 >> > MeanSquares_2_threads 10 0.686028 0.728621 >> > 0.768821 0.799675 7.28621 >> > MeanSquares_3_threads 10 0.534885 0.554387 >> > 0.5846 0.578133 5.54387 >> > MeanSquares_4_threads 10 0.441551 0.463463 >> > 0.488903 0.502502 4.63463 >> > >> > >> > itk3 Linux 64 bit gcc 4.6.1 Build Release >> > System: morrigan >> > Processor: Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz >> > Serial #: >> > Cache: 2048 >> > Clock: 2003 >> > Cores: 4 cpus x 4 Cores = 16 >> > OSName: Linux >> > Release: 3.0.0-21-generic >> > Version: #35-Ubuntu SMP Fri May 25 17:57:41 UTC 2012 >> > Platform: x86_64 >> > Operating System is 64 bit >> > ITK Version: 3.20.1 >> > Virtual Memory: Total: 20795 Available: 20795 >> > Physical Memory: Total:8001 Available: 3416 >> > Probe Name: Count Min Mean >> > Stdev >> > Max Total >> > FilterChain_1_threads 10 0.0309269 0.031097 >> > 0.0327796 0.031528 0.31097 >> > FilterChain_2_threads 10 0.021672 0.0248219 >> > 0.0261976 0.026264 0.248219 >> > FilterChain_3_threads 10 0.0213931 0.0217796 >> > 0.0229634 0.0228851 0.217796 >> > FilterChain_4_threads 10 0.0198419 0.0207421 >> > 0.0218725 0.0216701 0.207421 >> > Image2D 10 0.0263629 0.0264159 >> > 0.0278451 0.0267591 0.264159 >> > Image3D 10 0.0263779 0.0263906 >> > 0.0278181 0.026463 0.263906 >> > MeanSquares_1_threads 10 0.463909 0.465901 >> > 0.491113 0.473644 4.65901 >> > MeanSquares_2_threads 10 0.349138 0.40529 >> > 0.429438 0.487785 4.0529 >> > MeanSquares_3_threads 10 0.331908 0.357685 >> > 0.377834 0.400941 3.57685 >> > MeanSquares_4_threads 10 0.299428 0.336914 >> > 0.356878 0.395144 3.36914 >> > >> > itk4 Linux 64-bit gcc 4.6.1 Build Release >> > System: morrigan >> > Processor: Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz >> > Serial #: >> > Cache: 2048 >> > Clock: 2003 >> > Cores: 4 cpus x 4 Cores = 16 >> > OSName: Linux >> > Release: 3.0.0-21-generic >> > Version: #35-Ubuntu SMP Fri May 25 17:57:41 UTC 2012 >> > Platform: x86_64 >> > Operating System is 64 bit >> > ITK Version: 4.3.0 >> > Virtual Memory: Total: 20795 Available: 20795 >> > Physical Memory: Total:8001 Available: 3400 >> > Probe Name: Count Min Mean >> > Stdev >> > Max Total >> > FilterChain_1_threads 10 0.0663331 0.0668247 >> > 0.0704416 0.0681288 0.668247 >> > FilterChain_2_threads 10 0.0353 0.0393441 >> > 0.042048 0.0534279 0.393441 >> > FilterChain_3_threads 10 0.025275 0.0265869 >> > 0.0282408 0.0364931 0.265869 >> > FilterChain_4_threads 10 0.0208249 0.0222288 >> > 0.0235333 0.028336 0.222288 >> > Image2D 10 0.026361 0.026421 >> > 0.0278503 0.0266371 0.26421 >> > Image3D 10 0.0263991 0.0266509 >> > 0.0280954 0.0274949 0.266509 >> > MeanSquares_1_threads 10 0.69005 0.69459 >> > 0.732169 0.701676 6.9459 >> > MeanSquares_2_threads 10 0.590396 0.699826 >> > 0.739994 0.766702 6.99826 >> > MeanSquares_3_threads 10 0.527146 0.538844 >> > 0.568047 0.548791 5.38844 >> > MeanSquares_4_threads 10 0.417008 0.454278 >> > 0.480139 0.533593 4.54278 >> > -------------------------------------------------------------- >> > Rupert Brooks >> > [email protected] >> > >> > >> > _______________________________________________ >> > Powered by www.kitware.com >> > >> > Visit other Kitware open-source projects at >> > http://www.kitware.com/opensource/opensource.html >> > >> > Kitware offers ITK Training Courses, for more information visit: >> > http://kitware.com/products/protraining.php >> > >> > Please keep messages on-topic and check the ITK FAQ at: >> > http://www.itk.org/Wiki/ITK_FAQ >> > >> > Follow this link to subscribe/unsubscribe: >> > http://www.itk.org/mailman/listinfo/insight-developers >> > > > _______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Kitware offers ITK Training Courses, for more information visit: http://kitware.com/products/protraining.php Please keep messages on-topic and check the ITK FAQ at: http://www.itk.org/Wiki/ITK_FAQ Follow this link to subscribe/unsubscribe: http://www.itk.org/mailman/listinfo/insight-developers
