[Bf-committers] massive cuda speed improvements with Cuda 5.0/5.5
Hi there, I did some tests with cuda 5.0 and 5.5 today and changed the nvcc optimization flags for cycles_kernel_cuda. I found out the following: - --opencc-options is deprecated for sm_20 and up and should be removed from compiler options - Stating -O3 and use_fast_math as nvcc options brings massive speedup on my system (more below) - We shouldnt complain about new cuda toolsets that are slow, we should find a solution as we cant use old software forever To the speedups: Example 1: system: i7-3820 @ 3.60GHz, GeForce GTK 660 Blender (cycles_cuda_kernel) compiled with standard settings: Mike_pan file took 02:06.60 to render Blender (cycles_cuda_kernel) compiled with O3 use-fast-math: Mike_pan took 01:39:93 There is no optical difference in the render results: Image1: http://www.pasteall.org/pic/52757 Image2: http://www.pasteall.org/pic/52758 I bet theres more potential in there. /Jürgen ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] massive cuda speed improvements with Cuda 5.0/5.5
Hi Jurgen, How does this times compare between CUDA 5.0 and 5.5? (is this a speedup from 5.5 but a slowdown in relation with 5.0? or it's an overall speed up ?) -- Dalai 2013/6/3 Jürgen Herrmann shadow...@me.com: Hi there, I did some tests with cuda 5.0 and 5.5 today and changed the nvcc optimization flags for cycles_kernel_cuda. I found out the following: - “--opencc-options “ is deprecated for sm_20 and up and should be removed from compiler options - Stating “-O3” and “—use_fast_math” as nvcc options brings massive speedup on my system (more below) - We shouldn’t complain about new cuda toolsets that are slow, we should find a solution as we can’t use old software forever… To the speedups: Example 1: system: i7-3820 @ 3.60GHz, GeForce GTK 660 Blender (cycles_cuda_kernel) compiled with standard settings: Mike_pan file took 02:06.60 to render Blender (cycles_cuda_kernel) compiled with –O3 –use-fast-math: Mike_pan took 01:39:93 There is no optical difference in the render results: Image1: http://www.pasteall.org/pic/52757 Image2: http://www.pasteall.org/pic/52758 I bet there’s more potential in there. /Jürgen ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] massive cuda speed improvements with Cuda 5.0/5.5
Hi Dalai, I tested 5.5 on a different system I don't have access to this machine right now, I'll deliver the complete benchmark results tomorrow. I plan to compare on as many different configurations with 32 and 64 bit and different cuda versions. This will take some time but I think it's worth it. ;) /Jürgen Am 03.06.2013 um 20:50 schrieb Dalai Felinto dfeli...@gmail.com: Hi Jurgen, How does this times compare between CUDA 5.0 and 5.5? (is this a speedup from 5.5 but a slowdown in relation with 5.0? or it's an overall speed up ?) -- Dalai 2013/6/3 Jürgen Herrmann shadow...@me.com: Hi there, I did some tests with cuda 5.0 and 5.5 today and changed the nvcc optimization flags for cycles_kernel_cuda. I found out the following: - “--opencc-options “ is deprecated for sm_20 and up and should be removed from compiler options - Stating “-O3” and “—use_fast_math” as nvcc options brings massive speedup on my system (more below) - We shouldn’t complain about new cuda toolsets that are slow, we should find a solution as we can’t use old software forever… To the speedups: Example 1: system: i7-3820 @ 3.60GHz, GeForce GTK 660 Blender (cycles_cuda_kernel) compiled with standard settings: Mike_pan file took 02:06.60 to render Blender (cycles_cuda_kernel) compiled with –O3 –use-fast-math: Mike_pan took 01:39:93 There is no optical difference in the render results: Image1: http://www.pasteall.org/pic/52757 Image2: http://www.pasteall.org/pic/52758 I bet there’s more potential in there. /Jürgen ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] massive cuda speed improvements with Cuda 5.0/5.5
Thanks for testing. I've also been doing some experimenting with
compile flags and other things here. So far it seems I can make my
650M render a few percentages faster compared to CUDA 4.2, but 460 GT
is still considerably slower with the BMW scene (2m30s with 5.5
compared to 2m01s with 4.2), and 580 GTX had a similar difference. It
seems you are testing with a 6xx card so that makes sense.
Patch attached for those who want to test this with 5.0/5.5.
On Mon, Jun 3, 2013 at 8:46 PM, Jürgen Herrmann shadow...@me.com wrote:
Hi there,
I did some tests with cuda 5.0 and 5.5 today and changed the nvcc
optimization flags for cycles_kernel_cuda.
I found out the following:
- “--opencc-options “ is deprecated for sm_20 and up and should be
removed from compiler options
- Stating “-O3” and “—use_fast_math” as nvcc options brings massive
speedup on my system (more below)
- We shouldn’t complain about new cuda toolsets that are slow, we
should find a solution as we can’t use old software forever…
To the speedups:
Example 1:
system: i7-3820 @ 3.60GHz, GeForce GTK 660
Blender (cycles_cuda_kernel) compiled with standard settings:
Mike_pan file took 02:06.60 to render
Blender (cycles_cuda_kernel) compiled with –O3 –use-fast-math:
Mike_pan took 01:39:93
There is no optical difference in the render results:
Image1: http://www.pasteall.org/pic/52757
Image2: http://www.pasteall.org/pic/52758
I bet there’s more potential in there.
/Jürgen
___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers
diff --git a/intern/cycles/device/device_cuda.cpp
b/intern/cycles/device/device_cuda.cpp
index f32c6dd..27978b9 100644
--- a/intern/cycles/device/device_cuda.cpp
+++ b/intern/cycles/device/device_cuda.cpp
@@ -46,6 +46,7 @@ public:
mapdevice_ptr, bool tex_interp_map;
int cuDevId;
bool first_error;
+ vectorCUstream cuStreams;
struct PixelMem {
GLuint cuPBO;
@@ -205,6 +206,12 @@ public:
if(cuda_error_(result, cuCtxCreate))
return;
+ const int num_streams = 8;
+ cuStreams.resize(num_streams);
+
+ for(int i = 0; i num_streams; i++)
+ cuStreamCreate(cuStreams[i], 0);
+
cuda_pop_context();
}
@@ -212,6 +219,9 @@ public:
{
task_pool.stop();
+ for(int i = 0; i cuStreams.size(); i++)
+ cuStreamDestroy(cuStreams[i]);
+
cuda_push_context();
cuda_assert(cuCtxDetach(cuContext))
}
@@ -514,7 +524,7 @@ public:
}
}
- void path_trace(RenderTile rtile, int sample)
+ void path_trace(RenderTile rtile, int sample, CUstream stream)
{
if(have_error())
return;
@@ -575,9 +585,9 @@ public:
cuda_assert(cuFuncSetCacheConfig(cuPathTrace,
CU_FUNC_CACHE_PREFER_L1))
cuda_assert(cuFuncSetBlockShape(cuPathTrace, xthreads,
ythreads, 1))
- cuda_assert(cuLaunchGrid(cuPathTrace, xblocks, yblocks))
+ cuda_assert(cuLaunchGridAsync(cuPathTrace, xblocks, yblocks,
stream))
- cuda_assert(cuCtxSynchronize())
+ //cuda_assert(cuCtxSynchronize())
cuda_pop_context();
}
@@ -882,12 +892,35 @@ public:
void thread_run(DeviceTask *task)
{
if(task-type == DeviceTask::PATH_TRACE) {
- RenderTile tile;
+ vectorRenderTile concurrent_tiles(cuStreams.size());
+ vectorbool have_tile(cuStreams.size());
/* keep rendering tiles until done */
- while(task-acquire_tile(this, tile)) {
- int start_sample = tile.start_sample;
- int end_sample = tile.start_sample +
tile.num_samples;
+ while(1) {
+ int start_sample = -1;
+ int end_sample = -1;
+
+ for(int i = 0; i concurrent_tiles.size();
i++) {
+ RenderTile tile = concurrent_tiles[i];
+
+ if(task-acquire_tile(this, tile)) {
+ have_tile[i] = true;
+
+ if(start_sample == -1) {
+ start_sample =
tile.start_sample;
+ end_sample =
tile.start_sample + tile.num_samples;
+ }
+ else {
+
Re: [Bf-committers] massive cuda speed improvements with Cuda 5.0/5.5
Hi Brecht, you're welcome ;) I think I'll have to include these compiler flag optimizations into vs2012 builds otherwise these builds will be significantly slower than other compilers :( Cycles has two problems when it comes to windows/vs2012 builds: 1. It is much slower on CPU 2. It is slower with cuda, if we don't use opt flags. As long as MinGW OpenMP isn't fixed we have to stick to vs2008/cuda 4.2 or find a decent solution for VS2012/Cuda 5.5. As blender seems to be used mostly by windows users this isn't optimal. /Jürgen Am 03.06.2013 um 21:20 schrieb Brecht Van Lommel brechtvanlom...@pandora.be: Thanks for testing. I've also been doing some experimenting with compile flags and other things here. So far it seems I can make my 650M render a few percentages faster compared to CUDA 4.2, but 460 GT is still considerably slower with the BMW scene (2m30s with 5.5 compared to 2m01s with 4.2), and 580 GTX had a similar difference. It seems you are testing with a 6xx card so that makes sense. Patch attached for those who want to test this with 5.0/5.5. On Mon, Jun 3, 2013 at 8:46 PM, Jürgen Herrmann shadow...@me.com wrote: Hi there, I did some tests with cuda 5.0 and 5.5 today and changed the nvcc optimization flags for cycles_kernel_cuda. I found out the following: - “--opencc-options “ is deprecated for sm_20 and up and should be removed from compiler options - Stating “-O3” and “—use_fast_math” as nvcc options brings massive speedup on my system (more below) - We shouldn’t complain about new cuda toolsets that are slow, we should find a solution as we can’t use old software forever… To the speedups: Example 1: system: i7-3820 @ 3.60GHz, GeForce GTK 660 Blender (cycles_cuda_kernel) compiled with standard settings: Mike_pan file took 02:06.60 to render Blender (cycles_cuda_kernel) compiled with –O3 –use-fast-math: Mike_pan took 01:39:93 There is no optical difference in the render results: Image1: http://www.pasteall.org/pic/52757 Image2: http://www.pasteall.org/pic/52758 I bet there’s more potential in there. /Jürgen ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers cuda_5.5_tests.txt ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
[Bf-committers] massive cuda speed improvements with Cuda 5.0/5.5
Hi all, As long as MinGW OpenMP isn't fixed it is fixed in http://tdm-gcc.tdragon.net/download ( see openmp download ) and patch to any other MinGW version for gomp library can be found here http://netcologne.dl.sourceforge.net/project/tdm-gcc/Sources/TDM%20Sources/gcc-4.7.1-tdmsrc-1.zip not sure how useful it is, but still - there is a patch which can be applied. Regards Sergey ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] massive cuda speed improvements with Cuda 5.0/5.5
Hi Jürgen, Am 03.06.2013 20:46, schrieb Jürgen Herrmann: - “--opencc-options “ is deprecated for sm_20 and up and should be removed from compiler options This option should not be harmful, we just kept it in for sm_1x architecture. Although I am not sure if sm_1x still builds at all, we dropped official support for it with Blender 2.67, so probably can be removed. - Stating “-O3” and “—use_fast_math” as nvcc options brings massive speedup on my system (more below) We cannot use fast math everywhere, I remember that Brecht changed the code to only use fast math functions selectively, to avoid some precision problems. http://projects.blender.org/scm/viewvc.php?view=revroot=bf-blenderrevision=47133 But that could not be valid with Toolkit 5.x anymore, needs further tests. - We shouldn’t complain about new cuda toolsets that are slow, we should find a solution as we can’t use old software forever… True, but still it's sad that an upgrade causes all those problems. :) Thanks for testing! Thomas -- Thomas Dinges Blender Developer, Artist and Musician www.dingto.org ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] massive cuda speed improvements with Cuda 5.0/5.5
Hi Brecht, this looks very promising. On my Geforce 540M (Windows x64, driver 320.20) I get those render times with the BMW scene (100 Samples, 128x128 tiles). Vanilla Trunk (Toolkit 4.2): 2.29 minutes Vanilla Trunk (Toolkit 5.5 RC): 3.54 minutes Trunk + patch (Toolkit 5.5 RC): 3.00 minutes So, that is definitely much better. :) Best regards, Thomas Am 03.06.2013 21:20, schrieb Brecht Van Lommel: Thanks for testing. I've also been doing some experimenting with compile flags and other things here. So far it seems I can make my 650M render a few percentages faster compared to CUDA 4.2, but 460 GT is still considerably slower with the BMW scene (2m30s with 5.5 compared to 2m01s with 4.2), and 580 GTX had a similar difference. It seems you are testing with a 6xx card so that makes sense. Patch attached for those who want to test this with 5.0/5.5. -- Thomas Dinges Blender Developer, Artist and Musician www.dingto.org ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] massive cuda speed improvements with Cuda 5.0/5.5
hi it is fixed in http://tdm-gcc.tdragon.net/download ( see openmp download ) and patch to any other MinGW version for gomp library can be found here http://netcologne.dl.sourceforge.net/project/tdm-gcc/Sources/TDM%20Sources/gcc-4.7.1-tdmsrc-1.zip can Antony maybe check this and enable openmp correctly and solves the mingw64 problem, this could be the answer guys! Regards Yousef Harfoush ba...@msn.com ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] massive cuda speed improvements with Cuda 5.0/5.5
hi again i had time so i did some tests with a simple smoke scene: gcc 4.7.1 wingw64 with openmp NOT compiled took 1:46 s gcc 4.7.1 wingw64 with openmp COMPILED crashes gcc 4.7.1 wingw64 with replacing the fixed dlls from the site and openmp COMPILED took 0:25 s it seems the problem has been fixed :) i hope some one DO a full blender regression test. Regards Yousef Harfoush ba...@msn.com ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] massive cuda speed improvements with Cuda 5.0/5.5
Sounds promising! If that really works we could just drop the whole MSVC stuff and build with mingw ;) Am 04.06.2013 um 04:09 schrieb Yousef Hurfoush ba...@msn.com: hi again i had time so i did some tests with a simple smoke scene: gcc 4.7.1 wingw64 with openmp NOT compiled took 1:46 s gcc 4.7.1 wingw64 with openmp COMPILED crashes gcc 4.7.1 wingw64 with replacing the fixed dlls from the site and openmp COMPILED took 0:25 s it seems the problem has been fixed :) i hope some one DO a full blender regression test. Regards Yousef Harfoush ba...@msn.com ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
