Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
Thanks for checking it! I'll update README for CURAND.jl On Monday, December 7, 2015 at 8:56:36 PM UTC+3, Joaquim Masset Lacombe Dias Garcia wrote: > > And it also works on windows if CUDA version is 7.5! > > On Wed, Nov 25, 2015 at 7:58 PM, Joaquim Masset Lacombe Dias Garcia < > joaquim...@gmail.com > wrote: > >> Wow! it seems that something is wrong in CUDA 7.0 ! >> >> I updated my MAC to cuda 7.5 and bith the C code and julia code worked >> fine! >> This other guy: >> >> http://stackoverflow.com/questions/33904554/curand-error-while-alternating-with-device-initialization-and-reset-in-cuda-7-0?noredirect=1#comment55573150_33904554 >> >> had the C code running smoothly on windows with CUDA 7.5 >> >> I will try the julia code on my windows notebook later this week with >> CUDA 7.5 >> >> If somebody else could confirm the problem on curand in cuda 7.0, we >> could add that in CURAND.jl documentation. >> >> >> >> Em quarta-feira, 25 de novembro de 2015 19:20:48 UTC-2, Andrei Zh >> escreveu: >>> >>> @Joaquim Would it be possible for you to upgrade or donwgrade your CUDA >>> version? >>> >>> Meanwhile, we are still waiting for other people with Windows or Mac :D >>> >>> On Wed, Nov 25, 2015 at 6:35 PM, Sergio Muniz>>> wrote: >>> It works fine here, on Ubuntu 14.04 and CUDA 6.5. Cheers, [S]. On Tuesday, November 24, 2015 at 9:30:32 PM UTC-2, Joaquim Masset Lacombe Dias Garcia wrote: > > Interesting, both my machines (windows and mac) have CUDA 7.0, > possibly thats the issue, since the C code in > https://github.com/JuliaGPU/CURAND.jl/issues/3#issuecomment-159319580 > fails in both. > > If some linux user could test this version, our statistics would be > more complete. I will try 6.5 and 7.5 > > Em terça-feira, 24 de novembro de 2015 21:21:41 UTC-2, Tim Holy > escreveu: >> >> 6.5 >> >> --Tim >> >> On Wednesday, November 25, 2015 01:37:22 AM Andrei wrote: >> > On Tue, Nov 24, 2015 at 8:03 PM, Kristoffer Carlsson < >> kcarl...@gmail.com> >> > wrote: >> > > The original code in the OP fails for me >> > >> > Yes, this is expected behavior: for convenience, CURAND.jl creates >> default >> > random number generator, which obviously becomes invalid after >> > `device_reset()`. At the same time, my last code snippet creates >> new and >> > explicit RNG, which fixes the issue on Linux. >> > >> > The problem is that on some platforms even creating new generator >> doesn't >> > help, so I'm trying to understand the difference. >> > >> > @Tim, @Kristoffer, could you also specify CUDA version in use, >> please? >> >> >>> >
Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
And it also works on windows if CUDA version is 7.5! On Wed, Nov 25, 2015 at 7:58 PM, Joaquim Masset Lacombe Dias Garcia < joaquimdgar...@gmail.com> wrote: > Wow! it seems that something is wrong in CUDA 7.0 ! > > I updated my MAC to cuda 7.5 and bith the C code and julia code worked > fine! > This other guy: > > http://stackoverflow.com/questions/33904554/curand-error-while-alternating-with-device-initialization-and-reset-in-cuda-7-0?noredirect=1#comment55573150_33904554 > > had the C code running smoothly on windows with CUDA 7.5 > > I will try the julia code on my windows notebook later this week with CUDA > 7.5 > > If somebody else could confirm the problem on curand in cuda 7.0, we could > add that in CURAND.jl documentation. > > > > Em quarta-feira, 25 de novembro de 2015 19:20:48 UTC-2, Andrei Zh escreveu: >> >> @Joaquim Would it be possible for you to upgrade or donwgrade your CUDA >> version? >> >> Meanwhile, we are still waiting for other people with Windows or Mac :D >> >> On Wed, Nov 25, 2015 at 6:35 PM, Sergio Munizwrote: >> >>> It works fine here, on Ubuntu 14.04 and CUDA 6.5. >>> >>> Cheers, >>> [S]. >>> >>> >>> >>> >>> On Tuesday, November 24, 2015 at 9:30:32 PM UTC-2, Joaquim Masset >>> Lacombe Dias Garcia wrote: Interesting, both my machines (windows and mac) have CUDA 7.0, possibly thats the issue, since the C code in https://github.com/JuliaGPU/CURAND.jl/issues/3#issuecomment-159319580 fails in both. If some linux user could test this version, our statistics would be more complete. I will try 6.5 and 7.5 Em terça-feira, 24 de novembro de 2015 21:21:41 UTC-2, Tim Holy escreveu: > > 6.5 > > --Tim > > On Wednesday, November 25, 2015 01:37:22 AM Andrei wrote: > > On Tue, Nov 24, 2015 at 8:03 PM, Kristoffer Carlsson < > kcarl...@gmail.com> > > wrote: > > > The original code in the OP fails for me > > > > Yes, this is expected behavior: for convenience, CURAND.jl creates > default > > random number generator, which obviously becomes invalid after > > `device_reset()`. At the same time, my last code snippet creates new > and > > explicit RNG, which fixes the issue on Linux. > > > > The problem is that on some platforms even creating new generator > doesn't > > help, so I'm trying to understand the difference. > > > > @Tim, @Kristoffer, could you also specify CUDA version in use, > please? > > >>
Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
It works fine here, on Ubuntu 14.04 and CUDA 6.5. Cheers, [S]. On Tuesday, November 24, 2015 at 9:30:32 PM UTC-2, Joaquim Masset Lacombe Dias Garcia wrote: > > Interesting, both my machines (windows and mac) have CUDA 7.0, possibly > thats the issue, since the C code in > https://github.com/JuliaGPU/CURAND.jl/issues/3#issuecomment-159319580 > fails in both. > > If some linux user could test this version, our statistics would be more > complete. I will try 6.5 and 7.5 > > Em terça-feira, 24 de novembro de 2015 21:21:41 UTC-2, Tim Holy escreveu: >> >> 6.5 >> >> --Tim >> >> On Wednesday, November 25, 2015 01:37:22 AM Andrei wrote: >> > On Tue, Nov 24, 2015 at 8:03 PM, Kristoffer Carlsson < >> kcarl...@gmail.com> >> > wrote: >> > > The original code in the OP fails for me >> > >> > Yes, this is expected behavior: for convenience, CURAND.jl creates >> default >> > random number generator, which obviously becomes invalid after >> > `device_reset()`. At the same time, my last code snippet creates new >> and >> > explicit RNG, which fixes the issue on Linux. >> > >> > The problem is that on some platforms even creating new generator >> doesn't >> > help, so I'm trying to understand the difference. >> > >> > @Tim, @Kristoffer, could you also specify CUDA version in use, please? >> >>
Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
@Joaquim Would it be possible for you to upgrade or donwgrade your CUDA version? Meanwhile, we are still waiting for other people with Windows or Mac :D On Wed, Nov 25, 2015 at 6:35 PM, Sergio Munizwrote: > It works fine here, on Ubuntu 14.04 and CUDA 6.5. > > Cheers, > [S]. > > > > > On Tuesday, November 24, 2015 at 9:30:32 PM UTC-2, Joaquim Masset Lacombe > Dias Garcia wrote: >> >> Interesting, both my machines (windows and mac) have CUDA 7.0, possibly >> thats the issue, since the C code in >> https://github.com/JuliaGPU/CURAND.jl/issues/3#issuecomment-159319580 >> fails in both. >> >> If some linux user could test this version, our statistics would be more >> complete. I will try 6.5 and 7.5 >> >> Em terça-feira, 24 de novembro de 2015 21:21:41 UTC-2, Tim Holy escreveu: >>> >>> 6.5 >>> >>> --Tim >>> >>> On Wednesday, November 25, 2015 01:37:22 AM Andrei wrote: >>> > On Tue, Nov 24, 2015 at 8:03 PM, Kristoffer Carlsson < >>> kcarl...@gmail.com> >>> > wrote: >>> > > The original code in the OP fails for me >>> > >>> > Yes, this is expected behavior: for convenience, CURAND.jl creates >>> default >>> > random number generator, which obviously becomes invalid after >>> > `device_reset()`. At the same time, my last code snippet creates new >>> and >>> > explicit RNG, which fixes the issue on Linux. >>> > >>> > The problem is that on some platforms even creating new generator >>> doesn't >>> > help, so I'm trying to understand the difference. >>> > >>> > @Tim, @Kristoffer, could you also specify CUDA version in use, please? >>> >>>
Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
Wow! it seems that something is wrong in CUDA 7.0 ! I updated my MAC to cuda 7.5 and bith the C code and julia code worked fine! This other guy: http://stackoverflow.com/questions/33904554/curand-error-while-alternating-with-device-initialization-and-reset-in-cuda-7-0?noredirect=1#comment55573150_33904554 had the C code running smoothly on windows with CUDA 7.5 I will try the julia code on my windows notebook later this week with CUDA 7.5 If somebody else could confirm the problem on curand in cuda 7.0, we could add that in CURAND.jl documentation. Em quarta-feira, 25 de novembro de 2015 19:20:48 UTC-2, Andrei Zh escreveu: > > @Joaquim Would it be possible for you to upgrade or donwgrade your CUDA > version? > > Meanwhile, we are still waiting for other people with Windows or Mac :D > > On Wed, Nov 25, 2015 at 6:35 PM, Sergio Muniz> wrote: > >> It works fine here, on Ubuntu 14.04 and CUDA 6.5. >> >> Cheers, >> [S]. >> >> >> >> >> On Tuesday, November 24, 2015 at 9:30:32 PM UTC-2, Joaquim Masset Lacombe >> Dias Garcia wrote: >>> >>> Interesting, both my machines (windows and mac) have CUDA 7.0, possibly >>> thats the issue, since the C code in >>> https://github.com/JuliaGPU/CURAND.jl/issues/3#issuecomment-159319580 >>> fails in both. >>> >>> If some linux user could test this version, our statistics would be more >>> complete. I will try 6.5 and 7.5 >>> >>> Em terça-feira, 24 de novembro de 2015 21:21:41 UTC-2, Tim Holy escreveu: 6.5 --Tim On Wednesday, November 25, 2015 01:37:22 AM Andrei wrote: > On Tue, Nov 24, 2015 at 8:03 PM, Kristoffer Carlsson < kcarl...@gmail.com> > wrote: > > The original code in the OP fails for me > > Yes, this is expected behavior: for convenience, CURAND.jl creates default > random number generator, which obviously becomes invalid after > `device_reset()`. At the same time, my last code snippet creates new and > explicit RNG, which fixes the issue on Linux. > > The problem is that on some platforms even creating new generator doesn't > help, so I'm trying to understand the difference. > > @Tim, @Kristoffer, could you also specify CUDA version in use, please? >
Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
On Tue, Nov 24, 2015 at 8:03 PM, Kristoffer Carlssonwrote: > The original code in the OP fails for me Yes, this is expected behavior: for convenience, CURAND.jl creates default random number generator, which obviously becomes invalid after `device_reset()`. At the same time, my last code snippet creates new and explicit RNG, which fixes the issue on Linux. The problem is that on some platforms even creating new generator doesn't help, so I'm trying to understand the difference. @Tim, @Kristoffer, could you also specify CUDA version in use, please?
Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
Interesting, both my machines (windows and mac) have CUDA 7.0, possibly thats the issue, since the C code in https://github.com/JuliaGPU/CURAND.jl/issues/3#issuecomment-159319580 fails in both. If some linux user could test this version, our statistics would be more complete. I will try 6.5 and 7.5 Em terça-feira, 24 de novembro de 2015 21:21:41 UTC-2, Tim Holy escreveu: > > 6.5 > > --Tim > > On Wednesday, November 25, 2015 01:37:22 AM Andrei wrote: > > On Tue, Nov 24, 2015 at 8:03 PM, Kristoffer Carlsson> > > wrote: > > > The original code in the OP fails for me > > > > Yes, this is expected behavior: for convenience, CURAND.jl creates > default > > random number generator, which obviously becomes invalid after > > `device_reset()`. At the same time, my last code snippet creates new and > > explicit RNG, which fixes the issue on Linux. > > > > The problem is that on some platforms even creating new generator > doesn't > > help, so I'm trying to understand the difference. > > > > @Tim, @Kristoffer, could you also specify CUDA version in use, please? > >
Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
It didn't throw any errors for me. julia> versioninfo() Julia Version 0.4.2-pre+1 Commit b8b3517* (2015-11-08 10:37 UTC) Platform Info: System: Linux (x86_64-linux-gnu) CPU: Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge) LAPACK: libopenblas64_ LIBM: libopenlibm LLVM: libLLVM-3.3 --Tim On Tuesday, November 24, 2015 08:17:33 AM Andrei Zh wrote: > While investigating this issue we found that CUDArt and CURAND may behave > differently on different platforms. Could somebody from user group with > enabled CUDA try the following code in Julia REPL and report: > > * operating system > * whether this code succeeded or failed; if failed, then what is the > message. > > The code is (just copy and paste): > > Pkg.add("CUDArt") # if it's not installed yet > Pkg.add("CURAND") # if it's not installed yet > > using CUDArt > using CURAND > > rng = create_generator() > d_a = curand(rng,Float64, 1000) > destroy_generator(rng) > a = to_host(d_a) > device_reset() > result = devices(dev->capability(dev)[1]>=2) do devlist > end > > rng = create_generator() > d_a = curand(rng,Float64, 1000) > destroy_generator(rng) > a = to_host(d_a) > device_reset() > result = devices(dev->capability(dev)[1]>=2) do devlist > end > > The goal of this experiment is to see if cuRAND can survive after > `cudaDeviceReset()`. So far it works on one Linux machine and fails on one > Windows machine, so some statistics will be very helpful. > > Thanks. > > > On Friday, November 6, 2015 at 10:46:54 PM UTC+3, Joaquim Masset Lacombe > > Dias Garcia wrote: > > I was playing with the GPU (in both windows and mac) libraries and I came > > up with the following errors: > > The first time I execute the do loop, every thing goes well. > > But If i try a second time in a row I get this invalid pointer error. > > > > I got the same error in all my do loops, any ideas? > > > > best, > > Joaquim > > > > *My code running on terminal:* > > > > julia> using CUDArt > > julia> using CURAND > > julia> result = devices(dev->capability(dev)[1]>=2) do devlist > > > >d_a = curand(Float32, 1000); > >a = to_host(d_a); > > > >end > > > > 1000-element Array{Float64,1}: > > 0.438451 > > 0.460365 > > 0.250215 > > 0.494744 > > 0.0530111 > > 0.337699 > > 0.396763 > > 0.874419 > > 0.482167 > > 0.0428398 > > ? > > 0.563937 > > 0.80706 > > 0.190015 > > 0.334969 > > 0.622164 > > 0.710596 > > 0.0125895 > > 0.990388 > > 0.467796 > > 0.24313 > > > > julia> result = devices(dev->capability(dev)[1]>=2) do devlist > > > >d_a = curand(Float32, 1000); > >a = to_host(d_a); > > > >end > > > > WARNING: CUDA error triggered from: > > in checkerror at > > > > C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\libcudart-6.5.jl > > > > :15 > > : > > in copy! at C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\arrays.jl:152 > > in to_host at C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\arrays.jl:87 > > in anonymous at none:3 > > in devices at > > > > C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\device.jl:61ERROR: > > Launch failed, perhaps due to an invalid pointer > > > > in checkdrv at C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\module.jl:6 > > in close at C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\device.jl:136 > > in devices at C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\device.jl:63
Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
6.5 --Tim On Wednesday, November 25, 2015 01:37:22 AM Andrei wrote: > On Tue, Nov 24, 2015 at 8:03 PM, Kristoffer Carlsson> wrote: > > The original code in the OP fails for me > > Yes, this is expected behavior: for convenience, CURAND.jl creates default > random number generator, which obviously becomes invalid after > `device_reset()`. At the same time, my last code snippet creates new and > explicit RNG, which fixes the issue on Linux. > > The problem is that on some platforms even creating new generator doesn't > help, so I'm trying to understand the difference. > > @Tim, @Kristoffer, could you also specify CUDA version in use, please?
Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
7.5 here
Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
> But I can replicate the bug if I use CURAND. I'd suggest filing an issue > with > that package. Most likely it needs to (and, you need to) initialize > resources > according to these instructions: > https://github.com/JuliaGPU/CUDArt.jl#initializing-and-freeing-ptx-modules > > CURAND.jl is a thin wrapper around the corresponding C library and doesn't use any custom modules, so I suppose initializing resources is not applicable to this case. Tim, except for loading utility functions (PtxUtils), is there anything special in `device() ... do` block that I should pay attention to?
Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
See the code in src/device.jl that implements the do-block syntax (especially functions that take a function as their first argument). It's not a big file, so shouldn't be too hard to figure out what's happening. --Tim On Tuesday, November 10, 2015 02:16:52 PM Andrei Zh wrote: > > But I can replicate the bug if I use CURAND. I'd suggest filing an issue > > with > > that package. Most likely it needs to (and, you need to) initialize > > resources > > according to these instructions: > > https://github.com/JuliaGPU/CUDArt.jl#initializing-and-freeing-ptx-modules > > CURAND.jl is a thin wrapper around the corresponding C library and doesn't > use any custom modules, so I suppose initializing resources is not > applicable to this case. > > Tim, except for loading utility functions (PtxUtils), is there anything > special in `device() ... do` block that I should pay attention to?
Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
if I load: using CUDArt using CURAND and then I run either: d_a = curand(Float64, 1000); a = to_host(d_a); OR: result = devices(dev->capability(dev)[1]>=2) do devlist end I can repeat the block as many time I want. However if I run both combined (or alternatedly in any order): d_a = curand(Float64, 1000); a = to_host(d_a); result = devices(dev->capability(dev)[1]>=2) do devlist end I get the error: WARNING: CUDA error triggered from: LoadError: "unspecified launch failure" while loading In[3], in expression starting on line 2 in checkerror at C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\libcudart-6.5.jl:16 in checkerror at C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\libcudart-6.5.jl:15 in copy! at C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\arrays.jl:152 in to_host at C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\arrays.jl:87 in include_string at loading.jl:266 in execute_request_0x535c5df2 at C:\Users\joaquimgarcia\.julia\v0.4\IJulia\src\execute_request.jl:177 in eventloop at C:\Users\joaquimgarcia\.julia\v0.4\IJulia\src\IJulia.jl:141 in anonymous at task.jl:447 I will file this as an issue in CURAND
Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
Are you sure this is due to CUDArt? For me, this works as many times as I care to try it (note that I'm not using CURAND): using CUDArt result = devices(dev->capability(dev)[1]>=2; nmax = 1) do devlist device(devlist[1]) h = rand(Float32, 1000) d_a = CudaArray(h) a = to_host(d_a) end But I can replicate the bug if I use CURAND. I'd suggest filing an issue with that package. Most likely it needs to (and, you need to) initialize resources according to these instructions: https://github.com/JuliaGPU/CUDArt.jl#initializing-and-freeing-ptx-modules Best, --Tim On Saturday, November 07, 2015 02:52:16 PM Jason Eckstein wrote: > I've had the same errors on MacOS, Windows 10, and Linux using the > devices() do loop with CUDArt package. All of the programs I run I place > inside the outer do loop including all the repetitions because as soon as > that block ends I cannot use any other CUDA functions without some type of > pointer or memory error. The only solution I've found is to restart Julia, > but I noticed that if I run a program like yours with a julia command in > the terminal rather than a REPL session, even if I just run one loop and I > close the session I still get the memory errors when the script ends. > Perhaps someone involved in developing CUDArt can give some insight as to > why the outer do-block loop cannot be ended cleanly. > > On Friday, November 6, 2015 at 12:46:54 PM UTC-7, Joaquim Masset Lacombe > > Dias Garcia wrote: > > I was playing with the GPU (in both windows and mac) libraries and I came > > up with the following errors: > > The first time I execute the do loop, every thing goes well. > > But If i try a second time in a row I get this invalid pointer error. > > > > I got the same error in all my do loops, any ideas? > > > > best, > > Joaquim > > > > *My code running on terminal:* > > > > julia> using CUDArt > > julia> using CURAND > > julia> result = devices(dev->capability(dev)[1]>=2) do devlist > > > >d_a = curand(Float32, 1000); > >a = to_host(d_a); > > > >end > > > > 1000-element Array{Float64,1}: > > 0.438451 > > 0.460365 > > 0.250215 > > 0.494744 > > 0.0530111 > > 0.337699 > > 0.396763 > > 0.874419 > > 0.482167 > > 0.0428398 > > ? > > 0.563937 > > 0.80706 > > 0.190015 > > 0.334969 > > 0.622164 > > 0.710596 > > 0.0125895 > > 0.990388 > > 0.467796 > > 0.24313 > > > > julia> result = devices(dev->capability(dev)[1]>=2) do devlist > > > >d_a = curand(Float32, 1000); > >a = to_host(d_a); > > > >end > > > > WARNING: CUDA error triggered from: > > in checkerror at > > > > C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\libcudart-6.5.jl > > > > :15 > > : > > in copy! at C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\arrays.jl:152 > > in to_host at C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\arrays.jl:87 > > in anonymous at none:3 > > in devices at > > > > C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\device.jl:61ERROR: > > Launch failed, perhaps due to an invalid pointer > > > > in checkdrv at C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\module.jl:6 > > in close at C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\device.jl:136 > > in devices at C:\Users\joaquimgarcia\.julia\v0.4\CUDArt\src\device.jl:63
Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
Interesting. I have the bug when using cublas.jl I haven't found a way to initialize resources to avoid these errors but if I just use cudart on it's own there aren't memory errors.
Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
Interesting. I have the bug when using cublas.jl I haven't found a way to initialize resources to avoid these errors but if I just use cudart on it's own there aren't memory errors.
Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice
Interesting. I have the bug when using cublas.jl I haven't found a way to initialize resources to avoid these errors but if I just use cudart on it's own there aren't memory errors.