Re: [petsc-dev] cuda failures of tests in master

2015-08-10 Thread Karl Rupp
Hi Dominic, > With the current implementation the following can happen (v is of type VECCUSP): - Originally data on GPU, v.valid_GPU_array == PETSC_CUSP_GPU - a call to VecPlaceArray(v, arr) unplaces the data on the host and sets v.valid_CPU_array=CPU. Note that the GPU data does not get stashed

Re: [petsc-dev] cuda failures of tests in master

2015-08-08 Thread Dominic Meiser
On 08/08/2015 12:13 PM, Barry Smith wrote: Understood. I found this in the manual page for BJACOBI :-) Developer Notes: This preconditioner does not currently work with CUDA/CUSP for a couple of reasons. (1) It creates seq vectors as work vectors that should be cusp (

Re: [petsc-dev] cuda failures of tests in master

2015-08-08 Thread Barry Smith
Understood. I found this in the manual page for BJACOBI :-) Developer Notes: This preconditioner does not currently work with CUDA/CUSP for a couple of reasons. (1) It creates seq vectors as work vectors that should be cusp (2) The use of VecPlaceArray() is not handled proper

Re: [petsc-dev] cuda failures of tests in master

2015-08-08 Thread Dominic Meiser
With the current implementation the following can happen (v is of type VECCUSP): - Originally data on GPU, v.valid_GPU_array == PETSC_CUSP_GPU - a call to VecPlaceArray(v, arr) unplaces the data on the host and sets v.valid_CPU_array=CPU. Note that the GPU data does not get stashed. - subsequent

Re: [petsc-dev] cuda failures of tests in master

2015-08-07 Thread Barry Smith
> On Aug 7, 2015, at 7:50 PM, Dominic Meiser wrote: > > FYI I've opened a pull request that addresses this issue. > > While going through the code I ran into a question regarding the semantics of > VecPlaceArray and VecResetArray: Is the contents of the "unplaced" array > supposed to be prese

Re: [petsc-dev] cuda failures of tests in master

2015-08-07 Thread Dominic Meiser
FYI I've opened a pull request that addresses this issue. While going through the code I ran into a question regarding the semantics of VecPlaceArray and VecResetArray: Is the contents of the "unplaced" array supposed to be preserved so that the vector is completely restored upon calling VecRe

Re: [petsc-dev] cuda failures of tests in master

2015-08-07 Thread Dominic Meiser
Thanks for pointing this out. I'll have a look. Dominic On 08/07/2015 02:42 PM, Barry Smith wrote: Guardians of CUDA/GPUs http://ftp.mcs.anl.gov/pub/petsc/nightlylogs/archive/2015/08/06/examples_master_arch-cuda-double_bb-proxy.log http://ftp.mcs.anl.gov/pub/petsc/nightlylogs/archive/2015/0

[petsc-dev] cuda failures of tests in master

2015-08-07 Thread Barry Smith
Guardians of CUDA/GPUs http://ftp.mcs.anl.gov/pub/petsc/nightlylogs/archive/2015/08/06/examples_master_arch-cuda-double_bb-proxy.log http://ftp.mcs.anl.gov/pub/petsc/nightlylogs/archive/2015/08/06/examples_master_arch-cuda_bb-proxy.log search for ex2_bjacobi note that this example does not fa