> El 25 feb 2016, a las 17:19, Dominic Meiser <dmei...@txcorp.com> escribió: > > On Thu, Feb 25, 2016 at 01:13:01PM +0100, Jose E. Roman wrote: >> We are trying to do some GPU developments on the SLEPc side, and we would >> need a way of placing the array of a VECCUSP vector, providing the GPU >> address. Specifically, what we want to do is have a large Vec on GPU and >> slice it in several smaller Vecs. >> >> For the GetArray/RestoreArray we have all possibilities: >> - VecGetArray: gets the pointer to the buffer stored in CPU memory >> - VecCUSPGetArray*: returns a CUSPARRAY object that contains some info, >> including the buffer allocated in GPU memory >> - VecCUSPGetCUDAArray*: returns a raw pointer of the GPU buffer >> >> The problem comes with PlaceArray equivalents. Using VecPlaceArray we can >> provide a new pointer to CPU memory. We wanted to implement the equivalent >> thing for GPU, but we found difficulties due to Thrust. If we wanted to >> provide a VecCUSPPlaceCUDAArray the problem is that Thrust does not allow >> wrapping an exisiting GPU buffer with a CUSPARRAY object (when creating a >> CUSPARRAY it always allocates new memory). On the other hand, >> VecCUSPPlaceArray is possible to implement, but the problem is that one >> should provide a CUSPARRAY obtained from a VecCUSPGetArray* without >> modification (it is not possible to do pointer arithmetic with a CUSPARRAY). >> >> Any thoughts? >> > > I think your and Karli's analysis is correct, this is currently > not supported. Besides Karli's proposal to use ViennaCL's cuda > backend a different option might be to use cusp's array views. > These have a constructor for sub-ranges of other cusp arrays: > > https://github.com/cusplibrary/cusplibrary/blob/master/cusp/array1d.h#L409 > > However, enabling cusp array views in something like > VecCUSPPlaceArray is not immediately possible. The CUSPARRAY > type, which is currently hardwired to be > cusp::array1d<PetscScalar,cusp::device_memory>, would have to > become a template parameter. I'm not sure if we want to go down > that path.
Yes, we do not like this. > > The alternative would be to use raw cuda pointers instead of cusp > arrays for GPU memory in VecCUSP. That would be a fairly > significant undertaking (certainly more than the 2-3 weeks Karli > is estimating for getting the ViennaCL cuda backend in). Do you mean creating a new class VECCUDA in addition to VECCUSP and VECVIENNACL? This could be a solution for us. It would mean maybe refactoring MATAIJCUSPARSE to work with these new Vecs? If there is interest we can help in adding this stuff. > > Cheers, > Dominic > > -- > Dominic Meiser > Tech-X Corporation - 5621 Arapahoe Avenue - Boulder, CO 80303
