Hi Barry, I'm sorry, I should have provided more details.
Matrix A defined in my problem is of size NxN and I have N vectors v_i, each of length N, which are to be multiplied by A. N is fairly small 10000-50000. Ideally speaking, it doesn't require parallel storage. But I will soon exhaust memory by storing both A and v_i's sequentially. So I planned to implement it through a sub-communiator as this Mat-Vec(s) operation(s) is part of a bigger problem which requires a greater degree of parallelization. My ghost points are physical grid points, FEM nodal points to be exact. So I have a list of nodes whose data is required on a given processor to perform evaluation of any physical quantity. It is through this list that I decide the ghost points (I check if the global ids of these nodes are within the ownership range for the PETSc partioning or not). Now depending upon the number of processors I put into my sub-communicator, the percentage of non-local data I'll require will range from 5-10%. I hope I'm able to put forth my problem properly and adequately. Thanks, Bikash On Oct 16, 2015 11:49 AM, "Barry Smith" <[email protected]> wrote: > > > On Oct 16, 2015, at 7:52 AM, Bikash Kanungo <[email protected]> wrote: > > > > Hi, > > > > I need to multiply a matrix A with several vectors v_i's and use the > resultant vectors in my calculation which involves access to non-local > nodes. > > What percentage of the "non-local" results do you need in "your > calculation", a few percent, all of them? How do you know which "non-local" > results you will need for "your calculation"? Are they ghost points on a > grid? Something else? > > What is "several vectors"? 5? 100? 1000? > > Barry > > > I can think of two possible ways to do it: > > > > 1. Store the vectors v_i's into an MPIDENSE matrix and perform > MatMatMult with A to get a resultant dense matrix B. I can now use > MatGetSubMatrices on B to access the non-local part required in my > subsequent calculations. > > > > 2. Use MatMult to evaluate A*v_i for each of the vectors and provide > ghost padding through VecCreateGhost for the resultant vector. > > > > Which of the two is a more efficient way to perform the task? Also, if > there is a better way possible to achieve the same, then kindly let me know. > > > > Thanks, > > Bikash > > > > -- > > Bikash S. Kanungo > > PhD Student > > Computational Materials Physics Group > > Mechanical Engineering > > University of Michigan > > > >
