Re: [petsc-users] difference between local and global vectors

[TAY wee-beng]

On 19/8/2015 1:17 PM, Dave May wrote:


On 19 August 2015 at 03:38, TAY wee-beng <zon...@gmail.com 
<mailto:zon...@gmail.com>> wrote:

    Hi,

    I am using DA. For e.g.

    DM da_u

    call
    
DMDACreate3d(MPI_COMM_WORLD,DM_BOUNDARY_NONE,DM_BOUNDARY_NONE,DM_BOUNDARY_NONE,DMDA_STENCIL_STAR,size_x,size_y,&

    
size_z,1,PETSC_DECIDE,PETSC_DECIDE,1,stencil_width,lx,PETSC_NULL_INTEGER,PETSC_NULL_INTEGER,da_u,ierr)

    call DMCreateGlobalVector(da_u,u_global,ierr)

    call DMCreateLocalVector(da_u,u_local,ierr)

    To update the ghost values, I use:

    call DMLocalToLocalBegin(da_u,u_local,INSERT_VALUES,u_local,ierr)

    call DMLocalToLocalEnd(da_u,u_local,INSERT_VALUES,u_local,ierr)



This is incorrect.
The manpage for DMLocalToLocal clearly says "Maps from a local vector 
(including ghost points that contain irrelevant values) to another 
local vector where the ghost points in the second are set correctly."
To update ghost values from a global vector (e.g. to perform the 
scatter) you need to use DMGlobalToLocalBegin() , DMGlobalToLocalEnd().

Hi Dave,

Thanks for the clarification although I'm still confused. Supposed I 
have a 1D vector da_u, It has size 8, so it's like da_u_array(8), with 
stencil width 1

So for 2 procs,

there will be 2 da_u_array - da_u_array(1:5) and da_u_array(4:8)

After performing some operations on each procs's da_u_array, I need to 
update 1st procs's da_u_array(5) and 2nd procs's da_u_array(4) from the 
2nd and 1st procs respectively. I simply call:

call DMLocalToLocalBegin(da_u,u_local,INSERT_VALUES,u_local,ierr)

call DMLocalToLocalEnd(da_u,u_local,INSERT_VALUES,u_local,ierr)

and it seems to be enough. I check the ghost values and they have been 
updated. So if I am not using the linear solvers, I do not need the 
global vector,is that so?



    It seems that I don't need to use global vector at all.

    So what's the difference between local and global vector?



* Local vectors contain ghost values from any neighbouring MPI 
processes. They are always defined over PETSC_COMM_SELF.
* Global vectors store the DOFs assigned to each sub-domain. These 
will parallel vectors defined over the same communicator as your DM

Thus, you use local vectors to compute things like the sub-domain 
contribution to (i) a non-linear residual evaluation or (ii) a 
sparse-matric vector product.
You use global vectors together with linear and non-linear solvers as 
these vectors.

If your stencil width was zero (in your DMDACreate3d() function call), 
then the would be no ghost values to communicate between neighbouring 
MPI processes. Hence, the entries in the following two arrays 
LA_u_local[], LA_u[] would be identical
  VecGetArrayRead(u_local,&LA_u_local);
and
  VecGetArrayRead(u,&LA_u);

That said, u_local would still be of type VECSEQ, where as u would be 
of type VECMPI.



    When will I need to use?:

    call DMGlobalToLocalBegin(da_u,u_global,INSERT_VALUES,u_local,ierr)

    call DMGlobalToLocalEnd(da_u,u_global,INSERT_VALUES,u_local,ierr)


See points (i) and (ii) above from common use cases.

Thanks,
  Dave




    -- 
    Thank you

    Yours sincerely,

    TAY wee-beng