Re: [petsc-users] Ghost points in DMSwarm

2024-02-25 Thread Matthew Knepley 
On Sun, Feb 25, 2024 at 1:00 PM MIGUEL MOLINOS PEREZ  wrote:

> I see, so I'm the one in charge of the synchronization between the
> particles and the replicas associated, right?
>

Yes. We could conceivably put this in during our rewrite of DMSwarm, but it
would change the semantics. Right now, there is no particle identity.

  Thanks,

Matt


> Best,
> Miguel
>
> On Feb 25, 2024, at 6:53 PM, Matthew Knepley  wrote:
>
> On Sun, Feb 25, 2024 at 7:21 AM MIGUEL MOLINOS PEREZ 
> wrote:
>
>> Dear Matthew,
>>
>> Thank you for answer me out of office hours and for the suggestion. If I
>> interpret it correctly, your point is using the background mesh halo to
>> identify and create ghosts particles in the overlapping region using
>> DMSwarmMigrate().
>>
>> Then two questions hits me:
>>
>>- If I change a variable in the “real” particle (say the position),
>>it should be updated in the replicas on the overlapped regions, right? 
>> This
>>sort of communicator is included in DMSwarmMigrate()?
>>
>> No, the particles that are migrated are copies. There is no current
> analogue of DMGlobalToLocal(). It would need to be constructed. It is
> possible, but would take writing some code. All applications we have
> supported to date do not use replicated particles.
>
>>
>>- The function DMSwarmCreateLocalVectorFromField() does not include
>>extra space for ghost particles. Therefore, I should resize the local size
>>for each rank to accommodate the replicas manually. Right?
>>
>> It uses all particles on the process. If some of those are replicas, they
> will be included.
>
>   Thanks,
>
>  Matt
>
>
>> Anyway, I think this this will take me more time I expected originally.
>> Meanwhile,  I will go ahead with my “humble” parallelization scheme which
>> consist in using AllReduce to create a serial vectors with the variables I
>> need to evaluate rank-wise the residual equations of my problem  (please,
>> don’t judge me, publish or perish...)
>>
>> Many thanks,
>> Miguel
>>
>>
>> On Feb 24, 2024, at 4:24 PM, Matthew Knepley  wrote:
>>
>> On Fri, Feb 23, 2024 at 5:14 PM MIGUEL MOLINOS PEREZ 
>> wrote:
>>
>>> Dear all, I am struggling on how to include ghost points in DMSwarm
>>> local vectors. According to PETSc documentation it seems “straightforward”
>>> for a DMDA mesh. However, I am not so sure on how to do it for a DMSwarm.
>>> In fact, if I add the result
>>> ZjQcmQRYFpfptBannerStart
>>> This Message Is From an External Sender
>>> This message came from outside your organization.
>>>
>>> ZjQcmQRYFpfptBannerEnd
>>> Dear all,
>>>
>>> I am struggling on how to include ghost points in DMSwarm local vectors.
>>> According to PETSc documentation it seems “straightforward” for a DMDA
>>> mesh. However, I am not so sure on how to do it for a DMSwarm. In fact,
>>> if I add the result of DMSwarmGetLocalSize for each rank, the result is
>>> the exact number of particles in the system. Which means that the halo
>>> is zero.
>>>
>>> Since there is a function called DMSwarmCreateLocalVectorFromField,I
>>> was wandering if there is a function already implemented in petsc (and
>>> I’m missing it) to include ghost points in DMSwarm and therefore don’t
>>> have to reinvent the wheel. If so, is there any example out there I can
>>> follow?
>>>
>>
>> DMSwarm is currently different from the other DMs. There is no idea of
>> identity for particles, so there is no idea of a shared particle. You can
>> create this by adding an ID to your particle fields, but it would be a
>> manual process. DMSwarmMigrate() can duplicate particles if you have an
>> overlapping mesh, which would mimic shared particles.
>>
>>   THanks,
>>
>>  Matt
>>
>>
>>> Thanks in advance.
>>>
>>> Regards,
>>> Miguel
>>>
>>> petsc.org
>>> 
>>>
>>> 
>>> 
>>>
>>>
>>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eDmkxFmYvuRK7K4mCXY1o6YNM-qQM7ujn4OiPKemscUGOZ5aL9E_0BY0NgwxYeuNOJ8AZTj4c29sKXzs_I05$
>>  
>> >  >
>>
>>
>>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting

Re: [petsc-users] Ghost points in DMSwarm

2024-02-25 Thread MIGUEL MOLINOS PEREZ 
I see, so I'm the one in charge of the synchronization between the particles 
and the replicas associated, right?

Best,
Miguel

On Feb 25, 2024, at 6:53 PM, Matthew Knepley  wrote:

On Sun, Feb 25, 2024 at 7:21 AM MIGUEL MOLINOS PEREZ 
mailto:mmoli...@us.es>> wrote:
Dear Matthew,

Thank you for answer me out of office hours and for the suggestion. If I 
interpret it correctly, your point is using the background mesh halo to 
identify and create ghosts particles in the overlapping region using 
DMSwarmMigrate().

Then two questions hits me:

  *   If I change a variable in the “real” particle (say the position), it 
should be updated in the replicas on the overlapped regions, right? This sort 
of communicator is included in DMSwarmMigrate()?

No, the particles that are migrated are copies. There is no current analogue of 
DMGlobalToLocal(). It would need to be constructed. It is possible, but would 
take writing some code. All applications we have supported to date do not use 
replicated particles.

  *   The function DMSwarmCreateLocalVectorFromField() does not include extra 
space for ghost particles. Therefore, I should resize the local size for each 
rank to accommodate the replicas manually. Right?

It uses all particles on the process. If some of those are replicas, they will 
be included.

  Thanks,

 Matt

Anyway, I think this this will take me more time I expected originally. 
Meanwhile,  I will go ahead with my “humble” parallelization scheme which 
consist in using AllReduce to create a serial vectors with the variables I need 
to evaluate rank-wise the residual equations of my problem  (please, don’t 
judge me, publish or perish...)

Many thanks,
Miguel


On Feb 24, 2024, at 4:24 PM, Matthew Knepley 
mailto:knep...@gmail.com>> wrote:

On Fri, Feb 23, 2024 at 5:14 PM MIGUEL MOLINOS PEREZ 
mailto:mmoli...@us.es>> wrote:
Dear all, I am struggling on how to include ghost points in DMSwarm local 
vectors. According to PETSc documentation it seems “straightforward” for a DMDA 
mesh. However, I am not so sure on how to do it for a DMSwarm. In fact, if I 
add the result
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.

ZjQcmQRYFpfptBannerEnd
Dear all,

I am struggling on how to include ghost points in DMSwarm local vectors. 
According to PETSc documentation it seems “straightforward” for a DMDA mesh. 
However, I am not so sure on how to do it for a DMSwarm. In fact, if I add the 
result of DMSwarmGetLocalSize for each rank, the result is the exact number of 
particles in the system. Which means that the halo is zero.

Since there is a function called DMSwarmCreateLocalVectorFromField,I was 
wandering if there is a function already implemented in petsc (and I’m missing 
it) to include ghost points in DMSwarm and therefore don’t have to reinvent the 
wheel. If so, is there any example out there I can follow?

DMSwarm is currently different from the other DMs. There is no idea of identity 
for particles, so there is no idea of a shared particle. You can create this by 
adding an ID to your particle fields, but it would be a manual process. 
DMSwarmMigrate() can duplicate particles if you have an overlapping mesh, which 
would mimic shared particles.

  THanks,

 Matt

Thanks in advance.

Regards,
Miguel


petsc.org
[X]




--
What most experimenters take for granted before they begin their experiments is 
infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!dxinoV9sMfUwzVekCnVvZ-35vwYSpmBeJrsrizWNZxVg6Q_Vsoi_-fVMaxO8IdGAV8Jlgec_1ZeY42E-Phe_sQ$
 




--
What most experimenters take for granted before they begin their experiments is 
infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!dxinoV9sMfUwzVekCnVvZ-35vwYSpmBeJrsrizWNZxVg6Q_Vsoi_-fVMaxO8IdGAV8Jlgec_1ZeY42E-Phe_sQ$

Re: [petsc-users] Ghost points in DMSwarm

2024-02-25 Thread Matthew Knepley 
On Sun, Feb 25, 2024 at 7:21 AM MIGUEL MOLINOS PEREZ  wrote:

> Dear Matthew,
>
> Thank you for answer me out of office hours and for the suggestion. If I
> interpret it correctly, your point is using the background mesh halo to
> identify and create ghosts particles in the overlapping region using
> DMSwarmMigrate().
>
> Then two questions hits me:
>
>- If I change a variable in the “real” particle (say the position), it
>should be updated in the replicas on the overlapped regions, right? This
>sort of communicator is included in DMSwarmMigrate()?
>
> No, the particles that are migrated are copies. There is no current
analogue of DMGlobalToLocal(). It would need to be constructed. It is
possible, but would take writing some code. All applications we have
supported to date do not use replicated particles.

>
>- The function DMSwarmCreateLocalVectorFromField() does not include
>extra space for ghost particles. Therefore, I should resize the local size
>for each rank to accommodate the replicas manually. Right?
>
> It uses all particles on the process. If some of those are replicas, they
will be included.

  Thanks,

 Matt


> Anyway, I think this this will take me more time I expected originally.
> Meanwhile,  I will go ahead with my “humble” parallelization scheme which
> consist in using AllReduce to create a serial vectors with the variables I
> need to evaluate rank-wise the residual equations of my problem  (please,
> don’t judge me, publish or perish...)
>
> Many thanks,
> Miguel
>
>
> On Feb 24, 2024, at 4:24 PM, Matthew Knepley  wrote:
>
> On Fri, Feb 23, 2024 at 5:14 PM MIGUEL MOLINOS PEREZ 
> wrote:
>
>> Dear all, I am struggling on how to include ghost points in DMSwarm local
>> vectors. According to PETSc documentation it seems “straightforward” for a
>> DMDA mesh. However, I am not so sure on how to do it for a DMSwarm. In
>> fact, if I add the result
>> ZjQcmQRYFpfptBannerStart
>> This Message Is From an External Sender
>> This message came from outside your organization.
>>
>> ZjQcmQRYFpfptBannerEnd
>> Dear all,
>>
>> I am struggling on how to include ghost points in DMSwarm local vectors.
>> According to PETSc documentation it seems “straightforward” for a DMDA
>> mesh. However, I am not so sure on how to do it for a DMSwarm. In fact,
>> if I add the result of DMSwarmGetLocalSize for each rank, the result is
>> the exact number of particles in the system. Which means that the halo
>> is zero.
>>
>> Since there is a function called DMSwarmCreateLocalVectorFromField,I was
>> wandering if there is a function already implemented in petsc (and I’m
>> missing it) to include ghost points in DMSwarm and therefore don’t have
>> to reinvent the wheel. If so, is there any example out there I can follow?
>>
>
> DMSwarm is currently different from the other DMs. There is no idea of
> identity for particles, so there is no idea of a shared particle. You can
> create this by adding an ID to your particle fields, but it would be a
> manual process. DMSwarmMigrate() can duplicate particles if you have an
> overlapping mesh, which would mimic shared particles.
>
>   THanks,
>
>  Matt
>
>
>> Thanks in advance.
>>
>> Regards,
>> Miguel
>>
>> petsc.org
>> 
>>
>> 
>> 
>>
>>
>>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!bRyZjT28s2cAZIFj69F3Bmv2X5NDZvT-prHmzArFuDq6aqqCaDpfaHH0ip-K53DfXi574ejpx04DurcseuAb$
>  
>   >
>
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!bRyZjT28s2cAZIFj69F3Bmv2X5NDZvT-prHmzArFuDq6aqqCaDpfaHH0ip-K53DfXi574ejpx04DurcseuAb$

Re: [petsc-users] Ghost points in DMSwarm

2024-02-25 Thread MIGUEL MOLINOS PEREZ 
Dear Matthew,

Thank you for answer me out of office hours and for the suggestion. If I 
interpret it correctly, your point is using the background mesh halo to 
identify and create ghosts particles in the overlapping region using 
DMSwarmMigrate().

Then two questions hits me:

  *   If I change a variable in the “real” particle (say the position), it 
should be updated in the replicas on the overlapped regions, right? This sort 
of communicator is included in DMSwarmMigrate()?
  *   The function DMSwarmCreateLocalVectorFromField() does not include extra 
space for ghost particles. Therefore, I should resize the local size for each 
rank to accommodate the replicas manually. Right?

Anyway, I think this this will take me more time I expected originally. 
Meanwhile,  I will go ahead with my “humble” parallelization scheme which 
consist in using AllReduce to create a serial vectors with the variables I need 
to evaluate rank-wise the residual equations of my problem  (please, don’t 
judge me, publish or perish...)

Many thanks,
Miguel


On Feb 24, 2024, at 4:24 PM, Matthew Knepley  wrote:

On Fri, Feb 23, 2024 at 5:14 PM MIGUEL MOLINOS PEREZ 
mailto:mmoli...@us.es>> wrote:
Dear all, I am struggling on how to include ghost points in DMSwarm local 
vectors. According to PETSc documentation it seems “straightforward” for a DMDA 
mesh. However, I am not so sure on how to do it for a DMSwarm. In fact, if I 
add the result
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.

ZjQcmQRYFpfptBannerEnd
Dear all,

I am struggling on how to include ghost points in DMSwarm local vectors. 
According to PETSc documentation it seems “straightforward” for a DMDA mesh. 
However, I am not so sure on how to do it for a DMSwarm. In fact, if I add the 
result of DMSwarmGetLocalSize for each rank, the result is the exact number of 
particles in the system. Which means that the halo is zero.

Since there is a function called DMSwarmCreateLocalVectorFromField,I was 
wandering if there is a function already implemented in petsc (and I’m missing 
it) to include ghost points in DMSwarm and therefore don’t have to reinvent the 
wheel. If so, is there any example out there I can follow?

DMSwarm is currently different from the other DMs. There is no idea of identity 
for particles, so there is no idea of a shared particle. You can create this by 
adding an ID to your particle fields, but it would be a manual process. 
DMSwarmMigrate() can duplicate particles if you have an overlapping mesh, which 
would mimic shared particles.

  THanks,

 Matt

Thanks in advance.

Regards,
Miguel


petsc.org
[X]




--
What most experimenters take for granted before they begin their experiments is 
infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!cQcTxz-cUaVBpc8qjPR-ggaw3NXqq0-rf5v5xtsKzEmm4uvOkRcw4LffQ4DSJX8Py_hU0auibW7oRimePeKAZA$

Re: [petsc-users] Ghost points in DMSwarm

2024-02-24 Thread Matthew Knepley 
On Fri, Feb 23, 2024 at 5:14 PM MIGUEL MOLINOS PEREZ  wrote:

> Dear all, I am struggling on how to include ghost points in DMSwarm local
> vectors. According to PETSc documentation it seems “straightforward” for a
> DMDA mesh. However, I am not so sure on how to do it for a DMSwarm. In
> fact, if I add the result
> ZjQcmQRYFpfptBannerStart
> This Message Is From an External Sender
> This message came from outside your organization.
>
> ZjQcmQRYFpfptBannerEnd
> Dear all,
>
> I am struggling on how to include ghost points in DMSwarm local vectors.
> According to PETSc documentation it seems “straightforward” for a DMDA
> mesh. However, I am not so sure on how to do it for a DMSwarm. In fact,
> if I add the result of DMSwarmGetLocalSize for each rank, the result is
> the exact number of particles in the system. Which means that the halo is
> zero.
>
> Since there is a function called DMSwarmCreateLocalVectorFromField,I was
> wandering if there is a function already implemented in petsc (and I’m
> missing it) to include ghost points in DMSwarm and therefore don’t have
> to reinvent the wheel. If so, is there any example out there I can follow?
>

DMSwarm is currently different from the other DMs. There is no idea of
identity for particles, so there is no idea of a shared particle. You can
create this by adding an ID to your particle fields, but it would be a
manual process. DMSwarmMigrate() can duplicate particles if you have an
overlapping mesh, which would mimic shared particles.

  THanks,

 Matt


> Thanks in advance.
>
> Regards,
> Miguel
>
> petsc.org
> 
>
> 
> 
>
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!aLpar0ltcRArMpZIgCFcyKDnezb9G3tPzvVdBNtiSHZD2-37Vdmfl-Kp4YrSLNP7wO3gZjhPcXCq2BitoRRm$