[petsc-users] Clarification of INSERT_VALUES for vec with ghost nodes
Hi, I have a vector with ghost nodes where each process may or may not change the value of a specific ghost node (using INSERT_VALUES). At the end I would like for each process, that see a particular ghost node, to have the smallest of the set values. I do not think there is a straightforward way to achieve this, but I would like to be wrong. Any suggestion? To build a work around I need to understand better the behavior of VecGhostUpdateBegin(...); VecGhostUpdateEnd(...). In particular in the documentation I do not see the option VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE); VecGhostUpdateEnd(v, INSERT_VALUES, SCATTER_REVERSE); In case this is possible to be used, what is the behavior of this call in the following two cases? 1) Assume that node-i belongs to proc0, and is ghosted in proc1 and proc2, also assume that the current value of node-i is value0 and proc0 does not modify it, but proc1 and proc2 do. start with: proc0 -> value0 proc1 -> value0 proc2 -> value0 change to: proc0 -> value0 proc1 -> value1 proc2 -> value2 I assume that calling VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE); VecGhostUpdateEnd(v, INSERT_VALUES, SCATTER_REVERSE); will have an unpredictable behavior as proc0 -> either value1 or value2 proc1 -> value1 proc2 -> value2 2) Assume now that node-i belongs to proc0, and is ghosted in proc1 and proc2, also assume that the current value of node-i is value0 and proc0 and proc1 do not modify it, but proc2 does. start with: proc0 -> value0 proc1 -> value0 proc2 -> value0 change to: proc0 -> value0 proc1 -> value0 proc2 -> value2 Is the call VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE); VecGhostUpdateEnd(v, INSERT_VALUES, SCATTER_REVERSE); still unpredictable? proc0 -> either value0 or value2 proc1 -> value0 proc2 -> value2 or proc0 -> value2 (since proc1 did not modify the original value, so it did not reverse scatter) proc1 -> value0 proc2 -> value2 Thanks a lot for your help Eugenio
Re: [petsc-users] Clarification of INSERT_VALUES for vec with ghost nodes
On Wed, Sep 25, 2019 at 9:11 AM Aulisa, Eugenio via petsc-users mailto:petsc-users@mcs.anl.gov>> wrote: Hi, I have a vector with ghost nodes where each process may or may not change the value of a specific ghost node (using INSERT_VALUES). At the end I would like for each process, that see a particular ghost node, to have the smallest of the set values. Do you mean owner of ghost nodes gets the smallest values. That is, in your example below, proc0 gets Min(value0, value1, value2)? I do not think there is a straightforward way to achieve this, but I would like to be wrong. Any suggestion? To build a work around I need to understand better the behavior of VecGhostUpdateBegin(...); VecGhostUpdateEnd(...). In particular in the documentation I do not see the option VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE); VecGhostUpdateEnd(v, INSERT_VALUES, SCATTER_REVERSE); In case this is possible to be used, what is the behavior of this call in the following two cases? 1) Assume that node-i belongs to proc0, and is ghosted in proc1 and proc2, also assume that the current value of node-i is value0 and proc0 does not modify it, but proc1 and proc2 do. start with: proc0 -> value0 proc1 -> value0 proc2 -> value0 change to: proc0 -> value0 proc1 -> value1 proc2 -> value2 I assume that calling VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE); VecGhostUpdateEnd(v, INSERT_VALUES, SCATTER_REVERSE); will have an unpredictable behavior as proc0 -> either value1 or value2 proc1 -> value1 proc2 -> value2 2) Assume now that node-i belongs to proc0, and is ghosted in proc1 and proc2, also assume that the current value of node-i is value0 and proc0 and proc1 do not modify it, but proc2 does. start with: proc0 -> value0 proc1 -> value0 proc2 -> value0 change to: proc0 -> value0 proc1 -> value0 proc2 -> value2 Is the call VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE); VecGhostUpdateEnd(v, INSERT_VALUES, SCATTER_REVERSE); still unpredictable? proc0 -> either value0 or value2 proc1 -> value0 proc2 -> value2 or proc0 -> value2 (since proc1 did not modify the original value, so it did not reverse scatter) proc1 -> value0 proc2 -> value2 Thanks a lot for your help Eugenio
Re: [petsc-users] Clarification of INSERT_VALUES for vec with ghost nodes
On Wed, Sep 25, 2019 at 9:11 AM Aulisa, Eugenio via petsc-users mailto:petsc-users@mcs.anl.gov>> wrote: Hi, I have a vector with ghost nodes where each process may or may not change the value of a specific ghost node (using INSERT_VALUES). At the end I would like for each process, that see a particular ghost node, to have the smallest of the set values. Do you mean owner of ghost nodes gets the smallest values. That is, in your example below, proc0 gets Min(value0, value1, value2)? If I can get the Min(value0, value1, value2) on the owner then I can scatter it forward with INSERT_VALUES to all processes that ghost it. And if there is a easy way to get Min(value0, value1, value2) on the owner (or on all processes) I would like to know. Since I do not think there is a straightforward way to achieve that, I was looking at a workaround, and to do that I need to know the behavior of scatter reverse in the cases described below. Notice that I used the option INSERT_VALUES which I am not even sure is allowed. I do not think there is a straightforward way to achieve this, but I would like to be wrong. Any suggestion? To build a work around I need to understand better the behavior of VecGhostUpdateBegin(...); VecGhostUpdateEnd(...). In particular in the documentation I do not see the option VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE); VecGhostUpdateEnd(v, INSERT_VALUES, SCATTER_REVERSE); In case this is possible to be used, what is the behavior of this call in the following two cases? 1) Assume that node-i belongs to proc0, and is ghosted in proc1 and proc2, also assume that the current value of node-i is value0 and proc0 does not modify it, but proc1 and proc2 do. start with: proc0 -> value0 proc1 -> value0 proc2 -> value0 change to: proc0 -> value0 proc1 -> value1 proc2 -> value2 I assume that calling VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE); VecGhostUpdateEnd(v, INSERT_VALUES, SCATTER_REVERSE); will have an unpredictable behavior as proc0 -> either value1 or value2 proc1 -> value1 proc2 -> value2 2) Assume now that node-i belongs to proc0, and is ghosted in proc1 and proc2, also assume that the current value of node-i is value0 and proc0 and proc1 do not modify it, but proc2 does. start with: proc0 -> value0 proc1 -> value0 proc2 -> value0 change to: proc0 -> value0 proc1 -> value0 proc2 -> value2 Is the call VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE); VecGhostUpdateEnd(v, INSERT_VALUES, SCATTER_REVERSE); still unpredictable? proc0 -> either value0 or value2 proc1 -> value0 proc2 -> value2 or proc0 -> value2 (since proc1 did not modify the original value, so it did not reverse scatter) proc1 -> value0 proc2 -> value2 Thanks a lot for your help Eugenio
Re: [petsc-users] Clarification of INSERT_VALUES for vec with ghost nodes
With VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE), the owner will get updated by ghost values. So in your case 1, proc0 gets either value1 or value2 from proc1/2; in case 2; proc0 gets either value0 or value2 from proc1/2. In short, you could not achieve your goal with INSERT_VALUES. Though you can do it with other interfaces in PETSc, e.g., PetscSFReduceBegin/End, I believe it is better to extend VecGhostUpdate to support MAX/MIN_VALUES, because it is a simpler interface for you and it is very easy to add. Could you try branch jczhang/feature-vscat-min-values to see if it works for you? See the end of src/vec/vec/examples/tutorials/ex9.c for an example of the new functionality. Use mpirun -n 2 ./ex9 -minvalues to test it and its expected output is output/ex9_2.out Petsc will have a new release this weekend. Let's see whether I can put it in the new release. Thanks. --Junchao Zhang On Thu, Sep 26, 2019 at 3:28 AM Aulisa, Eugenio mailto:eugenio.aul...@ttu.edu>> wrote: On Wed, Sep 25, 2019 at 9:11 AM Aulisa, Eugenio via petsc-users mailto:petsc-users@mcs.anl.gov>> wrote: Hi, I have a vector with ghost nodes where each process may or may not change the value of a specific ghost node (using INSERT_VALUES). At the end I would like for each process, that see a particular ghost node, to have the smallest of the set values. Do you mean owner of ghost nodes gets the smallest values. That is, in your example below, proc0 gets Min(value0, value1, value2)? If I can get the Min(value0, value1, value2) on the owner then I can scatter it forward with INSERT_VALUES to all processes that ghost it. And if there is a easy way to get Min(value0, value1, value2) on the owner (or on all processes) I would like to know. Since I do not think there is a straightforward way to achieve that, I was looking at a workaround, and to do that I need to know the behavior of scatter reverse in the cases described below. Notice that I used the option INSERT_VALUES which I am not even sure is allowed. I do not think there is a straightforward way to achieve this, but I would like to be wrong. Any suggestion? To build a work around I need to understand better the behavior of VecGhostUpdateBegin(...); VecGhostUpdateEnd(...). In particular in the documentation I do not see the option VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE); VecGhostUpdateEnd(v, INSERT_VALUES, SCATTER_REVERSE); In case this is possible to be used, what is the behavior of this call in the following two cases? 1) Assume that node-i belongs to proc0, and is ghosted in proc1 and proc2, also assume that the current value of node-i is value0 and proc0 does not modify it, but proc1 and proc2 do. start with: proc0 -> value0 proc1 -> value0 proc2 -> value0 change to: proc0 -> value0 proc1 -> value1 proc2 -> value2 I assume that calling VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE); VecGhostUpdateEnd(v, INSERT_VALUES, SCATTER_REVERSE); will have an unpredictable behavior as proc0 -> either value1 or value2 proc1 -> value1 proc2 -> value2 2) Assume now that node-i belongs to proc0, and is ghosted in proc1 and proc2, also assume that the current value of node-i is value0 and proc0 and proc1 do not modify it, but proc2 does. start with: proc0 -> value0 proc1 -> value0 proc2 -> value0 change to: proc0 -> value0 proc1 -> value0 proc2 -> value2 Is the call VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE); VecGhostUpdateEnd(v, INSERT_VALUES, SCATTER_REVERSE); still unpredictable? proc0 -> either value0 or value2 proc1 -> value0 proc2 -> value2 or proc0 -> value2 (since proc1 did not modify the original value, so it did not reverse scatter) proc1 -> value0 proc2 -> value2 Thanks a lot for your help Eugenio
Re: [petsc-users] Clarification of INSERT_VALUES for vec with ghost nodes
Yes it worked. I get exactly the same vector as reported in the output file I will test it now with my problem and if I see anything strange I will let you know. Thank you so much Eugenio Eugenio Aulisa Department of Mathematics and Statistics, Texas Tech University Lubbock TX, 79409-1042 room: 226 http://www.math.ttu.edu/~eaulisa/ phone: (806) 834-6684 fax: (806) 742-1112 From: Zhang, Junchao Sent: Thursday, September 26, 2019 11:02 AM To: Aulisa, Eugenio Cc: petsc-users@mcs.anl.gov Subject: Re: [petsc-users] Clarification of INSERT_VALUES for vec with ghost nodes With VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE), the owner will get updated by ghost values. So in your case 1, proc0 gets either value1 or value2 from proc1/2; in case 2; proc0 gets either value0 or value2 from proc1/2. In short, you could not achieve your goal with INSERT_VALUES. Though you can do it with other interfaces in PETSc, e.g., PetscSFReduceBegin/End, I believe it is better to extend VecGhostUpdate to support MAX/MIN_VALUES, because it is a simpler interface for you and it is very easy to add. Could you try branch jczhang/feature-vscat-min-values to see if it works for you? See the end of src/vec/vec/examples/tutorials/ex9.c for an example of the new functionality. Use mpirun -n 2 ./ex9 -minvalues to test it and its expected output is output/ex9_2.out Petsc will have a new release this weekend. Let's see whether I can put it in the new release. Thanks. --Junchao Zhang On Thu, Sep 26, 2019 at 3:28 AM Aulisa, Eugenio mailto:eugenio.aul...@ttu.edu>> wrote: On Wed, Sep 25, 2019 at 9:11 AM Aulisa, Eugenio via petsc-users mailto:petsc-users@mcs.anl.gov>> wrote: Hi, I have a vector with ghost nodes where each process may or may not change the value of a specific ghost node (using INSERT_VALUES). At the end I would like for each process, that see a particular ghost node, to have the smallest of the set values. Do you mean owner of ghost nodes gets the smallest values. That is, in your example below, proc0 gets Min(value0, value1, value2)? If I can get the Min(value0, value1, value2) on the owner then I can scatter it forward with INSERT_VALUES to all processes that ghost it. And if there is a easy way to get Min(value0, value1, value2) on the owner (or on all processes) I would like to know. Since I do not think there is a straightforward way to achieve that, I was looking at a workaround, and to do that I need to know the behavior of scatter reverse in the cases described below. Notice that I used the option INSERT_VALUES which I am not even sure is allowed. I do not think there is a straightforward way to achieve this, but I would like to be wrong. Any suggestion? To build a work around I need to understand better the behavior of VecGhostUpdateBegin(...); VecGhostUpdateEnd(...). In particular in the documentation I do not see the option VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE); VecGhostUpdateEnd(v, INSERT_VALUES, SCATTER_REVERSE); In case this is possible to be used, what is the behavior of this call in the following two cases? 1) Assume that node-i belongs to proc0, and is ghosted in proc1 and proc2, also assume that the current value of node-i is value0 and proc0 does not modify it, but proc1 and proc2 do. start with: proc0 -> value0 proc1 -> value0 proc2 -> value0 change to: proc0 -> value0 proc1 -> value1 proc2 -> value2 I assume that calling VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE); VecGhostUpdateEnd(v, INSERT_VALUES, SCATTER_REVERSE); will have an unpredictable behavior as proc0 -> either value1 or value2 proc1 -> value1 proc2 -> value2 2) Assume now that node-i belongs to proc0, and is ghosted in proc1 and proc2, also assume that the current value of node-i is value0 and proc0 and proc1 do not modify it, but proc2 does. start with: proc0 -> value0 proc1 -> value0 proc2 -> value0 change to: proc0 -> value0 proc1 -> value0 proc2 -> value2 Is the call VecGhostUpdateBegin(v, INSERT_VALUES, SCATTER_REVERSE); VecGhostUpdateEnd(v, INSERT_VALUES, SCATTER_REVERSE); still unpredictable? proc0 -> either value0 or value2 proc1 -> value0 proc2 -> value2 or proc0 -> value2 (since proc1 did not modify the original value, so it did not reverse scatter) proc1 -> value0 proc2 -> value2 Thanks a lot for your help Eugenio
