Okay, thanks. I'll look into it. Dmitry. On Mon, May 21, 2012 at 2:11 PM, Hui Zhang <mike.hui.zhang at hotmail.com>wrote:
> Sorry, I uses PCASMSetSortIndices for GASM. Now, I correct it and the > output becomes the > same as PCASM. See attached. > > > > > On May 21, 2012, at 8:59 PM, Hui Zhang wrote: > > Dmitry, > > finally, I am able to make a simple program and reproduce the error. Since > I had moved to > petsc-dev (before I used petsc-3.2), the error is a little different. > > Download the program attached and run the run.sh. You will see the output > from PCASM. > There is a submatrix (submat[1] on rank 0) which has a row like > > row 1: (0, -1) (1, 0 + 0.4 i) (4, -1) (2, -1) > > Note that the column 2 is placed in the last and the column numbers are > not in the normal order. > Following that, we can also see the error message like > > [0]PETSC ERROR: New nonzero at (1,2) caused a malloc! > > You can also try with GASM, use > > mpirun -np 2 ./gasm_test -dd_type gasm -n 4 -px 1 -py 2 -sx 2 -sy 1 > > which partitions the domain to 1 * 2 and mapped to the two processors, and > for each processor > we further partition its local domain to 2 * 1 subdomains. > > Then, you will see that the row 1 of submat[1] on rank 1 becomes > > row 1: (0, -1) (1, 0 + 0.4 i) (3, -1) (6, -1) > > which is totally wrong because from the IS for this overlapping subdomain > and the big matrix A, > row 1 should have non-zeros like the output from PCASM. > > I guess the problem is due to that we set PCASMSetSortIndices to FALSE and > something goes wrong > in asm.c and gasm.c. > > Thanks! > Hui > > > <GASM_test.zip> > > > > > > On Sun, May 20, 2012 at 5:06 AM, Hui Zhang <mike.hui.zhang at > hotmail.com>wrote: > >> I have the following MatView output: >> >> row 0: (0, -0.0351719 - 0.638814 i) (1, -0.517586 - 0.617823 i) (4, >> -0.517586 - 0.617823 i) (1, -1.03517) (5, -0.508793) >> >> Note that there are two values for the column 1, is this normal? >> >> The above matrix is obtained from PCASMGetSubKSP(pc,subksp) and >> KSPGetOperators(subksp[0], ...). >> > > This is odd. Can you explain a bit more what leads up to this, so we can > try and reproduce the problem? > Thanks. > Dmitry. > >> >> Thanks! >> >> >> >> On May 11, 2012, at 7:17 PM, Dmitry Karpeev wrote: >> >> You can call PCSetUp(pc) on either ASM or GASM, and that will destroy and >> recreate the matrices (including calling >> your modification subroutine), but not the subdomains or the subdomain >> solvers. >> If you just want to modify the submatrices, you can call >> PC(G)ASMGetSubmatrices() and modify the matrices it returns >> (in the same order as the subdomains were set). That's a bit of a >> hack, since you will essentially be modifying the PC's internal data >> structures. As long as you are careful, you should be okay, since you >> already effectively have the same type of access to the submatrices through >> the Modify callback. >> >> Dmitry. >> >> On Fri, May 11, 2012 at 11:52 AM, Hui Zhang <mike.hui.zhang at >> hotmail.com>wrote: >> >>> I just have a question about reuse of PCASM or PCGASM. >>> Suppose I have seted up the PCASM and related KSP and I solved one time. >>> Next for the same linear system (matrix and RHS), I just want PCASM >>> modify the submatrices (PCSetModifySubmatrices) in a different way, using >>> the same routine for modifying but with >>> different user context for the modifying routine. >>> >>> What can I do for this task? Currently, I destroy the KSP and >>> re-construct it. I guess >>> even for PCASM I can re-use it because the partition of subdomains >>> remain the same. >>> >>> Thanks! >>> >>> >>> On May 10, 2012, at 6:37 PM, Dmitry Karpeev wrote: >>> >>> Hui, >>> There've been several changes to PCGASM ahead of the new release. >>> Let me go back and see if it affected the convergence problem. >>> Dmitry. >>> >>> On Thu, May 10, 2012 at 4:16 AM, Hui Zhang <mike.hui.zhang at >>> hotmail.com>wrote: >>> >>>> Hi Dmitry, >>>> >>>> is there any news about PCGASM? >>>> >>>> thanks, >>>> Hui >>>> >>>> On Feb 20, 2012, at 6:38 PM, Dmitry Karpeev wrote: >>>> >>>> Okay, thanks. >>>> I'll take a look. >>>> >>>> Dmitry. >>>> >>>> On Mon, Feb 20, 2012 at 11:30 AM, Hui Zhang <mike.hui.zhang at hotmail.com >>>> > wrote: >>>> >>>>> For reference, my results are attached. >>>>> >>>>> asm1.txt for asm with 1 process, >>>>> asm2.txt for asm with 2 processes, >>>>> gasm1.txt for gasm with 1 process, (with the iteration numbers >>>>> different from others) >>>>> gasm2.txt for gasm with 2 processes >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> thank you, >>>>> Hui >>>>> >>>>> On Feb 20, 2012, at 3:06 PM, Dmitry Karpeev wrote: >>>>> >>>>> >>>>> >>>>> On Mon, Feb 20, 2012 at 12:59 AM, Hui Zhang < >>>>> mike.hui.zhang at hotmail.com> wrote: >>>>> >>>>>> >>>>>> On Feb 20, 2012, at 12:41 AM, Dmitry Karpeev wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Sun, Feb 19, 2012 at 3:08 PM, Hui Zhang < >>>>>> mike.hui.zhang at hotmail.com> wrote: >>>>>> >>>>>>> I have a new problem: the results from ASM and GASM are different >>>>>>> and it seems >>>>>>> GASM has something wrong with SetModifySubMatrices. Numerical tests >>>>>>> are with >>>>>>> each subdomain supported only by one subdomain. There are no >>>>>>> problems when >>>>>>> I did not modify submatrices. But when I modify submatrices, there >>>>>>> are problems >>>>>>> with GASM but no problems with ASM. >>>>>>> >>>>>>> For example, I use two subdomains. In the first case each subdomain >>>>>>> is supported by >>>>>>> one processor and there seems no problem with GASM. But when I use >>>>>>> run my program >>>>>>> with only one proc. so that it supports both of the two subdomains, >>>>>>> the iteration >>>>>>> number is different from the first case and is much larger. On the >>>>>>> other hand >>>>>>> ASM has no such problem. >>>>>>> >>>>>> >>>>>> Are the solutions the same? >>>>>> What problem are you solving? >>>>>> >>>>>> >>>>>> Yes, the solutions are the same. That's why ASM gives the same >>>>>> results with one or >>>>>> two processors. But GASM did not. >>>>>> >>>>> Sorry, I wasn't clear: ASM and GASM produced different solutions in >>>>> the case of two domains per processor? >>>>> >>>>>> I'm solving the Helmholtz equation. Maybe >>>>>> I can prepare a simpler example to show this difference. >>>>>> >>>>> That would be helpful. >>>>> Thanks. >>>>> >>>>> Dmitry. >>>>> >>>>>> >>>>>> >>>>>> Dmitry. >>>>>> >>>>>>> >>>>>>> >>>>>>> On Feb 15, 2012, at 6:46 PM, Dmitry Karpeev wrote: >>>>>>> >>>>>>> You should be able to. >>>>>>> This behavior is the same as in PCASM, >>>>>>> except in GASM the matrices live on subcommunicators. >>>>>>> I am in transit right now, but I can take a closer look in Friday. >>>>>>> >>>>>>> Dmitry >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Feb 15, 2012, at 8:07, Hui Zhang <mike.hui.zhang at hotmail.com> >>>>>>> wrote: >>>>>>> >>>>>>> On Feb 15, 2012, at 11:19 AM, Hui Zhang wrote: >>>>>>> >>>>>>> Hi Dmitry, >>>>>>> >>>>>>> thanks a lot! Currently, I'm not using ISColoring. Just comes >>>>>>> another question >>>>>>> on PCGASMSetModifySubMatrices(). The user provided function has the >>>>>>> prototype >>>>>>> >>>>>>> func (PC pc,PetscInt nsub,IS *row,IS *col,Mat *submat,void *ctx); >>>>>>> >>>>>>> I think the coloumns from the parameter 'col' are always the same as >>>>>>> the rows >>>>>>> from the parameter 'row'. Because PCGASMSetLocalSubdomains() only >>>>>>> accepts >>>>>>> index sets but not rows and columns. Has I misunderstood something? >>>>>>> >>>>>>> >>>>>>> As I tested, the row and col are always the same. >>>>>>> >>>>>>> I have a new question. Am I allowed to SetLocalToGlobalMapping() for >>>>>>> the submat's >>>>>>> in the above func()? >>>>>>> >>>>>>> thanks, >>>>>>> Hui >>>>>>> >>>>>>> >>>>>>> thanks, >>>>>>> Hui >>>>>>> >>>>>>> >>>>>>> On Feb 11, 2012, at 3:36 PM, Dmitry Karpeev wrote: >>>>>>> >>>>>>> Yes, that's right. >>>>>>> There is no good way to help the user assemble the subdomains at the >>>>>>> moment beyond the 2D stuff. >>>>>>> It is expected that they are generated from mesh subdomains. >>>>>>> Each IS does carry the subdomains subcomm. >>>>>>> >>>>>>> There is ISColoringToList() that is supposed to convert a "coloring" >>>>>>> of indices to an array of ISs, >>>>>>> each having the indices with the same color and the subcomm that >>>>>>> supports that color. It is >>>>>>> largely untested, though. You could try using it and give us >>>>>>> feedback on any problems you encounter. >>>>>>> >>>>>>> Dmitry. >>>>>>> >>>>>>> >>>>>>> On Sat, Feb 11, 2012 at 6:06 AM, Hui Zhang <<mike.hui.zhang at >>>>>>> hotmail.com> >>>>>>> mike.hui.zhang at hotmail.com> wrote: >>>>>>> >>>>>>>> About PCGASMSetLocalSubdomains(), in the case of one subdomain >>>>>>>> supported by >>>>>>>> multiple processors, shall I always create the arguments 'is[s]' >>>>>>>> and 'is_local[s]' >>>>>>>> in a subcommunicator consisting of processors supporting the >>>>>>>> subdomain 's'? >>>>>>>> >>>>>>>> The source code of PCGASMCreateSubdomains2D() seemingly does so. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Hui >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>> >>>> >>> >>> >> >> > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120521/7b569045/attachment-0001.html>
