Dmitry, thanks! For the moment, I can use the sorted IS and I just need to pass through user context some AO or ISLocalToGlobalMapping.
Hope you will get the unsorted IS case work soon! Thanks! On May 22, 2012, at 8:04 PM, Dmitry Karpeev wrote: > Hui, > There appears to be a bug in MatGetSubMatrices() which inserts submatrix > column indices in the same order > as they appeared in the original matrix. If a column IS has an inversion -- > IS[2] = 8 and IS[6] = 4, > as in your case -- then the submatrix column index 2 will appear after the > submatrix column index 4, since that's > how they were ordered in the original matrix (4 < 8, and the column indices > were properly sorted). This violates > the ordering of column indices, which prevents binary search from finding > column index 2. That results in an > apparent new nonzero insertion, which is in turn caught, since by default no > new nonzeros are allowed. > > I'm looking into this, and I'm not sure that there is an easy workaround for > you at this time: if you sort the IS indices, > it will violate the implicit local-to-global submatrix mapping you are using > in your ModifySubMatrices(). > > Thanks. > Dmitry. > > On Mon, May 21, 2012 at 2:33 PM, Hui Zhang <mike.hui.zhang at hotmail.com> > wrote: > Hi, > > It might be unclear. The error here is that the submat[1] does have a (1,2) > entry, > but MatSetValues at (1,2) gives an error message. GASM and ASM give the same > error. > > Thanks! > > On May 21, 2012, at 9:11 PM, Hui Zhang wrote: > >> Sorry, I uses PCASMSetSortIndices for GASM. Now, I correct it and the output >> becomes the >> same as PCASM. See attached. >> >> <GASM_test.zip> >> >> >> On May 21, 2012, at 8:59 PM, Hui Zhang wrote: >> >>> Dmitry, >>> >>> finally, I am able to make a simple program and reproduce the error. Since >>> I had moved to >>> petsc-dev (before I used petsc-3.2), the error is a little different. >>> >>> Download the program attached and run the run.sh. You will see the output >>> from PCASM. >>> There is a submatrix (submat[1] on rank 0) which has a row like >>> >>> row 1: (0, -1) (1, 0 + 0.4 i) (4, -1) (2, -1) >>> >>> Note that the column 2 is placed in the last and the column numbers are not >>> in the normal order. >>> Following that, we can also see the error message like >>> >>> [0]PETSC ERROR: New nonzero at (1,2) caused a malloc! >>> >>> You can also try with GASM, use >>> >>> mpirun -np 2 ./gasm_test -dd_type gasm -n 4 -px 1 -py 2 -sx 2 -sy 1 >>> >>> which partitions the domain to 1 * 2 and mapped to the two processors, and >>> for each processor >>> we further partition its local domain to 2 * 1 subdomains. >>> >>> Then, you will see that the row 1 of submat[1] on rank 1 becomes >>> >>> row 1: (0, -1) (1, 0 + 0.4 i) (3, -1) (6, -1) >>> >>> which is totally wrong because from the IS for this overlapping subdomain >>> and the big matrix A, >>> row 1 should have non-zeros like the output from PCASM. >>> >>> I guess the problem is due to that we set PCASMSetSortIndices to FALSE and >>> something goes wrong >>> in asm.c and gasm.c. >>> >>> Thanks! >>> Hui >>> >>> >>> <GASM_test.zip> >>> >>> >>> >>> >>> >>>> On Sun, May 20, 2012 at 5:06 AM, Hui Zhang <mike.hui.zhang at hotmail.com> >>>> wrote: >>>> I have the following MatView output: >>>> >>>> row 0: (0, -0.0351719 - 0.638814 i) (1, -0.517586 - 0.617823 i) (4, >>>> -0.517586 - 0.617823 i) (1, -1.03517) (5, -0.508793) >>>> >>>> Note that there are two values for the column 1, is this normal? >>>> >>>> The above matrix is obtained from PCASMGetSubKSP(pc,subksp) and >>>> KSPGetOperators(subksp[0], ...). >>>> >>>> This is odd. Can you explain a bit more what leads up to this, so we can >>>> try and reproduce the problem? >>>> Thanks. >>>> Dmitry. >>>> >>>> Thanks! >>>> >>>> >>>> >>>> On May 11, 2012, at 7:17 PM, Dmitry Karpeev wrote: >>>> >>>>> You can call PCSetUp(pc) on either ASM or GASM, and that will destroy and >>>>> recreate the matrices (including calling >>>>> your modification subroutine), but not the subdomains or the subdomain >>>>> solvers. >>>>> If you just want to modify the submatrices, you can call >>>>> PC(G)ASMGetSubmatrices() and modify the matrices it returns >>>>> (in the same order as the subdomains were set). That's a bit of a hack, >>>>> since you will essentially be modifying the PC's internal data >>>>> structures. As long as you are careful, you should be okay, since you >>>>> already effectively have the same type of access to the submatrices >>>>> through the Modify callback. >>>>> >>>>> Dmitry. >>>>> >>>>> On Fri, May 11, 2012 at 11:52 AM, Hui Zhang <mike.hui.zhang at >>>>> hotmail.com> wrote: >>>>> I just have a question about reuse of PCASM or PCGASM. >>>>> Suppose I have seted up the PCASM and related KSP and I solved one time. >>>>> Next for the same linear system (matrix and RHS), I just want PCASM >>>>> modify the submatrices (PCSetModifySubmatrices) in a different way, using >>>>> the same routine for modifying but with >>>>> different user context for the modifying routine. >>>>> >>>>> What can I do for this task? Currently, I destroy the KSP and >>>>> re-construct it. I guess >>>>> even for PCASM I can re-use it because the partition of subdomains remain >>>>> the same. >>>>> >>>>> Thanks! >>>>> >>>>> >>>>> On May 10, 2012, at 6:37 PM, Dmitry Karpeev wrote: >>>>> >>>>>> Hui, >>>>>> There've been several changes to PCGASM ahead of the new release. >>>>>> Let me go back and see if it affected the convergence problem. >>>>>> Dmitry. >>>>>> >>>>>> On Thu, May 10, 2012 at 4:16 AM, Hui Zhang <mike.hui.zhang at >>>>>> hotmail.com> wrote: >>>>>> Hi Dmitry, >>>>>> >>>>>> is there any news about PCGASM? >>>>>> >>>>>> thanks, >>>>>> Hui >>>>>> >>>>>> On Feb 20, 2012, at 6:38 PM, Dmitry Karpeev wrote: >>>>>> >>>>>>> Okay, thanks. >>>>>>> I'll take a look. >>>>>>> >>>>>>> Dmitry. >>>>>>> >>>>>>> On Mon, Feb 20, 2012 at 11:30 AM, Hui Zhang <mike.hui.zhang at >>>>>>> hotmail.com> wrote: >>>>>>> For reference, my results are attached. >>>>>>> >>>>>>> asm1.txt for asm with 1 process, >>>>>>> asm2.txt for asm with 2 processes, >>>>>>> gasm1.txt for gasm with 1 process, (with the iteration numbers >>>>>>> different from others) >>>>>>> gasm2.txt for gasm with 2 processes >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> thank you, >>>>>>> Hui >>>>>>> >>>>>>> On Feb 20, 2012, at 3:06 PM, Dmitry Karpeev wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Feb 20, 2012 at 12:59 AM, Hui Zhang <mike.hui.zhang at >>>>>>>> hotmail.com> wrote: >>>>>>>> >>>>>>>> On Feb 20, 2012, at 12:41 AM, Dmitry Karpeev wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Feb 19, 2012 at 3:08 PM, Hui Zhang <mike.hui.zhang at >>>>>>>>> hotmail.com> wrote: >>>>>>>>> I have a new problem: the results from ASM and GASM are different and >>>>>>>>> it seems >>>>>>>>> GASM has something wrong with SetModifySubMatrices. Numerical tests >>>>>>>>> are with >>>>>>>>> each subdomain supported only by one subdomain. There are no problems >>>>>>>>> when >>>>>>>>> I did not modify submatrices. But when I modify submatrices, there >>>>>>>>> are problems >>>>>>>>> with GASM but no problems with ASM. >>>>>>>>> >>>>>>>>> For example, I use two subdomains. In the first case each subdomain >>>>>>>>> is supported by >>>>>>>>> one processor and there seems no problem with GASM. But when I use >>>>>>>>> run my program >>>>>>>>> with only one proc. so that it supports both of the two subdomains, >>>>>>>>> the iteration >>>>>>>>> number is different from the first case and is much larger. On the >>>>>>>>> other hand >>>>>>>>> ASM has no such problem. >>>>>>>>> >>>>>>>>> Are the solutions the same? >>>>>>>>> What problem are you solving? >>>>>>>> >>>>>>>> Yes, the solutions are the same. That's why ASM gives the same results >>>>>>>> with one or >>>>>>>> two processors. But GASM did not. >>>>>>>> Sorry, I wasn't clear: ASM and GASM produced different solutions in >>>>>>>> the case of two domains per processor? >>>>>>>> I'm solving the Helmholtz equation. Maybe >>>>>>>> I can prepare a simpler example to show this difference. >>>>>>>> That would be helpful. >>>>>>>> Thanks. >>>>>>>> >>>>>>>> Dmitry. >>>>>>>> >>>>>>>>> >>>>>>>>> Dmitry. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Feb 15, 2012, at 6:46 PM, Dmitry Karpeev wrote: >>>>>>>>> >>>>>>>>>> You should be able to. >>>>>>>>>> This behavior is the same as in PCASM, >>>>>>>>>> except in GASM the matrices live on subcommunicators. >>>>>>>>>> I am in transit right now, but I can take a closer look in Friday. >>>>>>>>>> >>>>>>>>>> Dmitry >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Feb 15, 2012, at 8:07, Hui Zhang <mike.hui.zhang at hotmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> On Feb 15, 2012, at 11:19 AM, Hui Zhang wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Dmitry, >>>>>>>>>>>> >>>>>>>>>>>> thanks a lot! Currently, I'm not using ISColoring. Just comes >>>>>>>>>>>> another question >>>>>>>>>>>> on PCGASMSetModifySubMatrices(). The user provided function has >>>>>>>>>>>> the prototype >>>>>>>>>>>> >>>>>>>>>>>> func (PC pc,PetscInt nsub,IS *row,IS *col,Mat *submat,void >>>>>>>>>>>> *ctx); >>>>>>>>>>>> >>>>>>>>>>>> I think the coloumns from the parameter 'col' are always the same >>>>>>>>>>>> as the rows >>>>>>>>>>>> from the parameter 'row'. Because PCGASMSetLocalSubdomains() only >>>>>>>>>>>> accepts >>>>>>>>>>>> index sets but not rows and columns. Has I misunderstood something? >>>>>>>>>>> >>>>>>>>>>> As I tested, the row and col are always the same. >>>>>>>>>>> >>>>>>>>>>> I have a new question. Am I allowed to SetLocalToGlobalMapping() >>>>>>>>>>> for the submat's >>>>>>>>>>> in the above func()? >>>>>>>>>>> >>>>>>>>>>> thanks, >>>>>>>>>>> Hui >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> thanks, >>>>>>>>>>>> Hui >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Feb 11, 2012, at 3:36 PM, Dmitry Karpeev wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Yes, that's right. >>>>>>>>>>>>> There is no good way to help the user assemble the subdomains at >>>>>>>>>>>>> the moment beyond the 2D stuff. >>>>>>>>>>>>> It is expected that they are generated from mesh subdomains. >>>>>>>>>>>>> Each IS does carry the subdomains subcomm. >>>>>>>>>>>>> >>>>>>>>>>>>> There is ISColoringToList() that is supposed to convert a >>>>>>>>>>>>> "coloring" of indices to an array of ISs, >>>>>>>>>>>>> each having the indices with the same color and the subcomm that >>>>>>>>>>>>> supports that color. It is >>>>>>>>>>>>> largely untested, though. You could try using it and give us >>>>>>>>>>>>> feedback on any problems you encounter. >>>>>>>>>>>>> >>>>>>>>>>>>> Dmitry. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Sat, Feb 11, 2012 at 6:06 AM, Hui Zhang <mike.hui.zhang at >>>>>>>>>>>>> hotmail.com> wrote: >>>>>>>>>>>>> About PCGASMSetLocalSubdomains(), in the case of one subdomain >>>>>>>>>>>>> supported by >>>>>>>>>>>>> multiple processors, shall I always create the arguments 'is[s]' >>>>>>>>>>>>> and 'is_local[s]' >>>>>>>>>>>>> in a subcommunicator consisting of processors supporting the >>>>>>>>>>>>> subdomain 's'? >>>>>>>>>>>>> >>>>>>>>>>>>> The source code of PCGASMCreateSubdomains2D() seemingly does so. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Hui >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120522/5fa8fe3c/attachment.html>
