Re: [petsc-users] reuse a real matrix for a second linear system with complex numbers

2021-05-14 Thread Matthew Knepley via petsc-users 
On Fri, May 14, 2021 at 4:23 AM feng wang  wrote:

> Dear All,
>
> I am solving a coupled system. One system is AX=B. A, X and B are all real
> numbers and it is solved with GMRES in petsc. Now I need to solve a second
> linear system, it can be represented as (A+i*w)*Z=C. i is the imaginary
> unit.  Z and C are also complex numbers.
>
> So the Jacobian matrix of the second system is just A plus a diagonal
> contribution i*w.  I would like solve the second system with GMRES, could
> petsc handle this? any comments are welcome.
>

Mixing real and complex numbers in the same code is somewhat difficult now.
You have two obvious choices:

1) Configure for complex numbers and solve your first system as complex but
with 0 imaginary part. This will work fine, but uses more memory for that
system. However, since you will already
 use that much memory for the second system, it does not seem like a
big deal to me.

2) You could solve the second system in its equivalent real form

  / A  w \ /Zr\ = /Cr\
 \ -w  A / \Zi/ \Ci/

   This uses more memory for the second system, but does not require
reconfiguring.

  THanks,

 Matt

Thanks,
> Feng
>
-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/

Re: [petsc-users] reuse a real matrix for a second linear system with complex numbers

2021-05-14 Thread feng wang 
Thanks for your comments. It is very helpful!

I might try the 1st approach first.   For the 2nd approach which uses an 
equivalent real-number system, I see potential issues when running in parallel. 
I have re-ordered my cells to allow each rank hold continuous rows in the first 
real system Ax=B. For the equivalent real-number system,  each rank now holds 
(or can assign values to) two patches of continuous rows, which are separated 
by N rows, N is the size of square matrix A. I can't see a straightforward way 
to allow each rank hold continuous rows in this case. or petsc can handle these 
two patches of continuous rows with fixed row index difference in this case?

By the way, could I still re-use my KSP object in my second system by simply 
changing the operators and setting new parameters?

Thanks,
Feng


From: Matthew Knepley 
Sent: 14 May 2021 10:00
To: feng wang 
Cc: petsc-users@mcs.anl.gov 
Subject: Re: [petsc-users] reuse a real matrix for a second linear system with 
complex numbers

On Fri, May 14, 2021 at 4:23 AM feng wang 
mailto:snails...@hotmail.com>> wrote:
Dear All,

I am solving a coupled system. One system is AX=B. A, X and B are all real 
numbers and it is solved with GMRES in petsc. Now I need to solve a second 
linear system, it can be represented as (A+i*w)*Z=C. i is the imaginary unit.  
Z and C are also complex numbers.

So the Jacobian matrix of the second system is just A plus a diagonal 
contribution i*w.  I would like solve the second system with GMRES, could petsc 
handle this? any comments are welcome.

Mixing real and complex numbers in the same code is somewhat difficult now. You 
have two obvious choices:

1) Configure for complex numbers and solve your first system as complex but 
with 0 imaginary part. This will work fine, but uses more memory for that 
system. However, since you will already
 use that much memory for the second system, it does not seem like a big 
deal to me.

2) You could solve the second system in its equivalent real form

  / A  w \ /Zr\ = /Cr\
 \ -w  A / \Zi/ \Ci/

   This uses more memory for the second system, but does not require 
reconfiguring.

  THanks,

 Matt

Thanks,
Feng
--
What most experimenters take for granted before they begin their experiments is 
infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/<http://www.cse.buffalo.edu/~knepley/>

Re: [petsc-users] reuse a real matrix for a second linear system with complex numbers

2021-05-14 Thread Matthew Knepley 
On Fri, May 14, 2021 at 10:36 AM feng wang  wrote:

> Thanks for your comments. It is very helpful!
>
> I might try the 1st approach first.   For the 2nd approach which uses an
> equivalent real-number system, I see potential issues when running in
> parallel. I have re-ordered my cells to allow each rank hold continuous
> rows in the first real system Ax=B. For the equivalent real-number system,
> each rank now holds (or can assign values to) two patches of continuous
> rows, which are separated by N rows, N is the size of square matrix A. I
> can't see a straightforward way to allow each rank hold continuous rows in
> this case. or petsc can handle these two patches of continuous rows with
> fixed row index difference in this case?
>

I just wrote it that way for ease of typing. You can imagine permuting into
2x2 blocks with

  /a  w\
  \-w a/

for each entry.

  Thanks,

 Matt


> By the way, could I still re-use my KSP object in my second system by
> simply changing the operators and setting new parameters?
>
> Thanks,
> Feng
>
> --
> *From:* Matthew Knepley 
> *Sent:* 14 May 2021 10:00
> *To:* feng wang 
> *Cc:* petsc-users@mcs.anl.gov 
> *Subject:* Re: [petsc-users] reuse a real matrix for a second linear
> system with complex numbers
>
> On Fri, May 14, 2021 at 4:23 AM feng wang  wrote:
>
> Dear All,
>
> I am solving a coupled system. One system is AX=B. A, X and B are all real
> numbers and it is solved with GMRES in petsc. Now I need to solve a second
> linear system, it can be represented as (A+i*w)*Z=C. i is the imaginary
> unit.  Z and C are also complex numbers.
>
> So the Jacobian matrix of the second system is just A plus a diagonal
> contribution i*w.  I would like solve the second system with GMRES, could
> petsc handle this? any comments are welcome.
>
>
> Mixing real and complex numbers in the same code is somewhat difficult
> now. You have two obvious choices:
>
> 1) Configure for complex numbers and solve your first system as complex
> but with 0 imaginary part. This will work fine, but uses more memory for
> that system. However, since you will already
>  use that much memory for the second system, it does not seem like a
> big deal to me.
>
> 2) You could solve the second system in its equivalent real form
>
>   / A  w \ /Zr\ = /Cr\
>  \ -w  A / \Zi/ \Ci/
>
>This uses more memory for the second system, but does not require
> reconfiguring.
>
>   THanks,
>
>  Matt
>
> Thanks,
> Feng
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

Re: [petsc-users] reuse a real matrix for a second linear system with complex numbers

2021-05-14 Thread feng wang 
Yes, you are right. I can do row permutations to make them continuous. I will 
try this.

could I re-use my KSP object from the 1st linear system in my 2nd system by 
simply changing the operators and setting new parameters? or I need a separate 
KSP object for the 2nd system?

Thanks,
Feng


From: Matthew Knepley 
Sent: 14 May 2021 15:20
To: feng wang 
Cc: petsc-users@mcs.anl.gov 
Subject: Re: [petsc-users] reuse a real matrix for a second linear system with 
complex numbers

On Fri, May 14, 2021 at 10:36 AM feng wang 
mailto:snails...@hotmail.com>> wrote:
Thanks for your comments. It is very helpful!

I might try the 1st approach first.   For the 2nd approach which uses an 
equivalent real-number system, I see potential issues when running in parallel. 
I have re-ordered my cells to allow each rank hold continuous rows in the first 
real system Ax=B. For the equivalent real-number system,  each rank now holds 
(or can assign values to) two patches of continuous rows, which are separated 
by N rows, N is the size of square matrix A. I can't see a straightforward way 
to allow each rank hold continuous rows in this case. or petsc can handle these 
two patches of continuous rows with fixed row index difference in this case?

I just wrote it that way for ease of typing. You can imagine permuting into 2x2 
blocks with

  /a  w\
  \-w a/

for each entry.

  Thanks,

 Matt

By the way, could I still re-use my KSP object in my second system by simply 
changing the operators and setting new parameters?

Thanks,
Feng


From: Matthew Knepley mailto:knep...@gmail.com>>
Sent: 14 May 2021 10:00
To: feng wang mailto:snails...@hotmail.com>>
Cc: petsc-users@mcs.anl.gov<mailto:petsc-users@mcs.anl.gov> 
mailto:petsc-users@mcs.anl.gov>>
Subject: Re: [petsc-users] reuse a real matrix for a second linear system with 
complex numbers

On Fri, May 14, 2021 at 4:23 AM feng wang 
mailto:snails...@hotmail.com>> wrote:
Dear All,

I am solving a coupled system. One system is AX=B. A, X and B are all real 
numbers and it is solved with GMRES in petsc. Now I need to solve a second 
linear system, it can be represented as (A+i*w)*Z=C. i is the imaginary unit.  
Z and C are also complex numbers.

So the Jacobian matrix of the second system is just A plus a diagonal 
contribution i*w.  I would like solve the second system with GMRES, could petsc 
handle this? any comments are welcome.

Mixing real and complex numbers in the same code is somewhat difficult now. You 
have two obvious choices:

1) Configure for complex numbers and solve your first system as complex but 
with 0 imaginary part. This will work fine, but uses more memory for that 
system. However, since you will already
 use that much memory for the second system, it does not seem like a big 
deal to me.

2) You could solve the second system in its equivalent real form

  / A  w \ /Zr\ = /Cr\
 \ -w  A / \Zi/ \Ci/

   This uses more memory for the second system, but does not require 
reconfiguring.

  THanks,

 Matt

Thanks,
Feng
--
What most experimenters take for granted before they begin their experiments is 
infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/<http://www.cse.buffalo.edu/~knepley/>


--
What most experimenters take for granted before they begin their experiments is 
infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/<http://www.cse.buffalo.edu/~knepley/>

Re: [petsc-users] reuse a real matrix for a second linear system with complex numbers

2021-05-14 Thread Matthew Knepley 
On Fri, May 14, 2021 at 1:36 PM feng wang  wrote:

> Yes, you are right. I can do row permutations to make them continuous. I
> will try this.
>
> could I re-use my KSP object from the 1st linear system in my 2nd system
> by simply changing the operators and setting new parameters? or I need a
> separate KSP object for the 2nd system?
>

I tink you want 2 KSP objects. You could reuse the settings of the first,
but since the system is a different size, all storage would have to be
deleted and recreated anyway.

  Thanks,

 Matt


> Thanks,
> Feng
>
> --
> *From:* Matthew Knepley 
> *Sent:* 14 May 2021 15:20
> *To:* feng wang 
> *Cc:* petsc-users@mcs.anl.gov 
> *Subject:* Re: [petsc-users] reuse a real matrix for a second linear
> system with complex numbers
>
> On Fri, May 14, 2021 at 10:36 AM feng wang  wrote:
>
> Thanks for your comments. It is very helpful!
>
> I might try the 1st approach first.   For the 2nd approach which uses an
> equivalent real-number system, I see potential issues when running in
> parallel. I have re-ordered my cells to allow each rank hold continuous
> rows in the first real system Ax=B. For the equivalent real-number system,
> each rank now holds (or can assign values to) two patches of continuous
> rows, which are separated by N rows, N is the size of square matrix A. I
> can't see a straightforward way to allow each rank hold continuous rows in
> this case. or petsc can handle these two patches of continuous rows with
> fixed row index difference in this case?
>
>
> I just wrote it that way for ease of typing. You can imagine permuting
> into 2x2 blocks with
>
>   /a  w\
>   \-w a/
>
> for each entry.
>
>   Thanks,
>
>  Matt
>
>
> By the way, could I still re-use my KSP object in my second system by
> simply changing the operators and setting new parameters?
>
> Thanks,
> Feng
>
> ------------------
> *From:* Matthew Knepley 
> *Sent:* 14 May 2021 10:00
> *To:* feng wang 
> *Cc:* petsc-users@mcs.anl.gov 
> *Subject:* Re: [petsc-users] reuse a real matrix for a second linear
> system with complex numbers
>
> On Fri, May 14, 2021 at 4:23 AM feng wang  wrote:
>
> Dear All,
>
> I am solving a coupled system. One system is AX=B. A, X and B are all real
> numbers and it is solved with GMRES in petsc. Now I need to solve a second
> linear system, it can be represented as (A+i*w)*Z=C. i is the imaginary
> unit.  Z and C are also complex numbers.
>
> So the Jacobian matrix of the second system is just A plus a diagonal
> contribution i*w.  I would like solve the second system with GMRES, could
> petsc handle this? any comments are welcome.
>
>
> Mixing real and complex numbers in the same code is somewhat difficult
> now. You have two obvious choices:
>
> 1) Configure for complex numbers and solve your first system as complex
> but with 0 imaginary part. This will work fine, but uses more memory for
> that system. However, since you will already
>  use that much memory for the second system, it does not seem like a
> big deal to me.
>
> 2) You could solve the second system in its equivalent real form
>
>   / A  w \ /Zr\ = /Cr\
>  \ -w  A / \Zi/ \Ci/
>
>This uses more memory for the second system, but does not require
> reconfiguring.
>
>   THanks,
>
>  Matt
>
> Thanks,
> Feng
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

Re: [petsc-users] reuse a real matrix for a second linear system with complex numbers

2021-05-17 Thread feng wang 
Hi Mat,

I have tried the 1st approach you suggested, which re-configures petsc with 
complex numbers. For my real-number system, Ax=B, it still works fine. I would 
like to save a copy a A, because for my complex system (A + i*w) Z = C, the 
value of "w" could be a list of values,  so I only need to do a MatShift for my 
saved copy of A to build the LHS for each "w".

I am trying to use MatDuplicate to create the copy of "A" and MatCopy to copy 
the values of "A". the following is what I do.  "petsc_A_pre" is "A" and 
"petsc_A_pre_copy" is its copy.

  ierr = MatCreateBAIJ(*A_COMM_WORLD, blocksize, nlocal*blocksize, 
nlocal*blocksize, PETSC_DETERMINE, PETSC_DETERMINE,
maxneig, NULL, maxneig, NULL, &petsc_A_pre); 
CHKERRQ(ierr);
  ierr = MatSetOption(petsc_A_pre, MAT_STRUCTURALLY_SYMMETRIC, PETSC_TRUE); 
CHKERRQ(ierr);
  ierr = MatDuplicate(petsc_A_pre, MAT_SHARE_NONZERO_PATTERN, 
&petsc_A_pre_copy); CHKERRQ(ierr);   //line 69

.some operations.

 ierr = MatAssemblyBegin(petsc_A_pre,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);
 ierr = MatAssemblyEnd(petsc_A_pre,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);

 ierr = MatCopy(petsc_A_pre, petsc_A_pre_copy, SAME_NONZERO_PATTERN);
 ierr = 
MatAssemblyBegin(petsc_A_pre_copy,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);
 ierr = 
MatAssemblyEnd(petsc_A_pre_copy,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);


I get the following error for MatDuplicate and MatCopy:

[0]PETSC ERROR: Petsc has generated inconsistent data
[0]PETSC ERROR: Corrupt matrix
[0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html for 
trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.14.4, unknown
[0]PETSC ERROR: cfd on a arch-linux2-c-debug named snail by snail Mon May 17 
09:34:46 2021
[0]PETSC ERROR: Configure options --with-cc=mpicc --with-cxx=mpicxx --with-fc=0 
--with-scalar-type=complex
[0]PETSC ERROR: #1 MatDuplicateNoCreate_SeqBAIJ() line 3103 in 
/home/feng/cfd/petsc.complex/src/mat/impls/baij/seq/baij.c
[0]PETSC ERROR: #2 MatDuplicate_SeqBAIJ() line 3215 in 
/home/feng/cfd/petsc.complex/src/mat/impls/baij/seq/baij.c
[0]PETSC ERROR: #3 MatDuplicate() line 4663 in 
/home/feng/cfd/petsc.complex/src/mat/interface/matrix.c
[0]PETSC ERROR: #4 petsc_nk_init() line 69 in domain/cfd/petsc_nk.cpp

[0]PETSC ERROR: - Error Message 
--
[0]PETSC ERROR: Object is in wrong state
[0]PETSC ERROR: Must call MatXXXSetPreallocation() or MatSetUp() on argument 2 
"B" before MatCopy()
[0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html for 
trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.14.4, unknown
[0]PETSC ERROR: cfd on a arch-linux2-c-debug named feng by feng Mon May 17 
09:34:46 2021
[0]PETSC ERROR: Configure options --with-cc=mpicc --with-cxx=mpicxx --with-fc=0 
--with-scalar-type=complex
[0]PETSC ERROR: #5 MatCopy() line 4087 in 
/home/feng/cfd/petsc.complex/src/mat/interface/matrix.c

 Is there a better way to do this? could you give any comments?

Thanks,
Feng


From: Matthew Knepley 
Sent: 14 May 2021 20:26
To: feng wang 
Cc: petsc-users@mcs.anl.gov 
Subject: Re: [petsc-users] reuse a real matrix for a second linear system with 
complex numbers

On Fri, May 14, 2021 at 1:36 PM feng wang 
mailto:snails...@hotmail.com>> wrote:
Yes, you are right. I can do row permutations to make them continuous. I will 
try this.

could I re-use my KSP object from the 1st linear system in my 2nd system by 
simply changing the operators and setting new parameters? or I need a separate 
KSP object for the 2nd system?

I tink you want 2 KSP objects. You could reuse the settings of the first, but 
since the system is a different size, all storage would have to be deleted and 
recreated anyway.

  Thanks,

 Matt

Thanks,
Feng


From: Matthew Knepley mailto:knep...@gmail.com>>
Sent: 14 May 2021 15:20
To: feng wang mailto:snails...@hotmail.com>>
Cc: petsc-users@mcs.anl.gov<mailto:petsc-users@mcs.anl.gov> 
mailto:petsc-users@mcs.anl.gov>>
Subject: Re: [petsc-users] reuse a real matrix for a second linear system with 
complex numbers

On Fri, May 14, 2021 at 10:36 AM feng wang 
mailto:snails...@hotmail.com>> wrote:
Thanks for your comments. It is very helpful!

I might try the 1st approach first.   For the 2nd approach which uses an 
equivalent real-number system, I see potential issues when running in parallel. 
I have re-ordered my cells to allow each rank hold continuous rows in the first 
real system Ax=B. For the equivalent real-number system,  each rank now holds 
(or can assign values to) two patches of continuous rows, which are separated 
by N rows, N is the size of square matrix A. I can't see a straightforward way 
to allow each ran

Re: [petsc-users] reuse a real matrix for a second linear system with complex numbers

2021-05-17 Thread Matthew Knepley 
On Mon, May 17, 2021 at 5:18 AM feng wang  wrote:

> Hi Mat,
>
> I have tried the 1st approach you suggested, which re-configures petsc
> with complex numbers. For my real-number system, Ax=B, it still works fine.
> I would like to save a copy a A, because for my complex system (A + i*w) Z
> = C, the value of "w" could be a list of values,  so I only need to do a
> MatShift for my saved copy of A to build the LHS for each "w".
>
> I am trying to use MatDuplicate to create the copy of "A" and MatCopy to
> copy the values of "A". the following is what I do.  "petsc_A_pre" is "A"
> and "petsc_A_pre_copy" is its copy.
>
>   ierr = MatCreateBAIJ(*A_COMM_WORLD, blocksize, nlocal*blocksize,
> nlocal*blocksize, PETSC_DETERMINE, PETSC_DETERMINE,
> maxneig, NULL, maxneig, NULL, &petsc_A_pre);
> CHKERRQ(ierr);
>   ierr = MatSetOption(petsc_A_pre, MAT_STRUCTURALLY_SYMMETRIC,
> PETSC_TRUE); CHKERRQ(ierr);
>   ierr = MatDuplicate(petsc_A_pre, MAT_SHARE_NONZERO_PATTERN,
> &petsc_A_pre_copy); CHKERRQ(ierr);   *//line 69*
>
> .some operations.
>
>  ierr =
> MatAssemblyBegin(petsc_A_pre,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);
>  ierr =
> MatAssemblyEnd(petsc_A_pre,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);
>
>  ierr = MatCopy(petsc_A_pre, petsc_A_pre_copy,
> SAME_NONZERO_PATTERN);
>  ierr =
> MatAssemblyBegin(petsc_A_pre_copy,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);
>  ierr =
> MatAssemblyEnd(petsc_A_pre_copy,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);
>
>
> I get the following error for MatDuplicate and MatCopy:
>
> [0]PETSC ERROR: Petsc has generated inconsistent data
> [0]PETSC ERROR: Corrupt matrix
> [0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html
> for trouble shooting.
> [0]PETSC ERROR: Petsc Release Version 3.14.4, unknown
> [0]PETSC ERROR: cfd on a arch-linux2-c-debug named snail by snail Mon May
> 17 09:34:46 2021
> [0]PETSC ERROR: Configure options --with-cc=mpicc --with-cxx=mpicxx
> --with-fc=0 --with-scalar-type=complex
> [0]PETSC ERROR: #1 MatDuplicateNoCreate_SeqBAIJ() line 3103 in
> /home/feng/cfd/petsc.complex/src/mat/impls/baij/seq/baij.c
> [0]PETSC ERROR: #2 MatDuplicate_SeqBAIJ() line 3215 in
> /home/feng/cfd/petsc.complex/src/mat/impls/baij/seq/baij.c
> [0]PETSC ERROR: #3 MatDuplicate() line 4663 in
> /home/feng/cfd/petsc.complex/src/mat/interface/matrix.c
> [0]PETSC ERROR: #4 petsc_nk_init() line 69 in domain/cfd/petsc_nk.cpp
>

Your matrix is actually corrupt. We have to fix this. It is either a PETSc
bug or a bug in your code. Would you be able to send us a stripped down
version of your code (say one that puts in all zeros, or similarly
simplified), so we can track down where this is?

Alternatively, you can run under valgrind, since this looks like a memory
overwrite.

  Thanks,

 Matt


> [0]PETSC ERROR: - Error Message
> --
> [0]PETSC ERROR: Object is in wrong state
> [0]PETSC ERROR: Must call MatXXXSetPreallocation() or MatSetUp() on
> argument 2 "B" before MatCopy()
> [0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html
> for trouble shooting.
> [0]PETSC ERROR: Petsc Release Version 3.14.4, unknown
> [0]PETSC ERROR: cfd on a arch-linux2-c-debug named feng by feng Mon May 17
> 09:34:46 2021
> [0]PETSC ERROR: Configure options --with-cc=mpicc --with-cxx=mpicxx
> --with-fc=0 --with-scalar-type=complex
> [0]PETSC ERROR: #5 MatCopy() line 4087 in
> /home/feng/cfd/petsc.complex/src/mat/interface/matrix.c
>
>  Is there a better way to do this? could you give any comments?
>
> Thanks,
> Feng
>
> --
> *From:* Matthew Knepley 
> *Sent:* 14 May 2021 20:26
> *To:* feng wang 
> *Cc:* petsc-users@mcs.anl.gov 
> *Subject:* Re: [petsc-users] reuse a real matrix for a second linear
> system with complex numbers
>
> On Fri, May 14, 2021 at 1:36 PM feng wang  wrote:
>
> Yes, you are right. I can do row permutations to make them continuous. I
> will try this.
>
> could I re-use my KSP object from the 1st linear system in my 2nd system
> by simply changing the operators and setting new parameters? or I need a
> separate KSP object for the 2nd system?
>
>
> I tink you want 2 KSP objects. You could reuse the settings of the first,
> but since the system is a different size, all storage would have to be
> deleted and recreated anyway.
>
>   Thanks,
>
>  Matt
>
>
> Thanks,
> Feng
>
> --
> *From:* Matthew Knepley 
> *Sent:* 14 May 2021 15:20
> *To:* feng wang 
>

Re: [petsc-users] reuse a real matrix for a second linear system with complex numbers

2021-05-17 Thread feng wang 
Thanks for your reply.

Ax=B
(A + i*w) Z = C

I have been using the matrix A for my real-number system, it is ok, so I 
believe the matrix "A" should not be corrupted.   It is a bit strange that 
MatDuplicate and MatCopy complain. I am quite puzzled by this. my code is a big 
library, it would take some effort to make a stripped-down version, I can have 
a try.

Maybe I don't actually need to make a copy of A. I can directly work on it to 
add the diagonal contribution i*w and then call KSPSolve.  For a different "w", 
I can undo the previous diagonal contribution and add the new one and so on. so 
I only need the memory space of "A" in the end for both systems.  I've tried 
this for one "w" and it seems working for me.

I am relying on the fact that "A" (its values and structure) is untouched by 
"KSPSolve" , am I correct on this or I am being lucky?

Thanks,
feng


From: Matthew Knepley 
Sent: 17 May 2021 11:37
To: feng wang 
Cc: petsc-users@mcs.anl.gov 
Subject: Re: [petsc-users] reuse a real matrix for a second linear system with 
complex numbers

On Mon, May 17, 2021 at 5:18 AM feng wang 
mailto:snails...@hotmail.com>> wrote:
Hi Mat,

I have tried the 1st approach you suggested, which re-configures petsc with 
complex numbers. For my real-number system, Ax=B, it still works fine. I would 
like to save a copy a A, because for my complex system (A + i*w) Z = C, the 
value of "w" could be a list of values,  so I only need to do a MatShift for my 
saved copy of A to build the LHS for each "w".

I am trying to use MatDuplicate to create the copy of "A" and MatCopy to copy 
the values of "A". the following is what I do.  "petsc_A_pre" is "A" and 
"petsc_A_pre_copy" is its copy.

  ierr = MatCreateBAIJ(*A_COMM_WORLD, blocksize, nlocal*blocksize, 
nlocal*blocksize, PETSC_DETERMINE, PETSC_DETERMINE,
maxneig, NULL, maxneig, NULL, &petsc_A_pre); 
CHKERRQ(ierr);
  ierr = MatSetOption(petsc_A_pre, MAT_STRUCTURALLY_SYMMETRIC, PETSC_TRUE); 
CHKERRQ(ierr);
  ierr = MatDuplicate(petsc_A_pre, MAT_SHARE_NONZERO_PATTERN, 
&petsc_A_pre_copy); CHKERRQ(ierr);   //line 69

.some operations.

 ierr = MatAssemblyBegin(petsc_A_pre,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);
 ierr = MatAssemblyEnd(petsc_A_pre,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);

 ierr = MatCopy(petsc_A_pre, petsc_A_pre_copy, SAME_NONZERO_PATTERN);
 ierr = 
MatAssemblyBegin(petsc_A_pre_copy,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);
 ierr = 
MatAssemblyEnd(petsc_A_pre_copy,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);


I get the following error for MatDuplicate and MatCopy:

[0]PETSC ERROR: Petsc has generated inconsistent data
[0]PETSC ERROR: Corrupt matrix
[0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html for 
trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.14.4, unknown
[0]PETSC ERROR: cfd on a arch-linux2-c-debug named snail by snail Mon May 17 
09:34:46 2021
[0]PETSC ERROR: Configure options --with-cc=mpicc --with-cxx=mpicxx --with-fc=0 
--with-scalar-type=complex
[0]PETSC ERROR: #1 MatDuplicateNoCreate_SeqBAIJ() line 3103 in 
/home/feng/cfd/petsc.complex/src/mat/impls/baij/seq/baij.c
[0]PETSC ERROR: #2 MatDuplicate_SeqBAIJ() line 3215 in 
/home/feng/cfd/petsc.complex/src/mat/impls/baij/seq/baij.c
[0]PETSC ERROR: #3 MatDuplicate() line 4663 in 
/home/feng/cfd/petsc.complex/src/mat/interface/matrix.c
[0]PETSC ERROR: #4 petsc_nk_init() line 69 in domain/cfd/petsc_nk.cpp

Your matrix is actually corrupt. We have to fix this. It is either a PETSc bug 
or a bug in your code. Would you be able to send us a stripped down
version of your code (say one that puts in all zeros, or similarly simplified), 
so we can track down where this is?

Alternatively, you can run under valgrind, since this looks like a memory 
overwrite.

  Thanks,

 Matt

[0]PETSC ERROR: - Error Message 
--
[0]PETSC ERROR: Object is in wrong state
[0]PETSC ERROR: Must call MatXXXSetPreallocation() or MatSetUp() on argument 2 
"B" before MatCopy()
[0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html for 
trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.14.4, unknown
[0]PETSC ERROR: cfd on a arch-linux2-c-debug named feng by feng Mon May 17 
09:34:46 2021
[0]PETSC ERROR: Configure options --with-cc=mpicc --with-cxx=mpicxx --with-fc=0 
--with-scalar-type=complex
[0]PETSC ERROR: #5 MatCopy() line 4087 in 
/home/feng/cfd/petsc.complex/src/mat/interface/matrix.c

 Is there a better way to do this? could you give any comments?

Thanks,
Feng


From: Matthew Knepley mailto:knep...@gmail.com>>
Sent: 14 May 2021 20:26
To: feng wang mailto:snails...@hotmail.com>>
Cc: p