Thanks Hong! Now the petsc4py script I sent works for theta methods as well. I will test it with firedrake-ts soon.
Miguel From: "Zhang, Hong" <[email protected]> Date: Tuesday, December 22, 2020 at 12:35 PM To: "Salazar De Troya, Miguel" <[email protected]> Cc: Satish Balay via petsc-users <[email protected]> Subject: Re: [petsc-users] Support for full jacobianP in TSSetIJacobianP Miguel, You can now use my branch hongzh/support-parameterized-mass-matrix. It may take a few days or weeks to be merged. Your original script should work out of box with any checkpointing scheme. Nothing needs to be changed. The IJacobianP is simply M_P*U_t. Hong On Dec 22, 2020, at 11:46 AM, Salazar De Troya, Miguel via petsc-users <[email protected]<mailto:[email protected]>> wrote: Thanks, Hong. Now it works! I can work with backwards Euler for now. With regards to the other two options, I think -ts_trajectory_solution_only is ok because backwards euler does not have intermediate stage. With respect to -ts_trajectory_type memory, can I still do checkpointing to be able to solve larger problems? I have also noticed that TSComputeIJacobianP() is only used by the theta methods. Are there plans to support higher order methods? Miguel From: "Zhang, Hong" <[email protected]<mailto:[email protected]>> Date: Monday, December 21, 2020 at 8:16 PM To: "Salazar De Troya, Miguel" <[email protected]<mailto:[email protected]>> Cc: Satish Balay via petsc-users <[email protected]<mailto:[email protected]>> Subject: Re: [petsc-users] Support for full jacobianP in TSSetIJacobianP Thank you for providing the example. This is very helpful. Sorry that I was not accurate about what should be in IJacobianP. With current API, a little hack is needed to get it work. In IJacobianP, we have to provide shift*M_P*dt if we expand the formula in the paper to accommodate parameters mass matrices. So I changed your code as follows: if self.deriv == "c": dt = ts.getTimeStep() # dt is negative in backward run Jp[0, 0] = -shift*udot[0]*dt I noticed that there is some problem with the input variable Xdot and have been working on a fix. But as a quick workaround, you can use backward Euler with the following options before the fix is ready: -ts_type beuler -ts_trajectory_type memory -ts_trajectory_solution_only Thanks, Hong On Dec 20, 2020, at 3:28 PM, Salazar De Troya, Miguel <[email protected]<mailto:[email protected]>> wrote: Hello Hong, Thank you. My apologies for rushing to blame the API instead of looking at my own code. I’ve put together a minimum example in petsc4py that I am attaching to this email. Here I am solving the simple ODE: c * xdot = b * x(t) with initial condition x(0) = a and the cost function J equal to the solution at the final time “T”, i.e. J = x(T). The analytical solution is x(t) = a * exp(b/c *t). In the example, there is the option to calculate the derivatives w.r.t “b” or “c” in the keyword argument “deriv” passed to “SimpleODE”. For “b”, the solver returns the correct derivatives (checked with the analytical expression), but this does not work for “c”. I might be building the wrong jacobian that I pass to “setIJacobianP”. Could you please take a look at it? Thanks. Miguel From: "Zhang, Hong" <[email protected]<mailto:[email protected]>> Date: Saturday, December 19, 2020 at 10:02 AM To: "Salazar De Troya, Miguel" <[email protected]<mailto:[email protected]>> Cc: Satish Balay via petsc-users <[email protected]<mailto:[email protected]>> Subject: Re: [petsc-users] Support for full jacobianP in TSSetIJacobianP On Dec 18, 2020, at 6:35 PM, Salazar De Troya, Miguel <[email protected]<mailto:[email protected]>> wrote: Ok, I was not able to get such case to work in my firedrake-ts implementation. Maybe I am missing something in my code. I looked at the TSAdjoint paper https://arxiv.org/pdf/1912.07696.pdf Equation 2.1 and at the adjoint method for the theta method (Equation 2.15) where the mass matrix is not differentiated w.r.t. the design parameter “p” and decided to ask the question. For notational brevity, the formula used in the paper does not assume that the mass matrix depends on the parameters p. But it can be easily extended for this case. Is the actual implementation different from what is in the paper? The actual implementation is more general than the formula presented in the paper. Note that PETSc TS takes the ODE problem as F(U_t,U,P,t) = G(U,P,t) and does not ask for a mass matrix explicitly from users. When users provide IFunction, which is F(Udot,U,P,t), IJacobian (dF/dU) and IJacobianP (dF/dP) are needed by TSAdjoint to compute the sensitivities. Differentiating the mass matrix (more precisely, the term M*U_t ) is needed when you prepare the call back function IJacobianP. So if we have M(P)*U_t - f(t,U,P) in IFunction, IJacobianP should be M_P*U_t - f_P where U_t is provided by PETSc as an input argument. Thanks, Hong Thanks Miguel From: "Zhang, Hong" <[email protected]<mailto:[email protected]>> Date: Friday, December 18, 2020 at 3:11 PM To: "Salazar De Troya, Miguel" <[email protected]<mailto:[email protected]>> Cc: Satish Balay via petsc-users <[email protected]<mailto:[email protected]>> Subject: Re: [petsc-users] Support for full jacobianP in TSSetIJacobianP The current interface is general and should be applicable to this case as soon as users can provide IJacobianP, which is dF(Udot,U,P,t)/dP. Were you able to generate it in firedrake? If so, could you provide an example that I can test? Thanks, Hong On Dec 18, 2020, at 10:58 AM, Salazar De Troya, Miguel <[email protected]<mailto:[email protected]>> wrote: Yes, that is the case I am considering. The special case I am concerned about is as following: the heat equation in variational form and in firedrake/UFL notation is as follows: p*u_t*v*dx + inner(grad(u), grad(v))*dx = 0, where u is the temperature, u_t is its time derivative, v is just the test function, dx is the integration domain and p is the design parameter. If “p” were discontinuous, one can’t just factor “p” into the second term due to the divergence theorem. Meaning that p*u_t*v*dx + inner(grad(u), grad(v))*dx = 0 is different than u_t*v*dx + inner(1.0 / p * grad(u), grad(v))*dx = 0, which is what ideally one would obtain in order to adapt to the current interface in TSAdjoint. Thanks Miguel From: "Zhang, Hong" <[email protected]<mailto:[email protected]>> Date: Thursday, December 17, 2020 at 7:25 PM To: "Salazar De Troya, Miguel" <[email protected]<mailto:[email protected]>> Cc: Satish Balay via petsc-users <[email protected]<mailto:[email protected]>> Subject: Re: [petsc-users] Support for full jacobianP in TSSetIJacobianP Hi Miguel, Thank you for the nice work. I do not understand what you propose to do here. What is the obstacle to using current TSSetIJacobianP() for the corner case you mentioned? Are you considering a case in which the mass matrix is parameterized, e.g. M(p) udot - f(t,u) = g(t,u) ? Thanks, Hong On Dec 17, 2020, at 3:38 PM, Salazar De Troya, Miguel via petsc-users <[email protected]<mailto:[email protected]>> wrote: Hello, I am working on hooking up TSAdjoint with pyadjoint through the firedrake-ts interface (https://github.com/IvanYashchuk/firedrake-ts). I have done most of the implementation and now I am just testing for corner cases. One of them is when the design variable is multiplying the first derivative term. It would be the case ofF(Udot,U,P,t) = G(U,P,t) in https://www.mcs.anl.gov/petsc/petsc-master/docs/manualpages/Sensitivity/TSSetIJacobianP.html . I imagine that one could think of refactoring the “P” in the left hand side to the right hand side, but this is not trivial when “P” is a discontinuous field over the domain. I think it would be better to include the case of F(Udot,U,P,t) = G(U,P,t) in TSSetIJacobianP and I am volunteering to do it. Given the current implementation of TSAdjoint, is this something feasible? Thanks Miguel Miguel A. Salazar de Troya Postdoctoral Researcher, Lawrence Livermore National Laboratory B141 Rm: 1085-5 Ph: 1(925) 422-6411 <simple-ode.py>
