Re: Question about sparsity-based implementation in MATPower

2015-10-19 Thread Shruti Rao
Thank you Dr. Zimmerman and Dr. Marin. I greatly appreciate your valuable
inputs.

Cheers!

Shruti

On Mon, Oct 19, 2015 at 10:15 AM, Jose Luis Marin 
wrote:

>
> I'd like to add that Matlab keeps incorporating the latest sparse direct
> solvers coming from Tim Davis and his group from TAMU / U. of Florida (
> SuiteSparse ) into
> their new versions.  I believe that if the Jacobian is symmetric, current
> versions of MATLAB will use CHOLMOD, while if it's not, they will use
> UMFPACK.
>
> This is great because these are solid, state of the art direct solvers;
> however, as far as I know, there is still no way in Matlab to tune the
> spparms in order to deactivate their multifrontal / supernodal variants and
> just use the "simplicial" variants instead.  In some testing we did a while
> ago on the C version of SuiteSparse, the multifrontal and supernodal
> approaches performed worse on the kind matrices that one typically obtains
> in power networks.  It made sense, because those techniques are essentially
> trying to find denser blocks in order to use the BLAS, and power systems
> matrices are just too sparse for that approach to pay off.  I hope Matlab
> implements the KLU solver as an option some day, because my hunch is that
> KLU is the fastest solver for power systems problems (it was used on Xyce,
> a SPICE-like simulator).
>
>
> --
> Jose L. Marin
> Gridquant España SL
> Grupo AIA
>
>
> On Mon, Oct 19, 2015 at 3:04 PM, Ray Zimmerman  wrote:
>
>> I would also mention, for those who are interested, that version 5.1 of
>> MATPOWER includes a wrapper function mplinsolve()
>> 
>>  that
>> allows you to choose between different linear solvers for computing the
>> Newton update step in the MIPS interior-point OPF algorithm. Currently this
>> includes only Matlab’s built-in \ operator or the optional PARDISO.
>>
>> If I remember correctly, for the Newton-Raphson power flow, I stuck with
>> using Matlab’s \ operator directly rather than mplinsolve()
>> ,
>> because even for the largest systems I tried, there was little or no
>> advantage to PARDISO, and the extra overhead was noticeable on small
>> systems.
>>
>>Ray
>>
>>
>> On Oct 19, 2015, at 1:05 AM, Shruti Rao  wrote:
>>
>> Thank you Dr. Abhyankar for the guidance. I appreciate your time and
>> effort.
>>
>> Shruti
>>
>> On Sun, Oct 18, 2015 at 10:02 PM, Abhyankar, Shrirang G. > > wrote:
>>
>>> Shruti,
>>>   MATPOWER does use “\” operator for the linear solves. However note
>>> that, internally, MATLAB does perform some sort of matrix reordering to
>>> reduce the fill-ins in the factored matrix. For instance, UMFPACK uses an
>>> approximate minimum degree reordering scheme by default.
>>>
>>> Shri
>>>
>>> From: Shruti Rao 
>>> Reply-To: MATPOWER discussion forum 
>>> Date: Sunday, October 18, 2015 at 8:31 PM
>>> To: MATPOWER discussion forum 
>>> Subject: Re: Question about sparsity-based implementation in MATPower
>>>
>>> Thank you Dr. Abhyakar,
>>>
>>> My main aim was to confirm that MATPower uses the inbuilt "\" to solve
>>> the matrix equations and not Tinney or some other form of reordering  and
>>> then LU factorization followed by forward,backward substitutions. From your
>>> response I assume that it is true that MATpower uses "\" right?
>>>
>>> Thank you for your response.
>>>
>>>
>>>
>>> On Sun, Oct 18, 2015 at 6:27 PM, Abhyankar, Shrirang G. >> > wrote:
>>>
 Hi Shruti,
   The direct linear solver used by MATLAB depends on the symmetry of
 the Jacobian matrix. For MATPOWER test cases that have symmetric Jacobians
 (due to inactive taps), a Cholesky factorization is used (LL^T = A). For
 cases that lead to non-symmetric Jacobian, MATLAB uses UMFPACK for
 performing the linear solve.

 Shri

 From: Shruti Rao 
 Reply-To: MATPOWER discussion forum 
 Date: Sunday, October 18, 2015 at 5:37 PM
 To: MATPOWER discussion forum 
 Subject: Question about sparsity-based implementation in MATPower

 Greetings MATPower community,

 I had a question about the way sparsity-based techniques are used in
 the Newton-Raphson solver of the power flow algorithm in MATPower.

 I ran the code step-by-step and from my understanding, the way the
 sparsity of the Jacobian matrix is exploited is that it is created as a
 MATLAB "sparse" matrix wherein only the non-zeros are stored with the
 respective matrix positions and then the MATLAB operator "\" is invoked
 while calculating dx = -(J \ F); where J is the Jacobian and F is the
 vector of mismatches.

 MATLAB "\" by default exploits the sparsity of the matrix by using a LU
 solver. The kind of solver "\" uses actually depends on the matrix
 structure if it is diagonal/tridiagonal/banded and so on

Re: About PV buses with negative P gens, and the implementation of dclines in powerflow (not OPF)

2015-10-19 Thread Ray Zimmerman
1. Yes and yes.
2. For the purpose of an OPF, a dispatchable load is defined as a generator 
with negative PMIN and zero PMAX. In this case, an extra constant power factor 
constraint is added to the OPF problem. This does not affect simple power flow.
3. Yes.

   Ray

> On Oct 19, 2015, at 2:09 PM, Jose Luis Marin  wrote:
> 
> Hello Ray,
> 
> I read section 7.5.3 of the manual about DC transmission lines, but I still 
> have some questions in what regards simple powerflow:
> As I understand it, the implementation will place dummy generators at each 
> end. Are those buses then switched to PV type, taking the voltage setpoints 
> from VF, VT in mpc.dcline?  Are the MVAR limits QMINF, QMAXF, QMINT, QMAXT 
> used in the outer loop when enforcing limits?
> What happens to the dummy gen on the from side, which has negative P?  I 
> mean, I also read that dispatchable loads are implemented as generators with 
> negative injection, but that MATPOWER automatically keeps their P/Q ratio 
> fixed at the initially specified value.  Does this affect all generators with 
> negative P?  Does this only happen in OPF?
> Does toggle_dcline(mpc, 'on') have to be used when computing simple 
> powerflows too?
> 
> Regards,
> 
> -- 
> Jose L. Marin
> Gridquant España SL
> Grupo AIA
> 
>  


About PV buses with negative P gens, and the implementation of dclines in powerflow (not OPF)

2015-10-19 Thread Jose Luis Marin
Hello Ray,

I read section 7.5.3 of the manual about *DC transmission lines*, but I
still have some questions in what regards simple *powerflow*:

   1. As I understand it, the implementation will place dummy generators at
   each end. Are those buses then switched to PV type, taking the voltage
   setpoints from VF, VT in mpc.dcline?  Are the MVAR limits QMINF, QMAXF,
   QMINT, QMAXT used in the outer loop when enforcing limits?
   2. What happens to the dummy gen on the from side, which has negative
   P?  I mean, I also read that dispatchable loads are implemented as
   generators with negative injection, but that MATPOWER *automatically*
   keeps their P/Q ratio fixed at the initially specified value.  Does this
   affect all generators with negative P?  Does this only happen in OPF?
   3. Does toggle_dcline(mpc, 'on') have to be used when computing simple
   powerflows too?


Regards,

-- 
Jose L. Marin
Gridquant España SL
Grupo AIA




Re: Question about sparsity-based implementation in MATPower

2015-10-19 Thread Jose Luis Marin
I'd like to add that Matlab keeps incorporating the latest sparse direct
solvers coming from Tim Davis and his group from TAMU / U. of Florida (
SuiteSparse ) into
their new versions.  I believe that if the Jacobian is symmetric, current
versions of MATLAB will use CHOLMOD, while if it's not, they will use
UMFPACK.

This is great because these are solid, state of the art direct solvers;
however, as far as I know, there is still no way in Matlab to tune the
spparms in order to deactivate their multifrontal / supernodal variants and
just use the "simplicial" variants instead.  In some testing we did a while
ago on the C version of SuiteSparse, the multifrontal and supernodal
approaches performed worse on the kind matrices that one typically obtains
in power networks.  It made sense, because those techniques are essentially
trying to find denser blocks in order to use the BLAS, and power systems
matrices are just too sparse for that approach to pay off.  I hope Matlab
implements the KLU solver as an option some day, because my hunch is that
KLU is the fastest solver for power systems problems (it was used on Xyce,
a SPICE-like simulator).


-- 
Jose L. Marin
Gridquant España SL
Grupo AIA


On Mon, Oct 19, 2015 at 3:04 PM, Ray Zimmerman  wrote:

> I would also mention, for those who are interested, that version 5.1 of
> MATPOWER includes a wrapper function mplinsolve()
>  
> that
> allows you to choose between different linear solvers for computing the
> Newton update step in the MIPS interior-point OPF algorithm. Currently this
> includes only Matlab’s built-in \ operator or the optional PARDISO.
>
> If I remember correctly, for the Newton-Raphson power flow, I stuck with
> using Matlab’s \ operator directly rather than mplinsolve()
> ,
> because even for the largest systems I tried, there was little or no
> advantage to PARDISO, and the extra overhead was noticeable on small
> systems.
>
>Ray
>
>
> On Oct 19, 2015, at 1:05 AM, Shruti Rao  wrote:
>
> Thank you Dr. Abhyankar for the guidance. I appreciate your time and
> effort.
>
> Shruti
>
> On Sun, Oct 18, 2015 at 10:02 PM, Abhyankar, Shrirang G. 
> wrote:
>
>> Shruti,
>>   MATPOWER does use “\” operator for the linear solves. However note
>> that, internally, MATLAB does perform some sort of matrix reordering to
>> reduce the fill-ins in the factored matrix. For instance, UMFPACK uses an
>> approximate minimum degree reordering scheme by default.
>>
>> Shri
>>
>> From: Shruti Rao 
>> Reply-To: MATPOWER discussion forum 
>> Date: Sunday, October 18, 2015 at 8:31 PM
>> To: MATPOWER discussion forum 
>> Subject: Re: Question about sparsity-based implementation in MATPower
>>
>> Thank you Dr. Abhyakar,
>>
>> My main aim was to confirm that MATPower uses the inbuilt "\" to solve
>> the matrix equations and not Tinney or some other form of reordering  and
>> then LU factorization followed by forward,backward substitutions. From your
>> response I assume that it is true that MATpower uses "\" right?
>>
>> Thank you for your response.
>>
>>
>>
>> On Sun, Oct 18, 2015 at 6:27 PM, Abhyankar, Shrirang G. 
>> wrote:
>>
>>> Hi Shruti,
>>>   The direct linear solver used by MATLAB depends on the symmetry of the
>>> Jacobian matrix. For MATPOWER test cases that have symmetric Jacobians (due
>>> to inactive taps), a Cholesky factorization is used (LL^T = A). For cases
>>> that lead to non-symmetric Jacobian, MATLAB uses UMFPACK for performing the
>>> linear solve.
>>>
>>> Shri
>>>
>>> From: Shruti Rao 
>>> Reply-To: MATPOWER discussion forum 
>>> Date: Sunday, October 18, 2015 at 5:37 PM
>>> To: MATPOWER discussion forum 
>>> Subject: Question about sparsity-based implementation in MATPower
>>>
>>> Greetings MATPower community,
>>>
>>> I had a question about the way sparsity-based techniques are used in the
>>> Newton-Raphson solver of the power flow algorithm in MATPower.
>>>
>>> I ran the code step-by-step and from my understanding, the way the
>>> sparsity of the Jacobian matrix is exploited is that it is created as a
>>> MATLAB "sparse" matrix wherein only the non-zeros are stored with the
>>> respective matrix positions and then the MATLAB operator "\" is invoked
>>> while calculating dx = -(J \ F); where J is the Jacobian and F is the
>>> vector of mismatches.
>>>
>>> MATLAB "\" by default exploits the sparsity of the matrix by using a LU
>>> solver. The kind of solver "\" uses actually depends on the matrix
>>> structure if it is diagonal/tridiagonal/banded and so on (Flowchart
>>> obtained from Mathworks website attached in the email). I assume based on
>>> the typical  structure of the Jacobian that an LU solver is most likely to
>>> be chosen.
>>>
>>> Is my understanding correct or am I missing something out? Thank you for
>>> your time and effort.
>>>
>>>

Re: Question about sparsity-based implementation in MATPower

2015-10-19 Thread Ray Zimmerman
I would also mention, for those who are interested, that version 5.1 of 
MATPOWER includes a wrapper function mplinsolve() 
 
that allows you to choose between different linear solvers for computing the 
Newton update step in the MIPS interior-point OPF algorithm. Currently this 
includes only Matlab’s built-in \ operator or the optional PARDISO.

If I remember correctly, for the Newton-Raphson power flow, I stuck with using 
Matlab’s \ operator directly rather than mplinsolve() 
, 
because even for the largest systems I tried, there was little or no advantage 
to PARDISO, and the extra overhead was noticeable on small systems.

   Ray


> On Oct 19, 2015, at 1:05 AM, Shruti Rao  wrote:
> 
> Thank you Dr. Abhyankar for the guidance. I appreciate your time and effort.
> 
> Shruti
> 
> On Sun, Oct 18, 2015 at 10:02 PM, Abhyankar, Shrirang G.  > wrote:
> Shruti,
>   MATPOWER does use “\” operator for the linear solves. However note that, 
> internally, MATLAB does perform some sort of matrix reordering to reduce the 
> fill-ins in the factored matrix. For instance, UMFPACK uses an approximate 
> minimum degree reordering scheme by default.
> 
> Shri
> 
> From: Shruti Rao mailto:sra...@asu.edu>>
> Reply-To: MATPOWER discussion forum  >
> Date: Sunday, October 18, 2015 at 8:31 PM
> To: MATPOWER discussion forum  >
> Subject: Re: Question about sparsity-based implementation in MATPower
> 
> Thank you Dr. Abhyakar,
> 
> My main aim was to confirm that MATPower uses the inbuilt "\" to solve the 
> matrix equations and not Tinney or some other form of reordering  and then LU 
> factorization followed by forward,backward substitutions. From your response 
> I assume that it is true that MATpower uses "\" right?
> 
> Thank you for your response.
> 
> 
> 
> On Sun, Oct 18, 2015 at 6:27 PM, Abhyankar, Shrirang G.  > wrote:
> Hi Shruti,
>   The direct linear solver used by MATLAB depends on the symmetry of the 
> Jacobian matrix. For MATPOWER test cases that have symmetric Jacobians (due 
> to inactive taps), a Cholesky factorization is used (LL^T = A). For cases 
> that lead to non-symmetric Jacobian, MATLAB uses UMFPACK for performing the 
> linear solve. 
> 
> Shri
> 
> From: Shruti Rao mailto:sra...@asu.edu>>
> Reply-To: MATPOWER discussion forum  >
> Date: Sunday, October 18, 2015 at 5:37 PM
> To: MATPOWER discussion forum  >
> Subject: Question about sparsity-based implementation in MATPower
> 
> Greetings MATPower community,
> 
> I had a question about the way sparsity-based techniques are used in the 
> Newton-Raphson solver of the power flow algorithm in MATPower. 
> 
> I ran the code step-by-step and from my understanding, the way the sparsity 
> of the Jacobian matrix is exploited is that it is created as a MATLAB 
> "sparse" matrix wherein only the non-zeros are stored with the respective 
> matrix positions and then the MATLAB operator "\" is invoked while 
> calculating dx = -(J \ F); where J is the Jacobian and F is the vector of 
> mismatches. 
> 
> MATLAB "\" by default exploits the sparsity of the matrix by using a LU 
> solver. The kind of solver "\" uses actually depends on the matrix structure 
> if it is diagonal/tridiagonal/banded and so on (Flowchart obtained from 
> Mathworks website attached in the email). I assume based on the typical  
> structure of the Jacobian that an LU solver is most likely to be chosen. 
> 
> Is my understanding correct or am I missing something out? Thank you for your 
> time and effort.
> 
> 
> -- 
> Best Regards,
> Shruti Dwarkanath Rao
> 
> Graduate Research Assistant
> School of Electrical, Computer and Energy Engineering
> Arizona State University
> Tempe, AZ, 85281
> 650 996 0116 
> 
> 
> -- 
> Best Regards,
> Shruti Dwarkanath Rao
> 
> Graduate Research Assistant
> School of Electrical, Computer and Energy Engineering
> Arizona State University
> Tempe, AZ, 85281
> 650 996 0116 
> 
> 
> -- 
> Best Regards,
> Shruti Dwarkanath Rao
> 
> Graduate Research Assistant
> School of Electrical, Computer and Energy Engineering
> Arizona State University
> Tempe, AZ, 85281
> 650 996 0116