[julia-users] Re: Optimal Control using Dynamic Programming

[Maxim Egorov]

If you formulate the problem as a Markov decision process, you can use 
POMDPs.jl  which supports value iteration 
. The POMDPs.jl 
framework requires a functional problem definition, and some examples can 
be found in POMDPModels.jl . 

[julia-users] Re: Optimal Control using Dynamic Programming

[Alex Dowling]

I'm also bias to recommend formulating a nonlinear programming problem and 
using JuMP with IPOPT. This approach has been successfully demonstrated in 
the chemical engineering controls/systems literature on some rather nasty 
large nonlinear systems.

Here is a sample problem for nonlinear control of a chemical reactor using 
the discretized NLP approach in GAMS:
http://cepac.cheme.cmu.edu/pasi2011/library/biegler/hicks.pdf

[julia-users] Re: Optimal Control using Dynamic Programming

[cdm]

you may find the Julia interface to apmonitor useful:

https://groups.google.com/forum/#!searchin/julia-opt/apmonitor/julia-opt/y5sc_xtLJfk/PKNBik_A82AJ


good luck,

cdm


On Thursday, September 24, 2015 at 10:25:19 PM UTC-7, NV wrote:
>
> Hi,
>   I am new to Julia. I need to solve an optimal control problem using 
> dynamic programming. Are there pre-defined functions/packages that I could 
> use?
>
> Thanks.
>

Re: [julia-users] Re: Optimal Control using Dynamic Programming

[Narayani Vedam]

I have 3 states and two inputs. State 1 is [10:1:20], state 2 is
[85:0.2:95] and state 3 is [1:1:12]. The two inputs are [0 1] and [-1 0].
On Sep 25, 2015 1:00 PM, "Tony Kelman"  wrote:

> If you want an uninitialized 61-by-61 array of double precision floating
> point numbers you can do Array(61, 61)
>
> If you want the array initialized to zero you can call zeros(61, 61)
>
> A few for loops should accomplish what you're after here. Depending how
> expenaive your cost function is to evaluate this doesn't sound very big at
> all. How many state and input dimensions do you have?

Re: [julia-users] Re: Optimal Control using Dynamic Programming

[Tony Kelman]

If you want an uninitialized 61-by-61 array of double precision floating point 
numbers you can do Array(61, 61)

If you want the array initialized to zero you can call zeros(61, 61)

A few for loops should accomplish what you're after here. Depending how 
expenaive your cost function is to evaluate this doesn't sound very big at all. 
How many state and input dimensions do you have?

Re: [julia-users] Re: Optimal Control using Dynamic Programming

[Narayani Vedam]

Thank you for your inputs. My concern was about calculating the cost to go
map. I have discretized the state space and have 61 possible states in each
stages and there are 20 such stages.

That roughly amounts to 61*61 values for one stage. I am overwhelmed by the
amount of computations that this would mean and how best I could initialize
a suitable array and index it for later use.
On Sep 25, 2015 8:30 AM, "Tony Kelman"  wrote:

> I'm biased, but for control problems with constraints I would pose it as a
> model predictive control problem and use JuMP.jl with the Ipopt.jl
> optimization solver. If you prefer to construct a cost to go map and do
> dynamic programming then I can't point you to any Julia code that I know of
> that solves your exact problem, though there may be some similar matlab
> code posted somewhere that you could translate. Without a complete
> mathematical or code description of your problem you're being too vague for
> this forum to help much.

[julia-users] Re: Optimal Control using Dynamic Programming

[Tony Kelman]

I'm biased, but for control problems with constraints I would pose it as a 
model predictive control problem and use JuMP.jl with the Ipopt.jl optimization 
solver. If you prefer to construct a cost to go map and do dynamic programming 
then I can't point you to any Julia code that I know of that solves your exact 
problem, though there may be some similar matlab code posted somewhere that you 
could translate. Without a complete mathematical or code description of your 
problem you're being too vague for this forum to help much.

[julia-users] Re: Optimal Control using Dynamic Programming

[NV]

Hi,
   I am trying to control the velocity of a system optimally. I have a cost 
function (DP formulation) to minimize subject to a few constraints.
Thanks

On Friday, 25 September 2015 00:25:19 UTC-5, NV wrote:
>
> Hi,
>   I am new to Julia. I need to solve an optimal control problem using 
> dynamic programming. Are there pre-defined functions/packages that I could 
> use?
>
> Thanks.
>

[julia-users] Re: Optimal Control using Dynamic Programming

[Narayani Vedam]

The system is non-linear. I have a cost function, that I need to minimize 
to find the optimal control to move from one state to another. There are a 
few constraints that the states should satisfy even.

On Friday, 25 September 2015 01:20:52 UTC-5, Uwe Fechner wrote:
>
> Hello,
> as far as I know, there is no ready-made package for optimal control.
> There is one package on control design:
> https://github.com/JuliaControl/Control.jl
>
> And there are a lot of packages for solving optimization problems.
>
> Could you be a little bit more specific about your problem?
>
> How do you want to describe the system, that you want to control?
> Is it a linear system?
>
> Uwe
>
> Am Freitag, 25. September 2015 07:25:19 UTC+2 schrieb Narayani Vedam:
>>
>> Hi,
>>   I am new to Julia. I need to solve an optimal control problem using 
>> dynamic programming. Are there pre-defined functions/packages that I could 
>> use?
>>
>> Thanks.
>>
>

[julia-users] Re: Optimal Control using Dynamic Programming

[Tony Kelman]

The closest pre-made package for doing dynamic programming for optimal 
control would likely be one of the quantitative economics packages. If you 
have discrete decisions or want to grid your input space, it should be 
fairly straightforward to write out a cost-to-go matrix and apply the 
Bellman equation iteratively.



On Thursday, September 24, 2015 at 10:25:19 PM UTC-7, NV wrote:
>
> Hi,
>   I am new to Julia. I need to solve an optimal control problem using 
> dynamic programming. Are there pre-defined functions/packages that I could 
> use?
>
> Thanks.
>

[julia-users] Re: Optimal Control using Dynamic Programming

[Uwe Fechner]

Hello,
as far as I know, there is no ready-made package for optimal control.
There is one package on control design:
https://github.com/JuliaControl/Control.jl

And there are a lot of packages for solving optimization problems.

Could you be a little bit more specific about your problem?

How do you want to describe the system, that you want to control?
Is it a linear system?

Uwe

Am Freitag, 25. September 2015 07:25:19 UTC+2 schrieb Narayani Vedam:
>
> Hi,
>   I am new to Julia. I need to solve an optimal control problem using 
> dynamic programming. Are there pre-defined functions/packages that I could 
> use?
>
> Thanks.
>