[openstack-dev] [Nova] Migration state machine proposal.

[Tang Chen]

Hi, all,

Please help to review this BP.

https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine


Currently, the migration_status field in Migration object is indicating the
status of migration process. But in the current code, it is represented
by pure string, like 'migrating', 'finished', and so on.

The strings could be confusing to different developers, e.g. there are 3
statuses representing the migration process is over successfully:
'finished', 'completed' and 'done'.
And 2 for migration in process: 'running' and 'migrating'.

So I think we should use constants or enum for these statuses.


Furthermore, Nikola has proposed to create a state machine for the 
statuses,
which is part of another abandoned BP. And this is also the work I'd 
like to go

on with. Please refer to:
https://review.openstack.org/#/c/197668/ 

https://review.openstack.org/#/c/197669/ 




Another proposal is: introduce a new member named "state" into Migration.
Use a state machine to handle this Migration.state, and leave 
migration_status

field a descriptive human readable free-form.


So how do you think ?

Thanks.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Zhenyu Zheng]

I think it will be better if you can submit a spec for your proposal, it
will be easier for people to give comment.

On Wed, Oct 14, 2015 at 10:05 AM, Tang Chen  wrote:

> Hi, all,
>
> Please help to review this BP.
>
> https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine
>
>
> Currently, the migration_status field in Migration object is indicating
> the
> status of migration process. But in the current code, it is represented
> by pure string, like 'migrating', 'finished', and so on.
>
> The strings could be confusing to different developers, e.g. there are 3
> statuses representing the migration process is over successfully:
> 'finished', 'completed' and 'done'.
> And 2 for migration in process: 'running' and 'migrating'.
>
> So I think we should use constants or enum for these statuses.
>
>
> Furthermore, Nikola has proposed to create a state machine for the
> statuses,
> which is part of another abandoned BP. And this is also the work I'd like
> to go
> on with. Please refer to:
> https://review.openstack.org/#/c/197668/
> https://review.openstack.org/#/c/197669/
>
>
> Another proposal is: introduce a new member named "state" into Migration.
> Use a state machine to handle this Migration.state, and leave
> migration_status
> field a descriptive human readable free-form.
>
>
> So how do you think ?
>
> Thanks.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Tang Chen]

On 10/14/2015 11:14 AM, Zhenyu Zheng wrote:
I think it will be better if you can submit a spec for your proposal, 
it will be easier for people to give comment.


OK, will submit one soon.

Thanks.



On Wed, Oct 14, 2015 at 10:05 AM, Tang Chen > wrote:


Hi, all,

Please help to review this BP.

https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine


Currently, the migration_status field in Migration object is
indicating the
status of migration process. But in the current code, it is
represented
by pure string, like 'migrating', 'finished', and so on.

The strings could be confusing to different developers, e.g. there
are 3
statuses representing the migration process is over successfully:
'finished', 'completed' and 'done'.
And 2 for migration in process: 'running' and 'migrating'.

So I think we should use constants or enum for these statuses.


Furthermore, Nikola has proposed to create a state machine for the
statuses,
which is part of another abandoned BP. And this is also the work
I'd like to go
on with. Please refer to:
https://review.openstack.org/#/c/197668/
https://review.openstack.org/#/c/197669/


Another proposal is: introduce a new member named "state" into
Migration.
Use a state machine to handle this Migration.state, and leave
migration_status
field a descriptive human readable free-form.


So how do you think ?

Thanks.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Nikola Đipanov]

On 10/14/2015 04:29 AM, Tang Chen wrote:
>>
>> On Wed, Oct 14, 2015 at 10:05 AM, Tang Chen > > wrote:
>>
>> Hi, all,
>>
>> Please help to review this BP.
>>
>> https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine
>>
>>
>> Currently, the migration_status field in Migration object is
>> indicating the
>> status of migration process. But in the current code, it is
>> represented
>> by pure string, like 'migrating', 'finished', and so on.
>>
>> The strings could be confusing to different developers, e.g. there
>> are 3
>> statuses representing the migration process is over successfully:
>> 'finished', 'completed' and 'done'.
>> And 2 for migration in process: 'running' and 'migrating'.
>>
>> So I think we should use constants or enum for these statuses.
>>
>>
>> Furthermore, Nikola has proposed to create a state machine for the
>> statuses,
>> which is part of another abandoned BP. And this is also the work
>> I'd like to go
>> on with. Please refer to:
>> https://review.openstack.org/#/c/197668/
>> https://review.openstack.org/#/c/197669/
>>

This is IMHO a worthwhile effort on it's own. I'd like to see it use a
defined state machine in addition to being a simple enum so that
transitions are clearly defined as well.

>>
>> Another proposal is: introduce a new member named "state" into
>> Migration.
>> Use a state machine to handle this Migration.state, and leave
>> migration_status
>> field a descriptive human readable free-form.
>>

This is a separate effort IMHO - we should do both if possible.

>
> On 10/14/2015 11:14 AM, Zhenyu Zheng wrote:
>> I think it will be better if you can submit a spec for your proposal,
>> it will be easier for people to give comment.
>
> OK, will submit one soon.

If you plan to just enumerate the possible states - that should not
require a spec. Adding automaton in the mix, and especially adding a new
'state' field probably does deserve some discussion so in that case feel
free to write up a spec.

N.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Tang Chen]

On 10/14/2015 04:53 PM, Nikola Đipanov wrote:

On 10/14/2015 04:29 AM, Tang Chen wrote:

On Wed, Oct 14, 2015 at 10:05 AM, Tang Chen mailto:tangc...@cn.fujitsu.com>> wrote:

 Hi, all,

 Please help to review this BP.

 https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine


 Currently, the migration_status field in Migration object is
 indicating the
 status of migration process. But in the current code, it is
 represented
 by pure string, like 'migrating', 'finished', and so on.

 The strings could be confusing to different developers, e.g. there
 are 3
 statuses representing the migration process is over successfully:
 'finished', 'completed' and 'done'.
 And 2 for migration in process: 'running' and 'migrating'.

 So I think we should use constants or enum for these statuses.


 Furthermore, Nikola has proposed to create a state machine for the
 statuses,
 which is part of another abandoned BP. And this is also the work
 I'd like to go
 on with. Please refer to:
 https://review.openstack.org/#/c/197668/
 https://review.openstack.org/#/c/197669/


This is IMHO a worthwhile effort on it's own. I'd like to see it use a
defined state machine in addition to being a simple enum so that
transitions are clearly defined as well.


Agreed.


 Another proposal is: introduce a new member named "state" into
 Migration.
 Use a state machine to handle this Migration.state, and leave
 migration_status
 field a descriptive human readable free-form.


This is a separate effort IMHO - we should do both if possible.


Yes, I do agree.

And I think migrate_status and migrate_state fields could share the
same state machine if we do both.




On 10/14/2015 11:14 AM, Zhenyu Zheng wrote:

I think it will be better if you can submit a spec for your proposal,
it will be easier for people to give comment.

OK, will submit one soon.

If you plan to just enumerate the possible states - that should not
require a spec. Adding automaton in the mix, and especially adding a new
'state' field probably does deserve some discussion so in that case feel
free to write up a spec.


No, I planed to introduce a state machine for migrate_status field first.
And this needs a spec, I think.

And then we will go on discussing the new state field things.

Thanks.



N.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Tang Chen]

Hi all,

The spec is now available here:
https://review.openstack.org/#/c/235169/

Please help to review.

Thanks.

On 10/14/2015 10:05 AM, Tang Chen wrote:

Hi, all,

Please help to review this BP.

https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine


Currently, the migration_status field in Migration object is 
indicating the

status of migration process. But in the current code, it is represented
by pure string, like 'migrating', 'finished', and so on.

The strings could be confusing to different developers, e.g. there are 3
statuses representing the migration process is over successfully:
'finished', 'completed' and 'done'.
And 2 for migration in process: 'running' and 'migrating'.

So I think we should use constants or enum for these statuses.


Furthermore, Nikola has proposed to create a state machine for the 
statuses,
which is part of another abandoned BP. And this is also the work I'd 
like to go

on with. Please refer to:
https://review.openstack.org/#/c/197668/ 

https://review.openstack.org/#/c/197669/ 




Another proposal is: introduce a new member named "state" into Migration.
Use a state machine to handle this Migration.state, and leave 
migration_status

field a descriptive human readable free-form.


So how do you think ?

Thanks.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Tang Chen]

Hi, all,

If you don't mind, how about approve the BP, and I can start this work.

Thanks.


On 10/15/2015 04:53 PM, Tang Chen wrote:

Hi all,

The spec is now available here:
https://review.openstack.org/#/c/235169/

Please help to review.

Thanks.

On 10/14/2015 10:05 AM, Tang Chen wrote:

Hi, all,

Please help to review this BP.

https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine


Currently, the migration_status field in Migration object is 
indicating the

status of migration process. But in the current code, it is represented
by pure string, like 'migrating', 'finished', and so on.

The strings could be confusing to different developers, e.g. there are 3
statuses representing the migration process is over successfully:
'finished', 'completed' and 'done'.
And 2 for migration in process: 'running' and 'migrating'.

So I think we should use constants or enum for these statuses.


Furthermore, Nikola has proposed to create a state machine for the 
statuses,
which is part of another abandoned BP. And this is also the work I'd 
like to go

on with. Please refer to:
https://review.openstack.org/#/c/197668/ 

https://review.openstack.org/#/c/197669/ 




Another proposal is: introduce a new member named "state" into Migration.
Use a state machine to handle this Migration.state, and leave 
migration_status

field a descriptive human readable free-form.


So how do you think ?

Thanks.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Nikola Đipanov]

On 10/19/2015 11:13 AM, Tang Chen wrote:
> Hi, all,
> 
> If you don't mind, how about approve the BP, and I can start this work.
> 

This is IMHO the biggest drawback of the current spec process (as I've
written before).

There is no reason why you should doubt that this particular spec will
get approved, yet due to the combination of extremely limited review
bandwidth and very aggressive deadlines, there is a good chance your
spec will miss Mitaka release purely on process grounds.

This makes you even further dissuaded from actually putting in the
development effort.

N.

> Thanks. 
> 
> 
> On 10/15/2015 04:53 PM, Tang Chen wrote:
>> Hi all,
>>
>> The spec is now available here:
>> https://review.openstack.org/#/c/235169/
>>
>> Please help to review.
>>
>> Thanks.
>>
>> On 10/14/2015 10:05 AM, Tang Chen wrote:
>>> Hi, all,
>>>
>>> Please help to review this BP.
>>>
>>> https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine
>>>
>>>
>>> Currently, the migration_status field in Migration object is
>>> indicating the
>>> status of migration process. But in the current code, it is represented
>>> by pure string, like 'migrating', 'finished', and so on.
>>>
>>> The strings could be confusing to different developers, e.g. there are 3
>>> statuses representing the migration process is over successfully:
>>> 'finished', 'completed' and 'done'.
>>> And 2 for migration in process: 'running' and 'migrating'.
>>>
>>> So I think we should use constants or enum for these statuses.
>>>
>>>
>>> Furthermore, Nikola has proposed to create a state machine for the
>>> statuses,
>>> which is part of another abandoned BP. And this is also the work I'd
>>> like to go
>>> on with. Please refer to:
>>> https://review.openstack.org/#/c/197668/
>>> 
>>> https://review.openstack.org/#/c/197669/
>>> 
>>>
>>>
>>> Another proposal is: introduce a new member named "state" into Migration.
>>> Use a state machine to handle this Migration.state, and leave
>>> migration_status
>>> field a descriptive human readable free-form.
>>>
>>>
>>> So how do you think ?
>>>
>>> Thanks.
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Tang Chen]

Hi Nikola,

On 10/19/2015 06:52 PM, Nikola Đipanov wrote:

On 10/19/2015 11:13 AM, Tang Chen wrote:

Hi, all,

If you don't mind, how about approve the BP, and I can start this work.


This is IMHO the biggest drawback of the current spec process (as I've
written before).

There is no reason why you should doubt that this particular spec will
get approved, yet due to the combination of extremely limited review
bandwidth and very aggressive deadlines, there is a good chance your
spec will miss Mitaka release purely on process grounds.

This makes you even further dissuaded from actually putting in the
development effort.


Sorry, I didn't realize the limited review bandwidth. Actually I don't
have a deadline of this.

And OK, let's keep its status so that more people could review it,
and minimize the waste of development effort.

Thanks.



N.


Thanks.


On 10/15/2015 04:53 PM, Tang Chen wrote:

Hi all,

The spec is now available here:
https://review.openstack.org/#/c/235169/

Please help to review.

Thanks.

On 10/14/2015 10:05 AM, Tang Chen wrote:

Hi, all,

Please help to review this BP.

https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine


Currently, the migration_status field in Migration object is
indicating the
status of migration process. But in the current code, it is represented
by pure string, like 'migrating', 'finished', and so on.

The strings could be confusing to different developers, e.g. there are 3
statuses representing the migration process is over successfully:
'finished', 'completed' and 'done'.
And 2 for migration in process: 'running' and 'migrating'.

So I think we should use constants or enum for these statuses.


Furthermore, Nikola has proposed to create a state machine for the
statuses,
which is part of another abandoned BP. And this is also the work I'd
like to go
on with. Please refer to:
https://review.openstack.org/#/c/197668/

https://review.openstack.org/#/c/197669/



Another proposal is: introduce a new member named "state" into Migration.
Use a state machine to handle this Migration.state, and leave
migration_status
field a descriptive human readable free-form.


So how do you think ?

Thanks.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
.




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Tang Chen]

Hi,

Please help to take a look at this problem. I was trying to raise it in 
the spec discussion.
But since we don't need a spec on this problem, so I want to discuss it 
here.

It is about what the new state machine will be.

http://paste.openstack.org/show/476954/

Thanks.




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Joshua Harlow]

Question on some things seen in the below paste.

What is with 'finished' -> 'reverted' and 'finished' -> 'confirmed'?

Why does it jump over 'reverting' or 'confirming'? Should it?

The other question is the difference between 'failed' and 'error' in the 
first diagram, any idea on why/how these are semantically different? The 
difference between 'done' and 'finished' are also in my mind 
semantically confusing.


Overall I'm very much inclined to have three state machines (one for 
each type), vs the mix-mash of all three into one state machine (which 
causes the confusion around states in the first diagram in that paste).


Josh

Tang Chen wrote:

Hi,

Please help to take a look at this problem. I was trying to raise it in
the spec discussion.
But since we don't need a spec on this problem, so I want to discuss it
here.
It is about what the new state machine will be.

http://paste.openstack.org/show/476954/

Thanks.




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Tang Chen]

Hi Joshua,

On 10/22/2015 05:17 AM, Joshua Harlow wrote:

Question on some things seen in the below paste.

What is with 'finished' -> 'reverted' and 'finished' -> 'confirmed'?


Let's take confirm as an example. Please refer to function confirm_resize().
Resize is one type of migration in the current implement. It needs some
confirmation, and should destroy the source VM after the migration finished.

I guess the 'finished' state means just the migration is finished. Not 
the whole

resize process. Just my guess.



Why does it jump over 'reverting' or 'confirming'? Should it?


Not sure. I think it is just the original design. The confirmation and 
destroying

of the source VM may take some time. So it needs a state to indicate the
operation is on going.



The other question is the difference between 'failed' and 'error' in 
the first diagram, any idea on why/how these are semantically 
different? The difference between 'done' and 'finished' are also in my 
mind semantically confusing.


Yes, they are confusing. That is also why I raised this question.

About 'failed' and 'error', by reading the code, I think 'error' means 
we caught
some known or defined exceptions, and we know how to handle it. 'failed' 
means

unexpected exceptions happened.

And 'finished', 'completed', 'done' are just used by different types of 
migrations.

I cannot tell the difference. So I suggest to unify them.



Overall I'm very much inclined to have three state machines (one for 
each type), vs the mix-mash of all three into one state machine (which 
causes the confusion around states in the first diagram in that paste).


That is an idea. But I would prefer to have one single state machine for 
migration,
because resize and evacuate are reusing migration. They can be in one 
state machine.


It would be very helpful if the designer of the migration process could 
share his idea.
But if it is just some code modified by many people many times, I think 
we should

remove the confusing states and give a easier, better state machine.

Thanks.



Josh

Tang Chen wrote:

Hi,

Please help to take a look at this problem. I was trying to raise it in
the spec discussion.
But since we don't need a spec on this problem, so I want to discuss it
here.
It is about what the new state machine will be.

http://paste.openstack.org/show/476954/

Thanks.




__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Nikola Đipanov]

On 10/21/2015 10:17 PM, Joshua Harlow wrote:
> Question on some things seen in the below paste.
> 
> What is with 'finished' -> 'reverted' and 'finished' -> 'confirmed'?
> 
> Why does it jump over 'reverting' or 'confirming'? Should it?
> 
> The other question is the difference between 'failed' and 'error' in the
> first diagram, any idea on why/how these are semantically different? The
> difference between 'done' and 'finished' are also in my mind
> semantically confusing.
> 
> Overall I'm very much inclined to have three state machines (one for
> each type), vs the mix-mash of all three into one state machine (which
> causes the confusion around states in the first diagram in that paste).
> 

So the problem here is that they (as you point out) grew organically,
and we are exposing these through the API. We need to keep them, and I
see this BP as simply documenting them with automaton thrown in for it's
validation and documenting features.

So - we _do not_ want to change these. Think of them as information for
human consumption.

What we may want to do is add an additional field (called state instead
of status maybe), that we can use to re-boot states, and define better
state machines that are easier to write tooling against. This is a
separate effort, that will surely need a spec and a discussion to get
the states right.

That's what we (or at least I) were talking about.
N.

> Josh
> 
> Tang Chen wrote:
>> Hi,
>>
>> Please help to take a look at this problem. I was trying to raise it in
>> the spec discussion.
>> But since we don't need a spec on this problem, so I want to discuss it
>> here.
>> It is about what the new state machine will be.
>>
>> http://paste.openstack.org/show/476954/
>>
>> Thanks.
>>
>>
>>
>>
>> __
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Markus Zoeller]

Tang Chen  wrote on 10/21/2015 10:22:15 AM:

> From: Tang Chen 
> To: "OpenStack Development Mailing List (not for usage questions)" 
> 
> Date: 10/21/2015 10:24 AM
> Subject: Re: [openstack-dev] [Nova] Migration state machine proposal.
> 
> Hi,
> 
> Please help to take a look at this problem. I was trying to raise it in 
> the spec discussion.
> But since we don't need a spec on this problem, so I want to discuss it 
> here.
> It is about what the new state machine will be.
> 
> http://paste.openstack.org/show/476954/
> 
> Thanks.


I'd like to make you aware of bp "split-different-live-migration-types":
https://review.openstack.org/#/c/225910/
It intends to split those 3 types you are talking about. This could have
some overlap to your work.

Regards, Markus Zoeller (markus_z)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[John Garbutt]

On 22 October 2015 at 10:20, Nikola Đipanov  wrote:
> On 10/21/2015 10:17 PM, Joshua Harlow wrote:
>> Question on some things seen in the below paste.
>>
>> What is with 'finished' -> 'reverted' and 'finished' -> 'confirmed'?
>>
>> Why does it jump over 'reverting' or 'confirming'? Should it?
>>
>> The other question is the difference between 'failed' and 'error' in the
>> first diagram, any idea on why/how these are semantically different? The
>> difference between 'done' and 'finished' are also in my mind
>> semantically confusing.
>>
>> Overall I'm very much inclined to have three state machines (one for
>> each type), vs the mix-mash of all three into one state machine (which
>> causes the confusion around states in the first diagram in that paste).
>>
>
> So the problem here is that they (as you point out) grew organically,
> and we are exposing these through the API. We need to keep them, and I
> see this BP as simply documenting them with automaton thrown in for it's
> validation and documenting features.

+1
That feels like step one.

> So - we _do not_ want to change these. Think of them as information for
> human consumption.
>
> What we may want to do is add an additional field (called state instead
> of status maybe), that we can use to re-boot states, and define better
> state machines that are easier to write tooling against. This is a
> separate effort, that will surely need a spec and a discussion to get
> the states right.

+1
Honestly, this feels very related to the Task related efforts.

Thanks,
John


>> Josh
>>
>> Tang Chen wrote:
>>> Hi,
>>>
>>> Please help to take a look at this problem. I was trying to raise it in
>>> the spec discussion.
>>> But since we don't need a spec on this problem, so I want to discuss it
>>> here.
>>> It is about what the new state machine will be.
>>>
>>> http://paste.openstack.org/show/476954/
>>>
>>> Thanks.
>>>
>>>
>>>
>>>
>>> __
>>>
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Joshua Harlow]

Nikola Đipanov wrote:

On 10/21/2015 10:17 PM, Joshua Harlow wrote:

Question on some things seen in the below paste.

What is with 'finished' ->  'reverted' and 'finished' ->  'confirmed'?

Why does it jump over 'reverting' or 'confirming'? Should it?

The other question is the difference between 'failed' and 'error' in the
first diagram, any idea on why/how these are semantically different? The
difference between 'done' and 'finished' are also in my mind
semantically confusing.

Overall I'm very much inclined to have three state machines (one for
each type), vs the mix-mash of all three into one state machine (which
causes the confusion around states in the first diagram in that paste).



So the problem here is that they (as you point out) grew organically,
and we are exposing these through the API. We need to keep them, and I
see this BP as simply documenting them with automaton thrown in for it's
validation and documenting features.

So - we _do not_ want to change these. Think of them as information for
human consumption.


Sure, I'm all for not changing them (right now), or changing them after 
we have them explicitly defined and can easily understand what they are 
(vs right now where it is implicit and not easily understandable); one 
step at a time I guess :)




What we may want to do is add an additional field (called state instead
of status maybe), that we can use to re-boot states, and define better
state machines that are easier to write tooling against. This is a
separate effort, that will surely need a spec and a discussion to get
the states right.

That's what we (or at least I) were talking about.
N.


Josh

Tang Chen wrote:

Hi,

Please help to take a look at this problem. I was trying to raise it in
the spec discussion.
But since we don't need a spec on this problem, so I want to discuss it
here.
It is about what the new state machine will be.

http://paste.openstack.org/show/476954/

Thanks.




__

OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Tang Chen]

On 10/22/2015 05:42 PM, Markus Zoeller wrote:

Tang Chen  wrote on 10/21/2015 10:22:15 AM:


From: Tang Chen 
To: "OpenStack Development Mailing List (not for usage questions)"

Date: 10/21/2015 10:24 AM
Subject: Re: [openstack-dev] [Nova] Migration state machine proposal.

Hi,

Please help to take a look at this problem. I was trying to raise it in
the spec discussion.
But since we don't need a spec on this problem, so I want to discuss it
here.
It is about what the new state machine will be.

http://paste.openstack.org/show/476954/

Thanks.


I'd like to make you aware of bp "split-different-live-migration-types":
https://review.openstack.org/#/c/225910/
It intends to split those 3 types you are talking about. This could have
some overlap to your work.


Hi Markus ,

Thank you for the info. Will look at it in detail.

And BTW, the direction came to this:

1. Keep the Migration.status field as it is now. This is because these 
statuses

have been exposed through APIs. Just use Enum to ensure the values are
valid. And introduce a state machine for it. This is just a small 
improvement.


2. The real improvement is here, will be in another BP. Introduce a new
Migration.state field, and a new state machine to fix the problems 
in the

current implementation, I mean remove redundant statuses, and make
the state machine more reasonable.

Thanks.




Regards, Markus Zoeller (markus_z)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Jay Pipes]

On 10/22/2015 11:13 AM, Tang Chen wrote:

On 10/22/2015 05:17 AM, Joshua Harlow wrote:

Overall I'm very much inclined to have three state machines (one
for each type), vs the mix-mash of all three into one state machine
(which causes the confusion around states in the first diagram in
that paste).


That is an idea. But I would prefer to have one single state machine
for migration, because resize and evacuate are reusing migration.
They can be in one state machine.


Evacuate does *not* migrate/move anything. Evacuate *rebuilds* VMs from 
their original source image.


I support Nikola in that I believe the different migration types should 
have different state machines entirely (but be as consistent as possible 
in the naming of terminal states like "finished" vs "done" etc)



It would be very helpful if the designer of the migration process
could share his idea. But if it is just some code modified by many
people many times, I think we should remove the confusing states and
give a easier, better state machine.


There isn't a designer of the migration process :( The original (crap, 
IMHO) API from Rackspace Cloud Servers API was used for the resize 
functionality in the compute API and it's been a source of confusion and 
frustration ever since. Relying on a manual confirmation or revert input 
from the user was and continues to be a horrible idea.


I believe strongly that we should deprecate the existing migrate, 
resize, an live-migrate APIs in favor of a single consolidated, 
consistent "move" REST API that would have the following characteristics:


* No manual or wait-input states in any FSM graph
* Removal of the term "resize" from the API entirely (the target 
resource sizing is an attribute of the move operation, not a different 
type of API operation in and of itself)
* Transition to a task-based API for poll-state requests. This means 
that in order for a caller to determine the state of a VM the caller 
would call something like GET /servers//tasks/ in order to 
see the history of state changes or subtask operations for a particular 
request to move a VM


Timofei Durakov (cc'd) has a blueprint for splitting the live-migration 
types into separate task classes here:


https://review.openstack.org/#/c/225910/

I think there's a lot of good ideas in that proposal. Please do have a 
look at it.


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Chris Friesen]

On 10/26/2015 06:02 PM, Jay Pipes wrote:


I believe strongly that we should deprecate the existing migrate, resize, an
live-migrate APIs in favor of a single consolidated, consistent "move" REST API
that would have the following characteristics:

* No manual or wait-input states in any FSM graph


Sounds good.


* Removal of the term "resize" from the API entirely (the target resource sizing
is an attribute of the move operation, not a different type of API operation in
and of itself)


I disagree on this one.

As an end-user, if my goal is to take an existing instance and give it bigger 
disks, my first instinct isn't going to be to look at the "move" operation.  I'm 
going to look for "scale", or "resize", or something like that.


And if an admin wants to migrate an instance away from its current host, why 
would they want to change its disk size in the process?


I do think it makes sense to combine the external APIs for live and cold 
migration.  Those two are fundamentally similar, logically separated only by 
whether the instance stays running or not.


And I'm perfectly fine with having the internal implementation of all three 
share a code path, I just don't think it makes sense for the *external* API.



* Transition to a task-based API for poll-state requests. This means that in
order for a caller to determine the state of a VM the caller would call
something like GET /servers//tasks/ in order to see the history of
state changes or subtask operations for a particular request to move a VM

enstack.org/cgi-bin/mailman/listinfo/openstack-dev

Sounds good.

Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Tang Chen]

Hi Jay, Timofei,

Thank you for the info.

On 10/27/2015 08:02 AM, Jay Pipes wrote:

On 10/22/2015 11:13 AM, Tang Chen wrote:

On 10/22/2015 05:17 AM, Joshua Harlow wrote:

Overall I'm very much inclined to have three state machines (one
for each type), vs the mix-mash of all three into one state machine
(which causes the confusion around states in the first diagram in
that paste).


That is an idea. But I would prefer to have one single state machine
for migration, because resize and evacuate are reusing migration.
They can be in one state machine.


Evacuate does *not* migrate/move anything. Evacuate *rebuilds* VMs 
from their original source image.


Well, I just dug into the source code. I think there could be some 
difference between evacuate in nova server side and client side. In nova 
compute, the evacuate API does call rebuild process as you said. But in 
novaclient, there is a command "nova host-evacuate-live", which will 
live-migrate all running VMs, which made me believe that evacuate also 
migrates VMs. Please refer to:


https://github.com/openstack/python-novaclient/blob/master/novaclient/v2/contrib/host_evacuate_live.py#L72

I think this is also a reason why I always got confused in all these 
concepts: cold-migrate, evacuate, evacuate-live, rebuild, resize.



About the migration type, I can see that Timofei has tried to split 
live-migration into 3 types:

1. block_live_migrate
2. live_migrate_file_level_storage
3. live_migrate_block_stroage

I think it is in driver level, not the user level. It is based on the 
type of the storage the VM is using. And I think migration type should 
be a multi-level thing.


Since I'm still a little confused with all the types of migration, I'd 
like to share some of my understanding and if they are correct, I think 
we can improve it like this.


1. OpenStack is now supporting resize a VM to another compute node. If 
we set "allow_resize_to_same-host", it also supports local resize. If we 
are not using memory/CPU hotplug, resize will result in a shutdown and 
reconfiguration of VM.
So, there should be 2 types of resize: live (using hotplug) and cold 
(often resizing the primary disk).


2. Evacuate also has 2 types: live (equals to live-migrate) and cold 
(rebuild). But evacuate itself does nothing, I mean there is no actual 
process called evacuate. evacuate() is just an API calling 
rebuild_instance().


This is from the user level.

So finally, the migration type would be like this:

  user computedriver

  live-migrate
  live-evacuate live-migrate
  live-resize  memory/CPU hotplug

  cold-migrate   storage type, etc
  clod-evacuate   cold-migrate
  cold-resize  (to self or not)

rebuild   rebuild
  (this is not a migration)

I mean maybe we should handle different things in different levels. In 
compute, if the flow is too complex, we can define some more helper 
functions to make the main flow easier to understand.




I support Nikola in that I believe the different migration types 
should have different state machines entirely (but be as consistent as 
possible in the naming of terminal states like "finished" vs "done" etc)


OK. Agreed. And maybe also introduce state machines for task_state and 
vm_state.





It would be very helpful if the designer of the migration process
could share his idea. But if it is just some code modified by many
people many times, I think we should remove the confusing states and
give a easier, better state machine.


There isn't a designer of the migration process :( The original (crap, 
IMHO) API from Rackspace Cloud Servers API was used for the resize 
functionality in the compute API and it's been a source of confusion 
and frustration ever since. Relying on a manual confirmation or revert 
input from the user was and continues to be a horrible idea.


Agreed.



I believe strongly that we should deprecate the existing migrate, 
resize, an live-migrate APIs in favor of a single consolidated, 
consistent "move" REST API that would have the following characteristics:


* No manual or wait-input states in any FSM graph


Yes.

* Removal of the term "resize" from the API entirely (the target 
resource sizing is an attribute of the move operation, not a different 
type of API operation in and of itself)


Maybe we can define it in a different level, as I said above. Not sure.

* Transition to a task-based API for poll-state requests. This means 
that in order for a caller to determine the state of a VM the caller 
would call something like GET /servers//tasks/ in order to 
see the history of state changes or subtask operations for a 
particular request to move a VM


Yes.



Timofei Durakov (cc'd) has a blueprint for splitting the 
live-migration types into separate task classes here:


https://review.openstack.org/#/c/225910/

I think

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Tang Chen]

Hi Chris,

On 10/27/2015 12:16 PM, Chris Friesen wrote:

On 10/26/2015 06:02 PM, Jay Pipes wrote:

I believe strongly that we should deprecate the existing migrate, 
resize, an
live-migrate APIs in favor of a single consolidated, consistent 
"move" REST API

that would have the following characteristics:

* No manual or wait-input states in any FSM graph


Sounds good.

* Removal of the term "resize" from the API entirely (the target 
resource sizing
is an attribute of the move operation, not a different type of API 
operation in

and of itself)


I disagree on this one.

As an end-user, if my goal is to take an existing instance and give it 
bigger disks, my first instinct isn't going to be to look at the 
"move" operation.  I'm going to look for "scale", or "resize", or 
something like that.


And if an admin wants to migrate an instance away from its current 
host, why would they want to change its disk size in the process?


This is also where I got confused before. So please refer to my previous 
mail. I think to user, nova service, and driver, they are in 3 different 
levels, and migration type may mean different things to them.




I do think it makes sense to combine the external APIs for live and 
cold migration.  Those two are fundamentally similar, logically 
separated only by whether the instance stays running or not.


Yes, this is in nova service level.

Thanks.



And I'm perfectly fine with having the internal implementation of all 
three share a code path, I just don't think it makes sense for the 
*external* API.


* Transition to a task-based API for poll-state requests. This means 
that in

order for a caller to determine the state of a VM the caller would call
something like GET /servers//tasks/ in order to see the 
history of
state changes or subtask operations for a particular request to move 
a VM

enstack.org/cgi-bin/mailman/listinfo/openstack-dev

Sounds good.

Chris

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Jay Pipes]

On 10/27/2015 05:27 PM, Tang Chen wrote:

On 10/27/2015 08:02 AM, Jay Pipes wrote:

Evacuate does *not* migrate/move anything. Evacuate *rebuilds* VMs
from their original source image.


Well, I just dug into the source code. I think there could be some
difference between evacuate in nova server side and client side. In nova
compute, the evacuate API does call rebuild process as you said. But in
novaclient, there is a command "nova host-evacuate-live", which will
live-migrate all running VMs, which made me believe that evacuate also
migrates VMs. Please refer to:

https://github.com/openstack/python-novaclient/blob/master/novaclient/v2/contrib/host_evacuate_live.py#L72

I think this is also a reason why I always got confused in all these
concepts: cold-migrate, evacuate, evacuate-live, rebuild, resize.


Yes, this is a most unfortunate issue :( host-evacuate in the 
python-novaclient is not related at all to the evacuate server API.


We need to clean up this mess.

Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Jay Pipes]

On 10/27/2015 01:16 PM, Chris Friesen wrote:

On 10/26/2015 06:02 PM, Jay Pipes wrote:


I believe strongly that we should deprecate the existing migrate,
resize, an
live-migrate APIs in favor of a single consolidated, consistent "move"
REST API
that would have the following characteristics:

* No manual or wait-input states in any FSM graph


Sounds good.


* Removal of the term "resize" from the API entirely (the target
resource sizing
is an attribute of the move operation, not a different type of API
operation in
and of itself)


I disagree on this one.

As an end-user, if my goal is to take an existing instance and give it
bigger disks, my first instinct isn't going to be to look at the "move"
operation.  I'm going to look for "scale", or "resize", or something
like that.

And if an admin wants to migrate an instance away from its current host,
why would they want to change its disk size in the process?


A fair point. However, I think that a generic update VM API, which would 
allow changes to the resources consumed by the VM along with capabiities 
like CPU model or local disk performance (SSD) is a better way to handle 
this than a resize-specific API.


So, in other words, I'd support this:

PATCH /servers/

with some corresponding request payload that would indicate the required 
changes.



I do think it makes sense to combine the external APIs for live and cold
migration.  Those two are fundamentally similar, logically separated
only by whether the instance stays running or not.

And I'm perfectly fine with having the internal implementation of all
three share a code path, I just don't think it makes sense for the
*external* API.


I think you meant to say you don't think it makes sense to have three 
separate external APIs for what is fundamentally the same operation 
(move a VM), right?


Best,
-jay


* Transition to a task-based API for poll-state requests. This means
that in
order for a caller to determine the state of a VM the caller would call
something like GET /servers//tasks/ in order to see the
history of
state changes or subtask operations for a particular request to move a VM

enstack.org/cgi-bin/mailman/listinfo/openstack-dev

Sounds good.

Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Andrew Laski]

On 10/27/15 at 09:02am, Jay Pipes wrote:

On 10/22/2015 11:13 AM, Tang Chen wrote:

On 10/22/2015 05:17 AM, Joshua Harlow wrote:

Overall I'm very much inclined to have three state machines (one
for each type), vs the mix-mash of all three into one state machine
(which causes the confusion around states in the first diagram in
that paste).


That is an idea. But I would prefer to have one single state machine
for migration, because resize and evacuate are reusing migration.
They can be in one state machine.


Evacuate does *not* migrate/move anything. Evacuate *rebuilds* VMs 
from their original source image.


I support Nikola in that I believe the different migration types 
should have different state machines entirely (but be as consistent 
as possible in the naming of terminal states like "finished" vs 
"done" etc)


+1.

It seems that there's also some work to do to disambiguate what the 
different operations are and what they actually do.


Migrate/resize share a code path, and then there's a big switch to 
change behavior if the migration is a live-migrate.  I agree with the 
comment below that we should deprecate the resize terminology and 
consolidate cold-migrate and resize under one umbrella with one state 
machine.  Then live-migrate could have a separate state machine.


Rebuild/evacuate share a code path though evacuate involves a scheduler 
decision which is why it seems like a move.  It's actually a bit tricky 
to classify because if the instance is volume backed it is essentially 
just a move operation, if it's image backed it's a destructive rebuild.  
But I think it makes sense to not consider this a move operation and 
think of it as an administrative rebuild when a host is down.





It would be very helpful if the designer of the migration process
could share his idea. But if it is just some code modified by many
people many times, I think we should remove the confusing states and
give a easier, better state machine.


There isn't a designer of the migration process :( The original 
(crap, IMHO) API from Rackspace Cloud Servers API was used for the 
resize functionality in the compute API and it's been a source of 
confusion and frustration ever since. Relying on a manual 
confirmation or revert input from the user was and continues to be a 
horrible idea.


Agreed.  In my experience with operating a public cloud I am not aware 
of anyone benefiting from the manual checkpoint in the middle of resize.  
But we should solicit feedback from the operator community on this.




I believe strongly that we should deprecate the existing migrate, 
resize, an live-migrate APIs in favor of a single consolidated, 
consistent "move" REST API that would have the following 
characteristics:


I'm not sure that we should abstract away live vs cold migrate behind a 
single move API, but I strongly agree with consolidate cold-migrate and 
resize.




* No manual or wait-input states in any FSM graph
* Removal of the term "resize" from the API entirely (the target 
resource sizing is an attribute of the move operation, not a 
different type of API operation in and of itself)
* Transition to a task-based API for poll-state requests. This means 
that in order for a caller to determine the state of a VM the caller 
would call something like GET /servers//tasks/ in order 
to see the history of state changes or subtask operations for a 
particular request to move a VM


Huge +1 from me.



Timofei Durakov (cc'd) has a blueprint for splitting the 
live-migration types into separate task classes here:


https://review.openstack.org/#/c/225910/

I think there's a lot of good ideas in that proposal. Please do have 
a look at it.


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Tang Chen]

Hi Andrew,

On 10/29/2015 12:53 AM, Andrew Laski wrote:

On 10/27/15 at 09:02am, Jay Pipes wrote:

On 10/22/2015 11:13 AM, Tang Chen wrote:

On 10/22/2015 05:17 AM, Joshua Harlow wrote:

Overall I'm very much inclined to have three state machines (one
for each type), vs the mix-mash of all three into one state machine
(which causes the confusion around states in the first diagram in
that paste).


That is an idea. But I would prefer to have one single state machine
for migration, because resize and evacuate are reusing migration.
They can be in one state machine.


Evacuate does *not* migrate/move anything. Evacuate *rebuilds* VMs 
from their original source image.


I support Nikola in that I believe the different migration types 
should have different state machines entirely (but be as consistent 
as possible in the naming of terminal states like "finished" vs 
"done" etc)


+1.

It seems that there's also some work to do to disambiguate what the 
different operations are and what they actually do.


Migrate/resize share a code path, and then there's a big switch to 
change behavior if the migration is a live-migrate.  I agree with the 
comment below that we should deprecate the resize terminology and 
consolidate cold-migrate and resize under one umbrella with one state 
machine.  Then live-migrate could have a separate state machine.


Rebuild/evacuate share a code path though evacuate involves a 
scheduler decision which is why it seems like a move.  It's actually a 
bit tricky to classify because if the instance is volume backed it is 
essentially just a move operation, if it's image backed it's a 
destructive rebuild.  But I think it makes sense to not consider this 
a move operation and think of it as an administrative rebuild when a 
host is down.


I'm little think of that we should define each migration type clearly 
first, and then improve migration state machine. If we don't agree on 
what type represents what operation,  the state machine won't be good.


Please also help to review this BP, although the idea may be not exactly 
the same as yours.


Thanks.






It would be very helpful if the designer of the migration process
could share his idea. But if it is just some code modified by many
people many times, I think we should remove the confusing states and
give a easier, better state machine.


There isn't a designer of the migration process :( The original 
(crap, IMHO) API from Rackspace Cloud Servers API was used for the 
resize functionality in the compute API and it's been a source of 
confusion and frustration ever since. Relying on a manual 
confirmation or revert input from the user was and continues to be a 
horrible idea.


Agreed.  In my experience with operating a public cloud I am not aware 
of anyone benefiting from the manual checkpoint in the middle of 
resize.  But we should solicit feedback from the operator community on 
this.




I believe strongly that we should deprecate the existing migrate, 
resize, an live-migrate APIs in favor of a single consolidated, 
consistent "move" REST API that would have the following 
characteristics:


I'm not sure that we should abstract away live vs cold migrate behind 
a single move API, but I strongly agree with consolidate cold-migrate 
and resize.




* No manual or wait-input states in any FSM graph
* Removal of the term "resize" from the API entirely (the target 
resource sizing is an attribute of the move operation, not a 
different type of API operation in and of itself)
* Transition to a task-based API for poll-state requests. This means 
that in order for a caller to determine the state of a VM the caller 
would call something like GET /servers//tasks/ in order 
to see the history of state changes or subtask operations for a 
particular request to move a VM


Huge +1 from me.



Timofei Durakov (cc'd) has a blueprint for splitting the 
live-migration types into separate task classes here:


https://review.openstack.org/#/c/225910/

I think there's a lot of good ideas in that proposal. Please do have 
a look at it.


Best,
-jay

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
.




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Tang Chen]

On 10/29/2015 09:26 AM, Tang Chen wrote:

Hi Andrew,

On 10/29/2015 12:53 AM, Andrew Laski wrote:

On 10/27/15 at 09:02am, Jay Pipes wrote:

On 10/22/2015 11:13 AM, Tang Chen wrote:

On 10/22/2015 05:17 AM, Joshua Harlow wrote:

Overall I'm very much inclined to have three state machines (one
for each type), vs the mix-mash of all three into one state machine
(which causes the confusion around states in the first diagram in
that paste).


That is an idea. But I would prefer to have one single state machine
for migration, because resize and evacuate are reusing migration.
They can be in one state machine.


Evacuate does *not* migrate/move anything. Evacuate *rebuilds* VMs 
from their original source image.


I support Nikola in that I believe the different migration types 
should have different state machines entirely (but be as consistent 
as possible in the naming of terminal states like "finished" vs 
"done" etc)


+1.

It seems that there's also some work to do to disambiguate what the 
different operations are and what they actually do.


Migrate/resize share a code path, and then there's a big switch to 
change behavior if the migration is a live-migrate.  I agree with the 
comment below that we should deprecate the resize terminology and 
consolidate cold-migrate and resize under one umbrella with one state 
machine.  Then live-migrate could have a separate state machine.


Rebuild/evacuate share a code path though evacuate involves a 
scheduler decision which is why it seems like a move.  It's actually 
a bit tricky to classify because if the instance is volume backed it 
is essentially just a move operation, if it's image backed it's a 
destructive rebuild.  But I think it makes sense to not consider this 
a move operation and think of it as an administrative rebuild when a 
host is down.


I'm little think of that we should define each migration type clearly 
first, and then improve migration state machine. If we don't agree on 
what type represents what operation,  the state machine won't be good.


Please also help to review this BP, although the idea may be not 
exactly the same as yours.


https://blueprints.launchpad.net/nova/+spec/migration-type-refactor

https://blueprints.launchpad.net/nova/+spec/migration-state-field-machine



Thanks.






It would be very helpful if the designer of the migration process
could share his idea. But if it is just some code modified by many
people many times, I think we should remove the confusing states and
give a easier, better state machine.


There isn't a designer of the migration process :( The original 
(crap, IMHO) API from Rackspace Cloud Servers API was used for the 
resize functionality in the compute API and it's been a source of 
confusion and frustration ever since. Relying on a manual 
confirmation or revert input from the user was and continues to be a 
horrible idea.


Agreed.  In my experience with operating a public cloud I am not 
aware of anyone benefiting from the manual checkpoint in the middle 
of resize.  But we should solicit feedback from the operator 
community on this.




I believe strongly that we should deprecate the existing migrate, 
resize, an live-migrate APIs in favor of a single consolidated, 
consistent "move" REST API that would have the following 
characteristics:


I'm not sure that we should abstract away live vs cold migrate behind 
a single move API, but I strongly agree with consolidate cold-migrate 
and resize.




* No manual or wait-input states in any FSM graph
* Removal of the term "resize" from the API entirely (the target 
resource sizing is an attribute of the move operation, not a 
different type of API operation in and of itself)
* Transition to a task-based API for poll-state requests. This means 
that in order for a caller to determine the state of a VM the caller 
would call something like GET /servers//tasks/ in order 
to see the history of state changes or subtask operations for a 
particular request to move a VM


Huge +1 from me.



Timofei Durakov (cc'd) has a blueprint for splitting the 
live-migration types into separate task classes here:


https://review.openstack.org/#/c/225910/

I think there's a lot of good ideas in that proposal. Please do have 
a look at it.


Best,
-jay

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
.




__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Tang Chen]

Hi all,

Just FYI, the WIP patch-set is now available here:

https://review.openstack.org/241476
https://review.openstack.org/241477
https://review.openstack.org/241478
https://review.openstack.org/241479
https://review.openstack.org/241480

Thanks.

On 10/14/2015 10:05 AM, Tang Chen wrote:

Hi, all,

Please help to review this BP.

https://blueprints.launchpad.net/nova/+spec/live-migration-state-machine


Currently, the migration_status field in Migration object is 
indicating the

status of migration process. But in the current code, it is represented
by pure string, like 'migrating', 'finished', and so on.

The strings could be confusing to different developers, e.g. there are 3
statuses representing the migration process is over successfully:
'finished', 'completed' and 'done'.
And 2 for migration in process: 'running' and 'migrating'.

So I think we should use constants or enum for these statuses.


Furthermore, Nikola has proposed to create a state machine for the 
statuses,
which is part of another abandoned BP. And this is also the work I'd 
like to go

on with. Please refer to:
https://review.openstack.org/#/c/197668/ 

https://review.openstack.org/#/c/197669/ 




Another proposal is: introduce a new member named "state" into Migration.
Use a state machine to handle this Migration.state, and leave 
migration_status

field a descriptive human readable free-form.


So how do you think ?

Thanks.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Murray, Paul (HP Cloud)]

> From: Jay Pipes [mailto:jaypi...@gmail.com]
> On 10/27/2015 01:16 PM, Chris Friesen wrote:
> > On 10/26/2015 06:02 PM, Jay Pipes wrote:
> >
> >> I believe strongly that we should deprecate the existing migrate,
> >> resize, an live-migrate APIs in favor of a single consolidated,
> >> consistent "move"
> >> REST API
> >> that would have the following characteristics:
> >>
> >> * No manual or wait-input states in any FSM graph
> >
> > Sounds good.
> >
> >> * Removal of the term "resize" from the API entirely (the target
> >> resource sizing is an attribute of the move operation, not a
> >> different type of API operation in and of itself)
> >
> > I disagree on this one.
> >
> > As an end-user, if my goal is to take an existing instance and give it
> > bigger disks, my first instinct isn't going to be to look at the "move"
> > operation.  I'm going to look for "scale", or "resize", or something
> > like that.
> >
> > And if an admin wants to migrate an instance away from its current
> > host, why would they want to change its disk size in the process?
> 
> A fair point. However, I think that a generic update VM API, which would
> allow changes to the resources consumed by the VM along with capabiities
> like CPU model or local disk performance (SSD) is a better way to handle this
> than a resize-specific API.


Sorry I am so late to this - but this stuck out for me. 

Resize is an operation that a cloud user would do to his VM. Usually the
cloud user does not know what host the VM is running on so a resize does 
not appear to be a move at all.

Migrate is an operation that a cloud operator does to a VM that is not normally
available to a cloud user. A cloud operator does not change the VM because 
the operator just provides what the user asked for. He only choses where he is 
going to put it.

It seems clear to me that resize and migrate are very definitely different 
things,
even if they are implemented using the same code path internally for 
convenience.
At the very least I believe they need to be kept separate at the API so we can 
apply
different policy to control access to them.



> 
> So, in other words, I'd support this:
> 
> PATCH /servers/
> 
> with some corresponding request payload that would indicate the required
> changes.
> 
> > I do think it makes sense to combine the external APIs for live and
> > cold migration.  Those two are fundamentally similar, logically
> > separated only by whether the instance stays running or not.
> >
> > And I'm perfectly fine with having the internal implementation of all
> > three share a code path, I just don't think it makes sense for the
> > *external* API.
> 
> I think you meant to say you don't think it makes sense to have three
> separate external APIs for what is fundamentally the same operation (move
> a VM), right?
> 
> Best,
> -jay
> 
> >> * Transition to a task-based API for poll-state requests. This means
> >> that in order for a caller to determine the state of a VM the caller
> >> would call something like GET /servers//tasks/ in order
> >> to see the history of state changes or subtask operations for a
> >> particular request to move a VM
> > enstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> > Sounds good.
> >
> > Chris
> >

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Jonathan D. Proulx]

On Wed, Nov 04, 2015 at 06:17:17PM +, Murray, Paul (HP Cloud) wrote:
:> From: Jay Pipes [mailto:jaypi...@gmail.com]
:> A fair point. However, I think that a generic update VM API, which would
:> allow changes to the resources consumed by the VM along with capabiities
:> like CPU model or local disk performance (SSD) is a better way to handle this
:> than a resize-specific API.
:
:
:Sorry I am so late to this - but this stuck out for me. 
:
:Resize is an operation that a cloud user would do to his VM. Usually the
:cloud user does not know what host the VM is running on so a resize does 
:not appear to be a move at all.
:
:Migrate is an operation that a cloud operator does to a VM that is not normally
:available to a cloud user. A cloud operator does not change the VM because 
:the operator just provides what the user asked for. He only choses where he is 
:going to put it.
:
:It seems clear to me that resize and migrate are very definitely different 
things,
:even if they are implemented using the same code path internally for 
convenience.
:At the very least I believe they need to be kept separate at the API so we can 
apply
:different policy to control access to them.

As an operator I'm with Paul on this.

By all means use the same code path becasue behind the scenes it *is*
the same thing.  

BUT, at the API level we do need the distinction particularly for access
control policy. The UX 'findablility' is important too, but if that
were the only issue a bit of syntactic sugar in the UI could take care
of it.

-Jon

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Tang Chen]

On 11/05/2015 02:36 AM, Jonathan D. Proulx wrote:

On Wed, Nov 04, 2015 at 06:17:17PM +, Murray, Paul (HP Cloud) wrote:
:> From: Jay Pipes [mailto:jaypi...@gmail.com]
:> A fair point. However, I think that a generic update VM API, which would
:> allow changes to the resources consumed by the VM along with capabiities
:> like CPU model or local disk performance (SSD) is a better way to handle this
:> than a resize-specific API.
:
:
:Sorry I am so late to this - but this stuck out for me.
:
:Resize is an operation that a cloud user would do to his VM. Usually the
:cloud user does not know what host the VM is running on so a resize does
:not appear to be a move at all.
:
:Migrate is an operation that a cloud operator does to a VM that is not normally
:available to a cloud user. A cloud operator does not change the VM because
:the operator just provides what the user asked for. He only choses where he is
:going to put it.
:
:It seems clear to me that resize and migrate are very definitely different 
things,
:even if they are implemented using the same code path internally for 
convenience.
:At the very least I believe they need to be kept separate at the API so we can 
apply
:different policy to control access to them.

As an operator I'm with Paul on this.

By all means use the same code path becasue behind the scenes it *is*


Hi Jonathan,

I'm sorry that I cannot understand why resize and migrate are the same 
thing behind.


I have some understanding of my own here, please help to review.
https://blueprints.launchpad.net/nova/+spec/migration-type-refactor

I'm not sure if I understand it right or wrong. In my understanding, 
resizing a VM doesn't need to migrate it.


Thanks.


the same thing.

BUT, at the API level we do need the distinction particularly for access
control policy. The UX 'findablility' is important too, but if that
were the only issue a bit of syntactic sugar in the UI could take care
of it.

-Jon

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Ed Leafe]

On Nov 5, 2015, at 2:43 AM, Tang Chen  wrote:

> I'm sorry that I cannot understand why resize and migrate are the same thing 
> behind.

Resize is essentially a migration to the same host, rather than a different 
host. The process is still taking an existing VM and using it to create another 
VM that appears to the user as the same (ID, networking, attached volumes, 
metadata, etc.)

-- Ed Leafe







signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Murray, Paul (HP Cloud)]

> From: Ed Leafe [mailto:e...@leafe.com]
> On Nov 5, 2015, at 2:43 AM, Tang Chen  wrote:
> 
> > I'm sorry that I cannot understand why resize and migrate are the same
> thing behind.
> 
> Resize is essentially a migration to the same host, rather than a different
> host. The process is still taking an existing VM and using it to create 
> another
> VM that appears to the user as the same (ID, networking, attached volumes,
> metadata, etc.)
> 



Or more specifically, the migrate and resize API actions both call the resize
function in the compute api. As Ed said, they are basically the same behind 
the scenes. (But the API difference is important.)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Andrew Laski]

On 11/05/15 at 01:28pm, Murray, Paul (HP Cloud) wrote:




From: Ed Leafe [mailto:e...@leafe.com]
On Nov 5, 2015, at 2:43 AM, Tang Chen  wrote:

> I'm sorry that I cannot understand why resize and migrate are the same
thing behind.

Resize is essentially a migration to the same host, rather than a different
host. The process is still taking an existing VM and using it to create another
VM that appears to the user as the same (ID, networking, attached volumes,
metadata, etc.)





Or more specifically, the migrate and resize API actions both call the resize
function in the compute api. As Ed said, they are basically the same behind
the scenes. (But the API difference is important.)


Can you be a little more specific on what API difference is important to 
you?  There are two differences currently between migrate and resize in 
the API:


1. There is a different policy check, but this only really protects the 
next bit.


2. Resize passes in a new flavor and migration does not.

Both actions result in an instance being scheduled to a new host.  If 
they were consolidated into a single action with a policy check to 
enforce that users specified a new flavor and admins could leave that 
off would that be problematic for you?





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Jonathan D. Proulx]

On Thu, Nov 05, 2015 at 09:33:37AM -0500, Andrew Laski wrote:

:Can you be a little more specific on what API difference is important
:to you?  There are two differences currently between migrate and
:resize in the API:
:
:1. There is a different policy check, but this only really protects
:the next bit.
:
:2. Resize passes in a new flavor and migration does not.
:
:Both actions result in an instance being scheduled to a new host.  If
:they were consolidated into a single action with a policy check to
:enforce that users specified a new flavor and admins could leave that
:off would that be problematic for you?

My typical use is live-migration (perhaps that is yet another code
path?) which involves:

3. specify the host to migrate to

This is what I really want to protect.

my use case if it helps:

The reason I want to specify the host (or if I could even better a
host aggregaate) is that I use 'cpu_mode=host-passthrough' and have a
few generations of hardware (and my instance types are not constrained
to a particular generation which I realize is an option as we do that
for other purposes) so left to the scheduler it might try to
live-migrate to an older cpu generation which would fail so we'r
ecurrently using human intelligence to try to migrate to same
generation and if that's full move newer.

This is an uncommon but important procedure mostly used for updates that
require hypervisor reboot in which we roll everything from node-0 to
node-N, update 0 then roll node-1 to node0 etc ...

If I could constrain migration by host aggregate in ways that didn't
map to instance type metadata constraints that would simplify this,
but the current situation is adequate for me.

This isn't an issue with non-live migration or rezize neither of which
requite CPU consistency.

-Jon

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Andrew Laski]

On 11/05/15 at 10:12am, Jonathan D. Proulx wrote:

On Thu, Nov 05, 2015 at 09:33:37AM -0500, Andrew Laski wrote:

:Can you be a little more specific on what API difference is important
:to you?  There are two differences currently between migrate and
:resize in the API:
:
:1. There is a different policy check, but this only really protects
:the next bit.
:
:2. Resize passes in a new flavor and migration does not.
:
:Both actions result in an instance being scheduled to a new host.  If
:they were consolidated into a single action with a policy check to
:enforce that users specified a new flavor and admins could leave that
:off would that be problematic for you?

My typical use is live-migration (perhaps that is yet another code
path?) which involves:


Yes, live-migration is completely separate from resize/(cold)migrate.  
I'm not convinced that we can or should consolidate live-migration with 
resize/cold-migrate.




3. specify the host to migrate to

This is what I really want to protect.

my use case if it helps:

The reason I want to specify the host (or if I could even better a
host aggregaate) is that I use 'cpu_mode=host-passthrough' and have a
few generations of hardware (and my instance types are not constrained
to a particular generation which I realize is an option as we do that
for other purposes) so left to the scheduler it might try to
live-migrate to an older cpu generation which would fail so we'r
ecurrently using human intelligence to try to migrate to same
generation and if that's full move newer.

This is an uncommon but important procedure mostly used for updates that
require hypervisor reboot in which we roll everything from node-0 to
node-N, update 0 then roll node-1 to node0 etc ...

If I could constrain migration by host aggregate in ways that didn't
map to instance type metadata constraints that would simplify this,
but the current situation is adequate for me.

This isn't an issue with non-live migration or rezize neither of which
requite CPU consistency.

-Jon

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Chris Friesen]

On 11/05/2015 08:33 AM, Andrew Laski wrote:

On 11/05/15 at 01:28pm, Murray, Paul (HP Cloud) wrote:



Or more specifically, the migrate and resize API actions both call the resize
function in the compute api. As Ed said, they are basically the same behind
the scenes. (But the API difference is important.)


Can you be a little more specific on what API difference is important to you?
There are two differences currently between migrate and resize in the API:

1. There is a different policy check, but this only really protects the next 
bit.

2. Resize passes in a new flavor and migration does not.

Both actions result in an instance being scheduled to a new host.  If they were
consolidated into a single action with a policy check to enforce that users
specified a new flavor and admins could leave that off would that be problematic
for you?



To me, the fact that resize and cold migration share the same implementation is 
just that, an implementation detail.


From the outside they are different things...one is "take this instance and 
move it somewhere else", and the other "take this instance and change its 
resource profile".


To me, the external API would make more sense as:

1) resize

2) migrate (with option of cold or live, and with option to specify a 
destination, and with option to override the scheduler if the specified 
destination doesn't pass filters)



And while we're talking, I don't understand why "allow_resize_to_same_host" 
defaults to False.  The comments in https://bugs.launchpad.net/nova/+bug/1251266 
say that it's not intended to be used in production, but doesn't give a 
rationale for that statement.  If you're using local storage and you just want 
to add some more CPUs/RAM to the instance, wouldn't it be beneficial to avoid 
the need to copy the rootfs?


Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Bhandaru, Malini K]

+1 on Chris comments on implementation and API.
Migrate, if all is ideal, should take the initial launch flavor.

-Original Message-
From: Chris Friesen [mailto:chris.frie...@windriver.com] 
Sent: Thursday, November 05, 2015 8:46 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Nova] Migration state machine proposal.

On 11/05/2015 08:33 AM, Andrew Laski wrote:
> On 11/05/15 at 01:28pm, Murray, Paul (HP Cloud) wrote:

>> Or more specifically, the migrate and resize API actions both call 
>> the resize function in the compute api. As Ed said, they are 
>> basically the same behind the scenes. (But the API difference is 
>> important.)
>
> Can you be a little more specific on what API difference is important to you?
> There are two differences currently between migrate and resize in the API:
>
> 1. There is a different policy check, but this only really protects the next 
> bit.
>
> 2. Resize passes in a new flavor and migration does not.
>
> Both actions result in an instance being scheduled to a new host.  If 
> they were consolidated into a single action with a policy check to 
> enforce that users specified a new flavor and admins could leave that 
> off would that be problematic for you?


To me, the fact that resize and cold migration share the same implementation is 
just that, an implementation detail.

 From the outside they are different things...one is "take this instance and 
move it somewhere else", and the other "take this instance and change its 
resource profile".

To me, the external API would make more sense as:

1) resize

2) migrate (with option of cold or live, and with option to specify a 
destination, and with option to override the scheduler if the specified 
destination doesn't pass filters)


And while we're talking, I don't understand why "allow_resize_to_same_host" 
defaults to False.  The comments in https://bugs.launchpad.net/nova/+bug/1251266
say that it's not intended to be used in production, but doesn't give a 
rationale for that statement.  If you're using local storage and you just want 
to add some more CPUs/RAM to the instance, wouldn't it be beneficial to avoid 
the need to copy the rootfs?

Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Tang Chen]

On 11/06/2015 12:45 PM, Chris Friesen wrote:

On 11/05/2015 08:33 AM, Andrew Laski wrote:

On 11/05/15 at 01:28pm, Murray, Paul (HP Cloud) wrote:


Or more specifically, the migrate and resize API actions both call 
the resize
function in the compute api. As Ed said, they are basically the same 
behind

the scenes. (But the API difference is important.)


Can you be a little more specific on what API difference is important 
to you?
There are two differences currently between migrate and resize in the 
API:


1. There is a different policy check, but this only really protects 
the next bit.


2. Resize passes in a new flavor and migration does not.

Both actions result in an instance being scheduled to a new host.  If 
they were
consolidated into a single action with a policy check to enforce that 
users
specified a new flavor and admins could leave that off would that be 
problematic

for you?



To me, the fact that resize and cold migration share the same 
implementation is just that, an implementation detail.


From the outside they are different things...one is "take this 
instance and move it somewhere else", and the other "take this 
instance and change its resource profile".


To me, the external API would make more sense as:

1) resize

2) migrate (with option of cold or live, and with option to specify a 
destination, and with option to override the scheduler if the 
specified destination doesn't pass filters)


OK. Conceptually speaking, only one case that resize could reuse 
migration code: the current host cannot match the resize condition.

And the VM should be migrated to another host, and do the resize.

So I don't think resize should be one type of migration.

May I understand it like this:  what we are talking about here is a 
3-level conception.


user APInova service driver
migrate  live-migration   o ff compute node 
storage---shared file system
resizecold-migration  o n compute node 
storage---shared file system
rebuild   o n 
compute node storage---nonshared file system

evacuate

Indeed it is a implementation detail. If we can refactor the source code 
as above, maybe it is more clear.





And while we're talking, I don't understand why 
"allow_resize_to_same_host" defaults to False.  The comments in 
https://bugs.launchpad.net/nova/+bug/1251266 say that it's not 
intended to be used in production, but doesn't give a rationale for 
that statement.  If you're using local storage and you just want to 
add some more CPUs/RAM to the instance, wouldn't it be beneficial to 
avoid the need to copy the rootfs?


I'm sorry I don't know why it is False by default. But if we can 
refactor the source code, and split resize and migrate conceptually, I 
think we don't need this option any more.


And another question about resize, shall we think about CPU/memory 
hotplug ?  AFAIK, Linux kerenl and qemu are now supporting memory 
hotplug. CPU hotplug in qemu is still being developed. I was thinking 
resize could use these functionalities.


Thanks.



Chris

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Migration state machine proposal.

[Chris Friesen]

On 11/06/2015 03:47 AM, Tang Chen wrote:


On 11/06/2015 12:45 PM, Chris Friesen wrote:



To me, the external API would make more sense as:

1) resize

2) migrate (with option of cold or live, and with option to specify a
destination, and with option to override the scheduler if the specified
destination doesn't pass filters)


OK. Conceptually speaking, only one case that resize could reuse migration code:
the current host cannot match the resize condition.


It's not quite that simple.  Generally speaking, in both cases the scheduler 
picks a new host, we stop the guest and maybe move the rootfs files around then 
restart the guest, and handle all the resource tracking along the way.  The 
difference is that resize starts the guest with a different flavor and doesn't 
necessarily need to move to a different host (though by default it will).


So there really is quite a bit of overlap in what needs to happen 
internally...but I don't think that overlap should extend to the external API 
because it's not obvious to the end-user.



So I don't think resize should be one type of migration.

May I understand it like this:  what we are talking about here is a 3-level
conception.

user APInova service driver
migrate live-migration   o ff compute node storage—shared file 
system
resize  cold-migration   o n compute node  storage—shared file 
system
rebuild  o n compute node storage—nonshared 
file system
evacuate


Actually, rebuild/evacuate share an underlying nova code path in much the same 
way that resize/cold-migrate share a code path.  What nova calls evacuate is 
basically "rebuild with the same image to a different host".


Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev