Re: [openstack-dev] [all] Versioned objects cross project sessions next steps

[Doug Hellmann]

On Nov 24, 2014, at 4:06 PM, Jay Pipes  wrote:

> On 11/24/2014 03:11 PM, Joshua Harlow wrote:
>> Dan Smith wrote:
 3. vish brought up one draw back of versioned objects: the difficulty in
 cherry picking commits for stable branches - Is this a show stopper?.
>>> 
>>> After some discussion with some of the interested parties, we're
>>> planning to add a third .z element to the version numbers and use that
>>> to handle backports in the same way that we do for RPC:
>>> 
>>> https://review.openstack.org/#/c/134623/
>>> 
 Next steps:
 - Jay suggested making a second spec that would lay out what it would
 look like if we used google protocol buffers.
 - Dan: do you need some help in making this happen, do we need some
 volunteers?
>>> 
>>> I'm not planning to look into this, especially since we discussed it a
>>> couple years ago when deciding to do what we're currently doing. If
>>> someone else does, creates a thing that is demonstrably more useful than
>>> what we have, and provides a migration plan, then cool. Otherwise, I'm
>>> not really planning to stop what I'm doing at the moment.
>>> 
 - Are there any other concrete things we can do to get this usable by
 other projects in a timely manner?
>>> 
>>> To be honest, since the summit, I've not done anything with the current
>>> oslo spec, given the potential for doing something different that was
>>> raised. I know that cinder folks (at least) are planning to start
>>> copying code into their tree to get moving.
>>> 
>>> I think we need a decision to either (a) dump what we've got into the
>>> proposed library (or incubator) and plan to move forward incrementally
>>> or (b) each continue doing our own thing(s) in our own trees while we
>>> wait for someone to create something based on GPB that does what we want.
>> 
>> I'd prefer (a); although I hope there is a owner/lead for this library
>> (dan?) and it's not just dumped on the oslo folks as that won't work out
>> so well I think. It'd be nice if said owner could also look into (b) but
>> that's at there own (or other library supporter) time I suppose (I
>> personally think (b) would probably allow for a larger community of
>> folks to get involved in this library, would potentially reduce the
>> amount of custom/overlapping code and other similar benefits...).
> 
> I gave some comments at the very end of the summit session on this, and I 
> want to be clear about something. I definitely like GPB, and there's definite 
> overlap with some things that GPB does and things that nova.objects does.
> 
> That said, I don't think it's wise to make oslo-versionedobjects be a totally 
> new thing. I think we should use nova.objects as the base of a new 
> oslo-versionedobjects library, and we should evolve oslo-versionedobjects 
> slowly over time, eventually allowing for nova, ironic, and whomever else is 
> currently using nova/objects, to align with an Oslo library vision for this.
> 
> So, in short, I also think a) is the appropriate path to take.

+1

When Dan and I talked about this, I said I would take care of exporting the 
nova objects git history into a new repository. We’ve had some other things 
blocking work in Oslo that I needed to handle, but I expect to be able to pick 
up this work soon. If someone else wants to jump in, I’ll be happy to give a 
brain dump of what I planned to do for the export, since the existing Oslo tool 
that we use on the incubator isn’t quite right for the job.

Doug

> 
> Best,
> -jay
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Versioned objects cross project sessions next steps

[Angus Salkeld]

On Tue, Nov 25, 2014 at 7:06 AM, Jay Pipes  wrote:

> On 11/24/2014 03:11 PM, Joshua Harlow wrote:
>
>> Dan Smith wrote:
>>
>>> 3. vish brought up one draw back of versioned objects: the difficulty in
 cherry picking commits for stable branches - Is this a show stopper?.

>>>
>>> After some discussion with some of the interested parties, we're
>>> planning to add a third .z element to the version numbers and use that
>>> to handle backports in the same way that we do for RPC:
>>>
>>> https://review.openstack.org/#/c/134623/
>>>
>>>  Next steps:
 - Jay suggested making a second spec that would lay out what it would
 look like if we used google protocol buffers.
 - Dan: do you need some help in making this happen, do we need some
 volunteers?

>>>
>>> I'm not planning to look into this, especially since we discussed it a
>>> couple years ago when deciding to do what we're currently doing. If
>>> someone else does, creates a thing that is demonstrably more useful than
>>> what we have, and provides a migration plan, then cool. Otherwise, I'm
>>> not really planning to stop what I'm doing at the moment.
>>>
>>>  - Are there any other concrete things we can do to get this usable by
 other projects in a timely manner?

>>>
>>> To be honest, since the summit, I've not done anything with the current
>>> oslo spec, given the potential for doing something different that was
>>> raised. I know that cinder folks (at least) are planning to start
>>> copying code into their tree to get moving.
>>>
>>> I think we need a decision to either (a) dump what we've got into the
>>> proposed library (or incubator) and plan to move forward incrementally
>>> or (b) each continue doing our own thing(s) in our own trees while we
>>> wait for someone to create something based on GPB that does what we want.
>>>
>>
>> I'd prefer (a); although I hope there is a owner/lead for this library
>> (dan?) and it's not just dumped on the oslo folks as that won't work out
>> so well I think. It'd be nice if said owner could also look into (b) but
>> that's at there own (or other library supporter) time I suppose (I
>> personally think (b) would probably allow for a larger community of
>> folks to get involved in this library, would potentially reduce the
>> amount of custom/overlapping code and other similar benefits...).
>>
>
> I gave some comments at the very end of the summit session on this, and I
> want to be clear about something. I definitely like GPB, and there's
> definite overlap with some things that GPB does and things that
> nova.objects does.
>
> That said, I don't think it's wise to make oslo-versionedobjects be a
> totally new thing. I think we should use nova.objects as the base of a new
> oslo-versionedobjects library, and we should evolve oslo-versionedobjects
> slowly over time, eventually allowing for nova, ironic, and whomever else
> is currently using nova/objects, to align with an Oslo library vision for
> this.
>
> So, in short, I also think a) is the appropriate path to take.
>

Yeah, my concern with "(b)" is the time it will take for other projects to
get to use it, esp. since no one is
jumping to take the work on.

-Angus


>
> Best,
> -jay
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Versioned objects cross project sessions next steps

[melanie witt]

On Nov 24, 2014, at 13:06, Jay Pipes  wrote:

> That said, I don't think it's wise to make oslo-versionedobjects be a totally 
> new thing. I think we should use nova.objects as the base of a new 
> oslo-versionedobjects library, and we should evolve oslo-versionedobjects 
> slowly over time, eventually allowing for nova, ironic, and whomever else is 
> currently using nova/objects, to align with an Oslo library vision for this.
> 
> So, in short, I also think a) is the appropriate path to take.

+1 I'd like to see oslo-versionedobjects start out with nova.objects as the 
implementation, with the ability to add support for protobuf later.

melanie (melwitt)






signature.asc
Description: Message signed with OpenPGP using GPGMail
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Versioned objects cross project sessions next steps

[Jay Pipes]

On 11/24/2014 03:11 PM, Joshua Harlow wrote:

Dan Smith wrote:

3. vish brought up one draw back of versioned objects: the difficulty in
cherry picking commits for stable branches - Is this a show stopper?.


After some discussion with some of the interested parties, we're
planning to add a third .z element to the version numbers and use that
to handle backports in the same way that we do for RPC:

https://review.openstack.org/#/c/134623/


Next steps:
- Jay suggested making a second spec that would lay out what it would
look like if we used google protocol buffers.
- Dan: do you need some help in making this happen, do we need some
volunteers?


I'm not planning to look into this, especially since we discussed it a
couple years ago when deciding to do what we're currently doing. If
someone else does, creates a thing that is demonstrably more useful than
what we have, and provides a migration plan, then cool. Otherwise, I'm
not really planning to stop what I'm doing at the moment.


- Are there any other concrete things we can do to get this usable by
other projects in a timely manner?


To be honest, since the summit, I've not done anything with the current
oslo spec, given the potential for doing something different that was
raised. I know that cinder folks (at least) are planning to start
copying code into their tree to get moving.

I think we need a decision to either (a) dump what we've got into the
proposed library (or incubator) and plan to move forward incrementally
or (b) each continue doing our own thing(s) in our own trees while we
wait for someone to create something based on GPB that does what we want.


I'd prefer (a); although I hope there is a owner/lead for this library
(dan?) and it's not just dumped on the oslo folks as that won't work out
so well I think. It'd be nice if said owner could also look into (b) but
that's at there own (or other library supporter) time I suppose (I
personally think (b) would probably allow for a larger community of
folks to get involved in this library, would potentially reduce the
amount of custom/overlapping code and other similar benefits...).


I gave some comments at the very end of the summit session on this, and 
I want to be clear about something. I definitely like GPB, and there's 
definite overlap with some things that GPB does and things that 
nova.objects does.


That said, I don't think it's wise to make oslo-versionedobjects be a 
totally new thing. I think we should use nova.objects as the base of a 
new oslo-versionedobjects library, and we should evolve 
oslo-versionedobjects slowly over time, eventually allowing for nova, 
ironic, and whomever else is currently using nova/objects, to align with 
an Oslo library vision for this.


So, in short, I also think a) is the appropriate path to take.

Best,
-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Versioned objects cross project sessions next steps

[Joshua Harlow]

Dan Smith wrote:

3. vish brought up one draw back of versioned objects: the difficulty in
cherry picking commits for stable branches - Is this a show stopper?.


After some discussion with some of the interested parties, we're
planning to add a third .z element to the version numbers and use that
to handle backports in the same way that we do for RPC:

https://review.openstack.org/#/c/134623/


Next steps:
- Jay suggested making a second spec that would lay out what it would
look like if we used google protocol buffers.
- Dan: do you need some help in making this happen, do we need some
volunteers?


I'm not planning to look into this, especially since we discussed it a
couple years ago when deciding to do what we're currently doing. If
someone else does, creates a thing that is demonstrably more useful than
what we have, and provides a migration plan, then cool. Otherwise, I'm
not really planning to stop what I'm doing at the moment.


- Are there any other concrete things we can do to get this usable by
other projects in a timely manner?


To be honest, since the summit, I've not done anything with the current
oslo spec, given the potential for doing something different that was
raised. I know that cinder folks (at least) are planning to start
copying code into their tree to get moving.

I think we need a decision to either (a) dump what we've got into the
proposed library (or incubator) and plan to move forward incrementally
or (b) each continue doing our own thing(s) in our own trees while we
wait for someone to create something based on GPB that does what we want.



I'd prefer (a); although I hope there is a owner/lead for this library 
(dan?) and it's not just dumped on the oslo folks as that won't work out 
so well I think. It'd be nice if said owner could also look into (b) but 
that's at there own (or other library supporter) time I suppose (I 
personally think (b) would probably allow for a larger community of 
folks to get involved in this library, would potentially reduce the 
amount of custom/overlapping code and other similar benefits...).



--Dan


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Versioned objects cross project sessions next steps

[Dan Smith]

> 3. vish brought up one draw back of versioned objects: the difficulty in
> cherry picking commits for stable branches - Is this a show stopper?.

After some discussion with some of the interested parties, we're
planning to add a third .z element to the version numbers and use that
to handle backports in the same way that we do for RPC:

https://review.openstack.org/#/c/134623/

> Next steps:
> - Jay suggested making a second spec that would lay out what it would
> look like if we used google protocol buffers.
> - Dan: do you need some help in making this happen, do we need some
> volunteers?

I'm not planning to look into this, especially since we discussed it a
couple years ago when deciding to do what we're currently doing. If
someone else does, creates a thing that is demonstrably more useful than
what we have, and provides a migration plan, then cool. Otherwise, I'm
not really planning to stop what I'm doing at the moment.

> - Are there any other concrete things we can do to get this usable by
> other projects in a timely manner?

To be honest, since the summit, I've not done anything with the current
oslo spec, given the potential for doing something different that was
raised. I know that cinder folks (at least) are planning to start
copying code into their tree to get moving.

I think we need a decision to either (a) dump what we've got into the
proposed library (or incubator) and plan to move forward incrementally
or (b) each continue doing our own thing(s) in our own trees while we
wait for someone to create something based on GPB that does what we want.

--Dan



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Versioned objects cross project sessions next steps

[Jay Pipes]

On 11/13/2014 01:12 AM, Robert Collins wrote:

On 11 November 2014 13:30, Angus Salkeld  wrote:

Hi all

I just wanted to make sure we are all under the same understanding of the
outcomes and what the next steps for the versioned objects session are.

1. There is a lot of interest in other projects using oslo versioned objects
and it is worth progressing with this
(https://review.openstack.org/#/c/127532).
2. jpipes and jharlow suggested experimenting/investigating google protocol
buffers (https://developers.google.com/protocol-buffers/) instead of  the
custom serialization and version code. This *could* be an implementation
detail, but also could make the adoption by nova more complicated (as it has
a different mechanism in place).
3. vish brought up one draw back of versioned objects: the difficulty in
cherry picking commits for stable branches - Is this a show stopper?.

Next steps:
- Jay suggested making a second spec that would lay out what it would look
like if we used google protocol buffers.
- Dan: do you need some help in making this happen, do we need some
volunteers?
- Are there any other concrete things we can do to get this usable by other
projects in a timely manner?


+1 on protocol buffers, but perhaps
http://kentonv.github.io/capnproto/ could be considered: its protocol
buffers v2 basically - from one of the originators of protocol
buffers. It has Python support available too, just like protocol
buffers.


Very nice indeed. Been reading through the Cap'n'proto documentation and 
it looks like a great improvement over GPB.


Definitely something to look into.

I sent an email privately to Angus and Dan this morning saying that I 
personally don't have the bandwidth to do a PoC that would use GPB as 
implementation of the serialization, schema representation, and 
versioning engine. I support the idea of using GPB, but I also recognize 
it's a large amount of work.


Best,
-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Versioned objects cross project sessions next steps

[Joshua Harlow]

Neat,
Didn't know that existed.
http://kentonv.github.io/capnproto/language.html does look pretty nice...

-Josh  From: Robert Collins 
 To: OpenStack Development Mailing List (not for usage questions) 
 
 Sent: Wednesday, November 12, 2014 10:12 PM
 Subject: Re: [openstack-dev] [all] Versioned objects cross project sessions 
next steps
   
On 11 November 2014 13:30, Angus Salkeld  wrote:
> Hi all
>
> I just wanted to make sure we are all under the same understanding of the
> outcomes and what the next steps for the versioned objects session are.
>
> 1. There is a lot of interest in other projects using oslo versioned objects
> and it is worth progressing with this
> (https://review.openstack.org/#/c/127532).
> 2. jpipes and jharlow suggested experimenting/investigating google protocol
> buffers (https://developers.google.com/protocol-buffers/) instead of  the
> custom serialization and version code. This *could* be an implementation
> detail, but also could make the adoption by nova more complicated (as it has
> a different mechanism in place).
> 3. vish brought up one draw back of versioned objects: the difficulty in
> cherry picking commits for stable branches - Is this a show stopper?.
>
> Next steps:
> - Jay suggested making a second spec that would lay out what it would look
> like if we used google protocol buffers.
> - Dan: do you need some help in making this happen, do we need some
> volunteers?
> - Are there any other concrete things we can do to get this usable by other
> projects in a timely manner?

+1 on protocol buffers, but perhaps
http://kentonv.github.io/capnproto/ could be considered: its protocol
buffers v2 basically - from one of the originators of protocol
buffers. It has Python support available too, just like protocol
buffers.

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


  ___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Versioned objects cross project sessions next steps

[Robert Collins]

On 11 November 2014 13:30, Angus Salkeld  wrote:
> Hi all
>
> I just wanted to make sure we are all under the same understanding of the
> outcomes and what the next steps for the versioned objects session are.
>
> 1. There is a lot of interest in other projects using oslo versioned objects
> and it is worth progressing with this
> (https://review.openstack.org/#/c/127532).
> 2. jpipes and jharlow suggested experimenting/investigating google protocol
> buffers (https://developers.google.com/protocol-buffers/) instead of  the
> custom serialization and version code. This *could* be an implementation
> detail, but also could make the adoption by nova more complicated (as it has
> a different mechanism in place).
> 3. vish brought up one draw back of versioned objects: the difficulty in
> cherry picking commits for stable branches - Is this a show stopper?.
>
> Next steps:
> - Jay suggested making a second spec that would lay out what it would look
> like if we used google protocol buffers.
> - Dan: do you need some help in making this happen, do we need some
> volunteers?
> - Are there any other concrete things we can do to get this usable by other
> projects in a timely manner?

+1 on protocol buffers, but perhaps
http://kentonv.github.io/capnproto/ could be considered: its protocol
buffers v2 basically - from one of the originators of protocol
buffers. It has Python support available too, just like protocol
buffers.

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Versioned objects cross project sessions next steps

[Morgan Fainberg]

> On Nov 10, 2014, at 4:30 PM, Angus Salkeld  wrote:
> 
> Hi all
> 
> I just wanted to make sure we are all under the same understanding of the 
> outcomes and what the next steps for the versioned objects session are.
> 
> 1. There is a lot of interest in other projects using oslo versioned objects 
> and it is worth progressing with this 
> (https://review.openstack.org/#/c/127532 
> ).
> 2. jpipes and jharlow suggested experimenting/investigating google protocol 
> buffers (https://developers.google.com/protocol-buffers/) 
>  instead of  the custom 
> serialization and version code. This *could* be an implementation detail, but 
> also could make the adoption by nova more complicated (as it has a different 
> mechanism in place).
> 3. vish brought up one draw back of versioned objects: the difficulty in 
> cherry picking commits for stable branches - Is this a show stopper?.
> 
> Next steps:
> - Jay suggested making a second spec that would lay out what it would look 
> like if we used google protocol buffers.
> - Dan: do you need some help in making this happen, do we need some 
> volunteers?
> - Are there any other concrete things we can do to get this usable by other 
> projects in a timely manner?
> 
> Regards
> Angus

I am personally very interested in seeing what the spec for adopting protobuf 
ends up looking like in comparison to the current Nova implementation. The main 
reason is that there is already a fairly well driven mechanism in place to 
utilize the objects across different languages. This would allow for better 
consumption of any/all notifications from outside of the OpenStack ecosystem as 
well as cases (such as tokens) that might legitimately need to evolve the 
format in the longer-term without breaking *everything*. I also think that it 
would largely be possible to provide a way to wrap the current Nova object 
model into protobuf as a conversion/transition plan.

This doesn’t discount that protobuf may in-fact not be a good fit for our use 
cases and the Nova model does have some reasonable drive-time on it. It very 
well might be a good solution for all the projects.

Regarding stable patches, largely if the object is versioned differently it 
would be more difficult to back port a change. I, however, am going to hazard a 
guess that most of the issues are not directly revolving around the 
serialization/deserialization code (nor the data directly driven by it) meaning 
that the incremental difference in back port headaches would be relatively 
minor. In short, I think breaking the hard-tie between the underlying storage 
model and the in-memory/serialization mechanism would be good for all of the 
projects.

Jay/Dan/Angus, please let me know if I can be any help with exploring protobuf.

Cheers,
Morgan___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all] Versioned objects cross project sessions next steps

[Angus Salkeld]

Hi all

I just wanted to make sure we are all under the same understanding of the
outcomes and what the next steps for the versioned objects session are.

1. There is a lot of interest in other projects using oslo versioned
objects and it is worth progressing with this (
https://review.openstack.org/#/c/127532).
2. jpipes and jharlow suggested experimenting/investigating google protocol
buffers (https://developers.google.com/protocol-buffers/) instead of  the
custom serialization and version code. This *could* be an implementation
detail, but also could make the adoption by nova more complicated (as it
has a different mechanism in place).
3. vish brought up one draw back of versioned objects: the difficulty in
cherry picking commits for stable branches - Is this a show stopper?.

Next steps:
- Jay suggested making a second spec that would lay out what it would look
like if we used google protocol buffers.
- Dan: do you need some help in making this happen, do we need some
volunteers?
- Are there any other concrete things we can do to get this usable by other
projects in a timely manner?

Regards
Angus
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev