Re: [openstack-dev] Policy around Requirements Adds

2014-07-08 Thread Sean Dague 
On 07/08/2014 04:33 AM, Mark McLoughlin wrote:
> On Mon, 2014-07-07 at 16:46 -0400, Sean Dague wrote:
>> This thread was unfortunately hidden under a project specific tag (I
>> have thus stripped all the tags).
>>
>> The crux of the argument here is the following:
>>
>> Is a stackforge project project able to propose additions to
>> global-requirements.txt that aren't used by any projects in OpenStack.
>>
>> I believe the answer is firmly *no*.
>>
>> global-requirements.txt provides a way for us to have a single point of
>> vetting for requirements for OpenStack. It lets us assess licensing,
>> maturity, current state of packaging, python3 support, all in one place.
>> And it lets us enforce that integration of OpenStack projects all run
>> under a well understood set of requirements.
> 
> Allowing Stackforge projects use this as their base set of dependencies,
> while still taking additional dependencies makes sense to me. I don't
> really understand this GTFO stance.
> 
> Solum wants to depend on mistralclient - that seems like a perfectly
> reasonable thing to want to do. And they also appear to not want to
> stray any further from the base set of dependencies shared by OpenStack
> projects - that also seems like a good thing.
> 
> Now, perhaps the mechanics are tricky, and perhaps we don't want to
> enable Stackforge projects do stuff like pin to a different version of
> SQLalchemy, and perhaps this proposal isn't the ideal solution, and
> perhaps infra/others don't want to spend a lot of energy on something
> specifically for Stackforge projects ... but I don't see something
> fundamentally wrong with what they want to do.

Once it's in global requirements, any OpenStack project can include it
in their requirements. Modifying that file for only stackforge projects
is what I'm against.

If the solum team would like to write up a partial sync mechanism,
that's fine. It just needs to not be impacting the enforcement mechanism
we actually need for OpenStack projects.

-Sean

-- 
Sean Dague
http://dague.net



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Policy around Requirements Adds

2014-07-08 Thread Mark McLoughlin 
On Tue, 2014-07-08 at 06:26 -0400, Sean Dague wrote:
> On 07/08/2014 04:33 AM, Mark McLoughlin wrote:
> > On Mon, 2014-07-07 at 16:46 -0400, Sean Dague wrote:
> >> This thread was unfortunately hidden under a project specific tag (I
> >> have thus stripped all the tags).
> >>
> >> The crux of the argument here is the following:
> >>
> >> Is a stackforge project project able to propose additions to
> >> global-requirements.txt that aren't used by any projects in OpenStack.
> >>
> >> I believe the answer is firmly *no*.
> >>
> >> global-requirements.txt provides a way for us to have a single point of
> >> vetting for requirements for OpenStack. It lets us assess licensing,
> >> maturity, current state of packaging, python3 support, all in one place.
> >> And it lets us enforce that integration of OpenStack projects all run
> >> under a well understood set of requirements.
> > 
> > Allowing Stackforge projects use this as their base set of dependencies,
> > while still taking additional dependencies makes sense to me. I don't
> > really understand this GTFO stance.
> > 
> > Solum wants to depend on mistralclient - that seems like a perfectly
> > reasonable thing to want to do. And they also appear to not want to
> > stray any further from the base set of dependencies shared by OpenStack
> > projects - that also seems like a good thing.
> > 
> > Now, perhaps the mechanics are tricky, and perhaps we don't want to
> > enable Stackforge projects do stuff like pin to a different version of
> > SQLalchemy, and perhaps this proposal isn't the ideal solution, and
> > perhaps infra/others don't want to spend a lot of energy on something
> > specifically for Stackforge projects ... but I don't see something
> > fundamentally wrong with what they want to do.
> 
> Once it's in global requirements, any OpenStack project can include it
> in their requirements. Modifying that file for only stackforge projects
> is what I'm against.
> 
> If the solum team would like to write up a partial sync mechanism,
> that's fine. It just needs to not be impacting the enforcement mechanism
> we actually need for OpenStack projects.

Totally agree. Solum taking a dependency on mistralclient shouldn't e.g.
allow glance to take a dependency on mistralclient.

Mark.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Policy around Requirements Adds (was: New class of requirements for Stackforge projects)

2014-07-07 Thread Sean Dague 
This thread was unfortunately hidden under a project specific tag (I
have thus stripped all the tags).

The crux of the argument here is the following:

Is a stackforge project project able to propose additions to
global-requirements.txt that aren't used by any projects in OpenStack.

I believe the answer is firmly *no*.

global-requirements.txt provides a way for us to have a single point of
vetting for requirements for OpenStack. It lets us assess licensing,
maturity, current state of packaging, python3 support, all in one place.
And it lets us enforce that integration of OpenStack projects all run
under a well understood set of requirements.

The requirements sync that happens after requirements land is basically
just a nicety for getting openstack projects to the tested state by
eventual consistency.

If a stackforge project wants to be limited by global-requirements,
that's cool. We have a mechanism for that. However, they are accepting
that they will be limited by it. That means they live with how the
OpenStack project establishes that list. It specifically means they
*don't* get to propose any new requirements.

Basically in this case Solum wants to have it's cake and eat it to. Both
be enforced on requirements, and not be enforced. Or some 3rd thing that
means the same as that.

The near term fix is to remove solum from projects.txt.

On 06/26/2014 02:00 AM, Adrian Otto wrote:
> Ok,
> 
> I submitted and abandoned a couple of reviews[1][2] for a solution aimed
> to meet my goals without adding a new per-project requirements file. The
> flaw with this approach is that pip may install other requirements when
> installing the one(s) loaded from the fallback mirror, and those may
> conflict with the ones loaded from the primary mirror.
> 
> After discussing this further in #openstack-infra this evening, we
> should give serious consideration to adding python-mistralclient to
> global requirements. I have posted a review[3] for that to get input
> from the requirements review team.
> 
> Thanks,
> 
> Adrian
> 
> [1] https://review.openstack.org/102716
> [2] https://review.openstack.org/102719
> [3] https://review.openstack.org/102738
> 
> 
> On Jun 25, 2014, at 9:51 PM, Matthew Oliver  > wrote:
> 
>>
>> On Jun 26, 2014 12:12 PM, "Angus Salkeld" > > wrote:
>> >
> On 25/06/14 15:13, Clark Boylan wrote:
>> On Tue, Jun 24, 2014 at 9:54 PM, Adrian Otto
>>> mailto:adrian.o...@rackspace.com>> wrote:
>>> Hello,
>>>
>>> Solum has run into a constraint with the current scheme for
>>> requirements management within the OpenStack CI system. We have a
>>> proposal for dealing with this constraint that involves making a
>>> contribution to openstack-infra. This message explains the constraint,
>>> and our proposal for addressing it.
>>>
>>> == Background ==
>>>
>>> OpenStack uses a list of global requirements in the requirements
>>> repo[1], and each project has it’s own requirements.txt and
>>> test-requirements.txt files. The requirements are satisfied by gate
>>> jobs using pip configured to use the pypi.openstack.org
>>>  mirror, which is periodically updated
>>> with new content from pypi.python.org . One
>>> motivation for doing this is that pypi.python.org
>>>  may not be as fast or as reliable as a local
>>> mirror. The gate/check jobs for the projects use the OpenStack
>>> internal pypi mirror to ensure stability.
>>>
>>> The OpenStack CI system will sync up the requirements across all
>>> the official projects and will create reviews in the participating
>>> projects for any mis-matches. Solum is one of these projects, and
>>> enjoys this feature.
>>>
>>> Another motivation is so that users of OpenStack will have one
>>> single set of python package requirements/dependencies to install and
>>> run the individual OpenStack components.
>>>
>>> == Problem ==
>>>
>>> Stackforge projects listed in openstack/requirements/projects.txt
>>> that decide to depend on each other (for example, Solum wanting to
>>> list mistralclient as a requirement) are unable to, because they are
>>> not yet integrated, and are not listed in
>>> openstack/requirements/global-requirements.txt yet. This means that in
>>> order to depend on each other, a project must withdraw from
>>> projects.txt and begin using pip with pypi.poython.org
>>>  to satisfy all of their requirements.I
>>> strongly dislike this option.
>>>
>>> Mistral is still evolving rapidly, and we don’t think it makes
>>> sense for them to pursue integration wight now. The upstream
>>> distributions who include packages to support OpenStack will also
>>> prefer not to deal with a requirement that will be cutting a new
>>> version every week or two in order to satisfy evolving needs as Solum
>>> and other consumers of Mistral help refine how it works.
>>>
>>> == Proposal =

Re: [openstack-dev] Policy around Requirements Adds (was: New class of requirements for Stackforge projects)

2014-07-07 Thread Joe Gordon 
On Jul 7, 2014 4:48 PM, "Sean Dague"  wrote:
>
> This thread was unfortunately hidden under a project specific tag (I
> have thus stripped all the tags).
>
> The crux of the argument here is the following:
>
> Is a stackforge project project able to propose additions to
> global-requirements.txt that aren't used by any projects in OpenStack.
>
> I believe the answer is firmly *no*.

++

>
> global-requirements.txt provides a way for us to have a single point of
> vetting for requirements for OpenStack. It lets us assess licensing,
> maturity, current state of packaging, python3 support, all in one place.
> And it lets us enforce that integration of OpenStack projects all run
> under a well understood set of requirements.
>
> The requirements sync that happens after requirements land is basically
> just a nicety for getting openstack projects to the tested state by
> eventual consistency.
>
> If a stackforge project wants to be limited by global-requirements,
> that's cool. We have a mechanism for that. However, they are accepting
> that they will be limited by it. That means they live with how the
> OpenStack project establishes that list. It specifically means they
> *don't* get to propose any new requirements.
>
> Basically in this case Solum wants to have it's cake and eat it to. Both
> be enforced on requirements, and not be enforced. Or some 3rd thing that
> means the same as that.
>
> The near term fix is to remove solum from projects.txt.

The email included below mentions an additional motivation for using
global-requirements is to avoid using pypi.python.org and instead use
pypi.openstack.org for speed and reliability. Perhaps there is a way we can
support use case for stackforge projects not in projects.txt? I thought I
saw something the other day about adding a full pypi mirror to OpenStack
infra.

>
> On 06/26/2014 02:00 AM, Adrian Otto wrote:
> > Ok,
> >
> > I submitted and abandoned a couple of reviews[1][2] for a solution aimed
> > to meet my goals without adding a new per-project requirements file. The
> > flaw with this approach is that pip may install other requirements when
> > installing the one(s) loaded from the fallback mirror, and those may
> > conflict with the ones loaded from the primary mirror.
> >
> > After discussing this further in #openstack-infra this evening, we
> > should give serious consideration to adding python-mistralclient to
> > global requirements. I have posted a review[3] for that to get input
> > from the requirements review team.
> >
> > Thanks,
> >
> > Adrian
> >
> > [1] https://review.openstack.org/102716
> > [2] https://review.openstack.org/102719
> > [3] https://review.openstack.org/102738
> > 
> >
> > On Jun 25, 2014, at 9:51 PM, Matthew Oliver  > > wrote:
> >
> >>
> >> On Jun 26, 2014 12:12 PM, "Angus Salkeld"  >> > wrote:
> >> >
> > On 25/06/14 15:13, Clark Boylan wrote:
> >> On Tue, Jun 24, 2014 at 9:54 PM, Adrian Otto
> >>> mailto:adrian.o...@rackspace.com>> wrote:
> >>> Hello,
> >>>
> >>> Solum has run into a constraint with the current scheme for
> >>> requirements management within the OpenStack CI system. We have a
> >>> proposal for dealing with this constraint that involves making a
> >>> contribution to openstack-infra. This message explains the constraint,
> >>> and our proposal for addressing it.
> >>>
> >>> == Background ==
> >>>
> >>> OpenStack uses a list of global requirements in the requirements
> >>> repo[1], and each project has it’s own requirements.txt and
> >>> test-requirements.txt files. The requirements are satisfied by gate
> >>> jobs using pip configured to use the pypi.openstack.org
> >>>  mirror, which is periodically updated
> >>> with new content from pypi.python.org . One
> >>> motivation for doing this is that pypi.python.org
> >>>  may not be as fast or as reliable as a local
> >>> mirror. The gate/check jobs for the projects use the OpenStack
> >>> internal pypi mirror to ensure stability.
> >>>
> >>> The OpenStack CI system will sync up the requirements across all
> >>> the official projects and will create reviews in the participating
> >>> projects for any mis-matches. Solum is one of these projects, and
> >>> enjoys this feature.
> >>>
> >>> Another motivation is so that users of OpenStack will have one
> >>> single set of python package requirements/dependencies to install and
> >>> run the individual OpenStack components.
> >>>
> >>> == Problem ==
> >>>
> >>> Stackforge projects listed in openstack/requirements/projects.txt
> >>> that decide to depend on each other (for example, Solum wanting to
> >>> list mistralclient as a requirement) are unable to, because they are
> >>> not yet integrated, and are not listed in
> >>> openstack/requirements/global-requirements.txt yet. This means that in
> >>> order to depend on each other, a project

Re: [openstack-dev] Policy around Requirements Adds (was: New class of requirements for Stackforge projects)

2014-07-07 Thread Clark Boylan 
On Mon, Jul 7, 2014 at 3:45 PM, Joe Gordon  wrote:
>
> On Jul 7, 2014 4:48 PM, "Sean Dague"  wrote:
>>
>> This thread was unfortunately hidden under a project specific tag (I
>> have thus stripped all the tags).
>>
>> The crux of the argument here is the following:
>>
>> Is a stackforge project project able to propose additions to
>> global-requirements.txt that aren't used by any projects in OpenStack.
>>
>> I believe the answer is firmly *no*.
>
> ++
>
>>
>> global-requirements.txt provides a way for us to have a single point of
>> vetting for requirements for OpenStack. It lets us assess licensing,
>> maturity, current state of packaging, python3 support, all in one place.
>> And it lets us enforce that integration of OpenStack projects all run
>> under a well understood set of requirements.
>>
>> The requirements sync that happens after requirements land is basically
>> just a nicety for getting openstack projects to the tested state by
>> eventual consistency.
>>
>> If a stackforge project wants to be limited by global-requirements,
>> that's cool. We have a mechanism for that. However, they are accepting
>> that they will be limited by it. That means they live with how the
>> OpenStack project establishes that list. It specifically means they
>> *don't* get to propose any new requirements.
>>
>> Basically in this case Solum wants to have it's cake and eat it to. Both
>> be enforced on requirements, and not be enforced. Or some 3rd thing that
>> means the same as that.
>>
>> The near term fix is to remove solum from projects.txt.
>
> The email included below mentions an additional motivation for using
> global-requirements is to avoid using pypi.python.org and instead use
> pypi.openstack.org for speed and reliability. Perhaps there is a way we can
> support use case for stackforge projects not in projects.txt? I thought I
> saw something the other day about adding a full pypi mirror to OpenStack
> infra.
>
This is done. All tests are now run against a bandersnatch built full
mirror of pypi. Enforcement of the global requirements is performed
via the enforcement jobs.
>>
>> On 06/26/2014 02:00 AM, Adrian Otto wrote:
>> > Ok,
>> >
>> > I submitted and abandoned a couple of reviews[1][2] for a solution aimed
>> > to meet my goals without adding a new per-project requirements file. The
>> > flaw with this approach is that pip may install other requirements when
>> > installing the one(s) loaded from the fallback mirror, and those may
>> > conflict with the ones loaded from the primary mirror.
>> >
>> > After discussing this further in #openstack-infra this evening, we
>> > should give serious consideration to adding python-mistralclient to
>> > global requirements. I have posted a review[3] for that to get input
>> > from the requirements review team.
>> >
>> > Thanks,
>> >
>> > Adrian
>> >
>> > [1] https://review.openstack.org/102716
>> > [2] https://review.openstack.org/102719
>> > [3] https://review.openstack.org/102738
>> > 
>> >
>> > On Jun 25, 2014, at 9:51 PM, Matthew Oliver > > > wrote:
>> >
>> >>
>> >> On Jun 26, 2014 12:12 PM, "Angus Salkeld" > >> > wrote:
>> >> >
>> > On 25/06/14 15:13, Clark Boylan wrote:
>> >> On Tue, Jun 24, 2014 at 9:54 PM, Adrian Otto
>> >>> mailto:adrian.o...@rackspace.com>> wrote:
>> >>> Hello,
>> >>>
>> >>> Solum has run into a constraint with the current scheme for
>> >>> requirements management within the OpenStack CI system. We have a
>> >>> proposal for dealing with this constraint that involves making a
>> >>> contribution to openstack-infra. This message explains the constraint,
>> >>> and our proposal for addressing it.
>> >>>
>> >>> == Background ==
>> >>>
>> >>> OpenStack uses a list of global requirements in the requirements
>> >>> repo[1], and each project has it’s own requirements.txt and
>> >>> test-requirements.txt files. The requirements are satisfied by gate
>> >>> jobs using pip configured to use the pypi.openstack.org
>> >>>  mirror, which is periodically updated
>> >>> with new content from pypi.python.org . One
>> >>> motivation for doing this is that pypi.python.org
>> >>>  may not be as fast or as reliable as a local
>> >>> mirror. The gate/check jobs for the projects use the OpenStack
>> >>> internal pypi mirror to ensure stability.
>> >>>
>> >>> The OpenStack CI system will sync up the requirements across all
>> >>> the official projects and will create reviews in the participating
>> >>> projects for any mis-matches. Solum is one of these projects, and
>> >>> enjoys this feature.
>> >>>
>> >>> Another motivation is so that users of OpenStack will have one
>> >>> single set of python package requirements/dependencies to install and
>> >>> run the individual OpenStack components.
>> >>>
>> >>> == Problem ==
>> >>>
>> >>> Stackforge projects listed in openstack/requiremen

Re: [openstack-dev] Policy around Requirements Adds (was: New class of requirements for Stackforge projects)

2014-07-08 Thread Mark McLoughlin 
On Mon, 2014-07-07 at 16:46 -0400, Sean Dague wrote:
> This thread was unfortunately hidden under a project specific tag (I
> have thus stripped all the tags).
> 
> The crux of the argument here is the following:
> 
> Is a stackforge project project able to propose additions to
> global-requirements.txt that aren't used by any projects in OpenStack.
> 
> I believe the answer is firmly *no*.
> 
> global-requirements.txt provides a way for us to have a single point of
> vetting for requirements for OpenStack. It lets us assess licensing,
> maturity, current state of packaging, python3 support, all in one place.
> And it lets us enforce that integration of OpenStack projects all run
> under a well understood set of requirements.

Allowing Stackforge projects use this as their base set of dependencies,
while still taking additional dependencies makes sense to me. I don't
really understand this GTFO stance.

Solum wants to depend on mistralclient - that seems like a perfectly
reasonable thing to want to do. And they also appear to not want to
stray any further from the base set of dependencies shared by OpenStack
projects - that also seems like a good thing.

Now, perhaps the mechanics are tricky, and perhaps we don't want to
enable Stackforge projects do stuff like pin to a different version of
SQLalchemy, and perhaps this proposal isn't the ideal solution, and
perhaps infra/others don't want to spend a lot of energy on something
specifically for Stackforge projects ... but I don't see something
fundamentally wrong with what they want to do.

Mark.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev