Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Andreas Scheuring]

I must admit, I really like this idea of getting rid of all the devstack
params. It's always a mess looking up the functionality of the various
variables in the code when trying out something new.

I also understand the concern that was raised by somebody (couldn't find
it anymore) that the Neutron config defaults must not necessarily be the
values that make sense for devstack. So what about having some code in
the neutron devstack plugin defining the default values without using
fancy variables - so just using the initset?

On the other hand I find it useful to have access to some variables like
the HOST_IP or PHYSICAL_NETWORK. I don't want to hardcode my IP into
every setting and maybe I want to define a interface or bridgemapping
that takes use of the PHYSICAL_NETWORK var.
However it's still possible to define you own var in the local.conf and
use them in post_config...

But you already kind of convinced me to to that post-config way with the
macvtap integration... cause then it's just a matter of documentation...

Thanks


-- 
-
Andreas (IRC: scheuran) 

On Fr, 2016-04-08 at 15:07 +, Sean M. Collins wrote:
> Prior to the introduction of local.conf, the only way to configure
> OpenStack components was to introduce code directly into DevStack, so
> that DevStack would pick it up then inject it into the configuration
> file.
> 
> This was because DevStack writes out new configuration files on each
> run, so it wasn't possible for you to make changes to any configuration
> file (nova.conf, neutron.conf, ml2_plugin.ini, etc..).
> 
> So, someone who wanted to set the Linux Bridge Agent's
> physical_interface_mappings setting for Neutron would have to use
> $LB_INTERFACE_MAPPINGS in DevStack, which would then be invoked by
> DevStack[1].
> 
> The local.conf functionality was introduced quite a while back, and
> I think it's time to have a conversation about why we should start
> moving away from the previous practice of declaring variables in
> DevStack, and then having them injected into the configuration files.
> 
> The biggest issue is: There is a disconnect between the developers
> using DevStack and someone who is an operator or who has been editing
> OpenStack conf files directly. So, for example I can tell you all about
> how DevStack has a bunch of variables for configuring Neutron (which is
> Not a Good Thing™), and how those go into DevStack and then end up coming
> out the other side in a Neutron configuration file.
> 
> Really, I would like to get rid of the intermediate layer (DevStack)
> and get both Devs and Deployers to be able to just say: Here's my
> neutron.conf - let's diff mine and yours and see what we need to sync.
> 
> Matt Kassawara and I have had this issue, since he's coming from the
> OSAD side, and I'm coming from the DevStack side. We both know what the
> Neutron configuration should end up as, but DevStack having its own set
> of variables and how those variables are handled and eventually rendered
> as a Neutron config file makes things more difficult than it needs to
> be, since Matt has to now go and learn about how DevStack handles all
> these Neutron specific variables.
> 
> The Neutron refactor[2] that I am working on, I am trying to configure
> as little as possible in DevStack. Neutron should be able to, out of the
> box, Just Work™. If it can't, then that needs to be fixed in Neutron.
> 
> Secondly, the Neutron refactor will be getting rid of all the things
> like $LB_INTERFACE_MAPPINGS - I would *much* prefer that someone using
> DevStack actually set the apropriate line in their local.conf
> 
> Such as:
> 
> [[post-config|/$Q_PLUGIN_CONF_FILE]]
> [linux_bridge]
> physical_interface_mappings = foo:bar
> 
> 
> The advantage of this is, when someone is working with DevStack, the
> things they are configuring are the same as all the other OpenStack 
> documentation.
> 
> For example, someone could read the Networking Guide, read the example
> configuration[3] and the only thing they'd need to learn is our syntax
> for specifying what file the contents go in (the 
> "[[post-config|/$Q_PLUGIN_CONF_FILE]]" piece).
> 
> Thoughts?
> 
> [1]: 
> https://github.com/openstack-dev/devstack/blob/1195a5b7394fc5b7a1cb1415978e9997701f5af1/lib/neutron_plugins/linuxbridge_agent#L63
> 
> [2]: https://review.openstack.org/168438
> 
> [3]: 
> http://docs.openstack.org/liberty/networking-guide/scenario-classic-lb.html#example-configuration
> 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Hirofumi Ichihara]

On 2016/04/12 8:02, Kevin Benton wrote:
Oh right, I'm definitely for eliminating these values from Devstack 
and just telling people to use post-config. I was just hesitant about 
advocating for their removal from neutron.
Yeah, my point is eliminating useless options from Devstack since we can 
change them in post-config of local.conf if need.





On Mon, Apr 11, 2016 at 3:55 PM, Brandon Logan 
> wrote:


On Mon, 2016-04-11 at 15:30 -0700, Kevin Benton wrote:
> >[1]:

https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L178
> >[2]:
https://github.com/openstack/nova/blob/master/nova/conf/virt.py#L164-L166
>
>
> This is a Nova option to decide how long to wait for Neutron to
> callback before considering a port failed to be wired. The time this
> will take will depend quite a bit on how heavily loaded the
system is.
> We can certainly try to get rid of it, but it means that we have to
> force assumptions about how quickly a system should give up waiting
> for wiring. It would be similar to getting rid of the option to
choose
> a timeout value for the API clients.
>
>
>
> >[3]:

https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L162
> >[4]:

https://github.com/openstack/neutron/blob/master/neutron/common/config.py#L53
>
>
> Neutron does not need to be deployed with keystone. This is how you
> disable it. Some operators do not have Neutron exposed to tenants so
> keystone is stripped away for performance since the only things
> communicating with Neutron are internal trusted services.

This is correct. In a large deployment the number of requests going to
keystone dramatically affects performance.  Do you think this needs to
be a devstack config option though?  I kind of don't think it does for
no better reason than it's easy to just change the option in the
neutron.conf and restart.

>
> On Mon, Apr 11, 2016 at 12:42 PM, Hirofumi Ichihara
> > wrote:
> I agree. Throughout I was reviewing Devstack over 3 cycles,
> I thought the same thing. Devstack often accepted
patches just
> adding option although we're not sure who really needs the
> options.
> There are many useless stuff in the options.
> For example, default value of devstack option is the same
> value as
> default in Projects. Please look at [1] and [2], [3] and
[4].
> Who uses these options?
>
> We can see such options in devstack throughout. I agree we
> will adjust default configurations and
> that documents in Neutron side. However, let's eliminate
such
> options are clearly useless first.
> And then we should do after we made necessary options clear.
>
> [1]:
>

https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L178
> [2]:
>
https://github.com/openstack/nova/blob/master/nova/conf/virt.py#L164-L166
> [3]:
>

https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L162
> [4]:
>

https://github.com/openstack/neutron/blob/master/neutron/common/config.py#L53
>
> Thanks,
> Hirofumi
>
>
> On 2016/04/09 0:07, Sean M. Collins wrote:
> Prior to the introduction of local.conf, the
only way
> to configure
> OpenStack components was to introduce code directly
> into DevStack, so
> that DevStack would pick it up then inject it
into the
> configuration
> file.
>
> This was because DevStack writes out new
configuration
> files on each
> run, so it wasn't possible for you to make
changes to
> any configuration
> file (nova.conf, neutron.conf, ml2_plugin.ini,
etc..).
>
> So, someone who wanted to set the Linux Bridge
Agent's
> physical_interface_mappings setting for Neutron
would
> have to use
> $LB_INTERFACE_MAPPINGS in DevStack, which would then
> be invoked by
> DevStack[1].
>
> The local.conf functionality was introduced quite a
> while back, and
> I think it's time to have a conversation about
why we
> should start
> moving away from the previous practice of declaring
>

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Kevin Benton]

Oh right, I'm definitely for eliminating these values from Devstack and
just telling people to use post-config. I was just hesitant about
advocating for their removal from neutron.

On Mon, Apr 11, 2016 at 3:55 PM, Brandon Logan 
wrote:

> On Mon, 2016-04-11 at 15:30 -0700, Kevin Benton wrote:
> > >[1]:
> https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L178
> > >[2]:
> https://github.com/openstack/nova/blob/master/nova/conf/virt.py#L164-L166
> >
> >
> > This is a Nova option to decide how long to wait for Neutron to
> > callback before considering a port failed to be wired. The time this
> > will take will depend quite a bit on how heavily loaded the system is.
> > We can certainly try to get rid of it, but it means that we have to
> > force assumptions about how quickly a system should give up waiting
> > for wiring. It would be similar to getting rid of the option to choose
> > a timeout value for the API clients.
> >
> >
> >
> > >[3]:
> https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L162
> > >[4]:
> https://github.com/openstack/neutron/blob/master/neutron/common/config.py#L53
> >
> >
> > Neutron does not need to be deployed with keystone. This is how you
> > disable it. Some operators do not have Neutron exposed to tenants so
> > keystone is stripped away for performance since the only things
> > communicating with Neutron are internal trusted services.
>
> This is correct. In a large deployment the number of requests going to
> keystone dramatically affects performance.  Do you think this needs to
> be a devstack config option though?  I kind of don't think it does for
> no better reason than it's easy to just change the option in the
> neutron.conf and restart.
>
> >
> > On Mon, Apr 11, 2016 at 12:42 PM, Hirofumi Ichihara
> >  wrote:
> > I agree. Throughout I was reviewing Devstack over 3 cycles,
> > I thought the same thing. Devstack often accepted patches just
> > adding option although we're not sure who really needs the
> > options.
> > There are many useless stuff in the options.
> > For example, default value of devstack option is the same
> > value as
> > default in Projects. Please look at [1] and [2], [3] and [4].
> > Who uses these options?
> >
> > We can see such options in devstack throughout. I agree we
> > will adjust default configurations and
> > that documents in Neutron side. However, let's eliminate such
> > options are clearly useless first.
> > And then we should do after we made necessary options clear.
> >
> > [1]:
> >
> https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L178
> > [2]:
> >
> https://github.com/openstack/nova/blob/master/nova/conf/virt.py#L164-L166
> > [3]:
> >
> https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L162
> > [4]:
> >
> https://github.com/openstack/neutron/blob/master/neutron/common/config.py#L53
> >
> > Thanks,
> > Hirofumi
> >
> >
> > On 2016/04/09 0:07, Sean M. Collins wrote:
> > Prior to the introduction of local.conf, the only way
> > to configure
> > OpenStack components was to introduce code directly
> > into DevStack, so
> > that DevStack would pick it up then inject it into the
> > configuration
> > file.
> >
> > This was because DevStack writes out new configuration
> > files on each
> > run, so it wasn't possible for you to make changes to
> > any configuration
> > file (nova.conf, neutron.conf, ml2_plugin.ini, etc..).
> >
> > So, someone who wanted to set the Linux Bridge Agent's
> > physical_interface_mappings setting for Neutron would
> > have to use
> > $LB_INTERFACE_MAPPINGS in DevStack, which would then
> > be invoked by
> > DevStack[1].
> >
> > The local.conf functionality was introduced quite a
> > while back, and
> > I think it's time to have a conversation about why we
> > should start
> > moving away from the previous practice of declaring
> > variables in
> > DevStack, and then having them injected into the
> > configuration files.
> >
> > The biggest issue is: There is a disconnect between
> > the developers
> > using DevStack and someone who is an operator or who
> > has been editing
> > OpenStack conf files directly. So, for example I can
> > tell you all about
> > how DevStack has a 

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Brandon Logan]

On Mon, 2016-04-11 at 15:30 -0700, Kevin Benton wrote:
> >[1]: 
> >https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L178
> >[2]: 
> >https://github.com/openstack/nova/blob/master/nova/conf/virt.py#L164-L166
> 
> 
> This is a Nova option to decide how long to wait for Neutron to
> callback before considering a port failed to be wired. The time this
> will take will depend quite a bit on how heavily loaded the system is.
> We can certainly try to get rid of it, but it means that we have to
> force assumptions about how quickly a system should give up waiting
> for wiring. It would be similar to getting rid of the option to choose
> a timeout value for the API clients.
> 
> 
> 
> >[3]: 
> >https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L162
> >[4]: 
> >https://github.com/openstack/neutron/blob/master/neutron/common/config.py#L53
> 
> 
> Neutron does not need to be deployed with keystone. This is how you
> disable it. Some operators do not have Neutron exposed to tenants so
> keystone is stripped away for performance since the only things
> communicating with Neutron are internal trusted services.

This is correct. In a large deployment the number of requests going to
keystone dramatically affects performance.  Do you think this needs to
be a devstack config option though?  I kind of don't think it does for
no better reason than it's easy to just change the option in the
neutron.conf and restart.

> 
> On Mon, Apr 11, 2016 at 12:42 PM, Hirofumi Ichihara
>  wrote:
> I agree. Throughout I was reviewing Devstack over 3 cycles,
> I thought the same thing. Devstack often accepted patches just
> adding option although we're not sure who really needs the
> options.
> There are many useless stuff in the options.
> For example, default value of devstack option is the same
> value as
> default in Projects. Please look at [1] and [2], [3] and [4].
> Who uses these options?
> 
> We can see such options in devstack throughout. I agree we
> will adjust default configurations and
> that documents in Neutron side. However, let's eliminate such
> options are clearly useless first.
> And then we should do after we made necessary options clear.
> 
> [1]:
> 
> https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L178
> [2]:
> 
> https://github.com/openstack/nova/blob/master/nova/conf/virt.py#L164-L166
> [3]:
> 
> https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L162
> [4]:
> 
> https://github.com/openstack/neutron/blob/master/neutron/common/config.py#L53
> 
> Thanks,
> Hirofumi
> 
> 
> On 2016/04/09 0:07, Sean M. Collins wrote:
> Prior to the introduction of local.conf, the only way
> to configure
> OpenStack components was to introduce code directly
> into DevStack, so
> that DevStack would pick it up then inject it into the
> configuration
> file.
> 
> This was because DevStack writes out new configuration
> files on each
> run, so it wasn't possible for you to make changes to
> any configuration
> file (nova.conf, neutron.conf, ml2_plugin.ini, etc..).
> 
> So, someone who wanted to set the Linux Bridge Agent's
> physical_interface_mappings setting for Neutron would
> have to use
> $LB_INTERFACE_MAPPINGS in DevStack, which would then
> be invoked by
> DevStack[1].
> 
> The local.conf functionality was introduced quite a
> while back, and
> I think it's time to have a conversation about why we
> should start
> moving away from the previous practice of declaring
> variables in
> DevStack, and then having them injected into the
> configuration files.
> 
> The biggest issue is: There is a disconnect between
> the developers
> using DevStack and someone who is an operator or who
> has been editing
> OpenStack conf files directly. So, for example I can
> tell you all about
> how DevStack has a bunch of variables for configuring
> Neutron (which is
> Not a Good Thing™), and how those go into DevStack and
> then end up coming
> out the other side in a Neutron configuration file.
> 
> Really, I would 

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Kevin Benton]

>[1]:
https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L178
>[2]:
https://github.com/openstack/nova/blob/master/nova/conf/virt.py#L164-L166

This is a Nova option to decide how long to wait for Neutron to callback
before considering a port failed to be wired. The time this will take will
depend quite a bit on how heavily loaded the system is. We can certainly
try to get rid of it, but it means that we have to force assumptions about
how quickly a system should give up waiting for wiring. It would be similar
to getting rid of the option to choose a timeout value for the API clients.

>[3]:
https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L162
>[4]:
https://github.com/openstack/neutron/blob/master/neutron/common/config.py#L53

Neutron does not need to be deployed with keystone. This is how you disable
it. Some operators do not have Neutron exposed to tenants so keystone is
stripped away for performance since the only things communicating with
Neutron are internal trusted services.

On Mon, Apr 11, 2016 at 12:42 PM, Hirofumi Ichihara <
ichihara.hirof...@lab.ntt.co.jp> wrote:

> I agree. Throughout I was reviewing Devstack over 3 cycles,
> I thought the same thing. Devstack often accepted patches just
> adding option although we're not sure who really needs the options.
> There are many useless stuff in the options.
> For example, default value of devstack option is the same value as
> default in Projects. Please look at [1] and [2], [3] and [4]. Who uses
> these options?
>
> We can see such options in devstack throughout. I agree we will adjust
> default configurations and
> that documents in Neutron side. However, let's eliminate such options are
> clearly useless first.
> And then we should do after we made necessary options clear.
>
> [1]:
> https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L178
> [2]:
> https://github.com/openstack/nova/blob/master/nova/conf/virt.py#L164-L166
> [3]:
> https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L162
> [4]:
> https://github.com/openstack/neutron/blob/master/neutron/common/config.py#L53
>
> Thanks,
> Hirofumi
>
>
> On 2016/04/09 0:07, Sean M. Collins wrote:
>
>> Prior to the introduction of local.conf, the only way to configure
>> OpenStack components was to introduce code directly into DevStack, so
>> that DevStack would pick it up then inject it into the configuration
>> file.
>>
>> This was because DevStack writes out new configuration files on each
>> run, so it wasn't possible for you to make changes to any configuration
>> file (nova.conf, neutron.conf, ml2_plugin.ini, etc..).
>>
>> So, someone who wanted to set the Linux Bridge Agent's
>> physical_interface_mappings setting for Neutron would have to use
>> $LB_INTERFACE_MAPPINGS in DevStack, which would then be invoked by
>> DevStack[1].
>>
>> The local.conf functionality was introduced quite a while back, and
>> I think it's time to have a conversation about why we should start
>> moving away from the previous practice of declaring variables in
>> DevStack, and then having them injected into the configuration files.
>>
>> The biggest issue is: There is a disconnect between the developers
>> using DevStack and someone who is an operator or who has been editing
>> OpenStack conf files directly. So, for example I can tell you all about
>> how DevStack has a bunch of variables for configuring Neutron (which is
>> Not a Good Thing™), and how those go into DevStack and then end up coming
>> out the other side in a Neutron configuration file.
>>
>> Really, I would like to get rid of the intermediate layer (DevStack)
>> and get both Devs and Deployers to be able to just say: Here's my
>> neutron.conf - let's diff mine and yours and see what we need to sync.
>>
>> Matt Kassawara and I have had this issue, since he's coming from the
>> OSAD side, and I'm coming from the DevStack side. We both know what the
>> Neutron configuration should end up as, but DevStack having its own set
>> of variables and how those variables are handled and eventually rendered
>> as a Neutron config file makes things more difficult than it needs to
>> be, since Matt has to now go and learn about how DevStack handles all
>> these Neutron specific variables.
>>
>> The Neutron refactor[2] that I am working on, I am trying to configure
>> as little as possible in DevStack. Neutron should be able to, out of the
>> box, Just Work™. If it can't, then that needs to be fixed in Neutron.
>>
>> Secondly, the Neutron refactor will be getting rid of all the things
>> like $LB_INTERFACE_MAPPINGS - I would *much* prefer that someone using
>> DevStack actually set the apropriate line in their local.conf
>>
>> Such as:
>>
>>  [[post-config|/$Q_PLUGIN_CONF_FILE]]
>>  [linux_bridge]
>>  physical_interface_mappings = foo:bar
>>
>>
>> The advantage of this is, when someone is working with DevStack, the
>> things they are configuring are the same as all the 

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Hirofumi Ichihara]

I agree. Throughout I was reviewing Devstack over 3 cycles,
I thought the same thing. Devstack often accepted patches just
adding option although we're not sure who really needs the options.
There are many useless stuff in the options.
For example, default value of devstack option is the same value as
default in Projects. Please look at [1] and [2], [3] and [4]. Who uses 
these options?


We can see such options in devstack throughout. I agree we will adjust 
default configurations and
that documents in Neutron side. However, let's eliminate such options 
are clearly useless first.

And then we should do after we made necessary options clear.

[1]: 
https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L178
[2]: 
https://github.com/openstack/nova/blob/master/nova/conf/virt.py#L164-L166
[3]: 
https://github.com/openstack-dev/devstack/blob/master/lib/neutron-legacy#L162
[4]: 
https://github.com/openstack/neutron/blob/master/neutron/common/config.py#L53


Thanks,
Hirofumi

On 2016/04/09 0:07, Sean M. Collins wrote:

Prior to the introduction of local.conf, the only way to configure
OpenStack components was to introduce code directly into DevStack, so
that DevStack would pick it up then inject it into the configuration
file.

This was because DevStack writes out new configuration files on each
run, so it wasn't possible for you to make changes to any configuration
file (nova.conf, neutron.conf, ml2_plugin.ini, etc..).

So, someone who wanted to set the Linux Bridge Agent's
physical_interface_mappings setting for Neutron would have to use
$LB_INTERFACE_MAPPINGS in DevStack, which would then be invoked by
DevStack[1].

The local.conf functionality was introduced quite a while back, and
I think it's time to have a conversation about why we should start
moving away from the previous practice of declaring variables in
DevStack, and then having them injected into the configuration files.

The biggest issue is: There is a disconnect between the developers
using DevStack and someone who is an operator or who has been editing
OpenStack conf files directly. So, for example I can tell you all about
how DevStack has a bunch of variables for configuring Neutron (which is
Not a Good Thing™), and how those go into DevStack and then end up coming
out the other side in a Neutron configuration file.

Really, I would like to get rid of the intermediate layer (DevStack)
and get both Devs and Deployers to be able to just say: Here's my
neutron.conf - let's diff mine and yours and see what we need to sync.

Matt Kassawara and I have had this issue, since he's coming from the
OSAD side, and I'm coming from the DevStack side. We both know what the
Neutron configuration should end up as, but DevStack having its own set
of variables and how those variables are handled and eventually rendered
as a Neutron config file makes things more difficult than it needs to
be, since Matt has to now go and learn about how DevStack handles all
these Neutron specific variables.

The Neutron refactor[2] that I am working on, I am trying to configure
as little as possible in DevStack. Neutron should be able to, out of the
box, Just Work™. If it can't, then that needs to be fixed in Neutron.

Secondly, the Neutron refactor will be getting rid of all the things
like $LB_INTERFACE_MAPPINGS - I would *much* prefer that someone using
DevStack actually set the apropriate line in their local.conf

Such as:

 [[post-config|/$Q_PLUGIN_CONF_FILE]]
 [linux_bridge]
 physical_interface_mappings = foo:bar


The advantage of this is, when someone is working with DevStack, the
things they are configuring are the same as all the other OpenStack 
documentation.

For example, someone could read the Networking Guide, read the example
configuration[3] and the only thing they'd need to learn is our syntax
for specifying what file the contents go in (the 
"[[post-config|/$Q_PLUGIN_CONF_FILE]]" piece).

Thoughts?

[1]: 
https://github.com/openstack-dev/devstack/blob/1195a5b7394fc5b7a1cb1415978e9997701f5af1/lib/neutron_plugins/linuxbridge_agent#L63

[2]: https://review.openstack.org/168438

[3]: 
http://docs.openstack.org/liberty/networking-guide/scenario-classic-lb.html#example-configuration






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Matt Kassawara]

Good news... the number of neutron options with a sane default value has
increased significantly in the last couple of releases. If you're looking
at the installation guide, it explicitly configures more options to
override bad values set by distribution packages. For deployment tools,
explicitly configuring certain options prevents surprises if default values
change. Automatic configuration file generation and the release notes
mechanism makes detection of these changes a bit easier, but building
confidence takes time.

On Mon, Apr 11, 2016 at 8:39 AM, Sean M. Collins  wrote:

> Armando M. wrote:
> > My understanding of the plan to overhaul the neutron (bloated) layer
> > present in DevStack being tackled in [1] has always been that this was
> > about trimming the layer rather than eliminating it altogether. Is this
> > email a reflection of a desire to change direction? If so, SeanC please
> > clarify because I am slightly confused.
>
> As part of the DevStack refactor, I am trying to shrink the amount of
> DevStack configuration variables that are carried around for Neutron.
> That is the part I am trying to eliminate. There must be support for
> Neutron in DevStack, if we ever wish to become the default networking
> project in OpenStack and successfully deprecate nova-network.
>
> > To the very minimum we'd need to find the right blend of config variables
> > which (in conjunction with some other *optional* local.conf extra juice)
> > produce the Neutron configuration files that we have in the gate, namely
> > OVS, LB and OVS+DVR, with their multi-node variants, and thus allow us to
> > get happy pass with Grenade/Tempest (if that means skipping some tests so
> > be it) across all the branches we currently gate against. The rest of the
> > layer can be stripped to the bare bone, but without it we're basically
> > gonna have to deal with long local.conf files with entire chunks of agent
> > files etc. thus making Neutron support for repos like devstack-gate and
> > project-config rather more painful (I am assuming we're gonna have to use
> > the new layer/approach at some point?). Bear in mind that the complexity
> > bubble needs to move/split, it's not just gonna burst and vanish :)
>
> It is my hope that we can start looking at some of these configurations,
> take a look at what puppet or ansible set, and realize that a lot of
> these options could just be defaults instead of making it the job of
> deployment tools to explicitly configure.
>
> > On another note, we'd have to keep in mind neutron_plugins that currently
> > have a place in the devstack tree and/or that rely on the existing
> > neutron_legacy bits. What's the plan for those (e.g. networking-[ovn,
> odl,
> > ...] et al)? Finally, what's the plan for switching in the gate?
>
> I think neutron_plugins will eventually be removed. Third party plugins
> like ovn, odl, et al most likely have DevStack plugins that supersede
> the code in neutron_plugins.
>
> For the OVS and LB agents, I think we need to clean them up, and again,
> see what can be configured by default.
>
> --
> Sean M. Collins
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Sean M. Collins]

Armando M. wrote:
> My understanding of the plan to overhaul the neutron (bloated) layer
> present in DevStack being tackled in [1] has always been that this was
> about trimming the layer rather than eliminating it altogether. Is this
> email a reflection of a desire to change direction? If so, SeanC please
> clarify because I am slightly confused.

As part of the DevStack refactor, I am trying to shrink the amount of
DevStack configuration variables that are carried around for Neutron.
That is the part I am trying to eliminate. There must be support for
Neutron in DevStack, if we ever wish to become the default networking
project in OpenStack and successfully deprecate nova-network.

> To the very minimum we'd need to find the right blend of config variables
> which (in conjunction with some other *optional* local.conf extra juice)
> produce the Neutron configuration files that we have in the gate, namely
> OVS, LB and OVS+DVR, with their multi-node variants, and thus allow us to
> get happy pass with Grenade/Tempest (if that means skipping some tests so
> be it) across all the branches we currently gate against. The rest of the
> layer can be stripped to the bare bone, but without it we're basically
> gonna have to deal with long local.conf files with entire chunks of agent
> files etc. thus making Neutron support for repos like devstack-gate and
> project-config rather more painful (I am assuming we're gonna have to use
> the new layer/approach at some point?). Bear in mind that the complexity
> bubble needs to move/split, it's not just gonna burst and vanish :)

It is my hope that we can start looking at some of these configurations,
take a look at what puppet or ansible set, and realize that a lot of
these options could just be defaults instead of making it the job of
deployment tools to explicitly configure.

> On another note, we'd have to keep in mind neutron_plugins that currently
> have a place in the devstack tree and/or that rely on the existing
> neutron_legacy bits. What's the plan for those (e.g. networking-[ovn, odl,
> ...] et al)? Finally, what's the plan for switching in the gate?

I think neutron_plugins will eventually be removed. Third party plugins
like ovn, odl, et al most likely have DevStack plugins that supersede
the code in neutron_plugins.

For the OVS and LB agents, I think we need to clean them up, and again,
see what can be configured by default.

-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Armando M.]

On 8 April 2016 at 11:06, Sean Dague  wrote:

> On 04/08/2016 12:05 PM, Morales, Victor wrote:
> > Agree, sometimes is hard to figure out what is the Devstack variable
> that will modify the configuration value.
> >
> > There is an effort to categorize the configuration options[1] of some of
> the projects.  I’m wondering if it could be possible to create category or
> field that specifies the Destack variable that changes this configuration
> value.
> >
> > [1]
> https://review.openstack.org/#/c/295543/8/specs/categorized-configuration-options.rst
>
> I really don't think that Devstack should leak that far back into real
> projects.
>
> Devstack variables make a ton of sense when you are communicating a
> higher level construct, and it needs to do some logic on it and possibly
> set multiple things.
>
> Devstack variables that are basically pass through for individual config
> vars aren't really a good idea. We try to -1 them all now. But a lot of
> leaked in.
>
> I think Sean Collins' plan is a good one.
>
> -Sean
>
>
The plethora of networking-specific config variables is a vestigial
presence of a time where local.conf and the plugin mechanism was not in
place in DevStack. Bear in mind that this is not a Neutron specific
problem: all projects are affected more or less equally.

I 100% agree with SeanD that the proposals of passthrough variables is to
be shot down. Those that are used to tune more than one variable at any
given time are more useful instead, as they reduce the number of moving
parts that will have to go into local.conf.

My understanding of the plan to overhaul the neutron (bloated) layer
present in DevStack being tackled in [1] has always been that this was
about trimming the layer rather than eliminating it altogether. Is this
email a reflection of a desire to change direction? If so, SeanC please
clarify because I am slightly confused.

To the very minimum we'd need to find the right blend of config variables
which (in conjunction with some other *optional* local.conf extra juice)
produce the Neutron configuration files that we have in the gate, namely
OVS, LB and OVS+DVR, with their multi-node variants, and thus allow us to
get happy pass with Grenade/Tempest (if that means skipping some tests so
be it) across all the branches we currently gate against. The rest of the
layer can be stripped to the bare bone, but without it we're basically
gonna have to deal with long local.conf files with entire chunks of agent
files etc. thus making Neutron support for repos like devstack-gate and
project-config rather more painful (I am assuming we're gonna have to use
the new layer/approach at some point?). Bear in mind that the complexity
bubble needs to move/split, it's not just gonna burst and vanish :)

On another note, we'd have to keep in mind neutron_plugins that currently
have a place in the devstack tree and/or that rely on the existing
neutron_legacy bits. What's the plan for those (e.g. networking-[ovn, odl,
...] et al)? Finally, what's the plan for switching in the gate?

Cheers,
Armando

[1] https://review.openstack.org/#/c/168438/


> --
> Sean Dague
> http://dague.net
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Dariusz Smigiel]

On 08.04.2016 14:47, Assaf Muller wrote:

On Fri, Apr 8, 2016 at 3:37 PM, Doug Wiegley
 wrote:

On Apr 8, 2016, at 1:28 PM, Sean M. Collins  wrote:

Assaf Muller wrote:

I do want to say that ML2's "mechanism_drivers" option probably does
not have a default for the same reason we do not have a default for
the core_plugin value, we don't want to play favorites. From Neutron's
point of view, ignoring the existence of Devstack and upstream CI, I
think that makes sense.


True, I do see your point.

I do however think, that if you do pick the ML2 plugin as your
core_plugin, it should have some mechanism drivers enabled by default. You
shouldn't have to pick core_plugin, then be forced to pick
mechanism_drivers. I'd rather see some mechanism_drivers already
enabled, and if you have a difference in opinion, set mechanism_drivers
in your local.conf.

I previously thought that a default there made no sense, but really, how is a 
default core plugin of ml2 with a default mech of local going to hurt anyone?

I was playing devil's advocate. I'm fine with picking ML2 and OVS+LB.
You will face resistance from people that have an interest in having
the ML2 reference implementation gone.

I think that idea of getting rid of devstack configuration is good thing.
Instead of learning how to setup something using internal devstack 
variables, would be good to use targeted values/settings.
But it also shows some kind of problem. We need to have good default 
configuration to setup devstack.

Right now, default All-in-one configuration is done by this [1]:

[[local|localrc]]
disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
# Optional, to enable tempest configuration as part of devstack
enable_service tempest


That's all what's necessary to run Neutron in devstack. For someone who 
doesn't want to learn about Neutron and how it works, it's enough (yes, 
there are people who don't want know anything about the best project, 
like Neutron :))
When Sean's idea will materialize, it's required to be sure, that we 
have default config, that can be just copy-pasted without thinking.


[1] https://wiki.openstack.org/wiki/NeutronDevstack

dasm
--
Darek Smigiel

We had a big argument of whether to have a default DNS resolver… 8.8.8.8 leaks 
internal info to a third-party, hypervisor default potentially leaks 
infrastructure details.  Not having a default there at least has some 
security/privacy implications.

There are likely things that we can start defaulting in a saner way.

doug




--
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Assaf Muller]

On Fri, Apr 8, 2016 at 3:37 PM, Doug Wiegley
 wrote:
>
>> On Apr 8, 2016, at 1:28 PM, Sean M. Collins  wrote:
>>
>> Assaf Muller wrote:
>>> I do want to say that ML2's "mechanism_drivers" option probably does
>>> not have a default for the same reason we do not have a default for
>>> the core_plugin value, we don't want to play favorites. From Neutron's
>>> point of view, ignoring the existence of Devstack and upstream CI, I
>>> think that makes sense.
>>>
>>
>> True, I do see your point.
>>
>> I do however think, that if you do pick the ML2 plugin as your
>> core_plugin, it should have some mechanism drivers enabled by default. You
>> shouldn't have to pick core_plugin, then be forced to pick
>> mechanism_drivers. I'd rather see some mechanism_drivers already
>> enabled, and if you have a difference in opinion, set mechanism_drivers
>> in your local.conf.
>
> I previously thought that a default there made no sense, but really, how is a 
> default core plugin of ml2 with a default mech of local going to hurt anyone?

I was playing devil's advocate. I'm fine with picking ML2 and OVS+LB.
You will face resistance from people that have an interest in having
the ML2 reference implementation gone.

>
> We had a big argument of whether to have a default DNS resolver… 8.8.8.8 
> leaks internal info to a third-party, hypervisor default potentially leaks 
> infrastructure details.  Not having a default there at least has some 
> security/privacy implications.
>
> There are likely things that we can start defaulting in a saner way.
>
> doug
>
>
>
>>
>> --
>> Sean M. Collins
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Doug Wiegley]

> On Apr 8, 2016, at 1:28 PM, Sean M. Collins  wrote:
> 
> Assaf Muller wrote:
>> I do want to say that ML2's "mechanism_drivers" option probably does
>> not have a default for the same reason we do not have a default for
>> the core_plugin value, we don't want to play favorites. From Neutron's
>> point of view, ignoring the existence of Devstack and upstream CI, I
>> think that makes sense.
>> 
> 
> True, I do see your point.
> 
> I do however think, that if you do pick the ML2 plugin as your
> core_plugin, it should have some mechanism drivers enabled by default. You
> shouldn't have to pick core_plugin, then be forced to pick
> mechanism_drivers. I'd rather see some mechanism_drivers already
> enabled, and if you have a difference in opinion, set mechanism_drivers
> in your local.conf.

I previously thought that a default there made no sense, but really, how is a 
default core plugin of ml2 with a default mech of local going to hurt anyone?

We had a big argument of whether to have a default DNS resolver… 8.8.8.8 leaks 
internal info to a third-party, hypervisor default potentially leaks 
infrastructure details.  Not having a default there at least has some 
security/privacy implications.

There are likely things that we can start defaulting in a saner way.

doug



> 
> -- 
> Sean M. Collins
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Sean M. Collins]

Assaf Muller wrote:
> I do want to say that ML2's "mechanism_drivers" option probably does
> not have a default for the same reason we do not have a default for
> the core_plugin value, we don't want to play favorites. From Neutron's
> point of view, ignoring the existence of Devstack and upstream CI, I
> think that makes sense.
> 

True, I do see your point.

I do however think, that if you do pick the ML2 plugin as your
core_plugin, it should have some mechanism drivers enabled by default. You
shouldn't have to pick core_plugin, then be forced to pick
mechanism_drivers. I'd rather see some mechanism_drivers already
enabled, and if you have a difference in opinion, set mechanism_drivers
in your local.conf.

-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Sean Dague]

On 04/08/2016 12:05 PM, Morales, Victor wrote:
> Agree, sometimes is hard to figure out what is the Devstack variable that 
> will modify the configuration value.
> 
> There is an effort to categorize the configuration options[1] of some of the 
> projects.  I’m wondering if it could be possible to create category or field 
> that specifies the Destack variable that changes this configuration value.
> 
> [1] 
> https://review.openstack.org/#/c/295543/8/specs/categorized-configuration-options.rst

I really don't think that Devstack should leak that far back into real
projects.

Devstack variables make a ton of sense when you are communicating a
higher level construct, and it needs to do some logic on it and possibly
set multiple things.

Devstack variables that are basically pass through for individual config
vars aren't really a good idea. We try to -1 them all now. But a lot of
leaked in.

I think Sean Collins' plan is a good one.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Assaf Muller]

On Fri, Apr 8, 2016 at 11:57 AM, Sean M. Collins  wrote:
> Edgar Magana wrote:
>> This is a very solid plan. Maybe to fair on the current state of the 
>> devstack with neutron functionality, what will be the disadvantage(s) of 
>> this change from your perspective?
>>
>
> A user's local.conf will probably get a little bigger - and I think a
> lot of the issues about Neutron's inability to run out of the box will
> be exposed.
>
> I mean let's face it - Neutron, installed from source, with no
> configuration Does Not Work™. There are not enough settings that have
> defaults set, for it to actually run.
>
> This was made painfully obvious to me when I had to make new revisions
> to the Neutron DevStack refactor, where I had to add more inisets, in
> order for Neutron to finish stacking correctly.
>
> Did you know, for example, that we rely on DevStack[1] to set the list
> of mechanism_drivers? Without this, you'll get an empty mechanism_driver
> list and nothing will ever be wired up.

I don't want to detract from what you're saying Sean, and I largely
agree that we can be more opinionated in Neutron and rely less on
Devstack. I also never liked Devstack's "macros" and have always
preferred configuring everything myself via local.conf when that was
made an option, simply because I already know how to configure Neutron
and I didn't want to learn Devstack's options. I do want to say that
ML2's "mechanism_drivers" option probably does not have a default for
the same reason we do not have a default for the core_plugin value, we
don't want to play favorites. From Neutron's point of view, ignoring
the existence of Devstack and upstream CI, I think that makes sense.

>
> I'm sure there is an argument that can be made about why there is no
> default for mechanism_drivers in ML2, since there are lots of options.
> But, I think that we can at least enable the ones that we have in
> Neutron's main tree. Packagers who make packages for each mechanism
> driver (LB, OVS, etc..) already had to handle things like
> mechanism_drivers in the Ml2 configuration already, so it shouldn't
> really impact them since we're only setting a default if nothing is set,
> and their packages should explicitly set it.
>
> [1]: 
> https://github.com/openstack-dev/devstack/blob/master/lib/neutron_plugins/ml2#L27
>
>
> --
> Sean M. Collins
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Mike Spreitzer]

I like this, for a reason not mentioned.  I am not a Neutron dev or 
operator and have never learned how to deploy Neutron; I have always 
driven it through DevStack.  The documentation for that has never been 
adequate, and I have concluded it will never be adequate.  With inadequate 
documentation, that layer isn't really doing its job anyway.  If there is 
no devstack layer getting in my way, I am willing to learn how to deploy 
Neutron from what I suppose is better documentation than I have been 
reading.  I understand that direct Neutron configuration is more 
troublesome than it should be.  Eliminating the devstack Neutron layer 
will only increase the pressure to both improve the documentation of 
Neutron configuration and simplify the subject, both of which are Good 
Things (TM).

Regards,
Mike



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Morales, Victor]

Agree, sometimes is hard to figure out what is the Devstack variable that will 
modify the configuration value.

There is an effort to categorize the configuration options[1] of some of the 
projects.  I’m wondering if it could be possible to create category or field 
that specifies the Destack variable that changes this configuration value.

[1] 
https://review.openstack.org/#/c/295543/8/specs/categorized-configuration-options.rst

Victor Morales






On 4/8/16, 10:07 AM, "Sean M. Collins"  wrote:

>Prior to the introduction of local.conf, the only way to configure
>OpenStack components was to introduce code directly into DevStack, so
>that DevStack would pick it up then inject it into the configuration
>file.
>
>This was because DevStack writes out new configuration files on each
>run, so it wasn't possible for you to make changes to any configuration
>file (nova.conf, neutron.conf, ml2_plugin.ini, etc..).
>
>So, someone who wanted to set the Linux Bridge Agent's
>physical_interface_mappings setting for Neutron would have to use
>$LB_INTERFACE_MAPPINGS in DevStack, which would then be invoked by
>DevStack[1].
>
>The local.conf functionality was introduced quite a while back, and
>I think it's time to have a conversation about why we should start
>moving away from the previous practice of declaring variables in
>DevStack, and then having them injected into the configuration files.
>
>The biggest issue is: There is a disconnect between the developers
>using DevStack and someone who is an operator or who has been editing
>OpenStack conf files directly. So, for example I can tell you all about
>how DevStack has a bunch of variables for configuring Neutron (which is
>Not a Good Thing™), and how those go into DevStack and then end up coming
>out the other side in a Neutron configuration file.
>
>Really, I would like to get rid of the intermediate layer (DevStack)
>and get both Devs and Deployers to be able to just say: Here's my
>neutron.conf - let's diff mine and yours and see what we need to sync.
>
>Matt Kassawara and I have had this issue, since he's coming from the
>OSAD side, and I'm coming from the DevStack side. We both know what the
>Neutron configuration should end up as, but DevStack having its own set
>of variables and how those variables are handled and eventually rendered
>as a Neutron config file makes things more difficult than it needs to
>be, since Matt has to now go and learn about how DevStack handles all
>these Neutron specific variables.
>
>The Neutron refactor[2] that I am working on, I am trying to configure
>as little as possible in DevStack. Neutron should be able to, out of the
>box, Just Work™. If it can't, then that needs to be fixed in Neutron.
>
>Secondly, the Neutron refactor will be getting rid of all the things
>like $LB_INTERFACE_MAPPINGS - I would *much* prefer that someone using
>DevStack actually set the apropriate line in their local.conf
>
>Such as:
>
>[[post-config|/$Q_PLUGIN_CONF_FILE]]
>[linux_bridge]
>physical_interface_mappings = foo:bar
>
>
>The advantage of this is, when someone is working with DevStack, the
>things they are configuring are the same as all the other OpenStack 
>documentation.
>
>For example, someone could read the Networking Guide, read the example
>configuration[3] and the only thing they'd need to learn is our syntax
>for specifying what file the contents go in (the 
>"[[post-config|/$Q_PLUGIN_CONF_FILE]]" piece).
>
>Thoughts?
>
>[1]: 
>https://github.com/openstack-dev/devstack/blob/1195a5b7394fc5b7a1cb1415978e9997701f5af1/lib/neutron_plugins/linuxbridge_agent#L63
>
>[2]: https://review.openstack.org/168438
>
>[3]: 
>http://docs.openstack.org/liberty/networking-guide/scenario-classic-lb.html#example-configuration
>
>-- 
>Sean M. Collins
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Sean M. Collins]

Edgar Magana wrote:
> This is a very solid plan. Maybe to fair on the current state of the devstack 
> with neutron functionality, what will be the disadvantage(s) of this change 
> from your perspective?
> 

A user's local.conf will probably get a little bigger - and I think a
lot of the issues about Neutron's inability to run out of the box will
be exposed.

I mean let's face it - Neutron, installed from source, with no
configuration Does Not Work™. There are not enough settings that have
defaults set, for it to actually run.

This was made painfully obvious to me when I had to make new revisions
to the Neutron DevStack refactor, where I had to add more inisets, in
order for Neutron to finish stacking correctly.

Did you know, for example, that we rely on DevStack[1] to set the list
of mechanism_drivers? Without this, you'll get an empty mechanism_driver
list and nothing will ever be wired up.

I'm sure there is an argument that can be made about why there is no
default for mechanism_drivers in ML2, since there are lots of options.
But, I think that we can at least enable the ones that we have in
Neutron's main tree. Packagers who make packages for each mechanism
driver (LB, OVS, etc..) already had to handle things like
mechanism_drivers in the Ml2 configuration already, so it shouldn't
really impact them since we're only setting a default if nothing is set,
and their packages should explicitly set it.

[1]: 
https://github.com/openstack-dev/devstack/blob/master/lib/neutron_plugins/ml2#L27


-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Edgar Magana]

This is a very solid plan. Maybe to fair on the current state of the devstack 
with neutron functionality, what will be the disadvantage(s) of this change 
from your perspective?

Edgar



On 4/8/16, 8:07 AM, "Sean M. Collins"  wrote:

>Prior to the introduction of local.conf, the only way to configure
>OpenStack components was to introduce code directly into DevStack, so
>that DevStack would pick it up then inject it into the configuration
>file.
>
>This was because DevStack writes out new configuration files on each
>run, so it wasn't possible for you to make changes to any configuration
>file (nova.conf, neutron.conf, ml2_plugin.ini, etc..).
>
>So, someone who wanted to set the Linux Bridge Agent's
>physical_interface_mappings setting for Neutron would have to use
>$LB_INTERFACE_MAPPINGS in DevStack, which would then be invoked by
>DevStack[1].
>
>The local.conf functionality was introduced quite a while back, and
>I think it's time to have a conversation about why we should start
>moving away from the previous practice of declaring variables in
>DevStack, and then having them injected into the configuration files.
>
>The biggest issue is: There is a disconnect between the developers
>using DevStack and someone who is an operator or who has been editing
>OpenStack conf files directly. So, for example I can tell you all about
>how DevStack has a bunch of variables for configuring Neutron (which is
>Not a Good Thing™), and how those go into DevStack and then end up coming
>out the other side in a Neutron configuration file.
>
>Really, I would like to get rid of the intermediate layer (DevStack)
>and get both Devs and Deployers to be able to just say: Here's my
>neutron.conf - let's diff mine and yours and see what we need to sync.
>
>Matt Kassawara and I have had this issue, since he's coming from the
>OSAD side, and I'm coming from the DevStack side. We both know what the
>Neutron configuration should end up as, but DevStack having its own set
>of variables and how those variables are handled and eventually rendered
>as a Neutron config file makes things more difficult than it needs to
>be, since Matt has to now go and learn about how DevStack handles all
>these Neutron specific variables.
>
>The Neutron refactor[2] that I am working on, I am trying to configure
>as little as possible in DevStack. Neutron should be able to, out of the
>box, Just Work™. If it can't, then that needs to be fixed in Neutron.
>
>Secondly, the Neutron refactor will be getting rid of all the things
>like $LB_INTERFACE_MAPPINGS - I would *much* prefer that someone using
>DevStack actually set the apropriate line in their local.conf
>
>Such as:
>
>[[post-config|/$Q_PLUGIN_CONF_FILE]]
>[linux_bridge]
>physical_interface_mappings = foo:bar
>
>
>The advantage of this is, when someone is working with DevStack, the
>things they are configuring are the same as all the other OpenStack 
>documentation.
>
>For example, someone could read the Networking Guide, read the example
>configuration[3] and the only thing they'd need to learn is our syntax
>for specifying what file the contents go in (the 
>"[[post-config|/$Q_PLUGIN_CONF_FILE]]" piece).
>
>Thoughts?
>
>[1]: 
>https://github.com/openstack-dev/devstack/blob/1195a5b7394fc5b7a1cb1415978e9997701f5af1/lib/neutron_plugins/linuxbridge_agent#L63
>
>[2]: https://review.openstack.org/168438
>
>[3]: 
>http://docs.openstack.org/liberty/networking-guide/scenario-classic-lb.html#example-configuration
>
>-- 
>Sean M. Collins
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [devstack][neutron] Eliminating the DevStack layer

[Sean M. Collins]

Prior to the introduction of local.conf, the only way to configure
OpenStack components was to introduce code directly into DevStack, so
that DevStack would pick it up then inject it into the configuration
file.

This was because DevStack writes out new configuration files on each
run, so it wasn't possible for you to make changes to any configuration
file (nova.conf, neutron.conf, ml2_plugin.ini, etc..).

So, someone who wanted to set the Linux Bridge Agent's
physical_interface_mappings setting for Neutron would have to use
$LB_INTERFACE_MAPPINGS in DevStack, which would then be invoked by
DevStack[1].

The local.conf functionality was introduced quite a while back, and
I think it's time to have a conversation about why we should start
moving away from the previous practice of declaring variables in
DevStack, and then having them injected into the configuration files.

The biggest issue is: There is a disconnect between the developers
using DevStack and someone who is an operator or who has been editing
OpenStack conf files directly. So, for example I can tell you all about
how DevStack has a bunch of variables for configuring Neutron (which is
Not a Good Thing™), and how those go into DevStack and then end up coming
out the other side in a Neutron configuration file.

Really, I would like to get rid of the intermediate layer (DevStack)
and get both Devs and Deployers to be able to just say: Here's my
neutron.conf - let's diff mine and yours and see what we need to sync.

Matt Kassawara and I have had this issue, since he's coming from the
OSAD side, and I'm coming from the DevStack side. We both know what the
Neutron configuration should end up as, but DevStack having its own set
of variables and how those variables are handled and eventually rendered
as a Neutron config file makes things more difficult than it needs to
be, since Matt has to now go and learn about how DevStack handles all
these Neutron specific variables.

The Neutron refactor[2] that I am working on, I am trying to configure
as little as possible in DevStack. Neutron should be able to, out of the
box, Just Work™. If it can't, then that needs to be fixed in Neutron.

Secondly, the Neutron refactor will be getting rid of all the things
like $LB_INTERFACE_MAPPINGS - I would *much* prefer that someone using
DevStack actually set the apropriate line in their local.conf

Such as:

[[post-config|/$Q_PLUGIN_CONF_FILE]]
[linux_bridge]
physical_interface_mappings = foo:bar


The advantage of this is, when someone is working with DevStack, the
things they are configuring are the same as all the other OpenStack 
documentation.

For example, someone could read the Networking Guide, read the example
configuration[3] and the only thing they'd need to learn is our syntax
for specifying what file the contents go in (the 
"[[post-config|/$Q_PLUGIN_CONF_FILE]]" piece).

Thoughts?

[1]: 
https://github.com/openstack-dev/devstack/blob/1195a5b7394fc5b7a1cb1415978e9997701f5af1/lib/neutron_plugins/linuxbridge_agent#L63

[2]: https://review.openstack.org/168438

[3]: 
http://docs.openstack.org/liberty/networking-guide/scenario-classic-lb.html#example-configuration

-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev