Re: [Openstack-operators] Openstack zun on centos???

[Fox, Kevin M]

kolla installs it via containers.

Thanks,
Kevin

From: Ignazio Cassano [ignaziocass...@gmail.com]
Sent: Wednesday, November 14, 2018 10:48 AM
To: Eduardo Gonzalez
Cc: OpenStack Operators
Subject: Re: [Openstack-operators] Openstack zun on centos???

Hi Edoardo,
does it mean openstack kolla installs zun using pip ?
I did not find any zun rpm package
Regards
Ignazio

Il giorno Mer 14 Nov 2018 18:38 Eduardo Gonzalez 
mailto:dabar...@gmail.com>> ha scritto:
Hi Cassano, you can use zun in centos deployed by kolla-ansible.

https://docs.openstack.org/kolla-ansible/latest/reference/zun-guide.html

Regards

El mié., 14 nov. 2018 17:11, Ignazio Cassano 
mailto:ignaziocass...@gmail.com>> escribió:
Hi All,
I'd like to know if openstack zun will be released for centos.
Reading documentation at docs.openstack.org only 
ubuntu installation is reported.
Many thanks
Ignazio
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [octavia][rocky] Octavia and VxLAN without DVR

[Fox, Kevin M]

Can you use a provider network to expose galera to the vm?

alternately, you could put a db out in the vm side. You don't strictly need to 
use the same db for every component. If crossing the streams is hard, maybe 
avoiding crossing at all is easier?

Thanks,
Kevin

From: Florian Engelmann [florian.engelm...@everyware.ch]
Sent: Thursday, October 25, 2018 8:37 AM
To: Fox, Kevin M; openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [octavia][rocky] Octavia and VxLAN without 
DVR

you mean deploy octavia into an openstack project? But I will than need
to connect the octavia services with my galera DBs... so same problem.

Am 10/25/18 um 5:31 PM schrieb Fox, Kevin M:
> Would it make sense to move the control plane for this piece into the 
> cluster? (vm in a mangement tenant?)
>
> Thanks,
> Kevin
> 
> From: Florian Engelmann [florian.engelm...@everyware.ch]
> Sent: Thursday, October 25, 2018 7:39 AM
> To: openstack-operators@lists.openstack.org
> Subject: Re: [Openstack-operators] [octavia][rocky] Octavia and VxLAN without 
> DVR
>
> It looks like devstack implemented some o-hm0 interface to connect the
> physical control host to a VxLAN.
> In our case there is no VxLAN at the control nodes nor is OVS.
>
> Is it a option to deploy those Octavia services needing this conenction
> to the compute or network nodes and use o-hm0?
>
> Am 10/25/18 um 10:22 AM schrieb Florian Engelmann:
>> Or could I create lb-mgmt-net as VxLAN and connect the control nodes to
>> this VxLAN? How to do something like that?
>>
>> Am 10/25/18 um 10:03 AM schrieb Florian Engelmann:
>>> Hmm - so right now I can't see any routed option because:
>>>
>>> The gateway connected to the VLAN provider networks (bond1 on the
>>> network nodes) is not able to route any traffic to my control nodes in
>>> the spine-leaf layer3 backend network.
>>>
>>> And right now there is no br-ex at all nor any "streched" L2 domain
>>> connecting all compute nodes.
>>>
>>>
>>> So the only solution I can think of right now is to create an overlay
>>> VxLAN in the spine-leaf backend network, connect all compute and
>>> control nodes to this overlay L2 network, create a OVS bridge
>>> connected to that network on the compute nodes and allow the Amphorae
>>> to get an IPin this network as well.
>>> Not to forget about DHCP... so the network nodes will need this bridge
>>> as well.
>>>
>>> Am 10/24/18 um 10:01 PM schrieb Erik McCormick:
>>>>
>>>>
>>>> On Wed, Oct 24, 2018, 3:33 PM Engelmann Florian
>>>> >>> <mailto:florian.engelm...@everyware.ch>> wrote:
>>>>
>>>>  On the network nodes we've got a dedicated interface to deploy VLANs
>>>>  (like the provider network for internet access). What about creating
>>>>  another VLAN on the network nodes, give that bridge a IP which is
>>>>  part of the subnet of lb-mgmt-net and start the octavia worker,
>>>>  healthmanager and controller on the network nodes binding to that
>>>> IP?
>>>>
>>>> The problem with that is you can't out an IP in the vlan interface
>>>> and also use it as an OVS bridge, so the Octavia processes would have
>>>> nothing to bind to.
>>>>
>>>>
>>>> 
>>>>  *From:* Erik McCormick >>>  <mailto:emccorm...@cirrusseven.com>>
>>>>  *Sent:* Wednesday, October 24, 2018 6:18 PM
>>>>  *To:* Engelmann Florian
>>>>  *Cc:* openstack-operators
>>>>  *Subject:* Re: [Openstack-operators] [octavia][rocky] Octavia and
>>>>  VxLAN without DVR
>>>>
>>>>
>>>>  On Wed, Oct 24, 2018, 12:02 PM Florian Engelmann
>>>>  >>>  <mailto:florian.engelm...@everyware.ch>> wrote:
>>>>
>>>>  Am 10/24/18 um 2:08 PM schrieb Erik McCormick:
>>>>   >
>>>>   >
>>>>   > On Wed, Oct 24, 2018, 3:14 AM Florian Engelmann
>>>>   > >>>  <mailto:florian.engelm...@everyware.ch>
>>>>  <mailto:florian.engelm...@everyware.ch
>>>>  <mailto:florian.engelm...@everyware.ch>>>
>>>>   > wrote:
>>>>   >
>>>>   

Re: [Openstack-operators] [octavia][rocky] Octavia and VxLAN without DVR

[Fox, Kevin M]

Would it make sense to move the control plane for this piece into the cluster? 
(vm in a mangement tenant?)

Thanks,
Kevin

From: Florian Engelmann [florian.engelm...@everyware.ch]
Sent: Thursday, October 25, 2018 7:39 AM
To: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [octavia][rocky] Octavia and VxLAN without 
DVR

It looks like devstack implemented some o-hm0 interface to connect the
physical control host to a VxLAN.
In our case there is no VxLAN at the control nodes nor is OVS.

Is it a option to deploy those Octavia services needing this conenction
to the compute or network nodes and use o-hm0?

Am 10/25/18 um 10:22 AM schrieb Florian Engelmann:
> Or could I create lb-mgmt-net as VxLAN and connect the control nodes to
> this VxLAN? How to do something like that?
>
> Am 10/25/18 um 10:03 AM schrieb Florian Engelmann:
>> Hmm - so right now I can't see any routed option because:
>>
>> The gateway connected to the VLAN provider networks (bond1 on the
>> network nodes) is not able to route any traffic to my control nodes in
>> the spine-leaf layer3 backend network.
>>
>> And right now there is no br-ex at all nor any "streched" L2 domain
>> connecting all compute nodes.
>>
>>
>> So the only solution I can think of right now is to create an overlay
>> VxLAN in the spine-leaf backend network, connect all compute and
>> control nodes to this overlay L2 network, create a OVS bridge
>> connected to that network on the compute nodes and allow the Amphorae
>> to get an IPin this network as well.
>> Not to forget about DHCP... so the network nodes will need this bridge
>> as well.
>>
>> Am 10/24/18 um 10:01 PM schrieb Erik McCormick:
>>>
>>>
>>> On Wed, Oct 24, 2018, 3:33 PM Engelmann Florian
>>> >> > wrote:
>>>
>>> On the network nodes we've got a dedicated interface to deploy VLANs
>>> (like the provider network for internet access). What about creating
>>> another VLAN on the network nodes, give that bridge a IP which is
>>> part of the subnet of lb-mgmt-net and start the octavia worker,
>>> healthmanager and controller on the network nodes binding to that
>>> IP?
>>>
>>> The problem with that is you can't out an IP in the vlan interface
>>> and also use it as an OVS bridge, so the Octavia processes would have
>>> nothing to bind to.
>>>
>>>
>>> 
>>> *From:* Erik McCormick >> >
>>> *Sent:* Wednesday, October 24, 2018 6:18 PM
>>> *To:* Engelmann Florian
>>> *Cc:* openstack-operators
>>> *Subject:* Re: [Openstack-operators] [octavia][rocky] Octavia and
>>> VxLAN without DVR
>>>
>>>
>>> On Wed, Oct 24, 2018, 12:02 PM Florian Engelmann
>>> >> > wrote:
>>>
>>> Am 10/24/18 um 2:08 PM schrieb Erik McCormick:
>>>  >
>>>  >
>>>  > On Wed, Oct 24, 2018, 3:14 AM Florian Engelmann
>>>  > >> 
>>> >> >>
>>>  > wrote:
>>>  >
>>>  > Ohoh - thank you for your empathy :)
>>>  > And those great details about how to setup this mgmt
>>> network.
>>>  > I will try to do so this afternoon but solving that
>>> routing "puzzle"
>>>  > (virtual network to control nodes) I will need our
>>> network guys to help
>>>  > me out...
>>>  >
>>>  > But I will need to tell all Amphorae a static route to
>>> the gateway that
>>>  > is routing to the control nodes?
>>>  >
>>>  >
>>>  > Just set the default gateway when you create the neutron
>>> subnet. No need
>>>  > for excess static routes. The route on the other connection
>>> won't
>>>  > interfere with it as it lives in a namespace.
>>>
>>>
>>> My compute nodes have no br-ex and there is no L2 domain spread
>>> over all
>>> compute nodes. As far as I understood lb-mgmt-net is a provider
>>> network
>>> and has to be flat or VLAN and will need a "physical" gateway
>>> (as there
>>> is no virtual router).
>>> So the question - is it possible to get octavia up and running
>>> without a
>>> br-ex (L2 domain spread over all compute nodes) on the compute
>>> nodes?
>>>
>>>
>>> Sorry, I only meant it was *like* br-ex on your network nodes. You
>>> don't need that on your computes.
>>>
>>> The router here would be whatever does routing in your physical
>>> network. Setting the gateway in the neutron subnet simply adds that
>>> to the DHCP information sent to the amphorae.
>>>
>>> This does bring up another thingI for

Re: [Openstack-operators] [openstack-dev] [goals][tc][ptl][uc] starting goal selection for T series

[Fox, Kevin M]

+1 :)

From: Tim Bell [tim.b...@cern.ch]
Sent: Wednesday, September 26, 2018 11:55 AM
To: OpenStack Development Mailing List (not for usage questions); 
openstack-operators; openstack-sigs
Subject: Re: [Openstack-sigs] [openstack-dev] [goals][tc][ptl][uc] starting 
goal selection for T series

Doug,

Thanks for raising this. I'd like to highlight the goal "Finish moving legacy 
python-*client CLIs to python-openstackclient" from the etherpad and propose 
this for a T/U series goal.

To give it some context and the motivation:

At CERN, we have more than 3000 users of the OpenStack cloud. We write an 
extensive end user facing documentation which explains how to use the OpenStack 
along with CERN specific features (such as workflows for requesting 
projects/quotas/etc.).

One regular problem we come across is that the end user experience is 
inconsistent. In some cases, we find projects which are not covered by the 
unified OpenStack client (e.g. Manila). In other cases, there are subsets of 
the function which require the native project client.

I would strongly support a goal which targets

- All new projects should have the end user facing functionality fully exposed 
via the unified client
- Existing projects should aim to close the gap within 'N' cycles (N to be 
defined)
- Many administrator actions would also benefit from integration (reader roles 
are end users too so list and show need to be covered too)
- Users should be able to use a single openrc for all interactions with the 
cloud (e.g. not switch between password for some CLIs and Kerberos for OSC)

The end user perception of a solution will be greatly enhanced by a single 
command line tool with consistent syntax and authentication framework.

It may be a multi-release goal but it would really benefit the cloud consumers 
and I feel that goals should include this audience also.

Tim

-Original Message-
From: Doug Hellmann 
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 

Date: Wednesday, 26 September 2018 at 18:00
To: openstack-dev , openstack-operators 
, openstack-sigs 

Subject: [openstack-dev] [goals][tc][ptl][uc] starting goal selection for T 
series

It's time to start thinking about community-wide goals for the T series.

We use community-wide goals to achieve visible common changes, push for
basic levels of consistency and user experience, and efficiently improve
certain areas where technical debt payments have become too high -
across all OpenStack projects. Community input is important to ensure
that the TC makes good decisions about the goals. We need to consider
the timing, cycle length, priority, and feasibility of the suggested
goals.

If you are interested in proposing a goal, please make sure that before
the summit it is described in the tracking etherpad [1] and that you
have started a mailing list thread on the openstack-dev list about the
proposal so that everyone in the forum session [2] has an opportunity to
consider the details.  The forum session is only one step in the
selection process. See [3] for more details.

Doug

[1] https://etherpad.openstack.org/p/community-goals
[2] https://www.openstack.org/summit/berlin-2018/vote-for-speakers#/22814
[3] https://governance.openstack.org/tc/goals/index.html

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
openstack-sigs mailing list
openstack-s...@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-sigs
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [Openstack-sigs] [openstack-dev] Open letter/request to TC candidates (and existing elected officials)

[Fox, Kevin M]

How about stated this way,
Its the tc's responsibility to get it done. Either by delegating the activity, 
or by doing it themselves. But either way, it needs to get done. Its a ball 
that has been dropped too much in OpenStacks history. If no one is ultimately 
responsible, balls will keep getting dropped.

Thanks,
Kevin

From: Matt Riedemann [mriede...@gmail.com]
Sent: Wednesday, September 12, 2018 4:00 PM
To: Dan Smith; Thierry Carrez
Cc: OpenStack Development Mailing List (not for usage questions); 
openstack-s...@lists.openstack.org; openstack-operators@lists.openstack.org
Subject: Re: [Openstack-sigs] [openstack-dev] Open letter/request to TC 
candidates (and existing elected officials)

On 9/12/2018 3:30 PM, Dan Smith wrote:
>> I'm just a bit worried to limit that role to the elected TC members. If
>> we say "it's the role of the TC to do cross-project PM in OpenStack"
>> then we artificially limit the number of people who would sign up to do
>> that kind of work. You mention Ildiko and Lance: they did that line of
>> work without being elected.
> Why would saying that we_expect_  the TC members to do that work limit
> such activities only to those that are on the TC? I would expect the TC
> to take on the less-fun or often-neglected efforts that we all know are
> needed but don't have an obvious champion or sponsor.
>
> I think we expect some amount of widely-focused technical or project
> leadership from TC members, and certainly that expectation doesn't
> prevent others from leading efforts (even in the areas of proposing TC
> resolutions, etc) right?

Absolutely. I'm not saying the cross-project project management should
be restricted to or solely the responsibility of the TC. It's obvious
there are people outside of the TC that have already been doing this -
and no it's not always elected PTLs either. What I want is elected TC
members to prioritize driving technical deliverables to completion based
on ranked input from operators/users/SIGs over philosophical debates and
politics/bureaucracy and help to complete the technical tasks if possible.

--

Thanks,

Matt

___
openstack-sigs mailing list
openstack-s...@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-sigs

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [OpenStack-Operators][OpenStack] Regarding production grade OpenStack deployment

[Fox, Kevin M]

I don't think openstack itself can meet full zero downtime requirements. But if 
it can, then I also think none of the deployment tools try and support that use 
case either.

Thanks,
Kevin

From: Amit Kumar [ebiib...@gmail.com]
Sent: Friday, May 18, 2018 3:46 AM
To: OpenStack Operators; Openstack
Subject: [Openstack-operators] [OpenStack-Operators][OpenStack] Regarding 
production grade OpenStack deployment

Hi All,

We want to deploy our private cloud using OpenStack as highly available (zero 
downtime (ZDT) - in normal course of action and during upgrades as well) 
production grade environment. We came across following tools.


  *   We thought of using Kolla-Kubernetes as deployment tool, but we got 
feedback from Kolla IRC channel that this project is being retired. Moreover, 
we couldn't find latest documents having multi-node deployment steps and, High 
Availability support was also not mentioned at all anywhere in the 
documentation.
  *   Another option to have Kubernetes based deployment is to use 
OpenStack-Helm, but it seems the OSH community has not made OSH 1.0 officially 
available yet.
  *   Last option, is to use Kolla-Ansible, although it is not a Kubernetes 
deployment, but seems to have good community support around it. Also, its 
documentation talks a little about production grade deployment, probably it is 
being used in production grade environments.

If you folks have used any of these tools for deploying OpenStack to fulfill 
these requirements: HA and ZDT, then please provide your inputs specifically 
about HA and ZDT support of the deployment tool, based on your experience. And 
please share if you have any reference links that you have used for achieving 
HA and ZDT for the respective tools.

Lastly, if you think we should think that we have missed another more viable 
and stable options of deployment tools which can serve our requirement: HA and 
ZDT, then please do suggest the same.

Regards,
Amit


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] Upstream LTS Releases

[Fox, Kevin M]

I can think of a few ideas, though some sound painful on paper Not really 
recommending anything, just thinking out loud...

One idea is that at the root of chaos monkey. If something is hard, do it 
frequently. If upgrading is hard, we need to be doing it constantly so the pain 
gets largely eliminated. One idea would be to discourage the use of standing up 
a fresh devstack all the time by devs and have them upgrade them instead. If 
its hard, then its likely someone will chip in to make it less hard.

Another is devstack in general. the tooling used by devs and that used by ops 
are so different as to isolate the devs from ops' pain. If they used more 
opsish tooling, then they would hit the same issues and would be more likely to 
find solutions that work for both parties.

A third one is supporting multiple version upgrades in the gate. I rarely have 
a problem with a cloud has a database one version back. I have seen lots of 
issues with databases that contain data back when the cloud was instantiated 
and then upgraded multiple times.

Another option is trying to unify/detangle the upgrade procedure. upgrading 
compute kit should be one or two commands if you can live with the defaults. 
Not weeks of poring through release notes, finding correct orders from pages of 
text and testing vigorously on test systems.

How about some tool that does the: dump database to somewhere temporary, 
iterate over all the upgrade job components, and see if it will successfully 
not corrupt your database. That takes a while to do manually. Ideally it could 
even upload stacktraces back a bug tracker for attention.

Thanks,
Kevin

From: Davanum Srinivas [dava...@gmail.com]
Sent: Tuesday, November 14, 2017 4:08 PM
To: OpenStack Development Mailing List (not for usage questions)
Cc: openstack-oper.
Subject: Re: [openstack-dev] [Openstack-operators] Upstream LTS Releases

On Wed, Nov 15, 2017 at 10:44 AM, John Dickinson  wrote:
>
>
> On 14 Nov 2017, at 15:18, Mathieu Gagné wrote:
>
>> On Tue, Nov 14, 2017 at 6:00 PM, Fox, Kevin M  wrote:
>>> The pressure for #2 comes from the inability to skip upgrades and the fact 
>>> that upgrades are hugely time consuming still.
>>>
>>> If you want to reduce the push for number #2 and help developers get their 
>>> wish of getting features into users hands sooner, the path to upgrade 
>>> really needs to be much less painful.
>>>
>>
>> +1000
>>
>> We are upgrading from Kilo to Mitaka. It took 1 year to plan and
>> execute the upgrade. (and we skipped a version)
>> Scheduling all the relevant internal teams is a monumental task
>> because we don't have dedicated teams for those projects and they have
>> other priorities.
>> Upgrading affects a LOT of our systems, some we don't fully have
>> control over. And it can takes months to get new deployment on those
>> systems. (and after, we have to test compatibility, of course)
>>
>> So I guess you can understand my frustration when I'm told to upgrade
>> more often and that skipping versions is discouraged/unsupported.
>> At the current pace, I'm just falling behind. I *need* to skip
>> versions to keep up.
>>
>> So for our next upgrades, we plan on skipping even more versions if
>> the database migration allows it. (except for Nova which is a huge
>> PITA to be honest due to CellsV1)
>> I just don't see any other ways to keep up otherwise.
>
> ?!?!
>
> What does it take for this to never happen again? No operator should need to 
> plan and execute an upgrade for a whole year to upgrade one year's worth of 
> code development.
>
> We don't need new policies, new teams, more releases, fewer releases, or 
> anything like that. The goal is NOT "let's have an LTS release". The goal 
> should be "How do we make sure Mattieu and everyone else in the world can 
> actually deploy and use the software we are writing?"
>
> Can we drop the entire LTS discussion for now and focus on "make upgrades 
> take less than a year" instead? After we solve that, let's come back around 
> to LTS versions, if needed. I know there's already some work around that. 
> Let's focus there and not be distracted about the best bureaucracy for not 
> deleting two-year-old branches.
>
>
> --John

John,

So... Any concrete ideas on how to achieve that?

Thanks,
Dims

>
>
> /me puts on asbestos pants
>
>>
>> --
>> Mathieu
>>
>> ___
>> OpenStack-operators mailing list
>> OpenStack-operators@lists.openstack.org
>

Re: [Openstack-operators] [openstack-dev] Upstream LTS Releases

[Fox, Kevin M]

The pressure for #2 comes from the inability to skip upgrades and the fact that 
upgrades are hugely time consuming still.

If you want to reduce the push for number #2 and help developers get their wish 
of getting features into users hands sooner, the path to upgrade really needs 
to be much less painful.

Thanks,
Kevin

From: Erik McCormick [emccorm...@cirrusseven.com]
Sent: Tuesday, November 14, 2017 9:21 AM
To: Blair Bethwaite
Cc: OpenStack Development Mailing List (not for usage questions); 
openstack-oper.
Subject: Re: [openstack-dev] [Openstack-operators]  Upstream LTS Releases

On Tue, Nov 14, 2017 at 11:30 AM, Blair Bethwaite
 wrote:
> Hi all - please note this conversation has been split variously across
> -dev and -operators.
>
> One small observation from the discussion so far is that it seems as
> though there are two issues being discussed under the one banner:
> 1) maintain old releases for longer
> 2) do stable releases less frequently
>
> It would be interesting to understand if the people who want longer
> maintenance windows would be helped by #2.
>

I would like to hear from people who do *not* want #2 and why not.
What are the benefits of 6 months vs. 1 year? I have heard objections
in the hallway track, but I have struggled to retain the rationale for
more than 10 seconds. I think this may be more of a religious
discussion that could take a while though.

#1 is something we can act on right now with the eventual goal of
being able to skip releases entirely. We are addressing the
maintenance of the old issue right now. As we get farther down the
road of fast-forward upgrade tooling, then we will be able to please
those wishing for a slower upgrade cadence, and those that want to
stay on the bleeding edge simultaneously.

-Erik

> On 14 November 2017 at 09:25, Doug Hellmann  wrote:
>> Excerpts from Bogdan Dobrelya's message of 2017-11-14 17:08:31 +0100:
>>> >> The concept, in general, is to create a new set of cores from these
>>> >> groups, and use 3rd party CI to validate patches. There are lots of
>>> >> details to be worked out yet, but our amazing UC (User Committee) will
>>> >> be begin working out the details.
>>> >
>>> > What is the most worrying is the exact "take over" process. Does it mean 
>>> > that
>>> > the teams will give away the +2 power to a different team? Or will our 
>>> > (small)
>>> > stable teams still be responsible for landing changes? If so, will they 
>>> > have to
>>> > learn how to debug 3rd party CI jobs?
>>> >
>>> > Generally, I'm scared of both overloading the teams and losing the 
>>> > control over
>>> > quality at the same time :) Probably the final proposal will clarify it..
>>>
>>> The quality of backported fixes is expected to be a direct (and only?)
>>> interest of those new teams of new cores, coming from users and
>>> operators and vendors. The more parties to establish their 3rd party
>>
>> We have an unhealthy focus on "3rd party" jobs in this discussion. We
>> should not assume that they are needed or will be present. They may be,
>> but we shouldn't build policy around the assumption that they will. Why
>> would we have third-party jobs on an old branch that we don't have on
>> master, for instance?
>>
>>> checking jobs, the better proposed changes communicated, which directly
>>> affects the quality in the end. I also suppose, contributors from ops
>>> world will likely be only struggling to see things getting fixed, and
>>> not new features adopted by legacy deployments they're used to maintain.
>>> So in theory, this works and as a mainstream developer and maintainer,
>>> you need no to fear of losing control over LTS code :)
>>>
>>> Another question is how to not block all on each over, and not push
>>> contributors away when things are getting awry, jobs failing and merging
>>> is blocked for a long time, or there is no consensus reached in a code
>>> review. I propose the LTS policy to enforce CI jobs be non-voting, as a
>>> first step on that way, and giving every LTS team member a core rights
>>> maybe? Not sure if that works though.
>>
>> I'm not sure what change you're proposing for CI jobs and their voting
>> status. Do you mean we should make the jobs non-voting as soon as the
>> branch passes out of the stable support period?
>>
>> Regarding the review team, anyone on the review team for a branch
>> that goes out of stable support will need to have +2 rights in that
>> branch. Otherwise there's no point in saying that they're maintaining
>> the branch.
>>
>> Doug
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> Cheers,
> ~Blairo
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@

Re: [Openstack-operators] Case studies on Openstack HA architecture

[Fox, Kevin M]

kolla has various containerization tools. one based on ansible, another based 
on kubernetes.

From: Imtiaz Chowdhury [imtiaz.chowdh...@workday.com]
Sent: Monday, August 28, 2017 5:24 PM
To: Curtis
Cc: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Case studies on Openstack HA architecture

Thanks Curtis, Robert, David and Mohammed for your responses.

As a follow up question, do you use any deployment automation tools for setting 
up the HA control plane? I can see the value of deploying each service in 
separate virtual environment or containers but automating such deployment 
requires developing some new tools. Openstack-ansible is one potential 
deployment tool that I am aware of but that had limited support CentOS.

Imtiaz

On 8/28/17, 2:23 PM, "Curtis"  wrote:

On Fri, Aug 25, 2017 at 6:11 PM, Imtiaz Chowdhury
 wrote:
> Hi Openstack operators,
>
>
>
> Most Openstack HA deployment use 3 node database cluster, 3 node rabbitMQ
> cluster and 3 Controllers. I am wondering whether there are any studies 
done
> that show the pros and cons of co-locating database and messaging service
> with the Openstack control services.  In other words, I am very interested
> in learning about advantages and disadvantages, in terms of ease of
> deployment, upgrade and overall API performance, of having 3 all-in-one
> Openstack controller over a more distributed deployment model.

I'm not aware of any actual case studies, but this is the (current)
default model for tripleo and its downstream product, so there would
be a lot of deployments like this out there in the wild. In the
default deployment everything but compute is on these 3 nodes running
on the physical OS.

Do you mean 3 physical servers with everything running on the physical OS?

My opinion is that 3 physical nodes to run all the control plane
services is quite common, but in custom deployments I either run vms
and containers on those or just containers. I'd use at least lxc to
segregate services into their own containers.

I would also suggest that using those same physical servers as
north/south "network nodes" (which you probably don't have as I
believe workday is a big opencontrail user) or hosts for stateful
metric systems (ie. mongodb) can cause issues performance wise, but
co-located mysql/galera and rabbit on the same nodes as the rest of
the openstack control plane hasn't been a problem for me yet, but with
containers I could split them out fairly easily if needed.

Thanks,
Curtis.

>
>
>
> References to any work done in this area will be highly appreciated.
>
>
>
> Thanks,
> Imtiaz
>
>
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> 
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Doperators&d=DwIBaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=FzzkP-wZtxwR_lHv11gV2RDLNwSuEtI-Ttdh3mloOUA&m=byM_0ToLQ8JCji20rvyQJYr-Zm4pHsZ5TK4CFkuZbbk&s=mLtaaDedoNufBf-kCreLrdZ-McNRAuuesR3xWIT76Vc&e=
>



--
Blog: serverascode.com


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] [nova][cinder] Is there interest in an admin-api to refresh volume connection info?

[Fox, Kevin M]

Oh, yes please!  We've had to go through a lot of hoops to migrate ceph-mon's 
around while keeping their ip's consistent to avoid vm breakage. All the rest 
of the ceph ecosystem (at least that we've dealt with) works fine without the 
level of effort the current nova/cinder implementation imposes.

Thanks,
Kevin

From: melanie witt [melwi...@gmail.com]
Sent: Thursday, June 08, 2017 11:39 AM
To: Matt Riedemann; openstack-operators@lists.openstack.org; 
openstack-...@lists.openstack.org
Subject: Re: [openstack-dev] [Openstack-operators] [nova][cinder] Is there 
interest in an admin-api to refresh volume connection info?

On Thu, 8 Jun 2017 08:58:20 -0500, Matt Riedemann wrote:
> Nova stores the output of the Cinder os-initialize_connection info API
> in the Nova block_device_mappings table, and uses that later for making
> volume connections.
>
> This data can get out of whack or need to be refreshed, like if your
> ceph server IP changes, or you need to recycle some secret uuid for your
> ceph cluster.
>
> I think the only ways to do this on the nova side today are via volume
> detach/re-attach, reboot, migrations, etc - all of which, except live
> migration, are disruptive to the running guest.

I believe the only way to work around this currently is by doing a 'nova
shelve' followed by a 'nova unshelve'. That will end up querying the
connection_info from Cinder and update the block device mapping record
for the instance. Maybe detach/re-attach would work too but I can't
remember trying it.

> I've kicked around the idea of adding some sort of admin API interface
> for refreshing the BDM.connection_info on-demand if needed by an
> operator. Does anyone see value in this? Are operators doing stuff like
> this already, but maybe via direct DB updates?
>
> We could have something in the compute API which calls down to the
> compute for an instance and has it refresh the connection_info from
> Cinder and updates the BDM table in the nova DB. It could be an admin
> action API, or part of the os-server-external-events API, like what we
> have for the 'network-changed' event sent from Neutron which nova uses
> to refresh the network info cache.
>
> Other ideas or feedback here?

We've discussed this a few times before and we were thinking it might be
best to handle this transparently and just do a connection_info refresh
+ record update inline with the request flows that will end up reading
connection_info from the block device mapping records. That way,
operators won't have to intervene when connection_info changes.

At least in the case of Ceph, as long as a guest is running, it will
continue to work fine if the monitor IPs or secrets change because it
will continue to use its existing connection to the Ceph cluster. Things
go wrong when an instance action such as resize, stop/start, or reboot
is done because when the instance is taken offline and being brought
back up, the stale connection_info is read from the block_device_mapping
table and injected into the instance, and so it loses contact with the
cluster. If we query Cinder and update the block_device_mapping record
at the beginning of those actions, the instance will get the new
connection_info.

-melanie



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [nova][ironic][scheduler][placement][heat] IMPORTANT: Getting rid of the automated reschedule functionality

[Fox, Kevin M]

So one gut reaction is this is going to make more heat stacks fail. If 
pushing the orchestration stuff out of nova's the goal, you probably should 
involve heat so that it knows the difference between a vm that failed because 
it was scheduled poorly and can just be resubmitted and a vm that failed for 
other reasons?

To your comment do one thing well comment, I agree, but not necessarily with 
your conclusion. if nova's api is to provide a way for a user to launch a vm, 
if it fails more often due to internal issues such as scheduling races, thats 
nova's failure (or at least a machine's job to handle the failure) not the 
users.  The same retry code has to get implemented either way. Retry in nova, 
or retry by the thing driving nova. If there are a lot of different things 
driving nova (a lot of things already), then it means reimplementing the same 
solution over and over again in all the things. That makes openstack harder to 
use, and its already hard to use. Thats not a good path to continue to be on.

Maybe your right, that all retries should be external to the individual 
projects and maybe done by a central openstack retry daemon or something, and 
maybe nova'api's a low level api users shouldn't actually be talking to, but 
instead talking to an 'openstack cli, but for rest' kind of api. Thats 
something that openstack as a whole probably needs to talk about anyway. Each 
project keeps evolving their api's separately without central discussion of 
these sorts of cross cutting concerns that maybe are better solved the same way.

Also, for the record, I do use anit-affinity groups all the time, but haven't 
touched nfv.

Thanks,
Kevin

From: Jay Pipes [jaypi...@gmail.com]
Sent: Monday, May 22, 2017 10:54 AM
To: openstack-operators@lists.openstack.org
Subject: [Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: 
Getting rid of the automated reschedule functionality

Hi Ops,

I need your feedback on a very important direction we would like to
pursue. I realize that there were Forum sessions about this topic at the
summit in Boston and that there were some decisions that were reached.

I'd like to revisit that decision and explain why I'd like your support
for getting rid of the automatic reschedule behaviour entirely in Nova
for Pike.

== The current situation and why it sucks ==

Nova currently attempts to "reschedule" instances when any of the
following events occur:

a) the "claim resources" process that occurs on the nova-compute worker
results in the chosen compute node exceeding its own capacity

b) in between the time a compute node was chosen by the scheduler,
another process launched an instance that would violate an affinity
constraint

c) an "unknown" exception occurs during the spawn process. In practice,
this really only is seen when the Ironic baremetal node that was chosen
by the scheduler turns out to be unreliable (IPMI issues, BMC failures,
etc) and wasn't able to launch the instance. [1]

The logic for handling these reschedules makes the Nova conductor,
scheduler and compute worker code very complex. With the new cellsv2
architecture in Nova, child cells are not able to communicate with the
Nova scheduler (and thus "ask for a reschedule").

We (the Nova team) would like to get rid of the automated rescheduling
behaviour that Nova currently exposes because we could eliminate a large
amount of complexity (which leads to bugs) from the already-complicated
dance of communication that occurs between internal Nova components.

== What we would like to do ==

With the move of the resource claim to the Nova scheduler [2], we can
entirely eliminate the a) class of Reschedule causes.

This leaves class b) and c) causes of Rescheduling.

For class b) causes, we should be able to solve this issue when the
placement service understands affinity/anti-affinity (maybe
Queens/Rocky). Until then, we propose that instead of raising a
Reschedule when an affinity constraint was last-minute violated due to a
racing scheduler decision, that we simply set the instance to an ERROR
state.

Personally, I have only ever seen anti-affinity/affinity use cases in
relation to NFV deployments, and in every NFV deployment of OpenStack
there is a VNFM or MANO solution that is responsible for the
orchestration of instances belonging to various service function chains.
I think it is reasonable to expect the MANO system to be responsible for
attempting a re-launch of an instance that was set to ERROR due to a
last-minute affinity violation.

**Operators, do you agree with the above?**

Finally, for class c) Reschedule causes, I do not believe that we should
be attempting automated rescheduling when "unknown" errors occur. I just
don't believe this is something Nova should be doing.

I recognize that large Ironic users expressed their concerns about
IPMI/BMC communication being unreliable and not wanting to have users
manually retry a baremetal instance l

Re: [Openstack-operators] Ceph upgrade causing health warnings

[Fox, Kevin M]

its a flag like noout. set with the ceph cli command.

Make sure all clients are to jewel (all vms restarted after the client 
iupgraded) before you set it though. We ran into some issues with that.

Thanks,
Kevin

From: Grant Morley [gr...@absolutedevops.io]
Sent: Wednesday, May 17, 2017 3:47 AM
To: OpenStack Operators
Subject: [Openstack-operators] Ceph upgrade causing health warnings


Hi All,

We have just upgraded our ceph cluster to Jewel-10.2.6 running on Ubuntu 14.04 
and all seems to have gone quite well. However every now and then we are 
getting health warnings for the cluster that state the following:

all OSDs are running jewel or later but the 'require_jewel_osds' osdmap flag is 
not set

That setting doesn't appear to be in the ceph.conf file. I was wondering if 
anyone else has come across this and if they have simply added the config and 
restarted ceph to fix the issue?

I am a little reluctant to do it at the moment as the cluster seems to be 
running fine. Just more of an annoyance that we are getting a lot of alerts 
from our monitoring systems.

Regards,

Grant

--
[cid:part1.2D46650E.4A1174BA@absolutedevops.io]
Grant Morley
Cloud Lead
Absolute DevOps Ltd
Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP
www.absolutedevops.io 
gr...@absolutedevops.io 0845 874 0580
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Flavors

[Fox, Kevin M]

I think the really short answer is something like: It greatly simplifies 
scheduling and billing.


From: Vladimir Prokofev [v...@prokofev.me]
Sent: Wednesday, March 15, 2017 2:41 PM
To: OpenStack Operators
Subject: [Openstack-operators] Flavors

A question of curiosity - why do we even need flavors?

I do realise that we need a way to provide instance configuration, but why use 
such a rigid construction? Wouldn't it be more flexible to provide instance 
configuration as a set of parameters(metadata), and if you need some presets - 
well, use a preconfigured set of them as a flavor in your front-end(web/CLI 
client parameters)?

Suppose commercial customer has an instance with high storage IO load. 
Currently they have only one option - upsize instance to a flavor that provides 
higher IOPS. But ususally provider has a limited amount of flavors for 
purchase, and they upscale everything for a price. So instead of paying only 
for IOPS customers are pushed to pay for whole package. This is good from 
revenue point of view, but bad for customer's bank account and marketing(i.e. 
product architecure limits).
This applies to every resource - vCPU, RAM, storage, networking, etc - 
everything is controlled by flavor.

This concept has never been questioned anywhere I can search, so I have a 
feeling I'm missing something big here. Maybe other ways are too complicated to 
implement?

So does anyone has any idea - why such rigid approach as flavors instead of 
something more flexible?
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Trove in production

[Fox, Kevin M]

We've run it in a test cloud meant to identify production issues before 
supporting them on our production cloud.

We ran into a few issues that may or may not apply in your situation:

There's a security issue with the Trove Rabbitmq. The easiest way around it is 
to use the feature that lets the vm's run in a trove tenant rather then the 
users tenant. Depending on your environment, this can cause other problems with 
quota's/billing/restricted flavors though. We have not been able to deploy it 
in production due to these restrictions tied in with the security issue.

Another pain point for us was management of the guest agents. We upgraded our 
control plane for Trove, and all the existing guest agents in the vm's broke on 
us.

A third issue was incompatibility with the radosgw. Trove functionality relies 
on namespacing working right and at the time radosgw didnt fully support it. I 
think that may be fixed in Jewel, but haven't had a chance to confirm the issue 
is resolved, so there may be other issues.

Thanks,
Kevin

From: Masha Atakova [masha.atak...@mail.com]
Sent: Wednesday, March 15, 2017 12:48 AM
To: openstack-operators@lists.openstack.org
Subject: [Openstack-operators] Trove in production

Hi everyone,

I'm curious if anybody is using Trove in production: what's your
experience? What are the problems you've encountered and how critical
are they?

Thanks in advance for your responses


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [kolla] deprecation/removal of Debian images

[Fox, Kevin M]

The issue is, as I understand it, that there are no tests currently to check if 
changes to the Kolla code base will break the Debian based containers, and no 
one has stepped up to write the tests in a long time.

So, no one can rely on the containers being in a usable state.

If someone is willing to step up and contribute the missing infrastructure, I 
think everyone would be happy to have Debian continue to be supported.

Is anyone interested in contributing the missing functionality so it can stay?

Thanks,
Kevin

From: Keith Farrar [kfar...@parc.com]
Sent: Tuesday, January 24, 2017 5:04 PM
To: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [kolla] deprecation/removal of Debian images

Removing Debian support seems odd, in that Debian testing generally has
been the upstream source for Ubuntu OS packages.



___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [Large deployments] Openstack Large deployment using DVR

[Fox, Kevin M]

We're running dvr on one of our clouds. ~70 hypervisors currently and more 
getting ready to join. We have not played with dvr+l3ha as it was unstable at 
the time but once there is a migration path, we would like to go there.

So far, DVR has seemed pretty stable and is performing well.

Thanks,
Kevin

From: Clayton O'Neill [clay...@oneill.net]
Sent: Tuesday, November 01, 2016 6:18 AM
To: Satyanarayana Patibandla
Cc: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [Large deployments] Openstack Large 
deployment using DVR

On Tue, Nov 1, 2016 at 7:19 AM, Satyanarayana Patibandla
 wrote:
> We are planning for a new Openstack large deployment ( Around 25 K VMs). For
> external routing we want to use DVR for instances with floating IP and
> Network node L3 agent for instances with private IP. Our deployment is DVR
> with L3 HA.It seems still many people not yet started using DVR in
> production for large deployments. Did anyone use DVR in their Large/Medium
> production deployment. If you can share your experiences that will be
> helpful to us and that will save most of our time.

I've asked this question in operator sessions at the Summits and at
the NYC operators mid-cycle with not much response.  In the Neutron
feedback session last week, there was one person that said he was
running DVR, but that it had architectural issues with IPv6.  I got
the impression that his environment was much smaller than what you're
considering.

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Rados Gateway to Swift migration

[Fox, Kevin M]

ah. ok. we were going to start looking into getting some metrics too, but 
hadn't yet. We're infernalis now, and going to jewel soon. Theres a huge 
difference between jewel and hammer. So maybe its better now... The whole 
single namespace for all tenants thing is supposed to be fixed in jewel too. 
which has bitten us on multiple occasions.

Thanks,
Kevin

From: Xav Paice [xavpa...@gmail.com]
Sent: Wednesday, October 05, 2016 12:39 PM
To: Fox, Kevin M
Cc: George Mihaiescu; OpenStack Operators
Subject: Re: [Openstack-operators] Rados Gateway to Swift migration

On Wed, 2016-10-05 at 19:29 +0000, Fox, Kevin M wrote:
> Did you try it with jewel? If not, what version?
>

>From Emperor up to Hammer, haven't upgraded since then.  I see that
there's a number of significant changes, some of the limitations we were
finding to be a problem may be gone now.

> Thanks,
> Kevin
> 
> From: Xav Paice [xavpa...@gmail.com]
> Sent: Wednesday, October 05, 2016 12:12 PM
> To: George Mihaiescu
> Cc: OpenStack Operators
> Subject: Re: [Openstack-operators] Rados Gateway to Swift migration
>
> On Wed, 2016-10-05 at 07:19 -0400, George Mihaiescu wrote:
> > Hi Xav,
> >
> > We are trying to get usage metrics for radosgw as well for internal cost 
> > recovery. Can you please share how far you got in that process and what it 
> > was missing?
> >
>
> To be honest, we didn't get very far and that's one of the reasons for
> using Swift instead.  There were limitations with permissions that we
> found very difficult to get around, without having a data collection
> user added to each and every project.
>
> > Thank you,
> > George
> >
> > > On Oct 5, 2016, at 1:57 AM, Xav Paice  wrote:
> > >
> > >> On Wed, 2016-10-05 at 15:42 +1100, Blair Bethwaite wrote:
> > >> Nice! But I'm curious, why the need to migrate?
> > >
> > > Hmm.  I want to be diplomatic since both are great for their thing.
> > >
> > > For us, the main reason was simply that we wanted replication of the
> > > object storage between regions (we started the process before that was a
> > > feature in RGW), but also being a public cloud we also wanted to be able
> > > to bill customers for their usage, and we were finding that incredibly
> > > difficult with Rados Gateway in comparison to Swift.
> > >
> > > That, and we found customers were using RGW as a backup, and that's on
> > > the same storage back end as our Cinder and Glance - moving to a
> > > different platform makes it separate.
> > >
> > > There's a few other features in Swift that aren't in RGW, and we have
> > > customers asking for them, which really matters a lot to us.
> > >
> > > There's pros and cons for both, I don't regret us using RGW but it just
> > > doesn't suit our needs right now.
> > >
> > >
> > > ___
> > > OpenStack-operators mailing list
> > > OpenStack-operators@lists.openstack.org
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators




___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Rados Gateway to Swift migration

[Fox, Kevin M]

Did you try it with jewel? If not, what version?

Thanks,
Kevin

From: Xav Paice [xavpa...@gmail.com]
Sent: Wednesday, October 05, 2016 12:12 PM
To: George Mihaiescu
Cc: OpenStack Operators
Subject: Re: [Openstack-operators] Rados Gateway to Swift migration

On Wed, 2016-10-05 at 07:19 -0400, George Mihaiescu wrote:
> Hi Xav,
>
> We are trying to get usage metrics for radosgw as well for internal cost 
> recovery. Can you please share how far you got in that process and what it 
> was missing?
>

To be honest, we didn't get very far and that's one of the reasons for
using Swift instead.  There were limitations with permissions that we
found very difficult to get around, without having a data collection
user added to each and every project.

> Thank you,
> George
>
> > On Oct 5, 2016, at 1:57 AM, Xav Paice  wrote:
> >
> >> On Wed, 2016-10-05 at 15:42 +1100, Blair Bethwaite wrote:
> >> Nice! But I'm curious, why the need to migrate?
> >
> > Hmm.  I want to be diplomatic since both are great for their thing.
> >
> > For us, the main reason was simply that we wanted replication of the
> > object storage between regions (we started the process before that was a
> > feature in RGW), but also being a public cloud we also wanted to be able
> > to bill customers for their usage, and we were finding that incredibly
> > difficult with Rados Gateway in comparison to Swift.
> >
> > That, and we found customers were using RGW as a backup, and that's on
> > the same storage back end as our Cinder and Glance - moving to a
> > different platform makes it separate.
> >
> > There's a few other features in Swift that aren't in RGW, and we have
> > customers asking for them, which really matters a lot to us.
> >
> > There's pros and cons for both, I don't regret us using RGW but it just
> > doesn't suit our needs right now.
> >
> >
> > ___
> > OpenStack-operators mailing list
> > OpenStack-operators@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators




___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Rados Gateway to Swift migration

[Fox, Kevin M]

OpenStack really needs a way to have a control api for selecting a swift 
"flavor", and letting you have multiple swift endpoints within, so swift the 
software, radowgw, and vendor endpoints can all co'exist.

Kevin

From: Xav Paice [xavpa...@gmail.com]
Sent: Tuesday, October 04, 2016 2:45 PM
To: Curtis
Cc: OpenStack Operators
Subject: Re: [Openstack-operators] Rados Gateway to Swift migration

On Tue, 2016-10-04 at 07:18 -0600, Curtis wrote:
> On Mon, Oct 3, 2016 at 3:31 PM, Xav Paice  wrote:
> > Hi,
> >
> > We're in the process of migrating our object storage from Rados Gateway
> > to Swift, and I was wondering if anyone has experiences with the data
> > migration steps from one to the other?
> >
> > In particular, we're currently copying each object and container one by
> > one, but the process of inspecting each and every object is incredibly
> > slow.  The logic we have is kind of rsync-like, so we can repeat and
> > iterate until it's close, but the final go-live cutover will still take
> > many hours to complete.  Ideas on how to get over that would be very
> > much appreciated!
>
> Could you load balance to on backend or another based on whether or
> not the account has been migrated yet? I haven't thought that through
> completely...
>

That would be the preferable situation, migrating one customer at a
time, but we couldn't figure out how to achieve that goal without
writing some kind of middleware/proxy type of thing, or being clever
with haproxy.  We may yet have to go down that route, but I'm hoping
not!


> Thanks,
> Curtis.
>
> >
> > Many thanks
> >
> >
> > ___
> > OpenStack-operators mailing list
> > OpenStack-operators@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
>



___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Openstack team size vs's deployment size

[Fox, Kevin M]

I'd also add it depends on feature set of the cloud. If you have extra 
services, or your users keep asking for more and more openstack features to be 
added to the cloud (dnsaas, dbaas, hadoopaas, coeaas,) then the ratio is much 
higher then say, with just a basic cloud with vmaas & naas.

Thanks,
Kevin

From: Clint Byrum [cl...@fewbar.com]
Sent: Monday, September 12, 2016 2:26 PM
To: openstack-operators
Subject: Re: [Openstack-operators] Openstack team size vs's deployment size

Excerpts from gustavo panizzo (gfa)'s message of 2016-09-09 22:07:49 +0800:
> On Thu, Sep 08, 2016 at 03:52:42PM +, Kris G. Lindgren wrote:
> > I completely agree about the general rule of thumb.  I am only looking at 
> > the team that specifically supports openstack.  For us frontend support for 
> > public clouds is handled by another team/org all together.
>
> in my previous job the ratio was 1 openstack guy / 300 prod hv and
> ~ 50 non prod hypervisors (non prod clouds).
>
> we had 5 different clouds, 2 biggest clouds shared keystone and
> glance (same dc, different buildings, almost lan latency). the biggest cloud 
> had 2 regions
> (different connectivity on same dc building)
>
> a different team took care of the underlying hw, live migrations (when
> necessary but usually escalated to the openstack team) and install the
> computes running a single salt stack command. another team developed a
> in-house horizon replacement
>
> that job definitively burned me, i'd say that the sweet spot is about
> 1 person every 200 hv, but if your regions are very similar and you have
> control of the whole stack (i didn't) i think 1 person every 300 hv is
> doable
>
> we only used nova, neutron (provider networks), glance and keystone.
>

This ratio is entirely dependent on the point of the cloud, and where
one's priorities lie.

If you have a moderately sized private cloud (< 200 hv's) with no AZ's,
and 1 region, and an uptime SLA of 99.99%, I'd expect 1 FTE would be
plenty. But larger clouds that expect to continuously grow should try to
drive it down as low as possible so that value can be extracted from the
equipment in as small a batch size as possible. The higher that ratio,
the more value we're getting from the servers themselves.

Basically, I'd expect this to scale logarithmically, with clouds in the
1 - 500 hv range being similar in total cost, but everything above that
leveling off in total cost growth, but continuing a more or less linear
path up in value proposition. The only way we get there is to attack
complexity with a vengeance.

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Shelving

[Fox, Kevin M]

+1

From: Tim Bell [tim.b...@cern.ch]
Sent: Thursday, August 18, 2016 10:50 AM
To: Jonathan D. Proulx
Cc: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Shelving

I was interested to establish a consensus that

- Shelved instances should not be part of the users quota
- Quota in Glance (and associated chargeback if appropriate) is needed

Glance space for us is much less expensive than people leaving their instances 
running. Equally, terminating a user’s inactive VM would not be popular so 
giving them a shelved instance would allow them to re-create it much more 
easily.

Any objections to a blueprint that proposes shelving should be handled with the 
same quota model as snapshotting ?

Tim

On 18/08/16 19:43, "Jonathan D. Proulx"  wrote:

On Thu, Aug 18, 2016 at 03:24:28PM +, Tim Bell wrote:
:
:We’re having a look at VM shelving for the CERN community and struggling 
to find a motivation for a private cloud user to shelve their instances (and 
free up resources they may be only using infrequently).
:
:The problem is that shelved instances seem to still be included in the 
user’s quota. Without internal billing, the best motivation for users to shelve 
would be to allow them to maximize the use of their quota.
:
:Have any other sites used shelving extensively ? How did you motivate your 
users to shelve unused resources?


Hi Tim,

We've just started looking at this and for simialar reasons.  I agree
we should remove shelved resources from project quota. Shelved
instances do still hold some storage resources so there may need to be
new quota to accoutn for that some how...

Currently the only motivation for our users to shelve is to get me to
stop pestering them.

We're considering policy based enforced shelving (based on some yet to
be defined utilization metrics) but that's only a idea at this point
not a plan.

-Jon


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Multiple DHCP Agents

[Fox, Kevin M]

we run 3, one per controller. 40 seems like you might run into some issues and 
really shouldn't hit your controllers that hard at only 40 nodes.

Thanks,
Kevin

From: David Wahlstrom [david.wahlst...@gmail.com]
Sent: Wednesday, August 17, 2016 2:27 PM
To: OpenStack Operators
Subject: [Openstack-operators] Multiple DHCP Agents

Hey everybody,

We're working on getting some dhcp-agent's running on our deployment.  We have 
40-some hypervisors and were thinking about deploying an agent to some/most/a 
few of them.  What I'm curious about is the recommended way to have multiple 
dhcp-agents running and if there is any notable downside to simply running a 
dhcp-agent on every hypervisor and tune the 'dhcp_agents_per_network' value to 
a sane value (something like 2 or 3).  We are using neutron DVR for our L3.

Thanks in advance for the feedback!

--
David W.
Unix, because every barista in Seattle has an MCSE.
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [kolla] question of default users for operators

[Fox, Kevin M]

+1


From: Melvin Hillsman
Sent: Friday, July 29, 2016 9:22:02 PM
To: Steven Dake (stdake); openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [kolla] question of default users for 
operators

Personally, if OpenStack default is to have a member or _member_ as an operator 
I would prefer there be a user according to the default.

Kind regards,
--
Melvin Hillsman
Ops Technical Lead
OpenStack Innovation Center

mrhills...@gmail.com
phone: (210) 312-1267
mobile: (210) 413-1659
Learner | Ideation | Belief | Responsibility | Command
http://osic.org


From: "Steven Dake (stdake)"
Date: Friday, July 29, 2016 at 10:35 PM
To: 
"openstack-operators@lists.openstack.org"
Subject: [Openstack-operators] [kolla] question of default users for operators

Hey folks,

In Kolla we have a significant bug in that Horizon can't be used because it 
requires a member user.  We have a few approaches to fixing this problem in 
mind, but want to understand what Operators want.  Devstack itself has switched 
between member and _member_ although Devstack isn't our reference: Operators 
are.

Which of those do operators prefer (if either).

Another question was whether we should create a default user named "user" and 
use this in place of member.

Thoughts welcome.

Regards
-steve
___ OpenStack-operators mailing 
list 
OpenStack-operators@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

[Fox, Kevin M]

yeah, they work well. but thats not what I'm trying to get at. My point is, the 
patch submitted works only if plumbed up by the server process to stop properly 
on exit. Are we sure every rpc listening service is doing that today? If not, 
how do we find and fix them?

Thanks,
Kevin

From: Davanum Srinivas [dava...@gmail.com]
Sent: Thursday, July 28, 2016 5:31 AM
To: Dmitry Mescheryakov
Cc: Fox, Kevin M; OpenStack Operators
Subject: Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to 
Liberty

Dima, Kevin,

There are PreStop hooks that can be used to gracefully bring down
stuff running in containers:
http://kubernetes.io/docs/user-guide/container-environment/

-- Dims

On Thu, Jul 28, 2016 at 8:22 AM, Dmitry Mescheryakov
 wrote:
>
> 2016-07-26 21:20 GMT+03:00 Fox, Kevin M :
>>
>> It only relates to Kubernetes in that Kubernetes can do automatic rolling
>> upgrades by destroying/replacing a service. If the services don't clean up
>> after themselves, then performing a rolling upgrade will break things.
>>
>> So, what do you think is the best approach to ensuring all the services
>> shut things down properly? Seems like its a cross project issue? Should a
>> spec be submitted?
>
>
> I think that it would be fair if Kubernates sends a sigterm to OpenStack
> service in a container, then wait for the service to shut down and only then
> destroy the container.
>
> It might be not very important for our case though, if we agree to split
> expiration time for fanout and reply queues. And I don't know of any other
> case where an OpenStack service needs to clean up on shutdown in some
> external place.
>
> Thanks,
>
> Dmitry
>
>>
>> Thanks,
>> Kevin
>> ____________
>> From: Dmitry Mescheryakov [dmescherya...@mirantis.com]
>> Sent: Tuesday, July 26, 2016 11:01 AM
>> To: Fox, Kevin M
>> Cc: Sam Morrison; OpenStack Operators
>>
>> Subject: Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving
>> to Liberty
>>
>>
>>
>> 2016-07-25 18:47 GMT+03:00 Fox, Kevin M :
>>>
>>> Ah. Interesting.
>>>
>>> The graceful shutdown would really help the Kubernetes situation too.
>>> Kubernetes can do easy rolling upgrades and having the processes being able
>>> to clean up after themselves as they are upgraded is important. Is this
>>> something that needs to go into oslo.messaging or does it have to be added
>>> to all projects using it?
>>
>>
>> It both needs to be fixed on oslo.messaging side (delete fanout queue on
>> RPC server stop, which is done by Kirill's CR) and on side of projects using
>> it, as they need to actually stop RPC server before shutting down. As I
>> wrote earlier, among Neutron processes right now only openvswitch and
>> metadata agents do not stop RPC server.
>>
>> I am not sure how that relates to Kubernates, as I not much familiar with
>> it.
>>
>> Thanks,
>>
>> Dmitry
>>
>>>
>>>
>>> Thanks,
>>> Kevin
>>> 
>>> From: Dmitry Mescheryakov [dmescherya...@mirantis.com]
>>> Sent: Monday, July 25, 2016 3:47 AM
>>> To: Sam Morrison
>>> Cc: OpenStack Operators
>>> Subject: Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues
>>> moving to Liberty
>>>
>>> Sam,
>>>
>>> For your case I would suggest to lower rabbit_transient_queues_ttl until
>>> you are comfortable with volume of messages which comes during that time.
>>> Setting the parameter to 1 will essentially replicate bahaviour of
>>> auto_delete queues. But I would suggest not to set it that low, as otherwise
>>> your OpenStack will suffer from the original bug. Probably a value like 20
>>> seconds should work in most cases.
>>>
>>> I think that there is a space for improvement here - we can delete reply
>>> and fanout queues on graceful shutdown. But I am not sure if it will be easy
>>> to implement, as it requires services (Nova, Neutron, etc.) to stop RPC
>>> server on sigint and I don't know if they do it right now.
>>>
>>> I don't think we can make case with sigkill any better. Other than that,
>>> the issue could be investigated on Neutron side, maybe number of messages
>>> could be reduced there.
>>>
>>> Thanks,
>>>
>>> Dmitry
>>>
>>> 2016-07-25 9:27 GMT+03:00 Sam Morrison :
>>>>
>>>> We recently upgraded to Liberty and hav

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

[Fox, Kevin M]

It does send a sigterm and wait.

I'm saying, I'm concerned the services aren't all cleaning up after themselves 
today.

Thanks,
Kevin

From: Dmitry Mescheryakov [dmescherya...@mirantis.com]
Sent: Thursday, July 28, 2016 5:22 AM
To: Fox, Kevin M
Cc: Sam Morrison; OpenStack Operators
Subject: Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to 
Liberty


2016-07-26 21:20 GMT+03:00 Fox, Kevin M 
mailto:kevin@pnnl.gov>>:
It only relates to Kubernetes in that Kubernetes can do automatic rolling 
upgrades by destroying/replacing a service. If the services don't clean up 
after themselves, then performing a rolling upgrade will break things.

So, what do you think is the best approach to ensuring all the services shut 
things down properly? Seems like its a cross project issue? Should a spec be 
submitted?

I think that it would be fair if Kubernates sends a sigterm to OpenStack 
service in a container, then wait for the service to shut down and only then 
destroy the container.

It might be not very important for our case though, if we agree to split 
expiration time for fanout and reply queues. And I don't know of any other case 
where an OpenStack service needs to clean up on shutdown in some external place.

Thanks,

Dmitry


Thanks,
Kevin

From: Dmitry Mescheryakov 
[dmescherya...@mirantis.com<mailto:dmescherya...@mirantis.com>]
Sent: Tuesday, July 26, 2016 11:01 AM
To: Fox, Kevin M
Cc: Sam Morrison; OpenStack Operators

Subject: Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to 
Liberty



2016-07-25 18:47 GMT+03:00 Fox, Kevin M 
mailto:kevin@pnnl.gov>>:
Ah. Interesting.

The graceful shutdown would really help the Kubernetes situation too. 
Kubernetes can do easy rolling upgrades and having the processes being able to 
clean up after themselves as they are upgraded is important. Is this something 
that needs to go into oslo.messaging or does it have to be added to all 
projects using it?

It both needs to be fixed on oslo.messaging side (delete fanout queue on RPC 
server stop, which is done by Kirill's CR) and on side of projects using it, as 
they need to actually stop RPC server before shutting down. As I wrote earlier, 
among Neutron processes right now only openvswitch and metadata agents do not 
stop RPC server.

I am not sure how that relates to Kubernates, as I not much familiar with it.

Thanks,

Dmitry


Thanks,
Kevin

From: Dmitry Mescheryakov 
[dmescherya...@mirantis.com<mailto:dmescherya...@mirantis.com>]
Sent: Monday, July 25, 2016 3:47 AM
To: Sam Morrison
Cc: OpenStack Operators
Subject: Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to 
Liberty

Sam,

For your case I would suggest to lower rabbit_transient_queues_ttl until you 
are comfortable with volume of messages which comes during that time. Setting 
the parameter to 1 will essentially replicate bahaviour of auto_delete queues. 
But I would suggest not to set it that low, as otherwise your OpenStack will 
suffer from the original bug. Probably a value like 20 seconds should work in 
most cases.

I think that there is a space for improvement here - we can delete reply and 
fanout queues on graceful shutdown. But I am not sure if it will be easy to 
implement, as it requires services (Nova, Neutron, etc.) to stop RPC server on 
sigint and I don't know if they do it right now.

I don't think we can make case with sigkill any better. Other than that, the 
issue could be investigated on Neutron side, maybe number of messages could be 
reduced there.

Thanks,

Dmitry

2016-07-25 9:27 GMT+03:00 Sam Morrison 
mailto:sorri...@gmail.com>>:
We recently upgraded to Liberty and have come across some issues with queue 
build ups.

This is due to changes in rabbit to set queue expiries as opposed to queue auto 
delete.
See https://bugs.launchpad.net/oslo.messaging/+bug/1515278 for more information.

The fix for this bug is in liberty and it does fix an issue however it causes 
another one.

Every time you restart something that has a fanout queue. Eg. cinder-scheduler 
or the neutron agents you will have
a queue in rabbit that is still bound to the rabbitmq exchange (and so still 
getting messages in) but no consumers.

These messages in these queues are basically rubbish and don’t need to exist. 
Rabbit will delete these queues after 10 mins (although the default in master 
is now changed to 30 mins)

During this time the queue will grow and grow with messages. This sets off our 
nagios alerts and our ops guys have to deal with something that isn’t really an 
issue. They basically delete the queue.

A bad scenario is when you make a change to your cloud that means all your 1000 
neutron agents are restarted, this causes a couple of dead queues per agent to 
hang around. (port updates and security group updates) We get ar

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

[Fox, Kevin M]

It only relates to Kubernetes in that Kubernetes can do automatic rolling 
upgrades by destroying/replacing a service. If the services don't clean up 
after themselves, then performing a rolling upgrade will break things.

So, what do you think is the best approach to ensuring all the services shut 
things down properly? Seems like its a cross project issue? Should a spec be 
submitted?

Thanks,
Kevin

From: Dmitry Mescheryakov [dmescherya...@mirantis.com]
Sent: Tuesday, July 26, 2016 11:01 AM
To: Fox, Kevin M
Cc: Sam Morrison; OpenStack Operators
Subject: Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to 
Liberty



2016-07-25 18:47 GMT+03:00 Fox, Kevin M 
mailto:kevin@pnnl.gov>>:
Ah. Interesting.

The graceful shutdown would really help the Kubernetes situation too. 
Kubernetes can do easy rolling upgrades and having the processes being able to 
clean up after themselves as they are upgraded is important. Is this something 
that needs to go into oslo.messaging or does it have to be added to all 
projects using it?

It both needs to be fixed on oslo.messaging side (delete fanout queue on RPC 
server stop, which is done by Kirill's CR) and on side of projects using it, as 
they need to actually stop RPC server before shutting down. As I wrote earlier, 
among Neutron processes right now only openvswitch and metadata agents do not 
stop RPC server.

I am not sure how that relates to Kubernates, as I not much familiar with it.

Thanks,

Dmitry


Thanks,
Kevin

From: Dmitry Mescheryakov 
[dmescherya...@mirantis.com<mailto:dmescherya...@mirantis.com>]
Sent: Monday, July 25, 2016 3:47 AM
To: Sam Morrison
Cc: OpenStack Operators
Subject: Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to 
Liberty

Sam,

For your case I would suggest to lower rabbit_transient_queues_ttl until you 
are comfortable with volume of messages which comes during that time. Setting 
the parameter to 1 will essentially replicate bahaviour of auto_delete queues. 
But I would suggest not to set it that low, as otherwise your OpenStack will 
suffer from the original bug. Probably a value like 20 seconds should work in 
most cases.

I think that there is a space for improvement here - we can delete reply and 
fanout queues on graceful shutdown. But I am not sure if it will be easy to 
implement, as it requires services (Nova, Neutron, etc.) to stop RPC server on 
sigint and I don't know if they do it right now.

I don't think we can make case with sigkill any better. Other than that, the 
issue could be investigated on Neutron side, maybe number of messages could be 
reduced there.

Thanks,

Dmitry

2016-07-25 9:27 GMT+03:00 Sam Morrison 
mailto:sorri...@gmail.com>>:
We recently upgraded to Liberty and have come across some issues with queue 
build ups.

This is due to changes in rabbit to set queue expiries as opposed to queue auto 
delete.
See https://bugs.launchpad.net/oslo.messaging/+bug/1515278 for more information.

The fix for this bug is in liberty and it does fix an issue however it causes 
another one.

Every time you restart something that has a fanout queue. Eg. cinder-scheduler 
or the neutron agents you will have
a queue in rabbit that is still bound to the rabbitmq exchange (and so still 
getting messages in) but no consumers.

These messages in these queues are basically rubbish and don’t need to exist. 
Rabbit will delete these queues after 10 mins (although the default in master 
is now changed to 30 mins)

During this time the queue will grow and grow with messages. This sets off our 
nagios alerts and our ops guys have to deal with something that isn’t really an 
issue. They basically delete the queue.

A bad scenario is when you make a change to your cloud that means all your 1000 
neutron agents are restarted, this causes a couple of dead queues per agent to 
hang around. (port updates and security group updates) We get around 25 
messages / second on these queues and so you can see after 10 minutes we have a 
ton of messages in these queues.

1000 x 2 x 25 x 600 = 30,000,000 messages in 10 minutes to be precise.

Has anyone else been suffering with this before a raise a bug?

Cheers,
Sam


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org<mailto:OpenStack-operators@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

[Fox, Kevin M]

Yeah, we've experienced it but hadn't had time yet to really dig in like this, 
or gotten a good workaround. If you file a bug, please let me know what number.

Thanks,
Kevin

From: Sam Morrison [sorri...@gmail.com]
Sent: Sunday, July 24, 2016 11:27 PM
To: OpenStack Operators
Subject: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to   
Liberty

We recently upgraded to Liberty and have come across some issues with queue 
build ups.

This is due to changes in rabbit to set queue expiries as opposed to queue auto 
delete.
See https://bugs.launchpad.net/oslo.messaging/+bug/1515278 for more information.

The fix for this bug is in liberty and it does fix an issue however it causes 
another one.

Every time you restart something that has a fanout queue. Eg. cinder-scheduler 
or the neutron agents you will have
a queue in rabbit that is still bound to the rabbitmq exchange (and so still 
getting messages in) but no consumers.

These messages in these queues are basically rubbish and don’t need to exist. 
Rabbit will delete these queues after 10 mins (although the default in master 
is now changed to 30 mins)

During this time the queue will grow and grow with messages. This sets off our 
nagios alerts and our ops guys have to deal with something that isn’t really an 
issue. They basically delete the queue.

A bad scenario is when you make a change to your cloud that means all your 1000 
neutron agents are restarted, this causes a couple of dead queues per agent to 
hang around. (port updates and security group updates) We get around 25 
messages / second on these queues and so you can see after 10 minutes we have a 
ton of messages in these queues.

1000 x 2 x 25 x 600 = 30,000,000 messages in 10 minutes to be precise.

Has anyone else been suffering with this before a raise a bug?

Cheers,
Sam


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

[Fox, Kevin M]

Ah. Interesting.

The graceful shutdown would really help the Kubernetes situation too. 
Kubernetes can do easy rolling upgrades and having the processes being able to 
clean up after themselves as they are upgraded is important. Is this something 
that needs to go into oslo.messaging or does it have to be added to all 
projects using it?

Thanks,
Kevin

From: Dmitry Mescheryakov [dmescherya...@mirantis.com]
Sent: Monday, July 25, 2016 3:47 AM
To: Sam Morrison
Cc: OpenStack Operators
Subject: Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to 
Liberty

Sam,

For your case I would suggest to lower rabbit_transient_queues_ttl until you 
are comfortable with volume of messages which comes during that time. Setting 
the parameter to 1 will essentially replicate bahaviour of auto_delete queues. 
But I would suggest not to set it that low, as otherwise your OpenStack will 
suffer from the original bug. Probably a value like 20 seconds should work in 
most cases.

I think that there is a space for improvement here - we can delete reply and 
fanout queues on graceful shutdown. But I am not sure if it will be easy to 
implement, as it requires services (Nova, Neutron, etc.) to stop RPC server on 
sigint and I don't know if they do it right now.

I don't think we can make case with sigkill any better. Other than that, the 
issue could be investigated on Neutron side, maybe number of messages could be 
reduced there.

Thanks,

Dmitry

2016-07-25 9:27 GMT+03:00 Sam Morrison 
mailto:sorri...@gmail.com>>:
We recently upgraded to Liberty and have come across some issues with queue 
build ups.

This is due to changes in rabbit to set queue expiries as opposed to queue auto 
delete.
See https://bugs.launchpad.net/oslo.messaging/+bug/1515278 for more information.

The fix for this bug is in liberty and it does fix an issue however it causes 
another one.

Every time you restart something that has a fanout queue. Eg. cinder-scheduler 
or the neutron agents you will have
a queue in rabbit that is still bound to the rabbitmq exchange (and so still 
getting messages in) but no consumers.

These messages in these queues are basically rubbish and don’t need to exist. 
Rabbit will delete these queues after 10 mins (although the default in master 
is now changed to 30 mins)

During this time the queue will grow and grow with messages. This sets off our 
nagios alerts and our ops guys have to deal with something that isn’t really an 
issue. They basically delete the queue.

A bad scenario is when you make a change to your cloud that means all your 1000 
neutron agents are restarted, this causes a couple of dead queues per agent to 
hang around. (port updates and security group updates) We get around 25 
messages / second on these queues and so you can see after 10 minutes we have a 
ton of messages in these queues.

1000 x 2 x 25 x 600 = 30,000,000 messages in 10 minutes to be precise.

Has anyone else been suffering with this before a raise a bug?

Cheers,
Sam


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack] [tools] OpenStack client in a Docker container

[Fox, Kevin M]

Cool. Maybe this could be contributed to the Kolla project?

Thanks,
Kevin


From: Gerard Braad
Sent: Monday, June 27, 2016 8:58:18 PM
To: openst...@lists.openstack.org; openstack-operators
Subject: [Openstack-operators] [openstack] [tools] OpenStack client in a Docker 
container

Hi all,


When you reinstall workstations or test environments as often as I do,
you would like to automate everything... or containerize it. So, I
packaged the OpenStack client in a Docker container on Ubuntu and
CentOS. And to make it more convenient, I added Lars's 'stack' helper
tool. Just have a look at the registry [1] or the source [2].

Usage is as easy as:

Store your stackrc in ~/.stack named as an endpoint; e.g. ~/.stack/trystack
$ docker pull gbraad/openstack-client:centos
$ alias stack='docker run -it  --rm -v ~/.stack:/root/.stack
gbraad/openstack-client:centos stack'
$ stack trystack openstack server list

Comments welcomed...

regards,


Gerard

[1] https://hub.docker.com/r/gbraad/openstack-client/
[2] https://github.com/gbraad/docker-openstack-client/

--

   Gerard Braad | http://gbraad.nl
   [ Doing Open Source Matters ]

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [glance] glance-registry deprecation: Request for feedback

[Fox, Kevin M]

Is there a copy-from-url method that's not deprecated yet?

The app catalog is still pointing users at the command line in v1 mode

Thanks,
Kevin

From: Matt Fischer [m...@mattfischer.com]
Sent: Thursday, May 12, 2016 4:43 PM
To: Flavio Percoco
Cc: openstack-...@lists.openstack.org; openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [glance] glance-registry deprecation: 
Request for feedback


On May 11, 2016 10:03 PM, "Flavio Percoco" 
mailto:fla...@redhat.com>> wrote:
>
> Greetings,
>
> The Glance team is evaluating the needs and usefulness of the Glance Registry
> service and this email is a request for feedback from the overall community
> before the team moves forward with anything.
>
> Historically, there have been reasons to create this service. Some deployments
> use it to hide database credentials from Glance public endpoints, others use 
> it
> for scaling purposes and others because v1 depends on it. This is a good time
> for the team to re-evaluate the need of these services since v2 doesn't depend
> on it.
>
> So, here's the big question:
>
> Why do you think this service should be kept around?

I've not seen any responses so far so wanted to just say we have no use case 
for it. I assume this also explains the silence from the rest of the ops. +1 to 
remove.
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Fox, Kevin M]

Its not horrible to do in other languages. Here's one in go I'm working on:
https://github.com/kubernetes/kubernetes/pull/25391

The complication does come in if you want to make it perform well...

If you have a cached admin token, then its only one additional keystone call. 
But if you don't have a caching mechanism or an expired token, that one call 
can turn into 2.

-

Tangential question. What auth token will be used when the vm is downloading 
the vendor data? Or does it pre-generate it and stick it into the db?

Thanks,
Kevin

From: mikalst...@gmail.com [mikalst...@gmail.com] on behalf of Michael Still 
[mi...@stillhq.com]
Sent: Thursday, May 12, 2016 4:06 PM
To: Fox, Kevin M
Cc: David Medberry; Ned Rhudy; openstack-operators@lists.openstack.org; Sean 
Dague
Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?

I'm just going to reply to myself here with another status update.

The design seems largely settled at this point, with one exception -- how does 
nova authenticate with the external microservice?

The current proposal is to have nova use the client's keystone token to 
authenticate with the external microservice. This is a neat solution because 
its what nova does when talking to other services in your OpenStack deployment, 
so its consistent and well understood.

The catch here is that it means your external microservice needs to know how to 
do keystone authentication. That's well understood for python microservices, 
and I can provide sample code for that case using the keystone wsgi middleware. 
On the other hand, its harder for things like Java where I'm not sure I'm aware 
of any keystone auth implementation. Is effectively requiring the microservices 
to be written in python a particular problem? I'm hoping not given that all the 
current plugins are written in python by definition.

Cheers,
Michael




On Wed, May 4, 2016 at 7:37 AM, Michael Still 
mailto:mi...@stillhq.com>> wrote:
Hey,

I just wanted to let people know that the review is progressing, but we have a 
question.

Do operators really need to call more than one external REST service to collect 
vendordata? We can implement that in nova, but it would be nice to reduce the 
complexity to only having one external REST service. If you needed to call more 
than one service you could of course write a REST service that aggregated REST 
services.

Does anyone in the operator community have strong feelings either way? Should 
nova be able to call more than one external vendordata REST service?

Thanks,
Michael




On Sat, Apr 30, 2016 at 4:11 AM, Michael Still 
mailto:mi...@stillhq.com>> wrote:
So, after a series of hallway track chats this week, I wrote this:

https://review.openstack.org/#/c/310904/

Which is a proposal for how to implement vendordata in a way which would 
(probably) be acceptable to nova, whilst also meeting the needs of operators. I 
should reinforce that because this week is so hectic nova core hasn't really 
talked about this yet, but I am pretty sure I understand and have addressed 
Sean's concerns.

I'd be curious as to if the proposed solution actually meets your needs.

Michael




On Mon, Apr 18, 2016 at 10:55 AM, Fox, Kevin M 
mailto:kevin@pnnl.gov>> wrote:
We've used it too to work around the lack of instance users in nova. Please 
keep it until a viable solution can be reached.

Thanks,
Kevin

From: David Medberry [openst...@medberry.net<mailto:openst...@medberry.net>]
Sent: Monday, April 18, 2016 7:16 AM
To: Ned Rhudy
Cc: 
openstack-operators@lists.openstack.org<mailto:openstack-operators@lists.openstack.org>

Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?

Hi Ned, Jay,

We use it also and I have to agree, it's onerous to require users to add that 
functionality back in. Where was this discussed?

On Mon, Apr 18, 2016 at 8:13 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) 
mailto:erh...@bloomberg.net>> wrote:
Requiring users to remember to pass specific userdata through to their instance 
at every launch in order to replace functionality that currently works 
invisible to them would be a step backwards. It's an alternative, yes, but it's 
an alternative that adds burden to our users and is not one we would pursue.

What is the rationale for desiring to remove this functionality?

From: jaypi...@gmail.com<mailto:jaypi...@gmail.com>
Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?
On 04/18/2016 09:24 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) wrote:
> I noticed while reading through Mitaka release notes that
> vendordata_driver has been deprecated in Mitaka
> (https://review.openstack.org/#/c/288107/) and is slated for removal at
> some point. This came as somewhat of a surprise to me - I search

Re: [Openstack-operators] vmware to openstack

[Fox, Kevin M]

There are a couple of reasons I think this may cause you problems even if it 
was technically feasible:

 * VMs deployed on vmware are built as pets and really need/benifit from the 
features of vmware. if migrating to anything but a vmware openstack cloud then 
those underlying expectations built into the vms no longer hold true and 
badness can ensue.
 * Users without experience in the cloud expect it to work the same way as 
regular vm's. Its very different. if you can "just move" a vm in, then it 
incorrectly strengthens the expectation. If users have to work a little bit to 
get it into the cloud, then there's opportunity for re-education on the 
differences and a chance to do the migration correctly, de-pet'ing the vm at 
migration time. If you don't do it at migration time, it tends not to happen at 
all, and as an operator, you then have  a sea of pets you end up having to deal 
with and not very many tools at your disposal to deal with them. :/

Yes, it seems like a good idea to help your users by transitioning 
automagically, but in the long run, it causes more pain then it saves.

Thanks,
Kevin

From: Jonathan Proulx [j...@csail.mit.edu]
Sent: Friday, May 06, 2016 8:50 AM
To: Silence Dogood
Cc: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] vmware to openstack

On Fri, May 06, 2016 at 11:39:03AM -0400, Silence Dogood wrote:
:this strikes me as a really bad idea from a security standpoint... in fact
:it would violently violate like every audit / policy requirement I am aware
:of.

I come from a research environment with admittedly low thresholds for
this sort of thing (we don't do classified or medical stuff so no hard
regulations), but...

If the same entity controls the source cloud, translation layer, and
destination cloud where's the harm?

I can (maybe) see an issue if you're piping all you images through a
third part not involved in the source or target clouds. That's similar
to running on a public cloud trust wise so if you can trust a public
provider even that should be surmountable, but I don't think that's
the model suggested here.

-Jon

:
:-matt
:
:On Fri, May 6, 2016 at 11:32 AM, suresh kumar  wrote:
:
:> Thanks Mariano, that really helps.
:>
:> On Fri, May 6, 2016 at 11:16 AM, Mariano Cunietti 
:> wrote:
:>
:>>
:>> From: suresh kumar 
:>> Date: Friday 6 May 2016 at 17:07
:>> To: "openstack-operators@lists.openstack.org" <
:>> openstack-operators@lists.openstack.org>
:>> Subject: [Openstack-operators] vmware to openstack
:>>
:>>
:>> I am working on a project to migrate vm's from vmware to openstack where
:>> am using tools to convert disk and later upload them to glance and spin
:>> instances are there any tools/scripts where I can migrate multiple vm's at
:>> a time.
:>>
:>>
:>> https://cloudbase.it/cloud-migration-as-a-service/
:>>
:>>
:>
:> ___
:> OpenStack-operators mailing list
:> OpenStack-operators@lists.openstack.org
:> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
:>
:>

:___
:OpenStack-operators mailing list
:OpenStack-operators@lists.openstack.org
:http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


--

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Fox, Kevin M]

For a tenant though, I may not want to have to write user-data to bind every 
thing I launch through horizon's nova workflow, heat, sahara, etc. Just having 
one place to put the hook and its always called has some major advantages.

Thanks,
Kevin

From: Mathieu Gagné [mga...@calavera.ca]
Sent: Tuesday, May 03, 2016 3:25 PM
To: Fox, Kevin M
Cc: Michael Still; openstack-operators@lists.openstack.org; Sean Dague
Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?

On Tue, May 3, 2016 at 5:51 PM, Fox, Kevin M  wrote:
>
> I think I see at least one use case for minimum 2 hooks...
>
> Cloud provider wants to inject some stuff.
>
> Cloud tenant wants their own hook called to inject stuff to point to the
> Config Management server in their own tenant.
>
> Maybe that's not vendor_data but tenant_data or something...
>

I think this could very well be addressed by cloud-init or any other
sane initialization service.
We just have to make sure instance identification and any other
reasonable information are made available to those tools so they can
be passed to their own Config Management server.
I see vendor_data as a non-intrusive way to pass additional data to
the instance without requiring the vendor/provider to inject an agent
or custom code within the customer's image.

--
Mathieu


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Fox, Kevin M]

Depends on what its used for... I can see it potentially being used with Chef 
or Puppet, for calling hooks into AD to bind to a domain. etc. Probably at the 
same time. We use it with our keyserver (something similar to Barbican but 
created before Barbican was a thing) to relay trust info between Nova and the 
Keyserver through the Instance.

I've done some careful inheriting of our vendor_data plug-in to get all the 
features in one plugin, but I could see it being difficult for some folks when 
more features are added.

I think I see at least one use case for minimum 2 hooks...

Cloud provider wants to inject some stuff.

Cloud tenant wants their own hook called to inject stuff to point to the Config 
Management server in their own tenant.

Maybe that's not vendor_data but tenant_data or something...

Thanks,
Kevin

From: mikalst...@gmail.com [mikalst...@gmail.com] on behalf of Michael Still 
[mi...@stillhq.com]
Sent: Tuesday, May 03, 2016 2:37 PM
To: Fox, Kevin M
Cc: David Medberry; Ned Rhudy; openstack-operators@lists.openstack.org; Sean 
Dague
Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?

Hey,

I just wanted to let people know that the review is progressing, but we have a 
question.

Do operators really need to call more than one external REST service to collect 
vendordata? We can implement that in nova, but it would be nice to reduce the 
complexity to only having one external REST service. If you needed to call more 
than one service you could of course write a REST service that aggregated REST 
services.

Does anyone in the operator community have strong feelings either way? Should 
nova be able to call more than one external vendordata REST service?

Thanks,
Michael




On Sat, Apr 30, 2016 at 4:11 AM, Michael Still 
mailto:mi...@stillhq.com>> wrote:
So, after a series of hallway track chats this week, I wrote this:

https://review.openstack.org/#/c/310904/

Which is a proposal for how to implement vendordata in a way which would 
(probably) be acceptable to nova, whilst also meeting the needs of operators. I 
should reinforce that because this week is so hectic nova core hasn't really 
talked about this yet, but I am pretty sure I understand and have addressed 
Sean's concerns.

I'd be curious as to if the proposed solution actually meets your needs.

Michael




On Mon, Apr 18, 2016 at 10:55 AM, Fox, Kevin M 
mailto:kevin@pnnl.gov>> wrote:
We've used it too to work around the lack of instance users in nova. Please 
keep it until a viable solution can be reached.

Thanks,
Kevin

From: David Medberry [openst...@medberry.net<mailto:openst...@medberry.net>]
Sent: Monday, April 18, 2016 7:16 AM
To: Ned Rhudy
Cc: 
openstack-operators@lists.openstack.org<mailto:openstack-operators@lists.openstack.org>

Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?

Hi Ned, Jay,

We use it also and I have to agree, it's onerous to require users to add that 
functionality back in. Where was this discussed?

On Mon, Apr 18, 2016 at 8:13 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) 
mailto:erh...@bloomberg.net>> wrote:
Requiring users to remember to pass specific userdata through to their instance 
at every launch in order to replace functionality that currently works 
invisible to them would be a step backwards. It's an alternative, yes, but it's 
an alternative that adds burden to our users and is not one we would pursue.

What is the rationale for desiring to remove this functionality?

From: jaypi...@gmail.com<mailto:jaypi...@gmail.com>
Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?
On 04/18/2016 09:24 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) wrote:
> I noticed while reading through Mitaka release notes that
> vendordata_driver has been deprecated in Mitaka
> (https://review.openstack.org/#/c/288107/) and is slated for removal at
> some point. This came as somewhat of a surprise to me - I searched
> openstack-dev for vendordata-related subject lines going back to January
> and found no discussion on the matter (IRC logs, while available on
> eavesdrop, are not trivially searchable without a little scripting to
> fetch them first, so I didn't check there yet).
>
> We at Bloomberg make heavy use of this particular feature to inject
> dynamically generated JSON into the metadata service of instances; the
> content of the JSON differs depending on the instance making the request
> to the metadata service. The functionality that adds the contents of a
> static JSON file, while remaining around, is not suitable for our use case.
>
> Please let me know if you use vendordata_driver so that I/we can present
> an organized case for why this option or equivalent functionality needs
> to remain around. The alternative is 

Re: [Openstack-operators] Anyone else use vendordata_driver in nova.conf?

[Fox, Kevin M]

We've used it too to work around the lack of instance users in nova. Please 
keep it until a viable solution can be reached.

Thanks,
Kevin

From: David Medberry [openst...@medberry.net]
Sent: Monday, April 18, 2016 7:16 AM
To: Ned Rhudy
Cc: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?

Hi Ned, Jay,

We use it also and I have to agree, it's onerous to require users to add that 
functionality back in. Where was this discussed?

On Mon, Apr 18, 2016 at 8:13 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) 
mailto:erh...@bloomberg.net>> wrote:
Requiring users to remember to pass specific userdata through to their instance 
at every launch in order to replace functionality that currently works 
invisible to them would be a step backwards. It's an alternative, yes, but it's 
an alternative that adds burden to our users and is not one we would pursue.

What is the rationale for desiring to remove this functionality?

From: jaypi...@gmail.com
Subject: Re: [Openstack-operators] Anyone else use vendordata_driver in 
nova.conf?
On 04/18/2016 09:24 AM, Ned Rhudy (BLOOMBERG/ 731 LEX) wrote:
> I noticed while reading through Mitaka release notes that
> vendordata_driver has been deprecated in Mitaka
> (https://review.openstack.org/#/c/288107/) and is slated for removal at
> some point. This came as somewhat of a surprise to me - I searched
> openstack-dev for vendordata-related subject lines going back to January
> and found no discussion on the matter (IRC logs, while available on
> eavesdrop, are not trivially searchable without a little scripting to
> fetch them first, so I didn't check there yet).
>
> We at Bloomberg make heavy use of this particular feature to inject
> dynamically generated JSON into the metadata service of instances; the
> content of the JSON differs depending on the instance making the request
> to the metadata service. The functionality that adds the contents of a
> static JSON file, while remaining around, is not suitable for our use case.
>
> Please let me know if you use vendordata_driver so that I/we can present
> an organized case for why this option or equivalent functionality needs
> to remain around. The alternative is that we end up patching the
> vendordata driver directly in Nova when we move to Mitaka, which I'd
> like to avoid; as a matter of principle I would rather see more
> classloader overrides, not fewer.

Wouldn't an alternative be to use something like Chef, Puppet, Ansible,
Saltstack, etc and their associated config variable storage services
like Hiera or something similar to publish custom metadata? That way,
all you need to pass to your instance (via userdata) is a URI or
connection string and some auth details for your config storage service
and the instance can grab whatever you need.

Thoughts?
-jay

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [puppet][kolla] Multi-node installation

[Fox, Kevin M]

Kolla folks, the word opinionated in the deploymentbconfig tool is usually seen 
by ops as heavily tieing your hands to the point of being very painful or a 
show stopper. I get that your trying to say that now its "opinionated" for easy 
install but supports being unopinionated, but some ops wont read past the first 
part of the sentence. The reaction to the word opinionated can be that strong. 
Best to just claim unopinionated with an easy install option or something like 
that.

Thanks,
Kevin


From: Steven Dake (stdake)
Sent: Wednesday, April 13, 2016 1:40:16 AM
To: Rayson Ho; Emilien Macchi
Cc: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [puppet][kolla] Multi-node installation

A suggestion to give Kolla a spin inside…

From: Rayson Ho mailto:raysonlo...@gmail.com>>
Date: Tuesday, April 12, 2016 at 1:58 PM
To: Emilien Macchi mailto:emil...@redhat.com>>
Cc: 
"openstack-operators@lists.openstack.org"
 
mailto:openstack-operators@lists.openstack.org>>
Subject: Re: [Openstack-operators] [puppet] Multi-node installation

On Tue, Apr 12, 2016 at 4:17 PM, Emilien Macchi 
mailto:emil...@redhat.com>> wrote:
> If you need to start a composition layer from scratch, you'll need to
> compose the manifests yourself.
> Puppet OpenStack is not an opinionated project like TripleO, Fuel,
> Kolla. We're a library of Puppet modules that you can use at wish.
> You need to be a bit familiar with Puppet.
> But if you look at wat we do in puppet-openstack-integration, you're
> missing a few parameters to make it work in multi-node, it should not
> be hard.

Thanks for the the prompt reply, Emilien! (And hello from Toronto!)

I am just trying to install OpenStack on a cluster of 3 CentOS machines. I 
tried the OpenStack Ansible installation, but there are 2 showstoppers:
 - OpenStack Ansible works great on Ubuntu, but doesn't support RHEL (we have 
RHEL licenses, we use CentOS and Oracle Linux for R & D. Our production 
machines will be running RHEL so Ubuntu doesn't cut it.)

Rayson,

Kolla deploys multinode with high availability.  Kolla is an opinionated 
deployment management tool unless the Operator has opinions of their own.  
Kolla offers complete customization of any OpenStack parameter via augmentation 
files which override the defaults specified by Kolla.  This permits an Operator 
to have something working right out o the box, but change things around as the 
Operator's experience grows with OpenStack.  Its essentially the best of both 
worlds :)


 - If we install OpenStack behind an HTTP Proxy & firewall, then some parts of 
OpenStack setup do not work well with the proxy. So we install OpenStack 
outside the firewall and then bring them into the datacenter. However, with the 
OpenStack Ansible setup, every time the nodes boot up, they try to contact the 
package servers again to check for newer versions, and thus they would get 
errors as they need to go through the proxy.

Kolla will work well for what I think you desire.  To upgrade kolla, you run 
"kolla-ansible upgrade".  There is no runtime automatic checking for new stuff. 
 Oracle also ships Kolla 1.0.0 as part of their OpenStack implementation.



I use Puppet OpenStack because it works well on RHEL-based OSes & the 
installation doesn't seem to contact the outside world once it is configured. 
And I used Puppet a few years ago (mainly for basic provisioning in AWS/EC2), 
so that's another plus for Puppet. A few weeks ago I looked at Packstack, which 
calls the Puppet installation internally, but last time I used it I encountered 
other issues. However, as it supports multi-node installations, I think I will 
look at how it calls the Puppet manifests.

If there is an easier way to install OpenStack (ideally 14.0, and we would like 
to stay as close to the official OpenStack source as possible) on RHEL, without 
trying to download newer packages once the installation is done, and runs on 
more than 1 machine, please let me know!

I feel Kolla is pretty easy for operators to use and get their heads around.  I 
always ask new Operators to point out their top 3 pain points after eval.  When 
we started the project it was all over the place.  Now there doesn’t seem to be 
much if any pain points pointed out by Operators when operating Kolla.  A 
majority of people find it intuitive, well designed, and easy to operate and 
manage an OpenStack cloud long  term.  I personally think our top problem at 
this point is lack of documentation around operating Kolla long term – which 
will be rectified in May/June after Austin Summit concludes.

Most folks get a deployment going on bare metal in 1-2 hours – sometimes with a 
little help from the kolla irc channel #kolla on freenode.  Our community runs 
24 hours a day – so drop in if you have questions.  OSAD and Kolla are 
completely different experiences – the only thing they really have in common is 
the us

Re: [Openstack-operators] Intel 10 GbE / bonding issue with hash policy layer3+4

[Fox, Kevin M]

We saw a problem recently with layer3+4. We're still working on it, but a 
possible datapoint:
We had them connected through a cisco switch and saw iperf only use 50% of 
capacity about 50% of the runs, and most of the rest of the time full. 
Seemingly randomly.

We connected the two nodes directly together with layer3+4 balancing and it hit 
full performance every time. so in our case, it has something to do with the 
switch in the middle.

Thanks,
Kevin

From: Stig Telfer [stig.openst...@telfer.org]
Sent: Thursday, March 24, 2016 3:10 AM
To: openstack-operators
Subject: Re: [Openstack-operators] Intel 10 GbE / bonding issue with hash   
policy  layer3+4

Hi Sascha -

I had a similar experience earlier this week.  I was testing bond performance 
on a dual-port Mellanox NIC, LACP-bonded with layer 3+4 transmit hash.  First 
run of iperf (8 streams), I saw a reasonable distribution across the links.  
Shortly after that, performance dropped to a level that suggested no 
distribution.  I didn’t look into it any further at the time.

This was on CoreOS beta (kernel 4.3-6 IIRC).

If our experiences have a common root cause, I have a different distro and NIC 
to you, which might eliminate those components.  I should have the kit back in 
this configuration within a week, I’ll try to probe and report back.

Best wishes,
Stig


> On 23 Mar 2016, at 15:45, MailingLists - EWS 
>  wrote:
>
> Sascha,
>
> What version of the ixgbe driver are you using? Is it the same on both
> kernels? Have you tried the latest "out of tree driver" from E1000 to see if
> the issue goes away?
>
> I follow the E1000 mailing list and I seem to recall some rather recent
> posts regarding bonding and the ixgbe along with some patches being applied
> to the driver, however I don't know what version of kernel these issues were
> on, or even if the patches were accepted.
>
> https://sourceforge.net/p/e1000/mailman/e1000-devel/thread/87618083B2453E4A8
> 714035B62D67992504FB5FF%40FMSMSX105.amr.corp.intel.com/#msg34727125
>
> Something about a timing issue with detecting the slave's link speed and
> passing that information to the bonding driver in a timely fashion.
>
> Tom Walsh
> ExpressHosting
> https://expresshosting.net/
>
>> -Original Message-
>> From: Sascha Vogt [mailto:sascha.v...@gmail.com]
>> Sent: Wednesday, March 23, 2016 5:54 AM
>> To: openstack-operators
>> Subject: [Openstack-operators] Intel 10 GbE / bonding issue with hash
> policy
>> layer3+4
>>
>> Hi all,
>>
>> I thought it might be of interest / get feedback from the operators
>> community about an oddity we experienced with Intel 10 GbE NICs and LACP
>> bonding.
>>
>> We have Ubuntu 14.04.4 as OS and Intel 10 GbE NICs with the ixgbe Kernel
>> module. We use VLANS for ceph-client, ceph-data, openstack-data,
>> openstack-client networks all on a single LACP bonding of two 10 GbE
> ports.
>>
>> As bonding hash policy we chose layer3+4 so we can use the full 20 Gb even
>> if only two servers communicate with each other. Typically we check that
> by
>> using iperf to a single server with -P 4 and see if we exceed the 10 Gb
> limit
>> (just a few times to check).
>>
>> Due to Ubuntus default of installing the latest Kernel our new host had
>> Kernel 4.2.0 instead of the Kernel 3.16 the other machines had and we
>> noticed that iperf only used 10 Gb.
>>
>>> # cat /proc/net/bonding/bond0
>>> Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
>>>
>>> Bonding Mode: IEEE 802.3ad Dynamic link aggregation Transmit Hash
>>> Policy: layer3+4 (1)
>>
>> This was shown on both - Kernel 3.16 and 4.2.0
>>
>> After downgrading to Kernel 3.16 we got the iperf results we expected.
>>
>> Does anyone have a similar setup? Anyone noticed the same things? To us
>> this looks like a bug in the Kernel (ixgbe module?), or are we
>> misunderstanding the hash policy layer3+4?
>>
>> Any feedback is welcome :) I have not yet posted this to the Kernel ML or
>> Ubuntus ML yet, so if no one here is having a similar setup I'll move over
>> there. I just thought OpenStack ops might be the place were it is most
> likely
>> that someone has a similar setup :)
>>
>> Greetings
>> -Sascha-
>>
>> ___
>> OpenStack-operators mailing list
>> OpenStack-operators@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/ope

Re: [Openstack-operators] [openstack-operators] Fernet key rotation

[Fox, Kevin M]

You can just rotate without restarting services.

We're rotating currently only once a day.

We rotate on one machine, then rsync the data to the others in a cron job. Has 
been working well for a couple of months now.

Thanks,
Kevin

From: Ajay Kalambur (akalambu) [akala...@cisco.com]
Sent: Wednesday, March 16, 2016 2:44 PM
To: OpenStack Operators
Subject: [Openstack-operators] [openstack-operators] Fernet key rotation

Hi
In a multi node HA deployment for production does key rotate need a keystone 
process reboot or should we just run the fernet rotate on one node and 
distribute it without restarting any process
I presume keystone can handle the rotation without a restart?

I also assume this key rotation can happen without a maintenance window

What do folks typically do in production and how often do you rotate keys

Ajay

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-operators][neutron] use-case for multiple routers within a tenant

[Fox, Kevin M]

We use them all the time, and openstack in one version actually broke them on 
us. (I wrote and contributed a unit test so it shouldn't happen again.)

Use case:

You have two external networks.
1. Internet - One that's directly connected to the internet.
2. One that is a private network space and is available to the whole cloud.

Each tenant gets a router on each network (two routers total), defaulting to 
the internet one, and the subnet has a static routing rule to make the external 
private net route to the right neutron router.

The user can then add floating ip's to one or both of the networks making the 
vm available on that network. If the service is internet facing, they can just 
put that type of floating ip on. If they want to share it with the other 
tenants but not with the internet, they just put that type of floating ip on.

We don't have many ip's on the internet side, so having it split like this 
allows us to conserve ip's.

Make sense?

Thanks,
Kevin


From: Rubab Syed [rubab.sye...@gmail.com]
Sent: Tuesday, March 08, 2016 2:20 PM
To: openstack-operators@lists.openstack.org
Subject: [Openstack-operators] [openstack-operators][neutron] use-case for 
multiple routers within a tenant

Hi all,

I am trying to get a general understanding of OpenStack networking. Can someone 
please point out a simple use-case for using multiple routers within same 
tenant?


Thanks,
Rubab
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] What are you excited for in Mitaka?

[Fox, Kevin M]

I guess my current focus is on network HA, since these popped into my head 
right away and all seem to be network related:

* l3+dvr in neutron's a huge, huge thing.
* neutron external rbac. (https://review.openstack.org/#/c/282295/ really 
useful for more then one reason)
* Octavia anti-affinity (keep your backup lb on a different host then the 
primary. this is really important)
* Octavia Active/Standby. (deal with node dying automatically)

Thanks,
Kevin


From: Jesse Keating [omg...@us.ibm.com]
Sent: Tuesday, March 08, 2016 8:15 AM
To: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] What are you excited for in Mitaka?

I found one. In Mitaka Neutron you can take an existing router and turn it into 
a HA router. That will make it quite easy to transition older environments into 
a more stable configuration. Previously we'd have to tear down the router and 
build a new one, then re-attach all the interfaces.
-jlk


- Original message -
From: Tom Fifield 
To: Hauke Bruno Wollentin , OpenStack 
Operators 
Cc:
Subject: Re: [Openstack-operators] What are you excited for in Mitaka?
Date: Tue, Mar 8, 2016 6:24 AM

Nice one!

I just found a good one: the `use_neutron`` option.

 From Mitaka, in nova.conf, instead of putting

network_api_class=nova.network.neutronv2.api.API
security_group_api = neutron

you can just go

use_neutron=True


On 07/03/16 14:12, Hauke Bruno Wollentin wrote:
> Hi Tom,
>
> Hard to choose but from my business driven ops view I love to see the
> ongoing work in Neutron DVR, fernet tokens and also Cells v2.
>
> cheers,
> hauke
>
>
> *From:* Tom Fifield 
> *Sent:* Mar 7, 2016 12:42 PM
> *To:* OpenStack Operators
> *Subject:* [Openstack-operators] What are you excited for in Mitaka?
>
> Hi Ops,
>
> Mitaka is swiftly upon us.
>
> As you know, there's a whole lot of communications that goes out around
> release time. It's nice to pull out a few of the 300 features/3000 bugs
> we've worked on that are a bit more interesting than standard and
> highlight them where we can.
>
> So, quick question: is there anything you've seen going in to any of the
> projects that is particularly awesome from the ops perspective?
>
>
> Regards,
>
>
> Tom
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators



___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Setting affinity based on instance type

[Fox, Kevin M]

you usually do that on an instance level with server groups. do you have an 
example where you might want to do it at the flavor level?

Thanks,
Kevin

From: Adam Lawson [alaw...@aqorn.com]
Sent: Wednesday, March 02, 2016 2:48 PM
To: openstack-operators@lists.openstack.org
Subject: [Openstack-operators] Setting affinity based on instance type

I'm sure this is possible but I'm trying to find the info I need in the docs so 
I figured I'd pitch this to you guys while I continue looking:

Is it possible to set an affinity/anti-affinity policy to ensure instance Type 
A is weighted for/against co-location on the same physical host with instance 
Type B?

Basically I have no requirement for server-group affinity but rather to ensure 
specific workloads are as separate as possible.

Thoughts?

//adam


Adam Lawson

AQORN, Inc.
427 North Tatnall Street
Ste. 58461
Wilmington, Delaware 19801-2230
Toll-free: (844) 4-AQORN-NOW ext. 101
International: +1 302-387-4660
Direct: +1 916-246-2072
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [kolla] Question about how Operators deploy

[Fox, Kevin M]

We usually use two vips.

Thanks,
Kevin


From: Steven Dake (stdake)
Sent: Friday, February 12, 2016 6:04:45 AM
To: openstack-operators@lists.openstack.org
Subject: [Openstack-operators] [kolla] Question about how Operators deploy

Hi folks,

Unfortunately I won't be able to make it to the Operator midcycle because of 
budget constraints or I would find the answer to this question there.  The 
Kolla upstream is busy sorting out external ssl termination and a question 
arose in the Kolla community around operator requirements for publicURL vs 
internalURL VIP management.

At present, Kolla creates 3 Haproxy containers across 3 HA nodes with one VIP 
managed by keepalived.  The VIP is used for internal communication only.  Our 
PUBLIC_URL is set to a DNS name, and we expect the Operator to sort out how to 
map that DNS name to the internal VIP used by Kolla.  The way I do this in my 
home lab is to use NAT to NAT my public_URL from the internet (hosted by 
dyndns) to my internal VIP that haproxies to my 3 HA control nodes.  This is 
secure assuming someone doesn't bust through my NAT.

An alternative has been suggested which is to use TWO vips.  One for 
internal_url, one for public_url.  Then the operator would only be responsible 
for selecting where to to allocate the public_url endpoint's VIP.  I think this 
allows more flexibility without necessarily requiring NAT while still 
delivering a secure solution.

Not having ever run an OpenStack cloud in production, how do the Operators want 
it?  Our deciding factor here is what Operators want, not what is necessarily 
currently in the code base.  We still have time to make this work differently 
for Mitaka, but I need feedback/advice quickly.

The security guide seems to imply two VIPs are the way to Operate: (big 
diagram):
http://docs.openstack.org/security-guide/networking/architecture.html

The IRC discussion is here for reference:
http://eavesdrop.openstack.org/irclogs/%23kolla/%23kolla.2016-02-12.log.html#t2016-02-12T12:09:08

Thanks in Advance!
-steve

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [keystone] Usage of trusts with v2.0 authentication

[Fox, Kevin M]

They were used indirectly I think when you had some services configured for v2 
only authentication support because the service didn't work with v2. For 
example nova->neutron was v2 only for a while. I think all services are 
supporting v3 these days so that would no longer be necessary?

Thanks,
Kevin

From: Lance Bragstad [lbrags...@gmail.com]
Sent: Tuesday, February 09, 2016 9:06 AM
To: OpenStack Development Mailing List (not for usage questions); 
openstack-operators@lists.openstack.org
Subject: [Openstack-operators] [keystone] Usage of trusts with v2.0 
authentication

When trusts were implemented, they were designed to work as an extension under 
the version 3 API. The implementation didn't prevent the use of a trust to 
authenticate against version 2.0, which was never officially documented in the 
v2.0 API docs.

The keystone team is curious if there is anyone creating trusts using v3 and 
then using them against version 2.0. If not, we'd like to remove/deprecate 
support for that case in v2.0. If so, then we'll have to add official 
documentation for trusts against v2.0 and incorporate that case into fernet.

Thanks!

Lance
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] RAID / stripe block storage volumes

[Fox, Kevin M]

We've used ceph to address the storage requirement in small clouds pretty well. 
it works pretty well with only two storage nodes with replication set to 2, and 
because of the radosgw, you can share your small amount of storage between the 
object store and the block store avoiding the need to overprovision swift-only 
or cinder-only to handle usage unknowns. Its just one pool of storage.

Your right, using lvm is like telling your users, don't do pets, but then 
having pets at the heart of your system. when you loose one, you loose a lot. 
With a small ceph, you can take out one of the nodes, burn it to the ground and 
put it back, and it just works. No pets.

Do consider ceph for the small use case.

Thanks,
Kevin


From: Robert Starmer [rob...@kumul.us]
Sent: Monday, February 08, 2016 1:30 PM
To: Ned Rhudy
Cc: OpenStack Operators
Subject: Re: [Openstack-operators] RAID / stripe block storage volumes

Ned's model is the model I meant by "multiple underlying storage services".  
Most of the systems I've built are LV/LVM only,  a few added Ceph as an 
alternative/live-migration option, and one where we used Gluster due to size.  
Note that the environments I have worked with in general are small (~20 
compute), so huge Ceph environments aren't common.  I am also working on a 
project where the storage backend is entirely NFS...

And I think users are more and more educated to assume that there is nothing 
guaranteed.  There is the realization, at least for a good set of the customers 
I've worked with (and I try to educate the non-believers), that the way you get 
best effect from a system like OpenStack is to consider everything disposable. 
The one gap I've seen is that there are plenty of folks who don't deploy SWIFT, 
and without some form of object store, there's still the question of where you 
place your datasets so that they can be quickly recovered (and how do you keep 
them up to date if you do have one).  With VMs, there's the concept that you 
can recover quickly because the "dataset" e.g. your OS, is already there for 
you, and in plenty of small environments, that's only as true as the glance 
repository (guess what's usually backing that when there's no SWIFT around...).

So I see the issue as a holistic one. How do you show operators/users that they 
should consider everything disposable if we only look at the current running 
instance as the "thing"   Somewhere you still likely need some form of 
distributed resilience (and yes, I can see using the distributed Canonical, 
Centos, RedHat, Fedora, Debian, etc. mirrors as your distributed Image backup 
but what about the database content, etc.).

Robert

On Mon, Feb 8, 2016 at 1:44 PM, Ned Rhudy (BLOOMBERG/ 731 LEX) 
mailto:erh...@bloomberg.net>> wrote:
In our environments, we offer two types of storage. Tenants can either use 
Ceph/RBD and trade speed/latency for reliability and protection against 
physical disk failures, or they can launch instances that are realized as LVs 
on an LVM VG that we create on top of a RAID 0 spanning all but the OS disk on 
the hypervisor. This lets the users elect to go all-in on speed and sacrifice 
reliability for applications where replication/HA is handled at the app level, 
if the data on the instance is sourced from elsewhere, or if they just don't 
care much about the data.

There are some further changes to our approach that we would like to make down 
the road, but in general our users seem to like the current system and being 
able to forgo reliability or speed as their circumstances demand.

From: j...@topjian.net
Subject: Re: [Openstack-operators] RAID / stripe block storage volumes
Hi Robert,

Can you elaborate on "multiple underlying storage services"?

The reason I asked the initial question is because historically we've made our 
block storage service resilient to failure. Historically we also made our 
compute environment resilient to failure, too, but over time, we've seen users 
become more educated to cope with compute failure. As a result, we've been able 
to become more lenient with regard to building resilient compute environments.

We've been discussing how possible it would be to translate that same idea to 
block storage. Rather than have a large HA storage cluster (whether Ceph, 
Gluster, NetApp, etc), is it possible to offer simple single LVM volume servers 
and push the failure handling on to the user?

Of course, this doesn't work for all types of use cases and environments. We 
still have projects which require the cloud to own most responsibility for 
failure than the users.

But for environments were we offer general purpose / best effort compute and 
storage, what methods are available to help the user be resilient to block 
storage failures?

Joe

On Mon, Feb 8, 2016 at 12:09 PM, Robert Starmer 
mailto:rob...@kumul.us>> wrote:
I've always recommended providing multiple underlying storage services to 
provide this rathe

Re: [Openstack-operators] DVR and public IP consumption

[Fox, Kevin M]

I wish I had time for it. :/ Maybe in a few months when I get this new 
production cloud deployed. Can't promise anything though


I'm very much in favor of floating ip's. I'd even argue for them for ipv6. Not 
because of SNAT, or saving IP's. Its because they are an important abstraction 
that lets you turn pets into cattle.

Let me explain... ip addresses tend to be state that gets placed into various 
places. dns, config files for other services, etc. As such it is expected to be 
somewhat long lived. A VM on the other hand, shouldn't be. You should be able 
to stand up a new one, get it ready, and once checked out, move load from the 
old to the new. A floating ip is a perfect way to do that. You leave it on the 
old vm until everything checks out, then move the floating ip. You can even use 
the trick to further scale things Its originally on a VM, but you make a 
load balancer, add the old vm to the load balancer, then move the floating ip 
to the loadbalancer. then you can add more vm's to the LB. Seamless. Without a 
floating ip, none of that is possible. You have to take painful downtimes.

So, please neutron team, floating ip's for v6 are a thing that should be 
possible. The implementation can be quite different. I think ipv6 has a 
mobility extension that maybe can be used instead of snatting. But the goal of 
having an abstraction for a network service endpoint that can be moved around 
(floating ip) really needs to stay.

I'd also really like to see them for tenant networks for similar reasons. Them 
only working on external networks is limiting.

Thanks,
Kevin




From: Robert Starmer [rob...@kumul.us]
Sent: Friday, January 29, 2016 1:21 AM
To: Fox, Kevin M
Cc: Carl Baldwin; OpenStack Operators; Tomas Vondra
Subject: Re: [Openstack-operators] DVR and public IP consumption

I don't think there's anything wrong with your suggestion, as I can't find a 
path where the extra address is actually used (it doesn't get used in any NAT 
mapping, so it is really vestigial). The question now is, will anyone in the 
community be interested in extending the DVR code in this fashion (interested 
in writing a spec?).

I personally am a bigger proponent of dropping the whole Floating IP charade, 
and moving wholesale to v6 and routing right to the VM/container endpoint.  But 
maybe that's just my own odd view.


On Thu, Jan 28, 2016 at 10:10 AM, Fox, Kevin M 
mailto:kevin@pnnl.gov>> wrote:
Ah. so it was done just to make it simple to reuse lots of existing code to get 
DVR working quickly and thus a current requirement, but there is nothing 
stopping further enhancements to be made to eliminate it in the future?

What about a step in between what's there now, and eliminating it completely. 
If the router code expects there to be an ip allocated for it on every compute 
node, could you share one external ip between all the compute node routers? 
Since the network will never actually use it, it probably doesn't matter if its 
conflicting but it would still allow the existing code to function the way it 
always has, greatly simplifying implementation?

Thanks,
Kevin


From: Robert Starmer [rob...@kumul.us<mailto:rob...@kumul.us>]
Sent: Wednesday, January 27, 2016 8:34 PM
To: Fox, Kevin M
Cc: Carl Baldwin; OpenStack Operators; Tomas Vondra

Subject: Re: [Openstack-operators] DVR and public IP consumption

I think I've created a bit of confusion, because I forgot that DVR still does 
SNAT (generic non Floating IP tied NAT) on a central network node just like in 
the non-DVR model.  The extra address that is consumed is allocated to a FIP 
specific namespace when a DVR is made responsible for supporting a tenant's 
floating IP, and the question then is: Why do I need this _extra_ external 
address from the floating IP pool for the FIP namespace, since it's the 
allocation of a tenant requested floating IP to a tenant VM that triggers the 
DVR to implement the FIP namespace function in the first place.

In both the Paris and Vancouver DVR presentations "We add distributed FIP 
support at the expense of an _extra_ external address per device, but the FIP 
namespace is then shared across all tenants". Given that there is no "external" 
interface for the DVR interface for floating IPs until at least one tenant 
allocates one, a new namespace needs to be created to act as the termination 
for the tenant's floating IP.  A normal tenant router would have an address 
allocated already, because it has a port allocated onto the external network 
(this is the address that SNAT overloads for those non-floating associated 
machines that lets them communicate with the Internet at large), but in this 
case, no such interface exists until the namespace is created and attached to 
the external network, so when the flo

Re: [Openstack-operators] DVR and public IP consumption

[Fox, Kevin M]

 be solved by introducing a
few more configuration options. I like the way it can use L2 and provider
networks to integrate with the rest of the datacenter. No BGP L3VPN tunnels,
which cannot be done in open-source.

Tomas

>
> On Wed, Jan 27, 2016 at 3:33 PM, Fox, Kevin M
 wrote:
>
>
>
>
>
> But there already is a second external address, the fip address that's
nating. Is there a double nat? I'm a little confused.
> Thanks,
> Kevin
> From: Robert Starmer [robert-irot69hj...@public.gmane.org]Sent: Wednesday,
January 27, 2016 3:20 PMTo: Carl BaldwinCc: OpenStack Operators; Tomas
VondraSubject: Re: [Openstack-operators] DVR and public IP consumption
>
>
>
>
> You can't get rid of the "External" address as it's used to direct return
traffic to the right router node.  DVR as implemented is really just a local
NAT gateway per physical compute node.  The outside of your NAT needs to be
publicly unique,
>  so it needs it's own address.  Some SDN solutions can provide a truly
distributed router model, because they globally know the inside state of the
NAT environment, and can forward packets back to the internal source
properly, regardless of which distributed
>  forwarder receives the incoming "external" packets.
>
> If the number of external addresses consumed is an issue, you may consider
the dual gateway HA model instead of DVR.  This uses classic multi-router
models where one router takes on the task of forwading packets, and the
other device just acts as a backup.
>  You do still have a software bottleneck at your router, unless you then
also use one of the plugins that supports hardware L3 (last I checked,
Juniper, Arista, Cisco, etc. all provide an L3 plugin that is HA capable),
but you only burn 3 External addresses
>  for the router (and 3 internal network addresses per tenant side
interface if that matters).
>
> Hope that clarifies a bit,
> Robert
>
>
> On Tue, Jan 26, 2016 at 4:14 AM, Carl Baldwin
>  ecbaldwin.net> wrote:
> On Thu, Jan 14, 2016 at 2:45 AM, Tomas Vondra
 wrote:
> > Hi!
> > I have just deployed an OpenStack Kilo installation with DVR and expected
> > that it will consume one Public IP per network node as per
> >
> http://assafmuller.com/2015/04/15/distributed-virtual-routing-floating-ips/,
> > but it still eats one per virtual Router.
> > What is the correct behavior?Regardless of DVR, a Neutron router burns
one IP per virtual router
> which it uses to SNAT traffic from instances that do not have floating
> IPs.
> When you use DVR, an additional IP is consumed for each compute host
> running an L3 agent in DVR mode.  There has been some discussion about
> how this can be eliminated but no action has been taken to do this.
> > Otherwise, it works as a DVR should according to documentation. There are
> > router namespaces at both compute and network nodes, snat namespaces at the
> > network nodes and fip namespaces at the compute nodes. Every router has a
> > router_interface_distributed and a router_centralized_snat with private IPs,
> > however the router_gateway has a public IP, which I would like to getr id of
> > to increase density.I'm not sure if it is possible to avoid burning
these IPs at this
> time.  Maybe someone else can chime in with more detail.
> Carl


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] DVR and public IP consumption

[Fox, Kevin M]

Ah. so it was done just to make it simple to reuse lots of existing code to get 
DVR working quickly and thus a current requirement, but there is nothing 
stopping further enhancements to be made to eliminate it in the future?

What about a step in between what's there now, and eliminating it completely. 
If the router code expects there to be an ip allocated for it on every compute 
node, could you share one external ip between all the compute node routers? 
Since the network will never actually use it, it probably doesn't matter if its 
conflicting but it would still allow the existing code to function the way it 
always has, greatly simplifying implementation?

Thanks,
Kevin


From: Robert Starmer [rob...@kumul.us]
Sent: Wednesday, January 27, 2016 8:34 PM
To: Fox, Kevin M
Cc: Carl Baldwin; OpenStack Operators; Tomas Vondra
Subject: Re: [Openstack-operators] DVR and public IP consumption

I think I've created a bit of confusion, because I forgot that DVR still does 
SNAT (generic non Floating IP tied NAT) on a central network node just like in 
the non-DVR model.  The extra address that is consumed is allocated to a FIP 
specific namespace when a DVR is made responsible for supporting a tenant's 
floating IP, and the question then is: Why do I need this _extra_ external 
address from the floating IP pool for the FIP namespace, since it's the 
allocation of a tenant requested floating IP to a tenant VM that triggers the 
DVR to implement the FIP namespace function in the first place.

In both the Paris and Vancouver DVR presentations "We add distributed FIP 
support at the expense of an _extra_ external address per device, but the FIP 
namespace is then shared across all tenants". Given that there is no "external" 
interface for the DVR interface for floating IPs until at least one tenant 
allocates one, a new namespace needs to be created to act as the termination 
for the tenant's floating IP.  A normal tenant router would have an address 
allocated already, because it has a port allocated onto the external network 
(this is the address that SNAT overloads for those non-floating associated 
machines that lets them communicate with the Internet at large), but in this 
case, no such interface exists until the namespace is created and attached to 
the external network, so when the floating IP port is created, an address is 
simply allocated from the External (e.g. floating) pool for the interface.  And 
_then_ the floating IP is allocated to the namespace as well. The fact that 
this extra address is used is a part of the normal port allocation process (and 
default port-security anti-spoofing processes) that exist already, and 
simplifies the process of moving tenant allocated floating addresses around 
(the port state for the floating namespace doesn't change, it keeps it's 
special mac and address regardless of what ever else goes on). So don't think 
of it as a Floating IP allocated to the DVR, it's just the DVR's local 
representative for it's port on the external network.  Tenant addresses are 
then "on top" of this setup.

So, in-efficient, yes.  Part of DVR history, yes.  Confusing to us mere network 
mortals, yes.  But that's how I see it. And sorry for the SNAT reference, just 
adding my own additional layer of "this is how it should be"  on top.

Robert

On Wed, Jan 27, 2016 at 3:33 PM, Fox, Kevin M 
mailto:kevin@pnnl.gov>> wrote:
But there already is a second external address, the fip address that's nating. 
Is there a double nat? I'm a little confused.

Thanks,
Kevin

From: Robert Starmer [rob...@kumul.us<mailto:rob...@kumul.us>]
Sent: Wednesday, January 27, 2016 3:20 PM
To: Carl Baldwin
Cc: OpenStack Operators; Tomas Vondra
Subject: Re: [Openstack-operators] DVR and public IP consumption

You can't get rid of the "External" address as it's used to direct return 
traffic to the right router node.  DVR as implemented is really just a local 
NAT gateway per physical compute node.  The outside of your NAT needs to be 
publicly unique, so it needs it's own address.  Some SDN solutions can provide 
a truly distributed router model, because they globally know the inside state 
of the NAT environment, and can forward packets back to the internal source 
properly, regardless of which distributed forwarder receives the incoming 
"external" packets.

If the number of external addresses consumed is an issue, you may consider the 
dual gateway HA model instead of DVR.  This uses classic multi-router models 
where one router takes on the task of forwading packets, and the other device 
just acts as a backup.  You do still have a software bottleneck at your router, 
unless you then also use one of the plugins that supports hardware L3 (last I 
checked, Juniper, Arista, C

Re: [Openstack-operators] DVR and public IP consumption

[Fox, Kevin M]

But there already is a second external address, the fip address that's nating. 
Is there a double nat? I'm a little confused.

Thanks,
Kevin

From: Robert Starmer [rob...@kumul.us]
Sent: Wednesday, January 27, 2016 3:20 PM
To: Carl Baldwin
Cc: OpenStack Operators; Tomas Vondra
Subject: Re: [Openstack-operators] DVR and public IP consumption

You can't get rid of the "External" address as it's used to direct return 
traffic to the right router node.  DVR as implemented is really just a local 
NAT gateway per physical compute node.  The outside of your NAT needs to be 
publicly unique, so it needs it's own address.  Some SDN solutions can provide 
a truly distributed router model, because they globally know the inside state 
of the NAT environment, and can forward packets back to the internal source 
properly, regardless of which distributed forwarder receives the incoming 
"external" packets.

If the number of external addresses consumed is an issue, you may consider the 
dual gateway HA model instead of DVR.  This uses classic multi-router models 
where one router takes on the task of forwading packets, and the other device 
just acts as a backup.  You do still have a software bottleneck at your router, 
unless you then also use one of the plugins that supports hardware L3 (last I 
checked, Juniper, Arista, Cisco, etc. all provide an L3 plugin that is HA 
capable), but you only burn 3 External addresses for the router (and 3 internal 
network addresses per tenant side interface if that matters).

Hope that clarifies a bit,
Robert

On Tue, Jan 26, 2016 at 4:14 AM, Carl Baldwin 
mailto:c...@ecbaldwin.net>> wrote:
On Thu, Jan 14, 2016 at 2:45 AM, Tomas Vondra 
mailto:von...@czech-itc.cz>> wrote:
> Hi!
> I have just deployed an OpenStack Kilo installation with DVR and expected
> that it will consume one Public IP per network node as per
> http://assafmuller.com/2015/04/15/distributed-virtual-routing-floating-ips/,
> but it still eats one per virtual Router.
> What is the correct behavior?

Regardless of DVR, a Neutron router burns one IP per virtual router
which it uses to SNAT traffic from instances that do not have floating
IPs.

When you use DVR, an additional IP is consumed for each compute host
running an L3 agent in DVR mode.  There has been some discussion about
how this can be eliminated but no action has been taken to do this.

> Otherwise, it works as a DVR should according to documentation. There are
> router namespaces at both compute and network nodes, snat namespaces at the
> network nodes and fip namespaces at the compute nodes. Every router has a
> router_interface_distributed and a router_centralized_snat with private IPs,
> however the router_gateway has a public IP, which I would like to getr id of
> to increase density.

I'm not sure if it is possible to avoid burning these IPs at this
time.  Maybe someone else can chime in with more detail.

Carl

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Storage backend for glance

[Fox, Kevin M]

ceph would work pretty well for that use case too. We've run a ceph with two 
ost's, with the replication set to 2, to back both cinder and glance for HA. 
Nothing complicated needed to get it working. Less complicated then drbd I 
think. You can then also easily scale it out as needed.

Thanks,
Kevin

From: Hauke Bruno Wollentin [hauke-bruno.wollen...@innovo-cloud.de]
Sent: Wednesday, January 27, 2016 7:04 AM
To: Sławek Kapłoński; openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Storage backend for glance

Hi Slawek,

we use a shared NFS Export to save images to Glance. That enables HA in (imho) 
the simplest way.

For your setting you could use something like a hourly/daily/whenever rsync job 
and set the 'second' Glance node to passive/standby in the load balancing. Also 
it will be possible to run some kind of cluster between the 2 nodes, like DRBD 
or GlusterFS (but that would be a bit overpowered maybe).

cheers,
hauke

-Ursprüngliche Nachricht-
Von: Sławek Kapłoński [mailto:sla...@kaplonski.pl]
Gesendet: Mittwoch, 27. Januar 2016 15:23
An: openstack-operators@lists.openstack.org
Betreff: [Openstack-operators] Storage backend for glance

Hello,

I want to install Openstack with at least two glance nodes (to have HA) but 
with local filesystem as glance storage. Is it possible to use something like 
that in setup with two glance nodes? Maybe someone of You already have 
something like that?
I'm asking because AFAIK is image will be stored on one glance server and 
nova-compute will ask other glance host to download image then image will not 
be available to download and instance will be in ERROR state.
So maybe someone used it somehow in similar setup (maybe some NFS or something 
like that?). What are You experience with it?

--
Best regards / Pozdrawiam
Sławek Kapłoński
sla...@kaplonski.pl

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Cinder-backup to swift

[Fox, Kevin M]

great. thanks for letting us know. :)

Kevin

From: raju [raju.r...@gmail.com]
Sent: Tuesday, January 26, 2016 11:25 AM
To: Fox, Kevin M
Cc: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Cinder-backup to swift

Kevin/Swami,

Now it is working fine, created new volume after changing the services to same 
host resolved the issue.

appreciate your help

On Tue, Jan 26, 2016 at 2:04 PM, Fox, Kevin M 
mailto:kevin@pnnl.gov>> wrote:
crank up debugging and look in the logs? anything interesting?

Thanks,
Kevin

From: raju [raju.r...@gmail.com<mailto:raju.r...@gmail.com>]
Sent: Tuesday, January 26, 2016 9:55 AM
To: Fox, Kevin M
Cc: 
openstack-operators@lists.openstack.org<mailto:openstack-operators@lists.openstack.org>
Subject: Re: [Openstack-operators] Cinder-backup to swift

now both services running on same host still same issue

 cinder service-list
+--+-+--+-+---++-+
|  Binary  |   Host  | Zone |  Status | State | 
Updated_at | Disabled Reason |
+--+-+--+-+---++-+
|  cinder-backup   |node-7..stack   | nova | enabled |   up  | 
2016-01-26T17:53:44.00 |   None  |
| cinder-scheduler |node-7..stack   | nova | enabled |   up  | 
2016-01-26T17:53:47.00 |   None  |
|  cinder-volume   | node-7..stack@pure | nova | enabled |   up  | 
2016-01-26T17:53:43.00 |   None  |
+--+-+--+-+---++-+
root@node-7:~# cinder backup-create --container Cinder_Backup 
0d2eb15f-77b9-408b-86c3-e80fb24948cf
ERROR: Service cinder-backup could not be found. (HTTP 500) (Request-ID: 
req-c7e7555a-4fe0-4d09-9a3b-3a71e3c7d962)


On Tue, Jan 26, 2016 at 11:37 AM, Fox, Kevin M 
mailto:kevin@pnnl.gov>> wrote:
There is a "feature" where cinder-backup and cinder-volume must be running on 
the same node. If they are not, you get an error like that. I tripped over it 
myself.

Thanks,
Kevin

From: raju [raju.r...@gmail.com<mailto:raju.r...@gmail.com>]
Sent: Tuesday, January 26, 2016 8:16 AM
To: 
openstack-operators@lists.openstack.org<mailto:openstack-operators@lists.openstack.org>
Subject: [Openstack-operators] Cinder-backup to swift


Hi,

I am trying to backup cinder volumes to swift and enabled backup driver in 
cinder conf and startred cinder-backup service, when am trying to create a 
backup it is failing with below error


Error:
cinder backup-create --container Cinder_Backup 
0d2eb15f-77b9-408b-86c3-e80fb24948cf

ERROR: Service cinder-backup could not be found. (HTTP 500) (Request-ID: 
req-2ce2bee4-d237-4191-b9ea-48a72ccbb579)

# cinder service-list
+--+-+--+--+---++-+
|  Binary  | Host| Zone |  Status  | State |
 Updated_at | Disabled Reason |
+--+-+--+--+---++-+
|  cinder-backup   |  node-7..stack | nova | enabled  |   up  | 
2016-01-25T22:03:09.00 |   None  |
| cinder-scheduler |  node-7..stack | nova | enabled  |   up  | 
2016-01-25T22:03:06.00 |   None  |
 |
|  cinder-volume   |  pure@pure  | nova | enabled  |   up  | 
2016-01-25T22:03:09.00 |   None  |
+--+-+--+--+---++-+

service cinder-backup status
cinder-backup start/running, process 21223

LOG:

INFO cinder.api.openstack.wsgi [req-6ea85659-aa48-4dfd-8ca4-bf21c34537f3 
7bc0cb1d6c394eb598b4612b49b6b891 56083ff01d5d42848f3c8d227ed70006 - - -] GET 
http://10.24.4.2:8776/v1/56083ff01d5d42848f3c8d227ed70006/volumes/0d2eb15f-77b9-408b-86c3-e80fb24948cf
2016-01-26 16:15:23.012 11301 INFO cinder.api.openstack.wsgi 
[req-6ea85659-aa48-4dfd-8ca4-bf21c34537f3 7bc0cb1d6c394eb598b4612b49b6b891 
56083ff01d5d42848f3c8d227ed70006 - - -] 
http://10.24.4.2:8776/v1/56083ff01d5d42848f3c8d227ed70006/volumes/0d2eb15f-77b9-408b-86c3-e80fb24948cf
 returned with HTTP 200
2016-01-26 16:15:23.014 11301 INFO eventlet.wsgi.server 
[req-6ea85659-aa48-4dfd-8ca4-bf21c34537f3 7bc0cb1d6c394eb598b4612b49b6b891 
56083ff01d5d42848f3c8d227ed70006 - - -] 10.24.4.2 - - [26/Jan/2016 16:15:23] 
"GET 
/v1/56083ff01d5d42848f3c8d227ed70006/volumes/0d2eb15f-77b9-408b-86c3-e80fb24948cf
 HTTP/1.1" 200 963 0.417786
2016-01-26 16:15:23.030 11299 INFO eventlet.wsgi.server [-] (11299) accepted 
('10.24.4.2', 37650)
2016-01-2

Re: [Openstack-operators] Cinder-backup to swift

[Fox, Kevin M]

crank up debugging and look in the logs? anything interesting?

Thanks,
Kevin

From: raju [raju.r...@gmail.com]
Sent: Tuesday, January 26, 2016 9:55 AM
To: Fox, Kevin M
Cc: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Cinder-backup to swift

now both services running on same host still same issue

 cinder service-list
+--+-+--+-+---++-+
|  Binary  |   Host  | Zone |  Status | State | 
Updated_at | Disabled Reason |
+--+-+--+-+---++-+
|  cinder-backup   |node-7..stack   | nova | enabled |   up  | 
2016-01-26T17:53:44.00 |   None  |
| cinder-scheduler |node-7..stack   | nova | enabled |   up  | 
2016-01-26T17:53:47.00 |   None  |
|  cinder-volume   | node-7..stack@pure | nova | enabled |   up  | 
2016-01-26T17:53:43.00 |   None  |
+--+-+--+-+---++-+
root@node-7:~# cinder backup-create --container Cinder_Backup 
0d2eb15f-77b9-408b-86c3-e80fb24948cf
ERROR: Service cinder-backup could not be found. (HTTP 500) (Request-ID: 
req-c7e7555a-4fe0-4d09-9a3b-3a71e3c7d962)


On Tue, Jan 26, 2016 at 11:37 AM, Fox, Kevin M 
mailto:kevin@pnnl.gov>> wrote:
There is a "feature" where cinder-backup and cinder-volume must be running on 
the same node. If they are not, you get an error like that. I tripped over it 
myself.

Thanks,
Kevin

From: raju [raju.r...@gmail.com<mailto:raju.r...@gmail.com>]
Sent: Tuesday, January 26, 2016 8:16 AM
To: 
openstack-operators@lists.openstack.org<mailto:openstack-operators@lists.openstack.org>
Subject: [Openstack-operators] Cinder-backup to swift


Hi,

I am trying to backup cinder volumes to swift and enabled backup driver in 
cinder conf and startred cinder-backup service, when am trying to create a 
backup it is failing with below error


Error:
cinder backup-create --container Cinder_Backup 
0d2eb15f-77b9-408b-86c3-e80fb24948cf

ERROR: Service cinder-backup could not be found. (HTTP 500) (Request-ID: 
req-2ce2bee4-d237-4191-b9ea-48a72ccbb579)

# cinder service-list
+--+-+--+--+---++-+
|  Binary  | Host| Zone |  Status  | State |
 Updated_at | Disabled Reason |
+--+-+--+--+---++-+
|  cinder-backup   |  node-7..stack | nova | enabled  |   up  | 
2016-01-25T22:03:09.00 |   None  |
| cinder-scheduler |  node-7..stack | nova | enabled  |   up  | 
2016-01-25T22:03:06.00 |   None  |
 |
|  cinder-volume   |  pure@pure  | nova | enabled  |   up  | 
2016-01-25T22:03:09.00 |   None  |
+--+-+--+--+---++-+

service cinder-backup status
cinder-backup start/running, process 21223

LOG:

INFO cinder.api.openstack.wsgi [req-6ea85659-aa48-4dfd-8ca4-bf21c34537f3 
7bc0cb1d6c394eb598b4612b49b6b891 56083ff01d5d42848f3c8d227ed70006 - - -] GET 
http://10.24.4.2:8776/v1/56083ff01d5d42848f3c8d227ed70006/volumes/0d2eb15f-77b9-408b-86c3-e80fb24948cf
2016-01-26 16:15:23.012 11301 INFO cinder.api.openstack.wsgi 
[req-6ea85659-aa48-4dfd-8ca4-bf21c34537f3 7bc0cb1d6c394eb598b4612b49b6b891 
56083ff01d5d42848f3c8d227ed70006 - - -] 
http://10.24.4.2:8776/v1/56083ff01d5d42848f3c8d227ed70006/volumes/0d2eb15f-77b9-408b-86c3-e80fb24948cf
 returned with HTTP 200
2016-01-26 16:15:23.014 11301 INFO eventlet.wsgi.server 
[req-6ea85659-aa48-4dfd-8ca4-bf21c34537f3 7bc0cb1d6c394eb598b4612b49b6b891 
56083ff01d5d42848f3c8d227ed70006 - - -] 10.24.4.2 - - [26/Jan/2016 16:15:23] 
"GET 
/v1/56083ff01d5d42848f3c8d227ed70006/volumes/0d2eb15f-77b9-408b-86c3-e80fb24948cf
 HTTP/1.1" 200 963 0.417786
2016-01-26 16:15:23.030 11299 INFO eventlet.wsgi.server [-] (11299) accepted 
('10.24.4.2', 37650)
2016-01-26 16:15:23.123 11299 INFO cinder.api.openstack.wsgi 
[req-b1d4c37a-70e7-40d0-ac39-e5f109d4f62f 7bc0cb1d6c394eb598b4612b49b6b891 
56083ff01d5d42848f3c8d227ed70006 - - -] POST 
http://10.24.4.2:8776/v1/56083ff01d5d42848f3c8d227ed70006/backups
2016-01-26 16:15:23.125 11299 INFO cinder.api.contrib.backups 
[req-b1d4c37a-70e7-40d0-ac39-e5f109d4f62f 7bc0cb1d6c394eb598b4612b49b6b891 
56083ff01d5d42848f3c8d227ed70006 - - -] Creating backup of volume 
0d2eb15f-77b9-408b-86c3-e80fb24948cf in container Cinder_Backup
2016-01-26 16:15:23.193 11299 INFO cinder.api.openstack.wsgi 
[req-f698

Re: [Openstack-operators] Cinder-backup to swift

[Fox, Kevin M]

There is a "feature" where cinder-backup and cinder-volume must be running on 
the same node. If they are not, you get an error like that. I tripped over it 
myself.

Thanks,
Kevin

From: raju [raju.r...@gmail.com]
Sent: Tuesday, January 26, 2016 8:16 AM
To: openstack-operators@lists.openstack.org
Subject: [Openstack-operators] Cinder-backup to swift


Hi,

I am trying to backup cinder volumes to swift and enabled backup driver in 
cinder conf and startred cinder-backup service, when am trying to create a 
backup it is failing with below error


Error:
cinder backup-create --container Cinder_Backup 
0d2eb15f-77b9-408b-86c3-e80fb24948cf

ERROR: Service cinder-backup could not be found. (HTTP 500) (Request-ID: 
req-2ce2bee4-d237-4191-b9ea-48a72ccbb579)

# cinder service-list
+--+-+--+--+---++-+
|  Binary  | Host| Zone |  Status  | State |
 Updated_at | Disabled Reason |
+--+-+--+--+---++-+
|  cinder-backup   |  node-7..stack | nova | enabled  |   up  | 
2016-01-25T22:03:09.00 |   None  |
| cinder-scheduler |  node-7..stack | nova | enabled  |   up  | 
2016-01-25T22:03:06.00 |   None  |
 |
|  cinder-volume   |  pure@pure  | nova | enabled  |   up  | 
2016-01-25T22:03:09.00 |   None  |
+--+-+--+--+---++-+

service cinder-backup status
cinder-backup start/running, process 21223

LOG:

INFO cinder.api.openstack.wsgi [req-6ea85659-aa48-4dfd-8ca4-bf21c34537f3 
7bc0cb1d6c394eb598b4612b49b6b891 56083ff01d5d42848f3c8d227ed70006 - - -] GET 
http://10.24.4.2:8776/v1/56083ff01d5d42848f3c8d227ed70006/volumes/0d2eb15f-77b9-408b-86c3-e80fb24948cf
2016-01-26 16:15:23.012 11301 INFO cinder.api.openstack.wsgi 
[req-6ea85659-aa48-4dfd-8ca4-bf21c34537f3 7bc0cb1d6c394eb598b4612b49b6b891 
56083ff01d5d42848f3c8d227ed70006 - - -] 
http://10.24.4.2:8776/v1/56083ff01d5d42848f3c8d227ed70006/volumes/0d2eb15f-77b9-408b-86c3-e80fb24948cf
 returned with HTTP 200
2016-01-26 16:15:23.014 11301 INFO eventlet.wsgi.server 
[req-6ea85659-aa48-4dfd-8ca4-bf21c34537f3 7bc0cb1d6c394eb598b4612b49b6b891 
56083ff01d5d42848f3c8d227ed70006 - - -] 10.24.4.2 - - [26/Jan/2016 16:15:23] 
"GET 
/v1/56083ff01d5d42848f3c8d227ed70006/volumes/0d2eb15f-77b9-408b-86c3-e80fb24948cf
 HTTP/1.1" 200 963 0.417786
2016-01-26 16:15:23.030 11299 INFO eventlet.wsgi.server [-] (11299) accepted 
('10.24.4.2', 37650)
2016-01-26 16:15:23.123 11299 INFO cinder.api.openstack.wsgi 
[req-b1d4c37a-70e7-40d0-ac39-e5f109d4f62f 7bc0cb1d6c394eb598b4612b49b6b891 
56083ff01d5d42848f3c8d227ed70006 - - -] POST 
http://10.24.4.2:8776/v1/56083ff01d5d42848f3c8d227ed70006/backups
2016-01-26 16:15:23.125 11299 INFO cinder.api.contrib.backups 
[req-b1d4c37a-70e7-40d0-ac39-e5f109d4f62f 7bc0cb1d6c394eb598b4612b49b6b891 
56083ff01d5d42848f3c8d227ed70006 - - -] Creating backup of volume 
0d2eb15f-77b9-408b-86c3-e80fb24948cf in container Cinder_Backup
2016-01-26 16:15:23.193 11299 INFO cinder.api.openstack.wsgi 
[req-f698019f-d0cf-4057-857e-37aef6c40bbe - - - - -] HTTP exception thrown: 
Service cinder-backup could not be found.
2016-01-26 16:15:23.196 11299 INFO cinder.api.openstack.wsgi 
[req-f698019f-d0cf-4057-857e-37aef6c40bbe - - - - -] 
http://10.24.4.2:8776/v1/56083ff01d5d42848f3c8d227ed70006/backups returned with 
HTTP 500
2016-01-26 16:15:23.199 11299 INFO eventlet.wsgi.server 
[req-f698019f-d0cf-4057-857e-37aef6c40bbe - - - - -] 10.24.4.2 - - [26/Jan/2016 
16:15:23] "POST /v1/56083ff01d5d42848f3c8d227ed70006/backups HTTP/1.1" 500 378 
0.158105

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Galera setup testing

[Fox, Kevin M]

Awsome news. Should there be a tag added for gallera multimaster safe to let us 
ops know about these things?

Thanks,
Kevin


From: Kevin Benton
Sent: Monday, December 07, 2015 6:08:12 PM
To: Matteo Panella
Cc: OpenStack Operators
Subject: Re: [Openstack-operators] Galera setup testing


Neutron has protection against these now as of liberty with API level retry 
operations so this shouldn't be a problem for neutron any more.

On Dec 7, 2015 4:06 PM, "Matteo Panella" 
mailto:matteo.pane...@cnaf.infn.it>> wrote:
On 2015-12-07 22:45, James Dempsey wrote:
+1 for designating one node as primary.  This helped us reduce some
deadlocks that we were seeing when balancing sessions between DB hosts.

Keystone most likely won't be affected by writeset certification failures,
but other services (especially Neutron) are going to be hit by one sooner
or later.

Unfortunately, Galera doesn't take very kindly "SELECT ... FOR UPDATE", so
you can't really do proper load balancing (aside from designating a different 
node
as master in multiple listen stanzas, each one for a different set of services).

Regards,
--
Matteo Panella
INFN CNAF
Via Ranzani 13/2 c - 40127 Bologna, Italy
Phone: +39 051 609 2903

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] DIB in container vs VM

[Fox, Kevin M]

Whats the host system? It looks like its trying to apply selinux stuff. I'm not 
sure that's supported at all in docker. :/

It may be if your running one of the el's or fedora's as the docker host and 
container?

Thanks,
Kevin

From: Abel Lopez [alopg...@gmail.com]
Sent: Monday, December 07, 2015 1:57 PM
To: Fox, Kevin M
Cc: Clint Byrum; openstack-operators
Subject: Re: [Openstack-operators] DIB in container vs VM

Thanks for the suggestion.
I am indeed running it privileged.

One of the examples I wanted to share was: building Fedora image on an ubuntu 
docker image.
The error I get is in the finalize.d phase
http://paste.openstack.org/show/481088/

Error connecting to audit system.

Anyone have an idea?

On Dec 2, 2015, at 4:14 PM, Fox, Kevin M 
mailto:kevin@pnnl.gov>> wrote:

Did you try launching the docker container with the privilege flag? dib does a 
lot of privileged things last I looked.

Thanks,
Kevin

From: Clint Byrum [cl...@fewbar.com<mailto:cl...@fewbar.com>]
Sent: Wednesday, December 02, 2015 3:49 PM
To: openstack-operators
Subject: Re: [Openstack-operators] DIB in container vs VM

Excerpts from Abel Lopez's message of 2015-12-01 16:16:08 -0800:
Hey everyone,
I've been running diskimage-builder just fine for over a year inside various 
VMs, but now, I'm having to run it inside a docker container.
I'm curious if anyone has experience with making the 'ubuntu latest' docker hub 
image more like a VM.

For example, I can `vagrant up` the trusty image from vagrant cloud, and every 
image I attempt to make "JUST WORKS"
When I try the same things with `docker run` things fail left right and center.

I figure there is some combination of packages that need to be installed, or 
services that need to be running, or mount points that need to exist to make 
this happen.

I can't really comment without seeing specifics, but I can say that
requirements _should_ be listed here:

http://docs.openstack.org/developer/diskimage-builder/user_guide/installation.html

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Service Catalog TNG urls

[Fox, Kevin M]

Yeah, switching them that way makes a lot of sense.

Thanks,
Kevin


From: Dan Sneddon
Sent: Thursday, December 03, 2015 12:39:25 PM
To: Fox, Kevin M; Jesse Keating; Sean Dague
Cc: openstack-operators
Subject: Re: [Openstack-operators] Service Catalog TNG urls

On 12/03/2015 09:46 AM, Fox, Kevin M wrote:
> We use internal to be a private network between the controllers and the
> compute nodes that no one else has access to. Without that, we'd be stuck.
>
> An OpenStack network that's where all the public services go, that
> isn't external to the cloud for billing purposes does make sense too
> though. Maybe all three should be clearly delineated. Maybe something like:
>
> * Public - Available outside. may incur costs to access internally
> * Internal - Available publicly by machines in the cloud. no cost is
> ever incurred from using them.
> * Provider - Not exposed to normal users and intended for backend sorts
> of things.
>
> Hopefully we can make it a strong enough convention that apps can rely
> on it between clouds.
>
> Thanks,
> Kevin
> ---
> *From:* Jesse Keating [j...@bluebox.net]
> *Sent:* Thursday, December 03, 2015 8:09 AM
> *To:* Sean Dague
> *Cc:* openstack-operators
> *Subject:* Re: [Openstack-operators] Service Catalog TNG urls
>
> We make use of http urls internally for services to talk to each other,
> but not for human users. All our human users should be using https
> public url. We don't actually utilize the internalURL framework,
> instead we use /etc/hosts entries to change the domain resolution of
> our publicURL entries, and use other config file controls to reflect
> http vs https.
>
> Partly this is because we don't want to advertise internal IP addresses
> in the catalog. Partly this is because we do not want to advertise an
> http link that a client might accidentally use and pass credentials in
> the clear over the Internet.
>
> I believe we would be perfectly happy to do away with adminURL and
> internalURL. It'd certainly reduce our per-site configuration entries,
> and reduce confusion around what purpose these entries serve.
>
>
> - jlk
>
> On Thu, Dec 3, 2015 at 6:14 AM, Sean Dague  <mailto:s...@dague.net>> wrote:
>
> For folks that don't know, we've got an effort under way to look at
> some
> of what's happened with the service catalog, how it's organically
> grown,
> and do some pruning and tuning to make sure it's going to support what
> we want to do with OpenStack for the next 5 years (wiki page to dive
> deeper here - https://wiki.openstack.org/wiki/ServiceCatalogTNG).
>
> One of the early Open Questions is about urls. Today there is a
> completely free form field to specify urls, and there are conventions
> about having publicURL, internalURL, adminURL. These are, however, only
> conventions.
>
> The only project that's ever really used adminURL has been Keystone, so
> that's something we feel we can phase out in new representations.
>
> The real question / concern is around public vs. internal. And
> something
> we'd love feedback from people on.
>
> When this was brought up in Tokyo the answer we got was that internal
> URL was important because:
>
> * users trusted it to mean "I won't get changed for bandwidth"
> * it is often http instead of https, which provides a 20% performance
> gain for transfering large amounts of data (i.e. glance images)
>
> The question is, how hard would it be for sites to be configured so
> that
> internal routing is used whenever possible? Or is this a concept we
> need
> to formalize and make user applications always need to make the
> decision
> about which interface they should access?
>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> <mailto:OpenStack-operators@lists.openstack.org>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

Kevin,

I'm not sure I'm on board with the three endpoints as listed, but at
the very least I would swap Internal and Provider in your example:

* Public - Available outside. may inc

Re: [Openstack-operators] Service Catalog TNG urls

[Fox, Kevin M]

We use internal to be a private network between the controllers and the compute 
nodes that no one else has access to. Without that, we'd be stuck.

An OpenStack network that's where all the public services go, that isn't 
external to the cloud for billing purposes does make sense too though. Maybe 
all three should be clearly delineated. Maybe something like:

* Public - Available outside. may incur costs to access internally
* Internal - Available publicly by machines in the cloud. no cost is ever 
incurred from using them.
* Provider - Not exposed to normal users and intended for backend sorts of 
things.

Hopefully we can make it a strong enough convention that apps can rely on it 
between clouds.

Thanks,
Kevin

From: Jesse Keating [j...@bluebox.net]
Sent: Thursday, December 03, 2015 8:09 AM
To: Sean Dague
Cc: openstack-operators
Subject: Re: [Openstack-operators] Service Catalog TNG urls

We make use of http urls internally for services to talk to each other, but not 
for human users. All our human users should be using https public url. We don't 
actually utilize the internalURL framework, instead we use /etc/hosts entries 
to change the domain resolution of our publicURL entries, and use other config 
file controls to reflect http vs https.

Partly this is because we don't want to advertise internal IP addresses in the 
catalog. Partly this is because we do not want to advertise an http link that a 
client might accidentally use and pass credentials in the clear over the 
Internet.

I believe we would be perfectly happy to do away with adminURL and internalURL. 
It'd certainly reduce our per-site configuration entries, and reduce confusion 
around what purpose these entries serve.


- jlk

On Thu, Dec 3, 2015 at 6:14 AM, Sean Dague 
mailto:s...@dague.net>> wrote:
For folks that don't know, we've got an effort under way to look at some
of what's happened with the service catalog, how it's organically grown,
and do some pruning and tuning to make sure it's going to support what
we want to do with OpenStack for the next 5 years (wiki page to dive
deeper here - https://wiki.openstack.org/wiki/ServiceCatalogTNG).

One of the early Open Questions is about urls. Today there is a
completely free form field to specify urls, and there are conventions
about having publicURL, internalURL, adminURL. These are, however, only
conventions.

The only project that's ever really used adminURL has been Keystone, so
that's something we feel we can phase out in new representations.

The real question / concern is around public vs. internal. And something
we'd love feedback from people on.

When this was brought up in Tokyo the answer we got was that internal
URL was important because:

* users trusted it to mean "I won't get changed for bandwidth"
* it is often http instead of https, which provides a 20% performance
gain for transfering large amounts of data (i.e. glance images)

The question is, how hard would it be for sites to be configured so that
internal routing is used whenever possible? Or is this a concept we need
to formalize and make user applications always need to make the decision
about which interface they should access?

-Sean

--
Sean Dague
http://dague.net

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] DIB in container vs VM

[Fox, Kevin M]

Did you try launching the docker container with the privilege flag? dib does a 
lot of privileged things last I looked.

Thanks,
Kevin

From: Clint Byrum [cl...@fewbar.com]
Sent: Wednesday, December 02, 2015 3:49 PM
To: openstack-operators
Subject: Re: [Openstack-operators] DIB in container vs VM

Excerpts from Abel Lopez's message of 2015-12-01 16:16:08 -0800:
> Hey everyone,
> I've been running diskimage-builder just fine for over a year inside various 
> VMs, but now, I'm having to run it inside a docker container.
> I'm curious if anyone has experience with making the 'ubuntu latest' docker 
> hub image more like a VM.
>
> For example, I can `vagrant up` the trusty image from vagrant cloud, and 
> every image I attempt to make "JUST WORKS"
> When I try the same things with `docker run` things fail left right and 
> center.
>
> I figure there is some combination of packages that need to be installed, or 
> services that need to be running, or mount points that need to exist to make 
> this happen.

I can't really comment without seeing specifics, but I can say that
requirements _should_ be listed here:

http://docs.openstack.org/developer/diskimage-builder/user_guide/installation.html

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] [stable][all] Keeping Juno "alive" for longer.

[Fox, Kevin M]

One thing more to consider, live upgrades still arn't a thing yet, but getting 
closer. Being able to do it with single version upgrades is a pretty hard 
thing. Doing it across LTS style releases wouldn't work without a huge amount 
of effort all on its own.

We may be better off waiting until we get a core OpenStack release that allows 
live upgrades to work smoothly before calling that thing the first LTS release.

Thanks,
Kevin

From: Tom Cameron [tom.came...@rackspace.com]
Sent: Monday, November 09, 2015 11:01 AM
To: Jeremy Stanley; openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [openstack-dev] [stable][all] Keeping Juno 
"alive" for longer.

>From your other thread...

>Or else you're saying you intend to fix the current inability of our projects 
>to skip intermediate releases entirely during upgrades

I think without knowing it, that's what most would be suggesting, yeah. Of 
course, like you mentioned, the real work is in how upgrades get refactored to 
skip intermediate releases (two or three of them).

DB schema changes can basically be rolled up and kept around for a while, so 
that's not too be a problem. Config files OTOH have no schema or schema 
validator, so that would require tooling and all kinds of fun (bug prone) 
wizardry.

This is all solvable, but it adds complexity for the sake of what I can only 
imagine are the extreme minority of users. What do the user/operator surveys 
say about the usage of older releases? What portion of the user base is 
actually on releases prior to Havana?

--
Tom Cameron



From: Jeremy Stanley 
Sent: Monday, November 9, 2015 12:35
To: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [openstack-dev] [stable][all] Keeping Juno 
"alive" for longer.

On 2015-11-09 17:11:35 + (+), Tom Cameron wrote:
[...]
> I support an LTS release strategy because it will allow more
> adoption for more sectors by offering that stability everyone's
> talking about. But, it shouldn't be a super-super long support
> offering. Maybe steal some of Ubuntu's game and do an LTS every 4
> releases or so (24 months), but then maybe Openstack only supports
> them for 24 months time? Again, my concern is that this is free,
> open source software and you're probably not going to get many
> community members to volunteer to offer their precious time fixing
> bugs in a 2-year-old codebase that have been fixed for 18 months
> in a newer version.
[...]

Because we want people to be able upgrade their deployments, the
problem runs deeper than just backporting some fixes to a particular
branch for longer periods of time. Unfortunately the original poster
cross-posted this thread to multiple mailing lists so the discussion
has rapidly bifurcated, but I addressed this particular topic in my
http://lists.openstack.org/pipermail/openstack-dev/2015-November/078735.html
reply.
--
Jeremy Stanley

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] [stable][all] Keeping Juno "alive" for longer.

[Fox, Kevin M]

Kind of related, as an op, we see a lot of 3rd party repositories that recently 
only supported rhel5 move to finally supporting rhel6 because rhel7 came out 
and rhel5 went to long term support contract only. This caused us to have to 
support rhel5 way longer then we would have liked. Now, we're stuck at 6 
instead of 7. :/

Some number of users will stick with juno until it is EOL and then move. 
Sometimes its because its a desire to not make a change. Sometimes its 
considered a good thing by the ops that they finally have a "good enough" 
excuse (EOL) to move forward "finally" (sigh of relief). :)

Thanks,
Kevin


From: Jesse Keating [j...@bluebox.net]
Sent: Friday, November 06, 2015 10:14 AM
To: Dan Smith
Cc: OpenStack Development Mailing List (not for usage questions); 
openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [openstack-dev] [stable][all] Keeping Juno 
"alive" for longer.

We (Blue Box, an IBM company) do have a lot of installs on Juno, however we'll 
be aggressively moving to Kilo, so we are not interested in keeping Juno alive.


- jlk

On Fri, Nov 6, 2015 at 9:37 AM, Dan Smith 
mailto:d...@danplanet.com>> wrote:
> Worth mentioning that OpenStack releases that come out at the same time
> as Ubuntu LTS releases (12.04 + Essex, 14.04 + Icehouse, 16.04 + Mitaka)
> are supported for 5 years by Canonical so are already kind of an LTS.
> Support in this context means patches, updates and commercial support
> (for a fee).
> For paying customers 3 years of patches, updates and commercial support
> for April releases, (Kilo, O, Q etc..) is also available.

Yeah. IMHO, this is what you pay your vendor for. I don't think upstream
maintaining an older release for so long is a good use of people or CI
resources, especially given how hard it can be for us to keep even
recent stable releases working and maintained.

--Dan

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Tokyo Ops Design Summit Tracks - Agenda on Sched

[Fox, Kevin M]

+1


From: Christopher Aedo
Sent: Monday, October 12, 2015 3:51:23 PM
To: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Tokyo Ops Design Summit Tracks - Agenda on 
Sched

On Sun, Oct 11, 2015 at 8:55 PM, Tom Fifield  wrote:
> Hi all,
>
> We are still lacking:
>
> * Moderator(s) for Quotas and Billing
> * Content for about 3 slots.
>
> If you have interest for any of this, please speak now - time is running
> out!

I'm very interested in having an operator focused discussion about
delivering apps to your users, and whether or not that matters to the
cloud operators out there.  Apps on OpenStack is something near and
dear to my heart (working on apps.openstack.org), is it important to
the operators of clouds though?

I know a few of the bigger public cloud providers have worked on their
own approaches to this (using Heat templates, Murano apps, or some
other PaaS layers).  I think there's a big benefit to a common catalog
of apps that everyone can use, but maybe the big operators want to
exert more control by keeping an independent catalog?  Or maybe there
are other reasons?

Please speak up if you're interested in this topic, I'd be very happy
to moderate, and I think if we can get some operators talking together
about this there would be a lot of value in the conversation!

-Christopher

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] what is the different in use Qcow2 or Raw in Ceph

[Fox, Kevin M]

Awesome. Thanks for the link. :)

Kevin

From: jonathan.pro...@gmail.com [jonathan.pro...@gmail.com] on behalf of 
Jonathan Proulx [j...@jonproulx.com]
Sent: Thursday, May 28, 2015 12:59 PM
To: Fox, Kevin M
Cc: Dmitry Borodaenko; David Medberry; openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] what is the different in use Qcow2 or Raw in 
Ceph

On Thu, May 28, 2015 at 3:21 PM, Fox, Kevin M  wrote:
> I've experienced the opposite problem though. Downloading raw images and 
> uploading them to the cloud is very slow. Doing it through qcow2 allows them 
> to be compressed over the slow links. Ideally, the Ceph driver would take a 
> qcow2 and convert it to raw on glance ingest rather then at boot.

There is a blueprint for this which if I read correctly has landed in
kilo (I'm still on juno and haven't started kilo testing yet):

https://blueprints.launchpad.net/glance/+spec/basic-import-conversion

https://review.openstack.org/#/c/159129/

I hope that it's landed because I too am really looking forward to that feature

-Jon

> Thanks,
> Kevin
> 
> From: Dmitry Borodaenko [dborodae...@mirantis.com]
> Sent: Thursday, May 28, 2015 12:10 PM
> To: David Medberry
> Cc: openstack-operators@lists.openstack.org
> Subject: Re: [Openstack-operators] what is the different in use Qcow2 or Raw 
> in Ceph
>
> David is right, Ceph implements volume snapshotting at the RBD level,
> not even RADOS level: whole 2 levels of abstraction above file system.
> It doesn't matter if it's XFS, BtrFS, Ext4, or VFAT (if Ceph supported
> VFAT): Ceph RBD takes care of it before individual chunks of an RBD
> volume are passed to RADOS as objects and get written into the file
> system as files by an OSD process.
>
> The reason Fuel documentation recommends to disable QCOW2 format for
> images is that RBD does not support QCOW2 disks directly, so Nova and
> Cinder have to _convert_ a QCOW2 image into RAW format before passing
> it to QEMU's RBD driver. This means that you end up downloading the
> QCOW2 image from Ceph to a nova-compute node (first full copy),
> converting it (second full copy), and uploading the resultant RAW
> image back to Ceph (third full copy) just to launch a VM or create a
> volume from an image.
>
>
> On Thu, May 28, 2015 at 8:33 AM, David Medberry  
> wrote:
>> yep. It's at the CEPH level (not the XFS level.)
>>
>> On Thu, May 28, 2015 at 8:40 AM, Stephen Cousins 
>> wrote:
>>>
>>> Hi David,
>>>
>>> So Ceph will use Copy-on-write even with XFS?
>>>
>>> Thanks,
>>>
>>> Steve
>>>
>>> On Thu, May 28, 2015 at 10:36 AM, David Medberry 
>>> wrote:
>>>>
>>>> This isn't remotely related to btrfs. It works fine with XFS. Not sure
>>>> how that works in Fuel, never used it.
>>>>
>>>> On Thu, May 28, 2015 at 8:01 AM, Forrest Flagg 
>>>> wrote:
>>>>>
>>>>> I'm also curious about this.  Here are some other pieces of information
>>>>> relevant to the discussion.  Maybe someone here can clear this up for me 
>>>>> as
>>>>> well.  The documentation for Fuel 6.0, not sure what they changed for 6.1,
>>>>> [1] states that when using Ceph one should disable qcow2 so that images 
>>>>> are
>>>>> stored in raw format.  This is due to the fact that Ceph includes its own
>>>>> mechanisms for copy-on-write and snapshots.  According to the Ceph
>>>>> documentation [2], this is true only when using a BTRFS file system, but 
>>>>> in
>>>>> Fuel 6.0 Ceph uses XFS which doesn't provide this functionality.  Also, 
>>>>> [2]
>>>>> recommends not using BTRFS for production as it isn't considered fully
>>>>> mature.  In addition, Fuel 6.0 [3] states that OpenStack with raw images
>>>>> doesn't support snapshotting.
>>>>>
>>>>> Given this, why does Fuel suggest not using qcow2 with Ceph?  How can
>>>>> Ceph be useful if snapshotting isn't an option with raw images and qcow2
>>>>> isn't recommended?  Are there other factors to take into consideration 
>>>>> that
>>>>> I'm missing?
>>>>>
>>>>> [1]
>>>>> https://docs.mirantis.com/openstack/fuel/fuel-6.0/terminology.html#qcow2
>>>>> [2]
>>>>> http://ceph.com/docs/master/rados/configuration/filesystem-recommendations/
>>>>>

Re: [Openstack-operators] Dynamic Policy

[Fox, Kevin M]

As an Op, I've ran into this problem and keep running into it. I would very 
much like a solution.

Its also quite related to the nova instance user issue I've been working on, 
that's needed by the App Catalog project.

So, yes, please keep fighting the good fight.

Thanks,
Kevin

From: Adam Young [ayo...@redhat.com]
Sent: Wednesday, August 05, 2015 7:50 AM
To: openstack-operators@lists.openstack.org
Subject: [Openstack-operators] Dynamic Policy

How do you delegate the ability to delegate?

Lets say you are running a large cloud (purely hypothetical here) and
you want to let a user manage their own project.  They are "admin" but
they should be able to invite or eject people.

In order to do this, an ordinary user needs to be able to make a role
assignment.  However, Keystone does not support this today:  if you are
admin somewhere, you are admin everywhere:

https://bugs.launchpad.net/keystone/+bug/968696

Access control in OpenStack is controlled by Policy.  An informal survey
of operators shows that most people run with the stock policies such as
the Nova policy:

http://git.openstack.org/cgit/openstack/nova/tree/etc/nova/policy.json

In order to scope admin to the proejct, we would need to have rules that
enforce this scoping:  Evey rule should check that the project_id in the
token provided matches the  project_id of the resource of the API.

If we manage to get the policy files rewritten this way, We then need a
way to limit what roles a user can assign.The default mechanism
would say that a user needs to have an administrative role on the
project (or domain) that they want to assign the role on.

I don't think anything I've written thus far is controversial. Then, why
has it not happened yet? here are the list of problems we need to solve:

1. Operators expect the existing policy files to work as is. Changing
them will break workflow.
2. If everything is scoped, we need a way to delete resources left
orphan when a project is deleted from Keystone and the service does not
get the notification (or know how to handle it).

Of the two, I think the top one is the more difficult to solve. Scoping
everything can be handled via one of two mechanism;  either allow a
power-admin user to get a token scoped to some random project (even if
it has been deleted) or allow a token scoped to an administrative
project to also delete resources for a random project.

Dealing with the existing policy file issues is trickier, and that is
the real reason for the Dynamic Policy  approach:  give the endpoints
the ability to fetch their policy files from Keystone.  If policy goes
from being a configuration file to data, it is managed outside of the
configuration management tools, and becomes much more fluid. This has
risks, and should be an Opt-In mechanism.

Fetching the policy files from Keystone also provides the start of a
richer set of management for policy rules.  Currently, each of the stock
policy files exists only in their seperate git repos.  There is no
sharing of policy rules across projects.  If the policy files were
managed, rule by rule, from a centralized repository,  rules could be
shared, providing, among other things, the ability to enforce
hierarchical roles, roles namespaced to a service, and other, richer
policy management.

Feedback on this approach from operators is greatly appreciated.  I need
to justify the effort that would go in to making this happen, so if you
want it, speak up.

If, on the other hand, you feel that this is needlessly complicated or
would make deployments more difficult, that is important too, and please
let me know.

Finally, if you can see some alternative methods of implementing a more
dynamic access control, please chime in.





___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [kilo-muktinode] [ubuntu cloud images]

[Fox, Kevin M]

You can find ubuntu images in the app catalog. For example:
http://apps.openstack.org/#tab=glance-images&asset=Ubuntu%20Trusty%2014

Thanks,
Kevin

From: Abhishek Talwar [abhishek.tal...@tcs.com]
Sent: Friday, July 31, 2015 4:05 AM
To: openstack-operators
Subject: [Openstack-operators] [kilo-muktinode] [ubuntu cloud images]


Hi,


I have a multinode kilo OpenStack setup with a controller node, network node 
and 2 compute nodes. I have installed a cirros image with the following steps:

  1.  Create a temporary local directory:

$ mkdir /tmp/images

  1.  Download the source image into it:

$ wget -P /tmp/images 
http://download.cirros-cloud.net/0.3

  1.  Upload the image to the Image service using the QCOW2 disk format, bare 
container format, and public visibility so all projects can access it:

$ glance image-create --name "cirros-0.3.4-x86_64" --file 
/tmp/images/cirros-0.3.4-x86_64-disk.img \ --disk-format qcow2 
--container-format bare --visibility public --progress

Now I want to install an Ubuntu image on my setup and boot VM's with it. So 
what would be the desired steps for that.

I have checked this link 
"http://docs.openstack.org/image-guide/content/ubuntu-image.html"; but couldnt 
get through.

=-=-=
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] adding postinstall scripts in cloud init

[Fox, Kevin M]

Vendor data can do that. See the json metadata plugin to the nova metadata 
server and the vendor data section of cloud init.

Thanks,
Kevin


From: Kris G. Lindgren
Sent: Thursday, July 23, 2015 8:38:33 AM
To: Openstack Guru; openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] adding postinstall scripts in cloud init

Do you mean outside of the standard supplying user_data when the VM boots?  Or 
do you mean that you (as the cloud provider) want every vm to always do x,y,z 
and to leave user_data open to your end users?


Kris Lindgren
Senior Linux Systems Engineer
GoDaddy, LLC.


From: Openstack Guru mailto:openstackg...@gmail.com>>
Date: Thursday, July 23, 2015 at 8:57 AM
To: 
"openstack-operators@lists.openstack.org"
 
mailto:openstack-operators@lists.openstack.org>>
Subject: [Openstack-operators] adding postinstall scripts in cloud init

Hello every one,

I would like to add some command's in cloud init to execute while building an 
instance. Please advice best way to accomplish this?

Thanks in advance.


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Nova cells v2 and operational impacts

[Fox, Kevin M]

Sounds like a good plan to me.

Thanks,
Kevin


From: David Medberry
Sent: Tuesday, July 21, 2015 7:51:50 AM
To: Michael Still
Cc: openstack-operators@lists.openstack.org; Andrew Laski
Subject: Re: [Openstack-operators] Nova cells v2 and operational impacts

Also, if there is feedback, getting it in today or tomorrow would be most 
effective.

Michael, this plan works for me/us. TWC. -d

On Tue, Jul 21, 2015 at 9:45 AM, Michael Still 
mailto:mi...@stillhq.com>> wrote:
Heya,

the nova developer mid-cycle meetup is happening this week. We've been
talking through the operational impacts of cells v2, and thought it
would be a good idea to mention them here and get your thoughts.

First off, what is cells v2? The plan is that _every_ nova deployment
will be running a new version of cells. The default will be a
deployment of a single cell, which will have the impact that existing
single cell deployments will end up having another mysql database that
is required by cells. However, you wont be required to bring up any
additional nova services at this point [1], as cells v2 lives inside
the nova-api service.

The advantage of this approach is that cells stops being a weird
special case run by big deployments. We're forced to implement
everything in cells, instead of the bits that a couple of bigger
players cared enough about, and we're also forced to test it better.
It also means that smaller deployments can grow into big deployments
much more easily. Finally, it also simplifies the nova code, which
will reduce our tech debt.

This is a large block of work, so cells v2 wont be fully complete in
Liberty. Cells v1 deployments will effective run both cells v2 and
cells v1 for this release, with the cells v2 code thinking that there
is a single very large cell. We'll continue the transition for cells
v1 deployments to pure cells v2 in the M release.

So what's the actual question? We're introducing an additional mysql
database that every nova deployment will need to possess in Liberty.
We talked through having this data be in the existing database, but
that wasn't a plan that made us comfortable for various reasons. This
means that operators would need to do two db_syncs instead of one
during upgrades. We worry that this will be annoying to single cell
deployments.

We therefore propose the following:

 - all operators when they hit Liberty will need to add a new
connection string to their nova.conf which configures this new mysql
database, there will be a release note to remind you to do this.
 - we will add a flag which indicates if a db_sync should imply a sync
of the cells database as well. The default for this flag will be true.

This means that you can still do these syncs separately if you want,
but we're not forcing you to remember to do it if you just want it to
always happen at the same time.

Does this sound acceptable? Or are we over thinking this? We'd
appreciate your thoughts.

Cheers,
Michael

1: there is some talk about having a separate pool of conductors to
handle the cells database, but this wont be implemented in Liberty.

--
Rackspace Australia

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-operators] Onboarding Legacy Apps into OpenStack

[Fox, Kevin M]

The nova instance user workflow could be used for that?
https://review.openstack.org/#/c/186617/

A template could start the vm, register the instance user id with the IdP, and 
then the instance can call the IdP to register.

Thanks,
Kevin

From: Adam Young [ayo...@redhat.com]
Sent: Monday, June 22, 2015 11:36 AM
To: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [openstack-operators] Onboarding Legacy Apps 
into OpenStack

On 06/15/2015 07:46 PM, Barrett, Carol L wrote:
Operators – The Enterprise Work Group (formerly known as Win The Enterprise) 
has a team working on a Proof Of Concept for using Metadata to describe 
requirements for legacy apps and workloads to be on boarded onto an OpenStack 
Cloud.

We have 2 use cases that we are planning to implement: Encrypted Storage and 
Workload Isolation.
You can find these use cases here:

I am asking for your help:

  1.  We’re looking for an Operator who has a real world example of either of 
these use cases and can share information about config and overall 
requirements. We want to make sure our PoC is realistic.
  2.  We need more use cases to run through our PoC. Do you have Legacy App or 
Workload that you can work with us to write up a use case around?



If you’re game for either of these please let me know.
Thanks
Carol Barrett




___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


I'm not an operator, but I play one on TV.  My examples are not real world 
except as a developer using OpenStack.

I want to be able to launch a VM and have it automatically register with an 
Identity management service.  To do this, I need to generate a one time 
password that gets passed to both the IdM and to the VM, and User-Data  seems 
to be the only tool.  However, I would ideally have something that perform this 
workflow, regardless of how the user kicked off the task, and that would not 
require the user to modify the user-data when launching the VM;  it would be a 
property of the project instead.


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] [nova] [neutron] Re: How do your end users use networking?

[Fox, Kevin M]

The biggest issue we have run into with multiple public networks is restricting 
which users can use which networks. We have the same issue, where we may have 
an internal public network for the datacenter, but also, say, a DMZ network we 
want to put some vm's on, but can't currently extend that network easily there 
because too many tenants will be able to launch vm's attached to the DMZ that 
don't have authorization. Quota's or acls or something on public networks are 
really needed.

Thanks,
Kevin

From: Neil Jerram [neil.jer...@metaswitch.com]
Sent: Wednesday, June 17, 2015 4:28 AM
To: Jay Pipes; openstack-operators@lists.openstack.org; 
openstack-...@lists.openstack.org; Kyle Mestery
Subject: Re: [openstack-dev] [Openstack-operators] [nova] [neutron] Re: How do 
your end users use networking?

Couple more dumb comments here - sorry that I'm processing this thread
backwards!

On 16/06/15 15:20, Jay Pipes wrote:
> Adding -dev because of the reference to the Neutron "Get me a network
> spec". Also adding [nova] and [neutron] subject markers.
>
> Comments inline, Kris.
>
> On 05/22/2015 09:28 PM, Kris G. Lindgren wrote:
>> During the Openstack summit this week I got to talk to a number of other
>> operators of large Openstack deployments about how they do networking.
>>   I was happy, surprised even, to find that a number of us are using a
>> similar type of networking strategy.  That we have similar challenges
>> around networking and are solving it in our own but very similar way.
>>   It is always nice to see that other people are doing the same things
>> as you or see the same issues as you are and that "you are not crazy".
>> So in that vein, I wanted to reach out to the rest of the Ops Community
>> and ask one pretty simple question.
>>
>> Would it be accurate to say that most of your end users want almost
>> nothing to do with the network?
>
> That was my experience at AT&T, yes. The vast majority of end users
> could not care less about networking, as long as the connectivity was
> reliable, performed well, and they could connect to the Internet (and
> have others connect from the Internet to their VMs) when needed.
>
>> In my experience what the majority of them (both internal and external)
>> want is to consume from Openstack a compute resource, a property of
>> which is it that resource has an IP address.  They, at most, care about
>> which "network" they are on.  Where a "network" is usually an arbitrary
>> definition around a set of real networks, that are constrained to a
>> location, in which the company has attached some sort of policy.  For
>> example, I want to be in the production network vs's the xyz lab
>> network, vs's the backup network, vs's the corp network.  I would say
>> for Godaddy, 99% of our use cases would be defined as: I want a compute
>> resource in the production network zone, or I want a compute resource in
>> this other network zone.

Kris - this looks like the answer to my question why you define multiple
networks.  If that's right, no need to answer that question there.

>>  The end user only cares that the IP the vm
>> receives works in that zone, outside of that they don't care any other
>> property of that IP.  They do not care what subnet it is in, what vlan
>> it is on, what switch it is attached to, what router its attached to, or
>> how data flows in/out of that network.  It just needs to work.

Agreed.  I'm not a deployer, but my team is in contact with many
deployers who say similar things.

Regards,
Neil

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Gentoo image availability

[Fox, Kevin M]

Awesome. Are they ready enough that they should go into the app catalog? 
(http://apps.openstack.org)

Thanks,
Kevin

From: Matthew Thode [prometheanf...@gentoo.org]
Sent: Monday, June 08, 2015 8:26 AM
To: openstack-operators@lists.openstack.org
Subject: [Openstack-operators] Gentoo image availability

Hi,

I'm the packager of Openstack on Gentoo and have just started generation
of Gentoo Openstack images.  Right now it is just a basic amd64 image,
but I plan on adding nomultilib and hardened variants (for a total of at
least 4 images).  I plan on generating these images at least weekly

These images are not yet sanctioned by our infra team, but I plan on
remedying that (being a member of said team should help).

I am currently using the scripts at
https://github.com/prometheanfire/gentoo-cloud-prep to generate the
images (based on a heavily modified version of Matt Vandermeulen's
scripts).  If you have any issues please submit bugs there or contact me
on irc (prometheanfire on freenode).

Here's the link to the images, I'm currently gpg signing them with the
same key I use to sign this email (offline master key smartcard setup
for security minded folk).

http://23.253.251.73/

Let me know if you have questions,

--
Matthew Thode (prometheanfire)


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [Tags] Tags Team Repo & our first tags!

[Fox, Kevin M]

Hi Tom,

Yeah, i've played with the trove packaging in particular and have experienced 
what you were talking about. Some software is green, some packages are green, 
and some things are just plain buggy in the OpenStack bigger tent.

Level of effort is a concern for sure.

Knowing if its in my distro of choice does save me work. Telling if a package 
is buggy or not is much harder to maintain since the packager can release many 
new versions frequently and if you arent keeping track some how, you wont know 
if its fixed or not, possibly turning folks away when there isnt a problem any 
more.

Maybe a compromise for level of effort is to just allow users to give it a 
number of stars on a website, and age them off every month or so? Or keep track 
of package version in the package repo and expire the stars when a new package 
comes out? That could be automated.

Thanks,
Kevin


From: Tom Fifield
Sent: Sunday, June 07, 2015 6:54:42 PM
To: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [Tags] Tags Team Repo & our first tags!

On 06/06/15 03:12, Fox, Kevin M wrote:
> With my op hat on, I'd very much prefer packaged-in-ubuntu/packaged-in-centos 
> or packaged=ubuntu,centos. If its just packaged=True, I'd still have to go 
> look up if its in my distro of choice.


Kevin,

Many thanks for the input.

May I ask, would you* be ok with a structurified version of this:

"Trove packaging: generally looks good as best we can tell. Though, it
seems like they screwed it up in Ubuntu this time and here's a link to
https://bugs.launchpad.net/ubuntu/+source/openstack-trove/+bug/1308523";

?

Then, from there, assuming you're using centos for the sake of example,
maybe one of two things would happen:

1) You'd use it, confident that at least someone had had a quick look
and said "yeah, looks OK at first glance", and it ends up working
2) You're the first person to discover this massive bug with centos
packaging, and there's some easy way to update the tag to say:

"Trove packaging is: pretty average, after some experience. There are
problems in multiple distros
(https://bugs.launchpad.net/ubuntu/+source/openstack-trove/+bug/1308523
, http://bugs.centos.org/view.php?id=";


Reason I'm asking - I think we can probably get this kind of system
going with fairly minimal effort, but I think going through every
distro(+distroversion) for every project is probably beyond our
volunteer numbers at the moment.


Regards,


Tom


* perhaps assuming that you're a new user considering adopting Trove?

> Thanks,
> Kevin
> 
> From: Jeremy Stanley [fu...@yuggoth.org]
> Sent: Friday, June 05, 2015 11:42 AM
> To: openstack-operators@lists.openstack.org
> Subject: Re: [Openstack-operators] [Tags] Tags Team Repo & our first tags!
>
> On 2015-06-05 18:34:02 + (+), Tim Bell wrote:
>> But if there is one package out of all of the OS options, does
>> that make true or false ? Or do we have a rule that says a 1 means
>> that at least CentOS and Ubuntu are packaged ?
>>
>> I remain to be convinced that a 0 or 1 can be achieved within the
>> constraints that we need something which is useful for the
>> operators rather than mathematically correct.
>>
>> Let’s not forget the target audience for the Ops tags.
>
> This doesn't seem orthogonal to the idea of tags as boolean flags.
> Is there any reason not to have separate tags like
> packaged-in-ubuntu and packaged-in-centos? Of course there's still
> some ambiguity over details like "how new is the packaged version?"
> and "in what release(s) of the distro?" but those are probably
> things you could build tag-qualifying rules around.
> --
> Jeremy Stanley
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [Tags] Tags Team Repo & our first tags!

[Fox, Kevin M]

With my op hat on, I'd very much prefer packaged-in-ubuntu/packaged-in-centos 
or packaged=ubuntu,centos. If its just packaged=True, I'd still have to go look 
up if its in my distro of choice.

Thanks,
Kevin

From: Jeremy Stanley [fu...@yuggoth.org]
Sent: Friday, June 05, 2015 11:42 AM
To: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [Tags] Tags Team Repo & our first tags!

On 2015-06-05 18:34:02 + (+), Tim Bell wrote:
> But if there is one package out of all of the OS options, does
> that make true or false ? Or do we have a rule that says a 1 means
> that at least CentOS and Ubuntu are packaged ?
>
> I remain to be convinced that a 0 or 1 can be achieved within the
> constraints that we need something which is useful for the
> operators rather than mathematically correct.
>
> Let’s not forget the target audience for the Ops tags.

This doesn't seem orthogonal to the idea of tags as boolean flags.
Is there any reason not to have separate tags like
packaged-in-ubuntu and packaged-in-centos? Of course there's still
some ambiguity over details like "how new is the packaged version?"
and "in what release(s) of the distro?" but those are probably
things you could build tag-qualifying rules around.
--
Jeremy Stanley

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[Openstack-operators] [neutron] DVR & other drivers

[Fox, Kevin M]

Does DVR work with other drivers such as the Mellenox or Cisco drivers? What 
about with anything SRIOV?

Thanks,
Kevin
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Help with multiple external network in openstack

[Fox, Kevin M]

Since you are passing the tagged physical network device eth1.803 into the 
bridge, I think you need to use a flat network in the config/ external network 
create command. Otherwise it may do nested vlan tags.

Thanks,
Kevin


From: Geo Varghese
Sent: Friday, June 05, 2015 5:38:33 AM
To: Miguel A Diaz Corchero
Cc: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Help with multiple external network in 
openstack

Hi Miguel,

I am adding my content of /etc/neutron/plugins/ml2/ml2_conf.ini of controller

[ml2]
type_drivers = gre,vlan
tenant_network_types = gre,vlan
mechanism_drivers = openvswitch

[ml2_type_flat]
flat_networks =

[ml2_type_vlan]
network_vlan_ranges = physnet1:803:803,physnet2:805:805

[ml2_type_gre]
tunnel_id_ranges = 1:1000

[ml2_type_vxlan]
vni_ranges =
vxlan_group =

[securitygroup]
enable_security_group = True
enable_ipset = True
firewall_driver = 
neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver

[ovs]
local_ip = 192.168.100.74
tunnel_type = gre
enable_tunneling = True
bridge_mappings = physnet1:br-ex803,physnet2:br-ex805

The content of /etc/neutron/l3_agent.ini of controller

[DEFAULT]
debug = True
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
use_namespaces = True
gateway_external_network_id =
handle_internal_only_routers = True
external_network_bridge =
metadata_port = 9697
send_arp_for_ha = 3
periodic_interval = 40
periodic_fuzzy_delay = 5
router_delete_namespaces = False


Content of ovs-vsctl show  after creating the bridges

root@Node25:/home/geo# ovs-vsctl show
cd31399c-bdb7-4b79-9d51-acbec1a41619
Bridge br-int
fail_mode: secure
Port int-br-ex
Interface int-br-ex
Port "qr-221f276d-81"
tag: 1
Interface "qr-221f276d-81"
type: internal
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port "int-br-ex805"
Interface "int-br-ex805"
Port "int-br-ex803"
Interface "int-br-ex803"
Port "qg-d3568030-c3"
tag: 2
Interface "qg-d3568030-c3"
type: internal
Port "tapd7bacca2-15"
tag: 1
Interface "tapd7bacca2-15"
type: internal
Port br-int
Interface br-int
type: internal
Bridge "br-ex803"
Port "eth1.803"
Interface "eth1.803"
Port "phy-br-ex803"
Interface "phy-br-ex803"
Port "br-ex803"
Interface "br-ex803"
type: internal
Bridge br-tun
Port "gre-c0a864fe"
Interface "gre-c0a864fe"
type: gre
options: {in_key=flow, local_ip="192.168.100.74", out_key=flow, 
remote_ip="192.168.100.254"}
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port br-tun
Interface br-tun
type: internal
Bridge "br-ex805"
Port "phy-br-ex805"
Interface "phy-br-ex805"
Port "br-ex805"
Interface "br-ex805"
type: internal
Port "eth1.805"
Interface "eth1.805"
ovs_version: "2.0.2"


External network added by following commands

External network 1 =>

 neutron net-create ext-net1 --shared --router:external=True 
--provider:physical_network physnet1 --provider:network_type vlan 
--provider:segmentation_id 803

neutron subnet-create ext-net1 --name ext-subnet1 --allocation-pool 
start=192.168.200.150,end=192.168.200.160 --disable-dhcp 
192.168.200.0/24

External network 2 =>
neutron net-create ext-net2 --shared --router:external=True 
--provider:physical_network physnet2 --provider:network_type vlan 
--provider:segmentation_id 805

neutron subnet-create ext-net2 --name ext-subnet1 --allocation-pool 
start=192.168.123.150,end=192.168.123.160 --disable-dhcp 
192.168.123.0/24

Please update if you need any other details.

Thanks again for your time.


On Fri, Jun 5, 2015 at 5:05 PM, Geo Varghese 
mailto:gvargh...@aqorn.com>> wrote:
Hi Miguel,

Thanks for getting back to me.

I have already restarted both machines - controller and compute

Do i need to paste any commands or conf to debug it?


On Fri, Jun 5, 2015 at 4:55 PM, Miguel A Diaz Corchero 
mailto:miguelangel.d...@externos.ciemat.es>>
 wrote:
El 05/06/15 13:07, Geo Varghese escribió:
I tried it but somethig missing still, ping to floating IP seems not working
please, try to restart the L3 agent linked with that floating IP and let us know

do we have to add any changes to compute node  for it?
it shouldn't be necessary

Miguel.



On Fri, Jun 5, 2015 at 12:16 PM, Geo Varghese 
mailto:gvargh...@aqorn.com>> wrote:
Miguel,

Thanks thats a great link. Let me try it.

On Fri, Jun 5, 2015 at 12:04 

Re: [Openstack-operators] Help with multiple external network in openstack

[Fox, Kevin M]

Those are the 4 external networks. In this cloud, they are all linux bridges.

I'm not using vlan tagging on this cloud, so I'm not sure what that would look 
like.

Thanks,
Kevin

From: Geo Varghese [gvargh...@aqorn.com]
Sent: Thursday, June 04, 2015 1:02 PM
To: Fox, Kevin M
Cc: openstack-operators@lists.openstack.org; openst...@lists.openstack.org
Subject: Re: Help with multiple external network in openstack

Kevin,

Thanks. Can you please explain these values

pub:br-pub,scz:br-scz,osg:br-osg,mgmt:br-mgmt

These 4 networks are external networks? How you created these bridges.

Can you please specify the value added for

network_vlan_ranges =

Are you using vlan tag fro external network.

Sorry for many questions :)





-- Forwarded message ------
From: Fox, Kevin M mailto:kevin@pnnl.gov>>
Date: Fri, Jun 5, 2015 at 1:24 AM
Subject: RE: Help with multiple external network in openstack
To: Geo Varghese mailto:gvargh...@aqorn.com>>
Cc: 
"openstack-operators@lists.openstack.org<mailto:openstack-operators@lists.openstack.org>"
 
mailto:openstack-operators@lists.openstack.org>>,
 "openst...@lists.openstack.org<mailto:openst...@lists.openstack.org>" 
mailto:openst...@lists.openstack.org>>


In /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini currently we have:
bridge_mappings = pub:br-pub,scz:br-scz,osg:br-osg,mgmt:br-mgmt

Thanks,
Kevin

From: Geo Varghese [gvargh...@aqorn.com<mailto:gvargh...@aqorn.com>]
Sent: Thursday, June 04, 2015 12:29 PM
To: Fox, Kevin M
Cc: 
openstack-operators@lists.openstack.org<mailto:openstack-operators@lists.openstack.org>;
 openst...@lists.openstack.org<mailto:openst...@lists.openstack.org>
Subject: Re: Help with multiple external network in openstack

Thanks for the reply Kevin.

Currently bridge mapping is empty string.

As I am not creating br-ex bridge due to multiple external network. Can you 
please explain what i have to do.

On Thursday, June 4, 2015, Fox, Kevin M 
mailto:kevin@pnnl.gov>> wrote:
Bridge mappings set in plugin.ini?

Thanks,
Kevin


From: Geo Varghese
Sent: Thursday, June 04, 2015 6:25:46 AM
To: openstack-operators@lists.openstack.org; openst...@lists.openstack.org
Subject: [Openstack-operators] Help with multiple external network in openstack

Hi Team,

I need some help to setup multiple external network

In normal single external network we create br-ex bridge and add it in

/etc/neutron/l3_agent.ini

As

external_network_bridge = br-ex

It is working for me.


But in the case of multiple external network, this variable to be set to empty 
according to the docs. I did that but seems working.

Any one please specify whta other changes i have to do to make it working.

Thanks for your support guys.


--
Regards,
Geo Varghese



--
--
Regards,
Geo Varghese
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Help with multiple external network in openstack

[Fox, Kevin M]

In /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini currently we have:
bridge_mappings = pub:br-pub,scz:br-scz,osg:br-osg,mgmt:br-mgmt

Thanks,
Kevin

From: Geo Varghese [gvargh...@aqorn.com]
Sent: Thursday, June 04, 2015 12:29 PM
To: Fox, Kevin M
Cc: openstack-operators@lists.openstack.org; openst...@lists.openstack.org
Subject: Re: Help with multiple external network in openstack

Thanks for the reply Kevin.

Currently bridge mapping is empty string.

As I am not creating br-ex bridge due to multiple external network. Can you 
please explain what i have to do.

On Thursday, June 4, 2015, Fox, Kevin M 
mailto:kevin@pnnl.gov>> wrote:
Bridge mappings set in plugin.ini?

Thanks,
Kevin


From: Geo Varghese
Sent: Thursday, June 04, 2015 6:25:46 AM
To: openstack-operators@lists.openstack.org; openst...@lists.openstack.org
Subject: [Openstack-operators] Help with multiple external network in openstack

Hi Team,

I need some help to setup multiple external network

In normal single external network we create br-ex bridge and add it in

/etc/neutron/l3_agent.ini

As

external_network_bridge = br-ex

It is working for me.


But in the case of multiple external network, this variable to be set to empty 
according to the docs. I did that but seems working.

Any one please specify whta other changes i have to do to make it working.

Thanks for your support guys.


--
Regards,
Geo Varghese
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Venom vulnerability

[Fox, Kevin M]

Great. Thanks for sharing. I'll have to try it myself. :)

Kevin

From: Cynthia Lopes [clsacrame...@gmail.com]
Sent: Thursday, June 04, 2015 9:08 AM
To: Fox, Kevin M
Cc: Steve Gordon; OpenStack Operations Mailing List
Subject: Re: [Openstack-operators] Venom vulnerability

Well it works with Ceph Giant here, I did not upgrade to CentOS7.1 though, and 
I had ceph client installed before updating qemu.
I did that, installed qemu from 7.1 and it didnt break my conf. I was able to 
restart old VMs and deploy new ones.
No need to re-compile qemu-kvm anymore \o/


Yeh, I can't find anything to prove the hosts are not vulnerable anymore, I 
think I'll just give it sometime for now...

Rgards,
Cynthia

2015-06-04 17:50 GMT+02:00 Fox, Kevin M 
mailto:kevin@pnnl.gov>>:
I was curious because we're running giant and built a custom qemu too. I just 
rebuilt a patched one to cover Venom, but want to switch to stock 7.1 qemu at 
some point if it supports Ceph Giant.

I'm not aware of any check that actually tests the vulnerability. Just checks 
package versions.

Thanks,
Kevin


From: Cynthia Lopes [clsacrame...@gmail.com<mailto:clsacrame...@gmail.com>]
Sent: Thursday, June 04, 2015 8:05 AM
To: Fox, Kevin M
Cc: Steve Gordon; OpenStack Operations Mailing List

Subject: Re: [Openstack-operators] Venom vulnerability

Hi,

I dit not update my ceph client. The version before and after is:

# ceph -v
ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578)


Apart from checking my qemu-kvm version and having shutdown/up my instances, 
any ideas on how to validate that my host is no longer vulnerable?

Regards,
Cynthia


2015-06-04 16:59 GMT+02:00 Fox, Kevin M 
mailto:kevin@pnnl.gov>>:
For the record, what version of ceph are you using before and after?

Thanks,
Kevin


From: Cynthia Lopes
Sent: Thursday, June 04, 2015 1:27:53 AM
To: Steve Gordon
Cc: OpenStack Operations Mailing List
Subject: Re: [Openstack-operators] Venom vulnerability

Hi guys,

Just for feedback and if somebody else has compute nodes on CentOS 7.0, 
IceHouse and uses Ceph.



--
What I did that worked for me:
--



#Remove all QEMU and Livirt related RPMs. I had recompiled QEMU for RBD and 
Libvirt that I had was not compatible with the patched QEMU.
#This removes openstack-nova-compute and so on, be careful...
yum remove -y `rpm -qa | grep qemu`
yum remove -y `rpm -qa | grep libvirt`

#I updated base and update centos repositories to gether from most up to date 
versions. I have local repositories, so the commands should be adapted...
sed -i "s|/centos7|/centos7.1|g" CentOS-Base7.repo
sed -i "s|/centos7update|/centos7.1update|g" CentOS-Base7.repo

#I had to do an update...
yum clean all
yum -y update #check problem only with ceph... I had some dependencies problems 
with the ceph packages. But just Ceph

yum -y update --skip-broken #but ignoring them worked just fine

cd /etc/yum.repos.d/
#The update added all these repos on my yum.repos.d so I deleted (because I use 
local repositories)
rm -f CentOS-Base.repo CentOS-Debuginfo.repo CentOS-fasttrack.repo 
CentOS-Sources.repo CentOS-Vault.repo

#Then I re-installed QEMU and Libvirt with CentOS7.1 repositories (base and 
update)
yum -y install kvm qemu-kvm python-virtinst libvirt libvirt-python virt-manager 
libguestfs-tools
service libvirtd start

#I use puppet to configure my host, so I just re-run it to re-install 
nova-compute and re-configure
puppet agent -t #so replace this with your procedure for configure your compute 
node

service openstack-nova-compute status #chek nova-compute is running...

#I had a console.log file in the instances directory that became owned by root. 
So be sure to have everything owned by nova
chown -R nova:nova /var/lib/nova/

#Of course, at this moment all my instances were shutoff, so just restart 
them...


source keystonerc* #credentials

vms=`nova list --all-tenants --minimal --host $host | grep -v ID | grep -v "+-" 
| awk '{print $2}'` #guest vms ids on the host...

for vm in $vms ; do nova start $vm; done  #start vms...





Hope this might be useful for someone...

Regards,
Cynthia Lopes do Sacramento

2015-06-03 2:35 GMT+02:00 Steve Gordon 
mailto:sgor...@redhat.com>>:
- Original Message -
> From: "Erik McCormick" 
> mailto:emccorm...@cirrusseven.com>>
> To: "Tim Bell" mailto:tim.b...@cern.ch>>
>
> On Tue, Jun 2, 2015 at 5:34 AM, Tim Bell 
> mailto:tim.b...@cern.ch>> wrote:
>
> >  I had understood that CentOS 7.1 qemu-kvm has RBD support built-in. It
> > was not there on 7.0 but http://tracker.ceph.com/issues/10480 implies it

Re: [Openstack-operators] Venom vulnerability

[Fox, Kevin M]

I was curious because we're running giant and built a custom qemu too. I just 
rebuilt a patched one to cover Venom, but want to switch to stock 7.1 qemu at 
some point if it supports Ceph Giant.

I'm not aware of any check that actually tests the vulnerability. Just checks 
package versions.

Thanks,
Kevin


From: Cynthia Lopes [clsacrame...@gmail.com]
Sent: Thursday, June 04, 2015 8:05 AM
To: Fox, Kevin M
Cc: Steve Gordon; OpenStack Operations Mailing List
Subject: Re: [Openstack-operators] Venom vulnerability

Hi,

I dit not update my ceph client. The version before and after is:

# ceph -v
ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578)


Apart from checking my qemu-kvm version and having shutdown/up my instances, 
any ideas on how to validate that my host is no longer vulnerable?

Regards,
Cynthia


2015-06-04 16:59 GMT+02:00 Fox, Kevin M 
mailto:kevin@pnnl.gov>>:
For the record, what version of ceph are you using before and after?

Thanks,
Kevin


From: Cynthia Lopes
Sent: Thursday, June 04, 2015 1:27:53 AM
To: Steve Gordon
Cc: OpenStack Operations Mailing List
Subject: Re: [Openstack-operators] Venom vulnerability

Hi guys,

Just for feedback and if somebody else has compute nodes on CentOS 7.0, 
IceHouse and uses Ceph.



--
What I did that worked for me:
--



#Remove all QEMU and Livirt related RPMs. I had recompiled QEMU for RBD and 
Libvirt that I had was not compatible with the patched QEMU.
#This removes openstack-nova-compute and so on, be careful...
yum remove -y `rpm -qa | grep qemu`
yum remove -y `rpm -qa | grep libvirt`

#I updated base and update centos repositories to gether from most up to date 
versions. I have local repositories, so the commands should be adapted...
sed -i "s|/centos7|/centos7.1|g" CentOS-Base7.repo
sed -i "s|/centos7update|/centos7.1update|g" CentOS-Base7.repo

#I had to do an update...
yum clean all
yum -y update #check problem only with ceph... I had some dependencies problems 
with the ceph packages. But just Ceph

yum -y update --skip-broken #but ignoring them worked just fine

cd /etc/yum.repos.d/
#The update added all these repos on my yum.repos.d so I deleted (because I use 
local repositories)
rm -f CentOS-Base.repo CentOS-Debuginfo.repo CentOS-fasttrack.repo 
CentOS-Sources.repo CentOS-Vault.repo

#Then I re-installed QEMU and Libvirt with CentOS7.1 repositories (base and 
update)
yum -y install kvm qemu-kvm python-virtinst libvirt libvirt-python virt-manager 
libguestfs-tools
service libvirtd start

#I use puppet to configure my host, so I just re-run it to re-install 
nova-compute and re-configure
puppet agent -t #so replace this with your procedure for configure your compute 
node

service openstack-nova-compute status #chek nova-compute is running...

#I had a console.log file in the instances directory that became owned by root. 
So be sure to have everything owned by nova
chown -R nova:nova /var/lib/nova/

#Of course, at this moment all my instances were shutoff, so just restart 
them...


source keystonerc* #credentials

vms=`nova list --all-tenants --minimal --host $host | grep -v ID | grep -v "+-" 
| awk '{print $2}'` #guest vms ids on the host...

for vm in $vms ; do nova start $vm; done  #start vms...





Hope this might be useful for someone...

Regards,
Cynthia Lopes do Sacramento

2015-06-03 2:35 GMT+02:00 Steve Gordon 
mailto:sgor...@redhat.com>>:
- Original Message -
> From: "Erik McCormick" 
> mailto:emccorm...@cirrusseven.com>>
> To: "Tim Bell" mailto:tim.b...@cern.ch>>
>
> On Tue, Jun 2, 2015 at 5:34 AM, Tim Bell 
> mailto:tim.b...@cern.ch>> wrote:
>
> >  I had understood that CentOS 7.1 qemu-kvm has RBD support built-in. It
> > was not there on 7.0 but http://tracker.ceph.com/issues/10480 implies it
> > is in 7.1.
> >
> >
> >
> > You could check on the centos mailing lists to be sure.
> >
> >
> >
> > Tim
> >
> >
> It's about time! Thanks for the pointer Tim.
>
> Cynthia, If for some reason it's not in the Centos ones yet, I've been
> using the RHEV SRPMs and building the packages. You don't have to mess with
> the spec or anything. Just run them through rpmbuild and push them out.
>
> http://ftp.redhat.com/pub/redhat/linux/enterprise/7Server/en/RHEV/SRPMS/
>
> -Erik

FWIW equivalents builds for use with oVirt, RDO, etc. are being created under 
the auspices of the CentOS Virt SIG:

http://cbs.centos.org/repos/virt7-kvm-common-testing/x86_64/os/Packages/

Thanks,

Steve

___
OpenStack-operat

Re: [Openstack-operators] Help with multiple external network in openstack

[Fox, Kevin M]

Bridge mappings set in plugin.ini?

Thanks,
Kevin


From: Geo Varghese
Sent: Thursday, June 04, 2015 6:25:46 AM
To: openstack-operators@lists.openstack.org; openst...@lists.openstack.org
Subject: [Openstack-operators] Help with multiple external network in openstack

Hi Team,

I need some help to setup multiple external network

In normal single external network we create br-ex bridge and add it in

/etc/neutron/l3_agent.ini

As

external_network_bridge = br-ex

It is working for me.


But in the case of multiple external network, this variable to be set to empty 
according to the docs. I did that but seems working.

Any one please specify whta other changes i have to do to make it working.

Thanks for your support guys.


--
Regards,
Geo Varghese
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Venom vulnerability

[Fox, Kevin M]

For the record, what version of ceph are you using before and after?

Thanks,
Kevin


From: Cynthia Lopes
Sent: Thursday, June 04, 2015 1:27:53 AM
To: Steve Gordon
Cc: OpenStack Operations Mailing List
Subject: Re: [Openstack-operators] Venom vulnerability

Hi guys,

Just for feedback and if somebody else has compute nodes on CentOS 7.0, 
IceHouse and uses Ceph.



--
What I did that worked for me:
--



#Remove all QEMU and Livirt related RPMs. I had recompiled QEMU for RBD and 
Libvirt that I had was not compatible with the patched QEMU.
#This removes openstack-nova-compute and so on, be careful...
yum remove -y `rpm -qa | grep qemu`
yum remove -y `rpm -qa | grep libvirt`

#I updated base and update centos repositories to gether from most up to date 
versions. I have local repositories, so the commands should be adapted...
sed -i "s|/centos7|/centos7.1|g" CentOS-Base7.repo
sed -i "s|/centos7update|/centos7.1update|g" CentOS-Base7.repo

#I had to do an update...
yum clean all
yum -y update #check problem only with ceph... I had some dependencies problems 
with the ceph packages. But just Ceph

yum -y update --skip-broken #but ignoring them worked just fine

cd /etc/yum.repos.d/
#The update added all these repos on my yum.repos.d so I deleted (because I use 
local repositories)
rm -f CentOS-Base.repo CentOS-Debuginfo.repo CentOS-fasttrack.repo 
CentOS-Sources.repo CentOS-Vault.repo

#Then I re-installed QEMU and Libvirt with CentOS7.1 repositories (base and 
update)
yum -y install kvm qemu-kvm python-virtinst libvirt libvirt-python virt-manager 
libguestfs-tools
service libvirtd start

#I use puppet to configure my host, so I just re-run it to re-install 
nova-compute and re-configure
puppet agent -t #so replace this with your procedure for configure your compute 
node

service openstack-nova-compute status #chek nova-compute is running...

#I had a console.log file in the instances directory that became owned by root. 
So be sure to have everything owned by nova
chown -R nova:nova /var/lib/nova/

#Of course, at this moment all my instances were shutoff, so just restart 
them...


source keystonerc* #credentials

vms=`nova list --all-tenants --minimal --host $host | grep -v ID | grep -v "+-" 
| awk '{print $2}'` #guest vms ids on the host...

for vm in $vms ; do nova start $vm; done  #start vms...





Hope this might be useful for someone...

Regards,
Cynthia Lopes do Sacramento

2015-06-03 2:35 GMT+02:00 Steve Gordon 
mailto:sgor...@redhat.com>>:
- Original Message -
> From: "Erik McCormick" 
> mailto:emccorm...@cirrusseven.com>>
> To: "Tim Bell" mailto:tim.b...@cern.ch>>
>
> On Tue, Jun 2, 2015 at 5:34 AM, Tim Bell 
> mailto:tim.b...@cern.ch>> wrote:
>
> >  I had understood that CentOS 7.1 qemu-kvm has RBD support built-in. It
> > was not there on 7.0 but http://tracker.ceph.com/issues/10480 implies it
> > is in 7.1.
> >
> >
> >
> > You could check on the centos mailing lists to be sure.
> >
> >
> >
> > Tim
> >
> >
> It's about time! Thanks for the pointer Tim.
>
> Cynthia, If for some reason it's not in the Centos ones yet, I've been
> using the RHEV SRPMs and building the packages. You don't have to mess with
> the spec or anything. Just run them through rpmbuild and push them out.
>
> http://ftp.redhat.com/pub/redhat/linux/enterprise/7Server/en/RHEV/SRPMS/
>
> -Erik

FWIW equivalents builds for use with oVirt, RDO, etc. are being created under 
the auspices of the CentOS Virt SIG:

http://cbs.centos.org/repos/virt7-kvm-common-testing/x86_64/os/Packages/

Thanks,

Steve

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] PXE chaining

[Fox, Kevin M]

If you disable discovery/unknown hosts on both of the pxe servers, you should 
be able to define the machine in only one of the pxe servers and boot it 
reliably. I've mixed Ironic and Cobbler managed hosts on the same network this 
way.

Thanks,
Kevin

From: Adam Lawson [alaw...@aqorn.com]
Sent: Friday, May 29, 2015 9:44 AM
To: openstack-operators@lists.openstack.org
Subject: [Openstack-operators] PXE chaining

Hey all, curious if anyone has explored ways to get a machine to boot from 
different optional PXE servers listening on the same or different NIC's? We 
manage clouds using PXE (at least) and we want to deploy ironic which also uses 
PXE. We just want to deploy Ironic clouds aside adjacent to other clouds that 
do not use Ironic.


Adam Lawson

AQORN, Inc.
427 North Tatnall Street
Ste. 58461
Wilmington, Delaware 19801-2230
Toll-free: (844) 4-AQORN-NOW ext. 101
International: +1 302-387-4660
Direct: +1 916-246-2072
[http://www.aqorn.com/images/logo.png]
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] what is the different in use Qcow2 or Raw in Ceph

[Fox, Kevin M]

I've experienced the opposite problem though. Downloading raw images and 
uploading them to the cloud is very slow. Doing it through qcow2 allows them to 
be compressed over the slow links. Ideally, the Ceph driver would take a qcow2 
and convert it to raw on glance ingest rather then at boot.

Thanks,
Kevin

From: Dmitry Borodaenko [dborodae...@mirantis.com]
Sent: Thursday, May 28, 2015 12:10 PM
To: David Medberry
Cc: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] what is the different in use Qcow2 or Raw in 
Ceph

David is right, Ceph implements volume snapshotting at the RBD level,
not even RADOS level: whole 2 levels of abstraction above file system.
It doesn't matter if it's XFS, BtrFS, Ext4, or VFAT (if Ceph supported
VFAT): Ceph RBD takes care of it before individual chunks of an RBD
volume are passed to RADOS as objects and get written into the file
system as files by an OSD process.

The reason Fuel documentation recommends to disable QCOW2 format for
images is that RBD does not support QCOW2 disks directly, so Nova and
Cinder have to _convert_ a QCOW2 image into RAW format before passing
it to QEMU's RBD driver. This means that you end up downloading the
QCOW2 image from Ceph to a nova-compute node (first full copy),
converting it (second full copy), and uploading the resultant RAW
image back to Ceph (third full copy) just to launch a VM or create a
volume from an image.


On Thu, May 28, 2015 at 8:33 AM, David Medberry  wrote:
> yep. It's at the CEPH level (not the XFS level.)
>
> On Thu, May 28, 2015 at 8:40 AM, Stephen Cousins 
> wrote:
>>
>> Hi David,
>>
>> So Ceph will use Copy-on-write even with XFS?
>>
>> Thanks,
>>
>> Steve
>>
>> On Thu, May 28, 2015 at 10:36 AM, David Medberry 
>> wrote:
>>>
>>> This isn't remotely related to btrfs. It works fine with XFS. Not sure
>>> how that works in Fuel, never used it.
>>>
>>> On Thu, May 28, 2015 at 8:01 AM, Forrest Flagg 
>>> wrote:

 I'm also curious about this.  Here are some other pieces of information
 relevant to the discussion.  Maybe someone here can clear this up for me as
 well.  The documentation for Fuel 6.0, not sure what they changed for 6.1,
 [1] states that when using Ceph one should disable qcow2 so that images are
 stored in raw format.  This is due to the fact that Ceph includes its own
 mechanisms for copy-on-write and snapshots.  According to the Ceph
 documentation [2], this is true only when using a BTRFS file system, but in
 Fuel 6.0 Ceph uses XFS which doesn't provide this functionality.  Also, [2]
 recommends not using BTRFS for production as it isn't considered fully
 mature.  In addition, Fuel 6.0 [3] states that OpenStack with raw images
 doesn't support snapshotting.

 Given this, why does Fuel suggest not using qcow2 with Ceph?  How can
 Ceph be useful if snapshotting isn't an option with raw images and qcow2
 isn't recommended?  Are there other factors to take into consideration that
 I'm missing?

 [1]
 https://docs.mirantis.com/openstack/fuel/fuel-6.0/terminology.html#qcow2
 [2]
 http://ceph.com/docs/master/rados/configuration/filesystem-recommendations/
 [3]
 https://docs.mirantis.com/openstack/fuel/fuel-6.0/user-guide.html#qcow-format-ug

 Thanks,

 Forrest

 On Thu, May 28, 2015 at 8:02 AM, David Medberry 
 wrote:
>
> and better explained here:
> http://ceph.com/docs/master/rbd/qemu-rbd/
>
> On Thu, May 28, 2015 at 6:02 AM, David Medberry
>  wrote:
>>
>> The primary difference is the ability for CEPH to make zero byte
>> copies. When you use qcow2, ceph must actually create a complete copy
>> instead of a zero byte copy as it cannot do its own copy-on-write tricks
>> with a qcow2 image.
>>
>> So, yes, it will work fine with qcow2 images but it won't be as
>> performant as it is with RAW. Also, it will actually use more of the 
>> native
>> underlying storage.
>>
>> This is also shown as an Important Note in the CEPH docs:
>> http://ceph.com/docs/master/rbd/rbd-openstack/
>>
>> On Thu, May 28, 2015 at 4:12 AM, Shake Chen 
>> wrote:
>>>
>>> Hi
>>>
>>> Now I try to use Fuel 6.1 deploy openstack Juno, use Ceph as cinder,
>>> nova and glance backend.
>>>
>>> In Fuel document suggest if use ceph, suggest use RAW format image.
>>>
>>> but if I upload qcow2 image, seem working well.
>>>
>>> what is the different use qcow2 and RAW in Ceph?
>>>
>>>
>>> --
>>> Shake Chen
>>>
>>>
>>> ___
>>> OpenStack-operators mailing list
>>> OpenStack-operators@lists.openstack.org
>>>
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>>
>>
>
>
> __

Re: [Openstack-operators] How do your end users use networking?

[Fox, Kevin M]

I would agree from the standpoint that most users don't want to care about the 
network but the majority of the users are folks that want a higher level of 
service then just raw compute resources too.

They want an app catalog where they can get fully featured, reliable, scalable, 
and secure cloud enabled apps that they can just deploy and consume, and not 
care about any of the details. The same way you go to McDonalds and get a 
hamburger and not want to care about the huge scalable pipelines they have had 
to develop to get it from cow/field all the way to you.

It is the cloud application developers, that need to write the cloud apps that 
the end users above will consume, those are the folks that will care that 
features like NaaS exists as a common feature on clouds since those types of 
apps usually require that sort of functionality. If its not common, its hard 
for their apps to get consumed. So simply asking if the majority of users 
want/care about feature X, when they don't know or don't care to look that 
deeply into it doesn't mean they don't need it and ops don't need to provide 
it, simply that they don't care to know how hamburger is made. As the app 
catalog progresses, I believe it will become increasingly important.

Thanks,
Kevin


From: Kris G. Lindgren
Sent: Friday, May 22, 2015 6:28:44 PM
To: openstack-operators@lists.openstack.org
Subject: [Openstack-operators] How do your end users use networking?

During the Openstack summit this week I got to talk to a number of other 
operators of large Openstack deployments about how they do networking.  I was 
happy, surprised even, to find that a number of us are using a similar type of 
networking strategy.  That we have similar challenges around networking and are 
solving it in our own but very similar way.  It is always nice to see that 
other people are doing the same things as you or see the same issues as you are 
and that "you are not crazy". So in that vein, I wanted to reach out to the 
rest of the Ops Community and ask one pretty simple question.

Would it be accurate to say that most of your end users want almost nothing to 
do with the network?

In my experience what the majority of them (both internal and external) want is 
to consume from Openstack a compute resource, a property of which is it that 
resource has an IP address.  They, at most, care about which "network" they are 
on.  Where a "network" is usually an arbitrary definition around a set of real 
networks, that are constrained to a location, in which the company has attached 
some sort of policy.  For example, I want to be in the production network vs's 
the xyz lab network, vs's the backup network, vs's the corp network.  I would 
say for Godaddy, 99% of our use cases would be defined as: I want a compute 
resource in the production network zone, or I want a compute resource in this 
other network zone.  The end user only cares that the IP the vm receives works 
in that zone, outside of that they don't care any other property of that IP.  
They do not care what subnet it is in, what vlan it is on, what switch it is 
attached to, what router its attached to, or how data flows in/out of that 
network.  It just needs to work. We have also found that by giving the users a 
floating ip address that can be moved between vm's (but still constrained 
within a "network" zone) we can solve almost all of our users asks.  Typically, 
the internal need for a floating ip is when a compute resource needs to talk to 
another protected internal or external resource. Where it is painful (read: 
slow) to have the acl's on that protected resource updated. The external need 
is from our hosting customers who have a domain name (or many) tied to an IP 
address and changing IP's/DNS is particularly painful.

Since the vast majority of our end users don't care about any of the technical 
network stuff, we spend a large amount of time/effort in abstracting or hiding 
the technical stuff from the users view. Which has lead to a number of patches 
that we carry on both nova and neutron (and are available on our public 
github). At the same time we also have a *very* small subset of (internal) 
users who are at the exact opposite end of the scale.  They care very much 
about the network details, possibly all the way down to that they want to boot 
a vm to a specific HV, with a specific IP address on a specific network 
segment.  The difference however, is that these users are completely aware of 
the topology of the network and know which HV's map to which network segments 
and are essentially trying to make a very specific ask for scheduling.


Kris Lindgren
Senior Linux Systems Engineer
GoDaddy, LLC.
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] OpenStack Dockerizing on CoreOS

[Fox, Kevin M]

Have you looked at the kolla project?

Thanks,
Kevin


From: CoreOS
Sent: Wednesday, April 29, 2015 5:08:12 AM
To: openstack-operators@lists.openstack.org
Subject: [Openstack-operators] OpenStack Dockerizing on CoreOS

Hello,

I’m trying to develop fault tolerance supporting OpenStack on Docker/CoreOS. I 
think this kind of approaching is to getting the following advantages:
   - Easy to Deploy
   - Easy to Test
   - Easy to Scale-out
   - Fault Tolerance

Those who are interested in non-stop operation and easy extension of OpenStack, 
please see the following link https://github.com/ContinUSE/openstack-on-coreos.

This work is currently in the beginning. Please contact me if you have any 
comments, or questions.

Thanks,
JW
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Sharing resources across OpenStack instances

[Fox, Kevin M]

> -Original Message-
> From: Richard Raseley [mailto:rich...@raseley.com]
> Sent: Thursday, April 23, 2015 2:34 PM
> To: Fox, Kevin M
> Cc: openstack-operators@lists.openstack.org
> Subject: Re: [Openstack-operators] Sharing resources across OpenStack
> instances
> 
> Fox, Kevin M wrote:
> > I'm thinking more like this use case: I'm a cloud user, I want a new
> > Trac site. or an Elastic Search cluster. Or a Jenkins system for my
> > project. Why do I have to take a lot of time to deploy these things
> > myself? Ideally, one person, or a small group of folks provide generic
> > templates a common deployment of a given software that can easily be
> > discovered/launched by end users of any cloud. Docker is starting to
> > do this with the Docker Hub. Docker's too low level to do it really
> > nicely though. Say for an ElasticSearch cluster, you want it scalable,
> > with between M and N instances, with a load balancer and a private
> > network. Heat can do that... Murano is starting to enable that
> > ability, but requires the cloud admin to preload up all the bits into
> > it. That doesn't scale well though.
> >
> > I guess the how of how that works is up for grabs. Murano might not be
> > the right place for it.
> >
> > What other use cases for transferring resources between clouds can you
> > think of?
> 
> I have thought of the following:
> 
> * Swift - Users want to transfer objects or containers.

Makes sense.
 
> * Glance - Users want to transfer images and their associated metadata.

Also makes sense.

> * Cinder - Users want to transfer block devices, though this could be
> plausibly covered by Glance (vols -> images -> vols).

Yeah, there are a lot of ways to do this one. There's also snapshots and 
backups too. :/
I've often wanted to download a cinder volume as a qcow2 file.

> * Heat - Users want to transfer stack templates. I know we don't have the
> ability to natively store these artifacts at this time (in the same way Glance
> stores images).

I haven’t tried it, but 
https://www.openstack.org/summit/openstack-paris-summit-2014/session-videos/presentation/glance-artifacts-it-and-39-s-not-just-for-images-anymore
 claims heat artifacts are working in juno. So maybe specifically heat resource 
transfer isn't needed if glance has it covered.

> 
> * Manilla - Users want to transfer files or folder hierarchies.

Hmm So is rsync out of the question, or are you more interested in 
scheduling/managing transfers via the cloud?

> 
> > Perhaps if the app store like functionality was simply http based, the
> > existing glance fetch image from http mechanism would already work.
> 
> Seems reasonable, as long as you could give it a feed or endpoint to poll on
> a regular interval.
> 
> >
> > Another option would be for it to work similarly to yum/apt. You have
> > a set of repo's. You can search though the enabled repo's and install
> > stuff. Perhaps it has a yum-cron like "update the images that are
> > 'installed' daily" like feature...
> 
> I think you're touching on a lot of important (and really good) ideas here, it
> seems like there are various plausible ways one could go about
> 
> You've also touched on something that I've never really understood, which
> is why does Murano exist at all?
> 
> This is obviously off-topic in the context of this particular discussion, but 
> it
> seems to me like we already have something that can store objects and
> metadata (Glance) and something that handles application deployment and
> orchestration (Heat). It seems that the functionality of Murano should've
> naturally fell into these two projects with (a) Glance becoming a more
> general 'artifact' storage and metadata service and (b) Heat gaining the
> 'catalog' capability of Murano. In this way, specific applications would be
> represented by their own Heat templates, those templates could then be
> combined in an ad hoc way to build environments (which I believe is the
> terminology used in Murano) and then the the resulting composition would
> be orchestrated by Heat.
> 
> Perhaps I am missing something fundamental which Murano does...

This was brought up precisely when Murano tried to incubate. And its one of the 
reasons a lot of functionality got pushed over into Glance I think. I'm not 
really sure where Murano fits long term in the picture either, but I do know 
for sure that the app catalog system needs some kind of UI, and if all the rest 
of the Murano bits get pushed into the other OpenStack projects, I'm guessing 
that will still be needed. Whether the result is still called Murano, or it 
ends up just being folded into Horizon's project, that may not matter. The 
important thing is there's an app catalog.

Thanks,
Kevin

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Sharing resources across OpenStack instances

[Fox, Kevin M]

I'm thinking more like this use case:
I'm a cloud user, I want a new Trac site. or an Elastic Search cluster. Or a 
Jenkins system for my project. Why do I have to take a lot of time to deploy 
these things myself? Ideally, one person, or a small group of folks provide 
generic templates a common deployment of a given software that can easily be 
discovered/launched by end users of any cloud. Docker is starting to do this 
with the Docker Hub. Docker's too low level to do it really nicely though. Say 
for an ElasticSearch cluster, you want it scalable, with between M and N 
instances, with a load balancer and a private network. Heat can do that... 
Murano is starting to enable that ability, but requires the cloud admin to 
preload up all the bits into it. That doesn't scale well though.

I guess the how of how that works is up for grabs. Murano might not be the 
right place for it.

What other use cases for transferring resources between clouds can you think of?

Perhaps if the app store like functionality was simply http based, the existing 
glance fetch image from http mechanism would already work.

Another option would be for it to work similarly to yum/apt. You have a set of 
repo's. You can search though the enabled repo's and install stuff. Perhaps it 
has a yum-cron like "update the images that are 'installed' daily" like 
feature...

Thanks,
Kevin

From: Richard Raseley [rich...@raseley.com]
Sent: Thursday, April 23, 2015 12:01 PM
To: Fox, Kevin M
Cc: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Sharing resources across OpenStack instances

Fox, Kevin M wrote:
> Some folks have been discussing an app store model. perhaps in
> Murano, but more global, that would allow images/template to be
> registered somewhere like on openstack.org and show up on all clouds
> that have the global repo enabled. Murano would be extended to fetch
> the images/templates to the local glance on the first launch of an
> app if its not already cached locally.
>
> Would that sort of thing fit better then trying to sync glances
> backing store between clouds?

My first take is that what you've outlined feels a little too ambitious
to me. I also have concerns (feature and scope creep) about whether or
not Murano (or any other project) should be in the business of managing
and auditing replication of data between various other (potentially
disparate) systems.

I think there is tremendous value at the core of the idea, which is
essentially the ability to easily (and granularly) share and consume
resources between clouds. To me that feels much more like something that
would be interested on a per-project basis (as has been done in Swift).

In the model I am imagining, the underlying components of the cloud
would be responsible for inter-cloud data replication.

In the specific case of Glance images, it could either take the form of
a pub / sub model at the Glance level (which would allow replication of
images between Glance systems utilizing different back-ends) or the form
of backend <-> backend replication (e.g. with Swift Container
Replication) and then Glance would simply have a process for discovering
new images which have appeared on the back-end.

Regards,

Richard Raseley

SysOps Engineer
Puppet Labs

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Sharing resources across OpenStack instances

[Fox, Kevin M]

Some folks have been discussing an app store model. perhaps in Murano, but more 
global, that would allow images/template to be registered somewhere like on 
openstack.org and show up on all clouds that have the global repo enabled. 
Murano would be extended to fetch the images/templates to the local glance on 
the first launch of an app if its not already cached locally.

Would that sort of thing fit better then trying to sync glances backing store 
between clouds?

Thanks,
Kevin


From: Richard Raseley [rich...@raseley.com]
Sent: Thursday, April 23, 2015 9:55 AM
To: Tim Bell
Cc: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Sharing resources across OpenStack   
instances

Tim Bell wrote:
> Has anyone found a good way to do this without needing the user to upload
> to each cloud (and handle the associated consistency themselves) ?

For the use-case of inter-cloud replication and Swift, it seems pretty
straight forward to configure 'Container to Container Synchronization'[0].

I haven't done this specifically before, but it seems like you would
just then need some sort of auditing script running on both sides to
make Glance aware of replicated images.

Regards,

Richard Raseley

SysOps Engineer
Puppet Labs

[0] - http://docs.openstack.org/developer/swift/overview_container_sync.html

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Sharing resources across OpenStack instances

[Fox, Kevin M]

This is a case for a cross project cloud (institutional?). It costs more to run 
two little clouds then one bigger one. Both in terms of man power, and in cases 
like these. under utilized resources.

#3 is interesting though. If there is to be an openstack app catalog, it would 
be inportant to be able to pull the needed images from outside the cloud easily.

Thanks,
Kevin


From: Adam Young
Sent: Wednesday, April 22, 2015 6:32:17 AM
To: openstack-operators@lists.openstack.org
Subject: [Openstack-operators] Sharing resources across OpenStack instances

Its been my understanding that many people are deploying small OpenStack
instances as a way to share the Hardware owned by their particular team,
group, or department.  The Keystone instance represents ownership, and
the identity of the users comes from a corporate LDAP server.

Is there much demand for the following scenarios?

1.  A project team crosses organizational boundaries and has to work
with VMs in two separate OpenStack instances.  They need to set up a
network that requires talking to two neutron instances.

2.  One group manages a powerful storage array.  Several OpenStack
instances need to be able to mount volumes from this array. Sometimes,
those volumes have to be transferred from VMs running in one instance to
another.

3.  A group is producing nightly builds.  Part of this is an image
building system that posts to glance.  Ideally, multiple OpenStack
instances would be able to pull their images from the same glance.

4. Hadoop ( or some other orchestrated task) requires more resources
than are in any single OpenStack instance, and needs to allocate
resources across two or more instances for a single job.


I suspect that these kinds of architectures are becoming more common.
Can some of the operators validate these assumptions?  Are there other,
more common cases where Operations need to span multiple clouds which
would require integration of one Nova server with multiple Cinder,
Glance, or Neutron  servers managed in other OpenStack instances?

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] No more identity groups?

[Fox, Kevin M]

Its part of the keystone v3 api I think. I've only seen it show up when I 
configure horizon to specifically talk v3 to keystone.

Thanks,
Kevin

From: Mike Smith [mism...@overstock.com]
Sent: Wednesday, April 15, 2015 4:33 PM
To: 
Subject: [Openstack-operators] No more identity groups?

Hello all.   On my Havana-version openstack cloud, Horizon has the option of 
creating groups of users and then adding groups of users to tenants instead of 
having to add users individually.  On my Juno-version cloud, this feature seems 
to no longer exist.I checked the keystone catalog and both are using v2.0 
of the identify API.

Does anybody know if this is something that we removed, or if there is 
something I need to do to enable this identity group functionality?

Thanks,
Mike




CONFIDENTIALITY NOTICE: This message is intended only for the use and review of 
the individual or entity to which it is addressed and may contain information 
that is privileged and confidential. If the reader of this message is not the 
intended recipient, or the employee or agent responsible for delivering the 
message solely to the intended recipient, you are hereby notified that any 
dissemination, distribution or copying of this communication is strictly 
prohibited. If you have received this communication in error, please notify 
sender immediately by telephone or return email. Thank you.

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Hyper-converged OpenStack with Ceph

[Fox, Kevin M]

On the shared cloud, no. There is only one nic and we havent tried doing any 
qos on it.

Thanks,
Kevin


From: Chris Friesen
Sent: Thursday, March 19, 2015 10:10:47 AM
To: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Hyper-converged OpenStack with Ceph

On 03/19/2015 10:33 AM, Fox, Kevin M wrote:
> We've running it both ways. We have clouds with dedicated storage nodes, and
> clouds sharing storage/compute.
>
> The storage/compute solution with ceph is working ok for us. But, that
> particular cloud is 1gigabit only and seems very slow compared to our other
> clouds. But because of the gigabit interconnect, while the others are
> 40gigabit, its not clear if its slow because of the storage/compute together,
> or simply because of the slower interconnect. Could be some of both.

Are you doing anything to isolate the ceph work from the nova compute work?
(Separate NICs, separate CPUs, different disks, etc.)

Chris

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Hypervisor decision

[Fox, Kevin M]

I don't believe they do, but its not about that. its about capacity. To get the 
most out of your really expensive hyperv datacenter license, you should load it 
up with as many windows vm's as you can. A physical machine can only handle a 
fixed number of vm's max. If you put a linux vm on it, thats one less windows 
vm you can launch there, meaning you have to buy more datacenter physical 
nodes/licenses, which adds cost.

While I havent explored this option, it might be possible to buy datacenter 
hyperv licenses for your windows vm's, and put them in one host aggrigate, and 
buy cheaper windows licenses for hyperv for free os's and put them in another, 
and run things that way. Though you will still be paying more for windows 
licenses then if you did kvm for the free os's.

Thanks,
Kevin


From: matt [m...@nycresistor.com]
Sent: Thursday, March 19, 2015 9:36 AM
To: Fox, Kevin M
Cc: maishsk+openst...@maishsk.com; openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Hypervisor decision

I was under the impression hyper-v didn't charge a per seat license on non 
windows instances?

On Thu, Mar 19, 2015 at 12:05 PM, Fox, Kevin M 
mailto:kevin@pnnl.gov>> wrote:
So, in the pets vs cattle cloud philosophy, you want to be able to have as many 
cattle as you need, rather then limit the sets to a smaller number of more pet 
like things.

kvm allows unlimited numbers of vm's, which is very cloudy. but due to Windows 
licensing, tends to only work well with linux/bsd VM's.

Windows is a whole nother kettle of fish. They either license it per vm, which 
is very pet like, or alternately, the more cattle friendly way is to buy a 
DataCenter* version of windows.

Each hypervisor needs to be the DataCenter version, but it allows you to run 
unlimited Windows VM's on that hypervisor. So if you want to run lots of 
windows cattle, its can be the way to go.

Due to its high cost, it does not usually make sense to run all your linux vm's 
on Windows DataCenter version, so you run both kvm for linux/bsd vm's and 
Windows DataCenter licensed hyperv for windows vm's.

* http://www.microsoft.com/licensing/about-licensing/virtualization.aspx

Thanks,
Kevin

From: Maish Saidel-Keesing [mais...@maishsk.com<mailto:mais...@maishsk.com>]
Sent: Thursday, March 19, 2015 12:19 AM
To: 
openstack-operators@lists.openstack.org<mailto:openstack-operators@lists.openstack.org>

Subject: Re: [Openstack-operators] Hypervisor decision

That is interesting Tim.

Why Hyper-V if I may ask? Why not stick just with KVM?

Maish

On 19/03/15 08:22, Tim Bell wrote:
At CERN, we run KVM and Hyper-V. Both work fine.

Depending on the size of your cluster, you may have other factors to consider 
such as monitoring and configuration management. We use Puppet to configure 
both environnments.

Images are tagged with a property hypervisor_type which is used to schedule 
workloads to the appropriate hypervisor.

Tim

From: matt [mailto:m...@nycresistor.com]
Sent: 18 March 2015 23:24
To: Abel Lopez
Cc: 
openstack-operators@lists.openstack.org<mailto:openstack-operators@lists.openstack.org>
Subject: Re: [Openstack-operators] Hypervisor decision

most openstack environments at kvm, so if you want to stick with the herd, 
that's the way to go.

On Wed, Mar 18, 2015 at 5:53 PM, Abel Lopez 
mailto:alopg...@gmail.com>> wrote:
Interesting topic, since you're already running Hyper-v and ESX, I'm inferring 
that your workload is heavy on windows VMs.
If you're doing majority windows, and minority linux, stick with hyper-v. The 
benchmarks I've read show that windows VMs run fastest on hyper-v VS all others.
If you expect an even split, it might make sense to create Host Aggregates of 
various hypervisiors like hyper-v and KVM, and utilize extra-specs in the 
flavors and guest images to aid in scheduling, for example "Windows images 
launch on the hyper-v pool"

> On Mar 18, 2015, at 2:41 PM, Vytenis Silgalis 
> mailto:vsilga...@outlook.com>> wrote:
>
> Hello,
>
> I'm looking to champion openstack at my company, we currently run both a 
> small hyper-v cluster and 3 VMware clusters.   However we are not married to 
> any specific hypervisor.  What I'm looking for is recommendations for which 
> hypervisor we should look at for our openstack environments and the 
> pros/con's people have run into with the various hypervisors supported by 
> openstack.
>
>
> Thanks,
> Vytenis
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org<mailto:OpenStack-operators@lists.openstack.org>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

__

Re: [Openstack-operators] Hyper-converged OpenStack with Ceph

[Fox, Kevin M]

We've running it both ways. We have clouds with dedicated storage nodes, and 
clouds sharing storage/compute.

The storage/compute solution with ceph is working ok for us. But, that 
particular cloud is 1gigabit only and seems very slow compared to our other 
clouds. But because of the gigabit interconnect, while the others are 
40gigabit, its not clear if its slow because of the storage/compute together, 
or simply because of the slower interconnect. Could be some of both.

I'd be very curious if anyone else had a feeling for storage/compute together 
on a faster interconnect.

Thanks,
Kevin


From: Jesse Keating [j...@bluebox.net]
Sent: Thursday, March 19, 2015 9:20 AM
To: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Hyper-converged OpenStack with Ceph

On 3/19/15 9:08 AM, Jared Cook wrote:
> Hi, I'm starting to see a number of vendors push hyper-converged
> OpenStack solutions where compute and Ceph OSD nodes are one in the
> same.  In addition, Ceph monitors are placed on OpenStack controller
> nodes in these architectures.
>
> Recommendations I have read in the past have been to keep these things
> separate, but some vendors are now saying that this actually works out
> OK in practice.
>
> The biggest concern I have is that the compute node functions will
> compete with Ceph functions, and one over utilized node will slow down
> the entire Ceph cluster, which will slow down the entire cloud.  Is this
> an unfounded concern?
>
> Does anyone have experience running in this mode?  Experience at scale?
>
>

Not CEPH related, but it's a known tradeoff that compute resource on
control nodes can cause resource competition. This is a tradeoff for the
total cost of the cluster and the expected use case. If the use case
plans to scale out to many compute nodes, we suggest upgrading to
dedicated control nodes. This is higher cost, but somewhat necessary for
matching performance to capacity.

We may start small, but we can scale up to match the (growing) needs.


--
-jlk

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Hypervisor decision

[Fox, Kevin M]

So, in the pets vs cattle cloud philosophy, you want to be able to have as many 
cattle as you need, rather then limit the sets to a smaller number of more pet 
like things.

kvm allows unlimited numbers of vm's, which is very cloudy. but due to Windows 
licensing, tends to only work well with linux/bsd VM's.

Windows is a whole nother kettle of fish. They either license it per vm, which 
is very pet like, or alternately, the more cattle friendly way is to buy a 
DataCenter* version of windows.

Each hypervisor needs to be the DataCenter version, but it allows you to run 
unlimited Windows VM's on that hypervisor. So if you want to run lots of 
windows cattle, its can be the way to go.

Due to its high cost, it does not usually make sense to run all your linux vm's 
on Windows DataCenter version, so you run both kvm for linux/bsd vm's and 
Windows DataCenter licensed hyperv for windows vm's.

* http://www.microsoft.com/licensing/about-licensing/virtualization.aspx

Thanks,
Kevin

From: Maish Saidel-Keesing [mais...@maishsk.com]
Sent: Thursday, March 19, 2015 12:19 AM
To: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Hypervisor decision

That is interesting Tim.

Why Hyper-V if I may ask? Why not stick just with KVM?

Maish

On 19/03/15 08:22, Tim Bell wrote:
At CERN, we run KVM and Hyper-V. Both work fine.

Depending on the size of your cluster, you may have other factors to consider 
such as monitoring and configuration management. We use Puppet to configure 
both environnments.

Images are tagged with a property hypervisor_type which is used to schedule 
workloads to the appropriate hypervisor.

Tim

From: matt [mailto:m...@nycresistor.com]
Sent: 18 March 2015 23:24
To: Abel Lopez
Cc: 
openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Hypervisor decision

most openstack environments at kvm, so if you want to stick with the herd, 
that's the way to go.

On Wed, Mar 18, 2015 at 5:53 PM, Abel Lopez 
mailto:alopg...@gmail.com>> wrote:
Interesting topic, since you're already running Hyper-v and ESX, I'm inferring 
that your workload is heavy on windows VMs.
If you're doing majority windows, and minority linux, stick with hyper-v. The 
benchmarks I've read show that windows VMs run fastest on hyper-v VS all others.
If you expect an even split, it might make sense to create Host Aggregates of 
various hypervisiors like hyper-v and KVM, and utilize extra-specs in the 
flavors and guest images to aid in scheduling, for example "Windows images 
launch on the hyper-v pool"

> On Mar 18, 2015, at 2:41 PM, Vytenis Silgalis 
> mailto:vsilga...@outlook.com>> wrote:
>
> Hello,
>
> I'm looking to champion openstack at my company, we currently run both a 
> small hyper-v cluster and 3 VMware clusters.   However we are not married to 
> any specific hypervisor.  What I'm looking for is recommendations for which 
> hypervisor we should look at for our openstack environments and the 
> pros/con's people have run into with the various hypervisors supported by 
> openstack.
>
>
> Thanks,
> Vytenis
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators




___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


--
Best Regards, Maish Saidel-Keesing
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Help with glance issue after upgrade from Icehouse to Juno

[Fox, Kevin M]

What about the other glance logfiles? It looks like it may be calling out to a 
different server and thats failing...
Thanks,
Kevin

From: Nathan Stratton [nat...@robotics.net]
Sent: Friday, March 06, 2015 11:42 AM
To: openstack-oper.
Subject: [Openstack-operators] Help with glance issue after upgrade from 
Icehouse to Juno

Everything is working but glance, I can't even glance image-list. My logs don't 
look like they are saying anything useful.


2015-03-06 14:39:19.625 3973 INFO glance.registry.client.v1.client 
[1e5b5ed8-430a-4aec-928b-ebe14573ce42 a7602f2a62f046cda415c2ef3ff0a91c 
b2b9a5f24d4b48d687efa67efde1dd6d - - -] Registry client request GET 
/images/detail raised ServerError
2015-03-06 14:39:19.626 3973 INFO glance.wsgi.server 
[1e5b5ed8-430a-4aec-928b-ebe14573ce42 a7602f2a62f046cda415c2ef3ff0a91c 
b2b9a5f24d4b48d687efa67efde1dd6d - - -] Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 433, in 
handle_one_response
result = self.application(self.environ, start_response)
  File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 394, in 
__call__
response = req.get_response(self.application)
  File "/usr/lib/python2.7/site-packages/webob/request.py", line 1296, in send
application, catch_exc_info=False)
  File "/usr/lib/python2.7/site-packages/webob/request.py", line 1260, in 
call_application
app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/osprofiler/web.py", line 99, in 
__call__
return request.get_response(self.application)
  File "/usr/lib/python2.7/site-packages/webob/request.py", line 1296, in send
application, catch_exc_info=False)
  File "/usr/lib/python2.7/site-packages/webob/request.py", line 1260, in 
call_application
app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token.py", 
line 748, in __call__
return self._call_app(env, start_response)
  File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token.py", 
line 684, in _call_app
return self._app(env, _fake_start_response)
  File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 394, in 
__call__
response = req.get_response(self.application)
  File "/usr/lib/python2.7/site-packages/webob/request.py", line 1296, in send
application, catch_exc_info=False)
  File "/usr/lib/python2.7/site-packages/webob/request.py", line 1260, in 
call_application
app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/site-packages/paste/urlmap.py", line 203, in __call__
return app(environ, start_response)
  File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__
return resp(environ, start_response)
  File "/usr/lib/python2.7/site-packages/routes/middleware.py", line 131, in 
__call__
response = self.app(environ, start_response)
  File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__
return resp(environ, start_response)
  File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 683, in 
__call__
request, **action_args)
  File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 707, in 
dispatch
return method(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/glance/api/v1/images.py", line 347, in 
detail
images = registry.get_images_detail(req.context, **params)
  File "/usr/lib/python2.7/site-packages/glance/registry/client/v1/api.py", 
line 150, in get_images_detail
return c.get_images_detailed(**kwargs)
  File "/usr/lib/python2.7/site-packages/glance/registry/client/v1/client.py", 
line 144, in get_images_detailed
res = self.do_request("GET", "/images/detail", params=params)
  File "/usr/lib/python2.7/site-packages/glance/registry/client/v1/client.py", 
line 130, in do_request
'exc_name': exc_name})
  File "/usr/lib/python2.7/site-packages/glance/openst

Re: [Openstack-operators] Migrating keystone from MySQL to LDAP

[Fox, Kevin M]

See the id_mapping table.

Thanks,
Kevin

From: Antonio Messina [antonio.s.mess...@gmail.com]
Sent: Tuesday, March 03, 2015 11:28 AM
To: Fox, Kevin M
Cc: Caius Howcroft; openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Migrating keystone from MySQL to LDAP

On Mon, Mar 2, 2015 at 5:31 PM, Fox, Kevin M  wrote:
> That leaves identity mapping. There is a table of ldap users to
> unique id's in the database.

I'm not an expert, but I have a Juno testbed that is using LDAP for
identity and SQL for assignment, and the 'id' of the user is, in my
case, the uid attribute of the ldap object (cfr. `user_id_attribute`
option in `keystone.conf`).

$ keystone  user-get antonio
+--+-+
| Property |  Value  |
+--+-+
|id| antonio |
|   name   | antonio |
| username | antonio |
+--+-+

I don't have anything in the `user` table, and the `assignment` table
is populated only when I actually assign a role to an user in a
tenant.

$ keystone user-list --tenant demo
+-+-+-+---+
|id   |   name  | enabled | email |
+-+-+-+---+
| antonio | antonio | |   |
|  sergio |  sergio | |   |
+-+-+-+---+

and in the DB:

mysql> select asgn.actor_id, proj.name as project, role.name as
role from keystone.assignment as asgn left join keystone.project as
proj on asgn.target_id=proj.id left join keystone.role on
asgn.role_id=role.id where proj.name='demo';
+--+-+--+
| actor_id | project | role |
+--+-+--+
| antonio  | demo| Member   |
| sergio   | demo| Member   |
+--+-+--+

.a.

--
antonio.s.mess...@gmail.com
antonio.mess...@uzh.ch +41 (0)44 635 42 22
S3IT: Service and Support for Science IT   http://www.s3it.uzh.ch/
University of Zurich
Winterthurerstrasse 190
CH-8057 Zurich Switzerland

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Migrating keystone from MySQL to LDAP

[Fox, Kevin M]

You can leave the roles/projects outside of ldap by just using the LDAP 
identity plugin, leaving the rest in sql. It sounds like they will be 
deprecating putting roles/projects in LDAP in the future anyway.

That leaves identity mapping. There is a table of ldap users to unique id's in 
the database. I haven't tried, but you might be able to import all your ldap 
users into the table, then before any usage, switch the id to the old id's. No 
idea if its safe to do that though. You will have to test it thoroughly.

Thanks,
Kevin

From: Caius Howcroft [caius.howcr...@gmail.com]
Sent: Monday, March 02, 2015 7:36 AM
To: openstack-operators@lists.openstack.org
Subject: [Openstack-operators] Migrating keystone from MySQL to LDAP

Hi,

We are in the process of migrating off MySQL backend for keystone and
into LDAP. Just wondering if anyone ad any experience with this? I'm
going to have to keep all the id's the same (or else go in and change
project ids etc in things like cinder db). Looks like keystone API
doesn't allow me to force a uuid at creation time for projects, roles
and users. I can go in and create the projects etc in a python script
directly, but thats a bit messy.

Just wondered if anyone had a done this and had a neater solution?

Caius
--

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Networking architecture question: communication between tenants

[Fox, Kevin M]

We have 2 public networks, one for the internet and one public nonroutable one. 
Then we use per tenant private networks and two routers per tenant. One on each 
net. Then default the internet one and the internet router config provides an 
extra route to the nonroutable net router. Has worked well in production for 6 
months with icehouse.

Thanks,
Kevin


From: Emrah Aslan
Sent: Monday, November 03, 2014 1:44:34 AM
To: Michaël Van de Borne; openstack-operators
Subject: Re: [Openstack-operators] Networking architecture question: 
communication between tenants

Hi all,

I've checked  but couldn’t find an exact solution  - somehow can not 
troubleshoot caused by lack of 3rd party troubleshooting sw. Cloud-wide Galera 
cluster  has lots of bugs for sure.

Has anyone tried the AWS amazon tier2 services. We are having exactly the same 
problem within the AWS cloud services.

Kind Regards

Emrah ASLAN
Cisco/Citrix System Engineer



www.logicom.com.tr
Noramin İş Merkezi  | No: 237 /114 | 34398 Maslak ,İstanbul ,TR

T: +90 212 2762720  | D: +90 850 2215821  | M: +90 533 2853803  | F: +90 212 
2762750



Değerli İş Ortaklarımız,
Logicom kampanyaları , fırsat, duyuru ve stok bilgilerinin sizlere düzenli 
ulaşması için  aşağıdaki linki tıklayarak  e-mail adresinizi güncellemenizi 
rica ediyoruz.
http://visitor.r20.constantcontact.com/manage/optin?v=001t9egDEMH10MEulnTu-Lzln0RXbiYIgR2HnLd_hpHmPb0K44ZxJOya0FvCOF3TI8c2qeErt1Xrn3PlZqntTSqiSTW40PTK2XQ8OlOUe4qYOE%3D

-Original Message-
From: Michaël Van de Borne [mailto:michael.vandebo...@cetic.be]
Sent: Monday, November 03, 2014 11:25 AM
To: openstack-operators
Subject: [Openstack-operators] Networking architecture question: communication 
between tenants

Hello,

I'm building a private cloud in which I'd like Application Server instances 
from separate tenants to access the same unique cloud-wide Galera cluster 
(which would have its own tenant).

I'm wondering what the best network topology would be to achieve this.
The constraint is that tenant A Application Server instances should not see 
Tenant B App Servers.
- should I go with a per-tenant router topology? and assign 2 NICs to App 
Server instances: first one in their tenant network,  second one in Galera 
cluster tenant? is that possible?
- should I go with one router for all tenants?
- should the Galera cluster only be accessed from its floating IPs in order to 
avoid all communication between tenants?

Am I missing something?

Your architectural thoughts are welcome.

thank you,

cheers,

michaël

--
Michaël Van de Borne
R&D Engineer, SOA team, CETIC
Phone: +32 (0)71 49 07 45 Mobile: +32 (0)472 69 57 16, Skype: mikemowgli 
www.cetic.be, rue des Frères Wright, 29/3, B-6041 Charleroi


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators