Re: [Openstack-operators] regression testing before upgrade (operators view)

2017-08-31 Thread George Shuklin
est regards, Boris Pavlovic On Tue, Aug 29, 2017 at 2:24 AM, George Shuklin mailto:george.shuk...@gmail.com>> wrote: Hello everyone. Does someone do regression testing before performing upgrade (within same major version)? How do you do this? Do you know any tools for such

[Openstack-operators] regression testing before upgrade (operators view)

2017-08-29 Thread George Shuklin
Hello everyone. Does someone do regression testing before performing upgrade (within same major version)? How do you do this? Do you know any tools for such tests? I started to research this area, and I see three openstack-specific tools: rally (with reduction of load to normal levels), tempe

Re: [Openstack-operators] Issue with snapshots of raw images

2017-03-14 Thread George Shuklin
If anyone curious about this bug: It was clearly linux-3.13 bug, issue was completely solved by moving to 4.4. On 02/15/2017 08:03 PM, George Shuklin wrote: Hello. We've upgraded to mitaka, qemu 2.5/linux-3.13 and found that raw images now have BIG issues with snapshots. Symptoms:

Re: [Openstack-operators] need input on log translations

2017-03-11 Thread George Shuklin
Whole idea with log translation is half-backed anyway. About the half of important log messages contain output of things outside openstack. Libvirt, ip, sudo, kernel, etc. In any i18n installation there going to be some amount of untranslated messages. This kills whole idea of localization. Modern

[Openstack-operators] Issue with snapshots of raw images

2017-02-15 Thread George Shuklin
Hello. We've upgraded to mitaka, qemu 2.5/linux-3.13 and found that raw images now have BIG issues with snapshots. Symptoms: When snapshot process reach 'fallocated' blocks (see below) all IO in guest start lagging, including network IO. Windows starts loosing pings for very long period of

[Openstack-operators] allowed_address_pairs for port in neutron

2017-01-23 Thread George Shuklin
Hello. I'm trying to allow more than one IP on interface for tenant, but neutron (Mitaka) rejects my requests: $ neutron port-update b59bc3bb-7d34-4fbb-8e55-a9f1c5c88411 --allowed-address-pairs type=dict list=true ip_address=10.254.15.4 Unrecognized attribute(s) 'allowed_address_pairs' Neut

Re: [Openstack-operators] Ironic with top-rack switches management

2017-01-17 Thread George Shuklin
On 01/04/2017 07:31 PM, Clint Byrum wrote: Excerpts from George Shuklin's message of 2016-12-26 00:22:38 +0200: Hello everyone. Did someone actually made Ironic running with ToR (top rack switches) under neutron in production? Which switch verdor/plugin (and OS version) do you use? Do you have

[Openstack-operators] Ironic with top-rack switches management

2016-12-25 Thread George Shuklin
Hello everyone. Did someone actually made Ironic running with ToR (top rack switches) under neutron in production? Which switch verdor/plugin (and OS version) do you use? Do you have some switch configuration with parts outside of Neutron reach? Is it worth spent efforts on integration, etc?

Re: [Openstack-operators] Using novaclient, glanceclient, etc, from python

2016-11-25 Thread George Shuklin
, otherwise shade cannot guess it. 2016-11-25 11:42 GMT+01:00 George Shuklin <mailto:george.shuk...@gmail.com>>: shade fails if see too duplicate images in account. o = shade.OpenStackCloud(**creds) o.create_image(name=’killme’, filename=’/tmp/random_junk’, disk_format=’qco

Re: [Openstack-operators] Using novaclient, glanceclient, etc, from python

2016-11-25 Thread George Shuklin
shade fails if see too duplicate images in account. o = shade.OpenStackCloud(**creds) o.create_image(name=’killme’, filename=’/tmp/random_junk’, disk_format=’qcow2', container_format=’bare’, wait=True) |Traceback (most recent call last): ... File "/usr/lib/python2.7/dist-packages/shade/opensta

Re: [Openstack-operators] Using novaclient, glanceclient, etc, from python

2016-11-25 Thread George Shuklin
e to your needs? BR, Konstantin [1] http://docs.openstack.org/user-guide/sdk.html On Nov 17, 2016, at 7:17 PM, George Shuklin wrote: Hello. I can't find proper documentation about how to use openstack clients from inside python application. All I can find is just examples and rather abst

[Openstack-operators] openstack sdk: how to update image (create image with properties)

2016-11-23 Thread George Shuklin
Hello. I'm trying to use openstack SDK in my python code. I want to upload image and set few properties. And I can't. My code (without properties): from openstack import connection import os con=connection.Connection(auth_url=os.environ['OS_AUTH_URL'], project_name = os.environ['OS_TENANT_NAM

[Openstack-operators] Using novaclient, glanceclient, etc, from python

2016-11-17 Thread George Shuklin
Hello. I can't find proper documentation about how to use openstack clients from inside python application. All I can find is just examples and rather abstract (autogenerated) reference. Is there any normal documentation about proper way to use openstack clients from python applications? T

Re: [Openstack-operators] Allow to investigate instance actions after instance deletion

2016-04-18 Thread George Shuklin
. ___ Kris Lindgren Senior Linux Systems Engineer GoDaddy From: Dina Belova mailto:dbel...@mirantis.com>> Date: Wednesday, April 13, 2016 at 4:08 AM To: George Shuklin <mailto:george.shuk...@gmail.com>> Cc: "openstack-operators@lists.openstack.or

[Openstack-operators] Allow to investigate instance actions after instance deletion

2016-04-13 Thread George Shuklin
I filed a bug (feature request) about ability to see deleted instances action list: https://bugs.launchpad.net/nova/+bug/1569779 Any ideas? I really want to see it like this: I filed a bug (feature request) about ability to see deleted instances action list: https://bugs.launchpad.net/nova/+b

Re: [Openstack-operators] OpenStack Architecture, horizon and keystone

2015-09-30 Thread George Shuklin
There is two approaches. 1) You make public endpoints and horizon public. White IP, resolvable FQDN for endpoints, etc. 2) You hide endpoints from user (boo... no api, no automation), but expose horizon. Second case: horizon will sits as middleware between 'internal' networks where endpoints

Re: [Openstack-operators] Live snapshots on the raw disks never ends

2015-09-30 Thread George Shuklin
x27;t be of more help. ___ Kris Lindgren Senior Linux Systems Engineer GoDaddy On 9/24/15, 7:02 AM, "George Shuklin" mailto:george.shuk...@gmail.com>> wrote: >Hello everyone. > >Is som

[Openstack-operators] Live snapshots on the raw disks never ends

2015-09-24 Thread George Shuklin
Hello everyone. Is someone ever saw 'endless snapshot' problem? Some instances (with raw disks and live snapshoting enabled) are stuck at image_uploading forever. It looks like this: +--+--+ | Property

Re: [Openstack-operators] KVM memory overcommit with fast swap

2015-07-03 Thread George Shuklin
One notice: Even on the super-super-fast SSD, there is a huge overhead on IO. Basically, you can't go lower than 50 us on IO, and this is 5 ns, almost eternity for the modern processors. And you get minor page fault, which is not the fastest thing in the world. Few context switching, filesy

Re: [Openstack-operators] Setting IOthreads on a Nova Instance

2015-06-12 Thread George Shuklin
Ceph is slow as a hell by itself, and qemu/libvirt/nova adds qcow2 on top of that. Try to run it with raw volumes, at least. And check if you have properly configured network (jumbo frames, no overloaded switch tables, etc). On 06/12/2015 09:49 AM, pushpesh sharma wrote: Hi list, I need som

Re: [Openstack-operators] Allow user to see instances of other users

2015-06-11 Thread George Shuklin
ybe someone will show me where because I was also looking for it last time :) -- Pozdrawiam / Best regards Sławek Kapłoński sla...@kaplonski.pl Dnia czwartek, 11 czerwca 2015 21:06:31 George Shuklin pisze: Hello. I'm trying to allow a user with special role to see all instances of all ten

[Openstack-operators] Allow user to see instances of other users

2015-06-11 Thread George Shuklin
Hello. I'm trying to allow a user with special role to see all instances of all tenants without giving him admin privileges. My initial attempt was to change policy.json for nova to "compute:get_all_tenants": "role:special_role or is_admin:True". But it didn't work well. The command (nova

Re: [Openstack-operators] Gentoo image availability

2015-06-09 Thread George Shuklin
On 06/09/2015 05:46 AM, Matthew Thode wrote: Ya, not sure how to do multi-interface yet. I'd love if the cloud-init static ip support would work with it. (hash with macs being the key and a list of IPs being the value for each interface). Then dhcp can go away (I tend to much prefer config-dri

Re: [Openstack-operators] Gentoo image availability

2015-06-08 Thread George Shuklin
Nice to hear. You're doing a great job! Few things to make Gentoo 'first class citizen' for openstack (guest). 1. Check if you supports for all eth's, not only eth0. If instance boots with two or more interfaces, it should be able to get all it addresses. 2. Add Gentoo 'element' to disk-ima

Re: [Openstack-operators] 100% CPU and hangs if syslog is restarted

2015-05-29 Thread George Shuklin
On 05/28/2015 07:56 PM, George Shuklin wrote: Hello. Today we've discover a very serious bug in juno: https://bugs.launchpad.net/nova/+bug/1459726 In short: if you're using syslog, and restart rsyslog, all APIs processes will eventually stuck with 100% CPU usage without doing

[Openstack-operators] 100% CPU and hangs if syslog is restarted

2015-05-28 Thread George Shuklin
Hello. Today we've discover a very serious bug in juno: https://bugs.launchpad.net/nova/+bug/1459726 In short: if you're using syslog, and restart rsyslog, all APIs processes will eventually stuck with 100% CPU usage without doing anything. Is anyone hits this bug before? It looks like very

Re: [Openstack-operators] Fw: VM Stuck in Error State

2015-05-22 Thread George Shuklin
Enable debug in nova.conf on compute2 host, restart nova-compute and try again. You should see the reason in the log. It can be bad connection to glance, or problem with networking on the host. On 05/22/2015 12:04 PM, Abhishek Talwar wrote: Hi Folks, I know this is not the place to ask usag

Re: [Openstack-operators] Raising the degree of the scandal

2015-05-22 Thread George Shuklin
On 05/17/2015 04:33 PM, Miguel Ángel Ajo wrote: Probably the solution is not selected to be backported because: * It’s an intrusive change * Introduces new dependencies * Probably it’s going to introduce a performance penalty because eatables is slow. I’m asking in reviews for this feat

Re: [Openstack-operators] Raising the degree of the scandal

2015-05-16 Thread George Shuklin
On 05/15/2015 07:48 PM, Jay Pipes wrote: On 05/15/2015 12:38 PM, George Shuklin wrote: Just to let everyone know: broken antispoofing is not an 'security issue' and the fix is not planned to be backported to Juno/kilo. https://bugs.launchpad.net/bugs/1274034 What can I say? All hai

[Openstack-operators] Raising the degree of the scandal

2015-05-15 Thread George Shuklin
Just to let everyone know: broken antispoofing is not an 'security issue' and the fix is not planned to be backported to Juno/kilo. https://bugs.launchpad.net/bugs/1274034 What can I say? All hail devstack! Who care about production? ___ OpenStack-op

Re: [Openstack-operators] Multiple vlan ranges on same physical interface [ml2]

2015-05-14 Thread George Shuklin
tried... Yes, I can! Thank you. Thanks, Kevin Benton On May 9, 2015 17:16, "George Shuklin" <mailto:george.shuk...@gmail.com>> wrote: Yes, that's result. My plan was to allow 'internal' networks in neutron (by tenants itself), but after some strugg

Re: [Openstack-operators] Multiple vlan ranges on same physical interface [ml2]

2015-05-09 Thread George Shuklin
l segment' for VLAN autoallocation for tenant networks (it just grabs first available). On 05/09/2015 03:08 AM, Kevin Benton wrote: So if you don't let tenants allocate networks, then why do the VLAN ranges in neutron matter? It can just be part of your net-create scripts. O

Re: [Openstack-operators] Multiple vlan ranges on same physical interface [ml2]

2015-05-08 Thread George Shuklin
k_vlan_ranges is 'local:1000:4000', you can still successfully run the following as an admin: neutron net-create --provider:network_type=vlan --provider:physical_network=local --provider:segmentation_id=40 myextnet --router:external On Thu, May 7, 2015 at 7:32 AM, George Shuklin m

Re: [Openstack-operators] Multiple vlan ranges on same physical interface [ml2]

2015-05-07 Thread George Shuklin
On 05/07/2015 06:17 PM, gustavo panizzo (gfa) wrote: On 2015-05-07 22:32, George Shuklin wrote: Hello everyone. Got a problem: we want to use same physical interface for external networks and virtual (tenant) networks. All inside vlans with different ranges. My expected config was: [ml2

[Openstack-operators] Multiple vlan ranges on same physical interface [ml2]

2015-05-07 Thread George Shuklin
Hello everyone. Got a problem: we want to use same physical interface for external networks and virtual (tenant) networks. All inside vlans with different ranges. My expected config was: [ml2] type_drivers = vlan tenant_network_types = vlan [ml2_type_vlan] network_vlan_ranges = external:1:10

Re: [Openstack-operators] expanding to 2nd location

2015-05-05 Thread George Shuklin
Second installation, IMHO. Everything else would make it less robust. On 05/04/2015 11:40 PM, Jonathan Proulx wrote: Hi All, We're about to expand our OpenStack Cloud to a second datacenter. Anyone one have opinions they'd like to share as to what I would and should be worrying about or how to

Re: [Openstack-operators] [neutron] multiple external networks on the same host NIC

2015-04-25 Thread George Shuklin
Can you put them to different vlans? After that it would be very easy task. If not, AFAIK, neutron does not allow this. Or you can trick it thinking it is (are) separate networks. Create brige (br-join), plug eth to it. Create to fake external bridges (br-ex1, br-ex2). Join them together to br

Re: [Openstack-operators] over commit ratios

2015-04-22 Thread George Shuklin
this worse. Tim -Original Message----- From: George Shuklin [mailto:george.shuk...@gmail.com] Sent: 21 April 2015 23:55 To: openstack-operators@lists.openstack.org Subject: Re: [Openstack-operators] over commit ratios It's very depend on production type. If you can control guests and predi

Re: [Openstack-operators] over commit ratios

2015-04-21 Thread George Shuklin
It's very depend on production type. If you can control guests and predict their memory consumption, use it as base for ratio. If you can't (typical for public clouds) - use 1 or smaller with reserved_host_memory_mb in nova.conf. And one more: some swap sapce is really necessary. Add at least

Re: [Openstack-operators] Draft Agenda for the Vancouver Ops Summit Sessions

2015-04-13 Thread George Shuklin
On 04/13/2015 12:32 PM, Tom Fifield wrote: What kind of projects will be a sessions 'Architecture Show and Tell' and 'Architecture Show and Tell - Special Edition' about? Thanks. On 04/13/2015 12:32 PM, Tom Fifield wrote: [cut] _*General Sessions*_ Tuesday Big Room 1 Big

[Openstack-operators] Hundreds of instances per host

2015-04-02 Thread George Shuklin
Hello. Did someone have experience with many-many instances on a single host? What kind of issues you find on '200 instances' (300) borderline? Any specific performance issues, stability? (KVM) I've just start playing with that idea and I see that 150 instances consume significant amount of

Re: [Openstack-operators] Juno Lost connectivity to instance and new instances not getting IP

2015-03-26 Thread George Shuklin
Hello. Simple hack to try: log to the network node and go to network namespace of dhcp agent or router. Use ping6 to ff02::1 (with interface, f.e.: ping6 -I qr-bda2b276-72 ff02::1). If network is fine, you should receive reply from all nodes on the segment (link local ipv6 works without dhcp)

Re: [Openstack-operators] [Neutron][Nova] No Valid Host when booting new VM with Public IP

2015-03-18 Thread George Shuklin
We have that configuration and it works fine. Even better than L3 NAT on neutron routers. Tenant's VM works perfect with external networks and white IPs, but you should make external network available on each compute node (ml2_conf.ini). On 03/18/2015 07:29 PM, Adam Lawson wrote: What I'm try

Re: [Openstack-operators] [Neutron][Nova] No Valid Host when booting new VM with Public IP

2015-03-18 Thread George Shuklin
check out if you allowed nova to use external networks. Somewhere around api-paste.ini, with 'external' in the name of permission. If nova compute rejects binding, it will rejects to start instance, and pass that error to nova-scheduler, and it will return 'NVHF'. On 03/16/2015 10:52 PM, Adam

Re: [Openstack-operators] MTU on router interface (neutron & GRE) without jumbo

2015-03-18 Thread George Shuklin
- 1520 MTU Switches - 1516 MTU OVS - 1500 MTU Result: Failure - Traffic will have issues passing through. So just make sure everything in-between is higher than your server and client. --Joe On Fri, Mar 13, 2015 at 9:28 AM, George Shuklin mailto:george.shuk...@gmail.com>> wrote:

[Openstack-operators] MTU on router interface (neutron & GRE) without jumbo

2015-03-13 Thread George Shuklin
Hello. We've hit badly changes in behaviour of OVS when we switched from 3.08 to 3.13 kernel. When runs on 3.11 or above, OVS starts to use kernel GRE services. And they copy DNF (do not fragment) flag from encapsulated packet to GRE packet. And this mess up all things, because ICMP messages

Re: [Openstack-operators] Is it possible to port mirror to a vm?

2015-02-15 Thread George Shuklin
The answer is 'yes' and 'no'. No, openstack (neutron/nova-networks) have no such abstraction. Yes, you can do it with openvswitch at the compute host manually (until VM reboot). Quote from ovs-vsctl manpage: *Port* *Mirroring* Mirror all packets received or sent on*eth0* or*eth1*

Re: [Openstack-operators] [Ceilometer] Real world experience with Ceilometer deployments - Feedback requested

2015-02-11 Thread George Shuklin
Ceilometer is in sad state. 1. Collector leaks memory. We ran it on same host with mongo, and it grab 29Gb out of 32, leaving mongo with less than gig memory available. 2. Metering agent cause huge load on neutron-server. o(n) of metering rules and tenants. Few bugs reported, one bugfix in revi

Re: [Openstack-operators] How to handle updates of public images?

2015-02-07 Thread George Shuklin
On 02/07/2015 08:36 PM, Igor Bolotin wrote: Going back to the idea of archiving images and not allowing launch of new VMs and hiding archived images by default in Horizon/CLI (maybe still can list/show if requested, possibly admin function only). Would it make sense to propose this as a bluepr

Re: [Openstack-operators] [openstack-dev] [Telco][NFV][infra] Review process of TelcoWG use cases

2015-02-06 Thread George Shuklin
On 02/06/2015 09:14 PM, Marcos Garcia wrote: It does look like that. However, the intent here is to allow non-developer members of a Telco provide the use cases they need to accomplish. This way the Telco WG can identify gaps and file a proper spec into each of the OpenStack projects. Indee

Re: [Openstack-operators] [openstack-dev] [Telco][NFV][infra] Review process of TelcoWG use cases

2015-02-06 Thread George Shuklin
On 02/06/2015 04:21 PM, Steve Gordon wrote: It does look like that. However, the intent here is to allow non-developer members of a Telco provide the use cases they need to accomplish. This way the Telco WG can identify gaps and file a proper spec into each of the OpenStack projects. Indeed, w

Re: [Openstack-operators] [Telco][NFV][infra] Review process of TelcoWG use cases

2015-02-06 Thread George Shuklin
On 02/06/2015 07:52 PM, Paul Belanger wrote: I think it not really possible. If you talking about 'openstack' as 'Openstack developers', may be. But for operators all telco stuff is just completely foreign. I do not understand what they doing and I don't need them for my job. Sorry. Interesti

Re: [Openstack-operators] [Telco][NFV][infra] Review process of TelcoWG use cases

2015-02-06 Thread George Shuklin
On 02/06/2015 04:12 PM, Steve Gordon wrote: - Original Message - From: "George Shuklin" To: openstack-operators@lists.openstack.org Sorry guys. I think most of the ops here have no idea what you talking about. Telcos is telcos, ops is ops. Different worlds, differen

Re: [Openstack-operators] [Telco][NFV][infra] Review process of TelcoWG use cases

2015-02-06 Thread George Shuklin
Sorry guys. I think most of the ops here have no idea what you talking about. Telcos is telcos, ops is ops. Different worlds, different problems, different terminology. On 02/06/2015 01:11 PM, Marc Koderer wrote: Hello everyone, we are currently facing the issue that we don’t know how to proc

Re: [Openstack-operators] How to handle updates of public images?

2015-02-06 Thread George Shuklin
Hello. We're forced to use _base because nova wants them. But disks are raw and _base is just overhead. Migration (we use cold migration with instance shutdown) with deleted images was is broken in havana, but we're using patch from icehouse (see attachment and https://bugs.launchpad.net/no

Re: [Openstack-operators] How to handle updates of public images?

2015-02-05 Thread George Shuklin
8590 I didn't test migrations yet. On 02/05/2015 03:09 PM, George Shuklin wrote: Hello everyone. We are updating our public images regularly (to provide them to customers in up-to-date state). But there is a problem: If some instance starts from image it becomes 'used'. That me

[Openstack-operators] How to handle updates of public images?

2015-02-05 Thread George Shuklin
Hello everyone. We are updating our public images regularly (to provide them to customers in up-to-date state). But there is a problem: If some instance starts from image it becomes 'used'. That means: * That image is used as _base for nova * If instance is reverted this image is used to recre

Re: [Openstack-operators] Deprecation of in tree EC2 API in Nova for Kilo release

2015-01-29 Thread George Shuklin
On 01/28/2015 09:56 PM, Sean Dague wrote: The following review for Kilo deprecates the EC2 API in Nova - https://review.openstack.org/#/c/150929/ There are a number of reasons for this. The EC2 API has been slowly rotting in the Nova tree, never was highly tested, implements a substantially olde

Re: [Openstack-operators] Small openstack

2015-01-29 Thread George Shuklin
lity zones, instance placement, aggregates, etc. On 01/29/2015 01:34 AM, Thomas Goirand wrote: On 12/20/2014 11:16 PM, George Shuklin wrote: do 'network node on compute' is kinda sad Why? Thomas ___ OpenStack-operators mailing list

Re: [Openstack-operators] Packaging sample config versions

2015-01-28 Thread George Shuklin
Yes! Just had have discussion about this with my colleague yesterday. Seems be perfect solution. On 01/28/2015 12:00 AM, Tom Fifield wrote: Hi all, Based on Gustavo's excellent work below, talking with many ops, and after a brief chats with Jeremey and a few other TC folks, here's what I'd pr

Re: [Openstack-operators] RHEL 7 / CentOS 7 instances losing their network gateway

2015-01-27 Thread George Shuklin
How many network interfaces have your instance? If more than one - check settings for second network (subnet). It can have own dhcp settings which may mess up with routes for the main network. On 01/27/2015 06:08 PM, Joe Topjian wrote: Hello, I have run into two different OpenStack clouds whe

Re: [Openstack-operators] :document an OpenStack production environment

2015-01-26 Thread George Shuklin
east management types did not like that they couldn't really edit our documentation. -matt On Mon, Jan 26, 2015 at 5:10 AM, George Shuklin mailto:george.shuk...@gmail.com>> wrote: We using chef to manage hosts. Data bags contains all data of all hosts. We keep hardware configuration an

Re: [Openstack-operators] :document an OpenStack production environment

2015-01-26 Thread George Shuklin
We using chef to manage hosts. Data bags contains all data of all hosts. We keep hardware configuration and DC-wide-name in databags too. For the flowcharts we mostly use markers and whiteboard, sometime I sketch stuff in dia [1] or with wacom tablet in mypaint. [1] http://sourceforge.net/pro

Re: [Openstack-operators] Small openstack (part 2), distributed glance

2015-01-21 Thread George Shuklin
lease? I understand that you have compute —> glance, and glance —> swift. But what’s the third transfer? Thanks! Mike On 1/21/15, 10:36 AM, "George Shuklin" wrote: Ok, news so far: It works like a magic. Nova have option [glance] host=127.0.0.1 And I do not need to cheat wi

Re: [Openstack-operators] Small openstack (part 2), distributed glance

2015-01-21 Thread George Shuklin
U and 10G links. With that solution CPU load on glance-api is distributed evenly on all compute nodes, and overall snapshot traffic (on ports) was cut down 3 times! Why I didn't thought about this earlier? On 01/16/2015 12:20 AM, George Shuklin wrote: Hello everyone. One more thing in the ligh

Re: [Openstack-operators] Small openstack (part 2), distributed glance

2015-01-15 Thread George Shuklin
hursday, January 15, 2015, George Shuklin mailto:george.shuk...@gmail.com>> wrote: Hello everyone. One more thing in the light of small openstack. I really dislike tripple network load caused by current glance snapshot operations. When compute do snapshot, it playing with fi

Re: [Openstack-operators] Lets talk capacity monitoring

2015-01-15 Thread George Shuklin
On 01/15/2015 06:43 PM, Jesse Keating wrote: We have a need to better manage the various openstack capacities across our numerous clouds. We want to be able to detect when capacity of one system or another is approaching the point where it would be a good idea to arrange to increase that capaci

[Openstack-operators] Small openstack (part 2), distributed glance

2015-01-15 Thread George Shuklin
Hello everyone. One more thing in the light of small openstack. I really dislike tripple network load caused by current glance snapshot operations. When compute do snapshot, it playing with files locally, than it sends them to glance-api, and (if glance API is linked to swift), glance sends t

Re: [Openstack-operators] Small openstack

2015-01-12 Thread George Shuklin
Wow. Real problem. I check it - it allows one tenant to interrupt traffic on other. I was not able to intercept TCP traffic, victim lost connection and TCP was struggling with retransmissions, but it was not good. But idea of 'no routers in software' is too appealing. I think I'll stick with

Re: [Openstack-operators] Small openstack

2015-01-09 Thread George Shuklin
On 01/09/2015 09:25 PM, Kris G. Lindgren wrote: > Also, If you are running this configuration you should be aware of the > following bug: > > https://bugs.launchpad.net/neutron/+bug/1274034 > > And the corresponding fix: https://review.openstack.org/#/c/141130/ > > Basically - Neutron security grou

Re: [Openstack-operators] glance directory traversal bug and havana

2015-01-09 Thread George Shuklin
Seems I was wrong. Thanks, I'll look at it again. On 01/08/2015 07:37 PM, Jesse Keating wrote: On 1/7/15 8:47 PM, George Shuklin wrote: I spend few hours trying to backport to Havana, but than I found, that Havana seems be immune to the bug. I'm not 100% sure, so someone else

Re: [Openstack-operators] glance directory traversal bug and havana

2015-01-07 Thread George Shuklin
I spend few hours trying to backport to Havana, but than I found, that Havana seems be immune to the bug. I'm not 100% sure, so someone else advised to look too. The bug was that icehouse+ accepts all supported schemas. Fix excludes 'bad' schemes. Although Havana have explicitly given list of ac

Re: [Openstack-operators] Small openstack

2014-12-25 Thread George Shuklin
It may be not so great from point of view of 'bleeding edge' part of openstack (heat/LB/etc), but I do not want my infrastructure to bleed... I love that thing and I to stick to it. Updates will follow. On 12/22/2014 11:35 PM, George Shuklin wrote: Thank you for everyone! After s

Re: [Openstack-operators] Small openstack

2014-12-22 Thread George Shuklin
some VLAN/VXLANs with few external networks (per tenant). Tenants will have no 'routers' and 'floatingips', but still will have DHCP and other yummy neutron things like private networks with overlapping numbering plans. Future reports follow. On 12/21/2014 12:16 AM,

Re: [Openstack-operators] Small openstack

2014-12-21 Thread George Shuklin
On 12/21/2014 10:42 AM, Xav Paice wrote: On 21/12/14 11:16, George Shuklin wrote: Hello. I've suddenly got request for small installation of openstack (about 3-5 computes). They need almost nothing (just a management panel to span simple instances, few friendly tennants), and I curiou

Re: [Openstack-operators] Small openstack

2014-12-20 Thread George Shuklin
o be removed in the next release. so yes it will work, and it may even be optimal for this size deployment, but it will NOT be at all future proofed. I'd go with neutron as of now. -matt On Sat, Dec 20, 2014 at 5:16 PM, George Shuklin mailto:george.shuk...@gmail.com>> wr

[Openstack-operators] Small openstack

2014-12-20 Thread George Shuklin
Hello. I've suddenly got request for small installation of openstack (about 3-5 computes). They need almost nothing (just a management panel to span simple instances, few friendly tennants), and I curious, is nova-network good solution for this? They don't want network node and do 'network n

Re: [Openstack-operators] Packaging sample config versions

2014-12-15 Thread George Shuklin
Oh! Ubuntu's guy. Can you say why you didn't ship 2013.2.4? It was available in announced support lifecycle for cloudarchive, and you just stops to do anything with 2013.2.3 (havana) somewhere in the middle of the summer. It was really bad, because few important fixes for neutron were landed

Re: [Openstack-operators] Packaging sample config versions

2014-12-15 Thread George Shuklin
On 12/15/2014 10:49 AM, Thomas Goirand wrote: and ubuntu just put files in proper places without changing configs. Ahem... Ubuntu simply doesn't care much about config files. See what they ship for Nova and Cinder. I wouldn't say "without changing configs" in this case. We using chef for confi

Re: [Openstack-operators] Packaging sample config versions

2014-12-13 Thread George Shuklin
On 12/14/2014 01:27 AM, Jeremy Stanley wrote: On 2014-12-14 00:39:41 +0200 (+0200), George Shuklin wrote: [...] debian heavily relies on answers to debconfig, and ubuntu just put files in proper places without changing configs. We using chef for configuration, so ubuntu approach is better (when

Re: [Openstack-operators] Packaging sample config versions

2014-12-13 Thread George Shuklin
On 12/13/2014 05:13 PM, Thomas Goirand wrote: If I can help somehow, I'm ready to do something, but What should I do, exactly? There's a lot that can be done. If you like working on CI stuff, then you could help me with building the package validation CI which I'm trying to (re-)work. All of

Re: [Openstack-operators] Packaging sample config versions

2014-12-12 Thread George Shuklin
Real hard work, man. Thanks. I think the problem is that most of ops does not know enough about whole stuff needed. I do some tiny CI in my company: repackaging ubuntu packages with debian-jenkins-glue (plus backported patches icehouse->havana), but I can't say I understand all stuff happens

Re: [Openstack-operators] Operations project: Packaging

2014-11-30 Thread George Shuklin
Nice page. Is jenkins-debian-glue with ubuntu './debian' part is counted as valid way to package openstack? They even have separate repo for patches and debian-specific files (like control). On 11/30/2014 04:38 PM, Michael Chapman wrote: I've started a wiki page here, please feel free to fles

Re: [Openstack-operators] Operations project: Packaging

2014-11-27 Thread George Shuklin
On Thu, Nov 20, 2014 at 8:51 AM, Craig Tracey > wrote: Great input Kris. We also took a look at Anvil, and as you mention it is heavily biased for RH based distros. With regard to your requirements: 1) Under the cover for Giftwrap we use fpm for p

Re: [Openstack-operators] neutron + ceilometer = memory-hungry neutron server

2014-11-26 Thread George Shuklin
Thanks, reported: https://bugs.launchpad.net/neutron/+bug/1396677 (and I think it's a neutron bug, because it affects neutron-server) On 11/26/2014 04:33 PM, Kyle Mestery wrote: Hello. We using ceilometer to meter tenants traffic. We have some complicated list of exclusion list (unlimited traf

[Openstack-operators] neutron + ceilometer = memory-hungry neutron server

2014-11-24 Thread George Shuklin
Hello. We using ceilometer to meter tenants traffic. We have some complicated list of exclusion list (unlimited traffic to our own networks). We've got about 100k of metering rules in all tenants, and neutron-server starts to eat memory as mad (900Mb+ per each worker). I've dumped memory of

Re: [Openstack-operators] cloud-init: ssh-keys for host changed after global reboot

2014-11-15 Thread George Shuklin
a bug in an older version of cloud-init that may have caused this so knowing the cloud-init version would be good to. Thanks, Josh On Nov 13, 2014, at 6:06 PM, George Shuklin wrote: Hello. We had planned power outage for one of our OS installation (havana). After everything booted back, we

Re: [Openstack-operators] cloud-init: ssh-keys for host changed after global reboot

2014-11-14 Thread George Shuklin
that myself, I wonder if there is a conflict between cloud-init and libvirt_inject_key. Also curious if you're using the metadata api or config_drive. On Thursday, November 13, 2014, George Shuklin mailto:george.shuk...@gmail.com>> wrote: Hello. We had planned power outag

[Openstack-operators] cloud-init: ssh-keys for host changed after global reboot

2014-11-13 Thread George Shuklin
Hello. We had planned power outage for one of our OS installation (havana). After everything booted back, we found every instance has change it's own ssh key (server key ssh-server presents upon connection). Is this bug or feature? Someone saw that? Is any way to prevent this? Thanks! _

Re: [Openstack-operators] floatin ip issue

2014-10-31 Thread George Shuklin
quot;} | | 0189699c-8ffc-44cb-aebc-054c8d6001ee | ext-subnet | 192.168.122.0/24 <http://192.168.122.0/24> | {"start": "192.168.122.193", "end": "192.168.122.222"} | +--+-+--+--------+ P.S: External subnet is 192

Re: [Openstack-operators] floatin ip issue

2014-10-27 Thread George Shuklin
packets:1719 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:27377 (27.3 KB) TX bytes:164541 (164.5 KB) -- Thanks Paras. On Sat, Oct 25, 2014 at 3:18 AM, George Shuklin mailto:george.shuk...@gmail.com>> wrote: Check out if qrouter go

Re: [Openstack-operators] floatin ip issue

2014-10-25 Thread George Shuklin
Check out if qrouter got floating inside network namespace (ip net exec qrouter... ip -4 a), or just bash in to it (stdbuf -e0 -o0 ip net exec qrouter... /bin/bash) and play with it like with normal server. On 10/24/2014 07:38 PM, Paras pradhan wrote: Hello, Assigned a floating ip to an ins

Re: [Openstack-operators] Disable neutron agents

2014-10-20 Thread George Shuklin
On 10/20/2014 05:30 PM, Christian Berendt wrote: On 10/20/2014 02:48 PM, George Shuklin wrote: How to disable agents in neutron? This should be possible with "neutron agent-update --admin-state-up False AGENT". You can list all available agents with "neutron agent-list&qu

[Openstack-operators] Disable neutron agents

2014-10-20 Thread George Shuklin
Hello. I can't find any option for neutron to disable agents (like nova service-disable). Now I just shutdown unwanted agents (service stop on network node). But if node reboot they will come back, and this is not really welcome. How to disable agents in neutron? Thanks. _

Re: [Openstack-operators] [glance] how to update the contents of an image

2014-10-09 Thread George Shuklin
uses behind the scenes, so it makes sense that it would fail as well. On Wed, Oct 8, 2014 at 8:30 PM, George Shuklin mailto:george.shuk...@gmail.com>> wrote: We using Havana, with raw images and non-live migration (rsync/scp on halted VMs). Migration happens, but during ins

Re: [Openstack-operators] [glance] how to update the contents of an image

2014-10-08 Thread George Shuklin
gt;>>>> >>>>>>> --- >>>>>>> Best regards >>>>>>> Sławek Kapłoński >>>>>>> sla...@kaplonski.pl >>>>>>> >>>>>>> Dnia wtorek, 7 października 2014 10:05:57 Abel Lopez pisze: >>&

Re: [Openstack-operators] [glance] how to update the contents of an image

2014-10-07 Thread George Shuklin
the new image using the desired name (e.g. Fedora-20 or like Fedora-20-LATEST ) Obviously I assume there was testing for viability of the image before it was uploaded to glance. For more information, be sure to catch my talk on Tuesday 9am at the summit. On Oct 7, 2014, at 9:58 AM, George Shu

Re: [Openstack-operators] [glance] how to update the contents of an image

2014-10-07 Thread George Shuklin
As far as I know, it is not possible to assign uuid from deleted image to the new one, because deleted images keeps their metadata in DB. On 09/26/2014 04:43 PM, Abel Lopez wrote: Glance images are immutable. In order to update it, you should do as you are doing, but then rename the old image,

Re: [Openstack-operators] Request for feedback on DHCP IP usage

2014-10-07 Thread George Shuklin
Single IP per DHCP is nice. And move dhcp agent away from network node gives important thing: you can create isolated tenant networks without headache with dhcp-agents scheduling. Neutron does not supports now AZ from nova, and DHCP is just yet another thing to mess up with isolation. Some i

Re: [Openstack-operators] Migration from openvswitch to linuxbridge

2014-09-30 Thread George Shuklin
For high CPU usage for openvswitchd there is a simple solution: upgrade to anything like 1.11 or higher. There is a huge problem with ovs 1.4, 1.9, 1.10 - they suck at real-world network activity. All new version (2.0, 2.1) are much better and works great. You can find more detail in google wi

  1   2   >