Re: [openstack-dev] [Nova] Nova compute reports the disk usage

2016-08-04 Thread Chris Friesen
On 08/04/2016 02:42 AM, Tuan Luong wrote: Hi Chris, Yes we used qcow image. However I still have something not clear. What I know is that: - disk_available_least = free_disk - disk_over_commit - disk_over_commit = int(virt_size) - physical_disk_size Before launching: - free_disk = 1025 -

Re: [openstack-dev] [Nova] Does this api modification need Microversion?

2016-08-03 Thread Chris Friesen
On 08/03/2016 10:53 AM, Andrew Laski wrote: I think the discussion about whether or not this needs a microversion is missing the bigger question of whether or not this should be in the API to begin with. I don't think it's actually in the API. If it's safe to rollback from this error state

Re: [openstack-dev] [Nova] Nova compute reports the disk usage

2016-08-03 Thread Chris Friesen
On 08/03/2016 09:13 AM, Tuan Luong wrote: Hi Jay, Thank you for the reply. Actually, I did ask this question to make sure that everything is still going well with disk usage report of Nova. We had the problem related to wrong report (IMHO it is wrong). The situation is below: - We launch an

Re: [openstack-dev] [nova] Removal of live_migration_flag and block_migration_flag config options

2016-08-02 Thread Chris Friesen
On 08/02/2016 09:14 AM, Timofei Durakov wrote: Hi, Taking into account everything above I'd prefer to see live_migration_tunnelled(that corresponds to VIR_MIGRATE_TUNNELLED) defaulted to False. We just need to make a release note for this change, and on the host startup do LOG.warning to notify

Re: [Openstack-operators] nova resize on shared storage

2016-07-28 Thread Chris Friesen
On 07/28/2016 02:34 AM, Marcus Furlong wrote: Hi, I've been trying to find some information about using nova resize on shared storage, without giving each compute node ssh access to every other compute node. As the VM is on shared storage, the compute node shouldn't need ssh access to another

Re: [openstack-dev] [Kolla] [Fuel] [tc] Looks like Mirantis is getting Fuel CCP (docker/k8s) kicked off

2016-07-27 Thread Chris Friesen
On 07/27/2016 09:59 AM, Ed Leafe wrote: On Jul 27, 2016, at 10:51 AM, Joshua Harlow wrote: Whether to have competing projects in the big tent was debated by the TC at the time and my recollection is that we decided that was a good thing -- if someone wanted to develop a

Re: [Openstack] Anti-Affinity Based on NUMA Node

2016-07-27 Thread Chris Friesen
Host aggregates apply to the host as a whole, not individual NUMA nodes. Chris On 07/27/2016 09:39 AM, Fawaz Mohammed wrote: Hi Brent, Consider aggregation group. On Jul 27, 2016 3:48 AM, "Brent Troge" > wrote: Instead of

Re: [Openstack] Anti-Affinity Based on NUMA Node

2016-07-27 Thread Chris Friesen
On 07/26/2016 05:33 PM, Brent Troge wrote: Instead of anti-affinity based on compute host, is there a way to filter based on NUMA node ? In my flavor I am using hw_numa_nodes=1 However, for the VMs within the same server group, I would like to ensure they do not end up in the same host NUMA

Re: [openstack-dev] FPGA as a dynamic nested resources

2016-07-19 Thread Chris Friesen
On 07/19/2016 01:40 PM, Ed Leafe wrote: On Jul 19, 2016, at 11:03 AM, Roman Dobosz wrote: It can identified 3 levels of FPGA resources, which can be nested one on the others: 1. Whole FPGA. If used discrete FPGA, than even today it might be pass through to the VM.

Re: [Openstack] add existing instance to nova scheduler server group

2016-07-18 Thread Chris Friesen
On 07/15/2016 06:03 PM, Frank Ritchie wrote: Hi all, Is it possible with the OpenStack or Nova CLI to add an existing instance to an existing Nova Scheduler server group? No. This has been proposed a number of times over the years but has never been accepted. I believe the problem is

Re: [openstack-dev] [Nova] Regarding fix for 1454165

2016-07-13 Thread Chris Friesen
On 07/13/2016 04:12 PM, Sean Wilcox wrote: With this bug fix the resize_instance definition was effectively changed that is passed into the virt drivers finish_migration() function, IMHO. Agreed. Is it really true that changing memory and/or cpu isn't a resize at all? No, it's definitely a

Re: [openstack-dev] [nova] focused review pipeline of bug fix changes?

2016-07-12 Thread Chris Friesen
On 07/12/2016 01:59 AM, Markus Zoeller wrote: After closing the old (>18months) bug reports nobody is working on a few days ago [1], it became clear that the "in progress" reports are the majority [2]. After asking Gerrit how long it usually takes to get a fix merged [3], this is the result:

Re: [openstack-dev] Neutron and MTU advertisements -- post newton

2016-07-11 Thread Chris Friesen
On 07/11/2016 10:39 AM, Jay Pipes wrote: Out of curiosity, in what scenarios is it better to limit the instance's MTU to a value lower than that of the maximum path MTU of the infrastructure? In other words, if the infrastructure supports jumbo frames, why artificially limit an instance's MTU

Re: [openstack-dev] [nova] gate testing with lvm imagebackend

2016-07-04 Thread Chris Friesen
On 05/10/2016 04:47 PM, Matt Riedemann wrote: On 5/10/2016 5:14 PM, Chris Friesen wrote: For what it's worth, we've got internal patches to enable cold migration and resize for LVM-backed instances. We've also got a proof of concept to enable thin-provisioned LVM to get rid of the huge wipe

Re: [openstack-dev] [Nova] Questions about instance actions' update and finish

2016-06-30 Thread Chris Friesen
On 06/29/2016 09:13 PM, Edward Leafe wrote: On Jun 29, 2016, at 10:05 PM, Matt Riedemann wrote: 2. The updated_at field is also empty, should we sync the updated_at time to the created_at time when we create the action and also update it whenever the action status

Re: [Openstack] Using multiple compute drivers in Nova?

2016-06-27 Thread Chris Friesen
On 06/27/2016 09:31 AM, Turbo Fredriksson wrote: On Jun 27, 2016, at 4:07 PM, Chris Friesen wrote: ow about running two containers on your host, with one nova-compute in each? And those two is configured as docker and kvm respectively? And in those containers i run either a container

Re: [Openstack] Using multiple compute drivers in Nova?

2016-06-27 Thread Chris Friesen
On 06/24/2016 05:47 PM, Turbo Fredriksson wrote: On Jun 25, 2016, at 12:22 AM, Clint Byrum wrote: Excerpts from Turbo Fredriksson's message of 2016-06-24 22:50:40 +0100: The page http://docs.openstack.org/mitaka/config-reference/compute/hypervisors.html states: Most installations use only

Re: [openstack-dev] [cinder] No middle-man - when does/will Nova directly connect iSCSI volumes?

2016-06-23 Thread Chris Friesen
On 06/23/2016 10:09 AM, Walter A. Boring IV wrote: volumes connected to QEMU instances eventually become directly connected? Our long term goal is that 100% of all network storage will be connected to directly by QEMU. We already have the ability to partially do this with iSCSI, but it is

Re: [openstack-dev] [nova] ability to set metadata on instances (but config drive is not updated)

2016-06-21 Thread Chris Friesen
On 06/19/2016 11:57 AM, Clint Byrum wrote: Excerpts from Joshua Harlow's message of 2016-06-17 15:04:38 -0700: Now if I am using the configdrive instead of the metadata server at that special/magic ip that same metadata never seems to change (I assume the configdrive would have to be

Re: [openstack-dev] [all] what do you work on upstream of OpenStack?

2016-06-14 Thread Chris Friesen
On 06/13/2016 01:11 PM, Doug Hellmann wrote: I'm trying to pull together some information about contributions that OpenStack community members have made *upstream* of OpenStack, via code, docs, bug reports, or anything else to dependencies that we have. If you've made a contribution of that

Re: [openstack-dev] [nova] theoretical race between live migration and resource audit?

2016-06-14 Thread Chris Friesen
it runs via its periodic task. Matt (not a scheduler person) On Thu, Jun 9, 2016 at 10:41 PM, Chris Friesen <chris.frie...@windriver.com <mailto:chris.frie...@windriver.com>> wrote: Hi, I'm wondering if we might have a race between live migration and the resource audit. I

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

2016-06-14 Thread Chris Friesen
On 06/13/2016 02:17 PM, Paul Michali wrote: Hmm... I tried Friday and again today, and I'm not seeing the VMs being evenly created on the NUMA nodes. Every Cirros VM is created on nodeid 0. I have the m1/small flavor (@GB) selected and am using hw:numa_nodes=1 and hw:mem_page_size=2048

[openstack-dev] [nova] theoretical race between live migration and resource audit?

2016-06-09 Thread Chris Friesen
Hi, I'm wondering if we might have a race between live migration and the resource audit. I've included a few people on the receiver list that have worked directly with this code in the past. In _update_available_resource() we have code that looks like this: instances =

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

2016-06-09 Thread Chris Friesen
On 06/09/2016 05:15 AM, Paul Michali wrote: 1) On the host, I was seeing 32768 huge pages, of 2MB size. Please check the number of huge pages _per host numa node_. 2) I changed mem_page_size from 1024 to 2048 in the flavor, and then when VMs were created, they were being evenly assigned to

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

2016-06-08 Thread Chris Friesen
On 06/03/2016 12:03 PM, Paul Michali wrote: Thanks for the link Tim! Right now, I have two things I'm unsure about... One is that I had 1945 huge pages left (of size 2048k) and tried to create a VM with a small flavor (2GB), which should need 1024 pages, but Nova indicated that it wasn't able

Re: [openstack-dev] Reasoning behind my vote on the Go topic

2016-06-08 Thread Chris Friesen
On 06/07/2016 04:26 PM, Ben Meyer wrote: On 06/07/2016 06:09 PM, Samuel Merritt wrote: On 6/7/16 12:00 PM, Monty Taylor wrote: [snip] I'd rather see us focus energy on Python3, asyncio and its pluggable event loops. The work in: http://magic.io/blog/uvloop-blazing-fast-python-networking/ is

Re: [openstack-dev] [nova][scheduler] bug in handling of ISOLATE thread policy

2016-06-08 Thread Chris Friesen
On 06/07/2016 11:36 AM, Chris Friesen wrote: Hi, The full details are available at https://bugs.launchpad.net/nova/+bug/1590091 but the short version is this: 1) I'm running stable/mitaka in devstack. I've got a small system with 2 pCPUs, both marked as available for pinning. They're two

[openstack-dev] [nova][scheduler] bug in handling of ISOLATE thread policy

2016-06-07 Thread Chris Friesen
Hi, The full details are available at https://bugs.launchpad.net/nova/+bug/1590091 but the short version is this: 1) I'm running stable/mitaka in devstack. I've got a small system with 2 pCPUs, both marked as available for pinning. They're two cores of a single processor, no threads. 2)

Re: [openstack-dev] [nova] Intel NFV CI failing all shelve/unshelve tests

2016-05-25 Thread Chris Friesen
On 05/22/2016 05:41 PM, Jay Pipes wrote: Hello Novaites, I've noticed that the Intel NFV CI has been failing all test runs for quite some time (at least a few days), always failing the same tests around shelve/unshelve operations. I looked through the conductor and compute logs to see if I

Re: [Openstack-operators] [openstack-dev] [nova] Is verification of images in the image cache necessary?

2016-05-24 Thread Chris Friesen
On 05/24/2016 09:54 AM, Dan Smith wrote: I like the idea of checking the md5 matches before each boot, as it mirrors the check we do after downloading from glance. Its possible thats very unlikely to spot anything that shouldn't already be worried about by something else. It may just be my love

Re: [Openstack-operators] [openstack-dev] [openstack-operators][cinder] max_concurrent_builds in Cinder

2016-05-24 Thread Chris Friesen
On 05/23/2016 08:46 PM, John Griffith wrote: On Mon, May 23, 2016 at 8:32 AM, Ivan Kolodyazhny > wrote: Hi developers and operators, I would like to get any feedback from you about my idea before I'll start work on spec. In Nova, we've

Re: [Openstack] KVM & Hyper-V Performance

2016-05-12 Thread Chris Friesen
On 05/12/2016 08:52 AM, gulseren bulut wrote: Hello, I done some tests for windows virtual server on KVM and Hyper-V. All resources of VMs is same. VM OS: Windows Server 2012 R2 SQL: MSSQL Server 2012 R2 4 GB RAM, 2 vCPUs (virtio is used for guestos on KVM) This is DB tpc test using

Re: [Openstack-operators] User Survey usage of QEMU (as opposed to KVM) ?

2016-05-11 Thread Chris Friesen
On 05/11/2016 01:29 PM, Robert Starmer wrote: I don't disagree, what we're really getting at is that any lookup (ask the system what it's using on a particular instance, look at the config, look at the output of a nova CLI request, querry via Horizon), should all return the same answer. So one

Re: [Openstack-operators] User Survey usage of QEMU (as opposed to KVM) ?

2016-05-11 Thread Chris Friesen
On 05/11/2016 11:46 AM, Ronald Bradford wrote: I have been curious as to why as mentioned in the thread virt_type=kvm, but os-hypervisors API call states QEMU. Arguably in both cases the hypervisor is qemu. When virt_type=kvm we simply enable some additional acceleration. So rather than

Re: [openstack-dev] [nova] gate testing with lvm imagebackend

2016-05-11 Thread Chris Friesen
On 05/10/2016 04:47 PM, Matt Riedemann wrote: On 5/10/2016 5:14 PM, Chris Friesen wrote: On 05/10/2016 03:51 PM, Matt Riedemann wrote: For the libvirt imagebackend refactor that mdbooth is working on, I have a POC devstack-gate change which runs with the lvm imagebackend in the libvirt driver

Re: [openstack-dev] [nova] gate testing with lvm imagebackend

2016-05-10 Thread Chris Friesen
On 05/10/2016 03:51 PM, Matt Riedemann wrote: For the libvirt imagebackend refactor that mdbooth is working on, I have a POC devstack-gate change which runs with the lvm imagebackend in the libvirt driver [1]. The test results are mostly happy except for anything related to migrate (including

Re: [openstack-dev] [tc] supporting Go

2016-05-10 Thread Chris Friesen
On 05/10/2016 12:10 PM, Hayes, Graham wrote: The way this component works makes it quite difficult to make any major improvement. MiniDNS (the component) takes data and sends a zone transfer every time a recordset gets updated. That is a full (AXFR) zone transfer, so every record in the zone

Re: [openstack-dev] [Nova] Live Migration: Austin summit update

2016-05-03 Thread Chris Friesen
On 05/03/2016 03:14 AM, Daniel P. Berrange wrote: There are currently many options for live migration with QEMU that can assist in completion Given this I've spent the last week creating an automated test harness for QEMU upstream which triggers migration with an extreme guest CPU load and

[Openstack-operators] [nova][scheduler] please review nova sched logging proposal

2016-05-03 Thread Chris Friesen
Hi all, There's a proposal for improving the nova scheduler logs up at https://review.openstack.org/#/c/306647/ If you would like to be able to more easily determine why no valid host was found, please review the proposal and leave feedback. Thanks, Chris

Re: [openstack-dev] [nova] Migration shared storage negotiation

2016-03-07 Thread Chris Friesen
On 03/07/2016 09:23 AM, Matthew Booth wrote: Timofey has posted a patch which aims to improve shared storage negotiation a little: https://review.openstack.org/#/c/286745/ I've been thinking for some time about going bigger. It occurs to me that the destination node has enough context to know

Re: [openstack-dev] [nova][cinder] Limits on volume read throughput?

2016-03-07 Thread Chris Friesen
ding tests, and collecting measures... On Thu, Mar 3, 2016 at 11:28 AM, Chris Friesen <chris.frie...@windriver.com <mailto:chris.frie...@windriver.com>> wrote: On 03/03/2016 01:13 PM, Preston L. Bannister wrote: > Scanning the same volume from within the inst

[Openstack-operators] Upstream linux kernel bug if CONFIG_VIRT_CPU_ACCOUNTING_GEN enabled in guest

2016-03-06 Thread Chris Friesen
Hi all, Just thought I'd mention that if anyone has been seeing odd idle/system/user results in /proc/stats or "top" in a guest with CONFIG_VIRT_CPU_ACCOUNTING_GEN enabled (it's automatically selected by CONFIG_NO_HZ_FULL) that it's not your imagination or anything you did wrong. I recently

Re: [openstack-dev] [nova] Non-Admin user can show deleted instances using changes-since parameter when calling list API

2016-03-04 Thread Chris Friesen
On 03/04/2016 03:42 PM, Matt Riedemann wrote: On 3/3/2016 9:14 PM, Zhenyu Zheng wrote: Hm, I found out the reason: https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L1139-L1145 here we filtered out parameters like "deleted", and that's why the API behavior

Re: [Openstack-operators] libvirt cpu type per instance?

2016-03-03 Thread Chris Friesen
On 03/03/2016 03:20 PM, Kris G. Lindgren wrote: I would be curious if specifing the cpu type would actually restrict performance. As far as I know, this only restricts the cpu features presented to a vm. You can present a vm that has the cpu instruction sets of a Pentium 3 – but runs and is as

Re: [openstack-dev] [nova][cinder] Limits on volume read throughput?

2016-03-03 Thread Chris Friesen
On 03/03/2016 01:13 PM, Preston L. Bannister wrote: > Scanning the same volume from within the instance still gets the same > ~450MB/s that I saw before. Hmmm, with iSCSI inbetween that could be the TCP memcpy limitation. Measuring iSCSI in isolation is next on my list. Both on

Re: [openstack-dev] [all] A proposal to separate the design summit

2016-03-01 Thread Chris Friesen
On 03/01/2016 03:52 PM, Clint Byrum wrote: Excerpts from Eoghan Glynn's message of 2016-03-01 02:08:00 -0800: There are a whole slew of folks who work fulltime on OpenStack but contribute mainly in the background: operating clouds, managing engineering teams, supporting customers, designing

Re: [openstack-dev] [all] A proposal to separate the design summit

2016-03-01 Thread Chris Friesen
On 03/01/2016 09:03 AM, Anita Kuno wrote: On 03/01/2016 05:08 AM, Eoghan Glynn wrote: In Vancouver I happened to be sitting behind someone who stated "I'm just here for the buzz." Which is lovely for that person. The problem is that the buzz that person is there for is partially created by me

Re: [openstack-dev] [nova] nova hooks - document & test or deprecate?

2016-02-29 Thread Chris Friesen
On 02/29/2016 12:22 PM, Daniel P. Berrange wrote: There's three core scenarios for hooks 1. Modifying some aspect of the Nova operation 2. Triggering an external action synchronously to some Nova operation 3. Triggering an external action asynchronously to some Nova operation The Rdo

Re: [openstack-dev] [nova] solving API case sensitivity issues

2016-02-24 Thread Chris Friesen
On 02/24/2016 02:34 PM, Sean Dague wrote: There are no urls stored here. This is content coming through the body. While I do appreciate many folks weighing in that haven't read the bug yet, I would be even better if people did read the bug first. We have the ability to decide at the API level

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

2016-02-23 Thread Chris Friesen
On 02/22/2016 03:23 AM, Sylvain Bauza wrote: See, I don't think that a compute node unable to start a request is an 'abnormal case'. There are many reasons why a request can't be honored by the compute node : - for example, the scheduler doesn't own all the compute resources and thus can

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

2016-02-23 Thread Chris Friesen
On 02/21/2016 01:56 PM, Jay Pipes wrote: Yingxin, sorry for the delay in responding to this thread. My comments inline. On 02/17/2016 12:45 AM, Cheng, Yingxin wrote: To better illustrate the differences between shared-state, resource-provider and legacy scheduler, I’ve drew 3 simplified

Re: [openstack-dev] [nova] [all] Excessively high greenlet default + excessively low connection pool defaults leads to connection pool latency, timeout errors, idle database connections / workers

2016-02-23 Thread Chris Friesen
On 02/23/2016 05:25 AM, Roman Podoliaka wrote: So looks like it's two related problems here: 1) the distribution of load between workers is uneven. One way to fix this is to decrease the default number of greenlets in pool [2], which will effectively cause a particular worker to give up new

Re: [openstack-dev] Do we need lock fencing?

2016-02-22 Thread Chris Friesen
On 02/19/2016 06:05 PM, Joshua Harlow wrote: Hi all, After reading over the following interesting article about redis and redlock (IMHO it's good overview of distributed locking in general):

Re: [openstack-dev] [nova] nova-compute blocking main thread under heavy disk IO

2016-02-22 Thread Chris Friesen
On 02/22/2016 11:20 AM, Daniel P. Berrange wrote: On Mon, Feb 22, 2016 at 12:07:37PM -0500, Sean Dague wrote: On 02/22/2016 10:43 AM, Chris Friesen wrote: But the fact remains that nova-compute is doing disk I/O from the main thread, and if the guests push that disk hard enough then nova

Re: [openstack-dev] [nova] nova-compute blocking main thread under heavy disk IO

2016-02-22 Thread Chris Friesen
On 02/22/2016 11:17 AM, Jay Pipes wrote: On 02/22/2016 10:43 AM, Chris Friesen wrote: Hi all, We've recently run into some interesting behaviour that I thought I should bring up to see if we want to do anything about it. Basically the problem seems to be that nova-compute is doing disk I/O

[openstack-dev] [nova] nova-compute blocking main thread under heavy disk IO

2016-02-22 Thread Chris Friesen
Hi all, We've recently run into some interesting behaviour that I thought I should bring up to see if we want to do anything about it. Basically the problem seems to be that nova-compute is doing disk I/O from the main thread, and if it blocks then it can block all of nova-compute (since all

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

2016-02-18 Thread Chris Friesen
On 02/17/2016 06:59 AM, Chris Dent wrote: The advantage of a centralized datastore for that information is that it provides administrative control (e.g. reserving resources for other needs) and visibility. That level of command and control seems to be something people really want

Re: [openstack-dev] [nova] ram size and huge page number for VM

2016-02-09 Thread Chris Friesen
On 02/09/2016 04:20 AM, Daniel P. Berrange wrote: It not actually a huge waste - in fact it is likely to be beneficial to the overall performance of your VM. You need to remember that there is no direct relationship between allocation of huge pages to KVM and usage of huge pages inside the

Re: [openstack-dev] [nova] ram size and huge page number for VM

2016-02-08 Thread Chris Friesen
On 02/08/2016 01:47 PM, Serguei Bezverkhi (sbezverk) wrote: Hello, I using the latest Liberty and I am trying to bring up a VM with some numa options configured using flavor. Specifically I need to give this VM 16GB of RAM and in addition it will need to use 12x1GB huge pages. What I found

Re: [openstack-dev] [nova][libvirt] VIR_MIGRATE_NON_SHARED_INC works more like full copy in block migration ?

2016-01-13 Thread Chris Friesen
On 01/12/2016 05:35 AM, Kashyap Chamarthy wrote: On Sun, Jan 10, 2016 at 06:07:33PM +0800, Luo Gangyi wrote: Hi devs, Do you test the difference between within and without VIR_MIGRATE_NON_SHARED_INC ? When I add VIR_MIGRATE_NON_SHARED_INC in block_migration_flags in nova, nova block

Re: [openstack-dev] [nova] [all] Excessively high greenlet default + excessively low connection pool defaults leads to connection pool latency, timeout errors, idle database connections / workers

2016-01-08 Thread Chris Friesen
On 01/07/2016 06:55 PM, Mike Bayer wrote: On 01/07/2016 11:02 AM, Sean Dague wrote: On 01/07/2016 09:56 AM, Brant Knudson wrote: On Thu, Jan 7, 2016 at 6:39 AM, Clayton O'Neill > wrote: On Thu, Jan 7, 2016 at 2:49 AM, Roman Podoliaka

Re: [openstack-dev] [nova] [all] Excessively high greenlet default + excessively low connection pool defaults leads to connection pool latency, timeout errors, idle database connections / workers

2016-01-08 Thread Chris Friesen
On 01/07/2016 05:44 PM, Mike Bayer wrote: On 01/07/2016 07:39 AM, Clayton O'Neill wrote: On Thu, Jan 7, 2016 at 2:49 AM, Roman Podoliaka wrote: Linux gurus please correct me here, but my understanding is that Linux kernel queues up to $backlog number of connections

Re: [openstack-dev] [nova] [all] Excessively high greenlet default + excessively low connection pool defaults leads to connection pool latency, timeout errors, idle database connections / workers

2016-01-08 Thread Chris Friesen
On 01/07/2016 09:49 AM, Roman Podoliaka wrote: Actually we already do that in the parent process. The parent process: 1) starts and creates a socket 2) binds the socket and calls listen() on it passing the backlog value (http://linux.die.net/man/2/listen) 3) passes the socket to the eventlet

Re: [Openstack] Looking for Neutron Python code

2015-12-21 Thread Chris Friesen
On 12/21/2015 12:12 PM, Atif Saeed wrote: I am searching for python code of " visualizing OpenStack Networking Service Traffic in the cloud" Chapter 12 Troubleshooting. Can anyone help please by sharing the link of python code of Neutron architecture. I shall be extremely grateful. Are you

Re: [Openstack] cpu pinning in realtime

2015-12-16 Thread Chris Friesen
On 12/16/2015 03:17 PM, Satish Patel wrote: Hello, I am running openstack JUNO and created instance and later i did manually set pinning on compute node like following virsh vcpupin instance-0053 0 0 virsh vcpupin instance-0053 1 1 Does it work? or we need to reboot instance etc?

Re: [Openstack] CPU pinning question

2015-12-16 Thread Chris Friesen
On 12/15/2015 12:23 PM, Satish Patel wrote: If i enable "NUMATopologyFilter", Does JUNO support pinning? FYI, i am following this link: http://redhatstackblog.redhat.com/2015/05/05/cpu-pinning-and-numa-topology-awareness-in-openstack-compute/ Ah, sorry. I think that pinning only properly

Re: [Openstack] CPU pinning question

2015-12-15 Thread Chris Friesen
Actually no, I don't think that's right. When pinning is enabled each vCPU will be affined to a single host CPU. What is showing below is what I would expect if the instance was using non-dedicated CPUs. To the original poster, you should be using 'hw:cpu_policy': 'dedicated' in your

Re: [Openstack] CPU pinning question

2015-12-15 Thread Chris Friesen
pinning”? Anyway, for performance we didn’t see a difference between 1:1 pinning and confining (?) the vCPUs to a a set as long as the instance is aware of the underlying NUMA topology. Cheers, Arne On 15 Dec 2015, at 17:11, Chris Friesen <chris.frie...@windriver.com> wrote: Actual

Re: [openstack-dev] [cinder] Dependencies of snapshots on volumes

2015-12-09 Thread Chris Friesen
On 12/09/2015 10:27 AM, John Griffith wrote: On Tue, Dec 8, 2015 at 9:10 PM, Li, Xiaoyan > wrote: Hi all, Currently when deleting a volume, it checks whether there are snapshots created from it. If yes deletion is prohibited.

[openstack-dev] [nova] no user/project validation in "nova quota-show"?

2015-12-07 Thread Chris Friesen
Can someone explain why nova doesn't seem to be doing any validation in the "nova quota-show" command? (At least in kilo/stable.) If I run: nova --debug quota-show --tenant aprojectdoesnotexist --user nosuchuser the debug info shows: DEBUG (connectionpool:383) "GET

Re: [openstack-dev] [nova][cinder] About rebuilding volume-backed instances.

2015-11-11 Thread Chris Friesen
On 11/11/2015 11:01 AM, Murray, Paul (HP Cloud) wrote: Unfortunately, you're trying to work around misuse of the cloud API's, not missing features from them. Don't use those volume types, and don't build systems that rely on single ports and interfaces. IMO rebuild is a misguided concept

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-11-06 Thread Chris Friesen
On 11/06/2015 03:47 AM, Tang Chen wrote: On 11/06/2015 12:45 PM, Chris Friesen wrote: To me, the external API would make more sense as: 1) resize 2) migrate (with option of cold or live, and with option to specify a destination, and with option to override the scheduler if the specified

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-11-05 Thread Chris Friesen
On 11/05/2015 08:33 AM, Andrew Laski wrote: On 11/05/15 at 01:28pm, Murray, Paul (HP Cloud) wrote: Or more specifically, the migrate and resize API actions both call the resize function in the compute api. As Ed said, they are basically the same behind the scenes. (But the API difference is

Re: [openstack-dev] [Nova] attaching and detaching volumes in the API

2015-11-05 Thread Chris Friesen
On 11/05/2015 12:13 PM, Murray, Paul (HP Cloud) wrote: As part of this spec: https://review.openstack.org/#/c/221732/ I want to attach/detach volumes (and so manipulate block device mappings) when an instance is not on any compute node (actually when in shelved). Normally this happens in a

Re: [Openstack-operators] cinder volume_clear=zero makes sense with rbd ?

2015-11-04 Thread Chris Friesen
On 11/04/2015 08:46 AM, Saverio Proto wrote: Hello there, I am using cinder with rbd, and most volumes are created from glance images on rbd as well. Because of ceph features, these volumes are CoW and only blocks different from the original parent image are really written. Today I am

Re: [openstack-dev] [all][api][tc][perfromance] API for getting only status of resources

2015-11-04 Thread Chris Friesen
On 11/03/2015 11:45 PM, John Griffith wrote: On Tue, Nov 3, 2015 at 3:20 PM, Boris Pavlovic > wrote: Hi stackers, Usually such projects like Heat, Tempest, Rally, Scalar, and other tool that works with OpenStack are working with

[openstack-dev] [nova] vif unplugging when deleting instance while resizing

2015-10-30 Thread Chris Friesen
Hi, I do have a situation where an instance was deleted while resizing, and there are vswitch ports remaining in use when they shouldn't be. I suspect that it's related to bug 1392527. (https://bugs.launchpad.net/nova/+bug/1392527) That bug was involved a scenario where we trigger a resize

Re: [openstack-dev] [nova] vif unplugging when deleting instance while resizing

2015-10-30 Thread Chris Friesen
On 10/30/2015 03:11 PM, Chris Friesen wrote: From what I can tell, in stable/kilo with neutron when doing a resize we will only unplug the vifs on the source side when confirming the resize. The code path looks like: compute.manager.ComputeManager.confirm_resize

Re: [openstack-dev] [Nova] Migration state machine proposal.

2015-10-26 Thread Chris Friesen
On 10/26/2015 06:02 PM, Jay Pipes wrote: I believe strongly that we should deprecate the existing migrate, resize, an live-migrate APIs in favor of a single consolidated, consistent "move" REST API that would have the following characteristics: * No manual or wait-input states in any FSM graph

Re: [Openstack] how to isolate user's view of VMs in one tenant?

2015-10-20 Thread Chris Friesen
On 10/20/2015 07:49 PM, senvenks wrote: Hi, I am a newbie on OpenStack and now encounter a simple question here. On the default dashboard UI, a VM is visible to all users in one tenant after being created. Is there any way to isolate each user's view of

[Openstack] keypair description in documentation wrong?

2015-10-16 Thread Chris Friesen
Someone recently asked me a question about keypairs and multiple users and I thought I'd clarify a few things: 1) Each keypair is associated with a specific user. 2) A user cannot see a keypair belonging to another user. 3) If a user is part of multiple projects then any keypair owned by

Re: [Openstack] resize lvm-backed instance

2015-10-16 Thread Chris Friesen
I think it should be possible. What errors do you see in the logs? Chris On 10/16/2015 01:44 AM, Ivan Derbenev wrote: bump Regards, IT engineer Farheap, Russia Ivan Derbenev *From:*Ivan Derbenev [mailto:ivan.derbe...@tech-corps.com] *Sent:* Tuesday, October 13, 2015 1:53 PM *To:*

Re: [openstack-dev] Scheduler proposal

2015-10-10 Thread Chris Friesen
On 10/09/2015 07:29 PM, Clint Byrum wrote: Even if you figured out how to make the in-memory scheduler crazy fast, There's still value in concurrency for other reasons. No matter how fast you make the scheduler, you'll be slave to the response time of a single scheduling request. If you take

Re: [openstack-dev] Scheduler proposal

2015-10-09 Thread Chris Friesen
On 10/09/2015 03:36 PM, Ian Wells wrote: On 9 October 2015 at 12:50, Chris Friesen <chris.frie...@windriver.com <mailto:chris.frie...@windriver.com>> wrote: Has anybody looked at why 1 instance is too slow and what it would take to make 1 scheduler instance work

Re: [openstack-dev] [Openstack-operators] Scheduler proposal

2015-10-09 Thread Chris Friesen
On 10/08/2015 01:37 AM, Clint Byrum wrote: Excerpts from Maish Saidel-Keesing's message of 2015-10-08 00:14:55 -0700: Forgive the top-post. Cross-posting to openstack-operators for their feedback as well. Ed the work seems very promising, and I am interested to see how this evolves. With my

Re: [openstack-dev] Scheduler proposal

2015-10-09 Thread Chris Friesen
On 10/09/2015 11:09 AM, Zane Bitter wrote: The optimal way to do this would be a weighted random selection, where the probability of any given host being selected is proportional to its weighting. (Obviously this is limited by the accuracy of the weighting function in expressing your actual

Re: [openstack-dev] Scheduler proposal

2015-10-09 Thread Chris Friesen
On 10/09/2015 12:55 PM, Gregory Haynes wrote: There is a more generalized version of this algorithm for concurrent scheduling I've seen a few times - Pick N options at random, apply heuristic over that N to pick the best, attempt to schedule at your choice, retry on failure. As long as you have

Re: [openstack-dev] Scheduler proposal

2015-10-09 Thread Chris Friesen
On 10/09/2015 12:25 PM, Alec Hothan (ahothan) wrote: Still the point from Chris is valid. I guess the main reason openstack is going with multiple concurrent schedulers is to scale out by distributing the load between multiple instances of schedulers because 1 instance is too slow. This

Re: [openstack-dev] Scheduler proposal

2015-10-07 Thread Chris Friesen
On 10/07/2015 11:36 AM, Ed Leafe wrote: I've finally gotten around to finishing writing up that proposal [1], and I'd like to hope that it would be the basis for future discussions about addressing some of the underlying issues that exist in OpenStack for historical reasons, and how we might

Re: [openstack-dev] [nova] live migration in Mitaka

2015-10-07 Thread Chris Friesen
On 10/07/2015 03:14 AM, Daniel P. Berrange wrote: For suspended instances, the scenario is really the same as with completely offline instances. The only extra step is that you need to migrate the saved image state file, as well as the disk images. This is trivial once you have done the code

Re: [openstack-dev] Scheduler proposal

2015-10-07 Thread Chris Friesen
On 10/07/2015 07:23 PM, Ian Wells wrote: On 7 October 2015 at 16:00, Chris Friesen <chris.frie...@windriver.com <mailto:chris.frie...@windriver.com>> wrote: 1) Some resources (RAM) only require tracking amounts. Other resources (CPUs, PCI devices) require trackin

Re: [openstack-dev] [nova] live migration in Mitaka

2015-10-06 Thread Chris Friesen
On 10/06/2015 11:27 AM, Paul Carlton wrote: On 06/10/15 17:30, Chris Friesen wrote: On 10/06/2015 08:11 AM, Daniel P. Berrange wrote: On Tue, Oct 06, 2015 at 02:54:21PM +0100, Paul Carlton wrote: https://review.openstack.org/#/c/85048/ was raised to address the migration of instances

Re: [openstack-dev] [nova] live migration in Mitaka

2015-10-06 Thread Chris Friesen
On 10/06/2015 08:11 AM, Daniel P. Berrange wrote: On Tue, Oct 06, 2015 at 02:54:21PM +0100, Paul Carlton wrote: https://review.openstack.org/#/c/85048/ was raised to address the migration of instances that are not running but people did not warm to the idea of bringing a stopped/suspended

Re: [openstack-dev] [nova] live migration in Mitaka

2015-09-30 Thread Chris Friesen
On 09/30/2015 06:03 AM, Koniszewski, Pawel wrote: From: Murray, Paul (HP Cloud) [mailto:pmur...@hpe.com] - migrate suspended instances I'm not sure I understand this correctly. When user calls 'nova suspend' I thought that it actually "hibernates" VM and saves memory state to disk [2][3].

Re: [openstack-dev] [cinder] should we use fsync when writing iscsi config file?

2015-09-27 Thread Chris Friesen
On 09/26/2015 02:48 AM, Julien Danjou wrote: On Tue, Sep 22 2015, Chris Friesen wrote: On 09/22/2015 05:48 PM, Joshua Harlow wrote: A present: >>> import contextlib >>> import os >>> >>> @contextlib.contextmanager ... def synced_file(path, mo

Re: [openstack-dev] [cinder] should we use fsync when writing iscsi config file?

2015-09-25 Thread Chris Friesen
On 09/24/2015 04:21 PM, Chris Friesen wrote: On 09/24/2015 12:18 PM, Chris Friesen wrote: I think what happened is that we took the SIGTERM after the open() call in create_iscsi_target(), but before writing anything to the file. f = open(volume_path, 'w+') f.write

Re: [openstack-dev] [cinder] should we use fsync when writing iscsi config file?

2015-09-25 Thread Chris Friesen
On 09/25/2015 12:30 PM, Mitsuhiro Tanino wrote: On 09/22/2015 06:43 PM, Robert Collins wrote: On 23 September 2015 at 09:52, Chris Friesen <chris.frie...@windriver.com> wrote: Hi, I recently had an issue with one file out of a dozen or so in "/opt/cgcs/cinder/data/volumes/&quo

Re: [Openstack-operators] Migrating an instance to a host with less cores fails

2015-09-25 Thread Chris Friesen
On 09/25/2015 12:34 PM, Steve Gordon wrote: Nikola's reply got bounced because he isn't subscribed, but: """ Thanks Steve! So the below is likely the same root cause as this bug: https://launchpad.net/bugs/1461777 Which has been fixed in Liberty and backported to stable/kilo (see

Re: [openstack-dev] [cinder] should we use fsync when writing iscsi config file?

2015-09-24 Thread Chris Friesen
On 09/24/2015 10:54 AM, Chris Friesen wrote: I took another look at the code and realized that the file *should* get rebuilt on restart after a power outage--if the file already exists it will print a warning message in the logs but it should still overwrite the contents of the file

Re: [openstack-dev] [cinder] should we use fsync when writing iscsi config file?

2015-09-24 Thread Chris Friesen
On 09/24/2015 12:18 PM, Chris Friesen wrote: I think what happened is that we took the SIGTERM after the open() call in create_iscsi_target(), but before writing anything to the file. f = open(volume_path, 'w+') f.write(volume_conf) f.close() The 'w+' causes

<    1   2   3   4   5   6   7   >