Re: [openstack-dev] cgroups cpu share allocation in grizzly seems incorrect

[Qiu Yu]

On Fri, Aug 23, 2013 at 6:00 AM, Chris Friesen
 wrote:
>
> I just noticed that in Grizzly regardless of the number of vCPUs the value
> of /sys/fs/cgroup/cpu/libvirt/qemu/instance-X/cpu.shares seems to be the
> same.  If we were overloaded, this would give all instances the same cpu
> time regardless of the number of vCPUs in the instance.
>
> Is this design intent?  It seems to me that it would be more correct to have
> the instance value be multiplied by the number of vCPUs.

I think it make sense to have each vCPU an equal weight for scheduling
sake. This makes each vCPU an equal entity with same computing power.

For limiting vCPU hard limit purpose, using CFS quota/period, please
check following BP for reference.
https://blueprints.launchpad.net/nova/+spec/quota-instance-resource

--
Qiu Yu

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [savanna] savanna-extra and hadoop-patch

[Sergey Lukjanov]

Hi Erik,

First of all, savanna-extra has been created exactly for such needs - to store 
all stuff that we need but couldn't be placed to another repos. Initially it 
contains elements and pre-builded jar with Swift HCFS. Now the last one has 
been moved to the CDN and it's a good idea to make separated project for 
elements.

As for Swift HCFS the core attached to the HADOOP-8545 is targeted to the 
Hadoop 2 version and should be patched to work with Hadoop 1.X correctly. So, 
that's why we add it to the extra repo. It looks like that it's ok to add one 
more repo for Swift HCFS near the savanna at stackforge like HCFS for 
Gluster[0].

So, let's discuss both of the migrations at the next IRC team meeting.

[0] https://github.com/gluster/hadoop-glusterfs

Sincerely yours,
Sergey Lukjanov
Savanna Technical Lead
Mirantis Inc.

On Aug 27, 2013, at 5:18, Matthew Farrellee  wrote:

> https://review.openstack.org/#/c/42926/
> 
> I didn't get back to this on Friday and it got merged this morning, so here's 
> my feedback.
> 
> The savanna-extra repository now appears to hold (a) DIB image elements as 
> well as (b) the source for the Swift backed HCFS (Hadoop Compatible File 
> System) implementation.
> 
> If I understand this correctly, (b) is actually the patch set that is being 
> proposed to the Apache Hadoop community. That patch set has not been accepted 
> and is being tracked in HADOOP-8545[0], which appears stalled since July 2013.
> 
> Let's break Savanna's DIB elements out of savanna-extra and into 
> savanna-image-elements. It has a clear path forward and a good definition of 
> scope.
> 
> Let's also leave savanna-extra as a grab bag, whose only occupant is 
> currently the Swift code. Eventually that code will need a proper home, 
> either contributed to Apache Hadoop or broken out as its own project.
> 
> Best,
> 
> 
> matt
> 
> [0] https://issues.apache.org/jira/browse/HADOOP-8545
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Nova] Requesting feedback on review 35759

[Wang, Shane]

Hi,

We submitted the patches for bp 
https://blueprints.launchpad.net/nova/+spec/utilization-aware-scheduling 1+ 
month ago.
The first patch is to add a column to save metrics collected by plugins - 
https://review.openstack.org/#/c/35759/.
Is there anyone who is interested in that, would it be possible to get some 
reviews for that?

Thanks.
--
Shane

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Scheduler sub-group meeting on 8/27

[Dugger, Donald D]

1) Scheduler design sessions at the Icehouse summit
2) Perspective for Nova scheduler (Boris' proposal at 
https://docs.google.com/document/d/1_DRv7it_mwalEZzLy5WO92TJcummpmWL4NWsWf0UWiQ/edit#heading=h.6ixj0ctv4rwu
  )


--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [savanna] savanna-extra and hadoop-patch

[Matthew Farrellee]

https://review.openstack.org/#/c/42926/

I didn't get back to this on Friday and it got merged this morning, so 
here's my feedback.


The savanna-extra repository now appears to hold (a) DIB image elements 
as well as (b) the source for the Swift backed HCFS (Hadoop Compatible 
File System) implementation.


If I understand this correctly, (b) is actually the patch set that is 
being proposed to the Apache Hadoop community. That patch set has not 
been accepted and is being tracked in HADOOP-8545[0], which appears 
stalled since July 2013.


Let's break Savanna's DIB elements out of savanna-extra and into 
savanna-image-elements. It has a clear path forward and a good 
definition of scope.


Let's also leave savanna-extra as a grab bag, whose only occupant is 
currently the Swift code. Eventually that code will need a proper home, 
either contributed to Apache Hadoop or broken out as its own project.


Best,


matt

[0] https://issues.apache.org/jira/browse/HADOOP-8545

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [savanna] tarballs of savanna-extra

[Nirmal Ranganathan]

The code hasn't been merged into hadoop-common trunk, once that happens
there's a possibility of getting the jars from there. For now the CDN
works.


On Thu, Aug 22, 2013 at 2:17 PM, Sergey Lukjanov wrote:

> Ok, let's start from storing jars in CDN.
>
> Sincerely yours,
> Sergey Lukjanov
> Savanna Technical Lead
> Mirantis Inc.
>
> On Aug 21, 2013, at 23:16, Matthew Farrellee  wrote:
>
> > IMHO, the jars should be served from the Apache Hadoop community. I
> don't know what hoops have to jumped through for that though. It may be far
> simpler to put them in the mirantis CDN.
> >
> > Best,
> >
> >
> > matt
> >
> > On 08/21/2013 02:21 PM, Sergey Lukjanov wrote:
> >> Agreed that storing Hadoop-Swift integration jars in the git repo is
> >> a good practice, any thoughts about where to store them? Currently I
> >> have only one option - we can store them at the public CDN
> >> (savanna-files.mirantis.com) near the images for vanilla plugin.
> >>
> >> As for publishing tarballs with the content of savanna-extra - looks
> >> like there are more pros than cons, so, we can do it.
> >>
> >> Sincerely yours,
> >> Sergey Lukjanov
> >> Savanna Technical Lead
> >> Mirantis Inc.
> >>
> >> On Aug 20, 2013, at 16:31, Matthew Farrellee  wrote:
> >>
> >>> Is there a downside to having it? A positive is it gives a snapshot of
> everything for each release.
> >>>
> >>> I'm not at fan of having a snapshot of the Hadoop swift patches
> compiled into a jar and stored in the repository. I'd prefer that it is
> hosted elsewhere.
> >>>
> >>> Best,
> >>>
> >>>
> >>> matt
> >>>
> >>> On 08/19/2013 04:37 PM, Sergey Lukjanov wrote:
>  Hi Matt,
> 
>  it is not an accident that savanna-extra has no tarballs at
>  tarballs.o.o, because this repo is used for storing some date that is
>  only needed for some stuff like building images for vanilla plugin,
>  storing Swift support patch for Hadoop and etc. So, it looks like
>  that we should not package all of them to one heterogeneous tarball.
> 
>  Sincerely yours,
>  Sergey Lukjanov
>  Savanna Technical Lead
>  Mirantis Inc.
> 
>  On Aug 20, 2013, at 0:25, Matthew Farrellee  wrote:
> 
> > Will someone setup a tarballs.os.o release of savanna-extra's master
> (https://github.com/stackforge/savanna-extra), and make sure it gets an
> official release for 0.3?
> >
> > Best,
> >
> >
> > matt
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
>  ___
>  OpenStack-dev mailing list
>  OpenStack-dev@lists.openstack.org
>  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> >>>
> >>
> >
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Nirmal

http://rnirmal.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] About multihost patch review

[Edgar Magana]

Hi Developers,

Let me explain my point of view on this topic and please share your thoughts
in order to merge this new feature ASAP.

My understanding is that multi-host is nova-network HA  and we are
implementing this bp
https://blueprints.launchpad.net/neutron/+spec/quantum-multihost for the
same reason.
So, If in neutron configuration admin enables multi-host:
etc/dhcp_agent.ini

# Support multi host networks
# enable_multihost = False

Why do tenants needs to be aware of this? They should just create networks
in the way they normally do and not by adding the "multihost" extension.
I could be totally wrong and crazy, so please provide some feedback.

Thanks,

Edgar


From:  Yongsheng Gong 
Date:  Monday, August 26, 2013 2:58 PM
To:  "Kyle Mestery (kmestery)" , Aaron Rosen
, Armando Migliaccio , Akihiro
MOTOKI , Edgar Magana , Maru Newby
, Nachi Ueno , Salvatore Orlando
, Sumit Naiksatam , Mark
McClain , Gary Kotton ,
Robert Kukura 
Cc:  OpenStack List 
Subject:  Re: About multihost patch review

Hi,
Edgar Magana has commented to say:
'This is the part that for me is confusing and I will need some
clarification from the community. Do we expect to have the multi-host
feature as an extension or something that will natural work as long as the
deployment include more than one Network Node. In my opinion, Neutron
deployments with more than one Network Node by default should call DHCP
agents in all those nodes without the need to use an extension. If the
community has decided to do this by extensions, then I am fine' at
https://review.openstack.org/#/c/37919/11/neutron/extensions/multihostnetwor
k.py

I have commented back, what is your opinion about it?

Regards,
Yong Sheng Gong


On Fri, Aug 16, 2013 at 9:28 PM, Kyle Mestery (kmestery)
 wrote:
> Hi Yong:
> 
> I'll review this and try it out today.
> 
> Thanks,
> Kyle
> 
> On Aug 15, 2013, at 10:01 PM, Yongsheng Gong  wrote:
> 
>> > The multihost patch is there for a long long time, can someone help to
>> review?
>> > https://review.openstack.org/#/c/37919/
> 



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] oauth and keystone

[Adam Young]

On 08/21/2013 12:22 AM, Yongsheng Gong wrote:

Hi,

I have seen the code about oauth in the keystone but I cannot find the 
document about how to setup keystone and other openstack services to 
enable oauth.


OAuth support is very new.  The other services are not yet set to take 
advantage of it.




Can anyone tell me how to setup an env like this?

Thanks
Yong Sheng Gong


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] live-snapshot/cloning of virtual machines

[Tim Smith]

Hi all,

On Mon, Aug 19, 2013 at 11:49 PM, Bob Ball  wrote:

> I agree with the below from a XenServer perspective.  As with vmware,
> XenServer supports live snapshotting and creating multiple clones from that
> live snapshot.
>
> I understand that there is a XenAPI equivalent in the works and therefore
> would argue the API changes need to be accepted as a minimum.
>

Can nova technical leadership provide clarification on the current standing
of this blueprint? Two hypervisor vendors have expressed plans for
supporting this feature, and one has specifically requested that the API
changes be merged, but it appears that both the API changeset [1] and
novaclient support [2] have both been rejected pending libvirt support
(which has assumedly been ruled out for the Havana release).

[1] https://review.openstack.org/#/c/34036/
[2] https://review.openstack.org/#/c/43777/


> In order to minimize the feature divergence between hypervisors, I'd also
> argue that we should accept the libvirt implementation even if it uses
> unsupported APIs - perhaps disabled by default with a suitable warning that
> it isn't considered safe by libvirt/QEmu.
>

It's understandable that changes to the libvirt driver would be held back
until libvirt/qemu-upstream support for live snapshotting is established
(if ever), but given that other vendors whose release cadences don't
necessarily align with the nova release schedule have expressed plans to
support the interface it's unclear why lack of libvirt driver support would
block the entire blueprint.

Thanks,
Tim



>
> Bob
> 
> From: Shawn Hartsock [hartso...@vmware.com]
> Sent: 19 August 2013 20:59
> To: Daniel P. Berrange; OpenStack Development Mailing List
> Subject: Re: [openstack-dev] [nova] live-snapshot/cloning of virtual
> machines
>
> For what it's worth... this doesn't seem too bad to me...
>
> I was planning on using this part of the vSphere API:
> *
> https://www.vmware.com/support/developer/vc-sdk/visdk400pubs/ReferenceGuide/vim.vm.CloneSpec.html
>
> ...to accomplish the clone part of the BP. The API contains a "spec"
> section where you tell the ESX hypervisor how to handle things like network
> identity...
> *
> https://www.vmware.com/support/developer/vc-sdk/visdk400pubs/ReferenceGuide/vim.vm.customization.IPSettings.html
>
> ... I was going to use this to plumb together the "live-snapshot" bit ...
> *
> https://www.vmware.com/support/developer/vc-sdk/visdk400pubs/ReferenceGuide/vim.VirtualMachine.html#createSnapshot
>
> ... which includes stuff about how to handle memory.
>
> So, I didn't read this blueprint as especially hard to accomplish in the
> vmwareapi driver. It just would eat more time than I have right now and
> would require a deeper level of understanding of how the vSphere hypervisor
> suite works than most of the other features currently use. I'm fully
> planning on playing with this in IceHouse just to see how it would go, it's
> probably one of the more nifty new features we could add.
>
> Note: these are old features for the API and they are a tad complicated,
> but it's all well within the realm of supported! In fact, it's standard
> operating procedure to use the clone feature to scale-out an application in
> some vSphere shops. (albeit, in production the admins I know personally,
> use clone with power-off from a 'template' and the system comes up with a
> new MAC/etc on first boot... cloning from a running system is possible, but
> I would recommend cloning from a power-off state unless you've got a
> hot-plug feature in your guest OS)
>
>
>
> # Shawn Hartsock
>
> - Original Message -
> > From: "Daniel P. Berrange" 
> > To: "OpenStack Development Mailing List" <
> openstack-dev@lists.openstack.org>
> > Sent: Monday, August 19, 2013 5:24:59 AM
> > Subject: Re: [openstack-dev] [nova] live-snapshot/cloning of virtual
> machines
> >
> > On Mon, Aug 19, 2013 at 08:28:58AM +1200, Robert Collins wrote:
> > > On 17 August 2013 07:01, Russell Bryant  wrote:
> > >
> > > >> Maybe we've grown up to the point where we have to be more careful
> and
> > > >> not introduce
> > > >> these kind of features and the maintenance cost of introducing
> > > >> experimental features is
> > > >> too great. If that is the community consensus, then I'm happy keep
> the
> > > >> live snapshot stuff
> > > >> in a branch on github for people to experiment with.
> > > >
> > > > My feeling after following this discussion is that it's probably
> best to
> > > > keep baking in another branch (github or whatever).  The biggest
> reason
> > > > is because of the last comment quoted from Daniel Berrange above.  I
> > > > feel that like that is a pretty big deal.
> > >
> > > So, reading between the lines here, I guess you're worried that we'd
> > > let code paths that violate what upstream will support leak into the
> > > main codepaths for libvirt - and thus we'd end up with a situation
> > > where we aren't supported by upstream for a

Re: [openstack-dev] [Nova] Interested in a mid-Icehouse-cycle Nova meet-up?

[Joe Gordon]

On Mon, Aug 19, 2013 at 5:42 PM, Russell Bryant  wrote:

> Greetings,
>
> Some OpenStack programs have started a nice trend of getting together in
> the middle of the development cycle.  These meetups can serve a number
> of useful purposes: community building, ramping up new contributors,
> tackling hard problems by getting together in the same room, and more.


++, I think this is a great idea especially considering how busy the
summits are becoming.


>
> I am in the early stages of attempting to plan a Nova meet-up for the
> middle of the Icehouse cycle.  To start, I need to get a rough idea of
> how much interest there is.
>
> I have very little detail at this point, other than I'm looking at
> locations in the US, and that it would be mid-cycle (January/February).
>
> If you would be interested in this, please fill out the following form:
>
> http://goo.gl/RPa6iG
>
> Thanks!
>
> --
> Russell Bryant
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Tempest now Running in Parallel in the Gate

[James E. Blair]

Matthew Treinish  writes:

> This should improve the run times for tempest jobs to between 20-30min
> on average which will definitely help with the upcoming milestone
> rush.

This is great!  Thanks to you and the Tempest team for getting this
_huge_ effort done!  I'm really excited that we're able to run existing
tests more realistically and faster, as well as opening the door to
running even more tests in the future!

> So I'm asking everyone to be diligent with watching (and using) the recheck 
> list
> and that all the devs just don't recheck and forget. We need a combined 
> effort from
> everyone to make things stable and debug seemingly random failures when they
> come up. If we can iron out most of the flakiness by the milestone rush having
> the faster runtime will definitely help make the surge easier for everyone.

It's not that much fun, but when we've had flakey tests in the past,
diligent use of the recheck system has _really_ helped the people
working on tracking those problems down.  For reference, here's the
procedure to use:

  https://wiki.openstack.org/wiki/GerritJenkinsGithub#Test_Failures

-Jim

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Tempest now Running in Parallel in the Gate

[Matthew Treinish]

Hi everyone,

So as the title indicates I pushed through the change that migrates tempest to
run by default in parallel. (except for the neutron job, which still needs work
to run in parallel) This should improve the run times for tempest jobs to
between 20-30min on average which will definitely help with the upcoming
milestone rush. However, the short term trade-off of this is that it adds a bit
more flakiness to the tempest runs. But, it's a good flaky albeit an annoying
one. :)

Running in parallel actually helps us simulate a real workload a bit better
than before. Since we're running 4 test cases at once it more closely resembles
a real OpenStack deployment (at least compared to running the tests one at a 
time)
Just in debugging things to get it ready for running in the gate, I uncovered
some race conditions that pre-existed in the projects, and I'm sure there are
some more. There are probably still races between tests in tempest as well that
will have to be sorted.

So I'm asking everyone to be diligent with watching (and using) the recheck list
and that all the devs just don't recheck and forget. We need a combined effort 
from
everyone to make things stable and debug seemingly random failures when they
come up. If we can iron out most of the flakiness by the milestone rush having
the faster runtime will definitely help make the surge easier for everyone.

Thanks,

Matt Treinish

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] About multihost patch review

[Yongsheng Gong]

Hi,
Edgar Magana has commented to say:
'This is the part that for me is confusing and I will need some
clarification from the community. Do we expect to have the multi-host
feature as an extension or something that will natural work as long as the
deployment include more than one Network Node. In my opinion, Neutron
deployments with more than one Network Node by default should call DHCP
agents in all those nodes without the need to use an extension. If the
community has decided to do this by extensions, then I am fine' at
https://review.openstack.org/#/c/37919/11/neutron/extensions/multihostnetwork.py

I have commented back, what is your opinion about it?

Regards,
Yong Sheng Gong


On Fri, Aug 16, 2013 at 9:28 PM, Kyle Mestery (kmestery)  wrote:

> Hi Yong:
>
> I'll review this and try it out today.
>
> Thanks,
> Kyle
>
> On Aug 15, 2013, at 10:01 PM, Yongsheng Gong 
> wrote:
>
> > The multihost patch is there for a long long time, can someone help to
> review?
> > https://review.openstack.org/#/c/37919/
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Secure live VM migration in cloud (openstack)

[Joshua Harlow]

Hi,

Those ideas sounds pretty good to me. Although I'm not an expert in the 
security area, have u talked with the libvirt folks. I wonder if they have any 
of this planned?

From: Naveed Ahmad 
<12msccsnah...@seecs.edu.pk>
Reply-To: OpenStack Development Mailing List 
mailto:openstack-dev@lists.openstack.org>>
Date: Monday, August 26, 2013 11:10 AM
To: OpenStack Development Mailing List 
mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] Secure live VM migration in cloud (openstack)

Respected Joshua Harlow,

Thanks for reply,

Based on literature survey i found that following techniques are used for 
secure live migration of vm.

1. RSA with SSL protocol for authentication and encryption.
As you mentioned earlier same problem is in RSA based authentication. we have 
to add public keys of all other hypervisors.

In Blackhat 2013, security research found vulnerability in SSL so it can be 
breakable in very short time.
please check
http://arstechnica.com/security/2013/08/gone-in-30-seconds-new-attack-plucks-secrets-from-https-protected-pages/

2. SSH is used for secure tunnel before live vm migration.

Authentication is not discussed, only secure tunnel is used to achieve  
confidentiality.

3. Openstack uses libvirtd with kvm to provide secure vm migration between src 
and dst machine.
SSL is used for encrypted channel and SASL  is used for authentication.



so i am interested to implement authentication level's in live vm migration.

1.no authentication
2. Certificate base
3.smart card based authentication

and similarly ssl provide secure channel but after that seaprate VLAN is used 
for vm migration traffic. if we use ipsec then we can achieve same goal on 
network layer to hide all communication of vm migration.



Regards
Naveed









On Mon, Aug 26, 2013 at 2:44 AM, Joshua Harlow 
mailto:harlo...@yahoo-inc.com>> wrote:
Arg, hit send to quick.

*likely these problems would require some managed migration "thing" that would 
temporarily open the network access, issue temporary auth keys and the initiate 
the migration between the 2 hypervisors. Is this in your scope, to make this 
thing??


Sent from my really tiny device...

On Aug 25, 2013, at 2:42 PM, "Joshua Harlow" 
mailto:harlo...@yahoo-inc.com>> wrote:

Hi,

I think it's a good idea, can u describe more what would be different, would 
there be a new auth and live migration mechanism?

I think one of the problems at least yahoo has is that live migration requires 
all ssh keys to be on all hypervisors since hypervisors (libvirtd) open up the 
connection to the hypervisor to be migrated to. This is obviously bad, as any 
hacker if they can get out of a vm now can start issuing these migration 
requests. Also at yahoo we don't allow hypervisors to communicate openly to 
each other, this is protected at the network level. Would u be working on 
solutions to these problems (likely involving

Sent from my really tiny device...

On Aug 25, 2013, at 6:33 AM, "Naveed Ahmad" 
<12msccsnah...@seecs.edu.pk> wrote:


thanks for replying Joshua,


VM migration is the process used to migrate vm from one physical server to 
another physical server due to many reasons like system maintenance, hardware 
failure ,

VM is important element in cloud as well, so we do same in the cloud. xen/kvm 
hypervisor used in the openstack dont provide security  in this process. i 
studied few paper on it  which are related to VM migration in DC instead of 
Cloud.   i also seen book on openstack security in which it is describe that 
xen/kvm could not provide security but libvirt can be used with xen/kvm to 
secure this process.

Currently libvirt is providing ssl for confidentiality of data between source 
and destination. and SASL for authentication. i want to add other 
authentication mechanism in it and in the end it would be added in the 
Dashboard of openstack so that administrator use it easily, Access control is 
also part of this thesis..


may you got my idea Mr. Joshua Harlow and now please comment on it. is it good 
or not? your comment will help me to choose good topic in cloud security,


Regards










On Sun, Aug 25, 2013 at 4:17 AM, Joshua Harlow 
mailto:harlo...@yahoo-inc.com>> wrote:
Is there any write up of what u want to do or is that not defined yet?

If u can write up some information I think that would help others provide 
feedback as well as help everyone (including yourself) see the goal too be 
accomplished. It's hard to tell what the desired outcome is otherwise, secure 
vm migration could mean a lot of things :)

Sent from my really tiny device...

On Aug 24, 2013, at 12:26 PM, "Naveed Ahmad" 
<12msccsnah...@seecs.edu.pk> wrote:

>
>
> Hi all,
>
>
>
> I am doing thesis in cloud computing security domain, i selected to secure vm 
> migration  process in openstack.
> Please let me know about this idea. i have 

Re: [openstack-dev] [Openstack][keystone][ldap] Bug unsupported operand type(s) for &: 'str' and 'int'

[Adam Young]

On 08/22/2013 05:37 AM, ZhiQiang Fan wrote:
actually, i only read the master branch, and there is no such file, 
you can report a bug on keystone grizzly and submit your code, if 
you're granted


Yes, please file the bug, sign the CLA, and submit to gerrit. Thanks



On Thu, Aug 22, 2013 at 5:21 PM, Qinglong.Meng > wrote:


I have fix it in my env.
and you will find some hardcore in /identity/backend/ldap/core.py,,


2013/8/22 ZhiQiang Fan mailto:aji.zq...@gmail.com>>

yes, i think it is a bug, because the api doesn't convert
string to correct type:
>>> True & 1
1
>>> 'True' & 1
Traceback (most recent call last):
  File "", line 1, in 
TypeError: unsupported operand type(s) for &: 'str' and 'int'

Feel free to report a bug and may be you can try to fix it.


On Thu, Aug 22, 2013 at 3:51 PM, Qinglong.Meng
mailto:mengql112...@gmail.com>> wrote:

oh, yeah,
here is the keystone log:
http://paste.openstack.org/show/44857/

Best Regards,





2013/8/22 ZhiQiang Fan mailto:aji.zq...@gmail.com>>

your first reference (the log) is incorrect.


On Thu, Aug 22, 2013 at 2:55 PM, Qinglong.Meng
mailto:mengql112...@gmail.com>> wrote:

Hi All,
Os: Ubuntu 12.04 LTS
keystone version: stable/grizzly
After I deploy keystone with ldap backend
(openldap), I got the issue about type in
keystone.log, when I create user by  keystoneclient.
Here are some docs:
1. keystone.log
http://paste.openstack.org/
2. keystone.conf
http://paste.openstack.org/show/44839/
3. slapd.conf
http://paste.openstack.org/show/44843/
4. openstack.ldif
http://paste.openstack.org/show/44844/

And I think this is one bug, it's right?

Best Regards,

-- 


Lawrency Meng
mail: mengql112...@gmail.com

___
Mailing list: https://launchpad.net/~openstack

Post to : openst...@lists.launchpad.net

Unsubscribe : https://launchpad.net/~openstack

More help   : https://help.launchpad.net/ListHelp

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org


http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
blog: zqfan.github.com 

git: github.com/zqfan 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org


http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 


Lawrency Meng
mail: mengql112...@gmail.com 
___
Mailing list: https://launchpad.net/~openstack

Post to : openst...@lists.launchpad.net

Unsubscribe : https://launchpad.net/~openstack

More help   : https://help.launchpad.net/ListHelp

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
blog: zqfan.github.com 

git: github.com/zqfan 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 


Lawrency Meng
mail: mengql112...@gmail.com 

Re: [openstack-dev] Incubation Request: Marconi

[Anne Gentle]

Hi Kurt,

There's a thread that John Griffith started about 3rd party storage
drivers, where the code lives, how to review, how to ensure quality and
maintenance, see
http://lists.openstack.org/pipermail/openstack-dev/2013-July/012557.html.

It won't answer all your questions but gives you an idea of pluggable
architecture and maintenance.
Anne


On Fri, Aug 23, 2013 at 10:25 AM, Kurt Griffiths <
kurt.griffi...@rackspace.com> wrote:

> > next in the queue is ZMQ - and it will also sit on top of some
> > existing MB - we've plan to use rabbit, proton and other
> > technologies as storage backend.
>
> I'd love to get everyone's thoughts on how many/what kinds of transport
> and storage drivers should be part of the project directly vs. maintained
> independently as "3rd-party" drivers. How do other projects handle this?
>
> @kgriffs
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Anne Gentle
annegen...@justwriteclick.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Incubation Request: Marconi

[Anne Gentle]

On Fri, Aug 23, 2013 at 10:25 AM, Kurt Griffiths <
kurt.griffi...@rackspace.com> wrote:

> > next in the queue is ZMQ - and it will also sit on top of some
> > existing MB - we've plan to use rabbit, proton and other
> > technologies as storage backend.
>
> I'd love to get everyone's thoughts on how many/what kinds of transport
> and storage drivers should be part of the project directly vs. maintained
> independently as "3rd-party" drivers. How do other projects handle this?
>


Hi Kurt,

There's a thread that John Griffith started about 3rd party storage drivers,
where the code lives, how to review, how to ensure quality and maintenance,
seehttp://lists.openstack.org/pipermail/openstack-dev/2013-July/012557.html.

It won't answer all your questions but gives you an idea of pluggable
architecture and maintenance for Block Storage.
Anne


>
> @kgriffs
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Need help with Alembic...

[Jay Pipes]

On 08/26/2013 03:40 PM, Herndon, John Luke (HPCS - Ft. Collins) wrote:

Jay -

It looks there is an error in the migration script that causes it to abort:

AttributeError: 'ForeignKeyConstraint' object has no attribute 'drop'

My guess is the migration runs on the first test, creates event types
table fine, but exits with the above error, so migration is not
"complete". Thus every subsequent test tries to migrate the db, and
notices that event types already exists.


I'd corrected that particular mistake and pushed an updated migration 
script.


Best,
-jay



-john

On 8/26/13 1:15 PM, "Jay Pipes"  wrote:


I just noticed that every single test case for SQL-driver storage is
executing every single migration upgrade before every single test case
run:

https://github.com/openstack/ceilometer/blob/master/ceilometer/tests/db.py
#L46

https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/imp
l_sqlalchemy.py#L153

instead of simply creating a new database schema from the models in the
current source code base using a call to sqlalchemy.MetaData.create_all().

This results in re-running migrations over and over again, instead of
having dedicated migration tests that would test each migration
individually, as is the case in projects like Glance...

Is this intentional?

Best,
-jay

On 08/26/2013 02:59 PM, Sandy Walsh wrote:

I'm getting the same problem with a different migration (mine is
complaining that a column already exists)

http://paste.openstack.org/show/44512/

I've compared it to the other migrations and it seems fine.

-S

On 08/26/2013 02:34 PM, Jay Pipes wrote:

Hey all,

I'm trying to figure out what is going wrong with my code for this
patch:

https://review.openstack.org/41316

I had previously added a sqlalchemy-migrate migration script to add an
event_type table, and had that working, but then was asked to instead
use Alembic for migrations. So, I removed the sqlalchemy-migrate
migration file and added an Alembic migration [1].

Unfortunately, I am getting the following error when running tests:

OperationalError: (OperationalError) table event_type already exists
u'\nCREATE TABLE event_type (\n\tid INTEGER NOT NULL, \n\t"desc"
VARCHAR(255), \n\tPRIMARY KEY (id), \n\tUNIQUE ("desc")\n)\n\n' ()

The migration adds the event_type table. I've seen this error occur
before when using SQLite due to SQLite's ALTER TABLE statement not
allowing the rename of a column. In the sqlalchemy-migrate migration, I
had a specialized SQLite migration upgrade [2] and downgrade [3]
script,
but I'm not sure how I am supposed to handle this in Alembic. Could
someone help me out?

Thanks,
-jay

[1]

https://review.openstack.org/#/c/41316/16/ceilometer/storage/sqlalchemy/
alembic/versions/49036dfd_add_event_types.py

[2]

https://review.openstack.org/#/c/41316/14/ceilometer/storage/sqlalchemy/
migrate_repo/versions/013_sqlite_upgrade.sql

[3]

https://review.openstack.org/#/c/41316/14/ceilometer/storage/sqlalchemy/
migrate_repo/versions/013_sqlite_downgrade.sql


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Need help with Alembic...

[Herndon, John Luke (HPCS - Ft. Collins)]

Jay - 

It looks there is an error in the migration script that causes it to abort:

AttributeError: 'ForeignKeyConstraint' object has no attribute 'drop'

My guess is the migration runs on the first test, creates event types
table fine, but exits with the above error, so migration is not
"complete". Thus every subsequent test tries to migrate the db, and
notices that event types already exists.

-john

On 8/26/13 1:15 PM, "Jay Pipes"  wrote:

>I just noticed that every single test case for SQL-driver storage is
>executing every single migration upgrade before every single test case
>run:
>
>https://github.com/openstack/ceilometer/blob/master/ceilometer/tests/db.py
>#L46
>
>https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/imp
>l_sqlalchemy.py#L153
>
>instead of simply creating a new database schema from the models in the
>current source code base using a call to sqlalchemy.MetaData.create_all().
>
>This results in re-running migrations over and over again, instead of
>having dedicated migration tests that would test each migration
>individually, as is the case in projects like Glance...
>
>Is this intentional?
>
>Best,
>-jay
>
>On 08/26/2013 02:59 PM, Sandy Walsh wrote:
>> I'm getting the same problem with a different migration (mine is
>> complaining that a column already exists)
>>
>> http://paste.openstack.org/show/44512/
>>
>> I've compared it to the other migrations and it seems fine.
>>
>> -S
>>
>> On 08/26/2013 02:34 PM, Jay Pipes wrote:
>>> Hey all,
>>>
>>> I'm trying to figure out what is going wrong with my code for this
>>>patch:
>>>
>>> https://review.openstack.org/41316
>>>
>>> I had previously added a sqlalchemy-migrate migration script to add an
>>> event_type table, and had that working, but then was asked to instead
>>> use Alembic for migrations. So, I removed the sqlalchemy-migrate
>>> migration file and added an Alembic migration [1].
>>>
>>> Unfortunately, I am getting the following error when running tests:
>>>
>>> OperationalError: (OperationalError) table event_type already exists
>>> u'\nCREATE TABLE event_type (\n\tid INTEGER NOT NULL, \n\t"desc"
>>> VARCHAR(255), \n\tPRIMARY KEY (id), \n\tUNIQUE ("desc")\n)\n\n' ()
>>>
>>> The migration adds the event_type table. I've seen this error occur
>>> before when using SQLite due to SQLite's ALTER TABLE statement not
>>> allowing the rename of a column. In the sqlalchemy-migrate migration, I
>>> had a specialized SQLite migration upgrade [2] and downgrade [3]
>>>script,
>>> but I'm not sure how I am supposed to handle this in Alembic. Could
>>> someone help me out?
>>>
>>> Thanks,
>>> -jay
>>>
>>> [1]
>>> 
>>>https://review.openstack.org/#/c/41316/16/ceilometer/storage/sqlalchemy/
>>>alembic/versions/49036dfd_add_event_types.py
>>>
>>> [2]
>>> 
>>>https://review.openstack.org/#/c/41316/14/ceilometer/storage/sqlalchemy/
>>>migrate_repo/versions/013_sqlite_upgrade.sql
>>>
>>> [3]
>>> 
>>>https://review.openstack.org/#/c/41316/14/ceilometer/storage/sqlalchemy/
>>>migrate_repo/versions/013_sqlite_downgrade.sql
>>>
>>>
>>> ___
>>> OpenStack-dev mailing list
>>> OpenStack-dev@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>___
>OpenStack-dev mailing list
>OpenStack-dev@lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Need help with Alembic...

[Jay Pipes]

On 08/26/2013 03:27 PM, Monsyne Dragon wrote:

You should be able to get the dialect_name from the context in the alembic
migration and do something like:

   if dialect_name == 'sqlite':
#Do whacky sqlite stuff hereŠ
   else:
#sane db migration hère...


Hmm, ok, thanks for the suggestion, Dragon, I'll check into that.

Best,
-jay


On 8/26/13 1:59 PM, "Sandy Walsh"  wrote:


I'm getting the same problem with a different migration (mine is
complaining that a column already exists)

http://paste.openstack.org/show/44512/

I've compared it to the other migrations and it seems fine.

-S

On 08/26/2013 02:34 PM, Jay Pipes wrote:

Hey all,

I'm trying to figure out what is going wrong with my code for this
patch:

https://review.openstack.org/41316

I had previously added a sqlalchemy-migrate migration script to add an
event_type table, and had that working, but then was asked to instead
use Alembic for migrations. So, I removed the sqlalchemy-migrate
migration file and added an Alembic migration [1].

Unfortunately, I am getting the following error when running tests:

OperationalError: (OperationalError) table event_type already exists
u'\nCREATE TABLE event_type (\n\tid INTEGER NOT NULL, \n\t"desc"
VARCHAR(255), \n\tPRIMARY KEY (id), \n\tUNIQUE ("desc")\n)\n\n' ()

The migration adds the event_type table. I've seen this error occur
before when using SQLite due to SQLite's ALTER TABLE statement not
allowing the rename of a column. In the sqlalchemy-migrate migration, I
had a specialized SQLite migration upgrade [2] and downgrade [3] script,
but I'm not sure how I am supposed to handle this in Alembic. Could
someone help me out?

Thanks,
-jay

[1]

https://review.openstack.org/#/c/41316/16/ceilometer/storage/sqlalchemy/a
lembic/versions/49036dfd_add_event_types.py

[2]

https://review.openstack.org/#/c/41316/14/ceilometer/storage/sqlalchemy/m
igrate_repo/versions/013_sqlite_upgrade.sql

[3]

https://review.openstack.org/#/c/41316/14/ceilometer/storage/sqlalchemy/m
igrate_repo/versions/013_sqlite_downgrade.sql


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Need help with Alembic...

[Monsyne Dragon]

You should be able to get the dialect_name from the context in the alembic
migration and do something like:

  if dialect_name == 'sqlite':
#Do whacky sqlite stuff hereŠ
  else:
#sane db migration hère...


On 8/26/13 1:59 PM, "Sandy Walsh"  wrote:

>I'm getting the same problem with a different migration (mine is
>complaining that a column already exists)
>
>http://paste.openstack.org/show/44512/
>
>I've compared it to the other migrations and it seems fine.
>
>-S
>
>On 08/26/2013 02:34 PM, Jay Pipes wrote:
>> Hey all,
>> 
>> I'm trying to figure out what is going wrong with my code for this
>>patch:
>> 
>> https://review.openstack.org/41316
>> 
>> I had previously added a sqlalchemy-migrate migration script to add an
>> event_type table, and had that working, but then was asked to instead
>> use Alembic for migrations. So, I removed the sqlalchemy-migrate
>> migration file and added an Alembic migration [1].
>> 
>> Unfortunately, I am getting the following error when running tests:
>> 
>> OperationalError: (OperationalError) table event_type already exists
>> u'\nCREATE TABLE event_type (\n\tid INTEGER NOT NULL, \n\t"desc"
>> VARCHAR(255), \n\tPRIMARY KEY (id), \n\tUNIQUE ("desc")\n)\n\n' ()
>> 
>> The migration adds the event_type table. I've seen this error occur
>> before when using SQLite due to SQLite's ALTER TABLE statement not
>> allowing the rename of a column. In the sqlalchemy-migrate migration, I
>> had a specialized SQLite migration upgrade [2] and downgrade [3] script,
>> but I'm not sure how I am supposed to handle this in Alembic. Could
>> someone help me out?
>> 
>> Thanks,
>> -jay
>> 
>> [1]
>> 
>>https://review.openstack.org/#/c/41316/16/ceilometer/storage/sqlalchemy/a
>>lembic/versions/49036dfd_add_event_types.py
>> 
>> [2]
>> 
>>https://review.openstack.org/#/c/41316/14/ceilometer/storage/sqlalchemy/m
>>igrate_repo/versions/013_sqlite_upgrade.sql
>> 
>> [3]
>> 
>>https://review.openstack.org/#/c/41316/14/ceilometer/storage/sqlalchemy/m
>>igrate_repo/versions/013_sqlite_downgrade.sql
>> 
>> 
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>___
>OpenStack-dev mailing list
>OpenStack-dev@lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Need help with Alembic...

[Jay Pipes]

I just noticed that every single test case for SQL-driver storage is 
executing every single migration upgrade before every single test case run:


https://github.com/openstack/ceilometer/blob/master/ceilometer/tests/db.py#L46

https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/impl_sqlalchemy.py#L153

instead of simply creating a new database schema from the models in the 
current source code base using a call to sqlalchemy.MetaData.create_all().


This results in re-running migrations over and over again, instead of 
having dedicated migration tests that would test each migration 
individually, as is the case in projects like Glance...


Is this intentional?

Best,
-jay

On 08/26/2013 02:59 PM, Sandy Walsh wrote:

I'm getting the same problem with a different migration (mine is
complaining that a column already exists)

http://paste.openstack.org/show/44512/

I've compared it to the other migrations and it seems fine.

-S

On 08/26/2013 02:34 PM, Jay Pipes wrote:

Hey all,

I'm trying to figure out what is going wrong with my code for this patch:

https://review.openstack.org/41316

I had previously added a sqlalchemy-migrate migration script to add an
event_type table, and had that working, but then was asked to instead
use Alembic for migrations. So, I removed the sqlalchemy-migrate
migration file and added an Alembic migration [1].

Unfortunately, I am getting the following error when running tests:

OperationalError: (OperationalError) table event_type already exists
u'\nCREATE TABLE event_type (\n\tid INTEGER NOT NULL, \n\t"desc"
VARCHAR(255), \n\tPRIMARY KEY (id), \n\tUNIQUE ("desc")\n)\n\n' ()

The migration adds the event_type table. I've seen this error occur
before when using SQLite due to SQLite's ALTER TABLE statement not
allowing the rename of a column. In the sqlalchemy-migrate migration, I
had a specialized SQLite migration upgrade [2] and downgrade [3] script,
but I'm not sure how I am supposed to handle this in Alembic. Could
someone help me out?

Thanks,
-jay

[1]
https://review.openstack.org/#/c/41316/16/ceilometer/storage/sqlalchemy/alembic/versions/49036dfd_add_event_types.py

[2]
https://review.openstack.org/#/c/41316/14/ceilometer/storage/sqlalchemy/migrate_repo/versions/013_sqlite_upgrade.sql

[3]
https://review.openstack.org/#/c/41316/14/ceilometer/storage/sqlalchemy/migrate_repo/versions/013_sqlite_downgrade.sql


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Need help with Alembic...

[Sandy Walsh]

I'm getting the same problem with a different migration (mine is
complaining that a column already exists)

http://paste.openstack.org/show/44512/

I've compared it to the other migrations and it seems fine.

-S

On 08/26/2013 02:34 PM, Jay Pipes wrote:
> Hey all,
> 
> I'm trying to figure out what is going wrong with my code for this patch:
> 
> https://review.openstack.org/41316
> 
> I had previously added a sqlalchemy-migrate migration script to add an
> event_type table, and had that working, but then was asked to instead
> use Alembic for migrations. So, I removed the sqlalchemy-migrate
> migration file and added an Alembic migration [1].
> 
> Unfortunately, I am getting the following error when running tests:
> 
> OperationalError: (OperationalError) table event_type already exists
> u'\nCREATE TABLE event_type (\n\tid INTEGER NOT NULL, \n\t"desc"
> VARCHAR(255), \n\tPRIMARY KEY (id), \n\tUNIQUE ("desc")\n)\n\n' ()
> 
> The migration adds the event_type table. I've seen this error occur
> before when using SQLite due to SQLite's ALTER TABLE statement not
> allowing the rename of a column. In the sqlalchemy-migrate migration, I
> had a specialized SQLite migration upgrade [2] and downgrade [3] script,
> but I'm not sure how I am supposed to handle this in Alembic. Could
> someone help me out?
> 
> Thanks,
> -jay
> 
> [1]
> https://review.openstack.org/#/c/41316/16/ceilometer/storage/sqlalchemy/alembic/versions/49036dfd_add_event_types.py
> 
> [2]
> https://review.openstack.org/#/c/41316/14/ceilometer/storage/sqlalchemy/migrate_repo/versions/013_sqlite_upgrade.sql
> 
> [3]
> https://review.openstack.org/#/c/41316/14/ceilometer/storage/sqlalchemy/migrate_repo/versions/013_sqlite_downgrade.sql
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Infra] Meeting Tuesday August 27th at 19:00 UTC

[Elizabeth Krumbach Joseph]

The OpenStack Infrastructure (Infra) team is hosting our weekly
meeting tomorrow, Tuesday August 27th, at 19:00 UTC in
#openstack-meeting

Meeting agenda available here:
https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting (anyone is
welcome to to add agenda items)

Everyone interested in infrastructure and process surrounding
automated testing and deployment is encouraged to attend.

-- 
Elizabeth Krumbach Joseph || Lyz || pleia2
http://www.princessleia.com

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [savanna] Fwd: Savanna - Hadoop on OpenStack

[Matthew Farrellee]

FYI


 Original Message 
Subject: Savanna - Hadoop on OpenStack
Date: Mon, 26 Aug 2013 14:29:44 -0400
From: Matthew Farrellee 
Reply-To: Fedora Big Data SIG 
To: Fedora Big Data SIG 

Hello Big Data SIG folks,

If you aren't familiar, Savanna is an OpenStack project that provides
Hadoop cluster and workload management. Cluster - construct and manage
the lifecycle of Hadoop clusters. Workload - workflow for big data
processing with Hadoop (similar to AWS EMR).

The project home page is https://launchpad.net/savanna

Savanna is made up of 4 sub-projects -
  . savanna, the main services
  . savannadashboard, web UI integration with OpenStack Horizon
  . python-savannaclient, python bindings for the REST API
  . savanna-extra - diskimage-builder elements for...image building

As of today all those are available in F19, F20, and EL6*

openstack-savanna - https://bugzilla.redhat.com/show_bug.cgi?id=986615
python-django-savanna - https://bugzilla.redhat.com/show_bug.cgi?id=998123
python-savannaclient - https://bugzilla.redhat.com/show_bug.cgi?id=998701
savanna-image-elements - https://bugzilla.redhat.com/show_bug.cgi?id=998702

Thanks for all the community help getting Savanna into Fedora,
especially the #rdo folks.

With any luck the project will have Fedora based cloud images with its
next release. Right now all the images are Ubuntu based.

Best,


matt

* openstack-savanna (package for savanna sub-project) has a dep on
pycrypto and isn't available on EL6 yet, savanna-image-elements depends
on diskimage-builder which isn't included in EL6 yet

-

Install -

# yum --enablerepo=updates-testing install openstack-savanna
python-django-savanna (make sure you get python-django-savanna-0.2-2)

Setup & start savanna-api service -

# sed -i "s/^#os_admin_password=/os_admin_password=$OS_PASSWORD/"
/etc/savanna/savanna.conf
# systemctl start openstack-savanna-api

Setup and load Dashboard plugin -

# echo "SAVANNA_URL = 'http://localhost:8386/v1.0'" >>
/etc/openstack-dashboard/local_settings

Edit /usr/share/openstack-dashboard/openstack_dashboard/settings.py -
  . HORIZON_CONFIG = {
 'dashboards': ('project', 'admin', 'settings', 'savanna'),
  . INSTALLED_APPS = (
 'savannadashboard',
 'openstack_dashboard',

# systemctl reload httpd
___
bigdata mailing list
bigd...@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/bigdata



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Pipeline Retry Semantics ...

[Monsyne Dragon]

I added some thoughts to this.

On 8/15/13 11:20 AM, "Sandy Walsh"  wrote:

>Recently I've been focused on ensuring we don't drop notifications in
>CM. But problems still exist downstream, after we've captured the raw
>event.
>
>From the efforts going on with the Ceilometer sample pipeline, the new
>dispatcher model and the upcoming trigger pipeline, the discussion
>around retry semantics has being coming up a lot.
>
>In other words "What happens when step 4 of a 10 step pipeline fails?"
>
>As we get more into processing billing events, we really need to have a
>solid understanding of how we prevent double-counting or dropping events.
>
>I've started writing down some thoughts here:
>https://wiki.openstack.org/wiki/DuplicateWorkCeilometer
>
>It's a little scattered and I'd like some help tuning it.
>
>Hopefully it'll help grease the skids for the Icehouse Summit talks.
>
>Thanks!
>-S
>
>cc/ Josh, I think the State Management team can really help out here.
>
>___
>OpenStack-dev mailing list
>OpenStack-dev@lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Secure live VM migration in cloud (openstack)

[Naveed Ahmad]

Respected Joshua Harlow,

Thanks for reply,

Based on literature survey i found that following techniques are used for
secure live migration of vm.

1. RSA with SSL protocol for authentication and encryption.
As you mentioned earlier same problem is in RSA based authentication. we
have to add public keys of all other hypervisors.

In Blackhat 2013, security research found vulnerability in SSL so it can be
breakable in very short time.
please check
http://arstechnica.com/security/2013/08/gone-in-30-seconds-new-attack-plucks-secrets-from-https-protected-pages/

2. SSH is used for secure tunnel before live vm migration.

Authentication is not discussed, only secure tunnel is used to achieve
confidentiality.

3. Openstack uses libvirtd with kvm to provide secure vm migration between
src and dst machine.
SSL is used for encrypted channel and SASL  is used for authentication.



so i am interested to implement authentication level's in live vm migration.

1.no authentication
2. Certificate base
3.smart card based authentication

and similarly ssl provide secure channel but after that seaprate VLAN is
used for vm migration traffic. if we use ipsec then we can achieve same
goal on network layer to hide all communication of vm migration.



Regards
Naveed









On Mon, Aug 26, 2013 at 2:44 AM, Joshua Harlow wrote:

>  Arg, hit send to quick.
>
>  *likely these problems would require some managed migration "thing" that
> would temporarily open the network access, issue temporary auth keys and
> the initiate the migration between the 2 hypervisors. Is this in your
> scope, to make this thing??
>
>
> Sent from my really tiny device...
>
> On Aug 25, 2013, at 2:42 PM, "Joshua Harlow" 
> wrote:
>
>   Hi,
>
>  I think it's a good idea, can u describe more what would be different,
> would there be a new auth and live migration mechanism?
>
>  I think one of the problems at least yahoo has is that live migration
> requires all ssh keys to be on all hypervisors since hypervisors (libvirtd)
> open up the connection to the hypervisor to be migrated to. This is
> obviously bad, as any hacker if they can get out of a vm now can start
> issuing these migration requests. Also at yahoo we don't allow hypervisors
> to communicate openly to each other, this is protected at the network
> level. Would u be working on solutions to these problems (likely involving
>
> Sent from my really tiny device...
>
> On Aug 25, 2013, at 6:33 AM, "Naveed Ahmad" <12msccsnah...@seecs.edu.pk>
> wrote:
>
>
>  thanks for replying Joshua,
>
>
>  VM migration is the process used to migrate vm from one physical server
> to another physical server due to many reasons like system maintenance,
> hardware failure ,
>
>  VM is important element in cloud as well, so we do same in the cloud.
> xen/kvm hypervisor used in the openstack dont provide security  in this
> process. i studied few paper on it  which are related to VM migration in DC
> instead of Cloud.   i also seen book on openstack security in which it is
> describe that xen/kvm could not provide security but libvirt can be used
> with xen/kvm to secure this process.
>
>  Currently libvirt is providing ssl for confidentiality of data between
> source and destination. and SASL for authentication. i want to add other
> authentication mechanism in it and in the end it would be added in the
> Dashboard of openstack so that administrator use it easily, Access control
> is also part of this thesis..
>
>
>  may you got my idea Mr. Joshua Harlow and now please comment on it. is
> it good or not? your comment will help me to choose good topic in cloud
> security,
>
>
>  Regards
>
>
>
>
>
>
>
>
>
>
> On Sun, Aug 25, 2013 at 4:17 AM, Joshua Harlow wrote:
>
>> Is there any write up of what u want to do or is that not defined yet?
>>
>> If u can write up some information I think that would help others provide
>> feedback as well as help everyone (including yourself) see the goal too be
>> accomplished. It's hard to tell what the desired outcome is otherwise,
>> secure vm migration could mean a lot of things :)
>>
>> Sent from my really tiny device...
>>
>> On Aug 24, 2013, at 12:26 PM, "Naveed Ahmad" <12msccsnah...@seecs.edu.pk>
>> wrote:
>>
>> >
>> >
>> > Hi all,
>> >
>> >
>> >
>> > I am doing thesis in cloud computing security domain, i selected to
>> secure vm migration  process in openstack.
>> > Please let me know about this idea. i have done some initial work on
>> it. i need comment of you people which will be helpful for me.
>> >
>> >
>> >
>> >
>> > Thanks and Regards
>> >
>> >
>>  > ___
>> > OpenStack-dev mailing list
>> > OpenStack-dev@lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>___

Re: [openstack-dev] [OpenStack][Cinder] Driver qualification

[Duncan Thomas]

On 30 July 2013 21:36, Mark Washenberger  wrote:
> - I'd love to use this as a way to OpenStack-validate plugins that live
> outside of the project--and then kick most plugins out of the project to
> some other semi-canonical area.

I don't think I'm alone within the cinder team in seeing significant
benefit to keeping plugins within the cinder code base - they benefit
not just from the eyes of reviewers, but also from the mindshare of
developers. Since we know (or can go and look) at how various plugins
are using (or abusing) the interfaces we provide, we can keep that in
mind when making changes, and fix up obvious cases. Once things are in
a separate repo, you get all sorts of painful version skew type
problems, and everybody has less insight into how the code behaves as
a whole.

> - I don't think the test infrastructure should depend wholly on
> devstack--while that makes a ton of sense for plugins that depend on other
> OpenStack services working, I think its pretty crufty that we have plugins
> that depend on other openstack services. I'd rather not to weigh the good
> down with the bad.

The nature of some of the cinder drivers is that they require some
other services - e.g. some drivers snapshot to glace, all drivers
(should be able to) backup to glance, some drivers are tenant aware,
etc. Devstack is as good a 'standard platform' as we've got at the
moment... You could allow the validation suite to be run against any
openstack install, but then automated collection of the cinder.conf
and similar niceties becomes much harder.

+1 for John's approach

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Pipeline Retry Semantics ...

[Joshua Harlow]

Hey, seems like taskflow could help here also.

Check out the twiki which should have some good details for u guys: 
https://wiki.openstack.org/wiki/TaskFlow

Hit me up on irc if u guys want to talk :-)

#openstack-state-management

From: Doug Hellmann 
mailto:doug.hellm...@dreamhost.com>>
Reply-To: OpenStack Development Mailing List 
mailto:openstack-dev@lists.openstack.org>>
Date: Friday, August 16, 2013 1:24 PM
To: OpenStack Development Mailing List 
mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [Ceilometer] Pipeline Retry Semantics ...

I added a couple of comments in the wiki page. We should have at least one 
summit session about this, I think, unless we work it out before then.


On Thu, Aug 15, 2013 at 12:20 PM, Sandy Walsh 
mailto:sandy.wa...@rackspace.com>> wrote:
Recently I've been focused on ensuring we don't drop notifications in
CM. But problems still exist downstream, after we've captured the raw
event.

>From the efforts going on with the Ceilometer sample pipeline, the new
dispatcher model and the upcoming trigger pipeline, the discussion
around retry semantics has being coming up a lot.

In other words "What happens when step 4 of a 10 step pipeline fails?"

As we get more into processing billing events, we really need to have a
solid understanding of how we prevent double-counting or dropping events.

I've started writing down some thoughts here:
https://wiki.openstack.org/wiki/DuplicateWorkCeilometer

It's a little scattered and I'd like some help tuning it.

Hopefully it'll help grease the skids for the Icehouse Summit talks.

Thanks!
-S

cc/ Josh, I think the State Management team can really help out here.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Blueprint for Nova native image building

[Russell Bryant]

I believe this is where the thread stopped a couple weeks ago.  I was
just curious what has happened since.  How did you interpret all of the
feedback given, and what direction have you decided to take next?

Thanks,

-- 
Russell Bryant

On 08/08/2013 12:42 PM, Mark Washenberger wrote:
> There is a current proposal in glance that is receiving development
> attention towards "importing" images asynchronously from a variety of
> sources. The import feature is plugin-based, so it would be easy to add
> on the ability and a plugin to do something like importing base os installs.
> 
> The blueprint
> is https://blueprints.launchpad.net/glance/+spec/new-upload-workflow. It
> is currently targeted for Havana-3 (but is probably the first blueprint
> on the chopping block due to other dependencies that have not yet landed).
> 
> I think this approach probably makes more sense than putting the code
> directly into Nova. But overall, I'm somewhat in favor of keeping this
> feature out of core OpenStack projects for now. It feels niche enough
> that it could live as its own project without burdening its users--most
> folks who build base images are probably operations anyway and can
> deploy stuff. And I think there are a number of other tools perhaps that
> make it easier for the smallest shops to build base images (?)
> 
> I don't buy the argument about it being a lot more work to implement
> this feature outside of OpenStack. Not that the argument is false, but
> the concern seems minor compared to the cost of weighing down core with
> yet another feature. From where I'm sitting, OpenStack is still in the
> "too many features coming too fast" regime and architecture hasn't
> caught up. So putting on the breaks wherever possible seems like the
> wisest course.
> 
> 
> On Thu, Aug 8, 2013 at 7:33 AM, Daniel P. Berrange  > wrote:
> 
> On Thu, Aug 08, 2013 at 09:28:51AM -0500, Ian McLeod wrote:
> > On Wed, 2013-08-07 at 12:05 -0400, Russell Bryant wrote:
> > > On 08/07/2013 10:46 AM, Daniel P. Berrange wrote:
> > > > On Tue, Aug 06, 2013 at 12:20:00PM -0400, Russell Bryant wrote:
> > > >> On 08/06/2013 11:53 AM, Ian Mcleod wrote:
> > > >>> Hello,
> > > >>>
> > > >>> A blueprint has been registered regarding API additions to
> Nova to
> > > >>> enable the creation of base images from external OS install
> sources.
> > > >>> This provides a way to build images from scratch via native
> OS installer
> > > >>> tools using only the resources provided through Nova.  These
> images can
> > > >>> then be further customized by other tools that expect an
> existing image
> > > >>> as an input, such as disk image builder.
> > > >>>
> > > >>> Blueprint -
> > > >>> https://blueprints.launchpad.net/nova/+spec/base-image-creation
> > > >>>
> > > >>> Specification -
> https://wiki.openstack.org/wiki/NovaImageCreationAPI
> > > >>>
> > > >>> If this is a topic that interests you, please have a look
> (the spec is
> > > >>> not very long) and join the conversation.
> > > >>>
> > > >>> Please note that this blueprint follows on from proof of
> concept work
> > > >>> for native image building discussed on this list in April:
> > > >>>
> > > >>>
> http://lists.openstack.org/pipermail/openstack-dev/2013-April/007157.html
> > > >>
> > > >> Thanks of the update on this work.
> > > >>
> > > >> I see that your proof of concept shows how this can work as a
> tool
> > > >> outside of Nova:
> > > >>
> > > >> https://github.com/redhat-openstack/image-building-poc
> > > >>
> > > >> So, my biggest question is whether or not it makes sense for
> this to be
> > > >> a Nova feature or not.  If something can be implemented as a
> consumer of
> > > >> Nova, my default answer is that it should stay outside of
> nova until I
> > > >> am convinced otherwise.  :-)
> > > >>
> > > >> It sounds like this is mostly an extension to nova that
> implements a
> > > >> series of operations that can be done just as well outside of
> Nova.  Are
> > > >> there enhancements you are making or scenarios that won't
> work at all
> > > >> unless it lives inside of Nova?
> > > >>
> > > >> If it doesn't end up on the server side, it could potentially be
> > > >> implemented as an extension to novaclient.
> > > >
> > > > I think the key thing is that want to make sure we don't have all
> > > > clients apps having to re-invent the wheel. The way the proof of
> > > > concept was done as a standalone tool, would entail such wheel
> > > > re-invention by any frontend to Nova like the 'nova' cli and
> > > > Horizon dashboard. Possibly you could mitigate that if you could
> > > > actually put all the smarts in the python-novaclient library API
> > > > so it was shared by all frontend

Re: [openstack-dev] [Cinder] Snapshot List support

[Duncan Thomas]

On 3 August 2013 17:23, Scott Devoid  wrote:

> Hijacking your thread for a moment: I would think that a
> "revert_volume_to_snapshot" function would be useful. Is this implemented
> via create_volume_from_snapshot, i.e. snapshot.volume == volume in the
> arguments? Or does this functionality not exist?

A request for this comes up from time to time, but I've never seen an
example use-case where creating a new volume from the snapshot, (and
potentially deleting the old one) has any disadvantage beyond 'that
isn't the way I thought about the problem'. In general, in the cloud
world, you shouldn't get overly attached to a volume id... the data
behind it is what matters.

The only differences between I can see between 'restore volume from
snapshot' and 'create new volume from snapshot and delete the old
one', other than the volume id changes are all disadvantages:
- It's a new API call that needs implementing and testing
- If the operation encounters an error for some reason, you have
neither to old nor the new volume to work with

Certainly there are a few people on the cinder team who have talked
about adding the same interface, so it is entirely possible I'm
missing something - please don't be shy in telling me so!


Regards

-- 
Duncan Thomas

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ceilometer] Need help with Alembic...

[Jay Pipes]

Hey all,

I'm trying to figure out what is going wrong with my code for this patch:

https://review.openstack.org/41316

I had previously added a sqlalchemy-migrate migration script to add an 
event_type table, and had that working, but then was asked to instead 
use Alembic for migrations. So, I removed the sqlalchemy-migrate 
migration file and added an Alembic migration [1].


Unfortunately, I am getting the following error when running tests:

OperationalError: (OperationalError) table event_type already exists 
u'\nCREATE TABLE event_type (\n\tid INTEGER NOT NULL, \n\t"desc" 
VARCHAR(255), \n\tPRIMARY KEY (id), \n\tUNIQUE ("desc")\n)\n\n' ()


The migration adds the event_type table. I've seen this error occur 
before when using SQLite due to SQLite's ALTER TABLE statement not 
allowing the rename of a column. In the sqlalchemy-migrate migration, I 
had a specialized SQLite migration upgrade [2] and downgrade [3] script, 
but I'm not sure how I am supposed to handle this in Alembic. Could 
someone help me out?


Thanks,
-jay

[1] 
https://review.openstack.org/#/c/41316/16/ceilometer/storage/sqlalchemy/alembic/versions/49036dfd_add_event_types.py
[2] 
https://review.openstack.org/#/c/41316/14/ceilometer/storage/sqlalchemy/migrate_repo/versions/013_sqlite_upgrade.sql
[3] 
https://review.openstack.org/#/c/41316/14/ceilometer/storage/sqlalchemy/migrate_repo/versions/013_sqlite_downgrade.sql


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Neutron + Grenade (or other upgrade testing)

[Dean Troyer]

On Mon, Aug 26, 2013 at 10:50 AM, Maru Newby  wrote:

> Is anyone working on/planning on adding support for neutron to grenade?
>  Or is there any other automated upgrade testing going on for neutron?
>

We deliberately avoided migrations in Grenade (like Nova Volume -> Cinder)
as we wanted to focus on upgrades within projects.  Migrations will
necessarily be much more complicated, especially Nova Network -> Neutron.
 At some point Neutron should be added to Grenade, but only as a release
upgrade step for some basic configuration.

That said, I'm sure there would be great appreciation for a recipe to
duplicate an existing Nova Network config in Neutron.  We can debate if
that belongs in Grenade should it ever exist...

dt

-- 

Dean Troyer
dtro...@gmail.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Neutron + Grenade (or other upgrade testing)

[Maru Newby]

Is anyone working on/planning on adding support for neutron to grenade?  Or is 
there any other automated upgrade testing going on for neutron?

Thanks,


m.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon] Add a library js for creating charts

[Maxime Vidori]

Hi,

Currently, the charts for Horizon are directly created with D3. Maybe if we add 
a js library on top of d3 it will be easier and development will be faster. A 
blueprint was created at 
https://blueprints.launchpad.net/horizon/+spec/horizon-chart.js We actually 
need some reviews or feedback.

Maxime Vidori

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Murano] Meeting Minutes 2013-08-26

[Denis Koryavov]

Hello,

Below, you can see the meeting minutes from today's Murano meeting.

Minutes:
http://eavesdrop.openstack.org/meetings/murano/2013/murano.2013-08-26-15.00.html
Minutes (text):
http://eavesdrop.openstack.org/meetings/murano/2013/murano.2013-08-26-15.00.txt
Log:
http://eavesdrop.openstack.org/meetings/murano/2013/murano.2013-08-26-15.00.log.html

The next meeting will be 2th Sep. Have a nice day.

--
Denis
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Updating global requirements -- pyflakes version conflict

[Joe Gordon]

Hi All,

Now that we have global requirements and projects are starting to sync up
with them, many are seeing an issue with pyflakes version conflicts.

Initially hacking did not pin pep8, pyflakes and flake8.  But as of hacking
0.5.5 this changed,
https://github.com/openstack-dev/hacking/commit/ea0dc8b7b08eeaf14609130a3ff3f3ddb5cf2b62,
only after we added pep8, pyflakes and flake8 to all test-requirements.txt

So now pep8, pyflakes and flake8 can be removed from test-requirements.txt,
as they are just dependencies of hacking and thus indirect dependencies of
any project that uses hacking.  Several projects including tempest have
begun doing this (https://review.openstack.org/#/c/43168/5).  And because
we have 53 repos in the openstack namespace (github.com/openstack) it is
easier to send this out to hunt down 53 patches and add comments to them.

best,
Joe Gordon
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Code review study

[Joe Gordon]

On Tue, Aug 20, 2013 at 2:21 PM, Clint Byrum  wrote:

> Excerpts from Mark McLoughlin's message of 2013-08-20 03:26:01 -0700:
> > On Thu, 2013-08-15 at 14:12 +1200, Robert Collins wrote:
> > > This may interest data-driven types here.
> > >
> > >
> https://www.ibm.com/developerworks/rational/library/11-proven-practices-for-peer-review/
> > >
> > > Note specifically the citation of 200-400 lines as the knee of the
> review
> > > effectiveness curve: that's lower than I thought - I thought 200 was
> > > clearly fine - but no.
> >
> > The full study is here:
> >
> >
> http://support.smartbear.com/resources/cc/book/code-review-cisco-case-study.pdf
> >
> > This is an important subject and I'm glad folks are studying it, but I'm
> > sceptical about whether the "Defect density vs LOC" is going to help us
> > come up with better guidelines than we have already.
> >
> > Obviously, a metric like LOC hides some serious subtleties. Not all
> > changes are of equal complexity. We see massive refactoring patches
> > (like s/assertEquals/assertEqual/) that are actually safer than gnarly,
> > single-line, head-scratcher bug-fixes. The only way the report addresses
> > that issue with the underlying data is by eliding >10k LOC patches.
> >
>
> I'm not so sure that it is obvious what these subtleties are, or they
> would not be subtleties, they would be glaring issues.
>
> I agree that LOC changed is an imperfect measure. However, so are the
> hacking rules. They, however, have allowed us to not spend time on these
> things. We whole-heartedly embrace an occasional imperfection by deferring
> to something that can be measured by automation and thus free up valuable
> time for other activities more suited to limited reviewer/developer time.
>
> I'd like to see automation enforce change size. And just like with
> hacking, it would not be possible without a switch like "#noqa" that
> one can put in the commit message that says "hey automation, this is
> a trivial change".  That is something a reviewer can also see as a cue
> that this change, while big, should be trivial.
>

++

I put this into a bug: https://bugs.launchpad.net/hacking/+bug/1216928, and
added a related bug about making hacking git-checks run as post-commit
hooks (https://bugs.launchpad.net/hacking/+bug/1216925).


>
> > The "one logical change per commit" is a more effective guideline than
> > any LOC based guideline:
> >
> >
> https://wiki.openstack.org/wiki/GitCommitMessages#Structural_split_of_changes
> >
> > IMHO, the number of distinct logical changes in a patch has a more
> > predictable impact on review effectiveness than the LOC metric.
>
> Indeed, however, automating a check for that may be very difficult. I
> have seen tools like PMD[1] that try very hard to measure the complexity
> of code and the risk of change, and those might be interesting to see,
> but I'm not sure they are reliable enough to put in the OpenStack gate.
>
> [1] http://pmd.sourceforge.net/
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] [Cinder] - Cannot create snapshots from Horizon

[Karajgi, Rohit]

Hi,

Refer Horizon bug[1]: - Cannot create a volume snapshot from Dashboard. 
This is a critical bug that is yet to be fixed in Horizon, but the patch[2] 
that is submitted is still under some discussion.
Patch: 

The real fix that fill address this issue only requires Horizon to use limits 
API template instead of quota API template.
However, the create snapshot page does not display the usage information 
accurately (as per comment 
https://bugs.launchpad.net/horizon/+bug/1194506/comments/1)
This issue could be addressed in multiple ways, but it IMO the right approach 
to fix it would be to implement the Cinder limits API blueprint[3]
and then fix the Horizon display issue in Horizon's patch[2]. 

I think these issues should get fixed before Havana milestone-3.
Opinions would be greatly appreciated.

[1] https://bugs.launchpad.net/horizon/+bug/1194506
[2] https://review.openstack.org/#/c/40588/
[3] https://blueprints.launchpad.net/cinder/+spec/add-volume-usages-to-limits


Best Regards,
Rohit Karajgi | Technical Analyst | NTT Data Global Technology Services Private 
Ltd | w. +91.20.6604.1500 x 627 |  m. +91 992.242.9639 | 
rohit.kara...@nttdata.com


__
Disclaimer:This email and any attachments are sent in strictest confidence for 
the sole use of the addressee and may contain legally privileged, confidential, 
and proprietary data.  If you are not the intended recipient, please advise the 
sender by replying promptly to this email and then delete and destroy this 
email and any attachments without any further use, copying or forwarding

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glance] import task meeting moved to 15:00 UTC Tues 27 August

[Brian Rosmaita]

Sorry for the repost, just want to make sure everyone is aware that the meeting 
was moved.

From: Brian Rosmaita [brian.rosma...@rackspace.com]
Sent: Friday, August 23, 2013 1:48 PM
To: OpenStack Development Mailing List; Joshua Harlow; Jessica Lucci
Subject: [openstack-dev] [Glance] import task meeting moved to 15:00 UTC
Tues 27 August

We'll now be holding the below meeting at 15:00 UTC Tuesday 27 August in 
#openstack-glance on Freenode.

From: Brian Rosmaita [brian.rosma...@rackspace.com]
Sent: Thursday, August 22, 2013 5:33 PM
To: Joshua Harlow; OpenStack Development Mailing List
Subject: [openstack-dev] [Glance] import task meeting 15:00 UTC Monday 26   
August

We're shooting for 15:00 UTC Monday 26 August.  (Let me know ASAP if you really 
want to be in on this and can't make it, but this is probably the best time 
available.)

Topics:
(1) taskflow seam for integration
(2) tasks api and executor interface
(3) indexable column in db for tasks to be queryable

See 
http://eavesdrop.openstack.org/meetings/glance/2013/glance.2013-08-22-20.02.log.html
 for brief discussion of these topics during today's glance meeting.

More info:
https://etherpad.openstack.org/havana-glance-requirements
https://etherpad.openstack.org/LG39UnQA7z
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glance] import task meeting 15:00 UTC Monday 26 August

[Brian Rosmaita]

This thread continued past the announcement that the meeting was moved to 
Tuesday ... so, no task meeting today (Monday), we will be meeting at 15:00 UTC 
Tuesday 27 August in #openstack-glance on Freenode.

Topics:
(1) taskflow seam for integration
(2) tasks api and executor interface
(3) indexable column in db for tasks to be queryable

See 
http://eavesdrop.openstack.org/meetings/glance/2013/glance.2013-08-22-20.02.log.html
 for brief discussion of these topics during today's glance meeting.

More info:
https://etherpad.openstack.org/havana-glance-requirements
https://etherpad.openstack.org/LG39UnQA7z


From: Flavio Percoco [fla...@redhat.com]
Sent: Monday, August 26, 2013 4:12 AM
To: jbres...@redhat.com; OpenStack Development Mailing List
Subject: Re: [openstack-dev] [Glance] import task meeting 15:00 UTC Monday 26 
August

On 23/08/13 07:52 -1000, John Bresnahan wrote:
>On 08/23/2013 07:15 AM, Joshua Harlow wrote:
>> I'm fine with either times. So maybe 1600 for all?
>
>That time works for me.

Works for me!

--
@flaper87
Flavio Percoco

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Glance revert to fix image deletes on F19

[Dan Prince]

Hi all,

I'd like to highlight this Glance revert for an issue I started seeing on 
Fedora 19 last week.

  https://review.openstack.org/#/c/43542/

A revert like this should be a very safe thing to do so long as we do it 
quickly. Especially given this is a performance improvement...

Dan

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] AUTO: Jarrod B Johnson is unavailable (returning 09/03/2013)

[Jarrod B Johnson]

I am out of the office until 09/03/2013.

Contact my mannager Lalecha Watkins for assisstance


Note: This is an automated response to your message  "[openstack-dev]
[Nova][vmware] VMwareAPI sub-team reviews update 2013-08-23" sent on
08/23/2013 17:50:33.

This is the only notification you will receive while this person is away.___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Code review study

[Gary Kotton]

> -Original Message-
> From: Flavio Percoco [mailto:fla...@redhat.com]
> Sent: Monday, August 26, 2013 11:41 AM
> To: OpenStack Development Mailing List
> Subject: Re: [openstack-dev] Code review study
> 
> On 20/08/13 11:24 -0400, Russell Bryant wrote:
> >On 08/20/2013 11:08 AM, Daniel P. Berrange wrote:
> >> On Tue, Aug 20, 2013 at 04:02:12PM +0100, Mark McLoughlin wrote:
> >>> On Tue, 2013-08-20 at 11:26 +0100, Mark McLoughlin wrote:
>  On Thu, 2013-08-15 at 14:12 +1200, Robert Collins wrote:
> > This may interest data-driven types here.
> >
> > https://www.ibm.com/developerworks/rational/library/11-proven-
> prac
> > tices-for-peer-review/
> >
> > Note specifically the citation of 200-400 lines as the knee of the
> > review effectiveness curve: that's lower than I thought - I
> > thought 200 was clearly fine - but no.
> 
>  The full study is here:
> 
>  http://support.smartbear.com/resources/cc/book/code-review-cisco-
> ca
>  se-study.pdf
> 
>  This is an important subject and I'm glad folks are studying it,
>  but I'm sceptical about whether the "Defect density vs LOC" is
>  going to help us come up with better guidelines than we have already.
> 
>  Obviously, a metric like LOC hides some serious subtleties. Not all
>  changes are of equal complexity. We see massive refactoring patches
>  (like s/assertEquals/assertEqual/) that are actually safer than
>  gnarly, single-line, head-scratcher bug-fixes. The only way the
>  report addresses that issue with the underlying data is by eliding >10k
> LOC patches.
> 
>  The "one logical change per commit" is a more effective guideline
>  than any LOC based guideline:
> 
>  https://wiki.openstack.org/wiki/GitCommitMessages#Structural_split_
>  of_changes
> 
>  IMHO, the number of distinct logical changes in a patch has a more
>  predictable impact on review effectiveness than the LOC metric.
> >>>
> >>> Wow, I didn't notice Joe had started to enforce that here:
> >>>
> >>>   https://review.openstack.org/41695
> >>>
> >>> and the exact example I mentioned above :)
> >>>
> >>> We should not enforce rules like this blindly.
> >>
> >> Agreed, lines of code is a particularly poor metric for evaluating
> >> commits on.
> >
> >Agreed, I would _strongly_ prefer no enforcement around LOC.  It's just
> >not the right metric to be looking at for a hard rule.
> >
> 
> Agreed. I think we should focus on other things like per feature patches,
> which make more sense. Huge patches can be split in several ones - most of
> the time - which will implicitly enforce a LOC limit but will let patches like
> s/assertEquals/assertEqual/ land, which make sense to me.
> 
> This should be evaluated in a case-by-case basis.

[Gary Kotton] Agreed. The reviewers should use their discretion when it comes 
to the patch size. The flip side is that there are times when small patches 
based on on top of another fail to provide a complete picture of the change.

> 
> --
> @flaper87
> Flavio Percoco
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Murano] Release v0.2 Feature Freeze

[Alexander Tivelkov]

Hi folks,

As Murano project is close to its next stable release (v0.2, which is
scheduled on September, 5), we are announcing a feature freeze for the
branch 'release-0.2' of Murano repository.

Effective now, all the new features and improvements are supposed to go to
'master' branch. Everything which is pushed to the 'release-0.2' should
address an open bug related to the upcoming release or will be rejected.

Thanks!

--
Kind Regards,
Alexander Tivelkov
Principal Software Engineer

OpenStack Platform Product division
Mirantis, Inc

+7(495) 640 4904, ext 0236
+7-926-267-37-97(cell)
Vorontsovskaya street 35 B, building 3,
Moscow, Russia.
ativel...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Code review study

[Flavio Percoco]

On 20/08/13 11:24 -0400, Russell Bryant wrote:

On 08/20/2013 11:08 AM, Daniel P. Berrange wrote:

On Tue, Aug 20, 2013 at 04:02:12PM +0100, Mark McLoughlin wrote:

On Tue, 2013-08-20 at 11:26 +0100, Mark McLoughlin wrote:

On Thu, 2013-08-15 at 14:12 +1200, Robert Collins wrote:

This may interest data-driven types here.

https://www.ibm.com/developerworks/rational/library/11-proven-practices-for-peer-review/

Note specifically the citation of 200-400 lines as the knee of the review
effectiveness curve: that's lower than I thought - I thought 200 was
clearly fine - but no.


The full study is here:

http://support.smartbear.com/resources/cc/book/code-review-cisco-case-study.pdf

This is an important subject and I'm glad folks are studying it, but I'm
sceptical about whether the "Defect density vs LOC" is going to help us
come up with better guidelines than we have already.

Obviously, a metric like LOC hides some serious subtleties. Not all
changes are of equal complexity. We see massive refactoring patches
(like s/assertEquals/assertEqual/) that are actually safer than gnarly,
single-line, head-scratcher bug-fixes. The only way the report addresses
that issue with the underlying data is by eliding >10k LOC patches.

The "one logical change per commit" is a more effective guideline than
any LOC based guideline:

https://wiki.openstack.org/wiki/GitCommitMessages#Structural_split_of_changes

IMHO, the number of distinct logical changes in a patch has a more
predictable impact on review effectiveness than the LOC metric.


Wow, I didn't notice Joe had started to enforce that here:

  https://review.openstack.org/41695

and the exact example I mentioned above :)

We should not enforce rules like this blindly.


Agreed, lines of code is a particularly poor metric for evaluating
commits on.


Agreed, I would _strongly_ prefer no enforcement around LOC.  It's just
not the right metric to be looking at for a hard rule.



Agreed. I think we should focus on other things like per feature
patches, which make more sense. Huge patches can be split in several
ones - most of the time - which will implicitly enforce a LOC limit
but will let patches like s/assertEquals/assertEqual/ land, which make
sense to me.

This should be evaluated in a case-by-case basis.

--
@flaper87
Flavio Percoco

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [IMPORTANT] The Gate around Feature Freeze

[Flavio Percoco]

On 22/08/13 21:37 -0500, Dolph Mathews wrote:


On Thu, Aug 22, 2013 at 7:48 PM, James E. Blair  wrote:
Wow, nice work! Thank you, infra!



You guys ROCK! Thanks for everything!
FF

--
@flaper87
Flavio Percoco

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glance] import task meeting 15:00 UTC Monday 26 August

[Flavio Percoco]

On 23/08/13 07:52 -1000, John Bresnahan wrote:

On 08/23/2013 07:15 AM, Joshua Harlow wrote:

I'm fine with either times. So maybe 1600 for all?


That time works for me.


Works for me!

--
@flaper87
Flavio Percoco

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev