Re: [Openstack] [openstack-dev] [cinder] Proposal for Ollie Leahy to join cinder-core

2013-07-17 Thread Sean Dague

On 07/17/2013 02:35 PM, John Griffith wrote:


Just to point out a few things here, first off there is no guideline
that states a company affiliation should have anything to do with the
decision on voting somebody as core.  I have ABSOLUTELY NO concern about
representation of company affiliation what so ever.

Quite frankly I wouldn't mind if there were 20 core members from HP, if
they're all actively engaged and participating then that's great.  I
don't think there has been ANY incidence of folks exerting inappropriate
influence based on their affiliated interest, and if there ever was I
think it would be easy to identify and address.

As far as "don't need more" I don't agree with that either, if there are
folks contributing and doing the work then there's no reason not to add
them.  Cinder IMO does NOT have an excess of reviewers by a very very
long stretch.

The criteria here should be review consistency and quality as well as
knowledge of the project, nothing more nothing less.  If there's an
objection to the individuals participation or contribution that's fine,
but company affiliation should have no bearing.


+1

The people that do great work on reviews, should really be your review 
team, regardless of affiliation.


-Sean

--
Sean Dague
http://dague.net

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Grizzly Devstack Installation - RHEL 6.4

2013-07-16 Thread Sean Dague
Devstack in grizzly doesn't work with RHEL. If you are interested in 
using devstack you should use Fedora instead.


-Sean

On 07/16/2013 05:58 AM, Subramanian K wrote:

Hello All,

In the process of running devstack installation script on RHEL 6.4 , I
am faced with an error.  Below is the error tracked section

+ /opt/stack/devstack/tools/create_userrc.sh -PA --target-dir
/opt/stack/devstack/accrc
Authorization Failed: Unable to sign token. (HTTP 500)
Authorization Failed: Unable to sign token. (HTTP 500)
ERROR: Unable to sign token. (HTTP 500)
Failed to update the root certificate: /opt/stack/devstack/accrc/cacert.pem
Authorization Failed: Unable to sign token. (HTTP 500)


If I manually try to run any commands , I get the similar error message

nova --os-username admin --os-password XX --os-auth-url
http://127.0.0.1:35357/v2.0 --os-tenant-name demo image-list
ERROR: Unable to sign token. (HTTP 500)


Could someone throw insights in actual cause of this issue ?


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp




--
Sean Dague
http://dague.net

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Problem running devstack / stack.sh

2013-06-18 Thread Sean Dague

On 06/18/2013 12:49 PM, zan tosh wrote:

I am getting stuck with the following errors while installing using
devstack (stable/grizzly).
2013-06-18 09:43:01 + export OS_SERVICE_TOKEN=password
2013-06-18 09:43:01 + OS_SERVICE_TOKEN=password
2013-06-18 09:43:01 + export OS_SERVICE_ENDPOINT=http://localhost:35357/v2.0
2013-06-18 09:43:01 + OS_SERVICE_ENDPOINT=http://localhost:35357/v2.0
2013-06-18 09:43:01 + create_keystone_accounts
2013-06-18 09:43:01 ++ keystone tenant-create --name admin
2013-06-18 09:43:01 ++ grep ' id '
2013-06-18 09:43:01 ++ get_field 2
2013-06-18 09:43:01 ++ read data
2013-06-18 09:43:01 An unexpected error prevented the server from
fulfilling your request. (OperationalError) (1045, "Access denied for
user 'root'@'localhost' (using password: YES)") None None (HTTP 500)



That is your issue. It's mysql connectivity.

-Sean

--
Sean Dague
http://dague.net

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] The OpenStack Community Welcomes Developers in All Programming Languages

2013-06-12 Thread Sean Dague

On 06/12/2013 05:01 PM, Stefano Maffulli wrote:

On Wed 12 Jun 2013 01:19:50 PM PDT, John Wong wrote:

Is there any way we can punish these people in the future?


Let's be clear: we're *nowhere* near having to think about using such
measures.

I would like to focus the discussion on how we can help developers
discover and use the existing SDKs for OpenStack. At this stage I think
it is much more important to encourage conversations around consuming
OpenStack than to threat people contributing on IRC.

I agree with what Christopher/radix said before, that one step is to
have more volunteers that hang out on the IRC channels regularly and
are helpful.

Other ideas?


I'd just +1 on the "more volunteers" front. We could deputize some folks 
to make sure they pay attention to the channel and voice them in it. The 
reality is that with so many channels, #openstack tends to get forgotten 
by most of the -dev community, so having a concerted effort to have 
helpful people in there seems like the best approach.


    -Sean

--
Sean Dague
http://dague.net

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Getting DevStack networking working

2013-06-04 Thread Sean Dague
If you are running on recent Ubuntu it appears that you basically need 
MULTI_HOST=True set for guest routing to work. Otherwise vhost_net 
corrupts the checksums of the dhcp packets, and you're kind of done.


Been trying to narrow this down as far as possible the last couple of 
days (as I have some environments where this works, and some where it 
doesn't), but I still don't have a specific narrow work around for it 
other than turning on MULTI_HOST.


-Sean

On 06/04/2013 01:52 PM, Christopher Armstrong wrote:

Hi all!

I've been struggling the past couple of days on getting a working
DevStack so I can start contributing to the Heat project. The problems
I've been running into are largely networking related. I've tried a
number of combinations, like using latest git checkouts vs
stable/grizzly for the various projects, and nova-network vs quantum.

The two common behaviors I'm seeing are

1. if I use nova-network, I can't route to the guest VMs at all, with
private or public IP
2. if I use quantum, I can route into the guest VMs, and they can route
to each other, but I can't route out to the Internet (via tthe host
network).

#2 is really close to working, and I wonder if it's just the thing to be
expected. If that's the case, what should I be doing to get the VMs to
be able to route to the outside world? For what it's worth, I followed
the instructions at https://wiki.openstack.org/wiki/QuantumDevstack to
get quantum-on-devstack working. I'm using the single-node setup.

I hope to start contributing a lot to the OpenStack project soon!

--
IRC: radix
Christopher Armstrong
Rackspace



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp




--
Sean Dague
http://dague.net

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] New code name for networks

2013-05-13 Thread Sean Dague

On 05/13/2013 11:03 AM, Doug Hellmann wrote:



+1

Use a namespace package "openstack" then each project has a unique
package under that for their meaningfully named package (compute,
networking, etc.).


Sounds great and all, except when you try to do something like quantum, 
or even cinder, where you break out function into another project.


from openstack import network <- wait, is that what used to be nova 
network, or quantum, or some abstraction?


from openstack.compute import network as compute_network ?

Code names are actually incredibly useful in developing this stuff, 
because it lets us think about an implementation separate from a 
concept, and work with non native english speakers a lot easier where 
concept vs. implementation.


Honestly, there is already incredible confusion when you talk with 
people about "compute". Where you have to be really paying attention to 
nuance to figure out if people meant "Nova as a whole", "the Nova 
compute daemon", "A nova compute node, which might also have 
nova-network on it". The number of times we had to explain no-db-compute 
blueprint because of that speaks to the fact that generic names do not 
make anything easier, they generate more confusion quite often.


Code names are useful because it gives us a whole other namespace of 
words to work with to be very specific about what we mean, that can't be 
confused for the generic concepts of networking or computing. Yes, it's 
inside baseball, but when you are dealing with code as complicated as 
OpenStack, not having that inside baseball really slows things down.


Just look at the regular confusion new people have about the 2 uses of 
the term migrations in Nova, one for the database schema, and one for 
moving guests around. :)


-Sean

--
Sean Dague
http://dague.net

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [OpenStack Marketing] New code name for networks

2013-05-12 Thread Sean Dague

On 05/12/2013 09:14 AM, Mark McLoughlin wrote:

On Sat, 2013-05-11 at 19:50 +, Jason Smith wrote:

Hello,
I understand why we had to give up Quantum code name but rather than
just refer to it as networking let's come up with a new code name!


Yes, this was discussed at the summit:

   https://etherpad.openstack.org/ProjectsReNaming

The conclusion was that a number of choices for a new name would be put
forward and that Quantum's contributors would vote for one of those
choices.


Just as a consideration to the Nova project (and probably others), it 
would be *really* nice if the new name was also 7 characters.


There are currently 351 occurrences of the word quantum in the Nova code 
in current master, some of them will have pep8 implications after 
changing, if the length of the word changes.


This probably bites quantum internally as well, but it's worth making 
sure it's highlighted.


    -Sean

--
Sean Dague
http://dague.net

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] New code name for networks

2013-05-12 Thread Sean Dague
Quark is also the trademark of a computer software company - 
http://tess2.uspto.gov/bin/showfield?f=doc&state=4808:7k4xqd.2.44


This stuff is trickier than you might realize. Hence why car names are 
now all made-ity-up words.


-Sean

On 05/12/2013 12:21 PM, Jacob Godin wrote:

I like this, +1

On 2013-05-11 8:08 PM, "David Shrewsbury" mailto:shrewsbury.d...@gmail.com>> wrote:

quark

Keeps the sciency theme going, same first two letters, and
represents something
fundamental to other sciency stuff (much like networking is
fundamental to
OpenStack).

http://en.wikipedia.org/wiki/Quark

-Dave


On May 11, 2013, at 4:13 PM, Monty Taylor mailto:mord...@inaugust.com>> wrote:

 > Jeremy Stanly on IRC just suggested kumquat... but to that I respond:
 >
 > qumkuat
 >
 > Same benefits as qumutna - except it's more pronouncable.
 >
 > On 05/11/2013 04:07 PM, Monty Taylor wrote:
 >> I have been arguing for:
 >>
 >> mutnuaq
 >>
 >> Granted, it takes a minute to learn how to type, but it's just a
little
 >> snarky, and it takes up the exact same number of letter. However, it
 >> does screw with sorting. SO - what about:
 >>
 >> qumutna
 >>
 >> It's a little bit easier to wrap your head around, it's still
clearly an
 >> homage, and it should be super easy to bulk cut/replace.
 >>
 >> On 05/11/2013 03:58 PM, Davanum Srinivas wrote:
 >>> Lattice
 >>>
 >>> -- dims
 >>>
 >>> On Sat, May 11, 2013 at 3:52 PM, Mark Turner mailto:m...@amerine.net>> wrote:
 >>>> Tubes
 >>>>
 >>>> ;-)
 >>>>
 >>>>
 >>>> On Sat, May 11, 2013 at 12:51 PM, Jason Smith
mailto:jason.sm...@rackspace.com>>
 >>>> wrote:
 >>>>>
 >>>>> Hello,
 >>>>> I understand why we had to give up Quantum code name but
rather than just
 >>>>> refer to it as networking let's come up with a new code name!
 >>>>>
 >>>>> Thoughts?
 >>>>>
 >>>>> Thanks,
 >>>>> -js
 >>>>> ___
 >>>>> Mailing list: https://launchpad.net/~openstack
 >>>>> Post to : openstack@lists.launchpad.net
<mailto:openstack@lists.launchpad.net>
 >>>>> Unsubscribe : https://launchpad.net/~openstack
 >>>>> More help : https://help.launchpad.net/ListHelp
 >>>>
 >>>>
 >>>>
 >>>> ___
 >>>> Mailing list: https://launchpad.net/~openstack
 >>>> Post to : openstack@lists.launchpad.net
<mailto:openstack@lists.launchpad.net>
 >>>> Unsubscribe : https://launchpad.net/~openstack
 >>>> More help   : https://help.launchpad.net/ListHelp
 >>>>
 >>>
 >>>
 >>>
 >>
 >> ___
 >> Mailing list: https://launchpad.net/~openstack
 >> Post to : openstack@lists.launchpad.net
<mailto:openstack@lists.launchpad.net>
 >> Unsubscribe : https://launchpad.net/~openstack
 >> More help   : https://help.launchpad.net/ListHelp
 >>
 >
 > ___
 > Mailing list: https://launchpad.net/~openstack
 > Post to : openstack@lists.launchpad.net
<mailto:openstack@lists.launchpad.net>
 > Unsubscribe : https://launchpad.net/~openstack
 > More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
<mailto:openstack@lists.launchpad.net>
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp




--
Sean Dague
http://dague.net

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] grizzly's cinder-api is not starting : Couldn't find urlmap package

2013-04-12 Thread Sean Dague
Please post the contents of your /etc/cinder directory, I think this is 
related to an issue I saw during grenade testing.


-Sean

On 04/12/2013 08:29 AM, Mohammed Amine SAYA wrote:

Hi All,

I am trying to start cinder-api but it complains about urlmap module.
I checked in "/usr/lib/python2.7/dist-packages/paste/" directory and I
found
urlmap.py and urlmap.pyc.

Have you heard about this behavior in cinder-api? I didn't have it with
FOLSOM.

Here is the error I found in cinder-api.log :

2013-04-12 14:20:29 CRITICAL [cinder] No module named urlmap
Traceback (most recent call last):
   File "/usr/bin/cinder-api", line 48, in 
 server = service.WSGIService('osapi_volume')
   File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 520,
in __init__
 self.app = self.loader.load_app(name)
   File "/usr/lib/python2.7/dist-packages/cinder/wsgi.py", line 490, in
load_app
 return deploy.loadapp("config:%s" % self.config_path, name=name)
   File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py",
line 247, in loadapp
 return loadobj(APP, uri, name=name, **kw)
   File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py",
line 271, in loadobj
 global_conf=global_conf)
   File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py",
line 296, in loadcontext
 global_conf=global_conf)
   File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py",
line 320, in _loadconfig
 return loader.get_context(object_type, name, global_conf)
   File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py",
line 454, in get_context
 section)
   File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py",
line 476, in _context_from_use
 object_type, name=use, global_conf=global_conf)
   File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py",
line 406, in get_context
 global_conf=global_conf)
   File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py",
line 296, in loadcontext
 global_conf=global_conf)
   File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py",
line 337, in _loadfunc
 return loader.get_context(object_type, name, global_conf)
   File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py",
line 681, in get_context
 obj = lookup_object(self.spec)
   File "/usr/lib/python2.7/dist-packages/paste/deploy/util.py", line
69, in lookup_object
 module = __import__(parts)
ImportError: No module named urlmap


Thanks a lot for your help.
Best regards,
Amine.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp




--
Sean Dague
http://dague.net

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Gerrit Review + SSH

2013-04-05 Thread Sean Dague

On 04/04/2013 05:31 PM, Jeremy Stanley wrote:

On 2013-04-04 22:11:10 +0100 (+0100), Daniel P. Berrange wrote:
[...]

I don't know how hard it would be for OpenStack Infrastructure team
to officially make Gerrit available via port 443, in addition to the
normal SSH port.


We'd need to use different hostnames mapped to different IP
addresses since 443/tcp is already in use on review.openstack.org
for, well, HTTPS (the availability of fancy proxies which can
differentiate SSH from SSL/TLS notwithstanding--do those exist?).

The bigger question is whether it's worth the effort to maintain a
workaround like that... are there companies who want their employees
contributing to OpenStack development but won't grant those same
developers access to our code review system over the Internet? If
so, maybe some brave soul will take pity on them and set up a TCP
bounce proxy somewhere on port 443 to forward to port 29418 on our
Gerrit server for Git+SSH access on an alternate address and port. I
don't think that would need any sort of buy-off from our
Infrastructure Team (we can discuss if someone's actually interested
in setting it up), but probably wouldn't be "official" all the same.


I'm with Jeremy, I think putting work arrounds into infrastructure to 
deal with companies setting up their networks in completely 
uncollaborative ways, is just a slipperly slope to craziness.


It's probably worth documenting the services one needs to be able to 
effectively collaborate with the community on a wiki, to give folks 
something to take back to their IT depts and say "punch these holes, 
otherwise we can't do our jobs". A page on openstack.org about that 
would probably give them more leverage than figuring it out themselves.


-Sean

--
Sean Dague
http://dague.net

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Name it Hood!

2013-01-24 Thread Sean Dague

On 01/24/2013 02:50 PM, Monty Taylor wrote:

Hey all!

Here's my pitch for Hood:

a) It's the tallest mountain in Oregon, and honestly, it's a pretty
kick-ass mountain in general
b) Being in the pacific northwest, the mountain itself is quite
regularly in the clouds. That's gotta count for something.
c) It's actually a volcano.
d) Mount Hood is CLEARLY an Oregon thing. Havana is clearly a town in
Cuba. (We should have a design summit in cuba!!!)
e) Harbor is super-problematic because of the US/UK clash in spelling.
Half of us will spell it wrong no matter what.
f) Hood is only 4 letters. Think about that when you think about typing
hatfield a lot. Also, if we name it hatfield, we're going to have to
have the M summit somewhere that has a town called McCoy.


Yes, but I'm totally cool with that. +1 for Hatfield.

Just means that we have to go to Florida for the M summit - 
http://en.wikipedia.org/wiki/Fort_McCoy,_Florida



g) I'll buy you a beer at the summit if you vote for Hood.

Monty

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp




--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Reinstating Trey Morris for Nova Core

2013-01-23 Thread Sean Dague

+1

On 01/22/2013 07:25 PM, Vishvananda Ishaya wrote:

+1

We mentioned previously that we would fast-track former core members back in.
I gess we can wait a couple of days to see if anyone objects and then add him 
back.

Vish
On Jan 22, 2013, at 3:38 PM, Matt Dietz  wrote:


All,

I think Trey Morris has been doing really well on reviews again, so I'd
like to propose him to be reinstated for Nova core. Thoughts?

-Dietz



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp




--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] running devstack

2013-01-11 Thread Sean Dague

From the README.md

...
We also provide an environment file that you can use to interact with 
your cloud via CLI:


# source openrc file to load your environment with osapi and ec2 creds
. openrc
# list instances
nova list
...

-Sean

On 01/11/2013 08:34 AM, Snider, Tim wrote:

I had a long saga of installing devstack:
1. Started with Ubuntu Maverick. Can't install devstack on this 
release. fake out apt-sources fails due to dependencies.
2. do-release-upgade on command lines fails in numerous ways:
   overflow error -- apparently known problem. patch out 
offending lines retry.
   1000s of packages obtained. several hangs during update 
process. restarted several times.
run overnight. grub-update hangs punt.
3. Clean 12.10 install from cd. --- yipee works. ;)

Finally after that I installed devstack to start getting hands dirty with 
openstack:

4. devstack install script works (is it really necessary to 
install/upgrade unrelated packages e.g. vim??)
5. devstack.sh install script completes.

But then trying to run any of the exercises fails with " X11 connection rejected 
because of wrong authentication". What didn't I get setup correctly?
   root@84Server:~/devstack/exercises# ./client-env.sh
   *
   Begin DevStack Exercise: ./client-env.sh
   *
   Test Keystone
   X11 connection rejected because of wrong authentication.
   Unable to communicate with identity service: {"error": {"message": "Invalid user / password", 
"code": 401, "title": "Not Authorized"}}. (HTTP 401)

Suggestions  are appreciated.
Thanks,
Tim

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp




--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] new mailing list for bare-metal provisioning

2012-10-29 Thread Sean Dague

On 10/28/2012 08:19 PM, David Kang wrote:


  I agree that subject prefix is a way.
There are pros and cons of either approach.
However, when I asked a few of the people who showed interest in bare-metal 
discussion,
a new mailing list was preferred by them.
And we thought a separate mailing list makes people easier to participate and 
to manage the discussion.

  We can discuss this issue again among the people who signed up the new 
mailing list.


I think the more general concern is that part of the challenges of 
getting the current bare-metal approach in has been that it's largely 
been developed on the side. I think the path for successful evolution of 
bare-metal in nova is to stop thinking about it as a side effort, and 
more a part of normal nova development, as other changes in nova will 
have implications for bare-metal for sure.


So driving the discussion back onto the main list would be really 
beneficial.


-Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Tracking triage statistics

2012-10-26 Thread Sean Dague

On 10/25/2012 09:30 PM, Michael Still wrote:

On 10/26/2012 12:24 PM, Russell Bryant wrote:

On 10/25/2012 08:18 PM, Michael Still wrote:

I'd be interested in comments people might have. The code is at
http://bazaar.launchpad.net/~mikalstill/+junk/openstack-lp-scripts/view/head:/triage-stats.py


Awesome, thanks!

One thing I think we should do for these stats is filter out cases where
the reporter == triager.  Developers filing bugs and triaging them for
their own patches shouldn't be counted.


I thought about this... Surely any triage is better than none? If we
don't reward self triage, then someone else will still have to triage
the bug, right?

I'd be interested in other people's thoughts on this.


Very cool.




You should move this to github so I can send you a pull request.  :-)


Heh. The code is on LP mainly because that's where the existing
launchpadlib code for openstack resides. I can move it to github if
people feel strongly about it.


+1 for github please, just simpler in the current openstack culture.

-Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] Current IRC channel list ... a bit out of date

2012-10-23 Thread Sean Dague
As I was pointing some folks towards IRC I realized that our canonical 
IRC channel list for the project here - http://wiki.openstack.org/IRC, 
is somewhat out of date with the 14 channels that show up with a search 
on OpenStack on freenode - 
http://irc.netsplit.de/channels/?net=freenode&chat=openstack


Not all of those are official team channels, but most of them are. A 
gentle reminder to folks, if a sub team is going to spin up a channel, 
it would be really great to remember to add it to that list so that it's 
discoverable by new folks to the project. :)


Takes only a minute to update - http://wiki.openstack.org/IRC

-Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Rechecking changes if jenkins fails

2012-10-15 Thread Sean Dague

Add a comment that just says "recheck" in the review.

-Sean

On 10/15/2012 05:51 AM, Pitucha, Stanislaw Izaak wrote:

Hi all,
Is there some way to trigger another check when Jenkins fails because of
issues unrelated to the change?
For example last time I submitted https://review.openstack.org/14374 Jenkins
failed job gate-nova-docs, but that was because some package could not be
downloaded properly:

01:31:58 Downloading/unpacking httplib2 (from -r
/home/jenkins/workspace/gate-nova-docs/tools/pip-requires (line 20))
01:31:58   Hash of the package
http://pypi.openstack.org/httplib2/httplib2-0.7.6.tar.gz#md5=3f440fff00e0d2d
3c2971693de283cdf (from http://pypi.openstack.org/httplib2/) () doesn't match the expected hash
3f440fff00e0d2d3c2971693de283cdf!
01:31:58 Bad md5 hash for package
http://pypi.openstack.org/httplib2/httplib2-0.7.6.tar.gz#md5=3f440fff00e0d2d
3c2971693de283cdf (from http://pypi.openstack.org/httplib2/)

It seems I could not do anything about it apart from waiting for another
change to be merged and triggering "Rebase change". It's the second time I
run into an issue like that.

I know some projects workaround it by running checks again after a specific
comment containing one word (recheck, rekick, ...). Is there any similar
system working / planned for Openstack?

Regards,
Stanisław Pitucha
Cloud Services
Hewlett Packard




___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp




--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] TC Candidacy

2012-09-19 Thread Sean Dague

Hey all,

I'd like throw my hat in the ring for a seat on the TC as well.

Like Matt, I think the current crew of TC candidates are excellent, and 
I as a community member would be happy with any of them, but more 
choices are always good (unless they are bubble gum jelly beans, then 
all bets are off).


-- OpenStack Credentials --

I've been involved with OpenStack since March of this year, when IBM 
first got engaged in the community. I've been primarily working on nova 
and testing code during that time. My reviews and contributions are here 
https://review.openstack.org/#/q/reviewer:sdague%2540linux.vnet.ibm.com+OR+owner:sdague%2540linux.vnet.ibm.com,n,z 



-- Non-OpenStack Credentials --

I've been a part of IBM's Linux Technology Center since it's creation in 
2001, and contributed to numerous OpenSource projects over that time. 
Some relevant highlights have been the SystemImager tool for cluster 
installation and the OSCAR HPC toolkit. Contributions to the management 
and testing stacking in Xen. A mostly complete and varied history can be 
found at ohloh - https://www.ohloh.net/accounts/sdague.


I'm also a personal contributor to many other smaller open source 
projects in varied languages - some of which is captured in github - 
https://github.com/sdague.


From a leadership perspective I'm the president and founder of our 
local area Linux and Open Source Users Group (created almost 10 years 
ago) - http://mhvlug.org.


-- Reasons for Interest in TC --

My particular areas of focus over the next year are upgrade and testing. 
I want very much for OpenStack to be able to not only upgrade cleanly, 
but also be able to rolling upgrade in the field from one release to the 
next. This is going to be a long road ahead to get there and I think 
having a voice with this particular lens on the TC would help get us 
there as a community.


Thank you for your consideration,

-Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Openstack summit hotel

2012-09-11 Thread Sean Dague
Looks like that block is sold out as well now, even though the first 
page says bookable through Sept 24th. Any idea if more are going to 
happen, or if everyone's on their own at this point?


-Sean

On 09/04/2012 02:19 PM, Mark T. Voelker wrote:

Trey,

FYI, there's another room block at the Embassy Suites.

http://embassysuites.hilton.com/en/es/groups/personalized/S/SANDNES-OPE-20121014/index.jhtml?WT.mc_id=POG

At Your Service,

Mark T. Voelker

On 09/04/2012 01:47 PM, Trey Morris wrote:

The Manchester Grand Hyatt is booked out for the conference. I made the
mistake of waiting to book hotel until I was surely going. Does anyone
have any reservations they can't keep?


thanks,
-tr3buchet


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp



This body part will be downloaded on demand.



--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [openstack-dev] Nova PTL candidacy

2012-09-06 Thread Sean Dague

Looks like thanks to bcwaldon you wont have to wait much.

Also, +1 for Vish.

On 09/06/2012 02:12 PM, Duncan McGreggor wrote:

/me waits for http://vishfacts.com/ ...

On Thu, Sep 6, 2012 at 10:54 AM, Matt Joyce  wrote:

Vish doesn't sleep.  He waits.


On Thu, Sep 6, 2012 at 10:32 AM, Blake Yeager 
wrote:


He also lives vicariously through himself.


On Thu, Sep 6, 2012 at 10:12 AM, Ravi Jagannathan 
wrote:


+1 . Plus he has a cool name.


On Thu, Sep 6, 2012 at 10:58 AM, Sam Su  wrote:


+1


On Wed, Sep 5, 2012 at 12:19 AM, Michael Still
 wrote:


On 09/05/2012 06:03 AM, Matt Joyce wrote:

Vish is also a pretty cool guy and doesn't afraid of anything.


Vish does a great job -- many hours a day of code review and mentoring,
puts up with criticism much more calmly than I think many would, and is
a pleasure to work with.

Mikal


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp




___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp




___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp




___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp




___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp




--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Making the RPC backend a required configuration parameter

2012-08-08 Thread Sean Dague

On 08/08/2012 05:00 PM, Andrew Clay Shafer wrote:

On Wed, Aug 8, 2012 at 4:35 PM, Eric Windisch mailto:e...@cloudscaling.com>> wrote:

I believe that the RPC backend should no longer have any default

Historically, it seems that the Kombu driver is default only because
it existed before all others and before there was an abstraction.
With multiple implementations now available, it may be time for a
change.

Why?
* A default skews the attitudes and subsequent architectures toward
a specific implementation


* A default skews the practical testing scenarios, ensuring maturity
of one driver over others.
* The kombu driver does not work "out of the box", so it is no more
reasonable as a default than impl_fake.
* The RPC code is now in openstack-common, so addressing this later
will only create additional technical debt.

My proposal is that for Folsom, we introduce a "future_required"
flag on the configuration option, "rpc_backend". This will trigger a
WARNING message if the rpc_backend configuration value is not set.
  In Grizzly, we would make the rpc_backend variable mandatory in
the configuration.


Regardless of the actual default in openstack-common, the devstack 
default is going to skew all of this as well (if not more so), and 
devstack does need a default. Much like db backend.


I would also assume that Packagers are going to need to set a default in 
their packages.


While I have no objection to this change, I'm not sure it accomplishes 
the goal if it just means the default is set elsewhere, and 90% of 
people are all running with the same implementation anyway.


-Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] A series of blog posts on OpenStack networking details

2012-08-03 Thread Sean Dague

On 08/03/2012 02:50 PM, Eugene Kirpichov wrote:

Hello community,

I'd like to advertise that me and my colleague Piotr Siwczak at
Mirantis have started a series of blog posts explaining the gory
details of OpenStack networking.

http://www.mirantis.com/tag/networking/

So far we have two posts:

http://www.mirantis.com/blog/openstack-networking-flatmanager-and-flatdhcpmanager/
- basics of FlatManager and FlatDHCPManager in multi-host mode.

http://www.mirantis.com/blog/openstack-networking-single-host-flatdhcpmanager/
- extremely detailed account of FlatDHCPManager in single-host mode;
down to a walkthrough of L2 packet flow for several scenarios. I wrote
this post in revenge for my own struggles when I dreamt "if only
someone had described in extreme detail how it is *supposed* to work"
but was not able to find anything like that :)

A few more will appear soon: two posts from Piotr on VLANManager,
eventually also analyzing the packet flow.

(me and Peter have slightly different styles: he prefers to cover
details across several posts while I prefer to write a huge post with
all the details at once)

Comments and especially corrections are extremely welcome! [and, well,
shares too :) ]


Look like a great thing to add to the OpenStack Planet if you are 
interested.


    -Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Default reply to behavior for mailing list

2012-07-31 Thread Sean Dague

On 07/31/2012 02:13 PM, Jay Pipes wrote:


As a counter-point, I'd prefer to keep the list as it is and *not* munge
Reply-To.

Since Chip summarizes it better than I can, I'll link to his article
'"Reply-To" Munging Considered Harmful':

http://www.unicom.com/pw/reply-to-harmful.html


++

Just use a decent email program (i.e. not Outlook) that gives you things
like Reply to List.


"Just switch email programs" isn't really an option for many, especially 
if they are working with their corporate email infrastructure.


My personal experience is Reply-to munging is important to keep the 
communication on list. Defaults matter. And changing 1 header on 1 
server, instead of making 3816 people (the active count on openstack 
group in launchpad) change their email client, seems the polite thing to 
do. :)


That being said, I realize we're entering mostly a land of religion 
here. So I'll just end with a "long live emacs!" to try to get us to the 
godwin rule as fast as possible.


-Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [nova] Proposal to add Yun Mao to nova-core

2012-07-19 Thread Sean Dague

On 07/18/2012 07:10 PM, Vishvananda Ishaya wrote:

Hello Everyone!

Yun has been putting a lot of effort into cleaning up our state management, and 
has been contributing a lot to reviews[1]. I think he would make a great 
addition to nova-core.


[1] https://review.openstack.org/#/dashboard/1711


+1

-Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Solaris Fails as Guest VM

2012-07-18 Thread Sean Dague
My experience is that solaris is incredibly fickle on kvm. I think one 
of the issues had to do with the boot screen and how it uses graphics 
and framebuffer.


-Sean

On 07/17/2012 08:55 PM, Narayan Desai wrote:

I suspect that you need the right solaris (more likely illumos) bits
to get guest side support for virtio. We tried a while ago and the
default openindiana at the time didn't work.
  -nld

On Tue, Jul 17, 2012 at 7:43 PM, Joshua  wrote:

I have tried with both KVM and qemu. Solaris starts to boot and hits
grub then cycles boot. Anyone experienced this?

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp




--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] debugging a db migration script

2012-07-17 Thread Sean Dague

Is it up for review somewhere so we can debug by inspection?

-Sean

On 07/16/2012 11:59 PM, Jim Fehlig wrote:

I'm working on a patch that adds a column to the compute_nodes table in
the nova db, but it seems my db migration script fails when calling 'db
sync' in stack.sh.  I tried running the command manually, same failure:

stack@virt71:~> /opt/stack/nova/bin/nova-manage --debug -v db
sync2012-07-16 21:42:52 DEBUG nova.utils [-] backend  from (pid=19230)
__get_backend /opt/stack/nova/nova/utils.py:484
/usr/lib64/python2.6/site-packages/sqlalchemy/pool.py:681:
SADeprecationWarning: The 'listeners' argument to Pool (and
create_engine()) is deprecated.  Use event.listen().
   Pool.__init__(self, creator, **kw)
/usr/lib64/python2.6/site-packages/sqlalchemy/pool.py:159:
SADeprecationWarning: Pool.add_listener is deprecated.  Use event.listen()
   self.add_listener(l)
Command failed, please check log for more info

I can't find anything useful in any log (/var/log/*, /opt/stack/log/*).
I ran the above in strace and saw my migration script was opened and
then shortly thereafter writing of "Command failed, please check log for
more info" and exit(1) :).

The patch also adds a 'hypervisor_type' column to the instances table,
and that migration script succeeds!

Any hints for debugging a db migration script?

Thanks,
Jim


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp




--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [nova] [cinder] Nova-volume vs. Cinder in Folsom

2012-07-16 Thread Sean Dague

On 07/12/2012 05:40 PM, Vishvananda Ishaya wrote:

Excellent points. Let me make the following proposal:

1) Leave the code in nova-volume for now.
2) Document and test a clear migration path to cinder.
3) Take the working example upgrade to the operators list and ask them for 
opinions.
4) Decide based on their feedback whether it is acceptable to cut the 
nova-volume code out for folsom.


+1

-Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [nova] [cinder] Nova-volume vs. Cinder in Folsom

2012-07-11 Thread Sean Dague
Before we completely pile on option 1, can we get devstack changed to 
run this way? I think the amount of pain / ease that transition is for 
users and the OpenStack CI team will greatly inform this decision, and 
give us some good data points on how tough this is for people to convert.


-Sean

On 07/11/2012 12:22 PM, Mike Perez wrote:

+1 for option 1

--
Mike Perez
DreamHost.com

On Wednesday, July 11, 2012 at 8:26 AM, Vishvananda Ishaya wrote:


Option 1 -- Remove Nova Volume
==



This body part will be downloaded on demand.




--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] best practices for merging common into specific projects

2012-07-05 Thread Sean Dague

On 07/03/2012 02:00 PM, Dan Prince wrote:

I don't see this as much different as any other patches to nova (or
whatever project is using common).  It should be a proper patch
series.
  If the person pulling it in doesn't understand the merge well enough
  to
produce the patch series with proper commit messages, then they are
the
wrong person to be doing the merge in the first place.


I went on a bit of a rant about this on IRC yesterday. While I agree a patch 
series is appropriate for many new features and bug fixes I don't think it 
should be required for keeping openstack-common in sync. Especially since we 
don't merge tests from openstack-common which would help verify that the person 
doing the merges doesn't mess up the order of the patchset. If we were to 
include the tests from openstack-common in each project this could change my 
mind.

If someone wants to split openstack-common changes into patchsets that might be 
Okay in small numbers. If you are merging say 5-10 changes from openstack 
common into all the various openstack projects that could translate into a 
rather large number of reviews (25+) for things that have been already reviewed 
once.  For me using patchsets to keep openstack-common in sync just causes 
thrashing of Jenkins, SmokeStack, etc. for things that have already been gated. 
Seems like an awful waste of review/CI time. In my opinion patchsets are the 
way to go with getting things into openstack-common... but not when syncing to 
projects.

Hopefully this situation is short lived however and we start using a proper 
library sooner rather than later.


+1. I think the bisectability isn't a clear win without the unit tests 
coming over, and the rest is just more reviews (which are already in 
scarce supply), and more jenkins time which just slows down everything else.


-Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Jenkins vs SmokeStack tests & Gerrit merge blockers

2012-06-28 Thread Sean Dague

On 06/28/2012 11:13 AM, Monty Taylor wrote:

> Fundamentally though - we're at a point of trying to have our cake and

eat it too. Either we want comprehensive testing of all of the unit
tests, or we want to be careful about not making the test environment to
hard for a developer to exactly mimic.

I'm obviously on the side of having us have gating tests that some devs
might not be able to do on their laptops - such as  running the libvirt
tests properly. We're working on cloud software - worst case scenario if
there's an intractable problem, as dev can always spin up an ubuntu
image somewhere.


I'm definitely in agreement here. Something as fundamental as libvirt is 
to openstack really needs to see testing in the main test environment.


I really feel like this is almost the same conversation we had on IRC on 
Friday about the mysql requirement. There is a deeper level of testing 
that we can do if we start requiring more of the base OS in the jenkins 
environment. That takes us out of a place where all the unit tests that 
run in jenkins can be run in any python environment, but that's ok. More 
validation here on the gate is worth that.


Maybe we need something that's different than the current @skip 
semantics, some @skip_unless_gate (or similar), because a big piece of 
this is also confusion about what the gate is actually testing, and what 
it's skipping.


As the person involved in the patch that slipped through, I was as 
surprised as anyone else that it landed, but had an issue in a real 
libvirt test case.


-Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Openstack special sauce checking tool??

2012-06-28 Thread Sean Dague

On 06/28/2012 01:15 PM, Joshua Harlow wrote:

Hi all,

I remember hearing once that someone had a openstack import/hacking
style checking tool.

I was wondering if such a thing existed to verify same the openstack way
of doing imports and other special checks to match the openstack style.

I know a lot of us run pep8/pylint, but those don’t catch these special
sauce checks

And it would be nice to not have to manually check these (visually...)
but be able to run a tool that would just do the basic checks.

Does such a thing exist??


In nova, ./run_tests.sh -p does it. It's run by default if you do 
./run_tests.sh with no additional args at the end of the unit tests.


-Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [OpenStack][Nova] Possible inconsistency between devstack/nova execution and test_virt_drivers.py

2012-06-25 Thread Sean Dague

On 06/25/2012 10:41 AM, Leander Bessa Beernaert wrote:

Hello,

I'm working on the diagnostics method for libvirt. I've
successfully managed to test it while running it manually and with
devstack. However, the test case in test_virt_drivers.py fails since it
supplies a different data type to the method.

Could it be possible that there's a certain mismatch between the two or
that this particular method accepts multiple sorts of data-types?


Can you be more specific with the issue? I've been in that code 
recently, so I might be able to help sort this out.


    -Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] The right way to deprecate things in nova?

2012-06-13 Thread Sean Dague

On 06/13/2012 01:35 PM, Mark Washenberger wrote:


So up until this point OpenStack has been a pretty much a rip and
replace model. You want to go from Diablo to Essex, shut everything
down, upgrade, bring back up. When I went to change this parameter
originally, the review comments included just ripping out the old
function, and not deprecating it.

But I think we are moving into a phase where real OpenStack deployments
are going to have N and N+1 release componets talking to each other. so
it's probably worth getting in the habbit of having a standard way to
deprecate out over a release. LOG.warning messages scattered about,
which may or may not be consistent, that someone might or might not
remember to remove later, with or without their associated function,
seems kind of error prone.



Logging sounds like a great way to communicate to deployers and operators,
but really doesn't seem the best way to communicate with developers. So
my question is, are we using this mechanism to deprecate things the deployers
can control? Or is it things that developers need to deal with? If its the
latter (which it seems), I'd prefer that we just use our various developer
coordinating communication channels, such as the team meetings, IRC, mailing
list, etc.


So for deprecating some piece of Operator facing interface, I agree we 
can do that without anything as heavy as a decorators. So how about this 
instead, have a user_deprecate(msg="") function.


It's a wrapper on the LOG function, with some standard formatting that 
makes sure all the user deprecated features have an easy grepable 
pattern in the log. Also add the fatal functionality, so that people can 
sniff test their system before upgrading to N+1 that they aren't using 
deprecated configs.


It wouldn't be a decorator, just a function that can be placed inside code.

-Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] The right way to deprecate things in nova?

2012-06-13 Thread Sean Dague

On 06/12/2012 05:53 PM, Dan Prince wrote:


Here's my current suggested path forward, which I'd like comments on:
   * keep the existing nova.utils deprecation functions (don't remove
   them)


My take is why keep a 200-300 line set of functions and tests (a small 
framework) to log messages about code we want to get rid of? As of today we 
aren't even making use of it and I'm not convinced peppering more decorators 
all over the place is the best idea. I suppose I have a slight preference for 
simply logging things here.


So up until this point OpenStack has been a pretty much a rip and 
replace model. You want to go from Diablo to Essex, shut everything 
down, upgrade, bring back up. When I went to change this parameter 
originally, the review comments included just ripping out the old 
function, and not deprecating it.


But I think we are moving into a phase where real OpenStack deployments 
are going to have N and N+1 release componets talking to each other. so 
it's probably worth getting in the habbit of having a standard way to 
deprecate out over a release. LOG.warning messages scattered about, 
which may or may not be consistent, that someone might or might not 
remember to remove later, with or without their associated function, 
seems kind of error prone.



   * add the fatal config option, and associated unit tests to make
   sure
it works correctly. This would be helpful for people to ensure they
weren't depending on deprecated functions towards the end of a
release.


I'm not apposed to this but it seems like grepping log files is also a fine 
tool. Presumably this would be off by default.


Grepping for log files assumes the deprecation messages all look the 
same. It also is a much more active process that people need to think 
about having to do, not something that can be just flagged in jenkins.


-Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] The right way to deprecate things in nova?

2012-06-12 Thread Sean Dague
I'm in the process of deprecating the old way that we do virt drivers so 
that it's fully dynamic - 
https://blueprints.launchpad.net/nova/+spec/virt-driver-cleanup


The way the code current exists in master is that a LOG.error is emitted 
when the deprecated method is hit. I set it to error level to make sure 
it got noticed, as it will require a user configuration change 
post-Folsom when the old option is removed. This seems very ad-hoc.


Yesterday I had a conversation with markmc on IRC about this, and he 
suggested an approach where we have a config option that makes 
deprecation fatal, which could be forced on to ensure cleanliness. This 
could be done either as a decorator or as a regular function.


It also turns out there already are some deprecation functions, which 
dprince pointed out to me today on IRC, because he was in process of 
removing them from nova because they weren't used - 
https://review.openstack.org/#/c/8412/.


Here's my current suggested path forward, which I'd like comments on:
 * keep the existing nova.utils deprecation functions (don't remove them)
 * add the fatal config option, and associated unit tests to make sure 
it works correctly. This would be helpful for people to ensure they 
weren't depending on deprecated functions towards the end of a release.
 * possibly move them to nova.common as they might make for good 
openstack-common material down the road

 * use this instead of the direct LOG.error in get_connection

This would have the side effect of making the message warning level, 
instead of error level, which I think is fine at this point.


-Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] the right way to deprecate a config option?

2012-06-01 Thread Sean Dague
I'm reworking the virt driver loading so that it's using importutils 
(and thus looking more like the other driver interfaces), which means 
eventually connection_type parameter in nova.conf goes away, and 
computer_driver is used directly (the support is already there, but it's 
not currently used by default).


Is there a standard mechanism for deprecating a conf option like this? 
Right now I'm just triggering a LOG.error() with a deprecation message, 
but it seems like there should be something more standard for that, 
especially as we get into a deprecate in N, remove in N+1. I'm assuming 
this is a deprecation in Folsom (where the docs are changed to the new 
method), then remove the backwards compat in G.


    -Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] RFC - dynamically loading virt drivers

2012-05-30 Thread Sean Dague

On 05/18/2012 02:14 PM, Jay Pipes wrote:

On 05/17/2012 06:38 PM, Vishvananda Ishaya wrote:

On May 17, 2012, at 1:52 PM, Sean Dague wrote:



So I guess this would be my strategy:

a) remove get_connection from the drivers (and just have it construct
the 'connection' class directly)
b) modify the global get_connection to construct the drivers for
backwards compatibilty
c) modify the documentation to suggest changing drivers by specifying
the full path to the driver instead of connection_type
d) rename the connection classes to something reasonable representing
drivers (libvirt.driver:LibvirtDriver() vs
libvirt.connection.LibvirtConnection)
e) bonus points if it could be done with a short path for ease of use
(compute_driver=libvirt.LibvirtDriver vs
compute_driver=nova.virt.libvirt.driver.LibvirtDriver)


A review which covers a, b, and part of d is up at - 
https://review.openstack.org/#/c/7930/. I'd like to get comments on the 
work so far, and handle c, the rest of d, and e in a second round once a 
version of this lands.


Reviews appreciated.

    -Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] RFC - dynamically loading virt drivers

2012-05-21 Thread Sean Dague

On 05/17/2012 06:38 PM, Vishvananda Ishaya wrote:

The main issue with changing this is breaking existing installs.

So I guess this would be my strategy:

a) remove get_connection from the drivers (and just have it construct the 
'connection' class directly)


I'm starting down this path, but a lot of the drivers do substantial 
amount of FLAG handling in get_connection before they create the driver 
objects.


Do you want to move that FLAGS handling into the object constructor, or 
keep having some factory layer to manipulate the flags before we 
construct the driver?


    -Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] RFC - dynamically loading virt drivers

2012-05-18 Thread Sean Dague

On 05/17/2012 06:38 PM, Vishvananda Ishaya wrote:
>

So we already have plugabillity by just specifying a different compute_driver 
config option.  I don't like that we defer another level in compute and call 
get_connection.  IMO the best cleanup would be to remove the get_connection 
altogether and just construct the driver directly based on compute_driver.

The main issue with changing this is breaking existing installs.

So I guess this would be my strategy:

a) remove get_connection from the drivers (and just have it construct the 
'connection' class directly)
b) modify the global get_connection to construct the drivers for backwards 
compatibilty
c) modify the documentation to suggest changing drivers by specifying the full 
path to the driver instead of connection_type
d) rename the connection classes to something reasonable representing drivers 
(libvirt.driver:LibvirtDriver() vs libvirt.connection.LibvirtConnection)
e) bonus points if it could be done with a short path for ease of use 
(compute_driver=libvirt.LibvirtDriver vs 
compute_driver=nova.virt.libvirt.driver.LibvirtDriver)


On point c), is the long term view that .conf options are going to 
specify full class names? It seems like this actually gets kind of 
confusing to admins.



What are your thoughts on the following approach, which is related, but 
a little different?


a) have compute_driver take a module name in nova.virt. which is loaded 
with some standard construction method that all drivers would implement 
in their __init__.py. Match all existing module names to connection_type 
names current in use. Basically just jump to e, but also make all 
drivers conform some factory interface so "libvirt" is actually enough 
to get you nova.virt.libvirt.connect()


b) if compute_driver is not specified, use connection_type, but spit out 
a deprecation warning that the option is going away. (Removed fully in 
G). Because compute_drivers map to existing connection_types this just 
works with only a little refactoring in the drivers.


c) remove nova/virt/connection.py

The end result is that every driver is a self contained subdir in 
nova/virt/DRIVERNAME/.



* one test fails for Fake in test_virt_drivers, but only when it's run as the 
full unit test, not when run on it's own. It looks like it has to do with 
FakeConnection.instance() caching, which actually confuses me a bit, as I would 
have assumed one unit test file couldn't affect another (i.e. they started a 
clean env each time).


Generally breakage like this is due to some global state that is not cleaned 
up, so if FakeConnection is caching globally, then this could happen.


It is keeping global state, I'll look at fixing that independently.

-Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] RFC - dynamically loading virt drivers

2012-05-17 Thread Sean Dague
Rationale: nova loads drivers for various subsystems in very different 
ways, and not all of them are truly dynamic (i.e. a new driver could be 
fully self contained and not have to change a core source code file or 
monkey patch to load itself). I'd like to get all the driver plug points 
fully dynamic, and eventually a common pattern and loading mechanism is 
used on all of them, from a consistency perspective.


I've got a first attempt at refactoring the nova.virt.connection module 
to stop having a pre-defined list of virt drivers as strings, and 
instead load modules dynamically using the the 
nova.openstack.commons.importutils. This is in a branch on github - 
https://github.com/sdague/nova


The changed files are:
git diff master | diffstat -w 20
 tests/test_virt_driver_loader.py |   54 ++
 utils.py |3
 virt/connection.py   |   51 ++---
 virt/fake.py |4
 virt/libvirt/__init__.py |1
 5 files changed, 75 insertions(+), 38 deletions(-)

This passes the unit test battery on fake and libvirt drivers (* on 
fake) Because the strings passed to connection_type do map to module 
names, this shouldn't cause any issues with existing configurations.


The xenapi and vmwareapi modules probably just need a similar 
__init__.py addition to make get_connection available on module load. 
Baremetal will be slightly more work (but not much) because it had some 
additional setup in nova/virt/connection.py.


What I'm mostly looking for is comments on approach. Is importutils the 
prefered way to go about this (which is the nova.volume approach) now, 
or should this be using utils.LazyPluggable as is in nova.db.api, or 
some other approach entirely? Comments, redirections, appreciated.


* one test fails for Fake in test_virt_drivers, but only when it's run 
as the full unit test, not when run on it's own. It looks like it has to 
do with FakeConnection.instance() caching, which actually confuses me a 
bit, as I would have assumed one unit test file couldn't affect another 
(i.e. they started a clean env each time).


-Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] questions on the dynamic loading of virt drivers in nova

2012-05-09 Thread Sean Dague
I'm familiarizing myself with the nova code and trying to reconcile that 
while there is dynamic class based loading in ComputeManager using 
import_utils in __init__() there is also a defaulting to the 
nova.virt.connection.get_connection function.


That's actually got a big if / else statement of string literals of 
known virt drivers, and then loads specific virt drivers from there.


Is there a reason for both approaches? Can we refactor to a point where 
we don't need need of a common file with driver specific imports and 
string literals? Is there a reason not to?


Thanks,

    -Sean

--
Sean Dague
IBM Linux Technology Center
email: sda...@linux.vnet.ibm.com
alt-email: slda...@us.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] Checking InnoDB table creation as part of unit tests for nova

2012-04-27 Thread Sean Dague
I've got a patch out for review that opportunistically tests the nova 
migrations on mysql, if a very specific mysql database name/password/db 
exists on the test system (fails gracefully if it doesn't). It then also 
checks to make sure that there are no "non-InnoDB" tables in the 
resultant migration (also failing gracefully if there isn't). We did a 
couple of draft reviews with the CI team to make sure this was something 
they could support as a gating test. This would prevent new migrations 
that don't specify engine from sneaking in.


I'd like feedback from Nova devs on the approach - 
https://review.openstack.org/#/c/6805/

comments, flames, etc are welcome.

Thanks,

-Sean

--
Sean Dague
IBM Linux Technology Center
email: slda...@us.ibm.com
alt-email: sda...@linux.vnet.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] database migration cleanup

2012-04-27 Thread Sean Dague

On 04/26/2012 03:24 PM, Dan Prince wrote:


I think this scheme would support users who follow stable releases as well as 
users who follow trunk very closely.

We talked about this at the conference but I thought this issue might be near 
and dear to some of our end users so it was worth discussing on the list.

What are general thoughts on this approach?


Is there any support in sqlalchemy, or related tools, to handle 
migrations the way rails does, where a schema file is created at the end 
of every migration? It would be ideal if we both had a full migration 
history, as well as a short cut at any snap shot to get to the end.


-Sean

--
Sean Dague
IBM Linux Technology Center
email: slda...@us.ibm.com
alt-email: sda...@linux.vnet.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [OpenStack][Nova] Minimum required code coverage per file

2012-04-27 Thread Sean Dague

On 04/27/2012 04:12 AM, Thierry Carrez wrote:

Joe Gordon wrote:

It would nice to initially see the code coverage delta per merge
proposal as a comment in gerrit (similar to SmokeStack), and not as a
gating factor.


+1

Sounds like a good way to evaluate how blocking it should be, and use it
to make more informed decisions on the quality of a patch.


+1

I think any new proposed commit gate should be run in an informative 
only state for a while first just to get a feel of how it would change 
the development process, and if the information is useful on a per 
commit basis. SmokeStack has been a great prover of that approach so far.


-Sean

--
Sean Dague
IBM Linux Technology Center
email: slda...@us.ibm.com
alt-email: sda...@linux.vnet.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Encrypted virtual machines

2012-04-26 Thread Sean Dague

On 04/26/2012 12:11 PM, Michael Grosser wrote:

Data left on broken disks would be unreadable. --> You don't have to
worry about data destruction before selling/throwing out your disks.
   (That could be realized via encrypting the whole compute-node disk,
but that's not quite what I want.)
Another benefit would be, that you as a cloud user wouldn't have to
worry about the provider accessing your data. (Encrypting every vms disk
for additional security.)

Or am I seeing this too worry-some?


No, I think that's the right level of worry-some - 
http://www.contextis.co.uk/research/blog/dirtydisks/


    -Sean

--
Sean Dague
IBM Linux Technology Center
email: slda...@us.ibm.com
alt-email: sda...@linux.vnet.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Using Foreign Keys

2012-04-26 Thread Sean Dague

On 04/25/2012 05:17 PM, Vishvananda Ishaya wrote:

The main issue is when the relevant tables are moved into a separate
service a la quantum or cinder. We can't keep referential integrity
across multiple databases, so the foreign keys in this case need to be
removed. It leads to an odd situation when there is still an internal
implementation in addition to the external implementation because the
internal implementation no longer has foreign keys.

As an example, we used to have foreign key relationships between
instances and networks. We can no longer have these because we support
networks declared externally. The internal network management now has no
referential integrity, but this is the price we pay for separation of
concerns. We are going through a similar set of relationship-breaking
with the volume code.


There are definitely the practical aspects of where this "can't" be done 
because the services have split out, and I think that's fine.


But enforcing the ref constraints where possible just provides another 
level of safety in the data. A policy where we break FK relationships if 
the preferred core model is 2 services (i.e. Nova / Quantum), but we add 
FK constraints within a service might be a good idea.


    -Sean

--
Sean Dague
IBM Linux Technology Center
email: slda...@us.ibm.com
alt-email: sda...@linux.vnet.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Coverage report generation crashes with UnicodeDecodeError (#987077)

2012-04-23 Thread Sean Dague

On 04/23/2012 10:01 AM, Renier Morales wrote:

Hello,

Anyone experiencing a UnicodeDecodeError crash when running the nova tests with 
coverage reporting turned on (i.e. run_tests.sh -c)? Might this be a common 
environment misconfiguration problem on my side?
More details at: https://bugs.lanchpad.net/nova/+bug/987077


Any chance you've just got a stale .venv in the repo? Coverage seems to 
be running fine for me this morning.


-Sean

--
Sean Dague
IBM Linux Technology Center
email: slda...@us.ibm.com
alt-email: sda...@linux.vnet.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] nova.conf query

2012-04-17 Thread Sean Dague

On 04/16/2012 06:17 AM, Salman Malik wrote:

Hi All,

A quick question regarding nova.conf: How can I modify nova.conf and get
it to work with devstack. The problem that I am facing is after
modifying nova.conf, I have to reboot so as to restart services. But
when I reboot, devstack needs to be reinstalled all over again using
stack.sh and in the process it rewrites /etc/nova/nova.conf.


devstack supports an EXTRA_OPTS flag in it's localrc that will let you 
set additional values in there. You can set more than one flag by using 
bash arrays, i.e.:


EXTRA_OPTS=(logdir=/var/log/nova auto_assign_floating_ip=True)

If you are using devstack, it's quite worth reading through the script 
as well. It's pretty well documented, and there are lots of gems in 
there like that.


    -Sean

--
Sean Dague
IBM Linux Technology Center
email: slda...@us.ibm.com
alt-email: sda...@linux.vnet.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] profiling nova-api

2012-04-12 Thread Sean Dague

On 04/11/2012 04:48 PM, Yun Mao wrote:


* Database access: Each "nova list" API call will issue 4 db APIs: 3
instance_get_all_by_filters(), 1
instance_fault_get_by_instance_uuids(), so 1200 db API calls total
(note: not necessarily 1200 SQL statements, could be more). The 900
instance_get_all_by_filters() calls took 30.2 seconds (i.e. 0.03s
each)! The 300 instance_fault_get_by_instance_uuids() calls only took
1.129 seconds (0.004 each).

>

You might think: MySQL sucks. Not so fast. Remember this is a tiny
database with only 10 VMs. Profile also shows that the actual
_mysql.connection.query() method only took 1.883 seconds in total. So,
we pretty much spend 29 seconds out of 60 seconds doing either
sqlalchemy stuff or our own wrapper. You can also see from the sheer
volume of sqlalchemy library calls involved.


Reducing the sqlalchemy 50% is probably going to involve being smarter 
about assembling the queries in a more complex way, that prevents us 
from going to the db quite so often.


On the MySQL query 50% of the time, it would be good if you can figure 
out if we are table scanning in the instance_get_all_by_filters. My 
inspection so far definitely shows a lot of things we do WHERE clauses 
on that don't have indexes, which is generally bad form.


This bug also has a previous, slightly different, look at digging out 
some of these issues - https://bugs.launchpad.net/nova/+bug/964824.


    -Sean

--
Sean Dague
IBM Linux Technology Center
email: slda...@us.ibm.com
alt-email: sda...@linux.vnet.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] I18n issue for OpenStack

2012-04-12 Thread Sean Dague

On 04/12/2012 04:52 AM, Thierry Carrez wrote:

Joshua Harlow wrote:

So there is a question here that I don’t understand.

There a different levels of I18N, for say user facing error messages, or
for other things I consider UI (horizon).

Those need to be I18N and all that. I think the larger part that I don’t
understand is why the things that are not the above (log messages) are
being internationalized.

So what level do we want to have ;) And what level is normal for people
to expect (do systems like hadoop do I18N on there error messages, do
other apache projects?)


I agree with you.

Documentation and user-facing interfaces should definitely support I18N.
That means Horizon, and (maybe) the Python client bindings.
Operators-facing log messages... not so much. By internationalizing them
we lose the ability to google for them, and I'm not convinced the
trade-off is favorable for that particular population.


As long as user facing interfaces also means command line interface, I 
think we're on the same page.


-Sean

--
Sean Dague
IBM Linux Technology Center
email: slda...@us.ibm.com
alt-email: sda...@linux.vnet.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] I18n issue for OpenStack

2012-04-11 Thread Sean Dague

On 04/11/2012 08:41 AM, Thierry Carrez wrote:

Sheng Bo Hou wrote:

After the research done by my colleage Edward Zhang and myself, we have
found the following issues for openstack. We have already raised bug
https://bugs.launchpad.net/nova/+bug/974810
<https://bugs.launchpad.net/nova/+bug/974810>  .
[...]


Thanks for your analysis. We plan to discuss how to fix and extend I18N
at the summit. One question that was raised on the ML in February was
whether internationalization was actually worth the effort for
infrastructure software like OpenStack.

I'll be the first to admit that there are other languages than English,
but all our open development is based in English already (bugs, reviews,
commit messages, mailing-lists, IRC...), so I don't think supporting
more languages in the software itself will help growing our developer
community.


I would tend to disagree with that. People are more likely to invest 
their time in software if they'll be able to use it better in their 
locale. I think this is definitely even more true in places where 
English has less of a dominant presence. It may even bring people to the 
table solely interested in helping with translation. I've seen that 
happen elsewhere.



On our users community, do operators of OpenStack need translated error
messages ? Given that translations are often incomplete, is it worth it
? What do comparable infrastructure open source software projects
provide ? The effort to provide them has proven non-trivial, I'd like to
make sure it's time well spent.


If we want to think about OpenStack as a basic building block like 
Apache, i18n is critical. Otherwise there are regions that won't adopt 
it solely because of a lack of i18n.


Is there a metric on the completeness so far? Something automated that 
could be a jenkins coverage kind of test?


-Sean

--
Sean Dague
IBM Linux Technology Center
email: slda...@us.ibm.com
alt-email: sda...@linux.vnet.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] Nova database optimizing

2012-04-09 Thread Sean Dague
There are a lot of possible issues contained in this bug - 
https://bugs.launchpad.net/nova/+bug/964824, which I'm happy to break 
out as separate issues to be handled in launchpad, so they can each be 
dealt with on their own merits.


But before I get started on this process, are there any objections to 
separating out the issues and just banging these out? Also, who are the 
right review folks for this? The first change is ready for review at - 
https://review.openstack.org/#change,5970


Thanks much,

-Sean

--
Sean Dague
IBM Linux Technology Center
email: slda...@us.ibm.com
alt-email: sda...@linux.vnet.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Performance diagnosis of metadata query

2012-03-30 Thread Sean Dague

On 03/29/2012 02:45 PM, Justin Santa Barbara wrote:

I think _we_ should stand behind OpenStack.

That said, I understand why it's a tough call.  Hopefully most problems
can be fixed with a simple-ish SQL script post-release.


I've got a fix for the first index issue up in gerrit for master. 
https://review.openstack.org/#change,5970


Decisions can be made later about backporting, but for right now I'd 
appreciate comments.


    -Sean

--
Sean Dague
IBM Linux Technology Center
email: slda...@us.ibm.com
alt-email: sda...@linux.vnet.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Performance diagnosis of metadata query

2012-03-29 Thread Sean Dague

On 03/25/2012 01:03 PM, Justin Santa Barbara wrote:

The performance of the metadata query with cloud-init has been causing
some people problems (it's so slow cloud-init times out!), and has led
to the suggestion that we need lots of caching.  (My hypothesis is that
we don't...)

By turning on SQL debugging in SQL Alchemy (for which I've proposed a
patch for Essex: https://review.openstack.org/#change,5783), I was able
to capture the SQL statements.

I'm focusing on the SQL statements for the metadata call.


Is there a good way to map back where in the code these calls are coming 
from?




My install in nowhere near big enough for any of these to actually cause
a real problem, so I'd love to get timings / a log from someone that is
having a problem.  Even the table scan of fixed_ips should be OK if you
have enough RAM.


It's not just enough RAM, but that mysql was specifically tuned to have 
big table caches. Regardless, large table scans should be eliminated, 
especially if the table is mostly read, as the hit on an extra index on 
insert will be completely offset by the speedups on select.


-Sean

--
Sean Dague
IBM Linux Technology Center
email: slda...@us.ibm.com
alt-email: sda...@linux.vnet.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] git review complains about Contributor Agreement not signed

2012-03-14 Thread Sean Dague

On 03/14/2012 10:17 AM, Russell Bryant wrote:



Referring to the above wiki page, have you added yourself to the
contributors wiki page?  Have you requested and been approved for
membership in the openstack-cla group on Launchpad?


There seems to be at least a few day's holdup on approvals into 
openstack-cla (I'm not sure how often those get looked at). I know I 
filled out everything on Monday and am still in the pending queue.


-Sean

--
Sean Dague
IBM Linux Technology Center
email: slda...@us.ibm.com
alt-email: sda...@linux.vnet.ibm.com


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp