Re: [Openstack] using objects returned from DB layer
I agree except I though the preference was for instance_uuid = instance['uuid'] not instance_uuid = instance.uuid (use dict's and don't assume sqlalchemy) On Wed, Dec 14, 2011 at 11:19 PM, Devin Carlen devin.car...@gmail.com wrote: Yes, we should absolutely push to make this more consistent across the board. We are severely limiting possible future implementations. So, yea. +1! Devin On Dec 14, 2011, at 11:10 PM, Chris Behrens wrote: I've seen a number of patches lately that have code like this: instance = db.instance_get(...) instance_uuid = instance.uuid instead of: instance_uuid = instance['uuid'] There's a mix of usage throughout the code, and I know some people are just matching the surrounding code. But, in a number of cases, I've asked for these to be corrected to the latter, on assumption that the DB layer will be returning dictionaries at some point vs the models. It also pushes the code towards consistent usage. But I might be the only Nova Core member looking at this and/or maybe my assumption is wrong. So, I ask here: Should Nova Core make an effort to reject patches with the former format? Or did I miss any DB layer plans where the former format is now preferred? - Chris ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
2011/12/15 Jesse Andrews anotherje...@gmail.com: I agree except I though the preference was for instance_uuid = instance['uuid'] not instance_uuid = instance.uuid (use dict's and don't assume sqlalchemy) Yes... That's what Chris is saying, isn't it? -- Soren Hansen | http://linux2go.dk/ Ubuntu Developer | http://www.ubuntu.com/ OpenStack Developer | http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
++ on moving to a consistent dict-style syntax.
We could attempt to rip the Band-Aid off here by:
1. Rejecting any new patches which use the dot-notation
2. Trying to switch out to using dot-notation access all at once in one big
'fix-it-up' patch.
I'm not convinced 2) is possible. Separating model objects from other kinds of
objects will be time-consuming and error prone. Given the scope of this issue,
I'm not sure this could be done without a real risk of prolonged, recurring
breakage.
Instead, I'd advocate a step-wise approach:
1. Reject any new patches which use the dot-notation
2. Add code which helps us identify and fix dot-notation usage throughout the
codebase.
The basic idea here is that we'd add a flag to fail loudly when a object is
accessed using non-dict methods.
This could be done by adding a decorator to the sqlalchemy/db/api functions
such that instead of returning SQLAlchemy objects directly they instead return
objects of type AttrAccessibleDict.
An AttrAccessibleDict would be defined something like:
class AttrAccessibleDict(dict)
def __getattr__(self, attr):
if FLAGS.enforce_model_dict_access:
raise ModelObjectAccessError('must use dict-style access on model
objects')
else:
return self[attr]
def dictify_model(f):
def wrapper(*args, **kwargs):
rv = f(*args, **kwargs)
attr_dict = convert_to_attr_accessible_dict(rv)
return attr_dict
return wrapper
@dictify_model
def instance_get(…..):
pass
3. Make a concerted effort to fix the code over a period of a few weeks.
Developers could set --enforce_model_dict_access to True and fix up as many
cases as they can find. When they're tired of fixing cases or when more
pressing things come along for the moment, they can temporarily set
--enforce_model_dict_access back to False to that everything goes back to
humming along smoothly.
4. Once the dot-notation has been excised, we can switch over to actually
returning real Python dictionaries instead of AttrAccessibleDict objects.
5. Final step would be removing the cleanup code.
Once everything is returning dictionaries, we would remove the decorator
entirely and any other cleanup code we added along the way.
Feels like this approach would be a bit roundabout, but each step feels safe,
and it seem like it would get us to where we want to be in the course of a few
weeks (perhaps a couple of months, worst case).
Thoughts?
-Rick
On Dec 15, 2011, at 1:10 AM, Chris Behrens wrote:
I've seen a number of patches lately that have code like this:
instance = db.instance_get(...)
instance_uuid = instance.uuid
instead of:
instance_uuid = instance['uuid']
There's a mix of usage throughout the code, and I know some people are just
matching the surrounding code. But, in a number of cases, I've asked for
these to be corrected to the latter, on assumption that the DB layer will be
returning dictionaries at some point vs the models. It also pushes the code
towards consistent usage. But I might be the only Nova Core member looking
at this and/or maybe my assumption is wrong.
So, I ask here: Should Nova Core make an effort to reject patches with the
former format? Or did I miss any DB layer plans where the former format is
now preferred?
- Chris
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Metadata and File Injection
Great question. Right now there are 3 approaches to metadata/runtime config: * ec2 metadata service - http://169.254.169.254/ (used by ubuntu's cloud-init for example) * config drive - added in diablo * xenstore via openstack agent - https://launchpad.net/openstack-guest-agents * injecting files into the filesystem prior to launch - on some hypervisors / filesystems combinations There are issues with each approach currently: * metadata service - requires that cloud uses DHCP to configure network * config drive - only sets values at runtime and no support from any guests * xenstore approach - hypervisor specific. * injecting into filesystem - brittle as it requires the host to support arbitrary filesystems If we assume that instance networking can be configured by DHCP, then a metadata service is probably our best choice for simplicity of implementation and user experience. Given that Rackspace Public Cloud is the only openstack developer/user (I know of) that doesn't use DHCP for network configuration of instances, I reached out to the team to see if DHCP would be an option. They are researching either a way use DHCP _or_ propose an approach to initial network configuration that can be executed by the openstack guest agent. Based on those conversations we've been writing a proposal that would recommend guests use a (currently non-existing) openstack metadata api - http://wiki.openstack.org/guest-configuration Hope this helps. Thoughts on the optimal experience for essex? On Wed, Dec 14, 2011 at 9:28 AM, Jay Pipes jaypi...@gmail.com wrote: On Wed, Dec 14, 2011 at 9:04 AM, McNally, Dave dave.mcna...@hp.com wrote: Hi, I've recently been looking at file and metadata injection in Nova and I have a question relating to it. (BTW this is based off what I have seen in nova/virt/disk.py) I notice that for key/value pairs specified as metadata during boot of an instance these values are injected into a file /meta.js in the instance. However if a file (and corresponding injection location) are specified when booting the instance the file does not get injected. Not on libvirt/KVM. It works on Xen, though: https://bugs.launchpad.net/nova/+bug/755168 I was wondering if there was an intentional decision not to use a similar method to that used when injecting meta.js to inject other files? Because it seems to me the addition of such functionality would be fairly straightforward. I'm interested in the answer to this question as well... Also on a vaguely related note why is the metadata injected into a file rather than stored in a location where it can be retrieved from the metadata service? No idea. :( Cheers, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
Yes, that was what I was saying. :) On Dec 15, 2011, at 12:06 AM, Soren Hansen wrote: 2011/12/15 Jesse Andrews anotherje...@gmail.com: I agree except I though the preference was for instance_uuid = instance['uuid'] not instance_uuid = instance.uuid (use dict's and don't assume sqlalchemy) Yes... That's what Chris is saying, isn't it? -- Soren Hansen| http://linux2go.dk/ Ubuntu Developer| http://www.ubuntu.com/ OpenStack Developer | http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
Agree with both #1s as a start. And I wouldn't try to 'rip off the band-aid'
either.
I was bringing this up initially as I want to enforce *something* when
reviewing, but if we want to start 'fixing' things, we can start hitting small
sections of code to limit conflicts. The 'enforce_model' thing might be a bit
extreme right now. We may be able to get to a place where we feel we've got
everything... then dictify it all at that time and see what raises in tests.,
fixing those last things. Hm... although, I guess I really do like the ability
to turn something off if it's broken. :)
- Chris
On Dec 15, 2011, at 12:29 AM, Rick Harris wrote:
++ on moving to a consistent dict-style syntax.
We could attempt to rip the Band-Aid off here by:
1. Rejecting any new patches which use the dot-notation
2. Trying to switch out to using dot-notation access all at once in one big
'fix-it-up' patch.
I'm not convinced 2) is possible. Separating model objects from other kinds
of objects will be time-consuming and error prone. Given the scope of this
issue, I'm not sure this could be done without a real risk of prolonged,
recurring breakage.
Instead, I'd advocate a step-wise approach:
1. Reject any new patches which use the dot-notation
2. Add code which helps us identify and fix dot-notation usage throughout the
codebase.
The basic idea here is that we'd add a flag to fail loudly when a object is
accessed using non-dict methods.
This could be done by adding a decorator to the sqlalchemy/db/api functions
such that instead of returning SQLAlchemy objects directly they instead
return objects of type AttrAccessibleDict.
An AttrAccessibleDict would be defined something like:
class AttrAccessibleDict(dict)
def __getattr__(self, attr):
if FLAGS.enforce_model_dict_access:
raise ModelObjectAccessError('must use dict-style access on model
objects')
else:
return self[attr]
def dictify_model(f):
def wrapper(*args, **kwargs):
rv = f(*args, **kwargs)
attr_dict = convert_to_attr_accessible_dict(rv)
return attr_dict
return wrapper
@dictify_model
def instance_get(…..):
pass
3. Make a concerted effort to fix the code over a period of a few weeks.
Developers could set --enforce_model_dict_access to True and fix up as many
cases as they can find. When they're tired of fixing cases or when more
pressing things come along for the moment, they can temporarily set
--enforce_model_dict_access back to False to that everything goes back to
humming along smoothly.
4. Once the dot-notation has been excised, we can switch over to actually
returning real Python dictionaries instead of AttrAccessibleDict objects.
5. Final step would be removing the cleanup code.
Once everything is returning dictionaries, we would remove the decorator
entirely and any other cleanup code we added along the way.
Feels like this approach would be a bit roundabout, but each step feels safe,
and it seem like it would get us to where we want to be in the course of a
few weeks (perhaps a couple of months, worst case).
Thoughts?
-Rick
On Dec 15, 2011, at 1:10 AM, Chris Behrens wrote:
I've seen a number of patches lately that have code like this:
instance = db.instance_get(...)
instance_uuid = instance.uuid
instead of:
instance_uuid = instance['uuid']
There's a mix of usage throughout the code, and I know some people are just
matching the surrounding code. But, in a number of cases, I've asked for
these to be corrected to the latter, on assumption that the DB layer will be
returning dictionaries at some point vs the models. It also pushes the code
towards consistent usage. But I might be the only Nova Core member looking
at this and/or maybe my assumption is wrong.
So, I ask here: Should Nova Core make an effort to reject patches with the
former format? Or did I miss any DB layer plans where the former format is
now preferred?
- Chris
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
I was bringing this up initially as I want to enforce *something* when
reviewing,
Yeah, I was just thinking that it could be a point of confusion if, for an
extended period, we're in a state where new code has to use dict-style instance
access, while nearby, older code still uses attr-accessing.
We may be able to get to a place where we feel we've got everything... then
dictify it all at that time and see what raises in tests
One issue with that is that many of our fakes will already be returning dicts,
so the unit-tests may pass, while the actual code may still trigger an
exception.
The functional tests will help here, but, I don't think there's a substitute
for having --enforce_model=True and having everybody hammering at it for a few
weeks.
If it can survive that with no problems, then I'll be somewhat confident….
On Dec 15, 2011, at 2:49 AM, Chris Behrens wrote:
Agree with both #1s as a start. And I wouldn't try to 'rip off the band-aid'
either.
I was bringing this up initially as I want to enforce *something* when
reviewing, but if we want to start 'fixing' things, we can start hitting
small sections of code to limit conflicts. The 'enforce_model' thing might
be a bit extreme right now. We may be able to get to a place where we feel
we've got everything... then dictify it all at that time and see what raises
in tests., fixing those last things. Hm... although, I guess I really do
like the ability to turn something off if it's broken. :)
- Chris
On Dec 15, 2011, at 12:29 AM, Rick Harris wrote:
++ on moving to a consistent dict-style syntax.
We could attempt to rip the Band-Aid off here by:
1. Rejecting any new patches which use the dot-notation
2. Trying to switch out to using dot-notation access all at once in one big
'fix-it-up' patch.
I'm not convinced 2) is possible. Separating model objects from other kinds
of objects will be time-consuming and error prone. Given the scope of this
issue, I'm not sure this could be done without a real risk of prolonged,
recurring breakage.
Instead, I'd advocate a step-wise approach:
1. Reject any new patches which use the dot-notation
2. Add code which helps us identify and fix dot-notation usage throughout
the codebase.
The basic idea here is that we'd add a flag to fail loudly when a object is
accessed using non-dict methods.
This could be done by adding a decorator to the sqlalchemy/db/api functions
such that instead of returning SQLAlchemy objects directly they instead
return objects of type AttrAccessibleDict.
An AttrAccessibleDict would be defined something like:
class AttrAccessibleDict(dict)
def __getattr__(self, attr):
if FLAGS.enforce_model_dict_access:
raise ModelObjectAccessError('must use dict-style access on model
objects')
else:
return self[attr]
def dictify_model(f):
def wrapper(*args, **kwargs):
rv = f(*args, **kwargs)
attr_dict = convert_to_attr_accessible_dict(rv)
return attr_dict
return wrapper
@dictify_model
def instance_get(…..):
pass
3. Make a concerted effort to fix the code over a period of a few weeks.
Developers could set --enforce_model_dict_access to True and fix up as many
cases as they can find. When they're tired of fixing cases or when more
pressing things come along for the moment, they can temporarily set
--enforce_model_dict_access back to False to that everything goes back to
humming along smoothly.
4. Once the dot-notation has been excised, we can switch over to actually
returning real Python dictionaries instead of AttrAccessibleDict objects.
5. Final step would be removing the cleanup code.
Once everything is returning dictionaries, we would remove the decorator
entirely and any other cleanup code we added along the way.
Feels like this approach would be a bit roundabout, but each step feels
safe, and it seem like it would get us to where we want to be in the course
of a few weeks (perhaps a couple of months, worst case).
Thoughts?
-Rick
On Dec 15, 2011, at 1:10 AM, Chris Behrens wrote:
I've seen a number of patches lately that have code like this:
instance = db.instance_get(...)
instance_uuid = instance.uuid
instead of:
instance_uuid = instance['uuid']
There's a mix of usage throughout the code, and I know some people are just
matching the surrounding code. But, in a number of cases, I've asked for
these to be corrected to the latter, on assumption that the DB layer will
be returning dictionaries at some point vs the models. It also pushes the
code towards consistent usage. But I might be the only Nova Core member
looking at this and/or maybe my assumption is wrong.
So, I ask here: Should Nova Core make an effort to reject patches with the
former format? Or did I miss any DB layer plans where the former format
is now preferred?
- Chris
[Openstack] Swift maximum recommended objects per container
Hi, While searching for the swift performance tuning tips I came across this post: http://adrianotto.com/2010/09/openstack-os-is-great-for/ In the comments, some users mention that it's better to keep max. number of objects per container less than 1M. As far as I understood, this is mainly due to sqlite limitation. Post is quite old though, and I was wondering if this was addressed in the latest 1.4.4 version? Can I store unlimited (or ~ 1B objects) in a single container? -- Rustam. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Metadata and File Injection
Thanks for the responses guys. Jesse I can see how there might be concerns around using file injection but it seems as if that was chosen as a method of inserting user-specified metadata into an instance which is why I was wondering why it wasn't expanded to use the same process to insert files? Dave -Original Message- From: Jesse Andrews [mailto:anotherje...@gmail.com] Sent: 15 December 2011 08:34 To: Jay Pipes Cc: McNally, Dave; openstack@lists.launchpad.net Subject: Re: [Openstack] Metadata and File Injection Great question. Right now there are 3 approaches to metadata/runtime config: * ec2 metadata service - http://169.254.169.254/ (used by ubuntu's cloud-init for example) * config drive - added in diablo * xenstore via openstack agent - https://launchpad.net/openstack-guest-agents * injecting files into the filesystem prior to launch - on some hypervisors / filesystems combinations There are issues with each approach currently: * metadata service - requires that cloud uses DHCP to configure network * config drive - only sets values at runtime and no support from any guests * xenstore approach - hypervisor specific. * injecting into filesystem - brittle as it requires the host to support arbitrary filesystems If we assume that instance networking can be configured by DHCP, then a metadata service is probably our best choice for simplicity of implementation and user experience. Given that Rackspace Public Cloud is the only openstack developer/user (I know of) that doesn't use DHCP for network configuration of instances, I reached out to the team to see if DHCP would be an option. They are researching either a way use DHCP _or_ propose an approach to initial network configuration that can be executed by the openstack guest agent. Based on those conversations we've been writing a proposal that would recommend guests use a (currently non-existing) openstack metadata api - http://wiki.openstack.org/guest-configuration Hope this helps. Thoughts on the optimal experience for essex? On Wed, Dec 14, 2011 at 9:28 AM, Jay Pipes jaypi...@gmail.com wrote: On Wed, Dec 14, 2011 at 9:04 AM, McNally, Dave dave.mcna...@hp.com wrote: Hi, I've recently been looking at file and metadata injection in Nova and I have a question relating to it. (BTW this is based off what I have seen in nova/virt/disk.py) I notice that for key/value pairs specified as metadata during boot of an instance these values are injected into a file /meta.js in the instance. However if a file (and corresponding injection location) are specified when booting the instance the file does not get injected. Not on libvirt/KVM. It works on Xen, though: https://bugs.launchpad.net/nova/+bug/755168 I was wondering if there was an intentional decision not to use a similar method to that used when injecting meta.js to inject other files? Because it seems to me the addition of such functionality would be fairly straightforward. I'm interested in the answer to this question as well... Also on a vaguely related note why is the metadata injected into a file rather than stored in a location where it can be retrieved from the metadata service? No idea. :( Cheers, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] swift acl's
Hi All, I am testing acl support in swift-1.4.5. According to the document http://swift.openstack.org/misc.html#module-swift.common.middleware.acl the syntax to allow all non-admin users read access to the container except for one is as following. Executing the curl following curl command from an admin user. curl -v -X POST -H 'X-Auth-Token: AUTH_tkea3fdbf40e5b40708a51db0377be3f47' http://127.0.0.1:8080/v1/AUTH_test/cont -H 'X-Container-Read: .r:*,.rlistings,.r:-test:tester3' curl -v -X HEAD -H 'X-Auth-Token: AUTH_tkea3fdbf40e5b40708a51db0377be3f47' http://127.0.0.1:8080/v1/AUTH_test/cont *About to connect() to 127.0.0.1 port 8080 (#0) * Trying 127.0.0.1... connected * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0) HEAD /v1/AUTH_test/cont HTTP/1.1 User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/ 3.12.9.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2 Host: 127.0.0.1:8080\ Accept: X-Auth-Token: AUTH_tkea3fdbf40e5b40708a51db0377be3f47 HTTP/1.1 204 No Content X-Container-Object-Count: 10 X-Container-Read: .r:*,.rlistings,.r:-test:tester3 X-Container-Bytes-Used: 1 Accept-Ranges: bytes Content-Length: 0 Date: Thu, 15 Dec 2011 18:38:25 GMT * Connection #0 to host 127.0.0.1 left intact * Closing connection #0 -- But GET operations still succeed for the user tester3. What else is required to make the swift-server deny this user from doing GET operations. Thanks in advance. regards, Junaid ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Glance 2012.1-dev! ERROR: NO 'app_factory' attribute
Hi,all, Glance version i am running is 2012.1-dev. But glance-api can't be started(maybe this version is just unstable for now?).Here is some details: *#sudo glance-api glance-api.conf --debug -v * *[1]27173* *Error trying to load config /etc/glance/glance-api.conf:module 'glance.api.v1' from * *'/usr/local/lib/python2.7/dist-packages/glance-2012.1-py2.7.egg/glance/api/v1/__init__.pyc' has no 'app_factory' attribute* * * Any ideas?Thanks very much. * * Yours, Jeff ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Inner Exception: cannot import name v1_1 from (pid=xxx) import_class
How about adding the following two flags in nova.conf file? We are running flat dhcp without problem. You need to customize the values of the flags. --flat_network_dhcp_start=10.99.1.2 --flat_interface=eth0 David. -- Dr. Dong-In David Kang Computer Scientist USC/ISI - Original Message - Anyone? On 14/12/2011, at 15:24, denmat tu2bg...@gmail.com wrote: Hi all, I have been following the installation instructions here: http://cssoss.files.wordpress.com/2011/08/openstackbookv1-0_csscorp.pdf And I had a successful install at one stage - then I tried to install the dashboard, which wanted keystone and now things have gone a little messy. Previously this returned successfully: localadmin@au-mel-kvm-3:~$ euca-describe-availability-zones verbose [Errno 111] Connection refused This is because the nova services cannot be started. Each nova service complains in a similar way: == /var/log/nova/nova-network.log == 2011-12-14 15:12:20,129 DEBUG nova.utils [-] Inner Exception: cannot import name v1_1 from (pid=3298) import_class /usr/lib/python2.7/dist-packages/nova/utils.py:65 2011-12-14 15:12:20,130 CRITICAL nova [-] Class FlatDHCPManager could not be found (nova): TRACE: Traceback (most recent call last): (nova): TRACE: File /usr/bin/nova-network, line 47, in module (nova): TRACE: server = service.Service.create(binary='nova-network') (nova): TRACE: File /usr/lib/python2.7/dist-packages/nova/service.py, line 216, in create (nova): TRACE: report_interval, periodic_interval) (nova): TRACE: File /usr/lib/python2.7/dist-packages/nova/service.py, line 125, in __init__ (nova): TRACE: manager_class = utils.import_class(self.manager_class_name) (nova): TRACE: File /usr/lib/python2.7/dist-packages/nova/utils.py, line 66, in import_class (nova): TRACE: raise exception.ClassNotFound(class_name=class_str) (nova): TRACE: ClassNotFound: Class FlatDHCPManager could not be found (nova): TRACE: Keystone currently works (curl tests respond) and the I can use the dashboard to sign in but I receive these (understandably): Error: Unable to get usage info: [Errno 111] ECONNREFUSED Below are my config files: nova.conf --allow_admin_api --api_paste_config=/etc/nova/nova-api-paste.ini --cc_host=10.61.2.246 --dhcpbridge_flagfile=/etc/nova/nova.conf --dhcpbridge=/usr/bin/nova-dhcpbridge --ec2_url=http://172.31.0.10:8773/services/Cloud --FAKE_subdomain=ec2 --fixed_range=172.31.0.0/16 --glance_api_servers=172.31.0.10:9292 --image_service=nova.image.glance.GlanceImageService --iscsi_ip_prefix=172.31. --lock_path=/var/lock/nova --logdir=/var/log/nova --network_manager=nova.network.manager.FlatDHCPManager --network_size=16 --osapi_extension=extensions.admin.Admin --osapi_extension=nova.api.openstack.v2.contrib.standard_extensions --osapi_extensions_path=/opt/build/horizon/openstack-dashboard/.dashboard-venv/src/openstackx/extensions --rabbit_host=10.61.2.246 --routing_source_ip=10.61.2.246 --s3_host=172.31.0.10 --scheduler_driver=nova.scheduler.simple.SimpleScheduler --sql_connection=mysql://root:removed@172.31.0.10/nova --state_path=/var/lib/nova --verbose nova-api-paste.ini [composite:ec2] use = egg:Paste#urlmap /: ec2versions /services/Cloud: ec2cloud /services/Admin: ec2admin /latest: ec2metadata /2007-01-19: ec2metadata /2007-03-01: ec2metadata /2007-08-29: ec2metadata /2007-10-10: ec2metadata /2007-12-15: ec2metadata /2008-02-01: ec2metadata /2008-09-01: ec2metadata /2009-04-04: ec2metadata /1.0: ec2metadata [pipeline:ec2cloud] pipeline = logrequest totoken authtoken keystonecontext cloudrequest authorizer ec2executor [pipeline:ec2admin] pipeline = logrequest totoken authtoken keystonecontext adminrequest authorizer ec2executor [pipeline:ec2metadata] pipeline = logrequest ec2md [pipeline:ec2versions] pipeline = logrequest ec2ver [filter:logrequest] paste.filter_factory = nova.api.ec2:RequestLogging.factory [filter:ec2lockout] paste.filter_factory = nova.api.ec2:Lockout.factory [filter:totoken] paste.filter_factory = keystone.middleware.ec2_token:EC2Token.factory [filter:ec2noauth] paste.filter_factory = nova.api.ec2:NoAuth.factory [filter:authenticate] paste.filter_factory = nova.api.ec2:Authenticate.factory [filter:cloudrequest] controller = nova.api.ec2.cloud.CloudController paste.filter_factory = nova.api.ec2:Requestify.factory [filter:adminrequest] controller = nova.api.ec2.admin.AdminController paste.filter_factory = nova.api.ec2:Requestify.factory [filter:authorizer] paste.filter_factory = nova.api.ec2:Authorizer.factory [app:ec2executor] paste.app_factory = nova.api.ec2:Executor.factory [app:ec2ver] paste.app_factory = nova.api.ec2:Versions.factory [app:ec2md] paste.app_factory =
Re: [Openstack] Glance 2012.1-dev! ERROR: NO 'app_factory' attribute
Dear Jeff , Could u check ur conf file for app_factory section! I think ur conf file is out of date to match latest glance code. Compare with the new sample will figure it out. Hope it help Hugo's iPhone jeffrey coho jeffreycohob...@gmail.com 於 2011/12/15 21:28 寫道： Hi,all, Glance version i am running is 2012.1-dev. But glance-api can't be started(maybe this version is just unstable for now?).Here is some details: #sudo glance-api glance-api.conf --debug -v [1]27173 Error trying to load config /etc/glance/glance-api.conf:module 'glance.api.v1' from '/usr/local/lib/python2.7/dist-packages/glance-2012.1-py2.7.egg/glance/api/v1/__init__.pyc' has no 'app_factory' attribute Any ideas?Thanks very much. Yours, Jeff ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift maximum recommended objects per container
Just for reference, question was answered here: https://answers.launchpad.net/swift/+question/181977 On 15/12/2011 10:02, Rustam Aliyev wrote: Hi, While searching for the swift performance tuning tips I came across this post: http://adrianotto.com/2010/09/openstack-os-is-great-for/ In the comments, some users mention that it's better to keep max. number of objects per container less than 1M. As far as I understood, this is mainly due to sqlite limitation. Post is quite old though, and I was wondering if this was addressed in the latest 1.4.4 version? Can I store unlimited (or ~ 1B objects) in a single container? -- Rustam. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
On Thu, Dec 15, 2011 at 3:29 AM, Rick Harris rick.har...@rackspace.com wrote: ++ on moving to a consistent dict-style syntax. +1 from me, too. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Expectations for tracebacks
Hello all. We've recently upgraded our cactus system to more recent code. In the process of doing this, we've started logging whenever we get tracebacks out of any of the openstack components we are running. Some of these are clearly bugs, while others correspond to normal operational conditions (like the case where all compute resources are occupied). My question is, what is the project position on tracebacks? Should they be considered to be expected, or should we be treating them as bugs and filing tickets/patches for them? thanks. -nld ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Expectations for tracebacks
I don't know if there's an official project policy, but personally I don't think an end-user should ever see a Python traceback in a logfile under normal operations (i.e., the user has configured all of the services correctly). Lorin -- Lorin Hochstein, Computer Scientist USC Information Sciences Institute 703.812.3710 http://www.east.isi.edu/~lorin On Dec 15, 2011, at 10:49 AM, Narayan Desai wrote: Hello all. We've recently upgraded our cactus system to more recent code. In the process of doing this, we've started logging whenever we get tracebacks out of any of the openstack components we are running. Some of these are clearly bugs, while others correspond to normal operational conditions (like the case where all compute resources are occupied). My question is, what is the project position on tracebacks? Should they be considered to be expected, or should we be treating them as bugs and filing tickets/patches for them? thanks. -nld ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Glance 2012.1-dev! ERROR: NO 'app_factory' attribute
Yeah, I was worried we would run into these kinds of problems with a recent commit... Basically, do what Hugo suggested. We recently overhauled the configuration and paste deploy factories in Glance to align with the common cfg work being done in Nova and Glance. This means that the glance-api.conf and glance-registry.conf paste deploy sections need to be updated. You can see the changes needed by looking at this diff: https://review.openstack.org/#patch,sidebyside,2139,1,etc/glance-api.conf https://review.openstack.org/#patch,sidebyside,2139,1,etc/glance-cache.conf https://review.openstack.org/#patch,sidebyside,2139,1,etc/glance-registry.conf Ping us back if you have issues figuring out the needed changes. Thanks! -jay On Thu, Dec 15, 2011 at 8:28 AM, jeffrey coho jeffreycohob...@gmail.com wrote: Hi,all, Glance version i am running is 2012.1-dev. But glance-api can't be started(maybe this version is just unstable for now?).Here is some details: #sudo glance-api glance-api.conf --debug -v [1]27173 Error trying to load config /etc/glance/glance-api.conf:module 'glance.api.v1' from '/usr/local/lib/python2.7/dist-packages/glance-2012.1-py2.7.egg/glance/api/v1/__init__.pyc' has no 'app_factory' attribute Any ideas?Thanks very much. Yours, Jeff ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Expectations for tracebacks
On Thu, Dec 15 2011, Narayan Desai wrote: Hello all. We've recently upgraded our cactus system to more recent code. In the process of doing this, we've started logging whenever we get tracebacks out of any of the openstack components we are running. Some of these are clearly bugs, while others correspond to normal operational conditions (like the case where all compute resources are occupied). My question is, what is the project position on tracebacks? Should they be considered to be expected, or should we be treating them as bugs and filing tickets/patches for them? thanks. I think all are bugs. Even if you understand some of them and considers them to be logical, you should not see ugly backtraces. You should see nice log lines any system administrator can read and understand clearly. -- Julien Danjou // eNovance http://enovance.com // ✉ julien.dan...@enovance.com ☎ +33 1 49 70 99 81 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
On Thu, 2011-12-15 at 07:10 +, Chris Behrens wrote: There's a mix of usage throughout the code, and I know some people are just matching the surrounding code. But, in a number of cases, I've asked for these to be corrected to the latter, on assumption that the DB layer will be returning dictionaries at some point vs the models. It also pushes the code towards consistent usage. But I might be the only Nova Core member looking at this and/or maybe my assumption is wrong. So, I ask here: Should Nova Core make an effort to reject patches with the former format? Or did I miss any DB layer plans where the former format is now preferred? I have two, diametrically opposed answers. 1. When doing reviews, I've generally tried to enforce the dict access format, because it's been my understanding that that is the direction we're going in. 2. However, I violently disagree with the idea that the DB layer must return dicts. It does not, even if you start talking about allowing use of other kinds of databases. We can, and should, wrap these things in objects, upon which we can call methods that do things—i.e., we should, you know, actually use object-oriented programming. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
-1 By using the dict style access to these records we are tying ourselves to the internal implementation of those records. If we want to be able to move the 'data model' logic out of the 'process' logic we will need to access the models as attributes and not dict items. We should confine the dict-style access to the db records to the data models. ~ Aaron On Dec 15, 2011, at 9:17 AM, Jay Pipes wrote: On Thu, Dec 15, 2011 at 3:29 AM, Rick Harris rick.har...@rackspace.com wrote: ++ on moving to a consistent dict-style syntax. +1 from me, too. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
On Dec 15, 2011, at 1:10 AM, Chris Behrens wrote: I've seen a number of patches lately that have code like this: instance = db.instance_get(...) instance_uuid = instance.uuid instead of: instance_uuid = instance['uuid'] There's a mix of usage throughout the code, and I know some people are just matching the surrounding code. But, in a number of cases, I've asked for these to be corrected to the latter, on assumption that the DB layer will be returning dictionaries at some point vs the models. It also pushes the code towards consistent usage. But I might be the only Nova Core member looking at this and/or maybe my assumption is wrong. Actually, what we should be working to is using plain python model objects, and having the sqlalchemy layer use it's mapper functionality (separated from the models) to map db rows to those models. THis will allow any persistence layer to map to the same model objects, and allow us to have actual, object-oriented design in our code. (like real methods, where appropriate) This would suggest the dot notation, although making a python model class support a dict interface for backwards compatability whilst things are refactored is easy to do. So, I ask here: Should Nova Core make an effort to reject patches with the former format? Or did I miss any DB layer plans where the former format is now preferred Myself, and 0x44 discussed this at the db team meetings. We've been writing some blueprints on this. - Chris ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Monsyne M. Dragon OpenStack/Nova cell 210-441-0965 work x 5014190 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Metadata and File Injection
On Thu, 15 Dec 2011, McNally, Dave wrote: Thanks for the responses guys. Jesse I can see how there might be concerns around using file injection but it seems as if that was chosen as a method of inserting user-specified metadata into an instance which is why I was wondering why it wasn't expanded to use the same process to insert files? The biggest flaw in this is that it expects that the host can read a filesystem of the guest. That is simply *guaranteed* to result in a case where this is broken for future guests (which use btrfs, btrfs2.0 or smoserFS-1.2). I'm happy that a network based metadata service is generally seen as the smartest solution. This way its up to the guest to fit into the cloud, no changes to the hypervisors or openstack have to be made to support specific guests. Dave -Original Message- From: Jesse Andrews [mailto:anotherje...@gmail.com] Sent: 15 December 2011 08:34 To: Jay Pipes Cc: McNally, Dave; openstack@lists.launchpad.net Subject: Re: [Openstack] Metadata and File Injection Great question. Right now there are 3 approaches to metadata/runtime config: * ec2 metadata service - http://169.254.169.254/ (used by ubuntu's cloud-init for example) * config drive - added in diablo * xenstore via openstack agent - https://launchpad.net/openstack-guest-agents * injecting files into the filesystem prior to launch - on some hypervisors / filesystems combinations There are issues with each approach currently: * metadata service - requires that cloud uses DHCP to configure network * config drive - only sets values at runtime and no support from any guests * xenstore approach - hypervisor specific. * injecting into filesystem - brittle as it requires the host to support arbitrary filesystems If we assume that instance networking can be configured by DHCP, then a metadata service is probably our best choice for simplicity of implementation and user experience. Given that Rackspace Public Cloud is the only openstack developer/user (I know of) that doesn't use DHCP for network configuration of instances, I reached out to the team to see if DHCP would be an option. They are researching either a way use DHCP _or_ propose an approach to initial network configuration that can be executed by the openstack guest agent. Based on those conversations we've been writing a proposal that would recommend guests use a (currently non-existing) openstack metadata api - http://wiki.openstack.org/guest-configuration Hope this helps. Thoughts on the optimal experience for essex? On Wed, Dec 14, 2011 at 9:28 AM, Jay Pipes jaypi...@gmail.com wrote: On Wed, Dec 14, 2011 at 9:04 AM, McNally, Dave dave.mcna...@hp.com wrote: Hi, I've recently been looking at file and metadata injection in Nova and I have a question relating to it. (BTW this is based off what I have seen in nova/virt/disk.py) I notice that for key/value pairs specified as metadata during boot of an instance these values are injected into a file /meta.js in the instance. However if a file (and corresponding injection location) are specified when booting the instance the file does not get injected. Not on libvirt/KVM. It works on Xen, though: https://bugs.launchpad.net/nova/+bug/755168 I was wondering if there was an intentional decision not to use a similar method to that used when injecting meta.js to inject other files? Because it seems to me the addition of such functionality would be fairly straightforward. I'm interested in the answer to this question as well... Also on a vaguely related note why is the metadata injected into a file rather than stored in a location where it can be retrieved from the metadata service? No idea. :( Cheers, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Metadata and File Injection
On Thu, 15 Dec 2011, Jesse Andrews wrote: Great question. Right now there are 3 approaches to metadata/runtime config: * ec2 metadata service - http://169.254.169.254/ (used by ubuntu's cloud-init for example) * config drive - added in diablo * xenstore via openstack agent - https://launchpad.net/openstack-guest-agents * injecting files into the filesystem prior to launch - on some hypervisors / filesystems combinations There are issues with each approach currently: * metadata service - requires that cloud uses DHCP to configure network * config drive - only sets values at runtime and no support from any guests * xenstore approach - hypervisor specific. * injecting into filesystem - brittle as it requires the host to support arbitrary filesystems If we assume that instance networking can be configured by DHCP, then a metadata service is probably our best choice for simplicity of implementation and user experience. Given that Rackspace Public Cloud is the only openstack developer/user (I know of) that doesn't use DHCP for network configuration of instances, I reached out to the team to see if DHCP would be an option. They are researching either a way use DHCP _or_ propose an approach to initial network configuration that can be executed by the openstack guest agent. Based on those conversations we've been writing a proposal that would recommend guests use a (currently non-existing) openstack metadata api - http://wiki.openstack.org/guest-configuration I'm just curious, what are the motivations behind inventing something other than the EC2 Metadata service? It is generally functional, and quite a lot can (and has) built atop this simple service. Secondly, There are 2 features that I feel are missing from the metadata service. And I'd hope that these could be accounted for if there is going to be invention done. a.) user-data is a single entity. There are potentially multiple sources that want to provide input to a guest (the end user might want to install some packages at boot, and the cloud infrastructure might want to tell the guest of a local mirror). cloud-init supports multipart-mime in userdata, so that there can be separate pieces inside that single source, but even then, all parties involved have to agree that they do not completely own that resource. b.) There is no way to disable it. cloud-init supports writing a null-route to the metadata service, which can make it inaccessible to any non-root entity on the system. However, it would be nicer if there was a way to disable it entirely. With that in place, passing credentials into the guest would be easier as once they're consumed they can be removed. c.) No way to modify contents of the service after instance launch. OK, I said 2 features, and really this one is wishlist. If we had an arbitrary key-value store that was available, the user could interact with the guest using this service. It'd have to be poll-based, but a in-guest hypervisor could watch for creation and deletion of keys. Potentially, the MD might be RW from inside guest, meaning it could even convey information back. I consider this wishlist because really, there are perfectly good solutions for communicating to in-guest agents, theres not a lot of reason to invent a new one. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Writes are faster than reads in Swift
On Wed, Dec 14, 2011 at 5:38 PM, Ewan Mellor ewan.mel...@eu.citrix.com wrote: Or static disk image files! Only if you've got enough RAM on the storage worker node to cache the entire disk image. Otherwise it's just going to get evicted straight away. The case where you've got so few, small, disk images that you can cache them all in RAM must be pretty rare. And if it's not rare in your use-case, I reckon giving that RAM to your Glance nodes and doing the caching there would be just as good if not better. There's no reason it couldn't be both. Generally, the Glance API servers are not installed on the same servers as Swift object servers... My point being is that Swift was not designed for (large) static files for readonly workloads, and of course, that is the use case for Glance with a Swift backend (where a single container contains few objects, but they tend to be large). It would be great to be able to control the disk flushing behaviour in Swift to control for these alternate workloads. Certain use cases, such as starting up a lot of servers across lots of Nova compute nodes from a base image stored in Swift (read through Glance or not through Glance, it doesn't matter here...), would benefit from not throwing away the disk blocks that will be read heavily over and over again. Even on my home desktop machine (nowhere near the types of servers I see typically provisioned for Swift), I've got a bunch of buffer space available: jpipes@uberbox:~$ free total used free sharedbuffers cached Mem: 247301527997080 16733072 024883603482836 -/+ buffers/cache:2025884 22704268 Swap: 25149436 0 25149436 We can simulate a situation where a 2GB image file is available for streaming through a Swift object server by creating a 2GB file and cat'ing it to /dev/null. The below timing results show the time taken to stream the image when the image's disk blocks are in the disk cache: jpipes@uberbox:~$ dd if=/dev/zero of=fakeimage bs=1M count=2000 2000+0 records in 2000+0 records out 2097152000 bytes (2.1 GB) copied, 1.25832 s, 1.7 GB/s jpipes@uberbox:~$ free total used free sharedbuffers cached Mem: 247301523821108 20909044 0 97522220208 -/+ buffers/cache:1591148 23139004 Swap: 25149436 0 25149436 jpipes@uberbox:~$ time cat fakeimage /dev/null real0m0.346s user0m0.000s sys 0m0.340s Now, we simulate the dropping of the disk buffer cache: jpipes@uberbox:~$ echo 3 | sudo tee /proc/sys/vm/drop_caches 3 And try streaming the image again: jpipes@uberbox:~$ time cat fakeimage /dev/null real0m8.813s user0m0.012s sys 0m1.360s 0.346s vs. 8.813s is a pretty massive difference and one that I think deserves at least a switch to control the behaviour in Swift. Cheers, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
On Thu, Dec 15, 2011, Kevin L. Mitchell kevin.mitch...@rackspace.com wrote: 2. However, I violently disagree with the idea that the DB layer must return dicts. It does not, even if you start talking about allowing use of other kinds of databases. We can, and should, wrap these things in objects, upon which we can call methods that do things—i.e., we should, you know, actually use object-oriented programming. What kinds of things? I'm not against returning back a standardized object that provides __getattr__ so we don't have to use dict notation. Any database backend can do something similar easily. I'm just trying to better understand what is object-oriented about the data returned from a database? What methods would we want to use? JE ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Expectations for tracebacks
Definitely we should be filing bugs. Jesse On Thu, Dec 15, 2011 at 8:00 AM, Julien Danjou julien.dan...@enovance.com wrote: On Thu, Dec 15 2011, Narayan Desai wrote: Hello all. We've recently upgraded our cactus system to more recent code. In the process of doing this, we've started logging whenever we get tracebacks out of any of the openstack components we are running. Some of these are clearly bugs, while others correspond to normal operational conditions (like the case where all compute resources are occupied). My question is, what is the project position on tracebacks? Should they be considered to be expected, or should we be treating them as bugs and filing tickets/patches for them? thanks. I think all are bugs. Even if you understand some of them and considers them to be logical, you should not see ugly backtraces. You should see nice log lines any system administrator can read and understand clearly. -- Julien Danjou // eNovance http://enovance.com // ✉ julien.dan...@enovance.com ☎ +33 1 49 70 99 81 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Metadata and File Injection
On Thu, Dec 15, 2011 at 8:43 AM, Scott Moser smo...@ubuntu.com wrote: I'm just curious, what are the motivations behind inventing something other than the EC2 Metadata service? It is generally functional, and quite a lot can (and has) built atop this simple service. I should clarify - the idea is that ec2 metadata service is a great starting point - but there are already a handful of additions that should be added. In the existing (ec2) metadata service the urls look like: http://169.254.169.254/(version)/(resource) My hope is that we can add features like: * injected files / personality (or you can think of them as multiple named userdata sections) * updatable metadata params (the key/value params in the openstack api) with CRUD (so you can remove/update from guest) Secondly, There are 2 features that I feel are missing from the metadata service. And I'd hope that these could be accounted for if there is going to be invention done. a.) user-data is a single entity. There are potentially multiple sources that want to provide input to a guest (the end user might want to install some packages at boot, and the cloud infrastructure might want to tell the guest of a local mirror). cloud-init supports multipart-mime in userdata, so that there can be separate pieces inside that single source, but even then, all parties involved have to agree that they do not completely own that resource. The multipart-mime stuff is a great hack but isn't very user friendly as you state. In the openstack api we have server personalities that are similar to userdata except you can have multiple of them and they are named. http://docs.openstack.org/api/openstack-compute/1.1/content/Server_Personality-d1e2543.html If the guest (client) is in charge of pulling the personality data instead of the host injecting it, would this fulfill the usecase? b.) There is no way to disable it. cloud-init supports writing a null-route to the metadata service, which can make it inaccessible to any non-root entity on the system. However, it would be nicer if there was a way to disable it entirely. With that in place, passing credentials into the guest would be easier as once they're consumed they can be removed. The usecase of passing credentials is one reason we want to have the metadata passed via the openstack API mutable via the metadata service http://docs.openstack.org/api/openstack-compute/1.1/content/Server_Metadata-d1e2529.html A hashed password could be provided to the guest via the metadata service in a key bcrypt_password. The guest agent could retrieve the password and then DELETE it: curl -X DELETE http://169.254.169.254/os/meta/bcrypt_password c.) No way to modify contents of the service after instance launch. OK, I said 2 features, and really this one is wishlist. If we had an arbitrary key-value store that was available, the user could interact with the guest using this service. It'd have to be poll-based, but a in-guest hypervisor could watch for creation and deletion of keys. Potentially, the MD might be RW from inside guest, meaning it could even convey information back. I consider this wishlist because really, there are perfectly good solutions for communicating to in-guest agents, theres not a lot of reason to invent a new one. The openstack API already has CRUD of the key/value metadata in the non-metadata (regular) API. http://docs.openstack.org/api/openstack-compute/1.1/content/MetadataSection.html By exposing this through the metadata service I think we get what you are asking for. My wishlist is the able to do long-polling against the metadata service to be alerted of changes from the guest :) Jesse ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Metadata and File Injection
On Thu, 15 Dec 2011, Jesse Andrews wrote: On Thu, Dec 15, 2011 at 8:43 AM, Scott Moser smo...@ubuntu.com wrote: I'm just curious, what are the motivations behind inventing something other than the EC2 Metadata service? It is generally functional, and quite a lot can (and has) built atop this simple service. I should clarify - the idea is that ec2 metadata service is a great starting point - but there are already a handful of additions that should be added. In the existing (ec2) metadata service the urls look like: http://169.254.169.254/(version)/(resource) My hope is that we can add features like: * injected files / personality (or you can think of them as multiple named userdata sections) See below, but I don't see a reason to call something injected files, as opposed to a key of: files/etc/passwd : contents here and letting the guest agents and consumers sort out naming conventions. * updatable metadata params (the key/value params in the openstack api) with CRUD (so you can remove/update from guest) Secondly, There are 2 features that I feel are missing from the metadata service. And I'd hope that these could be accounted for if there is going to be invention done. a.) user-data is a single entity. There are potentially multiple sources that want to provide input to a guest (the end user might want to install some packages at boot, and the cloud infrastructure might want to tell the guest of a local mirror). cloud-init supports multipart-mime in userdata, so that there can be separate pieces inside that single source, but even then, all parties involved have to agree that they do not completely own that resource. The multipart-mime stuff is a great hack but isn't very user friendly as you state. In the openstack api we have server personalities that are similar to userdata except you can have multiple of them and they are named. http://docs.openstack.org/api/openstack-compute/1.1/content/Server_Personality-d1e2543.html If the guest (client) is in charge of pulling the personality data instead of the host injecting it, would this fulfill the usecase? I find specific designed use like this to be unnecessary. EC2 basically says heres a place to store a blob of data, do with it what you want. From that, best practices, good ideas, bad ideas, and even standards will evolve. What does a Server Personality give me that cannot be accomplished accomplished by guest and lauching-entity agreeing on a key-value pair with keyname server-personality. Basically, I'm saying dont bother with special keys/locations in a spec if there is no reason to. b.) There is no way to disable it. cloud-init supports writing a null-route to the metadata service, which can make it inaccessible to any non-root entity on the system. However, it would be nicer if there was a way to disable it entirely. With that in place, passing credentials into the guest would be easier as once they're consumed they can be removed. The usecase of passing credentials is one reason we want to have the metadata passed via the openstack API mutable via the metadata service http://docs.openstack.org/api/openstack-compute/1.1/content/Server_Metadata-d1e2529.html A hashed password could be provided to the guest via the metadata service in a key bcrypt_password. The guest agent could retrieve the password and then DELETE it: curl -X DELETE http://169.254.169.254/os/meta/bcrypt_password Well, theres no reason it can't be plaintext either. but, yes, thats basically what i'm hoping for. c.) No way to modify contents of the service after instance launch. OK, I said 2 features, and really this one is wishlist. If we had an arbitrary key-value store that was available, the user could interact with the guest using this service. It'd have to be poll-based, but a in-guest hypervisor could watch for creation and deletion of keys. Potentially, the MD might be RW from inside guest, meaning it could even convey information back. I consider this wishlist because really, there are perfectly good solutions for communicating to in-guest agents, theres not a lot of reason to invent a new one. The openstack API already has CRUD of the key/value metadata in the non-metadata (regular) API. http://docs.openstack.org/api/openstack-compute/1.1/content/MetadataSection.html By exposing this through the metadata service I think we get what you are asking for. My wishlist is the able to do long-polling against the metadata service to be alerted of changes from the guest :) I dont know that its necessary is the only thing. What is the scenario where polling this metadata service is better than the instance polling another network resource? Or listening to a SQS or zookeeper, or anything. I agree that it seems useful (I brought it up), but I honestly
[Openstack] Help with python-novaclient and keystone : expecting AUTH (HTTP 400).
Hi!
I'm testing a new installation (virtual environment) as following:
(1) I've installed Openstack using the Devstack Script (with some
modification to work here), so, all in one installation.
(2) Then, I started to install Openstack from a clean Ubuntu Oneiric
instalation, but using packages from launchpad, not the Ubuntu packages.
What I'm doing is follow the Devstack Script but trying to install the
components separately (one host with nova-network, other with nova-api,
dashboard, and so on).
Some months ago I've used the euca2ools command line tools. Now we had to
use the nova tool, from python-novaclient and the keystone authentication
system. I was getting some problems with my (2) installation when trying to
run the nova command (for example, to see my nodes, as we got with
euca-describe-availability-zones verbose some time ago. Not working. So, I
tried to run the same tool on the Devstack instalation and it didn't work
too.
user@ubuntu:~$ nova --debug list
connect: (192.168.122.100, 5000)
send: 'POST /v2.0/tokens HTTP/1.1\r\nHost:
192.168.122.100:5000\r\nContent-Length:
90\r\ncontent-type: application/json\r\naccept-encoding: gzip,
deflate\r\nuser-agent: python-novaclient\r\n\r\n{passwordCredentials:
{username: admin, password: Password, tenantId: admin}}'
reply: 'HTTP/1.1 400 Bad Request\r\n'
header: Content-Type: application/json; charset=UTF-8
header: Content-Length: 60
header: Date: Thu, 15 Dec 2011 18:34:20 GMT
Traceback (most recent call last):
File /usr/local/bin/nova, line 9, in module
load_entry_point('python-novaclient==2.6.5', 'console_scripts',
'nova')()
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/shell.py,
line 225, in main
OpenStackComputeShell().main(sys.argv[1:])
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/shell.py,
line 182, in main
self.cs.authenticate()
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/v1_1/client.py,
line 64, in authenticate
self.client.authenticate()
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/client.py,
line 204, in authenticate
auth_url = self._v2_auth(auth_url)
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/client.py,
line 255, in _v2_auth
resp, body = self.request(token_url, POST, body=body)
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/client.py,
line 99, in request
raise exceptions.from_response(resp, body)
novaclient.exceptions.BadRequest: Expecting auth (HTTP 400)
== novaclient.exceptions.BadRequest: Expecting auth (HTTP 400)
I've seen some curl examples that changed some sintax, the way that the URL
is composed, or something like that. With the new examples I could test
keystone and it is working. But I don't know how the nova command makes the
query.
I was basing myself on the Devstack installation but I'm seeing that not
all the components really works. Someone could help me?
My versions are those from Devstack (master branch from the most
components).
Finally, I've understood that the novaclient is now the way we had to
manage our private cloud. On the Devstack installation I can play with all
the things using Horizon Dashboard. But, in the (2) installation I can't
because it doesn't work yet and neither the nova command.
I appreciate any help!
Thanks a bunch!
:)
--
- MSc. Correa, J.L.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
On Dec 15, 2011, at 10:54 AM, Johannes Erdfelt wrote: What kinds of things? I'm just trying to better understand what is object-oriented about the data returned from a database? What methods would we want to use? JE Any direct access to the data within a record should be encapsulated within the model object. For example, changing an instance's progress should be a method on Instance, not a method buried in vmops. ~ Aaron ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack] Default User name and Password for Horizon i.e., Openstack-dashboard that i have been installed using [DevStack] script
The script asks you for the password ... or you put it in a file called localrc Here is my localrc MYSQL_PASSWORD=nova RABBIT_PASSWORD=nova SERVICE_TOKEN=nova ADMIN_PASSWORD=nova ENABLED_SERVICES=g-api,g-reg,key,n-api,n-cpu,n-net,n-sch,n-vnc,horizon,m ysql,rabbit,openstackx,quantum,q-svc,q-agt Q_PLUGIN=openvswitch debo From: openstack-bounces+dedutta=cisco@lists.launchpad.net [mailto:openstack-bounces+dedutta=cisco@lists.launchpad.net] On Behalf Of sn Sent: Thursday, December 15, 2011 10:28 AM To: Openstack Mailing List Subject: [Openstack] [OpenStack] Default User name and Password for Horizon i.e., Openstack-dashboard that i have been installed using [DevStack] script hi experts... i have just now installed successfully openstack without any errors.with the scripts mention in the devstack.org. but i could not login to my dashboard...if anyone know what are the default password for openstack dashboard that has been installed using devstack thanks -- I am on Twitter. Follow Me @SanjibNarzary http://www.twitter.com/SanjibNarzary ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack] Default User name and Password for Horizon i.e., Openstack-dashboard that i have been installed using [DevStack] script
reading the script.. you either have it in localrc file as *ADMIN_PASSWORD, *or you were prompted during install to type it in. * * On Thu, Dec 15, 2011 at 1:38 PM, sn alaya...@gmail.com wrote: and also the default username of openstack that has been installed using devstack script... On Thu, Dec 15, 2011 at 11:58 PM, sn alaya...@gmail.com wrote: hi experts... i have just now installed successfully openstack without any errors.with the scripts mention in the devstack.org. but i could not login to my dashboard...if anyone know what are the default password for openstack dashboard that has been installed using devstack thanks -- I am on Twitter. Follow Me @SanjibNarzaryhttp://www.twitter.com/SanjibNarzary -- I am on Twitter. Follow Me @SanjibNarzaryhttp://www.twitter.com/SanjibNarzary ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Writes are faster than reads in Swift
Your simulation is of the one case that I said would work: when you have enough RAM to cache the entire image. When you _don't_ have enough RAM, then the images will just immediately be evicted, and the performance with the cache will be worse, not better. This break point is going to be when your cloud has ~20 gold images, which doesn't sound very many to me. Ewan. -Original Message- From: Jay Pipes [mailto:jaypi...@gmail.com] Sent: Thursday, December 15, 2011 8:44 AM To: Ewan Mellor Cc: Michael Barton; openstack@lists.launchpad.net Subject: Re: [Openstack] Writes are faster than reads in Swift On Wed, Dec 14, 2011 at 5:38 PM, Ewan Mellor ewan.mel...@eu.citrix.com wrote: Or static disk image files! Only if you've got enough RAM on the storage worker node to cache the entire disk image. Otherwise it's just going to get evicted straight away. The case where you've got so few, small, disk images that you can cache them all in RAM must be pretty rare. And if it's not rare in your use-case, I reckon giving that RAM to your Glance nodes and doing the caching there would be just as good if not better. There's no reason it couldn't be both. Generally, the Glance API servers are not installed on the same servers as Swift object servers... My point being is that Swift was not designed for (large) static files for readonly workloads, and of course, that is the use case for Glance with a Swift backend (where a single container contains few objects, but they tend to be large). It would be great to be able to control the disk flushing behaviour in Swift to control for these alternate workloads. Certain use cases, such as starting up a lot of servers across lots of Nova compute nodes from a base image stored in Swift (read through Glance or not through Glance, it doesn't matter here...), would benefit from not throwing away the disk blocks that will be read heavily over and over again. Even on my home desktop machine (nowhere near the types of servers I see typically provisioned for Swift), I've got a bunch of buffer space available: jpipes@uberbox:~$ free total used free sharedbuffers cached Mem: 247301527997080 16733072 024883603482836 -/+ buffers/cache:2025884 22704268 Swap: 25149436 0 25149436 We can simulate a situation where a 2GB image file is available for streaming through a Swift object server by creating a 2GB file and cat'ing it to /dev/null. The below timing results show the time taken to stream the image when the image's disk blocks are in the disk cache: jpipes@uberbox:~$ dd if=/dev/zero of=fakeimage bs=1M count=2000 2000+0 records in 2000+0 records out 2097152000 bytes (2.1 GB) copied, 1.25832 s, 1.7 GB/s jpipes@uberbox:~$ free total used free sharedbuffers cached Mem: 247301523821108 20909044 0 97522220208 -/+ buffers/cache:1591148 23139004 Swap: 25149436 0 25149436 jpipes@uberbox:~$ time cat fakeimage /dev/null real0m0.346s user0m0.000s sys 0m0.340s Now, we simulate the dropping of the disk buffer cache: jpipes@uberbox:~$ echo 3 | sudo tee /proc/sys/vm/drop_caches 3 And try streaming the image again: jpipes@uberbox:~$ time cat fakeimage /dev/null real0m8.813s user0m0.012s sys 0m1.360s 0.346s vs. 8.813s is a pretty massive difference and one that I think deserves at least a switch to control the behaviour in Swift. Cheers, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
On Dec 15, 2011, at 8:21 AM, Monsyne Dragon wrote: Actually, what we should be working to is using plain python model objects, and having the sqlalchemy layer use it's mapper functionality (separated from the models) to map db rows to those models. THis will allow any persistence layer to map to the same model objects, and allow us to have actual, object-oriented design in our code. (like real methods, where appropriate) This would suggest the dot notation, although making a python model class support a dict interface for backwards compatability whilst things are refactored is easy to do. For what its worth, I've done extensive work with SQLAlchemy in the past with a variety of products and services now in production, and am active in the SQLAlchemy community (and friends with the creator/maintainer). Monsyne's suggestion here is excellent. SQLAlchemy implements the Data Mapper pattern for a reason. It gives you the ability to define your domain objects in a nice, object-oriented fashion, with appropriate state and behavior, and use attribute-based access for data, rather than assuming a low-level datatype like a dictionary. Then, you can utilize the SQLAlchemy ORM to map these domain objects to a fairly arbitrary database schema for persistence purposes. This gives you all of the power of the relational database under the hood, without its rules and assumptions leaking into your object design. SQLAlchemy, right on its front page of its website, says: SQLAlchemy is most famous for its object-relational mapper (ORM), an optional component that provides the data mapper pattern, where classes can be mapped to the database in open ended, multiple ways - allowing the object model and database schema to develop in a cleanly decoupled way from the beginning. I'm fairly new to the project and community, but I wanted to pipe in and share my experience with SQLAlchemy. If you actually use it as designed and to its fullest, it really offers a very powerful and clean approach to a difficult problem. -- Jonathan LaCour VP, Software Development DreamHost (W) 323.372.5984 (C) 404.784.1081 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
For me, it's not one particular notation versus the other, I'd be happy with either--it's having both. It just needlessly complicates things. Now we're complaining that the ORM we likely aren't using correctly isn't working for us I don't think anyone is complaining that the *ORM* is at fault here. We're just complaining that our abstraction is leaky, and since we went through all of the trouble to have this DB API indirection, we may as well seal it up so we can eventually support other data stores. As it stands now, we have all of the complexity of abstraction without any of the benefits. On Dec 15, 2011, at 12:35 PM, Matt Dietz wrote: I have to confess to being confused here. We deliberately chose sqlalchemy. Then we mapped everything away so it didn't look like the ORM in question when in reality, we partially took some of said ORM's job away from it. Now we're complaining that the ORM we likely aren't using correctly isn't working for us. In short, we chose to use an ORM, and now we're complaining about the O I'm not seeing what taking everything to a dictionary-centric model buys us, and I also don't see anyone actually justifying it. Can we get some actual examples of why one approach is better than the other? On 12/15/11 10:54 AM, Johannes Erdfelt johan...@erdfelt.com wrote: On Thu, Dec 15, 2011, Kevin L. Mitchell kevin.mitch...@rackspace.com wrote: 2. However, I violently disagree with the idea that the DB layer must return dicts. It does not, even if you start talking about allowing use of other kinds of databases. We can, and should, wrap these things in objects, upon which we can call methods that do things‹i.e., we should, you know, actually use object-oriented programming. What kinds of things? I'm not against returning back a standardized object that provides __getattr__ so we don't have to use dict notation. Any database backend can do something similar easily. I'm just trying to better understand what is object-oriented about the data returned from a database? What methods would we want to use? JE ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack] Default User name and Password for Horizon i.e., Openstack-dashboard that i have been installed using [DevStack] script
This should have been asked to you when running the stack.sh script and this should be stored in localrc file. Chmouel. On Thu, Dec 15, 2011 at 6:28 PM, sn alaya...@gmail.com wrote: hi experts... i have just now installed successfully openstack without any errors.with the scripts mention in the devstack.org. but i could not login to my dashboard...if anyone know what are the default password for openstack dashboard that has been installed using devstack thanks -- I am on Twitter. Follow Me @SanjibNarzary ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
Johannes Erdfelt johan...@erdfelt.com said: On Thu, Dec 15, 2011, Kevin L. Mitchell kevin.mitch...@rackspace.com wrote: 2. However, I violently disagree with the idea that the DB layer must return dicts. It does not, even if you start talking about allowing use of other kinds of databases. We can, and should, wrap these things in objects, upon which we can call methods that do things—i.e., we should, you know, actually use object-oriented programming. What kinds of things? I'm not against returning back a standardized object that provides __getattr__ so we don't have to use dict notation. Any database backend can do something similar easily. I'm just trying to better understand what is object-oriented about the data returned from a database? What methods would we want to use? JE I think the OO approach gets interesting when we start thinking about instances as objects rather than as data records passed to monolithic manager interfaces. For example, instead of the following snippet: instance = self.compute_api.get(uuid) self.compute_api.resize(instance, new_flavor) I might do something more like instance = self.compute_api.get(uuid) instance.resize(instance, new_flavor) This doesn't seem like a big deal probably--it isn't. But its powerful in that it has now separated out the interface for finding instances from the interface for interacting with a given instance. No longer do I need a single object that knows how to find _and_ how to talk to _all_ the instances in the system. That means we can be more flexible with less complicated code paths. Now, I could also rewrite it as instance = self.instance_registry.get(uuid) instance.resize(instance, new_flavor) And in this case, I'm now skipping the service-level middleman (compute_api) entirely. It also creates the possibility of dividing up the db api. No longer would we need a 9000+ function db.api--so I can stub in or fake out useful subsections of the db api with simple dependency injection rather than an arbitrarily long list of monkey-patches. We could also separate out the sqlalchemy/models.py as joins permit. I'm not trying to promote anything here as a panacea, however. OO approaches like this just seem better to me because I think the big interfaces nova has right now are code smell. But I'm not sure that this flexibility is something that Nova really needs in the near future, so refactoring to this approach could be a wasted effort or harmful if done only partially. For now dict-based consistency seems okay, although I'd also be in favor of turning off lazy-loading for good and fixing whatever breaks. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
Hm. I've just been going off of this doc string in db/api.py: Functions in this module are imported into the nova.db namespace. Call these functions from nova.db namespace, not the nova.db.api namespace. All functions in this module return objects that implement a dictionary-like interface. Currently, many of these objects are sqlalchemy objects that implement a dictionary interface. However, a future goal is to have all ofthese objects be simple dictionaries. I didn't put any thought into it, but I can see our own objects being better. I would just want to make sure they are easily serializable so they can be sent across AMQP. I admit that I haven't paid attention to any of the DB discussions. I guess that's why I brought up my question in the first place. ;) - Chris On Dec 15, 2011, at 10:52 AM, Aaron Lee wrote: On Dec 15, 2011, at 10:54 AM, Johannes Erdfelt wrote: What kinds of things? I'm just trying to better understand what is object-oriented about the data returned from a database? What methods would we want to use? JE Any direct access to the data within a record should be encapsulated within the model object. For example, changing an instance's progress should be a method on Instance, not a method buried in vmops. ~ Aaron ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Expectations for tracebacks
I think all are bugs. Even if you understand some of them and considers them to be logical, you should not see ugly backtraces. You should see nice log lines any system administrator can read and understand clearly. I agree. There are some other practical reasons for it too: - Exceptions change between releases (line numbers, file names, call stack, etc), so it can be hard to tell sometimes if you're looking at the same thing or different. - Exceptions take multiple lines and may actually contain the reason only in the last one. That means `grep 'CRIT|ERROR' ...` will not show the real message. - It might not be a critical situation actually. Most of exceptions will probably be, but there are also possible cases of couldn't remove X, because it's not there anymore which might be warnings. I'd be happy to see exceptions disappear completely from logs, unless something really unexpected happens - then the stacktrace and message help a lot with debugging. Regards, Stanisław Pitucha Cloud Services Hewlett Packard ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
On Dec 15, 2011, at 1:27 PM, Devin Carlen wrote: Matt, that answer is simple: so we can use things other than sqlalchemy. Except that we don't need to do that to use other things that sqlalchemy. We can have sqlalchemy map to plain python objects, and use those. And any other persistence layer can then use the same model objects. As it currently is, the orm gets dicts back from the DB API, sqlalchemy maps them to objects, then we take pains to pretend they are dicts identical to what the DBAPI returned. On Dec 15, 2011, at 10:35 AM, Matt Dietz wrote: I have to confess to being confused here. We deliberately chose sqlalchemy. Then we mapped everything away so it didn't look like the ORM in question when in reality, we partially took some of said ORM's job away from it. Now we're complaining that the ORM we likely aren't using correctly isn't working for us. In short, we chose to use an ORM, and now we're complaining about the O I'm not seeing what taking everything to a dictionary-centric model buys us, and I also don't see anyone actually justifying it. Can we get some actual examples of why one approach is better than the other? On 12/15/11 10:54 AM, Johannes Erdfelt johan...@erdfelt.com wrote: On Thu, Dec 15, 2011, Kevin L. Mitchell kevin.mitch...@rackspace.com wrote: 2. However, I violently disagree with the idea that the DB layer must return dicts. It does not, even if you start talking about allowing use of other kinds of databases. We can, and should, wrap these things in objects, upon which we can call methods that do things‹i.e., we should, you know, actually use object-oriented programming. What kinds of things? I'm not against returning back a standardized object that provides __getattr__ so we don't have to use dict notation. Any database backend can do something similar easily. I'm just trying to better understand what is object-oriented about the data returned from a database? What methods would we want to use? JE ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Monsyne M. Dragon OpenStack/Nova cell 210-441-0965 work x 5014190 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Writes are faster than reads in Swift
On Thu, Dec 15, 2011 at 2:55 PM, Ewan Mellor ewan.mel...@eu.citrix.com wrote: Your simulation is of the one case that I said would work: when you have enough RAM to cache the entire image. When you _don't_ have enough RAM, then the images will just immediately be evicted, and the performance with the cache will be worse, not better. This break point is going to be when your cloud has ~20 gold images, which doesn't sound very many to me. I'll go ahead and benchmark it on some real world systems and get some hard numbers so we can discuss facts over opinions. If it turns out that turning off the default cache flushing behaviour leads to an increase in performance and throughput for a very common use case (as I believe I have described in this thread...) I hope you won't mind having a switch to control the cache flush behaviour... Cheers, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] novaclient + keystone (HTTP 404, The resource could not be found.)
On Thu, Dec 15, 2011 at 2:29 PM, Dong-In David Kang dk...@isi.edu wrote:
I'm trying to make novaclient work with keystone.
It looks like authentication is working, but actual interaction between
novaclient and nova does not work.
Here is what I get (with added debugging messages I added in novaclient)
I'll appreciate any help.
David.
$ nova list
initial auth_url: http://10.0.10.1:5000/v2.0/
POST with body = {'auth': {'tenantName': 'demo', 'passwordCredentials':
{'username': 'demo', 'password': 'guest'}}}
request: args = ('http://10.0.10.1:5000/v2.0/tokens', 'POST')
request: kwargs = {'body': '{auth: {tenantName: demo,
passwordCredentials: {username: demo, password: guest}}}',
'headers': {'Content-Type': 'application/json', 'User-Agent':
'python-novaclient'}}
request: resp = {'date': 'Thu, 15 Dec 2011 19:20:10 GMT', 'status': '200',
'content-length': '993', 'content-type': 'application/json; charset=UTF-8'}
request: body = {access: {token: {expires:
2011-12-16T11:30:09.098995, id: 6ad85291-8d26-4bd5-a667-525322fd66a8,
tenant: {id: 2, name: demo}}, serviceCatalog: [{endpoints:
[{region: RegionOne, internalURL: http://10.0.10.1:8774/v1.1/;,
publicURL: http://10.0.10.1:8774/v1.1/}], type: compute, name:
nova}, {endpoints: [{region: RegionOne, internalURL:
http://10.0.11.1:9292/v1.1/;, publicURL: http://10.0.11.1:9292/v1.1/},
{region: RegionOne, internalURL: http://10.0.1.3:9292/v1.1/;,
publicURL: http://10.0.1.3:9292/v1.1/}, {region: RegionOne,
internalURL: http://10.0.11.1:9292/v1/;, publicURL:
http://10.0.11.1:9292/v1/}], type: image, name: glance},
{endpoints: [{region: RegionOne, internalURL:
http://10.0.10.1:5000/v2.0;, publicURL: http://10.0.10.1:5000/v2.0}],
type: identity, name: keystone}], user: {id: 2, roles:
[{tenantId: 2, id: 4, name: Member}], name: demo}}}
resp of POST = {'date': 'Thu, 15 Dec 2011 19:20:10 GMT', 'status': '200',
'content-length': '993', 'content-type': 'application/json; charset=UTF-8'}
service_catalog = module 'novaclient.service_catalog' from
'/home/dkang/venv/lib/python2.6/site-packages/novaclient/service_catalog.pyc'
auth_token = 6ad85291-8d26-4bd5-a667-525322fd66a8
management_url = http://10.0.10.1:8774/v1.1/
v2_auth is done
request: args =
(u'http://10.0.10.1:8774/v1.1//servers/detail?fresh=1323976810.06', 'GET')
Unless I'm mistaken, shouldn't the above URI include the TenantId
between /v1.1/ and /servers?
-jay
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Metadata and File Injection
On Dec 15, 2011, at 10:37 AM, Scott Moser wrote: On Thu, 15 Dec 2011, Jesse Andrews wrote: On Thu, Dec 15, 2011 at 8:43 AM, Scott Moser smo...@ubuntu.com wrote: I'm just curious, what are the motivations behind inventing something other than the EC2 Metadata service? It is generally functional, and quite a lot can (and has) built atop this simple service. I should clarify - the idea is that ec2 metadata service is a great starting point - but there are already a handful of additions that should be added. In the existing (ec2) metadata service the urls look like: http://169.254.169.254/(version)/(resource) My hope is that we can add features like: * injected files / personality (or you can think of them as multiple named userdata sections) See below, but I don't see a reason to call something injected files, as opposed to a key of: files/etc/passwd : contents here and letting the guest agents and consumers sort out naming conventions. * updatable metadata params (the key/value params in the openstack api) with CRUD (so you can remove/update from guest) Secondly, There are 2 features that I feel are missing from the metadata service. And I'd hope that these could be accounted for if there is going to be invention done. a.) user-data is a single entity. There are potentially multiple sources that want to provide input to a guest (the end user might want to install some packages at boot, and the cloud infrastructure might want to tell the guest of a local mirror). cloud-init supports multipart-mime in userdata, so that there can be separate pieces inside that single source, but even then, all parties involved have to agree that they do not completely own that resource. The multipart-mime stuff is a great hack but isn't very user friendly as you state. In the openstack api we have server personalities that are similar to userdata except you can have multiple of them and they are named. http://docs.openstack.org/api/openstack-compute/1.1/content/Server_Personality-d1e2543.html If the guest (client) is in charge of pulling the personality data instead of the host injecting it, would this fulfill the usecase? I find specific designed use like this to be unnecessary. EC2 basically says heres a place to store a blob of data, do with it what you want. From that, best practices, good ideas, bad ideas, and even standards will evolve. What does a Server Personality give me that cannot be accomplished accomplished by guest and lauching-entity agreeing on a key-value pair with keyname server-personality. I think you are stating the same thing here. Basically repurposing the existing personality code to be key, value instead of filename, data If the guest agent wants to interpret keys as filenames it can. we could repurpose metadata as well, but it multiple key/value with potentially large values seems ideal. Basically, I'm saying dont bother with special keys/locations in a spec if there is no reason to. b.) There is no way to disable it. cloud-init supports writing a null-route to the metadata service, which can make it inaccessible to any non-root entity on the system. However, it would be nicer if there was a way to disable it entirely. With that in place, passing credentials into the guest would be easier as once they're consumed they can be removed. The usecase of passing credentials is one reason we want to have the metadata passed via the openstack API mutable via the metadata service http://docs.openstack.org/api/openstack-compute/1.1/content/Server_Metadata-d1e2529.html A hashed password could be provided to the guest via the metadata service in a key bcrypt_password. The guest agent could retrieve the password and then DELETE it: curl -X DELETE http://169.254.169.254/os/meta/bcrypt_password Well, theres no reason it can't be plaintext either. but, yes, thats basically what i'm hoping for. c.) No way to modify contents of the service after instance launch. OK, I said 2 features, and really this one is wishlist. If we had an arbitrary key-value store that was available, the user could interact with the guest using this service. It'd have to be poll-based, but a in-guest hypervisor could watch for creation and deletion of keys. Potentially, the MD might be RW from inside guest, meaning it could even convey information back. I consider this wishlist because really, there are perfectly good solutions for communicating to in-guest agents, theres not a lot of reason to invent a new one. The openstack API already has CRUD of the key/value metadata in the non-metadata (regular) API. http://docs.openstack.org/api/openstack-compute/1.1/content/MetadataSection.html By exposing this through the metadata service I think we get what you are asking for. My wishlist is the able to do
Re: [Openstack] Help with python-novaclient and keystone : expecting AUTH (HTTP 400).
Hmm, looks like you are using an old version of python-novaclient. You might
try grabbing from source. The packages from launchpad are not supported in any
way. We try to provide relatively current packages, but we don't really have
the resources to maintain and support them at this point.
Vish
On Dec 15, 2011, at 10:46 AM, Jorge Luiz Correa wrote:
Hi!
I'm testing a new installation (virtual environment) as following:
(1) I've installed Openstack using the Devstack Script (with some
modification to work here), so, all in one installation.
(2) Then, I started to install Openstack from a clean Ubuntu Oneiric
instalation, but using packages from launchpad, not the Ubuntu packages. What
I'm doing is follow the Devstack Script but trying to install the components
separately (one host with nova-network, other with nova-api, dashboard, and
so on).
Some months ago I've used the euca2ools command line tools. Now we had to use
the nova tool, from python-novaclient and the keystone authentication system.
I was getting some problems with my (2) installation when trying to run the
nova command (for example, to see my nodes, as we got with
euca-describe-availability-zones verbose some time ago. Not working. So, I
tried to run the same tool on the Devstack instalation and it didn't work
too.
user@ubuntu:~$ nova --debug list
connect: (192.168.122.100, 5000)
send: 'POST /v2.0/tokens HTTP/1.1\r\nHost:
192.168.122.100:5000\r\nContent-Length: 90\r\ncontent-type:
application/json\r\naccept-encoding: gzip, deflate\r\nuser-agent:
python-novaclient\r\n\r\n{passwordCredentials: {username: admin,
password: Password, tenantId: admin}}'
reply: 'HTTP/1.1 400 Bad Request\r\n'
header: Content-Type: application/json; charset=UTF-8
header: Content-Length: 60
header: Date: Thu, 15 Dec 2011 18:34:20 GMT
Traceback (most recent call last):
File /usr/local/bin/nova, line 9, in module
load_entry_point('python-novaclient==2.6.5', 'console_scripts', 'nova')()
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/shell.py,
line 225, in main
OpenStackComputeShell().main(sys.argv[1:])
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/shell.py,
line 182, in main
self.cs.authenticate()
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/v1_1/client.py,
line 64, in authenticate
self.client.authenticate()
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/client.py,
line 204, in authenticate
auth_url = self._v2_auth(auth_url)
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/client.py,
line 255, in _v2_auth
resp, body = self.request(token_url, POST, body=body)
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/client.py,
line 99, in request
raise exceptions.from_response(resp, body)
novaclient.exceptions.BadRequest: Expecting auth (HTTP 400)
==
Re: [Openstack] Help with python-novaclient and keystone : expecting AUTH (HTTP 400).
Hum, ok, I'll try some versions tomorrow and post the results!
Thanks Vish!
:)
On Dec 15, 2011, at 8:53 PM, Vishvananda Ishaya wrote:
Hmm, looks like you are using an old version of python-novaclient. You might
try grabbing from source. The packages from launchpad are not supported in
any way. We try to provide relatively current packages, but we don't really
have the resources to maintain and support them at this point.
Vish
On Dec 15, 2011, at 10:46 AM, Jorge Luiz Correa wrote:
Hi!
I'm testing a new installation (virtual environment) as following:
(1) I've installed Openstack using the Devstack Script (with some
modification to work here), so, all in one installation.
(2) Then, I started to install Openstack from a clean Ubuntu Oneiric
instalation, but using packages from launchpad, not the Ubuntu packages.
What I'm doing is follow the Devstack Script but trying to install the
components separately (one host with nova-network, other with nova-api,
dashboard, and so on).
Some months ago I've used the euca2ools command line tools. Now we had to
use the nova tool, from python-novaclient and the keystone authentication
system. I was getting some problems with my (2) installation when trying to
run the nova command (for example, to see my nodes, as we got with
euca-describe-availability-zones verbose some time ago. Not working. So, I
tried to run the same tool on the Devstack instalation and it didn't work
too.
user@ubuntu:~$ nova --debug list
connect: (192.168.122.100, 5000)
send: 'POST /v2.0/tokens HTTP/1.1\r\nHost:
192.168.122.100:5000\r\nContent-Length: 90\r\ncontent-type:
application/json\r\naccept-encoding: gzip, deflate\r\nuser-agent:
python-novaclient\r\n\r\n{passwordCredentials: {username: admin,
password: Password, tenantId: admin}}'
reply: 'HTTP/1.1 400 Bad Request\r\n'
header: Content-Type: application/json; charset=UTF-8
header: Content-Length: 60
header: Date: Thu, 15 Dec 2011 18:34:20 GMT
Traceback (most recent call last):
File /usr/local/bin/nova, line 9, in module
load_entry_point('python-novaclient==2.6.5', 'console_scripts',
'nova')()
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/shell.py,
line 225, in main
OpenStackComputeShell().main(sys.argv[1:])
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/shell.py,
line 182, in main
self.cs.authenticate()
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/v1_1/client.py,
line 64, in authenticate
self.client.authenticate()
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/client.py,
line 204, in authenticate
auth_url = self._v2_auth(auth_url)
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/client.py,
line 255, in _v2_auth
resp, body = self.request(token_url, POST, body=body)
File
/usr/local/lib/python2.7/dist-packages/python_novaclient-2.6.5-py2.7.egg/novaclient/client.py,
line 99, in request
Re: [Openstack] novaclient + keystone (HTTP 404, The resource could not be found.)
On Thu, Dec 15, 2011 at 2:29 PM, Dong-In David Kang dk...@isi.edu
wrote:
I'm trying to make novaclient work with keystone.
It looks like authentication is working, but actual interaction
between novaclient and nova does not work.
Here is what I get (with added debugging messages I added in
novaclient)
I'll appreciate any help.
David.
$ nova list
initial auth_url: http://10.0.10.1:5000/v2.0/
POST with body = {'auth': {'tenantName': 'demo',
'passwordCredentials': {'username': 'demo', 'password': 'guest'}}}
request: args = ('http://10.0.10.1:5000/v2.0/tokens', 'POST')
request: kwargs = {'body': '{auth: {tenantName: demo,
passwordCredentials: {username: demo, password:
guest}}}', 'headers': {'Content-Type': 'application/json',
'User-Agent': 'python-novaclient'}}
request: resp = {'date': 'Thu, 15 Dec 2011 19:20:10 GMT', 'status':
'200', 'content-length': '993', 'content-type': 'application/json;
charset=UTF-8'}
request: body = {access: {token: {expires:
2011-12-16T11:30:09.098995, id:
6ad85291-8d26-4bd5-a667-525322fd66a8, tenant: {id: 2,
name: demo}}, serviceCatalog: [{endpoints: [{region:
RegionOne, internalURL: http://10.0.10.1:8774/v1.1/;,
publicURL: http://10.0.10.1:8774/v1.1/}], type: compute,
name: nova}, {endpoints: [{region: RegionOne,
internalURL: http://10.0.11.1:9292/v1.1/;, publicURL:
http://10.0.11.1:9292/v1.1/}, {region: RegionOne,
internalURL: http://10.0.1.3:9292/v1.1/;, publicURL:
http://10.0.1.3:9292/v1.1/}, {region: RegionOne,
internalURL: http://10.0.11.1:9292/v1/;, publicURL:
http://10.0.11.1:9292/v1/}], type: image, name: glance},
{endpoints: [{region: RegionOne, internalURL:
http://10.0.10.1:5000/v2.0;, publicURL:
http://10.0.10.1:5000/v2.0}], type: identity, name:
keystone}], user: {id: 2, roles: [{tenantId: 2,
id: 4, name: Member}], name: demo}}}
resp of POST = {'date': 'Thu, 15 Dec 2011 19:20:10 GMT', 'status':
'200', 'content-length': '993', 'content-type': 'application/json;
charset=UTF-8'}
service_catalog = module 'novaclient.service_catalog' from
'/home/dkang/venv/lib/python2.6/site-packages/novaclient/service_catalog.pyc'
auth_token = 6ad85291-8d26-4bd5-a667-525322fd66a8
management_url = http://10.0.10.1:8774/v1.1/
v2_auth is done
request: args =
(u'http://10.0.10.1:8774/v1.1//servers/detail?fresh=1323976810.06',
'GET')
Unless I'm mistaken, shouldn't the above URI include the TenantId
between /v1.1/ and /servers?
-jay
Thank you for the catch.
I've modified the keystone db, and rerun $nova list.
Now I have HTTP 500 error.
$ nova list
REQ: curl -i http://10.0.10.1:5000/v2.0/tokens -X POST -H Content-Type:
application/json -H User-Agent: python-novaclient
REQ BODY: {auth: {tenantName: demo, passwordCredentials: {username:
demo, password: guest}}}
RESP:{'date': 'Thu, 15 Dec 2011 22:54:31 GMT', 'status': '200',
'content-length': '1229', 'content-type': 'application/json; charset=UTF-8'}
{access: {token: {expires: 2011-12-16T11:30:09.098995, id:
6ad85291-8d26-4bd5-a667-525322fd6
6a8, tenant: {id: 2, name: demo}}, serviceCatalog: [{endpoints:
[{region: RegionOne, internalURL: http://10.0.10.1:8774/v1.1/1;,
publicURL: http://10.0.10.1:8774/v1.1/1}, {region: RegionOne,
internalURL: http://
10.0.10.1:8774/v1.1/2, publicURL: http://10.0.10.1:8774/v1.1/2},
{region: RegionOne, internalURL: http://10.0.10.1:8774/v1.1/3;,
publicURL: http://10.0.10.1:8774/v1.1/3}], type: compute, name:
nova}, {endpoints: [{reg
ion: RegionOne, internalURL: http://10.0.11.1:9292/v1.1/;, publicURL:
http://10.0.11.1:9292/v1.1/}, {region: RegionOne, internalURL:
http://10.0.1.3:9292/v1.1/;, publicURL: http://10.0.1.3:9292/v1.1/},
{region: RegionOne
, internalURL: http://10.0.11.1:9292/v1/;, publicURL:
http://10.0.11.1:9292/v1/}], type: image, name: glance},
{endpoints: [{region: RegionOne, internalURL:
http://10.0.10.1:5000/v2.0;, publicURL: http://10.0.10.1:50
00/v2.0}], type: identity, name: keystone}], user: {id: 2,
roles: [{tenantId: 2, id: 4, name: Member}], name: demo}}}
REQ: curl -i http://10.0.10.1:8774/v1.1/1/servers/detail?fresh=1323989671.41 -X
GET -H X-Auth-Project-Id: demo -H User-Agent: python-novaclient -H
X-Auth-Token: 6ad85291-8d26-4bd5-a667-525322fd66a8
REQ: curl -i http://10.0.10.1:8774/v1.1/1/servers/detail?fresh=1323989671.41 -X
GET -H X-Auth-Project-Id: demo -H User-Agent: python-novaclient -H
X-Auth-Token: 6ad85291-8d26-4bd5-a667-525322fd66a8
RESP:{'date': 'Thu, 15 Dec 2011 22:54:31 GMT', 'status': '500',
'content-length': '128', 'content-type': 'application/json; charset=UTF-8'}
{computeFault: {message: The server has either erred or is incapable of
performing the requested op
eration., code: 500}}
RESP:{'date': 'Thu, 15 Dec 2011 22:54:31 GMT', 'status': '500',
'content-length': '128', 'content-type': 'application/json; charset=UTF-8'}
{computeFault: {message: The server has either erred or is incapable of
performing the requested op
Re: [Openstack] using objects returned from DB layer
On Thu, Dec 15, 2011, Mark Washenberger mark.washenber...@rackspace.com wrote: 1. regular objects (which is what I thought Kevin was talking about. . . maybe?) I think this has been poorly defined so far. I've seen some quick proposals that include moving instance actions to an Instance class, etc. It's hard to make a good model for this. I think everyone would like to see a more complete proposal that includes some examples. I'm not saying you need to do it, but this is something that doesn't have an obvious design and implementation. It would be easier to understand and discuss with some real meat behind it. JE ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Windows2008 VM Shut off automatically
I use nova on KVM. I create Windows 2008 R2 image and upload to glance. $ glance index ID Name Disk Format Container Format Size -- -- 9Windows2008-X86_64 raw ovf 10737418240 When I boot the instance, I can ping and connect to remote desktop However, The problem is that the instance is shut down automatically after about a hour since it is launched. Nova doesn't know the state of instance because the KVM shut off the instance. nova@nova5:~$ nova list | grep Console_Server | bfff2565-7fcc-46b6-a97f-62018c33517b | Console_Server| ACTIVE | sds=10.101.0.227, 182.194.2.240 | Here is a qemu log $ vi /var/log/libvirt/qemu/instance-0160.log 1 2011-12-15 13:53:24.304: starting up 2 LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-0.14 -enable-kvm -m 10240 -smp 4,sockets=4,cores=1,threads=1 -name instance-0160 -uuid 95be861b-b3ae-6afc-d5f3-a7a773cb8da4 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-0160.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -boot c -drive file=/dev/disk/by-path/ip-10.245.34.2:3260-iscsi-iqn.2003-10.com.lefthandnetworks:c2-z1-swn:1112:volume-02a 2-lun-0,if=none,id=drive-virtio-disk0,boot=on,format=raw -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 -drive file=/dev/disk/by-path/ip-10.245.34.2:3 260-iscsi-iqn.2003-10.com.lefthandnetworks:c2-z1-swn:1114:volume-02a3-lun-0,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk 1,id=virtio-disk1 -netdev tap,fd=18,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=02:16:3e:0b:a4:ac,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/spcs/instances/instanc e-0160/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -usb -vnc 0.0.0.0:1 -k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 3 char device redirected to /dev/pts/2 4 kvm: -device rtl8139,netdev=hostnet0,id=net0,mac=02:16:3e:0b:a4:ac,bus=pci.0,addr=0x3: pci_add_option_rom: *failed to find romfile pxe-rtl8139.bin* 5 2011-12-15 14:53:55.335: shutting down At first, I tought the problem is cause by pxe-rtl8139.bin failed problem So I googled and I solved the problem. Her is the URL I found. http://ubuntuforums.org/archive/index.php/t-1814487.html http://hugokuo-hugo.blogspot.com/2011/07/build-winxp-image-into-nova.html Howerver, I coundn't solve the shut-off problem. $ vi /var/log/libvirt/qemu/instance-0161.log 1 2011-12-16 07:35:14.328: starting up 2 LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-0.14 -enable-kvm -m 10240 -smp 4,sockets=4,cores=1,threads=1 -name instance-0161 -uuid aedfe3a5-0925-aca8-38cb-f5ed9d3c0f42 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-0161.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -boot c -drive file=/dev/disk/by-path/ip-10.245.34.1:3260-iscsi-iqn.2003-10.com.lefthandnetworks:c1-z1-swn:1905:volume-02a 4-lun-0,if=none,id=drive-virtio-disk0,boot=on,format=raw -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 -drive file=/dev/disk/by-path/ip-10.245.34.1:3 260-iscsi-iqn.2003-10.com.lefthandnetworks:c1-z1-swn:1907:volume-02a5-lun-0,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk 1,id=virtio-disk1 -netdev tap,fd=17,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=02:16:3e:25:1c:d2,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/spcs/instances/instanc e-0161/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -usb -vnc 0.0.0.0:0 -k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 3 char device redirected to /dev/pts/1 4 2011-12-16 08:35:41.770: shutting down 5 2011-12-16 08:48:57.058: starting up 6 LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-0.14 -enable-kvm -m 10240 -smp 4,sockets=4,cores=1,threads=1 -name instance-0161 -uuid aedfe3a5-0925-aca8-38cb-f5ed9d3c0f42 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-0161.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -boot c -drive
[Openstack] Swift slow write performance
Hi, I'm testing swift 1.4.4 setup with 4 nodes/zones on RHEL 5.7. I ran into the problem of slow writes. Using swift-bench we generated load, writing hundreds of 4K files. Results: - Writes - ~3 PUTs/sec (very slow) - Reads - ~25 GETs/sec (ok) It's clear that writes are not limited by I/O. To prove that, I increased file size to 8K, 32K and 64K. In all cases, even with 64K, I had same result - 3 PUTs/sec. In the logs of the object servers I can see that each PUT operation actually took around 0.3 sec which is inline with 3 PUTs/ sec reported by swift-bench. Is that expected performance for such small cluster? Can I break down those 0.3 seconds spent for PUT operation to see where's bottleneck? Any advice for troubleshooting this is welcome. Regards, Rustam. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
+1 On 12/15/11 3:19 PM, Johannes Erdfelt johan...@erdfelt.com wrote: On Thu, Dec 15, 2011, Mark Washenberger mark.washenber...@rackspace.com wrote: 1. regular objects (which is what I thought Kevin was talking about. . . maybe?) I think this has been poorly defined so far. I've seen some quick proposals that include moving instance actions to an Instance class, etc. It's hard to make a good model for this. I think everyone would like to see a more complete proposal that includes some examples. I'm not saying you need to do it, but this is something that doesn't have an obvious design and implementation. It would be easier to understand and discuss with some real meat behind it. JE ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] swift acl's
In my case, I want to set access permissions to all the users except for this user tester3 and according to the documentation - is to be prefixed to deny access to the user. But even after setting the - for the user tester3, read access is granted to it. Can anyone who has used it provide some inputs. On Fri, Dec 16, 2011 at 4:29 AM, pf shineyear shin...@gmail.com wrote: if u set .r:* all user can GET, so try to not set .r:* just set .r:-test:tester3 is enough On Fri, Dec 16, 2011 at 12:19 AM, Mohammed Junaid mohdjunaid...@gmail.com wrote: Hi All, I am testing acl support in swift-1.4.5. According to the document http://swift.openstack.org/misc.html#module-swift.common.middleware.acl the syntax to allow all non-admin users read access to the container except for one is as following. Executing the curl following curl command from an admin user. curl -v -X POST -H 'X-Auth-Token: AUTH_tkea3fdbf40e5b40708a51db0377be3f47' http://127.0.0.1:8080/v1/AUTH_test/cont -H 'X-Container-Read: .r:*,.rlistings,.r:-test:tester3' curl -v -X HEAD -H 'X-Auth-Token: AUTH_tkea3fdbf40e5b40708a51db0377be3f47' http://127.0.0.1:8080/v1/AUTH_test/cont *About to connect() to 127.0.0.1 port 8080 (#0) * Trying 127.0.0.1... connected * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0) HEAD /v1/AUTH_test/cont HTTP/1.1 User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/ 3.12.9.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2 Host: 127.0.0.1:8080\ Accept: X-Auth-Token: AUTH_tkea3fdbf40e5b40708a51db0377be3f47 HTTP/1.1 204 No Content X-Container-Object-Count: 10 X-Container-Read: .r:*,.rlistings,.r:-test:tester3 X-Container-Bytes-Used: 1 Accept-Ranges: bytes Content-Length: 0 Date: Thu, 15 Dec 2011 18:38:25 GMT * Connection #0 to host 127.0.0.1 left intact * Closing connection #0 -- But GET operations still succeed for the user tester3. What else is required to make the swift-server deny this user from doing GET operations. Thanks in advance. regards, Junaid ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- regards, Junaid ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] swift acl's
hello, please Unsubscribe my account from open stack... On Fri, Dec 16, 2011 at 8:02 AM, Mohammed Junaid mohdjunaid...@gmail.comwrote: In my case, I want to set access permissions to all the users except for this user tester3 and according to the documentation - is to be prefixed to deny access to the user. But even after setting the - for the user tester3, read access is granted to it. Can anyone who has used it provide some inputs. On Fri, Dec 16, 2011 at 4:29 AM, pf shineyear shin...@gmail.com wrote: if u set .r:* all user can GET, so try to not set .r:* just set .r:-test:tester3 is enough On Fri, Dec 16, 2011 at 12:19 AM, Mohammed Junaid mohdjunaid...@gmail.com wrote: Hi All, I am testing acl support in swift-1.4.5. According to the document http://swift.openstack.org/misc.html#module-swift.common.middleware.acl the syntax to allow all non-admin users read access to the container except for one is as following. Executing the curl following curl command from an admin user. curl -v -X POST -H 'X-Auth-Token: AUTH_tkea3fdbf40e5b40708a51db0377be3f47' http://127.0.0.1:8080/v1/AUTH_test/cont -H 'X-Container-Read: .r:*,.rlistings,.r:-test:tester3' curl -v -X HEAD -H 'X-Auth-Token: AUTH_tkea3fdbf40e5b40708a51db0377be3f47' http://127.0.0.1:8080/v1/AUTH_test/cont *About to connect() to 127.0.0.1 port 8080 (#0) * Trying 127.0.0.1... connected * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0) HEAD /v1/AUTH_test/cont HTTP/1.1 User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/ 3.12.9.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2 Host: 127.0.0.1:8080\ Accept: X-Auth-Token: AUTH_tkea3fdbf40e5b40708a51db0377be3f47 HTTP/1.1 204 No Content X-Container-Object-Count: 10 X-Container-Read: .r:*,.rlistings,.r:-test:tester3 X-Container-Bytes-Used: 1 Accept-Ranges: bytes Content-Length: 0 Date: Thu, 15 Dec 2011 18:38:25 GMT * Connection #0 to host 127.0.0.1 left intact * Closing connection #0 -- But GET operations still succeed for the user tester3. What else is required to make the swift-server deny this user from doing GET operations. Thanks in advance. regards, Junaid ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- regards, Junaid ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack] Default User name and Password for Horizon i.e., Openstack-dashboard that i have been installed using [DevStack] script
thanks you everyone the default user name was admin and yes i have configured the password as password during installation. thanks for helping and replying all [DevStack] script is the best way to test Openstack. #devstack #openstack #ubuntu ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Newby question
Hi, For my company, I'd like to set-up a nova-compute All-In-One node, to replace a eucalyptus cluster which is showing age. Unfortunately I'm not well versed in networking, and have trouble grasping the difference between the multiple network managers, but I think I would need a Flat mode in my case. I have a simple question: could someone please provide a functional euca.conf, that is using only one NIC, and the relative /etc/network/interfaces, for Ubuntu 11.10/Diablo? Help much appreciated! alex p.s.: as a side node, I tried everywhere the web for an explanation of how 'nova-manage network' relates to the flags passed inside nova.conf. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
