date:20120815

[Openstack] Implement network creation in compute API

2012-08-15 Thread Alessio Ababilov

Hi!

I have implemented API for network creation a month ago
(https://review.openstack.org/#/c/9847/). The patch is still waiting
for approval.
Could somebody pay attention to it since it's a blocker for our
company, Grid Dynamics?

-- 
Alessio Ababilov
Software Engineer
Grid Dynamics

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [nova] Disk attachment consistency

2012-08-15 Thread John Garbutt

You can see what XenAPI exposes here:
 http://docs.vmd.citrix.com/XenServer/6.0.0/1.0/en_gb/api/?c=VBD

I think the only thing you can influence when plugging in the disk is the 
“userdevice” which is the disk position: 0,1,2…
When you have attached the disk you can find out the “device” name, such as 
/dev/xvda

I don't know about Xen with libvirt. But from the previous discussion it seems 
using the disk position would also work with KVM?

It seems disk position is also suitably OS agnostic, but I may have missed 
something there.

For backwards compatibility, we could make a best effort of translating the 
specified device name to a position. But as mentioned already, it seems fraught 
with danger in the general case.

I like the idea of an extra field to help report to the user what the likely 
device name is, if available. It would allow us to spot when the above 
guessing did not gone the way we had hoped.

Related to this, there is a limitation in Xen (a limitation in the 
blkbk/blkfrnt protocol I am told) that means you can't cancel the operation of 
removing a disk from a VM. So if the disk is in use, it may return with an 
exception saying the disk is in-use, but as soon as the disk can be released, 
it will be removed anyway. Currently, nova isn't very good at expressing to the 
user that this is what is happening:
https://bugs.launchpad.net/nova/+bug/1030108

Cheers,
John
 
 From: openstack-bounces+john.garbutt=citrix@lists.launchpad.net 
 [mailto:openstack-bounces+john.garbutt=citrix@lists.launchpad.net] On 
 Behalf Of Wangpan
 Sent: 15 August 2012 5:11
 To: Vishvananda Ishaya
 Cc: openstack
 Subject: Re: [Openstack] [nova] Disk attachment consistency
  This is definitely another solution, although it seems less usable than the 
  device serial number which can be an arbitrary string. If this works for 
  xen though, that would be a plus.
  Vish
 I don't have a Xen hypervisor in hand, so anybody else can try it on Xen ? 
 thanks
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [nova] Disk attachment consistency

2012-08-15 Thread Daniel P. Berrange

On Wed, Aug 15, 2012 at 03:49:45PM +0100, John Garbutt wrote:
 You can see what XenAPI exposes here:
  http://docs.vmd.citrix.com/XenServer/6.0.0/1.0/en_gb/api/?c=VBD
 
 I think the only thing you can influence when plugging in the disk is the 
 “userdevice”
 which is the disk position: 0,1,2…  When you have attached the disk you can 
 find out
 the “device” name, such as /dev/xvda
 
 I don't know about Xen with libvirt. But from the previous discussion it 
 seems using
 the disk position would also work with KVM?

No, this doesn't really work in general. Virtio disks get assigned SCSI device
numbers on a first-come first served basis. In the configuration you only have
control over the PCI device slot/function. You might assume that your disks
are probed in PCI device order, and thus get SCSI device numbers in that same
order. This is not really safe though. Further more if the guest has any
other kinds of devices, eg perhaps they logged into an iSCSI target, then all
bets are off for what SCSI device you get assigned.

All the host can safely say is

  - Virtio-blk disks get PCI address domain:bus:slot:function
  - Virtio-SCSI disks get SCSI address A.B.C.D
  - Disks have an unique serial string ZZZ

As a guest OS admin you can use this info get reliable disk names
in /dev/disk/by-{path,id}.

If your disk has a filesystem on it, you can also get a unique UUID
and /or filesystem label, which means you can refer to the device
from /dev/disk/by-{uuid,label} too.

Relying on /dev/sdXXX is doomed to failure in the long term, even on
bare metal, and should be avoided wherever possible.

Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Discussion about where to put database for bare-metal provisioning (review 10726)

2012-08-15 Thread Alex Glikson

Can you elaborate what is the purpose of this database?
If we compare it to KVM support, the 'primary' location of VMs' metadata 
is in libvirt internal store (outside of Nova), and then it is cached in 
Nova DB, for Nova purposes.
A similar approach might make for bare-metal machines too -- keep 
'primary' metadata store outside of Nova, and a cache in Nova DB.

Regards,
Alex




From:   David Kang dk...@isi.edu
To: OpenStack Development Mailing List 
openstack-...@lists.openstack.org, openstack@lists.launchpad.net 
(openstack@lists.launchpad.net) openstack@lists.launchpad.net, 
Date:   15/08/2012 06:32 PM
Subject:[Openstack] Discussion about where to put database for 
bare-metal provisioning (review 10726)
Sent by:openstack-bounces+glikson=il.ibm@lists.launchpad.net





 Hi,

 This is call for discussion about the code review 10726.
https://review.openstack.org/#/c/10726/
Mark asked why we implemented a separata database for bare-metal 
provisioning.
Here we describe our thought. 
We are open to discussion and to the changes that the community 
recommends.
Please give us your thoughts.

 NTT Docomo and USC/ISI have developed bare-metal provisioning.
We created separate database to describe bare-metal nodes, which consists 
of 5 tables now.
Our initial implementation assumes the database is not a part of nova 
database.
In addition to the reasons described in the comments of the code review,
here is another reason we decided a separate database for baremetal 
provisioning.

Bare-metal database is mainly used by bare-metal nova-compute.
Since bare-metal nova-compute manages multiple bare-metal machines, 
it needs to keep/update the information of bare-metal machines.
If the bare-metal database is in the main nova db, accessing nova db 
remotely by
bare-metal nova-compute is inevitable.
Once Vish told us that shared db access from nova-compute is not 
desirable.

It is possible to make the scheduler do the job of bare-metal 
nova-compute.
However, it would need a big changes in how the scheduler and a 
nova-compute
communicates. For example, currently, the scheduler casts an instance to a
nova-compute. But for bare-metal node, the scheduler should cast an 
instance 
to a bare-metal machine through bare-metal nova-compute.
Bare-metal nova-compute should boot the machine, transfer kernel, fs, etc.
So, bare-metal nova-compute should know the id of bare-metal node and 
other information 
for booting (PXE ip address, ...) and more.
That information should be sent to bare-metal nova-compute by the 
scheduler.

If frequent access of bare-metal tables in nova db from bare-metal 
nova-compute
is OK, we are OK to put the bare-metal tables into nova db.

 Please let us know your opinions.

 Thanks,
 David, Mikyung @ USC/ISI

--
Dr. Dong-In David Kang
Computer Scientist
USC/ISI


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [nova] Disk attachment consistency

2012-08-15 Thread John Garbutt

 From: Daniel P. Berrange [mailto:berra...@redhat.com]
 On Wed, Aug 15, 2012 at 03:49:45PM +0100, John Garbutt wrote:
  You can see what XenAPI exposes here:
   http://docs.vmd.citrix.com/XenServer/6.0.0/1.0/en_gb/api/?c=VBD
 
  I think the only thing you can influence when plugging in the disk is the
 “userdevice”
  which is the disk position: 0,1,2…  When you have attached the disk
  you can find out the “device” name, such as /dev/xvda
 
  I don't know about Xen with libvirt. But from the previous discussion
  it seems using the disk position would also work with KVM?
 
 No, this doesn't really work in general. Virtio disks get assigned SCSI device
 numbers on a first-come first served basis. In the configuration you only have
 control over the PCI device slot/function. You might assume that your disks 
 are
 probed in PCI device order, and thus get SCSI device numbers in that same 
 order.
 This is not really safe though. Further more if the guest has any other kinds 
 of
 devices, eg perhaps they logged into an iSCSI target, then all bets are off 
 for
 what SCSI device you get assigned.

 All the host can safely say is
 
   - Virtio-blk disks get PCI address domain:bus:slot:function
   - Virtio-SCSI disks get SCSI address A.B.C.D
   - Disks have an unique serial string ZZZ
 
 As a guest OS admin you can use this info get reliable disk names in
 /dev/disk/by-{path,id}.
Doh, I guess my plan doesn't work then. After checking, apparently the same 
problem is also present with how Xen deals with exposing the position to the 
guest VM.

 Relying on /dev/sdXXX is doomed to failure in the long term, even on bare
 metal, and should be avoided wherever possible.
I agree, long term, this is not the way forward. I was just thinking in terms 
of backwards compatibility.

 If your disk has a filesystem on it, you can also get a unique UUID and /or
 filesystem label, which means you can refer to the device from /dev/disk/by-
 {uuid,label} too.
That sounds the interesting for those attaching volumes that have a file system 
on it. Would it be reasonable to make this the best practice way for users to 
discover where the volume has been attached?

Maybe we should simply leave nova to report where the disk has been attached? 
XenAPI driver can simply attach in the next available slot, and report back 
what it can about the device location. Sounds like the libvirt driver could do 
the same?

We could document the current device (or whatever we call it) as a driver 
specific hint. Although this doesn't seem very satisfying.

Cheers,
John
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] Review required: https://review.openstack.org/#/c/11016/

2012-08-15 Thread Gurjar, Unmesh

Hi All,

Please review: https://review.openstack.org/#/c/11016/ - HTTP POST Notifier for 
OpenStack projects.


Thanks  Regards,
Unmesh Gurjar | Lead Engineer | NTT DATA Global Technology Services Private 
Limited | w. +91.20.6604.1500 x 379 | m. +91.982.324.7631 | 
unmesh.gur...@nttdata.com | Learn more at nttdata.com/americas

__
Disclaimer:This email and any attachments are sent in strictest confidence for 
the sole use of the addressee and may contain legally privileged, confidential, 
and proprietary data.  If you are not the intended recipient, please advise the 
sender by replying promptly to this email and then delete and destroy this 
email and any attachments without any further use, copying or forwarding

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Discussion about where to put database for bare-metal provisioning (review 10726)

2012-08-15 Thread David Kang


 
 The bare-metal database includes five tables:

1. bm_nodes   // This is similar to compute_node table
2. bm_deployments   // The status of deployment of bare-metal nodes
3. bm_pxe_ips   // PXE information for bare-metal nodes
4. bm_interfaces   // network information of bare-metal nodes
5. migrate_version  // for database migration

 The information of bare-metal nodes and their status 
are sent to the scheduler as an aggregate capability set of the bare-metal 
machines.
Our current approach is to have a new BaremetalHostManager 
(nova/nova/scheduler/baremetal_host_manager.py)
that caches the information.
BaremetalHostManager gets the information by accessing bare-metal db directly 
for now. (Proposed patch 4)
It works only when there is one shared bare-metal db exists.
But, it looks like that non-shared bare-metal db is preferred.
We will change BaremetalHostManager to use RPC (instead of db access) to access 
those information from multiple bare-metal nova-compute nodes. 

 Thanks,
 David




--
Dr. Dong-In David Kang
Computer Scientist
USC/ISI

- Original Message -
 Can you elaborate what is the purpose of this database?
 If we compare it to KVM support, the 'primary' location of VMs'
 metadata is in libvirt internal store (outside of Nova), and then it
 is cached in Nova DB, for Nova purposes.
 A similar approach might make for bare-metal machines too -- keep
 'primary' metadata store outside of Nova, and a cache in Nova DB.
 
 Regards,
 Alex
 
 
 
 
 From: David Kang dk...@isi.edu
 To: OpenStack Development Mailing List
 openstack-...@lists.openstack.org, openstack@lists.launchpad.net
 (openstack@lists.launchpad.net) openstack@lists.launchpad.net,
 Date: 15/08/2012 06:32 PM
 Subject: [Openstack] Discussion about where to put database for
 bare-metal provisioning (review 10726)
 Sent by: openstack-bounces+glikson=il.ibm@lists.launchpad.net
 
 
 
 
 
 
 Hi,
 
 This is call for discussion about the code review 10726.
 https://review.openstack.org/#/c/10726/
 Mark asked why we implemented a separata database for bare-metal
 provisioning.
 Here we describe our thought.
 We are open to discussion and to the changes that the community
 recommends.
 Please give us your thoughts.
 
 NTT Docomo and USC/ISI have developed bare-metal provisioning.
 We created separate database to describe bare-metal nodes, which
 consists of 5 tables now.
 Our initial implementation assumes the database is not a part of nova
 database.
 In addition to the reasons described in the comments of the code
 review,
 here is another reason we decided a separate database for baremetal
 provisioning.
 
 Bare-metal database is mainly used by bare-metal nova-compute.
 Since bare-metal nova-compute manages multiple bare-metal machines,
 it needs to keep/update the information of bare-metal machines.
 If the bare-metal database is in the main nova db, accessing nova db
 remotely by
 bare-metal nova-compute is inevitable.
 Once Vish told us that shared db access from nova-compute is not
 desirable.
 
 It is possible to make the scheduler do the job of bare-metal
 nova-compute.
 However, it would need a big changes in how the scheduler and a
 nova-compute
 communicates. For example, currently, the scheduler casts an instance
 to a
 nova-compute. But for bare-metal node, the scheduler should cast an
 instance
 to a bare-metal machine through bare-metal nova-compute.
 Bare-metal nova-compute should boot the machine, transfer kernel, fs,
 etc.
 So, bare-metal nova-compute should know the id of bare-metal node and
 other information
 for booting (PXE ip address, ...) and more.
 That information should be sent to bare-metal nova-compute by the
 scheduler.
 
 If frequent access of bare-metal tables in nova db from bare-metal
 nova-compute
 is OK, we are OK to put the bare-metal tables into nova db.
 
 Please let us know your opinions.
 
 Thanks,
 David, Mikyung @ USC/ISI
 
 --
 Dr. Dong-In David Kang
 Computer Scientist
 USC/ISI
 
 
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Devstack multinode Installation network configuration help and bridging explanation

2012-08-15 Thread Vishvananda Ishaya


On Aug 14, 2012, at 10:52 PM, Kurt Scholtens burnbrigh...@hotmail.com wrote:

 Vish - 
 
 Thank you for responding.  So, when using multiple interfaces, will setting 
 the PUBLIC_INTERFACE to the separate interface w/o VLANing work?  Is that 
 what you are saying below?  Are any additional configuration parameters then 
 required such as flat_network _bridge?

It should work on two separate interfaces with just that change.

 
 Something still puzzling me:
 From my routing table below, it appears both the public interface AND the 
 flat interface are on br100 (which is bound to eth2).  I didn't think a 
 bridge could have more than one subnet attached to it at any one time. I was 
 apparently wrong.
multiple unrelated ips on an interface is fine
 My br100 has an associated address of 11.0.4.12, but eth2 has the IP address 
 of 192.168.12.105br100 then, has the public_interface traffic (floating_range 
 - 192.168.12.x) and the fixed_address traffic traversing eth2, correct? 
 Is my understanding of what is going on correct?

if the interface specified in flat_interface has an ip on it, it moves it to 
the bridge. So you will often have multiple ips on the bridge: 1) the hosts 
original ip 2) an ip in the fixed_range for dnsmasq 3) all floating ips 
(assuming public_interface=br100) In addition, all guest vnics are on the same 
bridge.

public_interface just controls which interface is used for the floating ips. If 
you want them to be usable from outside the host, you have to make sure that is 
an interface (ir a bridge on an interface) that your router knows about.

Vish___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [openstack-dev] [nova] Disk attachment consistency

2012-08-15 Thread Vishvananda Ishaya


On Aug 14, 2012, at 10:00 PM, Chuck Thier cth...@gmail.com wrote:

 snip
 I could get behind this, and was was brought up by others in our group
 as a more feasible short term solution.  I have a couple of concerns
 with this.  It may cause just as much confusion if the api can't
 reliably determine which device a volume is attached to.  I'm also
 curious as to how well this will work with Xen, and hope some of the
 citrix folks will chime in.  From an api standpoint, I think it would
 be fine to make it optional, as any client that is using old api
 contract will still work as intended.

This will continue to work as well as it currently does with Xen. We can 
reliably determine which devices are known about and pass back the next one my 
patch under review below does that
 
 (review at https://review.openstack.org/#/c/10908/)
 
 snip
 First question: should we return this magic path somewhere via the api? It 
 would be pretty easy to have horizon generate it but it might be nice to 
 have it show up. If we do return it, do we mangle the device to always show 
 the consistent one, or do we return it as another parameter? guest_device 
 perhaps?
 
 Second question: what should happen if someone specifies /dev/xvda against a 
 kvm cloud or /dev/vda against a xen cloud?
 I see two options:
 a) automatically convert it to the right value and return it
 
 I thought that it already did this, but I would have to go back and
 double check.  But it seemed like for xen at least, if you specify
 /dev/vda, Nova would change it to /dev/xvda.

That may be true, I believe for libvirt we just accept /dev/xvdc since it is 
just interpreted as a label.
 
 
 snip
 
 Xen Server 6.0 has a limit of 16 virtual devices per guest instance.
 Experimentally it also expects those to be /dev/xvda - /dev/xvdp.  You
 can't for example attach a device to /dev/xvdq, even if there are no
 other devices attached to the instance.  If you attempt to do this,
 the volume will go in to the attaching state, fail to attach, and then
 fall back to the available state (This can be a bit confusing to new
 users who try to do so).  Does anyone know if there are similar
 limitations for KVM?

There are no limitations like this AFAIK, however in KVM it is possible
to exhaust virtio minor device numbers by continually detaching and
attaching a device if the guest kernel  3.2

 
 Also if you attempt to attach a volume to a deivce that already
 exists, it will silently fail and go back to available as well.  In
 this new scheme should it fail like that, or should it attempt to
 attach it to the next available device, or error out?  Perhaps a
 better question here, is for this initial consistency, is the goal to
 try to be consistent just when there is no device sent, or to also be
 consistent when the device is sent as well.

my review above addresses this by raising an error if you try to attach
to an existing device. I think this is preferrable: i.e. only do the
auto-assign if it is specifically requested.

 
 There was another idea, also brought up in our group.  Would it be
 possible to add a call that would return a list of available devices
 to be attached to?  In the case of Xen, it would return a list of
 devices /dev/xvda-p that were not used.  In the case of KVM, it would
 just return the next available device name.  At least in this case,
 user interfaces and command line tools could use this to validate the
 input the user provides (or auto generate the device to be used if the
 user doesn't select a device).

This is definitely a possibility, although it seems like a separate feature.

Vish


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Help with meta-data

2012-08-15 Thread Eugene Kirpichov

Hi Anne,

I accidentally found this email of yours while looking for links to my post.
I'd probably have found it earlier if you cc'd me on
ekirpic...@gmail.com or ekirpic...@mirantis.com [yes, that's two
different spellings...] :)

I support the idea that this should be somehow integrated in the docs,
but I'm not sure where exactly in the docs is a good place for
information of this style. Would it help if I just linked to the posts
from some wiki page or from the docs? Or would it only help if I (or
somebody) actually merged the relevant parts of the posts into
official documentation?

On Thu, Aug 9, 2012 at 8:07 AM, Anne Gentle a...@openstack.org wrote:
All, sorry for top posting, but this is a fine example of why we
really need bloggers to help with the documentation. These fragmented
instructions are difficult to rely on - we need maintainable,
process-oriented treatment of content.

Mirantis peeps, you have added in your blog entries to the docs in the
past, let's find ways to continually do that and maintain going
forward.

I'm not so interested in more install guides, but definitely
interested in more configuration guides. So Kord, while I like the
idea (and execution!) of the StackGeek 10-minute guide, it's not one
to bring into the official docs. But we would definitely welcome your
reviews of incoming updates to the docs!

Thanks Simon for bringing your difficulties to the list - we
continually work on improving the docs. What you learn now could help
hundreds if not thousands of others, so I'd love for you to improve
the official docs with your findings.
Thanks,
Anne

On Thu, Aug 9, 2012 at 4:42 AM, Simon Walter si...@gikaku.com wrote:

On 08/09/2012 12:59 PM, Scott Moser wrote:

On Aug 8, 2012, at 8:20 PM, Simon Walter si...@gikaku.com wrote:

On 08/09/2012 06:45 AM, Jay Pipes
I guess I'll have to build a VM from scratch, as I was relying on the ssh
key to be able to ssh into the VM, which apparently is supplied by the
meta-data service.

use cirros.
load an image, ssh on with 'cirros' user. pass is 'cubswin:)'

Thank you. That was good advice.

Somehow I was not able to connect via ssh. I managed to get novnc working
and logged into the VM. I can't find anything about connecting via serial or
the like as you can with Xen. I need to read more about KVM I guess.

Anyway, I think my networking setup is stuffed. I thought the 10 minutes
install would be the quickest way to get and running. Now I find myself
pouring over documentation trying to understand how best to setup
FlatDHCPManager with two network interfaces. I understand many things have
changed. So I don't want to go reading something out of date. I found these
blog posts which explained a lot:
http://www.mirantis.com/blog/openstack-networking-flatmanager-and-flatdhcpmanager/#comments
http://www.mirantis.com/blog/openstack-networking-single-host-flatdhcpmanager/
But am I reading the wrong thing? I like the way Stackgeek had it set up:
http://stackgeek.com/guides/gettingstarted.html

But I think they are missing details or it's out dated. For example, with
their setup the vnc console in horizon does not work because nova-vncproxy
is installed rather than novnc.

I'm pretty sure I can figure the networking out if I have the right
documentation in the first place. Is there a clear instructions for this
anywhere? Or would someone mind walking me through it again. So far I've
followed the stackgeek setup above, but the networking is obviously stuffed.

Must I have the flat_interface in promiscuous mode?
Or does it actually need an IP address?
Why are my VMs picking up an IP address from the public_interface DHCP
server and not from the flat_network_bridge?

Too many questions to ask. So I thought I should just ask: what is missing
or incorrect from Stackgeeks 10 minute scripts?

Many thanks for any advice, tips, docs, etc.

Cheers,

Simon

--
simonsmicrophone.com

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

--
Eugene Kirpichov
http://www.linkedin.com/in/eugenekirpichov

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

Re: [Openstack] Help with meta-data

2012-08-15 Thread Anne Gentle

Hi Eugene -
But I thought everyone was on the openstack list! :) Thanks for following up.

On Wed, Aug 15, 2012 at 4:35 PM, Eugene Kirpichov ekirpic...@gmail.com wrote:
Hi Anne,

I do like to put relevant blog posts on the wiki at
http://wiki.openstack.org/BloggersTips, so you can certainly add to
that page. If it's really missing information in the docs, though, it
should be added to the docs. I know that's a tough judgement call but
we all have to encourage that call.

Or would it only help if I (or
somebody) actually merged the relevant parts of the posts into
official documentation?

I wouldn't say only help but I prefer that you merge the relevant
parts of the posts. It's tougher for a doc team member to merge only
parts in without violating the license of the content - you as content
owner can certainly choose which parts to move into the official
documentation though.

Thanks for asking for clarifications - these are certainly gray areas
that I'd like to shine light upon.
Anne

Mirantis peeps, you have added in your blog entries to the docs in the
past, let's find ways to continually do that and maintain going
forward.

On Thu, Aug 9, 2012 at 4:42 AM, Simon Walter si...@gikaku.com wrote:

On 08/09/2012 12:59 PM, Scott Moser wrote:

On Aug 8, 2012, at 8:20 PM, Simon Walter si...@gikaku.com wrote:

On 08/09/2012 06:45 AM, Jay Pipes
I guess I'll have to build a VM from scratch, as I was relying on the ssh
key to be able to ssh into the VM, which apparently is supplied by the
meta-data service.

use cirros.
load an image, ssh on with 'cirros' user. pass is 'cubswin:)'

Thank you. That was good advice.

But I think they are missing details or it's out dated. For example, with
their setup the vnc console in horizon does not work because nova-vncproxy
is installed rather than novnc.

Too many questions to ask. So I thought I should just ask: what is missing
or incorrect from Stackgeeks 10 minute scripts?

Many thanks for any advice, tips, docs, etc.

Cheers,

Simon

--
simonsmicrophone.com

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net

Re: [Openstack] [openstack-dev] Discussion about where to put database for bare-metal provisioning (review 10726)

2012-08-15 Thread Michael J Fork


I am interested in finding a solution that enables bare-metal and
virtualized requests to be serviced through the same scheduler where the
compute_nodes table has a full view of schedulable resources.  This would
seem to simplify the end-to-end flow while opening up some additional use
cases (e.g. dynamic allocation of a node from bare-metal to hypervisor and
back).

One approach would be to have a proxy running a single nova-compute daemon
fronting the bare-metal nodes .  That nova-compute daemon would report up
many HostState objects (1 per bare-metal node) to become entries in the
compute_nodes table and accessible through the scheduler HostManager
object.  The HostState object would set cpu_info, vcpus, member_mb and
local_gb values to be used for scheduling with the hypervisor_host field
holding the bare-metal machine address (e.g. for IPMI based commands) and
hypervisor_type = NONE.  The bare-metal Flavors are created with an
extra_spec of hypervisor_type= NONE and the corresponding
compute_capabilities_filter would reduce the available hosts to those
bare_metal nodes.  The scheduler would need to understand that
hypervisor_type = NONE means you need an exact fit (or best-fit) host vs
weighting them (perhaps through the multi-scheduler).  The scheduler would
cast out the message to the topic.service-hostname (code today uses the
HostState hostname), with the compute driver having to understand if it
must be serviced elsewhere (but does not break any existing implementations
since it is 1 to 1).

Does this solution seem workable? Anything I missed?

Michael

-
Michael Fork
Cloud Architect, Emerging Solutions
IBM Systems  Technology Group

David Kang dk...@isi.edu wrote on 08/15/2012 11:28:34 AM:

 From: David Kang dk...@isi.edu
 To: OpenStack Development Mailing List openstack-
 d...@lists.openstack.org, openstack@lists.launchpad.net
 (openstack@lists.launchpad.net) openstack@lists.launchpad.net,
 Cc: mkk...@isi.edu, Ken Ash 50ba...@gmail.com, VTJ NOTSU Arata
 no...@virtualtech.jp
 Date: 08/15/2012 02:08 PM
 Subject: [openstack-dev] Discussion about where to put database for
 bare-metal provisioning (review 10726)



  Hi,

  This is call for discussion about the code review 10726.
 https://review.openstack.org/#/c/10726/
 Mark asked why we implemented a separata database for bare-metal
provisioning.
 Here we describe our thought.
 We are open to discussion and to the changes that the community
recommends.
 Please give us your thoughts.

  NTT Docomo and USC/ISI have developed bare-metal provisioning.
 We created separate database to describe bare-metal nodes, which
 consists of 5 tables now.
 Our initial implementation assumes the database is not a part of
 nova database.
 In addition to the reasons described in the comments of the code review,
 here is another reason we decided a separate database for baremetal
 provisioning.

 Bare-metal database is mainly used by bare-metal nova-compute.
 Since bare-metal nova-compute manages multiple bare-metal machines,
 it needs to keep/update the information of bare-metal machines.
 If the bare-metal database is in the main nova db, accessing nova db
 remotely by
 bare-metal nova-compute is inevitable.
 Once Vish told us that shared db access from nova-compute is not
desirable.

 It is possible to make the scheduler do the job of bare-metal
nova-compute.
 However, it would need a big changes in how the scheduler and a
nova-compute
 communicates. For example, currently, the scheduler casts an instance to
a
 nova-compute. But for bare-metal node, the scheduler should cast an
instance
 to a bare-metal machine through bare-metal nova-compute.
 Bare-metal nova-compute should boot the machine, transfer kernel, fs,
etc.
 So, bare-metal nova-compute should know the id of bare-metal node
 and other information
 for booting (PXE ip address, ...) and more.
 That information should be sent to bare-metal nova-compute by the
scheduler.

 If frequent access of bare-metal tables in nova db from bare-metal
 nova-compute
 is OK, we are OK to put the bare-metal tables into nova db.

  Please let us know your opinions.

  Thanks,
  David, Mikyung @ USC/ISI

 --
 Dr. Dong-In David Kang
 Computer Scientist
 USC/ISI


 ___
 OpenStack-dev mailing list
 openstack-...@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [openstack-dev] Discussion about where to put database for bare-metal provisioning (review 10726)

2012-08-15 Thread Vishvananda Ishaya


On Aug 15, 2012, at 3:17 PM, Michael J Fork mjf...@us.ibm.com wrote:

 I am interested in finding a solution that enables bare-metal and virtualized 
 requests to be serviced through the same scheduler where the compute_nodes 
 table has a full view of schedulable resources.  This would seem to simplify 
 the end-to-end flow while opening up some additional use cases (e.g. dynamic 
 allocation of a node from bare-metal to hypervisor and back).  
 
 One approach would be to have a proxy running a single nova-compute daemon 
 fronting the bare-metal nodes .  That nova-compute daemon would report up 
 many HostState objects (1 per bare-metal node) to become entries in the 
 compute_nodes table and accessible through the scheduler HostManager object.
 
 The HostState object would set cpu_info, vcpus, member_mb and local_gb values 
 to be used for scheduling with the hypervisor_host field holding the 
 bare-metal machine address (e.g. for IPMI based commands) and hypervisor_type 
 = NONE.  The bare-metal Flavors are created with an extra_spec of 
 hypervisor_type= NONE and the corresponding compute_capabilities_filter would 
 reduce the available hosts to those bare_metal nodes.  The scheduler would 
 need to understand that hypervisor_type = NONE means you need an exact fit 
 (or best-fit) host vs weighting them (perhaps through the multi-scheduler).  
 The scheduler would cast out the message to the topic.service-hostname 
 (code today uses the HostState hostname), with the compute driver having to 
 understand if it must be serviced elsewhere (but does not break any existing 
 implementations since it is 1 to 1).
 
 
 Does this solution seem workable? Anything I missed?
 
The bare metal driver already is proxying for the other nodes so it sounds like 
we need a couple of things to make this happen:

a) modify driver.get_host_stats to be able to return a list of host stats 
instead of just one. Report the whole list back to the scheduler. We could 
modify the receiving end to accept a list as well or just make multiple calls 
to 
self.update_service_capabilities(capabilities)

b) make a few minor changes to the scheduler to make sure filtering still 
works. Note the changes here may be very helpful:

https://review.openstack.org/10327

c) we have to make sure that instances launched on those nodes take up the 
entire host state somehow. We could probably do this by making sure that the 
instance_type ram, mb, gb etc. matches what the node has, but we may want a new 
boolean field used if those aren't sufficient.

I This approach seems pretty good. We could potentially get rid of the shared 
bare_metal_node table. I guess the only other concern is how you populate the 
capabilities that the bare metal nodes are reporting. I guess an api extension 
that rpcs to a baremetal node to add the node. Maybe someday this could be 
autogenerated by the bare metal host looking in its arp table for dhcp 
requests! :)

Vish

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [openstack-dev] Discussion about where to put database for bare-metal provisioning (review 10726)

2012-08-15 Thread David Kang


 Hi Vish,

 Is this discussion for long-term goal or for this Folsom release?

 We still believe that bare-metal database is needed
because there is not an automated way how bare-metal nodes report their 
capabilities
to their bare-metal nova-compute node. 

 Thanks,
 David
 
 
 I am interested in finding a solution that enables bare-metal and
 virtualized requests to be serviced through the same scheduler where
 the compute_nodes table has a full view of schedulable resources. This
 would seem to simplify the end-to-end flow while opening up some
 additional use cases (e.g. dynamic allocation of a node from
 bare-metal to hypervisor and back).
 
 One approach would be to have a proxy running a single nova-compute
 daemon fronting the bare-metal nodes . That nova-compute daemon would
 report up many HostState objects (1 per bare-metal node) to become
 entries in the compute_nodes table and accessible through the
 scheduler HostManager object.
 
 
 
 
 The HostState object would set cpu_info, vcpus, member_mb and local_gb
 values to be used for scheduling with the hypervisor_host field
 holding the bare-metal machine address (e.g. for IPMI based commands)
 and hypervisor_type = NONE. The bare-metal Flavors are created with an
 extra_spec of hypervisor_type= NONE and the corresponding
 compute_capabilities_filter would reduce the available hosts to those
 bare_metal nodes. The scheduler would need to understand that
 hypervisor_type = NONE means you need an exact fit (or best-fit) host
 vs weighting them (perhaps through the multi-scheduler). The scheduler
 would cast out the message to the topic.service-hostname (code
 today uses the HostState hostname), with the compute driver having to
 understand if it must be serviced elsewhere (but does not break any
 existing implementations since it is 1 to 1).
 
 
 
 
 
 Does this solution seem workable? Anything I missed?
 
 The bare metal driver already is proxying for the other nodes so it
 sounds like we need a couple of things to make this happen:
 
 
 a) modify driver.get_host_stats to be able to return a list of host
 stats instead of just one. Report the whole list back to the
 scheduler. We could modify the receiving end to accept a list as well
 or just make multiple calls to
 self.update_service_capabilities(capabilities)
 
 
 b) make a few minor changes to the scheduler to make sure filtering
 still works. Note the changes here may be very helpful:
 
 
 https://review.openstack.org/10327
 
 
 c) we have to make sure that instances launched on those nodes take up
 the entire host state somehow. We could probably do this by making
 sure that the instance_type ram, mb, gb etc. matches what the node
 has, but we may want a new boolean field used if those aren't
 sufficient.
 
 
 I This approach seems pretty good. We could potentially get rid of the
 shared bare_metal_node table. I guess the only other concern is how
 you populate the capabilities that the bare metal nodes are reporting.
 I guess an api extension that rpcs to a baremetal node to add the
 node. Maybe someday this could be autogenerated by the bare metal host
 looking in its arp table for dhcp requests! :)
 
 
 Vish
 
 ___
 OpenStack-dev mailing list
 openstack-...@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [openstack-dev] Discussion about where to put database for bare-metal provisioning (review 10726)

2012-08-15 Thread Vishvananda Ishaya

This was an immediate goal, the bare-metal nova-compute node could keep an 
internal database, but report capabilities through nova in the common way with 
the changes below. Then the scheduler wouldn't need access to the bare metal 
database at all.

On Aug 15, 2012, at 4:23 PM, David Kang dk...@isi.edu wrote:

 
 Hi Vish,
 
 Is this discussion for long-term goal or for this Folsom release?
 
 We still believe that bare-metal database is needed
 because there is not an automated way how bare-metal nodes report their 
 capabilities
 to their bare-metal nova-compute node. 
 
 Thanks,
 David
 
 
 I am interested in finding a solution that enables bare-metal and
 virtualized requests to be serviced through the same scheduler where
 the compute_nodes table has a full view of schedulable resources. This
 would seem to simplify the end-to-end flow while opening up some
 additional use cases (e.g. dynamic allocation of a node from
 bare-metal to hypervisor and back).
 
 One approach would be to have a proxy running a single nova-compute
 daemon fronting the bare-metal nodes . That nova-compute daemon would
 report up many HostState objects (1 per bare-metal node) to become
 entries in the compute_nodes table and accessible through the
 scheduler HostManager object.
 
 
 
 
 The HostState object would set cpu_info, vcpus, member_mb and local_gb
 values to be used for scheduling with the hypervisor_host field
 holding the bare-metal machine address (e.g. for IPMI based commands)
 and hypervisor_type = NONE. The bare-metal Flavors are created with an
 extra_spec of hypervisor_type= NONE and the corresponding
 compute_capabilities_filter would reduce the available hosts to those
 bare_metal nodes. The scheduler would need to understand that
 hypervisor_type = NONE means you need an exact fit (or best-fit) host
 vs weighting them (perhaps through the multi-scheduler). The scheduler
 would cast out the message to the topic.service-hostname (code
 today uses the HostState hostname), with the compute driver having to
 understand if it must be serviced elsewhere (but does not break any
 existing implementations since it is 1 to 1).
 
 
 
 
 
 Does this solution seem workable? Anything I missed?
 
 The bare metal driver already is proxying for the other nodes so it
 sounds like we need a couple of things to make this happen:
 
 
 a) modify driver.get_host_stats to be able to return a list of host
 stats instead of just one. Report the whole list back to the
 scheduler. We could modify the receiving end to accept a list as well
 or just make multiple calls to
 self.update_service_capabilities(capabilities)
 
 
 b) make a few minor changes to the scheduler to make sure filtering
 still works. Note the changes here may be very helpful:
 
 
 https://review.openstack.org/10327
 
 
 c) we have to make sure that instances launched on those nodes take up
 the entire host state somehow. We could probably do this by making
 sure that the instance_type ram, mb, gb etc. matches what the node
 has, but we may want a new boolean field used if those aren't
 sufficient.
 
 
 I This approach seems pretty good. We could potentially get rid of the
 shared bare_metal_node table. I guess the only other concern is how
 you populate the capabilities that the bare metal nodes are reporting.
 I guess an api extension that rpcs to a baremetal node to add the
 node. Maybe someday this could be autogenerated by the bare metal host
 looking in its arp table for dhcp requests! :)
 
 
 Vish
 
 ___
 OpenStack-dev mailing list
 openstack-...@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 ___
 OpenStack-dev mailing list
 openstack-...@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [openstack-dev] Discussion about where to put database for bare-metal provisioning (review 10726)

2012-08-15 Thread David Kang


 I see.
We (NTT and USC/ISI) will discuss more about that.
And we will implement according to your input.
From now, we assume to use current db.

 Here are my understanding of your comment. Please correct me if it is not 
correct.

 a) is clear.
 b) NTT Docomo has implemented some features that you mentioned in a new file 
baremetal_host_manager.py.
  We will not use baremetal_host_manager.py but will modify host_manager.py 
directly.
  I'm not sure if we have to change the scheduler. 
 c) I think used field my work. We'll look into it.

 Thanks,
 David

--
Dr. Dong-In David Kang
Computer Scientist
USC/ISI

- Original Message -
 This was an immediate goal, the bare-metal nova-compute node could
 keep an internal database, but report capabilities through nova in the
 common way with the changes below. Then the scheduler wouldn't need
 access to the bare metal database at all.
 
 On Aug 15, 2012, at 4:23 PM, David Kang dk...@isi.edu wrote:
 
 
  Hi Vish,
 
  Is this discussion for long-term goal or for this Folsom release?
 
  We still believe that bare-metal database is needed
  because there is not an automated way how bare-metal nodes report
  their capabilities
  to their bare-metal nova-compute node.
 
  Thanks,
  David
 
 
  I am interested in finding a solution that enables bare-metal and
  virtualized requests to be serviced through the same scheduler
  where
  the compute_nodes table has a full view of schedulable resources.
  This
  would seem to simplify the end-to-end flow while opening up some
  additional use cases (e.g. dynamic allocation of a node from
  bare-metal to hypervisor and back).
 
  One approach would be to have a proxy running a single nova-compute
  daemon fronting the bare-metal nodes . That nova-compute daemon
  would
  report up many HostState objects (1 per bare-metal node) to become
  entries in the compute_nodes table and accessible through the
  scheduler HostManager object.
 
 
 
 
  The HostState object would set cpu_info, vcpus, member_mb and
  local_gb
  values to be used for scheduling with the hypervisor_host field
  holding the bare-metal machine address (e.g. for IPMI based
  commands)
  and hypervisor_type = NONE. The bare-metal Flavors are created with
  an
  extra_spec of hypervisor_type= NONE and the corresponding
  compute_capabilities_filter would reduce the available hosts to
  those
  bare_metal nodes. The scheduler would need to understand that
  hypervisor_type = NONE means you need an exact fit (or best-fit)
  host
  vs weighting them (perhaps through the multi-scheduler). The
  scheduler
  would cast out the message to the topic.service-hostname (code
  today uses the HostState hostname), with the compute driver having
  to
  understand if it must be serviced elsewhere (but does not break any
  existing implementations since it is 1 to 1).
 
 
 
 
 
  Does this solution seem workable? Anything I missed?
 
  The bare metal driver already is proxying for the other nodes so it
  sounds like we need a couple of things to make this happen:
 
 
  a) modify driver.get_host_stats to be able to return a list of host
  stats instead of just one. Report the whole list back to the
  scheduler. We could modify the receiving end to accept a list as
  well
  or just make multiple calls to
  self.update_service_capabilities(capabilities)
 
 
  b) make a few minor changes to the scheduler to make sure filtering
  still works. Note the changes here may be very helpful:
 
 
  https://review.openstack.org/10327
 
 
  c) we have to make sure that instances launched on those nodes take
  up
  the entire host state somehow. We could probably do this by making
  sure that the instance_type ram, mb, gb etc. matches what the node
  has, but we may want a new boolean field used if those aren't
  sufficient.
 
 
  I This approach seems pretty good. We could potentially get rid of
  the
  shared bare_metal_node table. I guess the only other concern is how
  you populate the capabilities that the bare metal nodes are
  reporting.
  I guess an api extension that rpcs to a baremetal node to add the
  node. Maybe someday this could be autogenerated by the bare metal
  host
  looking in its arp table for dhcp requests! :)
 
 
  Vish
 
  ___
  OpenStack-dev mailing list
  openstack-...@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
  ___
  OpenStack-dev mailing list
  openstack-...@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 ___
 OpenStack-dev mailing list
 openstack-...@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe :

51 matches

Mail list logo