Re: [openstack-dev] [Nova][Manila] Nova API to attach/detach file volume/share

2015-10-28 Thread Thomas Bechtold 
On Mon, 2015-10-26 at 22:13 -0700, Sage Weil wrote:
[snipped]
> If you're at the summit, I'll be talking a bit about this on
> Wednesday at 
> 4:40 (http://sched.co/4A03).

The link doesn't work for me. Can you resent or post the room?

TIA

Tom

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Manila] Nova API to attach/detach file volume/share

2015-10-27 Thread Shinobu Kinjo 
Sorry, using vsock is:

-1

Need more discussion.

On Tue, Oct 27, 2015 at 2:13 PM, Sage Weil  wrote:

> On Mon, 26 Oct 2015, Ben Swartzlander wrote:
> > On 10/25/2015 01:25 PM, Sage Weil wrote:
> > > Hi everyone,
> > >
> > > We're currently working on an improved mechanism for plumbing file
> access
> > > into a VM or container and in most cases some interaction and
> > > configuration on the hypervisor/host is required to make it happen.
> The
> > > goal is to create something that generalizes beyond the current NFS and
> > > CIFS-only assumptions that are baked into the current crop of Manila
> > > drivers.
> > >
> > > The current target is to use the new VSOCK zero-configuration sockets,
> > > which I will be talking about at the summit on Wednesday:
> > >
> > >   - mount distributed fs on host, export via knfsd to guest over VSOCK
> > >   - run Ganesha with distributed fs FSAL on host, export to guest over
> VSOCK
> >
> > NFS-over-vsock is a VERY interesting idea. I understood that it wasn't
> > implemented yet in Linux... If that's no longer true then I'm excited to
> find
> > a way to make it work.
>
> There is a lot of enabling work, but Stefan Hajnoczi has patches in flight
> for qemu and the Linux NFS client+server, and Matt Benjamin is adding
> Ganesha support and will work on getting the needed nfs-utils changes in
> place.  It'll take a bit for this all to land upstream and in supported
> distros, but we think the overall stack in very promising!
>
> The challenge, I suspect, will be getting Manila and Nova to orchestrate
> it properly...
>
> If you're at the summit, I'll be talking a bit about this on Wednesday at
> 4:40 (http://sched.co/4A03).
>
> sage
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Email:
 - shin...@linux.com 
Blog:
 - Life with Distributed Computational System based on OpenSource

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Manila] Nova API to attach/detach file volume/share

2015-10-27 Thread Steve Gordon 
- Original Message -
> From: "Shinobu Kinjo" 
> To: "OpenStack Development Mailing List (not for usage questions)" 
> 
> 
> Sorry, using vsock is:
> 
> -1
> 
> Need more discussion.

Can you elaborate?

-Steve

> On Tue, Oct 27, 2015 at 2:13 PM, Sage Weil  wrote:
> 
> > On Mon, 26 Oct 2015, Ben Swartzlander wrote:
> > > On 10/25/2015 01:25 PM, Sage Weil wrote:
> > > > Hi everyone,
> > > >
> > > > We're currently working on an improved mechanism for plumbing file
> > access
> > > > into a VM or container and in most cases some interaction and
> > > > configuration on the hypervisor/host is required to make it happen.
> > The
> > > > goal is to create something that generalizes beyond the current NFS and
> > > > CIFS-only assumptions that are baked into the current crop of Manila
> > > > drivers.
> > > >
> > > > The current target is to use the new VSOCK zero-configuration sockets,
> > > > which I will be talking about at the summit on Wednesday:
> > > >
> > > >   - mount distributed fs on host, export via knfsd to guest over VSOCK
> > > >   - run Ganesha with distributed fs FSAL on host, export to guest over
> > VSOCK
> > >
> > > NFS-over-vsock is a VERY interesting idea. I understood that it wasn't
> > > implemented yet in Linux... If that's no longer true then I'm excited to
> > find
> > > a way to make it work.
> >
> > There is a lot of enabling work, but Stefan Hajnoczi has patches in flight
> > for qemu and the Linux NFS client+server, and Matt Benjamin is adding
> > Ganesha support and will work on getting the needed nfs-utils changes in
> > place.  It'll take a bit for this all to land upstream and in supported
> > distros, but we think the overall stack in very promising!
> >
> > The challenge, I suspect, will be getting Manila and Nova to orchestrate
> > it properly...
> >
> > If you're at the summit, I'll be talking a bit about this on Wednesday at
> > 4:40 (http://sched.co/4A03).
> >
> > sage
> >
> > __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> 
> 
> 
> --
> Email:
>  - shin...@linux.com 
> Blog:
>  - Life with Distributed Computational System based on OpenSource
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

-- 
Steve Gordon, RHCE
Sr. Technical Product Manager,
Red Hat Enterprise Linux OpenStack Platform

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Manila] Nova API to attach/detach file volume/share

2015-10-27 Thread Shinobu Kinjo 
Yes, I will.

On Wed, Oct 28, 2015 at 4:37 AM, Steve Gordon  wrote:

> - Original Message -
> > From: "Shinobu Kinjo" 
> > To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> >
> > Sorry, using vsock is:
> >
> > -1
> >
> > Need more discussion.
>
> Can you elaborate?
>
> -Steve
>
> > On Tue, Oct 27, 2015 at 2:13 PM, Sage Weil  wrote:
> >
> > > On Mon, 26 Oct 2015, Ben Swartzlander wrote:
> > > > On 10/25/2015 01:25 PM, Sage Weil wrote:
> > > > > Hi everyone,
> > > > >
> > > > > We're currently working on an improved mechanism for plumbing file
> > > access
> > > > > into a VM or container and in most cases some interaction and
> > > > > configuration on the hypervisor/host is required to make it happen.
> > > The
> > > > > goal is to create something that generalizes beyond the current
> NFS and
> > > > > CIFS-only assumptions that are baked into the current crop of
> Manila
> > > > > drivers.
> > > > >
> > > > > The current target is to use the new VSOCK zero-configuration
> sockets,
> > > > > which I will be talking about at the summit on Wednesday:
> > > > >
> > > > >   - mount distributed fs on host, export via knfsd to guest over
> VSOCK
> > > > >   - run Ganesha with distributed fs FSAL on host, export to guest
> over
> > > VSOCK
> > > >
> > > > NFS-over-vsock is a VERY interesting idea. I understood that it
> wasn't
> > > > implemented yet in Linux... If that's no longer true then I'm
> excited to
> > > find
> > > > a way to make it work.
> > >
> > > There is a lot of enabling work, but Stefan Hajnoczi has patches in
> flight
> > > for qemu and the Linux NFS client+server, and Matt Benjamin is adding
> > > Ganesha support and will work on getting the needed nfs-utils changes
> in
> > > place.  It'll take a bit for this all to land upstream and in supported
> > > distros, but we think the overall stack in very promising!
> > >
> > > The challenge, I suspect, will be getting Manila and Nova to
> orchestrate
> > > it properly...
> > >
> > > If you're at the summit, I'll be talking a bit about this on Wednesday
> at
> > > 4:40 (http://sched.co/4A03).
> > >
> > > sage
> > >
> > >
> __
> > > OpenStack Development Mailing List (not for usage questions)
> > > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >
> >
> >
> >
> > --
> > Email:
> >  - shin...@linux.com 
> > Blog:
> >  - Life with Distributed Computational System based on OpenSource
> > 
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> --
> Steve Gordon, RHCE
> Sr. Technical Product Manager,
> Red Hat Enterprise Linux OpenStack Platform
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Email:
 - shin...@linux.com 
Blog:
 - Life with Distributed Computational System based on OpenSource

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Manila] Nova API to attach/detach file volume/share

2015-10-27 Thread Sage Weil 
On Wed, 28 Oct 2015, Thomas Bechtold wrote:
> On Mon, 2015-10-26 at 22:13 -0700, Sage Weil wrote:
> [snipped]
> > If you're at the summit, I'll be talking a bit about this on
> > Wednesday at 
> > 4:40 (http://sched.co/4A03).
> 
> The link doesn't work for me. Can you resent or post the room?

Weird, maybe sched needs you to e logged in or something.

It's 4:40pm in Gyoko.

sage


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Manila] Nova API to attach/detach file volume/share

2015-10-27 Thread Thomas Bechtold 
On Mon, 2015-10-26 at 22:13 -0700, Sage Weil wrote:
[snipped]
> If you're at the summit, I'll be talking a bit about this on
> Wednesday at 
> 4:40 (http://sched.co/4A03).

The link doesn't work for me. Can you resent or post the room?

TIA

Tom

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Manila] Nova API to attach/detach file volume/share

2015-10-27 Thread Thomas Bechtold 
On Mon, 2015-10-26 at 22:13 -0700, Sage Weil wrote:
[snipped]
> If you're at the summit, I'll be talking a bit about this on
> Wednesday at 
> 4:40 (http://sched.co/4A03).

The link doesn't work for me. Can you resent or post the room?

TIA

Tom

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Manila] Nova API to attach/detach file volume/share

2015-10-26 Thread Ben Swartzlander 

On 10/25/2015 01:25 PM, Sage Weil wrote:

Hi everyone,

We're currently working on an improved mechanism for plumbing file access
into a VM or container and in most cases some interaction and
configuration on the hypervisor/host is required to make it happen.  The
goal is to create something that generalizes beyond the current NFS and
CIFS-only assumptions that are baked into the current crop of Manila
drivers.

The current target is to use the new VSOCK zero-configuration sockets,
which I will be talking about at the summit on Wednesday:

  - mount distributed fs on host, export via knfsd to guest over VSOCK
  - run Ganesha with distributed fs FSAL on host, export to guest over VSOCK


NFS-over-vsock is a VERY interesting idea. I understood that it wasn't 
implemented yet in Linux... If that's no longer true then I'm excited to 
find a way to make it work.



(In short, VSOCK requires no configuration on the guest--so we can
continue to treat it as a black box--and only a simple network id
assignment on the host.  This reduces driver complexity, reduces the
potential for user error, and improves security.)

Or, the same can be done using a more configuration-intensive private
network:

  - mount distributed fs on host, export via knfsd to guest over private
host/guest network
  - run ganesha with distributed fs FSAL on host, export to guest over
private host/guest network

We are also interested in container use-cases (e.g., for Nova-managed
containers using lxc or nova-docker):

  - mount distributed fs on host, bind mount a volume/directory into the
guest/container namespace

 From a plumbing perspective, these approaches appear to be the most
attractive as far as efficiency and security.  When it comes to making
openstack actually orchestrate it, however, things get complicated (and
as a developer in a peripheral project I don't have the clearest view of
how things should work).

The problem is that in any of these scenarios, the default/naive strategy
of having Manila go poking around on the host machine is very fragile:

  - What if the host is down when the Manila configuration is made?
  - What if the VM is migrated to another host?
  - What if Manila doesn't understand what (the) Nova (driver) is doing?

For block volumes, there is a simple separation of roles: Cinder manages
the volumes themselves, and Nova attaches them to guests.  This means
there is some volume-related code in Nova, but it also means that that
code is running in a context where it can Do The Right Thing.  I believe
the same is true of Manila file volumes as well... as soon as
you look beyond the current assumption that all volumes must be
reached via a network file protocol (and usually a Neutron
network).

That is, we would love to see a new set of Nova APIs to attach and detach
a manila fs to the guest, along with a "get share info" call that returns
any details required for the user/tenant to do the final mount within the
guest.

This enables our new topologies:

  - Attach API might mount CephFS on the host and exportfs it over VSOCK to
the guest VM.  Get-info API would direct the user to mount -t nfs vsock:1/
with -o nfsvers=4.1,proto=vsock,clientaddr=N ..
  - Attach API might configure/start a Ganesha instance doing the same
  - Attach API might mount CephFS on the host and then mount --bind the
share to $guestroot/dev/manila/$shareid.  Get-info API would direct the
user to mount --bind /dev/manila/$shareid.

I suspect it would also clean up some existing Manila topologies (though
I'm not sure).  For example, the current the NFS-based Manila drivers will
help you configure a Neutron net that can access the storage server.  The
attach API might attach the guest to that same network.  (Maybe.. I'm
super fuzzy on how this stuff works!)

What do people (especially the Nova folks) think about this?  I would love
sit down with anyone to talk about this this week in Tokyo on either/both
the Nova and Manila side of the picture.

Thanks!
sage

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Manila] Nova API to attach/detach file volume/share

2015-10-26 Thread Sage Weil 
On Mon, 26 Oct 2015, Ben Swartzlander wrote:
> On 10/25/2015 01:25 PM, Sage Weil wrote:
> > Hi everyone,
> > 
> > We're currently working on an improved mechanism for plumbing file access
> > into a VM or container and in most cases some interaction and
> > configuration on the hypervisor/host is required to make it happen.  The
> > goal is to create something that generalizes beyond the current NFS and
> > CIFS-only assumptions that are baked into the current crop of Manila
> > drivers.
> > 
> > The current target is to use the new VSOCK zero-configuration sockets,
> > which I will be talking about at the summit on Wednesday:
> > 
> >   - mount distributed fs on host, export via knfsd to guest over VSOCK
> >   - run Ganesha with distributed fs FSAL on host, export to guest over VSOCK
> 
> NFS-over-vsock is a VERY interesting idea. I understood that it wasn't
> implemented yet in Linux... If that's no longer true then I'm excited to find
> a way to make it work.

There is a lot of enabling work, but Stefan Hajnoczi has patches in flight 
for qemu and the Linux NFS client+server, and Matt Benjamin is adding 
Ganesha support and will work on getting the needed nfs-utils changes in 
place.  It'll take a bit for this all to land upstream and in supported 
distros, but we think the overall stack in very promising!

The challenge, I suspect, will be getting Manila and Nova to orchestrate 
it properly...

If you're at the summit, I'll be talking a bit about this on Wednesday at 
4:40 (http://sched.co/4A03).

sage

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Nova][Manila] Nova API to attach/detach file volume/share

2015-10-25 Thread Sage Weil 
Hi everyone,

We're currently working on an improved mechanism for plumbing file access 
into a VM or container and in most cases some interaction and 
configuration on the hypervisor/host is required to make it happen.  The 
goal is to create something that generalizes beyond the current NFS and 
CIFS-only assumptions that are baked into the current crop of Manila 
drivers.

The current target is to use the new VSOCK zero-configuration sockets, 
which I will be talking about at the summit on Wednesday:

 - mount distributed fs on host, export via knfsd to guest over VSOCK 
 - run Ganesha with distributed fs FSAL on host, export to guest over VSOCK

(In short, VSOCK requires no configuration on the guest--so we can 
continue to treat it as a black box--and only a simple network id 
assignment on the host.  This reduces driver complexity, reduces the 
potential for user error, and improves security.)

Or, the same can be done using a more configuration-intensive private 
network:

 - mount distributed fs on host, export via knfsd to guest over private 
   host/guest network
 - run ganesha with distributed fs FSAL on host, export to guest over 
   private host/guest network

We are also interested in container use-cases (e.g., for Nova-managed 
containers using lxc or nova-docker):

 - mount distributed fs on host, bind mount a volume/directory into the 
guest/container namespace

>From a plumbing perspective, these approaches appear to be the most 
attractive as far as efficiency and security.  When it comes to making 
openstack actually orchestrate it, however, things get complicated (and 
as a developer in a peripheral project I don't have the clearest view of 
how things should work).

The problem is that in any of these scenarios, the default/naive strategy 
of having Manila go poking around on the host machine is very fragile:

 - What if the host is down when the Manila configuration is made?
 - What if the VM is migrated to another host?
 - What if Manila doesn't understand what (the) Nova (driver) is doing?

For block volumes, there is a simple separation of roles: Cinder manages 
the volumes themselves, and Nova attaches them to guests.  This means 
there is some volume-related code in Nova, but it also means that that 
code is running in a context where it can Do The Right Thing.  I believe 
the same is true of Manila file volumes as well... as soon as 
you look beyond the current assumption that all volumes must be 
reached via a network file protocol (and usually a Neutron 
network).

That is, we would love to see a new set of Nova APIs to attach and detach 
a manila fs to the guest, along with a "get share info" call that returns 
any details required for the user/tenant to do the final mount within the 
guest.

This enables our new topologies:

 - Attach API might mount CephFS on the host and exportfs it over VSOCK to 
the guest VM.  Get-info API would direct the user to mount -t nfs vsock:1/ 
with -o nfsvers=4.1,proto=vsock,clientaddr=N ..
 - Attach API might configure/start a Ganesha instance doing the same
 - Attach API might mount CephFS on the host and then mount --bind the 
share to $guestroot/dev/manila/$shareid.  Get-info API would direct the 
user to mount --bind /dev/manila/$shareid.

I suspect it would also clean up some existing Manila topologies (though 
I'm not sure).  For example, the current the NFS-based Manila drivers will 
help you configure a Neutron net that can access the storage server.  The 
attach API might attach the guest to that same network.  (Maybe.. I'm 
super fuzzy on how this stuff works!)

What do people (especially the Nova folks) think about this?  I would love 
sit down with anyone to talk about this this week in Tokyo on either/both 
the Nova and Manila side of the picture.

Thanks!
sage

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev