Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

2016-11-29 Thread Fabian Grünbichler 
On Tue, Nov 29, 2016 at 09:26:50AM +0100, Dietmar Maurer wrote:
> > I think this would also be the first change to the Storage plugin API
> > that would warrant bumping its version (in PVE/Storage.pm:37)..
> 
> Why exactly? I think this change is fully backward compatible?

my initial reaction was probably overboard here - as long as Plugin.pm
does not implement it external plugins can just fallback to the default
and it should be caught by has_feature checks anyway.

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

2016-11-29 Thread Dietmar Maurer 
> I think this would also be the first change to the Storage plugin API
> that would warrant bumping its version (in PVE/Storage.pm:37)..

Why exactly? I think this change is fully backward compatible?

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

2016-11-28 Thread Fabian Grünbichler 
On Mon, Nov 28, 2016 at 09:21:02AM +0100, Dietmar Maurer wrote:
> 
> 
> > On November 28, 2016 at 8:26 AM Alexandre DERUMIER 
> > wrote:
> > 
> > 
> > >>I think it is basically a good idea, but we first need 
> > >>to implement "copy_image" for all (common) storage types. 
> > 
> > ok, I'll try to implement some storage
> > 
> > 
> > >>If it does not make sense, the drivers should raise an 
> > >>exception. 
> > 
> > do we need to raise an exception, if we define has_feature(storagecopy)  ?
> 
> Yes, just as additional protection.

I think this would also be the first change to the Storage plugin API
that would warrant bumping its version (in PVE/Storage.pm:37)..

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

2016-11-28 Thread Dietmar Maurer 


> On November 28, 2016 at 8:26 AM Alexandre DERUMIER 
> wrote:
> 
> 
> >>I think it is basically a good idea, but we first need 
> >>to implement "copy_image" for all (common) storage types. 
> 
> ok, I'll try to implement some storage
> 
> 
> >>If it does not make sense, the drivers should raise an 
> >>exception. 
> 
> do we need to raise an exception, if we define has_feature(storagecopy)  ?

Yes, just as additional protection.

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

2016-11-27 Thread Alexandre DERUMIER 
>>I think it is basically a good idea, but we first need 
>>to implement "copy_image" for all (common) storage types. 

ok, I'll try to implement some storage


>>If it does not make sense, the drivers should raise an 
>>exception. 

do we need to raise an exception, if we define has_feature(storagecopy)  ?


- Mail original -
De: "dietmar" <diet...@proxmox.com>
À: "aderumier" <aderum...@odiso.com>
Cc: "pve-devel" <pve-devel@pve.proxmox.com>
Envoyé: Samedi 26 Novembre 2016 07:30:04
Objet: Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

> any news to apply this patch ? 

I think it is basically a good idea, but we first need 
to implement "copy_image" for all (common) storage types. 
If it does not make sense, the drivers should raise an 
exception. 


> 
> - Mail original - 
> De: "dietmar" <diet...@proxmox.com> 
> À: "aderumier" <aderum...@odiso.com> 
> Cc: "pve-devel" <pve-devel@pve.proxmox.com> 
> Envoyé: Mardi 15 Novembre 2016 19:36:40 
> Objet: Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin 
> 
> > >>To be more precise, on lvmthin a copy is the same as 
> > >>a snapshot or a linked clone. We just copy metadata. 
> > 
> > but you can remove the parent image ? 
> 
> yes, I think that works. 
> 
> > if yes, same thing apply to sheepdog clones. 
> 
> 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

2016-11-25 Thread Dietmar Maurer 
> any news to apply this patch ?

I think it is basically a good idea, but we first need
to implement "copy_image" for all (common) storage types. 
If it does not make sense, the drivers should raise an
exception.


> 
> - Mail original -
> De: "dietmar" <diet...@proxmox.com>
> À: "aderumier" <aderum...@odiso.com>
> Cc: "pve-devel" <pve-devel@pve.proxmox.com>
> Envoyé: Mardi 15 Novembre 2016 19:36:40
> Objet: Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin
> 
> > >>To be more precise, on lvmthin a copy is the same as 
> > >>a snapshot or a linked clone. We just copy metadata. 
> > 
> > but you can remove the parent image ? 
> 
> yes, I think that works. 
> 
> > if yes, same thing apply to sheepdog clones. 
> 
> 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

2016-11-24 Thread Alexandre DERUMIER 
Hi,

any news to apply this patch ?

- Mail original -
De: "dietmar" <diet...@proxmox.com>
À: "aderumier" <aderum...@odiso.com>
Cc: "pve-devel" <pve-devel@pve.proxmox.com>
Envoyé: Mardi 15 Novembre 2016 19:36:40
Objet: Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

> >>To be more precise, on lvmthin a copy is the same as 
> >>a snapshot or a linked clone. We just copy metadata. 
> 
> but you can remove the parent image ? 

yes, I think that works. 

> if yes, same thing apply to sheepdog clones. 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

2016-11-15 Thread Dietmar Maurer 
> >>To be more precise, on lvmthin a copy is the same as
> >>a snapshot or a linked clone. We just copy metadata.
> 
> but you can remove the parent image ?

yes, I think that works.

> if yes, same thing apply to sheepdog clones.

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

2016-11-15 Thread Michael Rasmussen 
On Tue, 15 Nov 2016 18:43:16 +0100 (CET)
Alexandre DERUMIER  wrote:

> >>To be more precise, on lvmthin a copy is the same as
> >>a snapshot or a linked clone. We just copy metadata.  
> 
> but you can remove the parent image ?
> 
> if yes, same thing apply to sheepdog clones.
> 
Maybe works like hard links in POSIX file systems. Eg. actual data is
first removed when reference counter < 1.

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael  rasmussen  cc
http://pgp.mit.edu:11371/pks/lookup?op=get=0xD3C9A00E
mir  datanom  net
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE501F51C
mir  miras  org
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE3E80917
--
/usr/games/fortune -es says:
Great American Axiom:
Some is good, more is better, too much is just right.


pgpMz8_6RUNDl.pgp
Description: OpenPGP digital signature
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

2016-11-15 Thread Alexandre DERUMIER 
>>To be more precise, on lvmthin a copy is the same as
>>a snapshot or a linked clone. We just copy metadata.

but you can remove the parent image ?

if yes, same thing apply to sheepdog clones.

- Mail original -
De: "dietmar" <diet...@proxmox.com>
À: "aderumier" <aderum...@odiso.com>
Cc: "pve-devel" <pve-devel@pve.proxmox.com>
Envoyé: Mardi 15 Novembre 2016 16:20:13
Objet: Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

> >>Yes, because it is much faster than a copy. Why not? 
> ok. I think it was same speed. If it's faster ,yes :) 

To be more precise, on lvmthin a copy is the same as 
a snapshot or a linked clone. We just copy metadata. 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

2016-11-15 Thread Alexandre DERUMIER 
>>Yes, because it is much faster than a copy. Why not? 
ok. I think it was same speed. If it's faster ,yes :)

- Mail original -
De: "dietmar" <diet...@proxmox.com>
À: "aderumier" <aderum...@odiso.com>
Cc: "pve-devel" <pve-devel@pve.proxmox.com>
Envoyé: Mardi 15 Novembre 2016 11:35:07
Objet: Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

> >>What other storages support this? 
> for zfs, using zfs send|receive through ssh on the remote storage 
> 
> 
> >>I guess it is easy to implement with lvm-thin? 
> 
> I don't known . Does it make sense for local storage ? 

Yes, because it is much faster than a copy. Why not? 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

2016-11-15 Thread Wolfgang Bumiller 
On Tue, Nov 15, 2016 at 10:37:38AM +0100, Alexandre DERUMIER wrote:
> >>What other storages support this?
> for zfs, using zfs send|receive through ssh on the remote storage
> 
> 
> >>I guess it is easy to implement with lvm-thin?
> 
> I don't known . Does it make sense for local storage ?

mh... `cp --sparse=always --reflink=auto` could be a nice default...
on btrfs you'd get COW-clones then ;-)

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

2016-11-15 Thread Dietmar Maurer 
> >>What other storages support this?
> for zfs, using zfs send|receive through ssh on the remote storage
> 
> 
> >>I guess it is easy to implement with lvm-thin?
> 
> I don't known . Does it make sense for local storage ?

Yes, because it is much faster than a copy. Why not?

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

2016-11-15 Thread Alexandre DERUMIER 
>>What other storages support this?
for zfs, using zfs send|receive through ssh on the remote storage


>>I guess it is easy to implement with lvm-thin?

I don't known . Does it make sense for local storage ?


- Mail original -
De: "dietmar" <diet...@proxmox.com>
À: "aderumier" <aderum...@odiso.com>, "pve-devel" <pve-devel@pve.proxmox.com>
Envoyé: Mardi 15 Novembre 2016 09:23:56
Objet: Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

What other storages support this? I guess it 
is easy to implement with lvm-thin? Any other types? 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

2016-11-15 Thread Dietmar Maurer 
What other storages support this? I guess it 
is easy to implement with lvm-thin? Any other types?

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH] add new storagecopy feature && add rbdplugin

2016-11-14 Thread Alexandre Derumier 
This allow fast volume copy from storage to same storage if storage support it.

Signed-off-by: Alexandre Derumier 
---
 PVE/Storage.pm   | 18 ++
 PVE/Storage/RBDPlugin.pm | 22 ++
 2 files changed, 40 insertions(+)

diff --git a/PVE/Storage.pm b/PVE/Storage.pm
index a904f4e..bbccc6c 100755
--- a/PVE/Storage.pm
+++ b/PVE/Storage.pm
@@ -651,6 +651,24 @@ sub vdisk_clone {
 });
 }
 
+sub vdisk_copy {
+my ($cfg, $volid, $vmid, $snap) = @_;
+
+my ($storeid, $volname) = parse_volume_id($volid);
+
+my $scfg = storage_config($cfg, $storeid);
+
+my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
+
+activate_storage($cfg, $storeid);
+
+# lock shared storage
+return $plugin->cluster_lock_storage($storeid, $scfg->{shared}, undef, sub 
{
+   my $volname = $plugin->copy_image($scfg, $storeid, $volname, $vmid, 
$snap);
+   return "$storeid:$volname";
+});
+}
+
 sub vdisk_create_base {
 my ($cfg, $volid) = @_;
 
diff --git a/PVE/Storage/RBDPlugin.pm b/PVE/Storage/RBDPlugin.pm
index 6459f98..9aac701 100644
--- a/PVE/Storage/RBDPlugin.pm
+++ b/PVE/Storage/RBDPlugin.pm
@@ -440,6 +440,27 @@ sub clone_image {
 return $newvol;
 }
 
+sub copy_image {
+my ($class, $scfg, $storeid, $volname, $vmid, $snapname) = @_;
+
+my $name = &$find_free_diskname($storeid, $scfg, $vmid);
+
+my $newvol = $name;
+
+$volname = $volname."@".$snapname if $snapname;
+
+warn "clone $volname to $name\n";
+
+my $cmd = &$rbd_cmd($scfg, $storeid, 'copy', &$add_pool_to_disk($scfg, 
$volname), 
+   &$add_pool_to_disk($scfg, $name));
+
+run_rbd_command($cmd, errmsg => "rbd copy '$volname' error");
+
+&$krdb_feature_disable($scfg, $storeid, $name);
+
+return $newvol;
+}
+
 sub alloc_image {
 my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_;
 
@@ -664,6 +685,7 @@ sub volume_has_feature {
clone => { base => 1, snap => 1},
template => { current => 1},
copy => { base => 1, current => 1, snap => 1},
+   storagecopy => { base => 1, current => 1, snap => 1},
sparseinit => { base => 1, current => 1},
 };
 
-- 
2.1.4

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel