Re: [ceph-users] export-diff behavior if an initial snapshot is NOT specified

2016-11-30 Thread Jason Dillaman
The underlying issue has existed forever AFAIK, but the issue has been masked since Infernalis due to the deep-flatten changes. If you were to use krbd or an older librbd client to write to the image, the same issue would appear (until the PR is merged and backported). On Nov 30, 2016, at 1:00 AM,

Re: [ceph-users] export-diff behavior if an initial snapshot is NOT specified

2016-11-29 Thread Zhongyan Gu
Thanks for the quick confirm and fix. I reviewed the pull request. Seems that this fix is going into Jewel and Hammer. So does this mean this issue also exists in Jewel? why I test Jewel and cannot reproduce this issue?? Zhongyan On Wed, Nov 30, 2016 at 12:32 PM, Venky Shankar wrote: > On 16-11

Re: [ceph-users] export-diff behavior if an initial snapshot is NOT specified

2016-11-29 Thread Venky Shankar
On 16-11-30 12:07:33, Zhongyan Gu wrote: > Jason, > I test Jewel and confirmed Jewel has no such issue. > Could you tell me what is the specific pull that can be backported to > hammer to fix this issue?? PR: https://github.com/ceph/ceph/pull/12218 > > Zhongyan > > On Tue, Nov 29, 2016 at 10:52

Re: [ceph-users] export-diff behavior if an initial snapshot is NOT specified

2016-11-29 Thread Zhongyan Gu
Jason, I test Jewel and confirmed Jewel has no such issue. Could you tell me what is the specific pull that can be backported to hammer to fix this issue?? Zhongyan On Tue, Nov 29, 2016 at 10:52 PM, Jason Dillaman wrote: > You are correct that there is an issue with the Hammer version when > ca

Re: [ceph-users] export-diff behavior if an initial snapshot is NOT specified

2016-11-29 Thread Jason Dillaman
You are correct that there is an issue with the Hammer version when calculating diffs. If the clone has an object that obscures an extent within the parent image but didn't exist within the first snapshot of the clone, the diff results from the parent image won't be included in the result. I'll ope

Re: [ceph-users] export-diff behavior if an initial snapshot is NOT specified

2016-11-28 Thread Zhongyan Gu
Thank you Jason. We are designing a backup system for production cluster based on ceph's export /import diff feature. We found this issue and hopefully it can be confirmed and then fixed soon. If you need any more information for debugging, Please just let me know. Thanks, Zhongyan On Mon, Nov 28

Re: [ceph-users] export-diff behavior if an initial snapshot is NOT specified

2016-11-28 Thread Jason Dillaman
OK, in that case, none of my previous explanation is relevant. I'll spin up a hammer cluster and try to reproduce. On Wed, Nov 23, 2016 at 9:13 PM, Zhongyan Gu wrote: > BTW, I used Hammer 0.94.5 to do the test. > > Zhongyan > > On Thu, Nov 24, 2016 at 10:07 AM, Zhongyan Gu wrote: >> >> Thank you

Re: [ceph-users] export-diff behavior if an initial snapshot is NOT specified

2016-11-23 Thread Zhongyan Gu
BTW, I used Hammer 0.94.5 to do the test. Zhongyan On Thu, Nov 24, 2016 at 10:07 AM, Zhongyan Gu wrote: > Thank you Jason. My test shows in the following case, image B will be > exactly same: > 1. clone image A from parent: > #rbd clone 1124-parent@snap1 A > > 2. create snap for A > #rbd snap c

Re: [ceph-users] export-diff behavior if an initial snapshot is NOT specified

2016-11-23 Thread Zhongyan Gu
Thank you Jason. My test shows in the following case, image B will be exactly same: 1. clone image A from parent: #rbd clone 1124-parent@snap1 A 2. create snap for A #rbd snap create A@snap1 3. create empty image B #rbd create B -s 1 4. export-diff A then impor-diff B: #rbd export-diff A@snap1 -

Re: [ceph-users] export-diff behavior if an initial snapshot is NOT specified

2016-11-23 Thread Jason Dillaman
What you are seeing sounds like a side-effect of deep-flatten support. If you write to an unallocated extent within a cloned image, the associated object extent must be read from the parent image, modified, and written to the clone image. Since the Infernalis release, this process has been tweaked

Re: [ceph-users] export-diff behavior if an initial snapshot is NOT specified

2016-11-23 Thread Zhongyan Gu
Let me make the issue more clear. Suppose I cloned image A from a parent image and create snap1 for image A and then make some change of image A. If I did the rbd export-diff @snap1. how should I prepare the existing image B to make sure it will be exactly same with image A@snap1 after import-dif

Re: [ceph-users] export-diff behavior if an initial snapshot is NOT specified

2016-11-22 Thread Zhongyan Gu
Thanks Jason, very clear explanation. However, I found some strange behavior when export-diff on a cloned image, not sure it is a bug on calc_snap_set_diff(). The test is, Image A is cloned from a parent image. then create snap1 for image A. The content of export-diff A@snap1 will be changed when u

Re: [ceph-users] export-diff behavior if an initial snapshot is NOT specified

2016-11-22 Thread Jason Dillaman
On Tue, Nov 22, 2016 at 5:31 AM, Zhongyan Gu wrote: > So if initial snapshot is NOT specified, then: > rbd export-diff image@snap1 will diff all data to snap1. this cmd equals to > : > rbd export image@snap1. Is my understand right or not?? While they will both export all data associated w/ imag

[ceph-users] export-diff behavior if an initial snapshot is NOT specified

2016-11-22 Thread Zhongyan Gu
Hi there, According to the official man page: http://docs.ceph.com/docs/jewel/man/8/rbd/ export-diff [–from-snap snap-name] [–whole-object] (image-spec | snap-spec) dest-path Exports an incremental diff for an image to dest path (use - for stdout). If an initial snapshot is specified, only changes