Re: [openstack-dev] [Openstack] [Swift] Erasure coding reconstructor doesn't work

2015-07-22 Thread Clay Gerrard
On Wed, Jul 22, 2015 at 12:37 PM, Luse, Paul E wrote: > > > Wrt why the replication code seems to work if you delete just a .data > no it doesn't -> https://gist.github.com/clayg/88950d77d25a441635e6 > forces a listing every 10 passes for some reason. Clay? > IIRC the every 10 passes trick

Re: [openstack-dev] [Openstack] [Swift] Erasure coding reconstructor doesn't work

2015-07-22 Thread Changbin Liu
Got it, thanks! Thanks Changbin On Wed, Jul 22, 2015 at 4:19 PM, Clay Gerrard wrote: > > > On Wed, Jul 22, 2015 at 12:24 PM, Changbin Liu > wrote: > >> >> But now I wonder: is it "by design" that EC does not handle an accidental >> deletion of just the data file? >> > > Well, the design goal

Re: [openstack-dev] [Openstack] [Swift] Erasure coding reconstructor doesn't work

2015-07-22 Thread Clay Gerrard
On Wed, Jul 22, 2015 at 12:24 PM, Changbin Liu wrote: > > But now I wonder: is it "by design" that EC does not handle an accidental > deletion of just the data file? > Well, the design goal was not "do not handle the accidental deletion of just the data file" - it was "make replication fast enou

Re: [openstack-dev] [Openstack] [Swift] Erasure coding reconstructor doesn't work

2015-07-22 Thread Luse, Paul E
[mailto:changbin@gmail.com] Sent: Wednesday, July 22, 2015 12:24 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Openstack] [Swift] Erasure coding reconstructor doesn't work Thanks, Paul and Clay. By "deleted one data fragment"

Re: [openstack-dev] [Openstack] [Swift] Erasure coding reconstructor doesn't work

2015-07-22 Thread Changbin Liu
Thanks, John. I am happy to try out Swift EC and will report bugs to launchpad if necessary. (The issue I found happens to be different from the ones listed in the bug list. See my other email) Thanks Changbin On Tue, Jul 21, 2015 at 5:05 PM, John Dickinson wrote: > Yes, it's supposed to w

Re: [openstack-dev] [Openstack] [Swift] Erasure coding reconstructor doesn't work

2015-07-22 Thread Changbin Liu
for usage questions) > *Subject:* Re: [openstack-dev] [Openstack] [Swift] Erasure coding > reconstructor doesn't work > > > > How did you "deleted one data fragment"? > > > > Like replication the EC consistency engine uses some sub directory hashing >

Re: [openstack-dev] [Openstack] [Swift] Erasure coding reconstructor doesn't work

2015-07-21 Thread Luse, Paul E
, July 21, 2015 2:22 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Openstack] [Swift] Erasure coding reconstructor doesn't work How did you "deleted one data fragment"? Like replication the EC consistency engine uses some sub direc

Re: [openstack-dev] [Openstack] [Swift] Erasure coding reconstructor doesn't work

2015-07-21 Thread Clay Gerrard
How did you "deleted one data fragment"? Like replication the EC consistency engine uses some sub directory hashing to accelerate replication requests in a consistent system - so if you just rm a file down in an hashdir somewhere you also need to delete the hashes.pkl up in the part dir (or call t

Re: [openstack-dev] [Openstack] [Swift] Erasure coding reconstructor doesn't work

2015-07-21 Thread John Dickinson
Yes, it's supposed to work, but you've run in to some errors we've been finding and fixing. Right now the top priority for the Swift dev community is to take care of the outstanding EC issues and make a release. The list of the known EC bugs right now is https://bugs.launchpad.net/swift/+bugs?f

[openstack-dev] [Openstack] [Swift] Erasure coding reconstructor doesn't work

2015-07-21 Thread Changbin Liu
Folks, To test the latest feature of Swift erasure coding, I followed this document ( http://docs.openstack.org/developer/swift/overview_erasure_code.html) to deploy a simple cluster. I used Swift 2.3.0. I am glad that operations like object PUT/GET/DELETE worked fine. I can see that objects were