Re: [Gluster-devel] [POC] disaster recovery: reconstruct all shards
That's why I'm asking for a native tool from gluster Il 1 mar 2017 2:46 PM, "Krutika Dhananjay" ha scritto: > The caveats are: > > 1. A non-existent/missing shard anywhere between offset $SHARD_BLOCK_SIZE > through ceiling ($FILE_SIZE/$SHARD_BLOCK_SIZE) > indicates a hole. When you reconstruct data from a sharded file of this > nature, you need to take care to retain this property. > > 2. The above is also true for partially filled shards between offset > $SHARD_BLOCK_SIZE through ceiling ($FILE_SIZE/$SHARD_BLOCK_SIZE). > What do I mean by partially filled shards? Shards whose sizes are not > equal to $SHARD_BLOCK_SIZE. > > In the above, $FILE_SIZE can be gotten from the > 'trusted.glusterfs.shard.file-size' extended attribute on the base file > (the 0th block). > > -Krutika > > On Mon, Feb 27, 2017 at 1:45 PM, Gandalf Corvotempesta < > gandalf.corvotempe...@gmail.com> wrote: > >> Which caveats? >> Anyway, having this recovery tool integrated in gluster could be an >> appreciable plus to guarantee data recovery natively >> >> Il 27 feb 2017 6:02 AM, "Krutika Dhananjay" ha >> scritto: >> >>> It should be possible to write a script that stitches the different >>> pieces of a single file together >>> (although with a few caveats). >>> >>> -Krutika >>> >>> On Sun, Feb 26, 2017 at 8:52 PM, Gandalf Corvotempesta < >>> gandalf.corvotempe...@gmail.com> wrote: >>> Would be possible to add a command to use in case of disaster recovery (where everything is broken) to recreate files from sharding ? In example, let's assume a totally down cluster. no trusted pools and so on but sysadmin knows which hdd is part of any distributed replica: hdd1 + hdd2 + hdd3 are distributed and replicated to hdd4 + hdd5 + hdd6 a CLI could traverse hdd1,hdd2,hdd3 and reconstruct all shards creating the original, unsharded file. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel >>> >>> > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] [POC] disaster recovery: reconstruct all shards
The caveats are: 1. A non-existent/missing shard anywhere between offset $SHARD_BLOCK_SIZE through ceiling ($FILE_SIZE/$SHARD_BLOCK_SIZE) indicates a hole. When you reconstruct data from a sharded file of this nature, you need to take care to retain this property. 2. The above is also true for partially filled shards between offset $SHARD_BLOCK_SIZE through ceiling ($FILE_SIZE/$SHARD_BLOCK_SIZE). What do I mean by partially filled shards? Shards whose sizes are not equal to $SHARD_BLOCK_SIZE. In the above, $FILE_SIZE can be gotten from the 'trusted.glusterfs.shard.file-size' extended attribute on the base file (the 0th block). -Krutika On Mon, Feb 27, 2017 at 1:45 PM, Gandalf Corvotempesta < gandalf.corvotempe...@gmail.com> wrote: > Which caveats? > Anyway, having this recovery tool integrated in gluster could be an > appreciable plus to guarantee data recovery natively > > Il 27 feb 2017 6:02 AM, "Krutika Dhananjay" ha > scritto: > >> It should be possible to write a script that stitches the different >> pieces of a single file together >> (although with a few caveats). >> >> -Krutika >> >> On Sun, Feb 26, 2017 at 8:52 PM, Gandalf Corvotempesta < >> gandalf.corvotempe...@gmail.com> wrote: >> >>> Would be possible to add a command to use in case of disaster recovery >>> (where everything is broken) to recreate files from sharding ? >>> >>> In example, let's assume a totally down cluster. no trusted pools and >>> so on but sysadmin knows which hdd is part of any distributed replica: >>> >>> hdd1 + hdd2 + hdd3 are distributed and replicated to hdd4 + hdd5 + hdd6 >>> >>> a CLI could traverse hdd1,hdd2,hdd3 and reconstruct all shards >>> creating the original, unsharded file. >>> ___ >>> Gluster-devel mailing list >>> Gluster-devel@gluster.org >>> http://lists.gluster.org/mailman/listinfo/gluster-devel >>> >> >> ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] [POC] disaster recovery: reconstruct all shards
Which caveats? Anyway, having this recovery tool integrated in gluster could be an appreciable plus to guarantee data recovery natively Il 27 feb 2017 6:02 AM, "Krutika Dhananjay" ha scritto: > It should be possible to write a script that stitches the different pieces > of a single file together > (although with a few caveats). > > -Krutika > > On Sun, Feb 26, 2017 at 8:52 PM, Gandalf Corvotempesta < > gandalf.corvotempe...@gmail.com> wrote: > >> Would be possible to add a command to use in case of disaster recovery >> (where everything is broken) to recreate files from sharding ? >> >> In example, let's assume a totally down cluster. no trusted pools and >> so on but sysadmin knows which hdd is part of any distributed replica: >> >> hdd1 + hdd2 + hdd3 are distributed and replicated to hdd4 + hdd5 + hdd6 >> >> a CLI could traverse hdd1,hdd2,hdd3 and reconstruct all shards >> creating the original, unsharded file. >> ___ >> Gluster-devel mailing list >> Gluster-devel@gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-devel >> > > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] [POC] disaster recovery: reconstruct all shards
It should be possible to write a script that stitches the different pieces of a single file together (although with a few caveats). -Krutika On Sun, Feb 26, 2017 at 8:52 PM, Gandalf Corvotempesta < gandalf.corvotempe...@gmail.com> wrote: > Would be possible to add a command to use in case of disaster recovery > (where everything is broken) to recreate files from sharding ? > > In example, let's assume a totally down cluster. no trusted pools and > so on but sysadmin knows which hdd is part of any distributed replica: > > hdd1 + hdd2 + hdd3 are distributed and replicated to hdd4 + hdd5 + hdd6 > > a CLI could traverse hdd1,hdd2,hdd3 and reconstruct all shards > creating the original, unsharded file. > ___ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-devel > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] [POC] disaster recovery: reconstruct all shards
Would be possible to add a command to use in case of disaster recovery (where everything is broken) to recreate files from sharding ? In example, let's assume a totally down cluster. no trusted pools and so on but sysadmin knows which hdd is part of any distributed replica: hdd1 + hdd2 + hdd3 are distributed and replicated to hdd4 + hdd5 + hdd6 a CLI could traverse hdd1,hdd2,hdd3 and reconstruct all shards creating the original, unsharded file. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
