Re: Git packs friendly to block-level deduplication

On Wed, Jan 24, 2018 at 07:10:15PM -0500, Jeff King wrote: > On Thu, Jan 25, 2018 at 01:03:25AM +0100, Ævar Arnfjörð Bjarmason wrote: > > > > You may also want to use "--threads=1" to avoid non-determinism in the > > > generated packs. In theory, both repos would then produce identical base > >

Re: Git packs friendly to block-level deduplication

On Thu, Jan 25, 2018 at 01:03:25AM +0100, Ævar Arnfjörð Bjarmason wrote: > > You may also want to use "--threads=1" to avoid non-determinism in the > > generated packs. In theory, both repos would then produce identical base > > packs, though it does not seem to do so in practice (I didn't dig in

Re: Git packs friendly to block-level deduplication

On Wed, Jan 24 2018, Jeff King jotted: > On Wed, Jan 24, 2018 at 11:03:47PM +0100, Ævar Arnfjörð Bjarmason wrote: > >> This produces a total of 0 blocks that are the same. If after the repack >> we throw this in there after the repack: >> >> echo 5be1f00a9a | git pack-objects

Re: Git packs friendly to block-level deduplication

On Thu, Jan 25, 2018 at 12:06:59AM +0100, Ævar Arnfjörð Bjarmason wrote: > >> Has anyone here barked up this tree before? Suggestions? Tips on where > >> to start hacking the repack code to accomplish this would be most > >> welcome. > > > > Does this overlap with the desire to have resumable

Re: Git packs friendly to block-level deduplication

On Wed, Jan 24, 2018 at 11:03:47PM +0100, Ævar Arnfjörð Bjarmason wrote: > This produces a total of 0 blocks that are the same. If after the repack > we throw this in there after the repack: > > echo 5be1f00a9a | git pack-objects --no-reuse-delta --no-reuse-object > --revs

Re: Git packs friendly to block-level deduplication

On Wed, Jan 24 2018, Elijah Newren jotted: > On Wed, Jan 24, 2018 at 2:03 PM, Ævar Arnfjörð Bjarmason > wrote: >> If you have a bunch of git repositories cloned of the same project on >> the same filesystem, it would be nice of the packs that are produced >> would be friendly

Re: Git packs friendly to block-level deduplication

On Wed, Jan 24 2018, Junio C. Hamano jotted: > Mike Hommey writes: > >> FWIW, I sidestep the problem entirely by using alternatives. > > That's a funny way to use the word "side-step", I would say, as the > alternate object store support is there exactly for this use case.

Re: Git packs friendly to block-level deduplication

On Wed, Jan 24, 2018 at 2:03 PM, Ævar Arnfjörð Bjarmason wrote: > If you have a bunch of git repositories cloned of the same project on > the same filesystem, it would be nice of the packs that are produced > would be friendly to block-level deduplication. > > This would save

Re: Git packs friendly to block-level deduplication

On Wed, Jan 24, 2018 at 02:23:57PM -0800, Junio C Hamano wrote: > Mike Hommey writes: > > > FWIW, I sidestep the problem entirely by using alternatives. > > That's a funny way to use the word "side-step", I would say, as the > alternate object store support is there exactly

Re: Git packs friendly to block-level deduplication

Ævar Arnfjörð Bjarmason wrote: > If you have a bunch of git repositories cloned of the same project on > the same filesystem, it would be nice of the packs that are produced > would be friendly to block-level deduplication. Fwiw, I currently get around this when mirroring by

Re: Git packs friendly to block-level deduplication

Mike Hommey writes: > FWIW, I sidestep the problem entirely by using alternatives. That's a funny way to use the word "side-step", I would say, as the alternate object store support is there exactly for this use case.

Re: Git packs friendly to block-level deduplication

On Wed, Jan 24, 2018 at 11:03:47PM +0100, Ævar Arnfjörð Bjarmason wrote: > If you have a bunch of git repositories cloned of the same project on > the same filesystem, it would be nice of the packs that are produced > would be friendly to block-level deduplication. > > This would save space, and

Git packs friendly to block-level deduplication

If you have a bunch of git repositories cloned of the same project on the same filesystem, it would be nice of the packs that are produced would be friendly to block-level deduplication. This would save space, and the blocks would be more likely to be in cache when you access them, likely