Re: [PATCH/RFC] index-pack: produce pack index version 3

2012-08-16 Thread Shawn Pearce
On Wed, Aug 15, 2012 at 10:42 PM, Junio C Hamano gits...@pobox.com wrote: Shawn Pearce spea...@spearce.org writes: ... But I think its worth giving him a few weeks to finish getting the code ready, vs. rushing something in that someone else thinks might help. We have waited more than 6 years

Re: [PATCH/RFC] index-pack: produce pack index version 3

2012-08-15 Thread Junio C Hamano
Shawn Pearce spea...@spearce.org writes: ... But I think its worth giving him a few weeks to finish getting the code ready, vs. rushing something in that someone else thinks might help. We have waited more than 6 years or whatever to improve packing. Colby's experiments are showing massive

Re: [PATCH/RFC] index-pack: produce pack index version 3

2012-08-13 Thread Junio C Hamano
Nguyen Thai Ngoc Duy pclo...@gmail.com writes: On Mon, Aug 13, 2012 at 2:49 AM, Junio C Hamano gits...@pobox.com wrote: For example, the reachability bitmap would want to say something like Traversing from commit A, these objects in this pack are reachable. The bitmap for one commit A would

Re: [PATCH/RFC] index-pack: produce pack index version 3

2012-08-13 Thread Shawn Pearce
Let me start by echoing Junio's remark... lacks sufficient justification. You don't give enough evidence to support even why it is worth looking at this commit, let alone why it should be included and cause a format change in the idx file format. At some point you start to hand-wave about how it

Re: [PATCH/RFC] index-pack: produce pack index version 3

2012-08-13 Thread Nguyen Thai Ngoc Duy
On Tue, Aug 14, 2012 at 7:46 AM, Shawn Pearce spea...@spearce.org wrote: Colby is nearly done prototyping the bitmap reachability implementation in JGit and will release the code under the BSD license there soon. I can't promise when yet because Colby will soon be heading out for some (much

[PATCH/RFC] index-pack: produce pack index version 3

2012-08-12 Thread Nguyễn Thái Ngọc Duy
The sha-1-offset table in v3 is sorted by object type first, then sha-1 (as opposed to sorted by sha-1 only in v2). There are four fan-out tables, one for each object type. So to look for the offset of an object, we first locate the fan-out table based on object type, then do binary search in that

Re: [PATCH/RFC] index-pack: produce pack index version 3

2012-08-12 Thread Junio C Hamano
Nguyễn Thái Ngọc Duy pclo...@gmail.com writes: The main reason to group objects by type is to make it possible to create another sha1-something mapping for a particular object type, without wasting space for storing sha-1 keys again. For example, we can store commit caches, tree caches... at

Re: [PATCH/RFC] index-pack: produce pack index version 3

2012-08-12 Thread Nguyen Thai Ngoc Duy
On Mon, Aug 13, 2012 at 2:49 AM, Junio C Hamano gits...@pobox.com wrote: For example, the reachability bitmap would want to say something like Traversing from commit A, these objects in this pack are reachable. The bitmap for one commit A would logically consist of N bits for a packfile that