Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-12-05 Thread Jeff King
On Wed, Dec 05, 2018 at 07:41:44PM +0100, René Scharfe wrote: > > If we start to convert those, there's a > > little bit of a rabbit hole, but it's actually not too bad. > > You don't need to crawl in just for quick_has_loose(), but eventually > everything has to be converted. It seems a bit

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-12-05 Thread René Scharfe
Am 05.12.2018 um 09:15 schrieb Jeff King: > On Wed, Dec 05, 2018 at 01:51:36AM -0500, Jeff King wrote: > >>> This >>> function is easily converted to struct object_id, though, as its single >>> caller can pass one on -- this makes the copy unnecessary. >> >> If you mean modifying

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-12-05 Thread Jeff King
On Wed, Dec 05, 2018 at 01:51:36AM -0500, Jeff King wrote: > > This > > function is easily converted to struct object_id, though, as its single > > caller can pass one on -- this makes the copy unnecessary. > > If you mean modifying sha1_loose_object_info() to take an oid, then > sure, I agree

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-12-04 Thread Jeff King
On Wed, Dec 05, 2018 at 07:02:17AM +0100, René Scharfe wrote: > > I actually wrote it that way initially, but doing the hashcpy() in the > > caller is a bit more awkward. My thought was to punt on that until the > > rest of the surrounding code starts handling oids. > > The level of awkwardness

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-12-04 Thread René Scharfe
Am 05.12.2018 um 05:46 schrieb Jeff King: > On Tue, Dec 04, 2018 at 10:45:13PM +0100, René Scharfe wrote: > >>> The comment in the context there is warning callers to remember to load >>> the cache first. Now that we have individual caches, might it make sense >>> to change the interface a bit,

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-12-04 Thread Jeff King
On Tue, Dec 04, 2018 at 10:45:13PM +0100, René Scharfe wrote: > > The comment in the context there is warning callers to remember to load > > the cache first. Now that we have individual caches, might it make sense > > to change the interface a bit, and make these members private. I.e., > >

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-12-04 Thread René Scharfe
Am 03.12.2018 um 23:04 schrieb Jeff King: > On Sun, Dec 02, 2018 at 11:52:50AM +0100, René Scharfe wrote: > >>> And for mu.git, a ~20k object repo: >>> >>> Test origin/master >>> peff/jk/loose-cache avar/check-collisions-config >>>

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-12-03 Thread Jeff King
On Sun, Dec 02, 2018 at 11:52:50AM +0100, René Scharfe wrote: > > And for mu.git, a ~20k object repo: > > > > Test origin/master > > peff/jk/loose-cache avar/check-collisions-config > > > >

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-12-02 Thread René Scharfe
Am 13.11.2018 um 11:02 schrieb Ævar Arnfjörð Bjarmason: > > On Mon, Nov 12 2018, Ævar Arnfjörð Bjarmason wrote: > >> On Mon, Nov 12 2018, Ævar Arnfjörð Bjarmason wrote: >> >>> I get: >>> >>> Test origin/master >>> peff/jk/loose-cache

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-12-01 Thread Jeff King
On Tue, Nov 27, 2018 at 09:48:57PM +0100, René Scharfe wrote: > > +static int quick_has_loose(struct repository *r, > > + const unsigned char *sha1) > > +{ > > + int subdir_nr = sha1[0]; > > + struct object_id oid; > > + struct object_directory *odb; > > + > > +

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-11-27 Thread René Scharfe
Am 12.11.2018 um 15:54 schrieb Jeff King: > diff --git a/sha1-file.c b/sha1-file.c > index 4aae716a37..e53da0b701 100644 > --- a/sha1-file.c > +++ b/sha1-file.c > @@ -921,6 +921,24 @@ static int open_sha1_file(struct repository *r, > return -1; > } > > +static int quick_has_loose(struct

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-11-14 Thread René Scharfe
Am 13.11.2018 um 11:02 schrieb Ævar Arnfjörð Bjarmason: > So here's the same test not against NFS, but the local ext4 fs (CO7; > Linux 3.10) for sha1collisiondetection.git: > > Test origin/master > peff/jk/loose-cache

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-11-13 Thread Ævar Arnfjörð Bjarmason
On Mon, Nov 12 2018, Ævar Arnfjörð Bjarmason wrote: > On Mon, Nov 12 2018, Ævar Arnfjörð Bjarmason wrote: > >> On Mon, Nov 12 2018, Jeff King wrote: >> >>> On Mon, Nov 12, 2018 at 05:01:02PM +0100, Ævar Arnfjörð Bjarmason wrote: >>> > There's some obvious hand-waving in the paragraphs

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-11-12 Thread Geert Jansen
On Mon, Nov 12, 2018 at 11:21:51AM -0500, Jeff King wrote: > No, but they don't even really need to be actual objects. So I suspect > something like: > > git init > for i in $(seq 256); do > i=$(printf %02x $i) > mkdir -p .git/objects/$i > for j in $(seq --format=%038g 1000); do

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-11-12 Thread Ævar Arnfjörð Bjarmason
On Mon, Nov 12 2018, Ævar Arnfjörð Bjarmason wrote: > On Mon, Nov 12 2018, Jeff King wrote: > >> On Mon, Nov 12, 2018 at 05:01:02PM +0100, Ævar Arnfjörð Bjarmason wrote: >> >>> > There's some obvious hand-waving in the paragraphs above. I would love >>> > it if somebody with an NFS system could

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-11-12 Thread Ævar Arnfjörð Bjarmason
On Mon, Nov 12 2018, Jeff King wrote: > On Mon, Nov 12, 2018 at 05:01:02PM +0100, Ævar Arnfjörð Bjarmason wrote: > >> > There's some obvious hand-waving in the paragraphs above. I would love >> > it if somebody with an NFS system could do some before/after timings >> > with various numbers of

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-11-12 Thread Jeff King
On Mon, Nov 12, 2018 at 05:01:02PM +0100, Ævar Arnfjörð Bjarmason wrote: > > There's some obvious hand-waving in the paragraphs above. I would love > > it if somebody with an NFS system could do some before/after timings > > with various numbers of loose objects, to get a sense of where the > >

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-11-12 Thread Ævar Arnfjörð Bjarmason
On Mon, Nov 12 2018, Jeff King wrote: > In cases where we expect to ask has_sha1_file() about a lot of objects > that we are not likely to have (e.g., during fetch negotiation), we > already use OBJECT_INFO_QUICK to sacrifice accuracy (due to racing with > a simultaneous write or repack) for

Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-11-12 Thread Derrick Stolee
On 11/12/2018 9:54 AM, Jeff King wrote: In cases where we expect to ask has_sha1_file() about a lot of objects that we are not likely to have (e.g., during fetch negotiation), we already use OBJECT_INFO_QUICK to sacrifice accuracy (due to racing with a simultaneous write or repack) for speed (we

[PATCH 8/9] sha1-file: use loose object cache for quick existence check

2018-11-12 Thread Jeff King
In cases where we expect to ask has_sha1_file() about a lot of objects that we are not likely to have (e.g., during fetch negotiation), we already use OBJECT_INFO_QUICK to sacrifice accuracy (due to racing with a simultaneous write or repack) for speed (we avoid re-scanning the pack directory).