performance data:
Test w/100,000 files reduced the time by 0.53%
Test w/1,000,000 files reduced the time by 27.78%
Signed-off-by: Ben Peart
---
read-cache.c | 97 +++-
1 file changed, 81 insertions(+), 16 deletions(-)
diff --git a/read
will set the maximum number of threads to use.
For testing purposes, this setting can be overwritten by setting the
GIT_TEST_INDEX_THREADS= environment variable to a value greater than 0.
Signed-off-by: Ben Peart
---
Documentation/config.txt | 7 +++
config.c | 18
This patch does a clean up pass to minimize the casting required to work
with the memory mapped index (mmap).
It also makes the decoding of network byte order more consistent by using
get_be32() where possible.
Signed-off-by: Ben Peart
---
read-cache.c | 23 +++
1 file
From: Nguyễn Thái Ngọc Duy
Index format v4 requires some more computation to assemble a path
based on a previous one. The current code is not very efficient
because
- it doubles memory copy, we assemble the final path in a temporary
first before putting it back to a cache_entry
-
(but not
their contents). E.g. if we have "TREE" extension that is N-bytes
long, "REUC" extension that is M-bytes long, followed by "EOIE",
then the hash would be:
SHA-1("TREE" + +
"REUC" + )
Signed-off-by: Ben Peart
---
Documentation/t
tensions is probably worth
looking into as a quick examination shows that the bulk of the time for both
of them is spent in xcalloc().
### Patches
Ben Peart (6):
read-cache: clean up casting and byte decoding
eoie: add End of Index Entry (EOIE) extension
config: add new index.threads config settin
On 9/25/2018 2:01 PM, Stefan Beller wrote:
On Tue, Sep 25, 2018 at 7:30 AM Derrick Stolee wrote:
On 9/25/2018 3:06 AM, Patrick Steinhardt wrote:
On Mon, Sep 24, 2018 at 11:32:23PM +0200, SZEDER Gábor wrote:
On Mon, Sep 24, 2018 at 02:15:30PM -0700, Derrick Stolee via GitGitGadget wrote:
On 9/25/2018 2:42 PM, Derrick Stolee wrote:
In an effort to ensure new code is reasonably covered by the test suite,
we now have contrib/coverage-diff.sh to combine the gcov output from
'make coverage-test ; make coverage-report' with the output from 'git
diff A B' to discover _new_ lines
On 9/20/2018 2:43 PM, Junio C Hamano wrote:
Ben Peart writes:
This round has one code change based on feedback. Other changes are just
rewording commit messages.
Thanks. I think the only remaining issue is what to do with the
interaction between extra/additional error message that comes
.
Signed-off-by: Ben Peart
---
preload-index.c | 2 +-
t/README| 3 +++
t/t7519-status-fsmonitor.sh | 4 ++--
t/test-lib.sh | 1 +
4 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/preload-index.c b/preload-index.c
index 0a4e2933bb
.
Signed-off-by: Ben Peart
---
config.c| 2 +-
t/README| 4
t/t1700-split-index.sh | 2 +-
t/t7519-status-fsmonitor.sh | 2 +-
t/test-lib.sh | 20
5 files changed, 27 insertions(+), 3 deletions(-)
diff --git
.
Signed-off-by: Ben Peart
---
Makefile | 6 +++---
t/README | 4
t/test-lib.sh | 14 --
3 files changed, 15 insertions(+), 9 deletions(-)
diff --git a/Makefile b/Makefile
index 5a969f5830..9e84ef02f7 100644
--- a/Makefile
+++ b/Makefile
@@ -400,7 +400,7 @@ all
GIT_FORCE_PRELOAD_TEST is only checked for presence by using getenv().
Use git_env_bool() instead so that GIT_FORCE_PRELOAD_TEST=false can
work as expected.
Signed-off-by: Ben Peart
---
preload-index.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/preload-index.c b
_INDEX_VERSION
+fi
+
# Add libc MALLOC and MALLOC_PERTURB test
# only if we are not executing the test with valgrind
if expr " $GIT_TEST_OPTS " : ".* --valgrind " >/dev/null ||
### Patches
Ben Peart (5):
t/README: correct spelling of "uncommon"
preload-inde
Correct a spelling error in the documentation for GIT_TEST_OE_DELTA_SIZE.
Signed-off-by: Ben Peart
---
t/README | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/t/README b/t/README
index 9028b47d92..56a417439c 100644
--- a/t/README
+++ b/t/README
@@ -315,7 +315,7 @@ packs
On 9/15/2018 7:09 AM, Duy Nguyen wrote:
On Sat, Sep 15, 2018 at 01:07:46PM +0200, Duy Nguyen wrote:
12:50:00.084237 read-cache.c:1721 start loading index
12:50:00.119941 read-cache.c:1943 performance: 0.034778758 s: loaded all
extensions (1667075 bytes)
12:50:00.185352
On 9/15/2018 6:31 AM, Duy Nguyen wrote:
On Wed, Sep 12, 2018 at 6:18 PM Ben Peart wrote:
This patch helps address the CPU cost of loading the index by creating
multiple threads to divide the work of loading and converting the cache
entries across all available CPU cores.
It accomplishes
On 9/15/2018 6:24 AM, Duy Nguyen wrote:
On Sat, Sep 15, 2018 at 12:22 PM Duy Nguyen wrote:
@@ -1944,6 +1993,26 @@ int do_read_index(struct index_state *istate, const char
*path, int must_exist)
istate->cache = xcalloc(istate->cache_alloc, sizeof(*istate->cache));
On 9/15/2018 6:22 AM, Duy Nguyen wrote:
+index.threads::
+ Specifies the number of threads to spawn when loading the index.
+ This is meant to reduce index load time on multiprocessor machines.
+ Specifying 0 or 'true' will cause Git to auto-detect the number of
+
On 9/15/2018 6:02 AM, Duy Nguyen wrote:
default:
if (*ext < 'A' || 'Z' < *ext)
return error("index uses %.4s extension, which we do not
understand",
@@ -1889,6 +1893,11 @@ static size_t estimate_cache_size(size_t ondisk_size,
unsigned int
On 9/16/2018 3:17 AM, Eric Sunshine wrote:
On Fri, Sep 14, 2018 at 7:29 PM Ramsay Jones
wrote:
At one time, the POSIX standard required the type used to represent
a thread handle (pthread_t) be an arithmetic type. This is no longer
the case, probably because different platforms used to
.
Signed-off-by: Ben Peart
---
Makefile | 6 +++---
t/README | 4
t/test-lib.sh | 5 +++--
3 files changed, 10 insertions(+), 5 deletions(-)
diff --git a/Makefile b/Makefile
index 5a969f5830..9e84ef02f7 100644
--- a/Makefile
+++ b/Makefile
@@ -400,7 +400,7 @@ all::
# (defaults
.
Signed-off-by: Ben Peart
---
config.c| 2 +-
t/README| 4
t/t1700-split-index.sh | 2 +-
t/t7519-status-fsmonitor.sh | 2 +-
t/test-lib.sh | 20
5 files changed, 27 insertions(+), 3 deletions(-)
diff --git
.
Signed-off-by: Ben Peart
---
preload-index.c | 2 +-
t/README| 3 +++
t/t7519-status-fsmonitor.sh | 4 ++--
t/test-lib.sh | 1 +
4 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/preload-index.c b/preload-index.c
index 0a4e2933bb
_PRELOAD_TEST variable has been
renamed to GIT_TEST_PRELOAD_INDEX"
- exit 1
- fi
-fi
+check_var_migration GIT_FSMONITOR_TEST GIT_TEST_FSMONITOR
+check_var_migration TEST_GIT_INDEX_VERSION GIT_TEST_INDEX_VERSION
+check_var_migration GIT_FORCE_PRELOAD_TEST GIT_TEST_PRELOAD_INDEX
Correct a spelling error in the documentation for GIT_TEST_OE_DELTA_SIZE
Signed-off-by: Ben Peart
---
t/README | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/t/README b/t/README
index 9028b47d92..56a417439c 100644
--- a/t/README
+++ b/t/README
@@ -315,7 +315,7 @@ packs
Teach GIT_FORCE_PRELOAD_TEST to take a boolean to turn on or off this test
feature instead of simply testing for existance.
Signed-off-by: Ben Peart
---
preload-index.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/preload-index.c b/preload-index.c
index 71cd2437a3
On 9/14/2018 1:15 PM, Junio C Hamano wrote:
Ben Peart writes:
diff --git a/config.c b/config.c
index 3461993f0a..3555c63f28 100644
--- a/config.c
+++ b/config.c
@@ -2278,7 +2278,7 @@ int git_config_get_max_percent_split_change(void)
int git_config_get_fsmonitor(void
. If the new variable is also set, just give a warning so they can
eventually remove the old variable.
Signed-off-by: Ben Peart
---
config.c| 2 +-
t/README| 4
t/t1700-split-index.sh | 2 +-
t/t7519-status-fsmonitor.sh | 2 +-
t/test-lib.sh
Correct a spelling error in the documentation for GIT_TEST_OE_DELTA_SIZE
Signed-off-by: Ben Peart
---
t/README | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/t/README b/t/README
index 9028b47d92..56a417439c 100644
--- a/t/README
+++ b/t/README
@@ -315,7 +315,7 @@ packs
. If the new variable is also set, just give a warning so they can
eventually remove the old variable.
Signed-off-by: Ben Peart
---
Makefile | 6 +++---
t/README | 4
t/test-lib.sh | 15 +--
3 files changed, 20 insertions(+), 5 deletions(-)
diff --git a/Makefile b
. If the new variable is also set, just give a warning so they can
eventually remove the old variable.
Signed-off-by: Ben Peart
---
preload-index.c | 3 ++-
t/README| 3 +++
t/t7519-status-fsmonitor.sh | 4 ++--
t/test-lib.sh | 11 +++
4
variable. If the
new variable is also set, just give a warning so they can eventually remove
the old variable.
Base Ref: v2.19.0
Web-Diff: https://github.com/benpeart/git/commit/eff73d737e
Checkout: git fetch https://github.com/benpeart/git git-test-cleanup-v1 && git
checkout eff73d737e
B
On 9/13/2018 6:08 PM, Junio C Hamano wrote:
Thomas Gummerer writes:
Thanks, I do think this is a good idea. I do however share Ævar's
concern in https://public-inbox.org/git/87h8itkz2h@evledraar.gmail.com/.
I have TEST_GIT_INDEX_VERSION=4 set in my config.mak since quite a
long time,
On 9/13/2018 2:54 PM, Ævar Arnfjörð Bjarmason wrote:
On Thu, Sep 13 2018, Ben Peart wrote:
diff --git a/config.c b/config.c
index 3461993f0a..3555c63f28 100644
--- a/config.c
+++ b/config.c
@@ -2278,7 +2278,7 @@ int git_config_get_max_percent_split_change(void)
int
Rename TEST_GIT_INDEX_VERSION to GIT_TEST_INDEX_VERSION for consistency with
the other GIT_TEST_ special setups and properly document its use.
Signed-off-by: Ben Peart
---
Notes:
Base Ref: v2.19.0
Web-Diff: https://github.com/benpeart/git/commit/e26ccb9004
Checkout: git fetch https
Rename GIT_FORCE_PRELOAD_TEST to GIT_TEST_PRELOAD for consistency with the
other GIT_TEST_ special setups and properly document its use.
Signed-off-by: Ben Peart
---
Notes:
Base Ref: v2.19.0
Web-Diff: https://github.com/benpeart/git/commit/dcd201b920
Checkout: git fetch https
Rename GIT_FSMONITOR_TEST to GIT_TEST_FSMONITOR for consistency with the
other GIT_TEST_ special setups and properly document its use.
Signed-off-by: Ben Peart
---
Notes:
Base Ref: v2.19.0
Web-Diff: https://github.com/benpeart/git/commit/311484a684
Checkout: git fetch https
On 9/12/2018 6:31 PM, Thomas Gummerer wrote:
On 09/12, Ben Peart wrote:
Teach get_index_format_default() to support running the test suite
with specific index versions. In particular, this enables the test suite
to be run using index version 4 which is not the default so gets less testing
Teach get_index_format_default() to support running the test suite
with specific index versions. In particular, this enables the test suite
to be run using index version 4 which is not the default so gets less testing.
Signed-off-by: Ben Peart
---
Notes:
Base Ref: v2.19.0
Web-Diff
On 9/12/2018 12:45 PM, Derrick Stolee via GitGitGadget wrote:
We have coverage targets in our Makefile for using gcov to display line
coverage based on our test suite. The way I like to do it is to run:
make coverage-test
make coverage-report
Very nice, I was unaware of the coverage test
(but not
their contents). E.g. if we have "TREE" extension that is N-bytes
long, "REUC" extension that is M-bytes long, followed by "EOIE",
then the hash would be:
SHA-1("TREE" + +
"REUC" + )
Signed-off-by: Ben Peart
---
Documentation/t
ng
v2 only takes 4.226 seconds, while v4 takes 5.711 seconds, 35% more
time. The patch reduces read time on v4 to 4.319 seconds.
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Ben Peart
---
read-cache.c | 132 ++-
1 file changed, 67 inserti
This patch does a clean up pass to minimize the casting required to work
with the memory mapped index (mmap).
It also makes the decoding of network byte order more consistent by using
get_be32() where possible.
Signed-off-by: Ben Peart
---
read-cache.c | 49
+0.06) -24.1%
Signed-off-by: Ben Peart
---
read-cache.c | 242 +--
t/README | 3 +
2 files changed, 217 insertions(+), 28 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index b203eebb44..880f627b4c 100644
--- a/read-cache.c
+++ b/read
Extensions
--
read_cache/discard_cache 1000 times 202.95(0.01+0.07) 154.14(0.03+0.06) -24.1%
Signed-off-by: Ben Peart
---
Documentation/config.txt | 6 +++
config.c | 18
config.h
ap + src_offset, 8);
src_offset += 8;
src_offset += extsize;
}
the_hash_algo->final_fn(hash, );
- if (hashcmp(hash, (unsigned char *)index))
+ if (hashcmp(hash, (const unsigned char *)index))
return 0;
/* Valida
On 9/11/2018 7:26 PM, Ben Peart wrote:
This version of the patch merges in Duy's work to speed up index v4 decoding.
I had to massage it a bit to get it to work with the multi-threading but its
still largely his code. It helps a little (3%-4%) when the cache entry thread(s)
take the longest
Extensions
--
read_cache/discard_cache 1000 times 202.95(0.01+0.07) 154.14(0.03+0.06) -24.1%
Signed-off-by: Ben Peart
---
Documentation/config.txt | 6 +++
config.c | 18
config.h
(but not
their contents). E.g. if we have "TREE" extension that is N-bytes
long, "REUC" extension that is M-bytes long, followed by "EOIE",
then the hash would be:
SHA-1("TREE" + +
"REUC" + )
Signed-off-by: Ben Peart
---
Documentation/t
://github.com/benpeart/git read-index-multithread-v4
&& git checkout 9d31d5fb20
### Patches
Ben Peart (4):
eoie: add End of Index Entry (EOIE) extension
read-cache: load cache extensions on a worker thread
read-cache: speed up index load through parallelization
read-cache: clean up casting
This patch does a clean up pass to minimize the casting required to work
with the memory mapped index (mmap).
It also makes the decoding of network byte order more consistent by using
get_be32() where possible.
Signed-off-by: Ben Peart
---
read-cache.c | 49
ng
v2 only takes 4.226 seconds, while v4 takes 5.711 seconds, 35% more
time. The patch reduces read time on v4 to 4.319 seconds.
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Ben Peart
---
read-cache.c | 136 +++
1 file changed, 71 inserti
+0.06) -24.1%
Signed-off-by: Ben Peart
---
read-cache.c | 240 +--
1 file changed, 212 insertions(+), 28 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index 9b97c29f5b..c01d34a71d 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1942,20
On 9/10/2018 1:07 PM, Stefan Beller wrote:
On Mon, Sep 10, 2018 at 9:29 AM Ben Peart wrote:
It was reported that
GIT_FSMONITOR_TEST=$PWD/t7519/fsmonitor-all ./t7411-submodule-config.sh
breaks as the fsmonitor data is out of sync with the state of the .gitmodules
file. Update
.
Reported-by: Ævar Arnfjörð Bjarmason
Helped-by: Stefan Beller
Signed-off-by: Ben Peart
---
Notes:
Base Ref: v2.19.0-rc2
Web-Diff: https://github.com/benpeart/git/commit/ed30e1a885
Checkout: git fetch https://github.com/benpeart/git fsmonitor-t7411-v1 &&
git checkout ed
.
Reported-by: Ævar Arnfjörð Bjarmason
Inspired-by: Ben Peart
Signed-off-by: Stefan Beller
---
I am not quite sure if this is the correct approach and handling of the
fsmonitor API, but it unbreaks the test.
Just naively adding mark_fsmonitor_invalid doesn't work, as then ...
Adding it before
On 9/7/2018 5:10 PM, Junio C Hamano wrote:
Ben Peart writes:
+struct load_index_extensions
+{
+#ifndef NO_PTHREADS
+ pthread_t pthread;
+#endif
+ struct index_state *istate;
+ void *mmap;
+ size_t mmap_size;
+ unsigned long src_offset;
If the file format
On 9/8/2018 2:29 AM, Martin Ågren wrote:
On Fri, 7 Sep 2018 at 22:24, Ben Peart wrote:
Ben Peart writes:
- 160-bit SHA-1 over the extension types and their sizes (but not
their contents). E.g. if we have "TREE" extension that is N-bytes
long, "REUC" extension
On 9/7/2018 1:55 PM, Junio C Hamano wrote:
Ben Peart writes:
The extension consists of:
- 32-bit offset to the end of the index entries
- 160-bit SHA-1 over the extension types and their sizes (but not
their contents). E.g. if we have "TREE" extension that is N-bytes
l
On 9/7/2018 1:21 PM, Junio C Hamano wrote:
Ben Peart writes:
On further investigation with the previous patch, I noticed that my test
repos didn't contain the cache tree extension in their index. After doing a
commit to ensure they existed, I realized that in some instances, the time
On 9/7/2018 12:16 AM, Torsten Bögershausen wrote:
diff --git a/read-cache.c b/read-cache.c
index fcc776aaf0..8537a55750 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1941,20 +1941,212 @@ static void *load_index_extensions(void *_data)
return NULL;
}
+/*
+ * A helper
(0.04+0.04) -32.7%
p0002-read-cache.sh w/1,000,000 files
BaselineThread entries
---
217.60(0.03+0.04) 199.00(0.00+0.10) -8.6%
Signed-off-by: Ben Peart
---
read-cache.c | 242 +--
t/README
(but not
their contents). E.g. if we have "TREE" extension that is N-bytes
long, "REUC" extension that is M-bytes long, followed by "EOIE",
then the hash would be:
SHA-1("TREE" + +
"REUC" + )
Signed-off-by: Ben Peart
---
Documentation/t
) 217.60(0.03+0.04) -26.3%
Signed-off-by: Ben Peart
---
Documentation/config.txt | 6 +++
config.c | 18
config.h | 1 +
read-cache.c | 94
4 files changed, 102 insertions(+), 17 deletions(-)
diff --git
expand_name_field()
-
306.44(0.04+0.07) 295.42(0.01+0.07) -3.6%
Suggested by: Nguyễn Thái Ngọc Duy
Signed-off-by: Ben Peart
---
read-cache.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index
checkout 325ec69299
### Patches
Ben Peart (4):
read-cache: optimize expand_name_field() to speed up V4 index parsing.
eoie: add End of Index Entry (EOIE) extension
read-cache: load cache extensions on a worker thread
read-cache: speed up index load through parallelization
Documentation/co
> -Original Message-
> From: Stefan Beller
> Sent: Thursday, September 6, 2018 12:57 PM
> To: Ævar Arnfjörð Bjarmason
> Cc: Junio C Hamano ; git ; Ben
> Peart
> Subject: Re: sb/submodule-move-nested breaks t7411 under
> GIT_FSMONITOR_TEST
>
> > > W
On 9/1/2018 4:29 PM, Ævar Arnfjörð Bjarmason wrote:
B.t.w. for Ben or anyone else who knows about the fsmonitor part of
this: I've long been running the whole test suite with
`GIT_FSMONITOR_TEST=$PWD/t7519/fsmonitor-all prove ...` (also along with
GIT_TEST_SPLIT_INDEX=) after all the main
On 8/29/2018 1:12 PM, Junio C Hamano wrote:
Ben Peart writes:
This is possible because the current extensions don't access the cache
entries in the index_state structure so are OK that they don't all exist
yet.
The CACHE_EXT_TREE, CACHE_EXT_RESOLVE_UNDO, and CACHE_EXT_UNTRACKED
extensions
On 8/29/2018 1:14 PM, Junio C Hamano wrote:
Ben Peart writes:
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 1c42364988..79f8296d9c 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -2391,6 +2391,12 @@ imap::
The configuration variables
p->mmap_size = mmap_size;
+ p->start_offset = src_offset;
+
+ if (pthread_create(>pthread, NULL, load_index_extensions_thread, p))
+ die("unable to create load_index_extensions_thread");
+
+ for (i = 0; i < nr_threads + 1; i++) {
HEAD~3HEAD~2
--
read_cache/discard_cache 1000 times 202.06(0.06+0.09) 155.72(0.03+0.06) -22.9%
Signed-off-by: Ben Peart
---
Documentation/config.txt | 6 +
config.c | 14 +++
config.h
files
TestHEAD~3HEAD
--
read_cache/discard_cache 1000 times 201.77(0.03+0.07) 149.68(0.04+0.07) -25.8%
Suggested by: Nguyễn Thái Ngọc Duy
Signed-off-by: Ben Peart
---
read
%
Signed-off-by: Ben Peart
---
read-cache.c | 60 +---
1 file changed, 48 insertions(+), 12 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index c30346388a..f768004617 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1959,16 +1959,13 @@ struct
On 8/28/2018 3:25 PM, Duy Nguyen wrote:
On Mon, Aug 27, 2018 at 9:36 PM Junio C Hamano wrote:
PS. I notice that v4 does not pad to align entries at 4 byte boundary
like v2/v3. This could cause a slight slow down on x86 and segfault on
some other platforms.
Care to elaborate?
Long time
On 8/24/2018 2:40 PM, Duy Nguyen wrote:
On Thu, Aug 23, 2018 at 7:33 PM Stefan Beller wrote:
+core.fastIndex::
+ Enable parallel index loading
++
+This can speed up operations like 'git diff' and 'git status' especially
+when the index is very large. When enabled, Git will do the
On 8/24/2018 3:00 PM, Duy Nguyen wrote:
On Fri, Aug 24, 2018 at 8:40 PM Ben Peart wrote:
On 8/24/2018 2:20 PM, Duy Nguyen wrote:
On Fri, Aug 24, 2018 at 5:37 PM Duy Nguyen wrote:
On Thu, Aug 23, 2018 at 10:36 PM Ben Peart wrote:
Nice to see this done without a new index extension
On 8/24/2018 2:20 PM, Duy Nguyen wrote:
On Fri, Aug 24, 2018 at 5:37 PM Duy Nguyen wrote:
On Thu, Aug 23, 2018 at 10:36 PM Ben Peart wrote:
Nice to see this done without a new index extension that records
offsets, so that we can load existing index files in parallel.
Yes, I prefer
On 8/24/2018 11:57 AM, Duy Nguyen wrote:
On Fri, Aug 24, 2018 at 05:37:20PM +0200, Duy Nguyen wrote:
Since we're cutting corners to speed things up, could you try
something like this?
I notice that reading v4 is significantly slower than v2 and
apparently strlen() (at least from glibc) is
On 8/23/2018 2:06 PM, Junio C Hamano wrote:
Ben Peart writes:
This patch helps address the CPU cost of loading the index by creating
multiple threads to divide the work of loading and converting the cache
entries across all available CPU cores.
Nice.
+int git_config_get_fast_index(void
On 8/23/2018 1:31 PM, Stefan Beller wrote:
On Thu, Aug 23, 2018 at 8:45 AM Ben Peart wrote:
This patch helps address the CPU cost of loading the index by creating
multiple threads to divide the work of loading and converting the cache
entries across all available CPU cores
entries
FALSE TRUESavings %Savings
0.59898098 0.4513169 0.14766408 24.65%
Signed-off-by: Ben Peart
---
Notes:
Base Ref: master
Web-Diff: https://github.com/benpeart/git/commit/67a700419b
Checkout: git fetch https://github.com/benpeart/git
read-index-multithread-v1
On 8/18/2018 9:44 PM, Elijah Newren wrote:
On Fri, Aug 17, 2018 at 5:41 AM Ben Peart wrote:
On 8/16/2018 2:37 PM, Duy Nguyen wrote:
On Thu, Aug 16, 2018 at 8:27 PM Ben Peart wrote:
From: Ben Peart
Skip merging the commit, updating the index and working directory if and
only if we
On 8/18/2018 10:41 AM, Nguyễn Thái Ngọc Duy wrote:
In order to merge one or many trees with the index, unpack-trees code
walks multiple trees in parallel with the index and performs n-way
merge. If we find out at start of a directory that all trees are the
same (by comparing OID) and
On 8/16/2018 2:37 PM, Duy Nguyen wrote:
On Thu, Aug 16, 2018 at 8:27 PM Ben Peart wrote:
From: Ben Peart
Skip merging the commit, updating the index and working directory if and
only if we are creating a new branch via "git checkout -b ."
Any other checkout options will still
From: Ben Peart
Skip merging the commit, updating the index and working directory if and
only if we are creating a new branch via "git checkout -b ."
Any other checkout options will still go through the former code path.
If sparse_checkout is on, require the user to ma
On 8/6/2018 10:25 AM, Ben Peart wrote:
On 8/3/2018 11:58 AM, Duy Nguyen wrote:
On Thu, Aug 02, 2018 at 02:02:00PM -0400, Ben Peart wrote:
But if you still want to push it further, this is something I have in
mind. It probably has bugs, but at least preliminary test shows me
On 8/12/2018 4:15 AM, Nguyễn Thái Ngọc Duy wrote:
v4 has a bunch of changes
- 1/5 is a new one to show indented tracing. This way it's less
misleading to read nested time measurements
- 3/5 now has the switch/restore cache_bottom logic. Junio suggested a
check instead in his final
On 8/13/2018 3:36 PM, Stefan Beller wrote:
On Mon, Aug 13, 2018 at 12:25 PM Jeff King wrote:
I can buy the argument that it's nice to have some form of profiling
that works everywhere, even if it's lowest-common-denominator. I just
wonder if we could be investing effort into tooling around
On 8/12/2018 4:15 AM, Nguyễn Thái Ngọc Duy wrote:
In order to merge one or many trees with the index, unpack-trees code
walks multiple trees in parallel with the index and performs n-way
merge. If we find out at start of a directory that all trees are the
same (by comparing OID) and
On 8/12/2018 4:15 AM, Nguyễn Thái Ngọc Duy wrote:
We're going to optimize unpack_trees() a bit in the following
patches. Let's add some tracing to measure how long it takes before
and after. This is the baseline ("git checkout -" on webkit.git, 275k
files on worktree)
performance:
On 8/12/2018 4:15 AM, Nguyễn Thái Ngọc Duy wrote:
Performance measurements are listed right now as a flat list, which is
fine when we measure big blocks. But when we start adding more and
more measurements, some of them could be just part of a bigger
measurement and a flat list gives a wrong
On 8/13/2018 12:25 PM, Duy Nguyen wrote:
On Mon, Aug 13, 2018 at 6:05 PM Ben Peart wrote:
I was part way through writing a patch that would copy the valid parts
of the cache-tree from the source index to the dest index
Yeah sorry about that. I make bad judgements all the time
On 8/13/2018 11:48 AM, Elijah Newren wrote:
On Sun, Aug 12, 2018 at 1:16 AM Nguyễn Thái Ngọc Duy wrote:
We do n-way merge by walking the source index and n trees at the same
time and add merge results to a new temporary index called o->result.
The merge result for any given path could be
On 8/8/2018 4:53 PM, Ben Peart wrote:
On 8/1/2018 12:38 PM, Duy Nguyen wrote:
On Tue, Jul 31, 2018 at 01:31:31PM -0400, Ben Peart wrote:
On 7/31/2018 12:50 PM, Ben Peart wrote:
On 7/31/2018 11:31 AM, Duy Nguyen wrote:
In the performance game of whack-a-mole, that call to repair
On 8/1/2018 12:38 PM, Duy Nguyen wrote:
On Tue, Jul 31, 2018 at 01:31:31PM -0400, Ben Peart wrote:
On 7/31/2018 12:50 PM, Ben Peart wrote:
On 7/31/2018 11:31 AM, Duy Nguyen wrote:
In the performance game of whack-a-mole, that call to repair cache-tree
is now looking quite expensive
On 8/6/2018 2:59 PM, Junio C Hamano wrote:
Duy Nguyen writes:
We require the unpacked entry from all input trees to be a tree
objects (the dirmask thing), so if one tree has 't' as a file,
Ah, OK, this is still part of that "all the trees match cache tree
so we walk the index instead"
On 8/3/2018 11:58 AM, Duy Nguyen wrote:
On Thu, Aug 02, 2018 at 02:02:00PM -0400, Ben Peart wrote:
But if you still want to push it further, this is something I have in
mind. It probably has bugs, but at least preliminary test shows me
that it could skip 99% work inside unpack_trees
On 7/13/2018 12:55 PM, g...@jeffhostetler.com wrote:
From: Jeff Hostetler
Signed-off-by: Jeff Hostetler
---
Documentation/technical/structured-logging.txt | 816 +
1 file changed, 816 insertions(+)
create mode 100644
101 - 200 of 658 matches
Mail list logo
