On 11/28/2018 8:31 AM, SZEDER Gábor wrote:
On Tue, Nov 27, 2018 at 02:50:57PM -0500, Ben Peart wrote:
diff --git a/t/t1092-virtualworkdir.sh b/t/t1092-virtualworkdir.sh
new file mode 100755
index 00..0cdfe9b362
--- /dev/null
+++ b/t/t1092-virtualworkdir.sh
@@ -0,0 +1,393 @@
+#!/bin
On 11/28/2018 4:37 AM, Johannes Schindelin wrote:
Hi Ben,
On Tue, 27 Nov 2018, Ben Peart wrote:
From: Ben Peart
Add tracing around initializing and discarding mempools. In discard report
on the amount of memory unused in the current block to help tune setting
the initial_size.
Signed-off
From: Ben Peart
Add tracing around initializing and discarding mempools. In discard report
on the amount of memory unused in the current block to help tune setting
the initial_size.
Signed-off-by: Ben Peart
---
Notes:
Base Ref: * git-trace-mempool
Web-Diff: https://github.com/benpeart
From: Ben Peart
To make git perform well on the very largest repos, we must make git
operations O(modified) instead of O(size of repo). This takes advantage of
the fact that the number of files a developer has modified (especially
in very large repos) is typically a tiny fraction of the overall
On 11/26/2018 2:59 PM, Stefan Beller wrote:
+static int record_ieot(void)
+{
+ int val;
+
Initialize stack val to zero to ensure proper default.
I don't think that is needed here, as we only use `val` when
we first write to it via git_config_get_bool.
Did you spot this via code
On 11/20/2018 1:11 AM, Jonathan Nieder wrote:
Since 3b1d9e04 (eoie: add End of Index Entry (EOIE) extension,
2018-10-10) Git defaults to writing the new EOIE section when writing
out an index file. Usually that is a good thing because it improves
threaded performance, but when a Git
On 11/20/2018 4:26 AM, Ævar Arnfjörð Bjarmason wrote:
On Tue, Nov 20 2018, Jonathan Nieder wrote:
Just commenting here on the end-state of this since it's easier than
each patch at a time:
First, do we still need to be doing %.4s instead of just %s? It would be
easier for translators / to
On 11/20/2018 1:14 AM, Jonathan Nieder wrote:
If a user explicitly sets
[index]
threads = true
to read the index using multiple threads, ensure that index writes
include the offset table by default to make that possible. This
ensures that the user's intent of
On 11/20/2018 1:12 AM, Jonathan Nieder wrote:
As with EOIE, popular versions of Git do not support the new IEOT
extension yet. When accessing a Git repository written by a more
modern version of Git, they correctly ignore the unrecognized section,
but in the process they loudly warn
On 11/20/2018 1:11 AM, Jonathan Nieder wrote:
Since 3b1d9e04 (eoie: add End of Index Entry (EOIE) extension,
2018-10-10) Git defaults to writing the new EOIE section when writing
out an index file. Usually that is a good thing because it improves
threaded performance, but when a Git
wrote:
There are a lot of lines introduced by the IEOT extension in these commits:
> Ben Peart 3255089ad: ieot: add Index Entry Offset Table (IEOT)
extension
> Ben Peart 3b1d9e045: eoie: add End of Index Entry (EOIE) extension
> Ben Peart 77ff1127a: read-cache: load cach
On 11/13/2018 10:24 PM, Junio C Hamano wrote:
Jonathan Nieder writes:
We cannot change the past, but for index extensions of the future,
there is a straightforward improvement: silence that message except
when tracing. This way, the message is still available when
debugging, but in
On 11/13/2018 4:08 PM, Jonathan Nieder wrote:
Hi again,
Ben Peart wrote:
On 11/13/2018 1:18 PM, Jonathan Nieder wrote:
Ben Peart wrote:
Why introduce a new setting to disable writing the IEOT extension instead of
just using the existing index.threads setting? If index.threads=1
On 11/13/2018 1:18 PM, Jonathan Nieder wrote:
Hi,
Ben Peart wrote:
On 11/12/2018 7:39 PM, Jonathan Nieder wrote:
As with EOIE, popular versions of Git do not support the new IEOT
extension yet. When accessing a Git repository written by a more
modern version of Git, they correctly
On 11/12/2018 7:40 PM, Jonathan Nieder wrote:
Documentation/technical/index-format explains:
4-byte extension signature. If the first byte is 'A'..'Z' the
extension is optional and can be ignored.
This allows gracefully introducing a new index extension without
having to rely on
On 11/12/2018 7:39 PM, Jonathan Nieder wrote:
As with EOIE, popular versions of Git do not support the new IEOT
extension yet. When accessing a Git repository written by a more
modern version of Git, they correctly ignore the unrecognized section,
but in the process they loudly warn
On 11/12/2018 8:05 PM, Junio C Hamano wrote:
Jonathan Nieder writes:
Since 3b1d9e04 (eoie: add End of Index Entry (EOIE) extension,
2018-10-10) Git defaults to writing the new EOIE section when writing
out an index file. Usually that is a good thing because it improves
threaded
On 11/4/2018 4:01 PM, brian m. carlson wrote:
On Sun, Nov 04, 2018 at 07:34:01AM +0100, Duy Nguyen wrote:
On Wed, Oct 31, 2018 at 9:53 PM Ben Peart wrote:
It's more than a dynamic sparse-checkout because the same list is also
used to exclude any file/folder not listed. That means any file
On 11/5/2018 10:22 AM, Duy Nguyen wrote:
On Sun, Nov 4, 2018 at 10:01 PM brian m. carlson
wrote:
On Sun, Nov 04, 2018 at 07:34:01AM +0100, Duy Nguyen wrote:
On Wed, Oct 31, 2018 at 9:53 PM Ben Peart wrote:
It's more than a dynamic sparse-checkout because the same list is also
used
On 11/5/2018 10:26 AM, Duy Nguyen wrote:
On Mon, Nov 5, 2018 at 12:40 PM Ævar Arnfjörð Bjarmason
wrote:
On Sun, Nov 04 2018, Duy Nguyen wrote:
On Wed, Oct 31, 2018 at 9:53 PM Ben Peart wrote:
+core.virtualFilesystem::
+ If set, the value of this variable is used as a command
On 11/4/2018 7:02 PM, Junio C Hamano wrote:
Ben Peart writes:
+ if (*dtype == DT_UNKNOWN)
+ *dtype = get_dtype(NULL, istate, pathname, pathlen);
We try to defer paying cost to determine unknown *dtype as late as
possible by having this call
From: Ben Peart
With refresh_index() learning to utilize preload_index() to speed up its
operation there is no longer any benefit to having the caller preload the
index first. Remove those unneeded calls by calling read_index() instead of
the preload variant.
There is no measurable performance
On 11/2/2018 11:23 AM, Junio C Hamano wrote:
Ben Peart writes:
From: Ben Peart
During an "add", a call is made to run_diff_files() which calls
check_remove() for each index-entry. The preload_index() code
distributes some of the costs across multiple threads.
Nice. I pee
From: Ben Peart
During an "add", a call is made to run_diff_files() which calls
check_remove() for each index-entry. The preload_index() code distributes
some of the costs across multiple threads.
Because the files checked are restricted to pathspec, adding individual
files makes no
On 10/31/2018 3:11 PM, Duy Nguyen wrote:
not really a review, just a couple quick notes..
Perfect! As an RFC, I'm more looking for high level thoughts/notes than
a style/syntax code review.
On Tue, Oct 30, 2018 at 9:40 PM Ben Peart wrote:
From: Ben Peart
On index load, clear/set
On 10/30/2018 7:07 PM, Junio C Hamano wrote:
Ben Peart writes:
diff --git a/config.c b/config.c
index 4051e38823..96e05ee0f1 100644
--- a/config.c
+++ b/config.c
...
@@ -2307,6 +2311,37 @@ int git_config_get_index_threads(void)
return 0; /* auto */
}
+int
From: Ben Peart
On index load, clear/set the skip worktree bits based on the virtual
file system data. Use virtual file system data to update skip-worktree
bit in unpack-trees. Use virtual file system data to exclude files and
folders not explicitly requested.
Signed-off-by: Ben Peart
---
We
From: Ben Peart
Speed up refresh_index() by utilizing preload_index() to do most of the work
spread across multiple threads. This works because most cache entries will
get marked CE_UPTODATE so that refresh_cache_ent() can bail out early when
called from within refresh_index().
On a Windows
On 10/29/2018 1:26 PM, Duy Nguyen wrote:
On Mon, Oct 29, 2018 at 6:21 PM Ben Peart wrote:
@@ -114,6 +104,9 @@ static void preload_index(struct index_state *index,
threads = index->cache_nr / THREAD_COST;
if ((index->cache_nr > 1) && (threads <
On 10/29/2018 1:21 PM, Duy Nguyen wrote:
On Mon, Oct 29, 2018 at 6:05 PM Ben Peart wrote:
@@ -2756,8 +2745,11 @@ static int do_write_index(struct index_state *istate,
struct tempfile *tempfile,
if (ce_write(, newfd, , sizeof(hdr)) < 0)
return -1;
-#ifndef NO_PTHRE
On 10/29/2018 10:30 AM, Jeff King wrote:
On Sat, Oct 27, 2018 at 07:30:06PM +0200, Nguyễn Thái Ngọc Duy wrote:
-#ifndef NO_PTHREADS
- nr_threads = git_config_get_index_threads();
+ if (HAVE_THREADS) {
+ nr_threads = git_config_get_index_threads();
- /* TODO:
On 10/27/2018 1:30 PM, Nguyễn Thái Ngọc Duy wrote:
Signed-off-by: Nguyễn Thái Ngọc Duy
---
preload-index.c | 15 ++-
1 file changed, 2 insertions(+), 13 deletions(-)
diff --git a/preload-index.c b/preload-index.c
index 9e7152ab14..0e24886aca 100644
--- a/preload-index.c
+++
On 10/29/2018 10:30 AM, Jeff King wrote:
On Sat, Oct 27, 2018 at 07:30:06PM +0200, Nguyễn Thái Ngọc Duy wrote:
-#ifndef NO_PTHREADS
- nr_threads = git_config_get_index_threads();
+ if (HAVE_THREADS) {
+ nr_threads = git_config_get_index_threads();
- /* TODO:
On 10/27/2018 3:10 AM, Nguyễn Thái Ngọc Duy wrote:
Signed-off-by: Nguyễn Thái Ngọc Duy
---
read-cache.c | 49 ++---
1 file changed, 18 insertions(+), 31 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index d57958233e..ba870bc3fd 100644
On 10/27/2018 3:10 AM, Nguyễn Thái Ngọc Duy wrote:
Signed-off-by: Nguyễn Thái Ngọc Duy
---
preload-index.c | 15 ++-
1 file changed, 2 insertions(+), 13 deletions(-)
diff --git a/preload-index.c b/preload-index.c
index 9e7152ab14..0e24886aca 100644
--- a/preload-index.c
+++
On 10/23/2018 4:28 PM, Jeff King wrote:
On Thu, Oct 18, 2018 at 08:05:22PM +0200, Nguyễn Thái Ngọc Duy wrote:
On Thu, Oct 18, 2018 at 7:09 PM Jeff King wrote:
In this particular case though I think we should be able to avoid so
much #if if we make a wrapper for pthread api that would
On 10/25/2018 5:26 AM, Junio C Hamano wrote:
Junio C Hamano writes:
To be honest, I find the second sentence in your rewrite even more
confusing. It reads as if `reset.quiet` configuration variable
can be used to restore the "show what is yet to be added"
behaviour, due to the
On 10/22/2018 7:05 PM, Junio C Hamano wrote:
Jeff King writes:
If nobody uses it, should we drop the return value, too? Like:
Yup.
I'm good with that.
At one point I also had the additional #ifndef NO_PTHREADS lines but it
was starting to get messy with the threaded vs non-threaded
From: Ben Peart
Updated the wording in the documentation and commit messages to (hopefully)
make it clearer. Added the warning about 'reset --quiet' to the advice
system so that it can be turned off.
Base Ref:
Web-Diff: https://github.com/benpeart/git/commit/8a2fef45d4
Checkout: git fetch
From: Ben Peart
When git reset is run with the --quiet flag, don't bother finding any
additional unstaged changes as they won't be output anyway. This speeds up
the git reset command by avoiding having to lstat() every file looking for
changes that aren't going to be reported anyway
From: Ben Peart
refresh_index() is done after a reset command as an optimization. Because
it can be an expensive call, warn the user if it takes more than 2 seconds
and tell them how to avoid it using the --quiet command line option or
reset.quiet config setting.
Signed-off-by: Ben Peart
From: Ben Peart
Add a reset.quiet config setting that sets the default value of the --quiet
flag when running the reset command. This enables users to change the
default behavior to take advantage of the performance advantages of
avoiding the scan for unstaged changes after reset. Defaults
On 10/22/2018 10:45 AM, Duy Nguyen wrote:
On Mon, Oct 22, 2018 at 3:38 PM Ben Peart wrote:
From: Ben Peart
Add a reset.quiet config setting that sets the default value of the --quiet
flag when running the reset command. This enables users to change the
default behavior to take advantage
On 10/23/2018 5:13 AM, Ævar Arnfjörð Bjarmason wrote:
On Wed, Oct 17 2018, Jeff King wrote:
On Wed, Oct 17, 2018 at 02:19:59PM -0400, Eric Sunshine wrote:
On Wed, Oct 17, 2018 at 12:40 PM Ben Peart wrote:
Add a reset.quietDefault config setting that sets the default value
On 10/22/2018 4:06 PM, Jeff King wrote:
On Mon, Oct 22, 2018 at 08:13:32PM +0100, Ramsay Jones wrote:
-q::
--quiet::
- Be quiet, only report errors.
+--no-quiet::
+ Be quiet, only report errors. The default behavior respects the
+ `reset.quiet` config option, or
On 10/22/2018 8:23 PM, Junio C Hamano wrote:
Ben Peart writes:
From: Ben Peart
refresh_index() is done after a reset command as an optimization. Because
it can be an expensive call, warn the user if it takes more than 2 seconds
and tell them how to avoid it using the --quiet command
> -Original Message-
> From: Johannes Schindelin
> Sent: Monday, October 22, 2018 4:45 PM
> To: Ben Peart
> Cc: git@vger.kernel.org; gits...@pobox.com; Ben Peart
> ; p...@peff.net; sunsh...@sunshineco.com
> Subject: Re: [PATCH v3 1/3] reset: don't compute unstaged
From: Ben Peart
Remove the src_offset parameter which is unused as a result of switching
to the IEOT table of offsets. Also stop incrementing src_offset in the
multi-threaded codepath as it is no longer used and could cause confusion.
Signed-off-by: Ben Peart
---
Notes:
Base Ref:
Web
On 10/21/2018 10:14 PM, Junio C Hamano wrote:
Jeff King writes:
On Wed, Oct 10, 2018 at 11:59:38AM -0400, Ben Peart wrote:
+static unsigned long load_cache_entries_threaded(struct index_state *istate,
const char *mmap, size_t mmap_size,
+ unsigned long src_offset
From: Ben Peart
Add a reset.quiet config setting that sets the default value of the --quiet
flag when running the reset command. This enables users to change the
default behavior to take advantage of the performance advantages of
avoiding the scan for unstaged changes after reset. Defaults
From: Ben Peart
When git reset is run with the --quiet flag, don't bother finding any
additional unstaged changes as they won't be output anyway. This speeds up
the git reset command by avoiding having to lstat() every file looking for
changes that aren't going to be reported anyway
From: Ben Peart
refresh_index() is done after a reset command as an optimization. Because
it can be an expensive call, warn the user if it takes more than 2 seconds
and tell them how to avoid it using the --quiet command line option or
reset.quiet config setting.
Signed-off-by: Ben Peart
From: Ben Peart
Reworded the documentation for git-reset per review feedback.
Base Ref:
Web-Diff: https://github.com/benpeart/git/commit/1228898917
Checkout: git fetch https://github.com/benpeart/git reset-refresh-index-v3 &&
git checkout 1228898917
### Interdiff (v2..v3):
di
On 10/19/2018 1:11 PM, Jeff King wrote:
On Fri, Oct 19, 2018 at 01:10:34PM -0400, Eric Sunshine wrote:
On Fri, Oct 19, 2018 at 12:46 PM Jeff King wrote:
On Fri, Oct 19, 2018 at 12:36:44PM -0400, Eric Sunshine wrote:
How does the user reverse this for a particular git-reset invocation?
On 10/19/2018 12:46 PM, Jeff King wrote:
On Fri, Oct 19, 2018 at 12:36:44PM -0400, Eric Sunshine wrote:
On Fri, Oct 19, 2018 at 12:12 PM Ben Peart wrote:
Add a reset.quiet config setting that sets the default value of the --quiet
flag when running the reset command. This enables users
From: Ben Peart
When git reset is run with the --quiet flag, don't bother finding any
additional unstaged changes as they won't be output anyway. This speeds up
the git reset command by avoiding having to lstat() every file looking for
changes that aren't going to be reported anyway
From: Ben Peart
refresh_index() is done after a reset command as an optimization. Because
it can be an expensive call, warn the user if it takes more than 2 seconds
and tell them how to avoid it using the --quiet command line option or
reset.quiet config setting.
Signed-off-by: Ben Peart
From: Ben Peart
Add a reset.quiet config setting that sets the default value of the --quiet
flag when running the reset command. This enables users to change the
default behavior to take advantage of the performance advantages of
avoiding the scan for unstaged changes after reset. Defaults
From: Ben Peart
This itteration avoids the refresh_index() call completely if 'quiet'.
The advantage of this is that "git refresh" without any pathspec is also
significantly sped up.
Also added a notification if finding unstaged changes after reset takes
longer than 2 seconds to
On 10/18/2018 2:26 PM, Duy Nguyen wrote:
On Thu, Oct 18, 2018 at 8:18 PM Ben Peart wrote:
I actually started my effort to speed up reset by attempting to
multi-thread refresh_index(). You can see a work in progress at:
https://github.com/benpeart/git/pull/new/refresh-index-multithread
On 10/18/2018 2:36 AM, Jeff King wrote:
On Thu, Oct 18, 2018 at 12:40:48PM +0900, Junio C Hamano wrote:
Jeff King writes:
Whereas for the new config variable, you'd probably set it not because
you want it quiet all the time, but because you want to get some time
savings. So there it does
From: Ben Peart
When git reset is run with the --quiet flag, don't bother finding any
additional unstaged changes as they won't be output anyway. This speeds up
the git reset command by avoiding having to lstat() every file looking for
changes that aren't going to be reported anyway
From: Ben Peart
The reset (mixed) command unstages the specified file(s) and then shows you
the remaining unstaged changes. This can make the command slow on larger
repos because at the end it calls refresh_index() which has a single thread
that loops through all the entries calling lstat
From: Ben Peart
Add a reset.quietDefault config setting that sets the default value of the
--quiet flag when running the reset command. This enables users to change
the default behavior to take advantage of the performance advantages of
avoiding the scan for unstaged changes after reset
fixup! IEOT error messages
Enable localizing new error messages and improve the error message for
invalid IEOT extension sizes.
Signed-off-by: Ben Peart
---
read-cache.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index 7acc2c86f4
From: Ben Peart
This patch enables addressing the CPU cost of loading the index by adding
additional data to the index that will allow us to efficiently multi-
thread the loading and conversion of cache entries.
It accomplishes this by adding an (optional) index extension that is a
table
From: Ben Peart
This patch helps address the CPU cost of loading the index by loading
the cache extensions on a worker thread in parallel with loading the cache
entries.
In some cases, loading the extensions takes longer than loading the
cache entries so this patch utilizes the new EOIE
From: Ben Peart
This patch helps address the CPU cost of loading the index by utilizing
the Index Entry Offset Table (IEOT) to divide loading and conversion of
the cache entries across multiple threads in parallel.
I used p0002-read-cache.sh to generate some performance data:
Test w/100,000
From: Ben Peart
Add support for a new index.threads config setting which will be used to
control the threading code in do_read_index(). A value of 0 will tell the
index code to automatically determine the correct number of threads to use.
A value of 1 will make the code single threaded
From: Ben Peart
This patch does a clean up pass to minimize the casting required to work
with the memory mapped index (mmap).
It also makes the decoding of network byte order more consistent by using
get_be32() where possible.
Signed-off-by: Ben Peart
---
read-cache.c | 23
From: Nguyễn Thái Ngọc Duy
Index format v4 requires some more computation to assemble a path
based on a previous one. The current code is not very efficient
because
- it doubles memory copy, we assemble the final path in a temporary
first before putting it back to a cache_entry
-
From: Ben Peart
The End of Index Entry (EOIE) is used to locate the end of the variable
length index entries and the beginning of the extensions. Code can take
advantage of this to quickly locate the index extensions without having
to parse through all of the index entries.
The EOIE extension
From: Ben Peart
Fixed issues identified in review the most impactful probably being plugging
some leaks and improved error handling. Also added better error messages
and some code cleanup to code I'd touched.
The biggest change in the interdiff is the impact of renaming ieot_offset
On 10/9/2018 5:30 AM, Junio C Hamano wrote:
Jonathan Tan writes:
@@ -1635,6 +1635,7 @@ int unpack_trees(unsigned len, struct tree_desc *t,
struct unpack_trees_options
o->result.cache_tree = cache_tree();
if
On 10/8/2018 5:48 PM, Jonathan Tan wrote:
Whenever a sparse checkout occurs, the existence of all blobs in the
index is verified, whether or not they are included or excluded by the
.git/info/sparse-checkout specification. This degrades performance,
significantly in the case of a partial
On 10/8/2018 10:19 AM, Ævar Arnfjörð Bjarmason wrote:
On Thu, Sep 06 2018, Ævar Arnfjörð Bjarmason wrote:
On Thu, Feb 01 2018, Ævar Arnfjörð Bjarmason wrote:
The GIT_FSMONITOR_TEST variable allows you to roundtrip the fsmonitor
codpath in the whole test suite. On both Debian & CentOS
On 10/1/2018 1:09 PM, Duy Nguyen wrote:
On Mon, Oct 1, 2018 at 3:46 PM Ben Peart wrote:
+/*
+ * A helper function that will load the specified range of cache entries
+ * from the memory mapped file and add them to the given index.
+ */
+static unsigned long load_cache_entry_block(struct
On 10/1/2018 12:27 PM, Duy Nguyen wrote:
On Mon, Oct 1, 2018 at 3:46 PM Ben Peart wrote:
@@ -1888,6 +1890,23 @@ static size_t estimate_cache_size(size_t ondisk_size,
unsigned int entries)
return ondisk_size + entries * per_entry;
}
+struct index_entry_offset
On 10/1/2018 11:30 AM, Duy Nguyen wrote:
On Mon, Oct 1, 2018 at 3:46 PM Ben Peart wrote:
@@ -2479,6 +2491,7 @@ static int do_write_index(struct index_state *istate,
struct tempfile *tempfile,
if (ce_write(, newfd, , sizeof(hdr)) < 0)
return -1;
+ off
On 10/1/2018 11:50 AM, Duy Nguyen wrote:
On Mon, Oct 1, 2018 at 3:46 PM Ben Peart wrote:
@@ -1890,6 +1891,46 @@ static size_t estimate_cache_size(size_t ondisk_size,
unsigned int entries)
static size_t read_eoie_extension(const char *mmap, size_t mmap_size);
static void
On 10/1/2018 11:17 AM, SZEDER Gábor wrote:
On Mon, Oct 01, 2018 at 09:45:52AM -0400, Ben Peart wrote:
From: Ben Peart
The End of Index Entry (EOIE) is used to locate the end of the variable
length index entries and the beginning of the extensions. Code can take
advantage of this to quickly
From: Ben Peart
This patch enables addressing the CPU cost of loading the index by adding
additional data to the index that will allow us to efficiently multi-
thread the loading and conversion of cache entries.
It accomplishes this by adding an (optional) index extension that is a
table
From: Ben Peart
This patch helps address the CPU cost of loading the index by loading
the cache extensions on a worker thread in parallel with loading the cache
entries.
In some cases, loading the extensions takes longer than loading the
cache entries so this patch utilizes the new EOIE
From: Ben Peart
Add support for a new index.threads config setting which will be used to
control the threading code in do_read_index(). A value of 0 will tell the
index code to automatically determine the correct number of threads to use.
A value of 1 will make the code single threaded
From: Ben Peart
This patch helps address the CPU cost of loading the index by utilizing
the Index Entry Offset Table (IEOT) to divide loading and conversion of
the cache entries across multiple threads in parallel.
I used p0002-read-cache.sh to generate some performance data:
Test w/100,000
checkout c1125a5d9a
### Patches
Ben Peart (6):
read-cache: clean up casting and byte decoding
eoie: add End of Index Entry (EOIE) extension
config: add new index.threads config setting
read-cache: load cache extensions on a worker thread
ieot: add Index Entry Offset Table (IEOT) extensio
From: Ben Peart
This patch does a clean up pass to minimize the casting required to work
with the memory mapped index (mmap).
It also makes the decoding of network byte order more consistent by using
get_be32() where possible.
Signed-off-by: Ben Peart
---
read-cache.c | 23
From: Ben Peart
The End of Index Entry (EOIE) is used to locate the end of the variable
length index entries and the beginning of the extensions. Code can take
advantage of this to quickly locate the index extensions without having
to parse through all of the index entries.
Because it must
From: Nguyễn Thái Ngọc Duy
Index format v4 requires some more computation to assemble a path
based on a previous one. The current code is not very efficient
because
- it doubles memory copy, we assemble the final path in a temporary
first before putting it back to a cache_entry
-
On 9/28/2018 6:15 PM, Junio C Hamano wrote:
Ramsay Jones writes:
if (!nr) {
ieot_blocks = istate->cache_nr / THREAD_COST;
- if (ieot_blocks < 1)
- ieot_blocks = 1;
cpus =
On 9/28/2018 1:07 PM, Junio C Hamano wrote:
Ben Peart writes:
Why does multithreading have to be disabled in this test?
If multi-threading is enabled, it will write out the IEOT extension
which changes the SHA and causes the test to fail.
I think it is a design mistake to let
On 9/27/2018 8:19 PM, SZEDER Gábor wrote:
On Wed, Sep 26, 2018 at 03:54:38PM -0400, Ben Peart wrote:
The End of Index Entry (EOIE) is used to locate the end of the variable
Nit: perhaps start with:
The End of Index Entry (EOIE) optional extension can be used to ...
to make it clearer
On 9/28/2018 10:21 AM, Ben Peart wrote:
On 9/28/2018 6:01 AM, SZEDER Gábor wrote:
On Tue, Sep 18, 2018 at 11:29:35PM +, Ben Peart wrote:
diff --git a/t/README b/t/README
index 56a417439c..47165f7eab 100644
--- a/t/README
+++ b/t/README
@@ -319,6 +319,10 @@ GIT_TEST_OE_DELTA_SIZE
On 9/28/2018 6:01 AM, SZEDER Gábor wrote:
On Tue, Sep 18, 2018 at 11:29:35PM +, Ben Peart wrote:
diff --git a/t/README b/t/README
index 56a417439c..47165f7eab 100644
--- a/t/README
+++ b/t/README
@@ -319,6 +319,10 @@ GIT_TEST_OE_DELTA_SIZE= exercises the uncommon
pack-objects code
On 9/27/2018 8:26 PM, SZEDER Gábor wrote:
On Wed, Sep 26, 2018 at 03:54:39PM -0400, Ben Peart wrote:
Add support for a new index.threads config setting which will be used to
control the threading code in do_read_index(). A value of 0 will tell the
index code to automatically determine
On 9/27/2018 6:24 PM, Ramsay Jones wrote:
commit 225df8a468 ("ieot: add Index Entry Offset Table (IEOT)
extension", 2018-09-26) added a 'DIV_ROUND_UP(entries, ieot_blocks)
expression, where ieot_blocks was set to zero for a single cpu
platform. This caused an SIGFPE and a core dump in
On 9/26/2018 2:54 PM, Derrick Stolee wrote:
On 9/26/2018 2:43 PM, Thomas Gummerer wrote:
On 09/26, Derrick Stolee wrote:
This is a bit tricky to do, but I will investigate. For some things, the
values can conflict with each other (GIT_TEST_SPLIT_INDEX doesn't play
nicely with other index
On 9/26/2018 2:44 PM, Derrick Stolee wrote:
On 9/26/2018 1:59 PM, Junio C Hamano wrote:
Derrick Stolee writes:
In an effort to ensure new code is reasonably covered by the test
suite, we now have contrib/coverage-diff.sh to combine the gcov output
from 'make coverage-test ; make
.
The high cost of parsing the index extensions is driven by the cache tree
and the untracked cache extensions. As this is currently the longest pole,
any reduction in this time will reduce the overall index load times so is
worth further investigation in another patch series.
Signed-off-by: Ben
n the IEOT. Basically, with V4 indexes, it
generates offsets into blocks of prefix-compressed entries.
Signed-off-by: Ben Peart
---
Documentation/technical/index-format.txt | 18 +++
read-cache.c | 166 +++
2 files changed, 184 insertions(+)
1 - 100 of 658 matches
Mail list logo
