Duy Nguyen pclo...@gmail.com writes:
On Sat, Feb 15, 2014 at 8:15 AM, Zachary Turner ztur...@chromium.org wrote:
...
2) Use TLS as you suggest and have one fd per pack thread. Probably
the most complicated code change (at least for me, being a first-time
contributor)
It's not so
It shouldn't be hard for us to run some tests with this patch applied.
Will report back in a day or two.
On Tue, Feb 18, 2014 at 9:55 AM, Junio C Hamano gits...@pobox.com wrote:
Duy Nguyen pclo...@gmail.com writes:
On Sat, Feb 15, 2014 at 8:15 AM, Zachary Turner ztur...@chromium.org wrote:
Am 14.02.2014 00:09, schrieb Zachary Turner:
To elaborate a little bit more, you can verify with a sample program
that ReadFile with OVERLAPPED does in fact modify the HANDLE's file
position. The documentation doesn't actually state one way or
another. My original attempt at a patch didn't
(Gah, sorry if you're receiving multiple emails to your personal
addresses, I need to get used to manually setting Plain-text mode
every time I send a message).
For the mixed read, we wouldn't be looking for another caller of
pread() (since it doesn't care what the file pointer is), but instead
a
On Fri, Feb 14, 2014 at 11:15 AM, Zachary Turner ztur...@chromium.org wrote:
For the mixed read, we wouldn't be looking for another caller of pread()
(since it doesn't care what the file pointer is), but instead a caller of
read() or lseek(). In index-pack.c, I see two possible culprits:
1)
On Fri, Feb 14, 2014 at 11:04 AM, Karsten Blees karsten.bl...@gmail.com wrote:
Damn...you're right, multi-threaded git-index-pack works fine, but some tests
fail badly. Mixed reads would have to be from git_mmap, which is the only
other caller of pread().
msysgit used git_mmap() as defined
Am 14.02.2014 20:16, schrieb Zachary Turner:
For the mixed read, we wouldn't be looking for another caller of
pread() (since it doesn't care what the file pointer is), but instead
a caller of read() or lseek() (since those do depend on the current
file pointer). In index-pack.c, I see two
On Sat, Feb 15, 2014 at 2:16 AM, Zachary Turner ztur...@chromium.org wrote:
(Gah, sorry if you're receiving multiple emails to your personal
addresses, I need to get used to manually setting Plain-text mode
every time I send a message).
For the mixed read, we wouldn't be looking for another
On Fri, Feb 14, 2014 at 4:45 PM, Duy Nguyen pclo...@gmail.com wrote:
On Sat, Feb 15, 2014 at 2:16 AM, Zachary Turner ztur...@chromium.org wrote:
(Gah, sorry if you're receiving multiple emails to your personal
addresses, I need to get used to manually setting Plain-text mode
every time I send
On Sat, Feb 15, 2014 at 7:50 AM, Stefan Zager sza...@google.com wrote:
On Fri, Feb 14, 2014 at 4:45 PM, Duy Nguyen pclo...@gmail.com wrote:
On Sat, Feb 15, 2014 at 2:16 AM, Zachary Turner ztur...@chromium.org wrote:
(Gah, sorry if you're receiving multiple emails to your personal
addresses, I
Even if we make that change to use TLS for this case, the
implementation of pread() will still change in such a way that the
semantics of pread() are different on Windows. Is that ok?
Just to summarize, here's the viable approaches I've seen discussed so far:
1) Use _WINVER at compile time to
On Sat, Feb 15, 2014 at 8:15 AM, Zachary Turner ztur...@chromium.org wrote:
Even if we make that change to use TLS for this case, the
implementation of pread() will still change in such a way that the
semantics of pread() are different on Windows. Is that ok?
Just to summarize, here's the
Am 2/12/2014 20:30, schrieb Stefan Zager:
On Wed, Feb 12, 2014 at 11:22 AM, Karsten Blees karsten.bl...@gmail.com
wrote:
Am 12.02.2014 19:37, schrieb Erik Faye-Lund:
ReOpenFile, that's fantastic. Thanks a lot!
...but should be loaded dynamically via GetProcAddress, or are we ready to
drop
Mike Hommey m...@glandium.org writes:
On Wed, Feb 12, 2014 at 08:15:24PM +0100, David Kastrup wrote:
Stefan Zager sza...@chromium.org writes:
On Wed, Feb 12, 2014 at 10:50 AM, David Kastrup d...@gnu.org wrote:
Really, give the above patch a try. I am taking longer to finish it
than
David Kastrup d...@gnu.org writes:
Again, that's with an SSD and ext4 filesystem on GNU/Linux, and there
are no improvements in system time (I/O) except for patch 4 of the
series which helps perhaps 20% or so.
So the benefits of the patch will come into play mostly for big, bad
files on
Johannes Sixt j.s...@viscovery.net writes:
Am 2/12/2014 20:30, schrieb Stefan Zager:
On Wed, Feb 12, 2014 at 11:22 AM, Karsten Blees karsten.bl...@gmail.com
wrote:
Am 12.02.2014 19:37, schrieb Erik Faye-Lund:
ReOpenFile, that's fantastic. Thanks a lot!
...but should be loaded dynamically
On Thu, Feb 13, 2014 at 07:04:02AM +0100, David Kastrup wrote:
Mike Hommey m...@glandium.org writes:
On Wed, Feb 12, 2014 at 08:15:24PM +0100, David Kastrup wrote:
Stefan Zager sza...@chromium.org writes:
On Wed, Feb 12, 2014 at 10:50 AM, David Kastrup d...@gnu.org wrote:
On Thu, Feb 13, 2014 at 06:34:39PM +0900, Mike Hommey wrote:
On Thu, Feb 13, 2014 at 07:04:02AM +0100, David Kastrup wrote:
Mike Hommey m...@glandium.org writes:
On Wed, Feb 12, 2014 at 08:15:24PM +0100, David Kastrup wrote:
Stefan Zager sza...@chromium.org writes:
On Wed, Feb
On Thu, Feb 13, 2014 at 12:27 AM, Johannes Sixt j.s...@viscovery.net wrote:
Am 2/12/2014 20:30, schrieb Stefan Zager:
On Wed, Feb 12, 2014 at 11:22 AM, Karsten Blees karsten.bl...@gmail.com
wrote:
Am 12.02.2014 19:37, schrieb Erik Faye-Lund:
ReOpenFile, that's fantastic. Thanks a lot!
Karsten Blees karsten.blees at gmail.com writes:
Am 12.02.2014 19:37, schrieb Erik Faye-Lund:
On Wed, Feb 12, 2014 at 7:34 PM, Stefan Zager szager at google.com
wrote:
On Wed, Feb 12, 2014 at 10:27 AM, Erik Faye-Lund kusmabite at
gmail.com wrote:
On Wed, Feb 12, 2014 at 7:20 PM, Stefan
Am 13.02.2014 19:38, schrieb Zachary Turner:
The only reason ReOpenFile is necessary at
all is because some code somewhere is mixing read-styles against the same
fd.
I don't understand...ReadFile with OVERLAPPED parameter doesn't modify the
HANDLE's file position, so you should be able
On Thu, Feb 13, 2014 at 2:51 PM, Karsten Blees karsten.bl...@gmail.com wrote:
Am 13.02.2014 19:38, schrieb Zachary Turner:
The only reason ReOpenFile is necessary at
all is because some code somewhere is mixing read-styles against the same
fd.
I don't understand...ReadFile with OVERLAPPED
To elaborate a little bit more, you can verify with a sample program
that ReadFile with OVERLAPPED does in fact modify the HANDLE's file
position. The documentation doesn't actually state one way or
another. My original attempt at a patch didn't have the ReOpenFile,
and we experienced regular
Am 12.02.2014 04:43, schrieb Duy Nguyen:
On Wed, Feb 12, 2014 at 9:02 AM, Robin H. Johnson robb...@gentoo.org wrote:
On Tue, Feb 11, 2014 at 05:54:51PM -0800, Stefan Zager wrote:
We in the chromium project have a keen interest in adding threading to
git in the pursuit of performance for
On Wed, Feb 12, 2014 at 2:54 AM, Stefan Zager sza...@chromium.org wrote:
We in the chromium project have a keen interest in adding threading to
git in the pursuit of performance for lengthy operations (checkout,
status, blame, ...). Our motivation comes from hitting some
performance walls
On Tue, Feb 11, 2014 at 6:11 PM, Duy Nguyen pclo...@gmail.com wrote:
I have no comments about thread safety improvements (well, not yet).
If you have investigated about git performance on chromium
repositories, could you please sum it up? Threading may be an option
to improve performance, but
On Tue, Feb 11, 2014 at 7:43 PM, Duy Nguyen pclo...@gmail.com wrote:
From v1.9 shallow clone should work for all push/pull/clone... so
history depth does not matter (on the client side). As for
gentoo-x86's large worktree, using index v4 and avoid full-tree
operations (e.g. status ., not
On Wed, Feb 12, 2014 at 3:59 AM, Erik Faye-Lund kusmab...@gmail.com wrote:
On Wed, Feb 12, 2014 at 2:54 AM, Stefan Zager sza...@chromium.org wrote:
We are particularly concerned with the performance of msysgit, and we
have already chalked up a significant performance gain by turning on
the
On Wed, Feb 12, 2014 at 7:20 PM, Stefan Zager sza...@google.com wrote:
On Wed, Feb 12, 2014 at 3:59 AM, Erik Faye-Lund kusmab...@gmail.com wrote:
On Wed, Feb 12, 2014 at 2:54 AM, Stefan Zager sza...@chromium.org wrote:
We are particularly concerned with the performance of msysgit, and we
have
Stefan Zager sza...@chromium.org writes:
I'm optimistic that parallelizing the stat calls will yield a further
improvement.
It has already been mentionned in the thread, but in case you overlooked
it: did you look at core.preloadindex? It seems at least very close to
what you want.
--
On Wed, Feb 12, 2014 at 10:27 AM, Erik Faye-Lund kusmab...@gmail.com wrote:
On Wed, Feb 12, 2014 at 7:20 PM, Stefan Zager sza...@google.com wrote:
I don't want to steal the thunder of my coworker, who wrote the
implementation. He plans to submit it upstream soon-ish. It relies
on using the
On Wed, Feb 12, 2014 at 10:33 AM, Matthieu Moy
matthieu@grenoble-inp.fr wrote:
Stefan Zager sza...@chromium.org writes:
I'm optimistic that parallelizing the stat calls will yield a further
improvement.
It has already been mentionned in the thread, but in case you overlooked
it: did you
On Wed, Feb 12, 2014 at 7:34 PM, Stefan Zager sza...@google.com wrote:
On Wed, Feb 12, 2014 at 10:27 AM, Erik Faye-Lund kusmab...@gmail.com wrote:
On Wed, Feb 12, 2014 at 7:20 PM, Stefan Zager sza...@google.com wrote:
I don't want to steal the thunder of my coworker, who wrote the
Stefan Zager sza...@chromium.org writes:
On Tue, Feb 11, 2014 at 6:11 PM, Duy Nguyen pclo...@gmail.com wrote:
I have no comments about thread safety improvements (well, not yet).
If you have investigated about git performance on chromium
repositories, could you please sum it up? Threading
On Wed, Feb 12, 2014 at 10:50 AM, David Kastrup d...@gnu.org wrote:
Stefan Zager sza...@chromium.org writes:
Anything on Windows that touches a lot of files is miserable due to
the usual file system slowness on Windows, and luafv.sys (the UAC file
virtualization driver) seems to make it much
Stefan Zager sza...@chromium.org writes:
On Wed, Feb 12, 2014 at 10:50 AM, David Kastrup d...@gnu.org wrote:
Really, give the above patch a try. I am taking longer to finish it
than anticipated (with a lot due to procrastination but that is,
unfortunately, a large part of my workflow), and
Am 12.02.2014 19:37, schrieb Erik Faye-Lund:
On Wed, Feb 12, 2014 at 7:34 PM, Stefan Zager sza...@google.com wrote:
On Wed, Feb 12, 2014 at 10:27 AM, Erik Faye-Lund kusmab...@gmail.com wrote:
On Wed, Feb 12, 2014 at 7:20 PM, Stefan Zager sza...@google.com wrote:
I don't want to steal the
On Wed, Feb 12, 2014 at 11:22 AM, Karsten Blees karsten.bl...@gmail.com wrote:
Am 12.02.2014 19:37, schrieb Erik Faye-Lund:
On Wed, Feb 12, 2014 at 7:34 PM, Stefan Zager sza...@google.com wrote:
On Wed, Feb 12, 2014 at 10:27 AM, Erik Faye-Lund kusmab...@gmail.com
wrote:
On Wed, Feb 12, 2014
Stefan Zager sza...@chromium.org writes:
... I used the Very Sleepy profiler
to see where all the time was spent on Windows: 55% of the time was
spent in OpenFile, and 25% in CloseFile (both in win32).
This is somewhat interesting.
When we check things out, checkout_paths() has a list of
On Wed, Feb 12, 2014 at 12:06 PM, Junio C Hamano gits...@pobox.com wrote:
Stefan Zager sza...@chromium.org writes:
... I used the Very Sleepy profiler
to see where all the time was spent on Windows: 55% of the time was
spent in OpenFile, and 25% in CloseFile (both in win32).
This is
On Wed, Feb 12, 2014 at 12:27 PM, Stefan Zager sza...@chromium.org wrote:
On Wed, Feb 12, 2014 at 12:06 PM, Junio C Hamano gits...@pobox.com wrote:
Stefan Zager sza...@chromium.org writes:
Calls to write (and preparation of data to be written) will then
remain single-threaded, but it sounds
On Wed, Feb 12, 2014 at 08:15:24PM +0100, David Kastrup wrote:
Stefan Zager sza...@chromium.org writes:
On Wed, Feb 12, 2014 at 10:50 AM, David Kastrup d...@gnu.org wrote:
Really, give the above patch a try. I am taking longer to finish it
than anticipated (with a lot due to
On Wed, Feb 12, 2014 at 12:00:19PM +0100, Karsten Blees wrote:
Am 12.02.2014 04:43, schrieb Duy Nguyen:
On Wed, Feb 12, 2014 at 9:02 AM, Robin H. Johnson robb...@gentoo.org
wrote:
On Tue, Feb 11, 2014 at 05:54:51PM -0800, Stefan Zager wrote:
We in the chromium project have a keen
Am 13.02.2014 00:03, schrieb Mike Hommey:
On Wed, Feb 12, 2014 at 12:00:19PM +0100, Karsten Blees wrote:
Am 12.02.2014 04:43, schrieb Duy Nguyen:
On Wed, Feb 12, 2014 at 9:02 AM, Robin H. Johnson robb...@gentoo.org
wrote:
On Tue, Feb 11, 2014 at 05:54:51PM -0800, Stefan Zager wrote:
We in
On Tue, Feb 11, 2014 at 05:54:51PM -0800, Stefan Zager wrote:
To this end, I'd like to start submitting patches that make the code
base generally more thread-safe and thread-friendly. Right after this
email, I'm going to send the first such patch, which makes the global
list of pack files
We in the chromium project have a keen interest in adding threading to
git in the pursuit of performance for lengthy operations (checkout,
status, blame, ...). Our motivation comes from hitting some
performance walls when working with repositories the size of chromium
and blink:
On Tue, Feb 11, 2014 at 05:54:51PM -0800, Stefan Zager wrote:
We in the chromium project have a keen interest in adding threading to
git in the pursuit of performance for lengthy operations (checkout,
status, blame, ...). Our motivation comes from hitting some
performance walls when working
On Wed, Feb 12, 2014 at 8:54 AM, Stefan Zager sza...@chromium.org wrote:
We in the chromium project have a keen interest in adding threading to
git in the pursuit of performance for lengthy operations (checkout,
status, blame, ...). Our motivation comes from hitting some
performance walls
On Wed, Feb 12, 2014 at 9:02 AM, Robin H. Johnson robb...@gentoo.org wrote:
On Tue, Feb 11, 2014 at 05:54:51PM -0800, Stefan Zager wrote:
We in the chromium project have a keen interest in adding threading to
git in the pursuit of performance for lengthy operations (checkout,
status, blame,
49 matches
Mail list logo
