Re: Re-Transmission of blobs?

2013-09-24 Thread Jeff King
On Fri, Sep 20, 2013 at 11:27:15AM +0200, Josef Wolf wrote: Yes. If you know that the receiver has commit X, and you want to know if it has some blob Y, the only way to know for sure is to look at every tree of every commit reachable from X, and see whether any of them references Y.

Re: Re-Transmission of blobs?

2013-09-24 Thread Josef Wolf
On Tue, Sep 24, 2013 at 03:36:13AM -0400, Jeff King wrote: On Fri, Sep 20, 2013 at 11:27:15AM +0200, Josef Wolf wrote: Even without asking, we can assume with great probability that origin/somebranch is available at origin. Bear in mind that the transfer process does not know about

Re: Re-Transmission of blobs?

2013-09-20 Thread Josef Wolf
On Mon, Sep 16, 2013 at 05:55:36PM -0400, Jeff King wrote: On Fri, Sep 13, 2013 at 12:09:35PM +0200, Josef Wolf wrote: I'm not sure I understand correctly. I see that bitmaps can be used to implement set operations. But how comes that walking the graph requires a lot of CPU?

Re: Re-Transmission of blobs?

2013-09-16 Thread Jeff King
On Fri, Sep 13, 2013 at 12:09:35PM +0200, Josef Wolf wrote: I'm not sure I understand correctly. I see that bitmaps can be used to implement set operations. But how comes that walking the graph requires a lot of CPU? Isn't it O(n)? Yes and no. Your n there is the entirety of

Re: Re-Transmission of blobs?

2013-09-13 Thread Josef Wolf
On Thu, Sep 12, 2013 at 03:44:53PM -0400, Jeff King wrote: On Thu, Sep 12, 2013 at 12:35:32PM +0200, Josef Wolf wrote: I'm not sure I understand correctly. I see that bitmaps can be used to implement set operations. But how comes that walking the graph requires a lot of CPU? Isn't it

Re: Re-Transmission of blobs?

2013-09-13 Thread Josef Wolf
On Thu, Sep 12, 2013 at 08:06:35PM +, Pyeron, Jason J CTR (US) wrote: Yes, but it is those awfully slow connections (slower that the looping issue) which happen to always drop while cloning from our office. And the round trip should be mitigated by http-keep-alives. [ ... ] But, again if

RE: Re-Transmission of blobs?

2013-09-13 Thread Jason Pyeron
-Original Message- From: Josef Wolf Sent: Friday, September 13, 2013 6:23 On Thu, Sep 12, 2013 at 08:06:35PM +, Pyeron, Jason J CTR (US) wrote: Yes, but it is those awfully slow connections (slower that the looping issue) which happen to always drop while cloning from

Re: Re-Transmission of blobs?

2013-09-13 Thread Duy Nguyen
On Fri, Sep 13, 2013 at 3:06 AM, Pyeron, Jason J CTR (US) jason.j.pyeron@mail.mil wrote: But, again if the connection drops, we have already lost the delta advantage. I would think the scenario would go like this: git clone url://blah/blah [fail] cd blah git clone --resume #uses normal

Re: Re-Transmission of blobs?

2013-09-12 Thread Josef Wolf
On Mi, Sep 11, 2013 at 10:14:54 -0700, Junio C Hamano wrote: Josef Wolf j...@raven.inka.de writes: On Di, Sep 10, 2013 at 10:51:02 -0700, Junio C Hamano wrote: Consider this simple history with only a handful of commits (as usual, time flows from left to right): E

Re: Re-Transmission of blobs?

2013-09-12 Thread Jeff King
On Thu, Sep 12, 2013 at 09:42:41AM +0200, Josef Wolf wrote: There are some work being done to optimize this further using various techniques, but they are not ready yet. And this still stands. Do you have a pointer or something? I'd like to check out whether I can contribute to

Re: Re-Transmission of blobs?

2013-09-12 Thread Josef Wolf
On Do, Sep 12, 2013 at 05:23:40 -0400, Jeff King wrote: On Thu, Sep 12, 2013 at 09:42:41AM +0200, Josef Wolf wrote: I think Junio is referring to the reachability bitmap work. We may know that the other side has commit E (and therefore every object reachable from it), but we do not walk the

RE: Re-Transmission of blobs?

2013-09-12 Thread Pyeron, Jason J CTR (US)
-Original Message- From: Jeff King Sent: Thursday, September 12, 2013 5:24 AM On Thu, Sep 12, 2013 at 09:42:41AM +0200, Josef Wolf wrote: There are some work being done to optimize this further using various techniques, but they are not ready yet. And this still

Re: Re-Transmission of blobs?

2013-09-12 Thread Jeff King
On Thu, Sep 12, 2013 at 12:35:32PM +0200, Josef Wolf wrote: I'm not sure I understand correctly. I see that bitmaps can be used to implement set operations. But how comes that walking the graph requires a lot of CPU? Isn't it O(n)? Yes and no. Your n there is the entirety of history. Whereas

Re: Re-Transmission of blobs?

2013-09-12 Thread Jeff King
On Thu, Sep 12, 2013 at 12:45:44PM +, Pyeron, Jason J CTR (US) wrote: If the rules of engagement are change a bit, the server side can be release from most of its work (CPU/IO). Client does the following, looping as needed: Heads=server-heads(); KnownCommits=Local-AllCommits();

RE: Re-Transmission of blobs?

2013-09-12 Thread Pyeron, Jason J CTR (US)
-Original Message- From: Jeff King Sent: Thursday, September 12, 2013 3:57 PM On Thu, Sep 12, 2013 at 12:45:44PM +, Pyeron, Jason J CTR (US) wrote: If the rules of engagement are change a bit, the server side can be release from most of its work (CPU/IO). Client does

Re: Re-Transmission of blobs?

2013-09-11 Thread Josef Wolf
On Di, Sep 10, 2013 at 10:51:02 -0700, Junio C Hamano wrote: Consider this simple history with only a handful of commits (as usual, time flows from left to right): E / A---B---C---D where D is at the tip of the sending side, E is at the tip of the

Re: Re-Transmission of blobs?

2013-09-11 Thread Junio C Hamano
Josef Wolf j...@raven.inka.de writes: On Di, Sep 10, 2013 at 10:51:02 -0700, Junio C Hamano wrote: Consider this simple history with only a handful of commits (as usual, time flows from left to right): E / A---B---C---D where D is at the tip of the

Re-Transmission of blobs?

2013-09-10 Thread Josef Wolf
Hello, as we all know, files are identified by their SHA. Thus I had the impression that when transfering files, git would know by the SHA whether a given file is already available in the destination repository and the transfer would be of no use. But this don't seem to be tha case. Lets see this

Re: Re-Transmission of blobs?

2013-09-10 Thread Junio C Hamano
Josef Wolf j...@raven.inka.de writes: as we all know, files are identified by their SHA. Thus I had the impression that when transfering files, git would know by the SHA whether a given file is already available in the destination repository and the transfer would be of no use. That is