Re: [PATCH v3 0/8] Hiding refs

2014-03-18 Thread Duy Nguyen
On Tue, Mar 18, 2014 at 9:27 PM, Duy Nguyen pclo...@gmail.com wrote: I thought about GIT_CAPABILITIES= git-upload-pack ... (and actually added it in pack-protocol.txt then deleted). The thing is, if you want to new upload-pack, you would need new git-upload-pack at the remote end that must

Re: [PATCH v3 0/8] Hiding refs

2014-03-18 Thread Duy Nguyen
On Tue, Mar 18, 2014 at 12:17:39AM -0400, Jeff King wrote: On Fri, Mar 14, 2014 at 05:09:45PM -0700, Shawn Pearce wrote: On Fri, Mar 14, 2014 at 4:30 PM, Duy Nguyen pclo...@gmail.com wrote: On Fri, Mar 14, 2014 at 11:45 PM, Shawn Pearce spea...@spearce.org wrote: You missed the

Re: [PATCH v3 0/8] Hiding refs

2014-03-17 Thread Jeff King
On Fri, Mar 14, 2014 at 05:09:45PM -0700, Shawn Pearce wrote: On Fri, Mar 14, 2014 at 4:30 PM, Duy Nguyen pclo...@gmail.com wrote: On Fri, Mar 14, 2014 at 11:45 PM, Shawn Pearce spea...@spearce.org wrote: You missed the SSH case. It doesn't have this slot to hide the data into. Right

Re: [PATCH v3 0/8] Hiding refs

2014-03-17 Thread Jeff King
On Sat, Mar 15, 2014 at 08:23:08AM +0700, Duy Nguyen wrote: The default would start at false, and people who know their server is very up-to-date can turn it on. And then when many server implementations support it, flip the default to auto. And either leave it there forever, or

Re: [PATCH v3 0/8] Hiding refs

2014-03-14 Thread Duy Nguyen
On Wed, Mar 12, 2014 at 3:36 AM, Jeff King p...@peff.net wrote: If the client is limited to setting a few flags, then something like http can get away with: GET foo.git/info/refs?service=git-upload-packadvertise-symrefsrefspec=refs/heads/* And it does not need to worry about upload-pack2

Re: [PATCH v3 0/8] Hiding refs

2014-03-14 Thread Shawn Pearce
On Fri, Mar 14, 2014 at 5:37 AM, Duy Nguyen pclo...@gmail.com wrote: On Wed, Mar 12, 2014 at 3:36 AM, Jeff King p...@peff.net wrote: If the client is limited to setting a few flags, then something like http can get away with: GET

Re: [PATCH v3 0/8] Hiding refs

2014-03-14 Thread Duy Nguyen
On Fri, Mar 14, 2014 at 11:45 PM, Shawn Pearce spea...@spearce.org wrote: On Fri, Mar 14, 2014 at 5:37 AM, Duy Nguyen pclo...@gmail.com wrote: On Wed, Mar 12, 2014 at 3:36 AM, Jeff King p...@peff.net wrote: If the client is limited to setting a few flags, then something like http can get away

Re: [PATCH v3 0/8] Hiding refs

2014-03-14 Thread Shawn Pearce
On Fri, Mar 14, 2014 at 4:30 PM, Duy Nguyen pclo...@gmail.com wrote: On Fri, Mar 14, 2014 at 11:45 PM, Shawn Pearce spea...@spearce.org wrote: You missed the SSH case. It doesn't have this slot to hide the data into. Right now we run this for ssh case: ssh host git-upload-pack repo-path. New

Re: [PATCH v3 0/8] Hiding refs

2014-03-14 Thread Duy Nguyen
On Tue, Mar 11, 2014 at 8:49 AM, Jeff King p...@peff.net wrote: Right, I recall the general feeling being that such a system would work, and the transition would be managed by a config variable like remote.*.useUploadPack2. Probably with settings like: true: always try, but allow fall

Re: [PATCH v3 0/8] Hiding refs

2014-03-11 Thread Junio C Hamano
Jeff King p...@peff.net writes: I think the main flag of interest is giving an fnmatch pattern to limit the advertised refs. There could potentially be others, but I do not know of any offhand. One thing that comes to mind is where symrefs point at, which we failed to add the last time around

Re: [PATCH v3 0/8] Hiding refs

2014-03-11 Thread Jeff King
On Tue, Mar 11, 2014 at 12:32:37PM -0700, Junio C Hamano wrote: Jeff King p...@peff.net writes: I think the main flag of interest is giving an fnmatch pattern to limit the advertised refs. There could potentially be others, but I do not know of any offhand. One thing that comes to

Re: [PATCH v3 0/8] Hiding refs

2014-03-11 Thread Junio C Hamano
Jeff King p...@peff.net writes: On Tue, Mar 11, 2014 at 12:32:37PM -0700, Junio C Hamano wrote: Jeff King p...@peff.net writes: I think the main flag of interest is giving an fnmatch pattern to limit the advertised refs. There could potentially be others, but I do not know of any

Re: [PATCH v3 0/8] Hiding refs

2014-03-11 Thread Jeff King
On Tue, Mar 11, 2014 at 01:25:23PM -0700, Junio C Hamano wrote: Yeah, good idea. I might be misremembering some complications, but we can probably do it with: 1. Teach the client to send an advertise-symrefs flag before the ref advertisement. 2. Teach the server to include

Re: [PATCH v3 0/8] Hiding refs

2014-03-10 Thread Jeff King
On Sun, Feb 23, 2014 at 09:44:14AM +0700, Duy Nguyen wrote: (Digging up an old thread about initial refs listing in git protocol) And now I am responding to it slowly. :) For that to work, the new server needs to wait for the client to speak first. How would that server handle old clients

Re: [PATCH v3 0/8] Hiding refs

2014-02-22 Thread Duy Nguyen
(Digging up an old thread about initial refs listing in git protocol) On Thu, Feb 7, 2013 at 7:12 AM, Junio C Hamano gits...@pobox.com wrote: Ævar Arnfjörð Bjarmason ava...@gmail.com writes: I think there's a simpler way to do this, which is that: * New clients supporting v2 of the protocol

Re: [PATCH v3 0/8] Hiding refs

2013-02-09 Thread Junio C Hamano
Jed Brown j...@59a2.org writes: I believe that my use case would be well supported if git could push and pull unadvertised refs, as long as basic operations were not slowed down by the existence of a very large number of such refs. I am not sure about pushing part, but the jc/fetch-raw-sha1

Re: [PATCH v3 0/8] Hiding refs

2013-02-09 Thread Jed Brown
Junio C Hamano gits...@pobox.com writes: I am not sure about pushing part, but the jc/fetch-raw-sha1 topic (split from the main jc/hidden-refs topic) should allow your script, after the client learns the set of smudged object names, to ask for git fetch $there $sha1_1 $sha1_2 ... Well,

Re: [PATCH v3 0/8] Hiding refs

2013-02-07 Thread Ævar Arnfjörð Bjarmason
On Thu, Feb 7, 2013 at 1:16 AM, Jeff King p...@peff.net wrote: On Wed, Feb 06, 2013 at 04:12:10PM -0800, Junio C Hamano wrote: Ævar Arnfjörð Bjarmason ava...@gmail.com writes: I think there's a simpler way to do this, which is that: * New clients supporting v2 of the protocol send some

Re: [PATCH v3 0/8] Hiding refs

2013-02-07 Thread Jed Brown
Michael Haggerty mhag...@alum.mit.edu writes: A first weakness of your proposal is that even though the hidden refs are (optionally) fetchable, there is *no* way to discover them remotely or to bulk-download them; they would have to be retrieved one by one using out-of-band information. And

Re: [PATCH v3 0/8] Hiding refs

2013-02-07 Thread Junio C Hamano
Jeff King p...@peff.net writes: If the new client can handle the old-style server's response, then the server can start blasting out refs (optionally after a timeout) and stop when the client interrupts with hey, wait, I can speak the new protocol. The server just has to include you can

Re: [PATCH v3 0/8] Hiding refs

2013-02-06 Thread Michael Haggerty
On 02/05/2013 06:27 PM, Junio C Hamano wrote: Michael Haggerty mhag...@alum.mit.edu writes: I would again like to express my discomfort about this feature, which is already listed as will merge to next. Do not take will merge to next too literally. One major purpose of marking a topic as

Re: [PATCH v3 0/8] Hiding refs

2013-02-06 Thread Duy Nguyen
On Tue, Feb 5, 2013 at 5:29 PM, Michael Haggerty mhag...@alum.mit.edu wrote: Hiderefs creates a dark corner of a remote git repo that can hold arbitrary content that is impossible for anybody to discover but nevertheless possible for anybody to download (if they know the name of a hidden

Re: [PATCH v3 0/8] Hiding refs

2013-02-06 Thread Junio C Hamano
Duy Nguyen pclo...@gmail.com writes: On Tue, Feb 5, 2013 at 5:29 PM, Michael Haggerty mhag...@alum.mit.edu wrote: Hiderefs creates a dark corner of a remote git repo that can hold arbitrary content that is impossible for anybody to discover but nevertheless possible for anybody to download

Re: [PATCH v3 0/8] Hiding refs

2013-02-06 Thread Jonathan Nieder
Junio C Hamano wrote: Duy Nguyen pclo...@gmail.com writes: On Tue, Feb 5, 2013 at 5:29 PM, Michael Haggerty mhag...@alum.mit.edu wrote: Hiderefs creates a dark corner of a remote git repo [...] Or you can think hiderefs is the first step to addressing the initial ref advertisment problem.

Re: [PATCH v3 0/8] Hiding refs

2013-02-06 Thread Jonathan Nieder
Michael Haggerty wrote: Scenario 1: Some providers junk up their users' repositories with content that is not created by the repository's owner and that the owner doesn't want to appear to vouch for (e.g., GitHub pull requests). These references might sometimes be useful to fetch, singly or

Re: [PATCH v3 0/8] Hiding refs

2013-02-06 Thread Michael Haggerty
On 02/06/2013 08:17 PM, Junio C Hamano wrote: Duy Nguyen pclo...@gmail.com writes: On Tue, Feb 5, 2013 at 5:29 PM, Michael Haggerty mhag...@alum.mit.edu wrote: Hiderefs creates a dark corner of a remote git repo that can hold arbitrary content that is impossible for anybody to discover but

Re: [PATCH v3 0/8] Hiding refs

2013-02-06 Thread Michael Haggerty
On 02/06/2013 08:55 PM, Jonathan Nieder wrote: Michael Haggerty wrote: [...] But now every time I do a gitk --all or git log --decorate, the output is cluttered with all of his references (most of which are just old versions of references from the upstream repository that we both use). I

Re: [PATCH v3 0/8] Hiding refs

2013-02-06 Thread Junio C Hamano
Michael Haggerty mhag...@alum.mit.edu writes: On 02/06/2013 08:17 PM, Junio C Hamano wrote: ... Yes, the first three patches sound much more reasonable if this is the goal. ... I think that a more useful interim solution would be to make it easy to have two URLs accessing a single git

Re: [PATCH v3 0/8] Hiding refs

2013-02-06 Thread Ævar Arnfjörð Bjarmason
On Wed, Feb 6, 2013 at 8:17 PM, Junio C Hamano gits...@pobox.com wrote: Maybe this should be split up into a different thread, but: The upload-pack-2 service sits on a port different from today's [...]. I think there's a simpler way to do this, which is that: * New clients supporting v2 of

Re: [PATCH v3 0/8] Hiding refs

2013-02-06 Thread Jeff King
On Wed, Feb 06, 2013 at 11:17:06AM -0800, Junio C Hamano wrote: Let me help unconfuse this thread. I think the series as 8-patch series was poorly presented, and separating it into two will help understanding what they are about. The first three: upload-pack: share more code

Re: [PATCH v3 0/8] Hiding refs

2013-02-06 Thread Junio C Hamano
Ævar Arnfjörð Bjarmason ava...@gmail.com writes: I think there's a simpler way to do this, which is that: * New clients supporting v2 of the protocol send some piece of data that would break old servers. * If that fails the new client goes oh jeeze, I guess it's an old server, and

Re: [PATCH v3 0/8] Hiding refs

2013-02-06 Thread Jeff King
On Wed, Feb 06, 2013 at 04:12:10PM -0800, Junio C Hamano wrote: Ævar Arnfjörð Bjarmason ava...@gmail.com writes: I think there's a simpler way to do this, which is that: * New clients supporting v2 of the protocol send some piece of data that would break old servers. * If that

Re: [PATCH v3 0/8] Hiding refs

2013-02-05 Thread Michael Haggerty
On 01/30/2013 07:45 PM, Junio C Hamano wrote: The third round. - Multi-valued variable transfer.hiderefs lists prefixes of ref hierarchies to be hidden from the requests coming over the network. - A configuration optionally allows uploadpack to accept fetch requests for an

Re: [PATCH v3 0/8] Hiding refs

2013-02-05 Thread Jonathan Nieder
Hi Michael, Michael Haggerty wrote: I would again like to express my discomfort about this feature, which is already listed as will merge to next. Frankly, I have the feeling that this feature is being steamrolled in before a community consensus has been reached and indeed before many valid

Re: [PATCH v3 0/8] Hiding refs

2013-02-05 Thread Michael Haggerty
On 02/05/2013 09:33 AM, Jonathan Nieder wrote: Michael Haggerty wrote: I would again like to express my discomfort about this feature, which is already listed as will merge to next. Frankly, I have the feeling that this feature is being steamrolled in before a community consensus has been

Re: Is anyone working on a next-gen Git protocol (Re: [PATCH v3 0/8] Hiding refs)

2013-02-05 Thread Ævar Arnfjörð Bjarmason
On Wed, Jan 30, 2013 at 7:45 PM, Junio C Hamano gits...@pobox.com wrote: The third round. - Multi-valued variable transfer.hiderefs lists prefixes of ref hierarchies to be hidden from the requests coming over the network. - A configuration optionally allows uploadpack to accept

Re: Is anyone working on a next-gen Git protocol (Re: [PATCH v3 0/8] Hiding refs)

2013-02-05 Thread Junio C Hamano
Ævar Arnfjörð Bjarmason ava...@gmail.com writes: Do you have any plans for something that *does* have the reduction of network bandwidth as a primary goal? Uncluttering gives reduction of bandwidth anyway, so I do not see much point in the distinction you seem to be making. Is this what

Re: [PATCH v3 0/8] Hiding refs

2013-02-05 Thread Junio C Hamano
Michael Haggerty mhag...@alum.mit.edu writes: I would again like to express my discomfort about this feature, which is already listed as will merge to next. Do not take will merge to next too literally. One major purpose of marking a topic as such is exactly to solicit comments like this ;-)

Re: [PATCH v3 0/8] Hiding refs

2013-02-05 Thread Junio C Hamano
Jonathan Nieder jrnie...@gmail.com writes: * I didn't see a response to Peff's convincing arguments that this should be a client-side feature rather than a server-side feature [1]. The client can't control the size of the ref advertisement. That is the main motivation if I understood

Re: [PATCH v3 0/8] Hiding refs

2013-02-05 Thread Junio C Hamano
Michael Haggerty mhag...@alum.mit.edu writes: On 02/05/2013 09:33 AM, Jonathan Nieder wrote: Michael Haggerty wrote: I would again like to express my discomfort about this feature, which is already listed as will merge to next. Frankly, I have the feeling that this feature is being

Re: Is anyone working on a next-gen Git protocol (Re: [PATCH v3 0/8] Hiding refs)

2013-02-05 Thread Ævar Arnfjörð Bjarmason
On Tue, Feb 5, 2013 at 5:03 PM, Junio C Hamano gits...@pobox.com wrote: Ævar Arnfjörð Bjarmason ava...@gmail.com writes: Do you have any plans for something that *does* have the reduction of network bandwidth as a primary goal? Uncluttering gives reduction of bandwidth anyway, so I do not

[PATCH v3 0/8] Hiding refs

2013-01-30 Thread Junio C Hamano
The third round. - Multi-valued variable transfer.hiderefs lists prefixes of ref hierarchies to be hidden from the requests coming over the network. - A configuration optionally allows uploadpack to accept fetch requests for an object at the tip of a hidden ref. Elsewhere, we