[PATCH v1 0/9] Introducing remote ODBs

2018-06-23 Thread Christian Couder
This is a follow up from the patch series called "odb remote" that I
sent earlier this year, which was itself a follow up from the patch
series called "Promisor remotes and external ODB support" that I sent
earlier this year, which was itself a follow up from previous series.
See the links section for more information.

Following discussions of the previous patch series, the mechanism is
renamed from "odb remote" to "remote odb". I hope that this will will
be the last name change.

As with the previous "odb remote" series, this series is only about
integrating with the promisor/narrow clone work and showing that it
makes it possible to use more than one promisor remote. Everything
that is not necessary for that integration has been removed for now
(though you can still find it in one of my branches on GitHub if you
want).

There is one test in patch 8/9 that shows that more than one promisor
remote can now be used, but I still feel that it could be interesting
to add other such tests, so I am open to ideas in this area.

Changes compared to previous patch series


  - big rename from "odb remote" to "remote odb"
  - rename of field from "dealer" to "remote" in struct odb_helper
  - add comments around struct odb_helper
  - improve a few commit messages
  - add a documentation patch

High level overview of this patch series


  - Patch 1/9:

This makes functions in fetch-object.c return an error code, which is
necessary to later tell that they failed and try another remote odb
when there is more than one. This could also just be seen as a fix to
these functions.

  - Patch 2/9:

This introduces the minimum infrastructure for remote odbs.

  - Patches 3/9 and 4/9:

These patches implement remote_odb_get_direct() and
remote_odb_get_many_direct() using the functions from fetch-object.c.
These new functions will be used in following patches to replace the
functions from fetch-object.c.

  - Patch 5/9:

This implement remote_odb_reinit() which will be needed to reparse the
remote odb configuration.

  - Patches 6/9 and 7/9:

These patches integrate the remote odb mechanism into the
promisor/narrow clone code. The "extensions.partialclone" config
option is replaced by "odb..promisorRemote" and
"core.partialclonefilter" is replaced by
"odb..partialclonefilter".

  - Patch 8/9:

This adds a test case that shows that now more than one promisor
remote can be used.

  - Patch 9/9:

This starts documenting the remote odb mechanism.

Discussion
~~

I am not sure that it is ok to completely replace the
"extensions.partialclone" config option. Even if it is fully replaced,
no "extensions.remoteodb" is implemented in these patches, as maybe
the "extensions.partialclone" name could be kept even if the
underlying mechanism is the remote odb mechanism.

Anyway I think that the remote odb mechanism is much more extensible,
so I think using "extensions.partialclone" to specify a promisor
remote should be at least deprecated.

Links
~

This patch series on GitHub:

https://github.com/chriscool/git/commits/remote-odb

Previous "odb remote" series:

https://public-inbox.org/git/20180513103232.17514-1-chrisc...@tuxfamily.org/
https://github.com/chriscool/git/commits/odb-remote

Version 1 and 2 of the "Promisor remotes and external ODB support" series:

https://public-inbox.org/git/20180103163403.11303-1-chrisc...@tuxfamily.org/
https://public-inbox.org/git/20180319133147.15413-1-chrisc...@tuxfamily.org/

Version 1 and 2 of the "Promisor remotes and external ODB support" series on 
GitHub:

https://github.com/chriscool/git/commits/gl-small-promisor-external-odb12
https://github.com/chriscool/git/commits/gl-small-promisor-external-odb71


Christian Couder (9):
  fetch-object: make functions return an error code
  Add initial remote odb support
  remote-odb: implement remote_odb_get_direct()
  remote-odb: implement remote_odb_get_many_direct()
  remote-odb: add remote_odb_reinit()
  Use remote_odb_get_direct() and has_remote_odb()
  Use odb.origin.partialclonefilter instead of core.partialclonefilter
  t0410: test fetching from many promisor remotes
  Documentation/config: add odb..promisorRemote

 Documentation/config.txt  |   5 ++
 Makefile  |   2 +
 builtin/cat-file.c|   5 +-
 builtin/fetch.c   |  13 ++--
 builtin/gc.c  |   3 +-
 builtin/repack.c  |   3 +-
 cache.h   |   2 -
 connected.c   |   3 +-
 environment.c |   1 -
 fetch-object.c|  15 +++--
 fetch-object.h|   6 +-
 list-objects-filter-options.c |  49 --
 list-objects-filter-options.h |   3 +-
 odb-helper.c  |  45 +
 odb-helper.h  |  24 +++
 packfile.c|   3 +-
 remote-odb.c  | 118 ++
 remote-odb.h  

Re: [PATCH v1 0/9] Introducing remote ODBs

2018-06-23 Thread Christian Couder
On Sat, Jun 23, 2018 at 2:18 PM, Christian Couder
 wrote:

> High level overview of this patch series
> 
>
>   - Patch 1/9:

Sorry the patches are numbered X/11 but they should be numbered X/9 as
only the first 9 should be in the series and have been sent.


Re: [PATCH v1 0/9] Introducing remote ODBs

2018-06-25 Thread Junio C Hamano
Christian Couder  writes:

> This is a follow up from the patch series called "odb remote" that I
> ...

Just an early warning, as I haven't even complained on patch titles
of these patches in the series ;-)

5702.20 and 5702.21 seems to fail in standalone test, when these are
directly queued on top of Git v2.18.0; I haven't looked into the
failure myself (yet).






Re: [PATCH v1 0/9] Introducing remote ODBs

2018-06-25 Thread Eric Sunshine
On Mon, Jun 25, 2018 at 5:49 PM Junio C Hamano  wrote:
> Christian Couder  writes:
> > This is a follow up from the patch series called "odb remote" [...]
>
> 5702.20 and 5702.21 seems to fail in standalone test, when these are
> directly queued on top of Git v2.18.0; I haven't looked into the
> failure myself (yet).

In addition to the t5702 failures, I'm also seeing failures of
t0410.1, t5616.6 and t5616.7 at the tip of 'pu' as of [1], all of
which seem to be related to these changes.

[1]: https://public-inbox.org/git/xmqqin66mql6@gitster-ct.c.googlers.com/


Re: [PATCH v1 0/9] Introducing remote ODBs

2018-06-28 Thread Christian Couder
On Mon, Jun 25, 2018 at 11:49 PM, Junio C Hamano  wrote:
>
> Just an early warning, as I haven't even complained on patch titles
> of these patches in the series ;-)
>
> 5702.20 and 5702.21 seems to fail in standalone test, when these are
> directly queued on top of Git v2.18.0; I haven't looked into the
> failure myself (yet).

Yeah sorry about that.

These tests can be fixed with only
s/extensions.partialClone/odb.magic.promisorRemote/ though.


Re: [PATCH v1 0/9] Introducing remote ODBs

2018-06-28 Thread Christian Couder
On Tue, Jun 26, 2018 at 2:37 AM, Eric Sunshine  wrote:

> In addition to the t5702 failures, I'm also seeing failures of
> t0410.1, t5616.6 and t5616.7 at the tip of 'pu' as of [1], all of
> which seem to be related to these changes.

Yeah but only s/core.partialclonefilter/odb.origin.partialclonefilter/
is needed on top of the fix for for the master branch.

Thanks for the report!