Un-paged commit messages in git filter-branch's commit-filter?
Hello, I am trying to do a major cleanup of the repository in one of my projects (and switch from git-svn to native git). I have developed a commit-filter script over the last months that massages partially dreadful commit messages into something acceptable. While I am not 100% sure I think that upgrading git has broken it partially. AFAICT since the update the commit-filter does not get the original message anymore but at least the subject/first paragraph is run through a pager or something similar: The first line is broken into multiple lines (i.e. some line breaks are inserted about every 72 characters where none have been before). I have tried to run "git --no-pager filter-branch ..." to no avail. I have briefly looked at the source but could not find any proofs... Any hints would be appreciated. This is how I run my script: tmpvar="$(http://vger.kernel.org/majordomo-info.html
Re: [PATCH 3/4] dir: introduce file_size() to check the size of file
Hey Torsten,
On Sun, Jun 12, 2016 at 4:14 PM, Torsten Bögershausen wrote:
>>> So what I understand, you want something like this:
>>>
>>> +ssize_t file_size_not_zero(const char *filename)
>>> +{
>>> + struct stat st;
>>> + if (stat(filename, &st) < 0)
>>> + return -1;
>>> + return !!st.st_size);
>>> +}
>>
>> For the purpose of bisect_reset(), Yes. BTW a similar function exist
>> in builtin/am.c with the name is_empty_file(). But as Christian points
>> out file_size() could help to refactor other parts of code.
>>
>
> Please allow one or more late comments:
That's perfectly fine.
> If is_empty_file() does what you need, then it can be moved into wrapper.c
> and simply be re-used in your code.
Thanks for informing. I was unaware about the use of wrapper.c
> If you want to introduce a new function, that can be used for other
> refactoring,
> then the whole thing would ideally go into a single commit,
> or into a single series.
> That may probably be out of the scope for your current efforts ?
On re-thinking, I think introducing file_size() is out of the scope
for the current efforts and I will stick to is_empty_file(). Will move
it to wrapper.c and then use it in my code. I am not sure but I think
a few other parts could also use is_empty_file(). I will check on that
probably after GSoC as a cleanup.
> What really makes me concern is the mixture of signed - and unsigned:
> ssize_t file_size(const char *filename)
> +{
> + struct stat st;
> + if (stat(filename, &st) < 0)
> + return -1;
> + return xsize_t(st.st_size);
> +}
>
> To my understanding a file size is either 0, or a positive integer.
> Returning -1 is of course impossible with a positive integer.
True.
> So either the function is changed like this:
>
> int file_size(const char *filename, size_t *len)
> +{
> + struct stat st;
> + if (stat(filename, &st) < 0)
> + return -1;
> + *len = xsize_t(st.st_size);
> + return 0;
> +}
>
> Or, if that works for you:
>
> size_t file_size(const char *filename)
> +{
> + struct stat st;
> + if (stat(filename, &st) < 0)
> + return 0;
> + return xsize_t(st.st_size);
> +}
>
> Or, more git-ish:
>
> size_t file_size(const char *filename)
> +{
> + struct stat st;
> + if (stat(filename, &st))
> + return 0;
> + return xsize_t(st.st_size);
> +}
>
> (And then builtin/am.c can be changed to use the new function.
I think I will just skip file_size() for now.
Thanks for your comments!
Regards,
Pranit Bauva
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
DONATION OF $ 1.5 MILLION DOLLARS!!!
My wife and I have awarded you with a donation of $ 1.5 million Dollars from part of our Jackpot Lottery of 161,653,000 Million Pounds, send your name,address, phone for claims. View http://www.bbc.co.uk/news/uk-scotland-glasgow-west-18801698 We await your earliest response and God Bless you. Best of luck. Colin & Chris Weir -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/3] interpret-trailers: don't duplicate option strings
There's no need to do so; the argv strings will last until
the end of the program.
Signed-off-by: Jeff King
---
builtin/interpret-trailers.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index b99ae4b..175f147 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -20,7 +20,7 @@ int cmd_interpret_trailers(int argc, const char **argv, const
char *prefix)
{
int in_place = 0;
int trim_empty = 0;
- struct string_list trailers = STRING_LIST_INIT_DUP;
+ struct string_list trailers = STRING_LIST_INIT_NODUP;
struct option options[] = {
OPT_BOOL(0, "in-place", &in_place, N_("edit files in place")),
--
2.9.0.rc2.149.gd580ccd
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/3] blame,shortlog: don't make local option variables static
There's no need for these option variables to be static,
except that they are referenced by the options array itself,
which is static. But having all of this static is simply
unnecessary and confusing (and inconsistent with most other
commands, which either use a static global option list or a
true function-local one).
Note that in some cases we may need to actually initialize
the variables (since we cannot rely on BSS to do so). This
is a net improvement to readability, though, as we can use
the more verbose initializers for our string_lists.
Signed-off-by: Jeff King
---
builtin/blame.c| 12 ++--
builtin/shortlog.c | 6 +++---
2 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/builtin/blame.c b/builtin/blame.c
index 21f42b0..80d2431 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -2522,12 +2522,12 @@ int cmd_blame(int argc, const char **argv, const char
*prefix)
enum object_type type;
struct commit *final_commit = NULL;
- static struct string_list range_list;
- static int output_option = 0, opt = 0;
- static int show_stats = 0;
- static const char *revs_file = NULL;
- static const char *contents_from = NULL;
- static const struct option options[] = {
+ struct string_list range_list = STRING_LIST_INIT_NODUP;
+ int output_option = 0, opt = 0;
+ int show_stats = 0;
+ const char *revs_file = NULL;
+ const char *contents_from = NULL;
+ const struct option options[] = {
OPT_BOOL(0, "incremental", &incremental, N_("Show blame entries
as we find them, incrementally")),
OPT_BOOL('b', NULL, &blank_boundary, N_("Show blank SHA-1 for
boundary commits (Default: off)")),
OPT_BOOL(0, "root", &show_root, N_("Do not treat root commits
as boundaries (Default: off)")),
diff --git a/builtin/shortlog.c b/builtin/shortlog.c
index bfc082e..f83984e 100644
--- a/builtin/shortlog.c
+++ b/builtin/shortlog.c
@@ -233,11 +233,11 @@ void shortlog_init(struct shortlog *log)
int cmd_shortlog(int argc, const char **argv, const char *prefix)
{
- static struct shortlog log;
- static struct rev_info rev;
+ struct shortlog log = { STRING_LIST_INIT_NODUP };
+ struct rev_info rev;
int nongit = !startup_info->have_repository;
- static const struct option options[] = {
+ const struct option options[] = {
OPT_BOOL('n', "numbered", &log.sort_by_number,
N_("sort output according to the number of commits per
author")),
OPT_BOOL('s', "summary", &log.summary,
--
2.9.0.rc2.149.gd580ccd
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/3] parse_opt_string_list: stop allocating new strings
The parse_opt_string_list callback is basically a thin wrapper to string_list_append() any string options we get. However, it calls: string_list_append(v, xstrdup(arg)); which duplicates the option value. This is wrong for two reasons: 1. If the string list has strdup_strings set, then we are making an extra copy, which is simply leaked. 2. If the string list does not have strdup_strings set, then we pass memory ownership to the string list, but it does not realize this. If we later call string_list_clear(), which can happen if "--no-foo" is passed, then we will leak all of the existing entries. Instead, we should just pass the argument straight to string_list_append, and it can decide whether to copy or not based on its strdup_strings flag. It's possible that some (buggy) caller could be relying on this extra copy (e.g., because it parses some options from an allocated argv array and then frees the array), but it's not likely. For one, we generally only use parse_options on the argv given to us in main(). And two, such a caller is broken anyway, because other option types like OPT_STRING() do not make such a copy. This patch brings us in line with them. Noticed-by: Nguyễn Thái Ngọc Duy Signed-off-by: Jeff King --- parse-options-cb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/parse-options-cb.c b/parse-options-cb.c index 239898d..ba5acf3 100644 --- a/parse-options-cb.c +++ b/parse-options-cb.c @@ -144,7 +144,7 @@ int parse_opt_string_list(const struct option *opt, const char *arg, int unset) if (!arg) return -1; - string_list_append(v, xstrdup(arg)); + string_list_append(v, arg); return 0; } -- 2.9.0.rc2.149.gd580ccd -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/3] fix parse-opt string_list leaks
On Mon, Jun 13, 2016 at 07:08:55AM +0700, Duy Nguyen wrote:
> > So if we are doing the conservative thing, then I think the resulting
> > code should either look like:
> >
> > if (!v->strdup_strings)
> > die("BUG: OPT_STRING_LIST should always use strdup_strings");
> > string_list_append(v, arg);
>
> I agree with the analysis. But this die() would hit all callers
> (except interpret-trailers) because they all initialize with _NODUP
> and setting strdup_strings may require auditing all access to the
> string list in question, e.g. to change string_list_append(v,
> xstrdup(xxx)) to string_list_append(xxx). it may cause side effects if
> we are not careful.
Yep. It is not really fixing anything, so much as alerting us to broken
callers. We'd still have to fix the callers. :)
> So far all callers are in builtin/, I think it will not take much time
> to verify that they all call parse_options() with global argv, then we
> can just lose extra xstrdup() and stick to string_list_append().
> OPTION_STRING already assumes that argument strings are stable because
> they are passed back as-is. Can we go with an easier route, adding a
> comment on top of parse_options() stating that argv[] pointers may be
> passed back as-is and it's up to the caller to xstrdup() appropriately
> before argv[] memory is freed?
Yeah, the two options I laid out were the "conservative" side, where we
didn't make any assumptions about what is in passed into parse_options.
But I agree in practice that it's not likely to be a problem to just
point to the existing strings, and the fact that OPTION_STRING does so
already makes me even more confident.
So I'd suggest these patches:
[1/3]: parse_opt_string_list: stop allocating new strings
[2/3]: interpret-trailers: don't duplicate option strings
[3/3]: blame,shortlog: don't make local option variables static
The first one is what we've been discussing, and the others are just
follow-on cleanups. I stopped short of a fourth patch to convert more
cases of:
static struct string_list foo;
to:
static struct string_list foo = STRING_LIST_INIT_NODUP;
The two are equivalent (mostly due to historical reasons). I tend to
think explicit is better than implicit for something like this (not
because BSS auto-initialization isn't OK, but because there is an
explicit choice of dup/nodup that the writer made, and it is good to
communicate that). But maybe people don't want the extra noise.
-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Repacking a repository uses up all available disk space
On Mon, Jun 13, 2016 at 07:24:51AM +0700, Duy Nguyen wrote: > >> - git fsck --full > >> - git repack -Adl -b --pack-kept-objects > >> - git pack-refs --all > >> - git prune > >> > >> The reason it's split into repack + prune instead of just gc is because > >> we use alternates to save on disk space and try not to prune repos that > >> are used as alternates by other repos in order to avoid potential > >> corruption. > > Isn't this what extensions.preciousObjects is for? It looks like prune > just refuses to run in precious objects mode though, and repack is > skipped by gc, but if that repack command works, maybe we should do > something like that in git-gc? Sort of. preciousObjects is a fail-safe so that you do not ever accidentally run an object-deleting operation where you shouldn't (e.g., in the shared repository used by others as an alternate). So the important step there is that before running "repack", you would want to make sure you have taken into account the reachability of anybody sharing from you. So you could do something like (in your shared repository): git config core.repositoryFormatVersion 1 git config extension.preciousObjects true # this will fail, because it's dangerous! git gc # but we can do it safely if we take into account the other repos for repo in $(somehow_get_list_of_shared_repos); do git fetch $repo +refs/*:refs/shared/$repo/* done git config extension.preciousObjects false git gc git config extension.preciousObjects true So it really is orthogonal to running the various gc commands yourself; it's just here to prevent you shooting yourself in the foot. It may still be useful in such a case to split up the commands in your own script, though. In my case, you'll note that the commands above are racy (what happens if somebody pushes a reference to a shared object between your fetch and the gc invocation?). So we use a custom "repack -k" to get around that (it just keeps everything). You _could_ have gc automatically switch to "-k" in a preciousObjects repository. That's at least safe. But note that it doesn't really solve all of the problems (you do still want to have ref tips from the leaf repositories, because it affects things like bitmaps, and packing order). > BTW Jeff, I think we need more documentation for > extensions.preciousObjects. It's only documented in technical/ which > is practically invisible to all users. Maybe > include::repository-version.txt in config.txt, or somewhere close to > alternates? I'm a little hesitant to document it for end users because it's still pretty experimental. In fact, even we are not using it at GitHub currently. We don't have a big problem with "oops, I accidentally ran something destructive in the shared repository", because nothing except the maintenance script ever even goes into the shared repository. The reason I introduced it in the first place is that I was experimenting with the idea of actually symlinking "objects/" in the leaf repos into the shared repository. That eliminates the object writing in the "fetch" step above, which can be a bottleneck in some cases (not just the I/O, but the shared repo ends up having a _lot_ of refs, and fetch can be pretty slow). But in that case, anything that deletes an object in one of the leaf repos is very dangerous, as it has no idea that its object store is shared with other leaf repos. So I really wanted a fail safe so that running "git gc" wasn't catastrophic. I still think that's a viable approach, but my experiments got side-tracked and I never produced anything worth looking at. So until there's something end users can actually make use of, I'm hesitant to push that stuff into the regular-user documentation. Anybody who is playing with it at this point probably _should_ be familiar with what's in Documentation/technical. -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/3] repack: extend --keep-unreachable to loose objects
If you use "repack -adk" currently, we will pack all objects
that are already packed into the new pack, and then drop the
old packs. However, loose unreachable objects will be left
as-is. In theory these are meant to expire eventually with
"git prune". But if you are using "repack -k", you probably
want to keep things forever and therefore do not run "git
prune" at all. Meaning those loose objects may build up over
time and end up fooling any object-count heuristics (such as
the one done by "gc --auto", though since git-gc does not
support "repack -k", this really applies to whatever custom
scripts people might have driving "repack -k").
With this patch, we instead stuff any loose unreachable
objects into the pack along with the already-packed
unreachable objects. This may seem wasteful, but it is
really no more so than using "repack -k" in the first place.
We are at a slight disadvantage, in that we have no useful
ordering for the result, or names to hand to the delta code.
However, this is again no worse than what "repack -k" is
already doing for the packed objects. The packing of these
objects doesn't matter much because they should not be
accessed frequently (unless they actually _do_ become
referenced, but then they would get moved to a different
part of the packfile during the next repack).
Signed-off-by: Jeff King
---
Documentation/git-repack.txt | 3 ++-
builtin/pack-objects.c | 31 +++
builtin/repack.c | 1 +
t/t7701-repack-unpack-unreachable.sh | 13 +
4 files changed, 47 insertions(+), 1 deletion(-)
diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt
index 68702ea..b58b6b5 100644
--- a/Documentation/git-repack.txt
+++ b/Documentation/git-repack.txt
@@ -138,7 +138,8 @@ other objects in that pack they already have locally.
--keep-unreachable::
When used with `-ad`, any unreachable objects from existing
packs will be appended to the end of the packfile instead of
- being removed.
+ being removed. In addition, any unreachable loose objects will
+ be packed (and their loose counterparts removed).
Configuration
-
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 8f5e358..a2f8cfd 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -44,6 +44,7 @@ static int non_empty;
static int reuse_delta = 1, reuse_object = 1;
static int keep_unreachable, unpack_unreachable, include_tag;
static unsigned long unpack_unreachable_expiration;
+static int pack_loose_unreachable;
static int local;
static int incremental;
static int ignore_packed_keep;
@@ -2378,6 +2379,32 @@ static void add_objects_in_unpacked_packs(struct
rev_info *revs)
free(in_pack.array);
}
+static int add_loose_object(const unsigned char *sha1, const char *path,
+ void *data)
+{
+ enum object_type type = sha1_object_info(sha1, NULL);
+
+ if (type < 0) {
+ warning("loose object at %s could not be examined", path);
+ return 0;
+ }
+
+ add_object_entry(sha1, type, "", 0);
+ return 0;
+}
+
+/*
+ * We actually don't even have to worry about reachability here.
+ * add_object_entry will weed out duplicates, so we just add every
+ * loose object we find.
+ */
+static void add_unreachable_loose_objects(void)
+{
+ for_each_loose_file_in_objdir(get_object_directory(),
+ add_loose_object,
+ NULL, NULL, NULL);
+}
+
static int has_sha1_pack_kept_or_nonlocal(const unsigned char *sha1)
{
static struct packed_git *last_found = (void *)1;
@@ -2547,6 +2574,8 @@ static void get_object_list(int ac, const char **av)
if (keep_unreachable)
add_objects_in_unpacked_packs(&revs);
+ if (pack_loose_unreachable)
+ add_unreachable_loose_objects();
if (unpack_unreachable)
loosen_unused_packed_objects(&revs);
@@ -2647,6 +2676,8 @@ int cmd_pack_objects(int argc, const char **argv, const
char *prefix)
N_("include tag objects that refer to objects to be
packed")),
OPT_BOOL(0, "keep-unreachable", &keep_unreachable,
N_("keep unreachable objects")),
+ OPT_BOOL(0, "pack-loose-unreachable", &pack_loose_unreachable,
+N_("pack loose unreachable objects")),
{ OPTION_CALLBACK, 0, "unpack-unreachable", NULL, N_("time"),
N_("unpack unreachable objects newer than "),
PARSE_OPT_OPTARG, option_parse_unpack_unreachable },
diff --git a/builtin/repack.c b/builtin/repack.c
index 573e66c..f7b7409 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -248,6 +248,7 @@ int cmd_repack(int argc, const char **argv, const char
*prefix)
"--unpack-unreachable");
[PATCH 2/3] repack: add --keep-unreachable option
The usual way to do a full repack (and what is done by
git-gc) is to run "repack -Ad --unpack-unreachable=",
which will loosen any unreachable objects newer than
"", and drop any older ones.
This is a safer alternative to "repack -ad", because
"" becomes a grace period during which we will not
drop any new objects that are about to be referenced.
However, it isn't perfectly safe. It's always possible that
a process is about to reference an old object. Even if that
process were to take care to update the timestamp on the
object, there is no atomicity with a simultaneously running
"repack" process.
So while unlikely, there is a small race wherein we may drop
an object that is in the process of being referenced. If you
do automated repacking on a large number of active
repositories, you may hit it eventually, and the result is a
corrupted repository.
It would be nice to fix that race in the long run, but it's
complicated. In the meantime, there is a much simpler
strategy for automated repository maintenance: do not drop
objects at all. We already have a "--keep-unreachable"
option in pack-objects; we just need to plumb it through
from git-repack.
Note that this _isn't_ plumbed through from git-gc, so at
this point it's strictly a tool for people doing their own
advanced repository maintenance strategy.
Signed-off-by: Jeff King
---
Documentation/git-repack.txt | 6 ++
builtin/repack.c | 9 +
t/t7701-repack-unpack-unreachable.sh | 15 +++
3 files changed, 30 insertions(+)
diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt
index cde7b44..68702ea 100644
--- a/Documentation/git-repack.txt
+++ b/Documentation/git-repack.txt
@@ -134,6 +134,12 @@ other objects in that pack they already have locally.
the write of any objects that would be immediately pruned by
a follow-up `git prune`.
+-k::
+--keep-unreachable::
+ When used with `-ad`, any unreachable objects from existing
+ packs will be appended to the end of the packfile instead of
+ being removed.
+
Configuration
-
diff --git a/builtin/repack.c b/builtin/repack.c
index 858db38..573e66c 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -146,6 +146,7 @@ int cmd_repack(int argc, const char **argv, const char
*prefix)
int pack_everything = 0;
int delete_redundant = 0;
const char *unpack_unreachable = NULL;
+ int keep_unreachable = 0;
const char *window = NULL, *window_memory = NULL;
const char *depth = NULL;
const char *max_pack_size = NULL;
@@ -175,6 +176,8 @@ int cmd_repack(int argc, const char **argv, const char
*prefix)
N_("write bitmap index")),
OPT_STRING(0, "unpack-unreachable", &unpack_unreachable,
N_("approxidate"),
N_("with -A, do not loosen objects older than
this")),
+ OPT_BOOL('k', "keep-unreachable", &keep_unreachable,
+ N_("with -a, repack unreachable objects")),
OPT_STRING(0, "window", &window, N_("n"),
N_("size of the window used for delta
compression")),
OPT_STRING(0, "window-memory", &window_memory, N_("bytes"),
@@ -196,6 +199,10 @@ int cmd_repack(int argc, const char **argv, const char
*prefix)
if (delete_redundant && repository_format_precious_objects)
die(_("cannot delete packs in a precious-objects repo"));
+ if (keep_unreachable &&
+ (unpack_unreachable || (pack_everything & LOOSEN_UNREACHABLE)))
+ die(_("--keep-unreachable and -A are incompatible"));
+
if (pack_kept_objects < 0)
pack_kept_objects = write_bitmaps;
@@ -239,6 +246,8 @@ int cmd_repack(int argc, const char **argv, const char
*prefix)
} else if (pack_everything & LOOSEN_UNREACHABLE) {
argv_array_push(&cmd.args,
"--unpack-unreachable");
+ } else if (keep_unreachable) {
+ argv_array_push(&cmd.args,
"--keep-unreachable");
} else {
argv_array_push(&cmd.env_array,
"GIT_REF_PARANOIA=1");
}
diff --git a/t/t7701-repack-unpack-unreachable.sh
b/t/t7701-repack-unpack-unreachable.sh
index b66e383..f13df43 100755
--- a/t/t7701-repack-unpack-unreachable.sh
+++ b/t/t7701-repack-unpack-unreachable.sh
@@ -122,4 +122,19 @@ test_expect_success 'keep packed objects found only in
index' '
git cat-file blob :file
'
+test_expect_success 'repack -k keeps unreachable packed objects' '
+ # create packed-but-unreachable object
+ sha1=$(echo unreachable-packed | git hash-object -w --stdin) &&
+ pack=$(echo $sha1 | git pack-objects .git/objects/pack/pack) &&
+ git prune-packed
[PATCH 1/3] repack: document --unpack-unreachable option
This was added back in 7e52f56 (gc: do not explode objects which will be immediately pruned, 2012-04-07), but not documented at the time, since it was an internal detail between git-gc and git-repack. However, as people with complicated setups may want to effectively reimplement the steps of git-gc themselves, it is nice for us to document these interfaces. Signed-off-by: Jeff King --- Documentation/git-repack.txt | 6 ++ 1 file changed, 6 insertions(+) diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt index b9c02ce..cde7b44 100644 --- a/Documentation/git-repack.txt +++ b/Documentation/git-repack.txt @@ -128,6 +128,12 @@ other objects in that pack they already have locally. with `-b` or `repack.writeBitmaps`, as it ensures that the bitmapped packfile has the necessary objects. +--unpack-unreachable=:: + When loosening unreachable objects, do not bother loosening any + objects older than ``. This can be used to optimize out + the write of any objects that would be immediately pruned by + a follow-up `git prune`. + Configuration - -- 2.9.0.rc2.149.gd580ccd -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/3] repack --keep-unreachable
On Sun, Jun 12, 2016 at 07:43:27PM -0600, Nasser Grainawi wrote: > On Jun 12, 2016, at 4:13 PM, Jeff King wrote: > > > >At GitHub we actually have a patch to `repack` that keeps all > >objects, reachable or not, in the pack, and use it for all of our > >automated maintenance. Since we don't drop objects at all, we can't > >ever have such a race. Aside from some pathological cases, it wastes > >much less space than you'd expect. We turn the flag off for special > >cases (e.g., somebody has rewound history and wants to expunge a > >sensitive object). > > > >I'm happy to share the "keep everything" patch if you're interested. > > We have the same kind of patch actually (for the same reason), but > back on the shell implementation of repack. It'd be great if you could > share your modern version. Here is a cleaned-up version of what we run at GitHub (so this is a concept that has been exercised for a few years in production, but I had to forward port the patches a bit; I _probably_ didn't introduce any bugs. :) ). The heavy lifting is done by the existing --keep-unreachable option to pack-objects, which Junio added a long time ago[1] in support of a safer "gc --auto". But it doesn't look like we ever documented or exercised it, and "gc --auto" ended up using the loosen-unreachable strategy instead. In fact, the rest of that series seems to have been dropped; I couldn't find any discussion on the list explaining it, or why this one patch was kept (so I don't think anybody upstream has ever used this code, but as I said, we have been doing so for a few years, so I feel confident in it). [1/3]: repack: document --unpack-unreachable option [2/3]: repack: add --keep-unreachable option [3/3]: repack: extend --keep-unreachable to loose objects -Peff [1] http://article.gmane.org/gmane.comp.version-control.git/58413 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Repacking a repository uses up all available disk space
On Jun 12, 2016, at 4:13 PM, Jeff King wrote: > >At GitHub we actually have a patch to `repack` that keeps all >objects, reachable or not, in the pack, and use it for all of our >automated maintenance. Since we don't drop objects at all, we can't >ever have such a race. Aside from some pathological cases, it wastes >much less space than you'd expect. We turn the flag off for special >cases (e.g., somebody has rewound history and wants to expunge a >sensitive object). > >I'm happy to share the "keep everything" patch if you're interested. We have the same kind of patch actually (for the same reason), but back on the shell implementation of repack. It'd be great if you could share your modern version. Nasser -- Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] l10n updates for 2.9.0 rc0
Jiang Xin writes: > Please pull the following git l10n updates. > > The following changes since commit 3a0f269e7c82aa3a87323cb7ae04ac5f129f036b: > > Git 2.9-rc0 (2016-05-23 15:02:48 -0700) > > are available in the git repository at: > > git://github.com/git-l10n/git-po tags/l10n-2.9.0-rc0 > > for you to fetch changes up to ad583ebe0813c5d6a8e7c263d72d934770083d83: > > l10n: ko.po: Update Korean translation (2016-06-12 01:25:58 +0900) Thanks, will do. > > > l10n-2.9.0-rc0 > > > Alexander Shopov (1): > l10n: Updated Bulgarian translation of git (2597t,0f,0u) > > Antonin (1): > l10n: fr.po Fixed grammar mistake > > Changwoo Ryu (1): > l10n: ko.po: Update Korean translation > > Dimitriy Ryazantcev (1): > l10n: ru.po: update Russian translation > > Jean-Noel Avila (1): > l10n: fr.po v2.9.0rnd1 > > Jiang Xin (5): > l10n: git.pot: v2.9.0 round 1 (104 new, 37 removed) > Merge branch 'fix_fr' of git://github.com/jnavila/git > Merge branch 'v2.9.0_rnd1_fr' of git://github.com/jnavila/git > l10n: zh_CN: for git v2.9.0 l10n round 1 > Merge branch 'russian-l10n' of https://github.com/DJm00n/git-po-ru > > Peter Krefting (1): > l10n: sv.po: Update Swedish translation (2597t0f0u) > > Ralf Thielow (1): > l10n: de.po: translate 104 new messages > > Ray Chen (1): > l10n: zh_CN: review for git v2.9.0 l10n round 1 > > Trần Ngọc Quân (1): > l10n: Updated Vietnamese translation (2597t) > > Vasco Almeida (3): > l10n: pt_PT: merge git.pot file > l10n: pt_PT: update according to git-gui glossary > l10n: pt_PT: update Portuguese translation > > po/bg.po| 4689 > +-- > po/de.po| 3310 +++-- > po/fr.po| 3252 +++-- > po/git.pot | 3078 ++- > po/ko.po| 3192 ++-- > po/pt_PT.po | 3893 +++-- > po/ru.po| 3171 ++-- > po/sv.po| 3269 +++-- > po/vi.po| 3274 +++-- > po/zh_CN.po | 3375 ++ > 10 files changed, 19378 insertions(+), 15125 deletions(-) > > -- > Jiang Xin -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Repacking a repository uses up all available disk space
On Mon, Jun 13, 2016 at 5:13 AM, Jeff King wrote: > On Sun, Jun 12, 2016 at 05:54:36PM -0400, Konstantin Ryabitsev wrote: > >> > git gc --prune=now >> >> You are correct, this solves the problem, however I'm curious. The usual >> maintenance for these repositories is a regular run of: >> >> - git fsck --full >> - git repack -Adl -b --pack-kept-objects >> - git pack-refs --all >> - git prune >> >> The reason it's split into repack + prune instead of just gc is because >> we use alternates to save on disk space and try not to prune repos that >> are used as alternates by other repos in order to avoid potential >> corruption. Isn't this what extensions.preciousObjects is for? It looks like prune just refuses to run in precious objects mode though, and repack is skipped by gc, but if that repack command works, maybe we should do something like that in git-gc? BTW Jeff, I think we need more documentation for extensions.preciousObjects. It's only documented in technical/ which is practically invisible to all users. Maybe include::repository-version.txt in config.txt, or somewhere close to alternates? > [2] It's unclear to me if you're passing any options to git-prune, but > you may want to pass "--expire" with a short grace period. Without > any options it prunes every unreachable thing, which can lead to > races if the repository is actively being used. > > At GitHub we actually have a patch to `repack` that keeps all > objects, reachable or not, in the pack, and use it for all of our > automated maintenance. Since we don't drop objects at all, we can't > ever have such a race. Aside from some pathological cases, it wastes > much less space than you'd expect. We turn the flag off for special > cases (e.g., somebody has rewound history and wants to expunge a > sensitive object). > > I'm happy to share the "keep everything" patch if you're interested. Ah ok, I guess this is why we just skip repack. I guess '-Adl -b --pack-kept-objects' is not enough then. -- Duy -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] parse-options-cb.c: use string_list_append_nodup in OPT_STRING_LIST()
On Mon, Jun 13, 2016 at 5:03 AM, Jeff King wrote:
> On Fri, Jun 10, 2016 at 06:57:26PM +0700, Nguyễn Thái Ngọc Duy wrote:
>
>> If the given string list has strdup_strings set (*), the string will be
>> duplicated again. Pointless and leak memory. Ignore that flag.
>>
>> (*) only interpret-trailers.c does it at the moment
>>
>> Signed-off-by: Nguyễn Thái Ngọc Duy
>> ---
>> parse-options-cb.c | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/parse-options-cb.c b/parse-options-cb.c
>> index 239898d..8a1b6e6 100644
>> --- a/parse-options-cb.c
>> +++ b/parse-options-cb.c
>> @@ -144,7 +144,7 @@ int parse_opt_string_list(const struct option *opt,
>> const char *arg, int unset)
>> if (!arg)
>> return -1;
>>
>> - string_list_append(v, xstrdup(arg));
>> + string_list_append_nodup(v, xstrdup(arg));
>
> Hmm. So I agree this is an improvement, in the sense that we are
> double-allocating when v->strdup_strings is set. But I think there's a
> deeper issue here. Why are we always allocating in the first place?
>
> If the memory we are getting in "arg" is not stable, then we _do_ need
> to make a copy of it. But in that case, we want "strdup_strings" to be
> set; without it any time we later run string_list_clear(), we leak the
> allocated memory, because the struct has no idea that it is the owner of
> the memory (and we do call string_list_clear() when we see "--no-foo").
>
> If the memory _is_ stable, then we are fine to add a direct reference to
> it, and can lose the extra xstrdup() here. Only the caller knows for
> sure, so we should be respecting their value of strdup_strings (so lose
> the xstrdup, but keep calling string_list_append()).
>
> In practice, I suspect the memory _is_ stable, because we are generally
> parsing command-line arguments. But it does not hurt to stay on the
> conservative side, and always make a copy (in case we are parsing
> something besides the global argv array) . Apparently I am the original
> author of this code, in c8ba163 (parse-options: add OPT_STRING_LIST
> helper, 2011-06-09), but there's no mention of this point there, in the
> list archives, or in my brain.
>
> So if we are doing the conservative thing, then I think the resulting
> code should either look like:
>
> if (!v->strdup_strings)
> die("BUG: OPT_STRING_LIST should always use strdup_strings");
> string_list_append(v, arg);
I agree with the analysis. But this die() would hit all callers
(except interpret-trailers) because they all initialize with _NODUP
and setting strdup_strings may require auditing all access to the
string list in question, e.g. to change string_list_append(v,
xstrdup(xxx)) to string_list_append(xxx). it may cause side effects if
we are not careful.
So far all callers are in builtin/, I think it will not take much time
to verify that they all call parse_options() with global argv, then we
can just lose extra xstrdup() and stick to string_list_append().
OPTION_STRING already assumes that argument strings are stable because
they are passed back as-is. Can we go with an easier route, adding a
comment on top of parse_options() stating that argv[] pointers may be
passed back as-is and it's up to the caller to xstrdup() appropriately
before argv[] memory is freed?
>
> or:
>
> /* silently enable for convenience */
> v->strdup_strings = 1;
> string_list_append(v, arg);
>
> Of the two, I like the top one as it is less magical, but it would
> require adjusting the initialization of the string-list for most of the
> callers.
>
> -Peff
--
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Repacking a repository uses up all available disk space
On Sun, Jun 12, 2016 at 05:54:36PM -0400, Konstantin Ryabitsev wrote: > > git gc --prune=now > > You are correct, this solves the problem, however I'm curious. The usual > maintenance for these repositories is a regular run of: > > - git fsck --full > - git repack -Adl -b --pack-kept-objects > - git pack-refs --all > - git prune > > The reason it's split into repack + prune instead of just gc is because > we use alternates to save on disk space and try not to prune repos that > are used as alternates by other repos in order to avoid potential > corruption. > > Am I not doing something that needs to be doing in order to avoid the > same problem? Your approach makes sense; we do the same thing at GitHub for the same reasons[1]. The main thing you are missing that gc will do is that it knows the prune-time it is going to feed to git-prune[2], and passes that along to repack. That's what enables the "don't bother ejecting these, because I'm about to delete them" optimization. That option is not documented, because it was always assumed to be an internal thing to git-gc, but it is: git repack ... --unpack-unreachable=5.minutes.ago or whatever. -Peff [1] We don't run the fsck at the front, though, because it's really expensive. I'm not sure it buys you much, either. The repack will do a full walk of the graph, so it gets you a connectivity check, as well as a full content check of the commits and trees. The blobs are copied as-is from the old pack, but there is a checksum on the pack data (to catch any bit flips by the disk storage). So the only thing the fsck is getting you is that it fully reconstructs the deltas for each blob and checks their sha1. That's more robust than a checksum, but it's a lot more expensive. [2] It's unclear to me if you're passing any options to git-prune, but you may want to pass "--expire" with a short grace period. Without any options it prunes every unreachable thing, which can lead to races if the repository is actively being used. At GitHub we actually have a patch to `repack` that keeps all objects, reachable or not, in the pack, and use it for all of our automated maintenance. Since we don't drop objects at all, we can't ever have such a race. Aside from some pathological cases, it wastes much less space than you'd expect. We turn the flag off for special cases (e.g., somebody has rewound history and wants to expunge a sensitive object). I'm happy to share the "keep everything" patch if you're interested. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] parse-options-cb.c: use string_list_append_nodup in OPT_STRING_LIST()
On Fri, Jun 10, 2016 at 06:57:26PM +0700, Nguyễn Thái Ngọc Duy wrote:
> If the given string list has strdup_strings set (*), the string will be
> duplicated again. Pointless and leak memory. Ignore that flag.
>
> (*) only interpret-trailers.c does it at the moment
>
> Signed-off-by: Nguyễn Thái Ngọc Duy
> ---
> parse-options-cb.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/parse-options-cb.c b/parse-options-cb.c
> index 239898d..8a1b6e6 100644
> --- a/parse-options-cb.c
> +++ b/parse-options-cb.c
> @@ -144,7 +144,7 @@ int parse_opt_string_list(const struct option *opt, const
> char *arg, int unset)
> if (!arg)
> return -1;
>
> - string_list_append(v, xstrdup(arg));
> + string_list_append_nodup(v, xstrdup(arg));
Hmm. So I agree this is an improvement, in the sense that we are
double-allocating when v->strdup_strings is set. But I think there's a
deeper issue here. Why are we always allocating in the first place?
If the memory we are getting in "arg" is not stable, then we _do_ need
to make a copy of it. But in that case, we want "strdup_strings" to be
set; without it any time we later run string_list_clear(), we leak the
allocated memory, because the struct has no idea that it is the owner of
the memory (and we do call string_list_clear() when we see "--no-foo").
If the memory _is_ stable, then we are fine to add a direct reference to
it, and can lose the extra xstrdup() here. Only the caller knows for
sure, so we should be respecting their value of strdup_strings (so lose
the xstrdup, but keep calling string_list_append()).
In practice, I suspect the memory _is_ stable, because we are generally
parsing command-line arguments. But it does not hurt to stay on the
conservative side, and always make a copy (in case we are parsing
something besides the global argv array) . Apparently I am the original
author of this code, in c8ba163 (parse-options: add OPT_STRING_LIST
helper, 2011-06-09), but there's no mention of this point there, in the
list archives, or in my brain.
So if we are doing the conservative thing, then I think the resulting
code should either look like:
if (!v->strdup_strings)
die("BUG: OPT_STRING_LIST should always use strdup_strings");
string_list_append(v, arg);
or:
/* silently enable for convenience */
v->strdup_strings = 1;
string_list_append(v, arg);
Of the two, I like the top one as it is less magical, but it would
require adjusting the initialization of the string-list for most of the
callers.
-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Repacking a repository uses up all available disk space
On Sun, Jun 12, 2016 at 05:38:04PM -0400, Jeff King wrote: > > - When attempting to repack, creates millions of files and eventually > > eats up all available disk space > > That means these objects fall into the unreachable category. Git will > prune unreachable loose objects after a grace period based on the > filesystem mtime of the objects; the default is 2 weeks. > > For unreachable packed objects, their mtime is jumbled in with the rest > of the objects in the packfile. So Git's strategy is to "eject" such > objects from the packfiles into individual loose objects, and let them > "age out" of the grace period individually. > > Generally this works just fine, but there are corner cases where you > might have a very large number of such objects, and the loose storage is > much more expensive than the packed (e.g., because each object is stored > individually, not as a delta). > > It sounds like this is the case you're running into. > > The solution is to lower the grace period time, with something like: > > git gc --prune=5.minutes.ago > > or even: > > git gc --prune=now You are correct, this solves the problem, however I'm curious. The usual maintenance for these repositories is a regular run of: - git fsck --full - git repack -Adl -b --pack-kept-objects - git pack-refs --all - git prune The reason it's split into repack + prune instead of just gc is because we use alternates to save on disk space and try not to prune repos that are used as alternates by other repos in order to avoid potential corruption. Am I not doing something that needs to be doing in order to avoid the same problem? Thanks for your help. Regards, -- Konstantin Ryabitsev Linux Foundation Collab Projects Montréal, Québec signature.asc Description: PGP signature
Re: Repacking a repository uses up all available disk space
On Sun, Jun 12, 2016 at 05:25:14PM -0400, Konstantin Ryabitsev wrote: > Hello: > > I have a problematic repository that: > > - Takes up 9GB on disk > - Passes 'git fsck --full' with no errors > - When cloned with --mirror, takes up 38M on the target system Cloning will only copy the objects that are reachable from the refs. So presumably the other 8.9GB is either reachable from reflogs, or not reachable at all (due to rewinding history or deleting branches). > - When attempting to repack, creates millions of files and eventually > eats up all available disk space That means these objects fall into the unreachable category. Git will prune unreachable loose objects after a grace period based on the filesystem mtime of the objects; the default is 2 weeks. For unreachable packed objects, their mtime is jumbled in with the rest of the objects in the packfile. So Git's strategy is to "eject" such objects from the packfiles into individual loose objects, and let them "age out" of the grace period individually. Generally this works just fine, but there are corner cases where you might have a very large number of such objects, and the loose storage is much more expensive than the packed (e.g., because each object is stored individually, not as a delta). It sounds like this is the case you're running into. The solution is to lower the grace period time, with something like: git gc --prune=5.minutes.ago or even: git gc --prune=now That will prune the unreachable objects immediately (and the packfile ejector is smart enough to skip ejecting any file that would just get deleted immediately anyway). -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Repacking a repository uses up all available disk space
Hello: I have a problematic repository that: - Takes up 9GB on disk - Passes 'git fsck --full' with no errors - When cloned with --mirror, takes up 38M on the target system - When attempting to repack, creates millions of files and eventually eats up all available disk space Repacking the result of 'git clone --mirror' shows no problem, so it's got to be something really weird with that particular instance of the repository. If anyone is interested in poking at this particular problem to figure out what causes the repack process to eat up all available disk space, you can find the tarball of the problematic repository here: http://mricon.com/misc/src.git.tar.xz (warning: 6.6GB) You can clone the non-problematic version of this repository from git://codeaurora.org/quic/chrome4sdp/breakpad/breakpad/src.git Best, -- Konstantin Ryabitsev Linux Foundation Collab Projects Montréal, Québec signature.asc Description: PGP signature
[PATCH] Document the 'svn propset' command.
Junio + all, A week ago I was requested to provide documentation for the 'svn propset' command. I have attached a diff off of the 'maint' branch for this, however it seems to apply cleanly to 'master' as well. Thank you for your patience. This is also available on my github here: https://github.com/splbio/git/tree/document_propset --- Documentation/git-svn.txt | 14 ++ 1 file changed, 14 insertions(+) diff --git a/Documentation/git-svn.txt b/Documentation/git-svn.txt index fb23a98..e104824 100644 --- a/Documentation/git-svn.txt +++ b/Documentation/git-svn.txt @@ -459,6 +459,20 @@ Any other arguments are passed directly to 'git log' Gets the Subversion property given as the first argument, for a file. A specific revision can be specified with -r/--revision. +'propset':: + Sets the Subversion property given as the first argument, to the + value given as the second argument for the file given as the + third argument. ++ +Example: ++ + +git svn propset svn:keywords "FreeBSD=%H" devel/py-tipper/Makefile + ++ +This will set the property 'svn:keywords' to 'FreeBSD=%H' for the file +'devel/py-tipper/Makefile'. + 'show-externals':: Shows the Subversion externals. Use -r/--revision to specify a specific revision. -- 2.7.1 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: lib-httpd/apache.conf and mod_unixd.so
On Sun, Jun 12, 2016 at 8:00 PM, Jeff King wrote: > On Sun, Jun 12, 2016 at 05:48:42PM +0700, Duy Nguyen wrote: > >> A quick search shows that loading mod_unixd.so will do (and did make >> httpd run for me). Problem is, can I just update apache.conf to load >> it when apache version >= 2.4? I don't know if doing that would cause >> failure for other people because mod_unixd has existed since 2.2 and >> people have run httpd tests fine so far... > > Are you running a version that has 5922322 (t/lib-httpd: load mod_unixd, > 2015-05-11)? Nope. This is nd/shallow-deepen which has a really old base. Good to know it's already fixed. -- Duy -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[no subject]
Greetings, Have you been looking for finance options for your new home purchase, construction, refinance, medical cash, family cash, personal or business purpose. Welcome to the future! Finance made easy with us. Contact us as we offer our finance service at a low and affordable interest rate for long and short cash term,reply to this Email for more Info -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: lib-httpd/apache.conf and mod_unixd.so
On Sun, Jun 12, 2016 at 05:48:42PM +0700, Duy Nguyen wrote: > A quick search shows that loading mod_unixd.so will do (and did make > httpd run for me). Problem is, can I just update apache.conf to load > it when apache version >= 2.4? I don't know if doing that would cause > failure for other people because mod_unixd has existed since 2.2 and > people have run httpd tests fine so far... Are you running a version that has 5922322 (t/lib-httpd: load mod_unixd, 2015-05-11)? -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] lib-httpd.sh: print error.log on error
On Sun, Jun 12, 2016 at 05:41:54PM +0700, Nguyễn Thái Ngọc Duy wrote:
> Failure to bring up httpd for testing is not considered an error, so the
> trash directory, which contains this error.log file, is removed and we
> don't know what made httpd fail to start. Improve the situation a bit.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy
> ---
> t/lib-httpd.sh | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/t/lib-httpd.sh b/t/lib-httpd.sh
> index f9f3e5f..5b8de38 100644
> --- a/t/lib-httpd.sh
> +++ b/t/lib-httpd.sh
> @@ -180,6 +180,7 @@ start_httpd() {
> if test $? -ne 0
> then
> trap 'die' EXIT
> + cat "$HTTPD_ROOT_PATH"/error.log 2>/dev/null
> test_skip_or_die $GIT_TEST_HTTPD "web server setup failed"
> fi
I like the idea of giving more data on error, but I think this will
break the TAP output and confuse anything parsing the output of the
tests, like prove (I think arbitrary output should have "#" prepended).
Also (or alternatively), it should probably only happen when we are in
verbose mode (it's not taken care of for us as usual because tests call
start_httpd outside of a test_expect_ block). I think this eliminates
the need to deal with the TAP thing (because our usual "-v" output is
not TAP-compliant).
-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 27/27] fetch, upload-pack: --deepen=N extends shallow boundary by N commits
In git-fetch, --depth argument is always relative with the latest
remote refs. This makes it a bit difficult to cover this use case,
where the user wants to make the shallow history, say 3 levels
deeper. It would work if remote refs have not moved yet, but nobody
can guarantee that, especially when that use case is performed a
couple months after the last clone or "git fetch --depth". Also,
modifying shallow boundary using --depth does not work well with
clones created by --since or --not.
This patch fixes that. A new argument --deepen= will add more (*)
parent commits to the current history regardless of where remote refs
are.
Have/Want negotiation is still respected. So if remote refs move, the
server will send two chunks: one between "have" and "want" and another
to extend shallow history. In theory, the client could send no "want"s
in order to get the second chunk only. But the protocol does not allow
that. Either you send no want lines, which means ls-remote; or you
have to send at least one want line that carries deep-relative to the
server..
The main work was done by Dongcan Jiang. I fixed it up here and there.
And of course all the bugs belong to me.
(*) We could even support --deepen= where is negative. In that
case we can cut some history from the shallow clone. This operation
(and --depth=) does not require interaction with remote
side (and more complicated to implement as a result).
Helped-by: Duy Nguyen
Helped-by: Eric Sunshine
Helped-by: Junio C Hamano
Signed-off-by: Dongcan Jiang
Signed-off-by: Nguyễn Thái Ngọc Duy
---
Documentation/fetch-options.txt | 5 +
Documentation/git-fetch-pack.txt | 5 +
Documentation/gitremote-helpers.txt | 4
Documentation/technical/protocol-capabilities.txt | 7 ++
builtin/fetch-pack.c | 4
builtin/fetch.c | 14 +++-
fetch-pack.c | 3 +++
fetch-pack.h | 1 +
remote-curl.c | 14 +++-
t/t5500-fetch-pack.sh | 23
t/t5539-fetch-http-shallow.sh | 26 +++
transport-helper.c| 1 +
transport.c | 4
transport.h | 4
upload-pack.c | 23
15 files changed, 132 insertions(+), 6 deletions(-)
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 7aa1285..3b91f15 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -14,6 +14,11 @@
linkgit:git-clone[1]), deepen or shorten the history to the specified
number of commits. Tags for the deepened commits are not fetched.
+--deepen=::
+ Similar to --depth, except it specifies the number of commits
+ from the current shallow boundary instead of from the tip of
+ each remote branch history.
+
--shallow-since=::
Deepen or shorten the history of a shallow repository to
include all reachable commits after .
diff --git a/Documentation/git-fetch-pack.txt b/Documentation/git-fetch-pack.txt
index 4d15b04..c20958f 100644
--- a/Documentation/git-fetch-pack.txt
+++ b/Documentation/git-fetch-pack.txt
@@ -96,6 +96,11 @@ be in a separate packet, and the list must end with a flush
packet.
exclude commits reachable from a specified remote branch or tag.
This option can be specified multiple times.
+--deepen-relative::
+ Argument --depth specifies the number of commits from the
+ current shallow boundary instead of from the tip of each
+ remote branch history.
+
--no-progress::
Do not show the progress.
diff --git a/Documentation/gitremote-helpers.txt
b/Documentation/gitremote-helpers.txt
index 75bb638..6fca268 100644
--- a/Documentation/gitremote-helpers.txt
+++ b/Documentation/gitremote-helpers.txt
@@ -422,6 +422,10 @@ set by Git if the remote helper has the 'option'
capability.
Deepens the history of a shallow repository excluding ref.
Multiple options add up.
+'option deepen-relative {'true'|'false'}::
+ Deepens the history of a shallow repository relative to
+ current boundary. Only valid when used with "option depth".
+
'option followtags' {'true'|'false'}::
If enabled the helper should automatically fetch annotated
tag objects if the object the tag points at was transferred
diff --git a/Documentation/technical/protocol-capabilities.txt
b/Documentation/technical/protocol-capabilities.txt
index 0e6b57d..4fd6dcc 100644
--- a/Documentation/technical/protocol-capabilities.txt
+++ b/Documentation/technical/protocol-capabilities.txt
@@ -197,6 +197,13 @@ specific revision, instead of depth. Internally it's
[PATCH v2 26/27] upload-pack: add get_reachable_list()
Signed-off-by: Nguyễn Thái Ngọc Duy
---
object.h | 2 +-
upload-pack.c | 52 +---
2 files changed, 50 insertions(+), 4 deletions(-)
diff --git a/object.h b/object.h
index f8b6442..614a006 100644
--- a/object.h
+++ b/object.h
@@ -31,7 +31,7 @@ struct object_array {
* revision.h: 0-1026
* fetch-pack.c:0---4
* walker.c:0-2
- * upload-pack.c: 1119
+ * upload-pack.c: 4 1119
* builtin/blame.c: 12-13
* bisect.c: 16
* bundle.c: 16
diff --git a/upload-pack.c b/upload-pack.c
index adb8e33..3227df8 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -456,7 +456,8 @@ static int is_our_ref(struct object *o)
* on successful case, it's up to the caller to close cmd->out
*/
static int do_reachable_revlist(struct child_process *cmd,
- struct object_array *src)
+ struct object_array *src,
+ struct object_array *reachable)
{
static const char *argv[] = {
"rev-list", "--stdin", NULL,
@@ -487,6 +488,8 @@ static int do_reachable_revlist(struct child_process *cmd,
o = get_indexed_object(--i);
if (!o)
continue;
+ if (reachable && o->type == OBJ_COMMIT)
+ o->flags &= ~TMP_MARK;
if (!is_our_ref(o))
continue;
memcpy(namebuf + 1, oid_to_hex(&o->oid), GIT_SHA1_HEXSZ);
@@ -496,8 +499,13 @@ static int do_reachable_revlist(struct child_process *cmd,
namebuf[40] = '\n';
for (i = 0; i < src->nr; i++) {
o = src->objects[i].item;
- if (is_our_ref(o))
+ if (is_our_ref(o)) {
+ if (reachable)
+ add_object_array(o, NULL, reachable);
continue;
+ }
+ if (reachable && o->type == OBJ_COMMIT)
+ o->flags |= TMP_MARK;
memcpy(namebuf, oid_to_hex(&o->oid), GIT_SHA1_HEXSZ);
if (write_in_full(cmd->in, namebuf, 41) < 0)
goto error;
@@ -518,13 +526,51 @@ error:
return -1;
}
+static int get_reachable_list(struct object_array *src,
+ struct object_array *reachable)
+{
+ struct child_process cmd = CHILD_PROCESS_INIT;
+ int i;
+ struct object *o;
+ char namebuf[42]; /* ^ + SHA-1 + LF */
+
+ if (do_reachable_revlist(&cmd, src, reachable) < 0)
+ return -1;
+
+ while ((i = read_in_full(cmd.out, namebuf, 41)) == 41) {
+ struct object_id sha1;
+
+ if (namebuf[40] != '\n' || get_oid_hex(namebuf, &sha1))
+ break;
+
+ o = lookup_object(sha1.hash);
+ if (o && o->type == OBJ_COMMIT) {
+ o->flags &= ~TMP_MARK;
+ }
+ }
+ for (i = get_max_object_index(); 0 < i; i--) {
+ o = get_indexed_object(i - 1);
+ if (o && o->type == OBJ_COMMIT &&
+ (o->flags & TMP_MARK)) {
+ add_object_array(o, NULL, reachable);
+ o->flags &= ~TMP_MARK;
+ }
+ }
+ close(cmd.out);
+
+ if (finish_command(&cmd))
+ return -1;
+
+ return 0;
+}
+
static int has_unreachable(struct object_array *src)
{
struct child_process cmd = CHILD_PROCESS_INIT;
char buf[1];
int i;
- if (do_reachable_revlist(&cmd, src) < 0)
+ if (do_reachable_revlist(&cmd, src, NULL) < 0)
return 1;
/*
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 25/27] upload-pack: split check_unreachable() in two, prep for get_reachable_list()
Signed-off-by: Nguyễn Thái Ngọc Duy
---
upload-pack.c | 56 ++--
1 file changed, 38 insertions(+), 18 deletions(-)
diff --git a/upload-pack.c b/upload-pack.c
index acc6d97..adb8e33 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -452,21 +452,24 @@ static int is_our_ref(struct object *o)
return o->flags & ((allow_hidden_ref ? HIDDEN_REF : 0) | OUR_REF);
}
-static int has_unreachable(struct object_array *src)
+/*
+ * on successful case, it's up to the caller to close cmd->out
+ */
+static int do_reachable_revlist(struct child_process *cmd,
+ struct object_array *src)
{
static const char *argv[] = {
"rev-list", "--stdin", NULL,
};
- static struct child_process cmd = CHILD_PROCESS_INIT;
struct object *o;
char namebuf[42]; /* ^ + SHA-1 + LF */
int i;
- cmd.argv = argv;
- cmd.git_cmd = 1;
- cmd.no_stderr = 1;
- cmd.in = -1;
- cmd.out = -1;
+ cmd->argv = argv;
+ cmd->git_cmd = 1;
+ cmd->no_stderr = 1;
+ cmd->in = -1;
+ cmd->out = -1;
/*
* If the next rev-list --stdin encounters an unknown commit,
@@ -475,7 +478,7 @@ static int has_unreachable(struct object_array *src)
*/
sigchain_push(SIGPIPE, SIG_IGN);
- if (start_command(&cmd))
+ if (start_command(cmd))
goto error;
namebuf[0] = '^';
@@ -487,7 +490,7 @@ static int has_unreachable(struct object_array *src)
if (!is_our_ref(o))
continue;
memcpy(namebuf + 1, oid_to_hex(&o->oid), GIT_SHA1_HEXSZ);
- if (write_in_full(cmd.in, namebuf, 42) < 0)
+ if (write_in_full(cmd->in, namebuf, 42) < 0)
goto error;
}
namebuf[40] = '\n';
@@ -496,17 +499,39 @@ static int has_unreachable(struct object_array *src)
if (is_our_ref(o))
continue;
memcpy(namebuf, oid_to_hex(&o->oid), GIT_SHA1_HEXSZ);
- if (write_in_full(cmd.in, namebuf, 41) < 0)
+ if (write_in_full(cmd->in, namebuf, 41) < 0)
goto error;
}
- close(cmd.in);
- cmd.in = -1;
+ close(cmd->in);
+ cmd->in = -1;
+ sigchain_pop(SIGPIPE);
+
+ return 0;
+
+error:
+ sigchain_pop(SIGPIPE);
+
+ if (cmd->in >= 0)
+ close(cmd->in);
+ if (cmd->out >= 0)
+ close(cmd->out);
+ return -1;
+}
+
+static int has_unreachable(struct object_array *src)
+{
+ struct child_process cmd = CHILD_PROCESS_INIT;
+ char buf[1];
+ int i;
+
+ if (do_reachable_revlist(&cmd, src) < 0)
+ return 1;
/*
* The commits out of the rev-list are not ancestors of
* our ref.
*/
- i = read_in_full(cmd.out, namebuf, 1);
+ i = read_in_full(cmd.out, buf, 1);
if (i)
goto error;
close(cmd.out);
@@ -520,16 +545,11 @@ static int has_unreachable(struct object_array *src)
if (finish_command(&cmd))
goto error;
- sigchain_pop(SIGPIPE);
-
/* All the non-tip ones are ancestors of what we advertised */
return 0;
error:
sigchain_pop(SIGPIPE);
-
- if (cmd.in >= 0)
- close(cmd.in);
if (cmd.out >= 0)
close(cmd.out);
return 1;
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 24/27] t5500, t5539: tests for shallow depth excluding a ref
Signed-off-by: Nguyễn Thái Ngọc Duy Signed-off-by: Junio C Hamano --- t/t5500-fetch-pack.sh | 21 + t/t5539-fetch-http-shallow.sh | 22 ++ 2 files changed, 43 insertions(+) diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh index 26f050d..145b370 100755 --- a/t/t5500-fetch-pack.sh +++ b/t/t5500-fetch-pack.sh @@ -661,4 +661,25 @@ test_expect_success 'fetch shallow since ...' ' test_cmp expected actual ' +test_expect_success 'shallow clone exclude tag two' ' + test_create_repo shallow-exclude && + ( + cd shallow-exclude && + test_commit one && + test_commit two && + test_commit three && + git clone --shallow-exclude two "file://$(pwd)/." ../shallow12 && + git -C ../shallow12 log --pretty=tformat:%s HEAD >actual && + echo three >expected && + test_cmp expected actual + ) +' + +test_expect_success 'fetch exclude tag one' ' + git -C shallow12 fetch --shallow-exclude one origin && + git -C shallow12 log --pretty=tformat:%s origin/master >actual && + test_write_lines three two >expected && + test_cmp expected actual +' + test_done diff --git a/t/t5539-fetch-http-shallow.sh b/t/t5539-fetch-http-shallow.sh index 704753c..8e38c1b 100755 --- a/t/t5539-fetch-http-shallow.sh +++ b/t/t5539-fetch-http-shallow.sh @@ -98,5 +98,27 @@ test_expect_success 'fetch shallow since ...' ' test_cmp expected actual ' +test_expect_success 'shallow clone exclude tag two' ' + test_create_repo shallow-exclude && + ( + cd shallow-exclude && + test_commit one && + test_commit two && + test_commit three && + mv .git "$HTTPD_DOCUMENT_ROOT_PATH/shallow-exclude.git" && + git clone --shallow-exclude two $HTTPD_URL/smart/shallow-exclude.git ../shallow12 && + git -C ../shallow12 log --pretty=tformat:%s HEAD >actual && + echo three >expected && + test_cmp expected actual + ) +' + +test_expect_success 'fetch exclude tag one' ' + git -C shallow12 fetch --shallow-exclude one origin && + git -C shallow12 log --pretty=tformat:%s origin/master >actual && + test_write_lines three two >expected && + test_cmp expected actual +' + stop_httpd test_done -- 2.8.2.524.g6ff3d78 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 21/27] upload-pack: support define shallow boundary by excluding revisions
This should allow the user to say "create a shallow clone of this branch
after version ".
Short refs are accepted and expanded at the server side with expand_ref()
because we cannot expand (unknown) refs from the client side.
Like deepen-since, deepen-not cannot be used with deepen. But deepen-not
can be mixed with deepen-since. The result is exactly how you do the
command "git rev-list --since=... --not ref".
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
Documentation/technical/pack-protocol.txt | 3 ++-
Documentation/technical/protocol-capabilities.txt | 9 +
upload-pack.c | 23 +--
3 files changed, 32 insertions(+), 3 deletions(-)
diff --git a/Documentation/technical/pack-protocol.txt
b/Documentation/technical/pack-protocol.txt
index 9251df1..dee33a6 100644
--- a/Documentation/technical/pack-protocol.txt
+++ b/Documentation/technical/pack-protocol.txt
@@ -220,7 +220,8 @@ out of what the server said it could do with the first
'want' line.
shallow-line = PKT-LINE("shallow" SP obj-id)
depth-request = PKT-LINE("deepen" SP depth) /
- PKT-LINE("deepen-since" SP timestamp)
+ PKT-LINE("deepen-since" SP timestamp) /
+ PKT-LINE("deepen-not" SP ref)
first-want= PKT-LINE("want" SP obj-id SP capability-list)
additional-want = PKT-LINE("want" SP obj-id)
diff --git a/Documentation/technical/protocol-capabilities.txt
b/Documentation/technical/protocol-capabilities.txt
index f08cc4e..0e6b57d 100644
--- a/Documentation/technical/protocol-capabilities.txt
+++ b/Documentation/technical/protocol-capabilities.txt
@@ -188,6 +188,15 @@ specific time, instead of depth. Internally it's
equivalent of doing
"rev-list --max-age=" on the server side. "deepen-since"
cannot be used with "deepen".
+deepen-not
+--
+
+This capability adds "deepen-not" command to fetch-pack/upload-pack
+protocol so the client can request shallow clones that are cut at a
+specific revision, instead of depth. Internally it's equivalent of
+doing "rev-list --not " on the server side. "deepen-not"
+cannot be used with "deepen", but can be used with "deepen-since".
+
no-progress
---
diff --git a/upload-pack.c b/upload-pack.c
index 5269461..acc6d97 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -645,6 +645,7 @@ static void deepen_by_rev_list(int ac, const char **av,
static void receive_needs(void)
{
struct object_array shallows = OBJECT_ARRAY_INIT;
+ struct string_list deepen_not = STRING_LIST_INIT_DUP;
int depth = 0;
int has_non_tip = 0;
unsigned long deepen_since = 0;
@@ -695,6 +696,16 @@ static void receive_needs(void)
deepen_rev_list = 1;
continue;
}
+ if (skip_prefix(line, "deepen-not ", &arg)) {
+ char *ref = NULL;
+ unsigned char sha1[20];
+ if (expand_ref(arg, strlen(arg), sha1, &ref) != 1)
+ die("git upload-pack: ambiguous deepen-not:
%s", line);
+ string_list_append(&deepen_not, ref);
+ free(ref);
+ deepen_rev_list = 1;
+ continue;
+ }
if (!skip_prefix(line, "want ", &arg) ||
get_sha1_hex(arg, sha1_buf))
die("git upload-pack: protocol error, "
@@ -749,7 +760,7 @@ static void receive_needs(void)
if (depth == 0 && !deepen_rev_list && shallows.nr == 0)
return;
if (depth > 0 && deepen_rev_list)
- die("git upload-pack: deepen and deepen-since cannot be used
together");
+ die("git upload-pack: deepen and deepen-since (or deepen-not)
cannot be used together");
if (depth > 0)
deepen(depth, &shallows);
else if (deepen_rev_list) {
@@ -759,6 +770,14 @@ static void receive_needs(void)
argv_array_push(&av, "rev-list");
if (deepen_since)
argv_array_pushf(&av, "--max-age=%lu", deepen_since);
+ if (deepen_not.nr) {
+ argv_array_push(&av, "--not");
+ for (i = 0; i < deepen_not.nr; i++) {
+ struct string_list_item *s = deepen_not.items +
i;
+ argv_array_push(&av, s->string);
+ }
+ argv_array_push(&av, "--not");
+ }
for (i = 0; i < want_obj.nr; i++) {
struct object *o = want_obj.objects[i].item;
argv_array_push(&av, oid_to_hex(&o->oid));
@@ -814,7 +833,7 @@ static int send_ref(const char *refname, const struct
object_id *oid,
int flag, void *cb_data)
{
s
[PATCH v2 23/27] clone: define shallow clone boundary with --shallow-exclude
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
Documentation/git-clone.txt | 5 +
builtin/clone.c | 10 +-
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index a410409..5049663 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -196,6 +196,11 @@ objects from the source repository into a pack in the
cloned repository.
--shallow-since=::
Create a shallow clone with a history after the specified time.
+--shallow-exclude=::
+ Create a shallow clone with a history, excluding commits
+ reachable from a specified remote branch or tag. This option
+ can be specified multiple times.
+
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index dc2ef4f..3849231 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -44,6 +44,7 @@ static int deepen;
static char *option_template, *option_depth, *option_since;
static char *option_origin = NULL;
static char *option_branch = NULL;
+static struct string_list option_not = STRING_LIST_INIT_NODUP;
static const char *real_git_dir;
static char *option_upload_pack = "git-upload-pack";
static int option_verbosity;
@@ -89,6 +90,8 @@ static struct option builtin_clone_options[] = {
N_("create a shallow clone of that depth")),
OPT_STRING(0, "shallow-since", &option_since, N_("time"),
N_("create a shallow clone since a specific time")),
+ OPT_STRING_LIST(0, "shallow-exclude", &option_not, N_("revision"),
+ N_("deepen history of shallow clone by excluding rev")),
OPT_BOOL(0, "single-branch", &option_single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
@@ -852,7 +855,7 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
- if (option_depth || option_since)
+ if (option_depth || option_since || option_not.nr)
deepen = 1;
if (option_single_branch == -1)
option_single_branch = deepen ? 1 : 0;
@@ -983,6 +986,8 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
warning(_("--depth is ignored in local clones; use
file:// instead."));
if (option_since)
warning(_("--shallow-since is ignored in local clones;
use file:// instead."));
+ if (option_not.nr)
+ warning(_("--shallow-exclude is ignored in local
clones; use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow,
ignoring --local"));
@@ -1004,6 +1009,9 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (option_since)
transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
option_since);
+ if (option_not.nr)
+ transport_set_option(transport, TRANS_OPT_DEEPEN_NOT,
+(const char *)&option_not);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 22/27] fetch: define shallow boundary with --shallow-exclude
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
Documentation/fetch-options.txt | 5 +
Documentation/git-fetch-pack.txt| 5 +
Documentation/gitremote-helpers.txt | 4
builtin/fetch-pack.c| 7 +++
builtin/fetch.c | 13 ++---
fetch-pack.c| 15 ++-
fetch-pack.h| 1 +
remote-curl.c | 9 +
transport-helper.c | 24
transport.c | 4
transport.h | 6 ++
11 files changed, 89 insertions(+), 4 deletions(-)
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 8738d3d..7aa1285 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -18,6 +18,11 @@
Deepen or shorten the history of a shallow repository to
include all reachable commits after .
+--shallow-exclude=::
+ Deepen or shorten the history of a shallow repository to
+ exclude commits reachable from a specified remote branch or tag.
+ This option can be specified multiple times.
+
--unshallow::
If the source repository is complete, convert a shallow
repository to a complete one, removing all the limitations
diff --git a/Documentation/git-fetch-pack.txt b/Documentation/git-fetch-pack.txt
index 99e6257..4d15b04 100644
--- a/Documentation/git-fetch-pack.txt
+++ b/Documentation/git-fetch-pack.txt
@@ -91,6 +91,11 @@ be in a separate packet, and the list must end with a flush
packet.
Deepen or shorten the history of a shallow'repository to
include all reachable commits after .
+--shallow-exclude=::
+ Deepen or shorten the history of a shallow repository to
+ exclude commits reachable from a specified remote branch or tag.
+ This option can be specified multiple times.
+
--no-progress::
Do not show the progress.
diff --git a/Documentation/gitremote-helpers.txt
b/Documentation/gitremote-helpers.txt
index 9971d9a..75bb638 100644
--- a/Documentation/gitremote-helpers.txt
+++ b/Documentation/gitremote-helpers.txt
@@ -418,6 +418,10 @@ set by Git if the remote helper has the 'option'
capability.
'option deepen-since ::
Deepens the history of a shallow repository based on time.
+'option deepen-not ::
+ Deepens the history of a shallow repository excluding ref.
+ Multiple options add up.
+
'option followtags' {'true'|'false'}::
If enabled the helper should automatically fetch annotated
tag objects if the object the tag points at was transferred
diff --git a/builtin/fetch-pack.c b/builtin/fetch-pack.c
index 0402e27..07570be 100644
--- a/builtin/fetch-pack.c
+++ b/builtin/fetch-pack.c
@@ -50,6 +50,7 @@ int cmd_fetch_pack(int argc, const char **argv, const char
*prefix)
struct child_process *conn;
struct fetch_pack_args args;
struct sha1_array shallow = SHA1_ARRAY_INIT;
+ struct string_list deepen_not = STRING_LIST_INIT_DUP;
packet_trace_identity("fetch-pack");
@@ -108,6 +109,10 @@ int cmd_fetch_pack(int argc, const char **argv, const char
*prefix)
args.deepen_since = xstrdup(arg);
continue;
}
+ if (skip_prefix(arg, "--shallow-exclude=", &arg)) {
+ string_list_append(&deepen_not, arg);
+ continue;
+ }
if (!strcmp("--no-progress", arg)) {
args.no_progress = 1;
continue;
@@ -135,6 +140,8 @@ int cmd_fetch_pack(int argc, const char **argv, const char
*prefix)
}
usage(fetch_pack_usage);
}
+ if (deepen_not.nr)
+ args.deepen_not = &deepen_not;
if (i < argc)
dest = argv[i++];
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 283aa95..147504d 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -41,6 +41,7 @@ static int max_children = 1;
static const char *depth;
static const char *deepen_since;
static const char *upload_pack;
+static struct string_list deepen_not = STRING_LIST_INIT_NODUP;
static struct strbuf default_rla = STRBUF_INIT;
static struct transport *gtransport;
static struct transport *gsecondary;
@@ -118,6 +119,8 @@ static struct option builtin_fetch_options[] = {
N_("deepen history of shallow clone")),
OPT_STRING(0, "shallow-since", &deepen_since, N_("time"),
N_("deepen history of shallow repository based on time")),
+ OPT_STRING_LIST(0, "shallow-exclude", &deepen_not, N_("revision"),
+ N_("deepen history of shallow clone by excluding rev")),
{ OPTION_SET_INT, 0, "unshallow", &unshallow, NULL,
N_("convert to a complete repository"),
PARSE_
[PATCH v2 20/27] refs: add expand_ref()
This is basically dwim_ref() without @{} support. To be used on the
server side where we want to expand abbreviated to full ref names and
nothing else. The first user is "git clone/fetch --shallow-exclude".
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
refs.c | 8 +++-
refs.h | 1 +
2 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/refs.c b/refs.c
index e2d34b2..842e4d8 100644
--- a/refs.c
+++ b/refs.c
@@ -392,6 +392,13 @@ static char *substitute_branch_name(const char **string,
int *len)
int dwim_ref(const char *str, int len, unsigned char *sha1, char **ref)
{
char *last_branch = substitute_branch_name(&str, &len);
+ int refs_found = expand_ref(str, len, sha1, ref);
+ free(last_branch);
+ return refs_found;
+}
+
+int expand_ref(const char *str, int len, unsigned char *sha1, char **ref)
+{
const char **p, *r;
int refs_found = 0;
@@ -417,7 +424,6 @@ int dwim_ref(const char *str, int len, unsigned char *sha1,
char **ref)
warning("ignoring broken ref %s.", fullref);
}
}
- free(last_branch);
return refs_found;
}
diff --git a/refs.h b/refs.h
index 3c3da29..31a2fa6 100644
--- a/refs.h
+++ b/refs.h
@@ -90,6 +90,7 @@ extern int resolve_gitlink_ref(const char *path, const char
*refname, unsigned c
*/
extern int refname_match(const char *abbrev_name, const char *full_name);
+extern int expand_ref(const char *str, int len, unsigned char *sha1, char
**ref);
extern int dwim_ref(const char *str, int len, unsigned char *sha1, char **ref);
extern int dwim_log(const char *str, int len, unsigned char *sha1, char **ref);
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 18/27] clone: define shallow clone boundary based on time with --shallow-since
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
Documentation/git-clone.txt | 3 +++
builtin/clone.c | 16 +---
2 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index b7c467a..a410409 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -193,6 +193,9 @@ objects from the source repository into a pack in the
cloned repository.
`--no-single-branch` is given to fetch the histories near the
tips of all branches.
+--shallow-since=::
+ Create a shallow clone with a history after the specified time.
+
--[no-]single-branch::
Clone only the history leading to the tip of a single branch,
either specified by the `--branch` option or the primary
diff --git a/builtin/clone.c b/builtin/clone.c
index bcba080..dc2ef4f 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -40,7 +40,8 @@ static const char * const builtin_clone_usage[] = {
static int option_no_checkout, option_bare, option_mirror,
option_single_branch = -1;
static int option_local = -1, option_no_hardlinks, option_shared,
option_recursive;
-static char *option_template, *option_depth;
+static int deepen;
+static char *option_template, *option_depth, *option_since;
static char *option_origin = NULL;
static char *option_branch = NULL;
static const char *real_git_dir;
@@ -86,6 +87,8 @@ static struct option builtin_clone_options[] = {
N_("path to git-upload-pack on the remote")),
OPT_STRING(0, "depth", &option_depth, N_("depth"),
N_("create a shallow clone of that depth")),
+ OPT_STRING(0, "shallow-since", &option_since, N_("time"),
+ N_("create a shallow clone since a specific time")),
OPT_BOOL(0, "single-branch", &option_single_branch,
N_("clone only one branch, HEAD or --branch")),
OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
@@ -849,8 +852,10 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
usage_msg_opt(_("You must specify a repository to clone."),
builtin_clone_usage, builtin_clone_options);
+ if (option_depth || option_since)
+ deepen = 1;
if (option_single_branch == -1)
- option_single_branch = option_depth ? 1 : 0;
+ option_single_branch = deepen ? 1 : 0;
if (option_mirror)
option_bare = 1;
@@ -976,6 +981,8 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (is_local) {
if (option_depth)
warning(_("--depth is ignored in local clones; use
file:// instead."));
+ if (option_since)
+ warning(_("--shallow-since is ignored in local clones;
use file:// instead."));
if (!access(mkpath("%s/shallow", path), F_OK)) {
if (option_local > 0)
warning(_("source repository is shallow,
ignoring --local"));
@@ -994,6 +1001,9 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
if (option_depth)
transport_set_option(transport, TRANS_OPT_DEPTH,
option_depth);
+ if (option_since)
+ transport_set_option(transport, TRANS_OPT_DEEPEN_SINCE,
+option_since);
if (option_single_branch)
transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
@@ -1001,7 +1011,7 @@ int cmd_clone(int argc, const char **argv, const char
*prefix)
transport_set_option(transport, TRANS_OPT_UPLOADPACK,
option_upload_pack);
- if (transport->smart_options && !option_depth)
+ if (transport->smart_options && !deepen)
transport->smart_options->check_self_contained_and_connected =
1;
refs = transport_get_remote_refs(transport);
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 19/27] t5500, t5539: tests for shallow depth since a specific date
Signed-off-by: Nguyễn Thái Ngọc Duy Signed-off-by: Junio C Hamano --- t/t5500-fetch-pack.sh | 24 t/t5539-fetch-http-shallow.sh | 25 + 2 files changed, 49 insertions(+) diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh index e5f83bf..26f050d 100755 --- a/t/t5500-fetch-pack.sh +++ b/t/t5500-fetch-pack.sh @@ -637,4 +637,28 @@ test_expect_success MINGW 'fetch-pack --diag-url c:repo' ' check_prot_path c:repo file c:repo ' +test_expect_success 'clone shallow since ...' ' + test_create_repo shallow-since && + ( + cd shallow-since && + GIT_COMMITTER_DATE="1 +0700" git commit --allow-empty -m one && + GIT_COMMITTER_DATE="2 +0700" git commit --allow-empty -m two && + GIT_COMMITTER_DATE="3 +0700" git commit --allow-empty -m three && + git clone --shallow-since "3 +0700" "file://$(pwd)/." ../shallow11 && + git -C ../shallow11 log --pretty=tformat:%s HEAD >actual && + echo three >expected && + test_cmp expected actual + ) +' + +test_expect_success 'fetch shallow since ...' ' + git -C shallow11 fetch --shallow-since "2 +0700" origin && + git -C shallow11 log --pretty=tformat:%s origin/master >actual && + cat >expected <<-\EOF && + three + two + EOF + test_cmp expected actual +' + test_done diff --git a/t/t5539-fetch-http-shallow.sh b/t/t5539-fetch-http-shallow.sh index 37a4335..704753c 100755 --- a/t/t5539-fetch-http-shallow.sh +++ b/t/t5539-fetch-http-shallow.sh @@ -73,5 +73,30 @@ test_expect_success 'no shallow lines after receiving ACK ready' ' ) ' +test_expect_success 'clone shallow since ...' ' + test_create_repo shallow-since && + ( + cd shallow-since && + GIT_COMMITTER_DATE="1 +0700" git commit --allow-empty -m one && + GIT_COMMITTER_DATE="2 +0700" git commit --allow-empty -m two && + GIT_COMMITTER_DATE="3 +0700" git commit --allow-empty -m three && + mv .git "$HTTPD_DOCUMENT_ROOT_PATH/shallow-since.git" && + git clone --shallow-since "3 +0700" $HTTPD_URL/smart/shallow-since.git ../shallow11 && + git -C ../shallow11 log --pretty=tformat:%s HEAD >actual && + echo three >expected && + test_cmp expected actual + ) +' + +test_expect_success 'fetch shallow since ...' ' + git -C shallow11 fetch --shallow-since "2 +0700" origin && + git -C shallow11 log --pretty=tformat:%s origin/master >actual && + cat >expected <<-\EOF && + three + two + EOF + test_cmp expected actual +' + stop_httpd test_done -- 2.8.2.524.g6ff3d78 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 10/27] upload-pack: move rev-list code out of check_non_tip()
Signed-off-by: Nguyễn Thái Ngọc Duy
---
upload-pack.c | 36 +++-
1 file changed, 23 insertions(+), 13 deletions(-)
diff --git a/upload-pack.c b/upload-pack.c
index 7ce97ec..93c0522 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -451,7 +451,7 @@ static int is_our_ref(struct object *o)
return o->flags & ((allow_hidden_ref ? HIDDEN_REF : 0) | OUR_REF);
}
-static void check_non_tip(void)
+static int has_unreachable(struct object_array *src)
{
static const char *argv[] = {
"rev-list", "--stdin", NULL,
@@ -461,14 +461,6 @@ static void check_non_tip(void)
char namebuf[42]; /* ^ + SHA-1 + LF */
int i;
- /*
-* In the normal in-process case without
-* uploadpack.allowReachableSHA1InWant,
-* non-tip requests can never happen.
-*/
- if (!stateless_rpc && !(allow_unadvertised_object_request &
ALLOW_REACHABLE_SHA1))
- goto error;
-
cmd.argv = argv;
cmd.git_cmd = 1;
cmd.no_stderr = 1;
@@ -498,8 +490,8 @@ static void check_non_tip(void)
goto error;
}
namebuf[40] = '\n';
- for (i = 0; i < want_obj.nr; i++) {
- o = want_obj.objects[i].item;
+ for (i = 0; i < src->nr; i++) {
+ o = src->objects[i].item;
if (is_our_ref(o))
continue;
memcpy(namebuf, oid_to_hex(&o->oid), GIT_SHA1_HEXSZ);
@@ -530,7 +522,7 @@ static void check_non_tip(void)
sigchain_pop(SIGPIPE);
/* All the non-tip ones are ancestors of what we advertised */
- return;
+ return 0;
error:
sigchain_pop(SIGPIPE);
@@ -539,10 +531,28 @@ error:
close(cmd.in);
if (cmd.out >= 0)
close(cmd.out);
+ return 1;
+}
+static void check_non_tip(void)
+{
+ int i;
+
+ /*
+* In the normal in-process case without
+* uploadpack.allowReachableSHA1InWant,
+* non-tip requests can never happen.
+*/
+ if (!stateless_rpc && !(allow_unadvertised_object_request &
ALLOW_REACHABLE_SHA1))
+ goto error;
+ if (!has_unreachable(&want_obj))
+ /* All the non-tip ones are ancestors of what we advertised */
+ return;
+
+error:
/* Pick one of them (we know there at least is one) */
for (i = 0; i < want_obj.nr; i++) {
- o = want_obj.objects[i].item;
+ struct object *o = want_obj.objects[i].item;
if (!is_our_ref(o))
die("git upload-pack: not our ref %s",
oid_to_hex(&o->oid));
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 05/27] upload-pack: remove unused variable "backup"
After the last patch, "result" and "backup" are the same. "result" used
to move, but the movement is now contained in send_shallow(). Delete
this redundant variable.
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
upload-pack.c | 9 -
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/upload-pack.c b/upload-pack.c
index 0eb9a0b..ee5d20b 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -554,7 +554,7 @@ static void send_shallow(struct commit_list *result)
static void deepen(int depth, const struct object_array *shallows)
{
- struct commit_list *result = NULL, *backup = NULL;
+ struct commit_list *result = NULL;
int i;
if (depth == INFINITE_DEPTH && !is_repository_shallow())
for (i = 0; i < shallows->nr; i++) {
@@ -562,11 +562,10 @@ static void deepen(int depth, const struct object_array
*shallows)
object->flags |= NOT_SHALLOW;
}
else
- backup = result =
- get_shallow_commits(&want_obj, depth,
- SHALLOW, NOT_SHALLOW);
+ result = get_shallow_commits(&want_obj, depth,
+SHALLOW, NOT_SHALLOW);
send_shallow(result);
- free_commit_list(backup);
+ free_commit_list(result);
for (i = 0; i < shallows->nr; i++) {
struct object *object = shallows->objects[i].item;
if (object->flags & NOT_SHALLOW) {
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 09/27] upload-pack: make check_non_tip() clean things up on error
On error check_non_tip() will die and not closing file descriptors is no
big deal. The next patch will split the majority of this function out
for reuse in other cases, where die() may not be the only outcome. Same
story for popping SIGPIPE out of the signal chain. So let's make sure we
clean things up properly first.
Signed-off-by: Nguyễn Thái Ngọc Duy
---
upload-pack.c | 23 ---
1 file changed, 16 insertions(+), 7 deletions(-)
diff --git a/upload-pack.c b/upload-pack.c
index 8f4d7f4..7ce97ec 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -475,16 +475,16 @@ static void check_non_tip(void)
cmd.in = -1;
cmd.out = -1;
- if (start_command(&cmd))
- goto error;
-
/*
-* If rev-list --stdin encounters an unknown commit, it
-* terminates, which will cause SIGPIPE in the write loop
+* If the next rev-list --stdin encounters an unknown commit,
+* it terminates, which will cause SIGPIPE in the write loop
* below.
*/
sigchain_push(SIGPIPE, SIG_IGN);
+ if (start_command(&cmd))
+ goto error;
+
namebuf[0] = '^';
namebuf[41] = '\n';
for (i = get_max_object_index(); 0 < i; ) {
@@ -507,8 +507,7 @@ static void check_non_tip(void)
goto error;
}
close(cmd.in);
-
- sigchain_pop(SIGPIPE);
+ cmd.in = -1;
/*
* The commits out of the rev-list are not ancestors of
@@ -518,6 +517,7 @@ static void check_non_tip(void)
if (i)
goto error;
close(cmd.out);
+ cmd.out = -1;
/*
* rev-list may have died by encountering a bad commit
@@ -527,10 +527,19 @@ static void check_non_tip(void)
if (finish_command(&cmd))
goto error;
+ sigchain_pop(SIGPIPE);
+
/* All the non-tip ones are ancestors of what we advertised */
return;
error:
+ sigchain_pop(SIGPIPE);
+
+ if (cmd.in >= 0)
+ close(cmd.in);
+ if (cmd.out >= 0)
+ close(cmd.out);
+
/* Pick one of them (we know there at least is one) */
for (i = 0; i < want_obj.nr; i++) {
o = want_obj.objects[i].item;
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 17/27] fetch: define shallow boundary with --shallow-since
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
Documentation/fetch-options.txt | 4
Documentation/git-fetch-pack.txt| 4
Documentation/gitremote-helpers.txt | 3 +++
builtin/fetch-pack.c| 4
builtin/fetch.c | 29 +++--
fetch-pack.c| 12 +++-
fetch-pack.h| 1 +
remote-curl.c | 11 +--
transport.c | 4
transport.h | 4
10 files changed, 67 insertions(+), 9 deletions(-)
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 952dfdf..8738d3d 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -14,6 +14,10 @@
linkgit:git-clone[1]), deepen or shorten the history to the specified
number of commits. Tags for the deepened commits are not fetched.
+--shallow-since=::
+ Deepen or shorten the history of a shallow repository to
+ include all reachable commits after .
+
--unshallow::
If the source repository is complete, convert a shallow
repository to a complete one, removing all the limitations
diff --git a/Documentation/git-fetch-pack.txt b/Documentation/git-fetch-pack.txt
index 8680f45..99e6257 100644
--- a/Documentation/git-fetch-pack.txt
+++ b/Documentation/git-fetch-pack.txt
@@ -87,6 +87,10 @@ be in a separate packet, and the list must end with a flush
packet.
'git-upload-pack' treats the special depth 2147483647 as
infinite even if there is an ancestor-chain that long.
+--shallow-since=::
+ Deepen or shorten the history of a shallow'repository to
+ include all reachable commits after .
+
--no-progress::
Do not show the progress.
diff --git a/Documentation/gitremote-helpers.txt
b/Documentation/gitremote-helpers.txt
index 78e0b27..9971d9a 100644
--- a/Documentation/gitremote-helpers.txt
+++ b/Documentation/gitremote-helpers.txt
@@ -415,6 +415,9 @@ set by Git if the remote helper has the 'option' capability.
'option depth' ::
Deepens the history of a shallow repository.
+'option deepen-since ::
+ Deepens the history of a shallow repository based on time.
+
'option followtags' {'true'|'false'}::
If enabled the helper should automatically fetch annotated
tag objects if the object the tag points at was transferred
diff --git a/builtin/fetch-pack.c b/builtin/fetch-pack.c
index 8332d3d..0402e27 100644
--- a/builtin/fetch-pack.c
+++ b/builtin/fetch-pack.c
@@ -104,6 +104,10 @@ int cmd_fetch_pack(int argc, const char **argv, const char
*prefix)
args.depth = strtol(arg, NULL, 0);
continue;
}
+ if (skip_prefix(arg, "--shallow-since=", &arg)) {
+ args.deepen_since = xstrdup(arg);
+ continue;
+ }
if (!strcmp("--no-progress", arg)) {
args.no_progress = 1;
continue;
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 8e74213..283aa95 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -36,9 +36,10 @@ static int prune = -1; /* unspecified */
static int all, append, dry_run, force, keep, multiple, update_head_ok,
verbosity;
static int progress = -1, recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
-static int tags = TAGS_DEFAULT, unshallow, update_shallow;
+static int tags = TAGS_DEFAULT, unshallow, update_shallow, deepen;
static int max_children = 1;
static const char *depth;
+static const char *deepen_since;
static const char *upload_pack;
static struct strbuf default_rla = STRBUF_INIT;
static struct transport *gtransport;
@@ -115,6 +116,8 @@ static struct option builtin_fetch_options[] = {
OPT_BOOL(0, "progress", &progress, N_("force progress reporting")),
OPT_STRING(0, "depth", &depth, N_("depth"),
N_("deepen history of shallow clone")),
+ OPT_STRING(0, "shallow-since", &deepen_since, N_("time"),
+ N_("deepen history of shallow repository based on time")),
{ OPTION_SET_INT, 0, "unshallow", &unshallow, NULL,
N_("convert to a complete repository"),
PARSE_OPT_NONEG | PARSE_OPT_NOARG, NULL, 1 },
@@ -754,7 +757,7 @@ static int quickfetch(struct ref *ref_map)
* really need to perform. Claiming failure now will ensure
* we perform the network exchange to deepen our history.
*/
- if (depth)
+ if (deepen)
return -1;
return check_everything_connected(iterate_ref_map, 1, &rm);
}
@@ -859,7 +862,7 @@ static void set_option(struct transport *transport, const
char *name, const char
name, transport->url);
}
-static struct transport *prepare_transport(struct remote *remote)
+static struct transport *
[PATCH v2 13/27] fetch-pack.c: mark strings for translating
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
fetch-pack.c | 75 ++--
1 file changed, 37 insertions(+), 38 deletions(-)
diff --git a/fetch-pack.c b/fetch-pack.c
index 4020744..08caf1d 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -208,7 +208,7 @@ static void consume_shallow_list(struct fetch_pack_args
*args, int fd)
continue;
if (starts_with(line, "unshallow "))
continue;
- die("git fetch-pack: expected shallow list");
+ die(_("git fetch-pack: expected shallow list"));
}
}
}
@@ -220,7 +220,7 @@ static enum ack_type get_ack(int fd, unsigned char
*result_sha1)
const char *arg;
if (!len)
- die("git fetch-pack: expected ACK/NAK, got EOF");
+ die(_("git fetch-pack: expected ACK/NAK, got EOF"));
if (!strcmp(line, "NAK"))
return NAK;
if (skip_prefix(line, "ACK ", &arg)) {
@@ -238,7 +238,7 @@ static enum ack_type get_ack(int fd, unsigned char
*result_sha1)
return ACK;
}
}
- die("git fetch_pack: expected ACK/NAK, got '%s'", line);
+ die(_("git fetch_pack: expected ACK/NAK, got '%s'"), line);
}
static void send_request(struct fetch_pack_args *args,
@@ -285,7 +285,7 @@ static int find_common(struct fetch_pack_args *args,
size_t state_len = 0;
if (args->stateless_rpc && multi_ack == 1)
- die("--stateless-rpc requires multi_ack_detailed");
+ die(_("--stateless-rpc requires multi_ack_detailed"));
if (marked)
for_each_ref(clear_marks, NULL);
marked = 1;
@@ -357,23 +357,23 @@ static int find_common(struct fetch_pack_args *args,
while ((line = packet_read_line(fd[0], NULL))) {
if (skip_prefix(line, "shallow ", &arg)) {
if (get_sha1_hex(arg, sha1))
- die("invalid shallow line: %s", line);
+ die(_("invalid shallow line: %s"),
line);
register_shallow(sha1);
continue;
}
if (skip_prefix(line, "unshallow ", &arg)) {
if (get_sha1_hex(arg, sha1))
- die("invalid unshallow line: %s", line);
+ die(_("invalid unshallow line: %s"),
line);
if (!lookup_object(sha1))
- die("object not found: %s", line);
+ die(_("object not found: %s"), line);
/* make sure that it is parsed as shallow */
if (!parse_object(sha1))
- die("error in object: %s", line);
+ die(_("error in object: %s"), line);
if (unregister_shallow(sha1))
- die("no shallow found: %s", line);
+ die(_("no shallow found: %s"), line);
continue;
}
- die("expected shallow/unshallow, got %s", line);
+ die(_("expected shallow/unshallow, got %s"), line);
}
} else if (!args->stateless_rpc)
send_request(args, fd[1], &req_buf);
@@ -412,8 +412,8 @@ static int find_common(struct fetch_pack_args *args,
do {
ack = get_ack(fd[0], result_sha1);
if (ack)
- print_verbose(args, "got ack %d %s",
ack,
- sha1_to_hex(result_sha1));
+ print_verbose(args, _("got %s %d %s"),
"ack",
+ ack,
sha1_to_hex(result_sha1));
switch (ack) {
case ACK:
flushes = 0;
@@ -426,7 +426,7 @@ static int find_common(struct fetch_pack_args *args,
struct commit *commit =
lookup_commit(result_sha1);
if (!commit)
- die("invalid commit %s",
sha1_to_hex(result_sha1));
+ die(_("invalid commit %s"),
sha1_to_hex(result_sha1));
if (args->stateless_rpc
&& ack == ACK_co
[PATCH v2 15/27] shallow.c: implement a generic shallow boundary finder based on rev-list
Instead of a custom commit walker like get_shallow_commits(), this new
function uses rev-list to mark NOT_SHALLOW to all reachable commits,
except borders. The definition of reachable is to be defined by the
protocol later. This makes it more flexible to define shallow boundary.
The way we find border is paint all reachable commits NOT_SHALLOW. Any
of them that "touches" commits without NOT_SHALLOW flag are considered
shallow (e.g. zero parents via grafting mechanism). Shallow commits and
their true parents are all marked SHALLOW. Then NOT_SHALLOW is removed
from shallow commits at the end.
There is an interesting observation. With a generic walker, we can
produce all kinds of shallow cutting. In the following graph, every
commit but "x" is reachable. "b" is a parent of "a".
x -- a -- o
//
x -- c -- b -- o
After this function is run, "a" and "c" are both considered shallow
commits. After grafting occurs at the client side, what we see is
a -- o
/
c -- b -- o
Notice that because of grafting, "a" has zero parents, so "b" is no
longer a parent of "a".
This is unfortunate and may be solved in two ways. The first is change
the way shallow grafting works and keep "a -- b" connection if "b"
exists and always ends at shallow commits (iow, no loose ends). This is
hard to detect, or at least not cheap to do.
The second way is mark one "x" as shallow commit instead of "a" and
produce this graph at client side:
x -- a -- o
//
c -- b -- o
More commits, but simpler grafting rules.
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
commit.h | 2 ++
shallow.c | 78 +++
2 files changed, 80 insertions(+)
diff --git a/commit.h b/commit.h
index 5d58be0..b717be1 100644
--- a/commit.h
+++ b/commit.h
@@ -258,6 +258,8 @@ extern int for_each_commit_graft(each_commit_graft_fn, void
*);
extern int is_repository_shallow(void);
extern struct commit_list *get_shallow_commits(struct object_array *heads,
int depth, int shallow_flag, int not_shallow_flag);
+extern struct commit_list *get_shallow_commits_by_rev_list(
+ int ac, const char **av, int shallow_flag, int
not_shallow_flag);
extern void set_alternate_shallow_file(const char *path, int override);
extern int write_shallow_commits(struct strbuf *out, int use_pack_protocol,
const struct sha1_array *extra);
diff --git a/shallow.c b/shallow.c
index 60f1505..40c2485 100644
--- a/shallow.c
+++ b/shallow.c
@@ -10,6 +10,8 @@
#include "diff.h"
#include "revision.h"
#include "commit-slab.h"
+#include "revision.h"
+#include "list-objects.h"
static int is_shallow = -1;
static struct stat_validity shallow_stat;
@@ -137,6 +139,82 @@ struct commit_list *get_shallow_commits(struct
object_array *heads, int depth,
return result;
}
+static void show_commit(struct commit *commit, void *data)
+{
+ commit_list_insert(commit, data);
+}
+
+/*
+ * Given rev-list arguments, run rev-list. All reachable commits
+ * except border ones are marked with not_shallow_flag. Border commits
+ * are marked with shallow_flag. The list of border/shallow commits
+ * are also returned.
+ */
+struct commit_list *get_shallow_commits_by_rev_list(int ac, const char **av,
+ int shallow_flag,
+ int not_shallow_flag)
+{
+ struct commit_list *result = NULL, *p;
+ struct commit_list *not_shallow_list = NULL;
+ struct rev_info revs;
+ int both_flags = shallow_flag | not_shallow_flag;
+
+ /*
+* SHALLOW (excluded) and NOT_SHALLOW (included) should not be
+* set at this point. But better be safe than sorry.
+*/
+ clear_object_flags(both_flags);
+
+ is_repository_shallow(); /* make sure shallows are read */
+
+ init_revisions(&revs, NULL);
+ save_commit_buffer = 0;
+ setup_revisions(ac, av, &revs, NULL);
+
+ if (prepare_revision_walk(&revs))
+ die("revision walk setup failed");
+ traverse_commit_list(&revs, show_commit, NULL, ¬_shallow_list);
+
+ /* Mark all reachable commits as NOT_SHALLOW */
+ for (p = not_shallow_list; p; p = p->next)
+ p->item->object.flags |= not_shallow_flag;
+
+ /*
+* mark border commits SHALLOW + NOT_SHALLOW.
+* We cannot clear NOT_SHALLOW right now. Imagine border
+* commit A is processed first, then commit B, whose parent is
+* A, later. If NOT_SHALLOW on A is cleared at step 1, B
+* itself is considered border at step 2, which is incorrect.
+*/
+ for (p = not_shallow_list; p; p = p->next) {
+ struct commit *c = p->item;
+ struct commit_list *parent;
+
+ if (parse_commit(c))
+
[PATCH v2 06/27] upload-pack: move "unshallow" sending code out of deepen()
Also add some more comments in this code because it takes too long to
understand what it does (to me, who should be familiar enough to
understand this code well!)
Signed-off-by: Nguyễn Thái Ngọc Duy
---
upload-pack.c | 43 ++-
1 file changed, 30 insertions(+), 13 deletions(-)
diff --git a/upload-pack.c b/upload-pack.c
index ee5d20b..73a8b28 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -552,20 +552,10 @@ static void send_shallow(struct commit_list *result)
}
}
-static void deepen(int depth, const struct object_array *shallows)
+static void send_unshallow(const struct object_array *shallows)
{
- struct commit_list *result = NULL;
int i;
- if (depth == INFINITE_DEPTH && !is_repository_shallow())
- for (i = 0; i < shallows->nr; i++) {
- struct object *object = shallows->objects[i].item;
- object->flags |= NOT_SHALLOW;
- }
- else
- result = get_shallow_commits(&want_obj, depth,
-SHALLOW, NOT_SHALLOW);
- send_shallow(result);
- free_commit_list(result);
+
for (i = 0; i < shallows->nr; i++) {
struct object *object = shallows->objects[i].item;
if (object->flags & NOT_SHALLOW) {
@@ -573,7 +563,13 @@ static void deepen(int depth, const struct object_array
*shallows)
packet_write(1, "unshallow %s",
oid_to_hex(&object->oid));
object->flags &= ~CLIENT_SHALLOW;
- /* make sure the real parents are parsed */
+ /*
+* We want to _register_ "object" as shallow, but we
+* also need to traverse object's parents to deepen a
+* shallow clone. Unregister it for now so we can
+* parse and add the parents to the want list, then
+* re-register it.
+*/
unregister_shallow(object->oid.hash);
object->parsed = 0;
parse_commit_or_die((struct commit *)object);
@@ -588,6 +584,27 @@ static void deepen(int depth, const struct object_array
*shallows)
/* make sure commit traversal conforms to client */
register_shallow(object->oid.hash);
}
+}
+
+static void deepen(int depth, const struct object_array *shallows)
+{
+ if (depth == INFINITE_DEPTH && !is_repository_shallow()) {
+ int i;
+
+ for (i = 0; i < shallows->nr; i++) {
+ struct object *object = shallows->objects[i].item;
+ object->flags |= NOT_SHALLOW;
+ }
+ } else {
+ struct commit_list *result;
+
+ result = get_shallow_commits(&want_obj, depth,
+SHALLOW, NOT_SHALLOW);
+ send_shallow(result);
+ free_commit_list(result);
+ }
+
+ send_unshallow(shallows);
packet_flush(1);
}
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 16/27] upload-pack: add deepen-since to cut shallow repos based on time
This should allow the user to say "create a shallow clone containing the
work from last year" (once the client side is fixed up, of course).
In theory deepen-since and deepen (aka --depth) can be used together to
draw the shallow boundary (whether it's intersection or union is up to
discussion, but if rev-list is used, it's likely intersection). However,
because deepen goes with a custom commit walker, we can't mix the two
yet.
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
Documentation/technical/pack-protocol.txt | 3 +-
Documentation/technical/protocol-capabilities.txt | 9 +
upload-pack.c | 45 ++-
3 files changed, 54 insertions(+), 3 deletions(-)
diff --git a/Documentation/technical/pack-protocol.txt
b/Documentation/technical/pack-protocol.txt
index c6977bb..9251df1 100644
--- a/Documentation/technical/pack-protocol.txt
+++ b/Documentation/technical/pack-protocol.txt
@@ -219,7 +219,8 @@ out of what the server said it could do with the first
'want' line.
shallow-line = PKT-LINE("shallow" SP obj-id)
- depth-request = PKT-LINE("deepen" SP depth)
+ depth-request = PKT-LINE("deepen" SP depth) /
+ PKT-LINE("deepen-since" SP timestamp)
first-want= PKT-LINE("want" SP obj-id SP capability-list)
additional-want = PKT-LINE("want" SP obj-id)
diff --git a/Documentation/technical/protocol-capabilities.txt
b/Documentation/technical/protocol-capabilities.txt
index eaab6b4..f08cc4e 100644
--- a/Documentation/technical/protocol-capabilities.txt
+++ b/Documentation/technical/protocol-capabilities.txt
@@ -179,6 +179,15 @@ This capability adds "deepen", "shallow" and "unshallow"
commands to
the fetch-pack/upload-pack protocol so clients can request shallow
clones.
+deepen-since
+
+
+This capability adds "deepen-since" command to fetch-pack/upload-pack
+protocol so the client can request shallow clones that are cut at a
+specific time, instead of depth. Internally it's equivalent of doing
+"rev-list --max-age=" on the server side. "deepen-since"
+cannot be used with "deepen".
+
no-progress
---
diff --git a/upload-pack.c b/upload-pack.c
index 93c0522..5269461 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -14,6 +14,7 @@
#include "sigchain.h"
#include "version.h"
#include "string-list.h"
+#include "argv-array.h"
static const char upload_pack_usage[] = "git upload-pack [--strict]
[--timeout=] ";
@@ -629,11 +630,25 @@ static void deepen(int depth, const struct object_array
*shallows)
packet_flush(1);
}
+static void deepen_by_rev_list(int ac, const char **av,
+ struct object_array *shallows)
+{
+ struct commit_list *result;
+
+ result = get_shallow_commits_by_rev_list(ac, av, SHALLOW, NOT_SHALLOW);
+ send_shallow(result);
+ free_commit_list(result);
+ send_unshallow(shallows);
+ packet_flush(1);
+}
+
static void receive_needs(void)
{
struct object_array shallows = OBJECT_ARRAY_INIT;
int depth = 0;
int has_non_tip = 0;
+ unsigned long deepen_since = 0;
+ int deepen_rev_list = 0;
shallow_nr = 0;
for (;;) {
@@ -670,6 +685,16 @@ static void receive_needs(void)
die("Invalid deepen: %s", line);
continue;
}
+ if (skip_prefix(line, "deepen-since ", &arg)) {
+ char *end = NULL;
+ deepen_since = strtoul(arg, &end, 0);
+ if (!end || *end || !deepen_since ||
+ /* revisions.c's max_age -1 is special */
+ deepen_since == -1)
+ die("Invalid deepen-since: %s", line);
+ deepen_rev_list = 1;
+ continue;
+ }
if (!skip_prefix(line, "want ", &arg) ||
get_sha1_hex(arg, sha1_buf))
die("git upload-pack: protocol error, "
@@ -721,10 +746,26 @@ static void receive_needs(void)
if (!use_sideband && daemon_mode)
no_progress = 1;
- if (depth == 0 && shallows.nr == 0)
+ if (depth == 0 && !deepen_rev_list && shallows.nr == 0)
return;
+ if (depth > 0 && deepen_rev_list)
+ die("git upload-pack: deepen and deepen-since cannot be used
together");
if (depth > 0)
deepen(depth, &shallows);
+ else if (deepen_rev_list) {
+ struct argv_array av = ARGV_ARRAY_INIT;
+ int i;
+
+ argv_array_push(&av, "rev-list");
+ if (deepen_since)
+ argv_array_pushf(&av, "--max-age=%lu", deepen_since);
+ for (i = 0; i < want_obj.nr; i++) {
+ struct object *o = want_obj.objects[i].item;
+
[PATCH v2 07/27] upload-pack: use skip_prefix() instead of starts_with()
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
upload-pack.c | 32 ++--
1 file changed, 18 insertions(+), 14 deletions(-)
diff --git a/upload-pack.c b/upload-pack.c
index 73a8b28..fa7ce09 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -276,7 +276,7 @@ static void create_pack_file(void)
die("git upload-pack: %s", abort_msg);
}
-static int got_sha1(char *hex, unsigned char *sha1)
+static int got_sha1(const char *hex, unsigned char *sha1)
{
struct object *o;
int we_knew_they_have = 0;
@@ -382,6 +382,8 @@ static int get_common_commits(void)
for (;;) {
char *line = packet_read_line(0, NULL);
+ const char *arg;
+
reset_timeout();
if (!line) {
@@ -403,8 +405,8 @@ static int get_common_commits(void)
got_other = 0;
continue;
}
- if (starts_with(line, "have ")) {
- switch (got_sha1(line+5, sha1)) {
+ if (skip_prefix(line, "have ", &arg)) {
+ switch (got_sha1(arg, sha1)) {
case -1: /* they have what we do not */
got_other = 1;
if (multi_ack && ok_to_give_up()) {
@@ -620,14 +622,16 @@ static void receive_needs(void)
const char *features;
unsigned char sha1_buf[20];
char *line = packet_read_line(0, NULL);
+ const char *arg;
+
reset_timeout();
if (!line)
break;
- if (starts_with(line, "shallow ")) {
+ if (skip_prefix(line, "shallow ", &arg)) {
unsigned char sha1[20];
struct object *object;
- if (get_sha1_hex(line + 8, sha1))
+ if (get_sha1_hex(arg, sha1))
die("invalid shallow line: %s", line);
object = parse_object(sha1);
if (!object)
@@ -640,19 +644,19 @@ static void receive_needs(void)
}
continue;
}
- if (starts_with(line, "deepen ")) {
+ if (skip_prefix(line, "deepen ", &arg)) {
char *end;
- depth = strtol(line + 7, &end, 0);
- if (end == line + 7 || depth <= 0)
+ depth = strtol(arg, &end, 0);
+ if (end == arg || depth <= 0)
die("Invalid deepen: %s", line);
continue;
}
- if (!starts_with(line, "want ") ||
- get_sha1_hex(line+5, sha1_buf))
+ if (!skip_prefix(line, "want ", &arg) ||
+ get_sha1_hex(arg, sha1_buf))
die("git upload-pack: protocol error, "
"expected to get sha, not '%s'", line);
- features = line + 45;
+ features = arg + 40;
if (parse_feature_request(features, "multi_ack_detailed"))
multi_ack = 2;
@@ -859,7 +863,7 @@ int main(int argc, char **argv)
check_replace_refs = 0;
for (i = 1; i < argc; i++) {
- char *arg = argv[i];
+ const char *arg = argv[i];
if (arg[0] != '-')
break;
@@ -875,8 +879,8 @@ int main(int argc, char **argv)
strict = 1;
continue;
}
- if (starts_with(arg, "--timeout=")) {
- timeout = atoi(arg+10);
+ if (skip_prefix(arg, "--timeout=", &arg)) {
+ timeout = atoi(arg);
daemon_mode = 1;
continue;
}
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 12/27] fetch-pack: use a common function for verbose printing
This reduces the number of "if (verbose)" which makes it a bit easier
to read imo. It also makes it easier to redirect all these printouts,
to a file for example.
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
fetch-pack.c | 88 +---
1 file changed, 42 insertions(+), 46 deletions(-)
diff --git a/fetch-pack.c b/fetch-pack.c
index 01e34b6..4020744 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -50,6 +50,21 @@ static int non_common_revs, multi_ack, use_sideband;
#define ALLOW_REACHABLE_SHA1 02
static unsigned int allow_unadvertised_object_request;
+__attribute__((format (printf, 2, 3)))
+static inline void print_verbose(const struct fetch_pack_args *args,
+const char *fmt, ...)
+{
+ va_list params;
+
+ if (!args->verbose)
+ return;
+
+ va_start(params, fmt);
+ vfprintf(stderr, fmt, params);
+ va_end(params);
+ fputc('\n', stderr);
+}
+
static void rev_list_push(struct commit *commit, int mark)
{
if (!(commit->object.flags & mark)) {
@@ -375,8 +390,7 @@ static int find_common(struct fetch_pack_args *args,
retval = -1;
while ((sha1 = get_rev())) {
packet_buf_write(&req_buf, "have %s\n", sha1_to_hex(sha1));
- if (args->verbose)
- fprintf(stderr, "have %s\n", sha1_to_hex(sha1));
+ print_verbose(args, "have %s", sha1_to_hex(sha1));
in_vain++;
if (flush_at <= ++count) {
int ack;
@@ -397,9 +411,9 @@ static int find_common(struct fetch_pack_args *args,
consume_shallow_list(args, fd[0]);
do {
ack = get_ack(fd[0], result_sha1);
- if (args->verbose && ack)
- fprintf(stderr, "got ack %d %s\n", ack,
-
sha1_to_hex(result_sha1));
+ if (ack)
+ print_verbose(args, "got ack %d %s",
ack,
+ sha1_to_hex(result_sha1));
switch (ack) {
case ACK:
flushes = 0;
@@ -438,8 +452,7 @@ static int find_common(struct fetch_pack_args *args,
} while (ack);
flushes--;
if (got_continue && MAX_IN_VAIN < in_vain) {
- if (args->verbose)
- fprintf(stderr, "giving up\n");
+ print_verbose(args, "giving up");
break; /* give up */
}
}
@@ -449,8 +462,7 @@ done:
packet_buf_write(&req_buf, "done\n");
send_request(args, fd[1], &req_buf);
}
- if (args->verbose)
- fprintf(stderr, "done\n");
+ print_verbose(args, "done");
if (retval != 0) {
multi_ack = 0;
flushes++;
@@ -462,9 +474,8 @@ done:
while (flushes || multi_ack) {
int ack = get_ack(fd[0], result_sha1);
if (ack) {
- if (args->verbose)
- fprintf(stderr, "got ack (%d) %s\n", ack,
- sha1_to_hex(result_sha1));
+ print_verbose(args, "got ack (%d) %s", ack,
+ sha1_to_hex(result_sha1));
if (ack == ACK)
return 0;
multi_ack = 1;
@@ -509,9 +520,8 @@ static void mark_recent_complete_commits(struct
fetch_pack_args *args,
unsigned long cutoff)
{
while (complete && cutoff <= complete->item->date) {
- if (args->verbose)
- fprintf(stderr, "Marking %s as complete\n",
- oid_to_hex(&complete->item->object.oid));
+ print_verbose(args, "Marking %s as complete",
+ oid_to_hex(&complete->item->object.oid));
pop_most_recent_commit(&complete, COMPLETE);
}
}
@@ -652,18 +662,12 @@ static int everything_local(struct fetch_pack_args *args,
o = lookup_object(remote);
if (!o || !(o->flags & COMPLETE)) {
retval = 0;
- if (!args->verbose)
- continue;
- fprintf(stderr,
- "want %s (%s)\n", sha1_to_hex(remote),
- ref->name);
+ print_verbose(args, "want %s (%s)", sha1_to_hex(remote),
+
[PATCH v2 11/27] fetch-pack: use skip_prefix() instead of starts_with()
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
builtin/fetch-pack.c | 12 ++--
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/builtin/fetch-pack.c b/builtin/fetch-pack.c
index 9b2a514..8332d3d 100644
--- a/builtin/fetch-pack.c
+++ b/builtin/fetch-pack.c
@@ -59,12 +59,12 @@ int cmd_fetch_pack(int argc, const char **argv, const char
*prefix)
for (i = 1; i < argc && *argv[i] == '-'; i++) {
const char *arg = argv[i];
- if (starts_with(arg, "--upload-pack=")) {
- args.uploadpack = arg + 14;
+ if (skip_prefix(arg, "--upload-pack=", &arg)) {
+ args.uploadpack = arg;
continue;
}
- if (starts_with(arg, "--exec=")) {
- args.uploadpack = arg + 7;
+ if (skip_prefix(arg, "--exec=", &arg)) {
+ args.uploadpack = arg;
continue;
}
if (!strcmp("--quiet", arg) || !strcmp("-q", arg)) {
@@ -100,8 +100,8 @@ int cmd_fetch_pack(int argc, const char **argv, const char
*prefix)
args.verbose = 1;
continue;
}
- if (starts_with(arg, "--depth=")) {
- args.depth = strtol(arg + 8, NULL, 0);
+ if (skip_prefix(arg, "--depth=", &arg)) {
+ args.depth = strtol(arg, NULL, 0);
continue;
}
if (!strcmp("--no-progress", arg)) {
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 08/27] upload-pack: tighten number parsing at "deepen" lines
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
upload-pack.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/upload-pack.c b/upload-pack.c
index fa7ce09..8f4d7f4 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -645,9 +645,9 @@ static void receive_needs(void)
continue;
}
if (skip_prefix(line, "deepen ", &arg)) {
- char *end;
+ char *end = NULL;
depth = strtol(arg, &end, 0);
- if (end == arg || depth <= 0)
+ if (!end || *end || depth <= 0)
die("Invalid deepen: %s", line);
continue;
}
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 14/27] fetch-pack: use a separate flag for fetch in deepening mode
The shallow repo could be deepened or shortened when then user gives
--depth. But in future that won't be the only way to deepen/shorten a
repo. Stop relying on args->depth in this mode. Future deepening
methods can simply set this flag on instead of updating all these if
expressions.
The new name "deepen" was chosen after the command to define shallow
boundary in pack protocol. New commands also follow this tradition.
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
fetch-pack.c | 14 --
fetch-pack.h | 1 +
2 files changed, 9 insertions(+), 6 deletions(-)
diff --git a/fetch-pack.c b/fetch-pack.c
index 08caf1d..a14d24a 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -197,7 +197,7 @@ enum ack_type {
static void consume_shallow_list(struct fetch_pack_args *args, int fd)
{
- if (args->stateless_rpc && args->depth > 0) {
+ if (args->stateless_rpc && args->deepen) {
/* If we sent a depth we will get back "duplicate"
* shallow and unshallow commands every time there
* is a block of have lines exchanged.
@@ -348,7 +348,7 @@ static int find_common(struct fetch_pack_args *args,
packet_buf_flush(&req_buf);
state_len = req_buf.len;
- if (args->depth > 0) {
+ if (args->deepen) {
char *line;
const char *arg;
unsigned char sha1[20];
@@ -557,7 +557,7 @@ static void filter_refs(struct fetch_pack_args *args,
}
if (!keep && args->fetch_all &&
- (!args->depth || !starts_with(ref->name, "refs/tags/")))
+ (!args->deepen || !starts_with(ref->name, "refs/tags/")))
keep = 1;
if (keep) {
@@ -627,7 +627,7 @@ static int everything_local(struct fetch_pack_args *args,
}
}
- if (!args->depth) {
+ if (!args->deepen) {
for_each_ref(mark_complete_oid, NULL);
for_each_alternate_ref(mark_alternate_complete, NULL);
commit_list_sort_by_date(&complete);
@@ -812,6 +812,8 @@ static struct ref *do_fetch_pack(struct fetch_pack_args
*args,
if ((args->depth > 0 || is_repository_shallow()) &&
!server_supports("shallow"))
die(_("Server does not support shallow clients"));
+ if (args->depth > 0)
+ args->deepen = 1;
if (server_supports("multi_ack_detailed")) {
print_verbose(args, _("Server supports multi_ack_detailed"));
multi_ack = 2;
@@ -872,7 +874,7 @@ static struct ref *do_fetch_pack(struct fetch_pack_args
*args,
if (args->stateless_rpc)
packet_flush(fd[1]);
- if (args->depth > 0)
+ if (args->deepen)
setup_alternate_shallow(&shallow_lock, &alternate_shallow_file,
NULL);
else if (si->nr_ours || si->nr_theirs)
@@ -939,7 +941,7 @@ static void update_shallow(struct fetch_pack_args *args,
int *status;
int i;
- if (args->depth > 0 && alternate_shallow_file) {
+ if (args->deepen && alternate_shallow_file) {
if (*alternate_shallow_file == '\0') { /* --unshallow */
unlink_or_warn(git_path_shallow());
rollback_lock_file(&shallow_lock);
diff --git a/fetch-pack.h b/fetch-pack.h
index bb7fd76..4d0adb0 100644
--- a/fetch-pack.h
+++ b/fetch-pack.h
@@ -25,6 +25,7 @@ struct fetch_pack_args {
unsigned self_contained_and_connected:1;
unsigned cloning:1;
unsigned update_shallow:1;
+ unsigned deepen:1;
};
/*
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 04/27] upload-pack: move "shallow" sending code out of deepen()
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
upload-pack.c | 25 +++--
1 file changed, 15 insertions(+), 10 deletions(-)
diff --git a/upload-pack.c b/upload-pack.c
index 97ed620..0eb9a0b 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -538,6 +538,20 @@ error:
}
}
+static void send_shallow(struct commit_list *result)
+{
+ while (result) {
+ struct object *object = &result->item->object;
+ if (!(object->flags & (CLIENT_SHALLOW|NOT_SHALLOW))) {
+ packet_write(1, "shallow %s",
+oid_to_hex(&object->oid));
+ register_shallow(object->oid.hash);
+ shallow_nr++;
+ }
+ result = result->next;
+ }
+}
+
static void deepen(int depth, const struct object_array *shallows)
{
struct commit_list *result = NULL, *backup = NULL;
@@ -551,16 +565,7 @@ static void deepen(int depth, const struct object_array
*shallows)
backup = result =
get_shallow_commits(&want_obj, depth,
SHALLOW, NOT_SHALLOW);
- while (result) {
- struct object *object = &result->item->object;
- if (!(object->flags & (CLIENT_SHALLOW|NOT_SHALLOW))) {
- packet_write(1, "shallow %s",
-oid_to_hex(&object->oid));
- register_shallow(object->oid.hash);
- shallow_nr++;
- }
- result = result->next;
- }
+ send_shallow(result);
free_commit_list(backup);
for (i = 0; i < shallows->nr; i++) {
struct object *object = shallows->objects[i].item;
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 02/27] transport-helper.c: refactor set_helper_option()
For now we can handle two types, string and boolean, in
set_helper_option(). Later on we'll add string_list support, which does
not fit well. The new function strbuf_set_helper_option() can be reused
for a separate function that handles string-list.
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
transport-helper.c | 37 +++--
1 file changed, 23 insertions(+), 14 deletions(-)
diff --git a/transport-helper.c b/transport-helper.c
index a6bff8b..27a34e9 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -260,6 +260,28 @@ static const char *boolean_options[] = {
TRANS_OPT_FOLLOWTAGS,
};
+static int strbuf_set_helper_option(struct helper_data *data,
+ struct strbuf *buf)
+{
+ int ret;
+
+ sendline(data, buf);
+ if (recvline(data, buf))
+ exit(128);
+
+ if (!strcmp(buf->buf, "ok"))
+ ret = 0;
+ else if (starts_with(buf->buf, "error"))
+ ret = -1;
+ else if (!strcmp(buf->buf, "unsupported"))
+ ret = 1;
+ else {
+ warning("%s unexpectedly said: '%s'", data->name, buf->buf);
+ ret = 1;
+ }
+ return ret;
+}
+
static int set_helper_option(struct transport *transport,
const char *name, const char *value)
{
@@ -291,20 +313,7 @@ static int set_helper_option(struct transport *transport,
quote_c_style(value, &buf, NULL, 0);
strbuf_addch(&buf, '\n');
- sendline(data, &buf);
- if (recvline(data, &buf))
- exit(128);
-
- if (!strcmp(buf.buf, "ok"))
- ret = 0;
- else if (starts_with(buf.buf, "error")) {
- ret = -1;
- } else if (!strcmp(buf.buf, "unsupported"))
- ret = 1;
- else {
- warning("%s unexpectedly said: '%s'", data->name, buf.buf);
- ret = 1;
- }
+ ret = strbuf_set_helper_option(data, &buf);
strbuf_release(&buf);
return ret;
}
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 01/27] remote-curl.c: convert fetch_git() to use argv_array
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
remote-curl.c | 46 ++
1 file changed, 18 insertions(+), 28 deletions(-)
diff --git a/remote-curl.c b/remote-curl.c
index c704857..fd030c1 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -725,38 +725,28 @@ static int fetch_git(struct discovery *heads,
{
struct rpc_state rpc;
struct strbuf preamble = STRBUF_INIT;
- char *depth_arg = NULL;
- int argc = 0, i, err;
- const char *argv[17];
-
- argv[argc++] = "fetch-pack";
- argv[argc++] = "--stateless-rpc";
- argv[argc++] = "--stdin";
- argv[argc++] = "--lock-pack";
+ int i, err;
+ struct argv_array args = ARGV_ARRAY_INIT;
+
+ argv_array_pushl(&args, "fetch-pack", "--stateless-rpc",
+"--stdin", "--lock-pack", NULL);
if (options.followtags)
- argv[argc++] = "--include-tag";
+ argv_array_push(&args, "--include-tag");
if (options.thin)
- argv[argc++] = "--thin";
- if (options.verbosity >= 3) {
- argv[argc++] = "-v";
- argv[argc++] = "-v";
- }
+ argv_array_push(&args, "--thin");
+ if (options.verbosity >= 3)
+ argv_array_pushl(&args, "-v", "-v", NULL);
if (options.check_self_contained_and_connected)
- argv[argc++] = "--check-self-contained-and-connected";
+ argv_array_push(&args, "--check-self-contained-and-connected");
if (options.cloning)
- argv[argc++] = "--cloning";
+ argv_array_push(&args, "--cloning");
if (options.update_shallow)
- argv[argc++] = "--update-shallow";
+ argv_array_push(&args, "--update-shallow");
if (!options.progress)
- argv[argc++] = "--no-progress";
- if (options.depth) {
- struct strbuf buf = STRBUF_INIT;
- strbuf_addf(&buf, "--depth=%lu", options.depth);
- depth_arg = strbuf_detach(&buf, NULL);
- argv[argc++] = depth_arg;
- }
- argv[argc++] = url.buf;
- argv[argc++] = NULL;
+ argv_array_push(&args, "--no-progress");
+ if (options.depth)
+ argv_array_pushf(&args, "--depth=%lu", options.depth);
+ argv_array_push(&args, url.buf);
for (i = 0; i < nr_heads; i++) {
struct ref *ref = to_fetch[i];
@@ -769,7 +759,7 @@ static int fetch_git(struct discovery *heads,
memset(&rpc, 0, sizeof(rpc));
rpc.service_name = "git-upload-pack",
- rpc.argv = argv;
+ rpc.argv = args.argv;
rpc.stdin_preamble = &preamble;
rpc.gzip_request = 1;
@@ -778,7 +768,7 @@ static int fetch_git(struct discovery *heads,
write_or_die(1, rpc.result.buf, rpc.result.len);
strbuf_release(&rpc.result);
strbuf_release(&preamble);
- free(depth_arg);
+ argv_array_clear(&args);
return err;
}
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 03/27] upload-pack: move shallow deepen code out of receive_needs()
This is a prep step for further refactoring. Besides reindentation and
s/shallows\./shallows->/g, no other changes are expected.
Signed-off-by: Nguyễn Thái Ngọc Duy
Signed-off-by: Junio C Hamano
---
upload-pack.c | 99 +++
1 file changed, 52 insertions(+), 47 deletions(-)
diff --git a/upload-pack.c b/upload-pack.c
index b3f6653..97ed620 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -538,6 +538,55 @@ error:
}
}
+static void deepen(int depth, const struct object_array *shallows)
+{
+ struct commit_list *result = NULL, *backup = NULL;
+ int i;
+ if (depth == INFINITE_DEPTH && !is_repository_shallow())
+ for (i = 0; i < shallows->nr; i++) {
+ struct object *object = shallows->objects[i].item;
+ object->flags |= NOT_SHALLOW;
+ }
+ else
+ backup = result =
+ get_shallow_commits(&want_obj, depth,
+ SHALLOW, NOT_SHALLOW);
+ while (result) {
+ struct object *object = &result->item->object;
+ if (!(object->flags & (CLIENT_SHALLOW|NOT_SHALLOW))) {
+ packet_write(1, "shallow %s",
+oid_to_hex(&object->oid));
+ register_shallow(object->oid.hash);
+ shallow_nr++;
+ }
+ result = result->next;
+ }
+ free_commit_list(backup);
+ for (i = 0; i < shallows->nr; i++) {
+ struct object *object = shallows->objects[i].item;
+ if (object->flags & NOT_SHALLOW) {
+ struct commit_list *parents;
+ packet_write(1, "unshallow %s",
+oid_to_hex(&object->oid));
+ object->flags &= ~CLIENT_SHALLOW;
+ /* make sure the real parents are parsed */
+ unregister_shallow(object->oid.hash);
+ object->parsed = 0;
+ parse_commit_or_die((struct commit *)object);
+ parents = ((struct commit *)object)->parents;
+ while (parents) {
+ add_object_array(&parents->item->object,
+NULL, &want_obj);
+ parents = parents->next;
+ }
+ add_object_array(object, NULL, &extra_edge_obj);
+ }
+ /* make sure commit traversal conforms to client */
+ register_shallow(object->oid.hash);
+ }
+ packet_flush(1);
+}
+
static void receive_needs(void)
{
struct object_array shallows = OBJECT_ARRAY_INIT;
@@ -630,53 +679,9 @@ static void receive_needs(void)
if (depth == 0 && shallows.nr == 0)
return;
- if (depth > 0) {
- struct commit_list *result = NULL, *backup = NULL;
- int i;
- if (depth == INFINITE_DEPTH && !is_repository_shallow())
- for (i = 0; i < shallows.nr; i++) {
- struct object *object =
shallows.objects[i].item;
- object->flags |= NOT_SHALLOW;
- }
- else
- backup = result =
- get_shallow_commits(&want_obj, depth,
- SHALLOW, NOT_SHALLOW);
- while (result) {
- struct object *object = &result->item->object;
- if (!(object->flags & (CLIENT_SHALLOW|NOT_SHALLOW))) {
- packet_write(1, "shallow %s",
- oid_to_hex(&object->oid));
- register_shallow(object->oid.hash);
- shallow_nr++;
- }
- result = result->next;
- }
- free_commit_list(backup);
- for (i = 0; i < shallows.nr; i++) {
- struct object *object = shallows.objects[i].item;
- if (object->flags & NOT_SHALLOW) {
- struct commit_list *parents;
- packet_write(1, "unshallow %s",
- oid_to_hex(&object->oid));
- object->flags &= ~CLIENT_SHALLOW;
- /* make sure the real parents are parsed */
- unregister_shallow(object->oid.hash);
- object->parsed = 0;
- parse_commit_or_die((struct commit *)object);
- parents = ((struct commit *)object)->parents;
-
[PATCH v2 00/27] nd/shallow-deepen updates
Second update to address Junio comments. Interdiff
diff --git a/upload-pack.c b/upload-pack.c
index ef693bd..e40d15a 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -453,6 +453,9 @@ static int is_our_ref(struct object *o)
return o->flags & ((allow_hidden_ref ? HIDDEN_REF : 0) | OUR_REF);
}
+/*
+ * on successful case, it's up to the caller to close cmd->out
+ */
static int do_reachable_revlist(struct child_process *cmd,
struct object_array *src,
struct object_array *reachable)
@@ -470,16 +473,16 @@ static int do_reachable_revlist(struct child_process *cmd,
cmd->in = -1;
cmd->out = -1;
- if (start_command(cmd))
- goto error;
-
/*
-* If rev-list --stdin encounters an unknown commit, it
-* terminates, which will cause SIGPIPE in the write loop
+* If the next rev-list --stdin encounters an unknown commit,
+* it terminates, which will cause SIGPIPE in the write loop
* below.
*/
sigchain_push(SIGPIPE, SIG_IGN);
+ if (start_command(cmd))
+ goto error;
+
namebuf[0] = '^';
namebuf[41] = '\n';
for (i = get_max_object_index(); 0 < i; ) {
@@ -491,10 +494,8 @@ static int do_reachable_revlist(struct child_process *cmd,
if (!is_our_ref(o))
continue;
memcpy(namebuf + 1, oid_to_hex(&o->oid), GIT_SHA1_HEXSZ);
- if (write_in_full(cmd->in, namebuf, 42) < 0) {
- sigchain_pop(SIGPIPE);
+ if (write_in_full(cmd->in, namebuf, 42) < 0)
goto error;
- }
}
namebuf[40] = '\n';
for (i = 0; i < src->nr; i++) {
@@ -507,18 +508,18 @@ static int do_reachable_revlist(struct child_process *cmd,
if (reachable && o->type == OBJ_COMMIT)
o->flags |= TMP_MARK;
memcpy(namebuf, oid_to_hex(&o->oid), GIT_SHA1_HEXSZ);
- if (write_in_full(cmd->in, namebuf, 41) < 0) {
- sigchain_pop(SIGPIPE);
+ if (write_in_full(cmd->in, namebuf, 41) < 0)
goto error;
- }
}
close(cmd->in);
cmd->in = -1;
-
sigchain_pop(SIGPIPE);
+
return 0;
error:
+ sigchain_pop(SIGPIPE);
+
if (cmd->in >= 0)
close(cmd->in);
if (cmd->out >= 0)
@@ -530,11 +531,11 @@ static int get_reachable_list(struct object_array *src,
struct object_array *reachable)
{
struct child_process cmd = CHILD_PROCESS_INIT;
- int i, ret = do_reachable_revlist(&cmd, src, reachable);
+ int i;
struct object *o;
char namebuf[42]; /* ^ + SHA-1 + LF */
- if (ret < 0)
+ if (do_reachable_revlist(&cmd, src, reachable) < 0)
return -1;
while ((i = read_in_full(cmd.out, namebuf, 41)) == 41) {
@@ -564,14 +565,14 @@ static int get_reachable_list(struct object_array *src,
return 0;
}
-static int check_unreachable(struct object_array *src)
+static int has_unreachable(struct object_array *src)
{
struct child_process cmd = CHILD_PROCESS_INIT;
char buf[1];
int i;
if (do_reachable_revlist(&cmd, src, NULL) < 0)
- return 0;
+ return 1;
/*
* The commits out of the rev-list are not ancestors of
@@ -592,14 +593,13 @@ static int check_unreachable(struct object_array *src)
goto error;
/* All the non-tip ones are ancestors of what we advertised */
- return 1;
+ return 0;
error:
- if (cmd.in >= 0)
- close(cmd.in);
+ sigchain_pop(SIGPIPE);
if (cmd.out >= 0)
close(cmd.out);
- return 0;
+ return 1;
}
static void check_non_tip(void)
@@ -613,7 +613,7 @@ static void check_non_tip(void)
*/
if (!stateless_rpc && !(allow_unadvertised_object_request &
ALLOW_REACHABLE_SHA1))
goto error;
- if (check_unreachable(&want_obj))
+ if (!has_unreachable(&want_obj))
/* All the non-tip ones are ancestors of what we advertised */
return;
@@ -678,25 +678,33 @@ static void send_unshallow(const struct object_array
*shallows)
static void deepen(int depth, int deepen_relative,
struct object_array *shallows)
{
- struct commit_list *result = NULL;
- int i;
- if (depth == INFINITE_DEPTH && !is_repository_shallow())
+ if (depth == INFINITE_DEPTH && !is_repository_shallow()) {
+ int i;
+
for (i = 0; i < shallows->nr; i++) {
struct object *object = shallows->objects[i].item;
object->flags |= NOT_SHALLOW;
}
- else if (deepen_relative) {
+ }
lib-httpd/apache.conf and mod_unixd.so
I need help from apache experts. On my system (gentoo 64bit, apache-2.4.20), running an httpd test gives me -- 8< -- Initialized empty Git repository in /home/pclouds/w/git/temp/t/trash directory.t5539-fetch-http-shallow/.git/ checking prerequisite: NOT_ROOT mkdir -p "$TRASH_DIRECTORY/prereq-test-dir" && ( cd "$TRASH_DIRECTORY/prereq-test-dir" && uid=$(id -u) && test "$uid" != 0 ) prerequisite NOT_ROOT ok [Sun Jun 12 10:42:19.958873 2016] [core:crit] [pid 31585] AH00136: Server MUST relinquish startup privileges before accepting connections. Please ensure mod_unixd or other system security module is loaded. AH00016: Configuration Failed 1..0 # SKIP web server setup failed -- 8< -- A quick search shows that loading mod_unixd.so will do (and did make httpd run for me). Problem is, can I just update apache.conf to load it when apache version >= 2.4? I don't know if doing that would cause failure for other people because mod_unixd has existed since 2.2 and people have run httpd tests fine so far... -- Duy -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 3/4] dir: introduce file_size() to check the size of file
>> So what I understand, you want something like this:
>>
>> +ssize_t file_size_not_zero(const char *filename)
>> +{
>> + struct stat st;
>> + if (stat(filename, &st) < 0)
>> + return -1;
>> + return !!st.st_size);
>> +}
>
> For the purpose of bisect_reset(), Yes. BTW a similar function exist
> in builtin/am.c with the name is_empty_file(). But as Christian points
> out file_size() could help to refactor other parts of code.
>
Please allow one or more late comments:
If is_empty_file() does what you need, then it can be moved into wrapper.c
and simply be re-used in your code.
If you want to introduce a new function, that can be used for other refactoring,
then the whole thing would ideally go into a single commit,
or into a single series.
That may probably be out of the scope for your current efforts ?
What really makes me concern is the mixture of signed - and unsigned:
ssize_t file_size(const char *filename)
+{
+ struct stat st;
+ if (stat(filename, &st) < 0)
+ return -1;
+ return xsize_t(st.st_size);
+}
To my understanding a file size is either 0, or a positive integer.
Returning -1 is of course impossible with a positive integer.
So either the function is changed like this:
int file_size(const char *filename, size_t *len)
+{
+ struct stat st;
+ if (stat(filename, &st) < 0)
+ return -1;
+ *len = xsize_t(st.st_size);
+ return 0;
+}
Or, if that works for you:
size_t file_size(const char *filename)
+{
+ struct stat st;
+ if (stat(filename, &st) < 0)
+ return 0;
+ return xsize_t(st.st_size);
+}
Or, more git-ish:
size_t file_size(const char *filename)
+{
+ struct stat st;
+ if (stat(filename, &st))
+ return 0;
+ return xsize_t(st.st_size);
+}
(And then builtin/am.c can be changed to use the new function.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] lib-httpd.sh: print error.log on error
Failure to bring up httpd for testing is not considered an error, so the
trash directory, which contains this error.log file, is removed and we
don't know what made httpd fail to start. Improve the situation a bit.
Signed-off-by: Nguyễn Thái Ngọc Duy
---
t/lib-httpd.sh | 1 +
1 file changed, 1 insertion(+)
diff --git a/t/lib-httpd.sh b/t/lib-httpd.sh
index f9f3e5f..5b8de38 100644
--- a/t/lib-httpd.sh
+++ b/t/lib-httpd.sh
@@ -180,6 +180,7 @@ start_httpd() {
if test $? -ne 0
then
trap 'die' EXIT
+ cat "$HTTPD_ROOT_PATH"/error.log 2>/dev/null
test_skip_or_die $GIT_TEST_HTTPD "web server setup failed"
fi
}
--
2.8.2.524.g6ff3d78
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: feature request: git svn dommit --preserve-timestamps
On Sat, Jun 11 2016, Eric Wong wrote: > I'm sorry for your situation and hoping you migrate off SVN > entirely, soon :) I've done so, but my customer won't. He wants the code in his svn-repo. > Maybe you could look at how the _use_log_author and > _add_author_from options work. I've forgotten their existence > until now and I've never used them myself; but apparently > they're still there. That would mean adding "Date: ..." to the log message. Of course, I could do that. But then my customer would get confused about these extra lines in the svn log. It seems really to complicated, I think I'll just make a script, that saves all important information (date, message, etc.) at every git-commit to some kind of personal log-file, that I can use when I need to search the history by date. Nevertheless thanks for your efforts. Kind regards, -- Peter -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
