Re: [PATCH v3 2/4] rev-list: add an option to mark fewer edges as uninteresting
On Tue, Dec 23, 2014 at 01:51:42PM -0500, Jeff King wrote: On Tue, Dec 23, 2014 at 12:55:48PM -0500, Michael Blume wrote: This patch causes an error on my mac, test 5500 fetch-pack errors on part 44 - fetch creating new shallow root. It looks for remote: Total 1 in the fetch output and gets 3 instead. It fails for me on Linux, too. Interestingly the tip of the series passes. I didn't investigate further. Yes, the tip of the series fixes that issue. I had trouble coming up with a good technique to split the patches in a way such that they were logically distinct yet addressed the failure. I had an epiphany on how to do that last night, so I'll reroll sometime today. -- brian m. carlson / brian with sandals: Houston, Texas, US +1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187 signature.asc Description: Digital signature
[PATCH v3 2/4] rev-list: add an option to mark fewer edges as uninteresting
In commit fbd4a70 (list-objects: mark more commits as edges in
mark_edges_uninteresting - 2013-08-16), we marked an increasing number
of edges uninteresting. This change, and the subsequent change to make
this conditional on --objects-edge, are used by --thin to make much
smaller packs for shallow clones.
Unfortunately, they cause a significant performance regression when
pushing non-shallow clones with lots of refs (23.322 seconds vs.
4.785 seconds with 22400 refs). Add an option to git rev-list,
--objects-edge-aggressive, that preserves this more aggressive behavior,
while leaving --objects-edge to provide more performant behavior.
Signed-off-by: brian m. carlson sand...@crustytoothpaste.net
---
Documentation/git-rev-list.txt | 3 ++-
Documentation/rev-list-options.txt | 4
list-objects.c | 4 ++--
revision.c | 6 ++
revision.h | 1 +
5 files changed, 15 insertions(+), 3 deletions(-)
diff --git a/Documentation/git-rev-list.txt b/Documentation/git-rev-list.txt
index fd7f8b5..5b11922 100644
--- a/Documentation/git-rev-list.txt
+++ b/Documentation/git-rev-list.txt
@@ -46,7 +46,8 @@ SYNOPSIS
[ \--extended-regexp | -E ]
[ \--fixed-strings | -F ]
[ \--date=(local|relative|default|iso|iso-strict|rfc|short) ]
-[ [\--objects | \--objects-edge] [ \--unpacked ] ]
+[ [ \--objects | \--objects-edge | \--objects-edge-aggressive ]
+ [ \--unpacked ] ]
[ \--pretty | \--header ]
[ \--bisect ]
[ \--bisect-vars ]
diff --git a/Documentation/rev-list-options.txt
b/Documentation/rev-list-options.txt
index 2277fcb..8cb6f92 100644
--- a/Documentation/rev-list-options.txt
+++ b/Documentation/rev-list-options.txt
@@ -657,6 +657,10 @@ These options are mostly targeted for packing of Git
repositories.
objects in deltified form based on objects contained in these
excluded commits to reduce network traffic.
+--objects-edge-aggressive::
+ Similar to `--objects-edge`, but it tries harder to find excluded
+ commits at the cost of increased time.
+
--unpacked::
Only useful with `--objects`; print the object IDs that are not
in packs.
diff --git a/list-objects.c b/list-objects.c
index 2910bec..2a139b6 100644
--- a/list-objects.c
+++ b/list-objects.c
@@ -157,7 +157,7 @@ void mark_edges_uninteresting(struct rev_info *revs,
show_edge_fn show_edge)
if (commit-object.flags UNINTERESTING) {
mark_tree_uninteresting(commit-tree);
- if (revs-edge_hint !(commit-object.flags SHOWN))
{
+ if (revs-edge_hint_aggressive
!(commit-object.flags SHOWN)) {
commit-object.flags |= SHOWN;
show_edge(commit);
}
@@ -165,7 +165,7 @@ void mark_edges_uninteresting(struct rev_info *revs,
show_edge_fn show_edge)
}
mark_edge_parents_uninteresting(commit, revs, show_edge);
}
- if (revs-edge_hint) {
+ if (revs-edge_hint_aggressive) {
for (i = 0; i revs-cmdline.nr; i++) {
struct object *obj = revs-cmdline.rev[i].item;
struct commit *commit = (struct commit *)obj;
diff --git a/revision.c b/revision.c
index 75dda92..753dd2f 100644
--- a/revision.c
+++ b/revision.c
@@ -1853,6 +1853,12 @@ static int handle_revision_opt(struct rev_info *revs,
int argc, const char **arg
revs-tree_objects = 1;
revs-blob_objects = 1;
revs-edge_hint = 1;
+ } else if (!strcmp(arg, --objects-edge-aggressive)) {
+ revs-tag_objects = 1;
+ revs-tree_objects = 1;
+ revs-blob_objects = 1;
+ revs-edge_hint = 1;
+ revs-edge_hint_aggressive = 1;
} else if (!strcmp(arg, --verify-objects)) {
revs-tag_objects = 1;
revs-tree_objects = 1;
diff --git a/revision.h b/revision.h
index 9cb5adc..033a244 100644
--- a/revision.h
+++ b/revision.h
@@ -93,6 +93,7 @@ struct rev_info {
blob_objects:1,
verify_objects:1,
edge_hint:1,
+ edge_hint_aggressive:1,
limited:1,
unpacked:1,
boundary:2,
--
2.2.1.209.g41e5f3a
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 2/4] rev-list: add an option to mark fewer edges as uninteresting
This patch causes an error on my mac, test 5500 fetch-pack errors on
part 44 - fetch creating new shallow root. It looks for remote: Total
1 in the fetch output and gets 3 instead.
On Tue, Dec 23, 2014 at 7:01 AM, brian m. carlson
sand...@crustytoothpaste.net wrote:
In commit fbd4a70 (list-objects: mark more commits as edges in
mark_edges_uninteresting - 2013-08-16), we marked an increasing number
of edges uninteresting. This change, and the subsequent change to make
this conditional on --objects-edge, are used by --thin to make much
smaller packs for shallow clones.
Unfortunately, they cause a significant performance regression when
pushing non-shallow clones with lots of refs (23.322 seconds vs.
4.785 seconds with 22400 refs). Add an option to git rev-list,
--objects-edge-aggressive, that preserves this more aggressive behavior,
while leaving --objects-edge to provide more performant behavior.
Signed-off-by: brian m. carlson sand...@crustytoothpaste.net
---
Documentation/git-rev-list.txt | 3 ++-
Documentation/rev-list-options.txt | 4
list-objects.c | 4 ++--
revision.c | 6 ++
revision.h | 1 +
5 files changed, 15 insertions(+), 3 deletions(-)
diff --git a/Documentation/git-rev-list.txt b/Documentation/git-rev-list.txt
index fd7f8b5..5b11922 100644
--- a/Documentation/git-rev-list.txt
+++ b/Documentation/git-rev-list.txt
@@ -46,7 +46,8 @@ SYNOPSIS
[ \--extended-regexp | -E ]
[ \--fixed-strings | -F ]
[ \--date=(local|relative|default|iso|iso-strict|rfc|short) ]
-[ [\--objects | \--objects-edge] [ \--unpacked ] ]
+[ [ \--objects | \--objects-edge | \--objects-edge-aggressive ]
+ [ \--unpacked ] ]
[ \--pretty | \--header ]
[ \--bisect ]
[ \--bisect-vars ]
diff --git a/Documentation/rev-list-options.txt
b/Documentation/rev-list-options.txt
index 2277fcb..8cb6f92 100644
--- a/Documentation/rev-list-options.txt
+++ b/Documentation/rev-list-options.txt
@@ -657,6 +657,10 @@ These options are mostly targeted for packing of Git
repositories.
objects in deltified form based on objects contained in these
excluded commits to reduce network traffic.
+--objects-edge-aggressive::
+ Similar to `--objects-edge`, but it tries harder to find excluded
+ commits at the cost of increased time.
+
--unpacked::
Only useful with `--objects`; print the object IDs that are not
in packs.
diff --git a/list-objects.c b/list-objects.c
index 2910bec..2a139b6 100644
--- a/list-objects.c
+++ b/list-objects.c
@@ -157,7 +157,7 @@ void mark_edges_uninteresting(struct rev_info *revs,
show_edge_fn show_edge)
if (commit-object.flags UNINTERESTING) {
mark_tree_uninteresting(commit-tree);
- if (revs-edge_hint !(commit-object.flags
SHOWN)) {
+ if (revs-edge_hint_aggressive
!(commit-object.flags SHOWN)) {
commit-object.flags |= SHOWN;
show_edge(commit);
}
@@ -165,7 +165,7 @@ void mark_edges_uninteresting(struct rev_info *revs,
show_edge_fn show_edge)
}
mark_edge_parents_uninteresting(commit, revs, show_edge);
}
- if (revs-edge_hint) {
+ if (revs-edge_hint_aggressive) {
for (i = 0; i revs-cmdline.nr; i++) {
struct object *obj = revs-cmdline.rev[i].item;
struct commit *commit = (struct commit *)obj;
diff --git a/revision.c b/revision.c
index 75dda92..753dd2f 100644
--- a/revision.c
+++ b/revision.c
@@ -1853,6 +1853,12 @@ static int handle_revision_opt(struct rev_info *revs,
int argc, const char **arg
revs-tree_objects = 1;
revs-blob_objects = 1;
revs-edge_hint = 1;
+ } else if (!strcmp(arg, --objects-edge-aggressive)) {
+ revs-tag_objects = 1;
+ revs-tree_objects = 1;
+ revs-blob_objects = 1;
+ revs-edge_hint = 1;
+ revs-edge_hint_aggressive = 1;
} else if (!strcmp(arg, --verify-objects)) {
revs-tag_objects = 1;
revs-tree_objects = 1;
diff --git a/revision.h b/revision.h
index 9cb5adc..033a244 100644
--- a/revision.h
+++ b/revision.h
@@ -93,6 +93,7 @@ struct rev_info {
blob_objects:1,
verify_objects:1,
edge_hint:1,
+ edge_hint_aggressive:1,
limited:1,
unpacked:1,
boundary:2,
--
2.2.1.209.g41e5f3a
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a
Re: [PATCH v3 2/4] rev-list: add an option to mark fewer edges as uninteresting
On Tue, Dec 23, 2014 at 12:55:48PM -0500, Michael Blume wrote: This patch causes an error on my mac, test 5500 fetch-pack errors on part 44 - fetch creating new shallow root. It looks for remote: Total 1 in the fetch output and gets 3 instead. It fails for me on Linux, too. Interestingly the tip of the series passes. I didn't investigate further. -Peff -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 2/4] rev-list: add an option to mark fewer edges as uninteresting
Jeff King p...@peff.net writes: On Tue, Dec 23, 2014 at 12:55:48PM -0500, Michael Blume wrote: This patch causes an error on my mac, test 5500 fetch-pack errors on part 44 - fetch creating new shallow root. It looks for remote: Total 1 in the fetch output and gets 3 instead. It fails for me on Linux, too. Interestingly the tip of the series passes. I didn't investigate further. Not surprised, as the first three are the same from the yesterday's one that failed for everybody. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
