Re: [PATCH] commit-graph: split up close_reachable() progress output

2018-11-19 Thread SZEDER Gábor
On Mon, Nov 19, 2018 at 08:23:00PM +, Ævar Arnfjörð Bjarmason wrote:
> Amend the progress output added in 7b0f229222 ("commit-graph write:
> add progress output", 2018-09-17) so that the total numbers it reports
> aren't higher than the total number of commits anymore. See [1] for a
> bug report pointing that out.

Please make the commit message more self-contained, i.e. describe the
issue this patch fixes in more detail, so readers won't have to follow
links to understand the problem.

> When I added this I wasn't intending to provide an accurate count, but
> just have some progress output to show the user the command wasn't
> hanging[2]. But since we are showing numbers, let's make them
> accurate. The progress descriptions were suggested by Derrick Stolee
> in [3].
> 
> As noted in [2] we are unlikely to show anything except the "Expanding
> reachable..." message even on fairly large repositories such as
> linux.git. On a test repository I have with north of 7 million commits
> all of these are displayed. Two of them don't show up for long, but as
> noted in [5] future-proofing this for if the loops become more
> expensive in the future makes sense.

In my opinion this is rather one of those "we'll cross that bridge
when (or if ever) we get there" situations.

> 1. https://public-inbox.org/git/20181010203738.ge23...@szeder.dev/
> 2. https://public-inbox.org/git/87pnwhea8y@evledraar.gmail.com/
> 3. 
> https://public-inbox.org/git/f7a0cbee-863c-61d3-4959-5cec8b43c...@gmail.com/
> 4. https://public-inbox.org/git/20181015160545.gg19...@szeder.dev/
> 5. https://public-inbox.org/git/87murle8da@evledraar.gmail.com/
> 
> Reported-by: SZEDER Gábor 
> Helped-by: Derrick Stolee 
> Signed-off-by: Ævar Arnfjörð Bjarmason 
> ---
> 
> On Mon, Nov 19 2018, SZEDER Gábor wrote:
> 
> > Ping?
> >
> > We are at -rc0, this progress output is a new feature since v2.19.0,
> > and the numbers shown are still way off.
> 
> I was under the impression after your
> https://public-inbox.org/git/20181015160545.gg19...@szeder.dev/ that
> you were going to do some more digging & report back, so I put it on
> my "waiting for feedback" list and then forgot about it.

No, after I managed to "get my numbers straight" I sent another bug
report in

  https://public-inbox.org/git/20181015165447.gh19...@szeder.dev/

but as a reply to your original patch.  Sorry about the confusion.

> But here's a patch that should address the issue you pointed out, but
> I don't know if it fixes whatever you were alluding to in the linked
> E-Mail above.

I'm afraid this patch doesn't address that issue, as it's limited to
close_reachable(), and that issue is related to the progress output in
add_packed_commits().

>  commit-graph.c | 13 ++---
>  1 file changed, 10 insertions(+), 3 deletions(-)
> 
> diff --git a/commit-graph.c b/commit-graph.c
> index 40c855f185..9c0d6914be 100644
> --- a/commit-graph.c
> +++ b/commit-graph.c
> @@ -641,26 +641,29 @@ static void add_missing_parents(struct packed_oid_list 
> *oids, struct commit *com
>  
>  static void close_reachable(struct packed_oid_list *oids, int 
> report_progress)
>  {
> - int i;
> + int i, j;
>   struct commit *commit;
>   struct progress *progress = NULL;
> - int j = 0;
>  
>   if (report_progress)
>   progress = start_delayed_progress(
> - _("Annotating commits in commit graph"), 0);
> + _("Loading known commits in commit graph"), j = 0);
>   for (i = 0; i < oids->nr; i++) {
>   display_progress(progress, ++j);
>   commit = lookup_commit(the_repository, >list[i]);
>   if (commit)
>   commit->object.flags |= UNINTERESTING;
>   }
> + stop_progress();
>  
>   /*
>* As this loop runs, oids->nr may grow, but not more
>* than the number of missing commits in the reachable
>* closure.
>*/
> + if (report_progress)
> + progress = start_delayed_progress(
> + _("Expanding reachable commits in commit graph"), j = 
> 0);
>   for (i = 0; i < oids->nr; i++) {
>   display_progress(progress, ++j);
>   commit = lookup_commit(the_repository, >list[i]);
> @@ -668,7 +671,11 @@ static void close_reachable(struct packed_oid_list 
> *oids, int report_progress)
>   if (commit && !parse_commit(commit))
>   add_missing_parents(oids, commit);
>   }
> + stop_progress();
>  
> + if (report_progress)
> + progress = start_delayed_progress(
> + _("Clearing commit marks in commit graph"), j = 0);
>   for (i = 0; i < oids->nr; i++) {
>   display_progress(progress, ++j);
>   commit = lookup_commit(the_repository, >list[i]);
> -- 
> 2.19.1.1182.g4ecb1133ce
> 


Re: [PATCH] commit-graph: split up close_reachable() progress output

2018-11-19 Thread Derrick Stolee

On 11/19/2018 3:23 PM, Ævar Arnfjörð Bjarmason wrote:

+   if (report_progress)
+   progress = start_delayed_progress(
+   _("Expanding reachable commits in commit graph"), j = 
0);


This should be the only one that shows up in all but the very largest of 
repositories.


LGTM.

Thanks,
-Stolee


[PATCH] commit-graph: split up close_reachable() progress output

2018-11-19 Thread Ævar Arnfjörð Bjarmason
Amend the progress output added in 7b0f229222 ("commit-graph write:
add progress output", 2018-09-17) so that the total numbers it reports
aren't higher than the total number of commits anymore. See [1] for a
bug report pointing that out.

When I added this I wasn't intending to provide an accurate count, but
just have some progress output to show the user the command wasn't
hanging[2]. But since we are showing numbers, let's make them
accurate. The progress descriptions were suggested by Derrick Stolee
in [3].

As noted in [2] we are unlikely to show anything except the "Expanding
reachable..." message even on fairly large repositories such as
linux.git. On a test repository I have with north of 7 million commits
all of these are displayed. Two of them don't show up for long, but as
noted in [5] future-proofing this for if the loops become more
expensive in the future makes sense.

1. https://public-inbox.org/git/20181010203738.ge23...@szeder.dev/
2. https://public-inbox.org/git/87pnwhea8y@evledraar.gmail.com/
3. https://public-inbox.org/git/f7a0cbee-863c-61d3-4959-5cec8b43c...@gmail.com/
4. https://public-inbox.org/git/20181015160545.gg19...@szeder.dev/
5. https://public-inbox.org/git/87murle8da@evledraar.gmail.com/

Reported-by: SZEDER Gábor 
Helped-by: Derrick Stolee 
Signed-off-by: Ævar Arnfjörð Bjarmason 
---

On Mon, Nov 19 2018, SZEDER Gábor wrote:

> Ping?
>
> We are at -rc0, this progress output is a new feature since v2.19.0,
> and the numbers shown are still way off.

I was under the impression after your
https://public-inbox.org/git/20181015160545.gg19...@szeder.dev/ that
you were going to do some more digging & report back, so I put it on
my "waiting for feedback" list and then forgot about it.

But here's a patch that should address the issue you pointed out, but
I don't know if it fixes whatever you were alluding to in the linked
E-Mail above.

 commit-graph.c | 13 ++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/commit-graph.c b/commit-graph.c
index 40c855f185..9c0d6914be 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -641,26 +641,29 @@ static void add_missing_parents(struct packed_oid_list 
*oids, struct commit *com
 
 static void close_reachable(struct packed_oid_list *oids, int report_progress)
 {
-   int i;
+   int i, j;
struct commit *commit;
struct progress *progress = NULL;
-   int j = 0;
 
if (report_progress)
progress = start_delayed_progress(
-   _("Annotating commits in commit graph"), 0);
+   _("Loading known commits in commit graph"), j = 0);
for (i = 0; i < oids->nr; i++) {
display_progress(progress, ++j);
commit = lookup_commit(the_repository, >list[i]);
if (commit)
commit->object.flags |= UNINTERESTING;
}
+   stop_progress();
 
/*
 * As this loop runs, oids->nr may grow, but not more
 * than the number of missing commits in the reachable
 * closure.
 */
+   if (report_progress)
+   progress = start_delayed_progress(
+   _("Expanding reachable commits in commit graph"), j = 
0);
for (i = 0; i < oids->nr; i++) {
display_progress(progress, ++j);
commit = lookup_commit(the_repository, >list[i]);
@@ -668,7 +671,11 @@ static void close_reachable(struct packed_oid_list *oids, 
int report_progress)
if (commit && !parse_commit(commit))
add_missing_parents(oids, commit);
}
+   stop_progress();
 
+   if (report_progress)
+   progress = start_delayed_progress(
+   _("Clearing commit marks in commit graph"), j = 0);
for (i = 0; i < oids->nr; i++) {
display_progress(progress, ++j);
commit = lookup_commit(the_repository, >list[i]);
-- 
2.19.1.1182.g4ecb1133ce