Re: [PATCH] run-command: do not pass child process data into callbacks

[Junio C Hamano]

Johannes Sixt  writes:

> Am 29.02.2016 um 22:57 schrieb Stefan Beller:
>> The expected way to pass data into the callback is to pass them via
>> the customizable callback pointer. The error reporting in
>> default_{start_failure, task_finished} is not user friendly enough, that
>> we want to encourage using the child data for such purposes.
>> 
>> Furthermore the struct child data is cleaned by the run-command API,
>> before we access them in the callbacks, leading to use-after-free
>> situations.
>
> Thanks. The code changes match what I had prototyped. But please squash
> in this documentation change:
>
> diff --git a/run-command.h b/run-command.h
> index c6a3e42..3d1e59e 100644
> --- a/run-command.h
> +++ b/run-command.h
> @@ -191,9 +191,8 @@ typedef int (*task_finished_fn)(int result,
>   * (both stdout and stderr) is routed to stderr in a manner that output
>   * from different tasks does not interleave.
>   *
> - * If start_failure_fn or task_finished_fn are NULL, default handlers
> - * will be used. The default handlers will print an error message on
> - * error without issuing an emergency stop.
> + * start_failure_fn and task_finished_fn can be NULL to omit any
> + * special handling.
>   */
>  int run_processes_parallel(int n,
>  get_next_task_fn,

Thanks for careful reading.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] run-command: do not pass child process data into callbacks

[Johannes Sixt]

Am 29.02.2016 um 22:57 schrieb Stefan Beller:
> The expected way to pass data into the callback is to pass them via
> the customizable callback pointer. The error reporting in
> default_{start_failure, task_finished} is not user friendly enough, that
> we want to encourage using the child data for such purposes.
> 
> Furthermore the struct child data is cleaned by the run-command API,
> before we access them in the callbacks, leading to use-after-free
> situations.

Thanks. The code changes match what I had prototyped. But please squash
in this documentation change:

diff --git a/run-command.h b/run-command.h
index c6a3e42..3d1e59e 100644
--- a/run-command.h
+++ b/run-command.h
@@ -191,9 +191,8 @@ typedef int (*task_finished_fn)(int result,
  * (both stdout and stderr) is routed to stderr in a manner that output
  * from different tasks does not interleave.
  *
- * If start_failure_fn or task_finished_fn are NULL, default handlers
- * will be used. The default handlers will print an error message on
- * error without issuing an emergency stop.
+ * start_failure_fn and task_finished_fn can be NULL to omit any
+ * special handling.
  */
 int run_processes_parallel(int n,
   get_next_task_fn,

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] run-command: do not pass child process data into callbacks

[Junio C Hamano]

Stefan Beller  writes:

> It applies on 2.8.0-rc0.
>
> (I tried backporting it to 2.7, but realized it is not an issue there)

Thanks.  I tried to be nice to rebase the parallel-update stuff on
top of this myself, but stopped it as I won't have enough time to
push the result out for the day anyway.

submodule-parallel-update, submodule-init and refs-backend-lmdb
topics will be missing from tonight's 'pu'.



--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] run-command: do not pass child process data into callbacks

[Stefan Beller]

It applies on 2.8.0-rc0.

(I tried backporting it to 2.7, but realized it is not an issue there)
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] run-command: do not pass child process data into callbacks

[Stefan Beller]

The expected way to pass data into the callback is to pass them via
the customizable callback pointer. The error reporting in
default_{start_failure, task_finished} is not user friendly enough, that
we want to encourage using the child data for such purposes.

Furthermore the struct child data is cleaned by the run-command API,
before we access them in the callbacks, leading to use-after-free
situations.

Signed-off-by: Stefan Beller 
---

 This is the proper fix to not access memory after freeing.

 run-command.c  | 24 +++-
 run-command.h  |  4 +---
 submodule.c|  7 +++
 test-run-command.c |  1 -
 4 files changed, 7 insertions(+), 29 deletions(-)

diff --git a/run-command.c b/run-command.c
index 863dad5..c726010 100644
--- a/run-command.c
+++ b/run-command.c
@@ -902,35 +902,18 @@ struct parallel_processes {
struct strbuf buffered_output; /* of finished children */
 };
 
-static int default_start_failure(struct child_process *cp,
-struct strbuf *err,
+static int default_start_failure(struct strbuf *err,
 void *pp_cb,
 void *pp_task_cb)
 {
-   int i;
-
-   strbuf_addstr(err, "Starting a child failed:");
-   for (i = 0; cp->argv[i]; i++)
-   strbuf_addf(err, " %s", cp->argv[i]);
-
return 0;
 }
 
 static int default_task_finished(int result,
-struct child_process *cp,
 struct strbuf *err,
 void *pp_cb,
 void *pp_task_cb)
 {
-   int i;
-
-   if (!result)
-   return 0;
-
-   strbuf_addf(err, "A child failed with return code %d:", result);
-   for (i = 0; cp->argv[i]; i++)
-   strbuf_addf(err, " %s", cp->argv[i]);
-
return 0;
 }
 
@@ -1048,8 +1031,7 @@ static int pp_start_one(struct parallel_processes *pp)
pp->children[i].process.no_stdin = 1;
 
if (start_command(>children[i].process)) {
-   code = pp->start_failure(>children[i].process,
->children[i].err,
+   code = pp->start_failure(>children[i].err,
 pp->data,
 >children[i].data);
strbuf_addbuf(>buffered_output, >children[i].err);
@@ -1117,7 +1099,7 @@ static int pp_collect_finished(struct parallel_processes 
*pp)
 
code = finish_command(>children[i].process);
 
-   code = pp->task_finished(code, >children[i].process,
+   code = pp->task_finished(code,
 >children[i].err, pp->data,
 >children[i].data);
 
diff --git a/run-command.h b/run-command.h
index 42917e8..c6a3e42 100644
--- a/run-command.h
+++ b/run-command.h
@@ -159,8 +159,7 @@ typedef int (*get_next_task_fn)(struct child_process *cp,
  * To send a signal to other child processes for abortion, return
  * the negative signal number.
  */
-typedef int (*start_failure_fn)(struct child_process *cp,
-   struct strbuf *err,
+typedef int (*start_failure_fn)(struct strbuf *err,
void *pp_cb,
void *pp_task_cb);
 
@@ -179,7 +178,6 @@ typedef int (*start_failure_fn)(struct child_process *cp,
  * the negative signal number.
  */
 typedef int (*task_finished_fn)(int result,
-   struct child_process *cp,
struct strbuf *err,
void *pp_cb,
void *pp_task_cb);
diff --git a/submodule.c b/submodule.c
index 24fb81a..62c4356 100644
--- a/submodule.c
+++ b/submodule.c
@@ -705,8 +705,7 @@ static int get_next_submodule(struct child_process *cp,
return 0;
 }
 
-static int fetch_start_failure(struct child_process *cp,
-  struct strbuf *err,
+static int fetch_start_failure(struct strbuf *err,
   void *cb, void *task_cb)
 {
struct submodule_parallel_fetch *spf = cb;
@@ -716,8 +715,8 @@ static int fetch_start_failure(struct child_process *cp,
return 0;
 }
 
-static int fetch_finish(int retvalue, struct child_process *cp,
-   struct strbuf *err, void *cb, void *task_cb)
+static int fetch_finish(int retvalue, struct strbuf *err,
+   void *cb, void *task_cb)
 {
struct submodule_parallel_fetch *spf = cb;
 
diff --git a/test-run-command.c b/test-run-command.c
index fbe0a27..30a64a9 100644
--- a/test-run-command.c
+++ b/test-run-command.c
@@ -41,7 +41,6 @@ static int no_job(struct child_process *cp,
 }
 
 static int task_finished(int result,
-struct child_process *cp,
 struct strbuf *err,