Re: [PATCH 1/4] travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output

[Johannes Schindelin]

Hi Gábor,

On Fri, 15 Dec 2017, SZEDER Gábor wrote:

> On Fri, Dec 15, 2017 at 1:10 PM, Johannes Schindelin
>  wrote:
> >
> >> There is a lot going on in 'run-windows-build.sh', so the output of
> >> 'set -x' might be useful or might be considered too much clutter, I
> >> don't know.  I put Dscho on Cc, I think it's mainly his call.
> >
> > Certainly it might be useful.
> >
> > However, please make sure that the secret token is not leaked that
> > way.  Like, *really* sure. Due to the failure of Git to use a portable
> > and performant test suite, it does take about 90 minutes to build and
> > test a revision, therefore it would be very easy to DOS my build
> > system, and I really, really need it not to be DOSed because I use it
> > in my day job, too.
> 
> Ugh, I was completely unaware of this issue, and the first version of
> this patch is already in 'pu'...  **runs to check the trace logs**
> 
> Great, it seems we are in luck, as the secret token was specified as an
> encrypted environment variable in git/git repository settings on Travis
> CI.  It's redacted in the trace log [...]

That's good enough.

Thanks for making sure,
Dscho

Re: [PATCH 1/4] travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output

[SZEDER Gábor]

On Fri, Dec 15, 2017 at 1:10 PM, Johannes Schindelin
 wrote:
> Hi,
>
>> There is a lot going on in 'run-windows-build.sh', so the output of 'set
>> -x' might be useful or might be considered too much clutter, I don't
>> know.  I put Dscho on Cc, I think it's mainly his call.
>
> Certainly it might be useful.
>
> However, please make sure that the secret token is not leaked that way.
> Like, *really* sure. Due to the failure of Git to use a portable and
> performant test suite, it does take about 90 minutes to build and test a
> revision, therefore it would be very easy to DOS my build system, and I
> really, really need it not to be DOSed because I use it in my day job, too.

Ugh, I was completely unaware of this issue, and the first version of
this patch is already in 'pu'...  **runs to check the trace logs**

Great, it seems we are in luck, as the secret token was specified as an
encrypted environment variable in git/git repository settings on Travis
CI.  It's redacted in the trace log and I only see lines like these:

  Setting environment variables from repository settings
  $ export GFW_CI_TOKEN=[secure]

  +test -z [secure]

  +++curl -H 'Authentication: Bearer [secure]' --silent --retry 5
--write-out '%{HTTP_CODE}' --output /dev/fd/63
'https://git-for-windows-ci.azurewebsites.net/api/TestNow?action=trigger=pu=1229713f78cd2883798e95f33c19c81b523413fd=false'

  https://travis-ci.org/git/git/jobs/316791071

Phew.

However, I don't feel competent enough with Travis CI's encrypted
environment variables to say that this qualifies as "*really* sure"
"that the secret token is not leaked".
Anyway, note, that that '$ export GFW_CI_TOKEN=[secure]' line is already
present in all 'git/git' trace logs independently of this 'set -x'
change in question.

Gábor

Re: [PATCH 1/4] travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output

[Johannes Schindelin]

Hi,

> There is a lot going on in 'run-windows-build.sh', so the output of 'set
> -x' might be useful or might be considered too much clutter, I don't
> know.  I put Dscho on Cc, I think it's mainly his call.

Certainly it might be useful.

However, please make sure that the secret token is not leaked that way.
Like, *really* sure. Due to the failure of Git to use a portable and
performant test suite, it does take about 90 minutes to build and test a
revision, therefore it would be very easy to DOS my build system, and I
really, really need it not to be DOSed because I use it in my day job, too.

Ciao,
Dscho

Re: [PATCH 1/4] travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output

[SZEDER Gábor]

On Thu, Dec 14, 2017 at 12:10 AM, Lars Schneider
 wrote:
>
>> On 12 Dec 2017, at 19:43, SZEDER Gábor  wrote:
>>
>> On Tue, Dec 12, 2017 at 7:00 PM, Lars Schneider
>>  wrote:
>>>
 On 12 Dec 2017, at 00:34, SZEDER Gábor  wrote:

 While the build logic was embedded in our '.travis.yml', Travis CI
 used to produce a nice trace log including all commands executed in
 those embedded scriptlets.  Since 657343a60 (travis-ci: move Travis CI
 code into dedicated scripts, 2017-09-10), however, we only see the
 name of the dedicated scripts, but not what those scripts are actually
 doing, resulting in a less useful trace log.  A patch later in this
 series will move setting environment variables from '.travis.yml' to
 the 'ci/*' scripts, so not even those will be included in the trace
 log.

 Use 'set -x' in 'ci/lib-travisci.sh', which is sourced in most other
 'ci/*' scripts, so we get trace log about the commands executed in all
 of those scripts.
>>>
>>> I kind of did that intentionally to avoid clutter in the logs.
>>> However, I also agree with your reasoning. Therefore, the change
>>> looks good to me!
>>
>> Great, 'cause I'm starting to have second thoughts about this change :)
>>
>> It sure helped a lot while I worked on this patch series and a couple of
>> other Travis CI related patches (will submit them later)...  OTOH it
>> definitely creates clutter in the trace log.  The worst offender might
>> be 'ci/print-test-failures.sh', which iterates over all
>> 't/test-results/*.exit' files to find which tests failed and to show
>> their output, and 'set -x' makes every iteration visible.  And we have
>> about 800 tests, which means 800 iterations.  Yuck.
>>
>> Perhaps we should use other means to show what's going on instead, e.g.
>> use more 'echo's and '--verbose' options, or just avoid using '--quiet'.
>> And if some brave souls really want to tweak '.travis.yml' or the 'ci/*'
>> scripts, then they can set 'set -x' for themselves during development...
>> as the patch below shows it's easy enough, just a single character :)
>
> Hm... in that case. Would it be an option to "set -x" only in the header
> of "install-dependencies.sh"?
>
> In "lib-travisci.sh" we could keep your "set -x" and execute
> "set +x" at the end of the file. Wouldn't that give us the
> interesting traces without much clutter (e.g. what is $PATH etc)?

Hmm, that's an idea worth considering.

Scripts like 'run-build.sh', 'run-tests.sh' and 'run-static-analysis.sh'
do basically nothing more than run make with different targets, so on
one hand 'set -x' doesn't cause any clutter in the trace log, on the
other hand there is no benefit from it either.
'run-linux32-docker.sh' runs docker (the command) twice, so it's
basically in the same boat.

I think both 'lib-travisci.sh' and 'install-dependencies.sh' benefit
from 'set -x'.
So does 'test-documentation.sh': it executes about 15 commands, among
them a bunch of 'test -s ' which fail quietly.  With 'set -x' we
would see the last executed command and know that that's the one that
failed.

As mentioned above, 'print-test-failures.sh' definitely suffers from
'set -x'.

There is a lot going on in 'run-windows-build.sh', so the output of 'set
-x' might be useful or might be considered too much clutter, I don't
know.  I put Dscho on Cc, I think it's mainly his call.

So it seems that there are more scripts that would benefit from tracing
executed command using 'set -x' than scripts that would suffer because
of it, and it doesn't matter for the rest.  This means we could issue a
'set -x' in 'lib-travisci.sh' and disable it only in
'print-test-failures.sh'.

There is one thing that triggers my OCD: whenever we echo something, it
ends up being duplicated in the trace log, e.g.:

  + echo foo bar baz
  foo bar baz

We could workaround it by writing 'echo "" >/dev/null', but it
feels hackish and I'm not sure it's worth it.


Gábor

 Signed-off-by: SZEDER Gábor 
 ---
 ci/lib-travisci.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

 diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh
 index ac05f1f46..a0c8ae03f 100755
 --- a/ci/lib-travisci.sh
 +++ b/ci/lib-travisci.sh
 @@ -23,7 +23,7 @@ skip_branch_tip_with_tag () {

 # Set 'exit on error' for all CI scripts to let the caller know that
 # something went wrong
 -set -e
 +set -ex

 skip_branch_tip_with_tag

 --
 2.15.1.421.gc469ca1de

>>>
>

Re: [PATCH 1/4] travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output

[Lars Schneider]

> On 12 Dec 2017, at 19:43, SZEDER Gábor  wrote:
> 
> On Tue, Dec 12, 2017 at 7:00 PM, Lars Schneider
>  wrote:
>> 
>>> On 12 Dec 2017, at 00:34, SZEDER Gábor  wrote:
>>> 
>>> While the build logic was embedded in our '.travis.yml', Travis CI
>>> used to produce a nice trace log including all commands executed in
>>> those embedded scriptlets.  Since 657343a60 (travis-ci: move Travis CI
>>> code into dedicated scripts, 2017-09-10), however, we only see the
>>> name of the dedicated scripts, but not what those scripts are actually
>>> doing, resulting in a less useful trace log.  A patch later in this
>>> series will move setting environment variables from '.travis.yml' to
>>> the 'ci/*' scripts, so not even those will be included in the trace
>>> log.
>>> 
>>> Use 'set -x' in 'ci/lib-travisci.sh', which is sourced in most other
>>> 'ci/*' scripts, so we get trace log about the commands executed in all
>>> of those scripts.
>> 
>> I kind of did that intentionally to avoid clutter in the logs.
>> However, I also agree with your reasoning. Therefore, the change
>> looks good to me!
> 
> Great, 'cause I'm starting to have second thoughts about this change :)
> 
> It sure helped a lot while I worked on this patch series and a couple of
> other Travis CI related patches (will submit them later)...  OTOH it
> definitely creates clutter in the trace log.  The worst offender might
> be 'ci/print-test-failures.sh', which iterates over all
> 't/test-results/*.exit' files to find which tests failed and to show
> their output, and 'set -x' makes every iteration visible.  And we have
> about 800 tests, which means 800 iterations.  Yuck.
> 
> Perhaps we should use other means to show what's going on instead, e.g.
> use more 'echo's and '--verbose' options, or just avoid using '--quiet'.
> And if some brave souls really want to tweak '.travis.yml' or the 'ci/*'
> scripts, then they can set 'set -x' for themselves during development...
> as the patch below shows it's easy enough, just a single character :)

Hm... in that case. Would it be an option to "set -x" only in the header
of "install-dependencies.sh"?

In "lib-travisci.sh" we could keep your "set -x" and execute
"set +x" at the end of the file. Wouldn't that give us the 
interesting traces without much clutter (e.g. what is $PATH etc)?

- Lars


> 
> 
> Gábor
> 
> 
>>> 
>>> Signed-off-by: SZEDER Gábor 
>>> ---
>>> ci/lib-travisci.sh | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>> 
>>> diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh
>>> index ac05f1f46..a0c8ae03f 100755
>>> --- a/ci/lib-travisci.sh
>>> +++ b/ci/lib-travisci.sh
>>> @@ -23,7 +23,7 @@ skip_branch_tip_with_tag () {
>>> 
>>> # Set 'exit on error' for all CI scripts to let the caller know that
>>> # something went wrong
>>> -set -e
>>> +set -ex
>>> 
>>> skip_branch_tip_with_tag
>>> 
>>> --
>>> 2.15.1.421.gc469ca1de
>>> 
>> 

Re: [PATCH 1/4] travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output

[SZEDER Gábor]

On Tue, Dec 12, 2017 at 7:00 PM, Lars Schneider
 wrote:
>
>> On 12 Dec 2017, at 00:34, SZEDER Gábor  wrote:
>>
>> While the build logic was embedded in our '.travis.yml', Travis CI
>> used to produce a nice trace log including all commands executed in
>> those embedded scriptlets.  Since 657343a60 (travis-ci: move Travis CI
>> code into dedicated scripts, 2017-09-10), however, we only see the
>> name of the dedicated scripts, but not what those scripts are actually
>> doing, resulting in a less useful trace log.  A patch later in this
>> series will move setting environment variables from '.travis.yml' to
>> the 'ci/*' scripts, so not even those will be included in the trace
>> log.
>>
>> Use 'set -x' in 'ci/lib-travisci.sh', which is sourced in most other
>> 'ci/*' scripts, so we get trace log about the commands executed in all
>> of those scripts.
>
> I kind of did that intentionally to avoid clutter in the logs.
> However, I also agree with your reasoning. Therefore, the change
> looks good to me!

Great, 'cause I'm starting to have second thoughts about this change :)

It sure helped a lot while I worked on this patch series and a couple of
other Travis CI related patches (will submit them later)...  OTOH it
definitely creates clutter in the trace log.  The worst offender might
be 'ci/print-test-failures.sh', which iterates over all
't/test-results/*.exit' files to find which tests failed and to show
their output, and 'set -x' makes every iteration visible.  And we have
about 800 tests, which means 800 iterations.  Yuck.

Perhaps we should use other means to show what's going on instead, e.g.
use more 'echo's and '--verbose' options, or just avoid using '--quiet'.
And if some brave souls really want to tweak '.travis.yml' or the 'ci/*'
scripts, then they can set 'set -x' for themselves during development...
as the patch below shows it's easy enough, just a single character :)


Gábor


>>
>> Signed-off-by: SZEDER Gábor 
>> ---
>> ci/lib-travisci.sh | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh
>> index ac05f1f46..a0c8ae03f 100755
>> --- a/ci/lib-travisci.sh
>> +++ b/ci/lib-travisci.sh
>> @@ -23,7 +23,7 @@ skip_branch_tip_with_tag () {
>>
>> # Set 'exit on error' for all CI scripts to let the caller know that
>> # something went wrong
>> -set -e
>> +set -ex
>>
>> skip_branch_tip_with_tag
>>
>> --
>> 2.15.1.421.gc469ca1de
>>
>

Re: [PATCH 1/4] travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output

[Lars Schneider]

> On 12 Dec 2017, at 00:34, SZEDER Gábor  wrote:
> 
> While the build logic was embedded in our '.travis.yml', Travis CI
> used to produce a nice trace log including all commands executed in
> those embedded scriptlets.  Since 657343a60 (travis-ci: move Travis CI
> code into dedicated scripts, 2017-09-10), however, we only see the
> name of the dedicated scripts, but not what those scripts are actually
> doing, resulting in a less useful trace log.  A patch later in this
> series will move setting environment variables from '.travis.yml' to
> the 'ci/*' scripts, so not even those will be included in the trace
> log.
> 
> Use 'set -x' in 'ci/lib-travisci.sh', which is sourced in most other
> 'ci/*' scripts, so we get trace log about the commands executed in all
> of those scripts.

I kind of did that intentionally to avoid clutter in the logs.
However, I also agree with your reasoning. Therefore, the change
looks good to me!

Thanks,
Lars

> 
> Signed-off-by: SZEDER Gábor 
> ---
> ci/lib-travisci.sh | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh
> index ac05f1f46..a0c8ae03f 100755
> --- a/ci/lib-travisci.sh
> +++ b/ci/lib-travisci.sh
> @@ -23,7 +23,7 @@ skip_branch_tip_with_tag () {
> 
> # Set 'exit on error' for all CI scripts to let the caller know that
> # something went wrong
> -set -e
> +set -ex
> 
> skip_branch_tip_with_tag
> 
> -- 
> 2.15.1.421.gc469ca1de
> 

[PATCH 1/4] travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output

[SZEDER Gábor]

While the build logic was embedded in our '.travis.yml', Travis CI
used to produce a nice trace log including all commands executed in
those embedded scriptlets.  Since 657343a60 (travis-ci: move Travis CI
code into dedicated scripts, 2017-09-10), however, we only see the
name of the dedicated scripts, but not what those scripts are actually
doing, resulting in a less useful trace log.  A patch later in this
series will move setting environment variables from '.travis.yml' to
the 'ci/*' scripts, so not even those will be included in the trace
log.

Use 'set -x' in 'ci/lib-travisci.sh', which is sourced in most other
'ci/*' scripts, so we get trace log about the commands executed in all
of those scripts.

Signed-off-by: SZEDER Gábor 
---
 ci/lib-travisci.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh
index ac05f1f46..a0c8ae03f 100755
--- a/ci/lib-travisci.sh
+++ b/ci/lib-travisci.sh
@@ -23,7 +23,7 @@ skip_branch_tip_with_tag () {
 
 # Set 'exit on error' for all CI scripts to let the caller know that
 # something went wrong
-set -e
+set -ex
 
 skip_branch_tip_with_tag
 
-- 
2.15.1.421.gc469ca1de