Re: [PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job

2020-09-01 Thread Thomas Huth 
On 01/09/2020 08.12, Philippe Mathieu-Daudé wrote:
> On Tue, Sep 1, 2020 at 8:05 AM Thomas Huth  wrote:
>>
>> On 31/08/2020 23.31, Philippe Mathieu-Daudé wrote:
>>> Hi Thomas,
>>>
>>> Le lun. 31 août 2020 17:33, Thomas Huth >> > a écrit :
>>>
>>> The fuzzer job finishes quite early, so we can run the unit tests and
>>> qtests with -fsanitize=address here without extending the total test
>>> time.
>>>
>>> Signed-off-by: Thomas Huth mailto:th...@redhat.com>>
>>> ---
>>>  .gitlab-ci.yml | 5 +++--
>>>  1 file changed, 3 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>>> index 17f1f8fad9..417fda6909 100644
>>> --- a/.gitlab-ci.yml
>>> +++ b/.gitlab-ci.yml
>>> @@ -256,13 +256,14 @@ build-oss-fuzz:
>>>  - mkdir build-oss-fuzz
>>>  - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address"
>>>./scripts/oss-fuzz/build.sh
>>> +- export ASAN_OPTIONS="fast_unwind_on_malloc=0"
>>>  - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable
>>> -type f
>>>| grep -v slirp); do
>>>  grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 ||
>>> continue ;
>>>  echo Testing ${fuzzer} ... ;
>>> -ASAN_OPTIONS="fast_unwind_on_malloc=0"
>>> - "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
>>> +"${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
>>>done
>>> +- cd build-oss-fuzz && make check-qtest-i386 check-unit
>>>
>>>
>>> As this does not use the fuzzer main entry point, what is the point of
>>> running that?
>>
>> Read the friendly patch description ;-) - the idea here is that we run
>> the qtests and unit tests with -fsanitize=address in one of the
>> pipelines.
> 
> Sorry, the description was not that obvious to me.
> Can you add a comment before the 'make check*' line?
> 
>   # Unrelated to fuzzer: run tests with -fsanitize=address

Sure, will do.

 Thomas

Re: [PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job

2020-09-01 Thread Philippe Mathieu-Daudé 
On Tue, Sep 1, 2020 at 8:05 AM Thomas Huth  wrote:
>
> On 31/08/2020 23.31, Philippe Mathieu-Daudé wrote:
> > Hi Thomas,
> >
> > Le lun. 31 août 2020 17:33, Thomas Huth  > > a écrit :
> >
> > The fuzzer job finishes quite early, so we can run the unit tests and
> > qtests with -fsanitize=address here without extending the total test
> > time.
> >
> > Signed-off-by: Thomas Huth mailto:th...@redhat.com>>
> > ---
> >  .gitlab-ci.yml | 5 +++--
> >  1 file changed, 3 insertions(+), 2 deletions(-)
> >
> > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > index 17f1f8fad9..417fda6909 100644
> > --- a/.gitlab-ci.yml
> > +++ b/.gitlab-ci.yml
> > @@ -256,13 +256,14 @@ build-oss-fuzz:
> >  - mkdir build-oss-fuzz
> >  - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address"
> >./scripts/oss-fuzz/build.sh
> > +- export ASAN_OPTIONS="fast_unwind_on_malloc=0"
> >  - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable
> > -type f
> >| grep -v slirp); do
> >  grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 ||
> > continue ;
> >  echo Testing ${fuzzer} ... ;
> > -ASAN_OPTIONS="fast_unwind_on_malloc=0"
> > - "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
> > +"${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
> >done
> > +- cd build-oss-fuzz && make check-qtest-i386 check-unit
> >
> >
> > As this does not use the fuzzer main entry point, what is the point of
> > running that?
>
> Read the friendly patch description ;-) - the idea here is that we run
> the qtests and unit tests with -fsanitize=address in one of the
> pipelines.

Sorry, the description was not that obvious to me.
Can you add a comment before the 'make check*' line?

  # Unrelated to fuzzer: run tests with -fsanitize=address

> We could also use a different task for this, but since the
> fuzzer tests finish quite early and we already compiled the code with
> -fsanitize=address here anyway, it's IMHO simply a good fit here.
>
>  Thomas
>
>

Re: [PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job

2020-09-01 Thread Thomas Huth 
On 31/08/2020 18.40, Alexander Bulekov wrote:
> On 200831 1732, Thomas Huth wrote:
>> The fuzzer job finishes quite early, so we can run the unit tests and
>> qtests with -fsanitize=address here without extending the total test time.
>>
> 
> Sounds good to me, though its too bad that this is limited to i386 and
> we aren't using the --enable-sanitizers configure argument due to the
> strange oss-fuzz build script.
> 
> Speaking of testing with sanitizers, does it make sense to have a job
> that does check-qtest with --enable-tsan, now that we have
> ThreadSanitizer support? 
> 
> Can --enable-sanitizers --enable-tsan be added to some existing job,
> without severely increasing the pipeline's exeuction time?

Good idea, I'll have a try when I've got some spare time again (probably
not this week, though, it's a pretty busy one for me).

> Reviewed-by: Alexander Bulekov 

Thanks!

 Thomas

Re: [PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job

2020-09-01 Thread Thomas Huth 
On 31/08/2020 23.31, Philippe Mathieu-Daudé wrote:
> Hi Thomas,
> 
> Le lun. 31 août 2020 17:33, Thomas Huth  > a écrit :
> 
> The fuzzer job finishes quite early, so we can run the unit tests and
> qtests with -fsanitize=address here without extending the total test
> time.
> 
> Signed-off-by: Thomas Huth mailto:th...@redhat.com>>
> ---
>  .gitlab-ci.yml | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 17f1f8fad9..417fda6909 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -256,13 +256,14 @@ build-oss-fuzz:
>      - mkdir build-oss-fuzz
>      - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address"
>        ./scripts/oss-fuzz/build.sh
> +    - export ASAN_OPTIONS="fast_unwind_on_malloc=0"
>      - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable
> -type f
>                        | grep -v slirp); do
>          grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 ||
> continue ;
>          echo Testing ${fuzzer} ... ;
> -        ASAN_OPTIONS="fast_unwind_on_malloc=0"
> -         "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
> +        "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
>        done
> +    - cd build-oss-fuzz && make check-qtest-i386 check-unit
> 
> 
> As this does not use the fuzzer main entry point, what is the point of
> running that?

Read the friendly patch description ;-) - the idea here is that we run
the qtests and unit tests with -fsanitize=address in one of the
pipelines. We could also use a different task for this, but since the
fuzzer tests finish quite early and we already compiled the code with
-fsanitize=address here anyway, it's IMHO simply a good fit here.

 Thomas

Re: [PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job

2020-08-31 Thread Philippe Mathieu-Daudé 
Hi Thomas,

Le lun. 31 août 2020 17:33, Thomas Huth  a écrit :

> The fuzzer job finishes quite early, so we can run the unit tests and
> qtests with -fsanitize=address here without extending the total test time.
>
> Signed-off-by: Thomas Huth 
> ---
>  .gitlab-ci.yml | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 17f1f8fad9..417fda6909 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -256,13 +256,14 @@ build-oss-fuzz:
>  - mkdir build-oss-fuzz
>  - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address"
>./scripts/oss-fuzz/build.sh
> +- export ASAN_OPTIONS="fast_unwind_on_malloc=0"
>  - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable -type f
>| grep -v slirp); do
>  grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 ||
> continue ;
>  echo Testing ${fuzzer} ... ;
> -ASAN_OPTIONS="fast_unwind_on_malloc=0"
> - "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
> +"${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
>done
> +- cd build-oss-fuzz && make check-qtest-i386 check-unit
>

As this does not use the fuzzer main entry point, what is the point of
running that?


>  build-tci:
><<: *native_build_job_definition
> --
> 2.18.2
>
>
>

Re: [PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job

2020-08-31 Thread Alexander Bulekov 
On 200831 1732, Thomas Huth wrote:
> The fuzzer job finishes quite early, so we can run the unit tests and
> qtests with -fsanitize=address here without extending the total test time.
> 

Sounds good to me, though its too bad that this is limited to i386 and
we aren't using the --enable-sanitizers configure argument due to the
strange oss-fuzz build script.

Speaking of testing with sanitizers, does it make sense to have a job
that does check-qtest with --enable-tsan, now that we have
ThreadSanitizer support? 

Can --enable-sanitizers --enable-tsan be added to some existing job,
without severely increasing the pipeline's exeuction time?

Reviewed-by: Alexander Bulekov 

> Signed-off-by: Thomas Huth 
> ---
>  .gitlab-ci.yml | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 17f1f8fad9..417fda6909 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -256,13 +256,14 @@ build-oss-fuzz:
>  - mkdir build-oss-fuzz
>  - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address"
>./scripts/oss-fuzz/build.sh
> +- export ASAN_OPTIONS="fast_unwind_on_malloc=0"
>  - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable -type f
>| grep -v slirp); do
>  grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 || continue 
> ;
>  echo Testing ${fuzzer} ... ;
> -ASAN_OPTIONS="fast_unwind_on_malloc=0"
> - "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
> +"${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
>done
> +- cd build-oss-fuzz && make check-qtest-i386 check-unit
>  
>  build-tci:
><<: *native_build_job_definition
> -- 
> 2.18.2
>

[PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job

2020-08-31 Thread Thomas Huth 
The fuzzer job finishes quite early, so we can run the unit tests and
qtests with -fsanitize=address here without extending the total test time.

Signed-off-by: Thomas Huth 
---
 .gitlab-ci.yml | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 17f1f8fad9..417fda6909 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -256,13 +256,14 @@ build-oss-fuzz:
 - mkdir build-oss-fuzz
 - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address"
   ./scripts/oss-fuzz/build.sh
+- export ASAN_OPTIONS="fast_unwind_on_malloc=0"
 - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable -type f
   | grep -v slirp); do
 grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 || continue ;
 echo Testing ${fuzzer} ... ;
-ASAN_OPTIONS="fast_unwind_on_malloc=0"
- "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
+"${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
   done
+- cd build-oss-fuzz && make check-qtest-i386 check-unit
 
 build-tci:
   <<: *native_build_job_definition
-- 
2.18.2