[ 
https://issues.apache.org/jira/browse/IMPALA-13128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17860284#comment-17860284
 ] 

ASF subversion and git services commented on IMPALA-13128:
----------------------------------------------------------

Commit 8d05f5134cc95f53e4e4bbd8ceb9de88b845fda1 in impala's branch 
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=8d05f5134 ]

IMPALA-13128: disk-file-test Hangs on ARM + UBSAN Test Jobs

The Jenkins jobs that run the UBSAN tests on ARM were occaisonally
hanging on the disk-file-test. This commit fixes these hangs by
upgrading Google Test and implementing the Death Test handling
functionality which safely runs tests that expect the process to die.
See https://github.com/google/googletest/blob/main/docs/advanced.md#death-tests
for details on known problems with running death tests and threads at
the same time causing tests to hang.

Testing was accomplished by running the disk-file-test repeatedly in a
loop on a RHEL 8.9 ARM machine. Before this fix was implemented, this
test would run up to 70 times before it hung. After the fix was
implemented, the test ran 2,490 times and was still running when it was
stopped. These test runs had durations between 18.7 and 19.9 seconds
which means disk-file-test now takes about 15 seconds longer than its
previous duration of about 4.4 seconds.

Change-Id: Ie01f7781f24644a66e9ec52652450116f5cb4297
Reviewed-on: http://gerrit.cloudera.org:8080/21544
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>


> disk-file-test hangs on ARM + UBSAN test jobs
> ---------------------------------------------
>
>                 Key: IMPALA-13128
>                 URL: https://issues.apache.org/jira/browse/IMPALA-13128
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Backend
>    Affects Versions: Impala 4.5.0
>            Reporter: Joe McDonnell
>            Priority: Critical
>              Labels: broken-build, flaky
>
> The UBSAN ARM job (running on Redhat 8) has been hanging then timing out with 
> this being the last output:
> {noformat}
> 23:06:47  63/147 Test  #63: disk-io-mgr-test .................   Passed   
> 43.42 sec
> 23:07:30         Start  64: disk-file-test
> 23:07:30 
> 18:47:00 
> 18:47:00 **** run-all-tests.sh TIMED OUT! ****{noformat}
> This has happened multiple times, but it looks limited to ARM + UBSAN. The 
> jobs take stack traces, but only of the running impalads / HMS.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

Reply via email to