[jira] [Created] (IMPALA-8248) Re-organize authorization tests
Fredy Wijaya created IMPALA-8248: Summary: Re-organize authorization tests Key: IMPALA-8248 URL: https://issues.apache.org/jira/browse/IMPALA-8248 Project: IMPALA Issue Type: Sub-task Components: Infrastructure Reporter: Fredy Wijaya We have authorization tests that are specific to Sentry and authorization tests that can be applicable to any authorization provider. We need to re-organize the authorization tests to easily differentiate between Sentry-specific tests vs generic authorization tests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-8247) Backend tests from bit-stream-utils-test.cc and system-state-info-test.cc are not running
Joe McDonnell created IMPALA-8247: - Summary: Backend tests from bit-stream-utils-test.cc and system-state-info-test.cc are not running Key: IMPALA-8247 URL: https://issues.apache.org/jira/browse/IMPALA-8247 Project: IMPALA Issue Type: Bug Components: Backend, Infrastructure Affects Versions: Impala 3.2.0 Reporter: Joe McDonnell Assignee: Joe McDonnell Several tests in be/src/util were converted to the unified backend test executable in IMPALA-8071. Unfortunately, some tests stopped running as part of this change. Specifically, the tests in bit-stream-utils-test.cc and system-state-info-test.cc were not included in the test binary and stopped running. The tests changed to use ADD_UNIFIED_BE_LSAN_TEST, but the files were not included in the UtilTests library. These two files should be included in the test executable and we need to update the validation of the unified test executable (bin/validate-unified-backend-test-filters.py) to catch this case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-8246) Add a table with Hadoop components / versions and the type of timestamp they write in Parquet, and the problems we can expect when reading these with Impala.
Alex Rodoni created IMPALA-8246: --- Summary: Add a table with Hadoop components / versions and the type of timestamp they write in Parquet, and the problems we can expect when reading these with Impala. Key: IMPALA-8246 URL: https://issues.apache.org/jira/browse/IMPALA-8246 Project: IMPALA Issue Type: Improvement Components: Docs Reporter: Alex Rodoni Assignee: Alex Rodoni -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8212) Crash during startup in kudu::security::CanonicalizeKrb5Principal()
[
https://issues.apache.org/jira/browse/IMPALA-8212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Ho resolved IMPALA-8212.
Resolution: Fixed
Fix Version/s: Impala 3.2.0
> Crash during startup in kudu::security::CanonicalizeKrb5Principal()
> ---
>
> Key: IMPALA-8212
> URL: https://issues.apache.org/jira/browse/IMPALA-8212
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
> Environment: CentOS Linux release 7.4.1708 (Core)
> Linux vc0512.halxg.cloudera.com 3.10.0-693.21.1.el7.x86_64 #1 SMP Wed Mar 7
> 19:03:37 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
>Reporter: Tim Armstrong
>Assignee: Michael Ho
>Priority: Blocker
> Labels: crash
> Fix For: Impala 3.2.0
>
> Attachments: gdb-core-60055.txt, gdb.txt, hs_err_pid60055.log,
> hs_err_pid65365.log,
> impalad.vc0512.halxg.cloudera.com.impala.log.INFO.20190218-140034.65365,
> impalad.vc0513.halxg.cloudera.com.impala.log.INFO.20190216-142536.60055
>
>
> I saw this crash twice will working on the stress test. It *seems* to happen
> when the stress infrastructure switches the service to a debug build,
> restarts the service, then starts running queries. I haven't seen it happen
> once the service is up and running for a while.
> {noformat}
> #0 0x7fb03e1fa1f7 in raise () from sysroot/lib64/libc.so.6
> #1 0x7fb03e1fb8e8 in abort () from sysroot/lib64/libc.so.6
> #2 0x7fb041159185 in os::abort(bool) () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #3 0x7fb0412fb593 in VMError::report_and_die() () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #4 0x7fb04115e68f in JVM_handle_linux_signal () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #5 0x7fb041154be3 in signalHandler(int, siginfo*, void*) () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #6
> #7 0x048d0a53 in
> tcmalloc::ThreadCache::ReleaseToCentralCache(tcmalloc::ThreadCache::FreeList*,
> unsigned long, int) ()
> #8 0x048d0aec in
> tcmalloc::ThreadCache::ListTooLong(tcmalloc::ThreadCache::FreeList*, unsigned
> long) ()
> #9 0x04a0b4c0 in tc_free ()
> #10 0x7fb040d32933 in ElfDecoder::demangle(char const*, char*, int) ()
> from sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #11 0x7fb040d3222a in Decoder::demangle(char const*, char*, int) () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #12 0x7fb04115695d in os::dll_address_to_function_name(unsigned char*,
> char*, int, int*) () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #13 0x7fb040dc0222 in frame::print_C_frame(outputStream*, char*, int,
> unsigned char*) () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #14 0x7fb040d2e925 in print_native_stack(outputStream*, frame, Thread*,
> char*, int) () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #15 0x7fb0412f9cc8 in VMError::report(outputStream*) () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #16 0x7fb0412fb18a in VMError::report_and_die() () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #17 0x7fb04115e68f in JVM_handle_linux_signal () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #18 0x7fb041154be3 in signalHandler(int, siginfo*, void*) () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #19
> #20 0x048d0a53 in
> tcmalloc::ThreadCache::ReleaseToCentralCache(tcmalloc::ThreadCache::FreeList*,
> unsigned long, int) ()
> #21 0x048d0aec in
> tcmalloc::ThreadCache::ListTooLong(tcmalloc::ThreadCache::FreeList*, unsigned
> long) ()
> #22 0x04a0b4c0 in tc_free ()
> #23 0x7fb03e5915dd in pthread_attr_destroy () from
> sysroot/lib64/libpthread.so.0
> #24 0x7fb04115e49f in current_stack_region(unsigned char**, unsigned
> long*) () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #25 0x7fb04115e535 in os::current_stack_base() () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #26 0x7fb0412faeb4 in VMError::report(outputStream*) () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #27 0x7fb0412fb18a in VMError::report_and_die() () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
> #28 0x7fb04115e68f in JVM_handle_linux_signal () from
> sysroot/usr/java/jdk1.8.0_141-cloudera/jre/lib/amd64/server/libjvm.so
>
[jira] [Created] (IMPALA-8245) Add hostname to timeout error message in HdfsMonitoredOps
Joe McDonnell created IMPALA-8245: - Summary: Add hostname to timeout error message in HdfsMonitoredOps Key: IMPALA-8245 URL: https://issues.apache.org/jira/browse/IMPALA-8245 Project: IMPALA Issue Type: Bug Components: Backend Affects Versions: Impala 3.2.0 Reporter: Joe McDonnell If a DiskIo operation times out, it generates a TErrorCode::THREAD_POOL_TASK_TIMED_OUT or TErrorCode::THREAD_POOL_SUBMIT_FAILED error codes. These call GetDescription() to get DiskIo related context. That information should include the hostname where the error occurred to allow tracking down a problematic host that is seeing DiskIo timeouts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8064) test_min_max_filters is flaky
[
https://issues.apache.org/jira/browse/IMPALA-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pooja Nilangekar resolved IMPALA-8064.
--
Resolution: Fixed
> test_min_max_filters is flaky
> --
>
> Key: IMPALA-8064
> URL: https://issues.apache.org/jira/browse/IMPALA-8064
> Project: IMPALA
> Issue Type: Bug
>Reporter: Pooja Nilangekar
>Assignee: Pooja Nilangekar
>Priority: Blocker
> Labels: broken-build, flaky-test
> Fix For: Impala 3.2.0
>
> Attachments: profile.txt
>
>
> The following configuration of the test_min_max_filters:
> {code:java}
> query_test.test_runtime_filters.TestMinMaxFilters.test_min_max_filters[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold':
> 0} | table_format: kudu/none]{code}
> It produces a higher aggregation of sum over the proberows than expected:
> {code:java}
> query_test/test_runtime_filters.py:113: in test_min_max_filters
> self.run_test_case('QueryTest/min_max_filters', vector)
> common/impala_test_suite.py:518: in run_test_case
> update_section=pytest.config.option.update_results)
> common/test_result_verifier.py:612: in verify_runtime_profile %
> (function, field, expected_value, actual_value, actual))
> E AssertionError: Aggregation of SUM over ProbeRows did not match expected
> results.
> E EXPECTED VALUE: E 619
> EACTUAL VALUE: E 652
> {code}
> This test was introduced in the patch for IMPALA-6533. The failure occurred
> during an ASAN build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IMPALA-8244) Toolchain build fails to publish binaries even if asked to do so
Laszlo Gaal created IMPALA-8244:
---
Summary: Toolchain build fails to publish binaries even if asked
to do so
Key: IMPALA-8244
URL: https://issues.apache.org/jira/browse/IMPALA-8244
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.2.0
Reporter: Laszlo Gaal
The recent commit to native-toolchain that enabled the publishing of binary
artifacts from Docker-based toolchain builds
([https://github.com/cloudera/native-toolchain/commit/eeccbcab224fca5c25a7b78898444037b94d88b7)]
changed the logic that gates the publishing calls.
Internal builds on Jenkins (using the same logic as before the change) now
don't even attempt publishing even if parameterized to do so. Excerpt from the
build log:
{code:java}
02:19:44.605 install -d
/data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2/lib
02:19:44.608 install -m 644 -p libcctz.a
/data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2/lib
02:19:44.860 install -d
/data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2/lib
02:19:44.863 install -m 644 -p libcctz.so.2.2
/data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2/lib
02:19:44.880 make: Leaving directory
`/data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/source/cctz/cctz-2.2/build'
02:19:45.399 Cleaning
/data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/source/cctz
...
02:19:45.400 Entering directory
/data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/source/cctz
02:19:45.457 Removing cctz-2.2.tar.gz
02:19:45.457 Removing cctz-2.2/
02:19:45.459 Returning to directory
/data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain
02:19:45.459 # Build Complete cctz-2.2
02:19:45.459
###
02:19:45.549 + popd
02:19:45.549
/data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem
02:19:45.579 Archiving artifacts
02:19:45.781 [description-setter] Description set:
***redacted**
02:19:46.010 [WS-CLEANUP] Deleting project workspace...
02:19:46.010 [WS-CLEANUP] Deferred wipeout is used...
02:19:46.057 [WS-CLEANUP] done
02:19:46.057 Email was triggered for: Success
02:19:46.057 Sending email for trigger: Success
02:19:46.523 Sending email to: laszlo.g...@cloudera.com
02:19:46.551 Finished: SUCCESS{code}
The happy case with the missing steps (copied from another job that published
the bits successfully):
{code:java}
02:26:22.083 install -d
/data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2/lib
02:26:22.085 install -m 644 -p libcctz.a
/data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2/lib
02:26:22.338 install -d
/data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2/lib
02:26:22.340 install -m 644 -p libcctz.so.2.2
/data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/build/cctz-2.2/lib
02:26:22.358 make: Leaving directory
`/data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent58-ec2-c3-4xl-ondem/toolchain/source/cctz/cctz-2.2/build'
02:26:24.297 [INFO] Scanning for projects...
02:26:24.545 [INFO]
02:26:24.545 [INFO]
02:26:24.545 [INFO] Building Maven Stub Project (No POM) 1
02:26:24.546 [INFO]
02:26:24.552 [INFO]
02:26:24.552 [INFO] --- maven-deploy-plugin:2.7:deploy-file (default-cli) @
standalone-pom ---
02:26:25.079 Uploading:
http://maven.redacted/toolchain/cctz/2.2-gcc-4.9.2-253-1ffcf70e3f/cctz-2.2-gcc-4.9.2-253-1ffcf70e3f-ec2-package-centos-5.tar.gz
02:26:25.400 Uploaded:
http://maven.redacted/toolchain/cctz/2.2-gcc-4.9.2-253-1ffcf70e3f/cctz-2.2-gcc-4.9.2-253-1ffcf70e3f-ec2-package-centos-5.tar.gz
(2681 KB at 8325.9 KB/sec)
02:26:25.401 Uploading:
http://maven.redacted/toolchain/cctz/2.2-gcc-4.9.2-253-1ffcf70e3f/cctz-2.2-gcc-4.9.2-253-1ffcf70e3f.pom
02:26:25.421 Uploaded:
http://maven.redacted/toolchain/cctz/2
