[jira] [Commented] (IMPALA-8895) Expose daemon health on /healthz endpoint
[ https://issues.apache.org/jira/browse/IMPALA-8895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192543#comment-17192543 ] ASF subversion and git services commented on IMPALA-8895: - Commit fc51cd3bc0fa5e57e7b15fc07dbff9ced5d089c6 in impala's branch refs/heads/master from Bikramjeet Vig [ https://gitbox.apache.org/repos/asf?p=impala.git;h=fc51cd3 ] IMPALA-10052: Expose daemon health endpoint for statestore and catalog This change exposes the daemon health of statestored and catalogd via an HTTP endpoint '/healthz'. If the server is healthy, this endpoint will return HTTP code 200 (OK). If it is unhealthy, it will return 503 (Service Unavailable). This is consistent with the endpoint added for impalads in IMPALA-8895. Testing: - Extended test in test_web_pages.py Change-Id: I7714734df8e50dabbbebcb77a86a5a00bd13bf7c Reviewed-on: http://gerrit.cloudera.org:8080/16295 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Expose daemon health on /healthz endpoint > - > > Key: IMPALA-8895 > URL: https://issues.apache.org/jira/browse/IMPALA-8895 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 3.4.0 >Reporter: Lars Volker >Assignee: Lars Volker >Priority: Major > Labels: observability > Fix For: Impala 3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10052) Expose daemon health on /healthz endpoint for catalogd and statestored as well
[ https://issues.apache.org/jira/browse/IMPALA-10052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192542#comment-17192542 ] ASF subversion and git services commented on IMPALA-10052: -- Commit fc51cd3bc0fa5e57e7b15fc07dbff9ced5d089c6 in impala's branch refs/heads/master from Bikramjeet Vig [ https://gitbox.apache.org/repos/asf?p=impala.git;h=fc51cd3 ] IMPALA-10052: Expose daemon health endpoint for statestore and catalog This change exposes the daemon health of statestored and catalogd via an HTTP endpoint '/healthz'. If the server is healthy, this endpoint will return HTTP code 200 (OK). If it is unhealthy, it will return 503 (Service Unavailable). This is consistent with the endpoint added for impalads in IMPALA-8895. Testing: - Extended test in test_web_pages.py Change-Id: I7714734df8e50dabbbebcb77a86a5a00bd13bf7c Reviewed-on: http://gerrit.cloudera.org:8080/16295 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Expose daemon health on /healthz endpoint for catalogd and statestored as well > -- > > Key: IMPALA-10052 > URL: https://issues.apache.org/jira/browse/IMPALA-10052 > Project: IMPALA > Issue Type: Improvement >Reporter: Bikramjeet Vig >Assignee: Bikramjeet Vig >Priority: Major > Labels: observability > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10124) admission-controller-test fails with no such file or directory error
[
https://issues.apache.org/jira/browse/IMPALA-10124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192541#comment-17192541
]
ASF subversion and git services commented on IMPALA-10124:
--
Commit 1e63722f8da7cd94a6937891832e51eae834c956 in impala's branch
refs/heads/master from Qifan Chen
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=1e63722 ]
IMPALA-10124 admission-controller-test fails with no such file or
directory error
This work addresses a failure by disabling undefined behavior sanitizer
testing for AdmissionControllerTest.TopNQueryCheck test. In the test,
std::regex_match() is used to verify the appearance of certain strings
and can produce a core with very long stack trace failling in
std::vector::operator[]().
Testing:
1. Ran the test in both regular and disabling undefined behavior
sanitizer check modes. No core was seen.
Change-Id: I16d6cff8fad8d0e93a24ec3fefa9cc1f8c471aad
Reviewed-on: http://gerrit.cloudera.org:8080/16404
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> admission-controller-test fails with no such file or directory error
>
>
> Key: IMPALA-10124
> URL: https://issues.apache.org/jira/browse/IMPALA-10124
> Project: IMPALA
> Issue Type: Bug
>Reporter: Yongzhi Chen
>Assignee: Qifan Chen
>Priority: Major
>
> In master-core-ubsan, the admission-controller-test fails :
> 03:12:04
> /data/jenkins/workspace/impala-asf-master-core-ubsan/repos/Impala/be/build/debug//scheduling/admission-controller-test:
> line 10: 29380 Segmentation fault (core dumped)
> ${IMPALA_HOME}/bin/run-jvm-binary.sh
> ${IMPALA_HOME}/be/build/latest/service/unifiedbetests
> --gtest_filter=${GTEST_FILTER}
> --gtest_output=xml:${IMPALA_BE_TEST_LOGS_DIR}/${TEST_EXEC_NAME}.xml
> -log_filename="${TEST_EXEC_NAME}" "$@"
> 03:12:04 Traceback (most recent call last):
> 03:12:04 File
> "/data/jenkins/workspace/impala-asf-master-core-ubsan/repos/Impala/bin/junitxml_prune_notrun.py",
> line 71, in
> 03:12:04 if __name__ == "__main__": main()
> 03:12:04 File
> "/data/jenkins/workspace/impala-asf-master-core-ubsan/repos/Impala/bin/junitxml_prune_notrun.py",
> line 68, in main
> 03:12:04 junitxml_prune_notrun(options.filename)
> 03:12:04 File
> "/data/jenkins/workspace/impala-asf-master-core-ubsan/repos/Impala/bin/junitxml_prune_notrun.py",
> line 31, in junitxml_prune_notrun
> 03:12:04 root = tree.parse(junitxml_filename)
> 03:12:04 File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 647, in
> parse
> 03:12:04 source = open(source, "rb")
> 03:12:04 IOError: [Errno 2] No such file or directory:
> '/data/jenkins/workspace/impala-asf-master-core-ubsan/repos/Impala/logs/be_tests/admission-controller-test.xml'
> ...
> 03:18:30 The following tests FAILED:
> 03:18:30 57 - admission-controller-test (Failed)
> 03:18:30 Errors while running CTest
> 03:18:30 make: *** [test] Error 8
> 03:18:30 ERROR in
> /data/jenkins/workspace/impala-asf-master-core-ubsan/repos/Impala/bin/run-backend-tests.sh
> at line 43: "${MAKE_CMD:-make}" test ARGS="${BE_TEST_ARGS}"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10090) Create aarch64 development environment on ubuntu 18.04
[ https://issues.apache.org/jira/browse/IMPALA-10090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192544#comment-17192544 ] ASF subversion and git services commented on IMPALA-10090: -- Commit 6aaea3216ca2fabe352b9f11a7d140e9d9c183c9 in impala's branch refs/heads/master from huangtianhua [ https://gitbox.apache.org/repos/asf?p=impala.git;h=6aaea32 ] IMPALA-10090 Pull newest code of native-toolchain before build it If native-toolchain exists we should pull the newest code before build it. Change-Id: I2da3ffce7abb88190be0a5ea0e2cf603f98ee15e Reviewed-on: http://gerrit.cloudera.org:8080/16402 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Create aarch64 development environment on ubuntu 18.04 > -- > > Key: IMPALA-10090 > URL: https://issues.apache.org/jira/browse/IMPALA-10090 > Project: IMPALA > Issue Type: Sub-task >Reporter: zhaorenhai >Assignee: zhaorenhai >Priority: Major > > Including following changes: > 1 build native-toolchain local by script on aarch64 platform > 2 change some native-toolchain's lib version number > 3 split SKIP_TOOLCHAIN_BOOTSTRAP and DOWNLOAD_CDH_COMPONETS to two things, > because on aarch64, just need to download cdp components , but not need to > download toolchain. > 4 download hadoop aarch64 nativelibs , impala building needs these libs. > With this commit, on ubuntu 18.04 aarch64 version, just need to run > bin/bootstrap_development.sh, just like x86. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9351) AnalyzeDDLTest.TestCreateTableLikeFileOrc failed due to non-existing path
[
https://issues.apache.org/jira/browse/IMPALA-9351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192522#comment-17192522
]
Sahil Takiar commented on IMPALA-9351:
--
Another instance:
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/11985/testReport/junit/org.apache.impala.analysis/AnalyzeDDLTest/TestCreateTableLikeFileOrc/
> AnalyzeDDLTest.TestCreateTableLikeFileOrc failed due to non-existing path
> -
>
> Key: IMPALA-9351
> URL: https://issues.apache.org/jira/browse/IMPALA-9351
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Reporter: Fang-Yu Rao
>Assignee: Norbert Luksa
>Priority: Blocker
> Labels: broken-build, flaky-test
> Fix For: Impala 3.4.0
>
>
> AnalyzeDDLTest.TestCreateTableLikeFileOrc failed due to a non-existing path.
> Specifically, we see the following error message.
> {code:java}
> Error Message
> Error during analysis:
> org.apache.impala.common.AnalysisException: Cannot infer schema, path does
> not exist:
> hdfs://localhost:20500/test-warehouse/functional_orc_def.db/complextypes_fileformat/00_0
> sql:
> create table if not exists newtbl_DNE like orc
> '/test-warehouse/functional_orc_def.db/complextypes_fileformat/00_0'
> {code}
> The stack trace is provided in the following.
> {code:java}
> Stacktrace
> java.lang.AssertionError:
> Error during analysis:
> org.apache.impala.common.AnalysisException: Cannot infer schema, path does
> not exist:
> hdfs://localhost:20500/test-warehouse/functional_orc_def.db/complextypes_fileformat/00_0
> sql:
> create table if not exists newtbl_DNE like orc
> '/test-warehouse/functional_orc_def.db/complextypes_fileformat/00_0'
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.impala.common.FrontendFixture.analyzeStmt(FrontendFixture.java:397)
> at
> org.apache.impala.common.FrontendTestBase.AnalyzesOk(FrontendTestBase.java:244)
> at
> org.apache.impala.common.FrontendTestBase.AnalyzesOk(FrontendTestBase.java:185)
> at
> org.apache.impala.analysis.AnalyzeDDLTest.TestCreateTableLikeFileOrc(AnalyzeDDLTest.java:2045)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)
> {code}
> This test was recently added by [~norbertluksa], and [~boroknagyz] gave a +2,
> maybe [~boroknagyz] could provide some insight into this? Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (IMPALA-6984) Coordinator should cancel backends when returning EOS
[
https://issues.apache.org/jira/browse/IMPALA-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar reassigned IMPALA-6984:
Assignee: Wenzhe Zhou
> Coordinator should cancel backends when returning EOS
> -
>
> Key: IMPALA-6984
> URL: https://issues.apache.org/jira/browse/IMPALA-6984
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
>Affects Versions: Impala 3.0
>Reporter: Daniel Hecht
>Assignee: Wenzhe Zhou
>Priority: Major
> Labels: query-lifecycle
> Fix For: Impala 4.0
>
>
> Currently, the Coordinator waits for backends rather than proactively
> cancelling them in the case of hitting EOS. There's a tangled mess that makes
> it tricky to proactively cancel the backends related to how
> {{Coordinator::ComputeQuerySummary()}} works – we can't update the summary
> until the profiles are no longer changing (which also makes sense given that
> we want the exec summary to be consistent with the final profile). But we
> current tie together the FIS status and the profile, and cancellation of
> backends causes the FIS to return CANCELLED, which then means that the
> remaining FIS on that backend won't produce a final profile.
> With the rework of the protocol for IMPALA-2990 we should make it possible to
> sort this out such that a final profile can be requested regardless of how a
> FIS ends execution.
> This also relates to IMPALA-5783.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-5119) Don't make RPCs from Coordinator::UpdateBackendExecStatus()
[
https://issues.apache.org/jira/browse/IMPALA-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar reassigned IMPALA-5119:
Assignee: Wenzhe Zhou
> Don't make RPCs from Coordinator::UpdateBackendExecStatus()
> ---
>
> Key: IMPALA-5119
> URL: https://issues.apache.org/jira/browse/IMPALA-5119
> Project: IMPALA
> Issue Type: Sub-task
> Components: Distributed Exec
>Affects Versions: Impala 2.9.0
>Reporter: Henry Robinson
>Assignee: Wenzhe Zhou
>Priority: Major
>
> If it reports a bad status, {{UpdateFragmentExecStatus()}} will call
> {{UpdateStatus()}}, which takes {{Coordinator::lock_}} and then calls
> {{Cancel()}}. That method issues one RPC per fragment instance.
> In KRPC, doing so much work from {{UpdateFragmentExecStatus()}} - which is an
> RPC handler - is a bad idea, even if the RPCs are issued asynchronously.
> There's still some serialization cost.
> It's also a bad idea to do all this work while holding {{lock_}}. We should
> address both of these to ensure scalability of the cancellation path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-8291) 'DESCRIBE EXTENDED ..' does not display constraint information
[ https://issues.apache.org/jira/browse/IMPALA-8291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shant Hovsepian reassigned IMPALA-8291: --- Assignee: Shant Hovsepian > 'DESCRIBE EXTENDED ..' does not display constraint information > -- > > Key: IMPALA-8291 > URL: https://issues.apache.org/jira/browse/IMPALA-8291 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Anurag Mantripragada >Assignee: Shant Hovsepian >Priority: Major > > Currently, DESCRIBE EXTENDED table_name command does not display constraint > information like primary key / Foreign key information for tables created > through Hive. > This work must also be extended to tables created through Impala once we have > support for pk/fk in create table syntax. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-10157) IllegalStateException when using grouping() or grouping_id() with no GROUP BY clause
[
https://issues.apache.org/jira/browse/IMPALA-10157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-10157:
---
Description:
{noformat}https://issues.apache.org/jira/browse/IMPALA-10157#
[localhost.EXAMPLE.COM:21000] default> select int_col, tinyint_col,
grouping(int_col), grouping(tinyint_col), grouping_id(int_col, tinyint_col)
from functional.alltypes;
Query: select int_col, tinyint_col, grouping(int_col), grouping(tinyint_col),
grouping_id(int_col, tinyint_col) from functional.alltypes
Query submitted at: 2020-09-08 10:25:12 (Coordinator:
http://tarmstrong-box2:25000)
ERROR: IllegalStateException: null
{noformat}
{noformat}
I0908 10:25:12.834306 19139 Frontend.java:1507]
a845d2d57577cc5c:b905608c] Analyzing query: select int_col,
tinyint_col, grouping(int_col), grouping(tinyint_col), grouping_id(int_col,
tinyint_col) from functional.alltypes db: default
I0908 10:25:12.839713 19139 jni-util.cc:288] a845d2d57577cc5c:b905608c]
java.lang.IllegalStateException
at
com.google.common.base.Preconditions.checkState(Preconditions.java:492)
at
org.apache.impala.analysis.MultiAggregateInfo.analyze(MultiAggregateInfo.java:274)
at
org.apache.impala.analysis.SelectStmt$SelectAnalyzer.buildAggregateExprs(SelectStmt.java:845)
at
org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyze(SelectStmt.java:269)
at
org.apache.impala.analysis.SelectStmt$SelectAnalyzer.access$100(SelectStmt.java:237)
at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:230)
at
org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:472)
at
org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:436)
at
org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1547)
at
org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1514)
at
org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1484)
at
org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:162)
I0908 10:25:12.839778 19139 status.cc:129] a845d2d57577cc5c:b905608c]
IllegalStateException: null
@ 0x1ce443b impala::Status::Status()
@ 0x2644a01 impala::JniUtil::GetJniExceptionMsg()
@ 0x2446c96 impala::JniCall::Call<>()
@ 0x2443e49 impala::JniUtil::CallJniMethod<>()
@ 0x2442150 impala::Frontend::GetExecRequest()
@ 0x2cc4d47 impala::QueryDriver::RunFrontendPlanner()
@ 0x246f844 impala::ImpalaServer::ExecuteInternal()
@ 0x246f202 impala::ImpalaServer::Execute()
@ 0x25098c6 impala::ImpalaServer::query()
@ 0x2afb7df beeswax::BeeswaxServiceProcessor::process_query()
@ 0x2afb52d beeswax::BeeswaxServiceProcessor::dispatchCall()
@ 0x2ac8976 impala::ImpalaServiceProcessor::dispatchCall()
@ 0x1c92db3 apache::thrift::TDispatchProcessor::process()
@ 0x2195f36
apache::thrift::server::TAcceptQueueServer::Task::run()
@ 0x218b436 impala::ThriftThread::RunRunnable()
@ 0x218ca72 boost::_mfi::mf2<>::operator()()
@ 0x218c906 boost::_bi::list3<>::operator()<>()
@ 0x218c64c boost::_bi::bind_t<>::operator()()
@ 0x218c55e
boost::detail::function::void_function_obj_invoker0<>::invoke()
@ 0x2107043 boost::function0<>::operator()()
@ 0x26d5ea5 impala::Thread::SuperviseThread()
@ 0x26dde42 boost::_bi::list5<>::operator()<>()
@ 0x26ddd66 boost::_bi::bind_t<>::operator()()
@ 0x26ddd27 boost::detail::thread_data<>::run()
@ 0x3ec7271 thread_proxy
@ 0x7fa5b50b76da start_thread
@ 0x7fa5b1aaea3e clone
{noformat}
was:
{noformat}
[localhost.EXAMPLE.COM:21000] default> select int_col, tinyint_col,
grouping(int_col), grouping(tinyint_col), grouping_id(int_col, tinyint_col)
from functional.alltypes;
Query: select int_col, tinyint_col, grouping(int_col), grouping(tinyint_col),
grouping_id(int_col, tinyint_col) from functional.alltypes
Query submitted at: 2020-09-08 10:25:12 (Coordinator:
http://tarmstrong-box2:25000)
ERROR: IllegalStateException: null
{noformat}
{noformat}
I0908 10:25:12.834306 19139 Frontend.java:1507]
a845d2d57577cc5c:b905608c] Analyzing query: select int_col,
tinyint_col, grouping(int_col), grouping(tinyint_col), grouping_id(int_col,
tinyint_col) from functional.alltypes db: default
I0908 10:25:12.839713 19139 jni-util.cc:288] a845d2d57577cc5c:b905608c]
java.lang.IllegalStateException
at
com.google.common.base.Preconditions.checkState(Preconditions.java:492)
at
org.apache.impala.analysis.MultiAggregateInfo.analyze(MultiAggregateInfo.java:274)
at
[jira] [Created] (IMPALA-10158) test_iceberg_query and test_iceberg_profile fail after IMPALA-9741
Fang-Yu Rao created IMPALA-10158:
Summary: test_iceberg_query and test_iceberg_profile fail after
IMPALA-9741
Key: IMPALA-10158
URL: https://issues.apache.org/jira/browse/IMPALA-10158
Project: IMPALA
Issue Type: Bug
Reporter: Fang-Yu Rao
Assignee: Fang-Yu Rao
We found that the tests of {{test_iceberg_query}} and {{test_iceberg_profile}}
fail after the patch for IMPALA-9741 has been merged.
After some investigation with the help of [~boroknagyz] and [~csringhofer], we
found that it is a timezone-related issue and that we should add {{SET
TIMEZONE=UTC}} in the corresponding test files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-9227) Test coverage for query retries when there is a network partition
[ https://issues.apache.org/jira/browse/IMPALA-9227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar reassigned IMPALA-9227: Assignee: Wenzhe Zhou > Test coverage for query retries when there is a network partition > - > > Key: IMPALA-9227 > URL: https://issues.apache.org/jira/browse/IMPALA-9227 > Project: IMPALA > Issue Type: Sub-task > Components: Backend >Reporter: Sahil Takiar >Assignee: Wenzhe Zhou >Priority: Major > > The initial version of transparent query retries just adds coverage for > retrying a query if an impalad crashes. Now that the Impala has an RPC fault > injection framework (IMPALA-8138) based on debug actions, integration tests > can introduce network partitions between two impalad processes. > Node blacklisting should cause the Impala Coordinator to blacklist the nodes > with the network partitions (IMPALA-9137), and then transparent query retries > should cause the query to be successfully retried. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10157) IllegalStateException when using grouping() or grouping_id() with no GROUP BY clause
Tim Armstrong created IMPALA-10157:
--
Summary: IllegalStateException when using grouping() or
grouping_id() with no GROUP BY clause
Key: IMPALA-10157
URL: https://issues.apache.org/jira/browse/IMPALA-10157
Project: IMPALA
Issue Type: Bug
Components: Frontend
Reporter: Tim Armstrong
Assignee: Tim Armstrong
{noformat}
[localhost.EXAMPLE.COM:21000] default> select int_col, tinyint_col,
grouping(int_col), grouping(tinyint_col), grouping_id(int_col, tinyint_col)
from functional.alltypes;
Query: select int_col, tinyint_col, grouping(int_col), grouping(tinyint_col),
grouping_id(int_col, tinyint_col) from functional.alltypes
Query submitted at: 2020-09-08 10:25:12 (Coordinator:
http://tarmstrong-box2:25000)
ERROR: IllegalStateException: null
{noformat}
{noformat}
I0908 10:25:12.834306 19139 Frontend.java:1507]
a845d2d57577cc5c:b905608c] Analyzing query: select int_col,
tinyint_col, grouping(int_col), grouping(tinyint_col), grouping_id(int_col,
tinyint_col) from functional.alltypes db: default
I0908 10:25:12.839713 19139 jni-util.cc:288] a845d2d57577cc5c:b905608c]
java.lang.IllegalStateException
at
com.google.common.base.Preconditions.checkState(Preconditions.java:492)
at
org.apache.impala.analysis.MultiAggregateInfo.analyze(MultiAggregateInfo.java:274)
at
org.apache.impala.analysis.SelectStmt$SelectAnalyzer.buildAggregateExprs(SelectStmt.java:845)
at
org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyze(SelectStmt.java:269)
at
org.apache.impala.analysis.SelectStmt$SelectAnalyzer.access$100(SelectStmt.java:237)
at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:230)
at
org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:472)
at
org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:436)
at
org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1547)
at
org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1514)
at
org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1484)
at
org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:162)
I0908 10:25:12.839778 19139 status.cc:129] a845d2d57577cc5c:b905608c]
IllegalStateException: null
@ 0x1ce443b impala::Status::Status()
@ 0x2644a01 impala::JniUtil::GetJniExceptionMsg()
@ 0x2446c96 impala::JniCall::Call<>()
@ 0x2443e49 impala::JniUtil::CallJniMethod<>()
@ 0x2442150 impala::Frontend::GetExecRequest()
@ 0x2cc4d47 impala::QueryDriver::RunFrontendPlanner()
@ 0x246f844 impala::ImpalaServer::ExecuteInternal()
@ 0x246f202 impala::ImpalaServer::Execute()
@ 0x25098c6 impala::ImpalaServer::query()
@ 0x2afb7df beeswax::BeeswaxServiceProcessor::process_query()
@ 0x2afb52d beeswax::BeeswaxServiceProcessor::dispatchCall()
@ 0x2ac8976 impala::ImpalaServiceProcessor::dispatchCall()
@ 0x1c92db3 apache::thrift::TDispatchProcessor::process()
@ 0x2195f36
apache::thrift::server::TAcceptQueueServer::Task::run()
@ 0x218b436 impala::ThriftThread::RunRunnable()
@ 0x218ca72 boost::_mfi::mf2<>::operator()()
@ 0x218c906 boost::_bi::list3<>::operator()<>()
@ 0x218c64c boost::_bi::bind_t<>::operator()()
@ 0x218c55e
boost::detail::function::void_function_obj_invoker0<>::invoke()
@ 0x2107043 boost::function0<>::operator()()
@ 0x26d5ea5 impala::Thread::SuperviseThread()
@ 0x26dde42 boost::_bi::list5<>::operator()<>()
@ 0x26ddd66 boost::_bi::bind_t<>::operator()()
@ 0x26ddd27 boost::detail::thread_data<>::run()
@ 0x3ec7271 thread_proxy
@ 0x7fa5b50b76da start_thread
@ 0x7fa5b1aaea3e clone
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10156) test_unmatched_schema flaky with wrong results
Thomas Tauber-Marshall created IMPALA-10156:
---
Summary: test_unmatched_schema flaky with wrong results
Key: IMPALA-10156
URL: https://issues.apache.org/jira/browse/IMPALA-10156
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 4.0
Reporter: Thomas Tauber-Marshall
{noformat}
Error Message
query_test/test_scanners.py:244: in test_unmatched_schema
self.run_test_case('QueryTest/test-unmatched-schema', vector)
common/impala_test_suite.py:693: in run_test_case
self.__verify_results_and_errors(vector, test_section, result, use_db)
common/impala_test_suite.py:529: in __verify_results_and_errors
replace_filenames_with_placeholder) common/test_result_verifier.py:456: in
verify_raw_results VERIFIER_MAP[verifier](expected, actual)
common/test_result_verifier.py:278: in verify_query_result_is_equal assert
expected_results == actual_results E assert Comparing QueryTestResults
(expected vs actual): E 1001,'Name1',94611,5000 == 1001,'Name1',94611,5000
E 1002,'Name2',94611,5000 != 1001,'Name1',94611,5000 E
1003,'Name3',94611,5000 != 1002,'Name2',94611,5000 E
1004,'Name4',94611,5000 != 1002,'Name2',94611,5000 E
1005,'Name5',94611,5000 != 1003,'Name3',94611,5000 E
1006,'Name16',94612,15000 != 1003,'Name3',94611,5000 E
1006,'Name16',94612,5000 != 1004,'Name4',94611,5000 E
1006,'Name16',94616,15000 != 1004,'Name4',94611,5000 E
1006,'Name16',94616,5000 != 1005,'Name5',94611,5000 E
1006,'Name6',94616,15000 != 1005,'Name5',94611,5000 E
1006,'Name6',94616,5000 != 1006,'Name16',94612,15000 E
1106,'Name16',94612,15000 != 1006,'Name16',94612,15000 E
1106,'Name16',94612,5000 != 1006,'Name16',94612,5000 E
1106,'Name16',94616,15000 != 1006,'Name16',94612,5000 E
1106,'Name16',94616,5000 != 1006,'Name16',94616,15000 E
1106,'Name6',94612,15000 != 1006,'Name16',94616,15000 E
1106,'Name6',94612,5000 != 1006,'Name16',94616,5000 E
1106,'Name6',94616,15000 != 1006,'Name16',94616,5000 E
1106,'Name6',94616,5000 != 1006,'Name6',94616,15000 E None !=
1006,'Name6',94616,15000 E None != 1006,'Name6',94616,5000 E None !=
1006,'Name6',94616,5000 E None != 1106,'Name16',94612,15000 E None !=
1106,'Name16',94612,15000 E None != 1106,'Name16',94612,5000 E None !=
1106,'Name16',94612,5000 E None != 1106,'Name16',94616,15000 E None !=
1106,'Name16',94616,15000 E None != 1106,'Name16',94616,5000 E None !=
1106,'Name16',94616,5000 E None != 1106,'Name6',94612,15000 E None !=
1106,'Name6',94612,15000 E None != 1106,'Name6',94612,5000 E None !=
1106,'Name6',94612,5000 E None != 1106,'Name6',94616,15000 E None !=
1106,'Name6',94616,15000 E None != 1106,'Name6',94616,5000 E None !=
1106,'Name6',94616,5000 E Number of rows returned (expected vs actual): 19
!= 38
Stacktrace
query_test/test_scanners.py:244: in test_unmatched_schema
self.run_test_case('QueryTest/test-unmatched-schema', vector)
common/impala_test_suite.py:693: in run_test_case
self.__verify_results_and_errors(vector, test_section, result, use_db)
common/impala_test_suite.py:529: in __verify_results_and_errors
replace_filenames_with_placeholder)
common/test_result_verifier.py:456: in verify_raw_results
VERIFIER_MAP[verifier](expected, actual)
common/test_result_verifier.py:278: in verify_query_result_is_equal
assert expected_results == actual_results
E assert Comparing QueryTestResults (expected vs actual):
E 1001,'Name1',94611,5000 == 1001,'Name1',94611,5000
E 1002,'Name2',94611,5000 != 1001,'Name1',94611,5000
E 1003,'Name3',94611,5000 != 1002,'Name2',94611,5000
E 1004,'Name4',94611,5000 != 1002,'Name2',94611,5000
E 1005,'Name5',94611,5000 != 1003,'Name3',94611,5000
E 1006,'Name16',94612,15000 != 1003,'Name3',94611,5000
E 1006,'Name16',94612,5000 != 1004,'Name4',94611,5000
E 1006,'Name16',94616,15000 != 1004,'Name4',94611,5000
E 1006,'Name16',94616,5000 != 1005,'Name5',94611,5000
E 1006,'Name6',94616,15000 != 1005,'Name5',94611,5000
E 1006,'Name6',94616,5000 != 1006,'Name16',94612,15000
E 1106,'Name16',94612,15000 != 1006,'Name16',94612,15000
E 1106,'Name16',94612,5000 != 1006,'Name16',94612,5000
E 1106,'Name16',94616,15000 != 1006,'Name16',94612,5000
E 1106,'Name16',94616,5000 != 1006,'Name16',94616,15000
E 1106,'Name6',94612,15000 != 1006,'Name16',94616,15000
E 1106,'Name6',94612,5000 != 1006,'Name16',94616,5000
E 1106,'Name6',94616,15000 != 1006,'Name16',94616,5000
E 1106,'Name6',94616,5000 != 1006,'Name6',94616,15000
E None != 1006,'Name6',94616,15000
E None != 1006,'Name6',94616,5000
E None != 1006,'Name6',94616,5000
E None != 1106,'Name16',94612,15000
E None != 1106,'Name16',94612,15000
E None != 1106,'Name16',94612,5000
[jira] [Created] (IMPALA-10155) Apparent data race in GetTopNQueriesAndUpdatePoolStats
Thomas Tauber-Marshall created IMPALA-10155:
---
Summary: Apparent data race in GetTopNQueriesAndUpdatePoolStats
Key: IMPALA-10155
URL: https://issues.apache.org/jira/browse/IMPALA-10155
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 4.0
Reporter: Thomas Tauber-Marshall
Assignee: Qifan Chen
>From a tsan build:
{noformat}
WARNING: ThreadSanitizer: data race (pid=6487)
Read of size 1 at 0x7b48001c2c28 by thread T320 (mutexes: write
M866233966607478128, write M867078391537609888, write M627824798772259232,
write M451058461859238408):
#0
impala::MemTracker::GetTopNQueriesAndUpdatePoolStats(std::priority_queue >,
std::greater >&, int, impala::TPoolStats&)
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/runtime/mem-tracker.cc:453:19
(impalad+0x20b2e51)
#1 impala::MemTracker::UpdatePoolStatsForQueries(int, impala::TPoolStats&)
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/runtime/mem-tracker.cc:432:3
(impalad+0x20b2cdd)
#2 impala::AdmissionController::PoolStats::UpdateMemTrackerStats()
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/admission-controller.cc:1642:14
(impalad+0x21cb7b0)
#3
impala::AdmissionController::AddPoolUpdates(std::vector >*)
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/admission-controller.cc:1662:18
(impalad+0x21c8af3)
#4
impala::AdmissionController::UpdatePoolStats(std::map, std::allocator >, impala::TTopicDelta,
std::less,
std::allocator > >,
std::allocator, std::allocator > const, impala::TTopicDelta> > >
const&, std::vector
>*)
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/admission-controller.cc:1355:5
(impalad+0x21c881d)
#5
impala::AdmissionController::Init()::$_4::operator()(std::map, std::allocator >, impala::TTopicDelta,
std::less,
std::allocator > >,
std::allocator, std::allocator > const, impala::TTopicDelta> > >
const&, std::vector
>*) const
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/admission-controller.cc:643:45
(impalad+0x21cfb81)
#6
boost::detail::function::void_function_obj_invoker2,
std::allocator >, impala::TTopicDelta,
std::less,
std::allocator > >,
std::allocator, std::allocator > const, impala::TTopicDelta> > >
const&, std::vector
>*>::invoke(boost::detail::function::function_buffer&,
std::map,
std::allocator >, impala::TTopicDelta,
std::less,
std::allocator > >,
std::allocator, std::allocator > const, impala::TTopicDelta> > >
const&, std::vector
>*)
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/function/function_template.hpp:159:11
(impalad+0x21cf9cc)
#7 boost::function2, std::allocator >, impala::TTopicDelta,
std::less,
std::allocator > >,
std::allocator, std::allocator > const, impala::TTopicDelta> > >
const&, std::vector
>*>::operator()(std::map, std::allocator >, impala::TTopicDelta,
std::less,
std::allocator > >,
std::allocator, std::allocator > const, impala::TTopicDelta> > >
const&, std::vector
>*) const
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/function/function_template.hpp:770:14
(impalad+0x23fc400)
#8
impala::StatestoreSubscriber::UpdateState(std::map, std::allocator >, impala::TTopicDelta,
std::less,
std::allocator > >,
std::allocator, std::allocator > const, impala::TTopicDelta> > >
const&, impala::TUniqueId const&, std::vector >*, bool*)
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/statestore/statestore-subscriber.cc:471:7
(impalad+0x23f9339)
#9
impala::StatestoreSubscriberThriftIf::UpdateState(impala::TUpdateStateResponse&,
impala::TUpdateStateRequest const&)
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/statestore/statestore-subscriber.cc:110:18
(impalad+0x23fc65f)
#10 impala::StatestoreSubscriberProcessor::process_UpdateState(int,
apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*,
void*)
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/generated-sources/gen-cpp/StatestoreSubscriber.cpp:543:13
(impalad+0x29b2fc4)
#11
impala::StatestoreSubscriberProcessor::dispatchCall(apache::thrift::protocol::TProtocol*,
apache::thrift::protocol::TProtocol*, std::__cxx11::basic_string, std::allocator > const&, int, void*)
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/generated-sources/gen-cpp/StatestoreSubscriber.cpp:516:3
(impalad+0x29b2da2)
#12
apache::thrift::TDispatchProcessor::process(boost::shared_ptr,
boost::shared_ptr, void*)
[jira] [Created] (IMPALA-10154) Data race on coord_backend_id
Sahil Takiar created IMPALA-10154:
-
Summary: Data race on coord_backend_id
Key: IMPALA-10154
URL: https://issues.apache.org/jira/browse/IMPALA-10154
Project: IMPALA
Issue Type: Bug
Reporter: Sahil Takiar
Assignee: Wenzhe Zhou
TSAN is reporting a data race on
{{ExecQueryFInstancesRequestPB#coord_backend_id}}
{code:java}
WARNING: ThreadSanitizer: data race (pid=15392)
Write of size 8 at 0x7b74001104a8 by thread T83 (mutexes: write
M871582266043729400):
#0 impala::ExecQueryFInstancesRequestPB::mutable_coord_backend_id()
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/generated-sources/gen-cpp/control_service.pb.h:6625:23
(impalad+0x20c03ed)
#1 impala::QueryState::Init(impala::ExecQueryFInstancesRequestPB const*,
impala::TExecPlanFragmentInfo const&)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:216:21
(impalad+0x20b8b29)
#2 impala::QueryExecMgr::StartQuery(impala::ExecQueryFInstancesRequestPB
const*, impala::TQueryCtx const&, impala::TExecPlanFragmentInfo const&)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-exec-mgr.cc:80:23
(impalad+0x20acb59)
#3
impala::ControlService::ExecQueryFInstances(impala::ExecQueryFInstancesRequestPB
const*, impala::ExecQueryFInstancesResponsePB*, kudu::rpc::RpcContext*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/control-service.cc:157:66
(impalad+0x22a621d)
#4
impala::ControlServiceIf::ControlServiceIf(scoped_refptr
const&, scoped_refptr
const&)::$_1::operator()(google::protobuf::Message const*,
google::protobuf::Message*, kudu::rpc::RpcContext*) const
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/generated-sources/gen-cpp/control_service.service.cc:70:13
(impalad+0x23622a4)
#5 std::_Function_handler
const&, scoped_refptr
const&)::$_1>::_M_invoke(std::_Any_data const&, google::protobuf::Message
const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:316:2
(impalad+0x23620ed)
#6 std::function::operator()(google::protobuf::Message const*,
google::protobuf::Message*, kudu::rpc::RpcContext*) const
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706:14
(impalad+0x2a4a453)
#7 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/kudu/rpc/service_if.cc:139:3
(impalad+0x2a49efe)
#8 impala::ImpalaServicePool::RunThread()
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/impala-service-pool.cc:272:15
(impalad+0x2011a12)
#9 boost::_mfi::mf0::operator()(impala::ImpalaServicePool*) const
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/mem_fn_template.hpp:49:29
(impalad+0x2017a16)
#10 void boost::_bi::list1
>::operator(),
boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:259:9
(impalad+0x201796a)
#11 boost::_bi::bind_t,
boost::_bi::list1 >
>::operator()()
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:1222:16
(impalad+0x20178f3)
#12
boost::detail::function::void_function_obj_invoker0,
boost::_bi::list1 > >,
void>::invoke(boost::detail::function::function_buffer&)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/function/function_template.hpp:159:11
(impalad+0x20176e9)
#13 boost::function0::operator()() const
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/function/function_template.hpp:770:14
(impalad+0x1f666f1)
#14 impala::Thread::SuperviseThread(std::__cxx11::basic_string, std::allocator > const&,
std::__cxx11::basic_string, std::allocator >
const&, boost::function, impala::ThreadDebugInfo const*,
impala::Promise*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3
(impalad+0x252644b)
#15 void
boost::_bi::list5, std::allocator > >,
boost::_bi::value,
std::allocator > >, boost::_bi::value >,
boost::_bi::value,
boost::_bi::value*>
>::operator(),
std::allocator > const&,
[jira] [Commented] (IMPALA-10154) Data race on coord_backend_id
[
https://issues.apache.org/jira/browse/IMPALA-10154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192308#comment-17192308
]
Sahil Takiar commented on IMPALA-10154:
---
This looks related to IMPALA-5746, so assigning to [~wzhou].
> Data race on coord_backend_id
> -
>
> Key: IMPALA-10154
> URL: https://issues.apache.org/jira/browse/IMPALA-10154
> Project: IMPALA
> Issue Type: Bug
>Reporter: Sahil Takiar
>Assignee: Wenzhe Zhou
>Priority: Major
>
> TSAN is reporting a data race on
> {{ExecQueryFInstancesRequestPB#coord_backend_id}}
> {code:java}
> WARNING: ThreadSanitizer: data race (pid=15392)
> Write of size 8 at 0x7b74001104a8 by thread T83 (mutexes: write
> M871582266043729400):
> #0 impala::ExecQueryFInstancesRequestPB::mutable_coord_backend_id()
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/generated-sources/gen-cpp/control_service.pb.h:6625:23
> (impalad+0x20c03ed)
> #1 impala::QueryState::Init(impala::ExecQueryFInstancesRequestPB const*,
> impala::TExecPlanFragmentInfo const&)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:216:21
> (impalad+0x20b8b29)
> #2 impala::QueryExecMgr::StartQuery(impala::ExecQueryFInstancesRequestPB
> const*, impala::TQueryCtx const&, impala::TExecPlanFragmentInfo const&)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-exec-mgr.cc:80:23
> (impalad+0x20acb59)
> #3
> impala::ControlService::ExecQueryFInstances(impala::ExecQueryFInstancesRequestPB
> const*, impala::ExecQueryFInstancesResponsePB*, kudu::rpc::RpcContext*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/control-service.cc:157:66
> (impalad+0x22a621d)
> #4
> impala::ControlServiceIf::ControlServiceIf(scoped_refptr
> const&, scoped_refptr
> const&)::$_1::operator()(google::protobuf::Message const*,
> google::protobuf::Message*, kudu::rpc::RpcContext*) const
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/generated-sources/gen-cpp/control_service.service.cc:70:13
> (impalad+0x23622a4)
> #5 std::_Function_handler google::protobuf::Message*, kudu::rpc::RpcContext*),
> impala::ControlServiceIf::ControlServiceIf(scoped_refptr
> const&, scoped_refptr
> const&)::$_1>::_M_invoke(std::_Any_data const&, google::protobuf::Message
> const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:316:2
> (impalad+0x23620ed)
> #6 std::function google::protobuf::Message*,
> kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*,
> google::protobuf::Message*, kudu::rpc::RpcContext*) const
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706:14
> (impalad+0x2a4a453)
> #7 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/kudu/rpc/service_if.cc:139:3
> (impalad+0x2a49efe)
> #8 impala::ImpalaServicePool::RunThread()
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/impala-service-pool.cc:272:15
> (impalad+0x2011a12)
> #9 boost::_mfi::mf0 impala::ImpalaServicePool>::operator()(impala::ImpalaServicePool*) const
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/mem_fn_template.hpp:49:29
> (impalad+0x2017a16)
> #10 void boost::_bi::list1
> >::operator(),
> boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0 impala::ImpalaServicePool>&, boost::_bi::list0&, int)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:259:9
> (impalad+0x201796a)
> #11 boost::_bi::bind_t impala::ImpalaServicePool>,
> boost::_bi::list1 >
> >::operator()()
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:1222:16
> (impalad+0x20178f3)
> #12
> boost::detail::function::void_function_obj_invoker0 boost::_mfi::mf0,
> boost::_bi::list1 > >,
> void>::invoke(boost::detail::function::function_buffer&)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/function/function_template.hpp:159:11
> (impalad+0x20176e9)
> #13 boost::function0::operator()() const
>
[jira] [Commented] (IMPALA-10073) Create shaded dependency for S3A and aws-java-sdk-bundle
[ https://issues.apache.org/jira/browse/IMPALA-10073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192259#comment-17192259 ] Sahil Takiar commented on IMPALA-10073: --- [https://github.com/apache/impala/blob/master/shaded-deps/s3a-aws-sdk/pom.xml#L58] contains the full list > Create shaded dependency for S3A and aws-java-sdk-bundle > > > Key: IMPALA-10073 > URL: https://issues.apache.org/jira/browse/IMPALA-10073 > Project: IMPALA > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Fix For: Impala 4.0 > > > One of the largest dependencies in Impala Docker containers is the > aws-java-sdk-bundle jar. One way to decrease the size of this dependency is > to apply a similar technique used for the hive-exec shaded jar: > [https://github.com/apache/impala/blob/master/shaded-deps/pom.xml] > The aws-java-sdk-bundle contains SDKs for all AWS services, even though > Impala-S3A only requires a few of the more basic SDKs. > IMPALA-10028 and HADOOP-17197 both discuss this a bit as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10073) Create shaded dependency for S3A and aws-java-sdk-bundle
[ https://issues.apache.org/jira/browse/IMPALA-10073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192126#comment-17192126 ] Steve Loughran commented on IMPALA-10073: - Which bits of the AWS SDK have you cut? > Create shaded dependency for S3A and aws-java-sdk-bundle > > > Key: IMPALA-10073 > URL: https://issues.apache.org/jira/browse/IMPALA-10073 > Project: IMPALA > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Fix For: Impala 4.0 > > > One of the largest dependencies in Impala Docker containers is the > aws-java-sdk-bundle jar. One way to decrease the size of this dependency is > to apply a similar technique used for the hive-exec shaded jar: > [https://github.com/apache/impala/blob/master/shaded-deps/pom.xml] > The aws-java-sdk-bundle contains SDKs for all AWS services, even though > Impala-S3A only requires a few of the more basic SDKs. > IMPALA-10028 and HADOOP-17197 both discuss this a bit as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7658) Proper codegen for HiveUdfCall
[ https://issues.apache.org/jira/browse/IMPALA-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192105#comment-17192105 ] ASF subversion and git services commented on IMPALA-7658: - Commit dc08b657e86c3bd81af27c7acf4a31b39b5b20e9 in impala's branch refs/heads/master from Daniel Becker [ https://gitbox.apache.org/repos/asf?p=impala.git;h=dc08b65 ] IMPALA-7658: Proper codegen for HiveUdfCall Implementing codegen for HiveUdfCall. Testing: Verified that java udf tests pass locally. Benchmarks: Used a UDF from TestUdf.java that adds three integers: create function tpch15_parquet.sum3(int, int, int) returns int location '/test-warehouse/impala-hive-udfs.jar' symbol='org.apache.impala.TestUdf'; Used the following query on the master branch and the change's branch: set num_nodes=1; set mt_dop=1; select min(tpch15_parquet.sum3(cast(l_orderkey as int), cast(l_partkey as int), cast(l_suppkey as int))) from tpch15_parquet.lineitem; Results averaged over 100 runs after warmup: Master: 20.6346s, stddev: 0.3132411856765332 This change: 19.0256s, stddev: 0.42039019873436 This is a ~7.8% improvement. Change-Id: I2f994dac550f297ed3c88491816403f237d4d747 Reviewed-on: http://gerrit.cloudera.org:8080/16314 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Proper codegen for HiveUdfCall > -- > > Key: IMPALA-7658 > URL: https://issues.apache.org/jira/browse/IMPALA-7658 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Tim Armstrong >Assignee: Daniel Becker >Priority: Major > Labels: codegen, performance > > This function uses GetCodegendComputeFnWrapper() to call the interpreted path > but instead we could codegen the Evaluate() function to reduce the overhead. > I think this is likely to be a little involved since there's a loop to > unroll, so the solution might end up looking like IMPALA-5168 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10012) ds_hll_sketch() results ascii codec decoding error
[
https://issues.apache.org/jira/browse/IMPALA-10012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192106#comment-17192106
]
ASF subversion and git services commented on IMPALA-10012:
--
Commit 75146c9138051758176d3d4482f45a9d1056127b in impala's branch
refs/heads/master from Adam Tamas
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=75146c9 ]
IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
While the ds_hll_sketch() generates a string value as output the data
is not an ascii encoded text but a bitsketch, because of this, when
the shell get this data it disconnect while it tries to decode it.
The issue can be reproduced with a simple method like using unhex
with a wrong input.
Example: SELECT unhex("aa");
This patch contains a solution, where we replace any not UTF-8
decodable characters if we run into an UnicodeDecodeError after
fetching it.
This solution is working with the Thrift 0.9.3 autogenerated gen-py
but still fails with Thrift 0.11.0.
For Thrift 0.11.0 the error is catched and an error message is sent
(not working with beeswax protocol, because it generates a different
error (TypeError) which can come for other reasons too).
Testing:
-manual testing with these protocols: 'hs2-http', 'hs2', 'beeswax'
Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Reviewed-on: http://gerrit.cloudera.org:8080/16305
Reviewed-by: Csaba Ringhofer
Tested-by: Csaba Ringhofer
> ds_hll_sketch() results ascii codec decoding error
> --
>
> Key: IMPALA-10012
> URL: https://issues.apache.org/jira/browse/IMPALA-10012
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: Gabor Kaszab
>Assignee: Adam Tamas
>Priority: Major
>
> {code:java}
> select ds_hll_sketch(l_orderkey) from tpch_parquet.lineitem;
> Unknown Exception : 'ascii' codec can't decode byte 0x86 in position 20:
> ordinal not in range(128)
> Traceback (most recent call last):
> File "/home/gaborkaszab/shadow/Impala-upstream/shell/impala_shell.py", line
> 1183, in _execute_stmt
> for rows in rows_fetched:
> File "/home/gaborkaszab/shadow/Impala-upstream/shell/impala_client.py",
> line 1098, in fetch
> yield [row.split('\t') for row in result.data]
> UnicodeDecodeError: 'ascii' codec can't decode byte 0x86 in position 20:
> ordinal not in range(128)
> [Not connected] >
> {code}
> Even though there is no point of calling ds_hll_sketch() on its own, we
> should still make this foolproof and prevent Impala from crashing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9741) Support query iceberg table by impala
[
https://issues.apache.org/jira/browse/IMPALA-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192109#comment-17192109
]
ASF subversion and git services commented on IMPALA-9741:
-
Commit fb6d96e001c1a04475a8fd01f757dd0605cf3279 in impala's branch
refs/heads/master from skyyws
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=fb6d96e ]
IMPALA-9741: Support querying Iceberg table by impala
This patch mainly realizes the querying of iceberg table through impala,
we can use the following sql to create an external iceberg table:
CREATE EXTERNAL TABLE default.iceberg_test (
level string,
event_time timestamp,
message string,
)
STORED AS ICEBERG
LOCATION 'hdfs://xxx'
TBLPROPERTIES ('iceberg_file_format'='parquet');
Or just including table name and location like this:
CREATE EXTERNAL TABLE default.iceberg_test
STORED AS ICEBERG
LOCATION 'hdfs://xxx'
TBLPROPERTIES ('iceberg_file_format'='parquet');
'iceberg_file_format' is the file format in iceberg, currently only
support PARQUET, other format would be supported in the future. And
if you don't specify this property in your SQL, default file format
is PARQUET.
We achieved this function by treating the iceberg table as normal
unpartitioned hdfs table. When querying iceberg table, we pushdown
partition column predicates to iceberg to decide which data files
need to be scanned, and then transfer this information to BE to
do the real scan operation.
Testing:
- Unit test for Iceberg in FileMetadataLoaderTest
- Create table tests in functional_schema_template.sql
- Iceberg table query test in test_scanners.py
Change-Id: I856cfee4f3397d1a89cf17650e8d4fbfe1f2b006
Reviewed-on: http://gerrit.cloudera.org:8080/16143
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Support query iceberg table by impala
> -
>
> Key: IMPALA-9741
> URL: https://issues.apache.org/jira/browse/IMPALA-9741
> Project: IMPALA
> Issue Type: Sub-task
>Reporter: WangSheng
>Assignee: WangSheng
>Priority: Major
> Labels: impala-iceberg
> Attachments: select-iceberg.jpg
>
>
> Since we have submit an patch of supporting create iceberg table by impala in
> IMPALA-9688, we are preparing to implement iceberg table query by impala. But
> we need to read the impala and iceberg code deeply to determine how to do
> this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10012) ds_hll_sketch() results ascii codec decoding error
[
https://issues.apache.org/jira/browse/IMPALA-10012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192107#comment-17192107
]
ASF subversion and git services commented on IMPALA-10012:
--
Commit b7965d8240f3a5cd6972351b7fe1116e4a0835c1 in impala's branch
refs/heads/master from Csaba Ringhofer
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b7965d8 ]
Revert "IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix"
This reverts commit 75146c9138051758176d3d4482f45a9d1056127b.
Change-Id: I57f790389a8c847877999d2b9b8185939b416c07
Reviewed-on: http://gerrit.cloudera.org:8080/16417
Reviewed-by: Csaba Ringhofer
Tested-by: Csaba Ringhofer
> ds_hll_sketch() results ascii codec decoding error
> --
>
> Key: IMPALA-10012
> URL: https://issues.apache.org/jira/browse/IMPALA-10012
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: Gabor Kaszab
>Assignee: Adam Tamas
>Priority: Major
>
> {code:java}
> select ds_hll_sketch(l_orderkey) from tpch_parquet.lineitem;
> Unknown Exception : 'ascii' codec can't decode byte 0x86 in position 20:
> ordinal not in range(128)
> Traceback (most recent call last):
> File "/home/gaborkaszab/shadow/Impala-upstream/shell/impala_shell.py", line
> 1183, in _execute_stmt
> for rows in rows_fetched:
> File "/home/gaborkaszab/shadow/Impala-upstream/shell/impala_client.py",
> line 1098, in fetch
> yield [row.split('\t') for row in result.data]
> UnicodeDecodeError: 'ascii' codec can't decode byte 0x86 in position 20:
> ordinal not in range(128)
> [Not connected] >
> {code}
> Even though there is no point of calling ds_hll_sketch() on its own, we
> should still make this foolproof and prevent Impala from crashing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10012) ds_hll_sketch() results ascii codec decoding error
[
https://issues.apache.org/jira/browse/IMPALA-10012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192108#comment-17192108
]
ASF subversion and git services commented on IMPALA-10012:
--
Commit fe6e6257475b521b3ceeae4f6a898aae6ce3eb94 in impala's branch
refs/heads/master from Adam Tamas
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=fe6e625 ]
IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
While the ds_hll_sketch() generates a string value as output the data
is not an ascii encoded text but a bitsketch, because of this, when
the shell get this data it disconnect while it tries to decode it.
The issue can be reproduced with a simple method like using unhex
with a wrong input.
Example: SELECT unhex("aa");
This patch contains a solution, where we replace any not UTF-8
decodable characters if we run into an UnicodeDecodeError after
fetching it.
This solution is working with the Thrift 0.9.3 autogenerated gen-py
but still fails with Thrift 0.11.0.
For Thrift 0.11.0 the error is catched and an error message is sent
(not working with beeswax protocol, because it generates a different
error (TypeError) which can come for other reasons too).
Testing:
-manual testing with these protocols: 'hs2-http', 'hs2', 'beeswax'
Change-Id: I0c5f1290356e21aed8ca7f896f953541942aed05
Reviewed-on: http://gerrit.cloudera.org:8080/16418
Tested-by: Impala Public Jenkins
Reviewed-by: Gabor Kaszab
> ds_hll_sketch() results ascii codec decoding error
> --
>
> Key: IMPALA-10012
> URL: https://issues.apache.org/jira/browse/IMPALA-10012
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: Gabor Kaszab
>Assignee: Adam Tamas
>Priority: Major
>
> {code:java}
> select ds_hll_sketch(l_orderkey) from tpch_parquet.lineitem;
> Unknown Exception : 'ascii' codec can't decode byte 0x86 in position 20:
> ordinal not in range(128)
> Traceback (most recent call last):
> File "/home/gaborkaszab/shadow/Impala-upstream/shell/impala_shell.py", line
> 1183, in _execute_stmt
> for rows in rows_fetched:
> File "/home/gaborkaszab/shadow/Impala-upstream/shell/impala_client.py",
> line 1098, in fetch
> yield [row.split('\t') for row in result.data]
> UnicodeDecodeError: 'ascii' codec can't decode byte 0x86 in position 20:
> ordinal not in range(128)
> [Not connected] >
> {code}
> Even though there is no point of calling ds_hll_sketch() on its own, we
> should still make this foolproof and prevent Impala from crashing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-10153) Support time travel for Iceberg tables
[ https://issues.apache.org/jira/browse/IMPALA-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltán Borók-Nagy updated IMPALA-10153: --- Issue Type: New Feature (was: Bug) > Support time travel for Iceberg tables > -- > > Key: IMPALA-10153 > URL: https://issues.apache.org/jira/browse/IMPALA-10153 > Project: IMPALA > Issue Type: New Feature >Reporter: Zoltán Borók-Nagy >Priority: Major > Labels: impala-iceberg > > Iceberg tables support snapshots/data versioning/time travel. > It means we can query an older version of the table. > Probably we'll need to extend Impala's SQL syntax to support such queries > (Hive will also support such queries, so we should use the same syntax). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10153) Support time travel for Iceberg tables
Zoltán Borók-Nagy created IMPALA-10153: -- Summary: Support time travel for Iceberg tables Key: IMPALA-10153 URL: https://issues.apache.org/jira/browse/IMPALA-10153 Project: IMPALA Issue Type: Bug Reporter: Zoltán Borók-Nagy Iceberg tables support snapshots/data versioning/time travel. It means we can query an older version of the table. Probably we'll need to extend Impala's SQL syntax to support such queries (Hive will also support such queries, so we should use the same syntax). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10152) Add support for Iceberg HiveCatalog
Zoltán Borók-Nagy created IMPALA-10152: -- Summary: Add support for Iceberg HiveCatalog Key: IMPALA-10152 URL: https://issues.apache.org/jira/browse/IMPALA-10152 Project: IMPALA Issue Type: Sub-task Reporter: Zoltán Borók-Nagy HadoopTables and HadoopCatalog only work on filesystems that support atomic rename. Therefore they are not safe on S3. HiveCatalog uses the Hive Metastore to keep track of Iceberg tables, so we should use this interface when we store our tables in object stores. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-10151) Upgrade Iceberg to a version that is compatible with Hive3
[ https://issues.apache.org/jira/browse/IMPALA-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltán Borók-Nagy updated IMPALA-10151: --- Labels: impala-iceberg (was: ) > Upgrade Iceberg to a version that is compatible with Hive3 > -- > > Key: IMPALA-10151 > URL: https://issues.apache.org/jira/browse/IMPALA-10151 > Project: IMPALA > Issue Type: Sub-task >Reporter: Zoltán Borók-Nagy >Priority: Major > Labels: impala-iceberg > > To support Iceberg on S3, we'll need to use HiveCatalog. The current Iceberg > jars only compatible with Hive2, but Impala uses Hive3. So we'll need an > Iceberg version that is compatible with Hive3 as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10151) Upgrade Iceberg to a version that is compatible with Hive3
Zoltán Borók-Nagy created IMPALA-10151: -- Summary: Upgrade Iceberg to a version that is compatible with Hive3 Key: IMPALA-10151 URL: https://issues.apache.org/jira/browse/IMPALA-10151 Project: IMPALA Issue Type: Sub-task Reporter: Zoltán Borók-Nagy To support Iceberg on S3, we'll need to use HiveCatalog. The current Iceberg jars only compatible with Hive2, but Impala uses Hive3. So we'll need an Iceberg version that is compatible with Hive3 as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work stopped] (IMPALA-2792) Syntactic sugar for computing aggregates over nested collections.
[
https://issues.apache.org/jira/browse/IMPALA-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-2792 stopped by Tamas Mate.
--
> Syntactic sugar for computing aggregates over nested collections.
> -
>
> Key: IMPALA-2792
> URL: https://issues.apache.org/jira/browse/IMPALA-2792
> Project: IMPALA
> Issue Type: New Feature
> Components: Frontend
>Affects Versions: Impala 2.3.0
>Reporter: Alexander Behm
>Assignee: Tamas Mate
>Priority: Major
> Labels: complextype, nested_types, planner, ramp-up, usability
>
> For user convenience and SQL brevity, we should add syntax extensions to
> concisely express aggregates over nested collections. Internally, we should
> re-write the concise versions into the more verbose equivalent with a
> correlated inline view.
> Example A:
> {code}
> New syntax:
> select count(c.orders) from customer c
> Internally rewrite to:
> select cnt from customer c, (select count(*) from c.orders) v
> {code}
> Example B:
> {code}
> New syntax:
> select avg(c.orders.items.price) from customer c
> Internally rewrite to:
> select a from customer c, (select avg(price) from c.orders.items) v
> {code}
> I suggest performing the rewrite inside StmtRewriter.java after rewriting all
> subqueries from the WHERE clause.
> Similar syntactic improvements should be considered for analytic functions on
> nested collections.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
