[jira] [Commented] (HIVE-15570) LLAP: Exception in HostAffinitySplitLocationProvider when running in container mode
[
https://issues.apache.org/jira/browse/HIVE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877676#comment-15877676
]
Lefty Leverenz commented on HIVE-15570:
---
Doc note: The new description and behavior of
*hive.llap.client.consistent.splits* need to be documented in the wiki for
release 2.2.0:
* [Configuration Properties -- hive.llap.client.consistent.splits |
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.llap.client.consistent.splits]
Added a TODOC2.2 label.
(By the way, the parameter description should have included newlines (\n) as
shown for *hive.llap.validate.acls* right after it, to avoid overlong lines in
the generated template file hive-default.xml.template.)
> LLAP: Exception in HostAffinitySplitLocationProvider when running in
> container mode
> ---
>
> Key: HIVE-15570
> URL: https://issues.apache.org/jira/browse/HIVE-15570
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Rajesh Balamohan
>Assignee: Zhiyuan Yang
>Priority: Minor
> Labels: TODOC2.2
> Fix For: 2.2.0
>
> Attachments: HIVE-15570.1.patch, HIVE-15570.2.patch,
> HIVE-15570.3.patch
>
>
> Sometimes user might prefer to run with "hive.execution.mode=container" mode
> when LLAP is stopped. If hive config for LLAP had
> "hive.llap.client.consistent.splits=true" in client side, it would end up
> throwing the following exception in {{Utils.java}}.
> {noformat}
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
> ... 25 more
> Caused by: java.lang.IllegalStateException:
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider needs at
> least 1 location to function
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider.(HostAffinitySplitLocationProvider.java:52)
> at
> org.apache.hadoop.hive.ql.exec.tez.Utils.getSplitLocationProvider(Utils.java:54)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.(HiveSplitGenerator.java:121)
> ... 30 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15570) LLAP: Exception in HostAffinitySplitLocationProvider when running in container mode
[
https://issues.apache.org/jira/browse/HIVE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877601#comment-15877601
]
Siddharth Seth commented on HIVE-15570:
---
Test failures are unrelated. Committing. Thanks [~aplusplus]
> LLAP: Exception in HostAffinitySplitLocationProvider when running in
> container mode
> ---
>
> Key: HIVE-15570
> URL: https://issues.apache.org/jira/browse/HIVE-15570
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Rajesh Balamohan
>Assignee: Zhiyuan Yang
>Priority: Minor
> Attachments: HIVE-15570.1.patch, HIVE-15570.2.patch,
> HIVE-15570.3.patch
>
>
> Sometimes user might prefer to run with "hive.execution.mode=container" mode
> when LLAP is stopped. If hive config for LLAP had
> "hive.llap.client.consistent.splits=true" in client side, it would end up
> throwing the following exception in {{Utils.java}}.
> {noformat}
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
> ... 25 more
> Caused by: java.lang.IllegalStateException:
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider needs at
> least 1 location to function
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider.(HostAffinitySplitLocationProvider.java:52)
> at
> org.apache.hadoop.hive.ql.exec.tez.Utils.getSplitLocationProvider(Utils.java:54)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.(HiveSplitGenerator.java:121)
> ... 30 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15570) LLAP: Exception in HostAffinitySplitLocationProvider when running in container mode
[
https://issues.apache.org/jira/browse/HIVE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876933#comment-15876933
]
Hive QA commented on HIVE-15570:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853219/HIVE-15570.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 10251 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3675/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3675/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3675/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853219 - PreCommit-HIVE-Build
> LLAP: Exception in HostAffinitySplitLocationProvider when running in
> container mode
> ---
>
> Key: HIVE-15570
> URL: https://issues.apache.org/jira/browse/HIVE-15570
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Rajesh Balamohan
>Assignee: Zhiyuan Yang
>Priority: Minor
> Attachments: HIVE-15570.1.patch, HIVE-15570.2.patch,
> HIVE-15570.3.patch
>
>
> Sometimes user might prefer to run with "hive.execution.mode=container" mode
> when LLAP is stopped. If hive config for LLAP had
> "hive.llap.client.consistent.splits=true" in client side, it would end up
> throwing the following exception in {{Utils.java}}.
> {noformat}
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
> ... 25 more
> Caused by: java.lang.IllegalStateException:
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider needs at
> least 1 location to function
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider.(HostAffinitySplitLocationProvider.java:52)
> at
> org.apache.hadoop.hive.ql.exec.tez.Utils.getSplitLocationProvider(Utils.java:54)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.(HiveSplitGenerator.java:121)
> ... 30 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15570) LLAP: Exception in HostAffinitySplitLocationProvider when running in container mode
[
https://issues.apache.org/jira/browse/HIVE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872639#comment-15872639
]
Sergey Shelukhin commented on HIVE-15570:
-
+1
> LLAP: Exception in HostAffinitySplitLocationProvider when running in
> container mode
> ---
>
> Key: HIVE-15570
> URL: https://issues.apache.org/jira/browse/HIVE-15570
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Rajesh Balamohan
>Assignee: Zhiyuan Yang
>Priority: Minor
> Attachments: HIVE-15570.1.patch, HIVE-15570.2.patch,
> HIVE-15570.3.patch
>
>
> Sometimes user might prefer to run with "hive.execution.mode=container" mode
> when LLAP is stopped. If hive config for LLAP had
> "hive.llap.client.consistent.splits=true" in client side, it would end up
> throwing the following exception in {{Utils.java}}.
> {noformat}
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
> ... 25 more
> Caused by: java.lang.IllegalStateException:
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider needs at
> least 1 location to function
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider.(HostAffinitySplitLocationProvider.java:52)
> at
> org.apache.hadoop.hive.ql.exec.tez.Utils.getSplitLocationProvider(Utils.java:54)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.(HiveSplitGenerator.java:121)
> ... 30 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15570) LLAP: Exception in HostAffinitySplitLocationProvider when running in container mode
[
https://issues.apache.org/jira/browse/HIVE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15843396#comment-15843396
]
Siddharth Seth commented on HIVE-15570:
---
Think it is ok to ignore the llap specific 'consistent split' setting when
running in container mode.
> LLAP: Exception in HostAffinitySplitLocationProvider when running in
> container mode
> ---
>
> Key: HIVE-15570
> URL: https://issues.apache.org/jira/browse/HIVE-15570
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Rajesh Balamohan
>Assignee: Zhiyuan Yang
>Priority: Minor
> Attachments: HIVE-15570.1.patch, HIVE-15570.2.patch
>
>
> Sometimes user might prefer to run with "hive.execution.mode=container" mode
> when LLAP is stopped. If hive config for LLAP had
> "hive.llap.client.consistent.splits=true" in client side, it would end up
> throwing the following exception in {{Utils.java}}.
> {noformat}
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
> ... 25 more
> Caused by: java.lang.IllegalStateException:
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider needs at
> least 1 location to function
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider.(HostAffinitySplitLocationProvider.java:52)
> at
> org.apache.hadoop.hive.ql.exec.tez.Utils.getSplitLocationProvider(Utils.java:54)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.(HiveSplitGenerator.java:121)
> ... 30 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15570) LLAP: Exception in HostAffinitySplitLocationProvider when running in container mode
[
https://issues.apache.org/jira/browse/HIVE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15843312#comment-15843312
]
Sergey Shelukhin commented on HIVE-15570:
-
Sounds good if the user intends the query to use LLAP.
I wonder if it should also fail if that's not the case? I.e. this shouldn't be
usable for a container query, since it's pointless.
> LLAP: Exception in HostAffinitySplitLocationProvider when running in
> container mode
> ---
>
> Key: HIVE-15570
> URL: https://issues.apache.org/jira/browse/HIVE-15570
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Rajesh Balamohan
>Assignee: Zhiyuan Yang
>Priority: Minor
> Attachments: HIVE-15570.1.patch, HIVE-15570.2.patch
>
>
> Sometimes user might prefer to run with "hive.execution.mode=container" mode
> when LLAP is stopped. If hive config for LLAP had
> "hive.llap.client.consistent.splits=true" in client side, it would end up
> throwing the following exception in {{Utils.java}}.
> {noformat}
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
> ... 25 more
> Caused by: java.lang.IllegalStateException:
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider needs at
> least 1 location to function
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider.(HostAffinitySplitLocationProvider.java:52)
> at
> org.apache.hadoop.hive.ql.exec.tez.Utils.getSplitLocationProvider(Utils.java:54)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.(HiveSplitGenerator.java:121)
> ... 30 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15570) LLAP: Exception in HostAffinitySplitLocationProvider when running in container mode
[
https://issues.apache.org/jira/browse/HIVE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15842411#comment-15842411
]
Siddharth Seth commented on HIVE-15570:
---
I don't think we should fallback to using locations provided by the FS, if
daemons are not available. That will pollute the cache once the daemons come up
- for subsequent queries. Think it's better to fail in this case, with a
message saying that LLAP is not running.
There's no check in HS2 at the moment to see if LLAP is up before handing it
off to an AM for execution. Building that out would be useful. Once that is in
place, the SplitLocationProvider could wait for LLAP to come up (with a
timeout) before proceeding.
[~sershe] - thoughts?
> LLAP: Exception in HostAffinitySplitLocationProvider when running in
> container mode
> ---
>
> Key: HIVE-15570
> URL: https://issues.apache.org/jira/browse/HIVE-15570
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Rajesh Balamohan
>Assignee: Zhiyuan Yang
>Priority: Minor
> Attachments: HIVE-15570.1.patch, HIVE-15570.2.patch
>
>
> Sometimes user might prefer to run with "hive.execution.mode=container" mode
> when LLAP is stopped. If hive config for LLAP had
> "hive.llap.client.consistent.splits=true" in client side, it would end up
> throwing the following exception in {{Utils.java}}.
> {noformat}
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
> ... 25 more
> Caused by: java.lang.IllegalStateException:
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider needs at
> least 1 location to function
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider.(HostAffinitySplitLocationProvider.java:52)
> at
> org.apache.hadoop.hive.ql.exec.tez.Utils.getSplitLocationProvider(Utils.java:54)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.(HiveSplitGenerator.java:121)
> ... 30 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15570) LLAP: Exception in HostAffinitySplitLocationProvider when running in container mode
[
https://issues.apache.org/jira/browse/HIVE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15833051#comment-15833051
]
Hive QA commented on HIVE-15570:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12848665/HIVE-15570.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10974 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver.testCliDriver[cascade_dbdrop]
(batchId=226)
org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver.testCliDriver[generatehfiles_require_family_path]
(batchId=226)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part]
(batchId=149)
org.apache.hive.service.server.TestHS2HttpServer.testContextRootUrlRewrite
(batchId=186)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3106/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3106/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3106/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12848665 - PreCommit-HIVE-Build
> LLAP: Exception in HostAffinitySplitLocationProvider when running in
> container mode
> ---
>
> Key: HIVE-15570
> URL: https://issues.apache.org/jira/browse/HIVE-15570
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Rajesh Balamohan
>Assignee: Zhiyuan Yang
>Priority: Minor
> Attachments: HIVE-15570.1.patch, HIVE-15570.2.patch
>
>
> Sometimes user might prefer to run with "hive.execution.mode=container" mode
> when LLAP is stopped. If hive config for LLAP had
> "hive.llap.client.consistent.splits=true" in client side, it would end up
> throwing the following exception in {{Utils.java}}.
> {noformat}
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
> ... 25 more
> Caused by: java.lang.IllegalStateException:
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider needs at
> least 1 location to function
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider.(HostAffinitySplitLocationProvider.java:52)
> at
> org.apache.hadoop.hive.ql.exec.tez.Utils.getSplitLocationProvider(Utils.java:54)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.(HiveSplitGenerator.java:121)
> ... 30 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15570) LLAP: Exception in HostAffinitySplitLocationProvider when running in container mode
[
https://issues.apache.org/jira/browse/HIVE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816506#comment-15816506
]
Rajesh Balamohan commented on HIVE-15570:
-
When locations is empty, it would fall back to getDefaultSplitLocationProvider.
> LLAP: Exception in HostAffinitySplitLocationProvider when running in
> container mode
> ---
>
> Key: HIVE-15570
> URL: https://issues.apache.org/jira/browse/HIVE-15570
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-15570.1.patch
>
>
> Sometimes user might prefer to run with "hive.execution.mode=container" mode
> when LLAP is stopped. If hive config for LLAP had
> "hive.llap.client.consistent.splits=true" in client side, it would end up
> throwing the following exception in {{Utils.java}}.
> {noformat}
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
> ... 25 more
> Caused by: java.lang.IllegalStateException:
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider needs at
> least 1 location to function
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider.(HostAffinitySplitLocationProvider.java:52)
> at
> org.apache.hadoop.hive.ql.exec.tez.Utils.getSplitLocationProvider(Utils.java:54)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.(HiveSplitGenerator.java:121)
> ... 30 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15570) LLAP: Exception in HostAffinitySplitLocationProvider when running in container mode
[
https://issues.apache.org/jira/browse/HIVE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815724#comment-15815724
]
Hive QA commented on HIVE-15570:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12846610/HIVE-15570.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 10898 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=122)
[groupby3_map.q,union26.q,mapreduce1.q,mapjoin_addjar.q,bucket_map_join_spark1.q,udf_example_add.q,multi_insert_with_join.q,sample7.q,auto_join_nulls.q,ppd_outer_join4.q,load_dyn_part8.q,alter_merge_orc.q,sample6.q,bucket_map_join_1.q,auto_sortmerge_join_9.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=98)
[groupby_map_ppr.q,nullgroup4_multi_distinct.q,join_rc.q,union14.q,smb_mapjoin_12.q,vector_cast_constant.q,union_remove_4.q,auto_join11.q,load_dyn_part7.q,udaf_collect_set.q,vectorization_12.q,groupby_sort_skew_1.q,groupby_sort_skew_1_23.q,smb_mapjoin_25.q,skewjoinopt12.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[case_sensitivity]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input_testxpath]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_coalesce]
(batchId=75)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[offset_limit_ppd_optimizer]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part]
(batchId=148)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=97)
org.apache.hadoop.hive.ql.io.orc.TestNewInputOutputFormat.testNewOutputFormatWithCompression
(batchId=253)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2861/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2861/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2861/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12846610 - PreCommit-HIVE-Build
> LLAP: Exception in HostAffinitySplitLocationProvider when running in
> container mode
> ---
>
> Key: HIVE-15570
> URL: https://issues.apache.org/jira/browse/HIVE-15570
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-15570.1.patch
>
>
> Sometimes user might prefer to run with "hive.execution.mode=container" mode
> when LLAP is stopped. If hive config for LLAP had
> "hive.llap.client.consistent.splits=true" in client side, it would end up
> throwing the following exception in {{Utils.java}}.
> {noformat}
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
> ... 25 more
> Caused by: java.lang.IllegalStateException:
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider needs at
> least 1 location to function
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider.(HostAffinitySplitLocationProvider.java:52)
> at
> org.apache.hadoop.hive.ql.exec.tez.Utils.getSplitLocationProvider(Utils.java:54)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.(HiveSplitGenerator.java:121)
> ... 30 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15570) LLAP: Exception in HostAffinitySplitLocationProvider when running in container mode
[
https://issues.apache.org/jira/browse/HIVE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815687#comment-15815687
]
Sergey Shelukhin commented on HIVE-15570:
-
Wouldn't this cause the method to return null if locations are not present?
> LLAP: Exception in HostAffinitySplitLocationProvider when running in
> container mode
> ---
>
> Key: HIVE-15570
> URL: https://issues.apache.org/jira/browse/HIVE-15570
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-15570.1.patch
>
>
> Sometimes user might prefer to run with "hive.execution.mode=container" mode
> when LLAP is stopped. If hive config for LLAP had
> "hive.llap.client.consistent.splits=true" in client side, it would end up
> throwing the following exception in {{Utils.java}}.
> {noformat}
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
> ... 25 more
> Caused by: java.lang.IllegalStateException:
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider needs at
> least 1 location to function
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider.(HostAffinitySplitLocationProvider.java:52)
> at
> org.apache.hadoop.hive.ql.exec.tez.Utils.getSplitLocationProvider(Utils.java:54)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.(HiveSplitGenerator.java:121)
> ... 30 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
