[jira] [Updated] (HIVE-25327) Mapjoins in HiveServer2 fail when jmxremote is used
[
https://issues.apache.org/jira/browse/HIVE-25327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunhui Yang updated HIVE-25327:
Description:
我也遇到了同样的问题，虽然这个问题在之前的版本中已经关闭了，跟踪当前版本的源代码，也有修正，但问题仍然存在
以下为上一期HIVE-11369的主要内容：
将 hive.auto.convert.join 设置为 true 在 CLI 中可以正常工作，但在启动时将 jmx 选项传递给服务时，在
HiveServer2 中失败。这（hive-env.sh）足以使其失败：
{noformat}
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.port=8009
{noformat}
一旦我删除该行，它就可以正常工作。我*没有*想法...
下面是来自服务的日志：Here's the log from the service：
{noformat}
2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]: ql.Driver
(SessionState.java:printInfo(912)) - Query ID =
hive_20150724171919_aaa88a89-dc6d-490b-821c-4eec6d4c0421
2015-07-24 17:19:27,457 INFO [HiveServer2-Handler-Pool: Thread-22]: ql.Driver
(SessionState.java:printInfo(912)) - Total jobs = 1
2015-07-24 17:19:27,465 INFO [HiveServer2-Handler-Pool: Thread-22]: ql.Driver
(Driver.java:launchTask(1638)) - Starting task [Stage-4:MAPREDLOCAL] in serial
mode
2015-07-24 17:19:27,467 INFO [HiveServer2-Handler-Pool: Thread-22]:
mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(159)) - Generating
plan file
file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
2015-07-24 17:19:27,625 WARN [HiveServer2-Handler-Pool: Thread-22]:
conf.HiveConf (HiveConf.java:initialize(2620)) - HiveConf of name
hive.files.umask.value does not exist
2015-07-24 17:19:27,708 INFO [HiveServer2-Handler-Pool: Thread-22]:
mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(288)) - Executing:
/usr/lib/hadoop/bin/hadoop jar /usr/lib/hive/lib/hive-common-1.1.0-cdh5.4.3.jar
org.apache.hadoop.hive.ql.exec.mr.ExecDriver -localtask -plan
file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10005/plan.xml
-jobconffile
file:/tmp/hive/8932c206-5420-4b6f-9f1f-5f1706f30df8/hive_2015-07-24_17-19-26_552_5082133674120283907-1/-local-10006/jobconf.xml
2015-07-24 17:19:28,499 ERROR [HiveServer2-Handler-Pool: Thread-22]: exec.Task
(SessionState.java:printError(921)) - Execution failed with exit status: 1
2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]: exec.Task
(SessionState.java:printError(921)) - Obtaining error information
2015-07-24 17:19:28,500 ERROR [HiveServer2-Handler-Pool: Thread-22]: exec.Task
(SessionState.java:printError(921)) -
Task failed!
Task ID:
Stage-4
Logs:
2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]: exec.Task
(SessionState.java:printError(921)) - /tmp/hiveserver2_manual/hive-server2.log
2015-07-24 17:19:28,501 ERROR [HiveServer2-Handler-Pool: Thread-22]:
mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(308)) - Execution
failed with exit status: 1
2015-07-24 17:19:28,518 ERROR [HiveServer2-Handler-Pool: Thread-22]: ql.Driver
(SessionState.java:printError(921)) - FAILED: Execution Error, return code 1
from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
2015-07-24 17:19:28,599 WARN [HiveServer2-Handler-Pool: Thread-22]:
security.UserGroupInformation (UserGroupInformation.java:doAs(1674)) -
PriviledgedActionException as:hive (auth:SIMPLE)
cause:org.apache.hive.service.cli.HiveSQLException: Error while processing
statement: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
2015-07-24 17:19:28,600 WARN [HiveServer2-Handler-Pool: Thread-22]:
thrift.ThriftCLIService (ThriftCLIService.java:ExecuteStatement(496)) - Error
executing statement:
org.apache.hive.service.cli.HiveSQLException: Error while processing statement:
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
at
org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:315)
at
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:146)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:173)
at
org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:398)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveSessionImpl.java:379)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
at
[jira] [Updated] (HIVE-3376) Local mode errors point to the wrong file
[ https://issues.apache.org/jira/browse/HIVE-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunhui Yang updated HIVE-3376: --- Description: 这是由于本地模式下的 udf 错误而失败的查询的经典示例： 已结束作业 = job_local_0001出现错误 作业期间出错，正在获取调试信息... 执行失败，退出状态：2 获取错误信息 任务失败！ 任务ID： Stage-1 原木： /tmp/gang/hive.log 失败：执行错误，从 org.apache.hadoop.hive.ql.exec.MapRedTask 返回代码 2 但实际上在 /tmp/gang/gang_20120808181010_b716b5fb-5b8f-49d2-94e9-95bb8f465342 中本地 mr 工作.log was: 这是由于本地模式下的 udf 错误而失败的查询的经典示例： Ended Job = job_local_0001 with errors Error during job, obtaining debugging information... Execution failed with exit status: 2 Obtaining error information Task failed! Task ID: Stage-1 原木： /tmp/gang/hive.log FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask 但实际上在 /tmp/gang/gang_20120808181010_b716b5fb-5b8f-49d2-94e9-95bb8f465342 中本地 mr 工作.log > Local mode errors point to the wrong file > - > > Key: HIVE-3376 > URL: https://issues.apache.org/jira/browse/HIVE-3376 > Project: Hive > Issue Type: Bug > Components: Diagnosability, Logging >Reporter: Gang Tim Liu >Assignee: Gang Tim Liu >Priority: Major > Attachments: HIVE-3376.2.patch, hive-3371 > > > 这是由于本地模式下的 udf 错误而失败的查询的经典示例： > 已结束作业 = job_local_0001出现错误 > 作业期间出错，正在获取调试信息... > 执行失败，退出状态：2 > 获取错误信息 > 任务失败！ > 任务ID： > Stage-1 > 原木： > /tmp/gang/hive.log > 失败：执行错误，从 org.apache.hadoop.hive.ql.exec.MapRedTask 返回代码 2 > 但实际上在 /tmp/gang/gang_20120808181010_b716b5fb-5b8f-49d2-94e9-95bb8f465342 中本地 > mr 工作.log -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-3376) Local mode errors point to the wrong file
[ https://issues.apache.org/jira/browse/HIVE-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunhui Yang updated HIVE-3376: --- Description: 这是由于本地模式下的 udf 错误而失败的查询的经典示例： Ended Job = job_local_0001 with errors Error during job, obtaining debugging information... Execution failed with exit status: 2 Obtaining error information Task failed! Task ID: Stage-1 原木： /tmp/gang/hive.log FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask 但实际上在 /tmp/gang/gang_20120808181010_b716b5fb-5b8f-49d2-94e9-95bb8f465342 中本地 mr 工作.log was: This is a classic example of a query that fails due to a bad udf in local mode: Ended Job = job_local_0001 with errors Error during job, obtaining debugging information... Execution failed with exit status: 2 Obtaining error information Task failed! Task ID: Stage-1 Logs: /tmp/gang/hive.log FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask But actually local mr job in /tmp/gang/gang_20120808181010_b716b5fb-5b8f-49d2-94e9-95bb8f465342.log > Local mode errors point to the wrong file > - > > Key: HIVE-3376 > URL: https://issues.apache.org/jira/browse/HIVE-3376 > Project: Hive > Issue Type: Bug > Components: Diagnosability, Logging >Reporter: Gang Tim Liu >Assignee: Gang Tim Liu >Priority: Major > Attachments: HIVE-3376.2.patch, hive-3371 > > > 这是由于本地模式下的 udf 错误而失败的查询的经典示例： > Ended Job = job_local_0001 with errors > Error during job, obtaining debugging information... > Execution failed with exit status: 2 > Obtaining error information > Task failed! > Task ID: > Stage-1 > 原木： > /tmp/gang/hive.log > FAILED: Execution Error, return code 2 from > org.apache.hadoop.hive.ql.exec.MapRedTask > 但实际上在 /tmp/gang/gang_20120808181010_b716b5fb-5b8f-49d2-94e9-95bb8f465342 中本地 > mr 工作.log -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791864#comment-17791864
]
Chunhui Yang commented on HIVE-21660:
-
But I use Hive-2.3.9 and run with MapReduce not Tez,also face this question.Can
I edit GenTezUtils.java to fix this bug？
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
> Fix For: 4.0.0-alpha-1
>
> Attachments: HIVE-21660.1.patch, HIVE-21660.2.patch, HIVE-21660.patch
>
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t2( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-1626) stop using java.util.Stack
[ https://issues.apache.org/jira/browse/HIVE-1626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunhui Yang updated HIVE-1626: --- Description: 我们目前使用 Stack 作为通用节点遍历库的一部分。不应将 Stack 用于此目的，因为它从 Vector 继承会产生多余的同步开销。 大多数项目最终都会添加一个 ArrayStack 实现并改用它。 was: We currently use Stack as part of the generic node walking library. Stack should not be used for this since its inheritance from Vector incurs superfluous synchronization overhead. Most projects end up adding an ArrayStack implementation and using that instead. > stop using java.util.Stack > -- > > Key: HIVE-1626 > URL: https://issues.apache.org/jira/browse/HIVE-1626 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.7.0 >Reporter: John Sichi >Assignee: Teddy Choi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-1626.2.patch, HIVE-1626.2.patch, HIVE-1626.3.patch, > HIVE-1626.3.patch, HIVE-1626.3.patch > > Time Spent: 2h 40m > Remaining Estimate: 0h > > 我们目前使用 Stack 作为通用节点遍历库的一部分。不应将 Stack 用于此目的，因为它从 Vector 继承会产生多余的同步开销。 > 大多数项目最终都会添加一个 ArrayStack 实现并改用它。 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-12155) hive exited with status 5
[ https://issues.apache.org/jira/browse/HIVE-12155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17161024#comment-17161024 ] Chunhui Yang commented on HIVE-12155: - As of July 20, 2020, has no one solved this problem yet? > hive exited with status 5 > - > > Key: HIVE-12155 > URL: https://issues.apache.org/jira/browse/HIVE-12155 > Project: Hive > Issue Type: Bug > Components: CLI, Clients >Affects Versions: 1.2.1 > Environment: sqoop 1.4.5 & hadoop 2.6 & hive 1.2.1 >Reporter: Qiuzhuang Lian >Priority: Major > > We run sqoop-hive import job via RunJar to harness parallelisms runnings. > Sqoop hive import works very well but suddenly the sqoop-hive import job JVM > exits with "Hive exited with status 5" error during hive import phrase which > invokes HIVE CLI via java Process. Futhermore, we can't find any related > hive logs under /tmp/hive/hive_*.log. The error blocks all futher sqoop > import jobs. As a result, we have to restart system and it works well again. > The log detail is as follows, > Encountered IOException running import job: java.io.IOException: Hive exited > with status 5 > at > org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:385) > at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:335) > at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:239) > at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:511) -- This message was sent by Atlassian Jira (v8.3.4#803005)
