[jira] [Commented] (HIVE-17809) Implement per pool trigger validation and move sessions across pools
[
https://issues.apache.org/jira/browse/HIVE-17809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248379#comment-16248379
]
Hive QA commented on HIVE-17809:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897158/HIVE-17809.6.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 11380 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeOnTezEdges (batchId=288)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7774/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7774/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7774/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897158 - PreCommit-HIVE-Build
> Implement per pool trigger validation and move sessions across pools
>
>
> Key: HIVE-17809
> URL: https://issues.apache.org/jira/browse/HIVE-17809
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-17809.1.patch, HIVE-17809.2.patch,
> HIVE-17809.3.patch, HIVE-17809.4.patch, HIVE-17809.5.patch, HIVE-17809.6.patch
>
>
> HIVE-17508 trigger validation is applied for all pools at once. This is
> follow up to implement trigger validation at per pool level.
> This should also implement resolution for multiple applicable actions, as per
> the RB discussion
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17856) MM tables - IOW is not ACID compliant
[
https://issues.apache.org/jira/browse/HIVE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Yeom updated HIVE-17856:
--
Attachment: HIVE-17856.8.patch
> MM tables - IOW is not ACID compliant
> -
>
> Key: HIVE-17856
> URL: https://issues.apache.org/jira/browse/HIVE-17856
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Steve Yeom
> Labels: mm-gap-1
> Attachments: HIVE-17856.1.patch, HIVE-17856.2.patch,
> HIVE-17856.3.patch, HIVE-17856.4.patch, HIVE-17856.5.patch,
> HIVE-17856.6.patch, HIVE-17856.7.patch, HIVE-17856.8.patch
>
>
> The following tests were removed from mm_all during "integration"... I should
> have never allowed such manner of intergration.
> MM logic should have been kept intact until ACID logic could catch up. Alas,
> here we are.
> {noformat}
> drop table iow0_mm;
> create table iow0_mm(key int) tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow0_mm select key from intermediate;
> insert into table iow0_mm select key + 1 from intermediate;
> select * from iow0_mm order by key;
> insert overwrite table iow0_mm select key + 2 from intermediate;
> select * from iow0_mm order by key;
> drop table iow0_mm;
> drop table iow1_mm;
> create table iow1_mm(key int) partitioned by (key2 int)
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow1_mm partition (key2)
> select key as k1, key from intermediate union all select key as k1, key from
> intermediate;
> insert into table iow1_mm partition (key2)
> select key + 1 as k1, key from intermediate union all select key as k1, key
> from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key from intermediate union all select key + 4 as k1,
> key from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key + 3 from intermediate union all select key + 2 as
> k1, key + 2 from intermediate;
> select * from iow1_mm order by key, key2;
> drop table iow1_mm;
> {noformat}
> {noformat}
> drop table simple_mm;
> create table simple_mm(key int) stored as orc tblproperties
> ("transactional"="true", "transactional_properties"="insert_only");
> insert into table simple_mm select key from intermediate;
> -insert overwrite table simple_mm select key from intermediate;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-15956) StackOverflowError when drop lots of partitions
[
https://issues.apache.org/jira/browse/HIVE-15956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248358#comment-16248358
]
Hive QA commented on HIVE-15956:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12856246/HIVE-15956.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7773/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7773/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7773/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-11-11 06:20:33.720
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-7773/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-11-11 06:20:33.723
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 67888cf HIVE-17995 Run checkstyle on standalone-metastore module
with proper configuration (Adam Szita via Alan Gates)
+ git clean -f -d
Removing ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommandsForMmTable.java
Removing ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommandsForOrcMmTable.java
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 67888cf HIVE-17995 Run checkstyle on standalone-metastore module
with proper configuration (Adam Szita via Alan Gates)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-11-11 06:20:36.072
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java: No
such file or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12856246 - PreCommit-HIVE-Build
> StackOverflowError when drop lots of partitions
> ---
>
> Key: HIVE-15956
> URL: https://issues.apache.org/jira/browse/HIVE-15956
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 1.3.0, 2.2.0
>Reporter: Niklaus Xiao
>Assignee: Niklaus Xiao
> Attachments: HIVE-15956.patch
>
>
> Repro steps:
> 1. Create partitioned table and add 1 partitions
> {code}
> create table test_partition(id int) partitioned by (dt int);
> alter table test_partition add partition(dt=1);
> alter table test_partition add partition(dt=3);
> alter table test_partition add partition(dt=4);
> ...
> alter table test_partition add partition(dt=1);
> {code}
> 2. Drop 9000 partitions:
> {code}
> alter table test_partition drop partition(dt<9000);
> {code}
> Step 2 will fail with StackOverflowError:
> {code}
> Exception in thread "pool-7-thread-161" java.lang.StackOverflowError
> at
> org.datanucleus.query.expression.ExpressionCompiler.isOperator(ExpressionCompiler.java:819)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:190)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:192)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:192)
> at
>
[jira] [Commented] (HIVE-17856) MM tables - IOW is not ACID compliant
[
https://issues.apache.org/jira/browse/HIVE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248356#comment-16248356
]
Hive QA commented on HIVE-17856:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897144/HIVE-17856.6.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 19 failed/errored test(s), 11384 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_all] (batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_loaddata] (batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_ppd_varchar]
(batchId=11)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_all]
(batchId=147)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dp_counter_mm]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.TestTxnCommandsForMmTable.testInsertOverwriteForPartitionedMmTable
(batchId=254)
org.apache.hadoop.hive.ql.TestTxnCommandsForMmTable.testInsertOverwriteWithDynamicPartition
(batchId=254)
org.apache.hadoop.hive.ql.TestTxnCommandsForOrcMmTable.testInsertOverwriteForPartitionedMmTable
(batchId=272)
org.apache.hadoop.hive.ql.TestTxnCommandsForOrcMmTable.testInsertOverwriteWithDynamicPartition
(batchId=272)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7772/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7772/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7772/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 19 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897144 - PreCommit-HIVE-Build
> MM tables - IOW is not ACID compliant
> -
>
> Key: HIVE-17856
> URL: https://issues.apache.org/jira/browse/HIVE-17856
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Steve Yeom
> Labels: mm-gap-1
> Attachments: HIVE-17856.1.patch, HIVE-17856.2.patch,
> HIVE-17856.3.patch, HIVE-17856.4.patch, HIVE-17856.5.patch,
> HIVE-17856.6.patch, HIVE-17856.7.patch
>
>
> The following tests were removed from mm_all during "integration"... I should
> have never allowed such manner of intergration.
> MM logic should have been kept intact until ACID logic could catch up. Alas,
> here we are.
> {noformat}
> drop table iow0_mm;
> create table iow0_mm(key int) tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow0_mm select key from intermediate;
> insert into table iow0_mm select key + 1 from intermediate;
> select * from iow0_mm order by key;
> insert overwrite table iow0_mm select key + 2 from intermediate;
> select * from iow0_mm order by key;
> drop table iow0_mm;
> drop table iow1_mm;
> create table iow1_mm(key int) partitioned by (key2 int)
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow1_mm partition (key2)
> select key as k1, key from intermediate union all select key as k1, key from
> intermediate;
> insert into table iow1_mm partition (key2)
> select key + 1 as k1, key from intermediate union all select key as k1, key
> from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key from intermediate union all select key + 4 as k1,
> key from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key +
[jira] [Updated] (HIVE-17856) MM tables - IOW is not ACID compliant
[
https://issues.apache.org/jira/browse/HIVE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Yeom updated HIVE-17856:
--
Attachment: HIVE-17856.7.patch
> MM tables - IOW is not ACID compliant
> -
>
> Key: HIVE-17856
> URL: https://issues.apache.org/jira/browse/HIVE-17856
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Steve Yeom
> Labels: mm-gap-1
> Attachments: HIVE-17856.1.patch, HIVE-17856.2.patch,
> HIVE-17856.3.patch, HIVE-17856.4.patch, HIVE-17856.5.patch,
> HIVE-17856.6.patch, HIVE-17856.7.patch
>
>
> The following tests were removed from mm_all during "integration"... I should
> have never allowed such manner of intergration.
> MM logic should have been kept intact until ACID logic could catch up. Alas,
> here we are.
> {noformat}
> drop table iow0_mm;
> create table iow0_mm(key int) tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow0_mm select key from intermediate;
> insert into table iow0_mm select key + 1 from intermediate;
> select * from iow0_mm order by key;
> insert overwrite table iow0_mm select key + 2 from intermediate;
> select * from iow0_mm order by key;
> drop table iow0_mm;
> drop table iow1_mm;
> create table iow1_mm(key int) partitioned by (key2 int)
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow1_mm partition (key2)
> select key as k1, key from intermediate union all select key as k1, key from
> intermediate;
> insert into table iow1_mm partition (key2)
> select key + 1 as k1, key from intermediate union all select key as k1, key
> from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key from intermediate union all select key + 4 as k1,
> key from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key + 3 from intermediate union all select key + 2 as
> k1, key + 2 from intermediate;
> select * from iow1_mm order by key, key2;
> drop table iow1_mm;
> {noformat}
> {noformat}
> drop table simple_mm;
> create table simple_mm(key int) stored as orc tblproperties
> ("transactional"="true", "transactional_properties"="insert_only");
> insert into table simple_mm select key from intermediate;
> -insert overwrite table simple_mm select key from intermediate;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17809) Implement per pool trigger validation and move sessions across pools
[
https://issues.apache.org/jira/browse/HIVE-17809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248334#comment-16248334
]
Hive QA commented on HIVE-17809:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897158/HIVE-17809.6.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 11380 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[setop_no_distinct]
(batchId=79)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testApplyPlanQpChanges
(batchId=281)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7771/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7771/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7771/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897158 - PreCommit-HIVE-Build
> Implement per pool trigger validation and move sessions across pools
>
>
> Key: HIVE-17809
> URL: https://issues.apache.org/jira/browse/HIVE-17809
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-17809.1.patch, HIVE-17809.2.patch,
> HIVE-17809.3.patch, HIVE-17809.4.patch, HIVE-17809.5.patch, HIVE-17809.6.patch
>
>
> HIVE-17508 trigger validation is applied for all pools at once. This is
> follow up to implement trigger validation at per pool level.
> This should also implement resolution for multiple applicable actions, as per
> the RB discussion
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HIVE-18012) fix ct_noperm_loc test
[
https://issues.apache.org/jira/browse/HIVE-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248325#comment-16248325
]
Akira Ajisaka edited comment on HIVE-18012 at 11/11/17 4:41 AM:
Here are two questions:
1. When runnning the test by {{mvn test -Dtest=TestNegativeCliDriver
-Dqfile=ct_noperm_loc.q}}, the test was skipped. I had to modify
testconfiguration.properties to run the test. Is it correct?
2. After modifying the testconfiguration.properties, the test fails when
creating {{hdfs:///tmp/ct_noperm_loc}}.
{noformat:title=ct_noperm_loc.q.out.orig}
Command -mkdir -p hdfs:///tmp/ct_noperm_loc failed with exit code = 1
Query returned non-zero code: 1, cause: null
{noformat}
Am I missing something?
was (Author: ajisakaa):
Here are two questions:
1. When runnning the test by {{mvn test -Dtest=TestNegativeCliDriver
-Dqfile=ct_noperm_loc.q}} the test was skipped. I had to modify
testconfiguration.properties to run the test. Is it correct?
2. After modifying the testconfiguration.properties, the test fails when
creating {{hdfs:///tmp/ct_noperm_loc}}.
{noformat:title=ct_noperm_loc.q.out.orig}
Command -mkdir -p hdfs:///tmp/ct_noperm_loc failed with exit code = 1
Query returned non-zero code: 1, cause: null
{noformat}
Am I missing something?
> fix ct_noperm_loc test
> --
>
> Key: HIVE-18012
> URL: https://issues.apache.org/jira/browse/HIVE-18012
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-18012.001.patch, HIVE-18012.02.patch
>
>
> the goal of the test is to check that hive doesn't let user1 to create a
> table with a location under an unowned path.
> I've bisected this test to be broken by
> 5250ef450430fcdeed0a2cb7a770f48647987cd3 (HIVE-12408).
> the original exception was (which have been by that sole masked line):
> {code}
> FAILED: HiveAccessControlException Permission denied: Principal [name=user1,
> type=USER] does not have following privileges for operation CREATETABLE
> [[OBJECT OWNERSHIP] on Object [type=DFS_URI,
> name=hdfs://localhost:35753/tmp/ct_noperm_loc_foo0]]
> {code}
> the current semanticexception shouldnt be accepted ; because it's unrelated
> to the tests goal.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18012) fix ct_noperm_loc test
[
https://issues.apache.org/jira/browse/HIVE-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248325#comment-16248325
]
Akira Ajisaka commented on HIVE-18012:
--
Here are two questions:
1. When runnning the test by {{mvn test -Dtest=TestNegativeCliDriver
-Dqfile=ct_noperm_loc.q}} the test was skipped. I had to modify
testconfiguration.properties to run the test. Is it correct?
2. After modifying the testconfiguration.properties, the test fails when
creating {{hdfs:///tmp/ct_noperm_loc}}.
{noformat:title=ct_noperm_loc.q.out.orig}
Command -mkdir -p hdfs:///tmp/ct_noperm_loc failed with exit code = 1
Query returned non-zero code: 1, cause: null
{noformat}
Am I missing something?
> fix ct_noperm_loc test
> --
>
> Key: HIVE-18012
> URL: https://issues.apache.org/jira/browse/HIVE-18012
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-18012.001.patch, HIVE-18012.02.patch
>
>
> the goal of the test is to check that hive doesn't let user1 to create a
> table with a location under an unowned path.
> I've bisected this test to be broken by
> 5250ef450430fcdeed0a2cb7a770f48647987cd3 (HIVE-12408).
> the original exception was (which have been by that sole masked line):
> {code}
> FAILED: HiveAccessControlException Permission denied: Principal [name=user1,
> type=USER] does not have following privileges for operation CREATETABLE
> [[OBJECT OWNERSHIP] on Object [type=DFS_URI,
> name=hdfs://localhost:35753/tmp/ct_noperm_loc_foo0]]
> {code}
> the current semanticexception shouldnt be accepted ; because it's unrelated
> to the tests goal.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17809) Implement per pool trigger validation and move sessions across pools
[
https://issues.apache.org/jira/browse/HIVE-17809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248308#comment-16248308
]
Hive QA commented on HIVE-17809:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897158/HIVE-17809.6.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 11380 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7770/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7770/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7770/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897158 - PreCommit-HIVE-Build
> Implement per pool trigger validation and move sessions across pools
>
>
> Key: HIVE-17809
> URL: https://issues.apache.org/jira/browse/HIVE-17809
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-17809.1.patch, HIVE-17809.2.patch,
> HIVE-17809.3.patch, HIVE-17809.4.patch, HIVE-17809.5.patch, HIVE-17809.6.patch
>
>
> HIVE-17508 trigger validation is applied for all pools at once. This is
> follow up to implement trigger validation at per pool level.
> This should also implement resolution for multiple applicable actions, as per
> the RB discussion
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17361) Support LOAD DATA for transactional tables
[ https://issues.apache.org/jira/browse/HIVE-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-17361: -- Status: Patch Available (was: Open) > Support LOAD DATA for transactional tables > -- > > Key: HIVE-17361 > URL: https://issues.apache.org/jira/browse/HIVE-17361 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Wei Zheng >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-17361.07.patch, HIVE-17361.1.patch, > HIVE-17361.2.patch, HIVE-17361.3.patch, HIVE-17361.4.patch > > > LOAD DATA was not supported since ACID was introduced. Need to fill this gap > between ACID table and regular hive table. > Current Documentation is under [DML > Operations|https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations] > and [Loading files into > tables|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Loadingfilesintotables]: > \\ > * Load Data performs very limited validations of the data, in particular it > uses the input file name which may not be in 0_0 which can break some > read logic. (Certainly will for Acid). > * It does not check the schema of the file. This may be a non issue for Acid > which requires ORC which is self describing so Schema Evolution may handle > this seamlessly. (Assuming Schema is not too different). > * It does check that _InputFormat_S are compatible. > * Bucketed (and thus sorted) tables don't support Load Data (but only if > hive.strict.checks.bucketing=true (default)). Will keep this restriction for > Acid. > * Load Data supports OVERWRITE clause > * What happens to file permissions/ownership: rename vs copy differences > \\ > The implementation will follow the same idea as in HIVE-14988 and use a > base_N/ dir for OVERWRITE clause. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17361) Support LOAD DATA for transactional tables
[ https://issues.apache.org/jira/browse/HIVE-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-17361: -- Attachment: HIVE-17361.07.patch HIVE-17361.07.patch - WIP > Support LOAD DATA for transactional tables > -- > > Key: HIVE-17361 > URL: https://issues.apache.org/jira/browse/HIVE-17361 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Wei Zheng >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-17361.07.patch, HIVE-17361.1.patch, > HIVE-17361.2.patch, HIVE-17361.3.patch, HIVE-17361.4.patch > > > LOAD DATA was not supported since ACID was introduced. Need to fill this gap > between ACID table and regular hive table. > Current Documentation is under [DML > Operations|https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations] > and [Loading files into > tables|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Loadingfilesintotables]: > \\ > * Load Data performs very limited validations of the data, in particular it > uses the input file name which may not be in 0_0 which can break some > read logic. (Certainly will for Acid). > * It does not check the schema of the file. This may be a non issue for Acid > which requires ORC which is self describing so Schema Evolution may handle > this seamlessly. (Assuming Schema is not too different). > * It does check that _InputFormat_S are compatible. > * Bucketed (and thus sorted) tables don't support Load Data (but only if > hive.strict.checks.bucketing=true (default)). Will keep this restriction for > Acid. > * Load Data supports OVERWRITE clause > * What happens to file permissions/ownership: rename vs copy differences > \\ > The implementation will follow the same idea as in HIVE-14988 and use a > base_N/ dir for OVERWRITE clause. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17871) Add non nullability flag to druid time column
[
https://issues.apache.org/jira/browse/HIVE-17871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248289#comment-16248289
]
Hive QA commented on HIVE-17871:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897126/HIVE-17871.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 11374 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7769/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7769/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7769/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897126 - PreCommit-HIVE-Build
> Add non nullability flag to druid time column
> -
>
> Key: HIVE-17871
> URL: https://issues.apache.org/jira/browse/HIVE-17871
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-17871.2.patch, HIVE-17871.patch
>
>
> Druid time column is non null all the time.
> Adding the non nullability flag will enable extra calcite goodness like
> transforming
> {code} select count(`__time`) from table {code} to {code} select count(*)
> from table {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18009) Multiple lateral view query is slow on hive on spark
[
https://issues.apache.org/jira/browse/HIVE-18009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248270#comment-16248270
]
Hive QA commented on HIVE-18009:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897115/HIVE-18009.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 11376 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeOnTezEdges (batchId=288)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testSyntheticComplexSchema[4]
(batchId=187)
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testTupleInBagInTupleInBag[3]
(batchId=187)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7768/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7768/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7768/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897115 - PreCommit-HIVE-Build
> Multiple lateral view query is slow on hive on spark
>
>
> Key: HIVE-18009
> URL: https://issues.apache.org/jira/browse/HIVE-18009
> Project: Hive
> Issue Type: Improvement
> Components: Spark
>Affects Versions: 3.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-18009.1.patch, HIVE-18009.2.patch,
> HIVE-18009.3.patch
>
>
> When running the query with multiple lateral view, HoS is busy with the
> compilation. GenSparkUtils has an efficient implementation of
> getChildOperator when we have diamond hierarchy in operator trees (lateral
> view in this case) since the node may be visited multiple times.
> {noformat}
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:442)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
>
[jira] [Updated] (HIVE-18002) add group support for pool mappings
[ https://issues.apache.org/jira/browse/HIVE-18002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18002: Attachment: HIVE-18002.01.patch > add group support for pool mappings > --- > > Key: HIVE-18002 > URL: https://issues.apache.org/jira/browse/HIVE-18002 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-18002.01.patch, HIVE-18002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17904) handle internal Tez AM restart in registry and WM
[ https://issues.apache.org/jira/browse/HIVE-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-17904: Attachment: HIVE-17904.03.patch Patch got dropped... > handle internal Tez AM restart in registry and WM > - > > Key: HIVE-17904 > URL: https://issues.apache.org/jira/browse/HIVE-17904 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-17904.01.patch, HIVE-17904.02.patch, > HIVE-17904.03.patch, HIVE-17904.patch, HIVE-17904.patch > > > After the plan update patch is committed. The current code doesn't account > very well for it; registry may have races, and an event needs to be added to > WM when some AM resets, at least to make sure we discard the update errors > that pertain to the old AM. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17906) use kill query mechanics to kill queries in WM
[ https://issues.apache.org/jira/browse/HIVE-17906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-17906: Attachment: HIVE-17906.04.patch > use kill query mechanics to kill queries in WM > -- > > Key: HIVE-17906 > URL: https://issues.apache.org/jira/browse/HIVE-17906 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-17906.01.patch, HIVE-17906.02.patch, > HIVE-17906.03.patch, HIVE-17906.03.patch, HIVE-17906.04.patch, > HIVE-17906.patch > > > Right now it just closes the session (see HIVE-17841). The sessions would > need to be reused after the kill, or closed after the kill if the total QP > has decreased -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-13567) Enable auto-gather column stats by default
[
https://issues.apache.org/jira/browse/HIVE-13567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248233#comment-16248233
]
Hive QA commented on HIVE-13567:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897098/HIVE-13567.23wip02.patch
{color:green}SUCCESS:{color} +1 due to 33 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 869 failed/errored test(s), 11352 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_joins]
(batchId=235)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown]
(batchId=235)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=235)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_single_sourced_multi_insert]
(batchId=235)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_11]
(batchId=243)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_12]
(batchId=243)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_13]
(batchId=243)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_7]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[explain]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_dynamic_partitions]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_table]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_directory]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_table]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[write_final_output_blobstore]
(batchId=246)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_insert_overwrite]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_vectorization]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alterColumnStatsPart]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alterColumnStats]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_numbuckets_partitioned_table2_h23]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_numbuckets_partitioned_table_h23]
(batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_coltype]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_add_partition]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_serde2]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[analyze_table_null_partition]
(batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_filter]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_3a]
(batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join14] (batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join17] (batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join19] (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join19_inclause]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join1] (batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join26] (batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join2] (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join3] (batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join4] (batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join5] (batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join6] (batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join7] (batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join8] (batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join9] (batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_reordering_values]
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_13]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=47)
[jira] [Updated] (HIVE-17809) Implement per pool trigger validation and move sessions across pools
[ https://issues.apache.org/jira/browse/HIVE-17809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-17809: - Attachment: HIVE-17809.6.patch Addressed review comments > Implement per pool trigger validation and move sessions across pools > > > Key: HIVE-17809 > URL: https://issues.apache.org/jira/browse/HIVE-17809 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-17809.1.patch, HIVE-17809.2.patch, > HIVE-17809.3.patch, HIVE-17809.4.patch, HIVE-17809.5.patch, HIVE-17809.6.patch > > > HIVE-17508 trigger validation is applied for all pools at once. This is > follow up to implement trigger validation at per pool level. > This should also implement resolution for multiple applicable actions, as per > the RB discussion -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18046) Metastore: default IS_REWRITE_ENABLED=false instead of NULL
[
https://issues.apache.org/jira/browse/HIVE-18046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248144#comment-16248144
]
Sergey Shelukhin commented on HIVE-18046:
-
[~jcamachorodriguez] can you take a look? probably an easy fix to sql scripts
> Metastore: default IS_REWRITE_ENABLED=false instead of NULL
> ---
>
> Key: HIVE-18046
> URL: https://issues.apache.org/jira/browse/HIVE-18046
> Project: Hive
> Issue Type: Bug
> Components: Materialized views, Metastore
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
>
> The materialized view impl breaks old metastore sql write access, by
> complaining that the new table creation does not set this column up.
> {code}
> `IS_REWRITE_ENABLED` bit(1) NOT NULL,
> {code}
> {{NOT NULL DEFAULT 0}} would allow old metastore direct sql compatibility
> (not thrift).
> {code}
> 2017-11-09T07:11:58,331 ERROR [HiveServer2-Background-Pool: Thread-2354]
> metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
> 10) with error: javax.jdo.JDODataStoreException: Insert of object
> "org.apache.hadoop.hive.metastore.model.MTable@249dbf1" using statement
> "INSERT INTO `TBLS`
> (`TBL_ID`,`CREATE_TIME`,`DB_ID`,`LAST_ACCESS_TIME`,`OWNER`,`RETENTION`,`SD_ID`,`TBL_NAME`,`TBL_TYPE`,`VIEW_EXPANDED_TEXT`,`VIEW_ORIGINAL_TEXT`)
> VALUES (?,?,?,?,?,?,?,?,?,?,?)" failed : Field 'IS_REWRITE_ENABLED' doesn't
> have a default value
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:720)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:740)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:1038)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18012) fix ct_noperm_loc test
[
https://issues.apache.org/jira/browse/HIVE-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248141#comment-16248141
]
Hive QA commented on HIVE-18012:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897063/HIVE-18012.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 11374 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testApplyPlanQpChanges
(batchId=281)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7766/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7766/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7766/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897063 - PreCommit-HIVE-Build
> fix ct_noperm_loc test
> --
>
> Key: HIVE-18012
> URL: https://issues.apache.org/jira/browse/HIVE-18012
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-18012.001.patch, HIVE-18012.02.patch
>
>
> the goal of the test is to check that hive doesn't let user1 to create a
> table with a location under an unowned path.
> I've bisected this test to be broken by
> 5250ef450430fcdeed0a2cb7a770f48647987cd3 (HIVE-12408).
> the original exception was (which have been by that sole masked line):
> {code}
> FAILED: HiveAccessControlException Permission denied: Principal [name=user1,
> type=USER] does not have following privileges for operation CREATETABLE
> [[OBJECT OWNERSHIP] on Object [type=DFS_URI,
> name=hdfs://localhost:35753/tmp/ct_noperm_loc_foo0]]
> {code}
> the current semanticexception shouldnt be accepted ; because it's unrelated
> to the tests goal.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18046) Metastore: default IS_REWRITE_ENABLED=false instead of NULL
[
https://issues.apache.org/jira/browse/HIVE-18046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin reassigned HIVE-18046:
---
Assignee: Jesus Camacho Rodriguez
> Metastore: default IS_REWRITE_ENABLED=false instead of NULL
> ---
>
> Key: HIVE-18046
> URL: https://issues.apache.org/jira/browse/HIVE-18046
> Project: Hive
> Issue Type: Bug
> Components: Materialized views, Metastore
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
>
> The materialized view impl breaks old metastore sql write access, by
> complaining that the new table creation does not set this column up.
> {code}
> `IS_REWRITE_ENABLED` bit(1) NOT NULL,
> {code}
> {{NOT NULL DEFAULT 0}} would allow old metastore direct sql compatibility
> (not thrift).
> {code}
> 2017-11-09T07:11:58,331 ERROR [HiveServer2-Background-Pool: Thread-2354]
> metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
> 10) with error: javax.jdo.JDODataStoreException: Insert of object
> "org.apache.hadoop.hive.metastore.model.MTable@249dbf1" using statement
> "INSERT INTO `TBLS`
> (`TBL_ID`,`CREATE_TIME`,`DB_ID`,`LAST_ACCESS_TIME`,`OWNER`,`RETENTION`,`SD_ID`,`TBL_NAME`,`TBL_TYPE`,`VIEW_EXPANDED_TEXT`,`VIEW_ORIGINAL_TEXT`)
> VALUES (?,?,?,?,?,?,?,?,?,?,?)" failed : Field 'IS_REWRITE_ENABLED' doesn't
> have a default value
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:720)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:740)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:1038)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18046) Metastore: default IS_REWRITE_ENABLED=false instead of NULL
[
https://issues.apache.org/jira/browse/HIVE-18046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248124#comment-16248124
]
Gopal V commented on HIVE-18046:
Current workaround is to go the metastore and do
{code}
mysql> ALTER TABLE TBLS ALTER IS_REWRITE_ENABLED SET DEFAULT 0;
{code}
> Metastore: default IS_REWRITE_ENABLED=false instead of NULL
> ---
>
> Key: HIVE-18046
> URL: https://issues.apache.org/jira/browse/HIVE-18046
> Project: Hive
> Issue Type: Bug
> Components: Materialized views, Metastore
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Priority: Minor
>
> The materialized view impl breaks old metastore sql write access, by
> complaining that the new table creation does not set this column up.
> {code}
> `IS_REWRITE_ENABLED` bit(1) NOT NULL,
> {code}
> {{NOT NULL DEFAULT 0}} would allow old metastore direct sql compatibility
> (not thrift).
> {code}
> 2017-11-09T07:11:58,331 ERROR [HiveServer2-Background-Pool: Thread-2354]
> metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
> 10) with error: javax.jdo.JDODataStoreException: Insert of object
> "org.apache.hadoop.hive.metastore.model.MTable@249dbf1" using statement
> "INSERT INTO `TBLS`
> (`TBL_ID`,`CREATE_TIME`,`DB_ID`,`LAST_ACCESS_TIME`,`OWNER`,`RETENTION`,`SD_ID`,`TBL_NAME`,`TBL_TYPE`,`VIEW_EXPANDED_TEXT`,`VIEW_ORIGINAL_TEXT`)
> VALUES (?,?,?,?,?,?,?,?,?,?,?)" failed : Field 'IS_REWRITE_ENABLED' doesn't
> have a default value
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:720)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:740)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:1038)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18046) Metastore: default IS_REWRITE_ENABLED=false instead of NULL
[
https://issues.apache.org/jira/browse/HIVE-18046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-18046:
---
Description:
The materialized view impl breaks old metastore sql write access, by
complaining that the new table creation does not set this column up.
{code}
`IS_REWRITE_ENABLED` bit(1) NOT NULL,
{code}
{{NOT NULL DEFAULT 0}} would allow old metastore direct sql compatibility (not
thrift).
{code}
2017-11-09T07:11:58,331 ERROR [HiveServer2-Background-Pool: Thread-2354]
metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
10) with error: javax.jdo.JDODataStoreException: Insert of object
"org.apache.hadoop.hive.metastore.model.MTable@249dbf1" using statement "INSERT
INTO `TBLS`
(`TBL_ID`,`CREATE_TIME`,`DB_ID`,`LAST_ACCESS_TIME`,`OWNER`,`RETENTION`,`SD_ID`,`TBL_NAME`,`TBL_TYPE`,`VIEW_EXPANDED_TEXT`,`VIEW_ORIGINAL_TEXT`)
VALUES (?,?,?,?,?,?,?,?,?,?,?)" failed : Field 'IS_REWRITE_ENABLED' doesn't
have a default value
at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:720)
at
org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:740)
at
org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:1038)
{code}
was:
The materialized view impl breaks old metastore sql write access, by
complaining that the new table creation does not set this column up.
{code}
`IS_REWRITE_ENABLED` bit(1) NOT NULL,
{code}
{{NOT NULL DEFAULT 0}} would allow old metastore directsql compatibility.
> Metastore: default IS_REWRITE_ENABLED=false instead of NULL
> ---
>
> Key: HIVE-18046
> URL: https://issues.apache.org/jira/browse/HIVE-18046
> Project: Hive
> Issue Type: Bug
> Components: Materialized views, Metastore
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Priority: Minor
>
> The materialized view impl breaks old metastore sql write access, by
> complaining that the new table creation does not set this column up.
> {code}
> `IS_REWRITE_ENABLED` bit(1) NOT NULL,
> {code}
> {{NOT NULL DEFAULT 0}} would allow old metastore direct sql compatibility
> (not thrift).
> {code}
> 2017-11-09T07:11:58,331 ERROR [HiveServer2-Background-Pool: Thread-2354]
> metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
> 10) with error: javax.jdo.JDODataStoreException: Insert of object
> "org.apache.hadoop.hive.metastore.model.MTable@249dbf1" using statement
> "INSERT INTO `TBLS`
> (`TBL_ID`,`CREATE_TIME`,`DB_ID`,`LAST_ACCESS_TIME`,`OWNER`,`RETENTION`,`SD_ID`,`TBL_NAME`,`TBL_TYPE`,`VIEW_EXPANDED_TEXT`,`VIEW_ORIGINAL_TEXT`)
> VALUES (?,?,?,?,?,?,?,?,?,?,?)" failed : Field 'IS_REWRITE_ENABLED' doesn't
> have a default value
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:720)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:740)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:1038)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-15956) StackOverflowError when drop lots of partitions
[
https://issues.apache.org/jira/browse/HIVE-15956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248098#comment-16248098
]
Ashutosh Chauhan commented on HIVE-15956:
-
[~niklaus.xiao] I can repro the issue with test script you provided. But your
patch doesnt seem to fix it. I tried with batch-size 10, 100, 1000
> StackOverflowError when drop lots of partitions
> ---
>
> Key: HIVE-15956
> URL: https://issues.apache.org/jira/browse/HIVE-15956
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 1.3.0, 2.2.0
>Reporter: Niklaus Xiao
>Assignee: Niklaus Xiao
> Attachments: HIVE-15956.patch
>
>
> Repro steps:
> 1. Create partitioned table and add 1 partitions
> {code}
> create table test_partition(id int) partitioned by (dt int);
> alter table test_partition add partition(dt=1);
> alter table test_partition add partition(dt=3);
> alter table test_partition add partition(dt=4);
> ...
> alter table test_partition add partition(dt=1);
> {code}
> 2. Drop 9000 partitions:
> {code}
> alter table test_partition drop partition(dt<9000);
> {code}
> Step 2 will fail with StackOverflowError:
> {code}
> Exception in thread "pool-7-thread-161" java.lang.StackOverflowError
> at
> org.datanucleus.query.expression.ExpressionCompiler.isOperator(ExpressionCompiler.java:819)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:190)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:192)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:192)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> {code}
> {code}
> Exception in thread "pool-7-thread-198" java.lang.StackOverflowError
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:83)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17970) MM LOAD DATA with OVERWRITE doesn't use base_n directory concept
[
https://issues.apache.org/jira/browse/HIVE-17970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-17970:
--
Summary: MM LOAD DATA with OVERWRITE doesn't use base_n directory concept
(was: LOAD DATA with OVERWRITE doesn't use base_n directory concept)
> MM LOAD DATA with OVERWRITE doesn't use base_n directory concept
>
>
> Key: HIVE-17970
> URL: https://issues.apache.org/jira/browse/HIVE-17970
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>
> Judging by
> {code:java}
> Hive.loadTable(Path loadPath, String tableName, LoadFileType loadFileType,
> boolean isSrcLocal,
> boolean isSkewedStoreAsSubdir, boolean isAcid, boolean
> hasFollowingStatsTask,
> Long txnId, int stmtId, boolean isMmTable)
> {code}
> LOAD DATA with OVERWRITE will delete all existing data then write new data
> into the table. This logic makes sense for non-acid tables but for Acid/MM
> it should work like INSERT OVERWRITE statement and write new data to base_n/.
> This way the lock manager can be used to either get an X lock for IOW and
> thus block all readers or let it run with SemiShared and let readers continue
> and make the system more concurrent.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18046) Metastore: default IS_REWRITE_ENABLED=false instead of NULL
[
https://issues.apache.org/jira/browse/HIVE-18046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248062#comment-16248062
]
Gopal V commented on HIVE-18046:
bq. Hmm... directsql does not have write statements as far as I know
I meant to say "direct sql compatibility" instead of this being about thrift
access (i.e older embedded metastore + new sql backend).
The error is that create table fails with a mysql constraint error - because
the old insert doesn't insert that col (and default is NULL and the col is
constrained as NOT NULL).
> Metastore: default IS_REWRITE_ENABLED=false instead of NULL
> ---
>
> Key: HIVE-18046
> URL: https://issues.apache.org/jira/browse/HIVE-18046
> Project: Hive
> Issue Type: Bug
> Components: Materialized views, Metastore
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Priority: Minor
>
> The materialized view impl breaks old metastore sql write access, by
> complaining that the new table creation does not set this column up.
> {code}
> `IS_REWRITE_ENABLED` bit(1) NOT NULL,
> {code}
> {{NOT NULL DEFAULT 0}} would allow old metastore directsql compatibility.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17856) MM tables - IOW is not ACID compliant
[
https://issues.apache.org/jira/browse/HIVE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Yeom updated HIVE-17856:
--
Attachment: HIVE-17856.6.patch
The same as patch 5. But reattaching because patch 5 is not tested upon by the
system.
> MM tables - IOW is not ACID compliant
> -
>
> Key: HIVE-17856
> URL: https://issues.apache.org/jira/browse/HIVE-17856
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Steve Yeom
> Labels: mm-gap-1
> Attachments: HIVE-17856.1.patch, HIVE-17856.2.patch,
> HIVE-17856.3.patch, HIVE-17856.4.patch, HIVE-17856.5.patch, HIVE-17856.6.patch
>
>
> The following tests were removed from mm_all during "integration"... I should
> have never allowed such manner of intergration.
> MM logic should have been kept intact until ACID logic could catch up. Alas,
> here we are.
> {noformat}
> drop table iow0_mm;
> create table iow0_mm(key int) tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow0_mm select key from intermediate;
> insert into table iow0_mm select key + 1 from intermediate;
> select * from iow0_mm order by key;
> insert overwrite table iow0_mm select key + 2 from intermediate;
> select * from iow0_mm order by key;
> drop table iow0_mm;
> drop table iow1_mm;
> create table iow1_mm(key int) partitioned by (key2 int)
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow1_mm partition (key2)
> select key as k1, key from intermediate union all select key as k1, key from
> intermediate;
> insert into table iow1_mm partition (key2)
> select key + 1 as k1, key from intermediate union all select key as k1, key
> from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key from intermediate union all select key + 4 as k1,
> key from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key + 3 from intermediate union all select key + 2 as
> k1, key + 2 from intermediate;
> select * from iow1_mm order by key, key2;
> drop table iow1_mm;
> {noformat}
> {noformat}
> drop table simple_mm;
> create table simple_mm(key int) stored as orc tblproperties
> ("transactional"="true", "transactional_properties"="insert_only");
> insert into table simple_mm select key from intermediate;
> -insert overwrite table simple_mm select key from intermediate;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17856) MM tables - IOW is not ACID compliant
[
https://issues.apache.org/jira/browse/HIVE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248057#comment-16248057
]
Steve Yeom commented on HIVE-17856:
---
Wow the system seems to have dropped my patch 5 submitted yesterday early
afternoon.
Reattaching the same patch as version number 6.
> MM tables - IOW is not ACID compliant
> -
>
> Key: HIVE-17856
> URL: https://issues.apache.org/jira/browse/HIVE-17856
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Steve Yeom
> Labels: mm-gap-1
> Attachments: HIVE-17856.1.patch, HIVE-17856.2.patch,
> HIVE-17856.3.patch, HIVE-17856.4.patch, HIVE-17856.5.patch
>
>
> The following tests were removed from mm_all during "integration"... I should
> have never allowed such manner of intergration.
> MM logic should have been kept intact until ACID logic could catch up. Alas,
> here we are.
> {noformat}
> drop table iow0_mm;
> create table iow0_mm(key int) tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow0_mm select key from intermediate;
> insert into table iow0_mm select key + 1 from intermediate;
> select * from iow0_mm order by key;
> insert overwrite table iow0_mm select key + 2 from intermediate;
> select * from iow0_mm order by key;
> drop table iow0_mm;
> drop table iow1_mm;
> create table iow1_mm(key int) partitioned by (key2 int)
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow1_mm partition (key2)
> select key as k1, key from intermediate union all select key as k1, key from
> intermediate;
> insert into table iow1_mm partition (key2)
> select key + 1 as k1, key from intermediate union all select key as k1, key
> from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key from intermediate union all select key + 4 as k1,
> key from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key + 3 from intermediate union all select key + 2 as
> k1, key + 2 from intermediate;
> select * from iow1_mm order by key, key2;
> drop table iow1_mm;
> {noformat}
> {noformat}
> drop table simple_mm;
> create table simple_mm(key int) stored as orc tblproperties
> ("transactional"="true", "transactional_properties"="insert_only");
> insert into table simple_mm select key from intermediate;
> -insert overwrite table simple_mm select key from intermediate;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18046) Metastore: default IS_REWRITE_ENABLED=false instead of NULL
[
https://issues.apache.org/jira/browse/HIVE-18046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248052#comment-16248052
]
Sergey Shelukhin commented on HIVE-18046:
-
Hmm... directsql does not have write statements as far as I know. What is the
error that this causes?
> Metastore: default IS_REWRITE_ENABLED=false instead of NULL
> ---
>
> Key: HIVE-18046
> URL: https://issues.apache.org/jira/browse/HIVE-18046
> Project: Hive
> Issue Type: Bug
> Components: Materialized views, Metastore
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Priority: Minor
>
> The materialized view impl breaks old metastore sql write access, by
> complaining that the new table creation does not set this column up.
> {code}
> `IS_REWRITE_ENABLED` bit(1) NOT NULL,
> {code}
> {{NOT NULL DEFAULT 0}} would allow old metastore directsql compatibility.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17937) llap_acid_fast test is flaky
[ https://issues.apache.org/jira/browse/HIVE-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248051#comment-16248051 ] Sergey Shelukhin commented on HIVE-17937: - Patch looks good to me... the result changed in the run above however. Which one is the correct result? :) Might make sense to compare locally without IO > llap_acid_fast test is flaky > > > Key: HIVE-17937 > URL: https://issues.apache.org/jira/browse/HIVE-17937 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Teddy Choi > Attachments: HIVE-17993.patch > > > See for example > https://builds.apache.org/job/PreCommit-HIVE-Build/7521/testReport/org.apache.hadoop.hive.cli/TestMiniLlapLocalCliDriver/testCliDriver_llap_acid_fast_/history/ > (the history link is the same from any build number with a test run, just > replace 7521 if this one expires). > Looks like results change, which may not be good. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17809) Implement per pool trigger validation and move sessions across pools
[ https://issues.apache.org/jira/browse/HIVE-17809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248046#comment-16248046 ] Sergey Shelukhin commented on HIVE-17809: - Some minor comments on RB. > Implement per pool trigger validation and move sessions across pools > > > Key: HIVE-17809 > URL: https://issues.apache.org/jira/browse/HIVE-17809 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-17809.1.patch, HIVE-17809.2.patch, > HIVE-17809.3.patch, HIVE-17809.4.patch, HIVE-17809.5.patch > > > HIVE-17508 trigger validation is applied for all pools at once. This is > follow up to implement trigger validation at per pool level. > This should also implement resolution for multiple applicable actions, as per > the RB discussion -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17995) Run checkstyle on standalone-metastore module with proper configuration
[
https://issues.apache.org/jira/browse/HIVE-17995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-17995:
--
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Patch committed. {{mvn checkstyle:check}} fails in standalone-metastore
currently. The version of checkstyle we're using doesn't like lambdas. I
tried bumping up the version to the latest, but it appears they made
significant changes because a bunch of the modules we use didn't exist in the
new version. This will need to be fixed before we can turn on Yetis builds.
> Run checkstyle on standalone-metastore module with proper configuration
> ---
>
> Key: HIVE-17995
> URL: https://issues.apache.org/jira/browse/HIVE-17995
> Project: Hive
> Issue Type: Sub-task
>Reporter: Adam Szita
>Assignee: Adam Szita
> Fix For: 3.0.0
>
> Attachments: HIVE-17995.0.patch, HIVE-17995.1.patch
>
>
> Maven module standalone-metastore is obviously not connected to Hive root
> pom, therefore if someone (or an automated Yetus check) runs {{mvn
> checkstyle}} it will not consider Hive-specific checkstyle settings (e.g.
> validates row lengths against 80, not 100)
> We need to make sure standalone-metastore pom has the proper checkstyle
> configuration
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17809) Implement per pool trigger validation and move sessions across pools
[ https://issues.apache.org/jira/browse/HIVE-17809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-17809: - Attachment: (was: HIVE-17809.5.patch) > Implement per pool trigger validation and move sessions across pools > > > Key: HIVE-17809 > URL: https://issues.apache.org/jira/browse/HIVE-17809 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-17809.1.patch, HIVE-17809.2.patch, > HIVE-17809.3.patch, HIVE-17809.4.patch, HIVE-17809.5.patch > > > HIVE-17508 trigger validation is applied for all pools at once. This is > follow up to implement trigger validation at per pool level. > This should also implement resolution for multiple applicable actions, as per > the RB discussion -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17809) Implement per pool trigger validation and move sessions across pools
[ https://issues.apache.org/jira/browse/HIVE-17809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-17809: - Attachment: HIVE-17809.5.patch > Implement per pool trigger validation and move sessions across pools > > > Key: HIVE-17809 > URL: https://issues.apache.org/jira/browse/HIVE-17809 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-17809.1.patch, HIVE-17809.2.patch, > HIVE-17809.3.patch, HIVE-17809.4.patch, HIVE-17809.5.patch, HIVE-17809.5.patch > > > HIVE-17508 trigger validation is applied for all pools at once. This is > follow up to implement trigger validation at per pool level. > This should also implement resolution for multiple applicable actions, as per > the RB discussion -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18045) can VectorizedOrcAcidRowBatchReader be used all the time
[ https://issues.apache.org/jira/browse/HIVE-18045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247987#comment-16247987 ] Sergey Shelukhin commented on HIVE-18045: - There's BatchToRow* functionality already that allows LLAP IO to be used for non-vectorized queries. It could be reused... > can VectorizedOrcAcidRowBatchReader be used all the time > > > Key: HIVE-18045 > URL: https://issues.apache.org/jira/browse/HIVE-18045 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > > Can we use VectorizedOrcAcidRowBatchReader for non-vectorized queries? > It would just need a wrapper on top of it to turn VRBs into rows. > This would mean there is just 1 acid reader to maintain - not 2. > Would this be an issue for sorted reader/SMB support? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18045) can VectorizedOrcAcidRowBatchReader be used all the time
[ https://issues.apache.org/jira/browse/HIVE-18045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-18045: -- Description: Can we use VectorizedOrcAcidRowBatchReader for non-vectorized queries? It would just need a wrapper on top of it to turn VRBs into rows. This would mean there is just 1 acid reader to maintain - not 2. Would this be an issue for sorted reader/SMB support? was: Can we use VectorizedOrcAcidRowBatchReader for non-vectorized queries? It would just need a wrapper on top of it to turn VRBs into rows. This would mean there is just 1 acid reader to maintain - not 2. > can VectorizedOrcAcidRowBatchReader be used all the time > > > Key: HIVE-18045 > URL: https://issues.apache.org/jira/browse/HIVE-18045 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > > Can we use VectorizedOrcAcidRowBatchReader for non-vectorized queries? > It would just need a wrapper on top of it to turn VRBs into rows. > This would mean there is just 1 acid reader to maintain - not 2. > Would this be an issue for sorted reader/SMB support? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-18045) can VectorizedOrcAcidRowBatchReader be used all the time
[ https://issues.apache.org/jira/browse/HIVE-18045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman reassigned HIVE-18045: - > can VectorizedOrcAcidRowBatchReader be used all the time > > > Key: HIVE-18045 > URL: https://issues.apache.org/jira/browse/HIVE-18045 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > > Can we use VectorizedOrcAcidRowBatchReader for non-vectorized queries? > It would just need a wrapper on top of it to turn VRBs into rows. > This would mean there is just 1 acid reader to maintain - not 2. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16669) Fine tune Compaction to take advantage of Acid 2.0
[ https://issues.apache.org/jira/browse/HIVE-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247974#comment-16247974 ] Eugene Koifman commented on HIVE-16669: --- Can Major compaction use Vectorized reader underneath? > Fine tune Compaction to take advantage of Acid 2.0 > -- > > Key: HIVE-16669 > URL: https://issues.apache.org/jira/browse/HIVE-16669 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-16669.wip.patch > > > * There is little point using 2.0 vectorized reader since there is no > operator pipeline in compaction > * If minor compaction just concats delete_delta files together, then the 2 > stage compaction should always ensure that we have a limited number of Orc > readers to do the merging and current OrcRawRecordMerger should be fine > * ... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17809) Implement per pool trigger validation and move sessions across pools
[ https://issues.apache.org/jira/browse/HIVE-17809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-17809: - Attachment: HIVE-17809.5.patch > Implement per pool trigger validation and move sessions across pools > > > Key: HIVE-17809 > URL: https://issues.apache.org/jira/browse/HIVE-17809 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-17809.1.patch, HIVE-17809.2.patch, > HIVE-17809.3.patch, HIVE-17809.4.patch, HIVE-17809.5.patch > > > HIVE-17508 trigger validation is applied for all pools at once. This is > follow up to implement trigger validation at per pool level. > This should also implement resolution for multiple applicable actions, as per > the RB discussion -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17937) llap_acid_fast test is flaky
[
https://issues.apache.org/jira/browse/HIVE-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247920#comment-16247920
]
Hive QA commented on HIVE-17937:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897078/HIVE-17993.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 11374 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7765/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7765/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7765/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897078 - PreCommit-HIVE-Build
> llap_acid_fast test is flaky
>
>
> Key: HIVE-17937
> URL: https://issues.apache.org/jira/browse/HIVE-17937
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Teddy Choi
> Attachments: HIVE-17993.patch
>
>
> See for example
> https://builds.apache.org/job/PreCommit-HIVE-Build/7521/testReport/org.apache.hadoop.hive.cli/TestMiniLlapLocalCliDriver/testCliDriver_llap_acid_fast_/history/
> (the history link is the same from any build number with a test run, just
> replace 7521 if this one expires).
> Looks like results change, which may not be good.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18029) user mapping - support proper usernames with doAs = false
[
https://issues.apache.org/jira/browse/HIVE-18029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247909#comment-16247909
]
Thejas M Nair commented on HIVE-18029:
--
bq. maybe make it configurable
Its highly unlikely that someone going to be relying on the 'anonymous' user
being the user. I think this is a slightly backward incompatible change we can
make in 3.0.0 release. We don't have to add the burden of maintaining a config
that no one is likely to use.
> user mapping - support proper usernames with doAs = false
> -
>
> Key: HIVE-18029
> URL: https://issues.apache.org/jira/browse/HIVE-18029
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>
> Right now what happens on unsecure cluster with doAs=false (not sure which
> one is to blame - didn't look into it, maybe both) is {noformat}
> 2017-11-08T21:39:49,404 INFO [HiveServer2-Background-Pool: Thread-205]
> tez.WorkloadManagerFederation: Getting a WM session for anonymous
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18029) user mapping - support proper usernames with doAs = false
[
https://issues.apache.org/jira/browse/HIVE-18029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247904#comment-16247904
]
Sergey Shelukhin commented on HIVE-18029:
-
Yeah I was going to do smth like that... maybe make it configurable :)
> user mapping - support proper usernames with doAs = false
> -
>
> Key: HIVE-18029
> URL: https://issues.apache.org/jira/browse/HIVE-18029
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>
> Right now what happens on unsecure cluster with doAs=false (not sure which
> one is to blame - didn't look into it, maybe both) is {noformat}
> 2017-11-08T21:39:49,404 INFO [HiveServer2-Background-Pool: Thread-205]
> tez.WorkloadManagerFederation: Getting a WM session for anonymous
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17871) Add non nullability flag to druid time column
[
https://issues.apache.org/jira/browse/HIVE-17871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247901#comment-16247901
]
Jesus Camacho Rodriguez commented on HIVE-17871:
[~bslim], nice, thanks!
> Add non nullability flag to druid time column
> -
>
> Key: HIVE-17871
> URL: https://issues.apache.org/jira/browse/HIVE-17871
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-17871.2.patch, HIVE-17871.patch
>
>
> Druid time column is non null all the time.
> Adding the non nullability flag will enable extra calcite goodness like
> transforming
> {code} select count(`__time`) from table {code} to {code} select count(*)
> from table {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17871) Add non nullability flag to druid time column
[
https://issues.apache.org/jira/browse/HIVE-17871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-17871:
--
Attachment: HIVE-17871.2.patch
This patch has some unwanted code to test
> Add non nullability flag to druid time column
> -
>
> Key: HIVE-17871
> URL: https://issues.apache.org/jira/browse/HIVE-17871
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-17871.2.patch, HIVE-17871.patch
>
>
> Druid time column is non null all the time.
> Adding the non nullability flag will enable extra calcite goodness like
> transforming
> {code} select count(`__time`) from table {code} to {code} select count(*)
> from table {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HIVE-17457) Acid Insert Overwrite when the transaction fails
[ https://issues.apache.org/jira/browse/HIVE-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159992#comment-16159992 ] Eugene Koifman edited comment on HIVE-17457 at 11/10/17 6:09 PM: - the files in new base_x may not even be readable so likely this needs base_x_x so that the whole dir is skipped on read Another issue to think about: if 'x' aborts when does the metadata for it get cleaned? There is nothing in (COMPLETED_)TXN_COMPONENTS about it. Add logic to write to txn_components and special handling for base_x when x is aborted to skip it? Normal compaction will never produce base_x with x opened since it never includes any open txns. was (Author: ekoifman): the files in new base_x may not even be readable so likely this needs base_x_x so that the whole dir is skipped on read > Acid Insert Overwrite when the transaction fails > > > Key: HIVE-17457 > URL: https://issues.apache.org/jira/browse/HIVE-17457 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > > HIVE-14988 adds support for Insert Overwrite for Acid tables. > once we have direct write to target dir (i.e. no move op) - how do we handle > the case where the txn running IOW aborts? See if getAcidState() does the > right thing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18029) user mapping - support proper usernames with doAs = false
[
https://issues.apache.org/jira/browse/HIVE-18029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247798#comment-16247798
]
Thejas M Nair commented on HIVE-18029:
--
A correction - the "anonymous" is coming from jdbc driver, not beeline. In
HiveConnection.java -
{code}
private String getUserName() {
return getSessionValue(JdbcConnectionParams.AUTH_USER,
JdbcConnectionParams.ANONYMOUS_USER);
}
{code}
[~sershe] Do you plan to change/fix something here ? If yes, what is it ?
I think it might be a good idea to do what other commands in hadoop ecosystem
do, ie pick the current OS user name if none is specified (in non kerberos
mode).
> user mapping - support proper usernames with doAs = false
> -
>
> Key: HIVE-18029
> URL: https://issues.apache.org/jira/browse/HIVE-18029
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>
> Right now what happens on unsecure cluster with doAs=false (not sure which
> one is to blame - didn't look into it, maybe both) is {noformat}
> 2017-11-08T21:39:49,404 INFO [HiveServer2-Background-Pool: Thread-205]
> tez.WorkloadManagerFederation: Getting a WM session for anonymous
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18009) Multiple lateral view query is slow on hive on spark
[
https://issues.apache.org/jira/browse/HIVE-18009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-18009:
Attachment: HIVE-18009.3.patch
> Multiple lateral view query is slow on hive on spark
>
>
> Key: HIVE-18009
> URL: https://issues.apache.org/jira/browse/HIVE-18009
> Project: Hive
> Issue Type: Improvement
> Components: Spark
>Affects Versions: 3.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-18009.1.patch, HIVE-18009.2.patch,
> HIVE-18009.3.patch
>
>
> When running the query with multiple lateral view, HoS is busy with the
> compilation. GenSparkUtils has an efficient implementation of
> getChildOperator when we have diamond hierarchy in operator trees (lateral
> view in this case) since the node may be visited multiple times.
> {noformat}
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:442)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
>
[jira] [Updated] (HIVE-18009) Multiple lateral view query is slow on hive on spark
[
https://issues.apache.org/jira/browse/HIVE-18009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-18009:
Status: Patch Available (was: In Progress)
> Multiple lateral view query is slow on hive on spark
>
>
> Key: HIVE-18009
> URL: https://issues.apache.org/jira/browse/HIVE-18009
> Project: Hive
> Issue Type: Improvement
> Components: Spark
>Affects Versions: 3.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-18009.1.patch, HIVE-18009.2.patch,
> HIVE-18009.3.patch
>
>
> When running the query with multiple lateral view, HoS is busy with the
> compilation. GenSparkUtils has an efficient implementation of
> getChildOperator when we have diamond hierarchy in operator trees (lateral
> view in this case) since the node may be visited multiple times.
> {noformat}
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:442)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
>
[jira] [Updated] (HIVE-18009) Multiple lateral view query is slow on hive on spark
[
https://issues.apache.org/jira/browse/HIVE-18009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-18009:
Status: In Progress (was: Patch Available)
> Multiple lateral view query is slow on hive on spark
>
>
> Key: HIVE-18009
> URL: https://issues.apache.org/jira/browse/HIVE-18009
> Project: Hive
> Issue Type: Improvement
> Components: Spark
>Affects Versions: 3.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-18009.1.patch, HIVE-18009.2.patch,
> HIVE-18009.3.patch
>
>
> When running the query with multiple lateral view, HoS is busy with the
> compilation. GenSparkUtils has an efficient implementation of
> getChildOperator when we have diamond hierarchy in operator trees (lateral
> view in this case) since the node may be visited multiple times.
> {noformat}
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:442)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
>
[jira] [Updated] (HIVE-18009) Multiple lateral view query is slow on hive on spark
[
https://issues.apache.org/jira/browse/HIVE-18009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-18009:
Attachment: (was: HIVE-18009.3.patch)
> Multiple lateral view query is slow on hive on spark
>
>
> Key: HIVE-18009
> URL: https://issues.apache.org/jira/browse/HIVE-18009
> Project: Hive
> Issue Type: Improvement
> Components: Spark
>Affects Versions: 3.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-18009.1.patch, HIVE-18009.2.patch,
> HIVE-18009.3.patch
>
>
> When running the query with multiple lateral view, HoS is busy with the
> compilation. GenSparkUtils has an efficient implementation of
> getChildOperator when we have diamond hierarchy in operator trees (lateral
> view in this case) since the node may be visited multiple times.
> {noformat}
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:442)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.getChildOperator(GenSparkUtils.java:438)
> at
>
[jira] [Commented] (HIVE-18041) Add SORT_QUERY_RESULTS to subquery_multi
[ https://issues.apache.org/jira/browse/HIVE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247766#comment-16247766 ] Zoltan Haindrich commented on HIVE-18041: - my only concern with SORT_QUERY_RESULTS is that it makes the jenkins test results harder to understand whats even the change...I just wanted to note that considering order by should also be an option; sorry for the noise > Add SORT_QUERY_RESULTS to subquery_multi > > > Key: HIVE-18041 > URL: https://issues.apache.org/jira/browse/HIVE-18041 > Project: Hive > Issue Type: Test >Reporter: Rui Li >Priority: Trivial > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18012) fix ct_noperm_loc test
[
https://issues.apache.org/jira/browse/HIVE-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247723#comment-16247723
]
Ashutosh Chauhan commented on HIVE-18012:
-
+1
> fix ct_noperm_loc test
> --
>
> Key: HIVE-18012
> URL: https://issues.apache.org/jira/browse/HIVE-18012
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-18012.001.patch, HIVE-18012.02.patch
>
>
> the goal of the test is to check that hive doesn't let user1 to create a
> table with a location under an unowned path.
> I've bisected this test to be broken by
> 5250ef450430fcdeed0a2cb7a770f48647987cd3 (HIVE-12408).
> the original exception was (which have been by that sole masked line):
> {code}
> FAILED: HiveAccessControlException Permission denied: Principal [name=user1,
> type=USER] does not have following privileges for operation CREATETABLE
> [[OBJECT OWNERSHIP] on Object [type=DFS_URI,
> name=hdfs://localhost:35753/tmp/ct_noperm_loc_foo0]]
> {code}
> the current semanticexception shouldnt be accepted ; because it's unrelated
> to the tests goal.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[
https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247705#comment-16247705
]
Hive QA commented on HIVE-18038:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897088/HIVE-18038.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 11374 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithMr.testFetchResultsOfLogWithOrientation
(batchId=227)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7764/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7764/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7764/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897088 - PreCommit-HIVE-Build
> org.apache.hadoop.hive.ql.session.OperationLog - Review
> ---
>
> Key: HIVE-18038
> URL: https://issues.apache.org/jira/browse/HIVE-18038
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Priority: Trivial
> Attachments: HIVE-18038.1.patch, HIVE-18038.2.patch
>
>
> Simplifications, improve readability
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-13567) Enable auto-gather column stats by default
[ https://issues.apache.org/jira/browse/HIVE-13567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-13567: Attachment: HIVE-13567.23wip02.patch > Enable auto-gather column stats by default > -- > > Key: HIVE-13567 > URL: https://issues.apache.org/jira/browse/HIVE-13567 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Zoltan Haindrich > Attachments: HIVE-13567.01.patch, HIVE-13567.02.patch, > HIVE-13567.03.patch, HIVE-13567.04.patch, HIVE-13567.05.patch, > HIVE-13567.06.patch, HIVE-13567.07.patch, HIVE-13567.08.patch, > HIVE-13567.09.patch, HIVE-13567.10.patch, HIVE-13567.11.patch, > HIVE-13567.12.patch, HIVE-13567.13.patch, HIVE-13567.14.patch, > HIVE-13567.15.patch, HIVE-13567.16.patch, HIVE-13567.17.patch, > HIVE-13567.18.patch, HIVE-13567.19.patch, HIVE-13567.20.patch, > HIVE-13567.21.patch, HIVE-13567.22.patch, HIVE-13567.23wip01.patch, > HIVE-13567.23wip02.patch > > > in phase 2, we are going to set auto-gather column on as default. This needs > to update golden files. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-13567) Enable auto-gather column stats by default
[
https://issues.apache.org/jira/browse/HIVE-13567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247600#comment-16247600
]
Hive QA commented on HIVE-13567:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897075/HIVE-13567.23wip01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7763/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7763/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7763/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-11-10 14:55:17.471
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-7763/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-11-10 14:55:17.475
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
9f653d2..5011da7 master -> origin/master
cd08cd6..f993147 branch-1 -> origin/branch-1
+ git reset --hard HEAD
HEAD is now at 9f653d2 HIVE-16075 : MetaStore needs to reinitialize log4j to
allow log specific settings via hiveconf take effect (Yunfei Liu via Prashanth
J)
+ git clean -f -d
Removing
itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestTriggersMoveWorkloadManager.java
Removing
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/KillMoveTriggerActionHandler.java
Removing
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/PerPoolTriggerValidatorRunnable.java
Removing ql/src/java/org/apache/hadoop/hive/ql/wm/Action.java
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 2 commits, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 5011da7 HIVE-17785: Encription tests are not running (Peter
Vary, reviewed by Barna Zsombor Klara)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-11-10 14:55:23.843
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestMTQueries.java:44
error:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestMTQueries.java:
patch does not apply
error: patch failed:
itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniHS2.java:201
error:
itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniHS2.java:
patch does not apply
error: patch failed: itests/src/test/resources/testconfiguration.properties:111
error: itests/src/test/resources/testconfiguration.properties: patch does not
apply
error: patch failed:
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java:853
error: ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java: patch does
not apply
error: patch failed:
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java:129
error: ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java: patch does
not apply
error: patch failed:
ql/src/test/org/apache/hadoop/hive/ql/TxnCommandsBaseForTests.java:64
error: ql/src/test/org/apache/hadoop/hive/ql/TxnCommandsBaseForTests.java:
patch does not apply
error: patch failed: ql/src/test/queries/clientpositive/autoColumnStats_1.q:60
error: ql/src/test/queries/clientpositive/autoColumnStats_1.q: patch does not
apply
error: ql/src/test/queries/clientpositive/autoColumnStats_10.q: already exists
in working directory
error: patch failed:
ql/src/test/queries/clientpositive/exec_parallel_column_stats.q:1
error: ql/src/test/queries/clientpositive/exec_parallel_column_stats.q: patch
does not apply
error: patch failed:
[jira] [Commented] (HIVE-18041) Add SORT_QUERY_RESULTS to subquery_multi
[ https://issues.apache.org/jira/browse/HIVE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247542#comment-16247542 ] Rui Li commented on HIVE-18041: --- Hi [~kgyrtkirk], could you elaborate on the problem you mentioned? My understanding is SORT_QUERY_RESULTS will have similar effect as order by, except that the records are sorted in alphabetical order. But I did find that some queries in subquery_multi already have order by, and SORT_QUERY_RESULTS might change the output of those queries. In that sense, maybe adding more order by is a better choice. > Add SORT_QUERY_RESULTS to subquery_multi > > > Key: HIVE-18041 > URL: https://issues.apache.org/jira/browse/HIVE-18041 > Project: Hive > Issue Type: Test >Reporter: Rui Li >Priority: Trivial > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[ https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247557#comment-16247557 ] Peter Vary commented on HIVE-18038: --- +1 pending tests > org.apache.hadoop.hive.ql.session.OperationLog - Review > --- > > Key: HIVE-18038 > URL: https://issues.apache.org/jira/browse/HIVE-18038 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: BELUGA BEHR >Priority: Trivial > Attachments: HIVE-18038.1.patch, HIVE-18038.2.patch > > > Simplifications, improve readability -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[
https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247550#comment-16247550
]
BELUGA BEHR commented on HIVE-18038:
[~pvary] Thanks!!!
\\
\\
* {{IOUtils.closeStream(in)}} already has a null check in it, so no harm there,
just reusing code.
https://github.com/apache/hadoop/blob/bc206806dadc5dc85f182d98d859307cfb33172b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/IOUtils.java#L294
* I reverted the change for the "Reading log file" message.
One of the things that I was attempting to address was there there is an
edge-case with {{nLines}} being equal to 1. In this case, if the "Reading log
file" message is included, {{nLines}} is decremented to 0 and then the
following code will interpret the zero to mean "no cap":
{code:java}
// if nLines <= 0, read all lines in log file.
for (int i = 0; i < nLines || nLines <= 0; i++) {
{code}
So, if {{isShortLogs}} is true and the user requests one row, should we only
provide the "Reading log file" message or should we always provide this line,
when requested, and then include {{nLines}} more lines to the results?
> org.apache.hadoop.hive.ql.session.OperationLog - Review
> ---
>
> Key: HIVE-18038
> URL: https://issues.apache.org/jira/browse/HIVE-18038
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Priority: Trivial
> Attachments: HIVE-18038.1.patch, HIVE-18038.2.patch
>
>
> Simplifications, improve readability
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[ https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247553#comment-16247553 ] Peter Vary commented on HIVE-18038: --- Good catch [~belugabehr]! :) > org.apache.hadoop.hive.ql.session.OperationLog - Review > --- > > Key: HIVE-18038 > URL: https://issues.apache.org/jira/browse/HIVE-18038 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: BELUGA BEHR >Priority: Trivial > Attachments: HIVE-18038.1.patch, HIVE-18038.2.patch > > > Simplifications, improve readability -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HIVE-18036) Stats: Remove usage of clone() methods
[
https://issues.apache.org/jira/browse/HIVE-18036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-18036 started by Bertalan Kondrat.
---
> Stats: Remove usage of clone() methods
> --
>
> Key: HIVE-18036
> URL: https://issues.apache.org/jira/browse/HIVE-18036
> Project: Hive
> Issue Type: Sub-task
> Components: Statistics
>Reporter: Zoltan Haindrich
>Assignee: Bertalan Kondrat
>
> {{Statistics}} and {{ColStats}} implements cloneable; however they never
> throw clonenotsupported; and this causes try / catch blocks which are just
> noise
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[ https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HIVE-18038: --- Attachment: HIVE-18038.2.patch > org.apache.hadoop.hive.ql.session.OperationLog - Review > --- > > Key: HIVE-18038 > URL: https://issues.apache.org/jira/browse/HIVE-18038 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: BELUGA BEHR >Priority: Trivial > Attachments: HIVE-18038.1.patch, HIVE-18038.2.patch > > > Simplifications, improve readability -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-18036) Stats: Remove usage of clone() methods
[
https://issues.apache.org/jira/browse/HIVE-18036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bertalan Kondrat reassigned HIVE-18036:
---
Assignee: Bertalan Kondrat
> Stats: Remove usage of clone() methods
> --
>
> Key: HIVE-18036
> URL: https://issues.apache.org/jira/browse/HIVE-18036
> Project: Hive
> Issue Type: Sub-task
> Components: Statistics
>Reporter: Zoltan Haindrich
>Assignee: Bertalan Kondrat
>
> {{Statistics}} and {{ColStats}} implements cloneable; however they never
> throw clonenotsupported; and this causes try / catch blocks which are just
> noise
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17937) llap_acid_fast test is flaky
[ https://issues.apache.org/jira/browse/HIVE-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi updated HIVE-17937: -- Status: Patch Available (was: Open) > llap_acid_fast test is flaky > > > Key: HIVE-17937 > URL: https://issues.apache.org/jira/browse/HIVE-17937 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Teddy Choi > Attachments: HIVE-17993.patch > > > See for example > https://builds.apache.org/job/PreCommit-HIVE-Build/7521/testReport/org.apache.hadoop.hive.cli/TestMiniLlapLocalCliDriver/testCliDriver_llap_acid_fast_/history/ > (the history link is the same from any build number with a test run, just > replace 7521 if this one expires). > Looks like results change, which may not be good. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17937) llap_acid_fast test is flaky
[ https://issues.apache.org/jira/browse/HIVE-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi updated HIVE-17937: -- Attachment: HIVE-17993.patch The ColumnVector array of ColumnVectorBatch is referenced as ColumnVector array of VectorizedRowBatch. So it seems like when CV array in CVB is changed, CV array in VRB is also changed. It may result unexpected behaviors as described in this issue. So I changed it to create a new CV array to make CVB and VRB independent from each other. It is hard to reproduce this issue in my laptop. If the test fails, I will fix it further. Thanks for reporting this issue. > llap_acid_fast test is flaky > > > Key: HIVE-17937 > URL: https://issues.apache.org/jira/browse/HIVE-17937 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Teddy Choi > Attachments: HIVE-17993.patch > > > See for example > https://builds.apache.org/job/PreCommit-HIVE-Build/7521/testReport/org.apache.hadoop.hive.cli/TestMiniLlapLocalCliDriver/testCliDriver_llap_acid_fast_/history/ > (the history link is the same from any build number with a test run, just > replace 7521 if this one expires). > Looks like results change, which may not be good. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17785) Encription tests are not running
[
https://issues.apache.org/jira/browse/HIVE-17785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-17785:
--
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Pushed to the master.
Thanks [~zsombor.klara] for the review!
> Encription tests are not running
>
>
> Key: HIVE-17785
> URL: https://issues.apache.org/jira/browse/HIVE-17785
> Project: Hive
> Issue Type: Bug
>Reporter: Peter Vary
>Assignee: Peter Vary
> Fix For: 3.0.0
>
> Attachments: HIVE-17785.patch
>
>
> The testconfiguration.properties contains multiple tests in
> {{encrypted.query.files}}.
> There is no comma at the end of the list, so the tests are not running
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18038) org.apache.hadoop.hive.ql.session.OperationLog - Review
[
https://issues.apache.org/jira/browse/HIVE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247424#comment-16247424
]
Peter Vary commented on HIVE-18038:
---
[~belugabehr]: I am in a little hurry (last working day before a week pto) so
my review is far from complete, but have a few concerns:
- Removing the null check from {{private void resetIn()}} - this is used at
multiple places, and might cause problems at other places. Like if the
following line throws an exception: {{in = new BufferedReader(new
InputStreamReader(new FileInputStream(file)));}}, and the user tries
{{read(boolean isFetchFirst, long maxRows)}} twice
- Moving {{logs.add("Reading log file: " + file);}} to other place might extra
lines to the logs not just to the first line, but into the middle as well, if
there are more lines in the log file, than the {{nLines}}
Thanks,
Peter
> org.apache.hadoop.hive.ql.session.OperationLog - Review
> ---
>
> Key: HIVE-18038
> URL: https://issues.apache.org/jira/browse/HIVE-18038
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Priority: Trivial
> Attachments: HIVE-18038.1.patch
>
>
> Simplifications, improve readability
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-13567) Enable auto-gather column stats by default
[ https://issues.apache.org/jira/browse/HIVE-13567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-13567: Attachment: HIVE-13567.23wip01.patch #23wip01) now that stats merge is prepared to work mostly ok; it would be usefull to see if enabling by default show any problems or not > Enable auto-gather column stats by default > -- > > Key: HIVE-13567 > URL: https://issues.apache.org/jira/browse/HIVE-13567 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Zoltan Haindrich > Attachments: HIVE-13567.01.patch, HIVE-13567.02.patch, > HIVE-13567.03.patch, HIVE-13567.04.patch, HIVE-13567.05.patch, > HIVE-13567.06.patch, HIVE-13567.07.patch, HIVE-13567.08.patch, > HIVE-13567.09.patch, HIVE-13567.10.patch, HIVE-13567.11.patch, > HIVE-13567.12.patch, HIVE-13567.13.patch, HIVE-13567.14.patch, > HIVE-13567.15.patch, HIVE-13567.16.patch, HIVE-13567.17.patch, > HIVE-13567.18.patch, HIVE-13567.19.patch, HIVE-13567.20.patch, > HIVE-13567.21.patch, HIVE-13567.22.patch, HIVE-13567.23wip01.patch > > > in phase 2, we are going to set auto-gather column on as default. This needs > to update golden files. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17947) Concurrent inserts might fail for ACID table since HIVE-17526 on branch-1
[
https://issues.apache.org/jira/browse/HIVE-17947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247410#comment-16247410
]
Daniel Voros commented on HIVE-17947:
-
Thank you all!
> Concurrent inserts might fail for ACID table since HIVE-17526 on branch-1
> -
>
> Key: HIVE-17947
> URL: https://issues.apache.org/jira/browse/HIVE-17947
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 1.3.0
>Reporter: Daniel Voros
>Assignee: Daniel Voros
>Priority: Blocker
> Fix For: 1.3.0
>
> Attachments: HIVE-17947.1-branch-1.patch,
> HIVE-17947.2-branch-1.patch, HIVE-17947.3-branch-1.patch
>
>
> HIVE-17526 (only on branch-1) disabled conversion to ACID if there are
> *_copy_N files under the table, but the filesystem checks introduced there
> are running for every insert since the MoveTask in the end of the insert will
> call alterTable eventually.
> The filename checking also recurses into staging directories created by other
> inserts. If those are removed while listing the files, it leads to the
> following exception and failing insert:
> {code}
> java.io.FileNotFoundException: File
> hdfs://mycluster/apps/hive/warehouse/dvoros.db/concurrent_insert/.hive-staging_hive_2017-10-30_13-23-35_056_2844419018556002410-2/-ext-10001
> does not exist.
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.(DistributedFileSystem.java:1081)
> ~[hadoop-hdfs-2.7.3.2.6.3.0-235.jar:?]
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.(DistributedFileSystem.java:1059)
> ~[hadoop-hdfs-2.7.3.2.6.3.0-235.jar:?]
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1004)
> ~[hadoop-hdfs-2.7.3.2.6.3.0-235.jar:?]
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1000)
> ~[hadoop-hdfs-2.7.3.2.6.3.0-235.jar:?]
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listLocatedStatus(DistributedFileSystem.java:1018)
> ~[hadoop-hdfs-2.7.3.2.6.3.0-235.jar:?]
> at
> org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1735)
> ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
> at
> org.apache.hadoop.fs.FileSystem$6.handleFileStat(FileSystem.java:1864)
> ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
> at org.apache.hadoop.fs.FileSystem$6.hasNext(FileSystem.java:1841)
> ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
> at
> org.apache.hadoop.hive.metastore.TransactionalValidationListener.containsCopyNFiles(TransactionalValidationListener.java:226)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at
> org.apache.hadoop.hive.metastore.TransactionalValidationListener.handleAlterTableTransactionalProp(TransactionalValidationListener.java:104)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at
> org.apache.hadoop.hive.metastore.TransactionalValidationListener.handle(TransactionalValidationListener.java:63)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at
> org.apache.hadoop.hive.metastore.TransactionalValidationListener.onEvent(TransactionalValidationListener.java:55)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.firePreEvent(HiveMetaStore.java:2478)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_table_core(HiveMetaStore.java:4145)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_table_with_environment_context(HiveMetaStore.java:4117)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at sun.reflect.GeneratedMethodAccessor107.invoke(Unknown Source)
> ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_144]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_144]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at
> com.sun.proxy.$Proxy32.alter_table_with_environment_context(Unknown Source)
> [?:?]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_table_with_environmentContext(HiveMetaStoreClient.java:299)
>
[jira] [Updated] (HIVE-13567) Enable auto-gather column stats by default
[ https://issues.apache.org/jira/browse/HIVE-13567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-13567: Summary: Enable auto-gather column stats by default (was: Auto-gather column stats - phase 2) > Enable auto-gather column stats by default > -- > > Key: HIVE-13567 > URL: https://issues.apache.org/jira/browse/HIVE-13567 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Zoltan Haindrich > Attachments: HIVE-13567.01.patch, HIVE-13567.02.patch, > HIVE-13567.03.patch, HIVE-13567.04.patch, HIVE-13567.05.patch, > HIVE-13567.06.patch, HIVE-13567.07.patch, HIVE-13567.08.patch, > HIVE-13567.09.patch, HIVE-13567.10.patch, HIVE-13567.11.patch, > HIVE-13567.12.patch, HIVE-13567.13.patch, HIVE-13567.14.patch, > HIVE-13567.15.patch, HIVE-13567.16.patch, HIVE-13567.17.patch, > HIVE-13567.18.patch, HIVE-13567.19.patch, HIVE-13567.20.patch, > HIVE-13567.21.patch, HIVE-13567.22.patch > > > in phase 2, we are going to set auto-gather column on as default. This needs > to update golden files. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-13748) TypeInfoParser cannot handle the dash in the field name of a complex type
[ https://issues.apache.org/jira/browse/HIVE-13748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247368#comment-16247368 ] Andreas Weise commented on HIVE-13748: -- We are facing the same issue. Is there any progress / discussion going on? Do you expect side-effects when allowing '-' ? Thanks in advance for details. > TypeInfoParser cannot handle the dash in the field name of a complex type > - > > Key: HIVE-13748 > URL: https://issues.apache.org/jira/browse/HIVE-13748 > Project: Hive > Issue Type: Bug >Reporter: Chaoyu Tang >Assignee: Chaoyu Tang >Priority: Minor > > hive> create table y(col struct<`a-b`:double> COMMENT 'type field has a > dash'); > FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.IllegalArgumentException: > Error: : expected at the position 8 of 'struct' but '-' is found. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-15939) Make cast expressions comply more to sql2011
[
https://issues.apache.org/jira/browse/HIVE-15939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247355#comment-16247355
]
Zoltan Haindrich commented on HIVE-15939:
-
[~teddy.choi] could you rebase the patch to the current master?
> Make cast expressions comply more to sql2011
>
>
> Key: HIVE-15939
> URL: https://issues.apache.org/jira/browse/HIVE-15939
> Project: Hive
> Issue Type: Sub-task
> Components: SQL
>Reporter: Zoltan Haindrich
>Assignee: Teddy Choi
> Attachments: HIVE-15939.1.patch, HIVE-15939.2.patch,
> HIVE-15939.3.patch, HIVE-15939.4.patch, HIVE-15939.5.patch
>
>
> in HIVE-6590 Jason have uncovered the fact that UDFToBoolean treats all
> non-empty strings as true.
> It would be great to have the cast expressions closer to the standard...at
> least when there is an expected behaviour from the user;
> like {{cast('false' as boolean)}} should be true.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17809) Implement per pool trigger validation and move sessions across pools
[
https://issues.apache.org/jira/browse/HIVE-17809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247350#comment-16247350
]
Hive QA commented on HIVE-17809:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897049/HIVE-17809.4.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 11380 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=111)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testApplyPlanQpChanges
(batchId=281)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7762/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7762/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7762/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897049 - PreCommit-HIVE-Build
> Implement per pool trigger validation and move sessions across pools
>
>
> Key: HIVE-17809
> URL: https://issues.apache.org/jira/browse/HIVE-17809
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-17809.1.patch, HIVE-17809.2.patch,
> HIVE-17809.3.patch, HIVE-17809.4.patch
>
>
> HIVE-17508 trigger validation is applied for all pools at once. This is
> follow up to implement trigger validation at per pool level.
> This should also implement resolution for multiple applicable actions, as per
> the RB discussion
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18012) fix ct_noperm_loc test
[
https://issues.apache.org/jira/browse/HIVE-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-18012:
Attachment: HIVE-18012.02.patch
thanks [~ajisakaa], originally I was just worried about that we are making
something accidentally unsupported. I think the {{create table}} statment
without the column definition was bad that it have been accepted earlier.
> fix ct_noperm_loc test
> --
>
> Key: HIVE-18012
> URL: https://issues.apache.org/jira/browse/HIVE-18012
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-18012.001.patch, HIVE-18012.02.patch
>
>
> the goal of the test is to check that hive doesn't let user1 to create a
> table with a location under an unowned path.
> I've bisected this test to be broken by
> 5250ef450430fcdeed0a2cb7a770f48647987cd3 (HIVE-12408).
> the original exception was (which have been by that sole masked line):
> {code}
> FAILED: HiveAccessControlException Permission denied: Principal [name=user1,
> type=USER] does not have following privileges for operation CREATETABLE
> [[OBJECT OWNERSHIP] on Object [type=DFS_URI,
> name=hdfs://localhost:35753/tmp/ct_noperm_loc_foo0]]
> {code}
> the current semanticexception shouldnt be accepted ; because it's unrelated
> to the tests goal.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17947) Concurrent inserts might fail for ACID table since HIVE-17526 on branch-1
[
https://issues.apache.org/jira/browse/HIVE-17947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-17947:
Resolution: Fixed
Fix Version/s: 1.3.0
Status: Resolved (was: Patch Available)
pushed to branch-1
Thank you [~dvoros] for fixing this and Eugene for the review!
> Concurrent inserts might fail for ACID table since HIVE-17526 on branch-1
> -
>
> Key: HIVE-17947
> URL: https://issues.apache.org/jira/browse/HIVE-17947
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 1.3.0
>Reporter: Daniel Voros
>Assignee: Daniel Voros
>Priority: Blocker
> Fix For: 1.3.0
>
> Attachments: HIVE-17947.1-branch-1.patch,
> HIVE-17947.2-branch-1.patch, HIVE-17947.3-branch-1.patch
>
>
> HIVE-17526 (only on branch-1) disabled conversion to ACID if there are
> *_copy_N files under the table, but the filesystem checks introduced there
> are running for every insert since the MoveTask in the end of the insert will
> call alterTable eventually.
> The filename checking also recurses into staging directories created by other
> inserts. If those are removed while listing the files, it leads to the
> following exception and failing insert:
> {code}
> java.io.FileNotFoundException: File
> hdfs://mycluster/apps/hive/warehouse/dvoros.db/concurrent_insert/.hive-staging_hive_2017-10-30_13-23-35_056_2844419018556002410-2/-ext-10001
> does not exist.
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.(DistributedFileSystem.java:1081)
> ~[hadoop-hdfs-2.7.3.2.6.3.0-235.jar:?]
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.(DistributedFileSystem.java:1059)
> ~[hadoop-hdfs-2.7.3.2.6.3.0-235.jar:?]
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1004)
> ~[hadoop-hdfs-2.7.3.2.6.3.0-235.jar:?]
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1000)
> ~[hadoop-hdfs-2.7.3.2.6.3.0-235.jar:?]
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listLocatedStatus(DistributedFileSystem.java:1018)
> ~[hadoop-hdfs-2.7.3.2.6.3.0-235.jar:?]
> at
> org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1735)
> ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
> at
> org.apache.hadoop.fs.FileSystem$6.handleFileStat(FileSystem.java:1864)
> ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
> at org.apache.hadoop.fs.FileSystem$6.hasNext(FileSystem.java:1841)
> ~[hadoop-common-2.7.3.2.6.3.0-235.jar:?]
> at
> org.apache.hadoop.hive.metastore.TransactionalValidationListener.containsCopyNFiles(TransactionalValidationListener.java:226)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at
> org.apache.hadoop.hive.metastore.TransactionalValidationListener.handleAlterTableTransactionalProp(TransactionalValidationListener.java:104)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at
> org.apache.hadoop.hive.metastore.TransactionalValidationListener.handle(TransactionalValidationListener.java:63)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at
> org.apache.hadoop.hive.metastore.TransactionalValidationListener.onEvent(TransactionalValidationListener.java:55)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.firePreEvent(HiveMetaStore.java:2478)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_table_core(HiveMetaStore.java:4145)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_table_with_environment_context(HiveMetaStore.java:4117)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at sun.reflect.GeneratedMethodAccessor107.invoke(Unknown Source)
> ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_144]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_144]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> [hive-exec-2.1.0.2.6.3.0-235.jar:2.1.0.2.6.3.0-235]
> at
> com.sun.proxy.$Proxy32.alter_table_with_environment_context(Unknown Source)
> [?:?]
>
[jira] [Updated] (HIVE-17969) Metastore to alter table in batches of partitions when renaming table
[
https://issues.apache.org/jira/browse/HIVE-17969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-17969:
--
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Pushed to master.
Thanks for the patch [~szita]!
> Metastore to alter table in batches of partitions when renaming table
> -
>
> Key: HIVE-17969
> URL: https://issues.apache.org/jira/browse/HIVE-17969
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Adam Szita
>Assignee: Adam Szita
> Fix For: 3.0.0
>
> Attachments: HIVE-17969.0.patch, HIVE-17969.1.patch,
> HIVE-17969.2.patch, batched.png, hive9447OptimizationOnly.png, original.png
>
>
> I'm currently trying to speed up the {{alter table rename to}} feature of
> HMS. The recently submitted change (HIVE-9447) already helps a lot especially
> on Oracle HMS DBs.
> This time I intend to gain throughput independently of DB types by enabling
> HMS to execute this alter table command on batches of partitions (rather than
> 1by1)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18041) Add SORT_QUERY_RESULTS to subquery_multi
[ https://issues.apache.org/jira/browse/HIVE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247317#comment-16247317 ] Zoltan Haindrich commented on HIVE-18041: - I think an order by clause might be also a valid candidate to fix this; SORT_QUERY_RESULTS makes the jenkins test out diffs somewhat unreadable > Add SORT_QUERY_RESULTS to subquery_multi > > > Key: HIVE-18041 > URL: https://issues.apache.org/jira/browse/HIVE-18041 > Project: Hive > Issue Type: Test >Reporter: Rui Li >Priority: Trivial > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18012) fix ct_noperm_loc test
[
https://issues.apache.org/jira/browse/HIVE-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247296#comment-16247296
]
Akira Ajisaka commented on HIVE-18012:
--
Thanks!
bq. I don't entirely agree with the removal of the test; because it seems to me
that the previously working sql statements became unsupported somehow; maybe
the test needs updating?
Yes. It's better to update the test instead of deleting it. I'll review your
patch.
> fix ct_noperm_loc test
> --
>
> Key: HIVE-18012
> URL: https://issues.apache.org/jira/browse/HIVE-18012
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Akira Ajisaka
> Attachments: HIVE-18012.001.patch
>
>
> the goal of the test is to check that hive doesn't let user1 to create a
> table with a location under an unowned path.
> I've bisected this test to be broken by
> 5250ef450430fcdeed0a2cb7a770f48647987cd3 (HIVE-12408).
> the original exception was (which have been by that sole masked line):
> {code}
> FAILED: HiveAccessControlException Permission denied: Principal [name=user1,
> type=USER] does not have following privileges for operation CREATETABLE
> [[OBJECT OWNERSHIP] on Object [type=DFS_URI,
> name=hdfs://localhost:35753/tmp/ct_noperm_loc_foo0]]
> {code}
> the current semanticexception shouldnt be accepted ; because it's unrelated
> to the tests goal.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18012) fix ct_noperm_loc test
[
https://issues.apache.org/jira/browse/HIVE-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka reassigned HIVE-18012:
Assignee: Zoltan Haindrich (was: Akira Ajisaka)
> fix ct_noperm_loc test
> --
>
> Key: HIVE-18012
> URL: https://issues.apache.org/jira/browse/HIVE-18012
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-18012.001.patch
>
>
> the goal of the test is to check that hive doesn't let user1 to create a
> table with a location under an unowned path.
> I've bisected this test to be broken by
> 5250ef450430fcdeed0a2cb7a770f48647987cd3 (HIVE-12408).
> the original exception was (which have been by that sole masked line):
> {code}
> FAILED: HiveAccessControlException Permission denied: Principal [name=user1,
> type=USER] does not have following privileges for operation CREATETABLE
> [[OBJECT OWNERSHIP] on Object [type=DFS_URI,
> name=hdfs://localhost:35753/tmp/ct_noperm_loc_foo0]]
> {code}
> the current semanticexception shouldnt be accepted ; because it's unrelated
> to the tests goal.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17809) Implement per pool trigger validation and move sessions across pools
[ https://issues.apache.org/jira/browse/HIVE-17809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-17809: - Attachment: HIVE-17809.4.patch > Implement per pool trigger validation and move sessions across pools > > > Key: HIVE-17809 > URL: https://issues.apache.org/jira/browse/HIVE-17809 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-17809.1.patch, HIVE-17809.2.patch, > HIVE-17809.3.patch, HIVE-17809.4.patch > > > HIVE-17508 trigger validation is applied for all pools at once. This is > follow up to implement trigger validation at per pool level. > This should also implement resolution for multiple applicable actions, as per > the RB discussion -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17809) Implement per pool trigger validation and move sessions across pools
[ https://issues.apache.org/jira/browse/HIVE-17809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-17809: - Attachment: (was: HIVE-17809.4.patch) > Implement per pool trigger validation and move sessions across pools > > > Key: HIVE-17809 > URL: https://issues.apache.org/jira/browse/HIVE-17809 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-17809.1.patch, HIVE-17809.2.patch, > HIVE-17809.3.patch, HIVE-17809.4.patch > > > HIVE-17508 trigger validation is applied for all pools at once. This is > follow up to implement trigger validation at per pool level. > This should also implement resolution for multiple applicable actions, as per > the RB discussion -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17809) Implement per pool trigger validation and move sessions across pools
[ https://issues.apache.org/jira/browse/HIVE-17809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-17809: - Attachment: HIVE-17809.4.patch Addressed review comments > Implement per pool trigger validation and move sessions across pools > > > Key: HIVE-17809 > URL: https://issues.apache.org/jira/browse/HIVE-17809 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-17809.1.patch, HIVE-17809.2.patch, > HIVE-17809.3.patch, HIVE-17809.4.patch > > > HIVE-17508 trigger validation is applied for all pools at once. This is > follow up to implement trigger validation at per pool level. > This should also implement resolution for multiple applicable actions, as per > the RB discussion -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18030) HCatalog can't be used with Pig on Spark
[
https://issues.apache.org/jira/browse/HIVE-18030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247219#comment-16247219
]
liyunzhang commented on HIVE-18030:
---
[~szita]: my question is same as Xuefu, can we set {{mapred.task.id}} in pig to
bypass the problem? If not ,can you tell me the reason, tks!
> HCatalog can't be used with Pig on Spark
>
>
> Key: HIVE-18030
> URL: https://issues.apache.org/jira/browse/HIVE-18030
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Reporter: Adam Szita
>Assignee: Adam Szita
> Attachments: HIVE-18030.0.patch
>
>
> When using Pig on Spark in cluster mode, all queries containing HCatalog
> access are failing:
> {code}
> 2017-11-03 12:39:19,268 [dispatcher-event-loop-19] INFO
> org.apache.spark.storage.BlockManagerInfo - Added broadcast_6_piece0 in
> memory on <>:<> (size: 83.0 KB, free: 408.5
> MB)
> 2017-11-03 12:39:19,277 [task-result-getter-0] WARN
> org.apache.spark.scheduler.TaskSetManager - Lost task 0.0 in stage 0.0 (TID
> 0, <>, executor 2): java.lang.NullPointerException
> at org.apache.hadoop.security.Credentials.addAll(Credentials.java:401)
> at org.apache.hadoop.security.Credentials.addAll(Credentials.java:388)
> at
> org.apache.hive.hcatalog.pig.HCatLoader.setLocation(HCatLoader.java:128)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.mergeSplitSpecificConf(PigInputFormat.java:147)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat$RecordReaderFactory.(PigInputFormat.java:115)
> at
> org.apache.pig.backend.hadoop.executionengine.spark.running.PigInputFormatSpark$SparkRecordReaderFactory.(PigInputFormatSpark.java:126)
> at
> org.apache.pig.backend.hadoop.executionengine.spark.running.PigInputFormatSpark.createRecordReader(PigInputFormatSpark.java:70)
> at
> org.apache.spark.rdd.NewHadoopRDD$$anon$1.liftedTree1$1(NewHadoopRDD.scala:180)
> at
> org.apache.spark.rdd.NewHadoopRDD$$anon$1.(NewHadoopRDD.scala:179)
> at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:134)
> at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:69)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
> at org.apache.spark.scheduler.Task.run(Task.scala:108)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:338)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HIVE-17931) Implement Parquet vectorization reader for Array type
[ https://issues.apache.org/jira/browse/HIVE-17931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246862#comment-16246862 ] Colin Ma edited comment on HIVE-17931 at 11/10/17 8:24 AM: --- [~vihangk1], I'm afraid complex types is not fully supported, and I got the problem because of the following code: [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java#L3261] The List and Map are not supported, and the MapWork can't be vectorized with these types. I created HIVE-18043 to track this problem. was (Author: colinma): [~vihangk1], I'm afraid complex types is not fully supported, and I got the problem because of the following code: [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java#L3261] The List and Map are not supported, and the MapWork can't be vectorized with these types. > Implement Parquet vectorization reader for Array type > - > > Key: HIVE-17931 > URL: https://issues.apache.org/jira/browse/HIVE-17931 > Project: Hive > Issue Type: Sub-task >Reporter: Colin Ma >Assignee: Colin Ma > Attachments: HIVE-17931.001.patch, HIVE-17931.002.patch, > HIVE-17931.003.patch, HIVE-17931.004.patch > > > Parquet vectorized reader can't support array type, it should be supported to > improve the performance when the query with array type. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-18043) Vectorization: Support List type in MapWork
[ https://issues.apache.org/jira/browse/HIVE-18043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Ma reassigned HIVE-18043: --- > Vectorization: Support List type in MapWork > --- > > Key: HIVE-18043 > URL: https://issues.apache.org/jira/browse/HIVE-18043 > Project: Hive > Issue Type: Improvement >Reporter: Colin Ma >Assignee: Colin Ma > > Support Complex Types in vectorization is finished in HIVE-16589, but List > type is still not support in MapWork. It should be supported to improve the > performance when vectorization is enable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18042) Correlation Optimizer lead to NPE when there is multi union all operation after join
[
https://issues.apache.org/jira/browse/HIVE-18042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hengyu Dai updated HIVE-18042:
--
Summary: Correlation Optimizer lead to NPE when there is multi union all
operation after join (was: Correlation Optimizer NPE when there is multi
union all operation after join )
> Correlation Optimizer lead to NPE when there is multi union all operation
> after join
> -
>
> Key: HIVE-18042
> URL: https://issues.apache.org/jira/browse/HIVE-18042
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Affects Versions: 2.1.1
> Environment:
>Reporter: Hengyu Dai
>
> test sql:
> {code:sql}
> SELECT DISTINCT a.logday AS push_day, a.mtype, a.t, If(b.msgid IS NULL, 'no',
> 'yes') AS isnotdaoda, a.platform
> , a.uid, a.dt
> FROM (SELECT DISTINCT If(tokentype = '7', msgid, If(tokentype = '6',
> regexp_extract(sendpushresult, 'msgId":"([^"]+)', 1),
> regexp_extract(sendpushresult, 'msgId=(.+?),', 1))) AS msgid, logday, If(vid
> LIKE '60%', 'adr', If(vid LIKE '8%', 'ios', 'other')) AS platform, mtype, t
> , If(vid LIKE '8%', uid, gid) AS uid, concat(substr(logday, 1, 4),
> '-', substr(logday, 5, 2), '-', substr(logday, 7, 2)) AS dt
> FROM wirelessdata.orig_push_client
> ) a
> LEFT JOIN (SELECT DISTINCT msgid
> FROM (
> SELECT DISTINCT msgid
> FROM wirelessdata.orig_push_return
> UNION ALL
> SELECT DISTINCT msgid
> FROM wirelessdata.orig_push_return_xiaomi
> UNION ALL
> SELECT DISTINCT regexp_extract(action, '"id"："([^"]+)', 1) AS
> msgid
> FROM wirelessdata.ods_client_behavior_hour4spark
> ) bb
> ) b ON lower(a.msgid) = lower(b.msgid)
> {code}
> the error stack
> {code:java}
> 2017-11-10T16:01:21,123 ERROR [9b7d82f5-dfc8-43ac-8d6f-a019d8677392 main]
> ql.Driver: FAILED: NullPointerException null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.optimizer.GenMapRedUtils.setUnionPlan(GenMapRedUtils.java:230)
> at
> org.apache.hadoop.hive.ql.optimizer.GenMapRedUtils.joinUnionPlan(GenMapRedUtils.java:287)
> at
> org.apache.hadoop.hive.ql.optimizer.GenMRRedSink3.process(GenMRRedSink3.java:100)
> at
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
> at
> org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:54)
> at
> org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:65)
> at
> org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:65)
> at
> org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:65)
> at
> org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:65)
> at
> org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:65)
> at
> org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:65)
> at
> org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:65)
> at
> org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:65)
> at
> org.apache.hadoop.hive.ql.parse.GenMapRedWalker.walk(GenMapRedWalker.java:65)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:120)
> at
> org.apache.hadoop.hive.ql.parse.MapReduceCompiler.generateTaskTree(MapReduceCompiler.java:323)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:267)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11008)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10547)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:250)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:483)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1254)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1396)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1181)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:229)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:180)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:396)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:770)
> at
