[jira] [Updated] (HIVE-1010) Implement INFORMATION_SCHEMA in Hive
[ https://issues.apache.org/jira/browse/HIVE-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-1010: - Status: Patch Available (was: Open) > Implement INFORMATION_SCHEMA in Hive > > > Key: HIVE-1010 > URL: https://issues.apache.org/jira/browse/HIVE-1010 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Jeff Hammerbacher >Assignee: Gunther Hagleitner > Attachments: HIVE-1010.1.patch, HIVE-1010.2.patch, HIVE-1010.3.patch, > HIVE-1010.4.patch, HIVE-1010.5.patch > > > INFORMATION_SCHEMA is part of the SQL92 standard and would be useful to > implement using our metastore. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15830) Allow additional view ACLs for tez jobs
[
https://issues.apache.org/jira/browse/HIVE-15830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873039#comment-15873039
]
Hive QA commented on HIVE-15830:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853394/HIVE-15830.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 10246 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=140)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31]
(batchId=133)
org.apache.hive.common.util.TestACLConfigurationParser.test (batchId=237)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3643/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3643/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3643/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853394 - PreCommit-HIVE-Build
> Allow additional view ACLs for tez jobs
> ---
>
> Key: HIVE-15830
> URL: https://issues.apache.org/jira/browse/HIVE-15830
> Project: Hive
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Attachments: HIVE-15830.01.patch, HIVE-15830.02.patch,
> HIVE-15830.03.patch
>
>
> Allow users to grant view access to additional users when running tez jobs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-1010) Implement INFORMATION_SCHEMA in Hive
[ https://issues.apache.org/jira/browse/HIVE-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-1010: - Attachment: HIVE-1010.5.patch > Implement INFORMATION_SCHEMA in Hive > > > Key: HIVE-1010 > URL: https://issues.apache.org/jira/browse/HIVE-1010 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Jeff Hammerbacher >Assignee: Gunther Hagleitner > Attachments: HIVE-1010.1.patch, HIVE-1010.2.patch, HIVE-1010.3.patch, > HIVE-1010.4.patch, HIVE-1010.5.patch > > > INFORMATION_SCHEMA is part of the SQL92 standard and would be useful to > implement using our metastore. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15971) LLAP: logs urls should use daemon container id instead of fake container id
[
https://issues.apache.org/jira/browse/HIVE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873019#comment-15873019
]
Hive QA commented on HIVE-15971:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853390/HIVE-15971.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 10231 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=140)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=133)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=211)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3642/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3642/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3642/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853390 - PreCommit-HIVE-Build
> LLAP: logs urls should use daemon container id instead of fake container id
> ---
>
> Key: HIVE-15971
> URL: https://issues.apache.org/jira/browse/HIVE-15971
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 2.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-15971.1.patch, HIVE-15971.2.patch
>
>
> The containerId used for log url generation is fake. It should be replaced by
> the container id of the llap daemon.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15970) Fix merge to work in presence of AST rewrites
[ https://issues.apache.org/jira/browse/HIVE-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873014#comment-15873014 ] Wei Zheng commented on HIVE-15970: -- +1 > Fix merge to work in presence of AST rewrites > - > > Key: HIVE-15970 > URL: https://issues.apache.org/jira/browse/HIVE-15970 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-15970.01.patch, HIVE-15970.02.patch > > > The issue is that Context.getDestNamePrefix() caches ASTNode objects to > figure out if a given insert branch of multi-insert is insert, update or > delete but objects change when the tree is modified. For example, > row-filtering/column-masking does this, but also other optimizations/features. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15938) position alias in order by fails for union queries
[
https://issues.apache.org/jira/browse/HIVE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873012#comment-15873012
]
Hive QA commented on HIVE-15938:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853384/HIVE-15938.04.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10246 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31]
(batchId=133)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3641/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3641/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3641/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853384 - PreCommit-HIVE-Build
> position alias in order by fails for union queries
> --
>
> Key: HIVE-15938
> URL: https://issues.apache.org/jira/browse/HIVE-15938
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-15938.01.patch, HIVE-15938.02.patch,
> HIVE-15938.03.patch, HIVE-15938.04.patch, HIVE-15938.patch
>
>
> The problem is that the query introduces a spurious ALLCOLREF
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15938) position alias in order by fails for union queries
[
https://issues.apache.org/jira/browse/HIVE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872989#comment-15872989
]
Hive QA commented on HIVE-15938:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853384/HIVE-15938.04.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 10246 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=140)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31]
(batchId=133)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=211)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=211)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3640/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3640/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3640/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853384 - PreCommit-HIVE-Build
> position alias in order by fails for union queries
> --
>
> Key: HIVE-15938
> URL: https://issues.apache.org/jira/browse/HIVE-15938
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-15938.01.patch, HIVE-15938.02.patch,
> HIVE-15938.03.patch, HIVE-15938.04.patch, HIVE-15938.patch
>
>
> The problem is that the query introduces a spurious ALLCOLREF
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15955) make explain formatted to dump JSONObject when user level explain is on
[ https://issues.apache.org/jira/browse/HIVE-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872985#comment-15872985 ] Gunther Hagleitner commented on HIVE-15955: --- [~pxiong]. (1) Still doesn't answer the first question. Can you please make it so that the operator id is always included? I *believe* you're saying it's dependent on the user flag. (2) If you turn on hive.auto.convert.sortmerge.join on tez, you can produce operations that will result in multi operator reduce tasks. Basically if you have join of two group bys (joined on the group by keys), tez can pack the 2 group bys and the join in the same vertex. Admittedly this is off by default, but nevertheless it is a real possibility. That's why I'm suggesting that if it is easy it might be more direct to connect operators. > make explain formatted to dump JSONObject when user level explain is on > --- > > Key: HIVE-15955 > URL: https://issues.apache.org/jira/browse/HIVE-15955 > Project: Hive > Issue Type: New Feature >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-15955.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15971) LLAP: logs urls should use daemon container id instead of fake container id
[
https://issues.apache.org/jira/browse/HIVE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872979#comment-15872979
]
Hive QA commented on HIVE-15971:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853390/HIVE-15971.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10245 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=140)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31]
(batchId=133)
org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteDate2 (batchId=173)
org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteDecimalXY
(batchId=173)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3639/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3639/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3639/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853390 - PreCommit-HIVE-Build
> LLAP: logs urls should use daemon container id instead of fake container id
> ---
>
> Key: HIVE-15971
> URL: https://issues.apache.org/jira/browse/HIVE-15971
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 2.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-15971.1.patch, HIVE-15971.2.patch
>
>
> The containerId used for log url generation is fake. It should be replaced by
> the container id of the llap daemon.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15846) Relocate more dependencies (e.g. org.apache.zookeeper) for JDBC uber jar
[
https://issues.apache.org/jira/browse/HIVE-15846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872950#comment-15872950
]
Hive QA commented on HIVE-15846:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853375/HIVE-15846.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10245 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31]
(batchId=133)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3638/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3638/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3638/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853375 - PreCommit-HIVE-Build
> Relocate more dependencies (e.g. org.apache.zookeeper) for JDBC uber jar
>
>
> Key: HIVE-15846
> URL: https://issues.apache.org/jira/browse/HIVE-15846
> Project: Hive
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-15846.1.patch, HIVE-15846.2.patch,
> HIVE-15846.3.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-15964) LLAP: Llap IO codepath not getting invoked due to file column id mismatch
[ https://issues.apache.org/jira/browse/HIVE-15964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-15964: Attachment: HIVE-15964.2.patch Thanks [~sershe]. Attaching .2 version. Removed isAcidScan and added TODO for acid. Added llapreader.q.out for testclidriver. > LLAP: Llap IO codepath not getting invoked due to file column id mismatch > - > > Key: HIVE-15964 > URL: https://issues.apache.org/jira/browse/HIVE-15964 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-15964.1.patch, HIVE-15964.2.patch > > > LLAP IO codepath is not getting invoked in certain cases when schema > evolution checks are done. Though "int --> long" (fileType to readerType) > conversions are allowed, the file type columns are not matched correctly when > such conversions need to happen. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15903) Compute table stats when user computes column stats
[ https://issues.apache.org/jira/browse/HIVE-15903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872928#comment-15872928 ] Pengcheng Xiong commented on HIVE-15903: [~ashutoshc], could u take a look? Thanks. > Compute table stats when user computes column stats > --- > > Key: HIVE-15903 > URL: https://issues.apache.org/jira/browse/HIVE-15903 > Project: Hive > Issue Type: Bug >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-15903.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15903) Compute table stats when user computes column stats
[ https://issues.apache.org/jira/browse/HIVE-15903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-15903: --- Attachment: HIVE-15903.01.patch > Compute table stats when user computes column stats > --- > > Key: HIVE-15903 > URL: https://issues.apache.org/jira/browse/HIVE-15903 > Project: Hive > Issue Type: Bug >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-15903.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15903) Compute table stats when user computes column stats
[ https://issues.apache.org/jira/browse/HIVE-15903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-15903: --- Status: Patch Available (was: Open) > Compute table stats when user computes column stats > --- > > Key: HIVE-15903 > URL: https://issues.apache.org/jira/browse/HIVE-15903 > Project: Hive > Issue Type: Bug >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-15903.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15953) better error messages for LLAP registry properties
[
https://issues.apache.org/jira/browse/HIVE-15953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872913#comment-15872913
]
Hive QA commented on HIVE-15953:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853371/HIVE-15953.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 22 failed/errored test(s), 10245 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acid_part_update]
(batchId=151)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join6]
(batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join8]
(batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketmapjoin1]
(batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketmapjoin5]
(batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby2_map_skew]
(batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby7_noskew]
(batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join28]
(batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31]
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[lateral_view_explode2]
(batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[load_dyn_part1]
(batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt17]
(batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multiinsert]
(batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_20]
(batchId=131)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=211)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=211)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3637/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3637/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3637/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 22 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853371 - PreCommit-HIVE-Build
> better error messages for LLAP registry properties
> --
>
> Key: HIVE-15953
> URL: https://issues.apache.org/jira/browse/HIVE-15953
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-15953.patch
>
>
> {noformat}
> int memory =
> Integer.parseInt(srv.get(ConfVars.LLAP_DAEMON_MEMORY_PER_INSTANCE_MB.varname));
> int vCores =
> Integer.parseInt(srv.get(ConfVars.LLAP_DAEMON_NUM_EXECUTORS.varname));
> {noformat}
> Needs to provide a useful error message when it fails because the config is
> bad. There may be other similar places.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15970) Fix merge to work in presence of AST rewrites
[ https://issues.apache.org/jira/browse/HIVE-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872898#comment-15872898 ] Eugene Koifman commented on HIVE-15970: --- no, we don't. Context.operation explicitly tells us if it's Merge. or update or delete > Fix merge to work in presence of AST rewrites > - > > Key: HIVE-15970 > URL: https://issues.apache.org/jira/browse/HIVE-15970 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-15970.01.patch, HIVE-15970.02.patch > > > The issue is that Context.getDestNamePrefix() caches ASTNode objects to > figure out if a given insert branch of multi-insert is insert, update or > delete but objects change when the tree is modified. For example, > row-filtering/column-masking does this, but also other optimizations/features. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15970) Fix merge to work in presence of AST rewrites
[ https://issues.apache.org/jira/browse/HIVE-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872897#comment-15872897 ] Eugene Koifman commented on HIVE-15970: --- no related failures; the 2 "new" test failures also failed in https://builds.apache.org/job/PreCommit-HIVE-Build/3634/#showFailuresLink > Fix merge to work in presence of AST rewrites > - > > Key: HIVE-15970 > URL: https://issues.apache.org/jira/browse/HIVE-15970 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-15970.01.patch, HIVE-15970.02.patch > > > The issue is that Context.getDestNamePrefix() caches ASTNode objects to > figure out if a given insert branch of multi-insert is insert, update or > delete but objects change when the tree is modified. For example, > row-filtering/column-masking does this, but also other optimizations/features. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-1010) Implement INFORMATION_SCHEMA in Hive
[ https://issues.apache.org/jira/browse/HIVE-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-1010: - Attachment: HIVE-1010.4.patch > Implement INFORMATION_SCHEMA in Hive > > > Key: HIVE-1010 > URL: https://issues.apache.org/jira/browse/HIVE-1010 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Jeff Hammerbacher >Assignee: Gunther Hagleitner > Attachments: HIVE-1010.1.patch, HIVE-1010.2.patch, HIVE-1010.3.patch, > HIVE-1010.4.patch > > > INFORMATION_SCHEMA is part of the SQL92 standard and would be useful to > implement using our metastore. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15970) Fix merge to work in presence of AST rewrites
[
https://issues.apache.org/jira/browse/HIVE-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872880#comment-15872880
]
Wei Zheng commented on HIVE-15970:
--
So in Context.getDestNamePrefix, if curNode is TOK_INSERT_INTO, we assume it's
part of MERGE?
nit: In Context.java, comment need to be fixed
{code}
* (not that Insert into Acid table is in OTHER category
{code}
> Fix merge to work in presence of AST rewrites
> -
>
> Key: HIVE-15970
> URL: https://issues.apache.org/jira/browse/HIVE-15970
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 2.2.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Critical
> Attachments: HIVE-15970.01.patch, HIVE-15970.02.patch
>
>
> The issue is that Context.getDestNamePrefix() caches ASTNode objects to
> figure out if a given insert branch of multi-insert is insert, update or
> delete but objects change when the tree is modified. For example,
> row-filtering/column-masking does this, but also other optimizations/features.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-15796) HoS: poor reducer parallelism when operator stats are not accurate
[ https://issues.apache.org/jira/browse/HIVE-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-15796: Attachment: HIVE-15796.4.patch Adding qtest. > HoS: poor reducer parallelism when operator stats are not accurate > -- > > Key: HIVE-15796 > URL: https://issues.apache.org/jira/browse/HIVE-15796 > Project: Hive > Issue Type: Improvement > Components: Statistics >Affects Versions: 2.2.0 >Reporter: Chao Sun >Assignee: Chao Sun > Attachments: HIVE-15796.1.patch, HIVE-15796.2.patch, > HIVE-15796.3.patch, HIVE-15796.4.patch, HIVE-15796.wip.1.patch, > HIVE-15796.wip.2.patch, HIVE-15796.wip.patch > > > In HoS we use currently use operator stats to determine reducer parallelism. > However, it is often the case that operator stats are not accurate, > especially if column stats are not available. This sometimes will generate > extremely poor reducer parallelism, and cause HoS query to run forever. > This JIRA tries to offer an alternative way to compute reducer parallelism, > similar to how MR does. Here's the approach we are suggesting: > 1. when computing the parallelism for a MapWork, use stats associated with > the TableScan operator; > 2. when computing the parallelism for a ReduceWork, use the *maximum* > parallelism from all its parents. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15970) Fix merge to work in presence of AST rewrites
[
https://issues.apache.org/jira/browse/HIVE-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872853#comment-15872853
]
Hive QA commented on HIVE-15970:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853372/HIVE-15970.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10245 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=140)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31]
(batchId=133)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=211)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3636/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3636/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3636/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853372 - PreCommit-HIVE-Build
> Fix merge to work in presence of AST rewrites
> -
>
> Key: HIVE-15970
> URL: https://issues.apache.org/jira/browse/HIVE-15970
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 2.2.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Critical
> Attachments: HIVE-15970.01.patch, HIVE-15970.02.patch
>
>
> The issue is that Context.getDestNamePrefix() caches ASTNode objects to
> figure out if a given insert branch of multi-insert is insert, update or
> delete but objects change when the tree is modified. For example,
> row-filtering/column-masking does this, but also other optimizations/features.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15891) Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
[ https://issues.apache.org/jira/browse/HIVE-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872851#comment-15872851 ] Eugene Koifman commented on HIVE-15891: --- +1 > Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast > --- > > Key: HIVE-15891 > URL: https://issues.apache.org/jira/browse/HIVE-15891 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Attachments: HIVE-15891.1.patch, HIVE-15891.2.patch, > HIVE-15891.3.patch > > > Currently ACID UpdateDeleteSemanticAnalyzer directly manipulates the AST tree > but it's different from the general approach of modifying the token stream > and thus will cause AST tree mismatch if there is any rewrite happening after > UpdateDeleteSemanticAnalyzer. > The long term solution will be to rewrite the AST handling logic in > UpdateDeleteSemanticAnalyzer, to make it consistent with the general approach. > This ticket will for now detect the error prone cases and fail early. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15972) Runtime filtering not vectorizing for decimal/timestamp/char/varchar
[ https://issues.apache.org/jira/browse/HIVE-15972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-15972: -- Attachment: HIVE-15972.1.patch > Runtime filtering not vectorizing for decimal/timestamp/char/varchar > > > Key: HIVE-15972 > URL: https://issues.apache.org/jira/browse/HIVE-15972 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-15972.1.patch > > > Looks like versions of vectorized BetweenDynamicValue that use Java objects > needs to be initialized with non-null values -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15972) Runtime filtering not vectorizing for decimal/timestamp/char/varchar
[ https://issues.apache.org/jira/browse/HIVE-15972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-15972: -- Status: Patch Available (was: Open) > Runtime filtering not vectorizing for decimal/timestamp/char/varchar > > > Key: HIVE-15972 > URL: https://issues.apache.org/jira/browse/HIVE-15972 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-15972.1.patch > > > Looks like versions of vectorized BetweenDynamicValue that use Java objects > needs to be initialized with non-null values -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15830) Allow additional view ACLs for tez jobs
[ https://issues.apache.org/jira/browse/HIVE-15830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-15830: -- Attachment: HIVE-15830.03.patch Updated again. New behaviour. Use tez properties to determine the initial ACL set. Add the submitting user. Add the hs2/proc owner if the config property is set. The same tez property - for AM acls - is re-used to set dag ACLs. cc [~thejas], [~daijy] > Allow additional view ACLs for tez jobs > --- > > Key: HIVE-15830 > URL: https://issues.apache.org/jira/browse/HIVE-15830 > Project: Hive > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15830.01.patch, HIVE-15830.02.patch, > HIVE-15830.03.patch > > > Allow users to grant view access to additional users when running tez jobs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (HIVE-15972) Runtime filtering not vectorizing for decimal/timestamp/char/varchar
[ https://issues.apache.org/jira/browse/HIVE-15972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere reassigned HIVE-15972: - > Runtime filtering not vectorizing for decimal/timestamp/char/varchar > > > Key: HIVE-15972 > URL: https://issues.apache.org/jira/browse/HIVE-15972 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Jason Dere >Assignee: Jason Dere > > Looks like versions of vectorized BetweenDynamicValue that use Java objects > needs to be initialized with non-null values -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15951) Make sure base persist directory is unique and deleted
[ https://issues.apache.org/jira/browse/HIVE-15951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872832#comment-15872832 ] Ashutosh Chauhan commented on HIVE-15951: - deleteOnExit() is called when process exists. For container mode, process will terminate immediately and clean up will happen. For llap, process won't exit once RecordWriter is done, since its long running process waiting to execute next query. So, clean up wont happen until llap deamon goes down, which likely is not useful. > Make sure base persist directory is unique and deleted > -- > > Key: HIVE-15951 > URL: https://issues.apache.org/jira/browse/HIVE-15951 > Project: Hive > Issue Type: Bug > Components: Druid integration >Affects Versions: 2.2.0 >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Critical > Fix For: 2.2.0 > > Attachments: HIVE-15951.patch > > > In some cases the base persist directory will contain old data or shared > between reducer in the same physical VM. > That will lead to the failure of the job till that the directory is cleaned. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15938) position alias in order by fails for union queries
[ https://issues.apache.org/jira/browse/HIVE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872822#comment-15872822 ] Pengcheng Xiong commented on HIVE-15938: Thanks for [~sershe]'s efforts. The patch LGTM. I just have one doubt. If you try, you will find out that postgres actually supports select * from t order by 1. I am thinking that after HIVE-15160, supporting order by number in select * would be straight forward. > position alias in order by fails for union queries > -- > > Key: HIVE-15938 > URL: https://issues.apache.org/jira/browse/HIVE-15938 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15938.01.patch, HIVE-15938.02.patch, > HIVE-15938.03.patch, HIVE-15938.04.patch, HIVE-15938.patch > > > The problem is that the query introduces a spurious ALLCOLREF -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15959) LLAP: fix headroom calculation and move it to daemon
[
https://issues.apache.org/jira/browse/HIVE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872804#comment-15872804
]
Hive QA commented on HIVE-15959:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853369/HIVE-15959.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10245 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31]
(batchId=133)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3635/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3635/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3635/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853369 - PreCommit-HIVE-Build
> LLAP: fix headroom calculation and move it to daemon
>
>
> Key: HIVE-15959
> URL: https://issues.apache.org/jira/browse/HIVE-15959
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-15959.01.patch, HIVE-15959.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15951) Make sure base persist directory is unique and deleted
[ https://issues.apache.org/jira/browse/HIVE-15951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872796#comment-15872796 ] slim bouguerra commented on HIVE-15951: --- [~ashutoshc] i am not sure i am getting this, is it the delete on exit that won't work ? > Make sure base persist directory is unique and deleted > -- > > Key: HIVE-15951 > URL: https://issues.apache.org/jira/browse/HIVE-15951 > Project: Hive > Issue Type: Bug > Components: Druid integration >Affects Versions: 2.2.0 >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Critical > Fix For: 2.2.0 > > Attachments: HIVE-15951.patch > > > In some cases the base persist directory will contain old data or shared > between reducer in the same physical VM. > That will lead to the failure of the job till that the directory is cleaned. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15255) LLAP: service_busy error should not be retried so fast
[
https://issues.apache.org/jira/browse/HIVE-15255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872776#comment-15872776
]
Sergey Shelukhin commented on HIVE-15255:
-
Looking at the disable/enable code, this should have worked. However, I wonder
if scheduling takes into account disabled nodes correctly? I cannot find
special processing for service busy.
> LLAP: service_busy error should not be retried so fast
> --
>
> Key: HIVE-15255
> URL: https://issues.apache.org/jira/browse/HIVE-15255
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>
> {noformat}
> 2016-11-18 20:28:20,605 FINISHED]: vertexName=Map 1,
> taskAttemptId=attempt_1478967587833_2622_1_06_000105_1328, timeTaken=5,
> status=KILLED, errorEnum=SERVICE_BUSY, diagnostics=Service Busy,
> nodeHttpAddress=(node3), counters=Counters: 1,
> org.apache.tez.common.counters.DAGCounter, DATA_LOCAL_TASKS=1
> 2016-11-18 20:28:20,612 STARTED]: vertexName=Map 1,
> taskAttemptId=attempt_1478967587833_2622_1_06_000105_1329,
> containerId=container_1_2622_01_012504, nodeId=(node3):15001
> 2016-11-18 20:28:20,628 FINISHED]: vertexName=Map 1,
> taskAttemptId=attempt_1478967587833_2622_1_06_000105_1329, timeTaken=16,
> status=KILLED, errorEnum=SERVICE_BUSY, diagnostics=Service Busy,
> nodeHttpAddress=(node3), counters=Counters: 1,
> org.apache.tez.common.counters.DAGCounter, DATA_LOCAL_TASKS=1
> 2016-11-18 20:28:20,634 STARTED]: vertexName=Map 1,
> taskAttemptId=attempt_1478967587833_2622_1_06_000105_1330,
> containerId=container_1_2622_01_012511, nodeId=(node3):15001
> 2016-11-18 20:28:20,751 FINISHED]: vertexName=Map 1,
> taskAttemptId=attempt_1478967587833_2622_1_06_000105_1330, timeTaken=117,
> status=KILLED, errorEnum=SERVICE_BUSY, diagnostics=Service Busy,
> nodeHttpAddress=(node3), counters=Counters: 1,
> org.apache.tez.common.counters.DAGCounter, DATA_LOCAL_TASKS=1
> 2016-11-18 20:28:20,757 STARTED]: vertexName=Map 1,
> taskAttemptId=attempt_1478967587833_2622_1_06_000105_1331,
> containerId=container_1_2622_01_012522, nodeId=(node3):15001
> 2016-11-18 20:28:20,771 FINISHED]: vertexName=Map 1,
> taskAttemptId=attempt_1478967587833_2622_1_06_000105_1331, timeTaken=14,
> status=KILLED, errorEnum=SERVICE_BUSY, diagnostics=Service Busy,
> nodeHttpAddress=(node3), counters=Counters: 1,
> org.apache.tez.common.counters.DAGCounter, DATA_LOCAL_TASKS=1
> 2016-11-18 20:28:20,777 STARTED]: vertexName=Map 1,
> taskAttemptId=attempt_1478967587833_2622_1_06_000105_1332,
> containerId=container_1_2622_01_012529, nodeId=(node3):15001
> 2016-11-18 20:28:20,783 FINISHED]: vertexName=Map 1,
> taskAttemptId=attempt_1478967587833_2622_1_06_000105_1332, timeTaken=6,
> status=KILLED, errorEnum=SERVICE_BUSY, diagnostics=Service Busy,
> nodeHttpAddress=(node3), counters=Counters: 1,
> org.apache.tez.common.counters.DAGCounter, DATA_LOCAL_TASKS=1
> {noformat}
> As you can see by the attempt number, this has been going on for a while. In
> fact I think other tasks could have been scheduled in the time (not sure),
> but the thread just kept at it for this one task until it was finally
> scheduled.
> There should be some fallback after initial failures; we should also make
> sure such retries do not take over all scheduling (not sure if they do, need
> to check).
> LLAP on the node was alive, just busy with other tasks. The task did
> eventually get scheduled.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15971) LLAP: logs urls should use daemon container id instead of fake container id
[ https://issues.apache.org/jira/browse/HIVE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872764#comment-15872764 ] Prasanth Jayachandran commented on HIVE-15971: -- Addressed review comments. Also added nm.id parameter to the url which is required if we are accessing logs from running containers which is mostly the case with LLAP. [~sershe] can you please take another look? > LLAP: logs urls should use daemon container id instead of fake container id > --- > > Key: HIVE-15971 > URL: https://issues.apache.org/jira/browse/HIVE-15971 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-15971.1.patch, HIVE-15971.2.patch > > > The containerId used for log url generation is fake. It should be replaced by > the container id of the llap daemon. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15971) LLAP: logs urls should use daemon container id instead of fake container id
[ https://issues.apache.org/jira/browse/HIVE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-15971: - Attachment: HIVE-15971.2.patch > LLAP: logs urls should use daemon container id instead of fake container id > --- > > Key: HIVE-15971 > URL: https://issues.apache.org/jira/browse/HIVE-15971 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-15971.1.patch, HIVE-15971.2.patch > > > The containerId used for log url generation is fake. It should be replaced by > the container id of the llap daemon. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15971) LLAP: logs urls should use daemon container id instead of fake container id
[ https://issues.apache.org/jira/browse/HIVE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-15971: - Status: Patch Available (was: Open) > LLAP: logs urls should use daemon container id instead of fake container id > --- > > Key: HIVE-15971 > URL: https://issues.apache.org/jira/browse/HIVE-15971 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-15971.1.patch, HIVE-15971.2.patch > > > The containerId used for log url generation is fake. It should be replaced by > the container id of the llap daemon. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-14901) HiveServer2: Use user supplied fetch size to determine #rows serialized in tasks
[
https://issues.apache.org/jira/browse/HIVE-14901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872755#comment-15872755
]
Vaibhav Gumashta commented on HIVE-14901:
-
[~norrisl] Thanks for the patch. My feedback:
# Address the case when thrift serde is not used (basically have an upper bound
check in the fetch call). This is to prevent OOMs on server if a user passes an
abnormally large value.
# For passing the user value to thrift serde, check HiveSessionImpl#
configureSession.
# Small nit: the name of HIVE_SERVER2_RESULTSET_DEFAULT_FETCH_SIZE/
hive.server2.resultset.default.fetch.size to be consistent with the other
params. I think this won't be backward incompat as this hasn't gone into any
release yet (but a quick check on the released 2.1 line would be great).
# Log the MAX_BUFFERED_ROWS in serde at INFO level.
# You might want to think of additional tests if you can to test these changes
and if they are needed.
> HiveServer2: Use user supplied fetch size to determine #rows serialized in
> tasks
>
>
> Key: HIVE-14901
> URL: https://issues.apache.org/jira/browse/HIVE-14901
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, JDBC, ODBC
>Affects Versions: 2.1.0
>Reporter: Vaibhav Gumashta
>Assignee: Norris Lee
> Attachments: HIVE-14901.1.patch, HIVE-14901.2.patch,
> HIVE-14901.3.patch, HIVE-14901.patch
>
>
> Currently, we use {{hive.server2.thrift.resultset.max.fetch.size}} to decide
> the max number of rows that we write in tasks. However, we should ideally use
> the user supplied value (which can be extracted from the
> ThriftCLIService.FetchResults' request parameter) to decide how many rows to
> serialize in a blob in the tasks. We should however use
> {{hive.server2.thrift.resultset.max.fetch.size}} to have an upper bound on
> it, so that we don't go OOM in tasks and HS2.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15953) better error messages for LLAP registry properties
[
https://issues.apache.org/jira/browse/HIVE-15953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872754#comment-15872754
]
Prasanth Jayachandran commented on HIVE-15953:
--
+1
> better error messages for LLAP registry properties
> --
>
> Key: HIVE-15953
> URL: https://issues.apache.org/jira/browse/HIVE-15953
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-15953.patch
>
>
> {noformat}
> int memory =
> Integer.parseInt(srv.get(ConfVars.LLAP_DAEMON_MEMORY_PER_INSTANCE_MB.varname));
> int vCores =
> Integer.parseInt(srv.get(ConfVars.LLAP_DAEMON_NUM_EXECUTORS.varname));
> {noformat}
> Needs to provide a useful error message when it fails because the config is
> bad. There may be other similar places.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[
https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872750#comment-15872750
]
Hive QA commented on HIVE-15904:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853361/HIVE-15904.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10246 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynamic_semijoin_reduction_2]
(batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=140)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31]
(batchId=133)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=211)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3634/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3634/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3634/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853361 - PreCommit-HIVE-Build
> select query throwing Null Pointer Exception from
> org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
> --
>
> Key: HIVE-15904
> URL: https://issues.apache.org/jira/browse/HIVE-15904
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Aswathy Chellammal Sreekumar
>Assignee: Jason Dere
> Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch,
> HIVE-15904.3.patch, HIVE-15904.4.patch, table_18.q, table_1.q
>
>
> Following query failing with Null Pointer Exception from
> org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
> Attaching create table statements for table_1 and table_18
> Query:
> SELECT
> COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY
> (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 +
> t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col,
> (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1,
> FLOOR(t1.double_col_16) AS float_col,
> COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 +
> t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC,
> FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48
> FOLLOWING), 704) AS int_col_2
> FROM table_1 t1
> INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND
> ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND
> ((t2.tinyint_col_20) = (t1.tinyint_col_3))
> WHERE (t2.smallint_col_19) IN (SELECT
> COALESCE(-92, -994) AS int_col
> FROM table_1 tt1
> INNER JOIN table_18 tt2 ON (tt2.decimal1911_col_16) = (tt1.decimal2612_col_77)
> WHERE (t1.timestamp_col_9) = (tt2.timestamp_col_18));
> Error Stack:
> org.apache.hive.service.cli.HiveSQLException: Error while compiling
> statement: FAILED: NullPointerException null
> at
> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387)
>
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:193)
>
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276)
>
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:324)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:507)
>
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495)
>
> at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:308)
>
> at
>
[jira] [Commented] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872747#comment-15872747 ] Jason Dere commented on HIVE-15904: --- Yeah, i took a couple of stabs at trying to trim simplify and failed. Will give it one more try. > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Jason Dere > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, > HIVE-15904.3.patch, HIVE-15904.4.patch, table_18.q, table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal1911_col_16) = (tt1.decimal2612_col_77) > WHERE (t1.timestamp_col_9) = (tt2.timestamp_col_18)); > Error Stack: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387) > > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:193) > > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:507) > > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495) > > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:308) > > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:506) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:599) > > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [?:1.8.0_112] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [?:1.8.0_112] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan(DynamicPartitionPruningOptimization.java:402) > > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.process(DynamicPartitionPruningOptimization.java:226) > > at > org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89) > > at >
[jira] [Commented] (HIVE-15938) position alias in order by fails for union queries
[ https://issues.apache.org/jira/browse/HIVE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872743#comment-15872743 ] Ashutosh Chauhan commented on HIVE-15938: - LGTM. [~pxiong] Can you please also take a look? > position alias in order by fails for union queries > -- > > Key: HIVE-15938 > URL: https://issues.apache.org/jira/browse/HIVE-15938 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15938.01.patch, HIVE-15938.02.patch, > HIVE-15938.03.patch, HIVE-15938.04.patch, HIVE-15938.patch > > > The problem is that the query introduces a spurious ALLCOLREF -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB
[ https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872741#comment-15872741 ] Wei Zheng commented on HIVE-15910: -- The code change looks good. Btw, did we do any performance analysis for this change? > Improvements in Hive Unit Test by using In-memory Derby DB > -- > > Key: HIVE-15910 > URL: https://issues.apache.org/jira/browse/HIVE-15910 > Project: Hive > Issue Type: Test > Components: Tests >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Attachments: HIVE-15910.01.patch, HIVE-15910.05.patch, > HIVE-15910.06.patch, HIVE-15910.2.patch, HIVE-15910.3.patch, > HIVE-15910.4.patch > > > Hive UT currently uses Derby DB with storage on disk which have some > practical problems. > 1. The run-time of Hive unit tests are high as need to operate on the disk > quite often. > 2. It can cause conflict if multiple test cases operates on the same table > name (such as table being created already exist). > To solve these problems, we shall use an in-memory storage option of Derby DB > which can be even persisted if the test case demands that. > https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872735#comment-15872735 ] Gunther Hagleitner commented on HIVE-15904: --- +1. Nit: I like that it has a test, but it would have been nice to trim the test down a bit. Hard to follow. > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Jason Dere > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, > HIVE-15904.3.patch, HIVE-15904.4.patch, table_18.q, table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal1911_col_16) = (tt1.decimal2612_col_77) > WHERE (t1.timestamp_col_9) = (tt2.timestamp_col_18)); > Error Stack: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387) > > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:193) > > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:507) > > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495) > > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:308) > > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:506) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:599) > > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [?:1.8.0_112] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [?:1.8.0_112] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan(DynamicPartitionPruningOptimization.java:402) > > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.process(DynamicPartitionPruningOptimization.java:226) > > at > org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89) > > at >
[jira] [Updated] (HIVE-15938) position alias in order by fails for union queries
[ https://issues.apache.org/jira/browse/HIVE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-15938: Attachment: HIVE-15938.04.patch > position alias in order by fails for union queries > -- > > Key: HIVE-15938 > URL: https://issues.apache.org/jira/browse/HIVE-15938 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15938.01.patch, HIVE-15938.02.patch, > HIVE-15938.03.patch, HIVE-15938.04.patch, HIVE-15938.patch > > > The problem is that the query introduces a spurious ALLCOLREF -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15971) LLAP: logs urls should use daemon container id instead of fake container id
[ https://issues.apache.org/jira/browse/HIVE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872731#comment-15872731 ] Sergey Shelukhin commented on HIVE-15971: - timelineServerUri could be checked earlier too, not just on the lowest level. In the user-support part, map is accessed 3 times (contains-get-get). Also, there's some other jira about that somewhere that needs to be resolved as dup. Rest looks good. > LLAP: logs urls should use daemon container id instead of fake container id > --- > > Key: HIVE-15971 > URL: https://issues.apache.org/jira/browse/HIVE-15971 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-15971.1.patch > > > The containerId used for log url generation is fake. It should be replaced by > the container id of the llap daemon. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (HIVE-15938) position alias in order by fails for union queries
[ https://issues.apache.org/jira/browse/HIVE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872708#comment-15872708 ] Sergey Shelukhin edited comment on HIVE-15938 at 2/17/17 11:10 PM: --- [~ashutoshc] it failed in 10s of the previous builds except one. It may have been the patch that fixed it that is not merged yet :) https://builds.apache.org/job/PreCommit-HIVE-Build/3632/testReport/org.apache.hadoop.hive.cli/TestNegativeCliDriver/testCliDriver_create_view_failure2_/history/ was (Author: sershe): [~ashutoshc] it failed in 10s of the previous builds except one. It may have been the patch that fixed it that is not merged yet :) > position alias in order by fails for union queries > -- > > Key: HIVE-15938 > URL: https://issues.apache.org/jira/browse/HIVE-15938 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15938.01.patch, HIVE-15938.02.patch, > HIVE-15938.03.patch, HIVE-15938.patch > > > The problem is that the query introduces a spurious ALLCOLREF -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15938) position alias in order by fails for union queries
[ https://issues.apache.org/jira/browse/HIVE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872708#comment-15872708 ] Sergey Shelukhin commented on HIVE-15938: - [~ashutoshc] it failed in 10s of the previous builds except one. It may have been the patch that fixed it that is not merged yet :) > position alias in order by fails for union queries > -- > > Key: HIVE-15938 > URL: https://issues.apache.org/jira/browse/HIVE-15938 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15938.01.patch, HIVE-15938.02.patch, > HIVE-15938.03.patch, HIVE-15938.patch > > > The problem is that the query introduces a spurious ALLCOLREF -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15938) position alias in order by fails for union queries
[ https://issues.apache.org/jira/browse/HIVE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-15938: Attachment: HIVE-15938.03.patch Revert the out file change for union10. Data size estimate changed for constprog test, without a change of number of rows. The AST only changes as expected (allcolref -> single col), so if anything it's an unrelated issue in estimation. > position alias in order by fails for union queries > -- > > Key: HIVE-15938 > URL: https://issues.apache.org/jira/browse/HIVE-15938 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15938.01.patch, HIVE-15938.02.patch, > HIVE-15938.03.patch, HIVE-15938.patch > > > The problem is that the query introduces a spurious ALLCOLREF -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15971) LLAP: logs urls should use daemon container id instead of fake container id
[ https://issues.apache.org/jira/browse/HIVE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-15971: - Status: Open (was: Patch Available) > LLAP: logs urls should use daemon container id instead of fake container id > --- > > Key: HIVE-15971 > URL: https://issues.apache.org/jira/browse/HIVE-15971 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-15971.1.patch > > > The containerId used for log url generation is fake. It should be replaced by > the container id of the llap daemon. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15938) position alias in order by fails for union queries
[ https://issues.apache.org/jira/browse/HIVE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872701#comment-15872701 ] Ashutosh Chauhan commented on HIVE-15938: - create_view_failure has age 1. Likely related. > position alias in order by fails for union queries > -- > > Key: HIVE-15938 > URL: https://issues.apache.org/jira/browse/HIVE-15938 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15938.01.patch, HIVE-15938.02.patch, > HIVE-15938.patch > > > The problem is that the query introduces a spurious ALLCOLREF -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15971) LLAP: logs urls should use daemon container id instead of fake container id
[ https://issues.apache.org/jira/browse/HIVE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-15971: - Status: Patch Available (was: Open) > LLAP: logs urls should use daemon container id instead of fake container id > --- > > Key: HIVE-15971 > URL: https://issues.apache.org/jira/browse/HIVE-15971 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-15971.1.patch > > > The containerId used for log url generation is fake. It should be replaced by > the container id of the llap daemon. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15971) LLAP: logs urls should use daemon container id instead of fake container id
[ https://issues.apache.org/jira/browse/HIVE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-15971: - Attachment: HIVE-15971.1.patch > LLAP: logs urls should use daemon container id instead of fake container id > --- > > Key: HIVE-15971 > URL: https://issues.apache.org/jira/browse/HIVE-15971 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-15971.1.patch > > > The containerId used for log url generation is fake. It should be replaced by > the container id of the llap daemon. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (HIVE-15971) LLAP: logs urls should use daemon container id instead of fake container id
[ https://issues.apache.org/jira/browse/HIVE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-15971: > LLAP: logs urls should use daemon container id instead of fake container id > --- > > Key: HIVE-15971 > URL: https://issues.apache.org/jira/browse/HIVE-15971 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > > The containerId used for log url generation is fake. It should be replaced by > the container id of the llap daemon. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15970) Fix merge to work in presence of AST rewrites
[
https://issues.apache.org/jira/browse/HIVE-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872690#comment-15872690
]
Hive QA commented on HIVE-15970:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853324/HIVE-15970.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 34 failed/errored test(s), 10244 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[interval_arithmetic]
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_external_acid]
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_not_acid]
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_view_failure1]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_view_failure2]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[external1]
(batchId=85)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31]
(batchId=133)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testAlterTable
(batchId=194)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testTransactionalValidation
(batchId=194)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testAlterTable
(batchId=197)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testTransactionalValidation
(batchId=197)
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testAlterTable
(batchId=193)
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testTransactionalValidation
(batchId=193)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testAlterTable
(batchId=191)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testTransactionalValidation
(batchId=191)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testAlterTable
(batchId=202)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testTransactionalValidation
(batchId=202)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeType2SCD01 (batchId=270)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeType2SCD02 (batchId=270)
org.apache.hadoop.hive.ql.security.TestMetastoreAuthorizationProvider.testSimplePrivileges
(batchId=210)
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProvider.testSimplePrivileges
(batchId=208)
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProviderWithACL.testSimplePrivileges
(batchId=220)
org.apache.hive.hcatalog.api.TestHCatClient.testBasicDDLCommands (batchId=170)
org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=170)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.createTable
(batchId=221)
org.apache.hive.service.cli.TestEmbeddedThriftBinaryCLIService.testExecuteStatementAsync
(batchId=213)
org.apache.hive.service.cli.thrift.TestThriftCLIServiceWithBinary.testExecuteStatementAsync
(batchId=213)
org.apache.hive.service.cli.thrift.TestThriftCLIServiceWithHttp.testExecuteStatementAsync
(batchId=213)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3633/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3633/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3633/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 34 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853324 - PreCommit-HIVE-Build
> Fix merge to work in presence of AST rewrites
> -
>
> Key: HIVE-15970
> URL: https://issues.apache.org/jira/browse/HIVE-15970
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 2.2.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Critical
> Attachments: HIVE-15970.01.patch, HIVE-15970.02.patch
>
>
> The issue is that Context.getDestNamePrefix() caches ASTNode objects to
>
[jira] [Commented] (HIVE-15938) position alias in order by fails for union queries
[ https://issues.apache.org/jira/browse/HIVE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872688#comment-15872688 ] Sergey Shelukhin commented on HIVE-15938: - No, they are not related actually. I see they were failing for a while, e.g. https://builds.apache.org/job/PreCommit-HIVE-Build/3632/testReport/org.apache.hadoop.hive.cli/TestNegativeCliDriver/testCliDriver_create_external_acid_/history/ > position alias in order by fails for union queries > -- > > Key: HIVE-15938 > URL: https://issues.apache.org/jira/browse/HIVE-15938 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15938.01.patch, HIVE-15938.02.patch, > HIVE-15938.patch > > > The problem is that the query introduces a spurious ALLCOLREF -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15846) Relocate more dependencies (e.g. org.apache.zookeeper) for JDBC uber jar
[ https://issues.apache.org/jira/browse/HIVE-15846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872668#comment-15872668 ] Tao Li commented on HIVE-15846: --- [~thejas] I have uploaded another patch to merge the conflicts. > Relocate more dependencies (e.g. org.apache.zookeeper) for JDBC uber jar > > > Key: HIVE-15846 > URL: https://issues.apache.org/jira/browse/HIVE-15846 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Attachments: HIVE-15846.1.patch, HIVE-15846.2.patch, > HIVE-15846.3.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (HIVE-15959) LLAP: fix headroom calculation and move it to daemon
[ https://issues.apache.org/jira/browse/HIVE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872665#comment-15872665 ] Sergey Shelukhin edited comment on HIVE-15959 at 2/17/17 10:37 PM: --- The latest patch switches to LLAP considering purely xmx headroom, assuming the caller takes care of non-xmx headroom that is needed due to JVM overhead/etc causing YARN kills. That headroom is configurable as either size or percentage, 5% by default. Tested on a cluster, works as expected. was (Author: sershe): The latest patch switches to LLAP considering purely xmx headroom, assuming the caller takes care of non-xmx headroom that is needed due to JVM overhead/etc causing YARN kills. > LLAP: fix headroom calculation and move it to daemon > > > Key: HIVE-15959 > URL: https://issues.apache.org/jira/browse/HIVE-15959 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15959.01.patch, HIVE-15959.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15846) Relocate more dependencies (e.g. org.apache.zookeeper) for JDBC uber jar
[ https://issues.apache.org/jira/browse/HIVE-15846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Li updated HIVE-15846: -- Attachment: HIVE-15846.3.patch > Relocate more dependencies (e.g. org.apache.zookeeper) for JDBC uber jar > > > Key: HIVE-15846 > URL: https://issues.apache.org/jira/browse/HIVE-15846 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Attachments: HIVE-15846.1.patch, HIVE-15846.2.patch, > HIVE-15846.3.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15959) LLAP: fix headroom calculation and move it to daemon
[ https://issues.apache.org/jira/browse/HIVE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872665#comment-15872665 ] Sergey Shelukhin commented on HIVE-15959: - The latest patch switches to LLAP considering purely xmx headroom, assuming the caller takes care of non-xmx headroom that is needed due to JVM overhead/etc causing YARN kills. > LLAP: fix headroom calculation and move it to daemon > > > Key: HIVE-15959 > URL: https://issues.apache.org/jira/browse/HIVE-15959 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15959.01.patch, HIVE-15959.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15919) Row count mismatch for count * query
[ https://issues.apache.org/jira/browse/HIVE-15919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-15919: -- Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed to master > Row count mismatch for count * query > > > Key: HIVE-15919 > URL: https://issues.apache.org/jira/browse/HIVE-15919 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Jason Dere > Fix For: 2.2.0 > > Attachments: HIVE-15919.1.patch, HIVE-15919.2.patch, > HIVE-15919.3.patch, table_14.q, table_6.q > > > The following query is returning different output when run against hive and > postgres. > Query: > SELECT COUNT (*) > FROM > (SELECT LAG(COALESCE(t2.int_col_14, t1.int_col_80),22) OVER (ORDER BY > t1.tinyint_col_52 DESC) AS int_col > FROM table_6 t1 > INNER JOIN table_14 t2 ON ((t2.decimal0101_col_55) = (t1.decimal0101_col_9))) > AS FOO; > From hive: 0 > From postgres: 66903279 > Attaching ddl files for the tables. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872654#comment-15872654 ] Jason Dere commented on HIVE-15904: --- [~hagleitn] can you review? > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Jason Dere > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, > HIVE-15904.3.patch, HIVE-15904.4.patch, table_18.q, table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal1911_col_16) = (tt1.decimal2612_col_77) > WHERE (t1.timestamp_col_9) = (tt2.timestamp_col_18)); > Error Stack: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387) > > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:193) > > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:507) > > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495) > > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:308) > > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:506) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:599) > > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [?:1.8.0_112] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [?:1.8.0_112] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan(DynamicPartitionPruningOptimization.java:402) > > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.process(DynamicPartitionPruningOptimization.java:226) > > at > org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89) > > at > org.apache.hadoop.hive.ql.lib.ForwardWalker.walk(ForwardWalker.java:74) > at >
[jira] [Commented] (HIVE-15938) position alias in order by fails for union queries
[ https://issues.apache.org/jira/browse/HIVE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872656#comment-15872656 ] Ashutosh Chauhan commented on HIVE-15938: - Failed -ve test looks related. > position alias in order by fails for union queries > -- > > Key: HIVE-15938 > URL: https://issues.apache.org/jira/browse/HIVE-15938 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15938.01.patch, HIVE-15938.02.patch, > HIVE-15938.patch > > > The problem is that the query introduces a spurious ALLCOLREF -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15938) position alias in order by fails for union queries
[ https://issues.apache.org/jira/browse/HIVE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872657#comment-15872657 ] Ashutosh Chauhan commented on HIVE-15938: - [~pxiong] Can you help review this? > position alias in order by fails for union queries > -- > > Key: HIVE-15938 > URL: https://issues.apache.org/jira/browse/HIVE-15938 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15938.01.patch, HIVE-15938.02.patch, > HIVE-15938.patch > > > The problem is that the query introduces a spurious ALLCOLREF -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15951) Make sure base persist directory is unique and deleted
[ https://issues.apache.org/jira/browse/HIVE-15951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872651#comment-15872651 ] Ashutosh Chauhan commented on HIVE-15951: - This will work in container mode, but may not in llap mode with long running process. There we may need to keep track of lifecycle of the file. > Make sure base persist directory is unique and deleted > -- > > Key: HIVE-15951 > URL: https://issues.apache.org/jira/browse/HIVE-15951 > Project: Hive > Issue Type: Bug > Components: Druid integration >Affects Versions: 2.2.0 >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Critical > Fix For: 2.2.0 > > Attachments: HIVE-15951.patch > > > In some cases the base persist directory will contain old data or shared > between reducer in the same physical VM. > That will lead to the failure of the job till that the directory is cleaned. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15953) better error messages for LLAP registry properties
[
https://issues.apache.org/jira/browse/HIVE-15953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-15953:
Assignee: Sergey Shelukhin
Status: Patch Available (was: Open)
> better error messages for LLAP registry properties
> --
>
> Key: HIVE-15953
> URL: https://issues.apache.org/jira/browse/HIVE-15953
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-15953.patch
>
>
> {noformat}
> int memory =
> Integer.parseInt(srv.get(ConfVars.LLAP_DAEMON_MEMORY_PER_INSTANCE_MB.varname));
> int vCores =
> Integer.parseInt(srv.get(ConfVars.LLAP_DAEMON_NUM_EXECUTORS.varname));
> {noformat}
> Needs to provide a useful error message when it fails because the config is
> bad. There may be other similar places.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-15970) Fix merge to work in presence of AST rewrites
[ https://issues.apache.org/jira/browse/HIVE-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-15970: -- Attachment: HIVE-15970.02.patch > Fix merge to work in presence of AST rewrites > - > > Key: HIVE-15970 > URL: https://issues.apache.org/jira/browse/HIVE-15970 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-15970.01.patch, HIVE-15970.02.patch > > > The issue is that Context.getDestNamePrefix() caches ASTNode objects to > figure out if a given insert branch of multi-insert is insert, update or > delete but objects change when the tree is modified. For example, > row-filtering/column-masking does this, but also other optimizations/features. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15953) better error messages for LLAP registry properties
[
https://issues.apache.org/jira/browse/HIVE-15953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-15953:
Attachment: HIVE-15953.patch
[~prasanth_j] [~rajesh.balamohan] can you take a look? tiny patch
> better error messages for LLAP registry properties
> --
>
> Key: HIVE-15953
> URL: https://issues.apache.org/jira/browse/HIVE-15953
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
> Attachments: HIVE-15953.patch
>
>
> {noformat}
> int memory =
> Integer.parseInt(srv.get(ConfVars.LLAP_DAEMON_MEMORY_PER_INSTANCE_MB.varname));
> int vCores =
> Integer.parseInt(srv.get(ConfVars.LLAP_DAEMON_NUM_EXECUTORS.varname));
> {noformat}
> Needs to provide a useful error message when it fails because the config is
> bad. There may be other similar places.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15570) LLAP: Exception in HostAffinitySplitLocationProvider when running in container mode
[
https://issues.apache.org/jira/browse/HIVE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872639#comment-15872639
]
Sergey Shelukhin commented on HIVE-15570:
-
+1
> LLAP: Exception in HostAffinitySplitLocationProvider when running in
> container mode
> ---
>
> Key: HIVE-15570
> URL: https://issues.apache.org/jira/browse/HIVE-15570
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Rajesh Balamohan
>Assignee: Zhiyuan Yang
>Priority: Minor
> Attachments: HIVE-15570.1.patch, HIVE-15570.2.patch,
> HIVE-15570.3.patch
>
>
> Sometimes user might prefer to run with "hive.execution.mode=container" mode
> when LLAP is stopped. If hive config for LLAP had
> "hive.llap.client.consistent.splits=true" in client side, it would end up
> throwing the following exception in {{Utils.java}}.
> {noformat}
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
> ... 25 more
> Caused by: java.lang.IllegalStateException:
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider needs at
> least 1 location to function
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider.(HostAffinitySplitLocationProvider.java:52)
> at
> org.apache.hadoop.hive.ql.exec.tez.Utils.getSplitLocationProvider(Utils.java:54)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.(HiveSplitGenerator.java:121)
> ... 30 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-15969) Failures in TestRemoteHiveMetaStore, TestSetUGIOnOnlyServer
[
https://issues.apache.org/jira/browse/HIVE-15969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-15969:
Resolution: Fixed
Fix Version/s: 2.2.0
Status: Resolved (was: Patch Available)
Pushed to master. Thanks, Slim & Thejas!
> Failures in TestRemoteHiveMetaStore, TestSetUGIOnOnlyServer
> ---
>
> Key: HIVE-15969
> URL: https://issues.apache.org/jira/browse/HIVE-15969
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 2.2.0
>Reporter: Thejas M Nair
>Assignee: slim bouguerra
> Fix For: 2.2.0
>
> Attachments: HIVE-15969.2.patch, HIVE-15969.patch
>
>
> Looks like the additional failures in TestRemoteHiveMetaStore,
> TestSetUGIOnOnlyServer
> are related to this patch in HIVE-15877.
> I don't think that change was intended.
> {code}
> ---
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> +++
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> @@ -739,7 +739,10 @@ public void createTable(Table tbl, EnvironmentContext
> envContext) throws Already
> hook.commitCreateTable(tbl);
>}
>success = true;
> -} finally {
> +} catch (Exception e){
> + LOG.error("Got exception from createTable", e);
> +}
> +finally {
>if (!success && (hook != null)) {
> hook.rollbackCreateTable(tbl);
>}
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-15959) LLAP: fix headroom calculation and move it to daemon
[ https://issues.apache.org/jira/browse/HIVE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-15959: Attachment: (was: HIVE-15959.01.patch) > LLAP: fix headroom calculation and move it to daemon > > > Key: HIVE-15959 > URL: https://issues.apache.org/jira/browse/HIVE-15959 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15959.01.patch, HIVE-15959.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15959) LLAP: fix headroom calculation and move it to daemon
[ https://issues.apache.org/jira/browse/HIVE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-15959: Attachment: HIVE-15959.01.patch > LLAP: fix headroom calculation and move it to daemon > > > Key: HIVE-15959 > URL: https://issues.apache.org/jira/browse/HIVE-15959 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15959.01.patch, HIVE-15959.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15959) LLAP: fix headroom calculation and move it to daemon
[ https://issues.apache.org/jira/browse/HIVE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-15959: Attachment: (was: HIVE-15959.01.patch) > LLAP: fix headroom calculation and move it to daemon > > > Key: HIVE-15959 > URL: https://issues.apache.org/jira/browse/HIVE-15959 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15959.01.patch, HIVE-15959.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15959) LLAP: fix headroom calculation and move it to daemon
[ https://issues.apache.org/jira/browse/HIVE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-15959: Attachment: HIVE-15959.01.patch > LLAP: fix headroom calculation and move it to daemon > > > Key: HIVE-15959 > URL: https://issues.apache.org/jira/browse/HIVE-15959 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15959.01.patch, HIVE-15959.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15959) LLAP: fix headroom calculation and move it to daemon
[ https://issues.apache.org/jira/browse/HIVE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-15959: Attachment: HIVE-15959.01.patch > LLAP: fix headroom calculation and move it to daemon > > > Key: HIVE-15959 > URL: https://issues.apache.org/jira/browse/HIVE-15959 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15959.01.patch, HIVE-15959.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-14476) Fix logging issue for branch-1
[ https://issues.apache.org/jira/browse/HIVE-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872631#comment-15872631 ] Tao Li commented on HIVE-14476: --- [~spena], [~thejas] Do you know why there are still 130+ test failures for branch-1? > Fix logging issue for branch-1 > -- > > Key: HIVE-14476 > URL: https://issues.apache.org/jira/browse/HIVE-14476 > Project: Hive > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li > Attachments: HIVE-14476.1-branch-1.2.patch > > > This issue is from branch-1 code when we decide if a log entry is an > operational log or not (the operational logs are visible to the client). The > problem is that the code is checking the logging mode at the beginning of the > decide() method, while the logging mode is updated after that check. Due to > this issue, we ran into an issue that an operational log could be filtered > out if it's the very first log being checked from the this method. As a > result, that particular log is not showing up for the end user. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15938) position alias in order by fails for union queries
[
https://issues.apache.org/jira/browse/HIVE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872613#comment-15872613
]
Hive QA commented on HIVE-15938:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853320/HIVE-15938.02.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 35 failed/errored test(s), 10245 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constant_prop_1]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[interval_arithmetic]
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union10] (batchId=10)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_external_acid]
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_not_acid]
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_view_failure1]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_view_failure2]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[external1]
(batchId=85)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31]
(batchId=133)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testAlterTable
(batchId=194)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testTransactionalValidation
(batchId=194)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testAlterTable
(batchId=197)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testTransactionalValidation
(batchId=197)
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testAlterTable
(batchId=193)
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testTransactionalValidation
(batchId=193)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testAlterTable
(batchId=191)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testTransactionalValidation
(batchId=191)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testAlterTable
(batchId=202)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testTransactionalValidation
(batchId=202)
org.apache.hadoop.hive.ql.security.TestMetastoreAuthorizationProvider.testSimplePrivileges
(batchId=210)
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProvider.testSimplePrivileges
(batchId=208)
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProviderWithACL.testSimplePrivileges
(batchId=220)
org.apache.hive.hcatalog.api.TestHCatClient.testBasicDDLCommands (batchId=170)
org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=170)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.createTable
(batchId=221)
org.apache.hive.service.cli.TestEmbeddedThriftBinaryCLIService.testExecuteStatementAsync
(batchId=213)
org.apache.hive.service.cli.thrift.TestThriftCLIServiceWithBinary.testExecuteStatementAsync
(batchId=213)
org.apache.hive.service.cli.thrift.TestThriftCLIServiceWithHttp.testExecuteStatementAsync
(batchId=213)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3632/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3632/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3632/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 35 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853320 - PreCommit-HIVE-Build
> position alias in order by fails for union queries
> --
>
> Key: HIVE-15938
> URL: https://issues.apache.org/jira/browse/HIVE-15938
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-15938.01.patch, HIVE-15938.02.patch,
> HIVE-15938.patch
>
>
> The problem is that the query introduces
[jira] [Commented] (HIVE-15919) Row count mismatch for count * query
[ https://issues.apache.org/jira/browse/HIVE-15919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872612#comment-15872612 ] Matt McCline commented on HIVE-15919: - +1 LGTM > Row count mismatch for count * query > > > Key: HIVE-15919 > URL: https://issues.apache.org/jira/browse/HIVE-15919 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Jason Dere > Attachments: HIVE-15919.1.patch, HIVE-15919.2.patch, > HIVE-15919.3.patch, table_14.q, table_6.q > > > The following query is returning different output when run against hive and > postgres. > Query: > SELECT COUNT (*) > FROM > (SELECT LAG(COALESCE(t2.int_col_14, t1.int_col_80),22) OVER (ORDER BY > t1.tinyint_col_52 DESC) AS int_col > FROM table_6 t1 > INNER JOIN table_14 t2 ON ((t2.decimal0101_col_55) = (t1.decimal0101_col_9))) > AS FOO; > From hive: 0 > From postgres: 66903279 > Attaching ddl files for the tables. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-15904: -- Attachment: HIVE-15904.4.patch Updating patch/test > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Jason Dere > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, > HIVE-15904.3.patch, HIVE-15904.4.patch, table_18.q, table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal1911_col_16) = (tt1.decimal2612_col_77) > WHERE (t1.timestamp_col_9) = (tt2.timestamp_col_18)); > Error Stack: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387) > > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:193) > > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:507) > > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495) > > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:308) > > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:506) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:599) > > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [?:1.8.0_112] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [?:1.8.0_112] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan(DynamicPartitionPruningOptimization.java:402) > > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.process(DynamicPartitionPruningOptimization.java:226) > > at > org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89) > > at > org.apache.hadoop.hive.ql.lib.ForwardWalker.walk(ForwardWalker.java:74) > at >
[jira] [Assigned] (HIVE-15904) select query throwing Null Pointer Exception from org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan
[ https://issues.apache.org/jira/browse/HIVE-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere reassigned HIVE-15904: - Assignee: Jason Dere (was: Deepak Jaiswal) > select query throwing Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > -- > > Key: HIVE-15904 > URL: https://issues.apache.org/jira/browse/HIVE-15904 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Aswathy Chellammal Sreekumar >Assignee: Jason Dere > Attachments: HIVE-15904.1.patch, HIVE-15904.2.patch, > HIVE-15904.3.patch, table_18.q, table_1.q > > > Following query failing with Null Pointer Exception from > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan > Attaching create table statements for table_1 and table_18 > Query: > SELECT > COALESCE(498, LEAD(COALESCE(-973, -684, 515)) OVER (PARTITION BY > (t2.int_col_10 + t1.smallint_col_50) ORDER BY (t2.int_col_10 + > t1.smallint_col_50), FLOOR(t1.double_col_16) DESC), 524) AS int_col, > (t2.int_col_10) + (t1.smallint_col_50) AS int_col_1, > FLOOR(t1.double_col_16) AS float_col, > COALESCE(SUM(COALESCE(62, -380, -435)) OVER (PARTITION BY (t2.int_col_10 + > t1.smallint_col_50) ORDER BY (t2.int_col_10 + t1.smallint_col_50) DESC, > FLOOR(t1.double_col_16) DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 48 > FOLLOWING), 704) AS int_col_2 > FROM table_1 t1 > INNER JOIN table_18 t2 ON (((t2.tinyint_col_15) = (t1.bigint_col_7)) AND > ((t2.decimal2709_col_9) = (t1.decimal2016_col_26))) AND > ((t2.tinyint_col_20) = (t1.tinyint_col_3)) > WHERE (t2.smallint_col_19) IN (SELECT > COALESCE(-92, -994) AS int_col > FROM table_1 tt1 > INNER JOIN table_18 tt2 ON (tt2.decimal1911_col_16) = (tt1.decimal2612_col_77) > WHERE (t1.timestamp_col_9) = (tt2.timestamp_col_18)); > Error Stack: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387) > > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:193) > > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) > > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:324) > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:507) > > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495) > > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:308) > > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:506) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:599) > > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [?:1.8.0_112] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [?:1.8.0_112] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan(DynamicPartitionPruningOptimization.java:402) > > at > org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.process(DynamicPartitionPruningOptimization.java:226) > > at > org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105) > > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89) > > at > org.apache.hadoop.hive.ql.lib.ForwardWalker.walk(ForwardWalker.java:74) > at >
[jira] [Commented] (HIVE-15969) Failures in TestRemoteHiveMetaStore, TestSetUGIOnOnlyServer
[
https://issues.apache.org/jira/browse/HIVE-15969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872531#comment-15872531
]
Hive QA commented on HIVE-15969:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853307/HIVE-15969.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 10244 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[interval_arithmetic]
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=140)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31]
(batchId=133)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3631/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3631/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3631/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853307 - PreCommit-HIVE-Build
> Failures in TestRemoteHiveMetaStore, TestSetUGIOnOnlyServer
> ---
>
> Key: HIVE-15969
> URL: https://issues.apache.org/jira/browse/HIVE-15969
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 2.2.0
>Reporter: Thejas M Nair
>Assignee: slim bouguerra
> Attachments: HIVE-15969.2.patch, HIVE-15969.patch
>
>
> Looks like the additional failures in TestRemoteHiveMetaStore,
> TestSetUGIOnOnlyServer
> are related to this patch in HIVE-15877.
> I don't think that change was intended.
> {code}
> ---
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> +++
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> @@ -739,7 +739,10 @@ public void createTable(Table tbl, EnvironmentContext
> envContext) throws Already
> hook.commitCreateTable(tbl);
>}
>success = true;
> -} finally {
> +} catch (Exception e){
> + LOG.error("Got exception from createTable", e);
> +}
> +finally {
>if (!success && (hook != null)) {
> hook.rollbackCreateTable(tbl);
>}
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-15961) Vectorization: Cleanup Vectorizer logging
[ https://issues.apache.org/jira/browse/HIVE-15961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-15961: Attachment: HIVE-15961.02.patch > Vectorization: Cleanup Vectorizer logging > - > > Key: HIVE-15961 > URL: https://issues.apache.org/jira/browse/HIVE-15961 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-15961.01.patch, HIVE-15961.02.patch > > > Make most logging DEBUG. Show TypeInfos of input to Map/Reduce vertex. INFO > logging for reasons vertex did not vectorize. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15961) Vectorization: Cleanup Vectorizer logging
[ https://issues.apache.org/jira/browse/HIVE-15961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-15961: Status: In Progress (was: Patch Available) > Vectorization: Cleanup Vectorizer logging > - > > Key: HIVE-15961 > URL: https://issues.apache.org/jira/browse/HIVE-15961 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-15961.01.patch > > > Make most logging DEBUG. Show TypeInfos of input to Map/Reduce vertex. INFO > logging for reasons vertex did not vectorize. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15970) Fix merge to work in presence of AST rewrites
[ https://issues.apache.org/jira/browse/HIVE-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-15970: -- Summary: Fix merge to work in presence of AST rewrites (was: Fix merge to work in presents of AST rewrites) > Fix merge to work in presence of AST rewrites > - > > Key: HIVE-15970 > URL: https://issues.apache.org/jira/browse/HIVE-15970 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-15970.01.patch > > > The issue is that Context.getDestNamePrefix() caches ASTNode objects to > figure out if a given insert branch of multi-insert is insert, update or > delete but objects change when the tree is modified. For example, > row-filtering/column-masking does this, but also other optimizations/features. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15970) Fix merge to work in presence of AST rewrites
[ https://issues.apache.org/jira/browse/HIVE-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872507#comment-15872507 ] Eugene Koifman commented on HIVE-15970: --- cc [~wzheng] > Fix merge to work in presence of AST rewrites > - > > Key: HIVE-15970 > URL: https://issues.apache.org/jira/browse/HIVE-15970 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-15970.01.patch > > > The issue is that Context.getDestNamePrefix() caches ASTNode objects to > figure out if a given insert branch of multi-insert is insert, update or > delete but objects change when the tree is modified. For example, > row-filtering/column-masking does this, but also other optimizations/features. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15970) Fix merge to work in presents of AST rewrites
[ https://issues.apache.org/jira/browse/HIVE-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-15970: -- Attachment: HIVE-15970.01.patch > Fix merge to work in presents of AST rewrites > - > > Key: HIVE-15970 > URL: https://issues.apache.org/jira/browse/HIVE-15970 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-15970.01.patch > > > The issue is that Context.getDestNamePrefix() caches ASTNode objects to > figure out if a given insert branch of multi-insert is insert, update or > delete but objects change when the tree is modified. For example, > row-filtering/column-masking does this, but also other optimizations/features. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15970) Fix merge to work in presents of AST rewrites
[ https://issues.apache.org/jira/browse/HIVE-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-15970: -- Status: Patch Available (was: Open) > Fix merge to work in presents of AST rewrites > - > > Key: HIVE-15970 > URL: https://issues.apache.org/jira/browse/HIVE-15970 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-15970.01.patch > > > The issue is that Context.getDestNamePrefix() caches ASTNode objects to > figure out if a given insert branch of multi-insert is insert, update or > delete but objects change when the tree is modified. For example, > row-filtering/column-masking does this, but also other optimizations/features. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15970) Fix merge to work in presents of AST rewrites
[ https://issues.apache.org/jira/browse/HIVE-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-15970: -- Issue Type: Sub-task (was: Bug) Parent: HIVE-10924 > Fix merge to work in presents of AST rewrites > - > > Key: HIVE-15970 > URL: https://issues.apache.org/jira/browse/HIVE-15970 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-15970.01.patch > > > The issue is that Context.getDestNamePrefix() caches ASTNode objects to > figure out if a given insert branch of multi-insert is insert, update or > delete but objects change when the tree is modified. For example, > row-filtering/column-masking does this, but also other optimizations/features. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (HIVE-15970) Fix merge to work in presents of AST rewrites
[ https://issues.apache.org/jira/browse/HIVE-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman reassigned HIVE-15970: - > Fix merge to work in presents of AST rewrites > - > > Key: HIVE-15970 > URL: https://issues.apache.org/jira/browse/HIVE-15970 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Critical > > The issue is that Context.getDestNamePrefix() caches ASTNode objects to > figure out if a given insert branch of multi-insert is insert, update or > delete but objects change when the tree is modified. For example, > row-filtering/column-masking does this, but also other optimizations/features. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15908) OperationLog's LogFile writer should have autoFlush turned on
[
https://issues.apache.org/jira/browse/HIVE-15908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872503#comment-15872503
]
Thejas M Nair commented on HIVE-15908:
--
Are you testing with the master branch ?
HiveStatemet.DEFAULT_FETCH_SIZE has been 1000 for a while. But I am not sure
why that would have an impact.
HIVE-14618 has changes to have shorter timeouts for the getOperationStatus long
polling calls, which has similar impact like what Hue is doing. That could be
what you are hitting.
But it looks like it didn't change the beeline sleep timeouts for log fetches.
We could have a step function for that as well.
> OperationLog's LogFile writer should have autoFlush turned on
> -
>
> Key: HIVE-15908
> URL: https://issues.apache.org/jira/browse/HIVE-15908
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 0.13.0
>Reporter: Harsh J
>Priority: Minor
> Attachments: HIVE-15908.000.patch
>
>
> The HS2 offers an API to fetch Operation Log results from the maintained
> OperationLog file. The reader used inside class OperationLog$LogFile class
> reads line-by-line on its input stream, for any lines available from the OS's
> file input perspective.
> The writer inside the same class uses PrintStream to write to the file in
> parallel. However, the PrintStream constructor used sets PrintStream's
> {{autoFlush}} feature in an OFF state. This causes the BufferedWriter used by
> PrintStream to accumulate 8k worth of bytes in memory as the buffer before
> flushing the writes to disk, causing a slowness in the logs streamed back to
> the client. Every line must be ideally flushed entirely as-its-written, for a
> smoother experience.
> I suggest changing the line inside {{OperationLog$LogFile}} that appears as
> below:
> {code}
> out = new PrintStream(new FileOutputStream(file));
> {code}
> Into:
> {code}
> out = new PrintStream(new FileOutputStream(file), true);
> {code}
> This will cause it to use the described autoFlush feature of PrintStream and
> make for a better reader-log-results-streaming experience:
> https://docs.oracle.com/javase/7/docs/api/java/io/PrintStream.html#PrintStream(java.io.OutputStream,%20boolean)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-15938) position alias in order by fails for union queries
[ https://issues.apache.org/jira/browse/HIVE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-15938: Attachment: HIVE-15938.02.patch > position alias in order by fails for union queries > -- > > Key: HIVE-15938 > URL: https://issues.apache.org/jira/browse/HIVE-15938 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15938.01.patch, HIVE-15938.02.patch, > HIVE-15938.patch > > > The problem is that the query introduces a spurious ALLCOLREF -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15969) Failures in TestRemoteHiveMetaStore, TestSetUGIOnOnlyServer
[
https://issues.apache.org/jira/browse/HIVE-15969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872464#comment-15872464
]
Hive QA commented on HIVE-15969:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853301/HIVE-15969.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 30 failed/errored test(s), 10244 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[interval_arithmetic]
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_external_acid]
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_not_acid]
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_view_failure1]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_view_failure2]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[external1]
(batchId=85)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31]
(batchId=133)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testAlterTable
(batchId=194)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testTransactionalValidation
(batchId=194)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testAlterTable
(batchId=197)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testTransactionalValidation
(batchId=197)
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testAlterTable
(batchId=193)
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testTransactionalValidation
(batchId=193)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testAlterTable
(batchId=191)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testTransactionalValidation
(batchId=191)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testAlterTable
(batchId=202)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testTransactionalValidation
(batchId=202)
org.apache.hadoop.hive.ql.security.TestMetastoreAuthorizationProvider.testSimplePrivileges
(batchId=210)
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProvider.testSimplePrivileges
(batchId=208)
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProviderWithACL.testSimplePrivileges
(batchId=220)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=211)
org.apache.hive.hcatalog.api.TestHCatClient.testBasicDDLCommands (batchId=170)
org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=170)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3630/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3630/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3630/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 30 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853301 - PreCommit-HIVE-Build
> Failures in TestRemoteHiveMetaStore, TestSetUGIOnOnlyServer
> ---
>
> Key: HIVE-15969
> URL: https://issues.apache.org/jira/browse/HIVE-15969
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 2.2.0
>Reporter: Thejas M Nair
>Assignee: slim bouguerra
> Attachments: HIVE-15969.2.patch, HIVE-15969.patch
>
>
> Looks like the additional failures in TestRemoteHiveMetaStore,
> TestSetUGIOnOnlyServer
> are related to this patch in HIVE-15877.
> I don't think that change was intended.
> {code}
> ---
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> +++
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> @@ -739,7 +739,10 @@ public void createTable(Table tbl, EnvironmentContext
> envContext) throws Already
> hook.commitCreateTable(tbl);
>}
>
[jira] [Updated] (HIVE-15489) Alternatively use table scan stats for HoS
[ https://issues.apache.org/jira/browse/HIVE-15489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-15489: Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed to the master branch. Thanks [~xuefuz] for the review. > Alternatively use table scan stats for HoS > -- > > Key: HIVE-15489 > URL: https://issues.apache.org/jira/browse/HIVE-15489 > Project: Hive > Issue Type: Improvement > Components: Spark, Statistics >Affects Versions: 2.2.0 >Reporter: Chao Sun >Assignee: Chao Sun > Fix For: 2.2.0 > > Attachments: HIVE-15489.1.patch, HIVE-15489.2.patch, > HIVE-15489.3.patch, HIVE-15489.6.patch, HIVE-15489.7.patch, > HIVE-15489.wip.patch > > > For MapJoin in HoS, we should provide an option to only use stats in the TS > rather than the populated stats in each of the join branch. This could be > pretty conservative but more reliable. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15489) Alternatively use table scan stats for HoS
[ https://issues.apache.org/jira/browse/HIVE-15489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872434#comment-15872434 ] Chao Sun commented on HIVE-15489: - Test failures are un-related. > Alternatively use table scan stats for HoS > -- > > Key: HIVE-15489 > URL: https://issues.apache.org/jira/browse/HIVE-15489 > Project: Hive > Issue Type: Improvement > Components: Spark, Statistics >Affects Versions: 2.2.0 >Reporter: Chao Sun >Assignee: Chao Sun > Attachments: HIVE-15489.1.patch, HIVE-15489.2.patch, > HIVE-15489.3.patch, HIVE-15489.6.patch, HIVE-15489.7.patch, > HIVE-15489.wip.patch > > > For MapJoin in HoS, we should provide an option to only use stats in the TS > rather than the populated stats in each of the join branch. This could be > pretty conservative but more reliable. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15969) Failures in TestRemoteHiveMetaStore, TestSetUGIOnOnlyServer
[
https://issues.apache.org/jira/browse/HIVE-15969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872430#comment-15872430
]
Thejas M Nair commented on HIVE-15969:
--
+1
> Failures in TestRemoteHiveMetaStore, TestSetUGIOnOnlyServer
> ---
>
> Key: HIVE-15969
> URL: https://issues.apache.org/jira/browse/HIVE-15969
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 2.2.0
>Reporter: Thejas M Nair
>Assignee: slim bouguerra
> Attachments: HIVE-15969.2.patch, HIVE-15969.patch
>
>
> Looks like the additional failures in TestRemoteHiveMetaStore,
> TestSetUGIOnOnlyServer
> are related to this patch in HIVE-15877.
> I don't think that change was intended.
> {code}
> ---
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> +++
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> @@ -739,7 +739,10 @@ public void createTable(Table tbl, EnvironmentContext
> envContext) throws Already
> hook.commitCreateTable(tbl);
>}
>success = true;
> -} finally {
> +} catch (Exception e){
> + LOG.error("Got exception from createTable", e);
> +}
> +finally {
>if (!success && (hook != null)) {
> hook.rollbackCreateTable(tbl);
>}
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15959) LLAP: fix headroom calculation and move it to daemon
[
https://issues.apache.org/jira/browse/HIVE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872423#comment-15872423
]
Siddharth Seth commented on HIVE-15959:
---
The code that has been moved from package.py to the ServiceDriver - package.py
was heapSize*1.2. Java is using *2
Computing the headroom in the daemon.
Didn't understand this part (will be 0 very often)
{code}
return Math.max(0, minHeadRoom - headRoom);
{code}
There's normally a 6+GB headroom between (xmx + cache) and container-size - to
account for off heap allocations.
This headroom was to carve out a part of the heap for llap data structures, and
the cache. Think the simples would be to take the configured value (5%) and use
that with JVM.maxAvailable.
> LLAP: fix headroom calculation and move it to daemon
>
>
> Key: HIVE-15959
> URL: https://issues.apache.org/jira/browse/HIVE-15959
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-15959.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15955) make explain formatted to dump JSONObject when user level explain is on
[ https://issues.apache.org/jira/browse/HIVE-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872418#comment-15872418 ] Pengcheng Xiong commented on HIVE-15955: (1) With this change, you will get different json object when you set hive.explain.user = true/false and run explain formatted. (2) I think it may not be hard to change the "outputname" to point to the actual operator. My point is that, it is derived information, and should not be part of the output. And I did not see a case where RS outputs to 2 operators. > make explain formatted to dump JSONObject when user level explain is on > --- > > Key: HIVE-15955 > URL: https://issues.apache.org/jira/browse/HIVE-15955 > Project: Hive > Issue Type: New Feature >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-15955.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15938) position alias in order by fails for union queries
[ https://issues.apache.org/jira/browse/HIVE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872415#comment-15872415 ] Sergey Shelukhin commented on HIVE-15938: - union10 test was changed accidentally in the patch. Couple more tests failed that may be related, will take a look. > position alias in order by fails for union queries > -- > > Key: HIVE-15938 > URL: https://issues.apache.org/jira/browse/HIVE-15938 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15938.01.patch, HIVE-15938.patch > > > The problem is that the query introduces a spurious ALLCOLREF -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15954) LLAP: some Tez INFO logs are too noisy
[ https://issues.apache.org/jira/browse/HIVE-15954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-15954: Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed to master > LLAP: some Tez INFO logs are too noisy > -- > > Key: HIVE-15954 > URL: https://issues.apache.org/jira/browse/HIVE-15954 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 2.2.0 > > Attachments: HIVE-15954.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15955) make explain formatted to dump JSONObject when user level explain is on
[ https://issues.apache.org/jira/browse/HIVE-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872402#comment-15872402 ] Gunther Hagleitner commented on HIVE-15955: --- [~pxiong] I still don't understand: With this change, is the operator id always included in the explain json, regardless of the setting for hive.explain.user? Is the json object different depending on the setting of hive.explain.user? As far as your question to [~jdere] goes: They can, but it's difficult to parse the vertices and hook things up, and also this isn't always possible (you can have multiple operator trees in a single vertex). How hard is it to change the "outputname" to point to the actual operator? (Should be simple because we break that connection explicitly in TezCompiler when we break the plan into tasks. > make explain formatted to dump JSONObject when user level explain is on > --- > > Key: HIVE-15955 > URL: https://issues.apache.org/jira/browse/HIVE-15955 > Project: Hive > Issue Type: New Feature >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-15955.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15964) LLAP: Llap IO codepath not getting invoked due to file column id mismatch
[ https://issues.apache.org/jira/browse/HIVE-15964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872393#comment-15872393 ] Sergey Shelukhin commented on HIVE-15964: - llap_reader test failed on CLI driver. Also, is isAcidScan variable poorly named - why does alter table involve acid? > LLAP: Llap IO codepath not getting invoked due to file column id mismatch > - > > Key: HIVE-15964 > URL: https://issues.apache.org/jira/browse/HIVE-15964 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-15964.1.patch > > > LLAP IO codepath is not getting invoked in certain cases when schema > evolution checks are done. Though "int --> long" (fileType to readerType) > conversions are allowed, the file type columns are not matched correctly when > such conversions need to happen. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15969) Failures in TestRemoteHiveMetaStore, TestSetUGIOnOnlyServer
[
https://issues.apache.org/jira/browse/HIVE-15969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-15969:
--
Attachment: HIVE-15969.2.patch
> Failures in TestRemoteHiveMetaStore, TestSetUGIOnOnlyServer
> ---
>
> Key: HIVE-15969
> URL: https://issues.apache.org/jira/browse/HIVE-15969
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 2.2.0
>Reporter: Thejas M Nair
>Assignee: slim bouguerra
> Attachments: HIVE-15969.2.patch, HIVE-15969.patch
>
>
> Looks like the additional failures in TestRemoteHiveMetaStore,
> TestSetUGIOnOnlyServer
> are related to this patch in HIVE-15877.
> I don't think that change was intended.
> {code}
> ---
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> +++
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> @@ -739,7 +739,10 @@ public void createTable(Table tbl, EnvironmentContext
> envContext) throws Already
> hook.commitCreateTable(tbl);
>}
>success = true;
> -} finally {
> +} catch (Exception e){
> + LOG.error("Got exception from createTable", e);
> +}
> +finally {
>if (!success && (hook != null)) {
> hook.rollbackCreateTable(tbl);
>}
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15969) Failures in TestRemoteHiveMetaStore, TestSetUGIOnOnlyServer
[
https://issues.apache.org/jira/browse/HIVE-15969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872386#comment-15872386
]
slim bouguerra commented on HIVE-15969:
---
After talking with [~thejas] offline we decided it is better to catch the
exception from the final block and bubble up the exception from the create
table function.
> Failures in TestRemoteHiveMetaStore, TestSetUGIOnOnlyServer
> ---
>
> Key: HIVE-15969
> URL: https://issues.apache.org/jira/browse/HIVE-15969
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 2.2.0
>Reporter: Thejas M Nair
>Assignee: slim bouguerra
> Attachments: HIVE-15969.patch
>
>
> Looks like the additional failures in TestRemoteHiveMetaStore,
> TestSetUGIOnOnlyServer
> are related to this patch in HIVE-15877.
> I don't think that change was intended.
> {code}
> ---
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> +++
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> @@ -739,7 +739,10 @@ public void createTable(Table tbl, EnvironmentContext
> envContext) throws Already
> hook.commitCreateTable(tbl);
>}
>success = true;
> -} finally {
> +} catch (Exception e){
> + LOG.error("Got exception from createTable", e);
> +}
> +finally {
>if (!success && (hook != null)) {
> hook.rollbackCreateTable(tbl);
>}
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
