[jira] [Commented] (HIVE-15891) Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
[
https://issues.apache.org/jira/browse/HIVE-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873928#comment-15873928
]
Hive QA commented on HIVE-15891:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853503/HIVE-15891.4.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 10249 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[interval_arithmetic]
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31]
(batchId=133)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=211)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3653/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3653/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3653/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853503 - PreCommit-HIVE-Build
> Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
> ---
>
> Key: HIVE-15891
> URL: https://issues.apache.org/jira/browse/HIVE-15891
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 2.2.0
>Reporter: Wei Zheng
>Assignee: Wei Zheng
> Attachments: HIVE-15891.1.patch, HIVE-15891.2.patch,
> HIVE-15891.3.patch, HIVE-15891.4.patch
>
>
> Currently ACID UpdateDeleteSemanticAnalyzer directly manipulates the AST tree
> but it's different from the general approach of modifying the token stream
> and thus will cause AST tree mismatch if there is any rewrite happening after
> UpdateDeleteSemanticAnalyzer.
> The long term solution will be to rewrite the AST handling logic in
> UpdateDeleteSemanticAnalyzer, to make it consistent with the general approach.
> This ticket will for now detect the error prone cases and fail early.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15891) Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
[ https://issues.apache.org/jira/browse/HIVE-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872851#comment-15872851 ] Eugene Koifman commented on HIVE-15891: --- +1 > Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast > --- > > Key: HIVE-15891 > URL: https://issues.apache.org/jira/browse/HIVE-15891 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Attachments: HIVE-15891.1.patch, HIVE-15891.2.patch, > HIVE-15891.3.patch > > > Currently ACID UpdateDeleteSemanticAnalyzer directly manipulates the AST tree > but it's different from the general approach of modifying the token stream > and thus will cause AST tree mismatch if there is any rewrite happening after > UpdateDeleteSemanticAnalyzer. > The long term solution will be to rewrite the AST handling logic in > UpdateDeleteSemanticAnalyzer, to make it consistent with the general approach. > This ticket will for now detect the error prone cases and fail early. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15891) Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
[ https://issues.apache.org/jira/browse/HIVE-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872349#comment-15872349 ] Wei Zheng commented on HIVE-15891: -- I think it should, because as long as query involves update operations on acid table we have no guarantee how the AST will be after the rewrite. So we should throw the error. Current row filtering implementation has a vague distinction of "not supporting acid". I think this should fall in the "unsupported" category for now, until the AST manipulation is consistent. > Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast > --- > > Key: HIVE-15891 > URL: https://issues.apache.org/jira/browse/HIVE-15891 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Attachments: HIVE-15891.1.patch, HIVE-15891.2.patch, > HIVE-15891.3.patch > > > Currently ACID UpdateDeleteSemanticAnalyzer directly manipulates the AST tree > but it's different from the general approach of modifying the token stream > and thus will cause AST tree mismatch if there is any rewrite happening after > UpdateDeleteSemanticAnalyzer. > The long term solution will be to rewrite the AST handling logic in > UpdateDeleteSemanticAnalyzer, to make it consistent with the general approach. > This ticket will for now detect the error prone cases and fail early. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15891) Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
[ https://issues.apache.org/jira/browse/HIVE-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872334#comment-15872334 ] Eugene Koifman commented on HIVE-15891: --- I would argue that it should not... > Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast > --- > > Key: HIVE-15891 > URL: https://issues.apache.org/jira/browse/HIVE-15891 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Attachments: HIVE-15891.1.patch, HIVE-15891.2.patch, > HIVE-15891.3.patch > > > Currently ACID UpdateDeleteSemanticAnalyzer directly manipulates the AST tree > but it's different from the general approach of modifying the token stream > and thus will cause AST tree mismatch if there is any rewrite happening after > UpdateDeleteSemanticAnalyzer. > The long term solution will be to rewrite the AST handling logic in > UpdateDeleteSemanticAnalyzer, to make it consistent with the general approach. > This ticket will for now detect the error prone cases and fail early. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15891) Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
[ https://issues.apache.org/jira/browse/HIVE-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872330#comment-15872330 ] Wei Zheng commented on HIVE-15891: -- Yes it will. I can add another test for that. > Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast > --- > > Key: HIVE-15891 > URL: https://issues.apache.org/jira/browse/HIVE-15891 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Attachments: HIVE-15891.1.patch, HIVE-15891.2.patch, > HIVE-15891.3.patch > > > Currently ACID UpdateDeleteSemanticAnalyzer directly manipulates the AST tree > but it's different from the general approach of modifying the token stream > and thus will cause AST tree mismatch if there is any rewrite happening after > UpdateDeleteSemanticAnalyzer. > The long term solution will be to rewrite the AST handling logic in > UpdateDeleteSemanticAnalyzer, to make it consistent with the general approach. > This ticket will for now detect the error prone cases and fail early. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15891) Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
[ https://issues.apache.org/jira/browse/HIVE-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872258#comment-15872258 ] Eugene Koifman commented on HIVE-15891: --- delete from acidTable where acidTable.t in (select a from nonAcidTable). suppose there is column masking or row-filtering on nonAcidTable but not on acidTable - will you still throw? > Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast > --- > > Key: HIVE-15891 > URL: https://issues.apache.org/jira/browse/HIVE-15891 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Attachments: HIVE-15891.1.patch, HIVE-15891.2.patch, > HIVE-15891.3.patch > > > Currently ACID UpdateDeleteSemanticAnalyzer directly manipulates the AST tree > but it's different from the general approach of modifying the token stream > and thus will cause AST tree mismatch if there is any rewrite happening after > UpdateDeleteSemanticAnalyzer. > The long term solution will be to rewrite the AST handling logic in > UpdateDeleteSemanticAnalyzer, to make it consistent with the general approach. > This ticket will for now detect the error prone cases and fail early. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15891) Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
[ https://issues.apache.org/jira/browse/HIVE-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872237#comment-15872237 ] Wei Zheng commented on HIVE-15891: -- Test failures not related. [~ekoifman] Can you take a look please? > Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast > --- > > Key: HIVE-15891 > URL: https://issues.apache.org/jira/browse/HIVE-15891 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Attachments: HIVE-15891.1.patch, HIVE-15891.2.patch, > HIVE-15891.3.patch > > > Currently ACID UpdateDeleteSemanticAnalyzer directly manipulates the AST tree > but it's different from the general approach of modifying the token stream > and thus will cause AST tree mismatch if there is any rewrite happening after > UpdateDeleteSemanticAnalyzer. > The long term solution will be to rewrite the AST handling logic in > UpdateDeleteSemanticAnalyzer, to make it consistent with the general approach. > This ticket will for now detect the error prone cases and fail early. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15891) Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
[
https://issues.apache.org/jira/browse/HIVE-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871132#comment-15871132
]
Hive QA commented on HIVE-15891:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853161/HIVE-15891.3.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 32 failed/errored test(s), 10247 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_external_acid]
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_not_acid]
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_view_failure1]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_view_failure2]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[external1]
(batchId=85)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31]
(batchId=133)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testAlterTable
(batchId=194)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testTransactionalValidation
(batchId=194)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testAlterTable
(batchId=197)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testTransactionalValidation
(batchId=197)
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testAlterTable
(batchId=193)
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testTransactionalValidation
(batchId=193)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testAlterTable
(batchId=191)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testTransactionalValidation
(batchId=191)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testAlterTable
(batchId=202)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testTransactionalValidation
(batchId=202)
org.apache.hadoop.hive.ql.security.TestMetastoreAuthorizationProvider.testSimplePrivileges
(batchId=210)
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProvider.testSimplePrivileges
(batchId=208)
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProviderWithACL.testSimplePrivileges
(batchId=220)
org.apache.hive.hcatalog.api.TestHCatClient.testBasicDDLCommands (batchId=170)
org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=170)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.createTable
(batchId=221)
org.apache.hive.jdbc.TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery
(batchId=217)
org.apache.hive.service.cli.TestEmbeddedThriftBinaryCLIService.testExecuteStatementAsync
(batchId=213)
org.apache.hive.service.cli.thrift.TestThriftCLIServiceWithBinary.testExecuteStatementAsync
(batchId=213)
org.apache.hive.service.cli.thrift.TestThriftCLIServiceWithHttp.testExecuteStatementAsync
(batchId=213)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3613/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3613/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3613/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 32 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853161 - PreCommit-HIVE-Build
> Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
> ---
>
> Key: HIVE-15891
> URL: https://issues.apache.org/jira/browse/HIVE-15891
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 2.2.0
>Reporter: Wei Zheng
>Assignee: Wei Zheng
> Attachments: HIVE-15891.1.patch, HIVE-15891.2.patch,
> HIVE-15891.3.patch
>
>
> Currently ACID UpdateDeleteSemanticAnalyzer directly manipulates the AST tree
> but it's different from the general approach of modifying the token stream
> and thus will cause AST tree mismatch if there is
[jira] [Commented] (HIVE-15891) Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
[ https://issues.apache.org/jira/browse/HIVE-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869118#comment-15869118 ] Wei Zheng commented on HIVE-15891: -- It fails for all merge queries > Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast > --- > > Key: HIVE-15891 > URL: https://issues.apache.org/jira/browse/HIVE-15891 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Attachments: HIVE-15891.1.patch, HIVE-15891.2.patch > > > Currently ACID UpdateDeleteSemanticAnalyzer directly manipulates the AST tree > but it's different from the general approach of modifying the token stream > and thus will cause AST tree mismatch if there is any rewrite happening after > UpdateDeleteSemanticAnalyzer. > The long term solution will be to rewrite the AST handling logic in > UpdateDeleteSemanticAnalyzer, to make it consistent with the general approach. > This ticket will for now detect the error prone cases and fail early. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15891) Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
[ https://issues.apache.org/jira/browse/HIVE-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868933#comment-15868933 ] Eugene Koifman commented on HIVE-15891: --- does it fail any Merge or some specific one? > Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast > --- > > Key: HIVE-15891 > URL: https://issues.apache.org/jira/browse/HIVE-15891 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Attachments: HIVE-15891.1.patch, HIVE-15891.2.patch > > > Currently ACID UpdateDeleteSemanticAnalyzer directly manipulates the AST tree > but it's different from the general approach of modifying the token stream > and thus will cause AST tree mismatch if there is any rewrite happening after > UpdateDeleteSemanticAnalyzer. > The long term solution will be to rewrite the AST handling logic in > UpdateDeleteSemanticAnalyzer, to make it consistent with the general approach. > This ticket will for now detect the error prone cases and fail early. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15891) Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
[
https://issues.apache.org/jira/browse/HIVE-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868906#comment-15868906
]
Wei Zheng commented on HIVE-15891:
--
I tried that, and it works for existing tests that contain subquery. But this
check will fail MERGE statement.
{code}
if (!hasSubQuery) {
throw new RuntimeException("A matching node should have been found,
otherwise it means " +
"this is an UPDATE/DELETE/MERGE query but the AST got modified!");
}
{code}
Do we want to add another condition to rule out MERGE?
> Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
> ---
>
> Key: HIVE-15891
> URL: https://issues.apache.org/jira/browse/HIVE-15891
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 2.2.0
>Reporter: Wei Zheng
>Assignee: Wei Zheng
> Attachments: HIVE-15891.1.patch, HIVE-15891.2.patch
>
>
> Currently ACID UpdateDeleteSemanticAnalyzer directly manipulates the AST tree
> but it's different from the general approach of modifying the token stream
> and thus will cause AST tree mismatch if there is any rewrite happening after
> UpdateDeleteSemanticAnalyzer.
> The long term solution will be to rewrite the AST handling logic in
> UpdateDeleteSemanticAnalyzer, to make it consistent with the general approach.
> This ticket will for now detect the error prone cases and fail early.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15891) Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
[
https://issues.apache.org/jira/browse/HIVE-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868523#comment-15868523
]
Eugene Koifman commented on HIVE-15891:
---
could you use qpb.getIsSubQ() and pass that to ctx.getDestNamePrefix() to
restore
if (getIsUpdateDeleteMerge()) { check at the end of the loop?
> Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
> ---
>
> Key: HIVE-15891
> URL: https://issues.apache.org/jira/browse/HIVE-15891
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 2.2.0
>Reporter: Wei Zheng
>Assignee: Wei Zheng
> Attachments: HIVE-15891.1.patch, HIVE-15891.2.patch
>
>
> Currently ACID UpdateDeleteSemanticAnalyzer directly manipulates the AST tree
> but it's different from the general approach of modifying the token stream
> and thus will cause AST tree mismatch if there is any rewrite happening after
> UpdateDeleteSemanticAnalyzer.
> The long term solution will be to rewrite the AST handling logic in
> UpdateDeleteSemanticAnalyzer, to make it consistent with the general approach.
> This ticket will for now detect the error prone cases and fail early.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15891) Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
[ https://issues.apache.org/jira/browse/HIVE-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868254#comment-15868254 ] Wei Zheng commented on HIVE-15891: -- [~ekoifman] Can you take a look please? > Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast > --- > > Key: HIVE-15891 > URL: https://issues.apache.org/jira/browse/HIVE-15891 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 2.2.0 >Reporter: Wei Zheng >Assignee: Wei Zheng > Attachments: HIVE-15891.1.patch, HIVE-15891.2.patch > > > Currently ACID UpdateDeleteSemanticAnalyzer directly manipulates the AST tree > but it's different from the general approach of modifying the token stream > and thus will cause AST tree mismatch if there is any rewrite happening after > UpdateDeleteSemanticAnalyzer. > The long term solution will be to rewrite the AST handling logic in > UpdateDeleteSemanticAnalyzer, to make it consistent with the general approach. > This ticket will for now detect the error prone cases and fail early. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15891) Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
[
https://issues.apache.org/jira/browse/HIVE-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867361#comment-15867361
]
Hive QA commented on HIVE-15891:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12852720/HIVE-15891.2.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 10237 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[external_table_with_space_in_location_path]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[index_bitmap3]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[infer_bucket_sort_map_operators]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[parallel_orderby]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[remote_script]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[schemeAuthority]
(batchId=161)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31]
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver
(batchId=230)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3559/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3559/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3559/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12852720 - PreCommit-HIVE-Build
> Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
> ---
>
> Key: HIVE-15891
> URL: https://issues.apache.org/jira/browse/HIVE-15891
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 2.2.0
>Reporter: Wei Zheng
>Assignee: Wei Zheng
> Attachments: HIVE-15891.1.patch, HIVE-15891.2.patch
>
>
> Currently ACID UpdateDeleteSemanticAnalyzer directly manipulates the AST tree
> but it's different from the general approach of modifying the token stream
> and thus will cause AST tree mismatch if there is any rewrite happening after
> UpdateDeleteSemanticAnalyzer.
> The long term solution will be to rewrite the AST handling logic in
> UpdateDeleteSemanticAnalyzer, to make it consistent with the general approach.
> This ticket will for now detect the error prone cases and fail early.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15891) Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
[
https://issues.apache.org/jira/browse/HIVE-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15864813#comment-15864813
]
Hive QA commented on HIVE-15891:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12852440/HIVE-15891.1.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 41 failed/errored test(s), 10241 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_subquery]
(batchId=36)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sqlmerge]
(batchId=153)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.ql.TestTxnCommands.testDeleteIn (batchId=270)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeDeleteUpdate (batchId=270)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeOnTezEdges (batchId=270)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeType2SCD01 (batchId=270)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeType2SCD02 (batchId=270)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeUpdateDelete (batchId=270)
org.apache.hadoop.hive.ql.TestTxnCommands.testQuotedIdentifier (batchId=270)
org.apache.hadoop.hive.ql.TestTxnCommands.testQuotedIdentifier2 (batchId=270)
org.apache.hadoop.hive.ql.TestTxnCommands2.testDeleteIn (batchId=258)
org.apache.hadoop.hive.ql.TestTxnCommands2.testDynamicPartitionsMerge
(batchId=258)
org.apache.hadoop.hive.ql.TestTxnCommands2.testDynamicPartitionsMerge2
(batchId=258)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMerge (batchId=258)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMerge2 (batchId=258)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMerge3 (batchId=258)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMergeWithPredicate (batchId=258)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdate.testDeleteIn
(batchId=268)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdate.testDynamicPartitionsMerge
(batchId=268)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdate.testDynamicPartitionsMerge2
(batchId=268)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdate.testMerge
(batchId=268)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdate.testMerge2
(batchId=268)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdate.testMerge3
(batchId=268)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdate.testMergeWithPredicate
(batchId=268)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testDeleteIn
(batchId=266)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testDynamicPartitionsMerge
(batchId=266)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testDynamicPartitionsMerge2
(batchId=266)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testMerge
(batchId=266)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testMerge2
(batchId=266)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testMerge3
(batchId=266)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testMergeWithPredicate
(batchId=266)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testLocksInSubquery
(batchId=269)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMerge3Way01
(batchId=269)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMerge3Way02
(batchId=269)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMergePartitioned01
(batchId=269)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMergePartitioned02
(batchId=269)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMergeUnpartitioned01
(batchId=269)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMergeUnpartitioned02
(batchId=269)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3524/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3524/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3524/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 41 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12852440 - PreCommit-HIVE-Build
> Detect query rewrite scenario for UPDATE/DELETE/MERGE and fail fast
> ---
>
> Key: HIVE-15891
> URL:
