[jira] [Commented] (HIVE-11097) HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases
[ https://issues.apache.org/jira/browse/HIVE-11097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15142094#comment-15142094 ] Shannon Ladymon commented on HIVE-11097: [~prasanth_j], thank you for the details on how to add a test case. I've added this to the wiki. Please let me know if there are any changes/clarifications needed to what I put in the wiki: * [Hive Developer FAQ - How do I add a test case? | https://cwiki.apache.org/confluence/display/Hive/HiveDeveloperFAQ#HiveDeveloperFAQ-HowdoIaddatestcase?] > HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases > - > > Key: HIVE-11097 > URL: https://issues.apache.org/jira/browse/HIVE-11097 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.13.0, 0.14.0, 0.13.1, 1.0.0, 1.2.0 > Environment: Hive 0.13.1, Hive 2.0.0, hadoop 2.4.1 >Reporter: Wan Chang >Assignee: Wan Chang >Priority: Critical > Fix For: 1.3.0, 2.1.0 > > Attachments: HIVE-11097.1.patch, HIVE-11097.2.patch, > HIVE-11097.3.patch, HIVE-11097.4.patch, HIVE-11097.5.patch > > > Say we have a sql as > {code} > create table if not exists test_orc_src (a int, b int, c int) stored as orc; > create table if not exists test_orc_src2 (a int, b int, d int) stored as orc; > insert overwrite table test_orc_src select 1,2,3 from src limit 1; > insert overwrite table test_orc_src2 select 1,2,4 from src limit 1; > set hive.auto.convert.join = false; > set hive.execution.engine=mr; > select > tb.c > from test.test_orc_src tb > join (select * from test.test_orc_src2) tm > on tb.a = tm.a > where tb.b = 2 > {code} > The correct result is 3 but it produced no result. > I find that in HiveInputFormat.pushProjectionsAndFilters > {code} > match = splitPath.startsWith(key) || splitPathWithNoSchema.startsWith(key); > {code} > It uses startsWith to combine aliases with path, so tm will match two alias > in this case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11097) HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases
[ https://issues.apache.org/jira/browse/HIVE-11097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15124728#comment-15124728 ] Hive QA commented on HIVE-11097: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12784980/HIVE-11097.5.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 10048 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniTezCliDriver.org.apache.hadoop.hive.cli.TestMiniTezCliDriver org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cbo_limit org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_hybridgrace_hashjoin_1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge3 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge9 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_smb_main org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union_multiinsert org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_update_two_cols org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_between_columns org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_date_1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_expressions org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_math_funcs org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_partition_diff_num_cols org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_10 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_6 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testGetPartitionSpecs_WithAndWithoutPartitionGrouping org.apache.hive.jdbc.TestSSL.testSSLVersion org.apache.hive.service.cli.TestEmbeddedThriftBinaryCLIService.testExecuteStatementAsync {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6795/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6795/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6795/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 20 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12784980 - PreCommit-HIVE-TRUNK-Build > HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases > - > > Key: HIVE-11097 > URL: https://issues.apache.org/jira/browse/HIVE-11097 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.13.0, 0.14.0, 0.13.1, 1.0.0, 1.2.0 > Environment: Hive 0.13.1, Hive 2.0.0, hadoop 2.4.1 >Reporter: Wan Chang >Assignee: Wan Chang >Priority: Critical > Attachments: HIVE-11097.1.patch, HIVE-11097.2.patch, > HIVE-11097.3.patch, HIVE-11097.4.patch, HIVE-11097.5.patch > > > Say we have a sql as > {code} > create table if not exists test_orc_src (a int, b int, c int) stored as orc; > create table if not exists test_orc_src2 (a int, b int, d int) stored as orc; > insert overwrite table test_orc_src select 1,2,3 from src limit 1; > insert overwrite table test_orc_src2 select 1,2,4 from src limit 1; > set hive.auto.convert.join = false; > set hive.execution.engine=mr; > select > tb.c > from test.test_orc_src tb > join (select * from test.test_orc_src2) tm > on tb.a = tm.a > where tb.b = 2 > {code} > The correct result is 3 but it produced no result. > I find that in HiveInputFormat.pushProjectionsAndFilters > {code} > match = splitPath.startsWith(key) || splitPathWithNoSchema.startsWith(key); > {code} > It uses startsWith to combine aliases with path, so tm will match two alias > in this case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11097) HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases
[ https://issues.apache.org/jira/browse/HIVE-11097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110204#comment-15110204 ] Wan Chang commented on HIVE-11097: -- [~prasanth_j] symlink_text_input_format.q test failure relates to the patch. I have fixed it and add some comments to specify the scene. > HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases > - > > Key: HIVE-11097 > URL: https://issues.apache.org/jira/browse/HIVE-11097 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.13.0, 0.14.0, 0.13.1, 1.0.0, 1.2.0 > Environment: Hive 0.13.1, Hive 2.0.0, hadoop 2.4.1 >Reporter: Wan Chang >Assignee: Wan Chang >Priority: Critical > Attachments: HIVE-11097.1.patch, HIVE-11097.2.patch, > HIVE-11097.3.patch, HIVE-11097.4.patch > > > Say we have a sql as > {code} > create table if not exists test_orc_src (a int, b int, c int) stored as orc; > create table if not exists test_orc_src2 (a int, b int, d int) stored as orc; > insert overwrite table test_orc_src select 1,2,3 from src limit 1; > insert overwrite table test_orc_src2 select 1,2,4 from src limit 1; > set hive.auto.convert.join = false; > set hive.execution.engine=mr; > select > tb.c > from test.test_orc_src tb > join (select * from test.test_orc_src2) tm > on tb.a = tm.a > where tb.b = 2 > {code} > The correct result is 3 but it produced no result. > I find that in HiveInputFormat.pushProjectionsAndFilters > {code} > match = splitPath.startsWith(key) || splitPathWithNoSchema.startsWith(key); > {code} > It uses startsWith to combine aliases with path, so tm will match two alias > in this case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11097) HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases
[ https://issues.apache.org/jira/browse/HIVE-11097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15109674#comment-15109674 ] Prasanth Jayachandran commented on HIVE-11097: -- [~wanchang] Thanks for adding the test case! symlink_text_input_format.q test failure looks related to this patch. Not sure why it did not fail the first time. Could you please take a look? If you think it's not related then I will go ahead and commit the path. > HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases > - > > Key: HIVE-11097 > URL: https://issues.apache.org/jira/browse/HIVE-11097 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.13.0, 0.14.0, 0.13.1, 1.0.0, 1.2.0 > Environment: Hive 0.13.1, Hive 2.0.0, hadoop 2.4.1 >Reporter: Wan Chang >Assignee: Wan Chang >Priority: Critical > Attachments: HIVE-11097.1.patch, HIVE-11097.2.patch, > HIVE-11097.3.patch > > > Say we have a sql as > {code} > create table if not exists test_orc_src (a int, b int, c int) stored as orc; > create table if not exists test_orc_src2 (a int, b int, d int) stored as orc; > insert overwrite table test_orc_src select 1,2,3 from src limit 1; > insert overwrite table test_orc_src2 select 1,2,4 from src limit 1; > set hive.auto.convert.join = false; > set hive.execution.engine=mr; > select > tb.c > from test.test_orc_src tb > join (select * from test.test_orc_src2) tm > on tb.a = tm.a > where tb.b = 2 > {code} > The correct result is 3 but it produced no result. > I find that in HiveInputFormat.pushProjectionsAndFilters > {code} > match = splitPath.startsWith(key) || splitPathWithNoSchema.startsWith(key); > {code} > It uses startsWith to combine aliases with path, so tm will match two alias > in this case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11097) HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases
[ https://issues.apache.org/jira/browse/HIVE-11097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107678#comment-15107678 ] Hive QA commented on HIVE-11097: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12783035/HIVE-11097.3.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10011 tests executed *Failed tests:* {noformat} TestHWISessionManager - did not produce a TEST-*.xml file TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testMultiSessionMultipleUse org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testSingleSessionMultipleUse org.apache.hive.jdbc.TestSSL.testSSLVersion {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6674/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6674/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6674/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 7 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12783035 - PreCommit-HIVE-TRUNK-Build > HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases > - > > Key: HIVE-11097 > URL: https://issues.apache.org/jira/browse/HIVE-11097 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.13.0, 0.14.0, 0.13.1, 1.0.0, 1.2.0 > Environment: Hive 0.13.1, Hive 2.0.0, hadoop 2.4.1 >Reporter: Wan Chang >Assignee: Wan Chang >Priority: Critical > Attachments: HIVE-11097.1.patch, HIVE-11097.2.patch, > HIVE-11097.3.patch > > > Say we have a sql as > {code} > create table if not exists test_orc_src (a int, b int, c int) stored as orc; > create table if not exists test_orc_src2 (a int, b int, d int) stored as orc; > insert overwrite table test_orc_src select 1,2,3 from src limit 1; > insert overwrite table test_orc_src2 select 1,2,4 from src limit 1; > set hive.auto.convert.join = false; > set hive.execution.engine=mr; > select > tb.c > from test.test_orc_src tb > join (select * from test.test_orc_src2) tm > on tb.a = tm.a > where tb.b = 2 > {code} > The correct result is 3 but it produced no result. > I find that in HiveInputFormat.pushProjectionsAndFilters > {code} > match = splitPath.startsWith(key) || splitPathWithNoSchema.startsWith(key); > {code} > It uses startsWith to combine aliases with path, so tm will match two alias > in this case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11097) HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases
[ https://issues.apache.org/jira/browse/HIVE-11097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15106468#comment-15106468 ] Wan Chang commented on HIVE-11097: -- Hi [~prasanth_j], I use hive0.13.1 and the bug occurs with some complex sql. But I didn't reproduce the case on the master branch. I don't know whether it has been fix yet. > HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases > - > > Key: HIVE-11097 > URL: https://issues.apache.org/jira/browse/HIVE-11097 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.13.0, 0.14.0, 0.13.1, 1.0.0, 1.2.0 > Environment: Hive 0.13.1, Hive 2.0.0, hadoop 2.4.1 >Reporter: Wan Chang >Assignee: Wan Chang >Priority: Critical > Attachments: HIVE-11097.1.patch, HIVE-11097.2.patch, > HIVE-11097.3.patch > > > Say we have a sql as > {code} > create table if not exists test_orc_src (a int, b int, c int) stored as orc; > create table if not exists test_orc_src2 (a int, b int, d int) stored as orc; > insert overwrite table test_orc_src select 1,2,3 from src limit 1; > insert overwrite table test_orc_src2 select 1,2,4 from src limit 1; > set hive.auto.convert.join = false; > set hive.execution.engine=mr; > select > tb.c > from test.test_orc_src tb > join (select * from test.test_orc_src2) tm > on tb.a = tm.a > where tb.b = 2 > {code} > The correct result is 3 but it produced no result. > I find that in HiveInputFormat.pushProjectionsAndFilters > {code} > match = splitPath.startsWith(key) || splitPathWithNoSchema.startsWith(key); > {code} > It uses startsWith to combine aliases with path, so tm will match two alias > in this case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11097) HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases
[ https://issues.apache.org/jira/browse/HIVE-11097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15104953#comment-15104953 ] Wan Chang commented on HIVE-11097: -- [~prasanth_j] Thanks for your information. I will update the patch soon. > HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases > - > > Key: HIVE-11097 > URL: https://issues.apache.org/jira/browse/HIVE-11097 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.13.0, 0.14.0, 0.13.1, 1.0.0, 1.2.0 > Environment: Hive 0.13.1, Hive 2.0.0, hadoop 2.4.1 >Reporter: Wan Chang >Assignee: Wan Chang >Priority: Critical > Attachments: HIVE-11097.1.patch > > > Say we have a sql as > {code} > create table if not exists test_orc_src (a int, b int, c int) stored as orc; > create table if not exists test_orc_src2 (a int, b int, d int) stored as orc; > insert overwrite table test_orc_src select 1,2,3 from src limit 1; > insert overwrite table test_orc_src2 select 1,2,4 from src limit 1; > set hive.auto.convert.join = false; > set hive.execution.engine=mr; > select > tb.c > from test.test_orc_src tb > join (select * from test.test_orc_src2) tm > on tb.a = tm.a > where tb.b = 2 > {code} > The correct result is 3 but it produced no result. > I find that in HiveInputFormat.pushProjectionsAndFilters > {code} > match = splitPath.startsWith(key) || splitPathWithNoSchema.startsWith(key); > {code} > It uses startsWith to combine aliases with path, so tm will match two alias > in this case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11097) HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases
[ https://issues.apache.org/jira/browse/HIVE-11097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15104913#comment-15104913 ] Prasanth Jayachandran commented on HIVE-11097: -- [~wanchang] The patch looks good to me. Can you please add the test case provided in the description to qfile test suite? You can do so by copying the tests to a new file under ql/src/test/queries/clientpositive/.q. Add the to itests/src/test/resources/testconfiguration.properties to variable "minimr.query.files". First time when you run the test case, you can generate the golden (output) files by doing the following steps 1) compile hive source from top level hive directory {code} mvn clean install -DskipTests {code} 2) compile itests {code} cd itests mvn clean install -DskipTests {code} 3) Run test and generate output file {code} cd qtest mvn test -Dtest=TestMinimrCliDriver -Dqfile=.q -Dtest.output.overwrite=true {code} With the above steps, your patch should be having java file, q file and q.out file. Let me know if you need more information. +1 for the current patch. > HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases > - > > Key: HIVE-11097 > URL: https://issues.apache.org/jira/browse/HIVE-11097 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.13.0, 0.14.0, 0.13.1, 1.0.0, 1.2.0 > Environment: Hive 0.13.1, Hive 2.0.0, hadoop 2.4.1 >Reporter: Wan Chang >Assignee: Wan Chang >Priority: Critical > Attachments: HIVE-11097.1.patch > > > Say we have a sql as > {code} > create table if not exists test_orc_src (a int, b int, c int) stored as orc; > create table if not exists test_orc_src2 (a int, b int, d int) stored as orc; > insert overwrite table test_orc_src select 1,2,3 from src limit 1; > insert overwrite table test_orc_src2 select 1,2,4 from src limit 1; > set hive.auto.convert.join = false; > set hive.execution.engine=mr; > select > tb.c > from test.test_orc_src tb > join (select * from test.test_orc_src2) tm > on tb.a = tm.a > where tb.b = 2 > {code} > The correct result is 3 but it produced no result. > I find that in HiveInputFormat.pushProjectionsAndFilters > {code} > match = splitPath.startsWith(key) || splitPathWithNoSchema.startsWith(key); > {code} > It uses startsWith to combine aliases with path, so tm will match two alias > in this case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11097) HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases
[ https://issues.apache.org/jira/browse/HIVE-11097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14613563#comment-14613563 ] Wan Chang commented on HIVE-11097: -- Hi [~ashutoshc], would you help to review this patch please? > HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases > - > > Key: HIVE-11097 > URL: https://issues.apache.org/jira/browse/HIVE-11097 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.13.0, 0.14.0, 0.13.1, 1.0.0, 1.2.0 > Environment: Hive 0.13.1, Hive 2.0.0, hadoop 2.4.1 >Reporter: Wan Chang >Assignee: Wan Chang >Priority: Critical > Attachments: HIVE-11097.1.patch > > > Say we have a sql as > {code} > create table if not exists test_orc_src (a int, b int, c int) stored as orc; > create table if not exists test_orc_src2 (a int, b int, d int) stored as orc; > insert overwrite table test_orc_src select 1,2,3 from src limit 1; > insert overwrite table test_orc_src2 select 1,2,4 from src limit 1; > set hive.auto.convert.join = false; > set hive.execution.engine=mr; > select > tb.c > from test.test_orc_src tb > join (select * from test.test_orc_src2) tm > on tb.a = tm.a > where tb.b = 2 > {code} > The correct result is 3 but it produced no result. > I find that in HiveInputFormat.pushProjectionsAndFilters > {code} > match = splitPath.startsWith(key) || splitPathWithNoSchema.startsWith(key); > {code} > It uses startsWith to combine aliases with path, so tm will match two alias > in this case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11097) HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases
[ https://issues.apache.org/jira/browse/HIVE-11097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600719#comment-14600719 ] Wan Chang commented on HIVE-11097: -- Hi [~jvs], would you help to review this? > HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases > - > > Key: HIVE-11097 > URL: https://issues.apache.org/jira/browse/HIVE-11097 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.13.0, 0.14.0, 0.13.1, 1.0.0, 1.2.0 > Environment: Hive 0.13.1, Hive 2.0.0, hadoop 2.4.1 >Reporter: Wan Chang >Priority: Critical > Attachments: HIVE-11097.1.patch > > > Say we have a sql as > {code} > create table if not exists test_orc_src (a int, b int, c int) stored as orc; > create table if not exists test_orc_src2 (a int, b int, d int) stored as orc; > insert overwrite table test_orc_src select 1,2,3 from src limit 1; > insert overwrite table test_orc_src2 select 1,2,4 from src limit 1; > set hive.auto.convert.join = false; > set hive.execution.engine=mr; > select > tb.c > from test.test_orc_src tb > join (select * from test.test_orc_src2) tm > on tb.a = tm.a > where tb.b = 2 > {code} > The correct result is 3 but it produced no result. > I find that in HiveInputFormat.pushProjectionsAndFilters > {code} > match = splitPath.startsWith(key) || splitPathWithNoSchema.startsWith(key); > {code} > It uses startsWith to combine aliases with path, so tm will match two alias > in this case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11097) HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases
[ https://issues.apache.org/jira/browse/HIVE-11097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600338#comment-14600338 ] Hive QA commented on HIVE-11097: {color:green}Overall{color}: +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12741659/HIVE-11097.1.patch {color:green}SUCCESS:{color} +1 9019 tests passed Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/4370/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/4370/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-4370/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12741659 - PreCommit-HIVE-TRUNK-Build > HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases > - > > Key: HIVE-11097 > URL: https://issues.apache.org/jira/browse/HIVE-11097 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.13.0, 0.14.0, 0.13.1, 1.0.0, 1.2.0 > Environment: Hive 0.13.1, Hive 2.0.0, hadoop 2.4.1 >Reporter: Wan Chang >Priority: Critical > Attachments: HIVE-11097.1.patch > > > Say we have a sql as > {code} > create table if not exists test_orc_src (a int, b int, c int) stored as orc; > create table if not exists test_orc_src2 (a int, b int, d int) stored as orc; > insert overwrite table test_orc_src select 1,2,3 from src limit 1; > insert overwrite table test_orc_src2 select 1,2,4 from src limit 1; > set hive.auto.convert.join = false; > set hive.execution.engine=mr; > select > tb.c > from test.test_orc_src tb > join (select * from test.test_orc_src2) tm > on tb.a = tm.a > where tb.b = 2 > {code} > The correct result is 3 but it produced no result. > I find that in HiveInputFormat.pushProjectionsAndFilters > {code} > match = splitPath.startsWith(key) || splitPathWithNoSchema.startsWith(key); > {code} > It uses startsWith to combine aliases with path, so tm will match two alias > in this case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)