Hive-trunk-h0.21 - Build # 2146 - Still Failing
Changes for Build #2117 Changes for Build #2118 Changes for Build #2119 Changes for Build #2120 Changes for Build #2121 [navis] HIVE-4572 ColumnPruner cannot preserve RS key columns corresponding to un-selected join keys in columnExprMap (Yin Huai via Navis) [navis] HIVE-4540 JOIN-GRP BY-DISTINCT fails with NPE when mapjoin.mapreduce=true (Gunther Hagleitner via Navis) Changes for Build #2122 Changes for Build #2123 Changes for Build #2124 [gates] HIVE-4543 Broken link in HCat doc (Reader and Writer Interfaces) (Lefty Leverenz via gates) Changes for Build #2125 [daijy] PIG-3337: Fix remaining Window e2e tests Changes for Build #2126 [hashutosh] HIVE-4615 : Invalid column names allowed when created dynamically by a SerDe (Gabriel Reid via Ashutosh Chauhan) [hashutosh] HIVE-3846 : alter view rename NPEs with authorization on. (Teddy Choi via Ashutosh Chauhan) [hashutosh] HIVE-4403 : Running Hive queries on Yarn (MR2) gives warnings related to overriding final parameters (Chu Tong via Ashutosh Chauhan) [hashutosh] HIVE-4610 : HCatalog checkstyle violation after HIVE4578 (Brock Noland via Ashutosh Chauhan) [hashutosh] HIVE-4636 : Failing on TestSemanticAnalysis.testAddReplaceCols in trunk (Navis via Ashutosh Chauhan) [hashutosh] HIVE-4626 : join_vc.q is not deterministic (Navis via Ashutosh Chauhan) [hashutosh] HIVE-4562 : HIVE3393 brought in Jackson library,and these four jars should be packed into hive-exec.jar (caofangkun via Ashutosh Chauhan) [hashutosh] HIVE-4489 : beeline always return the same error message twice (Chaoyu Tang via Ashutosh Chauhan) [hashutosh] HIVE-4510 : HS2 doesn't nest exceptions properly (fun debug times) (Thejas Nair via Ashutosh Chauhan) [hashutosh] HIVE-4535 : hive build fails with hadoop 0.20 (Thejas Nair via Ashutosh Chauhan) Changes for Build #2127 [hashutosh] HIVE-4585 : Remove unused MR Temp file localization from Tasks (Gunther Hagleitner via Ashutosh Chauhan) [hashutosh] HIVE-4418 : TestNegativeCliDriver failure message if cmd succeeds is misleading (Thejas Nair via Ashutosh Chauhan) [navis] HIVE-4620 MR temp directory conflicts in case of parallel execution mode (Prasad Mujumdar via Navis) Changes for Build #2128 [hashutosh] HIVE-4646 : skewjoin.q is failing in hadoop2 (Navis via Ashutosh Chauhan) [hashutosh] HIVE-4377 : Add more comment to https://reviews.facebook.net/D1209 (HIVE2340) : (Navis via Ashutosh Chauhan) [hashutosh] HIVE-4546 : Hive CLI leaves behind the per session resource directory on non-interactive invocation (Prasad Mujumdar via Ashutosh Chauhan) [gates] HIVE-2670 A cluster test utility for Hive (gates and Johnny Zhang via gates) Changes for Build #2129 [hashutosh] HIVE-2304 : Support PreparedStatement.setObject (Ido Hadanny via Ashutosh Chauhan) [hashutosh] HIVE-4526 : auto_sortmerge_join_9.q throws NPE but test is succeeded (Navis via Ashutosh Chauhan) [hashutosh] HIVE-4516 : Fix concurrency bug in serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java (Jon Hartlaub and Navis via Ashutosh Chauhan) [hashutosh] HIVE-4566 : NullPointerException if typeinfo and nativesql commands are executed at beeline before a DB connection is established (Xuefu Zhang via Ashutosh Chauhan) Changes for Build #2130 Changes for Build #2131 [hashutosh] HIVE-4422 : Test output need to be updated for Windows only unit test in TestCliDriver (Shuaishuai Nie via Ashutosh Chauhan) [hashutosh] HIVE-4657 : HCatalog checkstyle violation after HIVE-2670 (Shreepadma Venugopalan via Ashutosh Chauhan) [hashutosh] HIVE-4348 : Unit test compile fail at hbase-handler project on Windows becuase of illegal escape character (Shuaishuai Nie via Ashutosh Chauhan) [hashutosh] HIVE-4554 : Failed to create a table from existing file if file path has spaces (Xuefu Zhang via Ashutosh Chauhan) [hashutosh] HIVE-4459 : Script hcat is overriding HIVE_CONF_DIR variable (Jarek Jarcec Cecho via Ashutosh Chauhan) Changes for Build #2132 [hashutosh] HIVE-4435 : Column stats: Distinct value estimator should use hash functions that are pairwise independent (Shreepadma Venugopalan via Ashutosh Chauhan) Changes for Build #2133 Changes for Build #2134 Changes for Build #2135 Changes for Build #2136 Changes for Build #2137 [navis] HIVE-4549 : JDBC compliance change TABLE_SCHEMA to TABLE_SCHEM (Prasad Mujumdar via Navis) Changes for Build #2138 [navis] HIVE-4691 : orc_createas1.q has minor inconsistency (Brock Noland via Navis) Changes for Build #2139 [hashutosh] HIVE-4712 : Fix TestCliDriver.truncate_* on 0.23 (Brock Noland via Ashutosh Chauhan) Changes for Build #2140 [navis] HIVE-4708 : Fix TestCliDriver.combine2.q on 0.23 (Brock Noland via Navis) Changes for Build #2141 Changes for Build #2142 [hashutosh] HIVE-4717 : Fix non-deterministic TestCliDriver on 0.23 (Brock Noland via Ashutosh Chauhan) Changes for Build #2143 Changes for Build #2144 [hashutosh] HIVE-4713 : Fix
[jira] [Commented] (HIVE-4730) Join on more than 2^31 records on single reducer failed (wrong results)
[ https://issues.apache.org/jira/browse/HIVE-4730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13684192#comment-13684192 ] Gabi Kazav commented on HIVE-4730: -- Hi Navis - on hive 0.7.x is works, on hive 0.10.0, i am compiling the exec jar and it build me hive-exec-0.10.0-SNAPSHOT.jar when i am running hive on ver 0.10.0, i got the following massage (also after clean build): [hdp@hive-1 gabi]$ hive /hive/hive/bin/hive: line 72: [: /hive/hive/lib/hive-exec-0.10.0.jar: binary operator expected Logging initialized using configuration in jar:file:/hive/hive-0.10.0/lib/hive-common-0.10.0.jar!/hive-log4j.properties Hive history file=/tmp/hdp/hive_job_log_hdp_201306151610_458251040.txt hive show tables; FAILED: Error in metadata: MetaException(message:Got exception: org.apache.hadoop.hive.metastore.api.MetaException javax.jdo.JDODataStoreException: An exception was thrown while adding/validating class(es) : Constraint 'COLUMNS_PK' already exists in Schema 'APP'. java.sql.SQLException: Constraint 'COLUMNS_PK' already exists in Schema 'APP'. at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) at org.apache.commons.dbcp.DelegatingStatement.execute(DelegatingStatement.java:264) at org.apache.commons.dbcp.DelegatingStatement.execute(DelegatingStatement.java:264) at org.datanucleus.store.rdbms.table.AbstractTable.executeDdlStatement(AbstractTable.java:730) at org.datanucleus.store.rdbms.table.AbstractTable.executeDdlStatementList(AbstractTable.java:681) at org.datanucleus.store.rdbms.table.AbstractTable.create(AbstractTable.java:402) at org.datanucleus.store.rdbms.table.AbstractTable.exists(AbstractTable.java:458) at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.performTablesValidation(RDBMSStoreManager.java:2689) at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.addClassTablesAndValidate(RDBMSStoreManager.java:2503) at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.run(RDBMSStoreManager.java:2148) at org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:113) at org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:986) at org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:952) at org.datanucleus.store.AbstractStoreManager.addClass(AbstractStoreManager.java:919) at org.datanucleus.store.mapped.MappedStoreManager.getDatastoreClass(MappedStoreManager.java:356) at org.datanucleus.store.rdbms.query.legacy.ExtentHelper.getExtent(ExtentHelper.java:48) at org.datanucleus.store.rdbms.RDBMSStoreManager.getExtent(RDBMSStoreManager.java:1332) at org.datanucleus.ObjectManagerImpl.getExtent(ObjectManagerImpl.java:4149) at org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileCandidates(JDOQLQueryCompiler.java:411) at org.datanucleus.store.rdbms.query.legacy.QueryCompiler.executionCompile(QueryCompiler.java:312) at org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compile(JDOQLQueryCompiler.java:225) at org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.compileInternal(JDOQLQuery.java:175) at org.datanucleus.store.query.Query.executeQuery(Query.java:1628) at org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.executeQuery(JDOQLQuery.java:245) at org.datanucleus.store.query.Query.executeWithArray(Query.java:1499) at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:243) at org.apache.hadoop.hive.metastore.ObjectStore.getTables(ObjectStore.java:781) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:111) at $Proxy6.getTables(Unknown Source) at
[jira] [Commented] (HIVE-4730) Join on more than 2^31 records on single reducer failed (wrong results)
[ https://issues.apache.org/jira/browse/HIVE-4730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13684193#comment-13684193 ] Gabi Kazav commented on HIVE-4730: -- ignore the script error line : /hive/hive/bin/hive: line 72: [: /hive/hive/lib/hive-exec-0.10.0.jar: binary operator expected i had 2 jar files (old and new), fixed it, but i still have the exception: hive show tables; FAILED: Error in metadata: MetaException(message:Got exception: org.apache.hadoop.hive.metastore.api.MetaException javax.jdo.JDODataStoreException: An exception was thrown while adding/validating class(es) : Constraint 'COLUMNS_PK' already exists in Schema 'APP'. java.sql.SQLException: Constraint 'COLUMNS_PK' already exists in Schema 'APP'. at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) at org.apache.commons.dbcp.DelegatingStatement.execute(DelegatingStatement.java:264) at org.apache.commons.dbcp.DelegatingStatement.execute(DelegatingStatement.java:264) at org.datanucleus.store.rdbms.table.AbstractTable.executeDdlStatement(AbstractTable.java:730) at org.datanucleus.store.rdbms.table.AbstractTable.executeDdlStatementList(AbstractTable.java:681) at org.datanucleus.store.rdbms.table.AbstractTable.create(AbstractTable.java:402) at org.datanucleus.store.rdbms.table.AbstractTable.exists(AbstractTable.java:458) at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.performTablesValidation(RDBMSStoreManager.java:2689) at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.addClassTablesAndValidate(RDBMSStoreManager.java:2503) at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.run(RDBMSStoreManager.java:2148) at org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:113) at org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:986) at org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:952) at org.datanucleus.store.AbstractStoreManager.addClass(AbstractStoreManager.java:919) at org.datanucleus.store.mapped.MappedStoreManager.getDatastoreClass(MappedStoreManager.java:356) at org.datanucleus.store.rdbms.query.legacy.ExtentHelper.getExtent(ExtentHelper.java:48) at org.datanucleus.store.rdbms.RDBMSStoreManager.getExtent(RDBMSStoreManager.java:1332) at org.datanucleus.ObjectManagerImpl.getExtent(ObjectManagerImpl.java:4149) at org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileCandidates(JDOQLQueryCompiler.java:411) at org.datanucleus.store.rdbms.query.legacy.QueryCompiler.executionCompile(QueryCompiler.java:312) at org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compile(JDOQLQueryCompiler.java:225) at org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.compileInternal(JDOQLQuery.java:175) at org.datanucleus.store.query.Query.executeQuery(Query.java:1628) at org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.executeQuery(JDOQLQuery.java:245) at org.datanucleus.store.query.Query.executeWithArray(Query.java:1499) at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:243) at org.apache.hadoop.hive.metastore.ObjectStore.getTables(ObjectStore.java:781) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:111) at $Proxy6.getTables(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_tables(HiveMetaStore.java:2327) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at
[jira] [Updated] (HIVE-2655) Ability to define functions in HQL
[ https://issues.apache.org/jira/browse/HIVE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-2655: --- Resolution: Fixed Fix Version/s: 0.12.0 Status: Resolved (was: Patch Available) Marking resolved as this was committed in 54ec1cb0d0540edf7946738bc113e90adcc09a6d. Ability to define functions in HQL -- Key: HIVE-2655 URL: https://issues.apache.org/jira/browse/HIVE-2655 Project: Hive Issue Type: New Feature Components: SQL Reporter: Jonathan Perlow Assignee: Brock Noland Fix For: 0.12.0 Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2655.D915.1.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2655.D915.2.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2655.D915.3.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2655.D915.4.patch, HIVE-2655-10.patch, HIVE-2655-10.patch, HIVE-2655-9.patch Ability to create functions in HQL as a substitute for creating them in Java. Jonathan Chang requested I create this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21 #404
See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/404/ -- [...truncated 58853 lines...] [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/jenkins/hive_2013-06-15_11-07-14_075_7906667758039792375/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/404/artifact/hive/build/service/tmp/hive_job_log_jenkins_201306151107_1611922950.txt [junit] Copying file: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: load data local inpath 'https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Output: default@testhivedrivertable [junit] Copying data from https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt [junit] Loading data to table default.testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select * from testhivedrivertable limit 10 [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/jenkins/hive_2013-06-15_11-07-18_311_6698120760933323630/-mr-1 [junit] POSTHOOK: query: select * from testhivedrivertable limit 10 [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/jenkins/hive_2013-06-15_11-07-18_311_6698120760933323630/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/404/artifact/hive/build/service/tmp/hive_job_log_jenkins_201306151107_2063944099.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/404/artifact/hive/build/service/tmp/hive_job_log_jenkins_201306151107_1038022079.txt [junit] Hive history file=https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/404/artifact/hive/build/service/tmp/hive_job_log_jenkins_201306151107_1455426645.txt [junit] Copying file: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK:
Hive-trunk-hadoop2 - Build # 241 - Still Failing
Changes for Build #210 Changes for Build #211 Changes for Build #212 Changes for Build #213 Changes for Build #214 [navis] HIVE-4572 ColumnPruner cannot preserve RS key columns corresponding to un-selected join keys in columnExprMap (Yin Huai via Navis) [navis] HIVE-4540 JOIN-GRP BY-DISTINCT fails with NPE when mapjoin.mapreduce=true (Gunther Hagleitner via Navis) Changes for Build #215 Changes for Build #216 Changes for Build #217 [gates] HIVE-4543 Broken link in HCat doc (Reader and Writer Interfaces) (Lefty Leverenz via gates) Changes for Build #218 Changes for Build #219 [daijy] PIG-3337: Fix remaining Window e2e tests Changes for Build #220 Changes for Build #221 Changes for Build #222 Changes for Build #223 [hashutosh] HIVE-4610 : HCatalog checkstyle violation after HIVE4578 (Brock Noland via Ashutosh Chauhan) [hashutosh] HIVE-4636 : Failing on TestSemanticAnalysis.testAddReplaceCols in trunk (Navis via Ashutosh Chauhan) [hashutosh] HIVE-4626 : join_vc.q is not deterministic (Navis via Ashutosh Chauhan) [hashutosh] HIVE-4562 : HIVE3393 brought in Jackson library,and these four jars should be packed into hive-exec.jar (caofangkun via Ashutosh Chauhan) [hashutosh] HIVE-4489 : beeline always return the same error message twice (Chaoyu Tang via Ashutosh Chauhan) [hashutosh] HIVE-4510 : HS2 doesn't nest exceptions properly (fun debug times) (Thejas Nair via Ashutosh Chauhan) [hashutosh] HIVE-4535 : hive build fails with hadoop 0.20 (Thejas Nair via Ashutosh Chauhan) Changes for Build #224 [hashutosh] HIVE-4615 : Invalid column names allowed when created dynamically by a SerDe (Gabriel Reid via Ashutosh Chauhan) [hashutosh] HIVE-3846 : alter view rename NPEs with authorization on. (Teddy Choi via Ashutosh Chauhan) [hashutosh] HIVE-4403 : Running Hive queries on Yarn (MR2) gives warnings related to overriding final parameters (Chu Tong via Ashutosh Chauhan) Changes for Build #225 [gates] HIVE-2670 A cluster test utility for Hive (gates and Johnny Zhang via gates) [hashutosh] HIVE-4585 : Remove unused MR Temp file localization from Tasks (Gunther Hagleitner via Ashutosh Chauhan) [hashutosh] HIVE-4418 : TestNegativeCliDriver failure message if cmd succeeds is misleading (Thejas Nair via Ashutosh Chauhan) [navis] HIVE-4620 MR temp directory conflicts in case of parallel execution mode (Prasad Mujumdar via Navis) Changes for Build #226 [hashutosh] HIVE-2304 : Support PreparedStatement.setObject (Ido Hadanny via Ashutosh Chauhan) [hashutosh] HIVE-4526 : auto_sortmerge_join_9.q throws NPE but test is succeeded (Navis via Ashutosh Chauhan) [hashutosh] HIVE-4516 : Fix concurrency bug in serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java (Jon Hartlaub and Navis via Ashutosh Chauhan) [hashutosh] HIVE-4566 : NullPointerException if typeinfo and nativesql commands are executed at beeline before a DB connection is established (Xuefu Zhang via Ashutosh Chauhan) [hashutosh] HIVE-4646 : skewjoin.q is failing in hadoop2 (Navis via Ashutosh Chauhan) [hashutosh] HIVE-4377 : Add more comment to https://reviews.facebook.net/D1209 (HIVE2340) : (Navis via Ashutosh Chauhan) [hashutosh] HIVE-4546 : Hive CLI leaves behind the per session resource directory on non-interactive invocation (Prasad Mujumdar via Ashutosh Chauhan) Changes for Build #227 [hashutosh] HIVE-4422 : Test output need to be updated for Windows only unit test in TestCliDriver (Shuaishuai Nie via Ashutosh Chauhan) [hashutosh] HIVE-4657 : HCatalog checkstyle violation after HIVE-2670 (Shreepadma Venugopalan via Ashutosh Chauhan) [hashutosh] HIVE-4348 : Unit test compile fail at hbase-handler project on Windows becuase of illegal escape character (Shuaishuai Nie via Ashutosh Chauhan) [hashutosh] HIVE-4554 : Failed to create a table from existing file if file path has spaces (Xuefu Zhang via Ashutosh Chauhan) [hashutosh] HIVE-4459 : Script hcat is overriding HIVE_CONF_DIR variable (Jarek Jarcec Cecho via Ashutosh Chauhan) Changes for Build #228 [hashutosh] HIVE-4435 : Column stats: Distinct value estimator should use hash functions that are pairwise independent (Shreepadma Venugopalan via Ashutosh Chauhan) Changes for Build #229 Changes for Build #230 Changes for Build #231 Changes for Build #232 Changes for Build #233 Changes for Build #234 [navis] HIVE-4691 : orc_createas1.q has minor inconsistency (Brock Noland via Navis) [navis] HIVE-4549 : JDBC compliance change TABLE_SCHEMA to TABLE_SCHEM (Prasad Mujumdar via Navis) Changes for Build #235 [hashutosh] HIVE-4712 : Fix TestCliDriver.truncate_* on 0.23 (Brock Noland via Ashutosh Chauhan) Changes for Build #236 [navis] HIVE-4708 : Fix TestCliDriver.combine2.q on 0.23 (Brock Noland via Navis) Changes for Build #237 [hashutosh] HIVE-4717 : Fix non-deterministic TestCliDriver on 0.23 (Brock Noland via Ashutosh Chauhan) Changes for Build #238 Changes for Build #239 [hashutosh] HIVE-4713 : Fix
Jenkins build is back to normal : Hive-0.10.0-SNAPSHOT-h0.20.1 #177
See https://builds.apache.org/job/Hive-0.10.0-SNAPSHOT-h0.20.1/177/
[jira] [Commented] (HIVE-2436) Update project naming and description in Hive website
[ https://issues.apache.org/jira/browse/HIVE-2436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13684527#comment-13684527 ] Brock Noland commented on HIVE-2436: Hi [~cwsteinbach] if you aren't working on this one, do you mind if I take it up? Update project naming and description in Hive website - Key: HIVE-2436 URL: https://issues.apache.org/jira/browse/HIVE-2436 Project: Hive Issue Type: Sub-task Reporter: John Sichi Assignee: Carl Steinbach http://www.apache.org/foundation/marks/pmcs.html#naming -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4113) Optimize select count(1) with RCFile and Orc
[
https://issues.apache.org/jira/browse/HIVE-4113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13684535#comment-13684535
]
Brock Noland commented on HIVE-4113:
[~owen.omalley] would you have some time to look at attached patch? Thanks!
Optimize select count(1) with RCFile and Orc
Key: HIVE-4113
URL: https://issues.apache.org/jira/browse/HIVE-4113
Project: Hive
Issue Type: Bug
Components: File Formats
Reporter: Gopal V
Assignee: Brock Noland
Fix For: 0.12.0
Attachments: HIVE-4113-0.patch
select count(1) loads up every column every row when used with RCFile.
select count(1) from store_sales_10_rc gives
{code}
Job 0: Map: 5 Reduce: 1 Cumulative CPU: 31.73 sec HDFS Read: 234914410
HDFS Write: 8 SUCCESS
{code}
Where as, select count(ss_sold_date_sk) from store_sales_10_rc; reads far
less
{code}
Job 0: Map: 5 Reduce: 1 Cumulative CPU: 29.75 sec HDFS Read: 28145994
HDFS Write: 8 SUCCESS
{code}
Which is 11% of the data size read by the COUNT(1).
This was tracked down to the following code in RCFile.java
{code}
} else {
// TODO: if no column name is specified e.g, in select count(1) from
tt;
// skip all columns, this should be distinguished from the case:
// select * from tt;
for (int i = 0; i skippedColIDs.length; i++) {
skippedColIDs[i] = false;
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 2147 - Still Failing
Changes for Build #2118
Changes for Build #2119
Changes for Build #2120
Changes for Build #2121
[navis] HIVE-4572 ColumnPruner cannot preserve RS key columns corresponding to
un-selected join keys in columnExprMap (Yin Huai via Navis)
[navis] HIVE-4540 JOIN-GRP BY-DISTINCT fails with NPE when
mapjoin.mapreduce=true (Gunther Hagleitner via Navis)
Changes for Build #2122
Changes for Build #2123
Changes for Build #2124
[gates] HIVE-4543 Broken link in HCat doc (Reader and Writer Interfaces) (Lefty
Leverenz via gates)
Changes for Build #2125
[daijy] PIG-3337: Fix remaining Window e2e tests
Changes for Build #2126
[hashutosh] HIVE-4615 : Invalid column names allowed when created dynamically
by a SerDe (Gabriel Reid via Ashutosh Chauhan)
[hashutosh] HIVE-3846 : alter view rename NPEs with authorization on. (Teddy
Choi via Ashutosh Chauhan)
[hashutosh] HIVE-4403 : Running Hive queries on Yarn (MR2) gives warnings
related to overriding final parameters (Chu Tong via Ashutosh Chauhan)
[hashutosh] HIVE-4610 : HCatalog checkstyle violation after HIVE4578 (Brock
Noland via Ashutosh Chauhan)
[hashutosh] HIVE-4636 : Failing on TestSemanticAnalysis.testAddReplaceCols in
trunk (Navis via Ashutosh Chauhan)
[hashutosh] HIVE-4626 : join_vc.q is not deterministic (Navis via Ashutosh
Chauhan)
[hashutosh] HIVE-4562 : HIVE3393 brought in Jackson library,and these four jars
should be packed into hive-exec.jar (caofangkun via Ashutosh Chauhan)
[hashutosh] HIVE-4489 : beeline always return the same error message twice
(Chaoyu Tang via Ashutosh Chauhan)
[hashutosh] HIVE-4510 : HS2 doesn't nest exceptions properly (fun debug times)
(Thejas Nair via Ashutosh Chauhan)
[hashutosh] HIVE-4535 : hive build fails with hadoop 0.20 (Thejas Nair via
Ashutosh Chauhan)
Changes for Build #2127
[hashutosh] HIVE-4585 : Remove unused MR Temp file localization from Tasks
(Gunther Hagleitner via Ashutosh Chauhan)
[hashutosh] HIVE-4418 : TestNegativeCliDriver failure message if cmd succeeds
is misleading (Thejas Nair via Ashutosh Chauhan)
[navis] HIVE-4620 MR temp directory conflicts in case of parallel execution
mode (Prasad Mujumdar via Navis)
Changes for Build #2128
[hashutosh] HIVE-4646 : skewjoin.q is failing in hadoop2 (Navis via Ashutosh
Chauhan)
[hashutosh] HIVE-4377 : Add more comment to https://reviews.facebook.net/D1209
(HIVE2340) : (Navis via Ashutosh Chauhan)
[hashutosh] HIVE-4546 : Hive CLI leaves behind the per session resource
directory on non-interactive invocation (Prasad Mujumdar via Ashutosh Chauhan)
[gates] HIVE-2670 A cluster test utility for Hive (gates and Johnny Zhang via
gates)
Changes for Build #2129
[hashutosh] HIVE-2304 : Support PreparedStatement.setObject (Ido Hadanny via
Ashutosh Chauhan)
[hashutosh] HIVE-4526 : auto_sortmerge_join_9.q throws NPE but test is
succeeded (Navis via Ashutosh Chauhan)
[hashutosh] HIVE-4516 : Fix concurrency bug in
serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java (Jon
Hartlaub and Navis via Ashutosh Chauhan)
[hashutosh] HIVE-4566 : NullPointerException if typeinfo and nativesql commands
are executed at beeline before a DB connection is established (Xuefu Zhang via
Ashutosh Chauhan)
Changes for Build #2130
Changes for Build #2131
[hashutosh] HIVE-4422 : Test output need to be updated for Windows only unit
test in TestCliDriver (Shuaishuai Nie via Ashutosh Chauhan)
[hashutosh] HIVE-4657 : HCatalog checkstyle violation after HIVE-2670
(Shreepadma Venugopalan via Ashutosh Chauhan)
[hashutosh] HIVE-4348 : Unit test compile fail at hbase-handler project on
Windows becuase of illegal escape character (Shuaishuai Nie via Ashutosh
Chauhan)
[hashutosh] HIVE-4554 : Failed to create a table from existing file if file
path has spaces (Xuefu Zhang via Ashutosh Chauhan)
[hashutosh] HIVE-4459 : Script hcat is overriding HIVE_CONF_DIR variable (Jarek
Jarcec Cecho via Ashutosh Chauhan)
Changes for Build #2132
[hashutosh] HIVE-4435 : Column stats: Distinct value estimator should use hash
functions that are pairwise independent (Shreepadma Venugopalan via Ashutosh
Chauhan)
Changes for Build #2133
Changes for Build #2134
Changes for Build #2135
Changes for Build #2136
Changes for Build #2137
[navis] HIVE-4549 : JDBC compliance change TABLE_SCHEMA to TABLE_SCHEM (Prasad
Mujumdar via Navis)
Changes for Build #2138
[navis] HIVE-4691 : orc_createas1.q has minor inconsistency (Brock Noland via
Navis)
Changes for Build #2139
[hashutosh] HIVE-4712 : Fix TestCliDriver.truncate_* on 0.23 (Brock Noland via
Ashutosh Chauhan)
Changes for Build #2140
[navis] HIVE-4708 : Fix TestCliDriver.combine2.q on 0.23 (Brock Noland via
Navis)
Changes for Build #2141
Changes for Build #2142
[hashutosh] HIVE-4717 : Fix non-deterministic TestCliDriver on 0.23 (Brock
Noland via Ashutosh Chauhan)
Changes for Build #2143
Changes for Build #2144
[hashutosh] HIVE-4713 : Fix TestCliDriver.skewjoin_union_remove_{1,2}.q on
[jira] [Commented] (HIVE-4730) Join on more than 2^31 records on single reducer failed (wrong results)
[ https://issues.apache.org/jira/browse/HIVE-4730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13684558#comment-13684558 ] Navis commented on HIVE-4730: - I think the exception above is from change of metastore schema between 0.7.0 and 0.10.0. You need upgrade metastore via script (I haven't do that, yet) or make a new metastore. Join on more than 2^31 records on single reducer failed (wrong results) --- Key: HIVE-4730 URL: https://issues.apache.org/jira/browse/HIVE-4730 Project: Hive Issue Type: Bug Affects Versions: 0.7.1, 0.8.0, 0.8.1, 0.9.0, 0.10.0, 0.11.0 Reporter: Gabi Kazav Assignee: Navis Priority: Critical Attachments: HIVE-4730.D11283.1.patch join on more than 2^31 rows leads to wrong results. for example: Create table small_table (p1 string) ROW FORMAT DELIMITEDLINES TERMINATED BY '\n'; Create table big_table (p1 string) ROW FORMAT DELIMITEDLINES TERMINATED BY '\n'; Loading 1 row to small_table (the value 1). Loading 2149580800 rows to big_table with the same value (1 on this case). create table output as select a.p1 from big_table a join small_table b on (a.p1=b.p1); select count(*) from output ; will return only 1 row... the reducer syslog: ... 2013-06-13 17:20:59,254 INFO ExecReducer: ExecReducer: processing 214700 rows: used memory = 32925960 2013-06-13 17:21:00,745 INFO ExecReducer: ExecReducer: processing 214800 rows: used memory = 12815184 2013-06-13 17:21:02,205 INFO ExecReducer: ExecReducer: processing 214900 rows: used memory = 26684552 -- looks like wrong value.. ... 2013-06-13 17:21:04,062 INFO ExecReducer: ExecReducer: processed 2149580801 rows: used memory = 17715896 2013-06-13 17:21:04,062 INFO org.apache.hadoop.hive.ql.exec.JoinOperator: 4 finished. closing... 2013-06-13 17:21:04,062 INFO org.apache.hadoop.hive.ql.exec.JoinOperator: 4 forwarded 1 rows 2013-06-13 17:21:05,791 INFO org.apache.hadoop.hive.ql.exec.JoinOperator: SKEWJOINFOLLOWUPJOBS:0 2013-06-13 17:21:05,792 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 5 finished. closing... 2013-06-13 17:21:05,792 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 5 forwarded 1 rows 2013-06-13 17:21:05,792 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: 6 finished. closing... 2013-06-13 17:21:05,792 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: 6 forwarded 0 rows 2013-06-13 17:21:05,946 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: TABLE_ID_1_ROWCOUNT:1 2013-06-13 17:21:05,946 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 5 Close done 2013-06-13 17:21:05,946 INFO org.apache.hadoop.hive.ql.exec.JoinOperator: 4 Close done -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4737) Allow access to MapredContext
[
https://issues.apache.org/jira/browse/HIVE-4737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13684561#comment-13684561
]
Navis commented on HIVE-4737:
-
If it would be exposed to client, how about make a interface?
Allow access to MapredContext
-
Key: HIVE-4737
URL: https://issues.apache.org/jira/browse/HIVE-4737
Project: Hive
Issue Type: Improvement
Affects Versions: 0.11.0
Reporter: Nicolas Lalevée
In order to tackle HIVE-1016, I have been pointed to HIVE-3628. But
MapredContext.get() is package protected.
Here is my current work around, a hack which I put in my code:
{code:java}
package org.apache.hadoop.hive.ql.exec;
public class MapredContextAccessor {
public static MapredContext get() {
return MapredContext.get();
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1662) Add file pruning into Hive.
[ https://issues.apache.org/jira/browse/HIVE-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-1662: -- Attachment: HIVE-1662.D8391.5.patch navis updated the revision HIVE-1662 [jira] Add file pruning into Hive.. Rebased to trunk Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D8391 CHANGE SINCE LAST DIFF https://reviews.facebook.net/D8391?vs=29433id=34785#toc AFFECTED FILES common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java ql/src/java/org/apache/hadoop/hive/ql/index/IndexPredicateAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java ql/src/java/org/apache/hadoop/hive/ql/metadata/FilePrunningPredicateHandler.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/AbstractJoinTaskDispatcher.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java ql/src/java/org/apache/hadoop/hive/ql/plan/TableScanDesc.java ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java ql/src/test/queries/clientpositive/file_pruning.q ql/src/test/results/clientpositive/file_pruning.q.out To: JIRA, navis Add file pruning into Hive. --- Key: HIVE-1662 URL: https://issues.apache.org/jira/browse/HIVE-1662 Project: Hive Issue Type: New Feature Reporter: He Yongqiang Assignee: Navis Attachments: HIVE-1662.D8391.1.patch, HIVE-1662.D8391.2.patch, HIVE-1662.D8391.3.patch, HIVE-1662.D8391.4.patch, HIVE-1662.D8391.5.patch now hive support filename virtual column. if a file name filter presents in a query, hive should be able to only add files which passed the filter to input paths. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4658) Make KW_OUTER optional in outer joins
[ https://issues.apache.org/jira/browse/HIVE-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13684567#comment-13684567 ] Phabricator commented on HIVE-4658: --- navis has commented on the revision HIVE-4658 [jira] Make KW_OUTER optional in outer joins. Two test you've mentioned passed in my trial. REVISION DETAIL https://reviews.facebook.net/D11091 To: JIRA, navis Cc: brock Make KW_OUTER optional in outer joins - Key: HIVE-4658 URL: https://issues.apache.org/jira/browse/HIVE-4658 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-4658.D11091.1.patch For really trivial migration issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2206) add a new optimizer for query correlation discovery and optimization
[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2206: -- Attachment: HIVE-2206.D11097.8.patch yhuai updated the revision HIVE-2206 [jira] add a new optimizer for query correlation discovery and optimization. All tests pass - new tests - Fix a bug related to UnionOperator; - wip - more comments in tests - Merge remote-tracking branch 'upstream/trunk' into HIVE-2206-3671-Improvement - update comments - Merge remote-tracking branch 'upstream/trunk' into HIVE-2206-3671-Improvement Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D11097 CHANGE SINCE LAST DIFF https://reviews.facebook.net/D11097?vs=34653id=34791#toc AFFECTED FILES common/src/java/org/apache/hadoop/hive/conf/HiveConf.java conf/hive-default.xml.template ql/if/queryplan.thrift ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/OperatorType.java ql/src/java/org/apache/hadoop/hive/ql/exec/CommonJoinOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/DemuxOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/ExecReducer.java ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/MuxOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRUnion1.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkDeDuplication.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/AbstractCorrelationProcCtx.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/CorrelationOptimizer.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/CorrelationUtilities.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/IntraQueryCorrelation.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/QueryPlanTreeTransformation.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/ReduceSinkDeDuplication.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/plan/DemuxDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/MuxDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/UnionDesc.java ql/src/test/queries/clientpositive/correlationoptimizer1.q ql/src/test/queries/clientpositive/correlationoptimizer2.q ql/src/test/queries/clientpositive/correlationoptimizer3.q ql/src/test/queries/clientpositive/correlationoptimizer4.q ql/src/test/queries/clientpositive/correlationoptimizer5.q ql/src/test/queries/clientpositive/correlationoptimizer6.q ql/src/test/queries/clientpositive/correlationoptimizer7.q ql/src/test/queries/clientpositive/correlationoptimizer8.q ql/src/test/queries/clientpositive/correlationoptimizer9.q ql/src/test/results/clientpositive/correlationoptimizer1.q.out ql/src/test/results/clientpositive/correlationoptimizer2.q.out ql/src/test/results/clientpositive/correlationoptimizer3.q.out ql/src/test/results/clientpositive/correlationoptimizer4.q.out ql/src/test/results/clientpositive/correlationoptimizer5.q.out ql/src/test/results/clientpositive/correlationoptimizer6.q.out ql/src/test/results/clientpositive/correlationoptimizer7.q.out ql/src/test/results/clientpositive/correlationoptimizer8.q.out ql/src/test/results/clientpositive/correlationoptimizer9.q.out ql/src/test/results/compiler/plan/groupby2.q.xml ql/src/test/results/compiler/plan/groupby3.q.xml To: JIRA, yhuai Cc: brock add a new optimizer for query correlation discovery and optimization Key: HIVE-2206 URL: https://issues.apache.org/jira/browse/HIVE-2206 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.12.0 Reporter: He Yongqiang Assignee: Yin Huai Attachments: HIVE-2206.10-r1384442.patch.txt, HIVE-2206.11-r1385084.patch.txt, HIVE-2206.12-r1386996.patch.txt, HIVE-2206.13-r1389072.patch.txt, HIVE-2206.14-r1389704.patch.txt, HIVE-2206.15-r1392491.patch.txt, HIVE-2206.16-r1399936.patch.txt, HIVE-2206.17-r1404933.patch.txt, HIVE-2206.18-r1407720.patch.txt, HIVE-2206.19-r1410581.patch.txt, HIVE-2206.1.patch.txt, HIVE-2206.20-r1434012.patch.txt, HIVE-2206.2.patch.txt, HIVE-2206.3.patch.txt, HIVE-2206.4.patch.txt,
