[jira] [Commented] (HIVE-2206) add a new optimizer for query correlation discovery and optimization
[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177318#comment-13177318 ] jirapos...@reviews.apache.org commented on HIVE-2206: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2001/ --- (Updated 2011-12-29 18:50:12.277210) Review request for hive. Summary --- This optimizer exploits intra-query correlations and merges multiple correlated MapReduce jobs into one jobs. This addresses bug HIVE-2206. https://issues.apache.org/jira/browse/HIVE-2206 Diffs (updated) - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/BaseReduceSinkOperator.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationCompositeOperator.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationLocalSimulativeReduceSinkOperator.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationReducerDispatchOperator.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecReducer.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizer.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizerUtils.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/BaseReduceSinkDesc.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CorrelationCompositeDesc.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CorrelationLocalSimulativeReduceSinkDesc.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CorrelationReducerDispatchDesc.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableScanDesc.java 1224666 trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/TestExecDriver.java 1224666 trunk/ql/src/test/results/compiler/plan/groupby1.q.xml 1224666 trunk/ql/src/test/results/compiler/plan/groupby2.q.xml 1224666 trunk/ql/src/test/results/compiler/plan/groupby3.q.xml 1224666 trunk/ql/src/test/results/compiler/plan/groupby5.q.xml 1224666 Diff: https://reviews.apache.org/r/2001/diff Testing (updated) --- Thanks, Yin > add a new optimizer for query correlation discovery and optimization > > > Key: HIVE-2206 > URL: https://issues.apache.org/jira/browse/HIVE-2206 > Project: Hive > Issue Type: New Feature >Reporter: He Yongqiang >Assignee: Yin Huai > Attachments: HIVE-2206.1.patch.txt, HIVE-2206.2.patch.txt, > HIVE-2206.3.patch.txt, HIVE-2206.4.patch.txt, HIVE-2206.5-1.patch.txt, > HIVE-2206.5.patch.txt, HIVE-2206.6.patch.txt, HIVE-2206.7.patch.txt, Queries, > YSmartPatchForHive.patch, testQueries.1.q, testQueries.q > > > reference: > http://www.cse.ohio-state.edu/hpcs/WWW/HTML/publications/papers/TR-11-7.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2206) add a new optimizer for query correlation discovery and optimization
[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated HIVE-2206: --- Attachment: HIVE-2206.8.r1224646.patch.txt diff in the review board has also been updated (https://reviews.apache.org/r/2001/). > add a new optimizer for query correlation discovery and optimization > > > Key: HIVE-2206 > URL: https://issues.apache.org/jira/browse/HIVE-2206 > Project: Hive > Issue Type: New Feature >Reporter: He Yongqiang >Assignee: Yin Huai > Attachments: HIVE-2206.1.patch.txt, HIVE-2206.2.patch.txt, > HIVE-2206.3.patch.txt, HIVE-2206.4.patch.txt, HIVE-2206.5-1.patch.txt, > HIVE-2206.5.patch.txt, HIVE-2206.6.patch.txt, HIVE-2206.7.patch.txt, > HIVE-2206.8.r1224646.patch.txt, YSmartPatchForHive.patch, testQueries.1.q > > > reference: > http://www.cse.ohio-state.edu/hpcs/WWW/HTML/publications/papers/TR-11-7.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2686) Test ppr_pushdown.q is failing on trunk
[
https://issues.apache.org/jira/browse/HIVE-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-2686:
---
Priority: Minor (was: Major)
Lowering priority since it passed on another machine for me. Keeping it open to
figure out the flakiness.
> Test ppr_pushdown.q is failing on trunk
> ---
>
> Key: HIVE-2686
> URL: https://issues.apache.org/jira/browse/HIVE-2686
> Project: Hive
> Issue Type: Test
> Components: Testing Infrastructure
>Affects Versions: 0.9.0
>Reporter: Ashutosh Chauhan
>Priority: Minor
> Fix For: 0.9.0
>
>
> {noformat}
> $ ant test -Dtestcase=TestCliDriver -Dqfile=ppr_pushdown.q
> [junit] See build/ql/tmp/hive.log, or try "ant test ...
> -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppr_pushdown(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> [junit] at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> [junit] at java.lang.reflect.Method.invoke(Method.java:597)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
> [junit] at
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
> [junit] at
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
> [junit] 178d177
> [junit] < 12341234
> [junit] 179a179
> [junit] > 12341234
> [junit] 196d195
> [junit] < 12241224
> [junit] 197a197
> [junit] > 12241224
> [junit] 214d213
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ...
> -Dtest.silent=false" to get more logs.
> [junit] < 12141214
> [junit] 215a215
> [junit] > 12141214
> [junit] 232d231
> [junit] < 12.412.4
> [junit] 233a233
> [junit] > 12.412.4
> [junit] 250d249
> [junit] < 12+412+4
> [junit] 251a251
> [junit] > 12+412+4
> [junit] 268d267
> [junit] < 12:412:4
> [junit] 269a269
> [junit] > 12:412:4
> [junit] 286d285
> [junit] < 12%412%4
> [junit] 287a287
> [junit] > 12%412%4
> [junit] 304d303
> [junit] < 12*412*4
> [junit] 305a305
> [junit] > 12*412*4
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 137.556 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2206) add a new optimizer for query correlation discovery and optimization
[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated HIVE-2206: --- Attachment: testQueries.2.q > add a new optimizer for query correlation discovery and optimization > > > Key: HIVE-2206 > URL: https://issues.apache.org/jira/browse/HIVE-2206 > Project: Hive > Issue Type: New Feature >Reporter: He Yongqiang >Assignee: Yin Huai > Attachments: HIVE-2206.1.patch.txt, HIVE-2206.2.patch.txt, > HIVE-2206.3.patch.txt, HIVE-2206.4.patch.txt, HIVE-2206.5-1.patch.txt, > HIVE-2206.5.patch.txt, HIVE-2206.6.patch.txt, HIVE-2206.7.patch.txt, > HIVE-2206.8.r1224646.patch.txt, YSmartPatchForHive.patch, testQueries.2.q > > > reference: > http://www.cse.ohio-state.edu/hpcs/WWW/HTML/publications/papers/TR-11-7.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2206) add a new optimizer for query correlation discovery and optimization
[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated HIVE-2206: --- Attachment: (was: Queries) > add a new optimizer for query correlation discovery and optimization > > > Key: HIVE-2206 > URL: https://issues.apache.org/jira/browse/HIVE-2206 > Project: Hive > Issue Type: New Feature >Reporter: He Yongqiang >Assignee: Yin Huai > Attachments: HIVE-2206.1.patch.txt, HIVE-2206.2.patch.txt, > HIVE-2206.3.patch.txt, HIVE-2206.4.patch.txt, HIVE-2206.5-1.patch.txt, > HIVE-2206.5.patch.txt, HIVE-2206.6.patch.txt, HIVE-2206.7.patch.txt, > HIVE-2206.8.r1224646.patch.txt, YSmartPatchForHive.patch, testQueries.1.q > > > reference: > http://www.cse.ohio-state.edu/hpcs/WWW/HTML/publications/papers/TR-11-7.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2681) SUCESS is misspelled
[ https://issues.apache.org/jira/browse/HIVE-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan resolved HIVE-2681. Resolution: Fixed Fix Version/s: 0.9.0 HIVE-2686 passed on another machine. So, looks like some problem in my local environment. Committed this one to trunk. > SUCESS is misspelled > > > Key: HIVE-2681 > URL: https://issues.apache.org/jira/browse/HIVE-2681 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Chang >Assignee: Jonathan Chang > Fix For: 0.9.0 > > Attachments: HIVE-2681.D1029.1.patch, HIVE-2681.D1029.1.patch > > > C'mon! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2001/ --- (Updated 2011-12-29 18:50:12.277210) Review request for hive. Summary --- This optimizer exploits intra-query correlations and merges multiple correlated MapReduce jobs into one jobs. This addresses bug HIVE-2206. https://issues.apache.org/jira/browse/HIVE-2206 Diffs (updated) - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/BaseReduceSinkOperator.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationCompositeOperator.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationLocalSimulativeReduceSinkOperator.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationReducerDispatchOperator.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecReducer.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizer.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizerUtils.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/BaseReduceSinkDesc.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CorrelationCompositeDesc.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CorrelationLocalSimulativeReduceSinkDesc.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CorrelationReducerDispatchDesc.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 1224666 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableScanDesc.java 1224666 trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/TestExecDriver.java 1224666 trunk/ql/src/test/results/compiler/plan/groupby1.q.xml 1224666 trunk/ql/src/test/results/compiler/plan/groupby2.q.xml 1224666 trunk/ql/src/test/results/compiler/plan/groupby3.q.xml 1224666 trunk/ql/src/test/results/compiler/plan/groupby5.q.xml 1224666 Diff: https://reviews.apache.org/r/2001/diff Testing (updated) --- Thanks, Yin
[jira] [Updated] (HIVE-2206) add a new optimizer for query correlation discovery and optimization
[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated HIVE-2206: --- Attachment: testQueries.2.q testQueries.2.q have three testing queries > add a new optimizer for query correlation discovery and optimization > > > Key: HIVE-2206 > URL: https://issues.apache.org/jira/browse/HIVE-2206 > Project: Hive > Issue Type: New Feature >Reporter: He Yongqiang >Assignee: Yin Huai > Attachments: HIVE-2206.1.patch.txt, HIVE-2206.2.patch.txt, > HIVE-2206.3.patch.txt, HIVE-2206.4.patch.txt, HIVE-2206.5-1.patch.txt, > HIVE-2206.5.patch.txt, HIVE-2206.6.patch.txt, HIVE-2206.7.patch.txt, > HIVE-2206.8.r1224646.patch.txt, YSmartPatchForHive.patch, testQueries.2.q > > > reference: > http://www.cse.ohio-state.edu/hpcs/WWW/HTML/publications/papers/TR-11-7.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2206) add a new optimizer for query correlation discovery and optimization
[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated HIVE-2206: --- Attachment: (was: testQueries.1.q) > add a new optimizer for query correlation discovery and optimization > > > Key: HIVE-2206 > URL: https://issues.apache.org/jira/browse/HIVE-2206 > Project: Hive > Issue Type: New Feature >Reporter: He Yongqiang >Assignee: Yin Huai > Attachments: HIVE-2206.1.patch.txt, HIVE-2206.2.patch.txt, > HIVE-2206.3.patch.txt, HIVE-2206.4.patch.txt, HIVE-2206.5-1.patch.txt, > HIVE-2206.5.patch.txt, HIVE-2206.6.patch.txt, HIVE-2206.7.patch.txt, > HIVE-2206.8.r1224646.patch.txt, YSmartPatchForHive.patch > > > reference: > http://www.cse.ohio-state.edu/hpcs/WWW/HTML/publications/papers/TR-11-7.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2206) add a new optimizer for query correlation discovery and optimization
[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated HIVE-2206: --- Attachment: (was: testQueries.2.q) > add a new optimizer for query correlation discovery and optimization > > > Key: HIVE-2206 > URL: https://issues.apache.org/jira/browse/HIVE-2206 > Project: Hive > Issue Type: New Feature >Reporter: He Yongqiang >Assignee: Yin Huai > Attachments: HIVE-2206.1.patch.txt, HIVE-2206.2.patch.txt, > HIVE-2206.3.patch.txt, HIVE-2206.4.patch.txt, HIVE-2206.5-1.patch.txt, > HIVE-2206.5.patch.txt, HIVE-2206.6.patch.txt, HIVE-2206.7.patch.txt, > HIVE-2206.8.r1224646.patch.txt, YSmartPatchForHive.patch, testQueries.2.q > > > reference: > http://www.cse.ohio-state.edu/hpcs/WWW/HTML/publications/papers/TR-11-7.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2681) SUCESS is misspelled
[ https://issues.apache.org/jira/browse/HIVE-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177325#comment-13177325 ] Hudson commented on HIVE-2681: -- Integrated in Hive-trunk-h0.23.0 #53 (See [https://builds.apache.org/job/Hive-trunk-h0.23.0/53/]) HIVE-2681 : SUCESS is misspelled (jonchang via Ashutosh Chauhan) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1225617 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/MapRedStats.java > SUCESS is misspelled > > > Key: HIVE-2681 > URL: https://issues.apache.org/jira/browse/HIVE-2681 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Chang >Assignee: Jonathan Chang > Fix For: 0.9.0 > > Attachments: HIVE-2681.D1029.1.patch, HIVE-2681.D1029.1.patch > > > C'mon! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.23.0 - Build # 53 - Still Failing
Changes for Build #9 [heyongqiang] HIVE-2634 [jira] revert HIVE-2566 (Namit Jain via Yongqiang He) Summary: HIVE-2634 This is leading to some problems. I will upload the offending testcase in a new jira. Test Plan: EMPTY Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang Differential Revision: 651 Changes for Build #10 [heyongqiang] HIVE-2632 [jira] ignore exception for external jars via reflection (Namit Jain via Yongqiang He) Summary: HIVE-2632 Test Plan: EMPTY Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang Differential Revision: 639 Changes for Build #11 Changes for Build #12 [heyongqiang] HIVE-2635 [jira] wrong class loader used for external jars (Namit Jain via Yongqiang He) Summary: HIVE-2635 Test Plan: EMPTY Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang Differential Revision: 687 [namit] HIVE-2329 Not using map aggregation, fails to execute group-by after cluster-by with same key (Navis via namit) Changes for Build #13 [namit] HIVE-2628 move one line log from MapOperator to HiveContextAwareRecordReader (He Yongqiang via namit) [namit] HIVE-2520 left semi join will duplicate data (binlijin via namit) [jvs] HIVE-1487 [jira] Add ptest script (Marek Sapota via John Sichi) Summary: HIVE-1464 speeded up serial runs somewhat - but looks like it's still too slow. we should use parallel junit or some similar setup to run test queries in parallel. this should be really easy as well need to just use a separate warehouse/metadb and potentiall mapred system dir location. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: mareksapotafb, jsichi Differential Revision: 603 Changes for Build #14 Changes for Build #15 Changes for Build #16 Changes for Build #17 [jvs] Fix executable permissions left out of commit for HIVE-1487. Changes for Build #18 [jvs] HIVE-2647 [jira] Force Bash shell on parallel test slave nodes (Marek Sapota via John Sichi) Summary: Forcing Bash on each remote command call. Force use of Bash shell, so user default shell choice doesn't impact the test script. Test Plan: EMPTY Reviewers: JIRA, jsichi, njain Reviewed By: jsichi CC: mareksapotafb, jsichi Differential Revision: 771 [jvs] HIVE-2648 [jira] Parallel tests fail if master directory is not present (Marek Sapota via John Sichi) Summary: Fix the task downloading Ant. Parallel tests should create directories as needed. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: mareksapotafb, jsichi Differential Revision: 777 Changes for Build #19 [jvs] HIVE-2649 [jira] Allow multiple ptest runs by the same person (Marek Sapota via John Sichi) Summary: Allow multiple ptest runs. Allow running ptest simultaneously, multiple times by one person on the same machines. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: mareksapotafb, jsichi Differential Revision: 783 [jvs] HIVE-2650 [jira] Parallel test commands that include cd fail (Marek Sapota via John Sichi) Summary: Add quotes around bash -c to fix commands using cd. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: mareksapotafb, jsichi Differential Revision: 801 [jvs] HIVE-2643. Recent patch prevents Hadoop confs from loading in 0.20.204 (Carl Steinbach via jvs) Changes for Build #20 Changes for Build #21 [jvs] HIVE-1040 [jira] use sed rather than diff for masking out noise in diff-based tests (Marek Sapota via John Sichi) Summary: Replace diff -I with regex masking in Java The current diff -I approach has two problems: (1) it does not allow resolution finer than line-level, so it's impossible to mask out pattern occurrences within a line, and (2) it produces unmasked files, so if you run diff on the command line to compare the result .q.out with the checked-in file, you see the noise. My suggestion is to first run sed to replace noise patterns with an unlikely-to-occur string like ZYZZYZVA, and then diff the pre-masked files without using any -I. This would require a one-time hit to update all existing .q.out files so that they would contain the pre-masked results. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 597 Changes for Build #22 Changes for Build #23 [namit] HIVE-2640 Add alterPartition to AlterHandler interface (Kevin Wilfong via namit) Changes for Build #24 [namit] HIVE-2617 Insert overwrite table db.tname fails if partition already exists (Chinna Rao Lalam via namit) Changes for Build #25 [namit] HIVE-2611 Make index table output of create index command if index is table based (Kevin Wilfong via namit) Changes for Build #26 [heyongqiang] HIVE-2651 [jira] The variable hive.exec.mode.local.auto.tasks.max should be changed (Namit Jain via Yongqiang He) Summary: HIVE-2651 It should be called hive.exec.mode.local.auto.input.files.max instead. The number of input files are checked currently. Test Plan: EMPTY Reviewer
[jira] [Updated] (HIVE-2206) add a new optimizer for query correlation discovery and optimization
[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated HIVE-2206: --- Attachment: (was: testQueries.q) > add a new optimizer for query correlation discovery and optimization > > > Key: HIVE-2206 > URL: https://issues.apache.org/jira/browse/HIVE-2206 > Project: Hive > Issue Type: New Feature >Reporter: He Yongqiang >Assignee: Yin Huai > Attachments: HIVE-2206.1.patch.txt, HIVE-2206.2.patch.txt, > HIVE-2206.3.patch.txt, HIVE-2206.4.patch.txt, HIVE-2206.5-1.patch.txt, > HIVE-2206.5.patch.txt, HIVE-2206.6.patch.txt, HIVE-2206.7.patch.txt, > HIVE-2206.8.r1224646.patch.txt, YSmartPatchForHive.patch, testQueries.1.q > > > reference: > http://www.cse.ohio-state.edu/hpcs/WWW/HTML/publications/papers/TR-11-7.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.23.0 - Build # 52 - Still Failing
Changes for Build #7 [heyongqiang] HIVE-1003 [jira] optimize metadata only queries (Namit Jain via Yongqiang He) Summary: testing Queries like: select max(ds) from T where ds is a partitioning column should be optimized. Test Plan: EMPTY Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: njain, heyongqiang Differential Revision: 105 Changes for Build #8 Changes for Build #9 [heyongqiang] HIVE-2634 [jira] revert HIVE-2566 (Namit Jain via Yongqiang He) Summary: HIVE-2634 This is leading to some problems. I will upload the offending testcase in a new jira. Test Plan: EMPTY Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang Differential Revision: 651 Changes for Build #10 [heyongqiang] HIVE-2632 [jira] ignore exception for external jars via reflection (Namit Jain via Yongqiang He) Summary: HIVE-2632 Test Plan: EMPTY Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang Differential Revision: 639 Changes for Build #11 Changes for Build #12 [heyongqiang] HIVE-2635 [jira] wrong class loader used for external jars (Namit Jain via Yongqiang He) Summary: HIVE-2635 Test Plan: EMPTY Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang Differential Revision: 687 [namit] HIVE-2329 Not using map aggregation, fails to execute group-by after cluster-by with same key (Navis via namit) Changes for Build #13 [namit] HIVE-2628 move one line log from MapOperator to HiveContextAwareRecordReader (He Yongqiang via namit) [namit] HIVE-2520 left semi join will duplicate data (binlijin via namit) [jvs] HIVE-1487 [jira] Add ptest script (Marek Sapota via John Sichi) Summary: HIVE-1464 speeded up serial runs somewhat - but looks like it's still too slow. we should use parallel junit or some similar setup to run test queries in parallel. this should be really easy as well need to just use a separate warehouse/metadb and potentiall mapred system dir location. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: mareksapotafb, jsichi Differential Revision: 603 Changes for Build #14 Changes for Build #15 Changes for Build #16 Changes for Build #17 [jvs] Fix executable permissions left out of commit for HIVE-1487. Changes for Build #18 [jvs] HIVE-2647 [jira] Force Bash shell on parallel test slave nodes (Marek Sapota via John Sichi) Summary: Forcing Bash on each remote command call. Force use of Bash shell, so user default shell choice doesn't impact the test script. Test Plan: EMPTY Reviewers: JIRA, jsichi, njain Reviewed By: jsichi CC: mareksapotafb, jsichi Differential Revision: 771 [jvs] HIVE-2648 [jira] Parallel tests fail if master directory is not present (Marek Sapota via John Sichi) Summary: Fix the task downloading Ant. Parallel tests should create directories as needed. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: mareksapotafb, jsichi Differential Revision: 777 Changes for Build #19 [jvs] HIVE-2649 [jira] Allow multiple ptest runs by the same person (Marek Sapota via John Sichi) Summary: Allow multiple ptest runs. Allow running ptest simultaneously, multiple times by one person on the same machines. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: mareksapotafb, jsichi Differential Revision: 783 [jvs] HIVE-2650 [jira] Parallel test commands that include cd fail (Marek Sapota via John Sichi) Summary: Add quotes around bash -c to fix commands using cd. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: mareksapotafb, jsichi Differential Revision: 801 [jvs] HIVE-2643. Recent patch prevents Hadoop confs from loading in 0.20.204 (Carl Steinbach via jvs) Changes for Build #20 Changes for Build #21 [jvs] HIVE-1040 [jira] use sed rather than diff for masking out noise in diff-based tests (Marek Sapota via John Sichi) Summary: Replace diff -I with regex masking in Java The current diff -I approach has two problems: (1) it does not allow resolution finer than line-level, so it's impossible to mask out pattern occurrences within a line, and (2) it produces unmasked files, so if you run diff on the command line to compare the result .q.out with the checked-in file, you see the noise. My suggestion is to first run sed to replace noise patterns with an unlikely-to-occur string like ZYZZYZVA, and then diff the pre-masked files without using any -I. This would require a one-time hit to update all existing .q.out files so that they would contain the pre-masked results. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 597 Changes for Build #22 Changes for Build #23 [namit] HIVE-2640 Add alterPartition to AlterHandler interface (Kevin Wilfong via namit) Changes for Build #24 [namit] HIVE-2617 Insert overwrite table db.tname fails if partition already exists (Chinna Rao Lalam via namit) Changes for Build #25 [namit] HIVE-2611 Make index table output of create index comm
Hive-trunk-h0.21 - Build # 1174 - Still Failing
Changes for Build #1144 [jvs] HIVE-1040 [jira] use sed rather than diff for masking out noise in diff-based tests (Marek Sapota via John Sichi) Summary: Replace diff -I with regex masking in Java The current diff -I approach has two problems: (1) it does not allow resolution finer than line-level, so it's impossible to mask out pattern occurrences within a line, and (2) it produces unmasked files, so if you run diff on the command line to compare the result .q.out with the checked-in file, you see the noise. My suggestion is to first run sed to replace noise patterns with an unlikely-to-occur string like ZYZZYZVA, and then diff the pre-masked files without using any -I. This would require a one-time hit to update all existing .q.out files so that they would contain the pre-masked results. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 597 Changes for Build #1145 Changes for Build #1146 [namit] HIVE-2640 Add alterPartition to AlterHandler interface (Kevin Wilfong via namit) Changes for Build #1147 [namit] HIVE-2617 Insert overwrite table db.tname fails if partition already exists (Chinna Rao Lalam via namit) Changes for Build #1148 [heyongqiang] HIVE-2651 [jira] The variable hive.exec.mode.local.auto.tasks.max should be changed (Namit Jain via Yongqiang He) Summary: HIVE-2651 It should be called hive.exec.mode.local.auto.input.files.max instead. The number of input files are checked currently. Test Plan: EMPTY Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang Differential Revision: 861 [cws] HIVE-727. Hive Server getSchema() returns wrong schema for 'Explain' queries (Prasad Mujumdar via cws) [namit] HIVE-2611 Make index table output of create index command if index is table based (Kevin Wilfong via namit) Changes for Build #1150 [jvs] HIVE-2657 [jira] builtins JAR is not being published to Maven repo & hive-cli POM does not depend on it either (Carl Steinbach via John Sichi) Summary: Make hive-cli and hive-ql depend on hive-builtins Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 897 [namit] HIVE-2654 "hive.querylog.location" requires parent directory to be exist or else folder creation fails (Chinna Rao Lalam via namit) Changes for Build #1151 [hashutosh] HIVE-1892 : show functions also returns internal operators (Priyadarshini via Ashutosh Chauhan) Changes for Build #1152 Changes for Build #1153 [namit] HIVE-2660 Need better exception handling in RCFile tolerate corruptions mode (Ramkumar Vadali via namit) Changes for Build #1154 [cws] HIVE-2631. Make Hive work with Hadoop 1.0.0 (Ashutosh Chauhan via cws) Changes for Build #1155 [cws] HIVE-BUILD. Update RELEASE_NOTES.txt with 0.8.0 release information (cws) Changes for Build #1156 Changes for Build #1157 Changes for Build #1158 [namit] HIVE-2602 add support for insert partition overwrite(...) if not exists (Chinna Rao Lalam via namit) Changes for Build #1159 Changes for Build #1160 [cws] HIVE-2005. Implement BETWEEN operator (Navis via cws) Changes for Build #1161 [jvs] HIVE-2433. add DOAP file for Hive Changes for Build #1162 Changes for Build #1163 Changes for Build #1164 [heyongqiang] HIVE-2666 [jira] StackOverflowError when using custom UDF in map join (Kevin Wilfong via Yongqiang He) Summary: Resource files are now added to the class path as soon as they are added via the CLI. This fixes the stack overflow error mentioned in the JIRA by ensuring a consistent class loader between serializers and deserializers for the same query. Note that now serdes which contain a static block to register themselves are now registered twice, once when adding the file to the class loader, and once when an instance of the class is created. Previously, registering a serde twice resulted in an exception, to avoid this, I have downgraded it to a warning. When a custom UDF is used as part of a join which is converted to a map join, the XMLEncoder enters an infinite loop when serializing the map reduce task for the second time, as part of sending it to be executed. This results in a stack overflow error. Test Plan: I ran the unit tests to verify nothing was broken. I ran several queries which used custom UDFs and involved a join which was converted to a map join. I verified these completed successfully consistently Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang, kevinwilfong Differential Revision: 957 [namit] HIVE-2642 fix Hive-2566 and make union optimization more aggressive (Yongqiang He via namit) Changes for Build #1166 Changes for Build #1167 Changes for Build #1168 [heyongqiang] HIVE-2600: Enable/Add type-specific compression for rcfile (Krishna Kumar via He Yongqiang) Changes for Build #1169 Changes for Build #1170 [cws] HIVE-1877. Add java_method() as a synonym for the reflect() UDF (Zhenxiao Luo via cws) Changes for Buil
Hive-0.8.0-SNAPSHOT-h0.21 - Build # 143 - Failure
Changes for Build #143 5 tests failed. REGRESSION: org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisherOneStat Error Message: null Stack Trace: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.Utilities.prepareWithRetry(Utilities.java:2163) at org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsAggregator.cleanUp(JDBCStatsAggregator.java:213) at org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.setUp(TestStatsPublisherEnhanced.java:60) at junit.framework.TestCase.runBare(TestCase.java:125) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at junit.framework.TestSuite.runTest(TestSuite.java:208) at junit.framework.TestSuite.run(TestSuite.java:203) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) REGRESSION: org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisher Error Message: null Stack Trace: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.Utilities.prepareWithRetry(Utilities.java:2163) at org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsAggregator.cleanUp(JDBCStatsAggregator.java:213) at org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.setUp(TestStatsPublisherEnhanced.java:60) at junit.framework.TestCase.runBare(TestCase.java:125) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at junit.framework.TestSuite.runTest(TestSuite.java:208) at junit.framework.TestSuite.run(TestSuite.java:203) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) REGRESSION: org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisherMultipleUpdates Error Message: null Stack Trace: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.Utilities.prepareWithRetry(Utilities.java:2163) at org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsAggregator.cleanUp(JDBCStatsAggregator.java:213) at org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.setUp(TestStatsPublisherEnhanced.java:60) at junit.framework.TestCase.runBare(TestCase.java:125) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at junit.framework.TestSuite.runTest(TestSuite.java:208) at junit.framework.TestSuite.run(TestSuite.java:203) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) REGRESSION: org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisherMultipleUpdatesSubsetStatistics Error Message: null Stack Trace: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.Utilities.prepareWithRetry(Utilities.java:2163) at org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsAggregator.cleanUp(JDBCStatsAggregator.java:213) at org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.setUp(TestStatsPublisherEnhanced.java:60) at junit.framework.TestCase.runBare(TestCase.java:125) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at junit.framework.TestSuite.runTest(TestSuite.java:208) at junit.framework.TestSuite.run(TestSuite.java:203) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunn
[jira] [Created] (HIVE-2687) Test classpath should resemble runtime classpath
Test classpath should resemble runtime classpath Key: HIVE-2687 URL: https://issues.apache.org/jira/browse/HIVE-2687 Project: Hive Issue Type: Bug Components: Testing Infrastructure Affects Versions: 0.8.0 Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2687) Test classpath should resemble runtime classpath
[ https://issues.apache.org/jira/browse/HIVE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177418#comment-13177418 ] Carl Steinbach commented on HIVE-2687: -- The scripts in the bin/ directory execute Hive using the 'hadoop jar' command. Consequently Hive inherits and gives precedence to the contents of Hadoop's lib/ directory. We need to make sure that Hive tests run with a similar classpath. > Test classpath should resemble runtime classpath > > > Key: HIVE-2687 > URL: https://issues.apache.org/jira/browse/HIVE-2687 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Affects Versions: 0.8.0 >Reporter: Carl Steinbach > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan resolved HIVE-2616. Resolution: Fixed Fix Version/s: 0.9.0 Thanks, Carl for the review. Committed to trunk. > Passing user identity from metastore client to server in non-secure mode > > > Key: HIVE-2616 > URL: https://issues.apache.org/jira/browse/HIVE-2616 > Project: Hive > Issue Type: Bug > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Fix For: 0.9.0 > > Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch, > hive-2616_4.patch, hive-2616_5.patch > > > Currently in unsecure mode client don't pass on user identity. As a result > hdfs and other operations done by server gets executed by user running > metastore process instead of being done in context of client. This results in > problem as reported here: > http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2681) SUCESS is misspelled
[ https://issues.apache.org/jira/browse/HIVE-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177473#comment-13177473 ] Hudson commented on HIVE-2681: -- Integrated in Hive-trunk-h0.21 #1175 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1175/]) HIVE-2681 : SUCESS is misspelled (jonchang via Ashutosh Chauhan) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1225617 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/MapRedStats.java > SUCESS is misspelled > > > Key: HIVE-2681 > URL: https://issues.apache.org/jira/browse/HIVE-2681 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Chang >Assignee: Jonathan Chang > Fix For: 0.9.0 > > Attachments: HIVE-2681.D1029.1.patch, HIVE-2681.D1029.1.patch > > > C'mon! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2589) Newly created partition should inherit properties from table
[ https://issues.apache.org/jira/browse/HIVE-2589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177474#comment-13177474 ] Ashutosh Chauhan commented on HIVE-2589: Can anyone take a look at this one? > Newly created partition should inherit properties from table > > > Key: HIVE-2589 > URL: https://issues.apache.org/jira/browse/HIVE-2589 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Fix For: 0.9.0 > > Attachments: hive-2589.patch, hive-2589.patch, hive-2589_1.patch, > hive-2589_2.patch > > > This will make all the info contained in table properties available to > partitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1175 - Still Failing
Changes for Build #1144 [jvs] HIVE-1040 [jira] use sed rather than diff for masking out noise in diff-based tests (Marek Sapota via John Sichi) Summary: Replace diff -I with regex masking in Java The current diff -I approach has two problems: (1) it does not allow resolution finer than line-level, so it's impossible to mask out pattern occurrences within a line, and (2) it produces unmasked files, so if you run diff on the command line to compare the result .q.out with the checked-in file, you see the noise. My suggestion is to first run sed to replace noise patterns with an unlikely-to-occur string like ZYZZYZVA, and then diff the pre-masked files without using any -I. This would require a one-time hit to update all existing .q.out files so that they would contain the pre-masked results. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 597 Changes for Build #1145 Changes for Build #1146 [namit] HIVE-2640 Add alterPartition to AlterHandler interface (Kevin Wilfong via namit) Changes for Build #1147 [namit] HIVE-2617 Insert overwrite table db.tname fails if partition already exists (Chinna Rao Lalam via namit) Changes for Build #1148 [heyongqiang] HIVE-2651 [jira] The variable hive.exec.mode.local.auto.tasks.max should be changed (Namit Jain via Yongqiang He) Summary: HIVE-2651 It should be called hive.exec.mode.local.auto.input.files.max instead. The number of input files are checked currently. Test Plan: EMPTY Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang Differential Revision: 861 [cws] HIVE-727. Hive Server getSchema() returns wrong schema for 'Explain' queries (Prasad Mujumdar via cws) [namit] HIVE-2611 Make index table output of create index command if index is table based (Kevin Wilfong via namit) Changes for Build #1150 [jvs] HIVE-2657 [jira] builtins JAR is not being published to Maven repo & hive-cli POM does not depend on it either (Carl Steinbach via John Sichi) Summary: Make hive-cli and hive-ql depend on hive-builtins Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 897 [namit] HIVE-2654 "hive.querylog.location" requires parent directory to be exist or else folder creation fails (Chinna Rao Lalam via namit) Changes for Build #1151 [hashutosh] HIVE-1892 : show functions also returns internal operators (Priyadarshini via Ashutosh Chauhan) Changes for Build #1152 Changes for Build #1153 [namit] HIVE-2660 Need better exception handling in RCFile tolerate corruptions mode (Ramkumar Vadali via namit) Changes for Build #1154 [cws] HIVE-2631. Make Hive work with Hadoop 1.0.0 (Ashutosh Chauhan via cws) Changes for Build #1155 [cws] HIVE-BUILD. Update RELEASE_NOTES.txt with 0.8.0 release information (cws) Changes for Build #1156 Changes for Build #1157 Changes for Build #1158 [namit] HIVE-2602 add support for insert partition overwrite(...) if not exists (Chinna Rao Lalam via namit) Changes for Build #1159 Changes for Build #1160 [cws] HIVE-2005. Implement BETWEEN operator (Navis via cws) Changes for Build #1161 [jvs] HIVE-2433. add DOAP file for Hive Changes for Build #1162 Changes for Build #1163 Changes for Build #1164 [heyongqiang] HIVE-2666 [jira] StackOverflowError when using custom UDF in map join (Kevin Wilfong via Yongqiang He) Summary: Resource files are now added to the class path as soon as they are added via the CLI. This fixes the stack overflow error mentioned in the JIRA by ensuring a consistent class loader between serializers and deserializers for the same query. Note that now serdes which contain a static block to register themselves are now registered twice, once when adding the file to the class loader, and once when an instance of the class is created. Previously, registering a serde twice resulted in an exception, to avoid this, I have downgraded it to a warning. When a custom UDF is used as part of a join which is converted to a map join, the XMLEncoder enters an infinite loop when serializing the map reduce task for the second time, as part of sending it to be executed. This results in a stack overflow error. Test Plan: I ran the unit tests to verify nothing was broken. I ran several queries which used custom UDFs and involved a join which was converted to a map join. I verified these completed successfully consistently Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang, kevinwilfong Differential Revision: 957 [namit] HIVE-2642 fix Hive-2566 and make union optimization more aggressive (Yongqiang He via namit) Changes for Build #1166 Changes for Build #1167 Changes for Build #1168 [heyongqiang] HIVE-2600: Enable/Add type-specific compression for rcfile (Krishna Kumar via He Yongqiang) Changes for Build #1169 Changes for Build #1170 [cws] HIVE-1877. Add java_method() as a synonym for the reflect() UDF (Zhenxiao Luo via cws) Changes for Buil
[jira] [Updated] (HIVE-2629) Make a single Hive binary work with both 0.20.x and 0.23.0
[ https://issues.apache.org/jira/browse/HIVE-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2629: -- Attachment: HIVE-2629.D711.2.patch thw updated the revision "HIVE-2629 [jira] Make a single Hive binary work with both 0.20.x and 0.23.0". Reviewers: JIRA, cwsteinbach Rebased patch. REVISION DETAIL https://reviews.facebook.net/D711 AFFECTED FILES shims/ivy.xml shims/src/commonSecure shims/src/commonSecure/java shims/src/commonSecure/java/org shims/src/commonSecure/java/org/apache shims/src/commonSecure/java/org/apache/hadoop shims/src/commonSecure/java/org/apache/hadoop/security shims/src/commonSecure/java/org/apache/hadoop/security/token shims/src/commonSecure/java/org/apache/hadoop/security/token/delegation shims/src/commonSecure/java/org/apache/hadoop/security/token/delegation/HiveDelegationTokenSupport.java shims/src/commonSecure/java/org/apache/hadoop/hive shims/src/commonSecure/java/org/apache/hadoop/hive/thrift shims/src/commonSecure/java/org/apache/hadoop/hive/thrift/DelegationTokenIdentifier.java shims/src/commonSecure/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java shims/src/commonSecure/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java shims/src/commonSecure/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java shims/src/commonSecure/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java shims/src/commonSecure/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java shims/src/commonSecure/java/org/apache/hadoop/hive/thrift/DelegationTokenSelector.java shims/src/commonSecure/java/org/apache/hadoop/hive/thrift/client shims/src/commonSecure/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java shims/src/commonSecure/java/org/apache/hadoop/hive/shims shims/src/commonSecure/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java shims/src/0.20S/java/org/apache/hadoop/security/token/delegation/HiveDelegationTokenSupport.java shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenIdentifier.java shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java shims/src/0.20S/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java shims/src/0.20S/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenSelector.java shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java shims/src/0.23/java/org/apache/hadoop/hive/thrift/DelegationTokenSelector23.java shims/src/0.23/java/org/apache/hadoop/hive/thrift/DelegationTokenIdentifier23.java shims/src/0.23/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java shims/src/common/java/org/apache/hadoop/hive/shims/ShimLoader.java shims/build.xml build.properties build-common.xml shims/src/0.20S/java/org/apache/hadoop/security/token/delegation/HiveDelegationTokenSupport.java@1225709 shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenIdentifier.java@1225709 shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java@1225709 shims/src/0.20S/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java@1225709 shims/src/0.20S/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java@1225709 shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java@1225709 shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java@1225709 shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenSelector.java@1225709 shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java@1225709 shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java@1225705 > Make a single Hive binary work with both 0.20.x and 0.23.0 > -- > > Key: HIVE-2629 > URL: https://issues.apache.org/jira/browse/HIVE-2629 > Project: Hive > Issue Type: Bug > Components: Shims >Reporter: Carl Steinbach >Assignee: Thomas Weise > Fix For: 0.9.0 > > Attachments: HIVE-2629.D711.1.patch, HIVE-2629.D711.2.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-0.8.1-SNAPSHOT-h0.21 - Build # 144 - Fixed
Changes for Build #143 Changes for Build #144 All tests passed The Apache Jenkins build system has built Hive-0.8.1-SNAPSHOT-h0.21 (build #144) Status: Fixed Check console output at https://builds.apache.org/job/Hive-0.8.1-SNAPSHOT-h0.21/144/ to view the results.
Phabricator CodeReview Problem
Hi, This is Zhenxiao Luo, from Cloudera, whose working on Hive, and would like to use Phabricator for code review. I am following the Hive wiki to setup Phabricator: 1. https://reviews.facebook.net/p/zhenxiao/ My account is setup. 2. My machine has php installed: [~]php --version PHP 5.3.8 (cli) (built: Sep 28 2011 17:34:42) Copyright (c) 1997-2011 The PHP Group Zend Engine v2.3.0, Copyright (c) 1998-2011 Zend Technologies 3. Getting arcanist and libphutil: git clone git://github.com/jsichi/libphutil.git git clone git://github.com/jsichi/arcanist.git [~/libphutil]git remote -v origin git://github.com/jsichi/libphutil.git (fetch) origin git://github.com/jsichi/libphutil.git (push) [~/arcanist]git remote -v origin git://github.com/jsichi/arcanist.git (fetch) origin git://github.com/jsichi/arcanist.git (push) 4. ant arc-setup successful 5. arc install-certificat successful 6. add archanist/bin to my path: [~/Code/hive]echo $PATH /home/cloudera/arcanist/bin:/home/cloudera/arcanist/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cloudera/bin 7. While, when I am trying to submit my patch for review, I have the following problem: [~/Code/hive]arc diff --jira HIVE-2203 Exception: ERR-BAD-VERSION: Your 'arc' client version is '2', but this server expects version '3'. Upgrade your 'arc' client. (Run with --trace for a full exception trace.) Any help are appreciated. Thanks, Zhenxiao
[jira] [Updated] (HIVE-2279) Implement sort(array) UDF
[ https://issues.apache.org/jira/browse/HIVE-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2279: -- Attachment: HIVE-2279.D1059.1.patch cwsteinbach requested code review of "HIVE-2279 [jira] Implement sort(array) UDF". Reviewers: JIRA HIVE-2279. Implement sort(array) UDF TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D1059 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSortArray.java ql/src/test/queries/clientpositive/udf_sort_array.q ql/src/test/results/clientpositive/show_functions.q.out ql/src/test/results/clientpositive/udf_sort_array.q.out MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/2241/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. > Implement sort(array) UDF > - > > Key: HIVE-2279 > URL: https://issues.apache.org/jira/browse/HIVE-2279 > Project: Hive > Issue Type: New Feature > Components: UDF >Reporter: Carl Steinbach >Assignee: Zhenxiao Luo > Attachments: HIVE-2279.D1059.1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2616) Passing user identity from metastore client to server in non-secure mode
[ https://issues.apache.org/jira/browse/HIVE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177573#comment-13177573 ] Hudson commented on HIVE-2616: -- Integrated in Hive-trunk-h0.21 #1176 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1176/]) HIVE-2616 : Passing user identity from metastore client to server in non-secure mode (Ashutosh Chauhan) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1225683 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml.template * /hive/trunk/metastore/if/hive_metastore.thrift * /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp * /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h * /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp * /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java * /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php * /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote * /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py * /hive/trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnBothClientServer.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyClient.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestSetUGIOnOnlyServer.java * /hive/trunk/shims/ivy.xml * /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client * /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/client/TUGIAssumingTransport.java * /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java * /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TFilterTransport.java * /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/TUGIContainingTransport.java > Passing user identity from metastore client to server in non-secure mode > > > Key: HIVE-2616 > URL: https://issues.apache.org/jira/browse/HIVE-2616 > Project: Hive > Issue Type: Bug > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Fix For: 0.9.0 > > Attachments: hive-2616.patch, hive-2616_1.patch, hive-2616_3.patch, > hive-2616_4.patch, hive-2616_5.patch > > > Currently in unsecure mode client don't pass on user identity. As a result > hdfs and other operations done by server gets executed by user running > metastore process instead of being done in context of client. This results in > problem as reported here: > http://mail-archives.apache.org/mod_mbox/hive-user/20.mbox/%3CCAK0mCrRC3aPqtRHDe2J25Rm0JX6TS1KXxd7KPjqJjoqBjg=a...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1176 - Still Failing
Changes for Build #1144 [jvs] HIVE-1040 [jira] use sed rather than diff for masking out noise in diff-based tests (Marek Sapota via John Sichi) Summary: Replace diff -I with regex masking in Java The current diff -I approach has two problems: (1) it does not allow resolution finer than line-level, so it's impossible to mask out pattern occurrences within a line, and (2) it produces unmasked files, so if you run diff on the command line to compare the result .q.out with the checked-in file, you see the noise. My suggestion is to first run sed to replace noise patterns with an unlikely-to-occur string like ZYZZYZVA, and then diff the pre-masked files without using any -I. This would require a one-time hit to update all existing .q.out files so that they would contain the pre-masked results. Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 597 Changes for Build #1145 Changes for Build #1146 [namit] HIVE-2640 Add alterPartition to AlterHandler interface (Kevin Wilfong via namit) Changes for Build #1147 [namit] HIVE-2617 Insert overwrite table db.tname fails if partition already exists (Chinna Rao Lalam via namit) Changes for Build #1148 [heyongqiang] HIVE-2651 [jira] The variable hive.exec.mode.local.auto.tasks.max should be changed (Namit Jain via Yongqiang He) Summary: HIVE-2651 It should be called hive.exec.mode.local.auto.input.files.max instead. The number of input files are checked currently. Test Plan: EMPTY Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang Differential Revision: 861 [cws] HIVE-727. Hive Server getSchema() returns wrong schema for 'Explain' queries (Prasad Mujumdar via cws) [namit] HIVE-2611 Make index table output of create index command if index is table based (Kevin Wilfong via namit) Changes for Build #1150 [jvs] HIVE-2657 [jira] builtins JAR is not being published to Maven repo & hive-cli POM does not depend on it either (Carl Steinbach via John Sichi) Summary: Make hive-cli and hive-ql depend on hive-builtins Test Plan: EMPTY Reviewers: JIRA, jsichi Reviewed By: jsichi CC: jsichi Differential Revision: 897 [namit] HIVE-2654 "hive.querylog.location" requires parent directory to be exist or else folder creation fails (Chinna Rao Lalam via namit) Changes for Build #1151 [hashutosh] HIVE-1892 : show functions also returns internal operators (Priyadarshini via Ashutosh Chauhan) Changes for Build #1152 Changes for Build #1153 [namit] HIVE-2660 Need better exception handling in RCFile tolerate corruptions mode (Ramkumar Vadali via namit) Changes for Build #1154 [cws] HIVE-2631. Make Hive work with Hadoop 1.0.0 (Ashutosh Chauhan via cws) Changes for Build #1155 [cws] HIVE-BUILD. Update RELEASE_NOTES.txt with 0.8.0 release information (cws) Changes for Build #1156 Changes for Build #1157 Changes for Build #1158 [namit] HIVE-2602 add support for insert partition overwrite(...) if not exists (Chinna Rao Lalam via namit) Changes for Build #1159 Changes for Build #1160 [cws] HIVE-2005. Implement BETWEEN operator (Navis via cws) Changes for Build #1161 [jvs] HIVE-2433. add DOAP file for Hive Changes for Build #1162 Changes for Build #1163 Changes for Build #1164 [heyongqiang] HIVE-2666 [jira] StackOverflowError when using custom UDF in map join (Kevin Wilfong via Yongqiang He) Summary: Resource files are now added to the class path as soon as they are added via the CLI. This fixes the stack overflow error mentioned in the JIRA by ensuring a consistent class loader between serializers and deserializers for the same query. Note that now serdes which contain a static block to register themselves are now registered twice, once when adding the file to the class loader, and once when an instance of the class is created. Previously, registering a serde twice resulted in an exception, to avoid this, I have downgraded it to a warning. When a custom UDF is used as part of a join which is converted to a map join, the XMLEncoder enters an infinite loop when serializing the map reduce task for the second time, as part of sending it to be executed. This results in a stack overflow error. Test Plan: I ran the unit tests to verify nothing was broken. I ran several queries which used custom UDFs and involved a join which was converted to a map join. I verified these completed successfully consistently Reviewers: JIRA, heyongqiang Reviewed By: heyongqiang CC: heyongqiang, kevinwilfong Differential Revision: 957 [namit] HIVE-2642 fix Hive-2566 and make union optimization more aggressive (Yongqiang He via namit) Changes for Build #1166 Changes for Build #1167 Changes for Build #1168 [heyongqiang] HIVE-2600: Enable/Add type-specific compression for rcfile (Krishna Kumar via He Yongqiang) Changes for Build #1169 Changes for Build #1170 [cws] HIVE-1877. Add java_method() as a synonym for the reflect() UDF (Zhenxiao Luo via cws) Changes for Buil
