Re: Propose to put JIRA traffic on separate hive list
... resolutions are more important to you than creations. Thanks for offering to help with filters, Alan, I might take you up on that someday. But I try not to wait for commits before checking doc issues. It seems rude to ask for parameter description changes after a commit. The filter I really want would check all *patches* for HiveConf.java parameters. That's not possible, is it? All commits are already sent to the commits list ... Um ... what is the commits list? (Assuming you don't mean the release list.) A general question: Would the non-creation JIRA list actually exclude creation messages, or would those go to both lists? Not sure if that matters, just curious. -- Lefty On Fri, Nov 21, 2014 at 5:44 PM, Gunther Hagleitner ghagleit...@hortonworks.com wrote: +1 Thanks, Gunther. On Fri, Nov 21, 2014 at 5:41 PM, Thejas Nair the...@hortonworks.com wrote: +1 On Fri, Nov 21, 2014 at 5:16 PM, Alan Gates ga...@hortonworks.com wrote: Do you mean RB creation? I wouldn't be in favor of sending all RB activity to it, since we're trying to create a low traffic list. Alan. Brock Noland br...@cloudera.com November 21, 2014 at 17:10 I think that RB activity could be put on this same list? Brock Noland br...@cloudera.com November 21, 2014 at 17:09 +1 Szehon Ho sze...@cloudera.com November 21, 2014 at 17:03 +1. Create + resolve might be a good idea too, as I imagine contributors might want to see what is changing in Hive, and not just issues created. Thanks, Szehon On Fri, Nov 21, 2014 at 1:15 PM, Prasanth Jayachandran Prasanth Jayachandran pjayachand...@hortonworks.com November 21, 2014 at 13:15 +1 - Prasanth Alan Gates ga...@hortonworks.com November 21, 2014 at 12:59 Poke, I got some scattered +1s but I'd like to get feedback from a higher percentage of committers before filing the infra ticket to do this. Or maybe a better way to put it is, is anyone opposed? Lefty, to answer your question, we could include resolutions as well. But that doubles the traffic going to the dev list. I get why you're concerned with it, since resolutions are more important to you than creations. I'm happy to help you set up a filter on the list with the non-creation JIRA traffic so you can just see the resolutions. Alan. -- Sent with Postbox http://www.getpostbox.com CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-7444) Update supported operating systems requirements in wikidoc
[ https://issues.apache.org/jira/browse/HIVE-7444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14221870#comment-14221870 ] Lefty Leverenz commented on HIVE-7444: -- +1 Thanks very much, [~thejas]! I put the same information in [Installing Hive| https://cwiki.apache.org/confluence/display/Hive/AdminManual+Installation#AdminManualInstallation-InstallingHive]. Update supported operating systems requirements in wikidoc Key: HIVE-7444 URL: https://issues.apache.org/jira/browse/HIVE-7444 Project: Hive Issue Type: Bug Components: Documentation Reporter: Lefty Leverenz The first sentence of Getting Started is outdated: {quote} DISCLAIMER: Hive has only been tested on Unix (Linux) and Mac systems using Java 1.6 for now – although it may very well work on other similar platforms. It does not work on Cygwin. {quote} The Requirements section also needs updating: {quote} Requirements * Java 1.6 * Hadoop 0.20.x, 0.23.x, or 2.0.x-alpha {quote} Quick reference: * [Getting Started | https://cwiki.apache.org/confluence/display/Hive/GettingStarted] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8929) incorrect error message for cbo path
[ https://issues.apache.org/jira/browse/HIVE-8929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14221872#comment-14221872 ] Hive QA commented on HIVE-8929: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12682766/HIVE-8929.patch {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), tests executed *Failed tests:* {noformat} org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Json {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1861/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1861/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1861/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12682766 - PreCommit-HIVE-TRUNK-Build incorrect error message for cbo path Key: HIVE-8929 URL: https://issues.apache.org/jira/browse/HIVE-8929 Project: Hive Issue Type: Test Components: CBO Affects Versions: 0.14.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-8929.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8922) CBO: assorted date and timestamp issues
[ https://issues.apache.org/jira/browse/HIVE-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14221874#comment-14221874 ] Hive QA commented on HIVE-8922: --- {color:red}Overall{color}: -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12682777/HIVE-8922.patch Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1862/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1862/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1862/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]] + export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera + JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera + export PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin + PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + cd /data/hive-ptest/working/ + tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-1862/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ svn = \s\v\n ]] + [[ -n '' ]] + [[ -d apache-svn-trunk-source ]] + [[ ! -d apache-svn-trunk-source/.svn ]] + [[ ! -d apache-svn-trunk-source ]] + cd apache-svn-trunk-source + svn revert -R . Reverted 'ql/src/java/org/apache/hadoop/hive/ql/optimizer/optiq/translator/JoinCondTypeCheckProcFactory.java' ++ awk '{print $2}' ++ egrep -v '^X|^Performing status on external' ++ svn status --no-ignore + rm -rf target datanucleus.log ant/target shims/target shims/0.20/target shims/0.20S/target shims/0.23/target shims/aggregator/target shims/common/target shims/common-secure/target shims/scheduler/target packaging/target hbase-handler/target testutils/target jdbc/target metastore/target itests/target itests/hcatalog-unit/target itests/test-serde/target itests/qtest/target itests/hive-unit-hadoop2/target itests/hive-minikdc/target itests/hive-unit/target itests/custom-serde/target itests/util/target hcatalog/target hcatalog/core/target hcatalog/streaming/target hcatalog/server-extensions/target hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target accumulo-handler/target hwi/target common/target common/src/gen contrib/target service/target serde/target beeline/target odbc/target cli/target ql/dependency-reduced-pom.xml ql/target + svn update Fetching external item into 'hcatalog/src/test/e2e/harness' External at revision 1641032. At revision 1641032. + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hive-ptest/working/scratch/build.patch + [[ -f /data/hive-ptest/working/scratch/build.patch ]] + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch The patch does not appear to apply with p0, p1, or p2 + exit 1 ' {noformat} This message is automatically generated. ATTACHMENT ID: 12682777 - PreCommit-HIVE-TRUNK-Build CBO: assorted date and timestamp issues --- Key: HIVE-8922 URL: https://issues.apache.org/jira/browse/HIVE-8922 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.15.0 Attachments: HIVE-8922.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8825) SQLCompletor catches Throwable and ignores it
[ https://issues.apache.org/jira/browse/HIVE-8825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14221916#comment-14221916 ] Hive QA commented on HIVE-8825: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12682799/HIVE-8825.1.patch {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1864/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1864/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1864/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12682799 - PreCommit-HIVE-TRUNK-Build SQLCompletor catches Throwable and ignores it - Key: HIVE-8825 URL: https://issues.apache.org/jira/browse/HIVE-8825 Project: Hive Issue Type: Bug Reporter: Brock Noland Assignee: Ferdinand Xu Attachments: HIVE-8825.1.patch, HIVE-8825.patch We should be catching the specific exception which is thrown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HIVE-8834) enable job progress monitoring of Remote Spark Context [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14220922#comment-14220922 ] Xuefu Zhang edited comment on HIVE-8834 at 11/22/14 11:49 AM: -- Thanks for working on this, [~lirui]. {quote} it involves some dependency issue between hive-exec and spark-client. {quote} Moving the classes from hive-exec to spark-client is fine. If Kryo makes life easier, we can also use it for serialization/de- by making a similar implementation of KryoSerializer here as well. was (Author: xuefuz): Thanks for working on this, [~lirui]. {quote} it involves some dependency issue between hive-exec and spark-client. {quote} Maybe this is obvious. I haven't go thru the patch in detail, but could you explain the dependency? Do you mean some classes from hive-exec are referenced on the remote spark client side? Can you gave an example? enable job progress monitoring of Remote Spark Context [Spark Branch] - Key: HIVE-8834 URL: https://issues.apache.org/jira/browse/HIVE-8834 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Chengxiang Li Assignee: Rui Li Labels: Spark-M3 Attachments: HIVE-8834.1-spark.patch, HIVE-8834.2-spark.patch, HIVE-8834.3-spark.patch We should enable job progress monitor in Remote Spark Context, the spark job progress info should fit into SparkJobStatus. SPARK-2321 supply new spark progress API, which should make this task easier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8916) Handle user@domain username under LDAP authentication
[ https://issues.apache.org/jira/browse/HIVE-8916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222011#comment-14222011 ] Mohit Sabharwal commented on HIVE-8916: --- Both tests pass for me locally. Handle user@domain username under LDAP authentication - Key: HIVE-8916 URL: https://issues.apache.org/jira/browse/HIVE-8916 Project: Hive Issue Type: Bug Components: Authentication Reporter: Mohit Sabharwal Assignee: Mohit Sabharwal Attachments: HIVE-8916.2.patch, HIVE-8916.3.patch, HIVE-8916.patch If LDAP is configured with multiple domains for authentication, users can be in different domains. Currently, LdapAuthenticationProviderImpl blindly appends the domain configured hive.server2.authentication.ldap.Domain to the username, which limits user to that domain. However, under multi-domain authentication, the username may already include the domain (ex: u...@domain.foo.com). We should not append a domain if one is already present. Also, if username already includes the domain, rest of Hive and authorization providers still expects the short name (user and not u...@domain.foo.com) for looking up privilege rules, etc. As such, any domain info in the username should be stripped off. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Hive-0.14 - Build # 741 - Still Failing
Changes for Build #696 [rohini] PIG-4186: Fix e2e run against new build of pig and some enhancements (rohini) Changes for Build #697 Changes for Build #698 Changes for Build #699 Changes for Build #700 Changes for Build #701 Changes for Build #702 Changes for Build #703 [daijy] HIVE-8484: HCatalog throws an exception if Pig job is of type 'fetch' (Lorand Bendig via Daniel Dai) Changes for Build #704 [gunther] HIVE-8781: Nullsafe joins are busted on Tez (Gunther Hagleitner, reviewed by Prasanth J) Changes for Build #705 [gunther] HIVE-8760: Pass a copy of HiveConf to hooks (Gunther Hagleitner, reviewed by Gopal V) Changes for Build #706 [thejas] HIVE-8772 : zookeeper info logs are always printed from beeline with service discovery mode (Thejas Nair, reviewed by Vaibhav Gumashta) Changes for Build #707 [gunther] HIVE-8782: HBase handler doesn't compile with hadoop-1 (Jimmy Xiang, reviewed by Xuefu and Sergey) Changes for Build #708 Changes for Build #709 [thejas] HIVE-8785 : HiveServer2 LogDivertAppender should be more selective for beeline getLogs (Thejas Nair, reviewed by Gopal V) Changes for Build #710 [vgumashta] HIVE-8764: Windows: HiveServer2 TCP SSL cannot recognize localhost (Vaibhav Gumashta reviewed by Thejas Nair) Changes for Build #711 [gunther] HIVE-8768: CBO: Fix filter selectivity for 'in clause' '' (Laljo John Pullokkaran via Gunther Hagleitner) Changes for Build #712 [gunther] HIVE-8794: Hive on Tez leaks AMs when killed before first dag is run (Gunther Hagleitner, reviewed by Gopal V) Changes for Build #713 [gunther] HIVE-8798: Some Oracle deadlocks not being caught in TxnHandler (Alan Gates via Gunther Hagleitner) Changes for Build #714 [gunther] HIVE-8800: Update release notes and notice for hive .14 (Gunther Hagleitner, reviewed by Prasanth J) [gunther] HIVE-8799: boatload of missing apache headers (Gunther Hagleitner, reviewed by Thejas M Nair) Changes for Build #715 [gunther] Preparing for release 0.14.0 Changes for Build #716 [gunther] Preparing for release 0.14.0 [gunther] Preparing for release 0.14.0 Changes for Build #717 Changes for Build #718 Changes for Build #719 Changes for Build #720 [gunther] HIVE-8811: Dynamic partition pruning can result in NPE during query compilation (Gunther Hagleitner, reviewed by Gopal V) Changes for Build #721 [gunther] HIVE-8805: CBO skipped due to SemanticException: Line 0:-1 Both left and right aliases encountered in JOIN 'avg_cs_ext_discount_amt' (Laljo John Pullokkaran via Gunther Hagleitner) [sershe] HIVE-8715 : Hive 14 upgrade scripts can fail for statistics if database was created using auto-create ADDENDUM (Sergey Shelukhin, reviewed by Ashutosh Chauhan and Gunther Hagleitner) Changes for Build #722 Changes for Build #723 Changes for Build #724 [gunther] HIVE-8845: Switch to Tez 0.5.2 (Gunther Hagleitner, reviewed by Gopal V) Changes for Build #725 [sershe] HIVE-8295 : Add batch retrieve partition objects for metastore direct sql (Selina Zhang and Sergey Shelukhin, reviewed by Ashutosh Chauhan) Changes for Build #726 Changes for Build #727 [gunther] HIVE-8873: Switch to calcite 0.9.2 (Gunther Hagleitner, reviewed by Gopal V) Changes for Build #728 [thejas] HIVE-8830 : hcatalog process don't exit because of non daemon thread (Thejas Nair, reviewed by Eugene Koifman, Sushanth Sowmyan) Changes for Build #729 Changes for Build #730 Changes for Build #731 Changes for Build #732 Changes for Build #733 Changes for Build #734 Changes for Build #735 Changes for Build #736 [sershe] HIVE-8876 : incorrect upgrade script for Oracle (13-14) (Sergey Shelukhin, reviewed by Ashutosh Chauhan) Changes for Build #737 Changes for Build #738 [gunther] HIVE-: Mapjoin with LateralViewJoin generates wrong plan in Tez (Prasanth J via Gunther Hagleitner) Changes for Build #739 Changes for Build #740 [cws] HIVE-8933. Check release builds for SNAPSHOT dependencies Changes for Build #741 No tests ran. The Apache Jenkins build system has built Hive-0.14 (build #741) Status: Still Failing Check console output at https://builds.apache.org/job/Hive-0.14/741/ to view the results.
[jira] [Commented] (HIVE-8910) Refactoring of PassThroughOutputFormat
[ https://issues.apache.org/jira/browse/HIVE-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222050#comment-14222050 ] Hive QA commented on HIVE-8910: --- {color:green}Overall{color}: +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12682806/HIVE-8910.3.patch.txt {color:green}SUCCESS:{color} +1 tests passed Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1865/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1865/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1865/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12682806 - PreCommit-HIVE-TRUNK-Build Refactoring of PassThroughOutputFormat --- Key: HIVE-8910 URL: https://issues.apache.org/jira/browse/HIVE-8910 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-8910.1.patch.txt, HIVE-8910.2.patch.txt, HIVE-8910.3.patch.txt It's overly complicated just for doing simple wrapping of output format. Before things get more worse, we should refactor this codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-4766) Support HS2 client login timeout when the thrift thread max# is reached
[ https://issues.apache.org/jira/browse/HIVE-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang reassigned HIVE-4766: - Assignee: Chaoyu Tang (was: Mohit Sabharwal) Support HS2 client login timeout when the thrift thread max# is reached --- Key: HIVE-4766 URL: https://issues.apache.org/jira/browse/HIVE-4766 Project: Hive Issue Type: Improvement Components: HiveServer2 Affects Versions: 0.10.0 Reporter: Chaoyu Tang Assignee: Chaoyu Tang HiveServer2 client (beeline) hangs in login if the thrift max thread# has been reached. It is because the server crashes due to a defect in currently used thrift 0.9.0. When hive is upgraded to use a new version of Thrift (say thrift 1.0), HS2 should support client login timeout instead of current hanging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-4766) Support HS2 client login timeout when the thrift thread max# is reached
[ https://issues.apache.org/jira/browse/HIVE-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-4766: -- Fix Version/s: 0.14.1 0.15.0 Labels: TODOC15 (was: ) Status: Patch Available (was: Open) Support HS2 client login timeout when the thrift thread max# is reached --- Key: HIVE-4766 URL: https://issues.apache.org/jira/browse/HIVE-4766 Project: Hive Issue Type: Improvement Components: HiveServer2 Affects Versions: 0.10.0 Reporter: Chaoyu Tang Assignee: Chaoyu Tang Labels: TODOC15 Fix For: 0.15.0, 0.14.1 Attachments: HIVE-4766.patch HiveServer2 client (beeline) hangs in login if the thrift max thread# has been reached. It is because the server crashes due to a defect in currently used thrift 0.9.0. When hive is upgraded to use a new version of Thrift (say thrift 1.0), HS2 should support client login timeout instead of current hanging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-4766) Support HS2 client login timeout when the thrift thread max# is reached
[ https://issues.apache.org/jira/browse/HIVE-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-4766: -- Attachment: HIVE-4766.patch Hive has upgraded to use Thrift-0.92 (See HIVE-8829) which includes THRIFT-2046 that provides the supports to loginTimeout and binary exponential backoff algorithm in login retry interval. Change Hive to adopt these features. Two new configurations are introduced: 1. hive.server2.thrift.login.timeout -- timeout for the thrift client to login (in seconds) 2. hive.server2.thrift.exponential.backoff.slotlength: Slot time of binary exponential backoff algorithm that HS2 Thrift Server uses for its login retry Support HS2 client login timeout when the thrift thread max# is reached --- Key: HIVE-4766 URL: https://issues.apache.org/jira/browse/HIVE-4766 Project: Hive Issue Type: Improvement Components: HiveServer2 Affects Versions: 0.10.0 Reporter: Chaoyu Tang Assignee: Chaoyu Tang Labels: TODOC15 Fix For: 0.15.0, 0.14.1 Attachments: HIVE-4766.patch HiveServer2 client (beeline) hangs in login if the thrift max thread# has been reached. It is because the server crashes due to a defect in currently used thrift 0.9.0. When hive is upgraded to use a new version of Thrift (say thrift 1.0), HS2 should support client login timeout instead of current hanging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8910) Refactoring of PassThroughOutputFormat
[ https://issues.apache.org/jira/browse/HIVE-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-8910: --- Resolution: Fixed Fix Version/s: 0.15.0 Status: Resolved (was: Patch Available) Committed to trunk. Refactoring of PassThroughOutputFormat --- Key: HIVE-8910 URL: https://issues.apache.org/jira/browse/HIVE-8910 Project: Hive Issue Type: Improvement Components: Query Processor, StorageHandler Reporter: Navis Assignee: Navis Priority: Minor Fix For: 0.15.0 Attachments: HIVE-8910.1.patch.txt, HIVE-8910.2.patch.txt, HIVE-8910.3.patch.txt It's overly complicated just for doing simple wrapping of output format. Before things get more worse, we should refactor this codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8910) Refactoring of PassThroughOutputFormat
[ https://issues.apache.org/jira/browse/HIVE-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-8910: --- Component/s: StorageHandler Refactoring of PassThroughOutputFormat --- Key: HIVE-8910 URL: https://issues.apache.org/jira/browse/HIVE-8910 Project: Hive Issue Type: Improvement Components: Query Processor, StorageHandler Reporter: Navis Assignee: Navis Priority: Minor Fix For: 0.15.0 Attachments: HIVE-8910.1.patch.txt, HIVE-8910.2.patch.txt, HIVE-8910.3.patch.txt It's overly complicated just for doing simple wrapping of output format. Before things get more worse, we should refactor this codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8929) incorrect error message for cbo path
[ https://issues.apache.org/jira/browse/HIVE-8929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-8929: --- Resolution: Fixed Fix Version/s: 0.15.0 Status: Resolved (was: Patch Available) Committed to trunk. incorrect error message for cbo path Key: HIVE-8929 URL: https://issues.apache.org/jira/browse/HIVE-8929 Project: Hive Issue Type: Test Components: CBO Affects Versions: 0.14.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.15.0 Attachments: HIVE-8929.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8923) HIVE-8512 needs to be fixed also for CBO
[ https://issues.apache.org/jira/browse/HIVE-8923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222113#comment-14222113 ] Hive QA commented on HIVE-8923: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12682804/HIVE-8923.patch {color:red}ERROR:{color} -1 due to 6 failed/errored test(s), tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_gby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_limit org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_semijoin org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cbo_gby org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cbo_limit org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cbo_semijoin {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1866/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1866/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1866/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12682804 - PreCommit-HIVE-TRUNK-Build HIVE-8512 needs to be fixed also for CBO Key: HIVE-8923 URL: https://issues.apache.org/jira/browse/HIVE-8923 Project: Hive Issue Type: Sub-task Components: CBO Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.15.0 Attachments: HIVE-8923.patch Queries reverted to incorrect results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8916) Handle user@domain username under LDAP authentication
[ https://issues.apache.org/jira/browse/HIVE-8916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-8916: --- Resolution: Fixed Fix Version/s: 0.15.0 Status: Resolved (was: Patch Available) Thank you very much Mohit! I have committed this to trunk. Handle user@domain username under LDAP authentication - Key: HIVE-8916 URL: https://issues.apache.org/jira/browse/HIVE-8916 Project: Hive Issue Type: Bug Components: Authentication Reporter: Mohit Sabharwal Assignee: Mohit Sabharwal Fix For: 0.15.0 Attachments: HIVE-8916.2.patch, HIVE-8916.3.patch, HIVE-8916.patch If LDAP is configured with multiple domains for authentication, users can be in different domains. Currently, LdapAuthenticationProviderImpl blindly appends the domain configured hive.server2.authentication.ldap.Domain to the username, which limits user to that domain. However, under multi-domain authentication, the username may already include the domain (ex: u...@domain.foo.com). We should not append a domain if one is already present. Also, if username already includes the domain, rest of Hive and authorization providers still expects the short name (user and not u...@domain.foo.com) for looking up privilege rules, etc. As such, any domain info in the username should be stripped off. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8825) SQLCompletor catches Throwable and ignores it
[ https://issues.apache.org/jira/browse/HIVE-8825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-8825: --- Resolution: Fixed Fix Version/s: 0.15.0 Status: Resolved (was: Patch Available) Thank you very much Ferdinand! I have committed this to trunk! SQLCompletor catches Throwable and ignores it - Key: HIVE-8825 URL: https://issues.apache.org/jira/browse/HIVE-8825 Project: Hive Issue Type: Bug Reporter: Brock Noland Assignee: Ferdinand Xu Fix For: 0.15.0 Attachments: HIVE-8825.1.patch, HIVE-8825.patch We should be catching the specific exception which is thrown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7444) Update supported operating systems requirements in wikidoc
[ https://issues.apache.org/jira/browse/HIVE-7444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222121#comment-14222121 ] Brock Noland commented on HIVE-7444: +1 on the updates. Thank you for taking care of this. We should state that version 0.15 will be Java 7 or newer since by the time it's release we'll actually only be a month or two from Java 7 EOL and users would be strongly advised to start moving to Java 8. Brock Update supported operating systems requirements in wikidoc Key: HIVE-7444 URL: https://issues.apache.org/jira/browse/HIVE-7444 Project: Hive Issue Type: Bug Components: Documentation Reporter: Lefty Leverenz The first sentence of Getting Started is outdated: {quote} DISCLAIMER: Hive has only been tested on Unix (Linux) and Mac systems using Java 1.6 for now – although it may very well work on other similar platforms. It does not work on Cygwin. {quote} The Requirements section also needs updating: {quote} Requirements * Java 1.6 * Hadoop 0.20.x, 0.23.x, or 2.0.x-alpha {quote} Quick reference: * [Getting Started | https://cwiki.apache.org/confluence/display/Hive/GettingStarted] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8935) Add debug logging around token stores
[ https://issues.apache.org/jira/browse/HIVE-8935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222162#comment-14222162 ] Hive QA commented on HIVE-8935: --- {color:green}Overall{color}: +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12682942/HIVE-8935.patch {color:green}SUCCESS:{color} +1 tests passed Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1867/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1867/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1867/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12682942 - PreCommit-HIVE-TRUNK-Build Add debug logging around token stores - Key: HIVE-8935 URL: https://issues.apache.org/jira/browse/HIVE-8935 Project: Hive Issue Type: Task Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-8935.patch, HIVE-8935.patch It's hard to debug issues related to delegation tokens due to a lack of debug logging. This jira is to add debug logging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8828) Remove hadoop 20 shims
[ https://issues.apache.org/jira/browse/HIVE-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-8828: --- Status: Open (was: Patch Available) Remove hadoop 20 shims -- Key: HIVE-8828 URL: https://issues.apache.org/jira/browse/HIVE-8828 Project: Hive Issue Type: Task Components: Shims Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-8828.1.patch, HIVE-8828.2.patch, HIVE-8828.3.patch, HIVE-8828.4.patch, HIVE-8828.patch CLEAR LIBRARY CACHE See : [mailing list discussion | http://mail-archives.apache.org/mod_mbox/hive-dev/201410.mbox/%3CCABgNGzfSB5VGTecONg0GgLCDdLLFfzLuZvP%2BGSBc0i0joqf3fg%40mail.gmail.com%3E] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8828) Remove hadoop 20 shims
[ https://issues.apache.org/jira/browse/HIVE-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-8828: --- Status: Patch Available (was: Open) Remove hadoop 20 shims -- Key: HIVE-8828 URL: https://issues.apache.org/jira/browse/HIVE-8828 Project: Hive Issue Type: Task Components: Shims Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-8828.1.patch, HIVE-8828.2.patch, HIVE-8828.3.patch, HIVE-8828.4.patch, HIVE-8828.5.patch, HIVE-8828.patch CLEAR LIBRARY CACHE See : [mailing list discussion | http://mail-archives.apache.org/mod_mbox/hive-dev/201410.mbox/%3CCABgNGzfSB5VGTecONg0GgLCDdLLFfzLuZvP%2BGSBc0i0joqf3fg%40mail.gmail.com%3E] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8828) Remove hadoop 20 shims
[ https://issues.apache.org/jira/browse/HIVE-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-8828: --- Attachment: HIVE-8828.5.patch More refactoring Remove hadoop 20 shims -- Key: HIVE-8828 URL: https://issues.apache.org/jira/browse/HIVE-8828 Project: Hive Issue Type: Task Components: Shims Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-8828.1.patch, HIVE-8828.2.patch, HIVE-8828.3.patch, HIVE-8828.4.patch, HIVE-8828.5.patch, HIVE-8828.patch CLEAR LIBRARY CACHE See : [mailing list discussion | http://mail-archives.apache.org/mod_mbox/hive-dev/201410.mbox/%3CCABgNGzfSB5VGTecONg0GgLCDdLLFfzLuZvP%2BGSBc0i0joqf3fg%40mail.gmail.com%3E] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 27895: Remove Hadoop 20 shims
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27895/ --- (Updated Nov. 22, 2014, 8:43 p.m.) Review request for hive and Thejas Nair. Changes --- deleted more methods from shims Bugs: HIVE-8828 https://issues.apache.org/jira/browse/HIVE-8828 Repository: hive-git Description --- Remove Hadoop 20 shims Diffs (updated) - beeline/src/test/org/apache/hive/beeline/ProxyAuthTest.java 95146e9 common/src/java/org/apache/hadoop/hive/common/FileUtils.java 95e8d7c common/src/java/org/apache/hadoop/hive/conf/HiveConf.java fafd78e hcatalog/core/src/main/java/org/apache/hive/hcatalog/common/HiveClientCache.java ffa648d hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/Security.java 39ef86e itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/MiniHiveKdc.java 9bf5e1f itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestJdbcWithMiniKdc.java 3e46bed itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestMiniHiveKdc.java 9d69952 itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/ql/security/TestStorageBasedMetastoreAuthorizationProviderWithACL.java 9982195 itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java b2bdafa itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/StorageBasedMetastoreTestBase.java 1d8ac24 itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestClientSideAuthorizationProvider.java f474d83 itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestMetastoreAuthorizationProvider.java 3bde2fc itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestStorageBasedMetastoreAuthorizationDrops.java c7b27a6 itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java faa51af jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java cfac55b metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 0224629 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java fecae97 metastore/src/java/org/apache/hadoop/hive/metastore/RetryingMetaStoreClient.java 0e1fafc metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java ef1eee2 metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java c99ce5f pom.xml 63d1e8a ql/pom.xml fa6c6d9 ql/src/java/org/apache/hadoop/hive/ql/Driver.java 42e1e20 ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java f834ad5 ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 56fd5a0 ql/src/java/org/apache/hadoop/hive/ql/exec/SecureCmdDoAs.java 832f84f ql/src/java/org/apache/hadoop/hive/ql/exec/errors/TaskLogProcessor.java 12433ca ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java 18e40b3 ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java d0c022b ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java 0e326cf ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java e5fce14 ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java 65a0090 ql/src/java/org/apache/hadoop/hive/ql/hooks/HookContext.java 260444f ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java 2ac60c0 ql/src/java/org/apache/hadoop/hive/ql/io/merge/MergeFileTask.java 4c2843c ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/stats/PartialScanTask.java 75e83b8 ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/truncate/ColumnTruncateTask.java 51a2cc6 ql/src/java/org/apache/hadoop/hive/ql/security/HadoopDefaultAuthenticator.java d68d19d ql/src/java/org/apache/hadoop/hive/ql/security/ProxyUserAuthenticator.java 95a98fe ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 2806bd1 ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestTezSessionState.java 63687eb ql/src/test/queries/clientpositive/alter_numbuckets_partitioned_table.q 5dda4c0 ql/src/test/queries/clientpositive/alter_numbuckets_partitioned_table2.q acc028b ql/src/test/queries/clientpositive/alter_numbuckets_partitioned_table2_h23.q d814304 ql/src/test/queries/clientpositive/alter_numbuckets_partitioned_table_h23.q a039925 ql/src/test/queries/clientpositive/archive.q a928a81 ql/src/test/queries/clientpositive/archive_corrupt.q cc9801d ql/src/test/queries/clientpositive/archive_excludeHadoop20.q 90757f2 ql/src/test/queries/clientpositive/auto_join14.q b282fb9 ql/src/test/queries/clientpositive/auto_join14_hadoop20.q 235b7c1 ql/src/test/queries/clientpositive/combine2.q 615986d ql/src/test/queries/clientpositive/combine2_hadoop20.q 9a9782a ql/src/test/queries/clientpositive/combine2_win.q f6090bb ql/src/test/queries/clientpositive/ctas.q 71af40e
[jira] [Updated] (HIVE-8774) CBO: enable groupBy index
[ https://issues.apache.org/jira/browse/HIVE-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-8774: -- Status: Patch Available (was: Open) CBO: enable groupBy index - Key: HIVE-8774 URL: https://issues.apache.org/jira/browse/HIVE-8774 Project: Hive Issue Type: Improvement Reporter: Pengcheng Xiong Assignee: Pengcheng Xiong Attachments: HIVE-8774.1.patch, HIVE-8774.2.patch, HIVE-8774.3.patch, HIVE-8774.4.patch, HIVE-8774.5.patch, HIVE-8774.6.patch Right now, even when groupby index is build, CBO is not able to use it. In this patch, we are trying to make it use groupby index that we build. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8774) CBO: enable groupBy index
[ https://issues.apache.org/jira/browse/HIVE-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-8774: -- Status: Open (was: Patch Available) CBO: enable groupBy index - Key: HIVE-8774 URL: https://issues.apache.org/jira/browse/HIVE-8774 Project: Hive Issue Type: Improvement Reporter: Pengcheng Xiong Assignee: Pengcheng Xiong Attachments: HIVE-8774.1.patch, HIVE-8774.2.patch, HIVE-8774.3.patch, HIVE-8774.4.patch, HIVE-8774.5.patch, HIVE-8774.6.patch Right now, even when groupby index is build, CBO is not able to use it. In this patch, we are trying to make it use groupby index that we build. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8926) Projections that only swap input columns are identified incorrectly as identity projections
[ https://issues.apache.org/jira/browse/HIVE-8926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesús Camacho Rodríguez updated HIVE-8926: -- Status: In Progress (was: Patch Available) Projections that only swap input columns are identified incorrectly as identity projections --- Key: HIVE-8926 URL: https://issues.apache.org/jira/browse/HIVE-8926 Project: Hive Issue Type: Bug Components: Logical Optimizer Affects Versions: 0.15.0 Reporter: Jesús Camacho Rodríguez Assignee: Jesús Camacho Rodríguez Fix For: 0.15.0 Attachments: HIVE-8926.01.patch, HIVE-8926.patch Projection operators that only swap the input columns in the tuples are identified as identity projections, and thus incorrectly deleted from the plan by the _identity project removal_ optimization. This bug was introduced in HIVE-8435. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8926) Projections that only swap input columns are identified incorrectly as identity projections
[ https://issues.apache.org/jira/browse/HIVE-8926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesús Camacho Rodríguez updated HIVE-8926: -- Status: Patch Available (was: In Progress) Projections that only swap input columns are identified incorrectly as identity projections --- Key: HIVE-8926 URL: https://issues.apache.org/jira/browse/HIVE-8926 Project: Hive Issue Type: Bug Components: Logical Optimizer Affects Versions: 0.15.0 Reporter: Jesús Camacho Rodríguez Assignee: Jesús Camacho Rodríguez Fix For: 0.15.0 Attachments: HIVE-8926.01.patch, HIVE-8926.patch Projection operators that only swap the input columns in the tuples are identified as identity projections, and thus incorrectly deleted from the plan by the _identity project removal_ optimization. This bug was introduced in HIVE-8435. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Propose to put JIRA traffic on separate hive list
Lefty Leverenz mailto:leftylever...@gmail.com November 22, 2014 at 0:02 ... resolutions are more important to you than creations. Thanks for offering to help with filters, Alan, I might take you up on that someday. But I try not to wait for commits before checking doc issues. It seems rude to ask for parameter description changes after a commit. The filter I really want would check all *patches* for HiveConf.java parameters. That's not possible, is it? I don't know of a way to do that. A JIRA query that shows all attachments newer than a given date might work better. All commits are already sent to the commits list ... Um ... what is the commits list? (Assuming you don't mean the release list.) comm...@hive.apache.org All committers should be subscribed to this. A general question: Would the non-creation JIRA list actually exclude creation messages, or would those go to both lists? Not sure if that matters, just curious. I've been wondering that myself. Assuming we go forward with this I'll work with infra to figure that out. Alan. -- Lefty On Fri, Nov 21, 2014 at 5:44 PM, Gunther Hagleitner ghagleit...@hortonworks.com wrote: +1 Thanks, Gunther. On Fri, Nov 21, 2014 at 5:41 PM, Thejas Nairthe...@hortonworks.com wrote: +1 On Fri, Nov 21, 2014 at 5:16 PM, Alan Gatesga...@hortonworks.com wrote: Do you mean RB creation? I wouldn't be in favor of sending all RB activity to it, since we're trying to create a low traffic list. Alan. Brock Nolandbr...@cloudera.com November 21, 2014 at 17:10 I think that RB activity could be put on this same list? Brock Nolandbr...@cloudera.com November 21, 2014 at 17:09 +1 Szehon Hosze...@cloudera.com November 21, 2014 at 17:03 +1. Create + resolve might be a good idea too, as I imagine contributors might want to see what is changing in Hive, and not just issues created. Thanks, Szehon On Fri, Nov 21, 2014 at 1:15 PM, Prasanth Jayachandran Prasanth Jayachandranpjayachand...@hortonworks.com November 21, 2014 at 13:15 +1 - Prasanth Alan Gatesga...@hortonworks.com November 21, 2014 at 12:59 Poke, I got some scattered +1s but I'd like to get feedback from a higher percentage of committers before filing the infra ticket to do this. Or maybe a better way to put it is, is anyone opposed? Lefty, to answer your question, we could include resolutions as well. But that doubles the traffic going to the dev list. I get why you're concerned with it, since resolutions are more important to you than creations. I'm happy to help you set up a filter on the list with the non-creation JIRA traffic so you can just see the resolutions. Alan. -- Sent with Postboxhttp://www.getpostbox.com CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. Gunther Hagleitner mailto:ghagleit...@hortonworks.com November 21, 2014 at 17:44 +1 Thanks, Gunther. Thejas Nair mailto:the...@hortonworks.com November 21, 2014 at 17:41 +1 Alan Gates mailto:ga...@hortonworks.com November 21, 2014 at 17:16 Do you mean RB creation? I wouldn't be in favor of sending all RB activity to it, since we're trying to create a low traffic list. Alan. Brock Noland mailto:br...@cloudera.com November 21, 2014 at 17:10 I think that RB activity could be put on this
[jira] [Commented] (HIVE-860) Persistent distributed cache
[ https://issues.apache.org/jira/browse/HIVE-860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1420#comment-1420 ] Hive QA commented on HIVE-860: -- {color:red}Overall{color}: -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12682902/HIVE-860.2.patch Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1868/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1868/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1868/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Tests exited with: ExecutionException: java.util.concurrent.ExecutionException: org.apache.hive.ptest.execution.ssh.SSHExecutionException: RSyncResult [localFile=/data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-1868/succeeded/TestCliDriver-avro_add_column.q-orc_wide_table.q-query_with_semi.q-and-12-more, remoteFile=/home/hiveptest/54.227.92.185-hiveptest-2/logs/, getExitCode()=12, getException()=null, getUser()=hiveptest, getHost()=54.227.92.185, getInstance()=2]: 'Address 54.227.92.185 maps to ec2-54-227-92-185.compute-1.amazonaws.com, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT! receiving incremental file list ./ TEST-TestCliDriver-avro_add_column.q-orc_wide_table.q-query_with_semi.q-and-12-more-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml 0 0%0.00kB/s0:00:00 7012 100%6.69MB/s0:00:00 (xfer#1, to-check=3/5) hive.log 0 0%0.00kB/s0:00:00 42008576 7% 40.06MB/s0:00:12 rsync: write failed on /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-1868/succeeded/TestCliDriver-avro_add_column.q-orc_wide_table.q-query_with_semi.q-and-12-more/hive.log: No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(301) [receiver=3.0.6] rsync: connection unexpectedly closed (244 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [generator=3.0.6] Address 54.227.92.185 maps to ec2-54-227-92-185.compute-1.amazonaws.com, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT! receiving incremental file list ./ hive.log 0 0%0.00kB/s0:00:00 rsync: write failed on /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-1868/succeeded/TestCliDriver-avro_add_column.q-orc_wide_table.q-query_with_semi.q-and-12-more/hive.log: No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(301) [receiver=3.0.6] rsync: connection unexpectedly closed (244 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [generator=3.0.6] Address 54.227.92.185 maps to ec2-54-227-92-185.compute-1.amazonaws.com, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT! receiving incremental file list ./ hive.log 0 0%0.00kB/s0:00:00 rsync: write failed on /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-1868/succeeded/TestCliDriver-avro_add_column.q-orc_wide_table.q-query_with_semi.q-and-12-more/hive.log: No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(301) [receiver=3.0.6] rsync: connection unexpectedly closed (244 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [generator=3.0.6] Address 54.227.92.185 maps to ec2-54-227-92-185.compute-1.amazonaws.com, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT! receiving incremental file list ./ hive.log 0 0%0.00kB/s0:00:00 rsync: write failed on /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-1868/succeeded/TestCliDriver-avro_add_column.q-orc_wide_table.q-query_with_semi.q-and-12-more/hive.log: No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(301) [receiver=3.0.6] rsync: connection unexpectedly closed (244 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [generator=3.0.6] Address 54.227.92.185 maps to ec2-54-227-92-185.compute-1.amazonaws.com, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT! receiving incremental file list ./ hive.log 0 0%0.00kB/s0:00:00 rsync: write failed on /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-1868/succeeded/TestCliDriver-avro_add_column.q-orc_wide_table.q-query_with_semi.q-and-12-more/hive.log: No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(301) [receiver=3.0.6] rsync: connection unexpectedly closed (244 bytes received so far) [generator] rsync error: error in rsync protocol
[jira] [Resolved] (HIVE-2638) Tests fail when Hive is run against Hadoop 0.23
[ https://issues.apache.org/jira/browse/HIVE-2638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan resolved HIVE-2638. Resolution: Fixed Fix Version/s: 0.14.0 Tests fail when Hive is run against Hadoop 0.23 --- Key: HIVE-2638 URL: https://issues.apache.org/jira/browse/HIVE-2638 Project: Hive Issue Type: Bug Components: Shims Reporter: Carl Steinbach Fix For: 0.14.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Propose to put JIRA traffic on separate hive list
Sorry I was unclear. Today RB traffic goes to the dev list. I'd like that mail to not go to the dev list, thus reducing traffic. On Fri, Nov 21, 2014 at 5:16 PM, Alan Gates ga...@hortonworks.com wrote: Do you mean RB creation? I wouldn't be in favor of sending all RB activity to it, since we're trying to create a low traffic list. Alan. Brock Noland br...@cloudera.com November 21, 2014 at 17:10 I think that RB activity could be put on this same list? Brock Noland br...@cloudera.com November 21, 2014 at 17:09 +1 Szehon Ho sze...@cloudera.com November 21, 2014 at 17:03 +1. Create + resolve might be a good idea too, as I imagine contributors might want to see what is changing in Hive, and not just issues created. Thanks, Szehon On Fri, Nov 21, 2014 at 1:15 PM, Prasanth Jayachandran Prasanth Jayachandran pjayachand...@hortonworks.com November 21, 2014 at 13:15 +1 - Prasanth Alan Gates ga...@hortonworks.com November 21, 2014 at 12:59 Poke, I got some scattered +1s but I'd like to get feedback from a higher percentage of committers before filing the infra ticket to do this. Or maybe a better way to put it is, is anyone opposed? Lefty, to answer your question, we could include resolutions as well. But that doubles the traffic going to the dev list. I get why you're concerned with it, since resolutions are more important to you than creations. I'm happy to help you set up a filter on the list with the non-creation JIRA traffic so you can just see the resolutions. Alan. -- Sent with Postbox http://www.getpostbox.com CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-8938) Compiler should save the transform URI as input entity
[ https://issues.apache.org/jira/browse/HIVE-8938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1460#comment-1460 ] Hive QA commented on HIVE-8938: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12683022/HIVE-8938.2.patch {color:red}ERROR:{color} -1 due to 120 failed/errored test(s), 6517 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.initializationError org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_archive_multi org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_binary_output_format org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynpart_sort_optimization_acid org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_escape_orderby1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_global_limit org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_bigdata org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_empty org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input14_limit org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input17 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input18 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input20 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input33 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input34 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input35 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input36 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input37 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input38 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_inputddl1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_into1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_into2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_into3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_into4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_into5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_into6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join19 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_load_binary_data org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_newline org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nonreserved_keywords_input37 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nonreserved_keywords_insert_into1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup4_multi_distinct org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullscript org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_createas1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_vectorization_ppd org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partcols1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_timestamp2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppr_pushdown3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_protectmode2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_query_properties org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_query_with_semi org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_rand_partitionpruner2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_rcfile_bigdata org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_rcfile_createas1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_rcfile_merge1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_rcfile_merge2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_rcfile_merge3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_rcfile_merge4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_script_env_var1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_script_env_var2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_script_pipe org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_shutdown org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoinopt19 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_12
[jira] [Commented] (HIVE-7073) Implement Binary in ParquetSerDe
[ https://issues.apache.org/jira/browse/HIVE-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1468#comment-1468 ] Hive QA commented on HIVE-7073: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12682840/HIVE-7073.3.patch {color:red}ERROR:{color} -1 due to 2 failed/errored test(s), tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan org.apache.hive.hcatalog.streaming.TestStreaming.testInterleavedTransactionBatchCommits {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1870/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1870/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1870/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12682840 - PreCommit-HIVE-TRUNK-Build Implement Binary in ParquetSerDe Key: HIVE-7073 URL: https://issues.apache.org/jira/browse/HIVE-7073 Project: Hive Issue Type: Sub-task Reporter: David Chen Assignee: Ferdinand Xu Attachments: HIVE-7073.1.patch, HIVE-7073.2.patch, HIVE-7073.3.patch, HIVE-7073.patch The ParquetSerDe currently does not support the BINARY data type. This ticket is to implement the BINARY data type. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Review Request 28372: HIVE-8950: Add support in ParquetHiveSerde to create table schema from a parquet file
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28372/ --- Review request for hive. Bugs: HIVE-8950 https://issues.apache.org/jira/browse/HIVE-8950 Repository: hive-git Description --- HIVE-8950: Add support in ParquetHiveSerde to create table schema from a parquet file Diffs - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java fafd78e63e9b41c9fdb0e017b567dc719d151784 data/files/data.parq PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ColInfoFromParquetFile.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java 4effe736fcf9d3715f03eed9885c299a7aa040dd ql/src/test/queries/clientpositive/parquet_create_gen_schema.q PRE-CREATION ql/src/test/queries/clientpositive/parquet_create_gen_schema1.q PRE-CREATION ql/src/test/results/clientpositive/parquet_create_gen_schema.q.out PRE-CREATION ql/src/test/results/clientpositive/parquet_create_gen_schema1.q.out PRE-CREATION Diff: https://reviews.apache.org/r/28372/diff/ Testing --- Tested by adding appropriate qTests. Thanks, Ashish Singh
[jira] [Updated] (HIVE-8950) Add support in ParquetHiveSerde to create table schema from a parquet file
[ https://issues.apache.org/jira/browse/HIVE-8950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Kumar Singh updated HIVE-8950: - Attachment: HIVE-8950.patch RB: https://reviews.apache.org/r/28372/ Add support in ParquetHiveSerde to create table schema from a parquet file -- Key: HIVE-8950 URL: https://issues.apache.org/jira/browse/HIVE-8950 Project: Hive Issue Type: Improvement Reporter: Ashish Kumar Singh Assignee: Ashish Kumar Singh Attachments: HIVE-8950.patch PARQUET-76 and PARQUET-47 ask for creating parquet backed tables without having to specify the column names and types. As, parquet files store schema in their footer, it is possible to generate hive schema from parquet file's metadata. This will improve usability of parquet backed tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8950) Add support in ParquetHiveSerde to create table schema from a parquet file
[ https://issues.apache.org/jira/browse/HIVE-8950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Kumar Singh updated HIVE-8950: - Status: Patch Available (was: Open) Add support in ParquetHiveSerde to create table schema from a parquet file -- Key: HIVE-8950 URL: https://issues.apache.org/jira/browse/HIVE-8950 Project: Hive Issue Type: Improvement Reporter: Ashish Kumar Singh Assignee: Ashish Kumar Singh Attachments: HIVE-8950.patch PARQUET-76 and PARQUET-47 ask for creating parquet backed tables without having to specify the column names and types. As, parquet files store schema in their footer, it is possible to generate hive schema from parquet file's metadata. This will improve usability of parquet backed tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8926) Projections that only swap input columns are identified incorrectly as identity projections
[ https://issues.apache.org/jira/browse/HIVE-8926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1493#comment-1493 ] Hive QA commented on HIVE-8926: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12682986/HIVE-8926.01.patch {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1871/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1871/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1871/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12682986 - PreCommit-HIVE-TRUNK-Build Projections that only swap input columns are identified incorrectly as identity projections --- Key: HIVE-8926 URL: https://issues.apache.org/jira/browse/HIVE-8926 Project: Hive Issue Type: Bug Components: Logical Optimizer Affects Versions: 0.15.0 Reporter: Jesús Camacho Rodríguez Assignee: Jesús Camacho Rodríguez Fix For: 0.15.0 Attachments: HIVE-8926.01.patch, HIVE-8926.patch Projection operators that only swap the input columns in the tuples are identified as identity projections, and thus incorrectly deleted from the plan by the _identity project removal_ optimization. This bug was introduced in HIVE-8435. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8920) SplitSparkWorkResolver doesn't work with UnionWork [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-8920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1497#comment-1497 ] Xuefu Zhang commented on HIVE-8920: --- It appears to me that the plan illustrated in the diagram is not quite right for the query. The source of the multi-insert is a result of UNION, but how come R5 only takes only part of the union input? To me, R5 should be connected to U3 as well. SplitSparkWorkResolver doesn't work with UnionWork [Spark Branch] - Key: HIVE-8920 URL: https://issues.apache.org/jira/browse/HIVE-8920 Project: Hive Issue Type: Sub-task Components: Spark Affects Versions: spark-branch Reporter: Chao The following query will not work: {code} from (select * from table0 union all select * from table1) s insert overwrite table table3 select s.x, count(1) group by s.x insert overwrite table table4 select s.y, count(1) group by s.y; {code} Currently, the plan for this query, before SplitSparkWorkResolver, looks like below: {noformat} M1M2 \ / \ U3 R5 | R4 {noformat} In {{SplitSparkWorkResolver#splitBaseWork}}, it assumes that the {{childWork}} is a ReduceWork, but for this case, you can see that for M2 the childWork could be UnionWork U3. Thus, the code will fail. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8834) enable job progress monitoring of Remote Spark Context [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-8834: - Attachment: HIVE-8834.4-spark.patch The new patch wraps requesting job/stage info into a {{Job}} so we can use the existing asynchronous API in {{SparkClient}}. I was trying to use {{KryoSerializer}} in the {{Job}}, but it turned out {{KryoSerializer}} can't properly ser/de {{SparkJobInfo}} and {{SparkStageInfo}}. I guess this is because the two classes are immutable, without zero-arg constructor. Others may help me verify this. Therefore, I added two wrappers for {{SparkJobInfo}} and {{SparkStageInfo}}. This may seem a little ugly but we don't have to do all the refactors. Hopefully this is a cleaner solution. enable job progress monitoring of Remote Spark Context [Spark Branch] - Key: HIVE-8834 URL: https://issues.apache.org/jira/browse/HIVE-8834 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Chengxiang Li Assignee: Rui Li Labels: Spark-M3 Attachments: HIVE-8834.1-spark.patch, HIVE-8834.2-spark.patch, HIVE-8834.3-spark.patch, HIVE-8834.4-spark.patch We should enable job progress monitor in Remote Spark Context, the spark job progress info should fit into SparkJobStatus. SPARK-2321 supply new spark progress API, which should make this task easier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8707) Fix ordering differences due to Java 8 HashMap function
[ https://issues.apache.org/jira/browse/HIVE-8707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222313#comment-14222313 ] Hive QA commented on HIVE-8707: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12682900/HIVE-8707.3.patch {color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 6651 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_acid_dynamic_partition org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Json {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1872/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1872/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1872/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12682900 - PreCommit-HIVE-TRUNK-Build Fix ordering differences due to Java 8 HashMap function --- Key: HIVE-8707 URL: https://issues.apache.org/jira/browse/HIVE-8707 Project: Hive Issue Type: Sub-task Reporter: Mohit Sabharwal Assignee: Mohit Sabharwal Attachments: HIVE-8707.1.patch, HIVE-8707.2.patch, HIVE-8707.3.patch, HIVE-8707.patch Java 8 uses a different hash function for HashMap, which is leading to iteration order differences in several cases. (See [Java8|http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/java/util/HashMap.java?av=f#336] vs [Java7|http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7-b147/java/util/HashMap.java#HashMap.hash%28int%29]) For example: - In Explain output: Stages are numbered differently - In PostExecutePrinter hook, table order is different in lineage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8834) enable job progress monitoring of Remote Spark Context [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222315#comment-14222315 ] Hive QA commented on HIVE-8834: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12683203/HIVE-8834.4-spark.patch {color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 7182 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testMultiSessionMultipleUse org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testSingleSessionMultipleUse org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchEmptyCommit {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/410/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/410/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-410/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 5 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12683203 - PreCommit-HIVE-SPARK-Build enable job progress monitoring of Remote Spark Context [Spark Branch] - Key: HIVE-8834 URL: https://issues.apache.org/jira/browse/HIVE-8834 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Chengxiang Li Assignee: Rui Li Labels: Spark-M3 Attachments: HIVE-8834.1-spark.patch, HIVE-8834.2-spark.patch, HIVE-8834.3-spark.patch, HIVE-8834.4-spark.patch We should enable job progress monitor in Remote Spark Context, the spark job progress info should fit into SparkJobStatus. SPARK-2321 supply new spark progress API, which should make this task easier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Propose to put JIRA traffic on separate hive list
Ok, +1, makes sense. Alan. Brock Noland mailto:br...@cloudera.com November 22, 2014 at 15:50 Sorry I was unclear. Today RB traffic goes to the dev list. I'd like that mail to not go to the dev list, thus reducing traffic. Alan Gates mailto:ga...@hortonworks.com November 21, 2014 at 17:16 Do you mean RB creation? I wouldn't be in favor of sending all RB activity to it, since we're trying to create a low traffic list. Alan. Brock Noland mailto:br...@cloudera.com November 21, 2014 at 17:10 I think that RB activity could be put on this same list? Brock Noland mailto:br...@cloudera.com November 21, 2014 at 17:09 +1 Szehon Ho mailto:sze...@cloudera.com November 21, 2014 at 17:03 +1. Create + resolve might be a good idea too, as I imagine contributors might want to see what is changing in Hive, and not just issues created. Thanks, Szehon On Fri, Nov 21, 2014 at 1:15 PM, Prasanth Jayachandran -- Sent with Postbox http://www.getpostbox.com -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-8574) Enhance metrics gathering in Spark Client [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-8574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222320#comment-14222320 ] Xuefu Zhang commented on HIVE-8574: --- Hi [~vanzin], any update on this issue? Thanks. Enhance metrics gathering in Spark Client [Spark Branch] Key: HIVE-8574 URL: https://issues.apache.org/jira/browse/HIVE-8574 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Marcelo Vanzin Assignee: Marcelo Vanzin The current implementation of metrics gathering in the Spark client is a little hacky. First, it's awkward to use (and the implementation is also pretty ugly). Second, it will just collect metrics indefinitely, so in the long term it turns into a huge memory leak. We need a simplified interface and some mechanism for disposing of old metrics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8707) Fix ordering differences due to Java 8 HashMap function
[ https://issues.apache.org/jira/browse/HIVE-8707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222324#comment-14222324 ] Brock Noland commented on HIVE-8707: All tests pass locally for me. Fix ordering differences due to Java 8 HashMap function --- Key: HIVE-8707 URL: https://issues.apache.org/jira/browse/HIVE-8707 Project: Hive Issue Type: Sub-task Reporter: Mohit Sabharwal Assignee: Mohit Sabharwal Attachments: HIVE-8707.1.patch, HIVE-8707.2.patch, HIVE-8707.3.patch, HIVE-8707.patch Java 8 uses a different hash function for HashMap, which is leading to iteration order differences in several cases. (See [Java8|http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/java/util/HashMap.java?av=f#336] vs [Java7|http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7-b147/java/util/HashMap.java#HashMap.hash%28int%29]) For example: - In Explain output: Stages are numbered differently - In PostExecutePrinter hook, table order is different in lineage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8707) Fix ordering differences due to Java 8 HashMap function
[ https://issues.apache.org/jira/browse/HIVE-8707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-8707: --- Resolution: Fixed Fix Version/s: 0.15.0 Status: Resolved (was: Patch Available) Thank you so much Mohit!! I have committed this to trunk! Fix ordering differences due to Java 8 HashMap function --- Key: HIVE-8707 URL: https://issues.apache.org/jira/browse/HIVE-8707 Project: Hive Issue Type: Sub-task Reporter: Mohit Sabharwal Assignee: Mohit Sabharwal Fix For: 0.15.0 Attachments: HIVE-8707.1.patch, HIVE-8707.2.patch, HIVE-8707.3.patch, HIVE-8707.patch Java 8 uses a different hash function for HashMap, which is leading to iteration order differences in several cases. (See [Java8|http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/java/util/HashMap.java?av=f#336] vs [Java7|http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7-b147/java/util/HashMap.java#HashMap.hash%28int%29]) For example: - In Explain output: Stages are numbered differently - In PostExecutePrinter hook, table order is different in lineage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8926) Projections that only swap input columns are identified incorrectly as identity projections
[ https://issues.apache.org/jira/browse/HIVE-8926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-8926: --- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Jesus! Projections that only swap input columns are identified incorrectly as identity projections --- Key: HIVE-8926 URL: https://issues.apache.org/jira/browse/HIVE-8926 Project: Hive Issue Type: Bug Components: Logical Optimizer Affects Versions: 0.15.0 Reporter: Jesús Camacho Rodríguez Assignee: Jesús Camacho Rodríguez Fix For: 0.15.0 Attachments: HIVE-8926.01.patch, HIVE-8926.patch Projection operators that only swap the input columns in the tuples are identified as identity projections, and thus incorrectly deleted from the plan by the _identity project removal_ optimization. This bug was introduced in HIVE-8435. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8395) CBO: enable by default
[ https://issues.apache.org/jira/browse/HIVE-8395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222330#comment-14222330 ] Hive QA commented on HIVE-8395: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12682936/HIVE-8395.25.patch {color:red}ERROR:{color} -1 due to 134 failed/errored test(s), tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_select org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join16 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join18 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join18_multi_distinct org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join22 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join26 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join27 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join_without_localtask org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_gby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_limit org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_semijoin org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ctas_colname org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_explain_logical org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_gby_star org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_position org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_ppd org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_resolution org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1_23 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1_23 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_having org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input30 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join22 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join28 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join29 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join31 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join32 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join32_lessSize org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join33 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join34 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join35 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join40 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_alt_syntax org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_cond_pushdown_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_cond_pushdown_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_cond_pushdown_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_cond_pushdown_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_limit_pushdown org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_oneskew_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_load_dyn_part14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_mapjoin org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_subquery org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_merge4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_metadataonly1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multiMapJoin2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_notable_alias1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_notable_alias2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup2