[jira] [Updated] (HIVE-5283) Merge vectorization branch to trunk
[ https://issues.apache.org/jira/browse/HIVE-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HIVE-5283: --- Attachment: HIVE-5283.2.patch The previous patch didn't apply cleanly, here is the updated patch. Merge vectorization branch to trunk --- Key: HIVE-5283 URL: https://issues.apache.org/jira/browse/HIVE-5283 Project: Hive Issue Type: Bug Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Attachments: HIVE-5283.1.patch, HIVE-5283.2.patch The purpose of this jira is to upload vectorization patch, run tests etc. The actual work will continue under HIVE-4160 umbrella jira. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5279) Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
[
https://issues.apache.org/jira/browse/HIVE-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766088#comment-13766088
]
Navis commented on HIVE-5279:
-
I've applied the patch and confirmed the query working. But,
reflectionUtils.newInstance() uses default constructor, which means it forces
UDAFEvaluator to have it (like GenericUDAFGroupOn$VersionedFloatGroupOnEval in
this case). I'm ok to make this to be a requirement of UDAFEvaluator. Other
ideas?
Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
---
Key: HIVE-5279
URL: https://issues.apache.org/jira/browse/HIVE-5279
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Priority: Critical
Attachments: 5279.patch
We didn't forced GenericUDAFEvaluator to be Serializable. I don't know how
previous serialization mechanism solved this but, kryo complaints that it's
not Serializable and fails the query.
The log below is the example,
{noformat}
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: Class
cannot be created (missing no-arg constructor):
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
Serialization trace:
inputOI
(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFGroupOn$VersionedFloatGroupOnEval)
genericUDAFEvaluator (org.apache.hadoop.hive.ql.plan.AggregationDesc)
aggregators (org.apache.hadoop.hive.ql.plan.GroupByDesc)
conf (org.apache.hadoop.hive.ql.exec.GroupByOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at
org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:312)
at
org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:261)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:256)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:383)
at org.apache.h
{noformat}
If this cannot be fixed in somehow, some UDAFs should be modified to be run
on hive-0.13.0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5283) Merge vectorization branch to trunk
[ https://issues.apache.org/jira/browse/HIVE-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HIVE-5283: --- Status: Patch Available (was: Open) Merge vectorization branch to trunk --- Key: HIVE-5283 URL: https://issues.apache.org/jira/browse/HIVE-5283 Project: Hive Issue Type: Bug Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Attachments: HIVE-5283.1.patch, HIVE-5283.2.patch The purpose of this jira is to upload vectorization patch, run tests etc. The actual work will continue under HIVE-4160 umbrella jira. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5283) Merge vectorization branch to trunk
[ https://issues.apache.org/jira/browse/HIVE-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HIVE-5283: --- Status: Open (was: Patch Available) Merge vectorization branch to trunk --- Key: HIVE-5283 URL: https://issues.apache.org/jira/browse/HIVE-5283 Project: Hive Issue Type: Bug Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Attachments: HIVE-5283.1.patch, HIVE-5283.2.patch The purpose of this jira is to upload vectorization patch, run tests etc. The actual work will continue under HIVE-4160 umbrella jira. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4844) Add varchar data type
[ https://issues.apache.org/jira/browse/HIVE-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766095#comment-13766095 ] Ashutosh Chauhan commented on HIVE-4844: +1 Add varchar data type - Key: HIVE-4844 URL: https://issues.apache.org/jira/browse/HIVE-4844 Project: Hive Issue Type: New Feature Components: Types Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-4844.10.patch, HIVE-4844.11.patch, HIVE-4844.12.patch, HIVE-4844.13.patch, HIVE-4844.14.patch, HIVE-4844.15.patch, HIVE-4844.1.patch.hack, HIVE-4844.2.patch, HIVE-4844.3.patch, HIVE-4844.4.patch, HIVE-4844.5.patch, HIVE-4844.6.patch, HIVE-4844.7.patch, HIVE-4844.8.patch, HIVE-4844.9.patch, HIVE-4844.D12699.1.patch, screenshot.png Add new varchar data types which have support for more SQL-compliant behavior, such as SQL string comparison semantics, max length, etc. Char type will be added as another task. NO PRECOMMIT TESTS - now dependent on HIVE-5203/5204/5206 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4690) stats_partscan_1.q makes different result with different hadhoop.mr.rev
[ https://issues.apache.org/jira/browse/HIVE-4690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766112#comment-13766112 ] Navis commented on HIVE-4690: - The patch made by arcanist seemed not applied cleanly with simple patch command. stats_partscan_1.q makes different result with different hadhoop.mr.rev Key: HIVE-4690 URL: https://issues.apache.org/jira/browse/HIVE-4690 Project: Hive Issue Type: Sub-task Affects Versions: 0.11.0 Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-4690.2.patch, HIVE-4690.D11163.1.patch stats_partscan_1.q uses mapred.min/max.split.size and logs number of files, which can be different with different hadoop.mr.rev. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5279) Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
[
https://issues.apache.org/jira/browse/HIVE-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766120#comment-13766120
]
Ashutosh Chauhan commented on HIVE-5279:
I think Kryo has other mechanisms of serialization which doesn't have that
requirement. I haven't explored Kryo fully yet. Probably, we should do that in
a follow-up. I think for now, we can get this one in and have this requirement
in the meanwhile.
[~navis] Since you have that custom udf, can you write a test case for this and
submit to let Hive QA run on the patch.
Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
---
Key: HIVE-5279
URL: https://issues.apache.org/jira/browse/HIVE-5279
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Priority: Critical
Attachments: 5279.patch
We didn't forced GenericUDAFEvaluator to be Serializable. I don't know how
previous serialization mechanism solved this but, kryo complaints that it's
not Serializable and fails the query.
The log below is the example,
{noformat}
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: Class
cannot be created (missing no-arg constructor):
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
Serialization trace:
inputOI
(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFGroupOn$VersionedFloatGroupOnEval)
genericUDAFEvaluator (org.apache.hadoop.hive.ql.plan.AggregationDesc)
aggregators (org.apache.hadoop.hive.ql.plan.GroupByDesc)
conf (org.apache.hadoop.hive.ql.exec.GroupByOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at
org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:312)
at
org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:261)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:256)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:383)
at org.apache.h
{noformat}
If this cannot be fixed in somehow, some UDAFs should be modified to be run
on hive-0.13.0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4388) HBase tests fail against Hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-4388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766125#comment-13766125 ] Brock Noland commented on HIVE-4388: Also I noticed with 0.96 we'll need to modify a test: TestHBaseNegativeCliDriver.cascade_dbdrop Change dfs -ls ../build/ql/tmp/hbase/hbase_table_0; to dfs -ls ../build/ql/tmp/hbase/data/default/hbase_table_0; HBase tests fail against Hadoop 2 - Key: HIVE-4388 URL: https://issues.apache.org/jira/browse/HIVE-4388 Project: Hive Issue Type: Bug Components: HBase Handler Reporter: Gunther Hagleitner Assignee: Brock Noland Attachments: HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388-wip.txt Currently we're building by default against 0.92. When you run against hadoop 2 (-Dhadoop.mr.rev=23) builds fail because of: HBASE-5963. HIVE-3861 upgrades the version of hbase used. This will get you past the problem in HBASE-5963 (which was fixed in 0.94.1) but fails with: HBASE-6396. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5079) Make Hive compile under Windows
[
https://issues.apache.org/jira/browse/HIVE-5079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766136#comment-13766136
]
Ashutosh Chauhan commented on HIVE-5079:
+1
Make Hive compile under Windows
---
Key: HIVE-5079
URL: https://issues.apache.org/jira/browse/HIVE-5079
Project: Hive
Issue Type: Bug
Components: Build Infrastructure
Reporter: Daniel Dai
Assignee: Daniel Dai
Fix For: 0.12.0
Attachments: HIVE-5079-1.patch, HIVE-5079-2.patch
Hive compilation failed under Windows. Error message:
{code}
compile:
[echo] Project: common
[exec] D:\Program Files (x86)\GnuWin32\bin\xargs.exe: md5sum: No such
file
or directory
[exec] md5sum:
../serde/src/java/org/apache/hadoop/hive/serde2/io/Timesta:
No such file or directory
[javac] Compiling 25 source files to
D:\Users\Administrator\hive\build\common\classes
[javac]
D:\Users\Administrator\hive\common\src\gen\org\apache\hive\common\package-info.java:4:
unclosed string literal
[javac] @HiveVersionAnnotation(version=0.12.0-SNAPSHOT,
revision=80eadd8fa2af5eeba61f921318ab8b2c19980ab3, branch=trunk
[javac]
^
[javac]
D:\Users\Administrator\hive\common\src\gen\org\apache\hive\common\package-info.java:5:
unclosed string literal
[javac] ,
[javac] ^
[javac]
D:\Users\Administrator\hive\common\src\gen\org\apache\hive\common\package-info.java:6:
class, interface, or enum expected
[javac] user=Administrator
[javac] ^
[javac]
D:\Users\Administrator\hive\common\src\gen\org\apache\hive\common\package-info.java:6:
unclosed string literal
[javac] user=Administrator
[javac] ^
[javac]
D:\Users\Administrator\hive\common\src\gen\org\apache\hive\common\package-info.java:10:
unclosed string literal
[javac] ,
[javac] ^
[javac]
D:\Users\Administrator\hive\common\src\gen\org\apache\hive\common\package-info.java:11:
unclosed string literal
[javac]
srcChecksum=aadceb95c37a1704aaf19501f46f6e84
[javac] ^
[javac]
D:\Users\Administrator\hive\common\src\gen\org\apache\hive\common\package-info.java:12:
unclosed string literal
[javac] )
[javac] ^
[javac] 7 errors
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5288) Perflogger should log under single class
[ https://issues.apache.org/jira/browse/HIVE-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-5288: --- Attachment: HIVE-5288.patch Phabricator is down. Anyway it's a simple patch. Perflogger should log under single class Key: HIVE-5288 URL: https://issues.apache.org/jira/browse/HIVE-5288 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor Attachments: HIVE-5288.patch Perflogger should log under single class, so that it could be turned on without mass logging spew. Right now the log is passed to it externally; this could be preserved by passing in a string to be logged as part of the message. Anyway most of the time it's called from Driver and Utilities, which is a pretty useless class name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4844) Add varchar data type
[ https://issues.apache.org/jira/browse/HIVE-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766149#comment-13766149 ] Phabricator commented on HIVE-4844: --- ashutoshc has commented on the revision HIVE-4844 [jira] Add varchar data type. Sorry.. missed while previewing earlier. INLINE COMMENTS serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveVarcharObjectInspector.java:34 You need to add no-arg constructor for Kryo to correctly serialize this. serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveVarcharObjectInspector.java:32 Need to add no-arg constructor for Kryo to work correctly. serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveVarcharObjectInspector.java:33 no-arg constructor. serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveVarcharObjectInspector.java:35 no-arg ctor. REVISION DETAIL https://reviews.facebook.net/D12699 To: JIRA, jdere Cc: ashutoshc Add varchar data type - Key: HIVE-4844 URL: https://issues.apache.org/jira/browse/HIVE-4844 Project: Hive Issue Type: New Feature Components: Types Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-4844.10.patch, HIVE-4844.11.patch, HIVE-4844.12.patch, HIVE-4844.13.patch, HIVE-4844.14.patch, HIVE-4844.15.patch, HIVE-4844.1.patch.hack, HIVE-4844.2.patch, HIVE-4844.3.patch, HIVE-4844.4.patch, HIVE-4844.5.patch, HIVE-4844.6.patch, HIVE-4844.7.patch, HIVE-4844.8.patch, HIVE-4844.9.patch, HIVE-4844.D12699.1.patch, screenshot.png Add new varchar data types which have support for more SQL-compliant behavior, such as SQL string comparison semantics, max length, etc. Char type will be added as another task. NO PRECOMMIT TESTS - now dependent on HIVE-5203/5204/5206 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5206) Support parameterized primitive types
[ https://issues.apache.org/jira/browse/HIVE-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-5206: - Attachment: HIVE-5206.4.patch patch 4 - rebasing with trunk Support parameterized primitive types - Key: HIVE-5206 URL: https://issues.apache.org/jira/browse/HIVE-5206 Project: Hive Issue Type: Improvement Components: Types Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-5206.1.patch, HIVE-5206.2.patch, HIVE-5206.3.patch, HIVE-5206.4.patch, HIVE-5206.D12693.1.patch Support for parameterized types is needed for char/varchar/decimal support. This adds a type parameters value to the PrimitiveTypeEntry/PrimitiveTypeInfo/PrimitiveObjectInspector objects. NO PRECOMMIT TESTS - dependent on HIVE-5203/HIVE-5204 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-5221) Issue in colun type with data type as BINARY
[ https://issues.apache.org/jira/browse/HIVE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Kamrul Islam reassigned HIVE-5221: --- Assignee: Mohammad Kamrul Islam Issue in colun type with data type as BINARY Key: HIVE-5221 URL: https://issues.apache.org/jira/browse/HIVE-5221 Project: Hive Issue Type: Bug Reporter: Arun Vasu Assignee: Mohammad Kamrul Islam Priority: Critical Hi, I am using Hive 10. When I create an external table with column type as Binary, the query result on the table is showing some junk values for the column with binary datatype. Please find below the query I have used to create the table: CREATE EXTERNAL TABLE BOOL1(NB BOOLEAN,email STRING, bitfld BINARY) ROW FORMAT DELIMITED FIELDS TERMINATED BY '^' LINES TERMINATED BY '\n' STORED AS TEXTFILE LOCATION '/user/hivetables/testbinary'; The query I have used is : select * from bool1 The sample data in the hdfs file is: 0^a...@abc.com^001 1^a...@abc.com^010 ^a...@abc.com^011 ^a...@abc.com^100 t^a...@abc.com^101 f^a...@abc.com^110 true^a...@abc.com^111 false^a...@abc.com^001 123^^01100010 12344^^0111 Please share your inputs if it is possible. Thanks, Arun -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5127) Upgrade xerces and xalan for WebHCat
[ https://issues.apache.org/jira/browse/HIVE-5127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766163#comment-13766163 ] Thejas M Nair commented on HIVE-5127: - [~ekoifman] Did the patch accidentally get created as a reverse patch ? Upgrade xerces and xalan for WebHCat Key: HIVE-5127 URL: https://issues.apache.org/jira/browse/HIVE-5127 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Fix For: 0.12.0 Attachments: HIVE-5127.patch Currently webhcat log files are full of exceptions like this, which obscures the real output and may cause perf issues. Upgrading to more recent versions of xerces/xalan fixes this. Add the following to hive/hcatalog/webhcat/svr/pom.xml dependency groupIdxerces/groupId artifactIdxercesImpl/artifactId version2.9.1/version /dependency dependency groupIdxalan/groupId artifactIdxalan/artifactId version2.7.1/version /dependency 13/08/20 16:54:04 ERROR conf.Configuration: Failed to set setXIncludeAware(true) for parser org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@48dbb335:java.lang.UnsupportedOperationException: This parser does not support specification null version null java.lang.UnsupportedOperationException: This parser does not support specification null version null at javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590) at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1892) at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1861) at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1778) at org.apache.hadoop.conf.Configuration.get(Configuration.java:870) at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:171) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:305) at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:288) at org.apache.hadoop.util.GenericOptionsParser.validateFiles(GenericOptionsParser.java:383) at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:281) at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:422) at org.apache.hadoop.util.GenericOptionsParser.init(GenericOptionsParser.java:168) at org.apache.hadoop.util.GenericOptionsParser.init(GenericOptionsParser.java:151) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.hcatalog.templeton.LauncherDelegator$1.run(LauncherDelegator.java:99) at org.apache.hcatalog.templeton.LauncherDelegator$1.run(LauncherDelegator.java:95) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1441) at org.apache.hcatalog.templeton.LauncherDelegator.queueAsUser(LauncherDelegator.java:95) at org.apache.hcatalog.templeton.LauncherDelegator.enqueueController(LauncherDelegator.java:77) at org.apache.hcatalog.templeton.JarDelegator.run(JarDelegator.java:52) at org.apache.hcatalog.templeton.StreamingDelegator.run(StreamingDelegator.java:53) at org.apache.hcatalog.templeton.Server.mapReduceStreaming(Server.java:596) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) at
[jira] [Commented] (HIVE-5241) Default log4j log level for WebHCat should be INFO not DEBUG
[ https://issues.apache.org/jira/browse/HIVE-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766165#comment-13766165 ] Thejas M Nair commented on HIVE-5241: - +1 . I will commit after running webhcat tests manually. Default log4j log level for WebHCat should be INFO not DEBUG Key: HIVE-5241 URL: https://issues.apache.org/jira/browse/HIVE-5241 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.11.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Fix For: 0.12.0 Attachments: HIVE-5241.patch webhcat-logj4.properties -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5209) JDBC support for varchar
[ https://issues.apache.org/jira/browse/HIVE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-5209: - Attachment: HIVE-5209.2.patch new patch HIVE-5209.2.patch, rebasing against dependent patches. JDBC support for varchar Key: HIVE-5209 URL: https://issues.apache.org/jira/browse/HIVE-5209 Project: Hive Issue Type: Improvement Components: JDBC, Types Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-5209.1.patch, HIVE-5209.2.patch, HIVE-5209.D12705.1.patch Support returning varchar length in result set metadata NO PRECOMMIT TESTS - dependent on HIVE-4844 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4492) Revert HIVE-4322
[ https://issues.apache.org/jira/browse/HIVE-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766167#comment-13766167 ] Thejas M Nair commented on HIVE-4492: - This looks like an api backward compatibility fix that we should include in 0.12 . [~ashutoshc] Do you agree ? Revert HIVE-4322 Key: HIVE-4492 URL: https://issues.apache.org/jira/browse/HIVE-4492 Project: Hive Issue Type: Bug Components: Metastore, Thrift API Reporter: Samuel Yuan Assignee: Samuel Yuan Fix For: 0.13.0 Attachments: HIVE-4492.1.patch.txt, HIVE-4492.patch See HIVE-4432 and HIVE-4433. It's possible to work around these issues but a better solution is probably to roll back the fix and change the API to use a primitive type as the map key (in a backwards-compatible manner). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4583) Make Hive compile and run with JDK7
[ https://issues.apache.org/jira/browse/HIVE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765618#comment-13765618 ] Xuefu Zhang commented on HIVE-4583: --- Given that the last dependent JIRA HIVE-5263 is closed, I think we are ready to close this JIRA as well. I ran a build based on trunk with JDK7, and all Hive tests passed. There were a few hcat test case failures, but I believe they are not related to JDK7, but are transient failures due to recent hcat rebase. Please raise any concern you might have. Otherwise, the JIRA will be closed as resolved in a day or two. Make Hive compile and run with JDK7 --- Key: HIVE-4583 URL: https://issues.apache.org/jira/browse/HIVE-4583 Project: Hive Issue Type: Task Affects Versions: 0.11.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang This is an umbrella ticket to cover all issues related to supporting JDK 7 in HIVE. Many such issues are expected. Of course, JDK 6 needs to continue to work. The major obstacles on the way supporting JDK7 are: 1. JDBC component needs to be upgraded because of JDBC interface changes in JDK7. 2. DataNucleus needs to be upgraded because the current version doesn't support JDK7. 3. Many test failures needs to be fixed. The majority of the failures are caused by JDK7 subtle behaviour change. 4. Build needs to be changed to accommodate JDK7 compiler. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5161) Additional SerDe support for varchar type
[ https://issues.apache.org/jira/browse/HIVE-5161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-5161: - Attachment: HIVE-5161.3.patch uploading patch v3: - rebase against trunk, dependent patches - regenerate OrcProto using protobuf-2.5 Additional SerDe support for varchar type - Key: HIVE-5161 URL: https://issues.apache.org/jira/browse/HIVE-5161 Project: Hive Issue Type: Bug Components: Serializers/Deserializers, Types Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-5161.1.patch, HIVE-5161.2.patch, HIVE-5161.3.patch Breaking out support for varchar for the various SerDes as an additional task. NO PRECOMMIT TESTS - can't run tests until HIVE-4844 is committed -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5161) Additional SerDe support for varchar type
[ https://issues.apache.org/jira/browse/HIVE-5161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766172#comment-13766172 ] Jason Dere commented on HIVE-5161: -- posted review at https://reviews.facebook.net/D12897 Additional SerDe support for varchar type - Key: HIVE-5161 URL: https://issues.apache.org/jira/browse/HIVE-5161 Project: Hive Issue Type: Bug Components: Serializers/Deserializers, Types Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-5161.1.patch, HIVE-5161.2.patch, HIVE-5161.3.patch Breaking out support for varchar for the various SerDes as an additional task. NO PRECOMMIT TESTS - can't run tests until HIVE-4844 is committed -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5281) Constructor for DelegationTokenInformation changed in HDFS-4680 which we use in HiveDelegationTokenSupport.
[ https://issues.apache.org/jira/browse/HIVE-5281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765604#comment-13765604 ] Brock Noland commented on HIVE-5281: [~thejas] after HDFS-4680 delegation tokens will have more information. Additionally as of 0.12 we will be storing them in the metastore (HIVE-3255). HIVE-2467 in 0.8 put them in ZK but I am sure that is less used than HIVE-3255 will be. My concern is that we have no way at present to version these things but we know more fields will be coming later. I feel we should add versioning in the 0.12 release, before we start storing them in the db, since we know they will need to be versioned in the future. Constructor for DelegationTokenInformation changed in HDFS-4680 which we use in HiveDelegationTokenSupport. --- Key: HIVE-5281 URL: https://issues.apache.org/jira/browse/HIVE-5281 Project: Hive Issue Type: Sub-task Reporter: Brock Noland Assignee: Brock Noland -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5036) [WebHCat] Add cmd script for WebHCat
[ https://issues.apache.org/jira/browse/HIVE-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766176#comment-13766176 ] Thejas M Nair commented on HIVE-5036: - +1 [WebHCat] Add cmd script for WebHCat Key: HIVE-5036 URL: https://issues.apache.org/jira/browse/HIVE-5036 Project: Hive Issue Type: Sub-task Components: HCatalog Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.12.0 Attachments: HIVE-5036-1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4910) Hadoop 2 archives broken
[ https://issues.apache.org/jira/browse/HIVE-4910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766177#comment-13766177 ] Vikram Dixit K commented on HIVE-4910: -- Hi Brock, I had a conversation with the HDFS folks some time back and it seems like there is a difference in the way mapreduce forms the URI vs how we form it. Mapreduce seems to have been able to form the URI correctly (with the ':' et. al.). Ideally, we need to use/replicate what mapreduce does. We could go with this fix for now but we may need to revisit this. Thanks Vikram. Hadoop 2 archives broken Key: HIVE-4910 URL: https://issues.apache.org/jira/browse/HIVE-4910 Project: Hive Issue Type: Bug Components: Query Processor, Tests Affects Versions: 0.11.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Priority: Minor Fix For: 0.11.1 Attachments: HIVE-4910.patch, HIVE-4910.patch Hadoop 2 archive tests are broken. The issue stems from the fact that har uri construction does not really have a port in the URI when unit tests are run. This means that an invalid uri is constructed resulting in failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5264) SQL generated by MetaStoreDirectSql.java not compliant with Postgres.
[ https://issues.apache.org/jira/browse/HIVE-5264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-5264: --- Attachment: HIVE-5264.patch preliminary patch, let me run the tests against different stores and then submit to Hive QA, there may be some bugs SQL generated by MetaStoreDirectSql.java not compliant with Postgres. - Key: HIVE-5264 URL: https://issues.apache.org/jira/browse/HIVE-5264 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.12.0 Environment: Ubuntu 12.04 PostgreSQL 9.1.8 Reporter: Alexander Behm Assignee: Sergey Shelukhin Attachments: HIVE-5264.patch Some operations against the Hive Metastore seem broken against Postgres. For example, when using HiveMetastoreClient.listPartitions() the Postgres logs show queries such as: 2013-09-09 19:10:01 PDT STATEMENT: select PARTITIONS.PART_ID from PARTITIONS inner join TBLS on PARTITIONS.TBL_ID = TBLS.TBL_ID inner join DBS on TBLS.DB_ID = DBS.DB_ID where TBLS.TBL_NAME = $1 and DBS.NAME = $2 order by PART_NAME asc with a somewhat cryptic (but correct) error: ERROR: relation partitions does not exist at character 32 Postgres identifiers are somewhat unusual. Unquoted identifiers are interpreted as lower case (there is no Postgres option to change this). Since the Metastore table schema uses upper case table names, the correct SQL requires escaped identifiers to those tables, i.e., select PARTITIONS.PART_ID from PARTITIONS... Hive sets metastore.try.direct.sql=true by default, so the above SQL is generated by hive/metastore/MetaStoreDirectSql.java, i.e., this is not a Datanucleus problem. When I set metastore.try.direct.sql=false, then the Metastore backed by Postgres works. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5035) [WebHCat] Hardening parameters for Windows
[ https://issues.apache.org/jira/browse/HIVE-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5035: Description: everything pass to pig/hive/hadoop command line will be quoted. That include: mapreducejar: libjars arg define mapreducestream: cmdenv define arg pig arg execute hive arg define execute NO PRECOMMIT TESTS was: everything pass to pig/hive/hadoop command line will be quoted. That include: mapreducejar: libjars arg define mapreducestream: cmdenv define arg pig arg execute hive arg define execute [WebHCat] Hardening parameters for Windows -- Key: HIVE-5035 URL: https://issues.apache.org/jira/browse/HIVE-5035 Project: Hive Issue Type: Sub-task Components: HCatalog Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.12.0 Attachments: HIVE-5035-1.patch everything pass to pig/hive/hadoop command line will be quoted. That include: mapreducejar: libjars arg define mapreducestream: cmdenv define arg pig arg execute hive arg define execute NO PRECOMMIT TESTS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5036) [WebHCat] Add cmd script for WebHCat
[ https://issues.apache.org/jira/browse/HIVE-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5036: Description: NO PRECOMMIT TESTS [WebHCat] Add cmd script for WebHCat Key: HIVE-5036 URL: https://issues.apache.org/jira/browse/HIVE-5036 Project: Hive Issue Type: Sub-task Components: HCatalog Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.12.0 Attachments: HIVE-5036-1.patch NO PRECOMMIT TESTS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5035) [WebHCat] Hardening parameters for Windows
[ https://issues.apache.org/jira/browse/HIVE-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766180#comment-13766180 ] Thejas M Nair commented on HIVE-5035: - +1 Will run the test manually as its webhcat-only change (running all hive tests is going to be an overkill, I think we should look at enabling hcat-only pre-commit tests). [WebHCat] Hardening parameters for Windows -- Key: HIVE-5035 URL: https://issues.apache.org/jira/browse/HIVE-5035 Project: Hive Issue Type: Sub-task Components: HCatalog Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.12.0 Attachments: HIVE-5035-1.patch everything pass to pig/hive/hadoop command line will be quoted. That include: mapreducejar: libjars arg define mapreducestream: cmdenv define arg pig arg execute hive arg define execute NO PRECOMMIT TESTS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5036) [WebHCat] Add cmd script for WebHCat
[ https://issues.apache.org/jira/browse/HIVE-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766182#comment-13766182 ] Thejas M Nair commented on HIVE-5036: - Will run the test manually as its webhcat-only change (running all hive tests is going to be an overkill, I think we should look at enabling hcat-only pre-commit tests). [WebHCat] Add cmd script for WebHCat Key: HIVE-5036 URL: https://issues.apache.org/jira/browse/HIVE-5036 Project: Hive Issue Type: Sub-task Components: HCatalog Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.12.0 Attachments: HIVE-5036-1.patch NO PRECOMMIT TESTS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5078) [WebHCat] Fix e2e tests on Windows plus test cases for new features
[ https://issues.apache.org/jira/browse/HIVE-5078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766186#comment-13766186 ] Thejas M Nair commented on HIVE-5078: - [~daijy] Can you please address Eugene's comment ? [WebHCat] Fix e2e tests on Windows plus test cases for new features --- Key: HIVE-5078 URL: https://issues.apache.org/jira/browse/HIVE-5078 Project: Hive Issue Type: Improvement Components: HCatalog Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.12.0 Attachments: HIVE-5078-1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4844) Add varchar data type
[ https://issues.apache.org/jira/browse/HIVE-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-4844: - Attachment: HIVE-4844.16.patch Apparently hadn't completely removed java files for GenericUDFConcat/Upper/Lower - they were still in the patch as empty files. Adding patch v16 which removes those empty files. Add varchar data type - Key: HIVE-4844 URL: https://issues.apache.org/jira/browse/HIVE-4844 Project: Hive Issue Type: New Feature Components: Types Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-4844.10.patch, HIVE-4844.11.patch, HIVE-4844.12.patch, HIVE-4844.13.patch, HIVE-4844.14.patch, HIVE-4844.15.patch, HIVE-4844.16.patch, HIVE-4844.1.patch.hack, HIVE-4844.2.patch, HIVE-4844.3.patch, HIVE-4844.4.patch, HIVE-4844.5.patch, HIVE-4844.6.patch, HIVE-4844.7.patch, HIVE-4844.8.patch, HIVE-4844.9.patch, HIVE-4844.D12699.1.patch, screenshot.png Add new varchar data types which have support for more SQL-compliant behavior, such as SQL string comparison semantics, max length, etc. Char type will be added as another task. NO PRECOMMIT TESTS - now dependent on HIVE-5203/5204/5206 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4444) [HCatalog] WebHCat Hive should support equivalent parameters as Pig
[ https://issues.apache.org/jira/browse/HIVE-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-: Description: Currently there is no files and args parameter in Hive. We shall add them to make them similar to Pig. NO PRECOMMIT TESTS was:Currently there is no files and args parameter in Hive. We shall add them to make them similar to Pig. [HCatalog] WebHCat Hive should support equivalent parameters as Pig Key: HIVE- URL: https://issues.apache.org/jira/browse/HIVE- Project: Hive Issue Type: Improvement Components: HCatalog Reporter: Daniel Dai Attachments: HIVE--1.patch, HIVE--2.patch Currently there is no files and args parameter in Hive. We shall add them to make them similar to Pig. NO PRECOMMIT TESTS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4444) [HCatalog] WebHCat Hive should support equivalent parameters as Pig
[ https://issues.apache.org/jira/browse/HIVE-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766189#comment-13766189 ] Thejas M Nair commented on HIVE-: - +1 I will run the hcat tests manually. [HCatalog] WebHCat Hive should support equivalent parameters as Pig Key: HIVE- URL: https://issues.apache.org/jira/browse/HIVE- Project: Hive Issue Type: Improvement Components: HCatalog Reporter: Daniel Dai Attachments: HIVE--1.patch, HIVE--2.patch Currently there is no files and args parameter in Hive. We shall add them to make them similar to Pig. NO PRECOMMIT TESTS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4443) [HCatalog] Have an option for GET queue to return all job information in single call
[ https://issues.apache.org/jira/browse/HIVE-4443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-4443: Description: Currently do display a summary of all jobs, one has to call GET queue to retrieve all the jobids and then call GET queue/:jobid for each job. It would be nice to do this in a single call. I would suggest: * GET queue - mark deprecate * GET queue/jobID - mark deprecate * DELETE queue/jobID - mark deprecate * GET jobs - return the list of JSON objects jobid but no detailed info * GET jobs/fields=* - return the list of JSON objects containing detailed Job info * GET jobs/jobID - return the single JSON object containing the detailed Job info for the job with the given ID (equivalent to GET queue/jobID) * DELETE jobs/jobID - equivalent to DELETE queue/jobID NO PRECOMMIT TESTS was: Currently do display a summary of all jobs, one has to call GET queue to retrieve all the jobids and then call GET queue/:jobid for each job. It would be nice to do this in a single call. I would suggest: * GET queue - mark deprecate * GET queue/jobID - mark deprecate * DELETE queue/jobID - mark deprecate * GET jobs - return the list of JSON objects jobid but no detailed info * GET jobs/fields=* - return the list of JSON objects containing detailed Job info * GET jobs/jobID - return the single JSON object containing the detailed Job info for the job with the given ID (equivalent to GET queue/jobID) * DELETE jobs/jobID - equivalent to DELETE queue/jobID [HCatalog] Have an option for GET queue to return all job information in single call - Key: HIVE-4443 URL: https://issues.apache.org/jira/browse/HIVE-4443 Project: Hive Issue Type: Improvement Components: HCatalog Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.12.0 Attachments: HIVE-4443-1.patch, HIVE-4443-2.patch, HIVE-4443-3.patch Currently do display a summary of all jobs, one has to call GET queue to retrieve all the jobids and then call GET queue/:jobid for each job. It would be nice to do this in a single call. I would suggest: * GET queue - mark deprecate * GET queue/jobID - mark deprecate * DELETE queue/jobID - mark deprecate * GET jobs - return the list of JSON objects jobid but no detailed info * GET jobs/fields=* - return the list of JSON objects containing detailed Job info * GET jobs/jobID - return the single JSON object containing the detailed Job info for the job with the given ID (equivalent to GET queue/jobID) * DELETE jobs/jobID - equivalent to DELETE queue/jobID NO PRECOMMIT TESTS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4443) [HCatalog] Have an option for GET queue to return all job information in single call
[ https://issues.apache.org/jira/browse/HIVE-4443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766192#comment-13766192 ] Thejas M Nair commented on HIVE-4443: - +1 . [~daijy] Is there a follow up jira with documentation update ? If not can you please create one (or link it with this one?) [HCatalog] Have an option for GET queue to return all job information in single call - Key: HIVE-4443 URL: https://issues.apache.org/jira/browse/HIVE-4443 Project: Hive Issue Type: Improvement Components: HCatalog Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.12.0 Attachments: HIVE-4443-1.patch, HIVE-4443-2.patch, HIVE-4443-3.patch Currently do display a summary of all jobs, one has to call GET queue to retrieve all the jobids and then call GET queue/:jobid for each job. It would be nice to do this in a single call. I would suggest: * GET queue - mark deprecate * GET queue/jobID - mark deprecate * DELETE queue/jobID - mark deprecate * GET jobs - return the list of JSON objects jobid but no detailed info * GET jobs/fields=* - return the list of JSON objects containing detailed Job info * GET jobs/jobID - return the single JSON object containing the detailed Job info for the job with the given ID (equivalent to GET queue/jobID) * DELETE jobs/jobID - equivalent to DELETE queue/jobID NO PRECOMMIT TESTS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4844) Add varchar data type
[ https://issues.apache.org/jira/browse/HIVE-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765858#comment-13765858 ] Ashutosh Chauhan commented on HIVE-4844: [~jdere] Left some comments on phabricator. Add varchar data type - Key: HIVE-4844 URL: https://issues.apache.org/jira/browse/HIVE-4844 Project: Hive Issue Type: New Feature Components: Types Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-4844.10.patch, HIVE-4844.11.patch, HIVE-4844.12.patch, HIVE-4844.13.patch, HIVE-4844.14.patch, HIVE-4844.1.patch.hack, HIVE-4844.2.patch, HIVE-4844.3.patch, HIVE-4844.4.patch, HIVE-4844.5.patch, HIVE-4844.6.patch, HIVE-4844.7.patch, HIVE-4844.8.patch, HIVE-4844.9.patch, HIVE-4844.D12699.1.patch, screenshot.png Add new varchar data types which have support for more SQL-compliant behavior, such as SQL string comparison semantics, max length, etc. Char type will be added as another task. NO PRECOMMIT TESTS - now dependent on HIVE-5203/5204/5206 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5084) Fix newline.q on Windows
[ https://issues.apache.org/jira/browse/HIVE-5084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-5084: - Status: Patch Available (was: Open) Fix newline.q on Windows Key: HIVE-5084 URL: https://issues.apache.org/jira/browse/HIVE-5084 Project: Hive Issue Type: Bug Components: Tests, Windows Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.12.0 Attachments: HIVE-5084-1.patch Test failed with vague error message: [junit] Error during job, obtaining debugging information... [junit] junit.framework.AssertionFailedError: Client Execution failed with error code = 2 hive.log doesn't show something interesting either: 2013-08-14 00:47:29,411 DEBUG zookeeper.ClientCnxn (ClientCnxn.java:readResponse(723)) - Got ping response for sessionid: 0x1407a49fc1e0003 after 1ms 2013-08-14 00:47:31,391 ERROR exec.Task (SessionState.java:printError(416)) - Execution failed with exit status: 2 2013-08-14 00:47:31,391 ERROR exec.Task (SessionState.java:printError(416)) - Obtaining error information 2013-08-14 00:47:31,392 ERROR exec.Task (SessionState.java:printError(416)) - Task failed! Task ID: Stage-1 Logs: -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5086) Fix scriptfile1.q on Windows
[
https://issues.apache.org/jira/browse/HIVE-5086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-5086:
-
Status: Patch Available (was: Open)
Fix scriptfile1.q on Windows
Key: HIVE-5086
URL: https://issues.apache.org/jira/browse/HIVE-5086
Project: Hive
Issue Type: Bug
Components: Tests, Windows
Reporter: Daniel Dai
Assignee: Daniel Dai
Fix For: 0.12.0
Attachments: HIVE-5086-1.patch
Test failed with error message:
[junit] Task with the most failures(4):
[junit] -
[junit] Task ID:
[junit] task_20130814023904691_0001_m_00
[junit]
[junit] URL:
[junit]
http://localhost:50030/taskdetails.jsp?jobid=job_20130814023904691_0001tipid=task_20130814023904691_0001_m_00
[junit] -
[junit] Diagnostic Messages for this Task:
[junit] java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {key:238,value:val_238}
[junit] at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175)
[junit] at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
[junit] at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
[junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:365)
[junit] at org.apache.hadoop.mapred.Child$4.run(Child.java:271)
[junit] at java.security.AccessController.doPrivileged(Native Method)
[junit] at javax.security.auth.Subject.doAs(Subject.java:396)
[junit] at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
[junit] at org.apache.hadoop.mapred.Child.main(Child.java:265)
[junit] Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
Runtime Error while processing row {key:238,value:val_238}
[junit] at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:538)
[junit] at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:157)
[junit] ... 8 more
[junit] Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
[Error 2]: Unable to initialize custom script.
[junit] at
org.apache.hadoop.hive.ql.exec.ScriptOperator.processOp(ScriptOperator.java:357)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:848)
[junit] at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:88)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:848)
[junit] at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:848)
[junit] at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:528)
[junit] ... 9 more
[junit] Caused by: java.io.IOException: Cannot run program
D:\tmp\hadoop-Administrator\mapred\local\3_0\taskTracker\Administrator\jobcache\job_20130814023904691_0001\attempt_20130814023904691_0001_m_00_3\work\.\testgrep:
CreateProcess error=193, %1 is not a valid Win32 application
[junit] at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
[junit] at
org.apache.hadoop.hive.ql.exec.ScriptOperator.processOp(ScriptOperator.java:316)
[junit] ... 18 more
[junit] Caused by: java.io.IOException: CreateProcess error=193, %1 is
not a valid Win32 application
[junit] at java.lang.ProcessImpl.create(Native Method)
[junit] at java.lang.ProcessImpl.init(ProcessImpl.java:81)
[junit] at java.lang.ProcessImpl.start(ProcessImpl.java:30)
[junit] at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
[junit] ... 19 more
[junit]
[junit]
[junit] Exception: Client Execution failed with error code = 2
[junit] See build/ql/tmp/hive.log, or try ant test ...
-Dtest.silent=false to get more logs.
[junit] junit.framework.AssertionFailedError: Client Execution failed
with error code = 2
[junit] See build/ql/tmp/hive.log, or try ant test ...
-Dtest.silent=false to get more logs.
[junit] at junit.framework.Assert.fail(Assert.java:47)
[junit] at
org.apache.hadoop.hive.cli.TestMinimrCliDriver.runTest(TestMinimrCliDriver.java:122)
[junit] at
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_scriptfile1(TestMinimrCliDriver.java:104)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[jira] [Updated] (HIVE-5079) Make Hive compile under Windows
[
https://issues.apache.org/jira/browse/HIVE-5079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-5079:
-
Status: Patch Available (was: Open)
Make Hive compile under Windows
---
Key: HIVE-5079
URL: https://issues.apache.org/jira/browse/HIVE-5079
Project: Hive
Issue Type: Bug
Components: Build Infrastructure
Reporter: Daniel Dai
Assignee: Daniel Dai
Fix For: 0.12.0
Attachments: HIVE-5079-1.patch, HIVE-5079-2.patch
Hive compilation failed under Windows. Error message:
{code}
compile:
[echo] Project: common
[exec] D:\Program Files (x86)\GnuWin32\bin\xargs.exe: md5sum: No such
file
or directory
[exec] md5sum:
../serde/src/java/org/apache/hadoop/hive/serde2/io/Timesta:
No such file or directory
[javac] Compiling 25 source files to
D:\Users\Administrator\hive\build\common\classes
[javac]
D:\Users\Administrator\hive\common\src\gen\org\apache\hive\common\package-info.java:4:
unclosed string literal
[javac] @HiveVersionAnnotation(version=0.12.0-SNAPSHOT,
revision=80eadd8fa2af5eeba61f921318ab8b2c19980ab3, branch=trunk
[javac]
^
[javac]
D:\Users\Administrator\hive\common\src\gen\org\apache\hive\common\package-info.java:5:
unclosed string literal
[javac] ,
[javac] ^
[javac]
D:\Users\Administrator\hive\common\src\gen\org\apache\hive\common\package-info.java:6:
class, interface, or enum expected
[javac] user=Administrator
[javac] ^
[javac]
D:\Users\Administrator\hive\common\src\gen\org\apache\hive\common\package-info.java:6:
unclosed string literal
[javac] user=Administrator
[javac] ^
[javac]
D:\Users\Administrator\hive\common\src\gen\org\apache\hive\common\package-info.java:10:
unclosed string literal
[javac] ,
[javac] ^
[javac]
D:\Users\Administrator\hive\common\src\gen\org\apache\hive\common\package-info.java:11:
unclosed string literal
[javac]
srcChecksum=aadceb95c37a1704aaf19501f46f6e84
[javac] ^
[javac]
D:\Users\Administrator\hive\common\src\gen\org\apache\hive\common\package-info.java:12:
unclosed string literal
[javac] )
[javac] ^
[javac] 7 errors
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5088) Fix udf_translate.q on Windows
[ https://issues.apache.org/jira/browse/HIVE-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-5088: - Status: Patch Available (was: Open) Fix udf_translate.q on Windows -- Key: HIVE-5088 URL: https://issues.apache.org/jira/browse/HIVE-5088 Project: Hive Issue Type: Bug Components: Tests, Windows Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.12.0 Attachments: HIVE-5088-1.patch Test failed with message: [junit] Begin query: udf_translate.q [junit] 13/08/14 03:23:57 FATAL conf.Configuration: error parsing conf file: com.sun.org.apache.xerces.internal.impl.io.MalformedByteSequenceException: Invalid byte 1 of 1-byte UTF-8 sequence. [junit] Exception in thread main java.lang.RuntimeException: com.sun.org.apache.xerces.internal.impl.io.MalformedByteSequenceException: Invalid byte 1 of 1-byte UTF-8 sequence. [junit] at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1255) [junit] at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1117) [junit] at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1053) [junit] at org.apache.hadoop.conf.Configuration.get(Configuration.java:397) [junit] at org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:594) [junit] at org.apache.hadoop.hive.conf.HiveConf.getBoolVar(HiveConf.java:1015) [junit] at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:659) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [junit] at java.lang.reflect.Method.invoke(Method.java:597) [junit] at org.apache.hadoop.util.RunJar.main(RunJar.java:160) [junit] Caused by: com.sun.org.apache.xerces.internal.impl.io.MalformedByteSequenceException: Invalid byte 1 of 1-byte UTF-8 sequence. [junit] at com.sun.org.apache.xerces.internal.impl.io.UTF8Reader.invalidByte(UTF8Reader.java:684) [junit] at com.sun.org.apache.xerces.internal.impl.io.UTF8Reader.read(UTF8Reader.java:554) [junit] at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.load(XMLEntityScanner.java:1742) [junit] at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.peekChar(XMLEntityScanner.java:487) [junit] at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2688) [junit] at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647) [junit] at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140) [junit] at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:511) [junit] at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808) [junit] at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737) [junit] at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119) [junit] at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:232) [junit] at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284) [junit] at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:124) [junit] at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1181) [junit] ... 11 more [junit] Exception: Client Execution failed with error code = 1 [junit] See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. [junit] junit.framework.AssertionFailedError: Client Execution failed with error code = 1 [junit] See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. [junit] at junit.framework.Assert.fail(Assert.java:47) [junit] at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:122) [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_translate(TestCliDriver.java:104) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [junit] at
[jira] [Updated] (HIVE-4844) Add varchar data type
[ https://issues.apache.org/jira/browse/HIVE-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-4844: - Description: Add new varchar data types which have support for more SQL-compliant behavior, such as SQL string comparison semantics, max length, etc. Char type will be added as another task. was: Add new varchar data types which have support for more SQL-compliant behavior, such as SQL string comparison semantics, max length, etc. Char type will be added as another task. NO PRECOMMIT TESTS - now dependent on HIVE-5203/5204/5206 Add varchar data type - Key: HIVE-4844 URL: https://issues.apache.org/jira/browse/HIVE-4844 Project: Hive Issue Type: New Feature Components: Types Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-4844.10.patch, HIVE-4844.11.patch, HIVE-4844.12.patch, HIVE-4844.13.patch, HIVE-4844.14.patch, HIVE-4844.15.patch, HIVE-4844.16.patch, HIVE-4844.1.patch.hack, HIVE-4844.2.patch, HIVE-4844.3.patch, HIVE-4844.4.patch, HIVE-4844.5.patch, HIVE-4844.6.patch, HIVE-4844.7.patch, HIVE-4844.8.patch, HIVE-4844.9.patch, HIVE-4844.D12699.1.patch, screenshot.png Add new varchar data types which have support for more SQL-compliant behavior, such as SQL string comparison semantics, max length, etc. Char type will be added as another task. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5278) Move some string UDFs to GenericUDFs, for better varchar support
[ https://issues.apache.org/jira/browse/HIVE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-5278: - Status: Patch Available (was: Open) Move some string UDFs to GenericUDFs, for better varchar support Key: HIVE-5278 URL: https://issues.apache.org/jira/browse/HIVE-5278 Project: Hive Issue Type: Improvement Components: Types, UDF Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-5278.1.patch To better support varchar/char types in string UDFs, select UDFs should be converted to GenericUDFs. This allows the UDF to return the resulting char/varchar length in the type metadata. This work is being split off as a separate task from HIVE-4844. The initial UDFs as part of this work are concat/lower/upper. NO PRECOMMIT TESTS - dependent on HIVE-4844 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5278) Move some string UDFs to GenericUDFs, for better varchar support
[ https://issues.apache.org/jira/browse/HIVE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-5278: - Attachment: HIVE-5278.1.patch Attaching patch v1 Posted review at https://reviews.facebook.net/D12909 Move some string UDFs to GenericUDFs, for better varchar support Key: HIVE-5278 URL: https://issues.apache.org/jira/browse/HIVE-5278 Project: Hive Issue Type: Improvement Components: Types, UDF Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-5278.1.patch To better support varchar/char types in string UDFs, select UDFs should be converted to GenericUDFs. This allows the UDF to return the resulting char/varchar length in the type metadata. This work is being split off as a separate task from HIVE-4844. The initial UDFs as part of this work are concat/lower/upper. NO PRECOMMIT TESTS - dependent on HIVE-4844 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4492) Revert HIVE-4322
[ https://issues.apache.org/jira/browse/HIVE-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766214#comment-13766214 ] Ashutosh Chauhan commented on HIVE-4492: Yup. Please apply to 0.12 Revert HIVE-4322 Key: HIVE-4492 URL: https://issues.apache.org/jira/browse/HIVE-4492 Project: Hive Issue Type: Bug Components: Metastore, Thrift API Reporter: Samuel Yuan Assignee: Samuel Yuan Fix For: 0.13.0 Attachments: HIVE-4492.1.patch.txt, HIVE-4492.patch See HIVE-4432 and HIVE-4433. It's possible to work around these issues but a better solution is probably to roll back the fix and change the API to use a primitive type as the map key (in a backwards-compatible manner). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4844) Add varchar data type
[ https://issues.apache.org/jira/browse/HIVE-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766216#comment-13766216 ] Phabricator commented on HIVE-4844: --- jdere has commented on the revision HIVE-4844 [jira] Add varchar data type. Most of the SerDe support has been broken out into HIVE-5161. The SerDe's I'm testing there are RegexSerDe, LazyBinarySerDe, LazySimpleSerDe, ColumnarSerDe, LazyBinaryColumnarSerDe, OrcSerDe. I'll need to take a look again, but if HBase/Avro SerDe's use reflection-based methods to get their Types/ObjectInspectors, then they won't be able to support varchars properly. INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java:629 will change in the next patch ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java:160 will change in next patch ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java:169 will change in next patch ql/src/java/org/apache/hadoop/hive/ql/udf/GenericUDFEncode.java:48 sure, will change ql/src/java/org/apache/hadoop/hive/ql/udf/GenericUDFEncode.java:56 will change this message too ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToVarchar.java:36 whoops forgot to add that .. will do ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToVarchar.java:39-40 Actually, my latest patch did mark these fields as transient .. I'll need to update this review with the updated patch. serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveVarcharObjectInspector.java:34 Do we need to serialize object inspectors? my impression is that should always have been transient, at least when they're used in UDFs since they can be set during initialize(). serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java:401 Yeah, this comment is gone in the updated patch. serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java:474 2011 SQL Standard (6.13 cast specification) does say that the value should be TRUE/FALSE. This also matches what we're doing with the string type. serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveVarcharObjectInspector.java:33 ok, will change serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveVarcharObjectInspector.java:35 will change REVISION DETAIL https://reviews.facebook.net/D12699 To: JIRA, jdere Cc: ashutoshc Add varchar data type - Key: HIVE-4844 URL: https://issues.apache.org/jira/browse/HIVE-4844 Project: Hive Issue Type: New Feature Components: Types Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-4844.10.patch, HIVE-4844.11.patch, HIVE-4844.12.patch, HIVE-4844.13.patch, HIVE-4844.14.patch, HIVE-4844.15.patch, HIVE-4844.16.patch, HIVE-4844.1.patch.hack, HIVE-4844.2.patch, HIVE-4844.3.patch, HIVE-4844.4.patch, HIVE-4844.5.patch, HIVE-4844.6.patch, HIVE-4844.7.patch, HIVE-4844.8.patch, HIVE-4844.9.patch, HIVE-4844.D12699.1.patch, screenshot.png Add new varchar data types which have support for more SQL-compliant behavior, such as SQL string comparison semantics, max length, etc. Char type will be added as another task. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765664#comment-13765664 ] Hudson commented on HIVE-5260: -- FAILURE: Integrated in Hive-trunk-hadoop1-ptest #163 (See [https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/163/]) HIVE-5260 : Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers (Viraj Bhat via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1522574) * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableOutputFormat.java * /hive/trunk/hbase-handler/src/test/results/positive/external_table_ppd.q.out * /hive/trunk/hbase-handler/src/test/results/positive/hbase_binary_storage_queries.q.out * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HivePassThroughOutputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HivePassThroughRecordWriter.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableDesc.java Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.13.0 Attachments: HIVE-5260.2.patch, HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5261) Make the Hive HBase storage handler work from HCatalog, and use HiveStorageHandlers instead of HCatStorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765663#comment-13765663 ] Hudson commented on HIVE-5261: -- FAILURE: Integrated in Hive-trunk-hadoop1-ptest #163 (See [https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/163/]) HIVE-5261 : Make the Hive HBase storage handler work from HCatalog, and use HiveStorageHandlers instead of HCatStorageHandlers (Viraj Bhat via Sushanth Sowmyan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1522583) * /hive/trunk/hcatalog/build-support/ant/checkstyle.xml * /hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/HCatStorageHandler.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/CreateTableHook.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/common/HCatUtil.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/DefaultRecordWriterContainer.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputFormatContainer.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileRecordWriterContainer.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FosterStorageHandler.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatBaseInputFormat.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatBaseOutputFormat.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatRecordReader.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatStorageHandler.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/InitializeInput.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/PartInfo.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/security/StorageDelegationAuthorizationProvider.java * /hive/trunk/hcatalog/core/src/test/java/org/apache/hive/hcatalog/cli/DummyStorageHandler.java * /hive/trunk/hcatalog/storage-handlers/hbase/pom.xml * /hive/trunk/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java * /hive/trunk/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java * /hive/trunk/hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/SkeletonHBaseTest.java * /hive/trunk/hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java * /hive/trunk/hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHCatHBaseInputFormat.java * /hive/trunk/hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHiveHBaseStorageHandler.java * /hive/trunk/hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHiveHBaseTableOutputFormat.java * /hive/trunk/hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestPigHBaseStorageHandler.java Make the Hive HBase storage handler work from HCatalog, and use HiveStorageHandlers instead of HCatStorageHandlers -- Key: HIVE-5261 URL: https://issues.apache.org/jira/browse/HIVE-5261 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.13.0 Attachments: HIVE-5261.patch This is a task being created for the HCat side of HIVE-4331 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Created] (HIVE-5112) Upgrade protobuf to 2.5 from 2.4
In conviencing current users to make developers happy of beta software happy is not a big win. Protobuf 2.4 has been the standard for a long time now. We and others have considerable amount of udfs that amd other things that need to be rebuilt. Forcing 12 to protobuf 2.5 will make it harder for me and others to upgrade. Really this dep should all be external to hive. There is no reason that a input or storage handler has to dictate the version of things. I do not see what the problem of waitng 3 month is. On one hand we are going to run 2x tests of hcat for 2 releases to guarentee comparability. On the otherhand we are willimlng to force people to rebuild the kits to be in sync with non release software, On Thursday, September 12, 2013, Sushanth Sowmyan (JIRA) j...@apache.org wrote: [ https://issues.apache.org/jira/browse/HIVE-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765916#comment-13765916] Sushanth Sowmyan commented on HIVE-5112: I'd also much rather have this in 0.12. Without this in 0.12, we either block HIVE-4388 inclusion in 0.12, or require two versions of HIVE-4388, one with protobuf 2.4 and one with 2.5. And that jira blocks being able to work with hbase 0.96, which is in an RC stage, and past beta. The term beta, imo, is not so much about experimentation nowadays as it is about being a preview. Upgrade protobuf to 2.5 from 2.4 Key: HIVE-5112 URL: https://issues.apache.org/jira/browse/HIVE-5112 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Owen O'Malley Fix For: 0.13.0 Attachments: HIVE-5112.2.patch, HIVE-5112.D12429.1.patch Hadoop and Hbase have both upgraded protobuf. We should as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-5289) PTest2 should disable checking of libraries during batch exec
Brock Noland created HIVE-5289: -- Summary: PTest2 should disable checking of libraries during batch exec Key: HIVE-5289 URL: https://issues.apache.org/jira/browse/HIVE-5289 Project: Hive Issue Type: Test Reporter: Brock Noland Assignee: Brock Noland PTest2 has two phases: 1) Build from source 2) Exec in parallel during phase two we don't want ivy making HTTP requests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5289) PTest2 should disable checking of libraries during batch exec
[ https://issues.apache.org/jira/browse/HIVE-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-5289: --- Attachment: HIVE-5289.patch The attached patch improved execution time quite a bit for traditional unit tests. PTest2 should disable checking of libraries during batch exec - Key: HIVE-5289 URL: https://issues.apache.org/jira/browse/HIVE-5289 Project: Hive Issue Type: Test Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-5289.patch PTest2 has two phases: 1) Build from source 2) Exec in parallel during phase two we don't want ivy making HTTP requests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5289) PTest2 should disable checking of libraries during batch exec
[ https://issues.apache.org/jira/browse/HIVE-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-5289: --- Fix Version/s: 0.13.0 Status: Patch Available (was: Open) PTest2 should disable checking of libraries during batch exec - Key: HIVE-5289 URL: https://issues.apache.org/jira/browse/HIVE-5289 Project: Hive Issue Type: Test Reporter: Brock Noland Assignee: Brock Noland Fix For: 0.13.0 Attachments: HIVE-5289.patch PTest2 has two phases: 1) Build from source 2) Exec in parallel during phase two we don't want ivy making HTTP requests. NO PRECOMMIT TESTS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5289) PTest2 should disable checking of libraries during batch exec
[ https://issues.apache.org/jira/browse/HIVE-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-5289: --- Description: PTest2 has two phases: 1) Build from source 2) Exec in parallel during phase two we don't want ivy making HTTP requests. NO PRECOMMIT TESTS was: PTest2 has two phases: 1) Build from source 2) Exec in parallel during phase two we don't want ivy making HTTP requests. PTest2 should disable checking of libraries during batch exec - Key: HIVE-5289 URL: https://issues.apache.org/jira/browse/HIVE-5289 Project: Hive Issue Type: Test Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-5289.patch PTest2 has two phases: 1) Build from source 2) Exec in parallel during phase two we don't want ivy making HTTP requests. NO PRECOMMIT TESTS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5127) Upgrade xerces and xalan for WebHCat
[ https://issues.apache.org/jira/browse/HIVE-5127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766241#comment-13766241 ] Eugene Koifman commented on HIVE-5127: -- no, the patch is fine. There was a block of text commented out. Now it just adds a latest version of Xerces as a dependency. Upgrade xerces and xalan for WebHCat Key: HIVE-5127 URL: https://issues.apache.org/jira/browse/HIVE-5127 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Fix For: 0.12.0 Attachments: HIVE-5127.patch Currently webhcat log files are full of exceptions like this, which obscures the real output and may cause perf issues. Upgrading to more recent versions of xerces/xalan fixes this. Add the following to hive/hcatalog/webhcat/svr/pom.xml dependency groupIdxerces/groupId artifactIdxercesImpl/artifactId version2.9.1/version /dependency dependency groupIdxalan/groupId artifactIdxalan/artifactId version2.7.1/version /dependency 13/08/20 16:54:04 ERROR conf.Configuration: Failed to set setXIncludeAware(true) for parser org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@48dbb335:java.lang.UnsupportedOperationException: This parser does not support specification null version null java.lang.UnsupportedOperationException: This parser does not support specification null version null at javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590) at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1892) at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1861) at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1778) at org.apache.hadoop.conf.Configuration.get(Configuration.java:870) at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:171) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:305) at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:288) at org.apache.hadoop.util.GenericOptionsParser.validateFiles(GenericOptionsParser.java:383) at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:281) at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:422) at org.apache.hadoop.util.GenericOptionsParser.init(GenericOptionsParser.java:168) at org.apache.hadoop.util.GenericOptionsParser.init(GenericOptionsParser.java:151) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.hcatalog.templeton.LauncherDelegator$1.run(LauncherDelegator.java:99) at org.apache.hcatalog.templeton.LauncherDelegator$1.run(LauncherDelegator.java:95) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1441) at org.apache.hcatalog.templeton.LauncherDelegator.queueAsUser(LauncherDelegator.java:95) at org.apache.hcatalog.templeton.LauncherDelegator.enqueueController(LauncherDelegator.java:77) at org.apache.hcatalog.templeton.JarDelegator.run(JarDelegator.java:52) at org.apache.hcatalog.templeton.StreamingDelegator.run(StreamingDelegator.java:53) at org.apache.hcatalog.templeton.Server.mapReduceStreaming(Server.java:596) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766243#comment-13766243 ] Viraj Bhat commented on HIVE-5260: -- Sushanth I think the patch to apply to Hive 0.12 as well. I would be good to have this in Hive 0.12 Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.13.0 Attachments: HIVE-5260.2.patch, HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4388) HBase tests fail against Hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-4388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765900#comment-13765900 ] Sushanth Sowmyan commented on HIVE-4388: Brock, HBASE-9477 you link to seems to have been closed. Are you working on respinning this? I would like for this patch to be part of 0.12 so that we have a more seamless hbase-hadoop2 integration in 0.12. I'm willing to help with the rebase if you need. HBase tests fail against Hadoop 2 - Key: HIVE-4388 URL: https://issues.apache.org/jira/browse/HIVE-4388 Project: Hive Issue Type: Bug Components: HBase Handler Reporter: Gunther Hagleitner Assignee: Brock Noland Attachments: HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388-wip.txt Currently we're building by default against 0.92. When you run against hadoop 2 (-Dhadoop.mr.rev=23) builds fail because of: HBASE-5963. HIVE-3861 upgrades the version of hbase used. This will get you past the problem in HBASE-5963 (which was fixed in 0.94.1) but fails with: HBASE-6396. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5253) Create component to compile and jar dynamic code
[ https://issues.apache.org/jira/browse/HIVE-5253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-5253: -- Attachment: HIVE-5253.patch.txt Create component to compile and jar dynamic code Key: HIVE-5253 URL: https://issues.apache.org/jira/browse/HIVE-5253 Project: Hive Issue Type: Sub-task Reporter: Edward Capriolo Attachments: HIVE-5253.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5253) Create component to compile and jar dynamic code
[ https://issues.apache.org/jira/browse/HIVE-5253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-5253: -- Attachment: (was: cli_driver.diff) Create component to compile and jar dynamic code Key: HIVE-5253 URL: https://issues.apache.org/jira/browse/HIVE-5253 Project: Hive Issue Type: Sub-task Reporter: Edward Capriolo Attachments: HIVE-5253.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5253) Create component to compile and jar dynamic code
[ https://issues.apache.org/jira/browse/HIVE-5253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-5253: -- Attachment: cli_driver.diff Create component to compile and jar dynamic code Key: HIVE-5253 URL: https://issues.apache.org/jira/browse/HIVE-5253 Project: Hive Issue Type: Sub-task Reporter: Edward Capriolo Attachments: HIVE-5253.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5253) Create component to compile and jar dynamic code
[ https://issues.apache.org/jira/browse/HIVE-5253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-5253: -- Assignee: Edward Capriolo Status: Patch Available (was: Open) Create component to compile and jar dynamic code Key: HIVE-5253 URL: https://issues.apache.org/jira/browse/HIVE-5253 Project: Hive Issue Type: Sub-task Reporter: Edward Capriolo Assignee: Edward Capriolo Attachments: HIVE-5253.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5112) Upgrade protobuf to 2.5 from 2.4
[ https://issues.apache.org/jira/browse/HIVE-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766251#comment-13766251 ] Sushanth Sowmyan commented on HIVE-5112: Compiled code for protobuf is supposed to be backward-compatible across protobuf releases(old classes with new protobuf jar) . The only case where protobuf backward compatibility is supposed to bite you is when you have generated code with an old protobuf jar that you're trying to compile with a new protobuf jar. It's supposed to cause compile-time hassles, not runtime hassles. So, I wanted to see if it was a difficult task upgrading hive-protobuf to a newer protobuf as well and so, tried to check out hive-protobuf to test to see if it was a simple update to 2.5 for that. I see only one .proto file, and it's used by the tests. I was able to generate new code for it that compiles, but I get test failures all over from the version of hive_test that I compiled (return code 9 and all that) That got me thinking - if the problem comes from a protobuf 2.5 is not production-certified stance, then shouldn't hive-0.12 also not match production-certification? Upgrade protobuf to 2.5 from 2.4 Key: HIVE-5112 URL: https://issues.apache.org/jira/browse/HIVE-5112 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Owen O'Malley Fix For: 0.13.0 Attachments: HIVE-5112.2.patch, HIVE-5112.D12429.1.patch Hadoop and Hbase have both upgraded protobuf. We should as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-5290) Some HCatalog tests have been behaving flaky
Brock Noland created HIVE-5290: -- Summary: Some HCatalog tests have been behaving flaky Key: HIVE-5290 URL: https://issues.apache.org/jira/browse/HIVE-5290 Project: Hive Issue Type: Test Affects Versions: 0.13.0 Reporter: Brock Noland Assignee: Brock Noland -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5066) [WebHCat] Other code fixes for Windows
[ https://issues.apache.org/jira/browse/HIVE-5066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-5066: - Status: Patch Available (was: Open) [WebHCat] Other code fixes for Windows -- Key: HIVE-5066 URL: https://issues.apache.org/jira/browse/HIVE-5066 Project: Hive Issue Type: Sub-task Components: HCatalog Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.12.0 Attachments: HIVE-5034-1.patch, HIVE-5066-2.patch, HIVE-5066-3.patch This is equivalent to HCATALOG-526, but updated to sync with latest trunk. NO PRECOMMIT TESTS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5066) [WebHCat] Other code fixes for Windows
[ https://issues.apache.org/jira/browse/HIVE-5066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-5066: - Description: This is equivalent to HCATALOG-526, but updated to sync with latest trunk. NO PRECOMMIT TESTS was:This is equivalent to HCATALOG-526, but updated to sync with latest trunk. [WebHCat] Other code fixes for Windows -- Key: HIVE-5066 URL: https://issues.apache.org/jira/browse/HIVE-5066 Project: Hive Issue Type: Sub-task Components: HCatalog Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.12.0 Attachments: HIVE-5034-1.patch, HIVE-5066-2.patch, HIVE-5066-3.patch This is equivalent to HCATALOG-526, but updated to sync with latest trunk. NO PRECOMMIT TESTS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4844) Add varchar data type
[ https://issues.apache.org/jira/browse/HIVE-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-4844: -- Attachment: HIVE-4844.D12891.1.patch jdere requested code review of HIVE-4844 [jira] Add varchar data type. Reviewers: JIRA Changes to HIVE-4844 per review from ashutosh Add new varchar data types which have support for more SQL-compliant behavior, such as SQL string comparison semantics, max length, etc. Char type will be added as another task. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D12891 AFFECTED FILES common/src/java/org/apache/hadoop/hive/common/type/HiveBaseChar.java common/src/java/org/apache/hadoop/hive/common/type/HiveVarchar.java common/src/test/org/apache/hadoop/hive/common/type/TestHiveVarchar.java data/files/datatypes.txt data/files/vc1.txt ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g ql/src/java/org/apache/hadoop/hive/ql/parse/ParseUtils.java ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeConstantDesc.java ql/src/java/org/apache/hadoop/hive/ql/udf/GenericUDFEncode.java ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFComputeStats.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseCompare.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFConcat.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFConcatWS.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLower.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFReflect2.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFStringToMap.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToDate.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToVarchar.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFUpper.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFUtils.java ql/src/test/org/apache/hadoop/hive/ql/exec/TestFunctionRegistry.java ql/src/test/queries/clientnegative/invalid_varchar_length_1.q ql/src/test/queries/clientnegative/invalid_varchar_length_2.q ql/src/test/queries/clientnegative/invalid_varchar_length_3.q ql/src/test/queries/clientpositive/alter_varchar1.q ql/src/test/queries/clientpositive/ctas_varchar.q ql/src/test/queries/clientpositive/partition_varchar1.q ql/src/test/queries/clientpositive/varchar_1.q ql/src/test/queries/clientpositive/varchar_2.q ql/src/test/queries/clientpositive/varchar_cast.q ql/src/test/queries/clientpositive/varchar_comparison.q ql/src/test/queries/clientpositive/varchar_join1.q ql/src/test/queries/clientpositive/varchar_nested_types.q ql/src/test/queries/clientpositive/varchar_udf1.q ql/src/test/queries/clientpositive/varchar_union1.q ql/src/test/results/clientnegative/invalid_varchar_length_1.q.out ql/src/test/results/clientnegative/invalid_varchar_length_2.q.out ql/src/test/results/clientnegative/invalid_varchar_length_3.q.out ql/src/test/results/clientpositive/alter_varchar1.q.out ql/src/test/results/clientpositive/ctas_varchar.q.out ql/src/test/results/clientpositive/partition_varchar1.q.out ql/src/test/results/clientpositive/varchar_1.q.out ql/src/test/results/clientpositive/varchar_2.q.out ql/src/test/results/clientpositive/varchar_cast.q.out ql/src/test/results/clientpositive/varchar_comparison.q.out ql/src/test/results/clientpositive/varchar_join1.q.out ql/src/test/results/clientpositive/varchar_nested_types.q.out ql/src/test/results/clientpositive/varchar_udf1.q.out ql/src/test/results/clientpositive/varchar_union1.q.out serde/if/serde.thrift serde/src/gen/thrift/gen-cpp/serde_constants.cpp serde/src/gen/thrift/gen-cpp/serde_constants.h serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/test/ThriftTestObj.java serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/Complex.java serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MegaStruct.java serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py serde/src/gen/thrift/gen-rb/serde_constants.rb serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroObjectInspectorGenerator.java
[jira] [Updated] (HIVE-5290) Some HCatalog tests have been behaving flaky
[ https://issues.apache.org/jira/browse/HIVE-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-5290: --- Attachment: HIVE-5290.patch Attached patch makes the TestHCatLoader* and TestSequenceFileReadWrite significantly more reliable for me. Some HCatalog tests have been behaving flaky Key: HIVE-5290 URL: https://issues.apache.org/jira/browse/HIVE-5290 Project: Hive Issue Type: Test Affects Versions: 0.13.0 Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-5290.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5290) Some HCatalog tests have been behaving flaky
[ https://issues.apache.org/jira/browse/HIVE-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-5290: --- Status: Patch Available (was: Open) Some HCatalog tests have been behaving flaky Key: HIVE-5290 URL: https://issues.apache.org/jira/browse/HIVE-5290 Project: Hive Issue Type: Test Affects Versions: 0.13.0 Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-5290.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request 14113: support standard UDFs and generic UDFs in vectorized mode with an adaptor
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14113/#review26071 --- Hey, First off let me say you've don't a massive job in the vectorization branch! Nice work! I obviously don't have a enough context to do a real review. However, I just noticed you guys wanted to merge this branch soon so I took a peak at this issue. There is a large amount of trailing whitespace in this patch. Can you remove that? It shows up in red on RB. ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java https://reviews.apache.org/r/14113/#comment50900 Does something require ArrayList to be on the LHS? - Brock Noland On Sept. 12, 2013, 11:57 p.m., Eric Hanson wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14113/ --- (Updated Sept. 12, 2013, 11:57 p.m.) Review request for hive. Bugs: HIVE-4961 https://issues.apache.org/jira/browse/HIVE-4961 Repository: hive-git Description --- Added support standard UDFs and generic UDFs in vectorized mode with an adaptor. Includes unit tests. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java 3d8ade6 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorUDFAdaptor.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorUDFArgDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 5b467bb ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 048824a ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeGenericFuncDesc.java 52fe47b ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/GenericUDFIsNull.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/LongUDF.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestUDF.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorUDFAdaptor.java PRE-CREATION Diff: https://reviews.apache.org/r/14113/diff/ Testing --- Also did extensive ad-hoc end-to end tests with a variety of UDFs. Thanks, Eric Hanson
[jira] [Commented] (HIVE-5251) Change the CLI's poor escapting for semi colon.
[
https://issues.apache.org/jira/browse/HIVE-5251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766264#comment-13766264
]
Edward Capriolo commented on HIVE-5251:
---
Cli escapes are really bad. I know I am half responsible for the current logic.
{quote}
hive set x=' \' ';
hive set x;
x=' ' '
hive compile
` public class x { }` as groovy named x.groovy;
Added /tmp/0.jar to class path
Added resource: /tmp/0.jar
{quote}
This patch should gives us better escaping.
Change the CLI's poor escapting for semi colon.
---
Key: HIVE-5251
URL: https://issues.apache.org/jira/browse/HIVE-5251
Project: Hive
Issue Type: Sub-task
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Currently the CLI has rather flaky ; handling. When specifying inline code
this character is important and we should fix the cli so ; escaping is better.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5251) Change the CLI's poor escapting for semi colon.
[ https://issues.apache.org/jira/browse/HIVE-5251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-5251: -- Status: Patch Available (was: Open) Change the CLI's poor escapting for semi colon. --- Key: HIVE-5251 URL: https://issues.apache.org/jira/browse/HIVE-5251 Project: Hive Issue Type: Sub-task Reporter: Edward Capriolo Assignee: Edward Capriolo Attachments: HIVE-5251.patch.txt Currently the CLI has rather flaky ; handling. When specifying inline code this character is important and we should fix the cli so ; escaping is better. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5251) Change the CLI's poor escapting for semi colon.
[ https://issues.apache.org/jira/browse/HIVE-5251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-5251: -- Attachment: HIVE-5251.patch.txt Change the CLI's poor escapting for semi colon. --- Key: HIVE-5251 URL: https://issues.apache.org/jira/browse/HIVE-5251 Project: Hive Issue Type: Sub-task Reporter: Edward Capriolo Assignee: Edward Capriolo Attachments: HIVE-5251.patch.txt Currently the CLI has rather flaky ; handling. When specifying inline code this character is important and we should fix the cli so ; escaping is better. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5251) Change the CLI's poor escapting for semi colon.
[ https://issues.apache.org/jira/browse/HIVE-5251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766266#comment-13766266 ] Edward Capriolo commented on HIVE-5251: --- Let's see what the patch breaks. Even if it does break some test cases I think we should move forward. Change the CLI's poor escapting for semi colon. --- Key: HIVE-5251 URL: https://issues.apache.org/jira/browse/HIVE-5251 Project: Hive Issue Type: Sub-task Reporter: Edward Capriolo Assignee: Edward Capriolo Attachments: HIVE-5251.patch.txt Currently the CLI has rather flaky ; handling. When specifying inline code this character is important and we should fix the cli so ; escaping is better. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4844) Add varchar data type
[ https://issues.apache.org/jira/browse/HIVE-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766271#comment-13766271 ] Phabricator commented on HIVE-4844: --- ashutoshc has accepted the revision HIVE-4844 [jira] Add varchar data type. +1 Thanks, Jason for making changes. REVISION DETAIL https://reviews.facebook.net/D12891 BRANCH HIVE-4844.2 ARCANIST PROJECT hive To: JIRA, ashutoshc, jdere Add varchar data type - Key: HIVE-4844 URL: https://issues.apache.org/jira/browse/HIVE-4844 Project: Hive Issue Type: New Feature Components: Types Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-4844.10.patch, HIVE-4844.11.patch, HIVE-4844.12.patch, HIVE-4844.13.patch, HIVE-4844.14.patch, HIVE-4844.15.patch, HIVE-4844.16.patch, HIVE-4844.1.patch.hack, HIVE-4844.2.patch, HIVE-4844.3.patch, HIVE-4844.4.patch, HIVE-4844.5.patch, HIVE-4844.6.patch, HIVE-4844.7.patch, HIVE-4844.8.patch, HIVE-4844.9.patch, HIVE-4844.D12699.1.patch, HIVE-4844.D12891.1.patch, screenshot.png Add new varchar data types which have support for more SQL-compliant behavior, such as SQL string comparison semantics, max length, etc. Char type will be added as another task. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5127) Upgrade xerces and xalan for WebHCat
[ https://issues.apache.org/jira/browse/HIVE-5127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766288#comment-13766288 ] Thejas M Nair commented on HIVE-5127: - Got it now. I failed to notice the comment being removed! +1 Will be useful to have this in 0.12 as well. Upgrade xerces and xalan for WebHCat Key: HIVE-5127 URL: https://issues.apache.org/jira/browse/HIVE-5127 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Fix For: 0.12.0 Attachments: HIVE-5127.patch Currently webhcat log files are full of exceptions like this, which obscures the real output and may cause perf issues. Upgrading to more recent versions of xerces/xalan fixes this. Add the following to hive/hcatalog/webhcat/svr/pom.xml dependency groupIdxerces/groupId artifactIdxercesImpl/artifactId version2.9.1/version /dependency dependency groupIdxalan/groupId artifactIdxalan/artifactId version2.7.1/version /dependency 13/08/20 16:54:04 ERROR conf.Configuration: Failed to set setXIncludeAware(true) for parser org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@48dbb335:java.lang.UnsupportedOperationException: This parser does not support specification null version null java.lang.UnsupportedOperationException: This parser does not support specification null version null at javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590) at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1892) at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1861) at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1778) at org.apache.hadoop.conf.Configuration.get(Configuration.java:870) at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:171) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:305) at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:288) at org.apache.hadoop.util.GenericOptionsParser.validateFiles(GenericOptionsParser.java:383) at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:281) at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:422) at org.apache.hadoop.util.GenericOptionsParser.init(GenericOptionsParser.java:168) at org.apache.hadoop.util.GenericOptionsParser.init(GenericOptionsParser.java:151) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.hcatalog.templeton.LauncherDelegator$1.run(LauncherDelegator.java:99) at org.apache.hcatalog.templeton.LauncherDelegator$1.run(LauncherDelegator.java:95) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1441) at org.apache.hcatalog.templeton.LauncherDelegator.queueAsUser(LauncherDelegator.java:95) at org.apache.hcatalog.templeton.LauncherDelegator.enqueueController(LauncherDelegator.java:77) at org.apache.hcatalog.templeton.JarDelegator.run(JarDelegator.java:52) at org.apache.hcatalog.templeton.StreamingDelegator.run(StreamingDelegator.java:53) at org.apache.hcatalog.templeton.Server.mapReduceStreaming(Server.java:596) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
[jira] [Updated] (HIVE-5161) Additional SerDe support for varchar type
[ https://issues.apache.org/jira/browse/HIVE-5161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-5161: -- Attachment: D12897.1.patch jdere requested code review of HIVE-5161 [jira] Additional SerDe support for varchar type. Reviewers: JIRA HIVE-5161: Additional SerDe support for varchar type Breaking out support for varchar for the various SerDes as an additional task. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D12897 AFFECTED FILES ql/src/gen/protobuf/gen-java/org/apache/hadoop/hive/ql/io/orc/OrcProto.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/ColumnStatisticsImpl.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcStruct.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java ql/src/java/org/apache/hadoop/hive/ql/io/orc/WriterImpl.java ql/src/protobuf/org/apache/hadoop/hive/ql/io/orc/orc_proto.proto ql/src/test/queries/clientpositive/varchar_serde.q ql/src/test/results/clientpositive/varchar_serde.q.out serde/src/java/org/apache/hadoop/hive/serde2/RegexSerDe.java serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java MANAGE HERALD RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/30867/ To: JIRA, jdere Additional SerDe support for varchar type - Key: HIVE-5161 URL: https://issues.apache.org/jira/browse/HIVE-5161 Project: Hive Issue Type: Bug Components: Serializers/Deserializers, Types Reporter: Jason Dere Assignee: Jason Dere Attachments: D12897.1.patch, HIVE-5161.1.patch, HIVE-5161.2.patch, HIVE-5161.3.patch Breaking out support for varchar for the various SerDes as an additional task. NO PRECOMMIT TESTS - can't run tests until HIVE-4844 is committed -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request 10180: HIVE-4171
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10180/#review26073 --- LGTM - Prasad Mujumdar On Sept. 12, 2013, 3:08 a.m., Thejas Nair wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10180/ --- (Updated Sept. 12, 2013, 3:08 a.m.) Review request for hive, Ashutosh Chauhan, Navis Ryu, and Prasad Mujumdar. Bugs: HIVE-4171 https://issues.apache.org/jira/browse/HIVE-4171 Repository: hive-git Description --- HIVE-4171 Diffs - cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java aeced48 cli/src/java/org/apache/hadoop/hive/cli/CliSessionState.java d11b873 cli/src/test/org/apache/hadoop/hive/cli/TestCliSessionState.java a95e206 hcatalog/core/src/main/java/org/apache/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzer.java 0e011e4 hcatalog/core/src/main/java/org/apache/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzerBase.java 3740795 hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzer.java 75837ef hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzerBase.java c144135 ql/src/java/org/apache/hadoop/hive/ql/Driver.java 0bc4425 ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsTask.java 9de7dcf ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 4dcb260 ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 344c9b1 ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 932ec66 ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java 3413f51 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 6d50109 ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 6888504 ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java d318f71 ql/src/test/org/apache/hadoop/hive/ql/exec/TestExecDriver.java 9588061 ql/src/test/org/apache/hadoop/hive/ql/history/TestHiveHistory.java 8b8c276 ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java 4ee5267 ql/src/test/org/apache/hadoop/hive/ql/parse/TestMacroSemanticAnalyzer.java 81a3396 ql/src/test/org/apache/hadoop/hive/ql/session/TestSessionState.java PRE-CREATION Diff: https://reviews.apache.org/r/10180/diff/ Testing --- Manually tested, unit test to be added. Thanks, Thejas Nair
[jira] [Commented] (HIVE-5283) Merge vectorization branch to trunk
[ https://issues.apache.org/jira/browse/HIVE-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766303#comment-13766303 ] Carl Steinbach commented on HIVE-5283: -- [~jnp] Can you please post a review request on phabricator or reviewboard? Also, is the plan to commit this patch or to merge the vectorization branch into trunk? Merge vectorization branch to trunk --- Key: HIVE-5283 URL: https://issues.apache.org/jira/browse/HIVE-5283 Project: Hive Issue Type: Bug Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Attachments: HIVE-5283.1.patch, HIVE-5283.2.patch The purpose of this jira is to upload vectorization patch, run tests etc. The actual work will continue under HIVE-4160 umbrella jira. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3764) Support metastore version consistency check
[ https://issues.apache.org/jira/browse/HIVE-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Mujumdar updated HIVE-3764: -- Description: Today there's no version/compatibility information stored in hive metastore. Also the datanucleus configuration property to automatically create missing tables is enabled by default. If you happen to start an older or newer hive or don't run the correct upgrade scripts during migration, the metastore would end up corrupted. The autoCreate schema is not always sufficient to upgrade metastore when migrating to newer release. It's not supported with all databases. Besides the migration often involves altering existing table, changing or moving data etc. Hence it's very useful to have some consistency check to make sure that hive is using correct metastore and for production systems the schema is not automatically by running hive. Besides it would be helpful to add a tool that can leverage this version information to figure out the required set of upgrade scripts, and execute those against the configured metastore. Now that Hive includes Beeline client, it can be used to execute the scripts. was: Today there's no version/compatibility information stored in hive metastore. Also the datanucleus configuration property to automatically create missing tables is enabled by default. If you happen to start an older or newer hive or don't run the correct upgrade scripts during migration, the metastore would end up corrupted. The autoCreate schema is not always sufficient to upgrade metastore when migrating to newer release. It's not supported with all databases. Besides the migration often involves altering existing table, changing or moving data etc. Hence it's very useful to have some consistency check to make sure that hive is using correct metastore and for production systems the schema is not automatically by running hive. Support metastore version consistency check --- Key: HIVE-3764 URL: https://issues.apache.org/jira/browse/HIVE-3764 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Today there's no version/compatibility information stored in hive metastore. Also the datanucleus configuration property to automatically create missing tables is enabled by default. If you happen to start an older or newer hive or don't run the correct upgrade scripts during migration, the metastore would end up corrupted. The autoCreate schema is not always sufficient to upgrade metastore when migrating to newer release. It's not supported with all databases. Besides the migration often involves altering existing table, changing or moving data etc. Hence it's very useful to have some consistency check to make sure that hive is using correct metastore and for production systems the schema is not automatically by running hive. Besides it would be helpful to add a tool that can leverage this version information to figure out the required set of upgrade scripts, and execute those against the configured metastore. Now that Hive includes Beeline client, it can be used to execute the scripts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request 14120: HIVE-3764: Support metastore version consistency check
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14120/ --- Review request for hive. Bugs: HIVE-3764 https://issues.apache.org/jira/browse/HIVE-3764 Repository: hive-git Description --- - Added a new table in the metastore schema to store the Hive version in the metastore. - Metastore handler compare the version stored in the schema with its own version. If there's a mismatch, then it can either record the correct version or raise error. The behavior is configurable via a new Hive config. This config when set, also restrict dataNucleus to auto upgrade the schema. - The new schema creation and upgrade scripts record the new version in the metastore version table. - Added 0.12 upgrade scripts for all supported DBs to creates the new table version tables in 0.12 metastore schema - Added a new schemaTool that can perform new schema initialization or upgrade based on the schema version and product version. Diffs - beeline/src/java/org/apache/hive/beeline/HiveSchemaHelper.java PRE-CREATION beeline/src/java/org/apache/hive/beeline/HiveSchemaTool.java PRE-CREATION beeline/src/test/org/apache/hive/beeline/src/test/TestSchemaTool.java PRE-CREATION bin/ext/schemaTool.sh PRE-CREATION bin/schematool PRE-CREATION build-common.xml ad5ac23 build.xml 3e87163 common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 22149e4 conf/hive-default.xml.template 9a3fc1d metastore/scripts/upgrade/derby/014-HIVE-3764.derby.sql PRE-CREATION metastore/scripts/upgrade/derby/hive-schema-0.12.0.derby.sql cce544f metastore/scripts/upgrade/derby/upgrade-0.10.0-to-0.11.0.derby.sql cae7936 metastore/scripts/upgrade/derby/upgrade-0.11.0-to-0.12.0.derby.sql 492cc93 metastore/scripts/upgrade/derby/upgrade.order.derby PRE-CREATION metastore/scripts/upgrade/mysql/014-HIVE-3764.mysql.sql PRE-CREATION metastore/scripts/upgrade/mysql/hive-schema-0.12.0.mysql.sql 22a77fe metastore/scripts/upgrade/mysql/upgrade-0.11.0-to-0.12.0.mysql.sql 375a05f metastore/scripts/upgrade/mysql/upgrade.order.mysql PRE-CREATION metastore/scripts/upgrade/oracle/014-HIVE-3764.oracle.sql PRE-CREATION metastore/scripts/upgrade/oracle/hive-schema-0.12.0.oracle.sql 85a0178 metastore/scripts/upgrade/oracle/upgrade-0.10.0-to-0.11.0.mysql.sql PRE-CREATION metastore/scripts/upgrade/oracle/upgrade-0.11.0-to-0.12.0.oracle.sql a2d0901 metastore/scripts/upgrade/oracle/upgrade.order.oracle PRE-CREATION metastore/scripts/upgrade/postgres/014-HIVE-3764.postgres.sql PRE-CREATION metastore/scripts/upgrade/postgres/hive-schema-0.12.0.postgres.sql 7b319ba metastore/scripts/upgrade/postgres/upgrade-0.11.0-to-0.12.0.postgres.sql 9da0a1b metastore/scripts/upgrade/postgres/upgrade.order.postgres PRE-CREATION metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 39dda92 metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java PRE-CREATION metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java a08c728 metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java e410c3a metastore/src/model/org/apache/hadoop/hive/metastore/model/MVersionTable.java PRE-CREATION metastore/src/model/package.jdo c42b5b0 metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java 8066784 metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 0f9b16c metastore/src/test/org/apache/hadoop/hive/metastore/TestMetastoreVersion.java PRE-CREATION Diff: https://reviews.apache.org/r/14120/diff/ Testing --- Added new tests for schema verification and schemaTool. Thanks, Prasad Mujumdar
[jira] [Updated] (HIVE-3764) Support metastore version consistency check
[ https://issues.apache.org/jira/browse/HIVE-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Mujumdar updated HIVE-3764: -- Affects Version/s: 0.8.0 0.9.0 0.10.0 0.11.0 Fix Version/s: 0.12.0 Support metastore version consistency check --- Key: HIVE-3764 URL: https://issues.apache.org/jira/browse/HIVE-3764 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.8.0, 0.9.0, 0.10.0, 0.11.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Fix For: 0.12.0 Today there's no version/compatibility information stored in hive metastore. Also the datanucleus configuration property to automatically create missing tables is enabled by default. If you happen to start an older or newer hive or don't run the correct upgrade scripts during migration, the metastore would end up corrupted. The autoCreate schema is not always sufficient to upgrade metastore when migrating to newer release. It's not supported with all databases. Besides the migration often involves altering existing table, changing or moving data etc. Hence it's very useful to have some consistency check to make sure that hive is using correct metastore and for production systems the schema is not automatically by running hive. Besides it would be helpful to add a tool that can leverage this version information to figure out the required set of upgrade scripts, and execute those against the configured metastore. Now that Hive includes Beeline client, it can be used to execute the scripts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3764) Support metastore version consistency check
[ https://issues.apache.org/jira/browse/HIVE-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Mujumdar updated HIVE-3764: -- Status: Open (was: Patch Available) Support metastore version consistency check --- Key: HIVE-3764 URL: https://issues.apache.org/jira/browse/HIVE-3764 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Today there's no version/compatibility information stored in hive metastore. Also the datanucleus configuration property to automatically create missing tables is enabled by default. If you happen to start an older or newer hive or don't run the correct upgrade scripts during migration, the metastore would end up corrupted. The autoCreate schema is not always sufficient to upgrade metastore when migrating to newer release. It's not supported with all databases. Besides the migration often involves altering existing table, changing or moving data etc. Hence it's very useful to have some consistency check to make sure that hive is using correct metastore and for production systems the schema is not automatically by running hive. Besides it would be helpful to add a tool that can leverage this version information to figure out the required set of upgrade scripts, and execute those against the configured metastore. Now that Hive includes Beeline client, it can be used to execute the scripts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3764) Support metastore version consistency check
[ https://issues.apache.org/jira/browse/HIVE-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Mujumdar updated HIVE-3764: -- Attachment: HIVE-3764-0.13-addional-file.patch Additional files for trunk (0.13) Support metastore version consistency check --- Key: HIVE-3764 URL: https://issues.apache.org/jira/browse/HIVE-3764 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.8.0, 0.9.0, 0.10.0, 0.11.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Fix For: 0.12.0 Attachments: HIVE-3764-0.13-addional-file.patch, HIVE-3764.4.patch Today there's no version/compatibility information stored in hive metastore. Also the datanucleus configuration property to automatically create missing tables is enabled by default. If you happen to start an older or newer hive or don't run the correct upgrade scripts during migration, the metastore would end up corrupted. The autoCreate schema is not always sufficient to upgrade metastore when migrating to newer release. It's not supported with all databases. Besides the migration often involves altering existing table, changing or moving data etc. Hence it's very useful to have some consistency check to make sure that hive is using correct metastore and for production systems the schema is not automatically by running hive. Besides it would be helpful to add a tool that can leverage this version information to figure out the required set of upgrade scripts, and execute those against the configured metastore. Now that Hive includes Beeline client, it can be used to execute the scripts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5261) Make the Hive HBase storage handler work from HCatalog, and use HiveStorageHandlers instead of HCatStorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766334#comment-13766334 ] Hudson commented on HIVE-5261: -- ABORTED: Integrated in Hive-trunk-hadoop2 #425 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/425/]) HIVE-5261 : Make the Hive HBase storage handler work from HCatalog, and use HiveStorageHandlers instead of HCatStorageHandlers (Viraj Bhat via Sushanth Sowmyan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1522583) * /hive/trunk/hcatalog/build-support/ant/checkstyle.xml * /hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/mapreduce/HCatStorageHandler.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/CreateTableHook.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/common/HCatUtil.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/DefaultRecordWriterContainer.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputFormatContainer.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileRecordWriterContainer.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FosterStorageHandler.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatBaseInputFormat.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatBaseOutputFormat.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatRecordReader.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatStorageHandler.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/InitializeInput.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/PartInfo.java * /hive/trunk/hcatalog/core/src/main/java/org/apache/hive/hcatalog/security/StorageDelegationAuthorizationProvider.java * /hive/trunk/hcatalog/core/src/test/java/org/apache/hive/hcatalog/cli/DummyStorageHandler.java * /hive/trunk/hcatalog/storage-handlers/hbase/pom.xml * /hive/trunk/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java * /hive/trunk/hcatalog/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java * /hive/trunk/hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/SkeletonHBaseTest.java * /hive/trunk/hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java * /hive/trunk/hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHCatHBaseInputFormat.java * /hive/trunk/hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHiveHBaseStorageHandler.java * /hive/trunk/hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHiveHBaseTableOutputFormat.java * /hive/trunk/hcatalog/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestPigHBaseStorageHandler.java Make the Hive HBase storage handler work from HCatalog, and use HiveStorageHandlers instead of HCatStorageHandlers -- Key: HIVE-5261 URL: https://issues.apache.org/jira/browse/HIVE-5261 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.13.0 Attachments: HIVE-5261.patch This is a task being created for the HCat side of HIVE-4331 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5260) Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers
[ https://issues.apache.org/jira/browse/HIVE-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766335#comment-13766335 ] Hudson commented on HIVE-5260: -- ABORTED: Integrated in Hive-trunk-hadoop2 #425 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/425/]) HIVE-5260 : Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers (Viraj Bhat via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1522574) * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableOutputFormat.java * /hive/trunk/hbase-handler/src/test/results/positive/external_table_ppd.q.out * /hive/trunk/hbase-handler/src/test/results/positive/hbase_binary_storage_queries.q.out * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HivePassThroughOutputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HivePassThroughRecordWriter.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableDesc.java Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers --- Key: HIVE-5260 URL: https://issues.apache.org/jira/browse/HIVE-5260 Project: Hive Issue Type: Sub-task Components: HBase Handler, HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Viraj Bhat Fix For: 0.13.0 Attachments: HIVE-5260.2.patch, HIVE-5260.patch This is a task being created to address the hive-side of HIVE-4331. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5278) Move some string UDFs to GenericUDFs, for better varchar support
[ https://issues.apache.org/jira/browse/HIVE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-5278: -- Attachment: D12909.1.patch jdere requested code review of HIVE-5278 [jira] Move some string UDFs to GenericUDFs, for better varchar support. Reviewers: JIRA HIVE-5278: Move some string UDFs to GenericUDFs, for better varchar support To better support varchar/char types in string UDFs, select UDFs should be converted to GenericUDFs. This allows the UDF to return the resulting char/varchar length in the type metadata. This work is being split off as a separate task from HIVE-4844. The initial UDFs as part of this work are concat/lower/upper. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D12909 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java ql/src/java/org/apache/hadoop/hive/ql/udf/UDFConcat.java ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLower.java ql/src/java/org/apache/hadoop/hive/ql/udf/UDFUpper.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFConcat.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLower.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFUpper.java ql/src/test/results/compiler/plan/groupby2.q.xml ql/src/test/results/compiler/plan/udf6.q.xml MANAGE HERALD RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/30879/ To: JIRA, jdere Move some string UDFs to GenericUDFs, for better varchar support Key: HIVE-5278 URL: https://issues.apache.org/jira/browse/HIVE-5278 Project: Hive Issue Type: Improvement Components: Types, UDF Reporter: Jason Dere Assignee: Jason Dere Attachments: D12909.1.patch, HIVE-5278.1.patch To better support varchar/char types in string UDFs, select UDFs should be converted to GenericUDFs. This allows the UDF to return the resulting char/varchar length in the type metadata. This work is being split off as a separate task from HIVE-4844. The initial UDFs as part of this work are concat/lower/upper. NO PRECOMMIT TESTS - dependent on HIVE-4844 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request 14121: Base64 decoding for binary type
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14121/ --- Review request for hive and Ashutosh Chauhan. Bugs: HIVE-5221 https://issues.apache.org/jira/browse/HIVE-5221 Repository: hive-git Description --- WIP Patch Copied from the JIRA: Hi, I am using Hive 10. When I create an external table with column type as Binary, the query result on the table is showing some junk values for the column with binary datatype. Please find below the query I have used to create the table: CREATE EXTERNAL TABLE BOOL1(NB BOOLEAN,email STRING, bitfld BINARY) ROW FORMAT DELIMITED FIELDS TERMINATED BY '^' LINES TERMINATED BY '\n' STORED AS TEXTFILE LOCATION '/user/hivetables/testbinary'; The query I have used is : select * from bool1 The sample data in the hdfs file is: 0^a...@abc.com^001 1^a...@abc.com^010 ^a...@abc.com^011 ^a...@abc.com^100 t^a...@abc.com^101 f^a...@abc.com^110 true^a...@abc.com^111 false^a...@abc.com^001 123^ ^01100010 12344^ ^0111 Please share your inputs if it is possible. Thanks, Arun Diffs - hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java 4900a41 serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java 28f8d6a serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarSerDe.java 11f5f07 serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyBinary.java ae12f20 serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java a08b4a8 serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazySimpleSerDe.java 606208c serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyBinaryObjectInspector.java dbd60f7 serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java afbf454 Diff: https://reviews.apache.org/r/14121/diff/ Testing --- WIP: working on writing new test cases. Thanks, Mohammad Islam
[jira] [Updated] (HIVE-5221) Issue in colun type with data type as BINARY
[ https://issues.apache.org/jira/browse/HIVE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Kamrul Islam updated HIVE-5221: Status: Patch Available (was: Open) Issue in colun type with data type as BINARY Key: HIVE-5221 URL: https://issues.apache.org/jira/browse/HIVE-5221 Project: Hive Issue Type: Bug Reporter: Arun Vasu Assignee: Mohammad Kamrul Islam Priority: Critical Attachments: HIVE-5221.1.patch Hi, I am using Hive 10. When I create an external table with column type as Binary, the query result on the table is showing some junk values for the column with binary datatype. Please find below the query I have used to create the table: CREATE EXTERNAL TABLE BOOL1(NB BOOLEAN,email STRING, bitfld BINARY) ROW FORMAT DELIMITED FIELDS TERMINATED BY '^' LINES TERMINATED BY '\n' STORED AS TEXTFILE LOCATION '/user/hivetables/testbinary'; The query I have used is : select * from bool1 The sample data in the hdfs file is: 0^a...@abc.com^001 1^a...@abc.com^010 ^a...@abc.com^011 ^a...@abc.com^100 t^a...@abc.com^101 f^a...@abc.com^110 true^a...@abc.com^111 false^a...@abc.com^001 123^^01100010 12344^^0111 Please share your inputs if it is possible. Thanks, Arun -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5221) Issue in colun type with data type as BINARY
[ https://issues.apache.org/jira/browse/HIVE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Kamrul Islam updated HIVE-5221: Attachment: HIVE-5221.1.patch WIP Patch. Can be reviewed at : https://reviews.apache.org/r/14121/ Issue in colun type with data type as BINARY Key: HIVE-5221 URL: https://issues.apache.org/jira/browse/HIVE-5221 Project: Hive Issue Type: Bug Reporter: Arun Vasu Assignee: Mohammad Kamrul Islam Priority: Critical Attachments: HIVE-5221.1.patch Hi, I am using Hive 10. When I create an external table with column type as Binary, the query result on the table is showing some junk values for the column with binary datatype. Please find below the query I have used to create the table: CREATE EXTERNAL TABLE BOOL1(NB BOOLEAN,email STRING, bitfld BINARY) ROW FORMAT DELIMITED FIELDS TERMINATED BY '^' LINES TERMINATED BY '\n' STORED AS TEXTFILE LOCATION '/user/hivetables/testbinary'; The query I have used is : select * from bool1 The sample data in the hdfs file is: 0^a...@abc.com^001 1^a...@abc.com^010 ^a...@abc.com^011 ^a...@abc.com^100 t^a...@abc.com^101 f^a...@abc.com^110 true^a...@abc.com^111 false^a...@abc.com^001 123^^01100010 12344^^0111 Please share your inputs if it is possible. Thanks, Arun -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5122) Add partition for multiple partition ignores locations for non-first partitions
[ https://issues.apache.org/jira/browse/HIVE-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766352#comment-13766352 ] Thejas M Nair commented on HIVE-5122: - [~navis] Your changes in this patch make the logic in both anltr grammar and DDLSemanticAnalyzer code much cleaner, it looks good. But I didn't understand what was causing this bug (and how the change fixes it). Was it some anltr weird behavior with old grammar that caused the issue? Add partition for multiple partition ignores locations for non-first partitions --- Key: HIVE-5122 URL: https://issues.apache.org/jira/browse/HIVE-5122 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-5122.D12411.1.patch, HIVE-5122.D12411.2.patch http://www.mail-archive.com/user@hive.apache.org/msg09151.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4998) support jdbc documented table types in default configuration
[ https://issues.apache.org/jira/browse/HIVE-4998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-4998: Attachment: HIVE-4998.1.patch support jdbc documented table types in default configuration Key: HIVE-4998 URL: https://issues.apache.org/jira/browse/HIVE-4998 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Thejas M Nair Attachments: HIVE-4998.1.patch The jdbc table types supported by hive server2 are not the documented typical types [1] in jdbc, they are hive specific types (MANAGED_TABLE, EXTERNAL_TABLE, VIRTUAL_VIEW). HIVE-4573 added support for the jdbc documented typical types, but the HS2 default configuration is to return the hive types The default configuration should result in the expected jdbc typical behavior. [1] http://docs.oracle.com/javase/6/docs/api/java/sql/DatabaseMetaData.html?is-external=true#getTables(java.lang.String, java.lang.String, java.lang.String, java.lang.String[]) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-5237) Incorrect group-by aggregation in 0.11.0
[
https://issues.apache.org/jira/browse/HIVE-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Børge Svingen resolved HIVE-5237.
-
Resolution: Duplicate
Incorrect group-by aggregation in 0.11.0
Key: HIVE-5237
URL: https://issues.apache.org/jira/browse/HIVE-5237
Project: Hive
Issue Type: Bug
Affects Versions: 0.11.0
Reporter: Børge Svingen
Priority: Critical
group by with sub queries does not correctly aggregate results in Hive 0.11.0.
To reproduce:
Put the file
{code}
1,b
2,c
2,b
3,a
3,c
4,a
{code}
in HDFS, and run
{code}
create external table abc (x int, y string) row format delimited fields
terminated by ',' location '/data/';
{code}
The query
{code}
select
x,
count(*)
from
(select
x,
y
from
abc
group by
x,
y
) a
group by
x;
{code}
will then give the result
{code}
2 1
3 1
2 1
4 1
3 1
1 1
{code}
instead of the correct
{code}
1 1
2 2
3 2
4 1
{code}
In 0.9.0 and 0.10.0 this is all working correctly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request 14122: support jdbc documented table types in default configuration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14122/ --- Review request for hive. Bugs: HIVE-4998 https://issues.apache.org/jira/browse/HIVE-4998 Repository: hive-git Description --- The jdbc table types supported by hive server2 are not the documented typical types [1] in jdbc, they are hive specific types (MANAGED_TABLE, EXTERNAL_TABLE, VIRTUAL_VIEW). HIVE-4573 added support for the jdbc documented typical types, but the HS2 default configuration is to return the hive types The default configuration should result in the expected jdbc typical behavior. [1] http://docs.oracle.com/javase/6/docs/api/java/sql/DatabaseMetaData.html?is-external=true#getTables(java.lang.String, java.lang.String, java.lang.String, java.lang.String[]) Diffs - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java c5a8ff3 conf/hive-default.xml.template 181011b jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java c658dbd Diff: https://reviews.apache.org/r/14122/diff/ Testing --- Thanks, Thejas Nair
[jira] [Commented] (HIVE-5112) Upgrade protobuf to 2.5 from 2.4
[ https://issues.apache.org/jira/browse/HIVE-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766292#comment-13766292 ] Thejas M Nair commented on HIVE-5112: - There are discussions in the hadoop mailing lists about having the hadoop 2 GA in few weeks. So I think it is worthwhile that we aim to have 0.12 to be compatible with the hadoop 2 line as well. [~appodictic] I don't think your -1 is reasonable for the following reasons- - I don't consider it feasible to stop upgrading jars in hive because the old version of jar was used by some udf/serde/inputformat. (But I do think that in long term we need to explore ways of isolating the hive jars/classes vs user jars/classes) - In the example you cited, as Sushanth mentioned, you don't need to make any code changes to use protobuf 2.5. I suspect most users who are using protobuf with dynamic schema would not choose to work with generated code. So the number of users this is likely to affect is small. - The upgrade is inevitable. With it being in trunk, it is in its way into the 0.13 release. I request you to please reconsider your -1. Upgrade protobuf to 2.5 from 2.4 Key: HIVE-5112 URL: https://issues.apache.org/jira/browse/HIVE-5112 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Owen O'Malley Fix For: 0.13.0 Attachments: HIVE-5112.2.patch, HIVE-5112.D12429.1.patch Hadoop and Hbase have both upgraded protobuf. We should as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5237) Incorrect group-by aggregation in 0.11.0
[
https://issues.apache.org/jira/browse/HIVE-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766354#comment-13766354
]
Børge Svingen commented on HIVE-5237:
-
Thanks, resolving this one, then.
Incorrect group-by aggregation in 0.11.0
Key: HIVE-5237
URL: https://issues.apache.org/jira/browse/HIVE-5237
Project: Hive
Issue Type: Bug
Affects Versions: 0.11.0
Reporter: Børge Svingen
Priority: Critical
group by with sub queries does not correctly aggregate results in Hive 0.11.0.
To reproduce:
Put the file
{code}
1,b
2,c
2,b
3,a
3,c
4,a
{code}
in HDFS, and run
{code}
create external table abc (x int, y string) row format delimited fields
terminated by ',' location '/data/';
{code}
The query
{code}
select
x,
count(*)
from
(select
x,
y
from
abc
group by
x,
y
) a
group by
x;
{code}
will then give the result
{code}
2 1
3 1
2 1
4 1
3 1
1 1
{code}
instead of the correct
{code}
1 1
2 2
3 2
4 1
{code}
In 0.9.0 and 0.10.0 this is all working correctly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3764) Support metastore version consistency check
[ https://issues.apache.org/jira/browse/HIVE-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Mujumdar updated HIVE-3764: -- Status: Patch Available (was: Open) Review request on https://reviews.apache.org/r/14120/ Support metastore version consistency check --- Key: HIVE-3764 URL: https://issues.apache.org/jira/browse/HIVE-3764 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.11.0, 0.10.0, 0.9.0, 0.8.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Fix For: 0.12.0 Attachments: HIVE-3764.4.patch Today there's no version/compatibility information stored in hive metastore. Also the datanucleus configuration property to automatically create missing tables is enabled by default. If you happen to start an older or newer hive or don't run the correct upgrade scripts during migration, the metastore would end up corrupted. The autoCreate schema is not always sufficient to upgrade metastore when migrating to newer release. It's not supported with all databases. Besides the migration often involves altering existing table, changing or moving data etc. Hence it's very useful to have some consistency check to make sure that hive is using correct metastore and for production systems the schema is not automatically by running hive. Besides it would be helpful to add a tool that can leverage this version information to figure out the required set of upgrade scripts, and execute those against the configured metastore. Now that Hive includes Beeline client, it can be used to execute the scripts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3764) Support metastore version consistency check
[ https://issues.apache.org/jira/browse/HIVE-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Mujumdar updated HIVE-3764: -- Attachment: (was: HIVE-3764-1.patch) Support metastore version consistency check --- Key: HIVE-3764 URL: https://issues.apache.org/jira/browse/HIVE-3764 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Today there's no version/compatibility information stored in hive metastore. Also the datanucleus configuration property to automatically create missing tables is enabled by default. If you happen to start an older or newer hive or don't run the correct upgrade scripts during migration, the metastore would end up corrupted. The autoCreate schema is not always sufficient to upgrade metastore when migrating to newer release. It's not supported with all databases. Besides the migration often involves altering existing table, changing or moving data etc. Hence it's very useful to have some consistency check to make sure that hive is using correct metastore and for production systems the schema is not automatically by running hive. Besides it would be helpful to add a tool that can leverage this version information to figure out the required set of upgrade scripts, and execute those against the configured metastore. Now that Hive includes Beeline client, it can be used to execute the scripts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4998) support jdbc documented table types in default configuration
[ https://issues.apache.org/jira/browse/HIVE-4998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-4998: Assignee: Thejas M Nair Status: Patch Available (was: Open) support jdbc documented table types in default configuration Key: HIVE-4998 URL: https://issues.apache.org/jira/browse/HIVE-4998 Project: Hive Issue Type: Bug Components: HiveServer2, JDBC Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-4998.1.patch The jdbc table types supported by hive server2 are not the documented typical types [1] in jdbc, they are hive specific types (MANAGED_TABLE, EXTERNAL_TABLE, VIRTUAL_VIEW). HIVE-4573 added support for the jdbc documented typical types, but the HS2 default configuration is to return the hive types The default configuration should result in the expected jdbc typical behavior. [1] http://docs.oracle.com/javase/6/docs/api/java/sql/DatabaseMetaData.html?is-external=true#getTables(java.lang.String, java.lang.String, java.lang.String, java.lang.String[]) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3764) Support metastore version consistency check
[ https://issues.apache.org/jira/browse/HIVE-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Mujumdar updated HIVE-3764: -- Attachment: HIVE-3764.4.patch Patch for 0.12 branch Support metastore version consistency check --- Key: HIVE-3764 URL: https://issues.apache.org/jira/browse/HIVE-3764 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.8.0, 0.9.0, 0.10.0, 0.11.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Fix For: 0.12.0 Attachments: HIVE-3764.4.patch Today there's no version/compatibility information stored in hive metastore. Also the datanucleus configuration property to automatically create missing tables is enabled by default. If you happen to start an older or newer hive or don't run the correct upgrade scripts during migration, the metastore would end up corrupted. The autoCreate schema is not always sufficient to upgrade metastore when migrating to newer release. It's not supported with all databases. Besides the migration often involves altering existing table, changing or moving data etc. Hence it's very useful to have some consistency check to make sure that hive is using correct metastore and for production systems the schema is not automatically by running hive. Besides it would be helpful to add a tool that can leverage this version information to figure out the required set of upgrade scripts, and execute those against the configured metastore. Now that Hive includes Beeline client, it can be used to execute the scripts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5291) join32_lessSize.q has ordering problem under hadoop-2
[
https://issues.apache.org/jira/browse/HIVE-5291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766602#comment-13766602
]
Ashutosh Chauhan commented on HIVE-5291:
Actually, I cannot repro the fail you are mentioning on latest trunk. I did
{noformat}
ant test -Dtestcase=TestCliDriver -Dmodule=ql -Dqfile=join32_lessSize.q
-Dhadoop.mr.rev=23
{noformat}
and I got:
{noformat}
[junit] Done query: join32_lessSize.q elapsedTime=433s
[junit] Cleaning up TestCliDriver
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 436.32 sec
{noformat}
So, I am not sure what is there to fix.
join32_lessSize.q has ordering problem under hadoop-2
-
Key: HIVE-5291
URL: https://issues.apache.org/jira/browse/HIVE-5291
Project: Hive
Issue Type: Sub-task
Components: Tests
Reporter: Brock Noland
Assignee: Brock Noland
Fix For: 0.13.0
Attachments: HIVE-5291.patch
Test just needs more ORDER BY and output updated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5289) PTest2 should disable checking of libraries during batch exec
[ https://issues.apache.org/jira/browse/HIVE-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766607#comment-13766607 ] Thejas M Nair commented on HIVE-5289: - +! LGTM PTest2 should disable checking of libraries during batch exec - Key: HIVE-5289 URL: https://issues.apache.org/jira/browse/HIVE-5289 Project: Hive Issue Type: Test Reporter: Brock Noland Assignee: Brock Noland Fix For: 0.13.0 Attachments: HIVE-5289.patch PTest2 has two phases: 1) Build from source 2) Exec in parallel during phase two we don't want ivy making HTTP requests. NO PRECOMMIT TESTS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4690) stats_partscan_1.q makes different result with different hadhoop.mr.rev
[
https://issues.apache.org/jira/browse/HIVE-4690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13766520#comment-13766520
]
Hive QA commented on HIVE-4690:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12602857/HIVE-4690.2.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 3109 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.listener.TestNotificationListener.testAMQListener
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/716/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/716/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
stats_partscan_1.q makes different result with different hadhoop.mr.rev
Key: HIVE-4690
URL: https://issues.apache.org/jira/browse/HIVE-4690
Project: Hive
Issue Type: Sub-task
Affects Versions: 0.11.0
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-4690.2.patch, HIVE-4690.D11163.1.patch
stats_partscan_1.q uses mapred.min/max.split.size and logs number of files,
which can be different with different hadoop.mr.rev.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-5291) join32_lessSize.q has ordering problem under hadoop-2
Brock Noland created HIVE-5291: -- Summary: join32_lessSize.q has ordering problem under hadoop-2 Key: HIVE-5291 URL: https://issues.apache.org/jira/browse/HIVE-5291 Project: Hive Issue Type: Sub-task Components: Tests Reporter: Brock Noland Assignee: Brock Noland Fix For: 0.13.0 Test just needs more ORDER BY and output updated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
