[jira] [Commented] (HIVE-3392) Hive unnecessarily validates table SerDes when dropping a table
[ https://issues.apache.org/jira/browse/HIVE-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452747#comment-13452747 ] Namit Jain commented on HIVE-3392: -- [~ajeshpg], I have added you as a contributor. You can assign jiras you are working on to yourself Hive unnecessarily validates table SerDes when dropping a table --- Key: HIVE-3392 URL: https://issues.apache.org/jira/browse/HIVE-3392 Project: Hive Issue Type: Bug Affects Versions: 0.9.0 Reporter: Jonathan Natkins Assignee: Ajesh Kumar Labels: patch Attachments: HIVE-3392.1.patch.txt natty@hadoop1:~$ hive hive add jar /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar; Added /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar to class path Added resource: /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar hive create table test (a int) row format serde 'hive.serde.JSONSerDe'; OK Time taken: 2.399 seconds natty@hadoop1:~$ hive hive drop table test; FAILED: Hive Internal Error: java.lang.RuntimeException(MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe hive.serde.JSONSerDe does not exist)) java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe hive.serde.JSONSerDe does not exist) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:262) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253) at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:490) at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:162) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:943) at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeDropTable(DDLSemanticAnalyzer.java:700) at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:210) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:243) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:430) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:889) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:208) Caused by: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe com.cloudera.hive.serde.JSONSerDe does not exist) at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:211) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:260) ... 20 more hive add jar /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar; Added /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar to class path Added resource: /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar hive drop table test; OK Time taken: 0.658 seconds hive -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3403) user should not specify mapjoin to perform sort-merge bucketed join
[ https://issues.apache.org/jira/browse/HIVE-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452814#comment-13452814 ] Namit Jain commented on HIVE-3403: -- This is a patch in progress. Please review. [~navis], it would be really helpful if you can review it thoroughly. Others are also welcome. I will let HIVE-3283 go in first. user should not specify mapjoin to perform sort-merge bucketed join --- Key: HIVE-3403 URL: https://issues.apache.org/jira/browse/HIVE-3403 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Currently, in order to perform a sort merge bucketed join, the user needs to set hive.optimize.bucketmapjoin.sortedmerge to true, and also specify the mapjoin hint. The user should not specify any hints. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3432) perform a map-only group by if grouping key matches the sorting properties of the table
[ https://issues.apache.org/jira/browse/HIVE-3432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3432: - Summary: perform a map-only group by if grouping key matches the sorting properties of the table (was: perform a map-only group by is grouping key matches the sorting properties of the table) perform a map-only group by if grouping key matches the sorting properties of the table --- Key: HIVE-3432 URL: https://issues.apache.org/jira/browse/HIVE-3432 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Sambavi Muthukrishnan There should be an option to use bucketizedinputformat and use map-only group by. There would be no need to perform a map-side aggregation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3152) Disallow certain character patterns in partition names
[ https://issues.apache.org/jira/browse/HIVE-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13453173#comment-13453173 ] Ivan Gorbachev commented on HIVE-3152: -- I couldn't connect the diff to this task via `arc diff --jira HIVE-3152` so provide diff link in comment - https://reviews.facebook.net/D5235. Disallow certain character patterns in partition names -- Key: HIVE-3152 URL: https://issues.apache.org/jira/browse/HIVE-3152 Project: Hive Issue Type: New Feature Components: Metastore Reporter: Andrew Poland Assignee: Andrew Poland Priority: Minor Labels: api-addition, configuration-addition New event listener to allow metastore to reject a partition name if it contains undesired character patterns such as unicode and commas. Match pattern is implemented as a regular expression Modifies append_partition to call a new MetaStorePreventListener implementation, PreAppendPartitionEvent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3152) Disallow certain character patterns in partition names
[ https://issues.apache.org/jira/browse/HIVE-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Gorbachev updated HIVE-3152: - Status: Patch Available (was: Open) Disallow certain character patterns in partition names -- Key: HIVE-3152 URL: https://issues.apache.org/jira/browse/HIVE-3152 Project: Hive Issue Type: New Feature Components: Metastore Reporter: Andrew Poland Assignee: Andrew Poland Priority: Minor Labels: api-addition, configuration-addition New event listener to allow metastore to reject a partition name if it contains undesired character patterns such as unicode and commas. Match pattern is implemented as a regular expression Modifies append_partition to call a new MetaStorePreventListener implementation, PreAppendPartitionEvent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3450) Hive maven-publish ant task should be configurable
Travis Crawford created HIVE-3450: - Summary: Hive maven-publish ant task should be configurable Key: HIVE-3450 URL: https://issues.apache.org/jira/browse/HIVE-3450 Project: Hive Issue Type: Improvement Reporter: Travis Crawford Assignee: Travis Crawford Hive has support in its build process for publishing artifacts to Maven repositories. However, the Maven repository to publish to is not configurable. Hive's build should be updated so users can specify what Maven repository to publish artifacts in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: HIVE-3217: Implement HiveDatabaseMetaData.getFunctions() to retrieve registered UDFs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7023/ --- Review request for hive and Ashutosh Chauhan. Description --- Hive JDBC support currently throws UnsupportedException when getFunctions() is called. Hive CL provides a SHOW FUNCTIONS command to return the names of all registered UDFs. By getting a SQL Statement from the connection, getFunctions can execute( SHOW FUNCTIONS) to retrieve all the registered functions (including those registered through create temporary function). This addresses bug HIVE-3217. https://issues.apache.org/jira/browse/HIVE-3217 Diffs - http://svn.apache.org/repos/asf/hive/trunk/jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveConnection.java 1355487 http://svn.apache.org/repos/asf/hive/trunk/jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveDatabaseMetaData.java 1355487 http://svn.apache.org/repos/asf/hive/trunk/jdbc/src/test/org/apache/hadoop/hive/jdbc/TestJdbcDriver.java 1355487 Diff: https://reviews.apache.org/r/7023/diff/ Testing --- ant test -Dtestcase=TestJdbcDriver passed Thanks, Richard Ding
Review Request: HIVE-3450 Hive maven-publish ant task should be configurable
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7024/ --- Review request for hive and Ashutosh Chauhan. Description --- Add Hive build support for publishing artifacts to a user-defined Maven repo. This is accomplished with: ant clean maven-build maven-publish -Dversion=0.10.0-mybuild01 -Dmvn.deploy.id=my.repo.id -Dmvn.deploy.url=http://my.maven.repo/path This feature will be useful for developers who maintain their own build and need to share those artifacts with others. This addresses bug HIVE-3450. https://issues.apache.org/jira/browse/HIVE-3450 Diffs - build.properties d2bfc30 build.xml add5a46 Diff: https://reviews.apache.org/r/7024/diff/ Testing --- Thanks, Travis Crawford
Re: Review Request: HIVE-3450 Hive maven-publish ant task should be configurable
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7024/ --- (Updated Sept. 11, 2012, 5:21 p.m.) Review request for hive, Ashutosh Chauhan, Dmitriy Ryaboy, and Feng Peng. Description --- Add Hive build support for publishing artifacts to a user-defined Maven repo. This is accomplished with: ant clean maven-build maven-publish -Dversion=0.10.0-mybuild01 -Dmvn.deploy.id=my.repo.id -Dmvn.deploy.url=http://my.maven.repo/path This feature will be useful for developers who maintain their own build and need to share those artifacts with others. This addresses bug HIVE-3450. https://issues.apache.org/jira/browse/HIVE-3450 Diffs - build.properties d2bfc30 build.xml add5a46 Diff: https://reviews.apache.org/r/7024/diff/ Testing --- Thanks, Travis Crawford
[jira] [Updated] (HIVE-3152) Disallow certain character patterns in partition names
[ https://issues.apache.org/jira/browse/HIVE-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Gorbachev updated HIVE-3152: - Attachment: unicode.patch Disallow certain character patterns in partition names -- Key: HIVE-3152 URL: https://issues.apache.org/jira/browse/HIVE-3152 Project: Hive Issue Type: New Feature Components: Metastore Reporter: Andrew Poland Assignee: Andrew Poland Priority: Minor Labels: api-addition, configuration-addition Attachments: unicode.patch New event listener to allow metastore to reject a partition name if it contains undesired character patterns such as unicode and commas. Match pattern is implemented as a regular expression Modifies append_partition to call a new MetaStorePreventListener implementation, PreAppendPartitionEvent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-3450 Hive maven-publish ant task should be configurable
On Sept. 11, 2012, 5:47 p.m., Feng Peng wrote: build.xml, line 23 https://reviews.apache.org/r/7024/diff/1/?file=152640#file152640line23 Any reason for this change? I changed the urn lines after reading http://maven.apache.org/ant-tasks/installation.html The following example is shown for how to setup these tasks. project ... xmlns:artifact=antlib:org.apache.maven.artifact.ant ... path id=maven-ant-tasks.classpath path=lib/maven-ant-tasks-2.1.3.jar / typedef resource=org/apache/maven/artifact/ant/antlib.xml uri=antlib:org.apache.maven.artifact.ant classpathref=maven-ant-tasks.classpath / ... /project However, I reverted the urn changes and was able to publish to a user-defined repo, so I'll update the patch to remove these changed lines. - Travis --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7024/#review11340 --- On Sept. 11, 2012, 5:21 p.m., Travis Crawford wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7024/ --- (Updated Sept. 11, 2012, 5:21 p.m.) Review request for hive, Ashutosh Chauhan, Dmitriy Ryaboy, and Feng Peng. Description --- Add Hive build support for publishing artifacts to a user-defined Maven repo. This is accomplished with: ant clean maven-build maven-publish -Dversion=0.10.0-mybuild01 -Dmvn.deploy.id=my.repo.id -Dmvn.deploy.url=http://my.maven.repo/path This feature will be useful for developers who maintain their own build and need to share those artifacts with others. This addresses bug HIVE-3450. https://issues.apache.org/jira/browse/HIVE-3450 Diffs - build.properties d2bfc30 build.xml add5a46 Diff: https://reviews.apache.org/r/7024/diff/ Testing --- Thanks, Travis Crawford
Re: Review Request: HIVE-3450 Hive maven-publish ant task should be configurable
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7024/ --- (Updated Sept. 11, 2012, 6:30 p.m.) Review request for hive, Ashutosh Chauhan, Dmitriy Ryaboy, and Feng Peng. Changes --- This updated diff reverts the maven-ant-task initialization changes that were made to match the docs. Testing shows the current configuration works too, so we'll leave it as-is. Description --- Add Hive build support for publishing artifacts to a user-defined Maven repo. This is accomplished with: ant clean maven-build maven-publish -Dversion=0.10.0-mybuild01 -Dmvn.deploy.id=my.repo.id -Dmvn.deploy.url=http://my.maven.repo/path This feature will be useful for developers who maintain their own build and need to share those artifacts with others. This addresses bug HIVE-3450. https://issues.apache.org/jira/browse/HIVE-3450 Diffs (updated) - build.properties d2bfc30 build.xml add5a46 Diff: https://reviews.apache.org/r/7024/diff/ Testing --- Thanks, Travis Crawford
[jira] [Updated] (HIVE-3450) Hive maven-publish ant task should be configurable
[ https://issues.apache.org/jira/browse/HIVE-3450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Travis Crawford updated HIVE-3450: -- Attachment: HIVE-3450_mvn_deploy_configure.2.patch This updated diff reverts the maven-ant-task initialization changes that were made to match the docs. Testing shows the current configuration works too, so we'll leave it as-is. Hive maven-publish ant task should be configurable -- Key: HIVE-3450 URL: https://issues.apache.org/jira/browse/HIVE-3450 Project: Hive Issue Type: Improvement Reporter: Travis Crawford Assignee: Travis Crawford Attachments: HIVE-3450_mvn_deploy_configure.1.patch, HIVE-3450_mvn_deploy_configure.2.patch Hive has support in its build process for publishing artifacts to Maven repositories. However, the Maven repository to publish to is not configurable. Hive's build should be updated so users can specify what Maven repository to publish artifacts in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3283) bucket information should be used from the partition instead of the table
[ https://issues.apache.org/jira/browse/HIVE-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-3283: Attachment: HIVE-3283.2.patch.txt bucket information should be used from the partition instead of the table - Key: HIVE-3283 URL: https://issues.apache.org/jira/browse/HIVE-3283 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Namit Jain Assignee: Kevin Wilfong Attachments: HIVE-3283.1.patch.txt, HIVE-3283.2.patch.txt Currently Hive uses the number of buckets from the table object. Ideally, the number of buckets from the partition should be used -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3283) bucket information should be used from the partition instead of the table
[ https://issues.apache.org/jira/browse/HIVE-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-3283: Status: Patch Available (was: Open) bucket information should be used from the partition instead of the table - Key: HIVE-3283 URL: https://issues.apache.org/jira/browse/HIVE-3283 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Namit Jain Assignee: Kevin Wilfong Attachments: HIVE-3283.1.patch.txt, HIVE-3283.2.patch.txt Currently Hive uses the number of buckets from the table object. Ideally, the number of buckets from the partition should be used -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3339) Change the rules in SemanticAnalyzer to use Operator.getName() instead of hardcoded names
[ https://issues.apache.org/jira/browse/HIVE-3339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenxiao Luo updated HIVE-3339: --- Attachment: HIVE-3339.1.patch.txt Change the rules in SemanticAnalyzer to use Operator.getName() instead of hardcoded names - Key: HIVE-3339 URL: https://issues.apache.org/jira/browse/HIVE-3339 Project: Hive Issue Type: Bug Reporter: Namit Jain Priority: Minor Attachments: HIVE-3339.1.patch.txt This should be done for code cleanup. Instead of the rule being: SEL% It should say SelectOperator.getName()% It would make the rules more readable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
run a test case for hadoop 23
Hello all, I am looking for help and thank first. How to run a test case for hadoop 23 only? Exact command? I tried the following and it didn't work: ant ... -Dhadoop.version=0.23.1 -Dhadoop-0.23.version=0.23.1 -Dhadoop.mr.rev=23 It still gets 0.20 as hadoop version. Thanks Tim
Re: Review Request: HIVE-3450 Hive maven-publish ant task should be configurable
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7024/#review11355 --- lgtm. non-committer +1 - Feng Peng On Sept. 11, 2012, 6:30 p.m., Travis Crawford wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7024/ --- (Updated Sept. 11, 2012, 6:30 p.m.) Review request for hive, Ashutosh Chauhan, Dmitriy Ryaboy, and Feng Peng. Description --- Add Hive build support for publishing artifacts to a user-defined Maven repo. This is accomplished with: ant clean maven-build maven-publish -Dversion=0.10.0-mybuild01 -Dmvn.deploy.id=my.repo.id -Dmvn.deploy.url=http://my.maven.repo/path This feature will be useful for developers who maintain their own build and need to share those artifacts with others. This addresses bug HIVE-3450. https://issues.apache.org/jira/browse/HIVE-3450 Diffs - build.properties d2bfc30 build.xml add5a46 Diff: https://reviews.apache.org/r/7024/diff/ Testing --- Thanks, Travis Crawford
Review Request: Modify TypeInfoUtils.java to tolerate empty struct in Thrift.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7042/ --- Review request for hive, Ashutosh Chauhan, Dmitriy Ryaboy, and Travis Crawford. Description --- Right now TypeInfoUtils expects at least one field in a STRUCT, which is not always true, e.g., empty struct is allowed in Thrift. Modified TypeInfoUtils so that empty struct can be correctly processed. This addresses bug https://issues.apache.org/jira/browse/HIVE-3398. https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/HIVE-3398 Diffs - serde/if/test/complex.thrift 308b64c serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/Complex.java f5a2986 serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/EmptyStruct.java PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils.java 6c07ab5 serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestThriftObjectInspectors.java 5f692fb serde/src/test/org/apache/hadoop/hive/serde2/thrift_test/CreateSequenceFile.java 8aef773 Diff: https://reviews.apache.org/r/7042/diff/ Testing --- Thanks, Feng Peng
Phabricator login problem
I tried to log in to Phabricator using my GitHub account, but after entering user name and password correctly, I'm landed on this page https://reviews.facebook.net/oauth/github/login/?code=c440e14489ef15596a83state= which reads: GitHub Auth Failed Unable to authenticate with GitHub. There are several reasons this might happen: * Phabricator may be configured with the wrong Application Secret; or * the GitHub OAuth access token may have expired; or * GitHub may have revoked authorization for the Application; or * GitHub may be having technical problems. You can try again, or login using another method.
Re: Review Request: Modify TypeInfoUtils.java to tolerate empty struct in Thrift.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7042/ --- (Updated Sept. 11, 2012, 9:19 p.m.) Review request for hive, Ashutosh Chauhan, Dmitriy Ryaboy, and Travis Crawford. Changes --- Fix the Jira link. Description --- Right now TypeInfoUtils expects at least one field in a STRUCT, which is not always true, e.g., empty struct is allowed in Thrift. Modified TypeInfoUtils so that empty struct can be correctly processed. This addresses bug HIVE-3398. https://issues.apache.org/jira/browse/HIVE-3398 Diffs - serde/if/test/complex.thrift 308b64c serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/Complex.java f5a2986 serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/EmptyStruct.java PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils.java 6c07ab5 serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestThriftObjectInspectors.java 5f692fb serde/src/test/org/apache/hadoop/hive/serde2/thrift_test/CreateSequenceFile.java 8aef773 Diff: https://reviews.apache.org/r/7042/diff/ Testing --- Thanks, Feng Peng
Re: run a test case for hadoop 23
You might try modifying the build.properties file as follows:
hadoop.version=${hadoop-0.23.version}
hadoop.security.version=${hadoop-0.23.version}
hadoop.mr.rev=23
This works for me.
chris
On 9/11/12 1:59 PM, Gang Tim Liu g...@fb.com wrote:
Hello all,
I am looking for help and thank first.
How to run a test case for hadoop 23 only? Exact command?
I tried the following and it didn't work:
ant ... -Dhadoop.version=0.23.1 -Dhadoop-0.23.version=0.23.1
-Dhadoop.mr.rev=23
It still gets 0.20 as hadoop version.
Thanks
Tim
[jira] [Commented] (HIVE-967) Implement show create table
[
https://issues.apache.org/jira/browse/HIVE-967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13453441#comment-13453441
]
Steven Wong commented on HIVE-967:
--
I'm having trouble with Fabricator login, so here's my comment on a piece of
the code:
bq. {code}String createTab_stmt = CREATE VIEW + tableName + AS +
tbl.getViewOriginalText();{code}
Shouldn't getViewExpandedText be used instead? Because views are supposed to be
frozen at the time the view is created.
Implement show create table
-
Key: HIVE-967
URL: https://issues.apache.org/jira/browse/HIVE-967
Project: Hive
Issue Type: New Feature
Components: Metastore, Query Processor
Reporter: Adam Kramer
Assignee: Feng Lu
Attachments: HIVE-967.2.patch.txt, HIVE-967.3.patch.txt,
HIVE-967.4.patch.txt, HIVE-967.5.patch.txt, HIVE-967.patch.txt,
HiveShowCreateTable.jar, show_create.txt
SHOW CREATE TABLE would be very useful in cases where you are trying to
figure out the partitioning and/or bucketing scheme for a table. Perhaps this
could be implemented by having new tables automatically SET PROPERTIES
(create_command='raw text of the create statement')?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: run a test case for hadoop 23
Not working.
I must miss something.
What command do you use after changing it?
thanks
On 9/11/12 2:31 PM, Chris Drome cdr...@yahoo-inc.com wrote:
You might try modifying the build.properties file as follows:
hadoop.version=${hadoop-0.23.version}
hadoop.security.version=${hadoop-0.23.version}
hadoop.mr.rev=23
This works for me.
chris
On 9/11/12 1:59 PM, Gang Tim Liu g...@fb.com wrote:
Hello all,
I am looking for help and thank first.
How to run a test case for hadoop 23 only? Exact command?
I tried the following and it didn't work:
ant ... -Dhadoop.version=0.23.1 -Dhadoop-0.23.version=0.23.1
-Dhadoop.mr.rev=23
It still gets 0.20 as hadoop version.
Thanks
Tim
Re: run a test case for hadoop 23
You can try cleaning and rebuilding:
ant clean package
Then you can try running your tests:
ant test
How do you know that it is using hadoop 20?
chris
On 9/11/12 3:57 PM, Gang Tim Liu g...@fb.com wrote:
Not working.
I must miss something.
What command do you use after changing it?
thanks
On 9/11/12 2:31 PM, Chris Drome cdr...@yahoo-inc.com wrote:
You might try modifying the build.properties file as follows:
hadoop.version=${hadoop-0.23.version}
hadoop.security.version=${hadoop-0.23.version}
hadoop.mr.rev=23
This works for me.
chris
On 9/11/12 1:59 PM, Gang Tim Liu g...@fb.com wrote:
Hello all,
I am looking for help and thank first.
How to run a test case for hadoop 23 only? Exact command?
I tried the following and it didn't work:
ant ... -Dhadoop.version=0.23.1 -Dhadoop-0.23.version=0.23.1
-Dhadoop.mr.rev=23
It still gets 0.20 as hadoop version.
Thanks
Tim
Re: run a test case for hadoop 23
I am running test in eclipse and it is printing
INCLUDE list does not contain Hadoop Version 0.20. SkippingŠ
It's from QTestUtil.java due to hadoopversion 0.20
Thanks
On 9/11/12 4:04 PM, Chris Drome cdr...@yahoo-inc.com wrote:
You can try cleaning and rebuilding:
ant clean package
Then you can try running your tests:
ant test
How do you know that it is using hadoop 20?
chris
On 9/11/12 3:57 PM, Gang Tim Liu g...@fb.com wrote:
Not working.
I must miss something.
What command do you use after changing it?
thanks
On 9/11/12 2:31 PM, Chris Drome cdr...@yahoo-inc.com wrote:
You might try modifying the build.properties file as follows:
hadoop.version=${hadoop-0.23.version}
hadoop.security.version=${hadoop-0.23.version}
hadoop.mr.rev=23
This works for me.
chris
On 9/11/12 1:59 PM, Gang Tim Liu g...@fb.com wrote:
Hello all,
I am looking for help and thank first.
How to run a test case for hadoop 23 only? Exact command?
I tried the following and it didn't work:
ant ... -Dhadoop.version=0.23.1 -Dhadoop-0.23.version=0.23.1
-Dhadoop.mr.rev=23
It still gets 0.20 as hadoop version.
Thanks
Tim
Re: run a test case for hadoop 23
This one will work and it doesn't need change in build.properties. Details
in HIVE-3389.
ant test -Dhadoop.mr.rev=23 -Dtest.print.classpath=true
-Dhadoop.version=2.0.0-alpha -Dhadoop.security.version=2.0.0-alpha
-Dtestcase=TestCliDriver -Dqfile=...q
I will update hive wiki so that people can save efforts.
Thanks
Tim
On 9/11/12 4:14 PM, Gang Tim Liu g...@fb.com wrote:
I am running test in eclipse and it is printing
INCLUDE list does not contain Hadoop Version 0.20. SkippingŠ
It's from QTestUtil.java due to hadoopversion 0.20
Thanks
On 9/11/12 4:04 PM, Chris Drome cdr...@yahoo-inc.com wrote:
You can try cleaning and rebuilding:
ant clean package
Then you can try running your tests:
ant test
How do you know that it is using hadoop 20?
chris
On 9/11/12 3:57 PM, Gang Tim Liu g...@fb.com wrote:
Not working.
I must miss something.
What command do you use after changing it?
thanks
On 9/11/12 2:31 PM, Chris Drome cdr...@yahoo-inc.com wrote:
You might try modifying the build.properties file as follows:
hadoop.version=${hadoop-0.23.version}
hadoop.security.version=${hadoop-0.23.version}
hadoop.mr.rev=23
This works for me.
chris
On 9/11/12 1:59 PM, Gang Tim Liu g...@fb.com wrote:
Hello all,
I am looking for help and thank first.
How to run a test case for hadoop 23 only? Exact command?
I tried the following and it didn't work:
ant ... -Dhadoop.version=0.23.1 -Dhadoop-0.23.version=0.23.1
-Dhadoop.mr.rev=23
It still gets 0.20 as hadoop version.
Thanks
Tim
Hive-trunk-h0.21 - Build # 1663 - Still Failing
Changes for Build #1638 [namit] HIVE-3393 get_json_object and json_tuple should use Jackson library (Kevin Wilfong via namit) Changes for Build #1639 Changes for Build #1640 [ecapriolo] HIVE-3068 Export table metadata as JSON on table drop (Andrew Chalfant via egc) Changes for Build #1641 Changes for Build #1642 [hashutosh] HIVE-3338 : Archives broken for hadoop 1.0 (Vikram Dixit via Ashutosh Chauhan) Changes for Build #1643 Changes for Build #1644 Changes for Build #1645 [cws] HIVE-3413. Fix pdk.PluginTest on hadoop23 (Zhenxiao Luo via cws) Changes for Build #1646 [cws] HIVE-3056. Ability to bulk update location field in Db/Table/Partition records (Shreepadma Venugopalan via cws) [cws] HIVE-3416 [jira] Fix TestAvroSerdeUtils.determineSchemaCanReadSchemaFromHDFS when running Hive on hadoop23 (Zhenxiao Luo via Carl Steinbach) Summary: HIVE-3416: Fix TestAvroSerdeUtils.determineSchemaCanReadSchemaFromHDFS when running Hive on hadoop23 TestAvroSerdeUtils determinSchemaCanReadSchemaFromHDFS is failing when running hive on hadoop23: $ant very-clean package -Dhadoop.version=0.23.1 -Dhadoop-0.23.version=0.23.1 -Dhadoop.mr.rev=23 $ant test -Dhadoop.version=0.23.1 -Dhadoop-0.23.version=0.23.1 -Dhadoop.mr.rev=23 -Dtestcase=TestAvroSerdeUtils testcase classname=org.apache.hadoop.hive.serde2.avro.TestAvroSerdeUtils name=determineSchemaCanReadSchemaFromHDFS time=0.21 error message=org/apache/hadoop/net/StaticMapping type=java.lang.NoClassDefFoundErrorjava.lang.NoClassDefFoundError: org/apache/hadoop/net/StaticMapping at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:534) at org.apache.hadoop.hdfs.MiniDFSCluster.init(MiniDFSCluster.java:489) at org.apache.hadoop.hdfs.MiniDFSCluster.init(MiniDFSCluster.java:360) at org.apache.hadoop.hive.serde2.avro.TestAvroSerdeUtils.determineSchemaCanReadSchemaFromHDFS(TestAvroSerdeUtils.java:187) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.net.StaticMapping at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) ... 25 more /error /testcase Test Plan: EMPTY Reviewers: JIRA Differential Revision: https://reviews.facebook.net/D5025 [cws] HIVE-3424. Error by upgrading a Hive 0.7.0 database to 0.8.0 (008-HIVE-2246.mysql.sql) (Alexander Alten-Lorenz via cws) [cws] HIVE-3412. Fix TestCliDriver.repair on Hadoop 0.23.3, 3.0.0, and 2.2.0-alpha (Zhenxiao Luo via cws) Changes for Build #1647 Changes for Build #1648 [namit] HIVE-3429 Bucket map join involving table with more than 1 partition column causes FileNotFoundException (Kevin Wilfong via namit) Changes for Build #1649 [hashutosh] HIVE-3075 : Improve HiveMetaStore logging (Travis Crawford via Ashutosh Chauhan) Changes for Build #1650 [hashutosh] HIVE-3340 : shims unit test failures fails further test progress (Giridharan Kesavan via Ashutosh Chauhan) Changes for Build #1651 [hashutosh]
Re: run a test case for hadoop 23
Hi,
The following seems working to build/test hadoop23:
$ant very-clean package -Dhadoop.version=0.23.1
-Dhadoop-0.23.version=0.23.1 -Dhadoop.mr.rev=23
$ant test -Dhadoop.version=0.23.1 -Dhadoop-0.23.version=0.23.1
-Dhadoop.mr.rev=23
Thanks,
Zhenxiao
On Tue, Sep 11, 2012 at 4:51 PM, Gang Tim Liu g...@fb.com wrote:
This one will work and it doesn't need change in build.properties. Details
in HIVE-3389.
ant test -Dhadoop.mr.rev=23 -Dtest.print.classpath=true
-Dhadoop.version=2.0.0-alpha -Dhadoop.security.version=2.0.0-alpha
-Dtestcase=TestCliDriver -Dqfile=...q
I will update hive wiki so that people can save efforts.
Thanks
Tim
On 9/11/12 4:14 PM, Gang Tim Liu g...@fb.com wrote:
I am running test in eclipse and it is printing
INCLUDE list does not contain Hadoop Version 0.20. SkippingŠ
It's from QTestUtil.java due to hadoopversion 0.20
Thanks
On 9/11/12 4:04 PM, Chris Drome cdr...@yahoo-inc.com wrote:
You can try cleaning and rebuilding:
ant clean package
Then you can try running your tests:
ant test
How do you know that it is using hadoop 20?
chris
On 9/11/12 3:57 PM, Gang Tim Liu g...@fb.com wrote:
Not working.
I must miss something.
What command do you use after changing it?
thanks
On 9/11/12 2:31 PM, Chris Drome cdr...@yahoo-inc.com wrote:
You might try modifying the build.properties file as follows:
hadoop.version=${hadoop-0.23.version}
hadoop.security.version=${hadoop-0.23.version}
hadoop.mr.rev=23
This works for me.
chris
On 9/11/12 1:59 PM, Gang Tim Liu g...@fb.com wrote:
Hello all,
I am looking for help and thank first.
How to run a test case for hadoop 23 only? Exact command?
I tried the following and it didn't work:
ant ... -Dhadoop.version=0.23.1 -Dhadoop-0.23.version=0.23.1
-Dhadoop.mr.rev=23
It still gets 0.20 as hadoop version.
Thanks
Tim
[jira] [Updated] (HIVE-3428) Fix log4j configuration errors when running hive on hadoop23
[ https://issues.apache.org/jira/browse/HIVE-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenxiao Luo updated HIVE-3428: --- Attachment: HIVE-3428.3.patch.txt Fix log4j configuration errors when running hive on hadoop23 Key: HIVE-3428 URL: https://issues.apache.org/jira/browse/HIVE-3428 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Zhenxiao Luo Assignee: Zhenxiao Luo Attachments: HIVE-3428.1.patch.txt, HIVE-3428.2.patch.txt, HIVE-3428.3.patch.txt There are log4j configuration errors when running hive on hadoop23, some of them may fail testcases, since the following log4j error message could printed to console, or to output file, which diffs from the expected output: [junit] log4j:ERROR Could not find value for key log4j.appender.NullAppender [junit] log4j:ERROR Could not instantiate appender named NullAppender. [junit] 12/09/04 11:34:42 WARN conf.HiveConf: hive-site.xml not found on CLASSPATH -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3428) Fix log4j configuration errors when running hive on hadoop23
[ https://issues.apache.org/jira/browse/HIVE-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenxiao Luo updated HIVE-3428: --- Status: Patch Available (was: Open) Fix log4j configuration errors when running hive on hadoop23 Key: HIVE-3428 URL: https://issues.apache.org/jira/browse/HIVE-3428 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Zhenxiao Luo Assignee: Zhenxiao Luo Attachments: HIVE-3428.1.patch.txt, HIVE-3428.2.patch.txt, HIVE-3428.3.patch.txt There are log4j configuration errors when running hive on hadoop23, some of them may fail testcases, since the following log4j error message could printed to console, or to output file, which diffs from the expected output: [junit] log4j:ERROR Could not find value for key log4j.appender.NullAppender [junit] log4j:ERROR Could not instantiate appender named NullAppender. [junit] 12/09/04 11:34:42 WARN conf.HiveConf: hive-site.xml not found on CLASSPATH -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3428) Fix log4j configuration errors when running hive on hadoop23
[ https://issues.apache.org/jira/browse/HIVE-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13453659#comment-13453659 ] Zhenxiao Luo commented on HIVE-3428: Updated patch, review request submitted at: https://reviews.facebook.net/D5133 Fix log4j configuration errors when running hive on hadoop23 Key: HIVE-3428 URL: https://issues.apache.org/jira/browse/HIVE-3428 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Zhenxiao Luo Assignee: Zhenxiao Luo Attachments: HIVE-3428.1.patch.txt, HIVE-3428.2.patch.txt, HIVE-3428.3.patch.txt There are log4j configuration errors when running hive on hadoop23, some of them may fail testcases, since the following log4j error message could printed to console, or to output file, which diffs from the expected output: [junit] log4j:ERROR Could not find value for key log4j.appender.NullAppender [junit] log4j:ERROR Could not instantiate appender named NullAppender. [junit] 12/09/04 11:34:42 WARN conf.HiveConf: hive-site.xml not found on CLASSPATH -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-3392 - Review request
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7004/ --- (Updated Sept. 12, 2012, 3:20 a.m.) Review request for hive. Changes --- Changing code per review comments. Description --- Added logic to skip the table schema validation if the command is a drop command. This addresses bug HIVE-3392. https://issues.apache.org/jira/browse/HIVE-3392 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 271554c ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java b4ee673 Diff: https://reviews.apache.org/r/7004/diff/ Testing --- Able to drop table even if the user defined SerDe jar is not added to classpath. Thanks, Ajesh kumar
[jira] [Commented] (HIVE-3440) Fix pdk PluginTest failing on trunk-h0.21
[ https://issues.apache.org/jira/browse/HIVE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13453680#comment-13453680 ] Kevin Wilfong commented on HIVE-3440: - +1 I ran ant very-clean package test after applying the patch, and did not get this exception (just running ant test directly still gives the exception) Thanks Zhenxiao, I will commit once the tests finish. Fix pdk PluginTest failing on trunk-h0.21 - Key: HIVE-3440 URL: https://issues.apache.org/jira/browse/HIVE-3440 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Zhenxiao Luo Assignee: Zhenxiao Luo Fix For: 0.10.0 Attachments: HIVE-3440.1.patch.txt Get the failure when running on hadoop21, triggered directly from pdk(when triggered from builtin, pdk test is passed). Here is the execution log: 2012-09-06 13:46:05,646 WARN mapred.LocalJobRunner (LocalJobRunner.java:run(256)) - job_local_0001 java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ... 5 more Caused by: java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) ... 10 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ... 13 more Caused by: java.lang.RuntimeException: Map operator initialization failed at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121) ... 18 more Caused by: java.lang.NoClassDefFoundError: org/codehaus/jackson/map/ObjectMapper at org.apache.hadoop.hive.ql.udf.generic.GenericUDTFJSONTuple.clinit(GenericUDTFJSONTuple.java:54) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:532) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.registerGenericUDTF(FunctionRegistry.java:545) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.registerGenericUDTF(FunctionRegistry.java:539) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.clinit(FunctionRegistry.java:472) at org.apache.hadoop.hive.ql.exec.DefaultUDFMethodResolver.getEvalMethod(DefaultUDFMethodResolver.java:59) at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:154) at org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:98) at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:137) at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:898) at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:924) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:358) at
[jira] [Updated] (HIVE-3339) Change the rules in SemanticAnalyzer to use Operator.getName() instead of hardcoded names
[ https://issues.apache.org/jira/browse/HIVE-3339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3339: - Status: Open (was: Patch Available) comments on phabricator Change the rules in SemanticAnalyzer to use Operator.getName() instead of hardcoded names - Key: HIVE-3339 URL: https://issues.apache.org/jira/browse/HIVE-3339 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Zhenxiao Luo Priority: Minor Attachments: HIVE-3339.1.patch.txt This should be done for code cleanup. Instead of the rule being: SEL% It should say SelectOperator.getName()% It would make the rules more readable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3283) bucket information should be used from the partition instead of the table
[ https://issues.apache.org/jira/browse/HIVE-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3283: - Status: Open (was: Patch Available) comments on phabricator bucket information should be used from the partition instead of the table - Key: HIVE-3283 URL: https://issues.apache.org/jira/browse/HIVE-3283 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Namit Jain Assignee: Kevin Wilfong Attachments: HIVE-3283.1.patch.txt, HIVE-3283.2.patch.txt Currently Hive uses the number of buckets from the table object. Ideally, the number of buckets from the partition should be used -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-3392 - Review request
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7004/#review11384 --- Please add a new testcase - There is no easy way to add a .q file with the semantics you want. Can you try the following - I am not sure it will work add jar foo.jar create table T using foo.jar; !mv bar.jar foo.jar add jar foo.jar again drop table T Can you think of a way to add a testcase for this, if the above does not work ? There is 'reset' command - I am not sure if it removes the jars also. Can you confirm ? ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java https://reviews.apache.org/r/7004/#comment24425 space after if - namit jain On Sept. 12, 2012, 3:20 a.m., Ajesh kumar wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7004/ --- (Updated Sept. 12, 2012, 3:20 a.m.) Review request for hive. Description --- Added logic to skip the table schema validation if the command is a drop command. This addresses bug HIVE-3392. https://issues.apache.org/jira/browse/HIVE-3392 Diffs - ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 271554c ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java b4ee673 Diff: https://reviews.apache.org/r/7004/diff/ Testing --- Able to drop table even if the user defined SerDe jar is not added to classpath. Thanks, Ajesh kumar
[jira] [Commented] (HIVE-3392) Hive unnecessarily validates table SerDes when dropping a table
[ https://issues.apache.org/jira/browse/HIVE-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13453714#comment-13453714 ] Namit Jain commented on HIVE-3392: -- comments [~ajeshpg], please mark 'submit patch' when you are done with the changes and want others to review Hive unnecessarily validates table SerDes when dropping a table --- Key: HIVE-3392 URL: https://issues.apache.org/jira/browse/HIVE-3392 Project: Hive Issue Type: Bug Affects Versions: 0.9.0 Reporter: Jonathan Natkins Assignee: Ajesh Kumar Labels: patch Attachments: HIVE-3392.1.patch.txt natty@hadoop1:~$ hive hive add jar /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar; Added /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar to class path Added resource: /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar hive create table test (a int) row format serde 'hive.serde.JSONSerDe'; OK Time taken: 2.399 seconds natty@hadoop1:~$ hive hive drop table test; FAILED: Hive Internal Error: java.lang.RuntimeException(MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe hive.serde.JSONSerDe does not exist)) java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe hive.serde.JSONSerDe does not exist) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:262) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253) at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:490) at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:162) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:943) at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeDropTable(DDLSemanticAnalyzer.java:700) at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:210) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:243) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:430) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:889) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:208) Caused by: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe com.cloudera.hive.serde.JSONSerDe does not exist) at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:211) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:260) ... 20 more hive add jar /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar; Added /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar to class path Added resource: /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar hive drop table test; OK Time taken: 0.658 seconds hive -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3392) Hive unnecessarily validates table SerDes when dropping a table
[ https://issues.apache.org/jira/browse/HIVE-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13453715#comment-13453715 ] Edward Capriolo commented on HIVE-3392: --- I have an issue with this. Serde's and storage handlers could have custom drop logic. If we allow a drop without the class how do we know the right think was done? Hive unnecessarily validates table SerDes when dropping a table --- Key: HIVE-3392 URL: https://issues.apache.org/jira/browse/HIVE-3392 Project: Hive Issue Type: Bug Affects Versions: 0.9.0 Reporter: Jonathan Natkins Assignee: Ajesh Kumar Labels: patch Attachments: HIVE-3392.1.patch.txt natty@hadoop1:~$ hive hive add jar /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar; Added /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar to class path Added resource: /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar hive create table test (a int) row format serde 'hive.serde.JSONSerDe'; OK Time taken: 2.399 seconds natty@hadoop1:~$ hive hive drop table test; FAILED: Hive Internal Error: java.lang.RuntimeException(MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe hive.serde.JSONSerDe does not exist)) java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe hive.serde.JSONSerDe does not exist) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:262) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253) at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:490) at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:162) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:943) at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeDropTable(DDLSemanticAnalyzer.java:700) at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:210) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:243) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:430) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:889) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:208) Caused by: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe com.cloudera.hive.serde.JSONSerDe does not exist) at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:211) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:260) ... 20 more hive add jar /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar; Added /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar to class path Added resource: /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar hive drop table test; OK Time taken: 0.658 seconds hive -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-933) Infer bucketing/sorting properties
[ https://issues.apache.org/jira/browse/HIVE-933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain reassigned HIVE-933: --- Assignee: Kevin Wilfong Infer bucketing/sorting properties -- Key: HIVE-933 URL: https://issues.apache.org/jira/browse/HIVE-933 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Namit Jain Assignee: Kevin Wilfong This is a long-term plan, and may require major changes. From the query, we can figure out the sorting/bucketing properties, and change the metadata of the destination at that time. However, this means that different partitions may have different metadata. Currently, the query plan is same for all the partitions of the table - we can do the following: 1. In the first cut, have a simple approach where you take the union all metadata, and create the most defensive plan. 2. Enhance mapredWork() to include partition specific operator trees. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-3392 - Review request
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7004/ --- (Updated Sept. 12, 2012, 5:18 a.m.) Review request for hive. Changes --- Modified Code per Review comments. I will upload the testcase in https://issues.apache.org/jira/browse/HIVE-3392. please review. Description --- Added logic to skip the table schema validation if the command is a drop command. This addresses bug HIVE-3392. https://issues.apache.org/jira/browse/HIVE-3392 Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 271554c ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java b4ee673 Diff: https://reviews.apache.org/r/7004/diff/ Testing --- Able to drop table even if the user defined SerDe jar is not added to classpath. Thanks, Ajesh kumar
[jira] [Updated] (HIVE-3392) Hive unnecessarily validates table SerDes when dropping a table
[ https://issues.apache.org/jira/browse/HIVE-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajesh Kumar updated HIVE-3392: -- Attachment: HIVE-3392.Test Case - Before Patch.txt HIVE-3392.Test Case - After Patch.txt HIVE-3392.2.patch.txt Hive unnecessarily validates table SerDes when dropping a table --- Key: HIVE-3392 URL: https://issues.apache.org/jira/browse/HIVE-3392 Project: Hive Issue Type: Bug Affects Versions: 0.9.0 Reporter: Jonathan Natkins Assignee: Ajesh Kumar Labels: patch Attachments: HIVE-3392.2.patch.txt, HIVE-3392.Test Case - After Patch.txt, HIVE-3392.Test Case - Before Patch.txt natty@hadoop1:~$ hive hive add jar /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar; Added /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar to class path Added resource: /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar hive create table test (a int) row format serde 'hive.serde.JSONSerDe'; OK Time taken: 2.399 seconds natty@hadoop1:~$ hive hive drop table test; FAILED: Hive Internal Error: java.lang.RuntimeException(MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe hive.serde.JSONSerDe does not exist)) java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe hive.serde.JSONSerDe does not exist) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:262) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253) at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:490) at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:162) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:943) at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeDropTable(DDLSemanticAnalyzer.java:700) at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:210) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:243) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:430) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:889) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:208) Caused by: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe com.cloudera.hive.serde.JSONSerDe does not exist) at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:211) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:260) ... 20 more hive add jar /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar; Added /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar to class path Added resource: /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar hive drop table test; OK Time taken: 0.658 seconds hive -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3392) Hive unnecessarily validates table SerDes when dropping a table
[ https://issues.apache.org/jira/browse/HIVE-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajesh Kumar updated HIVE-3392: -- Attachment: (was: HIVE-3392.1.patch.txt) Hive unnecessarily validates table SerDes when dropping a table --- Key: HIVE-3392 URL: https://issues.apache.org/jira/browse/HIVE-3392 Project: Hive Issue Type: Bug Affects Versions: 0.9.0 Reporter: Jonathan Natkins Assignee: Ajesh Kumar Labels: patch Attachments: HIVE-3392.2.patch.txt, HIVE-3392.Test Case - After Patch.txt, HIVE-3392.Test Case - Before Patch.txt natty@hadoop1:~$ hive hive add jar /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar; Added /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar to class path Added resource: /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar hive create table test (a int) row format serde 'hive.serde.JSONSerDe'; OK Time taken: 2.399 seconds natty@hadoop1:~$ hive hive drop table test; FAILED: Hive Internal Error: java.lang.RuntimeException(MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe hive.serde.JSONSerDe does not exist)) java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe hive.serde.JSONSerDe does not exist) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:262) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253) at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:490) at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:162) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:943) at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeDropTable(DDLSemanticAnalyzer.java:700) at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:210) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:243) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:430) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:889) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:208) Caused by: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe com.cloudera.hive.serde.JSONSerDe does not exist) at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:211) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:260) ... 20 more hive add jar /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar; Added /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar to class path Added resource: /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar hive drop table test; OK Time taken: 0.658 seconds hive -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3392) Hive unnecessarily validates table SerDes when dropping a table
[ https://issues.apache.org/jira/browse/HIVE-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajesh Kumar updated HIVE-3392: -- Status: Patch Available (was: Open) New Review request added at https://reviews.apache.org/r/7004/diff/3/. As we can not use Junit/.q file for this scenario, I have added testcase in a step by step format. Please review code and Testcases. Hive unnecessarily validates table SerDes when dropping a table --- Key: HIVE-3392 URL: https://issues.apache.org/jira/browse/HIVE-3392 Project: Hive Issue Type: Bug Affects Versions: 0.9.0 Reporter: Jonathan Natkins Assignee: Ajesh Kumar Labels: patch Attachments: HIVE-3392.2.patch.txt, HIVE-3392.Test Case - After Patch.txt, HIVE-3392.Test Case - Before Patch.txt natty@hadoop1:~$ hive hive add jar /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar; Added /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar to class path Added resource: /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar hive create table test (a int) row format serde 'hive.serde.JSONSerDe'; OK Time taken: 2.399 seconds natty@hadoop1:~$ hive hive drop table test; FAILED: Hive Internal Error: java.lang.RuntimeException(MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe hive.serde.JSONSerDe does not exist)) java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe hive.serde.JSONSerDe does not exist) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:262) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253) at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:490) at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:162) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:943) at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeDropTable(DDLSemanticAnalyzer.java:700) at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:210) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:243) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:430) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:889) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:208) Caused by: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe com.cloudera.hive.serde.JSONSerDe does not exist) at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:211) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:260) ... 20 more hive add jar /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar; Added /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar to class path Added resource: /home/natty/source/sample-code/custom-serdes/target/custom-serdes-1.0-SNAPSHOT.jar hive drop table test; OK Time taken: 0.658 seconds hive -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3451) map-reduce jobs does not work for a partition containing sub-directories
Namit Jain created HIVE-3451:
Summary: map-reduce jobs does not work for a partition containing
sub-directories
Key: HIVE-3451
URL: https://issues.apache.org/jira/browse/HIVE-3451
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Namit Jain
Consider the following test:
-- The test verifies that sub-directories are supported for versions of hadoop
-- where MAPREDUCE-1501 is fixed. So, enable this test only for hadoop 23.
-- INCLUDE_HADOOP_MAJOR_VERSIONS(0.23)
CREATE TABLE fact_daily(x int) PARTITIONED BY (ds STRING);
CREATE TABLE fact_tz(x int) PARTITIONED BY (ds STRING, hr STRING)
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz';
INSERT OVERWRITE TABLE fact_tz PARTITION (ds='1', hr='1')
SELECT key+11 FROM src WHERE key=484;
ALTER TABLE fact_daily SET TBLPROPERTIES('EXTERNAL'='TRUE');
ALTER TABLE fact_daily ADD PARTITION (ds='1')
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz/ds=1';
set mapred.input.dir.recursive=true;
SELECT * FROM fact_daily WHERE ds='1';
SELECT count(1) FROM fact_daily WHERE ds='1';
Say, the above file was named: recursive_dir.q
and we ran the test for hadoop 23:
by executing:
ant test -Dhadoop.mr.rev=23 -Dtest.print.classpath=true
-Dhadoop.version=2.0.0-alpha -Dhadoop.security.version=2.0.0-alpha
-Dtestcase=TestCliDriver -Dqfile=recursive_dir.q
The select * from the table works fine, but the last command does not work
since it requires a map-reduce job.
This will prevent other features which are creating sub-directories to add
any tests which requires a map-reduce job. The work-around is to issue
queries which do not require map-reduce jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3141) Bug in SELECT query
[ https://issues.apache.org/jira/browse/HIVE-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13453736#comment-13453736 ] Ajesh Kumar commented on HIVE-3141: --- Hi Carl, when we execute 'select' query , it is not invoking SemanticAnalyzer.class. Let me know if I am missing something in my analysis. Bug in SELECT query --- Key: HIVE-3141 URL: https://issues.apache.org/jira/browse/HIVE-3141 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.9.0 Environment: OS: Ubuntu Hive version: hive-0.7.1-cdh3u2 Hadoop : hadoop-0.20.2 Reporter: ASK Priority: Minor Labels: patch Attachments: HIVE-3141.2.patch.txt, Hive_bug_3141_resolution.pdf, select_syntax.q, select_syntax.q.out When i try to execute select *(followed by any alphanumeric character) from table , query is throwing some issues. It display the result for select * It doesnot happen when only numbers follow the * -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-3141) Bug in SELECT query
[ https://issues.apache.org/jira/browse/HIVE-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain reassigned HIVE-3141: Assignee: Ajesh Kumar Bug in SELECT query --- Key: HIVE-3141 URL: https://issues.apache.org/jira/browse/HIVE-3141 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.9.0 Environment: OS: Ubuntu Hive version: hive-0.7.1-cdh3u2 Hadoop : hadoop-0.20.2 Reporter: ASK Assignee: Ajesh Kumar Priority: Minor Labels: patch Attachments: HIVE-3141.2.patch.txt, Hive_bug_3141_resolution.pdf, select_syntax.q, select_syntax.q.out When i try to execute select *(followed by any alphanumeric character) from table , query is throwing some issues. It display the result for select * It doesnot happen when only numbers follow the * -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
