[jira] Updated: (HIVE-1413) bring a table/partition offline
[ https://issues.apache.org/jira/browse/HIVE-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1413: -- Attachment: (was: HIVE-1414.1.patch) bring a table/partition offline --- Key: HIVE-1413 URL: https://issues.apache.org/jira/browse/HIVE-1413 Project: Hadoop Hive Issue Type: New Feature Components: Metastore, Query Processor Reporter: Namit Jain Assignee: Siying Dong Fix For: 0.7.0 Attachments: HIVE-1413.1.patch There should be a way to bring a table/partition offline. At that time, no read/write operations should be supported on that table. It would be very useful for housekeeping operations -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1413) bring a table/partition offline
[ https://issues.apache.org/jira/browse/HIVE-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1413: -- Attachment: HIVE-1413.2.patch 1. use parameters without making any change to metastore 2. add support of no_drop mode. Make every mode like bitmap instead of levels 3. cover the case of dropping tables bring a table/partition offline --- Key: HIVE-1413 URL: https://issues.apache.org/jira/browse/HIVE-1413 Project: Hadoop Hive Issue Type: New Feature Components: Metastore, Query Processor Reporter: Namit Jain Assignee: Siying Dong Fix For: 0.7.0 Attachments: HIVE-1413.1.patch, HIVE-1413.2.patch There should be a way to bring a table/partition offline. At that time, no read/write operations should be supported on that table. It would be very useful for housekeeping operations -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
SAXParseException on local mode?
I seem to get this error when hive decides to use local mode. If I disable it the problem is fixed: set hive.exec.mode.local.auto=false; I was running a large integration test so I'm not exactly sure which calls to make to reproduce this but perhaps someone else know's what's going on? java.lang.RuntimeException: org.xml.sax.SAXParseException: Content is not allowed in trailing section. at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1168) at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1040) at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:980) at org.apache.hadoop.conf.Configuration.get(Configuration.java:382) at org.apache.hadoop.mapred.JobConf.checkAndWarnDeprecation(JobConf.java:1662) at org.apache.hadoop.mapred.JobConf.init(JobConf.java:215) 10/08/03 15:40:36 INFO parse.ParseDriver: Parsing command: show functions loglinecleanup at org.apache.hadoop.mapred.LocalJobRunner$Job.init(LocalJobRunner.java:93) at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:373) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:800) at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730) 10/08/03 15:40:36 INFO parse.ParseDriver: Parse Completed at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:602) at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:1021) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) 10/08/03 15:40:36 INFO ql.Driver: Semantic Analysis Completed Caused by: org.xml.sax.SAXParseException: Content is not allowed in trailing section. at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:249) at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284) at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:124) at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1092) ... 16 more
[jira] Updated: (HIVE-1509) Monitor the working set of the number of files
[ https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-1509: - Attachment: HIVE-1509.patch Monitor the working set of the number of files --- Key: HIVE-1509 URL: https://issues.apache.org/jira/browse/HIVE-1509 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Reporter: Namit Jain Assignee: Ning Zhang Attachments: HIVE-1509.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1509) Monitor the working set of the number of files
[ https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-1509: - Status: Patch Available (was: Open) Monitor the working set of the number of files --- Key: HIVE-1509 URL: https://issues.apache.org/jira/browse/HIVE-1509 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Reporter: Namit Jain Assignee: Ning Zhang Attachments: HIVE-1509.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1482) Not all jdbc calls are threadsafe.
[
https://issues.apache.org/jira/browse/HIVE-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895344#action_12895344
]
John Sichi commented on HIVE-1482:
--
Yes, synchronized is the way to go.
I think the synchronization has to be at the connection level. For example,
HiveStatement also needs to make calls on the thrift interface. It's not just
DatabaseMetaData.
So we should add a new data member to HiveConnection:
Object connectionMutex = new Object();
Then pass connectionMutex to constructors of sub-objects which need to
participate in synchronization. They can then do
synchronized(connectionMutex) {
...
}
around their critical sections.
Creating a separate object for this purpose allows us to keep control over
synchronization (e.g. so it doesn't get mixed up with user-level or
thrift-level synchronization code later). We'll also need to be able to skip
synchronization in the case of asynchronous cancel, but that's a separate task.
We should also review to see if there is any client-side state which needs
protection.
Not all jdbc calls are threadsafe.
--
Key: HIVE-1482
URL: https://issues.apache.org/jira/browse/HIVE-1482
Project: Hadoop Hive
Issue Type: Bug
Components: Drivers
Affects Versions: 0.7.0
Reporter: Bennie Schut
Fix For: 0.7.0
As per jdbc spec they should be threadsafe:
http://download.oracle.com/docs/cd/E17476_01/javase/1.3/docs/guide/jdbc/spec/jdbc-spec.frame9.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1511) Hive plan serialization is slow
[ https://issues.apache.org/jira/browse/HIVE-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895343#action_12895343 ] Ning Zhang commented on HIVE-1511: -- The issue seems to be the fact that we serialize the plan by writing to HDFS file directly. We probably should cache it locally and then write it to HDFS. Hive plan serialization is slow --- Key: HIVE-1511 URL: https://issues.apache.org/jira/browse/HIVE-1511 Project: Hadoop Hive Issue Type: Improvement Affects Versions: 0.7.0 Reporter: Ning Zhang As reported by Edward Capriolo: For reference I did this as a test case SELECT * FROM src where key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR ...(100 more of these) No OOM but I gave up after the test case did not go anywhere for about 2 minutes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1511) Hive plan serialization is slow
[ https://issues.apache.org/jira/browse/HIVE-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895352#action_12895352 ] Edward Capriolo commented on HIVE-1511: --- Also possibly a clever way to remove duplicate expressions that evaluate to the same result such as multiple key=0 Hive plan serialization is slow --- Key: HIVE-1511 URL: https://issues.apache.org/jira/browse/HIVE-1511 Project: Hadoop Hive Issue Type: Improvement Affects Versions: 0.7.0 Reporter: Ning Zhang As reported by Edward Capriolo: For reference I did this as a test case SELECT * FROM src where key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR ...(100 more of these) No OOM but I gave up after the test case did not go anywhere for about 2 minutes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1509) Monitor the working set of the number of files
[ https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-1509: - Attachment: HIVE-1509.3.patch Uploading HIVE-1509.3.patch which fixed a bug. It passed hadoop 0.17 tests. I'm running 0.20 tests. Monitor the working set of the number of files --- Key: HIVE-1509 URL: https://issues.apache.org/jira/browse/HIVE-1509 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Reporter: Namit Jain Assignee: Ning Zhang Attachments: HIVE-1509.2.patch, HIVE-1509.3.patch, HIVE-1509.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
hive_hbase-handleer.jar
Hi, There: I took the latest code from the hive trunk and compiled it against cloudera CDH3 release of hbase/hadoop. I was able to get to the step that I can create an external table successfully. but when I select * from the table, the hive CLI console just hangs and nothing happens. I waited for a long time and it still goes that way. I wonder if anybody have gotten hive_hbase-handleer.jar to work. I understand that current hive_hbase-handleer.jar only works with hbase 0.20.3, I debugged the code and there is only some simple change needed for it to compile against cloudera CDH3. I wonder if there is anything I can do to figure out why the select * doesn't work. Jimmy.
[jira] Commented: (HIVE-1510) HiveCombineInputFormat should not use prefix matching to find the partitionDesc for a given path
[ https://issues.apache.org/jira/browse/HIVE-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895370#action_12895370 ] Namit Jain commented on HIVE-1510: -- Some minor comments TestHiveFileFormatUtils: 1. Use a different PartitionDesc every time instead of partDesc_1 for the partitions. 2. Spelling mistakes: forth group Otherwise, it looks good to me. Ning, can you also OK it , since we spent a lot of time debugging in the past. Also, before checking it, can you try the following 4 types of queries (with CombineHiveInputFormat): 1. hadoop 17 normal query 2. hadoop 17 sampling query 3. hadoop 20 normal query 4. hadoop 20 sampling query HiveCombineInputFormat should not use prefix matching to find the partitionDesc for a given path Key: HIVE-1510 URL: https://issues.apache.org/jira/browse/HIVE-1510 Project: Hadoop Hive Issue Type: Bug Reporter: He Yongqiang Assignee: He Yongqiang Attachments: hive-1510.1.patch set hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat; drop table combine_3_srcpart_seq_rc; create table combine_3_srcpart_seq_rc (key int , value string) partitioned by (ds string, hr string) stored as sequencefile; insert overwrite table combine_3_srcpart_seq_rc partition (ds=2010-08-03, hr=00) select * from src; alter table combine_3_srcpart_seq_rc set fileformat rcfile; insert overwrite table combine_3_srcpart_seq_rc partition (ds=2010-08-03, hr=001) select * from src; desc extended combine_3_srcpart_seq_rc partition(ds=2010-08-03, hr=00); desc extended combine_3_srcpart_seq_rc partition(ds=2010-08-03, hr=001); select * from combine_3_srcpart_seq_rc where ds=2010-08-03 order by key; drop table combine_3_srcpart_seq_rc; will fail. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1510) HiveCombineInputFormat should not use prefix matching to find the partitionDesc for a given path
[ https://issues.apache.org/jira/browse/HIVE-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895381#action_12895381 ] Ning Zhang commented on HIVE-1510: -- In HiveFileFormatUtils, removing scheme and authorities from the URI and only retain the path part may cause problem when the URI is har:// rather than hdfs://. This is one of the bugs that Paul fixed in hadoop for HAR to be able to work with CHIF. As a general comment, is it easier to just modify the pathToPartitionInfo to add a Path.SEPARATER at the end? You don't need to introduce the recursive checking and still can use the prefix matching. HiveCombineInputFormat should not use prefix matching to find the partitionDesc for a given path Key: HIVE-1510 URL: https://issues.apache.org/jira/browse/HIVE-1510 Project: Hadoop Hive Issue Type: Bug Reporter: He Yongqiang Assignee: He Yongqiang Attachments: hive-1510.1.patch set hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat; drop table combine_3_srcpart_seq_rc; create table combine_3_srcpart_seq_rc (key int , value string) partitioned by (ds string, hr string) stored as sequencefile; insert overwrite table combine_3_srcpart_seq_rc partition (ds=2010-08-03, hr=00) select * from src; alter table combine_3_srcpart_seq_rc set fileformat rcfile; insert overwrite table combine_3_srcpart_seq_rc partition (ds=2010-08-03, hr=001) select * from src; desc extended combine_3_srcpart_seq_rc partition(ds=2010-08-03, hr=00); desc extended combine_3_srcpart_seq_rc partition(ds=2010-08-03, hr=001); select * from combine_3_srcpart_seq_rc where ds=2010-08-03 order by key; drop table combine_3_srcpart_seq_rc; will fail. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1509) Monitor the working set of the number of files
[ https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895384#action_12895384 ] Joydeep Sen Sarma commented on HIVE-1509: - let me know once the tests pass 0.20 and i can commit. one more question: +MAXCREATEDFILES(hive.exec.max.created.files, 10), i think u may have to append a 'L' to 10 since u are trying to later on do a: + long upperLimit = HiveConf.getLongVar(job, HiveConf.ConfVars.MAXCREATEDFILES); (or switch to using getIntVar). i am a little surprised how this is working because the 10 would be interpreted as Integer, go to the integer constructor which should leave the long default to -1. (or i guess i have forgotten how this works) Monitor the working set of the number of files --- Key: HIVE-1509 URL: https://issues.apache.org/jira/browse/HIVE-1509 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Reporter: Namit Jain Assignee: Ning Zhang Attachments: HIVE-1509.2.patch, HIVE-1509.3.patch, HIVE-1509.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1510) HiveCombineInputFormat should not use prefix matching to find the partitionDesc for a given path
[ https://issues.apache.org/jira/browse/HIVE-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895386#action_12895386 ] He Yongqiang commented on HIVE-1510: will also test against har. even the old code also remove the scheme and authorities part when try to match partitionDesc. no offense -- the old code is not very clear, and not efficient. The new code does the same thing with simplified logic. HiveCombineInputFormat should not use prefix matching to find the partitionDesc for a given path Key: HIVE-1510 URL: https://issues.apache.org/jira/browse/HIVE-1510 Project: Hadoop Hive Issue Type: Bug Reporter: He Yongqiang Assignee: He Yongqiang Attachments: hive-1510.1.patch set hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat; drop table combine_3_srcpart_seq_rc; create table combine_3_srcpart_seq_rc (key int , value string) partitioned by (ds string, hr string) stored as sequencefile; insert overwrite table combine_3_srcpart_seq_rc partition (ds=2010-08-03, hr=00) select * from src; alter table combine_3_srcpart_seq_rc set fileformat rcfile; insert overwrite table combine_3_srcpart_seq_rc partition (ds=2010-08-03, hr=001) select * from src; desc extended combine_3_srcpart_seq_rc partition(ds=2010-08-03, hr=00); desc extended combine_3_srcpart_seq_rc partition(ds=2010-08-03, hr=001); select * from combine_3_srcpart_seq_rc where ds=2010-08-03 order by key; drop table combine_3_srcpart_seq_rc; will fail. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: hive_hbase-handleer.jar
Under the source for hive/hbase-handler/lib, delete the hbase-0.20.3 jars there, replace them with the jars matching your HBase version from CDH3, and then run ant package in hive. JVS On Aug 4, 2010, at 12:03 PM, Jinsong Hu wrote: Hi, There: I took the latest code from the hive trunk and compiled it against cloudera CDH3 release of hbase/hadoop. I was able to get to the step that I can create an external table successfully. but when I select * from the table, the hive CLI console just hangs and nothing happens. I waited for a long time and it still goes that way. I wonder if anybody have gotten hive_hbase-handleer.jar to work. I understand that current hive_hbase-handleer.jar only works with hbase 0.20.3, I debugged the code and there is only some simple change needed for it to compile against cloudera CDH3. I wonder if there is anything I can do to figure out why the select * doesn't work. Jimmy.
[jira] Updated: (HIVE-1509) Monitor the working set of the number of files
[ https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-1509: - Attachment: HIVE-1509.4.patch Attaching HIVE-1509.4.patch to change 10 to 10L. I think it passed because we have the parameter set up in hive-default.xml as well. All unit tests passed on 0.20 except index_compact_2.q. It is strange that this particular test passed when I run it individually. It also passed when I ran the whole tests again. Monitor the working set of the number of files --- Key: HIVE-1509 URL: https://issues.apache.org/jira/browse/HIVE-1509 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Reporter: Namit Jain Assignee: Ning Zhang Attachments: HIVE-1509.2.patch, HIVE-1509.3.patch, HIVE-1509.4.patch, HIVE-1509.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1509) Monitor the working set of the number of files
[ https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895411#action_12895411 ] Joydeep Sen Sarma commented on HIVE-1509: - ok - i will run tests on 20 and commit if all clear. Monitor the working set of the number of files --- Key: HIVE-1509 URL: https://issues.apache.org/jira/browse/HIVE-1509 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Reporter: Namit Jain Assignee: Ning Zhang Attachments: HIVE-1509.2.patch, HIVE-1509.3.patch, HIVE-1509.4.patch, HIVE-1509.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Review Request: HIVE-675: Add database/scheme support Hive QL
--- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/508/ --- Review request for Hive Developers. Summary --- Database/Scheme support for Hive. * Implemented 'SHOW DATABASES' command * Refactored TestHiveMetaStore and enabled tests for remote metastore client. * Added launch configurations for TestHiveMetaStore and TestHiveMetaStoreRemote This addresses bug HIVE-675. http://issues.apache.org/jira/browse/HIVE-675 Diffs - build-common.xml d4ff895 eclipse-templates/TestHive.launchtemplate 24efc12 eclipse-templates/TestHiveMetaStore.launchtemplate PRE-CREATION eclipse-templates/TestHiveMetaStoreRemote.launchtemplate PRE-CREATION metastore/if/hive_metastore.thrift 478d0af metastore/src/gen-cpp/ThriftHiveMetastore.h e2538fb metastore/src/gen-cpp/ThriftHiveMetastore.cpp f945a3a metastore/src/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp ed2bb99 metastore/src/gen-cpp/hive_metastore_types.h 1b0c706 metastore/src/gen-cpp/hive_metastore_types.cpp b5a403d metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java 78c78d9 metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 25408d9 metastore/src/gen-php/ThriftHiveMetastore.php ea4add5 metastore/src/gen-php/hive_metastore_types.php 61872a0 metastore/src/gen-py/hive_metastore/ThriftHiveMetastore-remote fc06cba metastore/src/gen-py/hive_metastore/ThriftHiveMetastore.py 4a0bc67 metastore/src/gen-py/hive_metastore/ttypes.py ea7269e metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 39dbd52 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 4fb296a metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java c6541af metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 6013644 metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 0818689 metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java a06384c metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 4951bd6 metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 4488f94 metastore/src/model/org/apache/hadoop/hive/metastore/model/MDatabase.java b3e098d metastore/src/model/package.jdo 206ba75 metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java fff6aad metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreBase.java PRE-CREATION metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java bc950b9 ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java bc268a4 ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java d59f48c ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 04dd9e3 ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 2ecda01 ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java eedf9e3 ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 0484c91 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 02bf926 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 70cd05f ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java eb079aa ql/src/java/org/apache/hadoop/hive/ql/plan/CreateDatabaseDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java ed4ed22 ql/src/java/org/apache/hadoop/hive/ql/plan/DropDatabaseDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/ShowDatabasesDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/SwitchDatabaseDesc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java b4651a2 ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java ab39ca4 ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveMetaStoreChecker.java 26cc71a ql/src/test/queries/clientnegative/create_database_bad_name.q PRE-CREATION ql/src/test/queries/clientnegative/database_switch_test.q PRE-CREATION ql/src/test/queries/clientpositive/database.q PRE-CREATION ql/src/test/results/clientnegative/create_database_bad_name.q.out PRE-CREATION ql/src/test/results/clientnegative/database_switch_test.q.out PRE-CREATION ql/src/test/results/clientpositive/database.q.out PRE-CREATION Diff: http://review.cloudera.org/r/508/diff Testing --- Thanks, Carl
[jira] Updated: (HIVE-675) add database/scheme support Hive QL
[ https://issues.apache.org/jira/browse/HIVE-675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-675: Attachment: HIVE-675-2010-8-4.patch.txt HIVE-675-2010-8-4.patch.txt: * Implemented 'SHOW DATABASES' command * Refactored TestHiveMetaStore and enabled tests for remote metastore client. * Added launch configurations for TestHiveMetaStore and TestHiveMetaStoreRemote add database/scheme support Hive QL --- Key: HIVE-675 URL: https://issues.apache.org/jira/browse/HIVE-675 Project: Hadoop Hive Issue Type: New Feature Components: Metastore, Query Processor Reporter: Prasad Chakka Assignee: Carl Steinbach Fix For: 0.6.0, 0.7.0 Attachments: hive-675-2009-9-16.patch, hive-675-2009-9-19.patch, hive-675-2009-9-21.patch, hive-675-2009-9-23.patch, hive-675-2009-9-7.patch, hive-675-2009-9-8.patch, HIVE-675-2010-7-16.patch.txt, HIVE-675-2010-8-4.patch.txt Currently all Hive tables reside in single namespace (default). Hive should support multiple namespaces (databases or schemas) such that users can create tables in their specific namespaces. These name spaces can have different warehouse directories (with a default naming scheme) and possibly different properties. There is already some support for this in metastore but Hive query parser should have this feature as well. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-675) add database/scheme support Hive QL
[ https://issues.apache.org/jira/browse/HIVE-675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-675: Status: Patch Available (was: Open) Posted on reviewboard: https://review.cloudera.org/r/508/ add database/scheme support Hive QL --- Key: HIVE-675 URL: https://issues.apache.org/jira/browse/HIVE-675 Project: Hadoop Hive Issue Type: New Feature Components: Metastore, Query Processor Reporter: Prasad Chakka Assignee: Carl Steinbach Fix For: 0.6.0, 0.7.0 Attachments: hive-675-2009-9-16.patch, hive-675-2009-9-19.patch, hive-675-2009-9-21.patch, hive-675-2009-9-23.patch, hive-675-2009-9-7.patch, hive-675-2009-9-8.patch, HIVE-675-2010-7-16.patch.txt, HIVE-675-2010-8-4.patch.txt Currently all Hive tables reside in single namespace (default). Hive should support multiple namespaces (databases or schemas) such that users can create tables in their specific namespaces. These name spaces can have different warehouse directories (with a default naming scheme) and possibly different properties. There is already some support for this in metastore but Hive query parser should have this feature as well. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-675) add database/scheme support Hive QL
[ https://issues.apache.org/jira/browse/HIVE-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895457#action_12895457 ] HBase Review Board commented on HIVE-675: - Message from: Carl Steinbach c...@cloudera.com --- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/508/ --- Review request for Hive Developers. Summary --- Database/Scheme support for Hive. * Implemented 'SHOW DATABASES' command * Refactored TestHiveMetaStore and enabled tests for remote metastore client. * Added launch configurations for TestHiveMetaStore and TestHiveMetaStoreRemote This addresses bug HIVE-675. http://issues.apache.org/jira/browse/HIVE-675 Diffs - build-common.xml d4ff895 eclipse-templates/TestHive.launchtemplate 24efc12 eclipse-templates/TestHiveMetaStore.launchtemplate PRE-CREATION eclipse-templates/TestHiveMetaStoreRemote.launchtemplate PRE-CREATION metastore/if/hive_metastore.thrift 478d0af metastore/src/gen-cpp/ThriftHiveMetastore.h e2538fb metastore/src/gen-cpp/ThriftHiveMetastore.cpp f945a3a metastore/src/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp ed2bb99 metastore/src/gen-cpp/hive_metastore_types.h 1b0c706 metastore/src/gen-cpp/hive_metastore_types.cpp b5a403d metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java 78c78d9 metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 25408d9 metastore/src/gen-php/ThriftHiveMetastore.php ea4add5 metastore/src/gen-php/hive_metastore_types.php 61872a0 metastore/src/gen-py/hive_metastore/ThriftHiveMetastore-remote fc06cba metastore/src/gen-py/hive_metastore/ThriftHiveMetastore.py 4a0bc67 metastore/src/gen-py/hive_metastore/ttypes.py ea7269e metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 39dbd52 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 4fb296a metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java c6541af metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 6013644 metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 0818689 metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java a06384c metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 4951bd6 metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 4488f94 metastore/src/model/org/apache/hadoop/hive/metastore/model/MDatabase.java b3e098d metastore/src/model/package.jdo 206ba75 metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java fff6aad metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreBase.java PRE-CREATION metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java bc950b9 ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java bc268a4 ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java d59f48c ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 04dd9e3 ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 2ecda01 ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java eedf9e3 ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 0484c91 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 02bf926 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 70cd05f ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java eb079aa ql/src/java/org/apache/hadoop/hive/ql/plan/CreateDatabaseDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java ed4ed22 ql/src/java/org/apache/hadoop/hive/ql/plan/DropDatabaseDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/ShowDatabasesDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/SwitchDatabaseDesc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java b4651a2 ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java ab39ca4 ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveMetaStoreChecker.java 26cc71a ql/src/test/queries/clientnegative/create_database_bad_name.q PRE-CREATION ql/src/test/queries/clientnegative/database_switch_test.q PRE-CREATION ql/src/test/queries/clientpositive/database.q PRE-CREATION ql/src/test/results/clientnegative/create_database_bad_name.q.out PRE-CREATION ql/src/test/results/clientnegative/database_switch_test.q.out PRE-CREATION ql/src/test/results/clientpositive/database.q.out PRE-CREATION Diff: http://review.cloudera.org/r/508/diff Testing --- Thanks, Carl add database/scheme support Hive QL --- Key: HIVE-675 URL:
can anybody help to check in this code into hive trunk ?
Please help to check in this code into http://svn.apache.org/repos/asf/hadoop/hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase The change is needed for the hive_hbase-handler.jar to be compatible with cloudera CDH3 version of hbase. they are also needed to be compatible with hbase 0.20.4 and 0.20.5. The change is very minimal tompared to today's trunk. only 2 type casting, and one change to use the getVaule() API that exists in all versions of hbase. Jimmy.
Re: can anybody help to check in this code into hive trunk ?
Please create a Hive JIRA issue and attach a patch file. http://wiki.apache.org/hadoop/Hive/HowToContribute#Creating_a_patch JVS On Aug 4, 2010, at 4:23 PM, Jinsong Hu wrote: Please help to check in this code into http://svn.apache.org/repos/asf/hadoop/hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase The change is needed for the hive_hbase-handler.jar to be compatible with cloudera CDH3 version of hbase. they are also needed to be compatible with hbase 0.20.4 and 0.20.5. The change is very minimal tompared to today's trunk. only 2 type casting, and one change to use the getVaule() API that exists in all versions of hbase. Jimmy.
[jira] Commented: (HIVE-1509) Monitor the working set of the number of files
[ https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895472#action_12895472 ] Joydeep Sen Sarma commented on HIVE-1509: - the test result for dyn_part3.q is not matching the one provided in the patch. it seems that testnegativeclidriver is not executing anything but the first query in the .q file: [junit] diff -a -I file: -I pfile: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I \ owner -I transient_lastDdlTime -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -\ I Caused by: -I [.][.][.] [0-9]* more /data/users/jssarma/hive_trunk/build/ql/test/logs/clientnegative/dy\ n_part3.q.out [junit] 9a10,27 [junit] PREHOOK: query: create table nzhang_part( key string) partitioned by (value string) [junit] PREHOOK: type: CREATETABLE [junit] POSTHOOK: query: create table nzhang_part( key string) partitioned by (value string) [junit] POSTHOOK: type: CREATETABLE [junit] POSTHOOK: Output: defa...@nzhang_part [junit] PREHOOK: query: insert overwrite table nzhang_part partition(value) select key, value from \ src [junit] PREHOOK: type: QUERY [junit] PREHOOK: Input: defa...@src [junit] FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask [junit] PREHOOK: query: create table nzhang_part( key string) partitioned by (value string) [junit] PREHOOK: type: CREATETABLE [junit] POSTHOOK: query: create table nzhang_part( key string) partitioned by (value string) [junit] POSTHOOK: type: CREATETABLE [junit] POSTHOOK: Output: defa...@nzhang_part [junit] PREHOOK: query: insert overwrite table nzhang_part partition(value) select key, value from \ src [junit] PREHOOK: type: QUERY [junit] PREHOOK: Input: defa...@src [junit] FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask Monitor the working set of the number of files --- Key: HIVE-1509 URL: https://issues.apache.org/jira/browse/HIVE-1509 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Reporter: Namit Jain Assignee: Ning Zhang Attachments: HIVE-1509.2.patch, HIVE-1509.3.patch, HIVE-1509.4.patch, HIVE-1509.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1413) bring a table/partition offline
[ https://issues.apache.org/jira/browse/HIVE-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1413: -- Status: Patch Available (was: Open) bring a table/partition offline --- Key: HIVE-1413 URL: https://issues.apache.org/jira/browse/HIVE-1413 Project: Hadoop Hive Issue Type: New Feature Components: Metastore, Query Processor Reporter: Namit Jain Assignee: Siying Dong Fix For: 0.7.0 Attachments: HIVE-1413.1.patch, HIVE-1413.2.patch, HIVE-1413.3.patch There should be a way to bring a table/partition offline. At that time, no read/write operations should be supported on that table. It would be very useful for housekeeping operations -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1413) bring a table/partition offline
[ https://issues.apache.org/jira/browse/HIVE-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1413: -- Attachment: HIVE-1413.3.patch use more consolidated approach to try to check it in validate() instead of everywhere. change syntax a little bit. Add some more unit test cases. bring a table/partition offline --- Key: HIVE-1413 URL: https://issues.apache.org/jira/browse/HIVE-1413 Project: Hadoop Hive Issue Type: New Feature Components: Metastore, Query Processor Reporter: Namit Jain Assignee: Siying Dong Fix For: 0.7.0 Attachments: HIVE-1413.1.patch, HIVE-1413.2.patch, HIVE-1413.3.patch There should be a way to bring a table/partition offline. At that time, no read/write operations should be supported on that table. It would be very useful for housekeeping operations -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-675) add database/scheme support Hive QL
[ https://issues.apache.org/jira/browse/HIVE-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895507#action_12895507 ] Namit Jain commented on HIVE-675: - Thanks Carl, I will take a look add database/scheme support Hive QL --- Key: HIVE-675 URL: https://issues.apache.org/jira/browse/HIVE-675 Project: Hadoop Hive Issue Type: New Feature Components: Metastore, Query Processor Reporter: Prasad Chakka Assignee: Carl Steinbach Fix For: 0.6.0, 0.7.0 Attachments: hive-675-2009-9-16.patch, hive-675-2009-9-19.patch, hive-675-2009-9-21.patch, hive-675-2009-9-23.patch, hive-675-2009-9-7.patch, hive-675-2009-9-8.patch, HIVE-675-2010-7-16.patch.txt, HIVE-675-2010-8-4.patch.txt Currently all Hive tables reside in single namespace (default). Hive should support multiple namespaces (databases or schemas) such that users can create tables in their specific namespaces. These name spaces can have different warehouse directories (with a default naming scheme) and possibly different properties. There is already some support for this in metastore but Hive query parser should have this feature as well. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1509) Monitor the working set of the number of files
[ https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895523#action_12895523 ] Ning Zhang commented on HIVE-1509: -- It's strange. I didn't get this diff on both 17 and 20. I'll run TestNegativeCliDriver again. Monitor the working set of the number of files --- Key: HIVE-1509 URL: https://issues.apache.org/jira/browse/HIVE-1509 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Reporter: Namit Jain Assignee: Ning Zhang Attachments: HIVE-1509.2.patch, HIVE-1509.3.patch, HIVE-1509.4.patch, HIVE-1509.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1509) Monitor the working set of the number of files
[ https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895526#action_12895526 ] Ning Zhang commented on HIVE-1509: -- The TestNegativeCliDriver ran successfully. In terms of the number of queries in the .q files in neg, I think as long as there is no queries after the first exception, it should be fine. At least this is what dyn_part[12].q was doing (only the last query thrown expected exception). Monitor the working set of the number of files --- Key: HIVE-1509 URL: https://issues.apache.org/jira/browse/HIVE-1509 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Reporter: Namit Jain Assignee: Ning Zhang Attachments: HIVE-1509.2.patch, HIVE-1509.3.patch, HIVE-1509.4.patch, HIVE-1509.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1413) bring a table/partition offline
[ https://issues.apache.org/jira/browse/HIVE-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1413: -- Attachment: HIVE-1413.4.patch The previous one missed two test output files... bring a table/partition offline --- Key: HIVE-1413 URL: https://issues.apache.org/jira/browse/HIVE-1413 Project: Hadoop Hive Issue Type: New Feature Components: Metastore, Query Processor Reporter: Namit Jain Assignee: Siying Dong Fix For: 0.7.0 Attachments: HIVE-1413.1.patch, HIVE-1413.2.patch, HIVE-1413.3.patch, HIVE-1413.4.patch There should be a way to bring a table/partition offline. At that time, no read/write operations should be supported on that table. It would be very useful for housekeeping operations -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1509) Monitor the working set of the number of files
[ https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895534#action_12895534 ] Joydeep Sen Sarma commented on HIVE-1509: - strange - let me retry. can u check the patch one last time? (perhaps it's not up to date with contents of ur tree?) Monitor the working set of the number of files --- Key: HIVE-1509 URL: https://issues.apache.org/jira/browse/HIVE-1509 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Reporter: Namit Jain Assignee: Ning Zhang Attachments: HIVE-1509.2.patch, HIVE-1509.3.patch, HIVE-1509.4.patch, HIVE-1509.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1509) Monitor the working set of the number of files
[ https://issues.apache.org/jira/browse/HIVE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895537#action_12895537 ] Ning Zhang commented on HIVE-1509: -- Yes, I check the svn diff in my working directory is the same as HIVE-1509.4.patch. Monitor the working set of the number of files --- Key: HIVE-1509 URL: https://issues.apache.org/jira/browse/HIVE-1509 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Reporter: Namit Jain Assignee: Ning Zhang Attachments: HIVE-1509.2.patch, HIVE-1509.3.patch, HIVE-1509.4.patch, HIVE-1509.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1413) bring a table/partition offline
[ https://issues.apache.org/jira/browse/HIVE-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-1413: - Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Resolution: Fixed committed. Thanks Siying bring a table/partition offline --- Key: HIVE-1413 URL: https://issues.apache.org/jira/browse/HIVE-1413 Project: Hadoop Hive Issue Type: New Feature Components: Metastore, Query Processor Reporter: Namit Jain Assignee: Siying Dong Fix For: 0.7.0 Attachments: HIVE-1413.1.patch, HIVE-1413.2.patch, HIVE-1413.3.patch, HIVE-1413.4.patch There should be a way to bring a table/partition offline. At that time, no read/write operations should be supported on that table. It would be very useful for housekeeping operations -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
