[jira] [Updated] (HIVE-2908) Hive: Extend ALTER TABLE DROP PARTITION syntax to use all comparators
[ https://issues.apache.org/jira/browse/HIVE-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2908: -- Attachment: HIVE-2908.D2523.2.patch sambavim updated the revision HIVE-2908 [jira] Hive: Extend ALTER TABLE DROP PARTITION syntax to use all comparators. Reviewers: JIRA, njain, kevinwilfong Updated the error message generated in DDLSemanticAnalyzer.java when the drop partition filter does not generate any partitions to drop. Tested the drop_partitions_filter test with a filter d 2 instead of d '2' - the former does not work, and the implementation of getPartitionsWithFilter says that it only works on string partition keys. Somehow providing a string value for the integer key seems to work although this seems hacky. The documentation should mention that this is only for string keys. When and how should I update the documentation? REVISION DETAIL https://reviews.facebook.net/D2523 AFFECTED FILES ql/src/test/results/clientpositive/drop_partitions_filter.q.out ql/src/test/results/clientpositive/drop_multi_partitions.q.out ql/src/test/results/clientnegative/drop_partition_filter_failure.q.out ql/src/test/results/clientnegative/drop_partition_failure.q.out ql/src/test/queries/clientpositive/drop_partitions_filter.q ql/src/test/queries/clientnegative/drop_partition_filter_failure.q ql/src/java/org/apache/hadoop/hive/ql/plan/DropTableDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionSpec.java ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java Hive: Extend ALTER TABLE DROP PARTITION syntax to use all comparators - Key: HIVE-2908 URL: https://issues.apache.org/jira/browse/HIVE-2908 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Sambavi Muthukrishnan Assignee: Sambavi Muthukrishnan Priority: Minor Attachments: HIVE-2908.D2523.1.patch, HIVE-2908.D2523.2.patch Original Estimate: 96h Remaining Estimate: 96h To drop a partition from a Hive table, this works: ALTER TABLE foo DROP PARTITION(ds = 'date') ...but it should also work to drop all partitions prior to date. ALTER TABLE foo DROP PARTITION(ds 'date') This task is to implement ALTER TABLE DROP PARTITION for all of the comparators, = = = != instead of just for =. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2908) Hive: Extend ALTER TABLE DROP PARTITION syntax to use all comparators
[ https://issues.apache.org/jira/browse/HIVE-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13241029#comment-13241029 ] Phabricator commented on HIVE-2908: --- sambavim has commented on the revision HIVE-2908 [jira] Hive: Extend ALTER TABLE DROP PARTITION syntax to use all comparators. Note: The committer will need to run the below test with -Doverwrite=true due to a problem with arc diff loading the differential on the test results. ant test -Dtestcase=TestCliDriver -Dqfile=escape1.q -Doverwrite=true REVISION DETAIL https://reviews.facebook.net/D2523 Hive: Extend ALTER TABLE DROP PARTITION syntax to use all comparators - Key: HIVE-2908 URL: https://issues.apache.org/jira/browse/HIVE-2908 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Sambavi Muthukrishnan Assignee: Sambavi Muthukrishnan Priority: Minor Attachments: HIVE-2908.D2523.1.patch, HIVE-2908.D2523.2.patch Original Estimate: 96h Remaining Estimate: 96h To drop a partition from a Hive table, this works: ALTER TABLE foo DROP PARTITION(ds = 'date') ...but it should also work to drop all partitions prior to date. ALTER TABLE foo DROP PARTITION(ds 'date') This task is to implement ALTER TABLE DROP PARTITION for all of the comparators, = = = != instead of just for =. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2906) Support providing some table properties by user via SQL
[
https://issues.apache.org/jira/browse/HIVE-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2906:
Status: Patch Available (was: Open)
Support providing some table properties by user via SQL
---
Key: HIVE-2906
URL: https://issues.apache.org/jira/browse/HIVE-2906
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Attachments: HIVE-2906.D2499.1.patch, HIVE-2906.D2499.2.patch
Some properties are needed to be provided to StorageHandler by user in
runtime. It might be an address for remote resource or retry count for access
or maximum version count(for hbase), etc.
For example,
{code}
select emp.empno, emp.ename from hbase_emp ('max.version'='3') emp;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2906) Support providing some table properties by user via SQL
[
https://issues.apache.org/jira/browse/HIVE-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-2906:
--
Attachment: HIVE-2906.D2499.2.patch
navis updated the revision HIVE-2906 [jira] Support providing some table
properties by user via SQL.
Reviewers: JIRA
1. Added 'hbase.max.versions' as a example of using table properties
2. Added test case 'hbase_max_versions.q'
REVISION DETAIL
https://reviews.facebook.net/D2499
AFFECTED FILES
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/ResultWrapper.java
hbase-handler/src/test/queries/hbase_max_versions.q
hbase-handler/src/test/results/hbase_max_versions.q.out
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java
ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
ql/src/test/queries/clientpositive/serde_user_properties.q
ql/src/test/results/clientpositive/serde_user_properties.q.out
Support providing some table properties by user via SQL
---
Key: HIVE-2906
URL: https://issues.apache.org/jira/browse/HIVE-2906
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Attachments: HIVE-2906.D2499.1.patch, HIVE-2906.D2499.2.patch
Some properties are needed to be provided to StorageHandler by user in
runtime. It might be an address for remote resource or retry count for access
or maximum version count(for hbase), etc.
For example,
{code}
select emp.empno, emp.ename from hbase_emp ('max.version'='3') emp;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2906) Support providing some table properties by user via SQL
[
https://issues.apache.org/jira/browse/HIVE-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-2906:
--
Attachment: HIVE-2906.D2499.3.patch
navis updated the revision HIVE-2906 [jira] Support providing some table
properties by user via SQL.
Reviewers: JIRA
1. Fixed test case to have deterministic result
REVISION DETAIL
https://reviews.facebook.net/D2499
AFFECTED FILES
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/ResultWrapper.java
hbase-handler/src/test/queries/hbase_max_versions.q
hbase-handler/src/test/results/hbase_max_versions.q.out
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java
ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
ql/src/test/queries/clientpositive/serde_user_properties.q
ql/src/test/results/clientpositive/serde_user_properties.q.out
Support providing some table properties by user via SQL
---
Key: HIVE-2906
URL: https://issues.apache.org/jira/browse/HIVE-2906
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Attachments: HIVE-2906.D2499.1.patch, HIVE-2906.D2499.2.patch,
HIVE-2906.D2499.3.patch
Some properties are needed to be provided to StorageHandler by user in
runtime. It might be an address for remote resource or retry count for access
or maximum version count(for hbase), etc.
For example,
{code}
select emp.empno, emp.ename from hbase_emp ('max.version'='3') emp;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2911) Move global .hiverc file
[
https://issues.apache.org/jira/browse/HIVE-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-2911:
--
Attachment: HIVE-2911.D2529.1.patch
omalley requested code review of HIVE-2911 [jira] Move global .hiverc file.
Reviewers: JIRA
HIVE-2911
Move the location of the global hiverc file to $HIVE_CONF_DIR/.hiverc.
Currently, the .hiverc files are loaded from:
$HIVE_HOME/bin/.hiverc
~/.hiverc
It seems more ops-friendly to have it in the config directory.
$HIVE_HOME/bin/.hiverc - for backwards compatibility
$HIVE_CONF_DIR/.hiverc
~/.hiverc
TEST PLAN
Can't test cli driver.
REVISION DETAIL
https://reviews.facebook.net/D2529
AFFECTED FILES
cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/5733/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
Move global .hiverc file
Key: HIVE-2911
URL: https://issues.apache.org/jira/browse/HIVE-2911
Project: Hive
Issue Type: Improvement
Components: CLI
Reporter: Owen O'Malley
Assignee: Owen O'Malley
Attachments: HIVE-2911.D2529.1.patch
Currently, the .hiverc files are loaded from:
{code}
$HIVE_HOME/bin/.hiverc
~/.hiverc
{code}
It seems more ops-friendly to have it in the config directory.
{code}
$HIVE_HOME/bin/.hiverc - for backwards compatibility
$HIVE_CONF_DIR/.hiverc
~/.hiverc
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2870) Throw an error when a nonexistent partition is accessed in strict mode
[ https://issues.apache.org/jira/browse/HIVE-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2870: -- Attachment: HIVE-2870.D2319.5.patch lucian updated the revision HIVE-2870 [jira] Throw an error when a nonexistent partition is accessed in strict mode. Reviewers: JIRA, njain, kevinwilfong, akramer add new confvar, throw exception or log message REVISION DETAIL https://reviews.facebook.net/D2319 AFFECTED FILES common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java ql/src/test/queries/clientnegative/nopart_after_pruning.q ql/src/test/queries/clientpositive/nopart_after_pruning.q ql/src/test/results/clientnegative/nopart_after_pruning.q.out ql/src/test/results/clientpositive/nopart_after_pruning.q.out Throw an error when a nonexistent partition is accessed in strict mode -- Key: HIVE-2870 URL: https://issues.apache.org/jira/browse/HIVE-2870 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Lucian Adrian Grijincu Priority: Minor Attachments: HIVE-2870.D2319.1.patch, HIVE-2870.D2319.2.patch, HIVE-2870.D2319.3.patch, HIVE-2870.D2319.4.patch, HIVE-2870.D2319.5.patch Original Estimate: 24h Remaining Estimate: 24h When a table does not exist and someone tries to read from it in a query, Hive throws an error. But if a partition is directly accessed that does not exist, an error is not thrown. This is inconsistent and also leads to a lot of confused users who get no output. This task is to cause Hive to throw an error when the partition pruner for a query eliminates ALL existing partitions for some table when running in strict mode. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Is HiveMetastoreClient a public interface?
No reponse from the user list. Reposting in the dev list: Hi, It seems like the HiveMetaStoreClient is the only way to programmatically access (and retreive) the metastore for DDL and metadata queries as well as retrieving delegation tokens. It isn't documented anywhere, so I'd like to know if it is advisable to have code make use of this directly or is this an internal API? If the latter what would be the recommended way of programmatically doing the previously mentioned operations? -Francis
[jira] [Commented] (HIVE-1721) use bloom filters to improve the performance of joins
[ https://issues.apache.org/jira/browse/HIVE-1721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13241517#comment-13241517 ] Mikalai Parafeniuk commented on HIVE-1721: -- Hello. I am Mikalaj Parafeniuk from Belarussian State University. I'm a third year student and I'm looking for a project to contribute in terms of GSoC 2012. I have already learned a basics about bloom filters from the internet and usefull links from this thread. But i need some further reading. Could you propose some? Also i need to learn Hive codebase. I want to achieve that by fixing small bugs. Could you give me the idea where can i start? use bloom filters to improve the performance of joins - Key: HIVE-1721 URL: https://issues.apache.org/jira/browse/HIVE-1721 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Namit Jain Labels: gsoc, gsoc2012, optimization In case of map-joins, it is likely that the big table will not find many matching rows from the small table. Currently, we perform a hash-map lookup for every row in the big table, which can be pretty expensive. It might be useful to try out a bloom-filter containing all the elements in the small table. Each element from the big table is first searched in the bloom filter, and only in case of a positive match, the small table hash table is explored. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2911) Move global .hiverc file
[
https://issues.apache.org/jira/browse/HIVE-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HIVE-2911:
Status: Patch Available (was: Open)
I tested this on a cluster and it worked as expected.
Move global .hiverc file
Key: HIVE-2911
URL: https://issues.apache.org/jira/browse/HIVE-2911
Project: Hive
Issue Type: Improvement
Components: CLI
Reporter: Owen O'Malley
Assignee: Owen O'Malley
Attachments: HIVE-2911.D2529.1.patch, HIVE-2911.D2529.2.patch
Currently, the .hiverc files are loaded from:
{code}
$HIVE_HOME/bin/.hiverc
~/.hiverc
{code}
It seems more ops-friendly to have it in the config directory.
{code}
$HIVE_HOME/bin/.hiverc - for backwards compatibility
$HIVE_CONF_DIR/.hiverc
~/.hiverc
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2911) Move global .hiverc file
[
https://issues.apache.org/jira/browse/HIVE-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-2911:
--
Attachment: HIVE-2911.D2529.2.patch
omalley updated the revision HIVE-2911 [jira] Move global .hiverc file.
Reviewers: JIRA
Changed the printInfo to printError so that the user actually sees the
warning.
REVISION DETAIL
https://reviews.facebook.net/D2529
AFFECTED FILES
cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java
Move global .hiverc file
Key: HIVE-2911
URL: https://issues.apache.org/jira/browse/HIVE-2911
Project: Hive
Issue Type: Improvement
Components: CLI
Reporter: Owen O'Malley
Assignee: Owen O'Malley
Attachments: HIVE-2911.D2529.1.patch, HIVE-2911.D2529.2.patch
Currently, the .hiverc files are loaded from:
{code}
$HIVE_HOME/bin/.hiverc
~/.hiverc
{code}
It seems more ops-friendly to have it in the config directory.
{code}
$HIVE_HOME/bin/.hiverc - for backwards compatibility
$HIVE_CONF_DIR/.hiverc
~/.hiverc
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Is HiveMetastoreClient a public interface?
Ideally you want to do everything through the hiveQL language. Calling the read only methods are safe and would be easier/better then Calling 'show tables' and attempting to parse. But I would consider the rest of the interface for experts only. Also this interface can change slightly between versions making it harder to build code against. Edward On Thu, Mar 29, 2012 at 2:50 PM, Francis Liu tof...@apache.org wrote: No reponse from the user list. Reposting in the dev list: Hi, It seems like the HiveMetaStoreClient is the only way to programmatically access (and retreive) the metastore for DDL and metadata queries as well as retrieving delegation tokens. It isn't documented anywhere, so I'd like to know if it is advisable to have code make use of this directly or is this an internal API? If the latter what would be the recommended way of programmatically doing the previously mentioned operations? -Francis
Re: Is HiveMetastoreClient a public interface?
Hi Edward, We would also need other apis such as dropping partitions and tables as well as retrieving delegation tokens. Experts only is doable. The main concern of it being public is wether in future releases, Hive developers will do their best to make sure the interface is backwards compatible. -Francis On 3/29/12 1:04 PM, Edward Capriolo edlinuxg...@gmail.com wrote: Ideally you want to do everything through the hiveQL language. Calling the read only methods are safe and would be easier/better then Calling 'show tables' and attempting to parse. But I would consider the rest of the interface for experts only. Also this interface can change slightly between versions making it harder to build code against. Edward On Thu, Mar 29, 2012 at 2:50 PM, Francis Liu tof...@apache.org wrote: No reponse from the user list. Reposting in the dev list: Hi, It seems like the HiveMetaStoreClient is the only way to programmatically access (and retreive) the metastore for DDL and metadata queries as well as retrieving delegation tokens. It isn't documented anywhere, so I'd like to know if it is advisable to have code make use of this directly or is this an internal API? If the latter what would be the recommended way of programmatically doing the previously mentioned operations? -Francis
[jira] [Created] (HIVE-2912) HiveServer shouldn't initialize HiveMetaStore if Metastore is remote.
HiveServer shouldn't initialize HiveMetaStore if Metastore is remote. - Key: HIVE-2912 URL: https://issues.apache.org/jira/browse/HIVE-2912 Project: Hive Issue Type: Bug Reporter: Ashutosh Chauhan In HiveServer.java HiveServerHandler constructor line 131, call to super() should be made only if hive.local.metastore = true in conf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2912) HiveServer shouldn't initialize HiveMetaStore if Metastore is remote.
[ https://issues.apache.org/jira/browse/HIVE-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2912: --- Description: Currently it always initializes MetaStore in-process. (was: In HiveServer.java HiveServerHandler constructor line 131, call to super() should be made only if hive.local.metastore = true in conf. ) HiveServer shouldn't initialize HiveMetaStore if Metastore is remote. - Key: HIVE-2912 URL: https://issues.apache.org/jira/browse/HIVE-2912 Project: Hive Issue Type: Bug Reporter: Ashutosh Chauhan Currently it always initializes MetaStore in-process. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Is HiveMetastoreClient a public interface?
Hi Francis, The HiveMetaStoreClient library is a thin wrapper layer written around the Thrift generated client code. It is not a public API and is subject to change in the future. If experts only is doable, then I recommend integrating directly at the Thrift API level (which is a public API), e.g. call the Thrift generated code directly instead of accessing it through the HiveMetaStoreClient library. Thanks. Carl On Thu, Mar 29, 2012 at 2:05 PM, Francis Liu tof...@apache.org wrote: Hi Edward, We would also need other apis such as dropping partitions and tables as well as retrieving delegation tokens. Experts only is doable. The main concern of it being public is wether in future releases, Hive developers will do their best to make sure the interface is backwards compatible. -Francis On 3/29/12 1:04 PM, Edward Capriolo edlinuxg...@gmail.com wrote: Ideally you want to do everything through the hiveQL language. Calling the read only methods are safe and would be easier/better then Calling 'show tables' and attempting to parse. But I would consider the rest of the interface for experts only. Also this interface can change slightly between versions making it harder to build code against. Edward On Thu, Mar 29, 2012 at 2:50 PM, Francis Liu tof...@apache.org wrote: No reponse from the user list. Reposting in the dev list: Hi, It seems like the HiveMetaStoreClient is the only way to programmatically access (and retreive) the metastore for DDL and metadata queries as well as retrieving delegation tokens. It isn't documented anywhere, so I'd like to know if it is advisable to have code make use of this directly or is this an internal API? If the latter what would be the recommended way of programmatically doing the previously mentioned operations? -Francis
[jira] [Updated] (HIVE-2906) Support providing some table properties by user via SQL
[
https://issues.apache.org/jira/browse/HIVE-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2906:
Status: Open (was: Patch Available)
Support providing some table properties by user via SQL
---
Key: HIVE-2906
URL: https://issues.apache.org/jira/browse/HIVE-2906
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Attachments: HIVE-2906.D2499.1.patch, HIVE-2906.D2499.2.patch,
HIVE-2906.D2499.3.patch
Some properties are needed to be provided to StorageHandler by user in
runtime. It might be an address for remote resource or retry count for access
or maximum version count(for hbase), etc.
For example,
{code}
select emp.empno, emp.ename from hbase_emp ('max.version'='3') emp;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Is HiveMetastoreClient a public interface?
Hi Carl, I see, thanks for clarifying that. It'd be brittle to write our own client as HiveMetaStoreClient is not really that thin since it is already performing callbacks for HiveMetaHook and someone might decide to add more (or remove) logic into it. The java API is an ask from one of our users for Hcatalog. Given that HiveMetaStoreClient is a thin wrapper around public apis, then change shouldn't be drastic? It seems the cleaner way to go about it is to create an hcat client wrapper around HiveMetaStoreClient and have HCatClient as public? -Francis On 3/29/12 3:34 PM, Carl Steinbach c...@cloudera.com wrote: Hi Francis, The HiveMetaStoreClient library is a thin wrapper layer written around the Thrift generated client code. It is not a public API and is subject to change in the future. If experts only is doable, then I recommend integrating directly at the Thrift API level (which is a public API), e.g. call the Thrift generated code directly instead of accessing it through the HiveMetaStoreClient library. Thanks. Carl On Thu, Mar 29, 2012 at 2:05 PM, Francis Liu tof...@apache.org wrote: Hi Edward, We would also need other apis such as dropping partitions and tables as well as retrieving delegation tokens. Experts only is doable. The main concern of it being public is wether in future releases, Hive developers will do their best to make sure the interface is backwards compatible. -Francis On 3/29/12 1:04 PM, Edward Capriolo edlinuxg...@gmail.com wrote: Ideally you want to do everything through the hiveQL language. Calling the read only methods are safe and would be easier/better then Calling 'show tables' and attempting to parse. But I would consider the rest of the interface for experts only. Also this interface can change slightly between versions making it harder to build code against. Edward On Thu, Mar 29, 2012 at 2:50 PM, Francis Liu tof...@apache.org wrote: No reponse from the user list. Reposting in the dev list: Hi, It seems like the HiveMetaStoreClient is the only way to programmatically access (and retreive) the metastore for DDL and metadata queries as well as retrieving delegation tokens. It isn't documented anywhere, so I'd like to know if it is advisable to have code make use of this directly or is this an internal API? If the latter what would be the recommended way of programmatically doing the previously mentioned operations? -Francis
[jira] [Updated] (HIVE-2906) Support providing some table properties by user via SQL
[
https://issues.apache.org/jira/browse/HIVE-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-2906:
--
Attachment: HIVE-2906.D2499.4.patch
navis updated the revision HIVE-2906 [jira] Support providing some table
properties by user via SQL.
Reviewers: JIRA
1. Fixed not applying table properties to fetch operator
2. Added 'hbase.versions.order' option for ordering of multi-versioned result
3. Removed debug message, added test case
REVISION DETAIL
https://reviews.facebook.net/D2499
AFFECTED FILES
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/ResultWrapper.java
hbase-handler/src/test/queries/hbase_max_versions.q
hbase-handler/src/test/results/hbase_max_versions.q.out
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java
ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
ql/src/test/queries/clientpositive/serde_user_properties.q
ql/src/test/results/clientpositive/serde_user_properties.q.out
Support providing some table properties by user via SQL
---
Key: HIVE-2906
URL: https://issues.apache.org/jira/browse/HIVE-2906
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Attachments: HIVE-2906.D2499.1.patch, HIVE-2906.D2499.2.patch,
HIVE-2906.D2499.3.patch, HIVE-2906.D2499.4.patch
Some properties are needed to be provided to StorageHandler by user in
runtime. It might be an address for remote resource or retry count for access
or maximum version count(for hbase), etc.
For example,
{code}
select emp.empno, emp.ename from hbase_emp ('max.version'='3') emp;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
