Re: Is Hive Index officially not recommended?

> Is anybody storing there index in a non-native table such as HBase? ... > Can you please point to implementations of HiveIndexHandler or >AbstractIndexHandler > that have usesIndexTable=false I don't think there are any publically available implementations yet. The Hive HBase-metastore project

Re: Is Hive Index officially not recommended?

Hi Gopal, As you suggested in your email above that *Part #1 of using hive indexes effectively is to write your ownHiveIndexHandler, with usesIndexTable=false;* *And then write a IndexPredicateAnalyzer, which lets you map arbitrarylookups into other range conditions.* Is anybody storing there

Re: Is Hive Index officially not recommended?

I'd like to revise the Indexing and IndexDev docs in the wiki to include this information (as well as information from a previous thread, if I can find it) so peopl

Re: Is Hive Index officially not recommended?

>So in a nutshell in Hive if "external" indexes are not used for improving >query response, what value they add and can we forget them for now? The builtin indexes - those that write data as smaller tables are only useful in a pre-columnar world, where the indexes offer a huge reduction in IO. P

RE: Is Hive Index officially not recommended?

--Original Message- From: Gopal Vijayaraghavan [mailto:go...@hortonworks.com] On Behalf Of Gopal Vijayaraghavan Sent: 05 January 2016 21:49 To: user@hive.apache.org Subject: Re: Is Hive Index officially not recommended? > I am going to run the same query in Hive. However, I only see

Re: Is Hive Index officially not recommended?

> I am going to run the same query in Hive. However, I only see a table >scan below and no mention of that index. May be I am missing something >here? Hive Indexes are an incomplete feature, because they are not maintained over an ACID storage & demand FileSystem access to check for validity.

RE: Is Hive Index officially not recommended?

ree, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility. From: Jörn Franke [mailto:jornfra...@gmail.com] Sent: 05 January 2016 19:59 To: user@hive.apache.org Subject: Re: Is Hive Index officially not recommended? Btw this is not Hive specific, but also fo

Re: Is Hive Index officially not recommended?

yes. we tried mr and it works fine. so it's more likely a tez issue. Thanks for your comments. On Tue, Jan 5, 2016 at 11:58 AM Jörn Franke wrote: > You can still use execution Engine mr for maintaining the index. Indeed > with the ORC or parquet format there are min/max indexes and bloom filters

Re: Is Hive Index officially not recommended?

Btw this is not Hive specific, but also for other relational database systems, such as Oracle Exadata. > On 05 Jan 2016, at 20:57, Jörn Franke wrote: > > You can still use execution Engine mr for maintaining the index. Indeed with > the ORC or parquet format there are min/max indexes and bloom

Re: Is Hive Index officially not recommended?

You can still use execution Engine mr for maintaining the index. Indeed with the ORC or parquet format there are min/max indexes and bloom filters, but you need to sort your data appropriately to benefit from performance. Alternatively you can create redundant tables sorted in different order. T

RE: Is Hive Index officially not recommended?

I don’t think Index on hive (as a separate entity) adds any value although you can create one You can create an ORC table which will have characteristics that can simulate index like behaviour CLUSTERED BY (object_id) INTO 256 BUCKETS STORED AS ORC TBLPROPERTIES ( "orc.compress"="SNAP