Re: Orc file and Hive Optimiser

2015-04-20 Thread Alan Gates
Mich Talebzadeh mailto:m...@peridale.co.uk April 19, 2015 at 12:32 Finally this is more of a speculative question. If we have ORC files that provide good functionality, is there any reason why one should deploy a columnar database such as Hbase or Cassandra If Hive can do the job as well?

Re: Orc file and Hive Optimiser

2015-04-19 Thread John Pullokkaran
user@hive.apache.orgmailto:user@hive.apache.org Date: Sunday, April 19, 2015 at 12:32 PM To: user@hive.apache.orgmailto:user@hive.apache.org user@hive.apache.orgmailto:user@hive.apache.org Subject: Orc file and Hive Optimiser My understanding is that the Optimized Row Columnar (ORC) file format

RE: Orc file and Hive Optimiser

2015-04-19 Thread Mich Talebzadeh
...@hortonworks.com] Sent: 19 April 2015 20:37 To: user@hive.apache.org Subject: Re: Orc file and Hive Optimiser ORC format is transparent to CBO. Currently we are working on a new cost model which might reflect ORC's performance advantages in optimization decisions. Thanks John From: Mich

Re: Orc file and Hive Optimiser

2015-04-19 Thread John Pullokkaran
] Sent: 19 April 2015 20:37 To: user@hive.apache.orgmailto:user@hive.apache.org Subject: Re: Orc file and Hive Optimiser ORC format is transparent to CBO. Currently we are working on a new cost model which might reflect ORC’s performance advantages in optimization decisions. Thanks John From: Mich

Orc file and Hive Optimiser

2015-04-19 Thread Mich Talebzadeh
My understanding is that the Optimized Row Columnar (ORC) file format provides a highly efficient way to store Hive data. https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC In a nutshell the columnar storage allows pretty efficient compression of columns on par with

RE: Orc file and Hive Optimiser

2015-04-19 Thread Mich Talebzadeh
that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility. From: John Pullokkaran [mailto:jpullokka...@hortonworks.com] Sent: 19 April 2015 20:53 To: user@hive.apache.org Subject: Re: Orc file and Hive Optimiser If you wish