Re: HBase => replication => Hive

2011-03-11 Thread Andrew Purtell
s easy to do. :) > > Otis > > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch > Lucene ecosystem search :: http://search-lucene.com/ > > > > Best regards, > > > >     - Andy > > > > Problems worthy  of attack prove their worth by > hitting

Re: HBase => replication => Hive

2011-03-11 Thread Otis Gospodnetic
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. > - Piet Hein (via Tom White) > > > --- On Thu, 3/10/11, Otis

Re: HBase => replication => Hive

2011-03-11 Thread Otis Gospodnetic
Hi, > So, you essentially want to dump HBase tables into sequence files/RC > files/text files and read it from Hive? I think that's a Q for J-D. I know that what I had in mind was not about creating periodic dumps because that means data in Hive would always be behind data in HBase, but a mor

Re: HBase => replication => Hive

2011-03-11 Thread Amandeep Khurana
So, you essentially want to dump HBase tables into sequence files/RC files/text files and read it from Hive? How do you plan to handle updates, deletes, IVS etc if you use the log edits to replicate from hbase to these files? Getting Hive to talk to HFiles gives you the same problem.. Isn't it eas

Re: HBase => replication => Hive

2011-03-11 Thread Jean-Daniel Cryans
Given Hive's support for RCFile, I'm pretty sure that writing a handler for HFiles would be pretty easy. That may work in your situation... or not. I personally found Hive's performance very adequate for ad hoc querying. We replicate some prod data on demand using CopyTables but we're moving to us

Re: HBase => replication => Hive

2011-03-11 Thread Lars George
t; --- On Thu, 3/10/11, Otis Gospodnetic wrote: > >> From: Otis Gospodnetic >> Subject: HBase => replication => Hive >> To: user@hbase.apache.org >> Date: Thursday, March 10, 2011, 10:43 PM >> Hi, >> >> Since HBase has a mechanism to replicate e

Re: HBase => replication => Hive

2011-03-11 Thread Andrew Purtell
Hein (via Tom White) --- On Thu, 3/10/11, Otis Gospodnetic wrote: > From: Otis Gospodnetic > Subject: HBase => replication => Hive > To: user@hbase.apache.org > Date: Thursday, March 10, 2011, 10:43 PM > Hi, > > Since HBase has a mechanism to replicate edit logs

HBase => replication => Hive

2011-03-10 Thread Otis Gospodnetic
Hi, Since HBase has a mechanism to replicate edit logs to another HBase cluster, I was wondering if people think it would be possible to implement HBase=>Hive replication? (and really make the destination pluggable later on) I'm asking because while one can integrate Hive and HBase by creating