Re: financial time series database

2009-04-03 Thread Ryan Rawson
Another reason to perhaps avoid tons of versions is there is no query mechanism, nor will there ever be. The mechanism is limited to asking for either the last N versions, or all of them. If you are querying a date range, this is obviously a problem. -ryan On Fri, Apr 3, 2009 at 7:25 AM, stack

Fwd: Shift entirely to HBase?

2009-04-03 Thread Vaibhav Puranik
Thanks Stack and Erik. I will look at the backup MR job in the issue. BTW, we won't be able to use the table based indexer as it's not useful whenever you use column name as data (so that we can store multiple columns per row). Most of our tables have column as data and hence we cannot specify th

Re: Shift entirely to HBase?

2009-04-03 Thread Erik Holstad
Hi Vaibhav! *https://issues.apache.org/jira/browse/HBASE-974 *is what Stack is talking about, just look at the last comment that I made so you know why it is a little bit slow at the moment. Internally on Streamy we use a setup class so that every table is only initiated one and you get and put th

Re: Shift entirely to HBase?

2009-04-03 Thread stack
Oh, you could also set your columns to flush more frequently than default so edits are presisted more often. St.Ack On Sat, Apr 4, 2009 at 12:16 AM, stack wrote: > Good. I'm glad you are doing the evaluations. My guess is that you'll > need to wait on 0.20.0 to get the realtime numbers you'll

Re: Shift entirely to HBase?

2009-04-03 Thread stack
Good. I'm glad you are doing the evaluations. My guess is that you'll need to wait on 0.20.0 to get the realtime numbers you'll be happy with. Look through hbase JIRAs for issues on backups. There are a few. I think Erik Holstad's the most up-to-date. Check it out (A google summer of code pro

Re: HQL and hbase shell

2009-04-03 Thread stack
On top of Eriks' message, this might help: http://wiki.apache.org/hadoop/Hbase/Shell St.Ack On Fri, Apr 3, 2009 at 9:50 PM, Jae Joo wrote: > Does anyone help me to run HQL? I ran bin/hbase shell, but it looks like > tha > it is not HQL. > > Jae Joo >

Re: HQL and hbase shell

2009-04-03 Thread Erik Holstad
Hi Jae! The HQL commands have been deprecated since I believe 0.18 or maybe earlier. Sorry for the inconvenience, I believe this was done so that people would understand that this is not a relational database and help them to start thinking in HBase terms instead. Regards Erik

RE: Novice Hbase user - Hbase restart problem solved

2009-04-03 Thread Taylor, Ronald C
Hi St. Ack, Erik, Thanks very much for the help. I now have Hbase back up and running. I actually completely deleted the HDFS directory, and reformatted from scratch. I also deleted everything pertaining to Hadoop and Hbase in the /tmp directory before doing a new invocation, as Erik suggested.

HQL and hbase shell

2009-04-03 Thread Jae Joo
Does anyone help me to run HQL? I ran bin/hbase shell, but it looks like tha it is not HQL. Jae Joo

Re: Shift entirely to HBase?

2009-04-03 Thread Vaibhav Puranik
Stack, We are still trying to explore answers to these questions. For example, we are at this moment doing performance testing on hbase to see whether it can be used as a real time database. We haven't finalized the new schema - I am exploring the table based indexes feature to see how it can he

Re: timestamp uses

2009-04-03 Thread Erik Holstad
Hi Genady! If everything goes as planned there will be a possibility to input a TimeRange into every get query in 0.20, so that you will be able to do the call, give me all data from row r, family f and column c in the timerange t2 to t1. The nice thing about the new implementation is also that you

Re: timestamp uses

2009-04-03 Thread Bradford Cross
I have another thread in progress re using HBase as a financial time series database. http://mail-archives.apache.org/mod_mbox/hadoop-hbase-user/200904.mbox/%3cea7d6a710904011948l2a79bf18hfbc7a6102676b...@mail.gmail.com%3e On Fri, Apr 3, 2009 at 8:38 AM, Jim Kellerman (POWERSET) < jim.keller...

RE: timestamp uses

2009-04-03 Thread Jim Kellerman (POWERSET)
There are a number of Jiras open to address this issue. See HBASE-33, HBASE-52 and HBASE-1182 --- Jim Kellerman, Powerset (Live Search, Microsoft Corporation) > -Original Message- > From: Genady [mailto:gena...@exelate.com] > Sent: Friday, April 03, 2009 2:56 AM > To: hbase-user@hadoop.ap

Re: try to run PerformanceEvaluation and encounter RetriesExhaustedException

2009-04-03 Thread stack
Thanks for the detailed description of your experiences. On Thu, Apr 2, 2009 at 10:03 AM, Jun Li wrote: > ... > > (1) I first changed HBASE_HEAPSIZE defined in hbase-env.sh from 1 GB to 2 > GB, and run: bin/hadoop org.apache.hadoop.hbase.PerformanceEvaluation > sequentialWrite 4. It fails at

Re: financial time series database

2009-04-03 Thread stack
On Thu, Apr 2, 2009 at 9:53 PM, Wesley Chow wrote: > > Are there technical limitations to the number of different timestamps per > cell? If it's the case that you're doing to be dealing with tens of > thousands to millions of entries all at one cell, perhaps you should check > that to make sure i

RE: timestamp uses

2009-04-03 Thread Genady
Jonathan, Please correct me If I wrong, but one of the features that HBase obviously missing is possibility to select records based on timestamp range(week, month, etc.), as far as understand, it's possible to make select with specified timestamps, but in a most cases you want to select ranges. To