Re: Giant sets of ordered data

2010-06-04 Thread Benjamin Black
Use index rows named for time intervals that contain columns named for the row keys of the base data rows from each interval. b On Wed, Jun 2, 2010 at 8:32 AM, David Boxenhorn wrote: > How do I handle giant sets of ordered data, e.g. by timestamps, which I want > to access by range?

Re: Giant sets of ordered data

2010-06-03 Thread yoshiyuki kanno
ured data to stick under a column (named by the > >>>> timestamp), then you can serialize and unserialize it yourself, or you > >>>> can use a supercolumn. It's effectively the same thing. Cassandra > >>>> only provides the super column s

Re: Giant sets of ordered data

2010-06-02 Thread Jonathan Shook
s the super column support as a convenience layer as it is >>>> currently implemented. That may change in the future. >>>> >>>> You didn't make clear in your question why a standard column would be >>>> less suitable. I presumed you had layered st

Re: Giant sets of ordered data

2010-06-02 Thread Jonathan Shook
idn't make clear in your question why a standard column would be >>> less suitable. I presumed you had layered structure within the >>> timestamp, hence my response. >>> How would you logically partition your dataset according to natural >>> application boundar

Re: Giant sets of ordered data

2010-06-02 Thread Ben Browning
ou had layered structure within the >> timestamp, hence my response. >> How would you logically partition your dataset according to natural >> application boundaries? This will answer most of your question. >> If you have a dataset which can't be partitioned into a reason

Re: Giant sets of ordered data

2010-06-02 Thread David Boxenhorn
can't be partitioned into a reasonable > size row, then you may want to use OPP and key concatenation. > > What do you mean by giant? > > On Wed, Jun 2, 2010 at 10:32 AM, David Boxenhorn > wrote: > > How do I handle giant sets of ordered data, e.g. by timestamps, which

Re: Giant sets of ordered data

2010-06-02 Thread Jonathan Shook
0 at 10:32 AM, David Boxenhorn wrote: > How do I handle giant sets of ordered data, e.g. by timestamps, which I want > to access by range? > > I can't put all the data into a supercolumn, because it's loaded into memory > at once, and it's too much data. > > Am

Re: Giant sets of ordered data

2010-06-02 Thread Ben Browning
's a tradeoff here between row width and read speed. Reading 1000 columns as a continuous slice from a single row will be very fast but reading 1000 columns as slices from 10 keys won't be as fast. Ben On Wed, Jun 2, 2010 at 11:32 AM, David Boxenhorn wrote: > How do I handle giant se

Giant sets of ordered data

2010-06-02 Thread David Boxenhorn
How do I handle giant sets of ordered data, e.g. by timestamps, which I want to access by range? I can't put all the data into a supercolumn, because it's loaded into memory at once, and it's too much data. Am I forced to use an order-preserving partitioner? I don't want th