Re: [mapreduce] ColumnFamilyRecordWriter hidden reuse

2011-01-25 Thread Mck
> > is "d.timestamp = System.currentTimeMillis();" ok? > > You are correct that microseconds would be better but for the test it > doesn't matter that much. Have you tried. I'm very new to cassandra as well, and always uncertain as to what to expect... > ByteBuffer bbKey = ByteBufferUtil.clo

Re: [mapreduce] ColumnFamilyRecordWriter hidden reuse

2011-01-25 Thread Patrik Modesto
On Tue, Jan 25, 2011 at 19:09, Mick Semb Wever wrote: > In fact i have another problem (trying to write an empty byte[], or > something, as a key, which put one whole row out of whack, ((one row in > 25 million...))). > > But i'm debugging along the same code. > > I don't quite understand how the

RE: the java client problem

2011-01-25 Thread Raoyixuan (Shandy)
I had find the loasschemafrom yaml by jconsole,How to load the schema ? From: Ashish [mailto:paliwalash...@gmail.com] Sent: Friday, January 21, 2011 8:10 PM To: user@cassandra.apache.org Subject: Re: the java client problem check cassandra-install-dir/conf/cassandra.yaml start cassandra connect

Re: Re-partitioning the cluster with nodetool: what's happening?

2011-01-25 Thread Aaron Morton
It can take a bit of thinking time for the nodes to work out what to stream, the bottom of this page http://wiki.apache.org/cassandra/Streaming talks about how to watch whats happening. If it does get stuck let us know. AaronOn 26 Jan, 2011,at 11:42 AM, buddhasystem wrote: Correction -- what I mea

Re: get_range_slices getting deleted rows

2011-01-25 Thread Roshan Dawrani
No, checking the key will not do. You will need to check if row.getColumnSlice().getColumns() is empty or not. That's what I do and it works for me. On Wed, Jan 26, 2011 at 4:53 AM, Nick Santini wrote: > thanks, > so I need to check the returned slice for the key to verify that is a valid > row

RE: Errors During Compaction

2011-01-25 Thread Dan Hendry
Limited joy I would say :) No long term damage at least. I ended up deleting (moving to another disk) all the sstables which fixed the problem. I ran in to even more problems during repair (detailed in another recent email) but it seems to have worked regardless. Just to be safe, I am in th

Re: get_range_slices getting deleted rows

2011-01-25 Thread Nick Santini
thanks, so I need to check the returned slice for the key to verify that is a valid row and not a deleted one? Nicolas Santini On Wed, Jan 26, 2011 at 12:16 PM, Narendra Sharma wrote: > Yes. See this http://wiki.apache.org/cassandra/FAQ#range_ghosts > > -Naren > > > On Tue, Jan 25, 2011 at 2:

Fwd: CFP - Berlin Buzzwords 2011 - Search, Score, Scale

2011-01-25 Thread David G. Boney
This might interest the Cassandra community. - Sincerely, David G. Boney dbon...@semanticartifacts.com http://www.semanticartifacts.com Begin forwarded message: > From: Isabel Drost > Date: January 25, 2011 2:53:28 PM CST > To: u...@mahout.apache.org > Cc: gene...@lucene.apache.or

Re: get_range_slices getting deleted rows

2011-01-25 Thread Narendra Sharma
Yes. See this http://wiki.apache.org/cassandra/FAQ#range_ghosts -Naren On Tue, Jan 25, 2011 at 2:59 PM, Nick Santini wrote: > Hi, > I'm trying a test scenario where I create 100 rows in a CF, then > use get_range_slices to get all the rows, and I get 100 rows, so far so good > then after the tes

get_range_slices getting deleted rows

2011-01-25 Thread Nick Santini
Hi, I'm trying a test scenario where I create 100 rows in a CF, then use get_range_slices to get all the rows, and I get 100 rows, so far so good then after the test I delete the rows using "remove" but without a column or super column, this deletes the row, I can confirm that cos if I try to get i

Re: Re-partitioning the cluster with nodetool: what's happening?

2011-01-25 Thread buddhasystem
Correction -- what I meant to say that I do see announcements about streaming in the output, but these are stuck at 0%. -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Re-partitioning-the-cluster-with-nodetool-what-s-happening-tp5960843p5960851.

Re-partitioning the cluster with nodetool: what's happening?

2011-01-25 Thread buddhasystem
I'm trying re-partition my 4-node cluster to make the load exactly 25% on each node. As per recipes found in documentation, I calculate: >>> for x in xrange(4): ... print 2**127/4*x ... 0 42535295865117307932921825928971026432 85070591730234615865843651857942052864 1276058875953519237987654777

Re: Stress test inconsistencies

2011-01-25 Thread Anthony John
Look at iostat -x 10 10 when he active par tof your test is running. there should be something called svc_t - that should be in the 10ms range, and await should be low. Will tell you if IO is slow, or if IO is not being issued. Also, ensure that you ain't swapping with something like "swapon -s"

Re: Schema Question

2011-01-25 Thread Aaron Morton
Yeah, I was talking about create a ColumnFamily definition via the API. Not inserting data into an already defined column family. The recommened approach to creating your schema is via the build in bin/cassandra-cli command line tool. It has loads of build in help and here is an example of how to c

Re: Files not deleted after compaction and GCed

2011-01-25 Thread Jonathan Ellis
the other component types are deleted by this line: SSTable.delete(desc, components); On Tue, Jan 25, 2011 at 3:11 PM, Ching-Cheng Chen wrote: > Nope, no exception at all. > But if the same class > (org.apache.cassandra.io.sstable.SSTableDeletingReference) is responsible > for delete

Re: Does Major Compaction work on dropped CFs? Doesn't seem so.

2011-01-25 Thread Aaron Morton
You can run JConsole on your workstation and connect remotely to the nodes, it does not need to be run on the node itself. Connecting is discussed here http://wiki.apache.org/cassandra/MemtableThresholds and some help for connecting is here http://wiki.apache.org/cassandra/JmxGotchasThere is also a

Re: Files not deleted after compaction and GCed

2011-01-25 Thread Ching-Cheng Chen
Nope, no exception at all. But if the same class (org.apache.cassandra.io.sstable.SSTableDeletingReference) is responsible for delete other files, then that's not right. I checked the source code for SSTableDeletingReference, doesn't looks like it will delete other files type. Regards, Chen On

Re: Files not deleted after compaction and GCed

2011-01-25 Thread Jonathan Ellis
No, that is not expected. All the sstable components are removed in the same method; did you check the log for exceptions? On Tue, Jan 25, 2011 at 2:58 PM, Ching-Cheng Chen wrote: > Using cassandra 0.7.0 > The class org.apache.cassandra.io.sstable.SSTableDeletingReference only > remove the -

Re: Stress test inconsistencies

2011-01-25 Thread Oleg Proudnikov
buddhasystem bnl.gov> writes: > > > Oleg, > > I'm a novice at this, but for what it's worth I can't imagine you can have a > _sustained_ 1kHz insertion rate on a single machine which also does some > reads. If I'm wrong, I'll be glad to learn that I was. It just doesn't seem > to square with a

Re: Errors During Compaction

2011-01-25 Thread Aaron Morton
Dan how did you go with this? More joy, less joy or a continuation of the current level of joy? Aaron On 24/01/2011, at 9:38 AM, Dan Hendry wrote: > I have run into a strange problem and was hoping for suggestions on how to > fix it (0.7.0). When compaction occurs on one node for what appear

Re: Stress test inconsistencies

2011-01-25 Thread Oleg Proudnikov
Brandon Williams gmail.com> writes: > > On Tue, Jan 25, 2011 at 1:23 PM, Oleg Proudnikov cloudorange.com> wrote: > > When I run contrib/stress with a higher thread count, the server does scale to > 200 inserts a second with latency of 200ms. At the same time Windows desktop > scales to 900 ins

Files not deleted after compaction and GCed

2011-01-25 Thread Ching-Cheng Chen
Using cassandra 0.7.0 The class org.apache.cassandra.io.sstable.SSTableDeletingReference only remove the -Data.db file, but leave the xxx-Compacted, xxx-Filter.db, xxx-Index.db and xxx-Statistics.db intact. And that's the behavior I saw.I ran manual compact then trigger a GC from jconsole

Re: Stress test inconsistencies

2011-01-25 Thread Brandon Williams
On Tue, Jan 25, 2011 at 1:23 PM, Oleg Proudnikov wrote: > When I run contrib/stress with a higher thread count, the server does scale > to > 200 inserts a second with latency of 200ms. At the same time Windows > desktop > scales to 900 inserts a second and latency of 120ms. There is a huge > diffe

Re: Stress test inconsistencies

2011-01-25 Thread buddhasystem
Oleg, I'm a novice at this, but for what it's worth I can't imagine you can have a _sustained_ 1kHz insertion rate on a single machine which also does some reads. If I'm wrong, I'll be glad to learn that I was. It just doesn't seem to square with a typical seek time on a hard drive. Maxim -- V

Re: Stress test inconsistencies

2011-01-25 Thread Oleg Proudnikov
Tyler Hobbs riptano.com> writes: > Try using something higher than -t 1, like -t 100.- Tyler > Thank you, Tyler! When I run contrib/stress with a higher thread count, the server does scale to 200 inserts a second with latency of 200ms. At the same time Windows desktop scales to 900 inserts a s

Re: Forcing GC w/o jconsole

2011-01-25 Thread buddhasystem
Thanks! It doesn't seem to have any effect on GCing dropped CFs, though. Maxim -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Forcing-GC-w-o-jconsole-tp5956747p5960100.html Sent from the cassandra-u...@incubator.apache.org mailing list archive

Re: [mapreduce] ColumnFamilyRecordWriter hidden reuse

2011-01-25 Thread Mick Semb Wever
On Tue, 2011-01-25 at 14:16 +0100, Patrik Modesto wrote: > The atttached file contains the working version with cloned key in > reduce() method. My other aproache was: > > > context.write(ByteBuffer.wrap(key.getBytes(), 0, key.getLength()), > > Collections.singletonList(getMutation(key))); > > Wh

Re: Stress test inconsistencies

2011-01-25 Thread Tyler Hobbs
Try using something higher than -t 1, like -t 100. - Tyler On Mon, Jan 24, 2011 at 9:38 PM, Oleg Proudnikov wrote: > Hi All, > > I am struggling to make sense of a simple stress test I ran against the > latest > Cassandra 0.7. My server performs very poorly compared to a desktop and > even a > n

Re: client threads locked up - JIRA ISSUE 1594

2011-01-25 Thread Nate McCall
What version of the Thrift API are you using? (In general, you should use an existing client library rather than rolling your own - I recommend Hector: https://github.com/rantav/hector). On Tue, Jan 25, 2011 at 12:38 AM, Arijit Mukherjee wrote: > I'm using Cassandra 0.6.8. I'm not using Hector -

Re: Schema Question

2011-01-25 Thread David McNelis
I'm fairly certain Aaron is referring to named families like BlogEntries, not named columns (i-got-a-new-guitar). On Tue, Jan 25, 2011 at 4:37 AM, Andy Burgess wrote: > Aaron, > > A question about one of your general points, "do not create CF's on the > fly" - what, exactly, does this mean? Do y

Re: [mapreduce] ColumnFamilyRecordWriter hidden reuse

2011-01-25 Thread Patrik Modesto
Hi Mick, attached is the very simple MR job, that deletes expired URL from my test Cassandra DB. The keyspace looks like this: Keyspace: Test: Replication Strategy: org.apache.cassandra.locator.SimpleStrategy Replication Factor: 2 Column Families: ColumnFamily: Url2 Columns sort

Re: [mapreduce] ColumnFamilyRecordWriter hidden reuse

2011-01-25 Thread Mick Semb Wever
On Tue, 2011-01-25 at 09:37 +0100, Patrik Modesto wrote: > While developing really simple MR task, I've found that a > combiantion of Hadoop optimalization and Cassandra > ColumnFamilyRecordWriter queue creates wrong keys to send to > batch_mutate(). I've seen similar behaviour (junk rows being w

Re: Upgrading from 0.6 to 0.7.0

2011-01-25 Thread Daniel Josefsson
Yes, it should be possible to try. We have not yet quite decided which way to go, think operations won't be happy with upgrading both server and client at the same time. Either we upgrade to 0.7.0 (currently does not look very likely), or we go to 0.6.9 and patch with TTL. I'm not too sure what a

Re: Schema Question

2011-01-25 Thread Andy Burgess
Aaron, A question about one of your general points, "do not create CF's on the fly" - what, exactly, does this mean? Do you mean named column families, like "BlogEntries" from Sam's example, or do you mean column family keys, like "i-got-a-new-guitar"? If it's the l

[mapreduce] ColumnFamilyRecordWriter hidden reuse

2011-01-25 Thread Patrik Modesto
Hi, I play with Cassandra 0.7.0 and Hadoop, developing simple MapReduce tasks. While developing really simple MR task, I've found that a combiantion of Hadoop optimalization and Cassandra ColumnFamilyRecordWriter queue creates wrong keys to send to batch_mutate(). The proble is in the reduce part,

Re: Basic question on a write operation immediately followed by a read

2011-01-25 Thread Roshan Dawrani
2011/1/25 Wangpei (Peter) > for your 1-node cluster, ANY is the only consistency level that client > may returns BEFORE node write to memory table. > > And read op on the node read both the memory table and SSTable. > > > > It real puzzle me. :( > Please don't be puzzled just yet. :-) As I sa

Re: Basic question on a write operation immediately followed by a read

2011-01-25 Thread Wangpei (Peter)
for your 1-node cluster, ANY is the only consistency level that client may returns BEFORE node write to memory table. And read op on the node read both the memory table and SSTable. It real puzzle me. :( 发件人: Roshan Dawrani [mailto:roshandawr...@gmail.com] 发送时间: 2011年1月25日 15:47 收件人: user@cassa