node vs node latency

2012-07-06 Thread Deno Vichas
all, what would explain a huge different (12ms vs 0.1ms) in read latency from node to node. i've got a 4 node cluster w/ replication factor of 3 using hector. i'm seeing these numbers with nodetool cfstats. thx, deno

how big can you slice

2012-07-06 Thread Deno Vichas
all, are there any guidelines to how much you can slice. how does total payload size vs # of column affect performance? thanks, deno

Re: Composite Slice Query returning non-sliced data

2012-07-06 Thread Sunit Randhawa
Aaron, For writing, i am using cli. Below is the piece of code that is reading column names of different types. Composite start = new Composite(); start.addComponent(0, beginTime, Composite.ComponentEquality.EQUAL);

Re: frequent node up/downs

2012-07-06 Thread feedly team
responses below. thanks! On Fri, Jul 6, 2012 at 3:09 PM, aaron morton wrote: > It looks like this happens when there is a promotion failure. > > > Java Heap is full. > Memory is fragmented. > Use C for web scale. > unfortunately i became too dumb to use C around 2004. camping accident. > > Also

Java heap space on Cassandra start up version 1.0.10

2012-07-06 Thread Jason Hill
Hello friends, I'm getting a: ERROR 22:50:29,695 Fatal exception in thread Thread[SSTableBatchOpen:2,5,main] java.lang.OutOfMemoryError: Java heap space error when I start Cassandra. This node was running fine and after some server work/upgrades it started throwing this error when I start the Ca

RE: Dynamic CF

2012-07-06 Thread Derek Williams
To clarify, I haven't tested it with compact storage, but it will only use the first part of the primary key without compact storage. On Jul 6, 2012 4:50 PM, "Derek Williams" wrote: > Actually, my solution only makes a row for each unique value of the first > part of the primary key. > On Jul 6,

RE: Dynamic CF

2012-07-06 Thread Derek Williams
Actually, my solution only makes a row for each unique value of the first part of the primary key. On Jul 6, 2012 3:07 PM, "Leonid Ilyevsky" wrote: > Thanks Derek, I actually saw your post, but this is not a good > substitution for a real dynamic table. Your trick will make a table with > even m

Random errors using phpcassa

2012-07-06 Thread Marco Matarazzo
Greetings. I am experiencing problems using a cassandra DB with phpcassa, but I am unable to understand if the error is on phpcassa client itself or on cassandra… As far as I understand, phpcassa just "pass" it to the thrift layer, and errors I am seeing are coming back from cassandra itself, s

RE: Dynamic CF

2012-07-06 Thread Leonid Ilyevsky
Thanks Derek, I actually saw your post, but this is not a good substitution for a real dynamic table. Your trick will make a table with even more rows; I wanted wide rows so the reads and writes would be more efficient. From: Derek Williams [mailto:de...@fyrie.net] Sent: Friday, July 06, 2012 4:

Re: Dynamic CF

2012-07-06 Thread Derek Williams
On Fri, Jul 6, 2012 at 2:49 PM, Leonid Ilyevsky wrote: > At this point I am really confused about what direction Cassandra is > going. CQL 3 has the benefit of composite keys, but no dynamic columns. > I thought, the whole point of Cassandra was to provide dynamic tables. > See my earlier post on

Re: Dynamic CF

2012-07-06 Thread Jeremy Hanna
Oh you're right - that has changed in cql 3: http://www.datastax.com/docs/1.1/configuration/storage_configuration#cf-storage-options I'm not sure then why that wouldn't work as it seems valid with the option and its value (double). On Jul 6, 2012, at 3:15 PM, Leonid Ilyevsky wrote: > Thanks Jer

RE: Dynamic CF

2012-07-06 Thread Leonid Ilyevsky
Indeed the cql 3.0 reference in cassandra11.pdf does not mention the dynamic tables whatsoever. Now I found the following posting: http://www.datastax.com/support-forums/topic/flexible-schema-in-cql-3 where the moderator said " Yes, CQL 3 expects all columns to be defined in the metadata". At

RE: Dynamic CF

2012-07-06 Thread Leonid Ilyevsky
Thanks Jeremy, but this doesn't work for me. I am using cql3, because I need new features like composite keys. The manual you pointed to is for 2.0. I have suspicion that cql3 does not support dynamic tables at all. Is there a manual for cql3? -Original Message- From: Jeremy Hanna [mailt

Re: Multiple keyspace question

2012-07-06 Thread Shahryar Sedghi
Aaron I am going to have many (over 50 eventually) keyspaces with limited number of CFs (5-6) do you think this one can cause a problem too. Thanks On Fri, Jul 6, 2012 at 2:28 PM, aaron morton wrote: > Also, all CF's in the same KS share one commit log. So all writes for the > row row key, acro

Re: Dynamic CF

2012-07-06 Thread Jeremy Hanna
you can use the cqlsh help but it will eventually refer you to a cql reference such as this one that says what the options are. Looks like you need just 'default_validation'. http://www.datastax.com/docs/1.0/references/cql/index#cql-column-family-storage-parameters On Jul 6, 2012, at 2:13 PM,

Dynamic CF

2012-07-06 Thread Leonid Ilyevsky
Can I create a dynamic Cf using cql3? What is the syntax? Here is what I tried: cqlsh:test> create table mytest ( ... asset ascii, datetime int, count int, primary key(asset, datetime) ) ... with default_validation_class = double; Bad Request: default_validation_class is not a v

Re: frequent node up/downs

2012-07-06 Thread aaron morton
> It looks like this happens when there is a promotion failure. Java Heap is full. Memory is fragmented. Use C for web scale. > Also is it normal to see the "Heap is xx full. You may need to reduce > memtable and/or cache sizes" message quite often? I haven't turned on row > caches or chan

Re: Composite key in thrift java api - still need help!!

2012-07-06 Thread aaron morton
AFAIK in all cases we consider values to be just bytes. They are passed to the ADT to turn into type aware instances. Can you try doing what you want to do with Hector or Astyanax? Then take a look at how they work if you still want to roll your own. Cheers - Aaron Morton Fre

Re: Cassandra and Tableau

2012-07-06 Thread aaron morton
Here are two links I've noticed in my travels, have not looked into what they offer. http://www.pentaho.com/big-data/nosql/cassandra/ http://www.jaspersoft.com/bigdata Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 7/07/2012, at 3:03 AM

Recommender system

2012-07-06 Thread Carlos Sanchez
All, Is anyone aware of a [distributed] Recommender system (i.e. amazon recommendations) that uses cassandra as a back-end? Thanks a lot Carlos

Re: cassandra on re-Start

2012-07-06 Thread aaron morton
> And when i try to recreate the keyspace 'XYZ' ,it gives a Ttransport > exception. Can you provide the error ? Having directories on disk will not stop keyspaces or column families from being created. If this is a test case, and you just want to keep moving, I would clear it all and start a

Re: Multiple keyspace question

2012-07-06 Thread aaron morton
Also, all CF's in the same KS share one commit log. So all writes for the row row key, across all CF's, are committed at the same time. Some other settings, such as caches in 1.1, are machine wide. If you have a small KS for something like app config, I'd say go with whatever feels right. If

Re: Tripling size of a cluster

2012-07-06 Thread aaron morton
If you have the time yes I would wait for the bootstrap to finish. It will make you life easier. good luck. - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 6/07/2012, at 7:12 PM, Mariusz Dymarek wrote: > Hi, > we`re in the middle of extending o

Re: GC freeze just after repair session

2012-07-06 Thread aaron morton
> I was thinking of decreasing concurrent_compactors and > in_memory_compaction_limit to go easy on GC I've used that technique to reduce gc pressure during compactions before. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 6/07/2012, at

Re: Composite Slice Query returning non-sliced data

2012-07-06 Thread aaron morton
Can you provide an example of writing and reading column names of a different type. Thanks - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 6/07/2012, at 11:30 AM, Sunit Randhawa wrote: > HI Aaron, > > It is > > create column family CF >

Re: frequent node up/downs

2012-07-06 Thread feedly team
I reduced the load and the problem hasn't been happening as much. After enabling gc logging, I see messages mentioning promotion failed when the pauses happen. It looks like this happens when there is a promotion failure. From reading on the web it looks like I could try reducing the CMSInitiatingO

RE: Composite key in thrift java api - still need help!!

2012-07-06 Thread Leonid Ilyevsky
I tried it, does not work for my batch_mutate (I tried both CompositeType.Builder and also my own implementation following the code examples). So it seems that the format with short length before the field and zero byte after does not work for the key. I found by experiment and debugger that in

Re: Cassandra and Tableau

2012-07-06 Thread Brian O'Neill
Robin, We have the same issue right now. We use Tableau for all of our reporting needs, but we couldn't find any acceptable bridge between it and Cassandra. We ended up using cassandra-triggers to replicate the data to Oracle. https://github.com/hmsonline/cassandra-triggers/ Let us know if you

RE: Composite key in thrift java api

2012-07-06 Thread Leonid Ilyevsky
Thanks Aaron, I guess, you suggest I just borrow the logic from that CompositeType.java, the "decompose" method, to comply with the protocol. I thought I can just call it, but I don't see the "decompose(Object... objects)" in the current 1.1.2 version, so probably your snippet is from the olde

Re: cassandra on re-Start

2012-07-06 Thread puneet loya
Hi, I think you did not get what i stated above: I will restate the complete problem again. I had created a keyspace named 'XYZ' in linux. I was able to access it through the java app. The system was then restarted once. Now when we tried to access the keyspace 'XYZ' ,it says it cannot locate

Re: Multiple keyspace question

2012-07-06 Thread Robin Verlangen
Hi Ben, The amount of keyspaces is not the problem: the amount of column families is. Each column family adds a certain amount of memory usage to the system. You can cope with this by adding memory or using generic column families that store different types of data. With kind regards, Robin Verl

Cassandra and Tableau

2012-07-06 Thread Robin Verlangen
Hi there, Is there anyone out there who's using Tableau in combination with a Cassandra cluster? There seems to be no standard solution to connect, at least I couldn't find one. Does anyone know how to tackle this problem? With kind regards, Robin Verlangen *Software engineer* * * W http://www.

Tripling size of a cluster

2012-07-06 Thread Mariusz Dymarek
Hi, we`re in the middle of extending our cluster from 10 to 30 nodes, we`re running cassandra 1.1.1... We`ve generated initial tokens for new nodes: "0": 0, # existing: node01-01 "1": 5671372782015641057722910123862803524, # new: node02-07 "2": 113427455640