Re: Creating namespace and column family from multiple nodes concurrently

2013-05-23 Thread Emalayan Vairavanathan
I am sorry if I was not clear. I was using nodes to refer machines (or vice versa). Let me put in another way...  The application is composed of multiple instances of an executable. The application runs on multiple machines concurrently. All the instances are going to issue the same CQL comman

Re: High performance disk io

2013-05-23 Thread aaron morton
> I am currently trying to really study the effect of the width of a row > (being in multiple sstables) vs its 95th percentile read time. I'd be interested to see your findings. Is use 3+ SSTables per read as (from cfhistograms) as a warning sign to dig deeper in the data model. Also the type

Re: For those using Cassandra from .Net

2013-05-23 Thread aaron morton
Thanks, when and were is the talk ? Cheers - Aaron Morton Freelance Cassandra Consultant New Zealand @aaronmorton http://www.thelastpickle.com On 23/05/2013, at 6:42 AM, Peter Lin wrote: > > NativeX is giving a talk about using Cassandra with .Net. Our firm created a > port

Re: bootstrapping a new node...

2013-05-23 Thread aaron morton
> 1. Is compaction supposed to go off during a bootstrapping node? When a new file is received during streaming it is added to the list of SSTables for the CF through the same process as a SSTable flush. Once the SStable count gets high enough compaction will do it's thing. > 2. I seem to rec

Re: Problem with streaming data from Hadoop: DecoratedKey(-1, )

2013-05-23 Thread aaron morton
> Any other ideas? Sounds like a nasty heisenbug, can you replace or rebuild the machine? Cheers - Aaron Morton Freelance Cassandra Consultant New Zealand @aaronmorton http://www.thelastpickle.com On 21/05/2013, at 9:36 PM, Michal Michalski wrote: > I've finally had some time

Re: Cassandra hangs on large hinted handoffs

2013-05-23 Thread Edward Capriolo
For some reason the 1.0.7 hints actually use a super column :) On Thu, May 23, 2013 at 6:18 PM, aaron morton wrote: > I know how this sounds, but upgrading to 1.1.11 is the best approach. > 1.0X is not getting any fixes, 1.1X is the most stable and still getting > some patches, and 1.2 is stable

Re: Cassandra hangs on large hinted handoffs

2013-05-23 Thread aaron morton
I know how this sounds, but upgrading to 1.1.11 is the best approach. 1.0X is not getting any fixes, 1.1X is the most stable and still getting some patches, and 1.2 is stable and in use. Hint storage has been redesigned in 1.2. > Any suggestions on how to make the cluster more tolerant to dow

Re: Cassandra read reapair

2013-05-23 Thread aaron morton
If you are reading and writing at CL QUOURM and getting inconsistent results that sounds like a bug. If you are mixing the CL levels such that R + W <= N then it's expected behaviour. Can you reproduce the issue outside of your app ? Cheers - Aaron Morton Freelance Cassandra

Re: column with TTL of 10 seconds lives very long...

2013-05-23 Thread Robert Coli
On Wed, May 22, 2013 at 11:32 PM, Tamar Fraenkel wrote: > I am using Hector HLockManagerImpl, which creates a keyspace named > HLockManagerImpl and CF HLocks. > For some reason I have a row with single column that should have expired > yesterday who is still there. > I tried deleting it using cli,

Re: Creating namespace and column family from multiple nodes concurrently

2013-05-23 Thread Arthur Zubarev
so where the multiple nodes are? I am just puzzled From: Emalayan Vairavanathan Sent: Thursday, May 23, 2013 3:43 PM To: Arthur Zubarev ; user@cassandra.apache.org Subject: Re: Creating namespace and column family from multiple nodes concurrently "Would each device/machine have its own keyspa

Re: Creating namespace and column family from multiple nodes concurrently

2013-05-23 Thread Robert Coli
On Thu, May 23, 2013 at 12:07 PM, Emalayan Vairavanathan wrote: > Do you have any idea how Cassandra is going to handle concurrent namespace > and column family creation (Here all the instances are going to create the > same namespace and column families concurrently)? > [...] > However I am not s

Re: Creating namespace and column family from multiple nodes concurrently

2013-05-23 Thread Emalayan Vairavanathan
"Would each device/machine have its own keyspace?" No. All the machines are going to run the exactly same CQL commands and going to create the same namespace and column families. Thank you Emalayan From: Arthur Zubarev To: Emalayan Vairavanathan ; user@cassan

Re: Creating namespace and column family from multiple nodes concurrently

2013-05-23 Thread Arthur Zubarev
Would each device/machine have its own keyspace? Basically, your client needs to take care of a successful creation of the schema and any other verifications and it is going to be time consuming. From: Emalayan Vairavanathan Sent: Thursday, May 23, 2013 3:07 PM To: user@cassandra.apache.org S

Re: Creating namespace and column family from multiple nodes concurrently

2013-05-23 Thread Emalayan Vairavanathan
Hi Arthur and Farraz, Thank you for getting back to me. I am trying to avoid sync among concurrent instances and this is why I am preferring Option - 2. Further in my application, I have reasonable window between the application initialization phase and the application runtime.  So as long as

Re: write time of CQL3 set items

2013-05-23 Thread Sylvain Lebresne
> Does anyone know I way I could expose the write time of set items? > You cannot currently unfortunately. The problem is really just an API one. Since currently you can only ever query a full collection, you cannot apply writeTime() to only an element, and applying it to the whole collectio

Re: exception causes streaming to hang forever

2013-05-23 Thread Yuki Morishita
What kind of error does the other end of streaming(/10.10.42.36) say? On Wed, May 22, 2013 at 5:19 PM, Hiller, Dean wrote: > We had 3 nodes roll on good and the next 2, we see a remote node with this > exception every time we start over and bootstrap the node > > ERROR [Streaming to /10.10.42.36

Re: Cassandra 1.2 TTL histogram problem

2013-05-23 Thread Yuki Morishita
> Are you sure that it is a good idea to estimate remainingKeys like that? Since we don't want to scan every row to check overlap and cause heavy IO automatically, the method can only do the best-effort type of calculation. In your case, try running user defined compaction on that sstable file. It

write time of CQL3 set items

2013-05-23 Thread Keith Wright
Hi all, I am using C* 1.2.4 with CQL3 and am taking advantage of the new collection support. One usage case I have is that I want a set of text and I need to know the time when each item in the set was written. If I understand CQL3 correctly, the underlying data engine utilizes composites

Re: High performance disk io

2013-05-23 Thread Edward Capriolo
I have used both rotation disks with lots of RAM as well as SSD devices. An important thing to consider is that SSD devices are not magic. You have big-o-notation in several places. 1) more data large bloom filters 2) more data (larger key caches) JVM overhead 3) more requests more young gen JVM ov

RE: High performance disk io

2013-05-23 Thread Christopher Wirt
Hi Igor, I was talking about 99th percentile from the Cassandra histograms when I said '1 or 2 ms for most cf'. But we have measured client side too and generally get a couple ms added on top.. as one might expect. Anyone interested - diskio (my original question) we have tried out t

Re: High performance disk io

2013-05-23 Thread Igor
Hello Christopher, BTW, are you talking about 99th percentiles on client side, or about percentiles from cassandra histograms for CF on cassandra side? Thanks! On 05/22/2013 05:41 PM, Christopher Wirt wrote: Hi Igor, Yea same here, 15ms for 99^th percentile is our max. Currently getting o

Re: Commit Log Magic

2013-05-23 Thread Jonathan Ellis
Sstables must be sorted by token, or we can't compact efficiently. Since writes usually do not arrive in token order, we stage them first in a memtable. (cc user@) On Thu, May 23, 2013 at 8:44 AM, Ansar Rafique wrote: > Hi Jonathan, > > I am Ansar Rafique and I asked you few questions 2 week ago

Re: column with TTL of 10 seconds lives very long...

2013-05-23 Thread Tamar Fraenkel
good point! *Tamar Fraenkel * Senior Software Engineer, TOK Media [image: Inline image 1] ta...@tok-media.com Tel: +972 2 6409736 Mob: +972 54 8356490 Fax: +972 2 5612956 On Thu, May 23, 2013 at 2:25 PM, wrote: > (Probably will not solve your problem, but worth mentioning): It’s not >

RE: column with TTL of 10 seconds lives very long...

2013-05-23 Thread moshe.kranc
(Probably will not solve your problem, but worth mentioning): It's not enough to check that the clocks of all the servers are synchronized - I believe that the client node sets the timestamp for a record being written. So, you should also check the timestamp on your Hector client nodes. From: T

Re: column with TTL of 10 seconds lives very long...

2013-05-23 Thread Tamar Fraenkel
Hi! TTL was set: [default@HLockingManager] get HLocks['/LockedTopic/31a30c12-652d-45b3-9ac2-0401cce85517']; => (column=69b057d4-3578-4326-a9d9-c975cb8316d2, value=36396230353764342d333537382d343332362d613964392d633937356362383331366432, timestamp=1369307815049000, ttl=10) Also, all other lock c

RE: column with TTL of 10 seconds lives very long...

2013-05-23 Thread moshe.kranc
Maybe you didn't set the TTL correctly. Check the TTL of the column using CQL, e.g.: SELECT TTL (colName) from colFamilyName WHERE ; From: Felipe Sere [mailto:felipe.s...@1und1.de] Sent: Thursday, May 23, 2013 1:28 PM To: user@cassandra.apache.org Subject: AW: column with TTL of 10 seconds lives v

AW: column with TTL of 10 seconds lives very long...

2013-05-23 Thread Felipe Sere
This is interesting as it might affect me too :) I have been observing deadlocks with HLockManagerImpl which dont get resolved for a long time even though the columns with the locks should only live for about 5-10secs. Any ideas how to investigate this further from the Cassandra-side? ___

Re: column with TTL of 10 seconds lives very long...

2013-05-23 Thread Tamar Fraenkel
Thanks for the response. Running date simultaneously on all nodes (using parallel ssh) shows that they are synced. Tamar *Tamar Fraenkel * Senior Software Engineer, TOK Media [image: Inline image 1] ta...@tok-media.com Tel: +972 2 6409736 Mob: +972 54 8356490 Fax: +972 2 5612956 On Thu,

Re: column with TTL of 10 seconds lives very long...

2013-05-23 Thread Nikolay Mihaylov
Did you synchronized the clocks between servers? On Thu, May 23, 2013 at 9:32 AM, Tamar Fraenkel wrote: > Hi! > I have Cassandra cluster with 3 node running version 1.0.11. > > I am using Hector HLockManagerImpl, which creates a keyspace named > HLockManagerImpl and CF HLocks. > For some reason