Frequent updates of freshly written columns

Are the very freshly written columns to a row in memtables, efficiently updated/overwritten by edited/new column values. After flushing of memtable, are those(edited + unedited ones) columns stored together on disk (in same blocks!?) as if they were written in one single operation or same time ??

Re: Inconsistent result in super range slice query (reversed order)

I'm unable to reproduce this in pycassa starting with a clean database. Are you doing anything else to these rows besides inserting them? Here's the complete script I'm using below. Could you confirm that this causes problems for you? - Tyler = import sys import pycassa pool = pycass

Re: Understand eventually consistent

But, the reason that it isn't safe to say that we are a strongly consistent store is that if 2 of your 3 replicas were to die and come back with no data, QUORUM might return the wrong result. A requirement of a strongly consistent store is that replicas cannot begin answering queries until they ar

Re: Inconsistent result in super range slice query (reversed order)

Hi Aaron, Range slice means get_range_slices() in thrift api, createSuperSliceQuery in hector, get_range() in pycassa. The example code in pycassa is attached below. The problem is a little bit complicated to explain. I'll try to describe in examples. Here are 8 super column names which exist in

Re: RE: Request For 0.6.12 Release

Thank the committers for doing the release. With regard to stability / communication in the last week or two - http://www.mail-archive.com/dev@cassandra.apache.org/msg01753.htmlAaron  On 18 Feb, 2011,at 03:17 PM, Gregory Szorc wrote:Aaron, Thank you very much for initiating the voting process. I’m

a question about the "token" and “key" in KeyRange

I saw on this page: http://wiki.apache.org/cassandra/API07 *KeyRange A KeyRange is used by get_range_slices to define the range of keys to get the slices for. The semantics of start keys and tokens are slightly different. Keys are start-inclusive; tokens are start-exclusive. Token ranges may als

RE: Request For 0.6.12 Release

Aaron, Thank you very much for initiating the voting process. I'm looking forward to running this release. Was there any discussion around improving the communication of known issues with releases? Gregory From: Aaron Morton [mailto:aa...@thelastpickle.com] Sent: Thursday, February 17, 2011 4

Re: Understand eventually consistent

For background...http://wiki.apache.org/cassandra/ArchitectureOverview(There is a section on consistency in there)For  deep background...http://www.allthingsdistributed.com/2008/12/eventually_consistent.htmlhttp://s3.amazonaws.com/AllThingsDistributed/sosp/amazon-dynamo-sosp2007.pdfIn short, yes (f

Understand eventually consistent

Why is Cassandra called eventually consistent data store? Wouldn't it be consistent if QUORAM is used? Another question is when I specify replication factor of 3 and write with factor of 2 and read with factor of 2 then what happens? 1. When write occurs cassandra will return to the client only

Re: memory consuption

Jonathan, When you get time could you please explain that a little more. Got a feeling I'm about to learn something :)ThanksAaronOn 18 Feb, 2011,at 01:36 PM, Jonathan Ellis wrote:Please note that this won't affect the actual memory in use, only how the OS reports it. On Thu, Feb 17, 2011 at 6:15

Re: memory consuption

Please note that this won't affect the actual memory in use, only how the OS reports it. On Thu, Feb 17, 2011 at 6:15 PM, Aaron Morton wrote: > Looks like you are using virtual memory for memmapped files. Change the > disk_access_mode to standard if you want to reduce the overall memory > usage.

Cassandra in Sydney or Melbourne ?

I'm thinking about visiting Sydney in the last week of March and Melbourne in the first week of April. Any user groups in those cities that would like to talk about Cassandra on around those times?Email me directly. cheersAaron

Re: memory consuption

Victor, I'm a bit confused here as we've now got two conversations in this thread. Can you please start a new thread about your memory problem, I'm not clear on the details. Please include...- cassandra version- JVM  heap settings from bin/cassandra.in.sh or conf/cassandra-env.sh - disk_access_mode

Re: Request For 0.6.12 Release

Gregory, There is a vote going on for 0.6.12 now http://www.mail-archive.com/dev@cassandra.apache.org/msg01808.htmlIf you have time grab the bin and give it a test http://people.apache.org/~eevans AaronOn 16 Feb, 2011,at 09:21 PM, Aaron Morton wrote:Have checked it's all in the 0.6 branch and aske

Re: memory consuption

Ruslan, Change your disk_access_mode standard to remove the virtual memory usage. Some info on JNAhttp://www.datastax.com/blog/whats-new-cassandra-066http://journal.paul.querna.org/articles/2010/11/11/enabling-jna-in-cassandra/AaronOn 18 Feb, 2011,at 09:00 AM, Aaron Morton wrote:What are you using

Re: memory consuption

Already done. The disk access mode is standard in storage-conf.xml (I am using 0.6.6 at the moment, I will upgrade to 0.7.x later). But this memory consumption is a real issue. 2011/2/17 Aaron Morton > Looks like you are using virtual memory for memmapped files. Change the > disk_access_mode to

Re: memory consuption

Looks like you are using virtual memory for memmapped files. Change the disk_access_mode to standard if you want to reduce the overall memory usage. AaronOn 18 Feb, 2011,at 09:34 AM, Victor Kabdebon wrote:Sorry I forgot to say that this is the partial result of : ps aux | grep cassandraBest regard

Re: cassandra & php

> > How does this connection pooling fit in with the TSocketPool.php classes? > Or am I off the wicket here? > Right now, TSocketPool is not being used (in pycassa or phpcassa); individual TSockets are managed within the library. TSocketPool may be a good alternative to this in the future, but I

Re: cassandra & php

Hi, How does this connection pooling fit in with the TSocketPool.php classes? Or am I off the wicket here? These are just a few of my observations in relation to what i have seen so far when working with PHP and Cassandra. I have been working with cassandra / php for the last 8 months now in

Health of the cluster

I have just started setting up Hector. I have 2 node cluster running in "foreground" for my test so that I can watch everything. One big question came to my mind that as I increase the cluster how do I verify the health of the cluster. How do I see that it is all good. Since I am new I can't qu

Re: cassandra & php

eager to test! thanks tyler ... On Thu, Feb 17, 2011 at 10:49 PM, Tyler Hobbs wrote: > what i'm not entirely happy with in using php versus java/hector is that >> there isn't any connection pooling. maybe that's just me and my poor >> skills. >> > > Better connection pooling and failover are o

Re: cassandra & php

> > what i'm not entirely happy with in using php versus java/hector is that > there isn't any connection pooling. maybe that's just me and my poor > skills. > Better connection pooling and failover are on the way. You can check on the progress in the connection-pooling branch here: https://gith

Re: Replica details

On Thu, Feb 17, 2011 at 1:41 PM, A J wrote: > Where can I get good detailed explanation of the various replication > options (Simple, Old Network and Network) along with snitches. I did > read the definitive guide but not really satisfied. > > Is there a good post somewhere explaining this ? > > I

Re: Updating/inserting into multiple column families using onemutator batch

You could try turning up the thrift_max_message_length_in_mb and thrift_framed_transport_size_in_mb (by default 16 and 15MB respectively) in cassandra.yaml to see if that helped. On Thu, Feb 17, 2011 at 2:46 PM, wrote: > Thanks. I will set the debug mode and see/share if it shows any relevant in

Re: Updating/inserting into multiple column families using onemutator batch

Thanks. I will set the debug mode and see/share if it shows any relevant info. The smaller batches of 20 or so column mutations had been working fine. After merging, the total # of mutations across all CFs must not be crossing 60-70. The problem is that it is not slow - it seems just hung there

Re: cant seem to figure out secondary index definition

How are you constructing the IndexSlicesQuery? Does it have an equals clause with that UUID as the column name? On Thu, Feb 17, 2011 at 11:32 AM, Roland Gude wrote: > Hi again, > > > > i am still having trouble with this. > > If I define the index using cli with these commands: > > create column

Re: memory consuption

2011/2/17 Aaron Morton > What are you using for disk_access_mode ? > I set config default: auto. Have you tried reducing the JVM head size? > No i set default settings > Have you added the Jna.jar file to lib/ ? This will allow Cassandra to lock > the JVM memory. > > No, and what does this me

Re: Updating/inserting into multiple column families using one mutator batch

log4-server.properties in the conf directory of cassandra (requires a restart) or via JMX through JConsole or similar on o.a.c.service.StorageService#setLog4jLevel Is there a threshold under which you can successfully insert in batch mode? Even with something low like 10 entries? On Thu, Feb 17,

Re: memory consuption

Sorry I forgot to say that this is the partial result of : ps aux | grep cassandra Best regards 2011/2/17 Victor Kabdebon > Oh right but Cassandra doesn't really respect that, I thought there was > another option to set that. > > Just for your information, I set xms and xmx very low with a smal

Re: memory consuption

Oh right but Cassandra doesn't really respect that, I thought there was another option to set that. Just for your information, I set xms and xmx very low with a small amount of data. I am waiting to be able to connect jconsole, I don't know why it is not reachable at the moment. Here is my result

Re: cluster size, several cluster on one node for multi-tenancy

Hector's virtual keyspaces would work well for what you describe. Ed Anuff, who added this feature to Hector, showed me a working multi-tennancy based app the other day and it worked quite well. On Thu, Feb 17, 2011 at 1:44 PM, Norman Maurer wrote: > Maybe you could make use of "Virtual Keyspaces

Re: memory consuption

bin/cassandra.in.sh set Xms and Xmx in the JVM_OPTSAaronOn 18 Feb, 2011,at 09:10 AM, Victor Kabdebon wrote:Is it possible to change the maximum JVM heap memory use in 0.6.X ?2011/2/17 Aaron Morton What are you using for disk_access_mode ?Have you tried reducing the JVM h

Re: Updating/inserting into multiple column families using one mutator batch

Hi, Thanks for replying. I have kept an eye on the Cassandra logs as well as my app server logs, and I didn't notice any unusual hector/casandra messages there. Where can I configure to see cassandra logs in debug mode? I am pretty sure I haven't touched the 500 mutations in a batch yet. What c

Re: frequent client exceptions on 0.7.0

Messages been dropped means the machine node is overloaded. Look at the thread pool stats to see which thread pools have queues. It may be IO related, so also check the read and write latency on the CF and use iostat. i would try those first, then jump into GC land. Aaron On 18/02/2011, at 4:4

Re: Updating/inserting into multiple column families using one mutator batch

It is fine to use multiple coumn families via batch_mutate. The size of the batch itself will take some tunning. In what you are describing below, it will help watch the cassandra logs in debug mode to diagnose the issue. In general though, I think a good rule with batch_mutate is to start with 50

Re: Inconsistent result in super range slice query (reversed order)

First some terminology, when you say range slice do you mean getting multiple rows? Or do you mean get_slice where you return multiple super columns from one row? Your examples looks like you want to get multiple super columns from one row. In which case the choice of partitioner is not importa

Updating/inserting into multiple column families using one mutator batch

Hi, Is it ok to update / insert into multiple column families (some regular, some related super column families) using in one batch? I earlier had a few separates mutator.execute() calls hitting these CFs, but I am trying to merge them into a bigger batch. The issue I am facing is that the small

Re: memory consuption

Is it possible to change the maximum JVM heap memory use in 0.6.X ? 2011/2/17 Aaron Morton > What are you using for disk_access_mode ? > Have you tried reducing the JVM head size? > Have you added the Jna.jar file to lib/ ? This will allow Cassandra to lock > the JVM memory. > > > Aaron > > > On

Re: memory consuption

What are you using for disk_access_mode ? Have you tried reducing the JVM head size? Have you added the Jna.jar file to lib/ ? This will allow Cassandra to lock the JVM memory. Aaron On 17/02/2011, at 9:20 PM, ruslan usifov wrote: > > > 2011/2/16 Aaron Morton > JVM heap memory is controlle

Re: cluster size, several cluster on one node for multi-tenancy

Maybe you could make use of "Virtual Keyspaces". See this wiki for the idea: https://github.com/rantav/hector/wiki/Virtual-Keyspaces Bye, Norman 2011/2/17 Frank LoVecchio : > Why not just create some sort of ACL on the client side and use one > Keyspace?  It's a lot less management. > > On Thu,

Re: cluster size, several cluster on one node for multi-tenancy

Why not just create some sort of ACL on the client side and use one Keyspace? It's a lot less management. On Thu, Feb 17, 2011 at 12:34 PM, Mimi Aluminium wrote: > Hi, > I really need your help in this matter. > I will try to simplify my problem and ask specific questions > > I am thinking of so

Re: cluster size, several cluster on one node for multi-tenancy

Hi, I really need your help in this matter. I will try to simplify my problem and ask specific questions I am thinking of solving the multi-tenancy problem by providing a separate cluster per each tenant. Does it sound reasonable? I can end-up with one node belongs to several clusters. Does Cassan

Re: Able to send only blank emails without contents to the group. What could be going on ?

Thanks ! Finally. Did several retries since morning. On Thu, Feb 17, 2011 at 1:39 PM, Jonathan Ellis wrote: > Maybe https://issues.apache.org/jira/browse/INFRA-3356? > > On Thu, Feb 17, 2011 at 12:37 PM, A J wrote: >> >> > > > > -- > Jonathan Ellis > Project Chair, Apache Cassandra > co-founder

Replica details

Where can I get good detailed explanation of the various replication options (Simple, Old Network and Network) along with snitches. I did read the definitive guide but not really satisfied. Is there a good post somewhere explaining this ? I will have 4 datacenters (assume) and 3 nodes in each DC.

Re: Able to send only blank emails without contents to the group. What could be going on ?

Maybe https://issues.apache.org/jira/browse/INFRA-3356? On Thu, Feb 17, 2011 at 12:37 PM, A J wrote: > > -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com

Re: Able to send only blank emails without contents to the group. What could be going on ?

try setting your email to plain text only, no rich text. On Thu, Feb 17, 2011 at 1:37 PM, A J wrote: > > -- Even the Magic 8 ball has an opinion on email clients: Outlook not so good.

Able to send only blank emails without contents to the group. What could be going on ?

Re: Pig not reading all cassandra data

Thanks a lot for the help on this! >From what I can tell that looks like a good solution. Created https://issues.apache.org/jira/browse/CASSANDRA-2184 to make that change. On Thu, Feb 17, 2011 at 11:52 AM, Matt Kennedy wrote: > I have a resolution for how I'm dealing with this problem for my pa

test

Re: Does counter columns support TTL

On Thu, Feb 17, 2011 at 6:19 PM, Utku Can Topçu wrote: > And I think this patch would still be useful and legitimate if the TTL of > the initial increment is taken into account. This is still broken for the same reason. The problem is that the live-time of a given update depends on when it get

Re: [RELEASE] 0.7.2

Looks like it's https://issues.apache.org/jira/browse/CASSANDRA-2172 Regards, Chen www.evidentsoftware.com On Thu, Feb 17, 2011 at 1:06 PM, Damick, Jeffrey wrote: > So after upgrade to 0.7.2, I see this on startup – should I just blow >

Re: [RELEASE] 0.7.2

https://issues.apache.org/jira/browse/CASSANDRA-2174 Yes, just clear the cache On Thu, Feb 17, 2011 at 1:06 PM, Damick, Jeffrey wrote: > So after upgrade to 0.7.2, I see this on startup – should I just blow > away these cache files? > > > WARN [main] 2011-02-17 18:03:24,161 ColumnFamilyStore.

Re: [RELEASE] 0.7.2

So after upgrade to 0.7.2, I see this on startup - should I just blow away these cache files? WARN [main] 2011-02-17 18:03:24,161 ColumnFamilyStore.java (line 281) error reading saved cache /var/lib/cassandra/saved_caches/xx-KeyCache java.io.EOFException at java.io.ObjectInputStream$Pe

Re: Pig not reading all cassandra data

I have a resolution for how I'm dealing with this problem for my particular situation and I'd like to throw it out there to see if you think it should be integrated into the core Cassandra code. Just to repeat, the immediate workaround for this is to set -Dpig.splitCombination=false when you launc

AW: cant seem to figure out secondary index definition

Hi again, i am still having trouble with this. If I define the index using cli with these commands: create column family A with column_type='Standard' and comparator='TimeUUIDType' and keys_cached=20 and read_repair_chance=1.0 and rows_cached=0.0 and column_metadata=[{column_name: -

Simple Compression Scheme

Below is a link for a simple client side compression scheme. I thought this might be of interest for some members of the list. While column values and column names are easy to handle on the client side, with the use of a custom column name comparator for the column names, the fact that there is

Re: Does counter columns support TTL

And I think this patch would still be useful and legitimate if the TTL of the initial increment is taken into account. On Thu, Feb 17, 2011 at 6:11 PM, Utku Can Topçu wrote: > Yes, I've read the discussion. My use-case is similar to the use-case of > the contributor. > > So that's the reason wh

Re: Does counter columns support TTL

Yes, I've read the discussion. My use-case is similar to the use-case of the contributor. So that's the reason why I've asked if it works or not. (with the flaw of course). On Thu, Feb 17, 2011 at 5:41 PM, Jonathan Ellis wrote: > If you read the discussion on that ticket, the point is that th

Re: memtable_flush_after_mins setting not working

https://issues.apache.org/jira/browse/CASSANDRA-2183 Regards, Chen www.evidentsoftware.com On Thu, Feb 17, 2011 at 11:47 AM, Ching-Cheng Chen < cc...@evidentsoftware.com> wrote: > Certainly, I'll open a ticket to track this issue. > > Regar

Re: memtable_flush_after_mins setting not working

Certainly, I'll open a ticket to track this issue. Regards, Chen www.evidentsoftware.com On Thu, Feb 17, 2011 at 11:42 AM, Jonathan Ellis wrote: > Your analysis sounds correct to me. Can you open a ticket on > https://issues.apache.org/jira/browse/CASSANDRA ? > > On Thu, Feb 17, 2011 at 10:1

Re: memtable_flush_after_mins setting not working

Your analysis sounds correct to me. Can you open a ticket on https://issues.apache.org/jira/browse/CASSANDRA ? On Thu, Feb 17, 2011 at 10:17 AM, Ching-Cheng Chen wrote: > We have observed the behavior that memtable_flush_after_mins setting not > working occasionally.   After some testing and cod

Re: Does counter columns support TTL

If you read the discussion on that ticket, the point is that the approach is fundamentally flawed. On Thu, Feb 17, 2011 at 10:16 AM, Utku Can Topçu wrote: > Can anyone confirm that this patch works with the current trunk? > > On Thu, Feb 17, 2011 at 4:16 PM, Sylvain Lebresne > wrote: >> >> https

memtable_flush_after_mins setting not working

We have observed the behavior that memtable_flush_after_mins setting not working occasionally. After some testing and code digging, we finally figured out what going on. The memtable_flush_after_mins won't work on certain condition with current implementation in Cassandra. In org.apache.cassandr

Re: Does counter columns support TTL

Can anyone confirm that this patch works with the current trunk? On Thu, Feb 17, 2011 at 4:16 PM, Sylvain Lebresne wrote: > https://issues.apache.org/jira/browse/CASSANDRA-2103 > > > On Thu, Feb 17, 2011 at 4:05 PM, Utku Can Topçu wrote: > >> Hi All, >> >> I'm experimenting and developing using

RE: frequent client exceptions on 0.7.0

Try turning on GC logging in Cassandra-env.sh, specifically: -XX:+PrintGCApplicationStoppedTime -Xloggc:/var/log/cassandra/gc.log Look for things like: "Total time for which application threads were stopped: 52.8795600 seconds". Anything over about a few seconds may be causing you

Inconsistent result in super range slice query (reversed order)

Hi, We are in trouble with a strange behavior in cassandra 0.7.2 (also happened in 0.7.0). Could someone help us? The problem happens on a column family of super column type named "Order". Data structure is something like: Order[ a_key ][ date + "/" + order_id + "/" (+ suffix) ][attribute] = va

Re: Does counter columns support TTL

https://issues.apache.org/jira/browse/CASSANDRA-2103 On Thu, Feb 17, 2011 at 4:05 PM, Utku Can Topçu wrote: > Hi All, > > I'm experimenting and developing using counters. However, I've come to a > usecase where I need counters to expire and get deleted after a certain time > of inactivity (i.e.

Re: Commercial support for cassandra

http://wiki.apache.org/cassandra/ThirdPartySupport On Thu, Feb 17, 2011 at 12:20 AM, Sal Fuentes wrote: > They also offer great training sessions. Have a look at their site for more > information: http://www.datastax.com/about-us > > > On Wed, Feb 16, 2011 at 3:13 PM, Michael Widmann < > michael

Does counter columns support TTL

Hi All, I'm experimenting and developing using counters. However, I've come to a usecase where I need counters to expire and get deleted after a certain time of inactivity (i.e. have countercolumn deleted one hour after the last increment). As far as I can tell counter columns don't have TTL in t

Re: cassandra & php

i think the quesiton you started out with was about using php & cassandra. the underlying cassandra configuration and implementation shouldn't change if you decide to go with pycassa, hector or phpcassa ... the thrift interface is suitably abstracted by these... what i'm not entirely happy with in

Re: rename index

I haven't tried this, but it should work in theory... Keep the wrongly named index for now. Create the rightly named index, then bring down the node and swap file names for the indexes. After that, you can safely drop the wrongly named index. Unfortunately, you need to do this for each node. G

Re: cassandra & php

2011/2/17 Sasha Dolgy > Developing on win7 w/single cassandra node. Deployed to aws large > instances for cassandra and micro for web, micro for php > > In production how many instances for cassandra do you use? And is it possible to provide your cassandra config?

Re: cassandra & php

Developing on win7 w/single cassandra node. Deployed to aws large instances for cassandra and micro for web, micro for php On 17 Feb 2011 13:36, "ruslan usifov" wrote: > And how many nodes of cassandra do you use? > > 2011/2/17 ruslan usifov > >> Hello >> >> >> If anybody use cassandra throw

Re: cassandra & php

And how many nodes of cassandra do you use? 2011/2/17 ruslan usifov > Hello > > > If anybody use cassandra throw php, please share you experience. How you > solve problem with thrift_protocol which have tons of bugs (memleaks, > impossible pass long time stamps, wrong work with ZTS, hung when wo

Re: cassandra & php

You main system is 64 or 32 bit?

Re: cassandra & php

I am having no problems using thobbs fork of phpcassa. Works a treat. On 17 Feb 2011 11:38, "ruslan usifov" wrote: > Hello > > > If anybody use cassandra throw php, please share you experience. How you > solve problem with thrift_protocol which have tons of bugs (memleaks, > impossible pass long

cassandra & php

Hello If anybody use cassandra throw php, please share you experience. How you solve problem with thrift_protocol which have tons of bugs (memleaks, impossible pass long time stamps, wrong work with ZTS, hung when work with framed transport) And did you sufficient with php -> Cassandra performan

AW: rename index

Thanks, Up to now i could not see any problems with the index names For now I will not touch it. If I encounter something I’ll let you know Von: Aaron Morton [mailto:aa...@thelastpickle.com] Gesendet: Mittwoch, 16. Februar 2011 21:00 An: user@cassandra.apache.org Betreff: Re: rename index There i

Re: frequent client exceptions on 0.7.0

On Thu, Feb 17, 2011 at 12:37 AM, Peter Schuller wrote: > Bottom line: Check /var/log/cassandra/system.log to begin with and see > if it's reporting anything or being restarted. Thanks, Peter. In the system.log, I see quite a few of these across several machines. Everything else in the log is I

Re: frequent client exceptions on 0.7.0

>   raise EOFError() > EOFError [snip] > error: [Errno 104] Connection reset by peer Sounds like you either have a firewalling/networking issues that is tearing down TCP connections, or your cassandra node is dying. Have you checked the Cassandra system log? A frequent mistake is configuring mem

Re: [RELEASE] 0.7.2

On 17 February 2011 00:56, Eric Evans wrote: > > CASSANDRA-2165[1] became evident almost as soon as 0.7.1 released, and > it's ugly enough that we didn't want to wait. > > Be sure you've read the changelog[2] and release notes[3], and let us > know[4] if you encounter any problems. > > Thanks! > >

Re: memory consuption

2011/2/16 Aaron Morton > JVM heap memory is controlled by the settings in conf/Cassandra-env.sh > > Memory mapped files will use additional virtual memory, is controlled in > conf/Cassandra.yaml disk_access_mode > > And??? JVM memory heap in cassandra 0.7 is by default half of memory is system in