Re: Backend application for Cassandra

2011-02-14 Thread Michal Augustýn
Hi,

it depends on your queries complexity - maybe secondary indexes would be
sufficient for you -
http://www.datastax.com/dev/blog/whats-new-cassandra-07-secondary-indexes

If your queries are too complex then you could use Pig (over Hadoop) -
http://www.slideshare.net/jeromatron/cassandrahadoop-integration

Augi

P.S.: I'm just user as you, not Cassandra developer.

2011/2/14 cbert...@libero.it 

> Hi all,
> I've build a web application using Cassandra.
> Data are stored in order to be quickly red/sorted due to my web-app needs.
> Everything is working quite good.
> Now the big "problem" is that the "other side" of my company needs to
> create
> reports over these data and the query they need to do would be very "heavy"
> in
> terms of client-side complexity.
> I'd like to know if you have any tips that may help ... I've red something
> about Kundera and Lucandra but I don't know these could be solutions ...
>
> Did you already face problems like this? Could you suggest any valid
> product/solution?
> I've heard (team-mates) some tips like "export all your CF into a
> relational
> model and query it" ... and I behaved like i didn't hear it :)
>
> TIA for any help
>
> Best Regards
>
> Carlo
>


Re: Extra Large Memtables

2011-02-14 Thread E S
 Already submitted and fixed!  Thanks Jonathan for your help on this.  I really 
appreciate it!

https://issues.apache.org/jira/browse/CASSANDRA-2158


  

Re: Data distribution

2011-02-14 Thread Matthew Dennis
regardless of increasing RF or not, RR happens based on the
read_repair_chance setting.  RR happens after the request has been replied
to though, so it's possible that if you increase the RF and then read that
the read might get stale/missing data.  RR would then put the correct value
on all the correct nodes so future reads would see the correct data, but the
initial read might not.

If you are already reading at CL.ONE then after increasing the RF you need
to read at CL.Q to maintain the same consistency.  If you're reading at CL.Q
or CL.ALL, then after increasing the RF you need to read at CL.ALL to
maintain the same consistency.  You have to do this until all the nodes are
consistent again.  If you depend on RR only this time is unbounded.  If you
run repair, then after repair the repair is complete you can go back to your
original CL.

tl;dr run nodetool repair after increasing the RF

On Mon, Feb 14, 2011 at 7:52 PM, mcasandra  wrote:

>
> When I increase the replication factor does the repair happen automatically
> in background when client first tries to access data from the node where
> data does not exist.
>
> Or the nodetool repair need to run after increasing the replication factor.
> --
> View this message in context:
> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Data-distribution-tp6025869p6025972.html
> Sent from the cassandra-u...@incubator.apache.org mailing list archive at
> Nabble.com.
>


Re: Cassandra documentation

2011-02-14 Thread mcasandra


Cassy Andra wrote:
> 
> Here is a blog my team is working on at Accenture which is intended to be
> a
> complete beginner's guide to Cassandra. I'm still updating a few posts
> based
> on DataStax's recommendations and I need to add the last three posts (will
> get this done soon), but you can start checking it out via this link:
> 
> https://sites.google.com/a/techlabs.accenture.com/cassandra
> 
> 
> - Sameer
> 
> 
I tried to follow steps to configure hector. Even after adding all the jar
files I see lot of errors in the project. For eg: apache commons object
mapper doesn't resolve.

Is that expected? Others have suggested using maven instead.
-- 
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cassandra-documentation-tp6012810p6025975.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.


RE: Data distribution

2011-02-14 Thread mcasandra

When I increase the replication factor does the repair happen automatically
in background when client first tries to access data from the node where
data does not exist.

Or the nodetool repair need to run after increasing the replication factor.
-- 
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Data-distribution-tp6025869p6025972.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.


Re: NFS instead of local storage

2011-02-14 Thread Matthew Dennis
no, it's actually worse to do that.

1) you're introducing single points of failure (your array).

2) you're introducing complexity and expense

3) you're introducing latency

4) you're introducing bottle necks

5) some other reasons...

You do want your commit log on a separate disk though.  The only reason I
wouldn't do that is if you just don't have that high of a write load, in
which case I would RAID whatever *local* disks you have on the box for
storage together and use that array for both commit log and data.

On Sun, Feb 13, 2011 at 5:25 PM, mcasandra  wrote:

>
> I just now watched some videos about performance tunning. And it looks like
> most of the bottleneck could be on reads. Also, it looks like it's
> advisable
> to put commit logs on separate drive.
>
> I was wondering if it makes sense to use NFS (if we can) with netapp array
> which provides it's own read and write caching.
> --
> View this message in context:
> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/NFS-instead-of-local-storage-tp6021959p6021959.html
> Sent from the cassandra-u...@incubator.apache.org mailing list archive at
> Nabble.com.
>


Re: Data distribution

2011-02-14 Thread Matthew Dennis
On Mon, Feb 14, 2011 at 6:58 PM, Dan Hendry wrote:

> > 1) If I insert a key and want to verify which node it went to then how do
> I
> > do that?
>
> I don't think you can and there should be no reason to care. Cassandra
> abstracts where data is being stored, think in terms of consistency levels
> not actual nodes.
>

If you actually mean which nodes a particular write went to, you'd have to
ask each node that it's supposed to belong on or watch the logs while on
debug.  If you just want to know which nodes it's *supposed* to go to, JMX
exposes getNaturalEndpoints and thrift exposes describe_ring. describe_ring
will give the token ranges for each node and then you can fit your key
inside those ranges and get a list of the nodes that key belongs on.


>
> > 2) How can I verify if the replication is working. That is how do I check
> > that CF row got inserted in 2 nodes if replication factor is set to 2.
>
> Perform a successful write at consistency level ALL via a thrift client.
>

Perform a successful write at CL.ALL will work.  You can also write at
whatever CL you want, then look in describe_ring to see where the data
should have ended up, then ask each node individually at CL.ONE to see if
that node has the data.  If you do this, make sure Read Repair (RR) is off,
otherwise the first read you do might repair it on another node and that
only tells you RR is working, not that the initially replication worked.


Re: Internal error processing insert

2011-02-14 Thread Matthew Dennis
I had actually meant to (and thought I did) type "greater than zero and less
than *or equal* to number of nodes".

That being said, you usually do want it less than the number of nodes in the
cluster because otherwise your cluster essentially has the same performance
as a single node.  In general (fuzzy) performance is num_nodes / RF.  If you
have 9 nodes and RF=3 you have performance roughly along the lines of having
3 boxen.  If you have N nodes and RF=N, you have roughly the same
performance as... one machine.

What Arron says about RF and Consistency Level (CL) is also correct.  RF=2
implies CL.Q requires both nodes meaning you can't lose a single node and
still read/write at CL.Q

Just to be pedantic though, you have have a cluster of N nodes (for large N)
and read/write at CL.Q even after losing some nodes.  For example, if you
have 10 nodes and RF=10 you only need 6 nodes up to read/write at CL.Q

On Mon, Feb 14, 2011 at 6:58 PM, Aaron Morton wrote:

> He probably meant in production. When playing around, and if you only have
> 2 nodes, you can set it to 2.
>
> From memory RF of 2 means the Quorum is also 2, so you cannot afford to
> lose one. Thats fine for playing.
>
> Aaron
>
>
> On 15 Feb, 2011,at 01:51 PM, mcasandra  wrote:
>
>
>
> mcasandra wrote:
> >
> > In earlier post same thread you mentioned that replication factor should
> > be set to less than N.
> >
> > Currently I am testing on 2 node cluster and I was able to set
> > replication_factor to 2 (=N) and also when I did cfstats (I don't quite
> > understand cfstats in detail) and see some activity on both nodes now
> >
> > [default@twissandra] update keyspace twissandra with
> replication_factor=2;
> > 3ed6b708-389b-11e0-993f-b7fa7ed61af9
> > [default@twissandra] set users['b']['add']='111';
> > Value inserted.
> >
> > On the other node when I ran nodetool with -h as localhost, I see:
> >
> >
> > Column Family: users
> > SSTable count: 0
> > Space used (live): 0
> > Space used (total): 0
> > Memtable Columns Count: 2
> > Memtable Data Size: 52
> > Memtable Switch Count: 0
> > Read Count: 3
> > Read Latency: NaN ms.
> > Write Count: 2
> > Write Latency: NaN ms.
> > Pending Tasks: 0
> > Key cache capacity: 20
> > Key cache size: 0
> > Key cache hit rate: NaN
> > Row cache: disabled
> > Compacted row minimum size: 0
> > Compacted row maximum size: 0
> > Compacted row mean size: 0
> >
> >
> Sorry I meant to say someone (Matthew Dennis) on the thread mentioned that
> replication factor should be less than no. of nodes.
> --
> View this message in context:
> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Internal-error-processing-insert-tp6025740p6025882.html
> Sent from the cassandra-u...@incubator.apache.org mailing list archive at
> Nabble.com.
>
>


Request For 0.6.12 Release

2011-02-14 Thread Gregory Szorc
The latest official 0.6.x releases, 0.6.10 and 0.6.11, have a very serious
bug/regression when performing some quorum reads (CASSANDRA-2081), which is
fixed in the head of the 0.6 branch. If there aren't any plans to cut 0.6.12
any time soon, as an end user, I request that an official and "blessed"
release of 0.6.x be made ASAP.

 

On a related note, I am frustrated that such a serious issue has lingered in
the "latest oldstable release." I would have liked to see one or more of the
following:

 

1)  The issue documented prominently on the apache.org web site and
inside the download archive so end users would know they are downloading and
running known-broken software

2)  The 0.6.10 and 0.6.11 builds pulled after identification of the
issue

3)  A 0.6.12 release cut immediately (with reasonable time for testing,
of course) to address the issue

 

I understand that releases may not always be as stable as we all desire.
But, I hope that when future bugs affecting the bread and butter properties
of a distributed storage engine surface (especially when they are
regressions) that the official project response (preferably via mailing list
and the web site) is swift and maximizes the potential for data integrity
and availability.

 

If there is anything I can do to help the process, I'd gladly give some of
my time to help the overall community.

 

Gregory Szorc

gregory.sz...@gmail.com

 



Re: Extra Large Memtables

2011-02-14 Thread Matthew Dennis
On Mon, Feb 14, 2011 at 2:54 PM, Robert Coli  wrote:

> Regarding very large memtables, it is important to recognize that
> throughput refers only to the size of the COLUMN VALUES, and not, for
> example, their names.
>

That would be a bug in it's own right.  There are lots of use cases that
only use column names and never use the values.

line 113 @
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/Column.java?revision=1063928&view=markup


Re: Fwd: SPA2011 - June 12th-15th - BCS London, UK - Call for Sessions

2011-02-14 Thread Courtney Robinson

Anyone else in London interested in this?


--
From: "Jonathan Ellis" 
Sent: Monday, February 14, 2011 10:30 PM
To: "user" 
Subject: Fwd: SPA2011 - June 12th-15th - BCS London, UK - Call for Sessions


In case any of the London crowd is interested:


-- Forwarded message --
From: Mike Hill 
Date: Mon, Feb 14, 2011 at 4:14 PM
Subject: SPA2011 - June 12th-15th - BCS London, UK - Call for Sessions
To: "mikewh...@gmail.com" 


SPA2011 - June 12th-15th - BCS London, UK - Call for Sessions

Submissions Deadline:  28th February 2011.

To find out more, and submit a proposal, visit 
http://www.spaconference.org



We would like to invite you to present a session at this leading
software development conference. SPA2011 will continue the well
established SPA tradition of learning through interaction, with sessions
exploring the latest advancements in software development practice.

We're looking for sessions which are interactive, thought provoking and
have not been seen before in this form (may be a topic you've covered
before, but it must be truly interactive).

They can be about technology or teams, practice or process - in fact
anything to do with advancing the state of the practice in software
development. We welcome submissions from everyone, if you're not
experienced with presenting sessions at SPA you'll be supported by our
well-established shepherding process which has ensured the standard
of sessions at SPA is exceptionally high.

Presenters will receive free attendance to the conference.
See the website for conditions.

Don't be shy! This year you can submit a rough proposal and get your
peers to give you feedback!

To find out more, and submit a proposal, visit 
http://www.spaconference.org



The submission deadline is 28th February 2011.

Willem van den Ende & Rob Bowley
Programme Chairs SPA 2011
progra...@spaconference.org 

--
You received this message because you are subscribed to the Google
Groups "NOSQL" group.
To post to this group, send email to nosql-discuss...@googlegroups.com.
To unsubscribe from this group, send email to
nosql-discussion+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/nosql-discussion?hl=en.



--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com



RE: Data distribution

2011-02-14 Thread Dan Hendry
> 1) If I insert a key and want to verify which node it went to then how do
I
> do that?

I don't think you can and there should be no reason to care. Cassandra
abstracts where data is being stored, think in terms of consistency levels
not actual nodes. 

> 2) How can I verify if the replication is working. That is how do I check
> that CF row got inserted in 2 nodes if replication factor is set to 2.

Perform a successful write at consistency level ALL via a thrift client.

> 3) What happens if I just update the keyspace and change the replication
> factor say 2 to 3. Would cassandra automatically replicate the old data to
> the 3rd node?

http://wiki.apache.org/cassandra/Operations#Replication

"but increasing it may be done if you (a) read at ConsistencyLevel.QUORUM or
ALL (depending on your existing replication factor) to make sure that a
replica that actually has the data is consulted, (b) are willing to accept
downtime while anti-entropy repair runs (see below), or (c) are willing to
live with some clients potentially being told no data exists if they read
from the new replica location(s) until repair is done."

Dan

-Original Message-
From: mcasandra [mailto:mohitanch...@gmail.com] 
Sent: February-14-11 19:45
To: cassandra-u...@incubator.apache.org
Subject: Data distribution


Couple of questions:

1) If I insert a key and want to verify which node it went to then how do I
do that?
2) How can I verify if the replication is working. That is how do I check
that CF row got inserted in 2 nodes if replication factor is set to 2.
3) What happens if I just update the keyspace and change the replication
factor say 2 to 3. Would cassandra automatically replicate the old data to
the 3rd node?
-- 
View this message in context:
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Data-distri
bution-tp6025869p6025869.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at
Nabble.com.
No virus found in this incoming message.
Checked by AVG - www.avg.com 
Version: 9.0.872 / Virus Database: 271.1.1/3441 - Release Date: 02/14/11
02:34:00



Re: Internal error processing insert

2011-02-14 Thread Aaron Morton
He probably meant in production. When playing around, and if you only have 2 nodes, you can set it to 2. From memory RF of 2 means the Quorum is also 2, so you cannot afford to lose one. Thats fine for playing. Aaron On 15 Feb, 2011,at 01:51 PM, mcasandra  wrote:

mcasandra wrote:
> 
> In earlier post same thread you mentioned that replication factor should
> be set to less than N.
> 
> Currently I am testing on 2 node cluster and I was able to set
> replication_factor to 2 (=N) and also when I did cfstats (I don't quite
> understand cfstats in detail) and see some activity on both nodes now
> 
> [default@twissandra] update keyspace twissandra with replication_factor=2;
> 3ed6b708-389b-11e0-993f-b7fa7ed61af9
> [default@twissandra] set users['b']['add']='111';
> Value inserted.
> 
> On the other node when I ran nodetool with -h as localhost, I see:
> 
> 
> Column Family: users
> SSTable count: 0
> Space used (live): 0
> Space used (total): 0
> Memtable Columns Count: 2
> Memtable Data Size: 52
> Memtable Switch Count: 0
> Read Count: 3
> Read Latency: NaN ms.
> Write Count: 2
> Write Latency: NaN ms.
> Pending Tasks: 0
> Key cache capacity: 20
> Key cache size: 0
> Key cache hit rate: NaN
> Row cache: disabled
> Compacted row minimum size: 0
> Compacted row maximum size: 0
> Compacted row mean size: 0
> 
> 
Sorry I meant to say someone (Matthew Dennis) on the thread mentioned that
replication factor should be less than no. of nodes.
-- 
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Internal-error-processing-insert-tp6025740p6025882.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.


Re: Fwd: SPA2011 - June 12th-15th - BCS London, UK - Call for Sessions

2011-02-14 Thread Courtney Robinson

Anyone else in London interested in this?


--
From: "Jonathan Ellis" 
Sent: Monday, February 14, 2011 10:30 PM
To: "user" 
Subject: Fwd: SPA2011 - June 12th-15th - BCS London, UK - Call for Sessions


In case any of the London crowd is interested:


-- Forwarded message --
From: Mike Hill 
Date: Mon, Feb 14, 2011 at 4:14 PM
Subject: SPA2011 - June 12th-15th - BCS London, UK - Call for Sessions
To: "mikewh...@gmail.com" 


SPA2011 - June 12th-15th - BCS London, UK - Call for Sessions

Submissions Deadline:  28th February 2011.

To find out more, and submit a proposal, visit 
http://www.spaconference.org



We would like to invite you to present a session at this leading
software development conference. SPA2011 will continue the well
established SPA tradition of learning through interaction, with sessions
exploring the latest advancements in software development practice.

We're looking for sessions which are interactive, thought provoking and
have not been seen before in this form (may be a topic you've covered
before, but it must be truly interactive).

They can be about technology or teams, practice or process - in fact
anything to do with advancing the state of the practice in software
development. We welcome submissions from everyone, if you're not
experienced with presenting sessions at SPA you'll be supported by our
well-established shepherding process which has ensured the standard
of sessions at SPA is exceptionally high.

Presenters will receive free attendance to the conference.
See the website for conditions.

Don't be shy! This year you can submit a rough proposal and get your
peers to give you feedback!

To find out more, and submit a proposal, visit 
http://www.spaconference.org



The submission deadline is 28th February 2011.

Willem van den Ende & Rob Bowley
Programme Chairs SPA 2011
progra...@spaconference.org 

--
You received this message because you are subscribed to the Google
Groups "NOSQL" group.
To post to this group, send email to nosql-discuss...@googlegroups.com.
To unsubscribe from this group, send email to
nosql-discussion+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/nosql-discussion?hl=en.



--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com



Re: Internal error processing insert

2011-02-14 Thread mcasandra


mcasandra wrote:
> 
> In earlier post same thread you mentioned that replication factor should
> be set to less than N.
> 
> Currently I am testing on 2 node cluster and I was able to set
> replication_factor to 2 (=N) and also when I did cfstats (I don't quite
> understand cfstats in detail) and see some activity on both nodes now.
> 
> [default@twissandra] update keyspace twissandra with replication_factor=2;
> 3ed6b708-389b-11e0-993f-b7fa7ed61af9
> [default@twissandra] set users['b']['add']='111';
> Value inserted.
> 
> On the other node when I ran nodetool with -h as localhost, I see:
> 
> 
> Column Family: users
> SSTable count: 0
> Space used (live): 0
> Space used (total): 0
> Memtable Columns Count: 2
> Memtable Data Size: 52
> Memtable Switch Count: 0
> Read Count: 3
> Read Latency: NaN ms.
> Write Count: 2
> Write Latency: NaN ms.
> Pending Tasks: 0
> Key cache capacity: 20
> Key cache size: 0
> Key cache hit rate: NaN
> Row cache: disabled
> Compacted row minimum size: 0
> Compacted row maximum size: 0
> Compacted row mean size: 0
> 
> 
Sorry I meant to say someone (Matthew Dennis) on the thread mentioned that
replication factor should be less than no. of nodes.
-- 
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Internal-error-processing-insert-tp6025740p6025882.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.


Re: Internal error processing insert

2011-02-14 Thread mcasandra

In earlier post same thread you mentioned that replication factor should be
set to less than N.

Currently I am testing on 2 node cluster and I was able to set
replication_factor to 2 (=N) and also when I did cfstats (I don't quite
understand cfstats in detail) and see some activity on both nodes now.

[default@twissandra] update keyspace twissandra with replication_factor=2;
3ed6b708-389b-11e0-993f-b7fa7ed61af9
[default@twissandra] set users['b']['add']='111';
Value inserted.

On the other node when I ran nodetool with -h as localhost, I see:


Column Family: users
SSTable count: 0
Space used (live): 0
Space used (total): 0
Memtable Columns Count: 2
Memtable Data Size: 52
Memtable Switch Count: 0
Read Count: 3
Read Latency: NaN ms.
Write Count: 2
Write Latency: NaN ms.
Pending Tasks: 0
Key cache capacity: 20
Key cache size: 0
Key cache hit rate: NaN
Row cache: disabled
Compacted row minimum size: 0
Compacted row maximum size: 0
Compacted row mean size: 0

-- 
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Internal-error-processing-insert-tp6025740p6025878.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.


Data distribution

2011-02-14 Thread mcasandra

Couple of questions:

1) If I insert a key and want to verify which node it went to then how do I
do that?
2) How can I verify if the replication is working. That is how do I check
that CF row got inserted in 2 nodes if replication factor is set to 2.
3) What happens if I just update the keyspace and change the replication
factor say 2 to 3. Would cassandra automatically replicate the old data to
the 3rd node?
-- 
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Data-distribution-tp6025869p6025869.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.


Re: Internal error processing insert

2011-02-14 Thread Matthew Dennis
On Mon, Feb 14, 2011 at 6:28 PM, Aaron Morton wrote:

> Will take a closer look at the code tonight, perhaps we should return an
> error if you try to using Network Topology it cannot detect any DC's .
>
>
+1


Re: Internal error processing insert

2011-02-14 Thread Eric Gilmore
For now, I have committed a change in the misleading documentation,
substituting SimpleStrategy for NTS.

Sorry you ran into trouble due to that, mcasandra.

On Mon, Feb 14, 2011 at 4:28 PM, Aaron Morton wrote:

> Will take a closer look at the code tonight, perhaps we should return an
> error if you try to using Network Topology it cannot detect any DC's .
>
> Cheers
> Aaron
>
>
> On 15 Feb, 2011,at 01:22 PM, mcasandra  wrote:
>
>
> That's what I thought might be happening since network topology will try to
> find one node on the other data center. Message is little confusing though.
>
> [default@unknown] update keyspace twissandra
> placement_strategy='org.apache.cassandra.locator.SimpleStrategy';
> Syntax error at position 28: missing EOF at 'placement_strategy'
> [default@unknown] update keyspace twissandra with
> placement_strategy='org.apache.cassandra.locator.SimpleStrategy';
> 5c487967-3899-11e0-993f-b7fa7ed61af9
> [default@unknown] use twissandra
> ... ;
> Authenticated to keyspace: twissandra
> [default@twissandra] set users['jsmith']['password']='ch@ngem3';
> Value inserted.
>
> --
> View this message in context:
> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Internal-error-processing-insert-tp6025740p6025813.html
> Sent from the cassandra-u...@incubator.apache.org mailing list archive at
> Nabble.com.
>
>


Re: Internal error processing insert

2011-02-14 Thread Aaron Morton
Will take a closer look at the code tonight, perhaps we should return an error if you try to using Network Topology it cannot detect any DC's . CheersAaron On 15 Feb, 2011,at 01:22 PM, mcasandra  wrote:
That's what I thought might be happening since network topology will try to
find one node on the other data center. Message is little confusing though.

[default@unknown] update keyspace twissandra 
placement_strategy='org.apache.cassandra.locator.SimpleStrategy';
Syntax error at position 28: missing EOF at 'placement_strategy'
[default@unknown]  update keyspace twissandra with
placement_strategy='org.apache.cassandra.locator.SimpleStrategy';
5c487967-3899-11e0-993f-b7fa7ed61af9
[default@unknown] use twissandra
... ;
Authenticated to keyspace: twissandra
[default@twissandra] set users['jsmith']['password']='ch@ngem3';
Value inserted.

-- 
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Internal-error-processing-insert-tp6025740p6025813.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.


Re: Internal error processing insert

2011-02-14 Thread mcasandra

That's what I thought might be happening since network topology will try to
find one node on the other data center. Message is little confusing though.

[default@unknown] update keyspace twissandra 
placement_strategy='org.apache.cassandra.locator.SimpleStrategy';
Syntax error at position 28: missing EOF at 'placement_strategy'
[default@unknown]  update keyspace twissandra with
placement_strategy='org.apache.cassandra.locator.SimpleStrategy';
5c487967-3899-11e0-993f-b7fa7ed61af9
[default@unknown] use twissandra
... ;
Authenticated to keyspace: twissandra
[default@twissandra] set users['jsmith']['password']='ch@ngem3';
Value inserted.

-- 
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Internal-error-processing-insert-tp6025740p6025813.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.


Re: Internal error processing insert

2011-02-14 Thread Aaron Morton
Not sure why the docs suggest to use the NetworkTopologyStrategy, if their are no data centres configured the NetworkTopologyStrategy will say the replication factor is 0. I think this is the source of the "invalid response count 1 for replication factor 0"  message. Can you try with the SimpleStrategy? create keyspace twissandra with replication_factor=1
and placement_strategy='org.apache.cassandra.locator.SimpleStrategy';AaronOn 15 Feb, 2011,at 01:02 PM, mcasandra  wrote:
No it's not set to 0. I am just following the example on datastax getting
started site. Here are all the commands:

[default@unknown] create keyspace twissandra with replication_factor=1
... and
placement_strategy='org.apache.cassandra.locator.NetworkTopologyStrategy';
22dea790-3893-11e0-9174-b7fa7ed61af9
[default@unknown] use twissandra;
Authenticated to keyspace: twissandra
[default@twissandra] create column family users with comparator = UTF8Type
... ... and column_metadata = [{column_name: password,
validation_class:
... ... UTF8Type}];
Syntax error at position 54: missing EOF at ''
[default@twissandra] create column family users with comparator = UTF8Type
... and column_metadata = [{column_name: password, validation_class:
UTF8Type}];
5ecbfe61-3893-11e0-9174-b7fa7ed61af9
[default@twissandra] create column family tweets with comparator = UTF8Type
and
... column_metadata = [{column_name: body, validation_class:
... UTF8Type}, {column_name: username, validation_class: UTF8Type}];
b468f492-3893-11e0-9174-b7fa7ed61af9
[default@twissandra] create column family friends with comparator =
UTF8Type;
c08ae1c3-3893-11e0-9174-b7fa7ed61af9
[default@twissandra] create column family followers with comparator =
UTF8Type;
c856bb44-3893-11e0-9174-b7fa7ed61af9
[default@twissandra] create column family userline with comparator =
LongType and
.. default_validation_class = TimeUUIDType;
cd9ef725-3893-11e0-9174-b7fa7ed61af9
[default@twissandra] create column family timeline with comparator =
LongType and
... default_validation_class = TimeUUIDType;
d34d6ee6-3893-11e0-9174-b7fa7ed61af9
[default@twissandra] set users['jsmith']['password']='ch@ngem3';
Internal error processing insert

-- 
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Internal-error-processing-insert-tp6025740p6025762.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.


Re: Internal error processing insert

2011-02-14 Thread mcasandra

No it's not set to 0. I am just following the example on datastax getting
started site. Here are all the commands:

[default@unknown] create keyspace twissandra with replication_factor=1
... and
placement_strategy='org.apache.cassandra.locator.NetworkTopologyStrategy';
22dea790-3893-11e0-9174-b7fa7ed61af9
[default@unknown] use twissandra;
Authenticated to keyspace: twissandra
[default@twissandra] create column family users with comparator = UTF8Type
... ... and column_metadata = [{column_name: password,
validation_class:
... ... UTF8Type}];
Syntax error at position 54: missing EOF at '.'
[default@twissandra] create column family users with comparator = UTF8Type
... and column_metadata = [{column_name: password, validation_class:
UTF8Type}];
5ecbfe61-3893-11e0-9174-b7fa7ed61af9
[default@twissandra] create column family tweets with comparator = UTF8Type
and
... column_metadata = [{column_name: body, validation_class:
... UTF8Type}, {column_name: username, validation_class: UTF8Type}];
b468f492-3893-11e0-9174-b7fa7ed61af9
[default@twissandra] create column family friends with comparator =
UTF8Type;
c08ae1c3-3893-11e0-9174-b7fa7ed61af9
[default@twissandra] create column family followers with comparator =
UTF8Type;
c856bb44-3893-11e0-9174-b7fa7ed61af9
[default@twissandra] create column family userline with comparator =
LongType and
... default_validation_class = TimeUUIDType;
cd9ef725-3893-11e0-9174-b7fa7ed61af9
[default@twissandra] create column family timeline with comparator =
LongType and
... default_validation_class = TimeUUIDType;
d34d6ee6-3893-11e0-9174-b7fa7ed61af9
[default@twissandra] set users['jsmith']['password']='ch@ngem3';
Internal error processing insert

-- 
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Internal-error-processing-insert-tp6025740p6025762.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.


Re: Internal error processing insert

2011-02-14 Thread Matthew Dennis
Is your ReplicationFactor (RF) really set to 0?  Don't do that, it needs to
be at least 1 and probably needs to be 3 in production if you care about
your data.  It must be greater than 0 and less than the number of nodes in
your ring.  It represents the number of nodes to copy/replicate data to.
Another way to look at it is I'm willing to tolerate this many failures
before I lose data.

On Mon, Feb 14, 2011 at 5:55 PM, mcasandra  wrote:

>
> I am following example posted on
>
> http://www.datastax.com/dev/tutorials/getting_started_0.7/using_cli#cassandra-cli
> cli
>
> I am seeing:
>
> $ set users['jsmith']['password']='ch@ngem3';
> Internal error processing insert
>
> In the logs I see:
>
> java.lang.AssertionError: invalid response count 1 for replication factor 0
>at
>
> org.apache.cassandra.service.WriteResponseHandler.determineBlockFor(WriteResponseHandler.java:98)
>at
>
> org.apache.cassandra.service.WriteResponseHandler.(WriteResponseHandler.java:48)
>at
>
> org.apache.cassandra.service.WriteResponseHandler.create(WriteResponseHandler.java:61)
>at
>
> org.apache.cassandra.locator.AbstractReplicationStrategy.getWriteResponseHandler(AbstractReplicationStrategy.java:125)
>at
>
> org.apache.cassandra.locator.NetworkTopologyStrategy.getWriteResponseHandler(NetworkTopologyStrategy.java:165)
>
> --
>
> My guess is because I am using network topology as specified in the
> example.
> Is that correct?
>
> What's the best way to fix the problem?
> --
> View this message in context:
> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Internal-error-processing-insert-tp6025740p6025740.html
> Sent from the cassandra-u...@incubator.apache.org mailing list archive at
> Nabble.com.
>


Internal error processing insert

2011-02-14 Thread mcasandra

I am following example posted on 
http://www.datastax.com/dev/tutorials/getting_started_0.7/using_cli#cassandra-cli
cli 

I am seeing:

$ set users['jsmith']['password']='ch@ngem3';
Internal error processing insert

In the logs I see:

java.lang.AssertionError: invalid response count 1 for replication factor 0
at
org.apache.cassandra.service.WriteResponseHandler.determineBlockFor(WriteResponseHandler.java:98)
at
org.apache.cassandra.service.WriteResponseHandler.(WriteResponseHandler.java:48)
at
org.apache.cassandra.service.WriteResponseHandler.create(WriteResponseHandler.java:61)
at
org.apache.cassandra.locator.AbstractReplicationStrategy.getWriteResponseHandler(AbstractReplicationStrategy.java:125)
at
org.apache.cassandra.locator.NetworkTopologyStrategy.getWriteResponseHandler(NetworkTopologyStrategy.java:165)

--

My guess is because I am using network topology as specified in the example.
Is that correct?

What's the best way to fix the problem?
-- 
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Internal-error-processing-insert-tp6025740p6025740.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.


Re: RandomPartitioner

2011-02-14 Thread Matthew Dennis
nodes contain data for (prevTokenInRing, nodesOwnToken] (i.e. exclusive from
previous token to inclusive of the nodes token).  So .179 will contain
things that hash in the range (152896308109140433971537345591636551711,0]
and .12 will contain things that hash in range
(0,152896308109140433971537345591636551711].  You can see this "visually" by
noticing that .179 has 10% and .12 has 90% of the ring (~
152896308109140433971537345591636551711 / 2^127).

On Mon, Feb 14, 2011 at 5:28 PM, mcasandra  wrote:

>
> I installed cassandra and started it in multi-node. I set the InitialToken
> to
> 0. I ran nodetool and see:
>
> $ nodetool -h localhost ring
> Address Status State   LoadOwnsToken
>
> 152896308109140433971537345591636551711
> 172.16.20.179  Up Normal  7.52 KB 10.14%  0
> 172.16.20.12   Up Normal  12.5 KB 89.86%
> 152896308109140433971537345591636551711
>
> --
> Does it mean any keys between 0 - 152896308109140433971537345591636551711
> will go in Node 1
> and anything > 152896308109140433971537345591636551711 will go in Node 2?
> --
> View this message in context:
> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/RandomPartitioner-tp6025203p6025659.html
> Sent from the cassandra-u...@incubator.apache.org mailing list archive at
> Nabble.com.
>


Re: RandomPartitioner

2011-02-14 Thread mcasandra

I installed cassandra and started it in multi-node. I set the InitialToken to
0. I ran nodetool and see:

$ nodetool -h localhost ring
Address Status State   LoadOwnsToken
  
152896308109140433971537345591636551711
172.16.20.179  Up Normal  7.52 KB 10.14%  0
172.16.20.12   Up Normal  12.5 KB 89.86% 
152896308109140433971537345591636551711

--
Does it mean any keys between 0 - 152896308109140433971537345591636551711
will go in Node 1
and anything > 152896308109140433971537345591636551711 will go in Node 2?
-- 
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/RandomPartitioner-tp6025203p6025659.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.


Fwd: SPA2011 - June 12th-15th - BCS London, UK - Call for Sessions

2011-02-14 Thread Jonathan Ellis
In case any of the London crowd is interested:


-- Forwarded message --
From: Mike Hill 
Date: Mon, Feb 14, 2011 at 4:14 PM
Subject: SPA2011 - June 12th-15th - BCS London, UK - Call for Sessions
To: "mikewh...@gmail.com" 


SPA2011 - June 12th-15th - BCS London, UK - Call for Sessions

Submissions Deadline:  28th February 2011.

To find out more, and submit a proposal, visit http://www.spaconference.org


We would like to invite you to present a session at this leading
software development conference. SPA2011 will continue the well
established SPA tradition of learning through interaction, with sessions
exploring the latest advancements in software development practice.

We're looking for sessions which are interactive, thought provoking and
have not been seen before in this form (may be a topic you've covered
before, but it must be truly interactive).

They can be about technology or teams, practice or process - in fact
anything to do with advancing the state of the practice in software
development. We welcome submissions from everyone, if you're not
experienced with presenting sessions at SPA you'll be supported by our
well-established shepherding process which has ensured the standard
of sessions at SPA is exceptionally high.

Presenters will receive free attendance to the conference.
See the website for conditions.

Don't be shy! This year you can submit a rough proposal and get your
peers to give you feedback!

To find out more, and submit a proposal, visit http://www.spaconference.org


The submission deadline is 28th February 2011.

Willem van den Ende & Rob Bowley
Programme Chairs SPA 2011
progra...@spaconference.org 

--
You received this message because you are subscribed to the Google
Groups "NOSQL" group.
To post to this group, send email to nosql-discuss...@googlegroups.com.
To unsubscribe from this group, send email to
nosql-discussion+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/nosql-discussion?hl=en.



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com


Re: RandomPartitioner

2011-02-14 Thread mcasandra


Dan Kuebrich wrote:
> 
> You may find this part of the wiki helpful:
> http://wiki.apache.org/cassandra/Operations#Range_changes
> 
> "If you explicitly specify an InitialToken in the configuration, the new
> node will bootstrap to that position on the ring. Otherwise, it will pick
> a
> Token that will give it half the keys from the node with the most disk
> space
> used, that does not already have another node bootstrapping into its
> Range."
> 
> 
So if I am starting with only one node and I specify InitialToken as 0 then
what does it mean?



> It is circular, and token selection (after first node) is determined by
> key
> distribution.
> 
> 

Didn't quite understand this statement. What do you mean by key
distribution? I don't have any keyspace or CF at this point.


One more question, how can I remove or change token to 0 if I started with
IntitialToken as default (unset).
-- 
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/RandomPartitioner-tp6025203p6025380.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.


Re: [RELEASE] 0.7.1

2011-02-14 Thread Eric Evans
On Mon, 2011-02-14 at 16:50 -0500, Jake Luciani wrote:
> It can take some time for the files to propagate to the mirrors.  It's
> Eventually Consistent though :) 

Preferably they'd 404 when that happens though. :)

-- 
Eric Evans
eev...@rackspace.com



Re: [RELEASE] 0.7.1

2011-02-14 Thread Eric Evans
On Mon, 2011-02-14 at 14:20 -0700, Frank LoVecchio wrote:
> Ah, I meant quite a few of the mirror links keep showing up as links
> to gossip sites and whatnot. 

I suspect those mirrors are broken.  Can you submit a ticket to 
https://issues.apache.org/jira/browse/INFRA for any like that you find?

-- 
Eric Evans
eev...@rackspace.com



Re: [RELEASE] 0.7.1

2011-02-14 Thread Jake Luciani
It can take some time for the files to propagate to the mirrors.  It's
Eventually Consistent though :)

On Mon, Feb 14, 2011 at 4:20 PM, Frank LoVecchio  wrote:

> Ah, I meant quite a few of the mirror links keep showing up as links to
> gossip sites and whatnot.
> On Feb 14, 2011 2:17 PM, "Norman Maurer"  wrote:
>


Re: RandomPartitioner

2011-02-14 Thread Dan Kuebrich
You may find this part of the wiki helpful:
http://wiki.apache.org/cassandra/Operations#Range_changes

"If you explicitly specify an InitialToken in the configuration, the new
node will bootstrap to that position on the ring. Otherwise, it will pick a
Token that will give it half the keys from the node with the most disk space
used, that does not already have another node bootstrapping into its Range."

It is circular, and token selection (after first node) is determined by key
distribution.

Sent from my iPhone

On Feb 14, 2011, at 4:13 PM, mcasandra  wrote:


I am trying to understand atleast to some level of detail about how random
partitioner works. With the text I have seen on the website I am not able to
clearly understand. Is there a place where it's described with an example,
for eg how nodes are assigned random tokens? Is the range picked randomly
and assigned to nodes in circular fashion, for eg: hash ABC to FGH goes to
node A and then hash IJKLM-OPQR goes to node B?
-- 
View this message in context:
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/RandomPartitioner-tp6025203p6025203.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at
Nabble.com.


Re: [RELEASE] 0.7.1

2011-02-14 Thread Frank LoVecchio
Ah, I meant quite a few of the mirror links keep showing up as links to
gossip sites and whatnot.
On Feb 14, 2011 2:17 PM, "Norman Maurer"  wrote:


Backend application for Cassandra

2011-02-14 Thread cbert...@libero.it
Hi all,
I've build a web application using Cassandra.
Data are stored in order to be quickly red/sorted due to my web-app needs. 
Everything is working quite good.
Now the big "problem" is that the "other side" of my company needs to create 
reports over these data and the query they need to do would be very "heavy" in 
terms of client-side complexity.
I'd like to know if you have any tips that may help ... I've red something 
about Kundera and Lucandra but I don't know these could be solutions ...

Did you already face problems like this? Could you suggest any valid 
product/solution?
I've heard (team-mates) some tips like "export all your CF into a relational 
model and query it" ... and I behaved like i didn't hear it :)

TIA for any help

Best Regards

Carlo


Re: [RELEASE] 0.7.1

2011-02-14 Thread Norman Maurer
Huh,

isn't that what mirrors are supposed to be for ?

Bye,
Norman

2011/2/14 Frank LoVecchio :
> Did the site get hacked?
> http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.7.1/apache-cassandra-0.7.1-bin.tar.gz
> Sources keep changing...
>
> On Mon, Feb 14, 2011 at 1:13 PM, Eric Evans  wrote:
>>
>> Today is Valentine's Day[1] in many parts of the world, an annual
>> commemoration of love and affection typically celebrated with candy,
>> stuffed animals, and floral arrangements.
>>
>> She may seem a bit a fickle at times, but Cassandra loves you, and since
>> most people would rather receive a gift of the heart than some
>> prefabricated sentiment, the project decided to give the gift of point
>> release, ala 0.7.1.
>>
>> Happy Valentine's Day! :)
>>
>> Redeem yours from the usual place[5] (or [6] for users of Debian and
>> derivatives).
>>
>> And, as usual, be sure to read through the changes[2] and release
>> notes[3]. Report any problems you find[4], and if you have any
>> questions, don't hesitate to ask.
>>
>> XOXOXO
>>
>> [1]: http://en.wikipedia.org/wiki/Valentine's_Day
>> [2]: http://goo.gl/5VAPP (CHANGES.txt)
>> [3]: http://goo.gl/C9M5W (NEWS.txt)
>> [4]: https://issues.apache.org/jira/browse/CASSANDRA
>> [5]: http://cassandra.apache.org/download
>> [6]: http://wiki.apache.org/cassandra/DebianPackaging
>>
>> --
>> Eric Evans
>> eev...@rackspace.com
>>
>
>
>
> --
> Frank LoVecchio
> Senior Software Engineer | Isidorey, LLC
> Google Voice +1.720.295.9179
> isidorey.com | facebook.com/franklovecchio | franklovecchio.com
>


How do I repair what looks to be a corrupt sstable index?

2011-02-14 Thread Scott McCarty
Hi,

I don't know how data on at least one of our five nodes in our cluster got
bad but what I'm seeing is that the key value output from sstable2json
doesn't match the key value that I specify on the command line.

For example, I do the following on one of the nodes:

bash$:/root Mon Feb 14 12:47:30pm
===> . /opt/cassandra/config/core/cassandra.in.sh &&
/opt/cassandra/bin/sstable2json
 /opt/cassandra/storage/core/data/p1/cf1-6239-Data.db -k
US15600be0-e8b4-4439-a7df-d2e21aace7c4:all
2011-02-14 12:47:47,315 INFO
[org.apache.cassandra.config.DatabaseDescriptor] - DiskAccessMode 'auto'
determined to be mmap, indexAccessMode is mmap
2011-02-14 12:47:47,951 INFO [org.apache.cassandra.io.SSTableReader] -
Opening /opt/cassandra/storage/core/data/p1/cf1-6239-Data.db
{
  "USc5494dfa-678a-4175-a1d5-65730065c69d:UNHIDE":
[["25636f6e763a50524331373331663131662d383466342d343463662d393037652d343565393732326561346161",
"05f49d9d252e01", 1297711209972000, false],
["256d6f64646174653a313239373730383732323632343a50524331373331663131662d383466342d343463662d393037652d343565393732326561346161",
"4d597f9f", 1297710990061000,
true],
["256d6f64646174653a313239373731303939303036313a50524331373331663131662d383466342d343463662d393037652d343565393732326561346161",
"4d59806b", 1297711209972000, true],
["256d6f64646174653a313239373731313230393937323a50524331373331663131662d383466342d343463662d393037652d343565393732326561346161",
"04555337303266363235652d643933362d343138342d613834392d376330616635323430396165",
1297711209972000, false]]
}

Notice that the key I specified on the command line
("US15600be0-e8b4-4439-a7df-d2e21aace7c4:all") doesn't match what comes back
as the JSON key ("USc5494dfa-678a-4175-a1d5-65730065c69d:UH").  These match
on a small sample of other nodes and other keys (as I would expect).

If this is a corrupt sstable index, how can I repair this?  We're running
0.6.11 and have RF=3 with the 5 nodes and have been using CL=QUORUM for
reads and writes for quite a while.  Something tells me it's not as easy as
deleting the index file :-)

Of course, the bigger question is how did it get this way...

Thanks!
  Scott


RandomPartitioner

2011-02-14 Thread mcasandra

I am trying to understand atleast to some level of detail about how random
partitioner works. With the text I have seen on the website I am not able to
clearly understand. Is there a place where it's described with an example,
for eg how nodes are assigned random tokens? Is the range picked randomly
and assigned to nodes in circular fashion, for eg: hash ABC to FGH goes to
node A and then hash IJKLM-OPQR goes to node B?
-- 
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/RandomPartitioner-tp6025203p6025203.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.


Re: [RELEASE] 0.7.1

2011-02-14 Thread Frank LoVecchio
Did the site get hacked?

http://www.apache.org/dyn/closer.cgi?path=/cassandra/0.7.1/apache-cassandra-0.7.1-bin.tar.gz

Sources keep changing...

On Mon, Feb 14, 2011 at 1:13 PM, Eric Evans  wrote:

>
> Today is Valentine's Day[1] in many parts of the world, an annual
> commemoration of love and affection typically celebrated with candy,
> stuffed animals, and floral arrangements.
>
> She may seem a bit a fickle at times, but Cassandra loves you, and since
> most people would rather receive a gift of the heart than some
> prefabricated sentiment, the project decided to give the gift of point
> release, ala 0.7.1.
>
> Happy Valentine's Day! :)
>
> Redeem yours from the usual place[5] (or [6] for users of Debian and
> derivatives).
>
> And, as usual, be sure to read through the changes[2] and release
> notes[3]. Report any problems you find[4], and if you have any
> questions, don't hesitate to ask.
>
> XOXOXO
>
> [1]: http://en.wikipedia.org/wiki/Valentine's_Day
> [2]: http://goo.gl/5VAPP (CHANGES.txt)
> [3]: http://goo.gl/C9M5W (NEWS.txt)
> [4]: https://issues.apache.org/jira/browse/CASSANDRA
> [5]: http://cassandra.apache.org/download
> [6]: http://wiki.apache.org/cassandra/DebianPackaging
>
> --
> Eric Evans
> eev...@rackspace.com
>
>


-- 
Frank LoVecchio
Senior Software Engineer | Isidorey, LLC
Google Voice +1.720.295.9179
isidorey.com | facebook.com/franklovecchio | franklovecchio.com


Re: JNA.jar

2011-02-14 Thread Aaron Morton
Ouch, that redesign is a bit nasty. jna.jar in this folder is the same as the one I last got 3.2.7http://java.net/projects/jna/sources/svn/show/trunk/jnalib/dist?rev=1182AaronOn 15 Feb, 2011,at 09:48 AM, mcasandra  wrote:
In Cassandra documentation it recommends downloading jna.jar. However I am
unable to see any jar files on the mentioned website

http://java.net/projects/jna/ http://java.net/projects/jna/ 

Do I need to download the source instead and then compile it?
-- 
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/JNA-jar-tp6025122p6025122.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabblecom.


Re: Extra Large Memtables

2011-02-14 Thread Robert Coli
On Sat, Feb 12, 2011 at 11:17 PM, E S  wrote:
> While experimenting with this, I found a bug where you can't have memtable
> throughput configured past 2 gigs without an integer overflow screwing up the
> flushes.  That makes me feel like I'm in uncharted territory :).

I am sure the project would appreciate a JIRA ticket detailing how to
reproduce this behavior, which sounds like a bug.

https://issues.apache.org/jira/browse/CASSANDRA

Regarding very large memtables, it is important to recognize that
throughput refers only to the size of the COLUMN VALUES, and not, for
example, their names. In cases where, for example, an empty column
value is being stored with a UUID column name, your memtable can be
much larger than the value in the cf definition. I have seen 1GB-sized
memtables flush to disk as 1.7GB, and they are likely significantly
larger in memory due to Java object overhead.

=Rob


JNA.jar

2011-02-14 Thread mcasandra

In Cassandra documentation it recommends downloading jna.jar. However I am
unable to see any jar files on the mentioned website

http://java.net/projects/jna/ http://java.net/projects/jna/ 

Do I need to download the source instead and then compile it?
-- 
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/JNA-jar-tp6025122p6025122.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.


[RELEASE] 0.7.1

2011-02-14 Thread Eric Evans

Today is Valentine's Day[1] in many parts of the world, an annual
commemoration of love and affection typically celebrated with candy,
stuffed animals, and floral arrangements.

She may seem a bit a fickle at times, but Cassandra loves you, and since
most people would rather receive a gift of the heart than some
prefabricated sentiment, the project decided to give the gift of point
release, ala 0.7.1.

Happy Valentine's Day! :)

Redeem yours from the usual place[5] (or [6] for users of Debian and
derivatives).

And, as usual, be sure to read through the changes[2] and release
notes[3]. Report any problems you find[4], and if you have any
questions, don't hesitate to ask.

XOXOXO

[1]: http://en.wikipedia.org/wiki/Valentine's_Day
[2]: http://goo.gl/5VAPP (CHANGES.txt)
[3]: http://goo.gl/C9M5W (NEWS.txt)
[4]: https://issues.apache.org/jira/browse/CASSANDRA
[5]: http://cassandra.apache.org/download
[6]: http://wiki.apache.org/cassandra/DebianPackaging

-- 
Eric Evans
eev...@rackspace.com



Re: cassandra as session store

2011-02-14 Thread Sasha Dolgy
hi,

a few weeks back this topic had some discussion (cassandra as a session
store).  subsequently, i threw together a quick hack to have PHP use
Cassandra as a session store.  A benefit I quickly found is that I could
rely on Cassandra to expire the sessions and not PHP session garbage
collection.  nice.

The code is at the following URL and uses phpcassa as the interface to
thrift.

http://pastebin.com/pKEKhCgg

Sessions were given their own column family, with the row key defined as the
site name.  each column is a session id with the column data being the
session data.

Works a treat.  Each time PHP session_start() is called, the expiry on the
column resets back to the original value.  So, if the expiry is set to 1
hour, it's 1 hour after the last session_start() was invoked.

-sd


-- 
Sasha Dolgy
sasha.do...@gmail.com


Re: Cassandra documentation

2011-02-14 Thread Sameer Farooqui
Here is a blog my team is working on at Accenture which is intended to be a
complete beginner's guide to Cassandra. I'm still updating a few posts based
on DataStax's recommendations and I need to add the last three posts (will
get this done soon), but you can start checking it out via this link:

https://sites.google.com/a/techlabs.accenture.com/cassandra


- Sameer


Re: time to live rows

2011-02-14 Thread Kallin Nagelberg
Huh... I usually insert, compact, then flush. Apparently I've been
doing it wrong my whole life. So it needs like a courtesy flush. Let
me try that :)

-Kal

On Thu, Feb 10, 2011 at 3:06 AM, Sylvain Lebresne  wrote:
> Kal, you may have to flush before compacting.
> If you insert then compact, then it's almost certain that the inserts
> are still in the memtable, and thus not compacted.
> On Tue, Feb 8, 2011 at 9:54 PM, Kallin Nagelberg
>  wrote:
>>
>> What's the secret recipe that I'm missing? I tried forcing compaction
>> on my column family's JMX bean
>> (org.apache.cassandra.db.ColumnFamilies.Main.Session) in jconsole,
>> after gc_grace had passed (i set it to 60).
>>
>> Thanks,
>> -Kal
>>
>> On Tue, Feb 8, 2011 at 3:46 PM, Benjamin Coverston
>>  wrote:
>> >
>> > On 2/8/11 1:23 PM, Kallin Nagelberg wrote:
>> >>
>> >> I did read those articles, but I didn't know know that deleting all
>> >> the columns on a row was equivalent to deleting the row. Like I
>> >> mentioned, I did delete all the columns from all my rows and then
>> >> forced compaction before and after gc_grace had passed, but all the
>> >> rows still exist. If they never disappear, then won't I run out of
>> >> resources eventually?
>> >>
>> >> -Kal
>> >
>> > You would, if there weren't a way to get rid of tombstones:
>> >
>> > http://wiki.apache.org/cassandra/DistributedDeletes
>> >
>> > --
>> > Ben Coverston
>> > DataStax -- The Apache Cassandra Company
>> >
>> >
>
>


Re: Indexes and hard disk

2011-02-14 Thread Javier Canillas
Well, we have had some problems when a compactation was kicking into the
system under heavy load on version 0.6.3. This caused some nodes to respond
slow and caused some application instability.

What we do, playing with compactation strategy, was to move the major
compactation to a fixed hour of the day when we know there won't be heavy
traffic. But all this was posible because we had enough disk space to
continue holding new SSTables storage without problem.

Hope this information throw some light on my recommendation.

On Sat, Feb 12, 2011 at 3:47 PM, mcasandra  wrote:

>
>
> Javier Canillas wrote:
> >
> >
> > On the other hand, if your application will use heavily cassandra i
> > recommend making a calcularon of the data volumen that will handle and
> > using a partition that is actually 3 to 5 times bigger. Doing so you
> > will be able to play around with compactation strategy to avoid
> > latency when your application is under heavy load.
> >
> >
>
> Can you please explain with an example? I just started reading about
> Cassandra, all I know is that Cassandra partitions based on the hash space
> and divides equally among all the nodes. The tokens (keys) are then
> distributed equally accross N replica nodes.
>
> --
> View this message in context:
> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Indexes-and-hard-disk-tp6018234p6019297.html
> Sent from the cassandra-u...@incubator.apache.org mailing list archive at
> Nabble.com.
>


Re: cassandra solaris x64 support

2011-02-14 Thread Xiaobo Gu
On Sun, Feb 13, 2011 at 2:28 AM, Sylvain Lebresne  wrote:
>
>
> On Sat, Feb 12, 2011 at 2:52 AM, Xiaobo Gu  wrote:
>>
>> On Fri, Feb 11, 2011 at 11:54 PM, Sylvain Lebresne 
>> wrote:
>> >
>> >
>> > On Fri, Feb 11, 2011 at 4:27 PM, Xiaobo Gu 
>> > wrote:
>> >>
>> >> On Fri, Feb 11, 2011 at 11:21 PM, Roland Gude
>> >> 
>> >> wrote:
>> >> > This is a problem with the start scripts, not with Cassandra itself
>> >> > (or
>> >> > any of its configuration)
>> >> > The shell you are using cannot start the cassandra shell script.
>> >> >
>> >> > Try
>> >> > #bash bin/cassandra -f
>> >> You are right, but there are other problems, but you help me make a
>> >> big step, I'll dig dipper tomorrow.
>> >>
>> >> -bash-3.00$ /bin/bash bin/cassandra -f
>> >> Error occurred during initialization of VM
>> >> Could not reserve enough space for object heap
>> >> -bash-3.00$
>> >
>> > You apparently don't have enough free memory available for the
>> > default setting (1G). Uncomment and change the value of
>> > MAX_HEAP_SIZE in conf/cassandra-env.sh.
>> >
>> >>
>> >> Does the start script automatically choose the 64bit Java virtual
>> >> machine, I have both 64bit and 32bit jdk installed.
>> >
>> > It either use $JAVA_HOME/bin/java if $JAVA_HOME is defined in
>> > your environment or the value returned by `which java`.
>>
>> 32bit and 64bit java binaries are mixed up in one repository, there is
>> a $JAVA_HOME/bin/java, and there is also a $JAVA_HOME/bin/amd64/java,
>> I think the first one is 32bit and the second is 64bit, how can I
>> choose the 64bit one.
>
> Look at bin/cassandra. It's a shell script that, in particular set the
> JAVA variable that is the used binary. A quick (though slightly dirty)
> way to force the binary would be to hardcode the value in there.
> Otherwise, if you unset the $JAVA_HOME variable and make sure the
> 64 bit binary is in your path (but not the 32 bit one, or "after"), that
> will
> work too.
I have changed the code like this, hoping it will help others too

JAVA_HOME=/usr/java

# Use JAVA_HOME if set, otherwise look for java in PATH
if [ -x $JAVA_HOME/bin/amd64/java ]; then
JAVA=$JAVA_HOME/bin/amd64/java
else
 if [ -x $JAVA_HOME/bin/java]; then
JAVA=$JAVA_HOME/bin/java
 else
JAVA=`which java`
 fi
fi
> --
> Sylvain
>
>>
>> >>
>> >> > As far as I know, it should work fine. Actually it should work with
>> >> > sh
>> >> > as well...
>> >> >
>> >> >
>> >> > -Ursprüngliche Nachricht-
>> >> > Von: Xiaobo Gu [mailto:guxiaobo1...@gmail.com]
>> >> > Gesendet: Freitag, 11. Februar 2011 16:12
>> >> > An: user@cassandra.apache.org
>> >> > Betreff: Re: cassandra solaris x64 support
>> >> >
>> >> > On Fri, Feb 11, 2011 at 10:51 PM, Jonathan Ellis 
>> >> > wrote:
>> >> >> The vast majority run on Linux, but there are a few people running
>> >> >> Cassandra on Solaris, FreeBSD, and Windows.
>> >> > But I failed to start the one node test cluster,
>> >> > # sh bin/cassandra -f
>> >> > bin/cassandra: syntax error at line 22: `MAX_HEAP_SIZE=$' unexpected
>> >> >
>> >> > My environemnt is as follwoing:
>> >> > # more /etc/release
>> >> >                       Solaris 10 10/09 s10x_u8wos_08a X86
>> >> >           Copyright 2009 Sun Microsystems, Inc.  All Rights Reserved.
>> >> >                        Use is subject to license terms.
>> >> >                           Assembled 16 September 2009
>> >> >
>> >> > # java -fullversion
>> >> > java full version "1.6.0_23-b05"
>> >> > # java -version
>> >> > java version "1.6.0_23"
>> >> > Java(TM) SE Runtime Environment (build 1.6.0_23-b05)
>> >> > Java HotSpot(TM) Client VM (build 19.0-b09, mixed mode, sharing)
>> >> >
>> >> > I changed initial_token:0
>> >> >
>> >> >
>> >> >> On Fri, Feb 11, 2011 at 4:40 AM, Xiaobo Gu 
>> >> >> wrote:
>> >> >>> Hi,
>> >> >>> Because I can't access the archives of the mailing list, so my
>> >> >>> apologies if someone have asked this before.
>> >> >>>
>> >> >>> Does any have successfully run Cassandra on Solaris 10 X64
>> >> >>> clusters?
>> >> >>>
>> >> >>> Regards,
>> >> >>>
>> >> >>> Xiaobo Gu
>> >> >>>
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Jonathan Ellis
>> >> >> Project Chair, Apache Cassandra
>> >> >> co-founder of DataStax, the source for professional Cassandra
>> >> >> support
>> >> >> http://www.datastax.com
>> >> >>
>> >> >
>> >> >
>> >> >
>> >
>> >
>
>


Re: Confused about get_slice SliceRange behavior with bloom filter

2011-02-14 Thread Aditya Narayan
Thanks for the clarifications..

On Mon, Feb 14, 2011 at 6:13 PM, Sylvain Lebresne wrote:

> On Mon, Feb 14, 2011 at 11:27 AM, Aditya Narayan  wrote:
>
>> Thanks Sylvain,
>>
>> I guess I might have misunderstood the meaning of column_index_size_in_kb,
>> My previous understanding about that was: it is the threshold size for a row
>> to pass, after which its columns will be indexed.
>>
>
> It is the size of the index 'bucket'. But given that there is no point to
> have an index with only one entry, it is true that it is also the threshold
> after wich row start to be indexed.
>
>
>>
>> If I have understood it correctly, it implies the size of the "blocks
>> (containing columns) that are kept together on the same index". So if you
>> make that high, a large no of columns will need to be deseralized for a
>> single column access, in that block. And it you make it lower than optimal
>> than indexes size will grow up, right?
>>
>
> yes
>
>
>> So I guess we should vary that depending on the size of our columns and
>> not the size of rows !? I have valueless columns for my usecase.
>
>
> Yes it depends mainly on the size of your columns. But if you have big
> rows, even with very tiny columns, you may still not want to put a too small
> value there. In general I would really make careful tests with your workload
> before changing the value of column_index_size_in_kb to see if it does make
> a difference. Not sure there is much to gain here.
>
> --
> Sylvain
>
>
>>
>>
>>
>>
>> On Mon, Feb 14, 2011 at 2:06 PM, Sylvain Lebresne 
>> wrote:
>>
>>> As said by aaron, if the whole row is under 64k, it won't matter. But
>>> since you spoke of very wide row, I'll assume the whole will be much more
>>> than 64k.
>>>
>>> If so, the row is indexed by block (of 64k, configurable). Then the read
>>> performance depends on how many of those block are needed for the query,
>>> since each block potentially means a seek (potentially because some block
>>> could happen to be sequential on disk). So if the columns you ask for are
>>> really randomly distributed, then yes, the biggest the row is, the biggest
>>> the chance is to have to hit many blocks and the biggest the chance is for
>>> these block to be far apart on disk.
>>>
>>> --
>>> Sylvain
>>>
>>> On Sun, Feb 13, 2011 at 10:19 PM, Aditya Narayan wrote:
>>>
 Jonathan,
 If I ask for around 150-200 columns (totally random not sequential) from
 a very wide row that contains more than a million or even more columns 
 then,
 is the read performance of the SliceQuery operation affected by or "depends
 on the length of the row" ?? (For my use case, I would use the column names
 list for this SliceQuery operation).


 Thanks
 Aditya


 On Sun, Feb 13, 2011 at 8:41 PM, Jonathan Ellis wrote:

> On Sun, Feb 13, 2011 at 12:37 AM, E S  wrote:
> > I've gotten myself really confused by
> > http://wiki.apache.org/cassandra/ArchitectureInternals and am hoping
> someone can
> > help me understand what the io behavior of this operation would be.
> >
> > When I do a get_slice for a column range, will it seek to every
> SSTable?  I had
> > thought that it would use the bloom filter on the row key so that it
> would only
> > do a seek to SSTables that have a very high probability of containing
> columns
> > for that row.
>
> Yes.
>
> > In the linked doc above, it seems to say that it is only used for
> > exact column names.  Am I misunderstanding this?
>
> Yes.  You may be confusing multi-row behavior with multi-column.
>
> > On a related note, if instead of using a SliceRange I provide an
> explicit list
> > of columns, will I have to read all SSTables that have values for the
> columns
>
> Yes.
>
> > or is it smart enough to stop after finding a value from the most
> recent
> > SSTable?
>
> There is no way to know which value is most recent without having to
> read it first.
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
>


>>>
>>
>


Re: Confused about get_slice SliceRange behavior with bloom filter

2011-02-14 Thread Sylvain Lebresne
On Mon, Feb 14, 2011 at 11:27 AM, Aditya Narayan  wrote:

> Thanks Sylvain,
>
> I guess I might have misunderstood the meaning of column_index_size_in_kb,
> My previous understanding about that was: it is the threshold size for a row
> to pass, after which its columns will be indexed.
>

It is the size of the index 'bucket'. But given that there is no point to
have an index with only one entry, it is true that it is also the threshold
after wich row start to be indexed.


>
> If I have understood it correctly, it implies the size of the "blocks
> (containing columns) that are kept together on the same index". So if you
> make that high, a large no of columns will need to be deseralized for a
> single column access, in that block. And it you make it lower than optimal
> than indexes size will grow up, right?
>

yes


> So I guess we should vary that depending on the size of our columns and not
> the size of rows !? I have valueless columns for my usecase.


Yes it depends mainly on the size of your columns. But if you have big rows,
even with very tiny columns, you may still not want to put a too small value
there. In general I would really make careful tests with your workload
before changing the value of column_index_size_in_kb to see if it does make
a difference. Not sure there is much to gain here.

--
Sylvain


>
>
>
>
> On Mon, Feb 14, 2011 at 2:06 PM, Sylvain Lebresne wrote:
>
>> As said by aaron, if the whole row is under 64k, it won't matter. But
>> since you spoke of very wide row, I'll assume the whole will be much more
>> than 64k.
>>
>> If so, the row is indexed by block (of 64k, configurable). Then the read
>> performance depends on how many of those block are needed for the query,
>> since each block potentially means a seek (potentially because some block
>> could happen to be sequential on disk). So if the columns you ask for are
>> really randomly distributed, then yes, the biggest the row is, the biggest
>> the chance is to have to hit many blocks and the biggest the chance is for
>> these block to be far apart on disk.
>>
>> --
>> Sylvain
>>
>> On Sun, Feb 13, 2011 at 10:19 PM, Aditya Narayan wrote:
>>
>>> Jonathan,
>>> If I ask for around 150-200 columns (totally random not sequential) from
>>> a very wide row that contains more than a million or even more columns then,
>>> is the read performance of the SliceQuery operation affected by or "depends
>>> on the length of the row" ?? (For my use case, I would use the column names
>>> list for this SliceQuery operation).
>>>
>>>
>>> Thanks
>>> Aditya
>>>
>>>
>>> On Sun, Feb 13, 2011 at 8:41 PM, Jonathan Ellis wrote:
>>>
 On Sun, Feb 13, 2011 at 12:37 AM, E S  wrote:
 > I've gotten myself really confused by
 > http://wiki.apache.org/cassandra/ArchitectureInternals and am hoping
 someone can
 > help me understand what the io behavior of this operation would be.
 >
 > When I do a get_slice for a column range, will it seek to every
 SSTable?  I had
 > thought that it would use the bloom filter on the row key so that it
 would only
 > do a seek to SSTables that have a very high probability of containing
 columns
 > for that row.

 Yes.

 > In the linked doc above, it seems to say that it is only used for
 > exact column names.  Am I misunderstanding this?

 Yes.  You may be confusing multi-row behavior with multi-column.

 > On a related note, if instead of using a SliceRange I provide an
 explicit list
 > of columns, will I have to read all SSTables that have values for the
 columns

 Yes.

 > or is it smart enough to stop after finding a value from the most
 recent
 > SSTable?

 There is no way to know which value is most recent without having to
 read it first.

 --
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder of DataStax, the source for professional Cassandra support
 http://www.datastax.com

>>>
>>>
>>
>


Re: Question about seeds in tow node cluster.

2011-02-14 Thread Xiaobo Gu
Thanks.

On Mon, Feb 14, 2011 at 7:59 PM, nicolas lattuada
 wrote:
> Hi
>
> I have a two nodes cluster running and have both of them in the seeds list.
>
> regards
>
>> Date: Sun, 13 Feb 2011 16:04:41 +0800
>> Subject: Question about seeds in tow node cluster.
>> From: guxiaobo1...@gmail.com
>> To: user@cassandra.apache.org
>>
>> Hi,
>> If the cluster only have tow nodes, should they both in the seeds list?
>>
>> Regards,
>>
>> Xiaobo Gu
>


RE: Question about seeds in tow node cluster.

2011-02-14 Thread nicolas lattuada

Hi 

I have a two nodes cluster running and have both of them in the seeds list.

regards

> Date: Sun, 13 Feb 2011 16:04:41 +0800
> Subject: Question about seeds in tow node cluster.
> From: guxiaobo1...@gmail.com
> To: user@cassandra.apache.org
> 
> Hi,
> If the cluster only have tow nodes, should they both in the seeds list?
> 
> Regards,
> 
> Xiaobo Gu
  

Re: Confused about get_slice SliceRange behavior with bloom filter

2011-02-14 Thread Aditya Narayan
Thanks Sylvain,

I guess I might have misunderstood the meaning of column_index_size_in_kb,
My previous understanding about that was: it is the threshold size for a row
to pass, after which its columns will be indexed.

If I have understood it correctly, it implies the size of the "blocks
(containing columns) that are kept together on the same index". So if you
make that high, a large no of columns will need to be deseralized for a
single column access, in that block. And it you make it lower than optimal
than indexes size will grow up, right?

So I guess we should vary that depending on the size of our columns and not
the size of rows !? I have valueless columns for my usecase.




On Mon, Feb 14, 2011 at 2:06 PM, Sylvain Lebresne wrote:

> As said by aaron, if the whole row is under 64k, it won't matter. But since
> you spoke of very wide row, I'll assume the whole will be much more than
> 64k.
>
> If so, the row is indexed by block (of 64k, configurable). Then the read
> performance depends on how many of those block are needed for the query,
> since each block potentially means a seek (potentially because some block
> could happen to be sequential on disk). So if the columns you ask for are
> really randomly distributed, then yes, the biggest the row is, the biggest
> the chance is to have to hit many blocks and the biggest the chance is for
> these block to be far apart on disk.
>
> --
> Sylvain
>
> On Sun, Feb 13, 2011 at 10:19 PM, Aditya Narayan  wrote:
>
>> Jonathan,
>> If I ask for around 150-200 columns (totally random not sequential) from a
>> very wide row that contains more than a million or even more columns then,
>> is the read performance of the SliceQuery operation affected by or "depends
>> on the length of the row" ?? (For my use case, I would use the column names
>> list for this SliceQuery operation).
>>
>>
>> Thanks
>> Aditya
>>
>>
>> On Sun, Feb 13, 2011 at 8:41 PM, Jonathan Ellis wrote:
>>
>>> On Sun, Feb 13, 2011 at 12:37 AM, E S  wrote:
>>> > I've gotten myself really confused by
>>> > http://wiki.apache.org/cassandra/ArchitectureInternals and am hoping
>>> someone can
>>> > help me understand what the io behavior of this operation would be.
>>> >
>>> > When I do a get_slice for a column range, will it seek to every
>>> SSTable?  I had
>>> > thought that it would use the bloom filter on the row key so that it
>>> would only
>>> > do a seek to SSTables that have a very high probability of containing
>>> columns
>>> > for that row.
>>>
>>> Yes.
>>>
>>> > In the linked doc above, it seems to say that it is only used for
>>> > exact column names.  Am I misunderstanding this?
>>>
>>> Yes.  You may be confusing multi-row behavior with multi-column.
>>>
>>> > On a related note, if instead of using a SliceRange I provide an
>>> explicit list
>>> > of columns, will I have to read all SSTables that have values for the
>>> columns
>>>
>>> Yes.
>>>
>>> > or is it smart enough to stop after finding a value from the most
>>> recent
>>> > SSTable?
>>>
>>> There is no way to know which value is most recent without having to
>>> read it first.
>>>
>>> --
>>> Jonathan Ellis
>>> Project Chair, Apache Cassandra
>>> co-founder of DataStax, the source for professional Cassandra support
>>> http://www.datastax.com
>>>
>>
>>
>


Re: Confused about get_slice SliceRange behavior with bloom filter

2011-02-14 Thread Sylvain Lebresne
As said by aaron, if the whole row is under 64k, it won't matter. But since
you spoke of very wide row, I'll assume the whole will be much more than
64k.

If so, the row is indexed by block (of 64k, configurable). Then the read
performance depends on how many of those block are needed for the query,
since each block potentially means a seek (potentially because some block
could happen to be sequential on disk). So if the columns you ask for are
really randomly distributed, then yes, the biggest the row is, the biggest
the chance is to have to hit many blocks and the biggest the chance is for
these block to be far apart on disk.

--
Sylvain

On Sun, Feb 13, 2011 at 10:19 PM, Aditya Narayan  wrote:

> Jonathan,
> If I ask for around 150-200 columns (totally random not sequential) from a
> very wide row that contains more than a million or even more columns then,
> is the read performance of the SliceQuery operation affected by or "depends
> on the length of the row" ?? (For my use case, I would use the column names
> list for this SliceQuery operation).
>
>
> Thanks
> Aditya
>
>
> On Sun, Feb 13, 2011 at 8:41 PM, Jonathan Ellis  wrote:
>
>> On Sun, Feb 13, 2011 at 12:37 AM, E S  wrote:
>> > I've gotten myself really confused by
>> > http://wiki.apache.org/cassandra/ArchitectureInternals and am hoping
>> someone can
>> > help me understand what the io behavior of this operation would be.
>> >
>> > When I do a get_slice for a column range, will it seek to every SSTable?
>>  I had
>> > thought that it would use the bloom filter on the row key so that it
>> would only
>> > do a seek to SSTables that have a very high probability of containing
>> columns
>> > for that row.
>>
>> Yes.
>>
>> > In the linked doc above, it seems to say that it is only used for
>> > exact column names.  Am I misunderstanding this?
>>
>> Yes.  You may be confusing multi-row behavior with multi-column.
>>
>> > On a related note, if instead of using a SliceRange I provide an
>> explicit list
>> > of columns, will I have to read all SSTables that have values for the
>> columns
>>
>> Yes.
>>
>> > or is it smart enough to stop after finding a value from the most recent
>> > SSTable?
>>
>> There is no way to know which value is most recent without having to
>> read it first.
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
>>
>
>