Re: possible content for C* wiki

[Russell Bradberry]

There is in depth documentation available here: http://www.datastax.com/docs , 
albeit not as visual as the blog post.



On May 15, 2014 at 10:33:08 AM, Daniel Shelepov (dan...@timefork.com) wrote:

Hello,  



I've recently undertook a study into Cassandra internals. One of the  
reasons was because the available documentation, such as pages like this  
(http://wiki.apache.org/cassandra/ArchitectureInternals ) weren't very  
complete. One of the outcomes of the study was a blog post about C*  
top-level packages (talking about Java here). If people think it's a useful  
overview, I could adapt that content to be put on the wiki so other people  
can find it. If it's not a good fit, no problem (not trying to be pushy  
here). The blog post is this one  
http://timefork.com/blog/apache-cassandra-internals .  



Regardless of whether it goes into the wiki or not, if someone has  
corrections or is otherwise opinionated, I'm happy to hear him out.  



Daniel  

possible content for C* wiki

[Daniel Shelepov]

Hello,

 

I've recently undertook a study into Cassandra internals.  One of the
reasons was because the available documentation, such as pages like this
(http://wiki.apache.org/cassandra/ArchitectureInternals ) weren't very
complete.  One of the outcomes of the study was a blog post about C*
top-level packages (talking about Java here).  If people think it's a useful
overview, I could adapt that content to be put on the wiki so other people
can find it.  If it's not a good fit, no problem (not trying to be pushy
here).  The blog post is this one
http://timefork.com/blog/apache-cassandra-internals .

 

Regardless of whether it goes into the wiki or not, if someone has
corrections or is otherwise opinionated, I'm happy to hear him out.

 

Daniel

 

Re: NPE in conditional updates w/ collections in 2.0.7

[Tyler Hobbs]

Hi Brian,

Thanks for the report.  This looks like
https://issues.apache.org/jira/browse/CASSANDRA-7155, which should be fixed
shortly.


On Thu, May 15, 2014 at 3:23 PM, Brian O'Neill wrote:

>
> OK ‹ we¹ve got some hyper data modeling going on, taking advantage of all
> the latest toys in CQL 2.  And we ran into some trouble using maps within
> conditional updates.  Specifically, when testing to see if a key exists in
> a
> map (with =null?), we encounter an NPE server-side.  We believe this worked
> in 2.0.4.
>
> With this schema:
> CREATE TABLE progress (
> key text,
> count int,
> partitions map,
> primary key (key)
> );
>
> When executing the following:
> cqlsh:hms> UPDATE foo SET count=4962 WHERE key='PA' IF
> partitions['a']=null;
>
>  [applied]
> ---
>  False
>
> cqlsh:hms> UPDATE foo SET count=4962 WHERE key='PA';
> cqlsh:hms> UPDATE foo SET count=4962 WHERE key='PA' IF
> partitions['a']=null;
> TSocket read 0 bytes
>
> We see the following NPE server-side:
> ERROR [Native-Transport-Requests:13353] 2014-05-15 15:10:00,154
> QueryMessage.java (line 131) Unexpected error during query
> java.lang.NullPointerException
> at
>
> org.apache.cassandra.cql3.ColumnCondition$WithVariables.collectionAppliesTo(
> ColumnCondition.java:168)
> at
>
> org.apache.cassandra.cql3.ColumnCondition$WithVariables.appliesTo(ColumnCond
> ition.java:142)
> at
>
> org.apache.cassandra.cql3.statements.CQL3CasConditions$ColumnsConditions.app
> liesTo(CQL3CasConditions.java:197)
> at
>
> org.apache.cassandra.cql3.statements.CQL3CasConditions.appliesTo(CQL3CasCond
> itions.java:108)
>
> Is there a better way to test for existence of a key?
> Or is this a bug?  (Regardless, we may want to protect against the NPE)
> Or am I missing something entirely?
>
> -brian
>
> ---
> Brian O'Neill
> Chief Technology Officer
>
>
> Health Market Science
> The Science of Better Results
> 2700 Horizon Drive € King of Prussia, PA € 19406
> M: 215.588.6024 € @boneill42    €
> healthmarketscience.com
>
>
> This information transmitted in this email message is for the intended
> recipient only and may contain confidential and/or privileged material. If
> you received this email in error and are not the intended recipient, or the
> person responsible to deliver it to the intended recipient, please contact
> the sender at the email above and delete this email and any attachments and
> destroy any copies thereof. Any review, retransmission, dissemination,
> copying or other use of, or taking any action in reliance upon, this
> information by persons or entities other than the intended recipient is
> strictly prohibited.
>
>
>
>


-- 
Tyler Hobbs
DataStax 

Re: NPE in conditional updates w/ collections in 2.0.7

[Brian O'Neill]

Perfect.  Thanks Tyler.

Great to hear you guys are already on top of it.  I’ll watch for the
resolution.

-brian

---
Brian O'Neill
Chief Technology Officer

Health Market Science
The Science of Better Results
2700 Horizon Drive • King of Prussia, PA • 19406
M: 215.588.6024 • @boneill42   •
healthmarketscience.com

This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material. If
you received this email in error and are not the intended recipient, or
the person responsible to deliver it to the intended recipient, please
contact the sender at the email above and delete this email and any
attachments and destroy any copies thereof. Any review, retransmission,
dissemination, copying or other use of, or taking any action in reliance
upon, this information by persons or entities other than the intended
recipient is strictly prohibited.
 






On 5/16/14, 12:25 PM, "Tyler Hobbs"  wrote:

>Hi Brian,
>
>Thanks for the report.  This looks like
>https://issues.apache.org/jira/browse/CASSANDRA-7155, which should be
>fixed
>shortly.
>
>
>On Thu, May 15, 2014 at 3:23 PM, Brian O'Neill
>wrote:
>
>>
>> OK ‹ we¹ve got some hyper data modeling going on, taking advantage of
>>all
>> the latest toys in CQL 2.  And we ran into some trouble using maps
>>within
>> conditional updates.  Specifically, when testing to see if a key exists
>>in
>> a
>> map (with =null?), we encounter an NPE server-side.  We believe this
>>worked
>> in 2.0.4.
>>
>> With this schema:
>> CREATE TABLE progress (
>> key text,
>> count int,
>> partitions map,
>> primary key (key)
>> );
>>
>> When executing the following:
>> cqlsh:hms> UPDATE foo SET count=4962 WHERE key='PA' IF
>> partitions['a']=null;
>>
>>  [applied]
>> ---
>>  False
>>
>> cqlsh:hms> UPDATE foo SET count=4962 WHERE key='PA';
>> cqlsh:hms> UPDATE foo SET count=4962 WHERE key='PA' IF
>> partitions['a']=null;
>> TSocket read 0 bytes
>>
>> We see the following NPE server-side:
>> ERROR [Native-Transport-Requests:13353] 2014-05-15 15:10:00,154
>> QueryMessage.java (line 131) Unexpected error during query
>> java.lang.NullPointerException
>> at
>>
>> 
>>org.apache.cassandra.cql3.ColumnCondition$WithVariables.collectionApplies
>>To(
>> ColumnCondition.java:168)
>> at
>>
>> 
>>org.apache.cassandra.cql3.ColumnCondition$WithVariables.appliesTo(ColumnC
>>ond
>> ition.java:142)
>> at
>>
>> 
>>org.apache.cassandra.cql3.statements.CQL3CasConditions$ColumnsConditions.
>>app
>> liesTo(CQL3CasConditions.java:197)
>> at
>>
>> 
>>org.apache.cassandra.cql3.statements.CQL3CasConditions.appliesTo(CQL3CasC
>>ond
>> itions.java:108)
>>
>> Is there a better way to test for existence of a key?
>> Or is this a bug?  (Regardless, we may want to protect against the NPE)
>> Or am I missing something entirely?
>>
>> -brian
>>
>> ---
>> Brian O'Neill
>> Chief Technology Officer
>>
>>
>> Health Market Science
>> The Science of Better Results
>> 2700 Horizon Drive € King of Prussia, PA € 19406
>> M: 215.588.6024 € @boneill42    €
>> healthmarketscience.com
>>
>>
>> This information transmitted in this email message is for the intended
>> recipient only and may contain confidential and/or privileged material.
>>If
>> you received this email in error and are not the intended recipient, or
>>the
>> person responsible to deliver it to the intended recipient, please
>>contact
>> the sender at the email above and delete this email and any attachments
>>and
>> destroy any copies thereof. Any review, retransmission, dissemination,
>> copying or other use of, or taking any action in reliance upon, this
>> information by persons or entities other than the intended recipient is
>> strictly prohibited.
>>
>>
>>
>>
>
>
>-- 
>Tyler Hobbs
>DataStax 

NPE in conditional updates w/ collections in 2.0.7

[Brian O'Neill]

OK ‹ we¹ve got some hyper data modeling going on, taking advantage of all
the latest toys in CQL 2.  And we ran into some trouble using maps within
conditional updates.  Specifically, when testing to see if a key exists in a
map (with =null?), we encounter an NPE server-side.  We believe this worked
in 2.0.4.

With this schema:
CREATE TABLE progress (
key text,
count int,
partitions map,
primary key (key)
);

When executing the following:
cqlsh:hms> UPDATE foo SET count=4962 WHERE key='PA' IF partitions['a']=null;

 [applied]
---
 False

cqlsh:hms> UPDATE foo SET count=4962 WHERE key='PA';
cqlsh:hms> UPDATE foo SET count=4962 WHERE key='PA' IF partitions['a']=null;
TSocket read 0 bytes

We see the following NPE server-side:
ERROR [Native-Transport-Requests:13353] 2014-05-15 15:10:00,154
QueryMessage.java (line 131) Unexpected error during query
java.lang.NullPointerException
at 
org.apache.cassandra.cql3.ColumnCondition$WithVariables.collectionAppliesTo(
ColumnCondition.java:168)
at 
org.apache.cassandra.cql3.ColumnCondition$WithVariables.appliesTo(ColumnCond
ition.java:142)
at 
org.apache.cassandra.cql3.statements.CQL3CasConditions$ColumnsConditions.app
liesTo(CQL3CasConditions.java:197)
at 
org.apache.cassandra.cql3.statements.CQL3CasConditions.appliesTo(CQL3CasCond
itions.java:108)

Is there a better way to test for existence of a key?
Or is this a bug?  (Regardless, we may want to protect against the NPE)
Or am I missing something entirely?

-brian

---
Brian O'Neill
Chief Technology Officer


Health Market Science
The Science of Better Results
2700 Horizon Drive € King of Prussia, PA € 19406
M: 215.588.6024 € @boneill42    €
healthmarketscience.com


This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material. If
you received this email in error and are not the intended recipient, or the
person responsible to deliver it to the intended recipient, please contact
the sender at the email above and delete this email and any attachments and
destroy any copies thereof. Any review, retransmission, dissemination,
copying or other use of, or taking any action in reliance upon, this
information by persons or entities other than the intended recipient is
strictly prohibited.
 

Re: FW: possible content for C* wiki

[Jonathan Ellis]

Sure, we'd be happy to add you as a wiki editor.  What's your login?

On Mon, May 12, 2014 at 4:28 PM, Daniel Shelepov  wrote:
> Resending this, because apparently this was undelivered according to
> http://mail-archives.apache.org/mod_mbox/cassandra-dev/201405.mbox/browser
>
>
>
> From: Daniel Shelepov [mailto:dan...@timefork.com]
> Sent: Friday, May 09, 2014 6:50 PM
> To: dev@cassandra.apache.org
> Subject: possible content for C* wiki
>
>
>
> Hello,
>
>
>
> I've recently undertook a study into Cassandra internals.  One of the
> reasons was because the available documentation, such as pages like this
> (http://wiki.apache.org/cassandra/ArchitectureInternals ) weren't very
> complete.  One of the outcomes of the study was a blog post about C*
> top-level packages (talking about Java here).  If people think it's a useful
> overview, I could adapt that content to be put on the wiki so other people
> can find it.  If it's not a good fit, no problem (not trying to be pushy
> here).  The blog post is this one
> http://timefork.com/blog/apache-cassandra-internals .
>
>
>
> Regardless of whether it goes into the wiki or not, if someone has
> corrections or is otherwise opinionated, I'm happy to hear him out.
>
>
>
> Daniel
>
>
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder, http://www.datastax.com
@spyced

Re: possible content for C* wiki

[Dave Brosius]

Feel free to add it to the WIKI, if you need an account let us know and 
we can add one for you.


Thanks!

---


On 2014-05-09 21:49, Daniel Shelepov wrote:

Hello,



I've recently undertook a study into Cassandra internals.  One of the
reasons was because the available documentation, such as pages like 
this

(http://wiki.apache.org/cassandra/ArchitectureInternals ) weren't very
complete.  One of the outcomes of the study was a blog post about C*
top-level packages (talking about Java here).  If people think it's a 
useful
overview, I could adapt that content to be put on the wiki so other 
people
can find it.  If it's not a good fit, no problem (not trying to be 
pushy

here).  The blog post is this one
http://timefork.com/blog/apache-cassandra-internals .



Regardless of whether it goes into the wiki or not, if someone has
corrections or is otherwise opinionated, I'm happy to hear him out.



Daniel

Things that are about 1M big

[graham sanderson]

So just throwing this out there for those for whom this might ring a bell.

I’m debugging some CMS memory fragmentation issues on 2.0.5 - and interestingly 
enough most of the objects giving us promotion failures are of size 131074 
(dwords) - GC logging obviously doesn’t say what those are, but I’d wager money 
they are either 1M big byte arrays, or less likely 256k entry object arrays 
backing large maps

So not strictly critical to solving my problem, but I was wondering if anyone 
can think of any heap allocated C* objects which are (with no significant 
changes to standard cassandra config) allocated in 1M chunks. (It would save me 
scouring the code, or a 9 gig heap dump if I need to figure it out!)

Thanks,

Graham

smime.p7s
Description: S/MIME cryptographic signature

RE: Things that are about 1M big

[Samuel CARRIERE]

Hi,
This is arena allocation of memtables. See here for more infos : 
http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-performance




De :graham sanderson 
A : dev@cassandra.apache.org, 
Date :  16/05/2014 14:03
Objet : Things that are about 1M big



So just throwing this out there for those for whom this might ring a bell.

I?m debugging some CMS memory fragmentation issues on 2.0.5 - and 
interestingly enough most of the objects giving us promotion failures are 
of size 131074 (dwords) - GC logging obviously doesn?t say what those are, 
but I?d wager money they are either 1M big byte arrays, or less likely 
256k entry object arrays backing large maps

So not strictly critical to solving my problem, but I was wondering if 
anyone can think of any heap allocated C* objects which are (with no 
significant changes to standard cassandra config) allocated in 1M chunks. 
(It would save me scouring the code, or a 9 gig heap dump if I need to 
figure it out!)

Thanks,

Graham

RE: FW: possible content for C* wiki

[Daniel Shelepov]

My username on apache.org is daniels.

-Original Message-
From: Jonathan Ellis [mailto:jbel...@gmail.com] 
Sent: Friday, May 16, 2014 1:52 PM
To: dev
Subject: Re: FW: possible content for C* wiki

Sure, we'd be happy to add you as a wiki editor.  What's your login?

On Mon, May 12, 2014 at 4:28 PM, Daniel Shelepov  wrote:
> Resending this, because apparently this was undelivered according to 
> http://mail-archives.apache.org/mod_mbox/cassandra-dev/201405.mbox/bro
> wser
>
>
>
> From: Daniel Shelepov [mailto:dan...@timefork.com]
> Sent: Friday, May 09, 2014 6:50 PM
> To: dev@cassandra.apache.org
> Subject: possible content for C* wiki
>
>
>
> Hello,
>
>
>
> I've recently undertook a study into Cassandra internals.  One of the 
> reasons was because the available documentation, such as pages like 
> this (http://wiki.apache.org/cassandra/ArchitectureInternals ) weren't 
> very complete.  One of the outcomes of the study was a blog post about 
> C* top-level packages (talking about Java here).  If people think it's 
> a useful overview, I could adapt that content to be put on the wiki so 
> other people can find it.  If it's not a good fit, no problem (not 
> trying to be pushy here).  The blog post is this one 
> http://timefork.com/blog/apache-cassandra-internals .
>
>
>
> Regardless of whether it goes into the wiki or not, if someone has 
> corrections or is otherwise opinionated, I'm happy to hear him out.
>
>
>
> Daniel
>
>
>



--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder, http://www.datastax.com
@spyced

Re: FW: possible content for C* wiki

[Brandon Williams]

I have added you.  Changes will take effect whenever moinmoin decides to
complete the call.


On Fri, May 16, 2014 at 4:48 PM, Daniel Shelepov wrote:

> My username on apache.org is daniels.
>
> -Original Message-
> From: Jonathan Ellis [mailto:jbel...@gmail.com]
> Sent: Friday, May 16, 2014 1:52 PM
> To: dev
> Subject: Re: FW: possible content for C* wiki
>
> Sure, we'd be happy to add you as a wiki editor.  What's your login?
>
> On Mon, May 12, 2014 at 4:28 PM, Daniel Shelepov 
> wrote:
> > Resending this, because apparently this was undelivered according to
> > http://mail-archives.apache.org/mod_mbox/cassandra-dev/201405.mbox/bro
> > wser
> >
> >
> >
> > From: Daniel Shelepov [mailto:dan...@timefork.com]
> > Sent: Friday, May 09, 2014 6:50 PM
> > To: dev@cassandra.apache.org
> > Subject: possible content for C* wiki
> >
> >
> >
> > Hello,
> >
> >
> >
> > I've recently undertook a study into Cassandra internals.  One of the
> > reasons was because the available documentation, such as pages like
> > this (http://wiki.apache.org/cassandra/ArchitectureInternals ) weren't
> > very complete.  One of the outcomes of the study was a blog post about
> > C* top-level packages (talking about Java here).  If people think it's
> > a useful overview, I could adapt that content to be put on the wiki so
> > other people can find it.  If it's not a good fit, no problem (not
> > trying to be pushy here).  The blog post is this one
> > http://timefork.com/blog/apache-cassandra-internals .
> >
> >
> >
> > Regardless of whether it goes into the wiki or not, if someone has
> > corrections or is otherwise opinionated, I'm happy to hear him out.
> >
> >
> >
> > Daniel
> >
> >
> >
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder, http://www.datastax.com
> @spyced
>
>

Re: Things that are about 1M big

[graham sanderson]

Excellent - thank you… 

On May 16, 2014, at 7:08 AM, Samuel CARRIERE  wrote:

> Hi,
> This is arena allocation of memtables. See here for more infos : 
> http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-performance
> 
> 
> 
> 
> De :graham sanderson 
> A : dev@cassandra.apache.org, 
> Date :  16/05/2014 14:03
> Objet : Things that are about 1M big
> 
> 
> 
> So just throwing this out there for those for whom this might ring a bell.
> 
> I?m debugging some CMS memory fragmentation issues on 2.0.5 - and 
> interestingly enough most of the objects giving us promotion failures are 
> of size 131074 (dwords) - GC logging obviously doesn?t say what those are, 
> but I?d wager money they are either 1M big byte arrays, or less likely 
> 256k entry object arrays backing large maps
> 
> So not strictly critical to solving my problem, but I was wondering if 
> anyone can think of any heap allocated C* objects which are (with no 
> significant changes to standard cassandra config) allocated in 1M chunks. 
> (It would save me scouring the code, or a 9 gig heap dump if I need to 
> figure it out!)
> 
> Thanks,
> 
> Graham



smime.p7s
Description: S/MIME cryptographic signature

RE: FW: possible content for C* wiki

[Daniel Shelepov]

Thanks.  I will upload the proposed content either this weekend or at some 
point next week.

-Original Message-
From: Brandon Williams [mailto:dri...@gmail.com] 
Sent: Friday, May 16, 2014 3:33 PM
To: dev@cassandra.apache.org
Subject: Re: FW: possible content for C* wiki

I have added you.  Changes will take effect whenever moinmoin decides to 
complete the call.


On Fri, May 16, 2014 at 4:48 PM, Daniel Shelepov wrote:

> My username on apache.org is daniels.
>
> -Original Message-
> From: Jonathan Ellis [mailto:jbel...@gmail.com]
> Sent: Friday, May 16, 2014 1:52 PM
> To: dev
> Subject: Re: FW: possible content for C* wiki
>
> Sure, we'd be happy to add you as a wiki editor.  What's your login?
>
> On Mon, May 12, 2014 at 4:28 PM, Daniel Shelepov 
> wrote:
> > Resending this, because apparently this was undelivered according to 
> > http://mail-archives.apache.org/mod_mbox/cassandra-dev/201405.mbox/b
> > ro
> > wser
> >
> >
> >
> > From: Daniel Shelepov [mailto:dan...@timefork.com]
> > Sent: Friday, May 09, 2014 6:50 PM
> > To: dev@cassandra.apache.org
> > Subject: possible content for C* wiki
> >
> >
> >
> > Hello,
> >
> >
> >
> > I've recently undertook a study into Cassandra internals.  One of 
> > the reasons was because the available documentation, such as pages 
> > like this (http://wiki.apache.org/cassandra/ArchitectureInternals ) 
> > weren't very complete.  One of the outcomes of the study was a blog 
> > post about
> > C* top-level packages (talking about Java here).  If people think 
> > it's a useful overview, I could adapt that content to be put on the 
> > wiki so other people can find it.  If it's not a good fit, no 
> > problem (not trying to be pushy here).  The blog post is this one 
> > http://timefork.com/blog/apache-cassandra-internals .
> >
> >
> >
> > Regardless of whether it goes into the wiki or not, if someone has 
> > corrections or is otherwise opinionated, I'm happy to hear him out.
> >
> >
> >
> > Daniel
> >
> >
> >
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder, http://www.datastax.com
> @spyced
>
>