[
https://issues.apache.org/jira/browse/CASSANDRA-7088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13981248#comment-13981248
]
Jacek Furmankiewicz edited comment on CASSANDRA-7088 at 4/25/14 5:05 PM:
-------------------------------------------------------------------------
Hi Brandon, we have good news. I am not sure if you still want to treat it as
an issue or not.
The core underlying problem is that we had a mismatch between a composite
column type in our Astyanax code (long,long,String)
vs how the the actual CF was created
(CompositeType(LongType,LongType,IntegerType)).
Now one ever noticed because it worked just fine in 1.2 (how I am not
sure....it should have bombed on the first query....I suspect Astyanax did some
raw binary serialization that somehow hid this underlying problem).
However, once we moved to 2.0, Cassandra must have detected that something does
not make sense in the query vs the underlying data representation and hence the
zero length BigInteger error.
So it is definitely an issue on our side.
The only potential issue on the Cassandra side is that this error caused the
entire query to freeze until we got an operation timeout exception, which sort
of hid the underlying issue for a while.
I will leave it up to you if you want to leave that particular issue open for
further investigation on your side or close it.
Thank you for your help along the way.
was (Author: jfurmankiewicz):
Hi Brandon, we have good news. I am not sure if you still want to treat it as
an issue or not.
The core underlying problem is that we had a mismatch between a composite
column type in our Astyanax code (long,long,String)
vs how the the actual CF was created
(CompositeType(LongType,LongType,IntegerType)).
Now one ever noticed because it worked just fine in 1.2 (how I am not
sure....it should have bombed on the first query....I suspect Astyanax did some
raw binary serialization that somehow hid this underlying problem).
However, once we moved to 2.0, Cassandra must have detected that something does
not make sense in the query vs the underlying data representation and hence the
zero length BigInteger error.
So it is definitely an issue on our side.
The only potential issue on the Cassandra side is that this error caused the
entire query to freeze until we got an operation timeout exception, which sort
of hid the underlying issue for a while.
I will leave it up to you if you want to leave the issue open for further
investigation on your side or close it.
Thank you for your help along the way.
> Zero length BigInteger exception causing processing to freeze
> --------------------------------------------------------------
>
> Key: CASSANDRA-7088
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7088
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Linux Mint 16
> Reporter: Jacek Furmankiewicz
> Attachments: cassandra.yaml
>
>
> We attempted to migrate our developers to Cassandra 2.0.7 from 1.2.
> Everything worked perfectly, but we have experienced a massive drop in
> developer velocity.
> We run integration tests with Cucumber BDD and 1000 BDDs went from 7 minutes
> (Cassandra 1.2) to 15 minutes (2.0.7),
> This is when we run Cassandra of the ramdisk (/dev/shm) to make it run faster
> on dev boxes.
> When we tried pointed to actual drives the difference was dramatic: the
> entire suite took over 70 minutes (!) vs 15 in Cassandra 1.2.
> After investigation, we found that most of the time is spent in the
> truncation logic between every scenario, where we truncate all the column
> families and start with a clean DB for the next test case.
> This used to be super fast in 1.2, is now very slow in 2.0.
> It may not seem important, but upgrading to 2.0 has basically cut down
> developer velocity by 100%, just by more than doubling the time it takes to
> run our BDD suite.
> We truncate the CFs using the Ruby driver:
> $cassandra.column_families.each do |column_family|
> name = column_family[0].to_s
> $cassandra.truncate! name
> end
> I am attaching our cassandra.yaml. Please note we already switched off
> auto_compaction before truncate, just as we did in 1.2 for dev boxes, Made no
> difference.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
