date:20150527

[jira] [Created] (CASSANDRA-9490) testcase failure : testWithDeletes(org.apache.cassandra.io.sstable.SSTableMetadataTest):

2015-05-27 Thread Pallavi Bhardwaj (JIRA)

Pallavi Bhardwaj created CASSANDRA-9490:
---

 Summary: testcase failure : 
testWithDeletes(org.apache.cassandra.io.sstable.SSTableMetadataTest):
 Key: CASSANDRA-9490
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9490
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
 Environment: Red Hat enterprise Linux ; Arch : PPC64le
Reporter: Pallavi Bhardwaj
 Fix For: 2.1.5


While executing the unit test cases, I observed the following failure,

[junit] Testcase: 
testWithDeletes(org.apache.cassandra.io.sstable.SSTableMetadataTest): FAILED
[junit] expected:-2.038078123E9 but was:1.432716678E9
[junit] junit.framework.AssertionFailedError: expected:-2.038078123E9 but 
was:1.432716678E9
[junit] at 
org.apache.cassandra.io.sstable.SSTableMetadataTest.testWithDeletes(SSTableMetadataTest.java:156)
[junit]




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-6977) attempting to create 10K column families fails with 100 node cluster

2015-05-27 Thread ZhaoYang (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560749#comment-14560749
]

ZhaoYang edited comment on CASSANDRA-6977 at 5/27/15 11:39 AM:
---

[~jkrupan] hi, In fact, what is the difference between 100 KS with 100 tables
each and 1 KS with 1 tables in a cluster? as far as I can tell, at least
the it's almost impossible to create 10K tables in one keyspace. What about
performance? Thanks.

was (Author: jasonstack):
[~jkrupan] hi, In factor, what is the difference between 100 KS with 100 tables
and 1 KS with 1 tables in a cluster? as far as I can tell, at least the
it's almost impossible to create 10K tables in one keyspace. What about
performance? Thanks.

attempting to create 10K column families fails with 100 node cluster

Key: CASSANDRA-6977
URL: https://issues.apache.org/jira/browse/CASSANDRA-6977
Project: Cassandra
Issue Type: Bug
Environment: 100 nodes, Ubuntu 12.04.3 LTS, AWS m1.large instances
Reporter: Daniel Meyer
Assignee: Rocco Varela
Priority: Minor
Fix For: 2.1.1

Attachments: 100_nodes_all_data.png, all_data_5_nodes.png,
keyspace_create.py, logs.tar, tpstats.txt, visualvm_tracer_data.csv

During this test we are attempting to create a total of 1K keyspaces with 10
column families each to bring the total column families to 10K. With a 5
node cluster this operation can be completed; however, it fails with 100
nodes. Please see the two charts. For the 5 node case the time required to
create each keyspace and subsequent 10 column families increases linearly
until the number of keyspaces is 1K. For a 100 node cluster there is a
sudden increase in latency between 450 keyspaces and 550 keyspaces. The test
ends when the test script times out. After the test script times out it is
impossible to reconnect to the cluster with the datastax python driver
because it cannot connect to the host:
cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers',
{'10.199.5.98': OperationTimedOut()}
It was found that running the following stress command does work from the
same machine the test script runs on.
cassandra-stress -d 10.199.5.98 -l 2 -e QUORUM -L3 -b -o INSERT
It should be noted that this test was initially done with DSE 4.0 and c*
version 2.0.5.24 and in that case it was not possible to run stress against
the cluster even locally on a node due to not finding the host.
Attached are system logs from one of the nodes, charts showing schema
creation latency for 5 and 100 node clusters and virtualvm tracer data for
cpu, memory, num_threads and gc runs, tpstat output and the test script.
The test script was on an m1.large aws instance outside of the cluster under
test.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8897) Remove FileCacheService, instead pooling the buffers

2015-05-27 Thread Stefania (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560734#comment-14560734
 ] 

Stefania commented on CASSANDRA-8897:
-

Thanks for the comments, as discussed earlier on, it should be clear now.

In the meantime I have been working on the cperf test:

bq. It would be nice to run some tests that show this change's strengths: if 
you disable auto-compaction on cstar (not sure if you can do this), and make 
the memtable size very small, then we can produce a workload with many sstables 
that still fit in memory. If we aim to write say 4G of data, but perhaps with 
many overwrites so each operation has to touch multiple files (so perhaps only 
400M of data written 10 times, or 4G of data with 10+ clustering values, 
written with visits=fixed(10)). If we set the memtable size to = 100Mb, this 
will likely result in files of = 30Mb, giving us 120+ sstables. If we 
constrain the size of the file cache to just 64Mb, say, we should on a random 
read workload see much more GC activity under the old scheme, and likely 
reduced throughput.

I can disable compaction in the schema and setup the memtable size parameters 
in cassandra.yaml so as to obtain sstables of approx 30MB. However I am having 
problems ensuring the overwrite part. Is it enough to run stress 10 times 
writing 400MB each time or do I need to do anything specific like using visits: 
fixed(10), which I cannot get to work, or using 10 clustering values (which 
generate batches too big and I cannot work out how to disable batching using 
stress). 

Here is the profile:

{code}
keyspace: perftesting

keyspace_definition: |

  CREATE KEYSPACE perftesting WITH replication = { 'class': 'SimpleStrategy', 
'replication_factor':1};

table: mytest

table_definition:

  CREATE TABLE mytest (
id text,
sub_id uuid,
data text,
PRIMARY KEY(id, sub_id)
  ) WITH compaction = {'class':'SizeTieredCompactionStrategy', 'enabled':false 
};

columnspec:
  - name: id
size: gaussian(1..50)
population: uniform(1..5000)
  - name: sub_id
cluster: fixed(5000)
  - name: data
size: fixed(1000)

insert:
  partitions: fixed(1)
  select: fixed(1)/5000
  batchtype: LOGGED

queries:
  read:
cql: select * from mytest where id = ? LIMIT 10
fields: samerow
{code}

Here is the command:

{code}
./tools/bin/cassandra-stress user profile=~/Documents/tests/8897.yaml 
ops\(insert=1\) n=30 -rate threads=50
{code}

Here are the warnings:

{code}
WARN  10:02:18 Batch of prepared statements for [perftesting.mytest] is of size 
5140, exceeding specified threshold of 5120 by 20.
{code}

And this is the problem when adding {{visits: fixed(10)}} to the profile insert 
section, plus I am not sure it is picked up even if I comment out the line 
generating the Exception:

{code}
Exception in thread main java.lang.IllegalArgumentException: Unrecognised 
insert option(s): {visits=fixed(10)}
at 
org.apache.cassandra.stress.StressProfile.getInsert(StressProfile.java:357)
at 
org.apache.cassandra.stress.settings.SettingsCommandUser$1.get(SettingsCommandUser.java:82)
at 
org.apache.cassandra.stress.settings.SettingsCommandUser$1.get(SettingsCommandUser.java:78)
at 
org.apache.cassandra.stress.operations.SampledOpDistributionFactory$1.get(SampledOpDistributionFactory.java:80)
at 
org.apache.cassandra.stress.StressAction$Consumer.init(StressAction.java:269)
at org.apache.cassandra.stress.StressAction.run(StressAction.java:204)
at 
org.apache.cassandra.stress.StressAction.warmup(StressAction.java:105)
at org.apache.cassandra.stress.StressAction.run(StressAction.java:61)
at org.apache.cassandra.stress.Stress.main(Stress.java:109)
{code}

Thanks! :)

 Remove FileCacheService, instead pooling the buffers
 

 Key: CASSANDRA-8897
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8897
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Stefania
 Fix For: 3.x

 Attachments: 9240_test_results.txt, 
 snapshot-1431582436640-cpu-backtraces.png, 
 snapshot-1431582436640-cpu-calltree-compression-8897.nps, 
 snapshot-1431582436640-cpu-calltree-compression-trunk.nps


 After CASSANDRA-8893, a RAR will be a very lightweight object and will not 
 need caching, so we can eliminate this cache entirely. Instead we should have 
 a pool of buffers that are page-aligned.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8897) Remove FileCacheService, instead pooling the buffers

2015-05-27 Thread Benedict (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560751#comment-14560751
]

Benedict commented on CASSANDRA-8897:
-

You can just overwrite it ten times, but it results in a bit of a different
profile. With a schema workload it should still require touching ten files, due
to implementation quirks, so this should be fine. There's also no absolute
requirement we have ten readers involved in a given query, either, it just
seemed like a nice addition to the test.

However the trick you're missing for the visits is to provide it under the
insert options group. i.e. {{ cassandra-stress -insert visits=fixed(10)}} . Its
on the roadmap to clarify all of this in the help, and most likely to deprecate
the old options, permitting us to amalgamate a lot of the options under the new
banner.

Remove FileCacheService, instead pooling the buffers

Key: CASSANDRA-8897
URL: https://issues.apache.org/jira/browse/CASSANDRA-8897
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Benedict
Assignee: Stefania
Fix For: 3.x

Attachments: 9240_test_results.txt,
snapshot-1431582436640-cpu-backtraces.png,
snapshot-1431582436640-cpu-calltree-compression-8897.nps,
snapshot-1431582436640-cpu-calltree-compression-trunk.nps

After CASSANDRA-8893, a RAR will be a very lightweight object and will not
need caching, so we can eliminate this cache entirely. Instead we should have
a pool of buffers that are page-aligned.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6977) attempting to create 10K column families fails with 100 node cluster

2015-05-27 Thread ZhaoYang (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560749#comment-14560749
]

ZhaoYang commented on CASSANDRA-6977:
-

[~jkrupan] hi, In factor, what is the difference between 100 KS with 100 tables
and 1 KS with 1 tables in a cluster? as far as I can tell, at least the
it's almost impossible to create 10K tables in one keyspace. What about
performance? Thanks.

attempting to create 10K column families fails with 100 node cluster

Attachments: 100_nodes_all_data.png, all_data_5_nodes.png,
keyspace_create.py, logs.tar, tpstats.txt, visualvm_tracer_data.csv

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-9479) Improve trace messages

2015-05-27 Thread Robert Stupp (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp reassigned CASSANDRA-9479:
---

Assignee: Robert Stupp

 Improve trace messages
 --

 Key: CASSANDRA-9479
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9479
 Project: Cassandra
  Issue Type: Improvement
Reporter: Robert Stupp
Assignee: Robert Stupp
Priority: Minor

 Currently, tracing only records lines like
 {{Enqueuing response to}} / {{Processing response from}} or
 {{Sending message to}} / {{Message received from}}.
 It would help if these messages also contain some information about the verb 
 and (if easily accessible) about kind of content.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9489) Testcase failure: testExpiredTombstones(org.apache.cassandra.cql3.SliceQueryFilterWithTombstonesTest)

2015-05-27 Thread Pallavi Bhardwaj (JIRA)

Pallavi Bhardwaj created CASSANDRA-9489:
---

 Summary: Testcase failure: 
testExpiredTombstones(org.apache.cassandra.cql3.SliceQueryFilterWithTombstonesTest)
 Key: CASSANDRA-9489
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9489
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
 Environment: Red Hat Enterprise Linux, Arch : PPC64le
Reporter: Pallavi Bhardwaj
 Fix For: 2.1.5


While executing the test cases, I observed the following failure,

[junit] ERROR 08:42:40 Scanned over 100 tombstones in 
cql_test_keyspace.table_4; query aborted (see tombstone_failure_threshold)
[junit] ERROR 08:42:40 Scanned over 100 tombstones in 
cql_test_keyspace.table_4; query aborted (see tombstone_failure_threshold)
[junit] -  ---
[junit] Testcase: 
testExpiredTombstones(org.apache.cassandra.cql3.SliceQueryFilterWithTombstonesTest):
  FAILED
[junit] null
[junit] junit.framework.AssertionFailedError: null
[junit] at 
org.apache.cassandra.cql3.SliceQueryFilterWithTombstonesTest.testExpiredTombstones(SliceQueryFilterWithTombstonesTest.java:155)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9491) Inefficient sequential repairs against vnode clusters

2015-05-27 Thread Jonathan Ellis (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-9491:
--
Assignee: Yuki Morishita

 Inefficient sequential repairs against vnode clusters
 -

 Key: CASSANDRA-9491
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9491
 Project: Cassandra
  Issue Type: Improvement
Reporter: Robert Stupp
Assignee: Yuki Morishita
Priority: Minor

 I've got a cluster with vnodes enabled. People regularly run sequential 
 repairs against that cluster.
 During such a sequential repair (just {{nodetool -pr}}, statistics show:
 * huge increase of live-sstable-count (approx doubling the amount),
 * huge amount of memtable-switches (approx 1200 per node per minute),
 * huge number of flushed (approx 25 per node per minute)
 * memtable-data-size drops to (nearly) 0
 * huge amount of compaction-completed-tasks (60k per minute) and 
 compacted-bytes (25GB per minute)
 These numbers do not match the real, tiny workload that the cluster really 
 has.
 The reason for these (IMO crazy) numbers is the way how sequential repairs 
 work on vnode clusters:
 Starting at {{StorageService.forceRepairAsync}} (from {{nodetool -pr}}, a 
 repair on the ranges from {{getLocalPrimaryRanges(keyspace)}} is initiated. 
 I'll express the schema in pseudo-code:
 {code}
 ranges = getLocalPrimaryRanges(keyspace)
 foreach range in ranges:
 {
   foreach columnFamily
   {
   start async RepairJob
   {
   if sequentialRepair:
   start SnapshotTask against each endpoint 
 (including self)
   send tree requests if snapshot successful
   else // if parallel repair
   send tree requests
   }
   }
 }
 {code}
 This means, that for each sequential repair, a snapshot (including all its 
 implications like flushes, tiny sstables, followup-compactions) is taken for 
 every range. That means 256 snapshots per column-family per repair on each 
 (involved) endpoint. For about 20 tables, this could mean 5120 snapshots 
 within a very short period of time. You do not realize that amount on the 
 file system, since the _tag_ for the snapshot is always the same - so all 
 snapshots end in the same directory.
 IMO it would be sufficient to snapshot only once per column-family. Or do I 
 miss something?
 So basically changing the pseudo-code to:
 {code}
 ranges = getLocalPrimaryRanges(keyspace)
 foreach range in ranges:
 {
   foreach columnFamily
   {
   if sequentialRepair:
   start SnapshotTask against each endpoint (including 
 self)
   start async RepairJob
   {
   send tree requests (if snapshot successful)
   }
   }
 }
 {code}
 NB: The code's similar in all versions (checked 2.0.11, 2.0.15, 2.1, 2.2, 
 trunk)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9490) testcase failure : testWithDeletes(org.apache.cassandra.io.sstable.SSTableMetadataTest):

2015-05-27 Thread Philip Thompson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561027#comment-14561027
 ] 

Philip Thompson commented on CASSANDRA-9490:


This test is currently passing on 2.1-head. Where are you experiencing this 
issue?

 testcase failure : 
 testWithDeletes(org.apache.cassandra.io.sstable.SSTableMetadataTest):
 

 Key: CASSANDRA-9490
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9490
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
 Environment: Red Hat enterprise Linux ; Arch : PPC64le
Reporter: Pallavi Bhardwaj
 Fix For: 2.1.5


 While executing the unit test cases, I observed the following failure,
 [junit] Testcase: 
 testWithDeletes(org.apache.cassandra.io.sstable.SSTableMetadataTest): 
 FAILED
 [junit] expected:-2.038078123E9 but was:1.432716678E9
 [junit] junit.framework.AssertionFailedError: expected:-2.038078123E9 
 but was:1.432716678E9
 [junit] at 
 org.apache.cassandra.io.sstable.SSTableMetadataTest.testWithDeletes(SSTableMetadataTest.java:156)
 [junit]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9491) Inefficient sequential repairs against vnode clusters

2015-05-27 Thread Robert Stupp (JIRA)

Robert Stupp created CASSANDRA-9491:
---

 Summary: Inefficient sequential repairs against vnode clusters
 Key: CASSANDRA-9491
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9491
 Project: Cassandra
  Issue Type: Improvement
Reporter: Robert Stupp
Priority: Minor


I've got a cluster with vnodes enabled. People regularly run sequential repairs 
against that cluster.
During such a sequential repair (just {{nodetool -pr}}, statistics show:
* huge increase of live-sstable-count (approx doubling the amount),
* huge amount of memtable-switches (approx 1200 per node per minute),
* huge number of flushed (approx 25 per node per minute)
* memtable-data-size drops to (nearly) 0
* huge amount of compaction-completed-tasks (60k per minute) and 
compacted-bytes (25GB per minute)

These numbers do not match the real, tiny workload that the cluster really has.

The reason for these (IMO crazy) numbers is the way how sequential repairs work 
on vnode clusters:
Starting at {{StorageService.forceRepairAsync}} (from {{nodetool -pr}}, a 
repair on the ranges from {{getLocalPrimaryRanges(keyspace)}} is initiated. 
I'll express the schema in pseudo-code:

{code}
ranges = getLocalPrimaryRanges(keyspace)
foreach range in ranges:
{
foreach columnFamily
{
start async RepairJob
{
if sequentialRepair:
start SnapshotTask against each endpoint 
(including self)
send tree requests if snapshot successful
else // if parallel repair
send tree requests
}
}
}
{code}

This means, that for each sequential repair, a snapshot (including all its 
implications like flushes, tiny sstables, followup-compactions) is taken for 
every range. That means 256 snapshots per column-family per repair on each 
(involved) endpoint. For about 20 tables, this could mean 5120 snapshots within 
a very short period of time. You do not realize that amount on the file system, 
since the _tag_ for the snapshot is always the same - so all snapshots end in 
the same directory.

IMO it would be sufficient to snapshot only once per column-family. Or do I 
miss something?

So basically changing the pseudo-code to:

{code}
ranges = getLocalPrimaryRanges(keyspace)
foreach range in ranges:
{
foreach columnFamily
{
if sequentialRepair:
start SnapshotTask against each endpoint (including 
self)

start async RepairJob
{
send tree requests (if snapshot successful)
}
}
}
{code}


NB: The code's similar in all versions (checked 2.0.11, 2.0.15, 2.1, 2.2, trunk)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9489) Testcase failure: testExpiredTombstones(org.apache.cassandra.cql3.SliceQueryFilterWithTombstonesTest)

2015-05-27 Thread Philip Thompson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561029#comment-14561029
 ] 

Philip Thompson commented on CASSANDRA-9489:


This test is currently passing on 2.1-head. Where are you experiencing this 
issue?

 Testcase failure: 
 testExpiredTombstones(org.apache.cassandra.cql3.SliceQueryFilterWithTombstonesTest)
 -

 Key: CASSANDRA-9489
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9489
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
 Environment: Red Hat Enterprise Linux, Arch : PPC64le
Reporter: Pallavi Bhardwaj
 Fix For: 2.1.5


 While executing the test cases, I observed the following failure,
 [junit] ERROR 08:42:40 Scanned over 100 tombstones in 
 cql_test_keyspace.table_4; query aborted (see tombstone_failure_threshold)
 [junit] ERROR 08:42:40 Scanned over 100 tombstones in 
 cql_test_keyspace.table_4; query aborted (see tombstone_failure_threshold)
 [junit] -  ---
 [junit] Testcase: 
 testExpiredTombstones(org.apache.cassandra.cql3.SliceQueryFilterWithTombstonesTest):
   FAILED
 [junit] null
 [junit] junit.framework.AssertionFailedError: null
 [junit] at 
 org.apache.cassandra.cql3.SliceQueryFilterWithTombstonesTest.testExpiredTombstones(SliceQueryFilterWithTombstonesTest.java:155)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8502) Static columns returning null for pages after first

2015-05-27 Thread Sylvain Lebresne (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560996#comment-14560996
 ] 

Sylvain Lebresne commented on CASSANDRA-8502:
-

Sorry for the delay, +1 on v2.

 Static columns returning null for pages after first
 ---

 Key: CASSANDRA-8502
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8502
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Flavien Charlon
Assignee: Tyler Hobbs
 Fix For: 2.1.x, 2.0.x

 Attachments: 8502-2.0-v2.txt, 8502-2.0.txt, 8502-2.1-v2.txt, 
 null-static-column.txt


 When paging is used for a query containing a static column, the first page 
 contains the right value for the static column, but subsequent pages have 
 null null for the static column instead of the expected value.
 Repro steps:
 - Create a table with a static column
 - Create a partition with 500 cells
 - Using cqlsh, query that partition
 Actual result:
 - You will see that first, the static column appears as expected, but if you 
 press a key after ---MORE---, the static columns will appear as null.
 See the attached file for a repro of the output.
 I am using a single node cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9458) Race condition causing StreamSession to get stuck in WAIT_COMPLETE

2015-05-27 Thread Omid Aladini (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560844#comment-14560844
]

Omid Aladini commented on CASSANDRA-9458:
-

Thanks for checking the log and the patch. You're right as all the relevant
calls to maybeCompleted are synchronised on the object.

{quote}
Do you have secondary indexes? Right now, streaming is considered completed
after secondary indexes are built in that finalise phase(CASSANDRA-9308).
{quote}

There are secondary indexes and I see a bunch of submitting index build of in
the full log so I guess it's possible that the index build is just taking
longer than the timeout. I'll disable the timeout (and enable TCP keep-alive
via CASSANDRA-9455) to see if it gets resolved.

Race condition causing StreamSession to get stuck in WAIT_COMPLETE
--

Key: CASSANDRA-9458
URL: https://issues.apache.org/jira/browse/CASSANDRA-9458
Project: Cassandra
Issue Type: Bug
Reporter: Omid Aladini
Assignee: Omid Aladini
Priority: Critical
Fix For: 2.1.x, 2.0.x

Attachments: 9458-v1.txt

I think there is a race condition in StreamSession where one side of the
stream could get stuck in WAIT_COMPLETE although both have sent COMPLETE
messages. Consider a scenario that node B is being bootstrapped and it only
receives files during the session:
1- During a stream session A sends some files to B and B sends no files to A.
2- Once B completes the last task (receiving), StreamSession::maybeComplete
is invoked.
3- While B is sending the COMPLETE message via StreamSession::maybeComplete,
it also receives the COMPLETE message from A and therefore
StreamSession::complete() is invoked.
4- Therefore both maybeComplete() and complete() functions have branched into
the state != State.WAIT_COMPLETE case and both set the state to WAIT_COMPLETE.
5- Now B is waiting to receive COMPLETE although it's already received it and
nothing triggers checking the state again, until it times out after
streaming_socket_timeout_in_ms.
In the log below:
https://gist.github.com/omidaladini/003de259958ad8dfb07e
although the node has received COMPLETE, SocketTimeoutException is thrown
after streaming_socket_timeout_in_ms (30 minutes here).

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9431) Static Analysis to warn on unsafe use of Autocloseable instances

2015-05-27 Thread T Jake Luciani (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560928#comment-14560928
 ] 

T Jake Luciani commented on CASSANDRA-9431:
---

Hmm, the problem is when I do that the tests fail in all sorts of glorious 
ways. see 
http://cassci.datastax.com/view/Dev/view/tjake/job/tjake-fix-leaks-testall/5/

 Static Analysis to warn on unsafe use of Autocloseable instances
 

 Key: CASSANDRA-9431
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9431
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: T Jake Luciani
 Fix For: 2.2.0 rc1


 The ideal goal would be to fail the build under any unsafe (and not annotated 
 as considered safe independently) uses of Autocloseable. It looks as though 
 eclipse (and hence, hopefully ecj) has support for this feature, so we should 
 investigate if it meets our requirements and we can get it integrated



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Fix error executing bound statement after adding a collection

2015-05-27 Thread slebresne

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 3e4ed9666 - 63165a719


Fix error executing bound statement after adding a collection

patch by blerer; reviewed by slebresne for CASSANDRA-9411


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/63165a71
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/63165a71
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/63165a71

Branch: refs/heads/cassandra-2.0
Commit: 63165a719cd8ec9d7f06c186f61d39403e192edc
Parents: 3e4ed96
Author: Benjamin Lerer benjamin.le...@datastax.com
Authored: Wed May 27 16:00:52 2015 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed May 27 16:00:52 2015 +0200

--
 CHANGES.txt |   1 +
 .../cql3/statements/ModificationStatement.java  |   6 +-
 .../cql3/statements/SelectStatement.java| 186 +++
 3 files changed, 110 insertions(+), 83 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/63165a71/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index af08802..709100b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.16:
+ * Fix failing bound statement after adding a collection (CASSANDRA-9411)
  * Fix counting cache serialization in request metrics (CASSANDRA-9466)
  * (cqlsh) Add LOGIN command to switch users (CASSANDRA-7212)
  * Clone SliceQueryFilter in AbstractReadCommand implementations 
(CASSANDRA-8940)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/63165a71/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
--
diff --git 
a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
index db22e7d..3852920 100644
--- a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
@@ -668,7 +668,8 @@ public abstract class ModificationStatement implements 
CQLStatement, MeasurableF
 private static ResultSet buildCasFailureResultSet(ByteBuffer key, 
ColumnFamily cf, IterableColumnIdentifier columnsWithConditions, boolean 
isBatch)
 throws InvalidRequestException
 {
-CFDefinition cfDef = cf.metadata().getCfDef();
+CFMetaData cfm = cf.metadata();
+CFDefinition cfDef = cfm.getCfDef();
 
 Selection selection;
 if (columnsWithConditions == null)
@@ -694,7 +695,8 @@ public abstract class ModificationStatement implements 
CQLStatement, MeasurableF
 
 long now = System.currentTimeMillis();
 Selection.ResultSetBuilder builder = selection.resultSetBuilder(now);
-SelectStatement.forSelection(cfDef, 
selection).processColumnFamily(key, cf, Collections.ByteBufferemptyList(), 
now, builder);
+SelectStatement.forSelection(cfm, selection)
+   .processColumnFamily(cfDef, key, cf, 
Collections.ByteBufferemptyList(), now, builder);
 
 return builder.build();
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/63165a71/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 8a4deb6..95e0441 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -46,6 +46,7 @@ import org.apache.cassandra.service.StorageProxy;
 import org.apache.cassandra.service.StorageService;
 import org.apache.cassandra.service.pager.*;
 import org.apache.cassandra.db.ConsistencyLevel;
+import org.apache.cassandra.thrift.ColumnDef;
 import org.apache.cassandra.thrift.IndexExpression;
 import org.apache.cassandra.thrift.IndexOperator;
 import org.apache.cassandra.thrift.ThriftValidation;
@@ -68,7 +69,7 @@ public class SelectStatement implements CQLStatement, 
MeasurableForPreparedCache
 private static final int DEFAULT_COUNT_PAGE_SIZE = 1;
 
 private final int boundTerms;
-public final CFDefinition cfDef;
+public final CFMetaData cfm;
 public final Parameters parameters;
 private final Selection selection;
 private final Term limit;
@@ -110,13 +111,13 @@ public class SelectStatement implements CQLStatement, 
MeasurableForPreparedCache
 }
 };
 
-public SelectStatement(CFDefinition cfDef, int boundTerms, Parameters 
parameters, Selection selection, Term limit)
+public

[jira] [Commented] (CASSANDRA-9482) SSTable leak after stress and repair

2015-05-27 Thread Philip Thompson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560998#comment-14560998
 ] 

Philip Thompson commented on CASSANDRA-9482:


Most likely bisecting will only point you to when leak detection was 
introduced, and not when the leak was.

 SSTable leak after stress and repair
 

 Key: CASSANDRA-9482
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9482
 Project: Cassandra
  Issue Type: Bug
Reporter: Jim Witschey

 I have a dtest that fails intermittently because of SSTable leaks. The test 
 logic leading to the error is:
 - create a 5-node cluster
 - insert 5000 records with {{stress}}, RF=3 at CL=ONE
 - run {{flush}} on all nodes 
 - run {{repair}} on a single node.
 The leak is detected on a different node than {{repair}} was run on.
 The failing test is 
 [here|https://github.com/mambocab/cassandra-dtest/blob/CASSANDRA-5839-squash/repair_test.py#L317].
  The relevant error his 
 [here|https://gist.github.com/mambocab/8aab7b03496e0b279bd3#file-node2-log-L256],
  along with the errors from the entire 5-node cluster. In these logs, the 
 {{repair}} was run on {{node1}} and the leak was found on {{node2}}.
 I can bisect, but I thought I'd get the ball rolling in case someone knows 
 where to look.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9482) SSTable leak after stress and repair

2015-05-27 Thread Philip Thompson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560999#comment-14560999
 ] 

Philip Thompson commented on CASSANDRA-9482:


/cc [~krummas]

 SSTable leak after stress and repair
 

 Key: CASSANDRA-9482
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9482
 Project: Cassandra
  Issue Type: Bug
Reporter: Jim Witschey

 I have a dtest that fails intermittently because of SSTable leaks. The test 
 logic leading to the error is:
 - create a 5-node cluster
 - insert 5000 records with {{stress}}, RF=3 at CL=ONE
 - run {{flush}} on all nodes 
 - run {{repair}} on a single node.
 The leak is detected on a different node than {{repair}} was run on.
 The failing test is 
 [here|https://github.com/mambocab/cassandra-dtest/blob/CASSANDRA-5839-squash/repair_test.py#L317].
  The relevant error his 
 [here|https://gist.github.com/mambocab/8aab7b03496e0b279bd3#file-node2-log-L256],
  along with the errors from the entire 5-node cluster. In these logs, the 
 {{repair}} was run on {{node1}} and the leak was found on {{node2}}.
 I can bisect, but I thought I'd get the ball rolling in case someone knows 
 where to look.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9494) Need to set TTL with COPY command

2015-05-27 Thread Ed Chen (JIRA)

Ed Chen created CASSANDRA-9494:
--

 Summary: Need to set TTL with COPY command
 Key: CASSANDRA-9494
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9494
 Project: Cassandra
  Issue Type: Improvement
  Components: API
Reporter: Ed Chen


I can import a chunk of data into Cassandra table with COPY command like:

COPY my_table (name, address) FROM my_file.csv WITH option='value' ... ;

But I am not able to specify a finite TTL in COPY command with USING TTL 
3600, for example. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7486) Migrate to G1GC by default

2015-05-27 Thread Albert P Tobey (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561432#comment-14561432
 ] 

Albert P Tobey commented on CASSANDRA-7486:
---

Updated patches with spelling and whitespace fixes:

https://github.com/tobert/cassandra/commits/g1gc-2

https://github.com/tobert/cassandra/commit/419d39814985a6ef165fdbafee5f1b84bf2f197b
https://github.com/tobert/cassandra/commit/89d40af978eaeb02185726a63257d979111ad317
https://github.com/tobert/cassandra/commit/0f70469985d62aeadc20b41dc9cdc9d72a035c64


 Migrate to G1GC by default
 --

 Key: CASSANDRA-7486
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
 Project: Cassandra
  Issue Type: New Feature
  Components: Config
Reporter: Jonathan Ellis
Assignee: Albert P Tobey
 Fix For: 3.0 beta 1


 See 
 http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
  and https://twitter.com/rbranson/status/482113561431265281
 May want to default 2.1 to G1.
 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
 Suspect this will help G1 even more than CMS.  (NB this is off by default but 
 needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9494) Need to set TTL with COPY command

2015-05-27 Thread Jonathan Ellis (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-9494:
--
Issue Type: Sub-task  (was: Improvement)
Parent: CASSANDRA-9303

 Need to set TTL with COPY command
 -

 Key: CASSANDRA-9494
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9494
 Project: Cassandra
  Issue Type: Sub-task
  Components: API
Reporter: Ed Chen

 I can import a chunk of data into Cassandra table with COPY command like:
 COPY my_table (name, address) FROM my_file.csv WITH option='value' ... ;
 But I am not able to specify a finite TTL in COPY command with USING TTL 
 3600, for example. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

1 2 >

1 - 100 of 153 matches

Mail list logo