date:20120111

Git Push Summary

2012-01-11 Thread slebresne

Updated Tags:  refs/tags/1.0.7-tentative [created] 10a8f6778

Git Push Summary

2012-01-11 Thread slebresne

Updated Tags:  refs/tags/1.0.7-tentative [deleted] 10a8f6778

git commit: Fix changelog/news and update version for 1.0.7 release

2012-01-11 Thread slebresne

Updated Branches:
  refs/heads/cassandra-1.0 10a8f6778 - d10da1552


Fix changelog/news and update version for 1.0.7 release


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d10da155
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d10da155
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d10da155

Branch: refs/heads/cassandra-1.0
Commit: d10da15526cef59ee4837930eb1a2c185bab2e7e
Parents: 10a8f67
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Jan 11 09:54:59 2012 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Jan 11 09:54:59 2012 +0100

--
 CHANGES.txt  |6 ++
 NEWS.txt |   15 +++
 build.xml|2 +-
 debian/changelog |6 ++
 4 files changed, 28 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d10da155/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b31f3a0..c0f2c66 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -20,6 +20,12 @@
  * Fix assertion error for CF with gc_grace=0 (CASSANDRA-3579)
  * Shutdown ParallelCompaction reducer executor after use (CASSANDRA-3711)
  * Avoid  0 value for pending tasks in leveled compaction (CASSANDRA-3693)
+ * Support TimeUUID in CassandraStorage (CASSANDRA-3327)
+ * Check schema is ready before continuin boostrapping (CASSANDRA-3629)
+ * Catch overflows during parsing of chunk_length_kb (CASSANDRA-3644)
+ * Improve stream protocol mismatch errors (CASSANDRA-3652)
+ * Avoid multiple thread doing HH to the same target (CASSANDRA-3681)
+ * Add JMX property for rp_timeout_in_ms (CASSANDRA-2940)
 Merged from 0.8:
  * avoid logging (harmless) exception when GC takes  1ms (CASSANDRA-3656)
  * prevent new nodes from thinking down nodes are up forever (CASSANDRA-3626)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d10da155/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 6d95650..2356faa 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -8,6 +8,14 @@ upgrade, just in case you need to roll back to the previous 
version.
 (Cassandra version X + 1 will always be able to read data files created
 by version X, but the inverse is not necessarily the case.)
 
+1.0.7
+=
+
+Upgrading
+-
+- Nothing specific to 1.0.7, please report to instruction for 1.0.6
+
+
 1.0.6
 =
 
@@ -21,6 +29,13 @@ Upgrading
   setting the right value and then run scrub on the column family.
 - Please report to instruction for 1.0.5 if coming from an older version.
 
+Other
+-
+- Adds new setstreamthroughput to nodetool to configure streaming
+  throttling
+- Adds JMX property to get/set rpc_timeout_in_ms at runtime
+- Allow configuring (per-CF) bloom_filter_fp_chance
+
 
 1.0.5
 =

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d10da155/build.xml
--
diff --git a/build.xml b/build.xml
index 0cb7f9e..7169ff0 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 property name=debuglevel value=source,lines,vars/
 
 !-- default version and SCM information (we need the default SCM info as 
people may checkout with git-svn) --
-property name=base.version value=1.0.6/
+property name=base.version value=1.0.7/
 property name=scm.default.path 
value=cassandra/branches/cassandra-1.0.0/
 property name=scm.default.connection 
value=scm:svn:http://svn.apache.org/repos/asf/${scm.default.path}/
 property name=scm.default.developerConnection 
value=scm:svn:https://svn.apache.org/repos/asf/${scm.default.path}/

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d10da155/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 9047f19..70578c8 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (1.0.7) unstable; urgency=low
+
+  * New release
+
+ -- Sylvain Lebresne slebre...@apache.org  Wed, 11 Jan 2012 09:53:43 +0100
+
 cassandra (1.0.6) unstable; urgency=low
 
   * New release

[jira] [Created] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Sylvain Lebresne (Created) (JIRA)

Fix unit tests failure
--

Key: CASSANDRA-3727
URL: https://issues.apache.org/jira/browse/CASSANDRA-3727
Project: Cassandra
Issue Type: Bug
Components: Tests
Affects Versions: 1.0.7
Reporter: Sylvain Lebresne
Priority: Blocker
Fix For: 1.0.7

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit first
prints a message with no failures in it), then hang until JUnit timeout and
fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading to
a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7) mostly
because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010. It's
likely just of problem of the test, but it's new and in the meantime removes
are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Pavel Yaskevich (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13183955#comment-13183955
]

Pavel Yaskevich commented on CASSANDRA-3727:

CliTest and others should be timeouting because of newly added shutdown hook.

Fix unit tests failure
--

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit
first prints a message with no failures in it), then hang until JUnit timeout
and fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading
to a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7)
mostly because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010.
It's likely just of problem of the test, but it's new and in the meantime
removes are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

[jira] [Commented] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Sylvain Lebresne (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13183957#comment-13183957
]

Sylvain Lebresne commented on CASSANDRA-3727:
-

Are you saying that they should be timeouting as in 'having them timeouting is
a feature' or are you just pointing out the likely source of the problem? In
the latter, do you remember the ticket that introduced those (or better, have
a fix for it)?

Fix unit tests failure
--

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit
first prints a message with no failures in it), then hang until JUnit timeout
and fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading
to a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7)
mostly because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010.
It's likely just of problem of the test, but it's new and in the meantime
removes are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

[jira] [Commented] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Pavel Yaskevich (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13183960#comment-13183960
]

Pavel Yaskevich commented on CASSANDRA-3727:

I'm saying that it's likely source of the problem and for CliTest fix would be
pretty straightforward, make CliMain to disconnect properly after tests are
done (I'm going to attach a patch fixing patch in a few minutes to this
ticket), I don't know about other tests tho.

Fix unit tests failure
--

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit
first prints a message with no failures in it), then hang until JUnit timeout
and fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading
to a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7)
mostly because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010.
It's likely just of problem of the test, but it's new and in the meantime
removes are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

[jira] [Updated] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Pavel Yaskevich (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Pavel Yaskevich updated CASSANDRA-3727:
---

Attachment: CASSANDRA-3727-CliTest-timeout-fix.patch

fixed CliTest timeout by correctly closing transport connection which allows
Cassandra shutdown hook to proceed without waiting for RPC connections to close.

Fix unit tests failure
--

Attachments: CASSANDRA-3727-CliTest-timeout-fix.patch

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit
first prints a message with no failures in it), then hang until JUnit timeout
and fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading
to a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7)
mostly because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010.
It's likely just of problem of the test, but it's new and in the meantime
removes are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

[jira] [Commented] (CASSANDRA-3693) strange values of pending tasks with compactionstats (below 0)

2012-01-11 Thread Zenek Kraweznik (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13183983#comment-13183983
 ] 

Zenek Kraweznik commented on CASSANDRA-3693:


ok, thanks. I'll chech it after ugprade to 1.0.7

And second question from my last comment: I wonder why 0 bytes total is 
visible on this node during compaction (nodetool ring is reporting 37.18GB).

 strange values of pending tasks with compactionstats (below 0)
 --

 Key: CASSANDRA-3693
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3693
 Project: Cassandra
  Issue Type: Bug
  Components: Core, Tools
Affects Versions: 1.0.0
 Environment: linux, oracle java 1.6.26
Reporter: Zenek Kraweznik
Assignee: Jonathan Ellis
Priority: Minor
  Labels: compaction
 Fix For: 1.0.7

 Attachments: 3693.txt


 during scrub:
 Every 2.0s: for i in 1 2 3; do nodetool -h 192.168.2.$i compactionstats; done 
   
   Wed Jan  4 13:48:13 2012
 pending tasks: 2147483646
   compaction typekeyspace   column family bytes compacted 
 bytes total  progress
Compaction ArchiveMessages 28034971475 
 7239313912038.73%
 pending tasks: -2147483647
   compaction typekeyspace   column family bytes compacted 
 bytes total  progress
Compaction ArchiveMessages 24575687282 
 7238530506733.95%
 pending tasks: 0

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2478) Custom CQL protocol/transport

2012-01-11 Thread Norman Maurer (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13183988#comment-13183988
 ] 

Norman Maurer commented on CASSANDRA-2478:
--

If you go with netty I may be able to help (I'm one of the netty 
committers/devs) with writting the code for it. Let me know if you are 
interested..

 Custom CQL protocol/transport
 -

 Key: CASSANDRA-2478
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2478
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Eric Evans
Priority: Minor
  Labels: cql

 A custom wire protocol would give us the flexibility to optimize for our 
 specific use-cases, and eliminate a troublesome dependency (I'm referring to 
 Thrift, but none of the others would be significantly better).  Additionally, 
 RPC is bad fit here, and we'd do better to move in the direction of something 
 that natively supports streaming.
 I don't think this is as daunting as it might seem initially.  Utilizing an 
 existing server framework like Netty, combined with some copy-and-paste of 
 bits from other FLOSS projects would probably get us 80% of the way there.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3578) Multithreaded commitlog

2012-01-11 Thread Updated

[
https://issues.apache.org/jira/browse/CASSANDRA-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Piotr Kołaczkowski updated CASSANDRA-3578:
--

Attachment: parallel_commit_log_2.patch

Changes contained in this patch:

1. Serialization and CRC is moved to a separate threadpool, making these
operations concurrent. However, appending serialized buffers to the commit log
is *still serial*, so this patch cannot be viewed as the final fix for the
issue.

2. Semantic of the CommitLog.add method when configured with periodic CLES is
slightly changed - in that case the add method enqueues the request and returns
immediately. It doesn't wait even for the serialization, CRC and copying the RM
into the commit log memory mapped segment. If this behaviour makes some
problems, the old behaviour can be easily brought back, with only a performance
penalty for additional synchronisation.

3. Segment syncing is done in parallel to CLS appending. This works perfectly
at least on Linux.

My observations while performing some limited performance testing while
developing this patch:

1. CRC calculation is the CPU-heaviest operation while saving the RM.

2. Writing to the memory mapped buffer is extremely fast. My Dell Latitude can
easily achieve copying speeds of several GB/s. The serial commit log executor
was not loaded fully, even when everything was running on a RAMDISK and with 4
parallel serializer threads running on all the 4 cores of the CPU.
Parallelizing CL appends might not improve performance by a huge factor,
because probably we hit the memory throughput limit first, not the CPU. But
anyway, I think it still makes sense to parallelize it in order to avoid
temporary serialized buffer creation, which would offload GC.

3. When tested on a RAMDISK, I was able to get some minor performance
improvement by being careful not-waiting unnecesarily on locks e.g. Future
objects. It is very important that for small RMs, queues are long enough. If
CL.add is blocking, the queues are usually short - their size is limited by the
number of active worker threads using CL. And with short queues, the frequency
of thread context switches rises.

4. I propose to limit the capacity of the BlockingQueues not by the number of
RMs, but by the predicted size of RMs. For large RMs, we probably don't want to
enqueue too many of them, not to waste memory or even get out of memory. On the
other hand, for small RMs, longer queues are better for keeping thread context
switches low.

Multithreaded commitlog
---

Key: CASSANDRA-3578
URL: https://issues.apache.org/jira/browse/CASSANDRA-3578
Project: Cassandra
Issue Type: Improvement
Reporter: Jonathan Ellis
Priority: Minor
Attachments: parallel_commit_log_2.patch

Brian Aker pointed out a while ago that allowing multiple threads to modify
the commitlog simultaneously (reserving space for each with a CAS first, the
way we do in the SlabAllocator.Region.allocate) can improve performance,
since you're not bottlenecking on a single thread to do all the copying and
CRC computation.
Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes
doable.
(moved from CASSANDRA-622, which was getting a bit muddled.)

[jira] [Updated] (CASSANDRA-2940) Make rpc_timeout_in_ms into a jmx mbean property

2012-01-11 Thread Jonathan Ellis (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2940:
--

Labels: jmx lhf  (was: lhf)

 Make rpc_timeout_in_ms into a jmx mbean property
 

 Key: CASSANDRA-2940
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2940
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jeremy Hanna
  Labels: jmx, lhf
 Fix For: 1.0.7

 Attachments: trunk-CASSANDRA-2940.txt, trunk-CASSANDRA-2940.txt


 When using the hadoop integration especially, experimenting with 
 rpc_timeout_in_ms is a pain if you have to restart every server in the 
 cluster for it to take effect.  This would be an improvement to make it into 
 a jmx mbean property to set it at runtime.  The yaml file could be updated 
 separately so it would be persistent still.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Pavel Yaskevich (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184094#comment-13184094
]

Pavel Yaskevich commented on CASSANDRA-3727:

I guess we can, but I thought that it's kind of feature that it does wait...

Fix unit tests failure
--

Attachments: CASSANDRA-3727-CliTest-timeout-fix.patch

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit
first prints a message with no failures in it), then hang until JUnit timeout
and fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading
to a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7)
mostly because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010.
It's likely just of problem of the test, but it's new and in the meantime
removes are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

[jira] [Assigned] (CASSANDRA-2940) Make rpc_timeout_in_ms into a jmx mbean property

2012-01-11 Thread Jonathan Ellis (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-2940:
-

Assignee: Ruben Terrazas

 Make rpc_timeout_in_ms into a jmx mbean property
 

 Key: CASSANDRA-2940
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2940
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jeremy Hanna
Assignee: Ruben Terrazas
  Labels: jmx, lhf
 Fix For: 1.0.7

 Attachments: trunk-CASSANDRA-2940.txt, trunk-CASSANDRA-2940.txt


 When using the hadoop integration especially, experimenting with 
 rpc_timeout_in_ms is a pain if you have to restart every server in the 
 cluster for it to take effect.  This would be an improvement to make it into 
 a jmx mbean property to set it at runtime.  The yaml file could be updated 
 separately so it would be persistent still.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Jonathan Ellis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184158#comment-13184158
]

Jonathan Ellis commented on CASSANDRA-3727:
---

Definitely not, otherwise there is no way to shut down if clients stay connected

Fix unit tests failure
--

Attachments: CASSANDRA-3727-CliTest-timeout-fix.patch

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit
first prints a message with no failures in it), then hang until JUnit timeout
and fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading
to a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7)
mostly because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010.
It's likely just of problem of the test, but it's new and in the meantime
removes are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

[jira] [Commented] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Jonathan Ellis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184159#comment-13184159
]

Jonathan Ellis commented on CASSANDRA-3727:
---

(The thrift shutdown was introduced for CASSANDRA-3335)

Fix unit tests failure
--

Attachments: CASSANDRA-3727-CliTest-timeout-fix.patch

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit
first prints a message with no failures in it), then hang until JUnit timeout
and fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading
to a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7)
mostly because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010.
It's likely just of problem of the test, but it's new and in the meantime
removes are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

[jira] [Commented] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Jonathan Ellis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184163#comment-13184163
 ] 

Jonathan Ellis commented on CASSANDRA-3727:
---

bq. Can we fix the thrift shutdown instead to not wait for sockets to be closed 
nicely

I'm not sure why this isn't how it already works.  From 
CustomTThreadPoolServer.WorkerProcess:

{code}
.   while (!stopped_  processor.process(inputProtocol, 
outputProtocol))
{
inputProtocol = 
inputProtocolFactory_.getProtocol(inputTransport);
outputProtocol = 
outputProtocolFactory_.getProtocol(outputTransport);
}
{code}

In other words, as soon as stopped_ is set (by the stop() method), each thread 
should finish the current request but not accept more.

 Fix unit tests failure
 --

 Key: CASSANDRA-3727
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3727
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Affects Versions: 1.0.7
Reporter: Sylvain Lebresne
Priority: Blocker
 Fix For: 1.0.7

 Attachments: CASSANDRA-3727-CliTest-timeout-fix.patch


 On current 1.0 branch (and on my machine: Linux), I have the following unit 
 test failures:
 * CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit 
 first prints a message with no failures in it), then hang until JUnit timeout 
 and fails with a 'Timeout occurred'. In other word, the tests themselves are 
 passing, but something they do prevents the process to exit cleanly leading 
 to a JUnit timeout. I don't want to discard that as not a problem, because if 
 something can make the process not exit cleanly, this can be a pain for 
 restarts (and in particular upgrades) and hence would be basically a 
 regression. I'm marking the ticket as blocker (for the release of 1.0.7) 
 mostly because of this one.
 * SystemTableTest: throws an assertionError. I haven't checked yet, so that 
 could be an easy one to fix.
 * RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another 
 process' (consistently). But I have no other process running on port 7010. 
 It's likely just of problem of the test, but it's new and in the meantime 
 removes are not tested.
 * I also see a bunch of stack trace with errors like:
 {noformat}
 [junit] ERROR 10:01:59,007 Fatal exception in thread 
 Thread[NonPeriodicTasks:1,5,main]
 [junit] java.lang.RuntimeException: java.io.IOException: Unable to create 
 hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to 
 /home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
  (errno 17)
 {noformat}
 (with SSTableReaderTest). This does not make the tests fail, but it is still 
 worth investigating. It may be due to CASSANDRA-3101.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Pavel Yaskevich (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184166#comment-13184166
]

Pavel Yaskevich commented on CASSANDRA-3727:

It's easy to test - run cassandra and in other terminal session connect to it
using CLI and try Ctrl-C Cassandra server without closing CLI.

Fix unit tests failure
--

Attachments: CASSANDRA-3727-CliTest-timeout-fix.patch

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit
first prints a message with no failures in it), then hang until JUnit timeout
and fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading
to a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7)
mostly because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010.
It's likely just of problem of the test, but it's new and in the meantime
removes are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

[jira] [Commented] (CASSANDRA-3715) Throw error when creating indexes with the same name as other CFs

2012-01-11 Thread Yuki Morishita (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184192#comment-13184192
 ] 

Yuki Morishita commented on CASSANDRA-3715:
---

I tried to reproduce this in 1.0.6 and cassandra-1.0 branch, but both display 
message correctly.
Looks like this is fixed in 1.0.6 by the patch to CASSANDRA-3573.

 Throw error when creating indexes with the same name as other CFs
 -

 Key: CASSANDRA-3715
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3715
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0.5
Reporter: Joaquin Casares
Assignee: Yuki Morishita

 0.8.8 throws: InvalidRequestException(why:Duplicate index name path)
 but 1.0.5 displays: null
 when running this:
 {noformat}
 create column family inode
   with column_type = 'Standard'
   and comparator = 
 'DynamicCompositeType(t=org.apache.cassandra.db.marshal.TimeUUIDType,s=org.apache.cassandra.db.marshal.UTF8Type,b=org.apache.cassandra.db.marshal.BytesType)'
   and default_validation_class = 'BytesType'
   and key_validation_class = 'BytesType'
   and rows_cached = 0.0
   and row_cache_save_period = 0
   and row_cache_keys_to_save = 2147483647
   and keys_cached = 100.0
   and key_cache_save_period = 14400
   and read_repair_chance = 1.0
   and gc_grace = 60
   and min_compaction_threshold = 4
   and max_compaction_threshold = 32
   and replicate_on_write = true
   and row_cache_provider = 'ConcurrentLinkedHashCacheProvider'
   and compaction_strategy = 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
   and comment = 'Stores file meta data'
   and column_metadata = [
 {column_name : 'b@70617468',
 validation_class : BytesType,
 index_name : 'path',
 index_type : 0
 },
 {column_name : 'b@73656e74696e656c',
 validation_class : BytesType,
 index_name : 'sentinel',
 index_type : 0
 },
 {column_name : 'b@706172656e745f70617468',
 validation_class : BytesType,
 index_name : 'parent_path',
 index_type : 0
 }];
 create column family inode_archive
   with column_type = 'Standard'
   and comparator = 
 'DynamicCompositeType(t=org.apache.cassandra.db.marshal.TimeUUIDType,s=org.apache.cassandra.db.marshal.UTF8Type,b=org.apache.cassandra.db.marshal.BytesType)'
   and default_validation_class = 'BytesType'
   and key_validation_class = 'BytesType'
   and rows_cached = 0.0
   and row_cache_save_period = 0
   and row_cache_keys_to_save = 2147483647
   and keys_cached = 100.0
   and key_cache_save_period = 14400
   and read_repair_chance = 1.0
   and gc_grace = 60
   and min_compaction_threshold = 4
   and max_compaction_threshold = 32
   and replicate_on_write = true
   and row_cache_provider = 'ConcurrentLinkedHashCacheProvider'
   and compaction_strategy = 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
   and comment = 'Stores file meta data'
   and column_metadata = [
 {column_name : 'b@70617468',
 validation_class : BytesType,
 index_name : 'path',
 index_type : 0
 },
 {column_name : 'b@73656e74696e656c',
 validation_class : BytesType,
 index_name : 'sentinel',
 index_type : 0
 },
 {column_name : 'b@706172656e745f70617468',
 validation_class : BytesType,
 index_name : 'parent_path',
 index_type : 0
 }];
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3712) Can't cleanup after I moved a token.

2012-01-11 Thread Herve Nicol (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184198#comment-13184198
 ] 

Herve Nicol commented on CASSANDRA-3712:


Thanks, that fixed cleanup.

 Can't cleanup after I moved a token.
 

 Key: CASSANDRA-3712
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3712
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0.6
 Environment: java version 1.6.0_26
 Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
 Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
 Ubuntu 10.04.2 LTS 64-Bit
 RAM: 2GB / 1GB free
 Data partition: 80% free on the most used server.
Reporter: Herve Nicol

 Before cleanup failed, I moved one node's token.
 My cluster had 10GB data on 2 nodes. Data repartition was bad, tokens were 
 165[...] and 155[...].
 I moved 155 to 075[...], then adjusted to 076[...]. The moves were correctly 
 processed, with no exception.
 But then, when I wanted to cleanup, it failed and keeps failing, on both 
 nodes.
 Other maintenance procedures like repair, compact or scrub work.
 All the data is in the URLs CF.
 Example session log:
 nodetool cleanup fails:
 $ ./nodetool --host cnode1 cleanup
 Error occured during cleanup
 java.util.concurrent.ExecutionException: java.lang.AssertionError
  at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
  at java.util.concurrent.FutureTask.get(FutureTask.java:83)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:203)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:237)
  at 
 org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:958)
  at 
 org.apache.cassandra.service.StorageService.forceTableCleanup(StorageService.java:1604)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  at java.lang.reflect.Method.invoke(Method.java:597)
  at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
  at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
  at 
 com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
  at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
  at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
  at 
 com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
  at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
  at 
 javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427)
  at 
 javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
  at 
 javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)
  at 
 javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360)
  at 
 javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  at java.lang.reflect.Method.invoke(Method.java:597)
  at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
  at sun.rmi.transport.Transport$1.run(Transport.java:159)
  at java.security.AccessController.doPrivileged(Native Method)
  at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
  at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
  at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
  at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
  at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  at java.lang.Thread.run(Thread.java:662)
 Caused by: java.lang.AssertionError
  at org.apache.cassandra.db.Memtable.put(Memtable.java:136)
  at 
 org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:780)
  at 
 org.apache.cassandra.db.index.keys.KeysIndex.deleteColumn(KeysIndex.java:82)
  at 
 org.apache.cassandra.db.index.SecondaryIndexManager.deleteFromIndexes(SecondaryIndexManager.java:438)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.doCleanupCompaction(CompactionManager.java:754)
  at

[jira] [Resolved] (CASSANDRA-3715) Throw error when creating indexes with the same name as other CFs

2012-01-11 Thread Jonathan Ellis (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-3715.
---

Resolution: Duplicate
  Assignee: (was: Yuki Morishita)

Thanks for looking into that, Yuki.

 Throw error when creating indexes with the same name as other CFs
 -

 Key: CASSANDRA-3715
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3715
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.0.5
Reporter: Joaquin Casares

 0.8.8 throws: InvalidRequestException(why:Duplicate index name path)
 but 1.0.5 displays: null
 when running this:
 {noformat}
 create column family inode
   with column_type = 'Standard'
   and comparator = 
 'DynamicCompositeType(t=org.apache.cassandra.db.marshal.TimeUUIDType,s=org.apache.cassandra.db.marshal.UTF8Type,b=org.apache.cassandra.db.marshal.BytesType)'
   and default_validation_class = 'BytesType'
   and key_validation_class = 'BytesType'
   and rows_cached = 0.0
   and row_cache_save_period = 0
   and row_cache_keys_to_save = 2147483647
   and keys_cached = 100.0
   and key_cache_save_period = 14400
   and read_repair_chance = 1.0
   and gc_grace = 60
   and min_compaction_threshold = 4
   and max_compaction_threshold = 32
   and replicate_on_write = true
   and row_cache_provider = 'ConcurrentLinkedHashCacheProvider'
   and compaction_strategy = 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
   and comment = 'Stores file meta data'
   and column_metadata = [
 {column_name : 'b@70617468',
 validation_class : BytesType,
 index_name : 'path',
 index_type : 0
 },
 {column_name : 'b@73656e74696e656c',
 validation_class : BytesType,
 index_name : 'sentinel',
 index_type : 0
 },
 {column_name : 'b@706172656e745f70617468',
 validation_class : BytesType,
 index_name : 'parent_path',
 index_type : 0
 }];
 create column family inode_archive
   with column_type = 'Standard'
   and comparator = 
 'DynamicCompositeType(t=org.apache.cassandra.db.marshal.TimeUUIDType,s=org.apache.cassandra.db.marshal.UTF8Type,b=org.apache.cassandra.db.marshal.BytesType)'
   and default_validation_class = 'BytesType'
   and key_validation_class = 'BytesType'
   and rows_cached = 0.0
   and row_cache_save_period = 0
   and row_cache_keys_to_save = 2147483647
   and keys_cached = 100.0
   and key_cache_save_period = 14400
   and read_repair_chance = 1.0
   and gc_grace = 60
   and min_compaction_threshold = 4
   and max_compaction_threshold = 32
   and replicate_on_write = true
   and row_cache_provider = 'ConcurrentLinkedHashCacheProvider'
   and compaction_strategy = 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
   and comment = 'Stores file meta data'
   and column_metadata = [
 {column_name : 'b@70617468',
 validation_class : BytesType,
 index_name : 'path',
 index_type : 0
 },
 {column_name : 'b@73656e74696e656c',
 validation_class : BytesType,
 index_name : 'sentinel',
 index_type : 0
 },
 {column_name : 'b@706172656e745f70617468',
 validation_class : BytesType,
 index_name : 'parent_path',
 index_type : 0
 }];
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: Update features section of NEWS.txt

2012-01-11 Thread brandonwilliams

Updated Branches:
  refs/heads/trunk ea4296932 - 5ad45de6d


Update features section of NEWS.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5ad45de6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5ad45de6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5ad45de6

Branch: refs/heads/trunk
Commit: 5ad45de6dbcfd163864676ad880de912b68de0ff
Parents: ea42969
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Jan 11 11:48:04 2012 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Jan 11 11:48:04 2012 -0600

--
 NEWS.txt |   12 
 1 files changed, 12 insertions(+), 0 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5ad45de6/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 868ff73..074f856 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -51,6 +51,18 @@ Features
   a single row have always been *atomic* (either all will be applied,
   or none) thanks to the CommitLog, but until 1.1 they were not *isolated*
   -- a reader may see mixed old and new values while the update happens.
+- Hadoop: a new BulkOutputFormat is included which will directly write
+  SSTables locally and then stream them into the cluster.
+- The bulk loader is not longer a fat client; it can be run from an
+  existing machine in a cluster.
+- A new write survey mode has been added, similar to bootstrap, but the
+  node will not automatically join the cluster.  This is useful for cases
+  such as testing different compaction strategies with live traffic without
+  affecting the cluster.
+- Key and row caches are now global, similar to the global memtable
+  threshold.
+- Off-heap caches no longer require JNA.
+- Streaming is now multithreaded.
 
 
 1.0.6

[jira] [Updated] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Jonathan Ellis (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3727:
--

Attachment: 3727.txt

So, I was over-optimistic in CASSANDRA-3335 when I thought I could get by 
without a MessagingService shutdown method.  The problem is that although my 
changes there do work to prevent accepting new connections, and to stop work on 
existing connections *after the first command post-shutdown*, that's not good 
enough in this case since the client is just sitting on its connection and 
never sends another command.

So, this patch renames MS.waitForCallbacks() back to shutdown(), and refuses to 
add new callbacks after that.  However, the analysis on 3335 that there's no 
good way to deal with an exception here, so we do this instead in ExpiringMap:

{code}
.   public V put(K key, V value, long timeout)
{
if (shutdown)
{
// StorageProxy isn't equipped to deal with I'm nominally alive, 
but I can't send any messages out.
// So we'll just sit on this thread for until the rest of the 
server shutdown completes.
//
// See comments in CustomTThreadPoolServer.serve, CASSANDRA-3335, 
and CASSANDRA-3727.
try
{
Thread.sleep(Long.MAX_VALUE);
}
catch (InterruptedException e)
{
throw new AssertionError(e);
}
}
CacheableObjectV previous = cache.put(key, new 
CacheableObjectV(value, timeout));
return (previous == null) ? null : previous.getValue();
}
{code}

Then, we switch the Thrift executor (and all DTPE instances) to use daemon 
threads, and remove the wait-for-WorkerProcess threads code from 
CustomTThreadPoolServer.serve:

{code}
.   // Thrift's default shutdown waits for the WorkerProcess threads to 
complete.  We do not,
// because doing that allows a client to hold our shutdown hostage by 
simply not sending
// another message after stop is called (since process will block 
indefinitely trying to read
// the next meessage header).
//
// The right fix would be to update thrift to set a socket timeout on 
client connections
// (and tolerate unintentional timeouts until stopped_ is set).  But 
this requires deep
// changes to the code generator, so simply setting these threads to 
daemon (in our custom
// CleaningThreadPool) and ignoring them after shutdown is good enough.
//
// Remember, our goal on shutdown is not necessarily that each client 
request we receive
// gets answered first [to do that, you should redirect clients to a 
different coordinator
// first], but rather (1) to make sure that for each update we ack as 
successful, we generate
// hints for any non-responsive replicas, and (2) to make sure that we 
quickly stop
// accepting client connections so shutdown can continue.  Not waiting 
for the WorkerProcess
// threads here accomplishes (2); MessagingService's shutdown method 
takes care of (1).
//
// See CASSANDRA-3335 and CASSANDRA-3727.
{code}

Finally, this patch also updates Memtable's memorymeter thread to use the newly 
daemonized DTPE for good measure, since there's no reason to ever block 
shutdown for that either.

 Fix unit tests failure
 --

 Key: CASSANDRA-3727
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3727
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Affects Versions: 1.0.7
Reporter: Sylvain Lebresne
Priority: Blocker
 Fix For: 1.0.7

 Attachments: 3727.txt, CASSANDRA-3727-CliTest-timeout-fix.patch


 On current 1.0 branch (and on my machine: Linux), I have the following unit 
 test failures:
 * CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit 
 first prints a message with no failures in it), then hang until JUnit timeout 
 and fails with a 'Timeout occurred'. In other word, the tests themselves are 
 passing, but something they do prevents the process to exit cleanly leading 
 to a JUnit timeout. I don't want to discard that as not a problem, because if 
 something can make the process not exit cleanly, this can be a pain for 
 restarts (and in particular upgrades) and hence would be basically a 
 regression. I'm marking the ticket as blocker (for the release of 1.0.7) 
 mostly because of this one.
 * SystemTableTest: throws an assertionError. I haven't checked yet, so that 
 could be an easy one to fix.
 * RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another 
 process' (consistently). But I have no other process running on port 7010. 
 It's likely just of problem of the test, but it's new and in the meantime

git commit: Also mention 2749

2012-01-11 Thread brandonwilliams

Updated Branches:
  refs/heads/trunk 5ad45de6d - f601dbfdf


Also mention 2749


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f601dbfd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f601dbfd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f601dbfd

Branch: refs/heads/trunk
Commit: f601dbfdf40b2e95622e5a1296e50dd043a4c83a
Parents: 5ad45de
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Jan 11 11:50:03 2012 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Jan 11 11:50:03 2012 -0600

--
 NEWS.txt |2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f601dbfd/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 074f856..493bb6a 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -51,6 +51,8 @@ Features
   a single row have always been *atomic* (either all will be applied,
   or none) thanks to the CommitLog, but until 1.1 they were not *isolated*
   -- a reader may see mixed old and new values while the update happens.
+- Finer-grained control over data directories, allowing a ColumnFamily to
+  be pinned to specfic media.
 - Hadoop: a new BulkOutputFormat is included which will directly write
   SSTables locally and then stream them into the cluster.
 - The bulk loader is not longer a fat client; it can be run from an

[jira] [Updated] (CASSANDRA-3725) Switch stress tool to using micros

2012-01-11 Thread Yuki Morishita (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-3725:
--

Attachment: 3725.txt

I found one place which uses milli sec for insertion.
I tested with the attached fix and verified that row got updated second time.

 Switch stress tool to using micros
 --

 Key: CASSANDRA-3725
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3725
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Cathy Daw
Assignee: Yuki Morishita
Priority: Trivial
 Fix For: 1.1

 Attachments: 3725.txt


 The situation encountered is that after deleting the columns for a row in the 
 cli new workloads don't update that columns for the rows. Brandon mentioned 
 that stress uses millis, cli uses micros and therefore row tombstone wins.
 Test Case: Before Delete
 {code}
 // Run: stress --operation=INSERT --num-keys=1  --columns=2 
 --consistency-level=QUORUM --column-size=1 --threads=1 --replication-factor=1 
 --nodes=localhost
 // Run cassandra-cli
 [default@Keyspace1] list Standard1;   
 Using default limit of 100
 ---
 RowKey: 30
 = (column=C0, value=63, timestamp=1326259090065)
 = (column=C1, value=63, timestamp=1326259090065)
 1 Row Returned.
 Elapsed time: 2 msec(s).
 [default@Keyspace1] del Standard1['30'];  
 row removed.
 [default@Keyspace1] list Standard1; 
 Using default limit of 100
 ---
 RowKey: 30
 {code}
 Test Case: After Delete
 {code}
 // Run: stress --operation=INSERT --num-keys=1  --columns=2 
 --consistency-level=QUORUM --column-size=1 --threads=1 --replication-factor=1 
 --nodes=localhost
 // Run cassandra-cli
 [default@Keyspace1] list Standard1;
 Using default limit of 100
 ---
 RowKey: 30
 1 Row Returned.
 Elapsed time: 1 msec(s).
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: expound on write survey mode

2012-01-11 Thread brandonwilliams

Updated Branches:
  refs/heads/trunk f601dbfdf - 980663a39


expound on write survey mode


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/980663a3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/980663a3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/980663a3

Branch: refs/heads/trunk
Commit: 980663a39267bb98397bb8dc43e335ea67e740dc
Parents: f601dbf
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Jan 11 11:57:06 2012 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Jan 11 11:57:06 2012 -0600

--
 NEWS.txt |8 
 1 files changed, 4 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/980663a3/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 493bb6a..036399b 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -57,10 +57,10 @@ Features
   SSTables locally and then stream them into the cluster.
 - The bulk loader is not longer a fat client; it can be run from an
   existing machine in a cluster.
-- A new write survey mode has been added, similar to bootstrap, but the
-  node will not automatically join the cluster.  This is useful for cases
-  such as testing different compaction strategies with live traffic without
-  affecting the cluster.
+- A new write survey mode has been added, similar to bootstrap (enabled via
+  -Dcassandra.write_survey=true), but the node will not automatically join
+  the cluster.  This is useful for cases such as testing different
+  compaction strategies with live traffic without affecting the cluster.
 - Key and row caches are now global, similar to the global memtable
   threshold.
 - Off-heap caches no longer require JNA.

[jira] [Commented] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Brandon Williams (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184249#comment-13184249
]

Brandon Williams commented on CASSANDRA-3727:
-

Fix unit tests failure
--

Attachments: 3727.txt, CASSANDRA-3727-CliTest-timeout-fix.patch

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit
first prints a message with no failures in it), then hang until JUnit timeout
and fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading
to a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7)
mostly because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010.
It's likely just of problem of the test, but it's new and in the meantime
removes are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

[jira] [Assigned] (CASSANDRA-3726) cqlsh and cassandra-cli show keys differently for data created via stress tool

2012-01-11 Thread paul cannon (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

paul cannon reassigned CASSANDRA-3726:
--

Assignee: paul cannon

 cqlsh and cassandra-cli show keys differently for data created via stress tool
 --

 Key: CASSANDRA-3726
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3726
 Project: Cassandra
  Issue Type: Bug
Reporter: Cathy Daw
Assignee: paul cannon
Priority: Minor

 {code}
 // Run: stress --operation=INSERT --num-keys=5  --columns=2 
 --consistency-level=QUORUM --column-size=1 --threads=1 --replication-factor=1 
 --nodes=localhost
 // cqlsh
 cqlsh:Keyspace1 select * from Standard1;
  KEY,3 | C0,c | C1,c | 
  KEY,0 | 
  KEY,2 | C0,c | C1,c | 
  KEY,1 | C0,c | C1,c | 
  KEY,4 | C0,c | C1,c | 
 cqlsh:Keyspace1 describe columnfamily Standard1;
 CREATE COLUMNFAMILY Standard1 (
   KEY blob PRIMARY KEY
 ) WITH
   comment='' AND
   comparator=ascii AND
   row_cache_provider='ConcurrentLinkedHashCacheProvider' AND
   key_cache_size=20.00 AND
   row_cache_size=0.00 AND
   read_repair_chance=1.00 AND
   gc_grace_seconds=864000 AND
   default_validation=blob AND
   min_compaction_threshold=4 AND
   max_compaction_threshold=32 AND
   row_cache_save_period_in_seconds=0 AND
   key_cache_save_period_in_seconds=14400 AND
   replicate_on_write=True;
 // cassandra-cli
 [default@Keyspace1] list Standard1;
 Using default limit of 100
 ---
 RowKey: 33
 = (column=C0, value=63, timestamp=1326259960705)
 = (column=C1, value=63, timestamp=1326259960705)
 ---
 RowKey: 30
 ---
 RowKey: 32
 = (column=C0, value=63, timestamp=1326259960704)
 = (column=C1, value=63, timestamp=1326259960704)
 ---
 RowKey: 31
 = (column=C0, value=63, timestamp=1326259960704)
 = (column=C1, value=63, timestamp=1326259960704)
 ---
 RowKey: 34
 = (column=C0, value=63, timestamp=1326259960705)
 = (column=C1, value=63, timestamp=1326259960705)
 [default@Keyspace1] describe Standard1;
 ColumnFamily: Standard1
   Key Validation Class: org.apache.cassandra.db.marshal.BytesType
   Default column value validator: 
 org.apache.cassandra.db.marshal.BytesType
   Columns sorted by: org.apache.cassandra.db.marshal.AsciiType
   Row cache size / save period in seconds / keys to save : 0.0/0/all
   Row Cache Provider: 
 org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider
   Key cache size / save period in seconds: 20.0/14400
   GC grace seconds: 864000
   Compaction min/max thresholds: 4/32
   Read repair chance: 1.0
   Replicate on write: true
   Built indexes: []
   Compaction Strategy: 
 org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: prevent slow clients from postponing shutdown indefinitely patch by jbellis; reviewed by brandonwilliams for CASSANDRA-3727

2012-01-11 Thread jbellis

Updated Branches:
  refs/heads/cassandra-1.0 d10da1552 - 185eca5d1


prevent slow clients from postponing shutdown indefinitely
patch by jbellis; reviewed by brandonwilliams for CASSANDRA-3727


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/185eca5d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/185eca5d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/185eca5d

Branch: refs/heads/cassandra-1.0
Commit: 185eca5d1fa7e384bb888c144d06abbced0fd577
Parents: d10da15
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Jan 11 11:44:32 2012 -0600
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Jan 11 12:48:40 2012 -0600

--
 .../concurrent/DebuggableThreadPoolExecutor.java   |2 +-
 .../cassandra/concurrent/NamedThreadFactory.java   |1 +
 src/java/org/apache/cassandra/db/Memtable.java |8 +++-
 .../org/apache/cassandra/net/MessagingService.java |   10 +---
 .../cassandra/service/AbstractCassandraDaemon.java |8 ++-
 .../apache/cassandra/service/StorageService.java   |9 ++--
 .../cassandra/thrift/CustomTThreadPoolServer.java  |   43 +++
 .../org/apache/cassandra/utils/ExpiringMap.java|   17 ++
 .../org/apache/cassandra/service/RemoveTest.java   |2 +-
 9 files changed, 59 insertions(+), 41 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/185eca5d/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
--
diff --git 
a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java 
b/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
index 7a344d8..f111d37 100644
--- a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
+++ b/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
@@ -108,7 +108,7 @@ public class DebuggableThreadPoolExecutor extends 
ThreadPoolExecutor
 protected void onFinalRejection(Runnable task) {}
 
 @Override
-public void afterExecute(Runnable r, Throwable t)
+protected void afterExecute(Runnable r, Throwable t)
 {
 super.afterExecute(r,t);
 logExceptionsAfterExecute(r, t);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/185eca5d/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
--
diff --git a/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java 
b/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
index 4cee8dc..a60a0d5 100644
--- a/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
+++ b/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
@@ -50,6 +50,7 @@ public class NamedThreadFactory implements ThreadFactory
 String name = id + : + n.getAndIncrement();
 Thread thread = new Thread(runnable, name);
 thread.setPriority(priority);
+thread.setDaemon(true);
 return thread;
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/185eca5d/src/java/org/apache/cassandra/db/Memtable.java
--
diff --git a/src/java/org/apache/cassandra/db/Memtable.java 
b/src/java/org/apache/cassandra/db/Memtable.java
index fdafc6d..412b800 100644
--- a/src/java/org/apache/cassandra/db/Memtable.java
+++ b/src/java/org/apache/cassandra/db/Memtable.java
@@ -31,6 +31,7 @@ import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
 import org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor;
+import org.apache.cassandra.concurrent.NamedThreadFactory;
 import org.apache.cassandra.db.columniterator.IColumnIterator;
 import org.apache.cassandra.db.columniterator.SimpleAbstractColumnIterator;
 import org.apache.cassandra.db.commitlog.ReplayPosition;
@@ -57,7 +58,12 @@ public class Memtable
 // we're careful to only allow one count to run at a time because counting 
is slow
 // (can be minutes, for a large memtable and a busy server), so we could 
keep memtables
 // alive after they're flushed and would otherwise be GC'd.
-private static final ExecutorService meterExecutor = new 
ThreadPoolExecutor(1, 1, Integer.MAX_VALUE, TimeUnit.MILLISECONDS, new 
SynchronousQueueRunnable())
+private static final ExecutorService meterExecutor = new 
DebuggableThreadPoolExecutor(1,
+   
   1,
+   
   Integer.MAX_VALUE,
+   
   TimeUnit.MILLISECONDS,
+

[jira] [Commented] (CASSANDRA-3634) compare string vs. binary prepared statement parameters

2012-01-11 Thread Brandon Williams (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184276#comment-13184276
 ] 

Brandon Williams commented on CASSANDRA-3634:
-

bq. It's not nearly so compelling to me. 10% is definitely on the high side of 
making me stand up and take notice, but it's not enormous.

It's more compelling to me when compared in the context of the existing RPC 
performance.  5% is gain okay (PS vs RPC), but 16% (BB vs RPC) is a fairly 
substantial improvement.

I was a little worried about the variance (even though the worst cases are 
pretty close) but I ran some tests with the commit log disable and the 
deviation is on par with the rest, I think it's just fast enough to push it 
that hard.

 compare string vs. binary prepared statement parameters
 ---

 Key: CASSANDRA-3634
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3634
 Project: Cassandra
  Issue Type: Sub-task
  Components: API, Core
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor
  Labels: cql
 Fix For: 1.1


 Perform benchmarks to compare the performance of string and pre-serialized 
 binary parameters to prepared statements.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3722) Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.

2012-01-11 Thread Brandon Williams (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184289#comment-13184289
 ] 

Brandon Williams commented on CASSANDRA-3722:
-

bq. Hi Brandon, Will it make sense for the remote node to avoid traffic for a 
given the start and end?

Sure, but my point is that's going to happen already:

* node X begins a repair, adversely affecting its read latency
* node Y tries to read data from X, and scores it badly due to the extra latency

At this point, node Y will not try to read from X again until either:

* all other members of X's replica set perform *worse* than X
* the RESET_INTERVAL_IN_MS elapses, and the entire process starts over again

Eventually, node X finishes the validation compaction and when the reset 
interval elapses, its reads are more equal to the rest of the replica set and 
everything is back to normal.

Telling the dsnitch when a remote node starts and stop compacting doesn't seem 
like it's going to improve on this a whole lot.


 Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.
 --

 Key: CASSANDRA-3722
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3722
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
Priority: Minor

 Currently Dynamic snitch looks at the latency for figuring out which node 
 will be better serving the requests, this works great but there is a part of 
 the traffic sent to collect this data... There is also a window when Snitch 
 doesn't know about some major event which are going to happen on the node 
 (Node which is going to receive the data request).
 It would be great if we can send some sort hints to the Snitch so they can 
 score based on known events causing higher latencies.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Reopened] (CASSANDRA-3579) AssertionError in hintedhandoff - 1.0.5

2012-01-11 Thread Jonathan Ellis (Reopened) (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reopened CASSANDRA-3579:
---


this is causing the SystemTableTest failure mentioned in CASSANDRA-3727

 AssertionError in hintedhandoff - 1.0.5
 ---

 Key: CASSANDRA-3579
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3579
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: RHEL 6.1 64 bit, 32 GB RAM, 8 GB allocated to JVM, 
 running XFS filesystem for commit/data directories
Reporter: Ramesh Natarajan
Assignee: Sylvain Lebresne
 Fix For: 1.0.7

 Attachments: 3579-v2.txt, 3579.patch


 We are running a 8 node cassandra cluster running cassandra 1.0.5.
 All our CF use leveled compaction.  We ran a test where we did a lot
 of inserts for 3 days. After that we started to run tests where some
 of the reads could ask for information that was inserted a while back.
 In this scenario we are seeing this assertion error in HintedHandoff.
 ERROR [HintedHandoff:3] 2011-12-05 15:42:04,324
 AbstractCassandraDaemon.java (line 133) Fatal exception in thread
 Thread[HintedHandoff:3,1,main]
 java.lang.RuntimeException: java.lang.RuntimeException:
 java.util.concurrent.ExecutionException: java.lang.AssertionError:
 originally calculated column size of 470937164 but now it is 470294247
at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
 Caused by: java.lang.RuntimeException:
 java.util.concurrent.ExecutionException: java.lang.AssertionError:
 originally calculated column size of 470937164 but now it is 470294247
at 
 org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:330)
at 
 org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81)
at 
 org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:353)
at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
 Caused by: java.util.concurrent.ExecutionException:
 java.lang.AssertionError: originally calculated column size of
 470937164 but now it is 470294247
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at 
 org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:326)
... 6 more
 Caused by: java.lang.AssertionError: originally calculated column size
 of 470937164 but now it is 470294247
at 
 org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:124)
at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:160)
at 
 org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:158)
at 
 org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:275)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
 ERROR [HintedHandoff:3] 2011-12-05 15:42:04,333
 AbstractCassandraDaemon.java (line 133) Fatal exception in thread
 Thread[HintedHandoff:3,1,main]
 java.lang.RuntimeException: java.lang.RuntimeException:
 java.util.concurrent.ExecutionException: java.lang.AssertionError:
 originally calculated column size of 470937164 but now it is 470294247
at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
 Caused by: java.lang.RuntimeException:
 java.util.concurrent.ExecutionException: java.lang.AssertionError:
 originally calculated column size of 470937164 but now it is 470294247
at 
 org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:330)
at 
 org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81)
at 
 org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:353)
at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
 Caused by: java.util.concurrent.ExecutionException:
 java.lang.AssertionError: originally calculated column size of
 470937164 but now it is 470294247

[jira] [Updated] (CASSANDRA-3579) AssertionError in hintedhandoff - 1.0.5

2012-01-11 Thread Jonathan Ellis (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3579:
--

Attachment: 3579-fix-text.txt

Switching from = to  means that getColumnFamily will now include tombstones 
for just-deleted column with gcgs == 0.

For clients this is harmless (since we drop tombstones in the coordinator after 
RR), but internal code needs to be more careful.  patch attached to have 
loadTokens expunge all tombstones.

 AssertionError in hintedhandoff - 1.0.5
 ---

 Key: CASSANDRA-3579
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3579
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: RHEL 6.1 64 bit, 32 GB RAM, 8 GB allocated to JVM, 
 running XFS filesystem for commit/data directories
Reporter: Ramesh Natarajan
Assignee: Sylvain Lebresne
 Fix For: 1.0.7

 Attachments: 3579-fix-text.txt, 3579-v2.txt, 3579.patch


 We are running a 8 node cassandra cluster running cassandra 1.0.5.
 All our CF use leveled compaction.  We ran a test where we did a lot
 of inserts for 3 days. After that we started to run tests where some
 of the reads could ask for information that was inserted a while back.
 In this scenario we are seeing this assertion error in HintedHandoff.
 ERROR [HintedHandoff:3] 2011-12-05 15:42:04,324
 AbstractCassandraDaemon.java (line 133) Fatal exception in thread
 Thread[HintedHandoff:3,1,main]
 java.lang.RuntimeException: java.lang.RuntimeException:
 java.util.concurrent.ExecutionException: java.lang.AssertionError:
 originally calculated column size of 470937164 but now it is 470294247
at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
 Caused by: java.lang.RuntimeException:
 java.util.concurrent.ExecutionException: java.lang.AssertionError:
 originally calculated column size of 470937164 but now it is 470294247
at 
 org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:330)
at 
 org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81)
at 
 org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:353)
at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
 Caused by: java.util.concurrent.ExecutionException:
 java.lang.AssertionError: originally calculated column size of
 470937164 but now it is 470294247
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at 
 org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:326)
... 6 more
 Caused by: java.lang.AssertionError: originally calculated column size
 of 470937164 but now it is 470294247
at 
 org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:124)
at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:160)
at 
 org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:158)
at 
 org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:275)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
 ERROR [HintedHandoff:3] 2011-12-05 15:42:04,333
 AbstractCassandraDaemon.java (line 133) Fatal exception in thread
 Thread[HintedHandoff:3,1,main]
 java.lang.RuntimeException: java.lang.RuntimeException:
 java.util.concurrent.ExecutionException: java.lang.AssertionError:
 originally calculated column size of 470937164 but now it is 470294247
at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
 Caused by: java.lang.RuntimeException:
 java.util.concurrent.ExecutionException: java.lang.AssertionError:
 originally calculated column size of 470937164 but now it is 470294247
at 
 org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:330)
at 
 org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81)
at

[jira] [Commented] (CASSANDRA-3579) AssertionError in hintedhandoff - 1.0.5

2012-01-11 Thread Sylvain Lebresne (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184323#comment-13184323
 ] 

Sylvain Lebresne commented on CASSANDRA-3579:
-

+1

 AssertionError in hintedhandoff - 1.0.5
 ---

 Key: CASSANDRA-3579
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3579
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: RHEL 6.1 64 bit, 32 GB RAM, 8 GB allocated to JVM, 
 running XFS filesystem for commit/data directories
Reporter: Ramesh Natarajan
Assignee: Sylvain Lebresne
 Fix For: 1.0.7

 Attachments: 3579-fix-text.txt, 3579-v2.txt, 3579.patch


 We are running a 8 node cassandra cluster running cassandra 1.0.5.
 All our CF use leveled compaction.  We ran a test where we did a lot
 of inserts for 3 days. After that we started to run tests where some
 of the reads could ask for information that was inserted a while back.
 In this scenario we are seeing this assertion error in HintedHandoff.
 ERROR [HintedHandoff:3] 2011-12-05 15:42:04,324
 AbstractCassandraDaemon.java (line 133) Fatal exception in thread
 Thread[HintedHandoff:3,1,main]
 java.lang.RuntimeException: java.lang.RuntimeException:
 java.util.concurrent.ExecutionException: java.lang.AssertionError:
 originally calculated column size of 470937164 but now it is 470294247
at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
 Caused by: java.lang.RuntimeException:
 java.util.concurrent.ExecutionException: java.lang.AssertionError:
 originally calculated column size of 470937164 but now it is 470294247
at 
 org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:330)
at 
 org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81)
at 
 org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:353)
at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
 Caused by: java.util.concurrent.ExecutionException:
 java.lang.AssertionError: originally calculated column size of
 470937164 but now it is 470294247
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at 
 org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:326)
... 6 more
 Caused by: java.lang.AssertionError: originally calculated column size
 of 470937164 but now it is 470294247
at 
 org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:124)
at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:160)
at 
 org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:158)
at 
 org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:275)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
 ERROR [HintedHandoff:3] 2011-12-05 15:42:04,333
 AbstractCassandraDaemon.java (line 133) Fatal exception in thread
 Thread[HintedHandoff:3,1,main]
 java.lang.RuntimeException: java.lang.RuntimeException:
 java.util.concurrent.ExecutionException: java.lang.AssertionError:
 originally calculated column size of 470937164 but now it is 470294247
at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
 Caused by: java.lang.RuntimeException:
 java.util.concurrent.ExecutionException: java.lang.AssertionError:
 originally calculated column size of 470937164 but now it is 470294247
at 
 org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:330)
at 
 org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81)
at 
 org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:353)
at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
 Caused by: java.util.concurrent.ExecutionException:
 java.lang.AssertionError: originally calculated column size of
 470937164 but now it is

[5/5] git commit: Add help for INSERT to cqlsh. Patch by Paul Cannon, reviewed by brandonwilliams for CASSANDRA-3718.

2012-01-11 Thread jbellis

Add help for INSERT to cqlsh.
Patch by Paul Cannon, reviewed by brandonwilliams for CASSANDRA-3718.


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2bb862c3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2bb862c3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2bb862c3

Branch: refs/heads/trunk
Commit: 2bb862c320987bcc696765f4f5d8e3029db44371
Parents: e2231a1
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Jan 10 13:59:50 2012 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Jan 10 13:59:50 2012 -0600

--
 bin/cqlsh |   27 +++
 1 files changed, 27 insertions(+), 0 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2bb862c3/bin/cqlsh
--
diff --git a/bin/cqlsh b/bin/cqlsh
index 72906b4..0f02a0c 100755
--- a/bin/cqlsh
+++ b/bin/cqlsh
@@ -1021,6 +1021,33 @@ class Shell(cmd.Cmd):
 Cassandra documentation.
 
 
+def help_insert(self):
+print 
+INSERT INTO columnFamily
+( keyname, colname1 [, colname2 [, ...]] )
+   VALUES ( keyval, colval1 [, colval2 [, ...]] )
+   [USING CONSISTENCY consistencylevel
+ [AND TIMESTAMP timestamp]
+ [AND TTL timeToLive]];
+
+An INSERT is used to write one or more columns to a record in a
+Cassandra column family. No results are returned.
+
+The first column name in the INSERT list must be the name of the
+column family key. Also, there must be more than one column name
+specified (Cassandra rows are not considered to exist with only
+a key and no associated columns).
+
+Unlike in SQL, the semantics of INSERT and UPDATE are identical.
+In either case a record is created if none existed before, and
+udpated when it does. For more information, see one of the
+following:
+
+  HELP UPDATE
+  HELP UPDATE_USING
+  HELP CONSISTENCYLEVEL
+
+
 def help_update(self):
 print 
 UPDATE columnFamily [USING CONSISTENCY consistencylevel

[3/5] git commit: Fix changelog/news and update version for 1.0.7 release

2012-01-11 Thread jbellis

Fix changelog/news and update version for 1.0.7 release


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d10da155
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d10da155
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d10da155

Branch: refs/heads/trunk
Commit: d10da15526cef59ee4837930eb1a2c185bab2e7e
Parents: 10a8f67
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Jan 11 09:54:59 2012 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Jan 11 09:54:59 2012 +0100

--
 CHANGES.txt  |6 ++
 NEWS.txt |   15 +++
 build.xml|2 +-
 debian/changelog |6 ++
 4 files changed, 28 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d10da155/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b31f3a0..c0f2c66 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -20,6 +20,12 @@
  * Fix assertion error for CF with gc_grace=0 (CASSANDRA-3579)
  * Shutdown ParallelCompaction reducer executor after use (CASSANDRA-3711)
  * Avoid  0 value for pending tasks in leveled compaction (CASSANDRA-3693)
+ * Support TimeUUID in CassandraStorage (CASSANDRA-3327)
+ * Check schema is ready before continuin boostrapping (CASSANDRA-3629)
+ * Catch overflows during parsing of chunk_length_kb (CASSANDRA-3644)
+ * Improve stream protocol mismatch errors (CASSANDRA-3652)
+ * Avoid multiple thread doing HH to the same target (CASSANDRA-3681)
+ * Add JMX property for rp_timeout_in_ms (CASSANDRA-2940)
 Merged from 0.8:
  * avoid logging (harmless) exception when GC takes  1ms (CASSANDRA-3656)
  * prevent new nodes from thinking down nodes are up forever (CASSANDRA-3626)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d10da155/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 6d95650..2356faa 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -8,6 +8,14 @@ upgrade, just in case you need to roll back to the previous 
version.
 (Cassandra version X + 1 will always be able to read data files created
 by version X, but the inverse is not necessarily the case.)
 
+1.0.7
+=
+
+Upgrading
+-
+- Nothing specific to 1.0.7, please report to instruction for 1.0.6
+
+
 1.0.6
 =
 
@@ -21,6 +29,13 @@ Upgrading
   setting the right value and then run scrub on the column family.
 - Please report to instruction for 1.0.5 if coming from an older version.
 
+Other
+-
+- Adds new setstreamthroughput to nodetool to configure streaming
+  throttling
+- Adds JMX property to get/set rpc_timeout_in_ms at runtime
+- Allow configuring (per-CF) bloom_filter_fp_chance
+
 
 1.0.5
 =

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d10da155/build.xml
--
diff --git a/build.xml b/build.xml
index 0cb7f9e..7169ff0 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 property name=debuglevel value=source,lines,vars/
 
 !-- default version and SCM information (we need the default SCM info as 
people may checkout with git-svn) --
-property name=base.version value=1.0.6/
+property name=base.version value=1.0.7/
 property name=scm.default.path 
value=cassandra/branches/cassandra-1.0.0/
 property name=scm.default.connection 
value=scm:svn:http://svn.apache.org/repos/asf/${scm.default.path}/
 property name=scm.default.developerConnection 
value=scm:svn:https://svn.apache.org/repos/asf/${scm.default.path}/

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d10da155/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 9047f19..70578c8 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (1.0.7) unstable; urgency=low
+
+  * New release
+
+ -- Sylvain Lebresne slebre...@apache.org  Wed, 11 Jan 2012 09:53:43 +0100
+
 cassandra (1.0.6) unstable; urgency=low
 
   * New release

[4/5] git commit: Avoid 0 value for pending tasks in leveled compaction

2012-01-11 Thread jbellis

Avoid  0 value for pending tasks in leveled compaction

patch by jbellis; reviewed by slebresne for CASSANDRA-3693


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/10a8f677
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/10a8f677
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/10a8f677

Branch: refs/heads/trunk
Commit: 10a8f67783d9987063424212129be29690628bca
Parents: 2bb862c
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Jan 11 08:52:42 2012 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Jan 11 08:54:44 2012 +0100

--
 CHANGES.txt|1 +
 .../cassandra/db/compaction/LeveledManifest.java   |   20 +-
 2 files changed, 14 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/10a8f677/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 09bb85d..b31f3a0 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -19,6 +19,7 @@
  * Don't ignore IOException during compaction (CASSANDRA-3655)
  * Fix assertion error for CF with gc_grace=0 (CASSANDRA-3579)
  * Shutdown ParallelCompaction reducer executor after use (CASSANDRA-3711)
+ * Avoid  0 value for pending tasks in leveled compaction (CASSANDRA-3693)
 Merged from 0.8:
  * avoid logging (harmless) exception when GC takes  1ms (CASSANDRA-3656)
  * prevent new nodes from thinking down nodes are up forever (CASSANDRA-3626)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/10a8f677/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java 
b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
index f61a26a..40a0a17 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
@@ -27,6 +27,7 @@ import java.io.IOException;
 import java.util.*;
 
 import com.google.common.collect.Iterables;
+import com.google.common.primitives.Ints;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -202,11 +203,14 @@ public class LeveledManifest
 return builder.toString();
 }
 
-private double maxBytesForLevel (int level)
+private long maxBytesForLevel(int level)
 {
-return level == 0
-   ? 4 * maxSSTableSizeInMB * 1024 * 1024
-   : Math.pow(10, level) * maxSSTableSizeInMB * 1024 * 1024;
+if (level == 0)
+return 4 * maxSSTableSizeInMB * 1024 * 1024;
+double bytes = Math.pow(10, level) * maxSSTableSizeInMB * 1024 * 1024;
+if (bytes  Long.MAX_VALUE)
+throw new RuntimeException(At most  + Long.MAX_VALUE +  bytes 
may be in a compaction level; your maxSSTableSize must be absurdly high to 
compute  + bytes);
+return (long) bytes;
 }
 
 public synchronized CollectionSSTableReader getCompactionCandidates()
@@ -424,12 +428,14 @@ public class LeveledManifest
 
 public int getEstimatedTasks()
 {
-int n = 0;
+long tasks = 0;
 for (int i = generations.length - 1; i = 0; i--)
 {
 ListSSTableReader sstables = generations[i];
-n += Math.max(0L, SSTableReader.getTotalBytes(sstables) - 
maxBytesForLevel(i)) / (maxSSTableSizeInMB * 1024 * 1024);
+long n = Math.max(0L, SSTableReader.getTotalBytes(sstables) - 
maxBytesForLevel(i)) / (maxSSTableSizeInMB * 1024 * 1024);
+logger.debug(Estimating  + n +  compaction tasks in level  + 
i);
+tasks += n;
 }
-return n;
+return Ints.checkedCast(tasks);
 }
 }

[1/5] git commit: merge from 1.0

2012-01-11 Thread jbellis

Updated Branches:
  refs/heads/trunk 980663a39 - a4fc7e270


merge from 1.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a4fc7e27
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a4fc7e27
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a4fc7e27

Branch: refs/heads/trunk
Commit: a4fc7e27045113ff680327558eef49f6d0c317d9
Parents: 980663a 185eca5
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Jan 11 13:50:29 2012 -0600
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Jan 11 13:50:29 2012 -0600

--
 CHANGES.txt|7 +++
 NEWS.txt   |   15 +
 bin/cqlsh  |   27 +
 debian/changelog   |6 ++
 .../concurrent/DebuggableThreadPoolExecutor.java   |2 +-
 .../cassandra/concurrent/NamedThreadFactory.java   |1 +
 src/java/org/apache/cassandra/db/Memtable.java |8 +++-
 .../cassandra/db/compaction/LeveledManifest.java   |   20 +---
 .../org/apache/cassandra/net/MessagingService.java |   10 +---
 .../cassandra/service/AbstractCassandraDaemon.java |   13 +++--
 .../apache/cassandra/service/StorageService.java   |9 ++--
 .../cassandra/thrift/CustomTThreadPoolServer.java  |   43 +++
 .../org/apache/cassandra/utils/ExpiringMap.java|   17 ++
 .../org/apache/cassandra/service/RemoveTest.java   |2 +-
 14 files changed, 129 insertions(+), 51 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a4fc7e27/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a4fc7e27/NEWS.txt
--
diff --cc NEWS.txt
index 036399b,2356faa..0f54918
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -8,65 -8,14 +8,73 @@@ upgrade, just in case you need to roll 
  (Cassandra version X + 1 will always be able to read data files created
  by version X, but the inverse is not necessarily the case.)
  
 +
 +1.1
 +===
 +
 +Upgrading
 +-
 +- If you are running a multi datacenter setup, you should upgrade to
 +  the latest 1.0.x (or 0.8.x) release before upgrading.  Versions
 +  0.8.8 and 1.0.3-1.0.5 generate cross-dc forwarding that is incompatible
 +  with 1.1.
 +- EACH_QUORUM ConsistencyLevel is only supported for writes and will now
 +  throw an InvalidRequestException when used for reads.  (Previous
 +  versions would silently perform a LOCAL_QUORUM read instead.)
 +- ANY ConsistencyLevel is only supported for writes and will now
 +  throw an InvalidRequestException when used for reads.  (Previous
 +  versions would silently perform a ONE read for range queries;
 +  single-row and multiget reads already rejected ANY.)
 +- The largest mutation batch accepted by the commitlog is now 128MB.  
 +  (In practice, batches larger than ~10MB always caused poor
 +  performance due to load volatility and GC promotion failures.)
 +  Larger batches will continue to be accepted but will not be
 +  durable.  Consider setting durable_writes=false if you really
 +  want to use such large batches.
 +- Make sure that global settings: key_cache_{size_in_mb, save_period}
 +  and row_cache_{size_in_mb, save_period} in conf/cassandra.yaml are
 +  used instead of per-ColumnFamily options.
 +- JMX methods no longer return custom Cassandra objects.  Any such methods
 +  will now return standard Maps, Lists, etc.
 +- Hadoop input and output details are now separated.  If you were
 +  previously using methods such as getRpcPort you now need to use
 +  getInputRpcPort or getOutputRpcPort depending on the circumstance.
 +- CQL changes:
 +  + Prior to 1.1, you could use KEY as the primary key name in some
 +select statements, even if the PK was actually given a different
 +name.  In 1.1+ you must use the defined PK name.
 +
 +
 +Features
 +
 +- Cassandra 1.1 adds row-level isolation.  Multi-column updates to
 +  a single row have always been *atomic* (either all will be applied,
 +  or none) thanks to the CommitLog, but until 1.1 they were not *isolated*
 +  -- a reader may see mixed old and new values while the update happens.
 +- Finer-grained control over data directories, allowing a ColumnFamily to
 +  be pinned to specfic media.
 +- Hadoop: a new BulkOutputFormat is included which will directly write
 +  SSTables locally and then stream them into the cluster.
 +- The bulk loader is not longer a fat client; it can be run from an
 +  existing machine in a cluster.
 +- A new write

[2/5] git commit: prevent slow clients from postponing shutdown indefinitely patch by jbellis; reviewed by brandonwilliams for CASSANDRA-3727

2012-01-11 Thread jbellis

prevent slow clients from postponing shutdown indefinitely
patch by jbellis; reviewed by brandonwilliams for CASSANDRA-3727


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/185eca5d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/185eca5d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/185eca5d

Branch: refs/heads/trunk
Commit: 185eca5d1fa7e384bb888c144d06abbced0fd577
Parents: d10da15
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Jan 11 11:44:32 2012 -0600
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Jan 11 12:48:40 2012 -0600

--
 .../concurrent/DebuggableThreadPoolExecutor.java   |2 +-
 .../cassandra/concurrent/NamedThreadFactory.java   |1 +
 src/java/org/apache/cassandra/db/Memtable.java |8 +++-
 .../org/apache/cassandra/net/MessagingService.java |   10 +---
 .../cassandra/service/AbstractCassandraDaemon.java |8 ++-
 .../apache/cassandra/service/StorageService.java   |9 ++--
 .../cassandra/thrift/CustomTThreadPoolServer.java  |   43 +++
 .../org/apache/cassandra/utils/ExpiringMap.java|   17 ++
 .../org/apache/cassandra/service/RemoveTest.java   |2 +-
 9 files changed, 59 insertions(+), 41 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/185eca5d/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
--
diff --git 
a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java 
b/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
index 7a344d8..f111d37 100644
--- a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
+++ b/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
@@ -108,7 +108,7 @@ public class DebuggableThreadPoolExecutor extends 
ThreadPoolExecutor
 protected void onFinalRejection(Runnable task) {}
 
 @Override
-public void afterExecute(Runnable r, Throwable t)
+protected void afterExecute(Runnable r, Throwable t)
 {
 super.afterExecute(r,t);
 logExceptionsAfterExecute(r, t);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/185eca5d/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
--
diff --git a/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java 
b/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
index 4cee8dc..a60a0d5 100644
--- a/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
+++ b/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
@@ -50,6 +50,7 @@ public class NamedThreadFactory implements ThreadFactory
 String name = id + : + n.getAndIncrement();
 Thread thread = new Thread(runnable, name);
 thread.setPriority(priority);
+thread.setDaemon(true);
 return thread;
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/185eca5d/src/java/org/apache/cassandra/db/Memtable.java
--
diff --git a/src/java/org/apache/cassandra/db/Memtable.java 
b/src/java/org/apache/cassandra/db/Memtable.java
index fdafc6d..412b800 100644
--- a/src/java/org/apache/cassandra/db/Memtable.java
+++ b/src/java/org/apache/cassandra/db/Memtable.java
@@ -31,6 +31,7 @@ import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
 import org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor;
+import org.apache.cassandra.concurrent.NamedThreadFactory;
 import org.apache.cassandra.db.columniterator.IColumnIterator;
 import org.apache.cassandra.db.columniterator.SimpleAbstractColumnIterator;
 import org.apache.cassandra.db.commitlog.ReplayPosition;
@@ -57,7 +58,12 @@ public class Memtable
 // we're careful to only allow one count to run at a time because counting 
is slow
 // (can be minutes, for a large memtable and a busy server), so we could 
keep memtables
 // alive after they're flushed and would otherwise be GC'd.
-private static final ExecutorService meterExecutor = new 
ThreadPoolExecutor(1, 1, Integer.MAX_VALUE, TimeUnit.MILLISECONDS, new 
SynchronousQueueRunnable())
+private static final ExecutorService meterExecutor = new 
DebuggableThreadPoolExecutor(1,
+   
   1,
+   
   Integer.MAX_VALUE,
+   
   TimeUnit.MILLISECONDS,
+   
   new SynchronousQueueRunnable(),

[3/3] git commit: have loadTokens expunge all tombstones patch by jbellis; reviewed by slebresne for CASSANDRA-3579

2012-01-11 Thread jbellis

have loadTokens expunge all tombstones
patch by jbellis; reviewed by slebresne for CASSANDRA-3579


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a95eafdc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a95eafdc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a95eafdc

Branch: refs/heads/cassandra-1.0
Commit: a95eafdccd398e775646688e9e37e0ae29e1114a
Parents: 185eca5
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Jan 11 13:41:59 2012 -0600
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Jan 11 13:51:58 2012 -0600

--
 src/java/org/apache/cassandra/db/SystemTable.java |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a95eafdc/src/java/org/apache/cassandra/db/SystemTable.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemTable.java 
b/src/java/org/apache/cassandra/db/SystemTable.java
index 93a0d05..19e9f15 100644
--- a/src/java/org/apache/cassandra/db/SystemTable.java
+++ b/src/java/org/apache/cassandra/db/SystemTable.java
@@ -223,7 +223,7 @@ public class SystemTable
 IPartitioner p = StorageService.getPartitioner();
 Table table = Table.open(Table.SYSTEM_TABLE);
 QueryFilter filter = QueryFilter.getIdentityFilter(decorate(RING_KEY), 
new QueryPath(STATUS_CF));
-ColumnFamily cf = 
table.getColumnFamilyStore(STATUS_CF).getColumnFamily(filter);
+ColumnFamily cf = 
ColumnFamilyStore.removeDeleted(table.getColumnFamilyStore(STATUS_CF).getColumnFamily(filter),
 Integer.MAX_VALUE);
 if (cf != null)
 {
 for (IColumn column : cf.getSortedColumns())

[1/3] git commit: Merge branch 'cassandra-1.0' into trunk

2012-01-11 Thread jbellis

Updated Branches:
  refs/heads/cassandra-1.0 185eca5d1 - a95eafdcc
  refs/heads/trunk a4fc7e270 - bfac658f0


Merge branch 'cassandra-1.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bfac658f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bfac658f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bfac658f

Branch: refs/heads/trunk
Commit: bfac658f0d19068d7ef922b3c3de631846b29ece
Parents: a4fc7e2 a95eafd
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Jan 11 13:52:35 2012 -0600
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Jan 11 13:52:35 2012 -0600

--
 src/java/org/apache/cassandra/db/SystemTable.java |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--

[2/3] git commit: have loadTokens expunge all tombstones patch by jbellis; reviewed by slebresne for CASSANDRA-3579

2012-01-11 Thread jbellis

have loadTokens expunge all tombstones
patch by jbellis; reviewed by slebresne for CASSANDRA-3579


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a95eafdc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a95eafdc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a95eafdc

Branch: refs/heads/trunk
Commit: a95eafdccd398e775646688e9e37e0ae29e1114a
Parents: 185eca5
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Jan 11 13:41:59 2012 -0600
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Jan 11 13:51:58 2012 -0600

--
 src/java/org/apache/cassandra/db/SystemTable.java |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a95eafdc/src/java/org/apache/cassandra/db/SystemTable.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemTable.java 
b/src/java/org/apache/cassandra/db/SystemTable.java
index 93a0d05..19e9f15 100644
--- a/src/java/org/apache/cassandra/db/SystemTable.java
+++ b/src/java/org/apache/cassandra/db/SystemTable.java
@@ -223,7 +223,7 @@ public class SystemTable
 IPartitioner p = StorageService.getPartitioner();
 Table table = Table.open(Table.SYSTEM_TABLE);
 QueryFilter filter = QueryFilter.getIdentityFilter(decorate(RING_KEY), 
new QueryPath(STATUS_CF));
-ColumnFamily cf = 
table.getColumnFamilyStore(STATUS_CF).getColumnFamily(filter);
+ColumnFamily cf = 
ColumnFamilyStore.removeDeleted(table.getColumnFamilyStore(STATUS_CF).getColumnFamily(filter),
 Integer.MAX_VALUE);
 if (cf != null)
 {
 for (IColumn column : cf.getSortedColumns())

[jira] [Resolved] (CASSANDRA-3579) AssertionError in hintedhandoff - 1.0.5

2012-01-11 Thread Jonathan Ellis (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-3579.
---

Resolution: Fixed

committed

 AssertionError in hintedhandoff - 1.0.5
 ---

 Key: CASSANDRA-3579
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3579
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: RHEL 6.1 64 bit, 32 GB RAM, 8 GB allocated to JVM, 
 running XFS filesystem for commit/data directories
Reporter: Ramesh Natarajan
Assignee: Sylvain Lebresne
 Fix For: 1.0.7

 Attachments: 3579-fix-text.txt, 3579-v2.txt, 3579.patch


 We are running a 8 node cassandra cluster running cassandra 1.0.5.
 All our CF use leveled compaction.  We ran a test where we did a lot
 of inserts for 3 days. After that we started to run tests where some
 of the reads could ask for information that was inserted a while back.
 In this scenario we are seeing this assertion error in HintedHandoff.
 ERROR [HintedHandoff:3] 2011-12-05 15:42:04,324
 AbstractCassandraDaemon.java (line 133) Fatal exception in thread
 Thread[HintedHandoff:3,1,main]
 java.lang.RuntimeException: java.lang.RuntimeException:
 java.util.concurrent.ExecutionException: java.lang.AssertionError:
 originally calculated column size of 470937164 but now it is 470294247
at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
 Caused by: java.lang.RuntimeException:
 java.util.concurrent.ExecutionException: java.lang.AssertionError:
 originally calculated column size of 470937164 but now it is 470294247
at 
 org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:330)
at 
 org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81)
at 
 org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:353)
at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
 Caused by: java.util.concurrent.ExecutionException:
 java.lang.AssertionError: originally calculated column size of
 470937164 but now it is 470294247
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at 
 org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:326)
... 6 more
 Caused by: java.lang.AssertionError: originally calculated column size
 of 470937164 but now it is 470294247
at 
 org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:124)
at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:160)
at 
 org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:158)
at 
 org.apache.cassandra.db.compaction.CompactionManager$6.call(CompactionManager.java:275)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
 ERROR [HintedHandoff:3] 2011-12-05 15:42:04,333
 AbstractCassandraDaemon.java (line 133) Fatal exception in thread
 Thread[HintedHandoff:3,1,main]
 java.lang.RuntimeException: java.lang.RuntimeException:
 java.util.concurrent.ExecutionException: java.lang.AssertionError:
 originally calculated column size of 470937164 but now it is 470294247
at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
 Caused by: java.lang.RuntimeException:
 java.util.concurrent.ExecutionException: java.lang.AssertionError:
 originally calculated column size of 470937164 but now it is 470294247
at 
 org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:330)
at 
 org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81)
at 
 org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:353)
at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
 Caused by: java.util.concurrent.ExecutionException:
 java.lang.AssertionError: originally calculated column size of
 470937164 but now it is 470294247
at

[jira] [Commented] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Jonathan Ellis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184328#comment-13184328
]

Jonathan Ellis commented on CASSANDRA-3727:
---

SystemTableTest failure taken care of on CASSANDRA-3579

Fix unit tests failure
--

Attachments: 3727.txt, CASSANDRA-3727-CliTest-timeout-fix.patch

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit
first prints a message with no failures in it), then hang until JUnit timeout
and fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading
to a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7)
mostly because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010.
It's likely just of problem of the test, but it's new and in the meantime
removes are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

[jira] [Commented] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Jonathan Ellis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184335#comment-13184335
]

Jonathan Ellis commented on CASSANDRA-3727:
---

bq. RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently).

FWIW, I get a timeout instead (on Windows).

Fix unit tests failure
--

Attachments: 3727.txt, CASSANDRA-3727-CliTest-timeout-fix.patch

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit
first prints a message with no failures in it), then hang until JUnit timeout
and fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading
to a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7)
mostly because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010.
It's likely just of problem of the test, but it's new and in the meantime
removes are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

[jira] [Commented] (CASSANDRA-1123) Allow tracing query details

2012-01-11 Thread Aaron Morton (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184339#comment-13184339
 ] 

Aaron Morton commented on CASSANDRA-1123:
-

Is there still interest in this ? 

 Allow tracing query details
 ---

 Key: CASSANDRA-1123
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1123
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
Assignee: Aaron Morton
 Fix For: 1.2

 Attachments: 1123-3.patch.gz


 In the spirit of CASSANDRA-511, it would be useful to tracing on queries to 
 see where latency is coming from: how long did row cache lookup take?  key 
 search in the index?  merging the data from the sstables?  etc.
 The main difference vs setting debug logging is that debug logging is too big 
 of a hammer; by turning on the flood of logging for everyone, you actually 
 distort the information you're looking for.  This would be something you 
 could set per-query (or more likely per connection).
 We don't need to be as sophisticated as the techniques discussed in the 
 following papers but they are interesting reading:
 http://research.google.com/pubs/pub36356.html
 http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/
 http://www.usenix.org/event/nsdi07/tech/fonseca.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Pavel Yaskevich (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184340#comment-13184340
]

Pavel Yaskevich commented on CASSANDRA-3727:

I get the same thing Sylvain does in RemoveTest : '/127.0.0.1:7010 is in use by
another process' on Mac OS X.

Fix unit tests failure
--

Attachments: 3727.txt, CASSANDRA-3727-CliTest-timeout-fix.patch

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit
first prints a message with no failures in it), then hang until JUnit timeout
and fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading
to a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7)
mostly because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010.
It's likely just of problem of the test, but it's new and in the meantime
removes are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

[cassandra-node] push by gdusba...@gmail.com - export bufferToString from driver, unexport Decoder on 2012-01-11 17:11 GMT

2012-01-11 Thread cassandra-node . apache-extras . org


Revision: 251aed763e01
Author:   Gary Dusbabek gdusba...@gmail.com
Date: Wed Jan 11 09:11:31 2012
Log:  export bufferToString from driver, unexport Decoder

http://code.google.com/a/apache-extras.org/p/cassandra-node/source/detail?r=251aed763e01

Modified:
 /lib/driver.js

===
--- /lib/driver.js  Wed Jan 11 09:06:46 2012
+++ /lib/driver.js  Wed Jan 11 09:11:31 2012
@@ -33,7 +33,10 @@

 var genericPool = require('generic-pool');

-var Decoder = module.exports.Decoder = require('./decoder').Decoder;
+var Decoder = require('./decoder').Decoder;
+
+var bufferToString = module.exports.bufferToString =  
require('./decoder').bufferToString;

+

 // used to parse the CF name out of a select statement.
 var selectRe = /\s*SELECT\s+.+\s+FROM\s+[\']?(\w+)/im;

[cassandra-node] push by gdusba...@gmail.com - export Decoder in driver on 2012-01-11 17:06 GMT

2012-01-11 Thread cassandra-node . apache-extras . org


Revision: e686ec8038f4
Author:   Gary Dusbabek gdusba...@gmail.com
Date: Wed Jan 11 09:06:46 2012
Log:  export Decoder in driver

http://code.google.com/a/apache-extras.org/p/cassandra-node/source/detail?r=e686ec8038f4

Modified:
 /lib/driver.js

===
--- /lib/driver.js  Wed Jan 11 09:01:40 2012
+++ /lib/driver.js  Wed Jan 11 09:06:46 2012
@@ -33,7 +33,7 @@

 var genericPool = require('generic-pool');

-var Decoder = require('./decoder').Decoder;
+var Decoder = module.exports.Decoder = require('./decoder').Decoder;

 // used to parse the CF name out of a select statement.
 var selectRe = /\s*SELECT\s+.+\s+FROM\s+[\']?(\w+)/im;

[cassandra-node] 3 new revisions pushed by gdusba...@gmail.com on 2012-01-11 17:04 GMT

2012-01-11 Thread cassandra-node . apache-extras . org


3 new revisions:

Revision: 974a3f8382e8
Author:   gdusbabek gdusba...@gmail.com
Date: Wed Jan 11 09:01:40 2012
Log:  detect select count(*) and decode appropriately.
http://code.google.com/a/apache-extras.org/p/cassandra-node/source/detail?r=974a3f8382e8

Revision: 474e16d83307
Author:   Gary Dusbabek gdusba...@gmail.com
Date: Wed Jan 11 08:05:25 2012
Log:  export bufferToHex and tests to show that binary cols/keys work
http://code.google.com/a/apache-extras.org/p/cassandra-node/source/detail?r=474e16d83307

Revision: 252c01c314aa
Author:   Gary Dusbabek gdusba...@gmail.com
Date: Wed Jan 11 09:03:09 2012
Log:  Merge branch '25/select_count'
http://code.google.com/a/apache-extras.org/p/cassandra-node/source/detail?r=252c01c314aa

==
Revision: 974a3f8382e8
Author:   gdusbabek gdusba...@gmail.com
Date: Wed Jan 11 09:01:40 2012
Log:  detect select count(*) and decode appropriately.

http://code.google.com/a/apache-extras.org/p/cassandra-node/source/detail?r=974a3f8382e8

Modified:
 /lib/decoder.js
 /lib/driver.js
 /test/test_driver.js

===
--- /lib/decoder.js Tue Jan 10 11:15:10 2012
+++ /lib/decoder.js Wed Jan 11 09:01:40 2012
@@ -129,6 +129,9 @@
   },
   'org.apache.cassandra.db.marshal.LexicalUUIDType': function(bytes) {
 return converters['org.apache.cassandra.db.marshal.UUIDType'](bytes);
+  },
+  'select count': function(bytes) {
+return converters['org.apache.cassandra.db.marshal.LongType'](bytes);
   }
 };

@@ -162,7 +165,9 @@
   } else if (which == 'comparator') {
 className = this.validators.comparator;
   } else if (which == 'validator') {
-if (column  this.validators.specificValidators[column]) {
+if (this.options.select_count) {
+  className = 'select count';
+} else if (column  this.validators.specificValidators[column]) {
   className = this.validators.specificValidators[column];
 } else {
   className = this.validators.defaultValidator;
===
--- /lib/driver.js  Mon Jan  9 03:12:55 2012
+++ /lib/driver.js  Wed Jan 11 09:01:40 2012
@@ -37,6 +37,7 @@

 // used to parse the CF name out of a select statement.
 var selectRe = /\s*SELECT\s+.+\s+FROM\s+[\']?(\w+)/im;
+var selectCountRe = /\s*SELECT\s+.*COUNT\(.+\)\s+FROM\s+[\']?(\w+)/im;

 var appExceptions =  
['InvalidRequestException', 'TimedOutException', 'UnavailableException',

   'SchemaDisagreementException'];
@@ -559,7 +560,8 @@
   } else {
 if (res.type === ttypes.CqlResultType.ROWS) {
   var cfName = selectRe.exec(cql)[1];
-  var decoder = new Decoder(self.validators[cfName], {use_bigints:  
self.connectionInfo.use_bigints});
+  var decoder = new Decoder(self.validators[cfName], {use_bigints:  
self.connectionInfo.use_bigints,
+   
select_count: selectCountRe.test(cql)});

   // for now, return results.
   var rows = [];
   for (var i = 0; i  res.rows.length; i++) {
===
--- /test/test_driver.jsMon Jan  9 12:44:46 2012
+++ /test/test_driver.jsWed Jan 11 09:01:40 2012
@@ -207,7 +207,8 @@
 function executeCountQuery(callback) {
   con.execute('SELECT COUNT(*) FROM CfLong', [], function(err, rows) {
 assert.ifError(err);
-assert.equal(rows[0].cols[0].value, 5);
+console.log(rows[0].cols);
+assert.strictEqual(rows[0].cols[0].value, 5);
 callback();
   });
 },

==
Revision: 474e16d83307
Author:   Gary Dusbabek gdusba...@gmail.com
Date: Wed Jan 11 08:05:25 2012
Log:  export bufferToHex and tests to show that binary cols/keys work

http://code.google.com/a/apache-extras.org/p/cassandra-node/source/detail?r=474e16d83307

Added:
 /test/util.js
Modified:
 /lib/decoder.js
 /test/test_decoder.js
 /test/test_driver.js

===
--- /dev/null
+++ /test/util.js   Wed Jan 11 08:05:25 2012
@@ -0,0 +1,23 @@
+
+// min is inclusive, max is exclusive.
+function randomInt(min, max) {
+  if (min === undefined) {
+min = -2147483648;
+  }
+  if (max === undefined) {
+max = 2147483647
+  }
+  return Math.round(Math.random() * (max - min) + min);
+}
+
+function randomBuffer(sz) {
+  sz = sz || randomInt(10, 100);
+  var buf = new Buffer(sz);
+  for (var i = 0; i  sz; i++) {
+buf[i] = randomInt(0, 255);
+  }
+  return buf;
+}
+
+exports.randomInt = randomInt;
+exports.randomBuffer = randomBuffer;
===
--- /lib/decoder.js Tue Jan 10 11:15:10 2012
+++ /lib/decoder.js Wed Jan 11 08:05:25 2012
@@ -63,6 +63,24 @@
   return bytesToBigInt(bytes);
 };

+// just what you think.
+function byteToHex(n) {
+  if (n  16) {
+return '0' + n.toString(16);

[jira] [Commented] (CASSANDRA-1123) Allow tracing query details

2012-01-11 Thread Jonathan Ellis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184347#comment-13184347
 ] 

Jonathan Ellis commented on CASSANDRA-1123:
---

Definitely.

I think we can do two things with this:

# Answer what is taking so long with query X
# Automatically enable tracing for say 1% of all queries and look for patterns

(2) is important because often (1) is not repeatable, if the slowness comes 
from hitting disk for instance it will usually be cached the next time.

I think we're 90% of the way there with this, but log files aren't very 
programatically accessible, i.e., it's difficult to implement scenario (2).  
I'd like to

- make enable_logging method return the query context id
- store trace results in a columnfamily keyed by the id so they can be 
retrieved and displayed (e.g. by adding EXPLAIN command to cqlsh, which would 
be a separate ticket)

What do you think?

 Allow tracing query details
 ---

 Key: CASSANDRA-1123
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1123
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
Assignee: Aaron Morton
 Fix For: 1.2

 Attachments: 1123-3.patch.gz


 In the spirit of CASSANDRA-511, it would be useful to tracing on queries to 
 see where latency is coming from: how long did row cache lookup take?  key 
 search in the index?  merging the data from the sstables?  etc.
 The main difference vs setting debug logging is that debug logging is too big 
 of a hammer; by turning on the flood of logging for everyone, you actually 
 distort the information you're looking for.  This would be something you 
 could set per-query (or more likely per connection).
 We don't need to be as sophisticated as the techniques discussed in the 
 following papers but they are interesting reading:
 http://research.google.com/pubs/pub36356.html
 http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/
 http://www.usenix.org/event/nsdi07/tech/fonseca.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Brandon Williams (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184361#comment-13184361
]

Brandon Williams commented on CASSANDRA-3727:
-

Same 'in use by another process' under linux. There is definitely no other
process.

Fix unit tests failure
--

Attachments: 3727.txt, CASSANDRA-3727-CliTest-timeout-fix.patch

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit
first prints a message with no failures in it), then hang until JUnit timeout
and fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading
to a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7)
mostly because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010.
It's likely just of problem of the test, but it's new and in the meantime
removes are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

[jira] [Commented] (CASSANDRA-3634) compare string vs. binary prepared statement parameters

2012-01-11 Thread Eric Evans (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184373#comment-13184373
]

Eric Evans commented on CASSANDRA-3634:
---

{quote}
It's more compelling to me when compared in the context of the existing RPC
performance. 5% is gain okay (PS vs RPC), but 16% (BB vs RPC) is a fairly
substantial improvement.

I was a little worried about the variance (even though the worst cases are
pretty close) but I ran some tests with the commit log disable and the
deviation is on par with the rest, I think it's just fast enough to push it
that hard.
{quote}

That's interesting. I got so wrapped up in the {{ByteBuffer}} vs. {{String}}
comparison that I lost track of the fact that your results put CQL w/ prepared
statements ahead of RPC _across the board_ (which is the most important
take-away from this I hope!). That would mean that you're willing to trade
that node-side abstraction for performance that is _already above-and-beyond_
RPC. I think I completely overestimated how compelling the
simplicity/abstraction vs performance trade-off argument would be to folks.

compare string vs. binary prepared statement parameters
---

Key: CASSANDRA-3634
URL: https://issues.apache.org/jira/browse/CASSANDRA-3634
Project: Cassandra
Issue Type: Sub-task
Components: API, Core
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor
Labels: cql
Fix For: 1.1

Perform benchmarks to compare the performance of string and pre-serialized
binary parameters to prepared statements.

[jira] [Created] (CASSANDRA-3728) Better error message when a column family creation fails

2012-01-11 Thread Eric Lubow (Created) (JIRA)

Better error message when a column family creation fails


 Key: CASSANDRA-3728
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3728
 Project: Cassandra
  Issue Type: Bug
Reporter: Eric Lubow
Priority: Minor


Since '-' characters are not allowed in column family names, there should be an 
error thrown on column family name validation.

[default@linkcurrent] create column family foo-bar;
null

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3728) Better error message when a column family creation fails

2012-01-11 Thread Jonathan Ellis (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3728:
--

 Reviewer: yukim
Fix Version/s: 1.0.8
 Assignee: Pavel Yaskevich

 Better error message when a column family creation fails
 

 Key: CASSANDRA-3728
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3728
 Project: Cassandra
  Issue Type: Bug
Reporter: Eric Lubow
Assignee: Pavel Yaskevich
Priority: Minor
  Labels: cli
 Fix For: 1.0.8


 Since '-' characters are not allowed in column family names, there should be 
 an error thrown on column family name validation.
 [default@linkcurrent] create column family foo-bar;
 null

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: Make stress.java insert operation to use microseconds patch by Yuki Morishita; reviewed by Pavel Yaskevich for CASSANDRA-3725

2012-01-11 Thread xedin

Updated Branches:
  refs/heads/trunk bfac658f0 - bb7b64312


Make stress.java insert operation to use microseconds
patch by Yuki Morishita; reviewed by Pavel Yaskevich for CASSANDRA-3725


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bb7b6431
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bb7b6431
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bb7b6431

Branch: refs/heads/trunk
Commit: bb7b64312d80d9dc3864cf6b09d565cbd6a14c14
Parents: bfac658
Author: Pavel Yaskevich pove...@gmail.com
Authored: Thu Jan 12 00:43:41 2012 +0200
Committer: Pavel Yaskevich pove...@gmail.com
Committed: Thu Jan 12 00:43:41 2012 +0200

--
 CHANGES.txt|2 ++
 .../cassandra/stress/operations/Inserter.java  |3 ++-
 2 files changed, 4 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/bb7b6431/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3bdd26f..edfb9e1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -42,6 +42,8 @@
  * finer-grained control over data directories (CASSANDRA-2749)
  * Fix ClassCastException during hinted handoff (CASSANDRA-3694)
  * Upgrade Thrift to 0.7 (CASSANDRA-3213)
+ * Make stress.java insert operation to use microseconds (CASSANDRA-3725)
+
 
 1.0.7
  * fix regression in HH page size calculation (CASSANDRA-3624)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/bb7b6431/tools/stress/src/org/apache/cassandra/stress/operations/Inserter.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/operations/Inserter.java 
b/tools/stress/src/org/apache/cassandra/stress/operations/Inserter.java
index c81df6f..a887724 100644
--- a/tools/stress/src/org/apache/cassandra/stress/operations/Inserter.java
+++ b/tools/stress/src/org/apache/cassandra/stress/operations/Inserter.java
@@ -22,6 +22,7 @@ import org.apache.cassandra.stress.util.Operation;
 import org.apache.cassandra.db.ColumnFamilyType;
 import org.apache.cassandra.thrift.*;
 import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.cassandra.utils.FBUtilities;
 
 import java.io.IOException;
 import java.nio.ByteBuffer;
@@ -54,7 +55,7 @@ public class Inserter extends Operation
 {
 columns.add(new Column(columnName(i, session.timeUUIDComparator))
 .setValue(values.get(i % values.size()))
-.setTimestamp(System.currentTimeMillis()));
+.setTimestamp(FBUtilities.timestampMicros()));
 }
 
 if (session.getColumnFamilyType() == ColumnFamilyType.Super)

[jira] [Resolved] (CASSANDRA-3725) Switch stress tool to using micros

2012-01-11 Thread Pavel Yaskevich (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich resolved CASSANDRA-3725.


Resolution: Fixed

Committed.

 Switch stress tool to using micros
 --

 Key: CASSANDRA-3725
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3725
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Cathy Daw
Assignee: Yuki Morishita
Priority: Trivial
 Fix For: 1.1

 Attachments: 3725.txt


 The situation encountered is that after deleting the columns for a row in the 
 cli new workloads don't update that columns for the rows. Brandon mentioned 
 that stress uses millis, cli uses micros and therefore row tombstone wins.
 Test Case: Before Delete
 {code}
 // Run: stress --operation=INSERT --num-keys=1  --columns=2 
 --consistency-level=QUORUM --column-size=1 --threads=1 --replication-factor=1 
 --nodes=localhost
 // Run cassandra-cli
 [default@Keyspace1] list Standard1;   
 Using default limit of 100
 ---
 RowKey: 30
 = (column=C0, value=63, timestamp=1326259090065)
 = (column=C1, value=63, timestamp=1326259090065)
 1 Row Returned.
 Elapsed time: 2 msec(s).
 [default@Keyspace1] del Standard1['30'];  
 row removed.
 [default@Keyspace1] list Standard1; 
 Using default limit of 100
 ---
 RowKey: 30
 {code}
 Test Case: After Delete
 {code}
 // Run: stress --operation=INSERT --num-keys=1  --columns=2 
 --consistency-level=QUORUM --column-size=1 --threads=1 --replication-factor=1 
 --nodes=localhost
 // Run cassandra-cli
 [default@Keyspace1] list Standard1;
 Using default limit of 100
 ---
 RowKey: 30
 1 Row Returned.
 Elapsed time: 1 msec(s).
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3674) add nodetool explicitgc

2012-01-11 Thread Peter Schuller (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184422#comment-13184422
 ] 

Peter Schuller commented on CASSANDRA-3674:
---

So is this a WONTFIX?

 add nodetool explicitgc
 ---

 Key: CASSANDRA-3674
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3674
 Project: Cassandra
  Issue Type: Improvement
Reporter: Peter Schuller
Assignee: Peter Schuller
Priority: Minor
 Attachments: CASSANDRA-3674-trunk.txt


 So that you can easily ask people run nodetool explicitgc and paste the 
 results.
 I'll file a separate JIRA suggesting that we ship with 
 -XX:+ExplicitGCInvokesConcurrent by default.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3722) Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.

2012-01-11 Thread Vijay (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184427#comment-13184427
]

Vijay commented on CASSANDRA-3722:
--

If we have thee end time we can avoid sending traffic back to the node untill
it recovers completely right? If compaction are for a second Is the worst case
is till ring delay we will not send traffic... alternatively we could do the
difference from the response time avg and throttle the request... or even T the
request just to conform if the even is complete Makes sense?

Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.
--

Key: CASSANDRA-3722
URL: https://issues.apache.org/jira/browse/CASSANDRA-3722
Project: Cassandra
Issue Type: Improvement
Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
Priority: Minor

Currently Dynamic snitch looks at the latency for figuring out which node
will be better serving the requests, this works great but there is a part of
the traffic sent to collect this data... There is also a window when Snitch
doesn't know about some major event which are going to happen on the node
(Node which is going to receive the data request).
It would be great if we can send some sort hints to the Snitch so they can
score based on known events causing higher latencies.

[jira] [Commented] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Jonathan Ellis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184436#comment-13184436
]

Jonathan Ellis commented on CASSANDRA-3727:
---

RemoveTest is trying to stop/start MessagingService multiple times in the same
suite (see setup/tearDown methods). My guess is that worked well enough
pre-CASSANDRA-3335. I'll see about making it happy again, although this feels
fragile.

Fix unit tests failure
--

Attachments: 3727.txt, CASSANDRA-3727-CliTest-timeout-fix.patch

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit
first prints a message with no failures in it), then hang until JUnit timeout
and fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading
to a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7)
mostly because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010.
It's likely just of problem of the test, but it's new and in the meantime
removes are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

[jira] [Commented] (CASSANDRA-3722) Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.

2012-01-11 Thread Brandon Williams (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184475#comment-13184475
 ] 

Brandon Williams commented on CASSANDRA-3722:
-

bq. If we have the end time we can avoid sending traffic back to the node until 
it recovers completely right?

I'm not sure if you mean writes or checksum requests for read repair here, but 
neither is avoidable (except for RR by adjusting the probability setting.)

If you mean the one or two reads after the reset interval, that doesn't seem 
like it's going to have a big enough impact to optimize for.

 Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.
 --

 Key: CASSANDRA-3722
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3722
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
Priority: Minor

 Currently Dynamic snitch looks at the latency for figuring out which node 
 will be better serving the requests, this works great but there is a part of 
 the traffic sent to collect this data... There is also a window when Snitch 
 doesn't know about some major event which are going to happen on the node 
 (Node which is going to receive the data request).
 It would be great if we can send some sort hints to the Snitch so they can 
 score based on known events causing higher latencies.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: attempt to humor tests that try to stop and restart MS

2012-01-11 Thread jbellis

Updated Branches:
  refs/heads/cassandra-1.0 a95eafdcc - 452ddf63c


attempt to humor tests that try to stop and restart MS


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/452ddf63
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/452ddf63
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/452ddf63

Branch: refs/heads/cassandra-1.0
Commit: 452ddf63c530fc573551e6fc9c79c1a876f11dd0
Parents: a95eafd
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Jan 11 16:59:33 2012 -0600
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Jan 11 16:59:33 2012 -0600

--
 .../org/apache/cassandra/net/MessagingService.java |   14 ++
 1 files changed, 14 insertions(+), 0 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/452ddf63/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java 
b/src/java/org/apache/cassandra/net/MessagingService.java
index 9ff110e..99037a1 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -482,7 +482,21 @@ public final class MessagingService implements 
MessagingServiceMBean
 logger_.info(Waiting for messaging service to quiesce);
 // We may need to schedule hints on the mutation stage, so it's 
erroneous to shut down the mutation stage first
 assert !StageManager.getStage(Stage.MUTATION).isShutdown();
+
+// the important part
 callbacks.shutdown();
+
+// attempt to humor tests that try to stop and restart MS
+try
+{
+for (SocketThread th : socketThreads)
+th.close();
+}
+catch (IOException e)
+{
+throw new IOError(e);
+}
+
 }
 
 public void receive(Message message, String id)

[jira] [Commented] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Jonathan Ellis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184495#comment-13184495
]

Jonathan Ellis commented on CASSANDRA-3727:
---

Pushed 452ddf63c530fc573551e6fc9c79c1a876f11dd0 with the socket teardown code
added back. Now it's hitting {{java.io.IOException: Failed to delete
c:\Users\Jonathan\projects\cassandra\git\build\test\cassandra\commitlog\CommitLog-1326319330071.log}}.
Progress?

Fix unit tests failure
--

Attachments: 3727.txt, CASSANDRA-3727-CliTest-timeout-fix.patch

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit
first prints a message with no failures in it), then hang until JUnit timeout
and fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading
to a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7)
mostly because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010.
It's likely just of problem of the test, but it's new and in the meantime
removes are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

[jira] [Commented] (CASSANDRA-3722) Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.

2012-01-11 Thread Vijay (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184497#comment-13184497
 ] 

Vijay commented on CASSANDRA-3722:
--

Hi Brandon, i am talking about Reads when the RR is disabled (0.0) It is 
not going to be 1 or 2 reads (we would be really happy if it is only 1 to 2 
reads) it will be as many reads it takes for the first read to come back, for 
example we have 1000 request per second work load when we see the node goes 
slower it might take a second or timeout (10 second) which means 1000 to 10k 
requests.

 Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.
 --

 Key: CASSANDRA-3722
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3722
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
Priority: Minor

 Currently Dynamic snitch looks at the latency for figuring out which node 
 will be better serving the requests, this works great but there is a part of 
 the traffic sent to collect this data... There is also a window when Snitch 
 doesn't know about some major event which are going to happen on the node 
 (Node which is going to receive the data request).
 It would be great if we can send some sort hints to the Snitch so they can 
 score based on known events causing higher latencies.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3634) compare string vs. binary prepared statement parameters

2012-01-11 Thread Jonathan Ellis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184510#comment-13184510
]

Jonathan Ellis commented on CASSANDRA-3634:
---

My reasoning is, there aren't a whole lot of places left to pick up an extra
10% performance... Two years ago, or one, maybe 10% isn't such a big deal
since there's so much left to optimize. That's no longer the case; I don't
think we should knowingly lock our next-gen interface into a lower-performing
design. Once made, we're stuck with this decision, or at least with a really,
really high barrier to change it.

On the other hand, we have the downside of extra complexity for the driver
authors. While this is a valid point, it's a finite one -- once a prepared
statement api has been created and debugged, binary vs strings isn't going to
matter. It's a one-time fee in exchange for better performance forever.
Additionally, sample binary marshalling code already exists for any language
with a Thrift driver. So we're really talking about a relatively small amount
of work to build a binary-based PS api, over a String one.

compare string vs. binary prepared statement parameters
---

Perform benchmarks to compare the performance of string and pre-serialized
binary parameters to prepared statements.

[jira] [Commented] (CASSANDRA-3722) Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.

2012-01-11 Thread Brandon Williams (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184511#comment-13184511
 ] 

Brandon Williams commented on CASSANDRA-3722:
-

Hmm, I see.  Instead of communicating various states the node is in over 
gossip, maybe penalizing hosts with pending reads would work better since it 
would work universally.

 Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.
 --

 Key: CASSANDRA-3722
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3722
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
Priority: Minor

 Currently Dynamic snitch looks at the latency for figuring out which node 
 will be better serving the requests, this works great but there is a part of 
 the traffic sent to collect this data... There is also a window when Snitch 
 doesn't know about some major event which are going to happen on the node 
 (Node which is going to receive the data request).
 It would be great if we can send some sort hints to the Snitch so they can 
 score based on known events causing higher latencies.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3722) Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.

2012-01-11 Thread Jonathan Ellis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184516#comment-13184516
 ] 

Jonathan Ellis commented on CASSANDRA-3722:
---

bq. maybe penalizing hosts with pending reads would work better since it would 
work universally

I like that idea.

 Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.
 --

 Key: CASSANDRA-3722
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3722
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
Priority: Minor

 Currently Dynamic snitch looks at the latency for figuring out which node 
 will be better serving the requests, this works great but there is a part of 
 the traffic sent to collect this data... There is also a window when Snitch 
 doesn't know about some major event which are going to happen on the node 
 (Node which is going to receive the data request).
 It would be great if we can send some sort hints to the Snitch so they can 
 score based on known events causing higher latencies.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3674) add nodetool explicitgc

2012-01-11 Thread Jonathan Ellis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184515#comment-13184515
 ] 

Jonathan Ellis commented on CASSANDRA-3674:
---

IMO it should be a wontfix -- I can't remember ever asking someone to force a 
GC when troubleshooting.  Taking a heap dump, yes, but hopefully we're not 
going to reimplement jmap next. :)

 add nodetool explicitgc
 ---

 Key: CASSANDRA-3674
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3674
 Project: Cassandra
  Issue Type: Improvement
Reporter: Peter Schuller
Assignee: Peter Schuller
Priority: Minor
 Attachments: CASSANDRA-3674-trunk.txt


 So that you can easily ask people run nodetool explicitgc and paste the 
 results.
 I'll file a separate JIRA suggesting that we ship with 
 -XX:+ExplicitGCInvokesConcurrent by default.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3722) Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.

2012-01-11 Thread Brandon Williams (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184527#comment-13184527
]

Brandon Williams commented on CASSANDRA-3722:
-

The tricky thing is, we don't know how much to penalize them per pending read.
We could choose an arbitrary static amount, and that would work as long as it's
the same for all hosts... except the badness threshold loses meaning.

Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.
--

[jira] [Commented] (CASSANDRA-3722) Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.

2012-01-11 Thread Vijay (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184576#comment-13184576
 ] 

Vijay commented on CASSANDRA-3722:
--

but the pending will be almost zero for the bad nodes as we already redirected 
traffic, right?

 Send Hints to Dynamic Snitch when Compaction or repair is going on for a node.
 --

 Key: CASSANDRA-3722
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3722
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1
Reporter: Vijay
Assignee: Vijay
Priority: Minor

 Currently Dynamic snitch looks at the latency for figuring out which node 
 will be better serving the requests, this works great but there is a part of 
 the traffic sent to collect this data... There is also a window when Snitch 
 doesn't know about some major event which are going to happen on the node 
 (Node which is going to receive the data request).
 It would be great if we can send some sort hints to the Snitch so they can 
 score based on known events causing higher latencies.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-3729) support counter debug mode on thrift interface

2012-01-11 Thread Peter Schuller (Created) (JIRA)

support counter debug mode on thrift interface
--

 Key: CASSANDRA-3729
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3729
 Project: Cassandra
  Issue Type: Improvement
Reporter: Peter Schuller
Assignee: Peter Schuller
Priority: Minor
 Attachments: trunk-3729.txt

Attaching a patch against trunk to add a counter debug mode on the thrift 
interface, allowing clients to decode and inspect counter contexts. This is all 
Stu's code, except that I generated the thrift stuff so any mistakes there are 
mine.

This was extremely useful internally on an 0.8. The patch is not yet tested on 
trunk, but if you think this can go in I will spend effort to test it soonish.

It's not very invasive (other than the generated thrift code), so it feels okay 
to have it if we maybe document that it is not a supported interface (clearly 
in the thrift spec).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3729) support counter debug mode on thrift interface

2012-01-11 Thread Peter Schuller (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Schuller updated CASSANDRA-3729:
--

Attachment: trunk-3729.txt

 support counter debug mode on thrift interface
 --

 Key: CASSANDRA-3729
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3729
 Project: Cassandra
  Issue Type: Improvement
Reporter: Peter Schuller
Assignee: Peter Schuller
Priority: Minor
 Attachments: trunk-3729.txt


 Attaching a patch against trunk to add a counter debug mode on the thrift 
 interface, allowing clients to decode and inspect counter contexts. This is 
 all Stu's code, except that I generated the thrift stuff so any mistakes 
 there are mine.
 This was extremely useful internally on an 0.8. The patch is not yet tested 
 on trunk, but if you think this can go in I will spend effort to test it 
 soonish.
 It's not very invasive (other than the generated thrift code), so it feels 
 okay to have it if we maybe document that it is not a supported interface 
 (clearly in the thrift spec).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Cassandra Wiki] Update of HowToContribute by DaveBrosius

2012-01-11 Thread Apache Wiki

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The HowToContribute page has been changed by DaveBrosius:
http://wiki.apache.org/cassandra/HowToContribute?action=diffrev1=41rev2=42

Comment:
show new git repository instructions (and for now don't show github mirror as 
it's not mirroring)

   1. Check if someone else has already begun work on the change you have in 
mind in the [[https://issues.apache.org/jira/browse/CASSANDRA|issue tracker]]
   1. If not, create a ticket describing the change you're proposing in the 
issue tracker
   1. Check out the latest version of the source code
-   * svn checkout http://svn.apache.org/repos/asf/cassandra/trunk 
cassandra-trunk
+   * git clone http://git-wip-us.apache.org/repos/asf/cassandra.git 
cassandra-trunk
-. or
-   * git clone git://git.apache.org/cassandra.git
   1. Modify the source to include the improvement/bugfix
* Verify that you follow Cassandra's CodeStyle.
* Verify that your change works by adding a unit test.
* Make sure all tests pass by running ant test in the project directory.
* For testing multi-node behavior, https://github.com/pcmanus/ccm is useful
   1. When you're happy with the result create a patch:
-   * svn add any new file
+   * git add any new file
-   * svn diff  branchname-issue.txt (e.g. trunk-123.txt, 
cassandra-0.6-123.txt)
+   * git diff  branchname-issue.txt (e.g. trunk-123.txt, 
cassandra-0.6-123.txt)
   1. Attach the newly generated patch to the issue and click Submit patch in 
the left side of the JIRA page
   1. Wait for other developers or committers to review it and hopefully +1 the 
ticket
   1. Wait for a committer to commit it.

[Cassandra Wiki] Update of RunningCassandraInEclipse by DaveBrosius

2012-01-11 Thread Apache Wiki

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The RunningCassandraInEclipse page has been changed by DaveBrosius:
http://wiki.apache.org/cassandra/RunningCassandraInEclipse?action=diffrev1=34rev2=35

Comment:
switch svn instructions to git

  
  These instructions were originally tested on Ubuntu 11.04, Eclipse Indigo 
SR1, on the trunk code shortly after the release of Cassandra 1.0.
  
- From the console, checkout the code using SVN. Here we assume you are 
checking out the latest trunk, but browse 
http://svn.apache.org/repos/asf/cassandra/ for all available versions...
+ From the console, checkout the code using GIT. Here we assume you are 
checking out the latest trunk, but browse 
http://git-wip-us.apache.org/repos/asf/cassandra.git for all available 
versions...
  
  {{{
- svn checkout http://svn.apache.org/repos/asf/cassandra/trunk/ cassandra-trunk
+ git clone http://git-wip-us.apache.org/repos/asf/cassandra.git cassandra-trunk
  
  cd cassandra-trunk

[Cassandra Wiki] Update of RunningCassandraInEclipse by DaveBrosius

2012-01-11 Thread Apache Wiki

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The RunningCassandraInEclipse page has been changed by DaveBrosius:
http://wiki.apache.org/cassandra/RunningCassandraInEclipse?action=diffrev1=35rev2=36

Comment:
more conversions for svn docs to git docs

  Eclipse is open source. Download Eclipse from 
http://www.eclipse.org/downloads/. There is no need for the Enterprise Edition 
(EE) version of Eclipse. Hence
  Eclipse IDE for Java Developers is good enough. 
  
- Cassandra is using Git and Subversion for version control. In this tutorial 
we will checkout Cassandra from Subversion. 
+ Cassandra is using Git for version control. In this tutorial we will checkout 
Cassandra from it. 
  
- The previous version of this guide used the Subclipse 
(http://subclipse.tigris.org/) Eclipse Subversion plugin. However, over time, 
the instructions became out-of-date and confusing - if you are currently using 
Subclipse or Subversive successfully with Cassandra then please add a new 
section below!
+ The previous version of this guide used the Subclipse 
(http://subclipse.tigris.org/) Eclipse Subversion plugin. However, over time, 
the instructions became out-of-date and confusing - if you are currently using 
Subclipse or Subversive successfully with Cassandra be aware as of December 
2011 Cassandra was switched to Git.
  
  For the moment, we will a command-line Subversion client.
  
@@ -15, +15 @@

  
  These instructions were originally tested on Ubuntu 11.04, Eclipse Indigo 
SR1, on the trunk code shortly after the release of Cassandra 1.0.
  
- From the console, checkout the code using GIT. Here we assume you are 
checking out the latest trunk, but browse 
http://git-wip-us.apache.org/repos/asf/cassandra.git for all available 
versions...
+ From the console, checkout the code using Git. Here we assume you are 
checking out the latest trunk, but browse 
http://git-wip-us.apache.org/repos/asf/cassandra.git for all available 
versions...
  
  {{{
  git clone http://git-wip-us.apache.org/repos/asf/cassandra.git cassandra-trunk

[jira] [Commented] (CASSANDRA-3634) compare string vs. binary prepared statement parameters

2012-01-11 Thread Eric Evans (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184680#comment-13184680
]

Eric Evans commented on CASSANDRA-3634:
---

{quote}
My reasoning is, there aren't a whole lot of places left to pick up an extra
10% performance... Two years ago, or one, maybe 10% isn't such a big deal since
there's so much left to optimize. That's no longer the case; I don't think we
should knowingly lock our next-gen interface into a lower-performing design.
Once made, we're stuck with this decision, or at least with a really, really
high barrier to change it.
{quote}

I think a custom protocol (planned for reasons unrelated to performance) could
easily be worth 10%. I take your point though, there isn't a lot of low
hanging fruit left.

{quote}
On the other hand, we have the downside of extra complexity for the driver
authors. While this is a valid point, it's a finite one – once a prepared
statement api has been created and debugged, binary vs strings isn't going to
matter. It's a one-time fee in exchange for better performance forever.
Additionally, sample binary marshalling code already exists for any language
with a Thrift driver. So we're really talking about a relatively small amount
of work to build a binary-based PS api, over a String one.
{quote}

I'm probably a little less optimistic about the amount of work or the potential
for bugs. A Pycassa bug that comes to mind caused integers to be mis-encoded
for more than a year before it was caught and fixed (and this being one of our
most (_the_ most?) battle-tested libraries).

That said, I do understand all of your points.

Considering the _kind_ of trade-off we're talking about, I wanted this issue to
be thoroughly thought through/discussed, with any relevant data readily at
hand. The scale is obviously quite different (I'm not citing a full swing of
the pendulum here), but the arguments for/against are basically the same ones
that spawned CQL in the first place. And, as you said, changing later is
prohibitively difficult; We're going to have to live with this decision.

I posted to client-dev@ earlier (I don't know why I didn't think of that a week
ago). They're basically our front-line users in this regard, and I think it
would be interesting to hear from some of them (particularly if I'm carrying a
mantle none of them care about :)).

compare string vs. binary prepared statement parameters
---

Perform benchmarks to compare the performance of string and pre-serialized
binary parameters to prepared statements.

[jira] [Commented] (CASSANDRA-3727) Fix unit tests failure

2012-01-11 Thread Jonathan Ellis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184681#comment-13184681
]

Jonathan Ellis commented on CASSANDRA-3727:
---

User error... it was my paused Cassandra instance in my IDE holding that CL
segment open. With that figured out, I also pushed
1e750138177e9cd9cbd6537451a4b5cd301dab3a which allows MS to be restarted by
RemoveTest.

All the tests now pass for me, except RecoveryManagerTruncateTest which has
failed on Windows for 0.8+.

Fix unit tests failure
--

Attachments: 3727.txt, CASSANDRA-3727-CliTest-timeout-fix.patch

On current 1.0 branch (and on my machine: Linux), I have the following unit
test failures:
* CliTest and EmbeddedCassandraTest: they both first kind of pass (JUnit
first prints a message with no failures in it), then hang until JUnit timeout
and fails with a 'Timeout occurred'. In other word, the tests themselves are
passing, but something they do prevents the process to exit cleanly leading
to a JUnit timeout. I don't want to discard that as not a problem, because if
something can make the process not exit cleanly, this can be a pain for
restarts (and in particular upgrades) and hence would be basically a
regression. I'm marking the ticket as blocker (for the release of 1.0.7)
mostly because of this one.
* SystemTableTest: throws an assertionError. I haven't checked yet, so that
could be an easy one to fix.
* RemoveTest: it fails, saying that '/127.0.0.1:7010 is in use by another
process' (consistently). But I have no other process running on port 7010.
It's likely just of problem of the test, but it's new and in the meantime
removes are not tested.
* I also see a bunch of stack trace with errors like:
{noformat}
[junit] ERROR 10:01:59,007 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: java.io.IOException: Unable to create
hard link from build/test/cassandra/data/Keyspace1/Indexed1-hc-1-Index.db to
/home/mcmanus/Git/cassandra/build/test/cassandra/data/Keyspace1/backups/Indexed1-hc-1-Index.db
(errno 17)
{noformat}
(with SSTableReaderTest). This does not make the tests fail, but it is still
worth investigating. It may be due to CASSANDRA-3101.

[2/4] git commit: reset callbacks map in MS.listen to allow RemoveTest to restart it

2012-01-11 Thread jbellis

reset callbacks map in MS.listen to allow RemoveTest to restart it


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1e750138
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1e750138
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1e750138

Branch: refs/heads/cassandra-1.0
Commit: 1e750138177e9cd9cbd6537451a4b5cd301dab3a
Parents: 452ddf6
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Jan 11 20:26:44 2012 -0600
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Jan 11 20:26:44 2012 -0600

--
 .../org/apache/cassandra/net/MessagingService.java |4 ++--
 .../org/apache/cassandra/utils/ExpiringMap.java|3 ++-
 2 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e750138/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java 
b/src/java/org/apache/cassandra/net/MessagingService.java
index 99037a1..e12f9ee 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -228,6 +228,7 @@ public final class MessagingService implements 
MessagingServiceMBean
  */
 public void listen(InetAddress localEp) throws IOException, 
ConfigurationException
 {
+callbacks.reset(); // hack to allow tests to stop/restart MS
 for (ServerSocket ss: getServerSocket(localEp))
 {
 SocketThread th = new SocketThread(ss, ACCEPT- + localEp);
@@ -471,7 +472,7 @@ public final class MessagingService implements 
MessagingServiceMBean
 
 public void clearCallbacksUnsafe()
 {
-callbacks.clear();
+callbacks.reset();
 }
 
 /**
@@ -496,7 +497,6 @@ public final class MessagingService implements 
MessagingServiceMBean
 {
 throw new IOError(e);
 }
-
 }
 
 public void receive(Message message, String id)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e750138/src/java/org/apache/cassandra/utils/ExpiringMap.java
--
diff --git a/src/java/org/apache/cassandra/utils/ExpiringMap.java 
b/src/java/org/apache/cassandra/utils/ExpiringMap.java
index 0672259..000af72 100644
--- a/src/java/org/apache/cassandra/utils/ExpiringMap.java
+++ b/src/java/org/apache/cassandra/utils/ExpiringMap.java
@@ -121,8 +121,9 @@ public class ExpiringMapK, V
 timer.cancel();
 }
 
-public void clear()
+public void reset()
 {
+shutdown = false;
 cache.clear();
 }

[3/4] git commit: reset callbacks map in MS.listen to allow RemoveTest to restart it

2012-01-11 Thread jbellis

reset callbacks map in MS.listen to allow RemoveTest to restart it


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1e750138
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1e750138
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1e750138

Branch: refs/heads/trunk
Commit: 1e750138177e9cd9cbd6537451a4b5cd301dab3a
Parents: 452ddf6
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Jan 11 20:26:44 2012 -0600
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Jan 11 20:26:44 2012 -0600

--
 .../org/apache/cassandra/net/MessagingService.java |4 ++--
 .../org/apache/cassandra/utils/ExpiringMap.java|3 ++-
 2 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e750138/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java 
b/src/java/org/apache/cassandra/net/MessagingService.java
index 99037a1..e12f9ee 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -228,6 +228,7 @@ public final class MessagingService implements 
MessagingServiceMBean
  */
 public void listen(InetAddress localEp) throws IOException, 
ConfigurationException
 {
+callbacks.reset(); // hack to allow tests to stop/restart MS
 for (ServerSocket ss: getServerSocket(localEp))
 {
 SocketThread th = new SocketThread(ss, ACCEPT- + localEp);
@@ -471,7 +472,7 @@ public final class MessagingService implements 
MessagingServiceMBean
 
 public void clearCallbacksUnsafe()
 {
-callbacks.clear();
+callbacks.reset();
 }
 
 /**
@@ -496,7 +497,6 @@ public final class MessagingService implements 
MessagingServiceMBean
 {
 throw new IOError(e);
 }
-
 }
 
 public void receive(Message message, String id)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e750138/src/java/org/apache/cassandra/utils/ExpiringMap.java
--
diff --git a/src/java/org/apache/cassandra/utils/ExpiringMap.java 
b/src/java/org/apache/cassandra/utils/ExpiringMap.java
index 0672259..000af72 100644
--- a/src/java/org/apache/cassandra/utils/ExpiringMap.java
+++ b/src/java/org/apache/cassandra/utils/ExpiringMap.java
@@ -121,8 +121,9 @@ public class ExpiringMapK, V
 timer.cancel();
 }
 
-public void clear()
+public void reset()
 {
+shutdown = false;
 cache.clear();
 }

[1/4] git commit: merge from 1.0

2012-01-11 Thread jbellis

Updated Branches:
  refs/heads/cassandra-1.0 452ddf63c - 1e7501381
  refs/heads/trunk bb7b64312 - b3a378c02


merge from 1.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b3a378c0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b3a378c0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b3a378c0

Branch: refs/heads/trunk
Commit: b3a378c02598f2150aede95daaae273ea356f127
Parents: bb7b643 1e75013
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Jan 11 20:48:40 2012 -0600
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Jan 11 20:48:40 2012 -0600

--
 .../org/apache/cassandra/net/MessagingService.java |   16 ++-
 .../org/apache/cassandra/utils/ExpiringMap.java|3 +-
 2 files changed, 17 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b3a378c0/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --cc src/java/org/apache/cassandra/net/MessagingService.java
index c54cdec,e12f9ee..d7613cb
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@@ -513,23 -464,15 +514,23 @@@ public final class MessagingService imp
  subscribers.add(subcriber);
  }
  
 -public void waitForStreaming() throws InterruptedException
 +public void clearCallbacksUnsafe()
  {
- callbacks.clear();
 -streamExecutor_.shutdown();
 -streamExecutor_.awaitTermination(24, TimeUnit.HOURS);
++callbacks.reset();
  }
  
 -public void clearCallbacksUnsafe()
 +public void waitForStreaming() throws InterruptedException
  {
 -callbacks.reset();
 +// this does not prevent new streams from beginning after a drain 
begins, but since streams are only
 +// started in response to explicit operator action 
(bootstrap/move/repair/etc) that feels like a feature.
 +for (DebuggableThreadPoolExecutor e : streamExecutors.values())
 +e.shutdown();
 +
 +for (DebuggableThreadPoolExecutor e : streamExecutors.values())
 +{
 +if (e.awaitTermination(24, TimeUnit.HOURS))
 +logger_.error(Stream took more than 24H to complete; 
skipping);
 +}
  }
  
  /**

[4/4] git commit: attempt to humor tests that try to stop and restart MS

2012-01-11 Thread jbellis

attempt to humor tests that try to stop and restart MS


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/452ddf63
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/452ddf63
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/452ddf63

Branch: refs/heads/trunk
Commit: 452ddf63c530fc573551e6fc9c79c1a876f11dd0
Parents: a95eafd
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Jan 11 16:59:33 2012 -0600
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Jan 11 16:59:33 2012 -0600

--
 .../org/apache/cassandra/net/MessagingService.java |   14 ++
 1 files changed, 14 insertions(+), 0 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/452ddf63/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java 
b/src/java/org/apache/cassandra/net/MessagingService.java
index 9ff110e..99037a1 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -482,7 +482,21 @@ public final class MessagingService implements 
MessagingServiceMBean
 logger_.info(Waiting for messaging service to quiesce);
 // We may need to schedule hints on the mutation stage, so it's 
erroneous to shut down the mutation stage first
 assert !StageManager.getStage(Stage.MUTATION).isShutdown();
+
+// the important part
 callbacks.shutdown();
+
+// attempt to humor tests that try to stop and restart MS
+try
+{
+for (SocketThread th : socketThreads)
+th.close();
+}
+catch (IOException e)
+{
+throw new IOError(e);
+}
+
 }
 
 public void receive(Message message, String id)

81 matches

Mail list logo