[jira] [Commented] (CASSANDRA-5019) Still too much object allocation on reads
[ https://issues.apache.org/jira/browse/CASSANDRA-5019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536849#comment-13536849 ] Sylvain Lebresne commented on CASSANDRA-5019: - And I don't think the replace we have is any problem for the flyweight pattern, since all we support is replacing a column by another with the same name. I.e. this is really an in-place replace only. > Still too much object allocation on reads > - > > Key: CASSANDRA-5019 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5019 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis > Fix For: 2.0 > > > ArrayBackedSortedColumns was a step in the right direction but it's still > relatively heavyweight thanks to allocating individual Columns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4564) MoveTest madness
[
https://issues.apache.org/jira/browse/CASSANDRA-4564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu Yu updated CASSANDRA-4564:
--
Attachment: cassandra-1.1-4564.txt
> MoveTest madness
>
>
> Key: CASSANDRA-4564
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4564
> Project: Cassandra
> Issue Type: Bug
> Components: Tests
>Reporter: Eric Evans
> Labels: lhf
> Attachments: cassandra-1.1-4564.txt
>
>
> I encountered what looks like bugs in
> {{o.a.c.service.MoveTest.newTestWriteEndpointsDuringMove()}} while doing
> something else; Here is a (poorly researched )ticket before I forget :)
> * There are two loops over non-system tables, and the first is a NOOP
> * In the second loop, a set exactly {{replicationFactor}} in size is compared
> against {{tmd.getWriteEndpoints()}}, which should produce greater than
> {{replicationFactor}} endpoints during a move (shouldn't it?); How does this
> pass?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4564) MoveTest madness
[
https://issues.apache.org/jira/browse/CASSANDRA-4564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536815#comment-13536815
]
Liu Yu commented on CASSANDRA-4564:
---
For the second issue, the root cause is: the key token is endpointToken+5, and
the endpoint token is moved to endpointToken+2, so there is no key need to be
moved. I have changed the move token to enpointToken+7 and updated the test
cases.
> MoveTest madness
>
>
> Key: CASSANDRA-4564
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4564
> Project: Cassandra
> Issue Type: Bug
> Components: Tests
>Reporter: Eric Evans
> Labels: lhf
>
> I encountered what looks like bugs in
> {{o.a.c.service.MoveTest.newTestWriteEndpointsDuringMove()}} while doing
> something else; Here is a (poorly researched )ticket before I forget :)
> * There are two loops over non-system tables, and the first is a NOOP
> * In the second loop, a set exactly {{replicationFactor}} in size is compared
> against {{tmd.getWriteEndpoints()}}, which should produce greater than
> {{replicationFactor}} endpoints during a move (shouldn't it?); How does this
> pass?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5079) Compaction deletes ExpiringColumns in Secondary Indexes
[
https://issues.apache.org/jira/browse/CASSANDRA-5079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-5079:
--
Reviewer: slebresne
Affects Version/s: (was: 1.2.0 rc1)
(was: 1.1.7)
0.7.0
Fix Version/s: 1.2.0 rc2
Thanks Aaron, that looks like good detective work.
(Setting affects-version to when 2I were added. Please correct if this was a
more recent regression.)
> Compaction deletes ExpiringColumns in Secondary Indexes
> ---
>
> Key: CASSANDRA-5079
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5079
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.7.0
>Reporter: amorton
>Assignee: amorton
> Fix For: 1.2.0 rc2
>
> Attachments: 5079.txt
>
>
> From this discussion
> http://www.mail-archive.com/user@cassandra.apache.org/msg26599.html
> CompactionManager.getDefaultGcBefore() set's the gc_before to be
> Integer.MAX_VALUE.
> In the example all entries in the secondary index have a TTL. In
> PreCompactedRow.removeDeletedAndOldShards() the CF is determined to have
> irrelevant data, the call to CFS.removeDeleted() results in the
> ExpiringColumns being removed and the row is treated as empty.
> CompactionTask.execute() exits at the {{if (!nni.hasNext())}} test, so the
> sstables are marked as compacted and soon deleted.
> In the example the localDeletionTime was Thu, 21 Mar 2013 08:25:22 GMT and
> should not have been purged.
> In the example when the first compaction on the secondary index runs the on
> disk data for the index is deleted. The logs show a compaction starting and
> no associated "Compacted to" message from that compaction thread.
> The impact is incorrect results from secondary indexes queries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2819) Split rpc timeout for read and write ops
[ https://issues.apache.org/jira/browse/CASSANDRA-2819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536780#comment-13536780 ] Jonathan Ellis commented on CASSANDRA-2819: --- Not at this point. 1.1 is very close to becoming "old-stable" with the release of 1.2. > Split rpc timeout for read and write ops > > > Key: CASSANDRA-2819 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2819 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Stu Hood >Assignee: Jonathan Ellis > Fix For: 1.2.0 beta 1 > > Attachments: 0001-additional-fix-for-2819.patch, 2819-v4.txt, > 2819-v5-rebased.txt, 2819-v8.txt, 2819-v9.txt, c2819.patch, c2819-v6, > c2819-v7, rpc-jira.patch > > > Given the vastly different latency characteristics of reads and writes, it > makes sense for them to have independent rpc timeouts internally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5079) Compaction deletes ExpiringColumns in Secondary Indexes
[
https://issues.apache.org/jira/browse/CASSANDRA-5079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
amorton updated CASSANDRA-5079:
---
Attachment: 5079.txt
> Compaction deletes ExpiringColumns in Secondary Indexes
> ---
>
> Key: CASSANDRA-5079
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5079
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 1.1.7, 1.2.0 rc1
>Reporter: amorton
>Assignee: amorton
> Attachments: 5079.txt
>
>
> From this discussion
> http://www.mail-archive.com/user@cassandra.apache.org/msg26599.html
> CompactionManager.getDefaultGcBefore() set's the gc_before to be
> Integer.MAX_VALUE.
> In the example all entries in the secondary index have a TTL. In
> PreCompactedRow.removeDeletedAndOldShards() the CF is determined to have
> irrelevant data, the call to CFS.removeDeleted() results in the
> ExpiringColumns being removed and the row is treated as empty.
> CompactionTask.execute() exits at the {{if (!nni.hasNext())}} test, so the
> sstables are marked as compacted and soon deleted.
> In the example the localDeletionTime was Thu, 21 Mar 2013 08:25:22 GMT and
> should not have been purged.
> In the example when the first compaction on the secondary index runs the on
> disk data for the index is deleted. The logs show a compaction starting and
> no associated "Compacted to" message from that compaction thread.
> The impact is incorrect results from secondary indexes queries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5079) Compaction deletes ExpiringColumns in Secondary Indexes
amorton created CASSANDRA-5079:
--
Summary: Compaction deletes ExpiringColumns in Secondary Indexes
Key: CASSANDRA-5079
URL: https://issues.apache.org/jira/browse/CASSANDRA-5079
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.2.0 rc1, 1.1.7
Reporter: amorton
Assignee: amorton
>From this discussion
>http://www.mail-archive.com/user@cassandra.apache.org/msg26599.html
CompactionManager.getDefaultGcBefore() set's the gc_before to be
Integer.MAX_VALUE.
In the example all entries in the secondary index have a TTL. In
PreCompactedRow.removeDeletedAndOldShards() the CF is determined to have
irrelevant data, the call to CFS.removeDeleted() results in the ExpiringColumns
being removed and the row is treated as empty. CompactionTask.execute() exits
at the {{if (!nni.hasNext())}} test, so the sstables are marked as compacted
and soon deleted.
In the example the localDeletionTime was Thu, 21 Mar 2013 08:25:22 GMT and
should not have been purged.
In the example when the first compaction on the secondary index runs the on
disk data for the index is deleted. The logs show a compaction starting and no
associated "Compacted to" message from that compaction thread.
The impact is incorrect results from secondary indexes queries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2819) Split rpc timeout for read and write ops
[ https://issues.apache.org/jira/browse/CASSANDRA-2819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536723#comment-13536723 ] Shotaro Kamio commented on CASSANDRA-2819: -- Can we expect backport of patches to 1.1.x? > Split rpc timeout for read and write ops > > > Key: CASSANDRA-2819 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2819 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Stu Hood >Assignee: Jonathan Ellis > Fix For: 1.2.0 beta 1 > > Attachments: 0001-additional-fix-for-2819.patch, 2819-v4.txt, > 2819-v5-rebased.txt, 2819-v8.txt, 2819-v9.txt, c2819.patch, c2819-v6, > c2819-v7, rpc-jira.patch > > > Given the vastly different latency characteristics of reads and writes, it > makes sense for them to have independent rpc timeouts internally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5044) Allow resizing of threadpools via JMX
[ https://issues.apache.org/jira/browse/CASSANDRA-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536658#comment-13536658 ] Carl Yeksigian commented on CASSANDRA-5044: --- We've been using this quite a bit to change the number of compaction threads while doing our development stage where we are switching between writing and reading. > Allow resizing of threadpools via JMX > - > > Key: CASSANDRA-5044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5044 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Carl Yeksigian >Assignee: Carl Yeksigian >Priority: Minor > Fix For: 1.2.1 > > Attachments: 5044.txt > > > Currently changing the size of threadpools is done using the config files. If > resizing a threadpool, the node needs to be taken offline. > Adding resizing via JMX allows online tweaking. TPEs allow for moving the > number of threads up and down; moving down will take effect after a thread in > the pool becomes idle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
buildbot failure in ASF Buildbot on cassandra-trunk
The Buildbot has detected a new failure on builder cassandra-trunk while building cassandra. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/2202 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: portunus_ubuntu Build Reason: scheduler Build Source Stamp: [branch trunk] dd14505f150c5543cf6853f71b93aef8817dd7b1 Blamelist: Jonathan Ellis BUILD FAILED: failed shell sincerely, -The Buildbot
[jira] [Commented] (CASSANDRA-5077) Simplify CompactionIterable.getReduced
[ https://issues.apache.org/jira/browse/CASSANDRA-5077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536542#comment-13536542 ] Jonathan Ellis commented on CASSANDRA-5077: --- committed! > Simplify CompactionIterable.getReduced > -- > > Key: CASSANDRA-5077 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5077 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis > Fix For: 1.2.1 > > Attachments: 5077.txt, 5077.txt > > > CompactionIterable turns empty rows into null, and > purge-compacted-rows-from-cache logic is duplicated (and different!) across > implementations. Let's simplify to just return empty rows as-is and let > CompactionTask handle the purge logic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[2/3] git commit: simplify CompactionIterable.getReduced patch by jbellis; reviewed by Carl Yeksigian for CASSANDRA-5077
simplify CompactionIterable.getReduced
patch by jbellis; reviewed by Carl Yeksigian for CASSANDRA-5077
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4e3d0491
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4e3d0491
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4e3d0491
Branch: refs/heads/trunk
Commit: 4e3d0491514225980972a2253330acf255687acd
Parents: f252842
Author: Jonathan Ellis
Authored: Wed Dec 19 16:52:18 2012 -0600
Committer: Jonathan Ellis
Committed: Wed Dec 19 16:54:22 2012 -0600
--
.../db/compaction/CompactionIterable.java | 15 +++---
.../cassandra/db/compaction/CompactionTask.java| 15 ++
.../db/compaction/ParallelCompactionIterable.java | 22 +--
.../cassandra/db/compaction/PrecompactedRow.java |2 -
4 files changed, 14 insertions(+), 40 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4e3d0491/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
b/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
index 9f4f7a0..32b4942 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
@@ -70,17 +70,10 @@ public class CompactionIterable extends
AbstractCompactionIterable
CompactionIterable.this.updateCounterFor(rows.size());
try
{
-AbstractCompactedRow compactedRow =
controller.getCompactedRow(new ArrayList(rows));
-if (compactedRow.isEmpty())
-{
-controller.invalidateCachedRow(compactedRow.key);
-return null;
-}
-
-// If the row is cached, we call removeDeleted on at read time
it to have coherent query returns,
-// but if the row is not pushed out of the cache, obsolete
tombstones will persist indefinitely.
-controller.removeDeletedInCache(compactedRow.key);
-return compactedRow;
+// create a new container for rows, since we're going to clear
ours for the next one,
+// and the AbstractCompactionRow code should be able to assume
that the collection it receives
+// won't be pulled out from under it.
+return controller.getCompactedRow(new
ArrayList(rows));
}
finally
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4e3d0491/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 481b7f4..3f6901a 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@ -129,7 +129,6 @@ public class CompactionTask extends AbstractCompactionTask
? new
ParallelCompactionIterable(compactionType, strategy.getScanners(toCompact),
controller)
: new CompactionIterable(compactionType,
strategy.getScanners(toCompact), controller);
CloseableIterator iter = ci.iterator();
-Iterator nni = Iterators.filter(iter,
Predicates.notNull());
Map cachedKeys = new
HashMap();
// we can't preheat until the tracker has been set. This doesn't
happen until we tell the cfs to
@@ -143,7 +142,7 @@ public class CompactionTask extends AbstractCompactionTask
collector.beginCompaction(ci);
try
{
-if (!nni.hasNext())
+if (!iter.hasNext())
{
// don't mark compacted in the finally block, since if there
_is_ nondeleted data,
// we need to sync it (via closeAndOpen) first, so there is no
period during which
@@ -154,17 +153,21 @@ public class CompactionTask extends AbstractCompactionTask
SSTableWriter writer = cfs.createCompactionWriter(keysPerSSTable,
cfs.directories.getLocationForDisk(dataDirectory), toCompact);
writers.add(writer);
-while (nni.hasNext())
+while (iter.hasNext())
{
if (ci.isStopRequested())
throw new
CompactionInterruptedException(ci.getCompactionInfo());
-AbstractCompactedRow row = nni.next();
+AbstractCompactedRow row = iter.next();
if (row.
[1/3] git commit: Merge branch 'cassandra-1.2' into trunk
Updated Branches: refs/heads/cassandra-1.2 f25284220 -> 4e3d04915 refs/heads/trunk 1e80bb899 -> dd14505f1 Merge branch 'cassandra-1.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dd14505f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dd14505f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dd14505f Branch: refs/heads/trunk Commit: dd14505f150c5543cf6853f71b93aef8817dd7b1 Parents: 1e80bb8 4e3d049 Author: Jonathan Ellis Authored: Wed Dec 19 16:54:50 2012 -0600 Committer: Jonathan Ellis Committed: Wed Dec 19 16:54:50 2012 -0600 -- .../db/compaction/CompactionIterable.java | 15 +++--- .../cassandra/db/compaction/CompactionTask.java| 15 ++ .../db/compaction/ParallelCompactionIterable.java | 22 +-- .../cassandra/db/compaction/PrecompactedRow.java |2 - 4 files changed, 14 insertions(+), 40 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/dd14505f/src/java/org/apache/cassandra/db/compaction/CompactionTask.java --
[3/3] git commit: simplify CompactionIterable.getReduced patch by jbellis; reviewed by Carl Yeksigian for CASSANDRA-5077
simplify CompactionIterable.getReduced
patch by jbellis; reviewed by Carl Yeksigian for CASSANDRA-5077
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4e3d0491
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4e3d0491
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4e3d0491
Branch: refs/heads/cassandra-1.2
Commit: 4e3d0491514225980972a2253330acf255687acd
Parents: f252842
Author: Jonathan Ellis
Authored: Wed Dec 19 16:52:18 2012 -0600
Committer: Jonathan Ellis
Committed: Wed Dec 19 16:54:22 2012 -0600
--
.../db/compaction/CompactionIterable.java | 15 +++---
.../cassandra/db/compaction/CompactionTask.java| 15 ++
.../db/compaction/ParallelCompactionIterable.java | 22 +--
.../cassandra/db/compaction/PrecompactedRow.java |2 -
4 files changed, 14 insertions(+), 40 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4e3d0491/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
b/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
index 9f4f7a0..32b4942 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
@@ -70,17 +70,10 @@ public class CompactionIterable extends
AbstractCompactionIterable
CompactionIterable.this.updateCounterFor(rows.size());
try
{
-AbstractCompactedRow compactedRow =
controller.getCompactedRow(new ArrayList(rows));
-if (compactedRow.isEmpty())
-{
-controller.invalidateCachedRow(compactedRow.key);
-return null;
-}
-
-// If the row is cached, we call removeDeleted on at read time
it to have coherent query returns,
-// but if the row is not pushed out of the cache, obsolete
tombstones will persist indefinitely.
-controller.removeDeletedInCache(compactedRow.key);
-return compactedRow;
+// create a new container for rows, since we're going to clear
ours for the next one,
+// and the AbstractCompactionRow code should be able to assume
that the collection it receives
+// won't be pulled out from under it.
+return controller.getCompactedRow(new
ArrayList(rows));
}
finally
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4e3d0491/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 481b7f4..3f6901a 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@ -129,7 +129,6 @@ public class CompactionTask extends AbstractCompactionTask
? new
ParallelCompactionIterable(compactionType, strategy.getScanners(toCompact),
controller)
: new CompactionIterable(compactionType,
strategy.getScanners(toCompact), controller);
CloseableIterator iter = ci.iterator();
-Iterator nni = Iterators.filter(iter,
Predicates.notNull());
Map cachedKeys = new
HashMap();
// we can't preheat until the tracker has been set. This doesn't
happen until we tell the cfs to
@@ -143,7 +142,7 @@ public class CompactionTask extends AbstractCompactionTask
collector.beginCompaction(ci);
try
{
-if (!nni.hasNext())
+if (!iter.hasNext())
{
// don't mark compacted in the finally block, since if there
_is_ nondeleted data,
// we need to sync it (via closeAndOpen) first, so there is no
period during which
@@ -154,17 +153,21 @@ public class CompactionTask extends AbstractCompactionTask
SSTableWriter writer = cfs.createCompactionWriter(keysPerSSTable,
cfs.directories.getLocationForDisk(dataDirectory), toCompact);
writers.add(writer);
-while (nni.hasNext())
+while (iter.hasNext())
{
if (ci.isStopRequested())
throw new
CompactionInterruptedException(ci.getCompactionInfo());
-AbstractCompactedRow row = nni.next();
+AbstractCompactedRow row = iter.next();
[jira] [Commented] (CASSANDRA-4897) Allow tiered compaction define max sstable size
[ https://issues.apache.org/jira/browse/CASSANDRA-4897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536522#comment-13536522 ] Jonathan Ellis commented on CASSANDRA-4897: --- Pushed this with some cleanup of the CompactionTask heirarchy to http://github.com/jbellis/cassandra/branches/4897 > Allow tiered compaction define max sstable size > --- > > Key: CASSANDRA-4897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4897 > Project: Cassandra > Issue Type: Improvement >Reporter: Radim Kolar >Assignee: Radim Kolar > Fix For: 1.2.1 > > Attachments: cass-maxsize1.txt, cass-maxsize2.txt > > > Lucene is doing same thing. Correctly configured max segment size will > recycle old data faster with less diskspace. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4897) Allow tiered compaction define max sstable size
[ https://issues.apache.org/jira/browse/CASSANDRA-4897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-4897: -- Reviewer: yukim Fix Version/s: 1.2.1 Assignee: Radim Kolar > Allow tiered compaction define max sstable size > --- > > Key: CASSANDRA-4897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4897 > Project: Cassandra > Issue Type: Improvement >Reporter: Radim Kolar >Assignee: Radim Kolar > Fix For: 1.2.1 > > Attachments: cass-maxsize1.txt, cass-maxsize2.txt > > > Lucene is doing same thing. Correctly configured max segment size will > recycle old data faster with less diskspace. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4598) describeOwnership() in Murmur3Partitioner.java doesn't work for close tokens
[
https://issues.apache.org/jira/browse/CASSANDRA-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuki Morishita resolved CASSANDRA-4598.
---
Resolution: Fixed
Fix Version/s: 1.2.0 rc2
We fixed desribeOwnership bug in CASSANDRA-5076, and the above issue was
resolved also. Closing this as 'Fixed'.
> describeOwnership() in Murmur3Partitioner.java doesn't work for close tokens
>
>
> Key: CASSANDRA-4598
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4598
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Julien Lambert
>Priority: Minor
> Fix For: 1.2.0 rc2
>
>
> On a 2 node-cluster, if the two tokens are close enough, the ownership
> information displayed will be 0.00% for each, instead of ~0% for one and
> ~100% for the other. The number of replicas displayed is then 0, even if you
> have more.
> Reproduce:
> - Create a 2-node cluster, using Murmur3Partitioner
> - Move the tokens to two consecutive values
> - Display ring with nodetool
> Problem:
> This line causing this problem is in {{describeOwnership()}} of
> {{Murmur3Partitioner.java}} (lines 117 and 123):
> {{float age = ((ti - tim1 + ri) % ri) / ri;}}
> If {{ti - tim1}} (the difference of the two consecutive tokens) is too small,
> then the precision of the float isn't enough to represent the exact numbers
> (because {{ri}}, the total range of the ring, is a very big number).
> For example, {{(float) (ri + 1) = (float) (ri - 1) = (float) ri =
> 9.223372E18}}, so that {{((ri+1)%ri)/ri = ((ri-1)%ri)/ri = (ri%ri)/ri = 0}}.
> Whereas with a correct precision, the exact value for {{(ri-1)%ri}} should be
> {{ri-1}} and {{(ri-1)/ri ~ 1.0 (100%)}} instead of 0%.
> Also, as the number of replica is determined by NodeCmd using the ownership
> percentages, it is wrong too.
> Solution:
> We might want to use BigInteger or BigDecimal somewhere?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3920) tests for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-3920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-3920: - Fix Version/s: 1.2.1 > tests for cqlsh > --- > > Key: CASSANDRA-3920 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3920 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: paul cannon >Assignee: paul cannon >Priority: Minor > Labels: cqlsh > Fix For: 1.2.1 > > Attachments: 3920-v1.01.txt > > > Cqlsh has become big enough and tries to cover enough situations that it's > time to start acting like a responsible adult and make this bugger some unit > tests to guard against regressions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
buildbot success in ASF Buildbot on cassandra-trunk
The Buildbot has detected a restored build on builder cassandra-trunk while building cassandra. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/2201 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: portunus_ubuntu Build Reason: scheduler Build Source Stamp: [branch trunk] 1e80bb89992cac590b01f0faea3c9dc6944d1ea0 Blamelist: Sylvain Lebresne ,Yuki Morishita Build succeeded! sincerely, -The Buildbot
[jira] [Comment Edited] (CASSANDRA-4894) log number of combined/merged rows during a compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536057#comment-13536057
]
Jonathan Ellis edited comment on CASSANDRA-4894 at 12/19/12 9:16 PM:
-
How about this?
Compacted 4 sstables to
[/Users/yuki/.ccm/1.2/node1/data/system/local/system-local-ia-18-Data.db,]. 957
bytes to 629 (~65% of original) in 35ms = 0.017139MB/s. 4 total rows, 1 unique.
Row merge counts are {1:0, 2:0, 3:0, 4:1}
was (Author: jbellis):
How about this?
Compacted 4 sstables to
[/Users/yuki/.ccm/1.2/node1/data/system/local/system-local-ia-18-Data.db,]. 957
bytes to 629 (~65% of original) in 35ms = 0.017139MB/s. 4 total rows, 1 unique.
Merge counts {1:0, 2:0, 3:0, 4:1}
> log number of combined/merged rows during a compaction
> --
>
> Key: CASSANDRA-4894
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4894
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Matthew F. Dennis
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 1.2.1
>
> Attachments: 4894-1.2.txt
>
>
> we already log some details about compactions but it would be useful to know
> how many rows were merged (resulting in "useful" work) and how many were
> unique (representing "wasted work").
> the simple approach requires two additional counters (one for unique rows,
> one for merged rows). As the merge join is progressing if two or more rows
> are combined, tick the joined counter. If a row is simply copied tick the
> unique counter.
> a more complete solution would be to keep a separate count for each number of
> merges. This would require number_of_files_being_merged counters. If no
> rows were merged, tick counters[0], if two rows were merged tick counters[1]
> and so on
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5019) Still too much object allocation on reads
[ https://issues.apache.org/jira/browse/CASSANDRA-5019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536426#comment-13536426 ] Jonathan Ellis commented on CASSANDRA-5019: --- Per the discussion on CASSANDRA-5043, add doesn't need to support out-of-order; only replace does. > Still too much object allocation on reads > - > > Key: CASSANDRA-5019 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5019 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis > Fix For: 2.0 > > > ArrayBackedSortedColumns was a step in the right direction but it's still > relatively heavyweight thanks to allocating individual Columns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[10/10] git commit: Update version to 1.2.0-rc2
Update version to 1.2.0-rc2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2c6b8ee1 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2c6b8ee1 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2c6b8ee1 Branch: refs/heads/cassandra-1.2 Commit: 2c6b8ee16b27404ece19ba3abfb443e410cb0adb Parents: 620a4b7 Author: Sylvain Lebresne Authored: Wed Dec 19 19:24:17 2012 +0100 Committer: Sylvain Lebresne Committed: Wed Dec 19 19:24:17 2012 +0100 -- build.xml|2 +- debian/changelog |6 ++ 2 files changed, 7 insertions(+), 1 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c6b8ee1/build.xml -- diff --git a/build.xml b/build.xml index e9b8e78..eca789a 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/> http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c6b8ee1/debian/changelog -- diff --git a/debian/changelog b/debian/changelog index cfa1781..0064b47 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (1.2.0~rc2) unstable; urgency=low + + * New RC release + + -- Sylvain Lebresne Wed, 19 Dec 2012 18:06:21 +0100 + cassandra (1.2.0~rc1) unstable; urgency=low * New RC release
[8/10] git commit: Merge branch 'cassandra-1.2.0' of https://git-wip-us.apache.org/repos/asf/cassandra into cassandra-1.2.0
Merge branch 'cassandra-1.2.0' of https://git-wip-us.apache.org/repos/asf/cassandra into cassandra-1.2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e2715f9e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e2715f9e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e2715f9e Branch: refs/heads/cassandra-1.2 Commit: e2715f9eaa9480fb316679a83404489dacfba74c Parents: 2c6b8ee e7ae7c9 Author: Sylvain Lebresne Authored: Wed Dec 19 19:49:24 2012 +0100 Committer: Sylvain Lebresne Committed: Wed Dec 19 19:49:24 2012 +0100 -- CHANGES.txt|1 + .../apache/cassandra/dht/Murmur3Partitioner.java | 14 +++-- .../apache/cassandra/dht/PartitionerTestCase.java | 40 ++- 3 files changed, 48 insertions(+), 7 deletions(-) --
[9/10] git commit: Update version to 1.2.0-rc2
Update version to 1.2.0-rc2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2c6b8ee1 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2c6b8ee1 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2c6b8ee1 Branch: refs/heads/trunk Commit: 2c6b8ee16b27404ece19ba3abfb443e410cb0adb Parents: 620a4b7 Author: Sylvain Lebresne Authored: Wed Dec 19 19:24:17 2012 +0100 Committer: Sylvain Lebresne Committed: Wed Dec 19 19:24:17 2012 +0100 -- build.xml|2 +- debian/changelog |6 ++ 2 files changed, 7 insertions(+), 1 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c6b8ee1/build.xml -- diff --git a/build.xml b/build.xml index e9b8e78..eca789a 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/> http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c6b8ee1/debian/changelog -- diff --git a/debian/changelog b/debian/changelog index cfa1781..0064b47 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (1.2.0~rc2) unstable; urgency=low + + * New RC release + + -- Sylvain Lebresne Wed, 19 Dec 2012 18:06:21 +0100 + cassandra (1.2.0~rc1) unstable; urgency=low * New RC release
[5/10] git commit: make sure OrderPreservingPartitionerTest runs clean
make sure OrderPreservingPartitionerTest runs clean
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/45310c06
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/45310c06
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/45310c06
Branch: refs/heads/cassandra-1.2.0
Commit: 45310c068fcc8868e3d4bdb877b2b62ec1bfdb98
Parents: e2715f9
Author: Yuki Morishita
Authored: Wed Dec 19 15:06:11 2012 -0600
Committer: Yuki Morishita
Committed: Wed Dec 19 15:06:11 2012 -0600
--
.../dht/OrderPreservingPartitionerTest.java|7 ++-
1 files changed, 6 insertions(+), 1 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/45310c06/test/unit/org/apache/cassandra/dht/OrderPreservingPartitionerTest.java
--
diff --git
a/test/unit/org/apache/cassandra/dht/OrderPreservingPartitionerTest.java
b/test/unit/org/apache/cassandra/dht/OrderPreservingPartitionerTest.java
index 6b3aed2..80021fd 100644
--- a/test/unit/org/apache/cassandra/dht/OrderPreservingPartitionerTest.java
+++ b/test/unit/org/apache/cassandra/dht/OrderPreservingPartitionerTest.java
@@ -20,10 +20,15 @@ package org.apache.cassandra.dht;
import org.junit.Test;
-public class OrderPreservingPartitionerTest extends
PartitionerTestCase {
+import org.apache.cassandra.SchemaLoader;
+
+public class OrderPreservingPartitionerTest extends
PartitionerTestCase
+{
public void initPartitioner()
{
partitioner = new OrderPreservingPartitioner();
+// need to clear data dir
+SchemaLoader.cleanupAndLeaveDirs();
}
@Test
[3/10] git commit: Merge branch 'cassandra-1.2.0' into cassandra-1.2
Merge branch 'cassandra-1.2.0' into cassandra-1.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f2528422 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f2528422 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f2528422 Branch: refs/heads/trunk Commit: f25284220ca2ba0bd28e753b4a1d554e468b85b0 Parents: de2495c 45310c0 Author: Yuki Morishita Authored: Wed Dec 19 15:06:30 2012 -0600 Committer: Yuki Morishita Committed: Wed Dec 19 15:06:30 2012 -0600 -- build.xml |2 +- debian/changelog |6 ++ .../dht/OrderPreservingPartitionerTest.java|7 ++- 3 files changed, 13 insertions(+), 2 deletions(-) --
[6/10] git commit: make sure OrderPreservingPartitionerTest runs clean
make sure OrderPreservingPartitionerTest runs clean
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/45310c06
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/45310c06
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/45310c06
Branch: refs/heads/cassandra-1.2
Commit: 45310c068fcc8868e3d4bdb877b2b62ec1bfdb98
Parents: e2715f9
Author: Yuki Morishita
Authored: Wed Dec 19 15:06:11 2012 -0600
Committer: Yuki Morishita
Committed: Wed Dec 19 15:06:11 2012 -0600
--
.../dht/OrderPreservingPartitionerTest.java|7 ++-
1 files changed, 6 insertions(+), 1 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/45310c06/test/unit/org/apache/cassandra/dht/OrderPreservingPartitionerTest.java
--
diff --git
a/test/unit/org/apache/cassandra/dht/OrderPreservingPartitionerTest.java
b/test/unit/org/apache/cassandra/dht/OrderPreservingPartitionerTest.java
index 6b3aed2..80021fd 100644
--- a/test/unit/org/apache/cassandra/dht/OrderPreservingPartitionerTest.java
+++ b/test/unit/org/apache/cassandra/dht/OrderPreservingPartitionerTest.java
@@ -20,10 +20,15 @@ package org.apache.cassandra.dht;
import org.junit.Test;
-public class OrderPreservingPartitionerTest extends
PartitionerTestCase {
+import org.apache.cassandra.SchemaLoader;
+
+public class OrderPreservingPartitionerTest extends
PartitionerTestCase
+{
public void initPartitioner()
{
partitioner = new OrderPreservingPartitioner();
+// need to clear data dir
+SchemaLoader.cleanupAndLeaveDirs();
}
@Test
[7/10] git commit: Merge branch 'cassandra-1.2.0' of https://git-wip-us.apache.org/repos/asf/cassandra into cassandra-1.2.0
Merge branch 'cassandra-1.2.0' of https://git-wip-us.apache.org/repos/asf/cassandra into cassandra-1.2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e2715f9e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e2715f9e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e2715f9e Branch: refs/heads/trunk Commit: e2715f9eaa9480fb316679a83404489dacfba74c Parents: 2c6b8ee e7ae7c9 Author: Sylvain Lebresne Authored: Wed Dec 19 19:49:24 2012 +0100 Committer: Sylvain Lebresne Committed: Wed Dec 19 19:49:24 2012 +0100 -- CHANGES.txt|1 + .../apache/cassandra/dht/Murmur3Partitioner.java | 14 +++-- .../apache/cassandra/dht/PartitionerTestCase.java | 40 ++- 3 files changed, 48 insertions(+), 7 deletions(-) --
[2/10] git commit: Merge branch 'cassandra-1.2.0' into cassandra-1.2
Merge branch 'cassandra-1.2.0' into cassandra-1.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f2528422 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f2528422 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f2528422 Branch: refs/heads/cassandra-1.2 Commit: f25284220ca2ba0bd28e753b4a1d554e468b85b0 Parents: de2495c 45310c0 Author: Yuki Morishita Authored: Wed Dec 19 15:06:30 2012 -0600 Committer: Yuki Morishita Committed: Wed Dec 19 15:06:30 2012 -0600 -- build.xml |2 +- debian/changelog |6 ++ .../dht/OrderPreservingPartitionerTest.java|7 ++- 3 files changed, 13 insertions(+), 2 deletions(-) --
[1/10] git commit: Merge branch 'cassandra-1.2' into trunk
Updated Branches: refs/heads/cassandra-1.2 de2495c2c -> f25284220 refs/heads/cassandra-1.2.0 e2715f9ea -> 45310c068 refs/heads/trunk ff92111c0 -> 1e80bb899 Merge branch 'cassandra-1.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1e80bb89 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1e80bb89 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1e80bb89 Branch: refs/heads/trunk Commit: 1e80bb89992cac590b01f0faea3c9dc6944d1ea0 Parents: ff92111 f252842 Author: Yuki Morishita Authored: Wed Dec 19 15:06:43 2012 -0600 Committer: Yuki Morishita Committed: Wed Dec 19 15:06:43 2012 -0600 -- build.xml |2 +- debian/changelog |6 ++ .../dht/OrderPreservingPartitionerTest.java|7 ++- 3 files changed, 13 insertions(+), 2 deletions(-) --
[4/10] git commit: make sure OrderPreservingPartitionerTest runs clean
make sure OrderPreservingPartitionerTest runs clean
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/45310c06
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/45310c06
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/45310c06
Branch: refs/heads/trunk
Commit: 45310c068fcc8868e3d4bdb877b2b62ec1bfdb98
Parents: e2715f9
Author: Yuki Morishita
Authored: Wed Dec 19 15:06:11 2012 -0600
Committer: Yuki Morishita
Committed: Wed Dec 19 15:06:11 2012 -0600
--
.../dht/OrderPreservingPartitionerTest.java|7 ++-
1 files changed, 6 insertions(+), 1 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/45310c06/test/unit/org/apache/cassandra/dht/OrderPreservingPartitionerTest.java
--
diff --git
a/test/unit/org/apache/cassandra/dht/OrderPreservingPartitionerTest.java
b/test/unit/org/apache/cassandra/dht/OrderPreservingPartitionerTest.java
index 6b3aed2..80021fd 100644
--- a/test/unit/org/apache/cassandra/dht/OrderPreservingPartitionerTest.java
+++ b/test/unit/org/apache/cassandra/dht/OrderPreservingPartitionerTest.java
@@ -20,10 +20,15 @@ package org.apache.cassandra.dht;
import org.junit.Test;
-public class OrderPreservingPartitionerTest extends
PartitionerTestCase {
+import org.apache.cassandra.SchemaLoader;
+
+public class OrderPreservingPartitionerTest extends
PartitionerTestCase
+{
public void initPartitioner()
{
partitioner = new OrderPreservingPartitioner();
+// need to clear data dir
+SchemaLoader.cleanupAndLeaveDirs();
}
@Test
[jira] [Commented] (CASSANDRA-4753) Timeout reporter writes hints for the local node
[ https://issues.apache.org/jira/browse/CASSANDRA-4753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536422#comment-13536422 ] Jonathan Ellis commented on CASSANDRA-4753: --- committed > Timeout reporter writes hints for the local node > > > Key: CASSANDRA-4753 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4753 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.2.0 beta 1 >Reporter: Aleksey Yeschenko >Assignee: Jonathan Ellis >Priority: Minor > Fix For: 1.2.1 > > Attachments: 4753.txt, 4753-v2.txt, 4753-v3.txt > > > MessagingService.java:330 calls StorageProxy.scheduleLocalHint() without > checking if the local node is the target. This causes > StorageProxy.scheduleLocalHint to throw AssertionError sometimes. > Got this error when some batches are timed out. This can happen because even > local batches now go through StorageProxy.sendMessages and aren't just > rm.apply()'d. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[2/3] git commit: fix hinting for dropped local writes patch by jbellis and aleksey for CASSANDRA-4753
fix hinting for dropped local writes
patch by jbellis and aleksey for CASSANDRA-4753
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/de2495c2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/de2495c2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/de2495c2
Branch: refs/heads/trunk
Commit: de2495c2c9bf8d8387d241df2962ae52f092b145
Parents: bebaf45
Author: Jonathan Ellis
Authored: Wed Dec 19 14:55:53 2012 -0600
Committer: Jonathan Ellis
Committed: Wed Dec 19 14:56:16 2012 -0600
--
CHANGES.txt|1 +
.../org/apache/cassandra/net/MessagingService.java |2 +-
.../org/apache/cassandra/service/StorageProxy.java | 187 ++-
3 files changed, 125 insertions(+), 65 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/de2495c2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 260aae5..d8caa14 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
1.2.1
+ * fix hinting for dropped local writes (CASSANDRA-4753)
* off-heap cache doesn't need mutable column container (CASSANDRA-5057)
* apply disk_failure_policy to bad disks on initial directory creation
(CASSANDRA-4847)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/de2495c2/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java
b/src/java/org/apache/cassandra/net/MessagingService.java
index 5329e1a..98495be 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -328,7 +328,7 @@ public final class MessagingService implements
MessagingServiceMBean
{
assert expiredCallbackInfo.sentMessage != null;
RowMutation rm = (RowMutation)
expiredCallbackInfo.sentMessage.payload;
-return StorageProxy.scheduleLocalHint(rm,
expiredCallbackInfo.target, null, null);
+return StorageProxy.submitHint(rm,
expiredCallbackInfo.target, null, null);
}
return null;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/de2495c2/src/java/org/apache/cassandra/service/StorageProxy.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageProxy.java
b/src/java/org/apache/cassandra/service/StorageProxy.java
index 1884132..fe427af 100644
--- a/src/java/org/apache/cassandra/service/StorageProxy.java
+++ b/src/java/org/apache/cassandra/service/StorageProxy.java
@@ -63,7 +63,6 @@ import org.apache.cassandra.tracing.Tracing;
import org.apache.cassandra.utils.ByteBufferUtil;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.Pair;
-import org.apache.cassandra.utils.WrappedRunnable;
public class StorageProxy implements StorageProxyMBean
{
@@ -314,16 +313,7 @@ public class StorageProxy implements StorageProxyMBean
Table.SYSTEM_KS,
null,
WriteType.BATCH_LOG);
-
-try
-{
-sendMessagesToOneDC(rm.createMessage(), endpoints, true, handler);
-}
-catch (IOException e)
-{
-throw new RuntimeException("Error writing to batchlog", e);
-}
-
+updateBatchlog(rm, endpoints, handler);
handler.get();
}
@@ -332,14 +322,19 @@ public class StorageProxy implements StorageProxyMBean
RowMutation rm = new RowMutation(Table.SYSTEM_KS,
UUIDType.instance.decompose(uuid));
rm.delete(new QueryPath(SystemTable.BATCHLOG_CF),
FBUtilities.timestampMicros());
AbstractWriteResponseHandler handler = new
WriteResponseHandler(endpoints, Collections.emptyList(),
ConsistencyLevel.ANY, Table.SYSTEM_KS, null, WriteType.SIMPLE);
+updateBatchlog(rm, endpoints, handler);
+}
-try
+private static void updateBatchlog(RowMutation rm, Collection
endpoints, AbstractWriteResponseHandler handler)
+{
+if (endpoints.contains(FBUtilities.getBroadcastAddress()))
{
-sendMessagesToOneDC(rm.createMessage(), endpoints, true, handler);
+assert endpoints.size() == 1;
+insertLocal(rm, handler);
}
-catch (IOException e)
+else
{
-throw new RuntimeException("Error deleting batch " + uuid, e);
+sendMessagesToOneDC(rm.cre
[3/3] git commit: fix hinting for dropped local writes patch by jbellis and aleksey for CASSANDRA-4753
fix hinting for dropped local writes
patch by jbellis and aleksey for CASSANDRA-4753
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/de2495c2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/de2495c2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/de2495c2
Branch: refs/heads/cassandra-1.2
Commit: de2495c2c9bf8d8387d241df2962ae52f092b145
Parents: bebaf45
Author: Jonathan Ellis
Authored: Wed Dec 19 14:55:53 2012 -0600
Committer: Jonathan Ellis
Committed: Wed Dec 19 14:56:16 2012 -0600
--
CHANGES.txt|1 +
.../org/apache/cassandra/net/MessagingService.java |2 +-
.../org/apache/cassandra/service/StorageProxy.java | 187 ++-
3 files changed, 125 insertions(+), 65 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/de2495c2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 260aae5..d8caa14 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
1.2.1
+ * fix hinting for dropped local writes (CASSANDRA-4753)
* off-heap cache doesn't need mutable column container (CASSANDRA-5057)
* apply disk_failure_policy to bad disks on initial directory creation
(CASSANDRA-4847)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/de2495c2/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java
b/src/java/org/apache/cassandra/net/MessagingService.java
index 5329e1a..98495be 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -328,7 +328,7 @@ public final class MessagingService implements
MessagingServiceMBean
{
assert expiredCallbackInfo.sentMessage != null;
RowMutation rm = (RowMutation)
expiredCallbackInfo.sentMessage.payload;
-return StorageProxy.scheduleLocalHint(rm,
expiredCallbackInfo.target, null, null);
+return StorageProxy.submitHint(rm,
expiredCallbackInfo.target, null, null);
}
return null;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/de2495c2/src/java/org/apache/cassandra/service/StorageProxy.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageProxy.java
b/src/java/org/apache/cassandra/service/StorageProxy.java
index 1884132..fe427af 100644
--- a/src/java/org/apache/cassandra/service/StorageProxy.java
+++ b/src/java/org/apache/cassandra/service/StorageProxy.java
@@ -63,7 +63,6 @@ import org.apache.cassandra.tracing.Tracing;
import org.apache.cassandra.utils.ByteBufferUtil;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.Pair;
-import org.apache.cassandra.utils.WrappedRunnable;
public class StorageProxy implements StorageProxyMBean
{
@@ -314,16 +313,7 @@ public class StorageProxy implements StorageProxyMBean
Table.SYSTEM_KS,
null,
WriteType.BATCH_LOG);
-
-try
-{
-sendMessagesToOneDC(rm.createMessage(), endpoints, true, handler);
-}
-catch (IOException e)
-{
-throw new RuntimeException("Error writing to batchlog", e);
-}
-
+updateBatchlog(rm, endpoints, handler);
handler.get();
}
@@ -332,14 +322,19 @@ public class StorageProxy implements StorageProxyMBean
RowMutation rm = new RowMutation(Table.SYSTEM_KS,
UUIDType.instance.decompose(uuid));
rm.delete(new QueryPath(SystemTable.BATCHLOG_CF),
FBUtilities.timestampMicros());
AbstractWriteResponseHandler handler = new
WriteResponseHandler(endpoints, Collections.emptyList(),
ConsistencyLevel.ANY, Table.SYSTEM_KS, null, WriteType.SIMPLE);
+updateBatchlog(rm, endpoints, handler);
+}
-try
+private static void updateBatchlog(RowMutation rm, Collection
endpoints, AbstractWriteResponseHandler handler)
+{
+if (endpoints.contains(FBUtilities.getBroadcastAddress()))
{
-sendMessagesToOneDC(rm.createMessage(), endpoints, true, handler);
+assert endpoints.size() == 1;
+insertLocal(rm, handler);
}
-catch (IOException e)
+else
{
-throw new RuntimeException("Error deleting batch " + uuid, e);
+sendMessagesToOneD
[1/3] git commit: Merge branch 'cassandra-1.2' into trunk
Updated Branches: refs/heads/cassandra-1.2 bebaf453c -> de2495c2c refs/heads/trunk 8e7abe0ad -> ff92111c0 Merge branch 'cassandra-1.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ff92111c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ff92111c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ff92111c Branch: refs/heads/trunk Commit: ff92111c0164f3c9aef61295681f80645674402f Parents: 8e7abe0 de2495c Author: Jonathan Ellis Authored: Wed Dec 19 14:56:26 2012 -0600 Committer: Jonathan Ellis Committed: Wed Dec 19 14:56:26 2012 -0600 -- CHANGES.txt|1 + .../org/apache/cassandra/net/MessagingService.java |2 +- .../org/apache/cassandra/service/StorageProxy.java | 187 ++- 3 files changed, 125 insertions(+), 65 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff92111c/CHANGES.txt -- diff --cc CHANGES.txt index efda024,d8caa14..86fae7c --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,10 -1,5 +1,11 @@@ +1.3 + * make index_interval configurable per columnfamily (CASSANDRA-3961) + * add default_tim_to_live (CASSANDRA-3974) + * add memtable_flush_period_in_ms (CASSANDRA-4237) + + 1.2.1 + * fix hinting for dropped local writes (CASSANDRA-4753) * off-heap cache doesn't need mutable column container (CASSANDRA-5057) * apply disk_failure_policy to bad disks on initial directory creation (CASSANDRA-4847) http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff92111c/src/java/org/apache/cassandra/service/StorageProxy.java --
[jira] [Commented] (CASSANDRA-4753) Timeout reporter writes hints for the local node
[ https://issues.apache.org/jira/browse/CASSANDRA-4753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536417#comment-13536417 ] Aleksey Yeschenko commented on CASSANDRA-4753: -- +1 > Timeout reporter writes hints for the local node > > > Key: CASSANDRA-4753 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4753 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.2.0 beta 1 >Reporter: Aleksey Yeschenko >Assignee: Jonathan Ellis >Priority: Minor > Fix For: 1.2.1 > > Attachments: 4753.txt, 4753-v2.txt, 4753-v3.txt > > > MessagingService.java:330 calls StorageProxy.scheduleLocalHint() without > checking if the local node is the target. This causes > StorageProxy.scheduleLocalHint to throw AssertionError sometimes. > Got this error when some batches are timed out. This can happen because even > local batches now go through StorageProxy.sendMessages and aren't just > rm.apply()'d. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
buildbot failure in ASF Buildbot on cassandra-trunk
The Buildbot has detected a new failure on builder cassandra-trunk while building cassandra. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/2199 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: portunus_ubuntu Build Reason: scheduler Build Source Stamp: [branch trunk] 8e7abe0ada0ec3d06da99771a1c3af35e3affd52 Blamelist: Yuki Morishita BUILD FAILED: failed shell sincerely, -The Buildbot
Git Push Summary
Updated Tags: refs/tags/1.2.0-rc2-tentative [created] e2715f9ea
[1/2] git commit: Merge branch 'cassandra-1.2.0' of https://git-wip-us.apache.org/repos/asf/cassandra into cassandra-1.2.0
Updated Branches: refs/heads/cassandra-1.2.0 e7ae7c904 -> e2715f9ea Merge branch 'cassandra-1.2.0' of https://git-wip-us.apache.org/repos/asf/cassandra into cassandra-1.2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e2715f9e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e2715f9e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e2715f9e Branch: refs/heads/cassandra-1.2.0 Commit: e2715f9eaa9480fb316679a83404489dacfba74c Parents: 2c6b8ee e7ae7c9 Author: Sylvain Lebresne Authored: Wed Dec 19 19:49:24 2012 +0100 Committer: Sylvain Lebresne Committed: Wed Dec 19 19:49:24 2012 +0100 -- CHANGES.txt|1 + .../apache/cassandra/dht/Murmur3Partitioner.java | 14 +++-- .../apache/cassandra/dht/PartitionerTestCase.java | 40 ++- 3 files changed, 48 insertions(+), 7 deletions(-) --
[2/2] git commit: Update version to 1.2.0-rc2
Update version to 1.2.0-rc2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2c6b8ee1 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2c6b8ee1 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2c6b8ee1 Branch: refs/heads/cassandra-1.2.0 Commit: 2c6b8ee16b27404ece19ba3abfb443e410cb0adb Parents: 620a4b7 Author: Sylvain Lebresne Authored: Wed Dec 19 19:24:17 2012 +0100 Committer: Sylvain Lebresne Committed: Wed Dec 19 19:24:17 2012 +0100 -- build.xml|2 +- debian/changelog |6 ++ 2 files changed, 7 insertions(+), 1 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c6b8ee1/build.xml -- diff --git a/build.xml b/build.xml index e9b8e78..eca789a 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/> http://git-wip-us.apache.org/repos/asf/cassandra/blob/2c6b8ee1/debian/changelog -- diff --git a/debian/changelog b/debian/changelog index cfa1781..0064b47 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (1.2.0~rc2) unstable; urgency=low + + * New RC release + + -- Sylvain Lebresne Wed, 19 Dec 2012 18:06:21 +0100 + cassandra (1.2.0~rc1) unstable; urgency=low * New RC release
[1/6] git commit: Merge branch 'cassandra-1.2' into trunk
Updated Branches: refs/heads/cassandra-1.2 0207bdf5f -> bebaf453c refs/heads/cassandra-1.2.0 620a4b7a4 -> e7ae7c904 refs/heads/trunk 9bc9853e2 -> 8e7abe0ad Merge branch 'cassandra-1.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8e7abe0a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8e7abe0a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8e7abe0a Branch: refs/heads/trunk Commit: 8e7abe0ada0ec3d06da99771a1c3af35e3affd52 Parents: 9bc9853 bebaf45 Author: Yuki Morishita Authored: Wed Dec 19 12:43:42 2012 -0600 Committer: Yuki Morishita Committed: Wed Dec 19 12:43:42 2012 -0600 -- CHANGES.txt|1 + .../apache/cassandra/dht/Murmur3Partitioner.java | 14 +++-- .../apache/cassandra/dht/PartitionerTestCase.java | 40 ++- 3 files changed, 48 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/8e7abe0a/CHANGES.txt --
[4/6] git commit: Fix M3P ownership calculation; patch by yukim reviewed by Sylvain Lebresne for CASSANDRA-5076
Fix M3P ownership calculation; patch by yukim reviewed by Sylvain Lebresne for
CASSANDRA-5076
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e7ae7c90
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e7ae7c90
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e7ae7c90
Branch: refs/heads/trunk
Commit: e7ae7c9040135b6ecae5204efd7e41ad2e6ba4ee
Parents: 620a4b7
Author: Yuki Morishita
Authored: Wed Dec 19 12:43:23 2012 -0600
Committer: Yuki Morishita
Committed: Wed Dec 19 12:43:23 2012 -0600
--
CHANGES.txt|1 +
.../apache/cassandra/dht/Murmur3Partitioner.java | 14 +++--
.../apache/cassandra/dht/PartitionerTestCase.java | 40 ++-
3 files changed, 48 insertions(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7ae7c90/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6edf7bb..7786b09 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -11,6 +11,7 @@ Merged from 1.1:
* Fix ALTER TABLE overriding compression options with defaults
(CASSANDRA-4996, 5066)
* fix specifying and altering crc_check_chance (CASSANDRA-5053)
+ * fix Murmur3Partitioner ownership% calculation (CASSANDRA-5076)
1.2-rc1
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7ae7c90/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
--
diff --git a/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
b/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
index 3e98e32..ad3579a 100644
--- a/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
+++ b/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
@@ -17,6 +17,7 @@
*/
package org.apache.cassandra.dht;
+import java.math.BigDecimal;
import java.math.BigInteger;
import java.nio.ByteBuffer;
import java.util.HashMap;
@@ -125,20 +126,21 @@ public class Murmur3Partitioner extends
AbstractPartitioner
// n-case
else
{
-final float ri = MAXIMUM;
// (used for addition later)
-Token start = (Token) i.next(); Long ti =
((LongToken)start).token; // The first token and its value
-Token t; Long tim1 = ti;
// The last token and its value (after loop)
+final BigInteger ri =
BigInteger.valueOf(MAXIMUM).subtract(BigInteger.valueOf(MINIMUM.token + 1));
// (used for addition later)
+final BigDecimal r = new BigDecimal(ri);
+Token start = (Token) i.next();BigInteger ti =
BigInteger.valueOf(((LongToken)start).token); // The first token and its value
+Token t; BigInteger tim1 = ti;
// The last token and its value (after loop)
while (i.hasNext())
{
-t = (Token) i.next(); ti = ((LongToken) t).token; // The next
token and its value
-float age = ((ti - tim1 + ri) % ri) / ri; // %age =
((T(i) - T(i-1) + R) % R) / R
+t = (Token) i.next(); ti = BigInteger.valueOf(((LongToken)
t).token); // The next token and its value
+float age = new
BigDecimal(ti.subtract(tim1).add(ri).mod(ri)).divide(r, 6,
BigDecimal.ROUND_HALF_EVEN).floatValue(); // %age = ((T(i) - T(i-1) + R) % R) /
R
ownerships.put(t, age); // save
(T(i) -> %age)
tim1 = ti;// ->
advance loop
}
// The start token's range extends backward to the last token,
which is why both were saved above.
-float x = LongToken) start).token - ti + ri) % ri) / ri;
+float x = new
BigDecimal(BigInteger.valueOf(((LongToken)start).token).subtract(ti).add(ri).mod(ri)).divide(r,
6, BigDecimal.ROUND_HALF_EVEN).floatValue();
ownerships.put(start, x);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7ae7c90/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
--
diff --git a/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
b/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
index 720ba3e..79b0d78 100644
--- a/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
+++ b/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
@@ -19,11 +19,14 @@
package org.apache.cassandra.dht;
import java.nio.ByteBuffer;
-import java.util.Random;
+import java.util.*;
import org.junit.Before;
import org.junit.Test;
+impo
[3/6] git commit: Merge branch 'cassandra-1.2.0' into cassandra-1.2
Merge branch 'cassandra-1.2.0' into cassandra-1.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bebaf453 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bebaf453 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bebaf453 Branch: refs/heads/trunk Commit: bebaf453c8e0f21b1be6c2ed962ae2f3e3c77373 Parents: 0207bdf e7ae7c9 Author: Yuki Morishita Authored: Wed Dec 19 12:43:33 2012 -0600 Committer: Yuki Morishita Committed: Wed Dec 19 12:43:33 2012 -0600 -- CHANGES.txt|1 + .../apache/cassandra/dht/Murmur3Partitioner.java | 14 +++-- .../apache/cassandra/dht/PartitionerTestCase.java | 40 ++- 3 files changed, 48 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/bebaf453/CHANGES.txt --
[6/6] git commit: Fix M3P ownership calculation; patch by yukim reviewed by Sylvain Lebresne for CASSANDRA-5076
Fix M3P ownership calculation; patch by yukim reviewed by Sylvain Lebresne for
CASSANDRA-5076
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e7ae7c90
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e7ae7c90
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e7ae7c90
Branch: refs/heads/cassandra-1.2
Commit: e7ae7c9040135b6ecae5204efd7e41ad2e6ba4ee
Parents: 620a4b7
Author: Yuki Morishita
Authored: Wed Dec 19 12:43:23 2012 -0600
Committer: Yuki Morishita
Committed: Wed Dec 19 12:43:23 2012 -0600
--
CHANGES.txt|1 +
.../apache/cassandra/dht/Murmur3Partitioner.java | 14 +++--
.../apache/cassandra/dht/PartitionerTestCase.java | 40 ++-
3 files changed, 48 insertions(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7ae7c90/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6edf7bb..7786b09 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -11,6 +11,7 @@ Merged from 1.1:
* Fix ALTER TABLE overriding compression options with defaults
(CASSANDRA-4996, 5066)
* fix specifying and altering crc_check_chance (CASSANDRA-5053)
+ * fix Murmur3Partitioner ownership% calculation (CASSANDRA-5076)
1.2-rc1
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7ae7c90/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
--
diff --git a/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
b/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
index 3e98e32..ad3579a 100644
--- a/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
+++ b/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
@@ -17,6 +17,7 @@
*/
package org.apache.cassandra.dht;
+import java.math.BigDecimal;
import java.math.BigInteger;
import java.nio.ByteBuffer;
import java.util.HashMap;
@@ -125,20 +126,21 @@ public class Murmur3Partitioner extends
AbstractPartitioner
// n-case
else
{
-final float ri = MAXIMUM;
// (used for addition later)
-Token start = (Token) i.next(); Long ti =
((LongToken)start).token; // The first token and its value
-Token t; Long tim1 = ti;
// The last token and its value (after loop)
+final BigInteger ri =
BigInteger.valueOf(MAXIMUM).subtract(BigInteger.valueOf(MINIMUM.token + 1));
// (used for addition later)
+final BigDecimal r = new BigDecimal(ri);
+Token start = (Token) i.next();BigInteger ti =
BigInteger.valueOf(((LongToken)start).token); // The first token and its value
+Token t; BigInteger tim1 = ti;
// The last token and its value (after loop)
while (i.hasNext())
{
-t = (Token) i.next(); ti = ((LongToken) t).token; // The next
token and its value
-float age = ((ti - tim1 + ri) % ri) / ri; // %age =
((T(i) - T(i-1) + R) % R) / R
+t = (Token) i.next(); ti = BigInteger.valueOf(((LongToken)
t).token); // The next token and its value
+float age = new
BigDecimal(ti.subtract(tim1).add(ri).mod(ri)).divide(r, 6,
BigDecimal.ROUND_HALF_EVEN).floatValue(); // %age = ((T(i) - T(i-1) + R) % R) /
R
ownerships.put(t, age); // save
(T(i) -> %age)
tim1 = ti;// ->
advance loop
}
// The start token's range extends backward to the last token,
which is why both were saved above.
-float x = LongToken) start).token - ti + ri) % ri) / ri;
+float x = new
BigDecimal(BigInteger.valueOf(((LongToken)start).token).subtract(ti).add(ri).mod(ri)).divide(r,
6, BigDecimal.ROUND_HALF_EVEN).floatValue();
ownerships.put(start, x);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7ae7c90/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
--
diff --git a/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
b/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
index 720ba3e..79b0d78 100644
--- a/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
+++ b/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
@@ -19,11 +19,14 @@
package org.apache.cassandra.dht;
import java.nio.ByteBuffer;
-import java.util.Random;
+import java.util.*;
import org.junit.Before;
import org.junit.Test;
[5/6] git commit: Fix M3P ownership calculation; patch by yukim reviewed by Sylvain Lebresne for CASSANDRA-5076
Fix M3P ownership calculation; patch by yukim reviewed by Sylvain Lebresne for
CASSANDRA-5076
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e7ae7c90
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e7ae7c90
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e7ae7c90
Branch: refs/heads/cassandra-1.2.0
Commit: e7ae7c9040135b6ecae5204efd7e41ad2e6ba4ee
Parents: 620a4b7
Author: Yuki Morishita
Authored: Wed Dec 19 12:43:23 2012 -0600
Committer: Yuki Morishita
Committed: Wed Dec 19 12:43:23 2012 -0600
--
CHANGES.txt|1 +
.../apache/cassandra/dht/Murmur3Partitioner.java | 14 +++--
.../apache/cassandra/dht/PartitionerTestCase.java | 40 ++-
3 files changed, 48 insertions(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7ae7c90/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6edf7bb..7786b09 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -11,6 +11,7 @@ Merged from 1.1:
* Fix ALTER TABLE overriding compression options with defaults
(CASSANDRA-4996, 5066)
* fix specifying and altering crc_check_chance (CASSANDRA-5053)
+ * fix Murmur3Partitioner ownership% calculation (CASSANDRA-5076)
1.2-rc1
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7ae7c90/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
--
diff --git a/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
b/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
index 3e98e32..ad3579a 100644
--- a/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
+++ b/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
@@ -17,6 +17,7 @@
*/
package org.apache.cassandra.dht;
+import java.math.BigDecimal;
import java.math.BigInteger;
import java.nio.ByteBuffer;
import java.util.HashMap;
@@ -125,20 +126,21 @@ public class Murmur3Partitioner extends
AbstractPartitioner
// n-case
else
{
-final float ri = MAXIMUM;
// (used for addition later)
-Token start = (Token) i.next(); Long ti =
((LongToken)start).token; // The first token and its value
-Token t; Long tim1 = ti;
// The last token and its value (after loop)
+final BigInteger ri =
BigInteger.valueOf(MAXIMUM).subtract(BigInteger.valueOf(MINIMUM.token + 1));
// (used for addition later)
+final BigDecimal r = new BigDecimal(ri);
+Token start = (Token) i.next();BigInteger ti =
BigInteger.valueOf(((LongToken)start).token); // The first token and its value
+Token t; BigInteger tim1 = ti;
// The last token and its value (after loop)
while (i.hasNext())
{
-t = (Token) i.next(); ti = ((LongToken) t).token; // The next
token and its value
-float age = ((ti - tim1 + ri) % ri) / ri; // %age =
((T(i) - T(i-1) + R) % R) / R
+t = (Token) i.next(); ti = BigInteger.valueOf(((LongToken)
t).token); // The next token and its value
+float age = new
BigDecimal(ti.subtract(tim1).add(ri).mod(ri)).divide(r, 6,
BigDecimal.ROUND_HALF_EVEN).floatValue(); // %age = ((T(i) - T(i-1) + R) % R) /
R
ownerships.put(t, age); // save
(T(i) -> %age)
tim1 = ti;// ->
advance loop
}
// The start token's range extends backward to the last token,
which is why both were saved above.
-float x = LongToken) start).token - ti + ri) % ri) / ri;
+float x = new
BigDecimal(BigInteger.valueOf(((LongToken)start).token).subtract(ti).add(ri).mod(ri)).divide(r,
6, BigDecimal.ROUND_HALF_EVEN).floatValue();
ownerships.put(start, x);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7ae7c90/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
--
diff --git a/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
b/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
index 720ba3e..79b0d78 100644
--- a/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
+++ b/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
@@ -19,11 +19,14 @@
package org.apache.cassandra.dht;
import java.nio.ByteBuffer;
-import java.util.Random;
+import java.util.*;
import org.junit.Before;
import org.junit.Tes
[2/6] git commit: Merge branch 'cassandra-1.2.0' into cassandra-1.2
Merge branch 'cassandra-1.2.0' into cassandra-1.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bebaf453 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bebaf453 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bebaf453 Branch: refs/heads/cassandra-1.2 Commit: bebaf453c8e0f21b1be6c2ed962ae2f3e3c77373 Parents: 0207bdf e7ae7c9 Author: Yuki Morishita Authored: Wed Dec 19 12:43:33 2012 -0600 Committer: Yuki Morishita Committed: Wed Dec 19 12:43:33 2012 -0600 -- CHANGES.txt|1 + .../apache/cassandra/dht/Murmur3Partitioner.java | 14 +++-- .../apache/cassandra/dht/PartitionerTestCase.java | 40 ++- 3 files changed, 48 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/bebaf453/CHANGES.txt --
[jira] [Commented] (CASSANDRA-5076) Murmur3Partitioner#describeOwnership calculates ownership% wrong
[
https://issues.apache.org/jira/browse/CASSANDRA-5076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536260#comment-13536260
]
Sylvain Lebresne commented on CASSANDRA-5076:
-
For the last range, I think the {{ti.subtract(tim1)}} part should be changed to
{{(BigIntegerToken)start).token.subtract(ti)}} as done in
RandomPartitionner.describeOwnership. Cause at the end of the loop, {{ti ==
tim1}}.
But +1 with that fixed.
> Murmur3Partitioner#describeOwnership calculates ownership% wrong
>
>
> Key: CASSANDRA-5076
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5076
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.2.0 rc1
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 1.2.0 rc2
>
> Attachments: 5076.txt
>
>
> When I issued 'nodetool status' on Murmur3-partitioned cluster I got the
> following output:
> {code}
> $ bin/nodetool -p 7100 status
>
>
>
> (git)-[5065]-[~/Developments/workspace/cassandra]
> Datacenter: datacenter1
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- Address Load Owns Host ID
> TokenRack
> UN 127.0.0.1 24.78 KB 66.7% ace7e54c-9fe1-4b23-83b0-949772b24c30
> -9223372036854775808 rack1
> UN 127.0.0.2 29.22 KB 66.7% 67146442-dbfd-449c-82e1-26729b8ac89c
> -3074457345618258603 rack1
> UN 127.0.0.3 6.19 KB66.7% 3fab9f18-daf3-4452-8b9c-204ea0ee2e15
> 3074457345618258602 rack1
> {code}
> Notice that 'Owns' percentages add up to 200%.
> I think the problem is that Murmur3Partitioner#describeOwnership currently
> calculate ownership% based on [0, Long.MAX_VALUE] range, but we have to
> consider about negative tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5076) Murmur3Partitioner#describeOwnership calculates ownership% wrong
[
https://issues.apache.org/jira/browse/CASSANDRA-5076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuki Morishita updated CASSANDRA-5076:
--
Attachment: 5076.txt
Patch attached to calculate ownership using BigInteger/BigDecimal for range
(Long.MIN_VALUE, Long.MAX_VALUE].
This will also resolve problem described in CASSANDRA-4598.
> Murmur3Partitioner#describeOwnership calculates ownership% wrong
>
>
> Key: CASSANDRA-5076
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5076
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.2.0 rc1
>Reporter: Yuki Morishita
>Priority: Minor
> Attachments: 5076.txt
>
>
> When I issued 'nodetool status' on Murmur3-partitioned cluster I got the
> following output:
> {code}
> $ bin/nodetool -p 7100 status
>
>
>
> (git)-[5065]-[~/Developments/workspace/cassandra]
> Datacenter: datacenter1
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- Address Load Owns Host ID
> TokenRack
> UN 127.0.0.1 24.78 KB 66.7% ace7e54c-9fe1-4b23-83b0-949772b24c30
> -9223372036854775808 rack1
> UN 127.0.0.2 29.22 KB 66.7% 67146442-dbfd-449c-82e1-26729b8ac89c
> -3074457345618258603 rack1
> UN 127.0.0.3 6.19 KB66.7% 3fab9f18-daf3-4452-8b9c-204ea0ee2e15
> 3074457345618258602 rack1
> {code}
> Notice that 'Owns' percentages add up to 200%.
> I think the problem is that Murmur3Partitioner#describeOwnership currently
> calculate ownership% based on [0, Long.MAX_VALUE] range, but we have to
> consider about negative tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5077) Simplify CompactionIterable.getReduced
[ https://issues.apache.org/jira/browse/CASSANDRA-5077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536197#comment-13536197 ] Carl Yeksigian commented on CASSANDRA-5077: --- +1 Rebased the patch against trunk because of the second changes from Cassandra-4492. Otherwise, patch LGTM. > Simplify CompactionIterable.getReduced > -- > > Key: CASSANDRA-5077 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5077 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis > Fix For: 1.2.1 > > Attachments: 5077.txt, 5077.txt > > > CompactionIterable turns empty rows into null, and > purge-compacted-rows-from-cache logic is duplicated (and different!) across > implementations. Let's simplify to just return empty rows as-is and let > CompactionTask handle the purge logic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5077) Simplify CompactionIterable.getReduced
[ https://issues.apache.org/jira/browse/CASSANDRA-5077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Yeksigian updated CASSANDRA-5077: -- Attachment: 5077.txt > Simplify CompactionIterable.getReduced > -- > > Key: CASSANDRA-5077 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5077 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis > Fix For: 1.2.1 > > Attachments: 5077.txt, 5077.txt > > > CompactionIterable turns empty rows into null, and > purge-compacted-rows-from-cache logic is duplicated (and different!) across > implementations. Let's simplify to just return empty rows as-is and let > CompactionTask handle the purge logic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5013) disallow bloom filter false positive chance of 0
[
https://issues.apache.org/jira/browse/CASSANDRA-5013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-5013:
--
Fix Version/s: (was: 1.2.0 rc2)
1.2.1
> disallow bloom filter false positive chance of 0
>
>
> Key: CASSANDRA-5013
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5013
> Project: Cassandra
> Issue Type: Bug
> Components: Config
>Reporter: Matthew F. Dennis
>Assignee: Brandon Williams
>Priority: Minor
> Fix For: 1.2.1
>
> Attachments: 5013.txt, 5013-v2.txt
>
>
> {pre}
> ERROR [CompactionExecutor:16] 2012-11-30 08:44:32,546 SSTableWriter.java
> (line 414) Bloom filter FP chance of zero isn't supposed to happen
> {pre}
> when attempting to set it to zero, C* should either disallow the change or
> should just interpret 0 as "make it the default" and not continually log the
> above error message
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2307) IndexOutOfBoundsException during compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536098#comment-13536098 ] Jonathan Ellis commented on CASSANDRA-2307: --- Commenting on a bug filed against a two-year-old release, for an eight-month-old .0 version, is probably not the best way to address this. If you can reproduce with 1.1.8 then please open a new ticket. Standard advice of running scrub first applies as above. > IndexOutOfBoundsException during compaction > --- > > Key: CASSANDRA-2307 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2307 > Project: Cassandra > Issue Type: Bug >Affects Versions: 0.7.2, 0.7.3 > Environment: Debian 5.0.8, Linux 2.6.26-2-amd64, 4GB of ram assigned > to Cassandra, JRE 1.6.0_24 >Reporter: Mike Nadeau > > We're getting an IndexOutOfBounds exception when compacting. > Here's the detailed error we get on-screen when running "nodetool -h > 10.3.133.10 compact": > Error occured while compacting keyspace test > java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException > at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source) > at java.util.concurrent.FutureTask.get(Unknown Source) > at > org.apache.cassandra.db.CompactionManager.performMajor(CompactionManager.java:186) > at > org.apache.cassandra.db.ColumnFamilyStore.forceMajorCompaction(ColumnFamilyStore.java:1678) > at > org.apache.cassandra.service.StorageService.forceTableCompaction(StorageService.java:1248) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) > at java.lang.reflect.Method.invoke(Unknown Source) > at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source) > at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source) > at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(Unknown Source) > at com.sun.jmx.mbeanserver.PerInterface.invoke(Unknown Source) > at com.sun.jmx.mbeanserver.MBeanSupport.invoke(Unknown Source) > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(Unknown > Source) > at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(Unknown Source) > at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown Source) > at javax.management.remote.rmi.RMIConnectionImpl.access$200(Unknown Source) > at > javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown > Source) > at > javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown > Source) > at javax.management.remote.rmi.RMIConnectionImpl.invoke(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) > at java.lang.reflect.Method.invoke(Unknown Source) > at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source) > at sun.rmi.transport.Transport$1.run(Unknown Source) > at java.security.AccessController.doPrivileged(Native Method) > at sun.rmi.transport.Transport.serviceCall(Unknown Source) > at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source) > at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source) > at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > at java.lang.Thread.run(Unknown Source) > Caused by: java.lang.IndexOutOfBoundsException > at java.nio.Buffer.checkIndex(Unknown Source) > at java.nio.HeapByteBuffer.getInt(Unknown Source) > at > org.apache.cassandra.db.DeletedColumn.getLocalDeletionTime(DeletedColumn.java:57) > at > org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:822) > at > org.apache.cassandra.db.ColumnFamilyStore.removeDeletedColumnsOnly(ColumnFamilyStore.java:809) > at > org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:800) > at org.apache.cassandra.io.PrecompactedRow.(PrecompactedRow.java:94) > at > org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:139) > at > org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108) > at > org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43) > at > org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73) > at > com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136) > at > com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131) > at > org.apache.commons.collections.iterators.
[jira] [Commented] (CASSANDRA-4858) Coverage analysis for low-CL queries
[ https://issues.apache.org/jira/browse/CASSANDRA-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536084#comment-13536084 ] Sylvain Lebresne commented on CASSANDRA-4858: - Is there a reason why this is limited to CL.ONE. Cause the problem is not necessarily limited to CL.ONE. Typically, in the degenerated case where RF == number of nodes, only one command need to be created ever, even for CL.ALL. More generally, as in the snippet of pseudo-code above, I think that as long as the intersection of endpoints for the current range and of endpoints for the next range is greater than what the consistency level require, then we can merge both range in one command. I'm also not sure why we would limit this to when RR is not true as long as the condition I've just described is true. As as side note, there is a typo in IncludingExcludingBounds.cloneWithRight (right is used twice). > Coverage analysis for low-CL queries > > > Key: CASSANDRA-4858 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4858 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: Vijay > Fix For: 1.2.1 > > Attachments: 0001-CASSANDRA-4858.patch > > > There are many cases where getRangeSlice creates more > RangeSliceCommand than it should, because it always creates one for each range > returned by getRestrictedRange. Especially for CL.ONE this does not take > the replication factor into account and is potentially pretty wasteful. > A range slice at CL.ONE on a 3 node cluster with RF=3 should only > ever create one RangeSliceCommand. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4858) Coverage analysis for low-CL queries
[ https://issues.apache.org/jira/browse/CASSANDRA-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-4858: -- Reviewer: slebresne > Coverage analysis for low-CL queries > > > Key: CASSANDRA-4858 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4858 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: Vijay > Fix For: 1.2.1 > > Attachments: 0001-CASSANDRA-4858.patch > > > There are many cases where getRangeSlice creates more > RangeSliceCommand than it should, because it always creates one for each range > returned by getRestrictedRange. Especially for CL.ONE this does not take > the replication factor into account and is potentially pretty wasteful. > A range slice at CL.ONE on a 3 node cluster with RF=3 should only > ever create one RangeSliceCommand. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4894) log number of combined/merged rows during a compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536057#comment-13536057
]
Jonathan Ellis commented on CASSANDRA-4894:
---
How about this?
Compacted 4 sstables to
[/Users/yuki/.ccm/1.2/node1/data/system/local/system-local-ia-18-Data.db,]. 957
bytes to 629 (~65% of original) in 35ms = 0.017139MB/s. 4 total rows, 1 unique.
Merge counts {1:0, 2:0, 3:0, 4:1}
> log number of combined/merged rows during a compaction
> --
>
> Key: CASSANDRA-4894
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4894
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Matthew F. Dennis
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 1.2.1
>
> Attachments: 4894-1.2.txt
>
>
> we already log some details about compactions but it would be useful to know
> how many rows were merged (resulting in "useful" work) and how many were
> unique (representing "wasted work").
> the simple approach requires two additional counters (one for unique rows,
> one for merged rows). As the merge join is progressing if two or more rows
> are combined, tick the joined counter. If a row is simply copied tick the
> unique counter.
> a more complete solution would be to keep a separate count for each number of
> merges. This would require number_of_files_being_merged counters. If no
> rows were merged, tick counters[0], if two rows were merged tick counters[1]
> and so on
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[2/3] git commit: Log compaction merge counts; patch by yukim reviewed by jbellis for CASSANDRA-4894
Log compaction merge counts; patch by yukim reviewed by jbellis for
CASSANDRA-4894
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0207bdf5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0207bdf5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0207bdf5
Branch: refs/heads/trunk
Commit: 0207bdf5ff07b60d9a44f7d592061a7b74d034e1
Parents: fca00ac
Author: Yuki Morishita
Authored: Wed Dec 19 08:56:52 2012 -0600
Committer: Yuki Morishita
Committed: Wed Dec 19 08:56:52 2012 -0600
--
CHANGES.txt|1 +
.../db/compaction/AbstractCompactionIterable.java | 24 +++
.../db/compaction/CompactionIterable.java |1 +
.../cassandra/db/compaction/CompactionTask.java|4 +-
.../db/compaction/ParallelCompactionIterable.java |1 +
5 files changed, 29 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0207bdf5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d3a0077..5f1ff2b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -9,6 +9,7 @@
* Add debug logging to list filenames processed by Directories.migrateFile
method (CASSANDRA-4939)
* Expose black-listed directories via JMX (CASSANDRA-4848)
+ * Log compaction merge counts (CASSANDRA-4894)
1.2.0-rc2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0207bdf5/src/java/org/apache/cassandra/db/compaction/AbstractCompactionIterable.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionIterable.java
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionIterable.java
index def7191..d7b9f63 100644
---
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionIterable.java
+++
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionIterable.java
@@ -18,6 +18,7 @@
package org.apache.cassandra.db.compaction;
import java.util.List;
+import java.util.concurrent.atomic.AtomicInteger;
import org.apache.cassandra.utils.CloseableIterator;
@@ -28,6 +29,12 @@ public abstract class AbstractCompactionIterable extends
CompactionInfo.Holder i
protected final long totalBytes;
protected volatile long bytesRead;
protected final List scanners;
+/*
+ * counters for merged rows.
+ * array index represents (number of merged rows - 1), so index 0 is
counter for no merge (1 row),
+ * index 1 is counter for 2 rows merged, and so on.
+ */
+protected final AtomicInteger[] mergeCounters;
public AbstractCompactionIterable(CompactionController controller,
OperationType type, List scanners)
{
@@ -40,6 +47,9 @@ public abstract class AbstractCompactionIterable extends
CompactionInfo.Holder i
for (ICompactionScanner scanner : scanners)
bytes += scanner.getLengthInBytes();
this.totalBytes = bytes;
+mergeCounters = new AtomicInteger[scanners.size()];
+for (int i = 0; i < mergeCounters.length; i++)
+mergeCounters[i] = new AtomicInteger();
}
public CompactionInfo getCompactionInfo()
@@ -50,5 +60,19 @@ public abstract class AbstractCompactionIterable extends
CompactionInfo.Holder i
totalBytes);
}
+protected void updateCounterFor(int rows)
+{
+assert rows > 0 && rows - 1 < mergeCounters.length;
+mergeCounters[rows - 1].incrementAndGet();
+}
+
+public int[] getMergedRowCounts()
+{
+int[] counters = new int[mergeCounters.length];
+for (int i = 0; i < counters.length; i++)
+counters[i] = mergeCounters[i].get();
+return counters;
+}
+
public abstract CloseableIterator iterator();
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0207bdf5/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
b/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
index f095a57..9f4f7a0 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
@@ -67,6 +67,7 @@ public class CompactionIterable extends
AbstractCompactionIterable
{
assert !rows.isEmpty();
+CompactionIterable.this.updateCounterFor(rows.size());
try
{
AbstractCompactedRow compactedRow =
controller.getCompactedRow(new ArrayList(rows));
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0207bdf5/
[1/3] git commit: Merge branch 'cassandra-1.2' into trunk
Updated Branches: refs/heads/cassandra-1.2 fca00acb5 -> 0207bdf5f refs/heads/trunk d3cd9adf9 -> 9bc9853e2 Merge branch 'cassandra-1.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9bc9853e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9bc9853e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9bc9853e Branch: refs/heads/trunk Commit: 9bc9853e295df3fc01fa60d0746ea293d651e635 Parents: d3cd9ad 0207bdf Author: Yuki Morishita Authored: Wed Dec 19 08:57:36 2012 -0600 Committer: Yuki Morishita Committed: Wed Dec 19 08:57:36 2012 -0600 -- CHANGES.txt|1 + .../db/compaction/AbstractCompactionIterable.java | 24 +++ .../db/compaction/CompactionIterable.java |1 + .../cassandra/db/compaction/CompactionTask.java|4 +- .../db/compaction/ParallelCompactionIterable.java |1 + 5 files changed, 29 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9bc9853e/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9bc9853e/src/java/org/apache/cassandra/db/compaction/CompactionTask.java --
[3/3] git commit: Log compaction merge counts; patch by yukim reviewed by jbellis for CASSANDRA-4894
Log compaction merge counts; patch by yukim reviewed by jbellis for
CASSANDRA-4894
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0207bdf5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0207bdf5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0207bdf5
Branch: refs/heads/cassandra-1.2
Commit: 0207bdf5ff07b60d9a44f7d592061a7b74d034e1
Parents: fca00ac
Author: Yuki Morishita
Authored: Wed Dec 19 08:56:52 2012 -0600
Committer: Yuki Morishita
Committed: Wed Dec 19 08:56:52 2012 -0600
--
CHANGES.txt|1 +
.../db/compaction/AbstractCompactionIterable.java | 24 +++
.../db/compaction/CompactionIterable.java |1 +
.../cassandra/db/compaction/CompactionTask.java|4 +-
.../db/compaction/ParallelCompactionIterable.java |1 +
5 files changed, 29 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0207bdf5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d3a0077..5f1ff2b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -9,6 +9,7 @@
* Add debug logging to list filenames processed by Directories.migrateFile
method (CASSANDRA-4939)
* Expose black-listed directories via JMX (CASSANDRA-4848)
+ * Log compaction merge counts (CASSANDRA-4894)
1.2.0-rc2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0207bdf5/src/java/org/apache/cassandra/db/compaction/AbstractCompactionIterable.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionIterable.java
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionIterable.java
index def7191..d7b9f63 100644
---
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionIterable.java
+++
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionIterable.java
@@ -18,6 +18,7 @@
package org.apache.cassandra.db.compaction;
import java.util.List;
+import java.util.concurrent.atomic.AtomicInteger;
import org.apache.cassandra.utils.CloseableIterator;
@@ -28,6 +29,12 @@ public abstract class AbstractCompactionIterable extends
CompactionInfo.Holder i
protected final long totalBytes;
protected volatile long bytesRead;
protected final List scanners;
+/*
+ * counters for merged rows.
+ * array index represents (number of merged rows - 1), so index 0 is
counter for no merge (1 row),
+ * index 1 is counter for 2 rows merged, and so on.
+ */
+protected final AtomicInteger[] mergeCounters;
public AbstractCompactionIterable(CompactionController controller,
OperationType type, List scanners)
{
@@ -40,6 +47,9 @@ public abstract class AbstractCompactionIterable extends
CompactionInfo.Holder i
for (ICompactionScanner scanner : scanners)
bytes += scanner.getLengthInBytes();
this.totalBytes = bytes;
+mergeCounters = new AtomicInteger[scanners.size()];
+for (int i = 0; i < mergeCounters.length; i++)
+mergeCounters[i] = new AtomicInteger();
}
public CompactionInfo getCompactionInfo()
@@ -50,5 +60,19 @@ public abstract class AbstractCompactionIterable extends
CompactionInfo.Holder i
totalBytes);
}
+protected void updateCounterFor(int rows)
+{
+assert rows > 0 && rows - 1 < mergeCounters.length;
+mergeCounters[rows - 1].incrementAndGet();
+}
+
+public int[] getMergedRowCounts()
+{
+int[] counters = new int[mergeCounters.length];
+for (int i = 0; i < counters.length; i++)
+counters[i] = mergeCounters[i].get();
+return counters;
+}
+
public abstract CloseableIterator iterator();
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0207bdf5/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
b/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
index f095a57..9f4f7a0 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionIterable.java
@@ -67,6 +67,7 @@ public class CompactionIterable extends
AbstractCompactionIterable
{
assert !rows.isEmpty();
+CompactionIterable.this.updateCounterFor(rows.size());
try
{
AbstractCompactedRow compactedRow =
controller.getCompactedRow(new ArrayList(rows));
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0
[jira] [Resolved] (CASSANDRA-5053) not possible to change crc_check_chance
[ https://issues.apache.org/jira/browse/CASSANDRA-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko resolved CASSANDRA-5053. -- Resolution: Fixed > not possible to change crc_check_chance > --- > > Key: CASSANDRA-5053 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5053 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.1, 1.2.0 rc1 >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 1.1.9, 1.2.0 rc2 > > Attachments: > 0001-CASSANDRA-5053-make-it-possible-to-change-crc_check_.patch, > 0001-fix-CASSANDRA-5053-not-possible-to-change-crc_check_.patch, 5053-v2.txt > > > It is not possible to change crc_check_chance using a schema modification > after CASSANDRA-4266 > This patch fixes that and moves the setting out into a configuration > parameter instead, you dont want to upgrade/scrub/.. all your sstables to > change the crc_check_chance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5053) not possible to change crc_check_chance
[ https://issues.apache.org/jira/browse/CASSANDRA-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13535932#comment-13535932 ] Aleksey Yeschenko commented on CASSANDRA-5053: -- Committed. > not possible to change crc_check_chance > --- > > Key: CASSANDRA-5053 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5053 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.1, 1.2.0 rc1 >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 1.1.9, 1.2.0 rc2 > > Attachments: > 0001-CASSANDRA-5053-make-it-possible-to-change-crc_check_.patch, > 0001-fix-CASSANDRA-5053-not-possible-to-change-crc_check_.patch, 5053-v2.txt > > > It is not possible to change crc_check_chance using a schema modification > after CASSANDRA-4266 > This patch fixes that and moves the setting out into a configuration > parameter instead, you dont want to upgrade/scrub/.. all your sstables to > change the crc_check_chance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[1/4] git commit: Merge branch 'cassandra-1.2' into trunk
Updated Branches: refs/heads/trunk 4a3794b7b -> d3cd9adf9 Merge branch 'cassandra-1.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d3cd9adf Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d3cd9adf Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d3cd9adf Branch: refs/heads/trunk Commit: d3cd9adf99129e74d0454f237101009eba625e63 Parents: 4a3794b fca00ac Author: Aleksey Yeschenko Authored: Wed Dec 19 16:10:21 2012 +0300 Committer: Aleksey Yeschenko Committed: Wed Dec 19 16:10:21 2012 +0300 -- CHANGES.txt|3 +- .../org/apache/cassandra/db/ColumnFamilyStore.java | 14 .../cassandra/db/ColumnFamilyStoreMBean.java |5 ++ .../io/compress/CompressedRandomAccessReader.java |2 +- .../io/compress/CompressionParameters.java | 54 -- .../streaming/compress/CompressedInputStream.java |2 +- 6 files changed, 69 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d3cd9adf/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d3cd9adf/src/java/org/apache/cassandra/db/ColumnFamilyStore.java --
[4/4] git commit: fix specifying and altering crc_check_chance; patch by Marcus Eriksson and Aleksey Yeschenko, reviewed by Aleksey Yeschenko for CASSANDRA-5053
fix specifying and altering crc_check_chance;
patch by Marcus Eriksson and Aleksey Yeschenko, reviewed by Aleksey
Yeschenko for CASSANDRA-5053
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1a66ee9a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1a66ee9a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1a66ee9a
Branch: refs/heads/trunk
Commit: 1a66ee9a94d67ae475ae287b280517aa9e3cc318
Parents: e792187
Author: Aleksey Yeschenko
Authored: Wed Dec 19 14:54:46 2012 +0300
Committer: Aleksey Yeschenko
Committed: Wed Dec 19 14:54:46 2012 +0300
--
CHANGES.txt|1 +
.../org/apache/cassandra/db/ColumnFamilyStore.java |7 ++
.../cassandra/db/ColumnFamilyStoreMBean.java |5 +
.../io/compress/CompressedRandomAccessReader.java |2 +-
.../io/compress/CompressionParameters.java | 62 --
5 files changed, 67 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index cc75791..38d9d47 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
1.1.9
* fix multithreaded compaction deadlock (CASSANDRA-4492)
+ * fix specifying and altering crc_check_chance (CASSANDRA-5053)
1.1.8
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 67a883d..8284d38 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -201,6 +201,13 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
metadata.compressionParameters = CompressionParameters.create(opts);
}
+public void setCrcCheckChance(double crcCheckChance) throws
ConfigurationException
+{
+for (SSTableReader sstable : table.getAllSSTables())
+if (sstable.compression)
+
sstable.getCompressionMetadata().parameters.setCrcCheckChance(crcCheckChance);
+}
+
private ColumnFamilyStore(Table table,
String columnFamilyName,
IPartitioner partitioner,
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
index 1d9959e..26da8be 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
@@ -215,6 +215,11 @@ public interface ColumnFamilyStoreMBean
public void setCompressionParameters(Map opts) throws
ConfigurationException;
/**
+ * Set new crc check chance
+ */
+public void setCrcCheckChance(double crcCheckChance) throws
ConfigurationException;
+
+/**
* Disable automatic compaction.
*/
public void disableAutoCompaction();
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
index 5403120..3d3b95b 100644
---
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
+++
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
@@ -90,7 +90,7 @@ public class CompressedRandomAccessReader extends
RandomAccessReader
validBufferBytes = metadata.compressor().uncompress(compressed, 0,
chunk.length, buffer, 0);
-if (metadata.parameters.crcChance >
FBUtilities.threadLocalRandom().nextDouble())
+if (metadata.parameters.getCrcCheckChance() >
FBUtilities.threadLocalRandom().nextDouble())
{
checksum.update(buffer, 0, validBufferBytes);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
b/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
index 9be71f6..05cc707 100644
---
[2/4] git commit: Merge branch 'cassandra-1.2.0' into cassandra-1.2
Merge branch 'cassandra-1.2.0' into cassandra-1.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fca00acb Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fca00acb Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fca00acb Branch: refs/heads/trunk Commit: fca00acb50eab46f133a5a17ddafe775f60bf604 Parents: 800b4f1 620a4b7 Author: Aleksey Yeschenko Authored: Wed Dec 19 16:08:40 2012 +0300 Committer: Aleksey Yeschenko Committed: Wed Dec 19 16:08:40 2012 +0300 -- CHANGES.txt|3 +- .../org/apache/cassandra/db/ColumnFamilyStore.java | 14 .../cassandra/db/ColumnFamilyStoreMBean.java |5 ++ .../io/compress/CompressedRandomAccessReader.java |2 +- .../io/compress/CompressionParameters.java | 54 -- .../streaming/compress/CompressedInputStream.java |2 +- 6 files changed, 69 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/fca00acb/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/fca00acb/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/fca00acb/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java --
[3/4] git commit: Merge branch 'cassandra-1.1' into cassandra-1.2.0
Merge branch 'cassandra-1.1' into cassandra-1.2.0 Conflicts: CHANGES.txt src/java/org/apache/cassandra/io/compress/CompressionParameters.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/620a4b7a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/620a4b7a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/620a4b7a Branch: refs/heads/trunk Commit: 620a4b7a4e86e1f31f9823e12a8d5473980c7f03 Parents: 08cc800 1a66ee9 Author: Aleksey Yeschenko Authored: Wed Dec 19 16:07:39 2012 +0300 Committer: Aleksey Yeschenko Committed: Wed Dec 19 16:07:39 2012 +0300 -- CHANGES.txt|3 +- .../org/apache/cassandra/db/ColumnFamilyStore.java | 14 .../cassandra/db/ColumnFamilyStoreMBean.java |5 ++ .../io/compress/CompressedRandomAccessReader.java |2 +- .../io/compress/CompressionParameters.java | 54 -- .../streaming/compress/CompressedInputStream.java |2 +- 6 files changed, 69 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/620a4b7a/CHANGES.txt -- diff --cc CHANGES.txt index 7a17236,38d9d47..6edf7bb --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,142 -1,29 +1,143 @@@ -1.1.9 +1.2.0-rc2 + * fix nodetool ownership display with vnodes (CASSANDRA-5065) + * cqlsh: add DESCRIBE KEYSPACES command (CASSANDRA-5060) + * Fix potential infinite loop when reloading CFS (CASSANDRA-5064) + * Fix SimpleAuthorizer example (CASSANDRA-5072) + * cqlsh: force CL.ONE for tracing and system.schema* queries (CASSANDRA-5070) + * Includes cassandra-shuffle in the debian package (CASSANDRA-5058) +Merged from 1.1: * fix multithreaded compaction deadlock (CASSANDRA-4492) + * fix temporarily missing schema after upgrade from pre-1.1.5 (CASSANDRA-5061) + * Fix ALTER TABLE overriding compression options with defaults -(CASSANDRA-4996, CASSANDRA-5066) ++ (CASSANDRA-4996, 5066) + * fix specifying and altering crc_check_chance (CASSANDRA-5053) -1.1.8 - * reset getRangeSlice filter after finishing a row for get_paged_slice - (CASSANDRA-4919) - * fix temporarily missing schema after upgrade from pre-1.1.5 (CASSANDRA-5061) +1.2-rc1 + * rename rpc_timeout settings to request_timeout (CASSANDRA-5027) + * add BF with 0.1 FP to LCS by default (CASSANDRA-5029) + * Fix preparing insert queries (CASSANDRA-5016) + * Fix preparing queries with counter increment (CASSANDRA-5022) + * Fix preparing updates with collections (CASSANDRA-5017) + * Don't generate UUID based on other node address (CASSANDRA-5002) + * Fix message when trying to alter a clustering key type (CASSANDRA-5012) + * Update IAuthenticator to match the new IAuthorizer (CASSANDRA-5003) + * Fix inserting only a key in CQL3 (CASSANDRA-5040) + * Fix CQL3 token() function when used with strings (CASSANDRA-5050) +Merged from 1.1: * reduce log spam from invalid counter shards (CASSANDRA-5026) * Improve schema propagation performance (CASSANDRA-5025) - * Fall back to old describe_splits if d_s_ex is not available (CASSANDRA-4803) - * Improve error reporting when streaming ranges fail (CASSANDRA-5009) + * Fix for IndexHelper.IndexFor throws OOB Exception (CASSANDRA-5030) + * cqlsh: make it possible to describe thrift CFs (CASSANDRA-4827) * cqlsh: fix timestamp formatting on some platforms (CASSANDRA-5046) - * Fix ALTER TABLE overriding compression options with defaults (CASSANDRA-4996, 5066) - * Avoid error opening data file on startup (CASSANDRA-4984) - * Fix wrong index_options in cli 'show schema' (CASSANDRA-5008) - * Allow overriding number of available processor (CASSANDRA-4790) -1.1.7 - * cqlsh: improve COPY FROM performance (CASSANDRA-4921) +1.2-beta3 + * make consistency level configurable in cqlsh (CASSANDRA-4829) + * fix cqlsh rendering of blob fields (CASSANDRA-4970) + * fix cqlsh DESCRIBE command (CASSANDRA-4913) + * save truncation position in system table (CASSANDRA-4906) + * Move CompressionMetadata off-heap (CASSANDRA-4937) + * allow CLI to GET cql3 columnfamily data (CASSANDRA-4924) + * Fix rare race condition in getExpireTimeForEndpoint (CASSANDRA-4402) + * acquire references to overlapping sstables during compaction so bloom filter + doesn't get free'd prematurely (CASSANDRA-4934) + * Don't share slice query filter in CQL3 SelectStatement (CASSANDRA-4928) + * Separate tracing from Log4J (CASSANDRA-4861) + * Exclude gcable tombstones from merkle-tree computation (CASSANDRA-4905) + * Better printing of AbstractBounds for tracing (CASSANDRA-4931) + * Optimize mostRecentTombstone check in CC.collectAllData (CASSANDRA-4883) + * Change stream session ID to UUID to
[3/3] git commit: fix specifying and altering crc_check_chance; patch by Marcus Eriksson and Aleksey Yeschenko, reviewed by Aleksey Yeschenko for CASSANDRA-5053
fix specifying and altering crc_check_chance;
patch by Marcus Eriksson and Aleksey Yeschenko, reviewed by Aleksey
Yeschenko for CASSANDRA-5053
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1a66ee9a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1a66ee9a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1a66ee9a
Branch: refs/heads/cassandra-1.2
Commit: 1a66ee9a94d67ae475ae287b280517aa9e3cc318
Parents: e792187
Author: Aleksey Yeschenko
Authored: Wed Dec 19 14:54:46 2012 +0300
Committer: Aleksey Yeschenko
Committed: Wed Dec 19 14:54:46 2012 +0300
--
CHANGES.txt|1 +
.../org/apache/cassandra/db/ColumnFamilyStore.java |7 ++
.../cassandra/db/ColumnFamilyStoreMBean.java |5 +
.../io/compress/CompressedRandomAccessReader.java |2 +-
.../io/compress/CompressionParameters.java | 62 --
5 files changed, 67 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index cc75791..38d9d47 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
1.1.9
* fix multithreaded compaction deadlock (CASSANDRA-4492)
+ * fix specifying and altering crc_check_chance (CASSANDRA-5053)
1.1.8
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 67a883d..8284d38 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -201,6 +201,13 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
metadata.compressionParameters = CompressionParameters.create(opts);
}
+public void setCrcCheckChance(double crcCheckChance) throws
ConfigurationException
+{
+for (SSTableReader sstable : table.getAllSSTables())
+if (sstable.compression)
+
sstable.getCompressionMetadata().parameters.setCrcCheckChance(crcCheckChance);
+}
+
private ColumnFamilyStore(Table table,
String columnFamilyName,
IPartitioner partitioner,
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
index 1d9959e..26da8be 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
@@ -215,6 +215,11 @@ public interface ColumnFamilyStoreMBean
public void setCompressionParameters(Map opts) throws
ConfigurationException;
/**
+ * Set new crc check chance
+ */
+public void setCrcCheckChance(double crcCheckChance) throws
ConfigurationException;
+
+/**
* Disable automatic compaction.
*/
public void disableAutoCompaction();
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
index 5403120..3d3b95b 100644
---
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
+++
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
@@ -90,7 +90,7 @@ public class CompressedRandomAccessReader extends
RandomAccessReader
validBufferBytes = metadata.compressor().uncompress(compressed, 0,
chunk.length, buffer, 0);
-if (metadata.parameters.crcChance >
FBUtilities.threadLocalRandom().nextDouble())
+if (metadata.parameters.getCrcCheckChance() >
FBUtilities.threadLocalRandom().nextDouble())
{
checksum.update(buffer, 0, validBufferBytes);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
b/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
index 9be71f6..05cc707 10
[2/3] git commit: Merge branch 'cassandra-1.1' into cassandra-1.2.0
Merge branch 'cassandra-1.1' into cassandra-1.2.0 Conflicts: CHANGES.txt src/java/org/apache/cassandra/io/compress/CompressionParameters.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/620a4b7a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/620a4b7a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/620a4b7a Branch: refs/heads/cassandra-1.2 Commit: 620a4b7a4e86e1f31f9823e12a8d5473980c7f03 Parents: 08cc800 1a66ee9 Author: Aleksey Yeschenko Authored: Wed Dec 19 16:07:39 2012 +0300 Committer: Aleksey Yeschenko Committed: Wed Dec 19 16:07:39 2012 +0300 -- CHANGES.txt|3 +- .../org/apache/cassandra/db/ColumnFamilyStore.java | 14 .../cassandra/db/ColumnFamilyStoreMBean.java |5 ++ .../io/compress/CompressedRandomAccessReader.java |2 +- .../io/compress/CompressionParameters.java | 54 -- .../streaming/compress/CompressedInputStream.java |2 +- 6 files changed, 69 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/620a4b7a/CHANGES.txt -- diff --cc CHANGES.txt index 7a17236,38d9d47..6edf7bb --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,142 -1,29 +1,143 @@@ -1.1.9 +1.2.0-rc2 + * fix nodetool ownership display with vnodes (CASSANDRA-5065) + * cqlsh: add DESCRIBE KEYSPACES command (CASSANDRA-5060) + * Fix potential infinite loop when reloading CFS (CASSANDRA-5064) + * Fix SimpleAuthorizer example (CASSANDRA-5072) + * cqlsh: force CL.ONE for tracing and system.schema* queries (CASSANDRA-5070) + * Includes cassandra-shuffle in the debian package (CASSANDRA-5058) +Merged from 1.1: * fix multithreaded compaction deadlock (CASSANDRA-4492) + * fix temporarily missing schema after upgrade from pre-1.1.5 (CASSANDRA-5061) + * Fix ALTER TABLE overriding compression options with defaults -(CASSANDRA-4996, CASSANDRA-5066) ++ (CASSANDRA-4996, 5066) + * fix specifying and altering crc_check_chance (CASSANDRA-5053) -1.1.8 - * reset getRangeSlice filter after finishing a row for get_paged_slice - (CASSANDRA-4919) - * fix temporarily missing schema after upgrade from pre-1.1.5 (CASSANDRA-5061) +1.2-rc1 + * rename rpc_timeout settings to request_timeout (CASSANDRA-5027) + * add BF with 0.1 FP to LCS by default (CASSANDRA-5029) + * Fix preparing insert queries (CASSANDRA-5016) + * Fix preparing queries with counter increment (CASSANDRA-5022) + * Fix preparing updates with collections (CASSANDRA-5017) + * Don't generate UUID based on other node address (CASSANDRA-5002) + * Fix message when trying to alter a clustering key type (CASSANDRA-5012) + * Update IAuthenticator to match the new IAuthorizer (CASSANDRA-5003) + * Fix inserting only a key in CQL3 (CASSANDRA-5040) + * Fix CQL3 token() function when used with strings (CASSANDRA-5050) +Merged from 1.1: * reduce log spam from invalid counter shards (CASSANDRA-5026) * Improve schema propagation performance (CASSANDRA-5025) - * Fall back to old describe_splits if d_s_ex is not available (CASSANDRA-4803) - * Improve error reporting when streaming ranges fail (CASSANDRA-5009) + * Fix for IndexHelper.IndexFor throws OOB Exception (CASSANDRA-5030) + * cqlsh: make it possible to describe thrift CFs (CASSANDRA-4827) * cqlsh: fix timestamp formatting on some platforms (CASSANDRA-5046) - * Fix ALTER TABLE overriding compression options with defaults (CASSANDRA-4996, 5066) - * Avoid error opening data file on startup (CASSANDRA-4984) - * Fix wrong index_options in cli 'show schema' (CASSANDRA-5008) - * Allow overriding number of available processor (CASSANDRA-4790) -1.1.7 - * cqlsh: improve COPY FROM performance (CASSANDRA-4921) +1.2-beta3 + * make consistency level configurable in cqlsh (CASSANDRA-4829) + * fix cqlsh rendering of blob fields (CASSANDRA-4970) + * fix cqlsh DESCRIBE command (CASSANDRA-4913) + * save truncation position in system table (CASSANDRA-4906) + * Move CompressionMetadata off-heap (CASSANDRA-4937) + * allow CLI to GET cql3 columnfamily data (CASSANDRA-4924) + * Fix rare race condition in getExpireTimeForEndpoint (CASSANDRA-4402) + * acquire references to overlapping sstables during compaction so bloom filter + doesn't get free'd prematurely (CASSANDRA-4934) + * Don't share slice query filter in CQL3 SelectStatement (CASSANDRA-4928) + * Separate tracing from Log4J (CASSANDRA-4861) + * Exclude gcable tombstones from merkle-tree computation (CASSANDRA-4905) + * Better printing of AbstractBounds for tracing (CASSANDRA-4931) + * Optimize mostRecentTombstone check in CC.collectAllData (CASSANDRA-4883) + * Change stream session ID to
[1/3] git commit: Merge branch 'cassandra-1.2.0' into cassandra-1.2
Updated Branches: refs/heads/cassandra-1.2 800b4f1f6 -> fca00acb5 Merge branch 'cassandra-1.2.0' into cassandra-1.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fca00acb Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fca00acb Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fca00acb Branch: refs/heads/cassandra-1.2 Commit: fca00acb50eab46f133a5a17ddafe775f60bf604 Parents: 800b4f1 620a4b7 Author: Aleksey Yeschenko Authored: Wed Dec 19 16:08:40 2012 +0300 Committer: Aleksey Yeschenko Committed: Wed Dec 19 16:08:40 2012 +0300 -- CHANGES.txt|3 +- .../org/apache/cassandra/db/ColumnFamilyStore.java | 14 .../cassandra/db/ColumnFamilyStoreMBean.java |5 ++ .../io/compress/CompressedRandomAccessReader.java |2 +- .../io/compress/CompressionParameters.java | 54 -- .../streaming/compress/CompressedInputStream.java |2 +- 6 files changed, 69 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/fca00acb/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/fca00acb/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/fca00acb/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java --
[2/2] git commit: fix specifying and altering crc_check_chance; patch by Marcus Eriksson and Aleksey Yeschenko, reviewed by Aleksey Yeschenko for CASSANDRA-5053
fix specifying and altering crc_check_chance;
patch by Marcus Eriksson and Aleksey Yeschenko, reviewed by Aleksey
Yeschenko for CASSANDRA-5053
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1a66ee9a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1a66ee9a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1a66ee9a
Branch: refs/heads/cassandra-1.2.0
Commit: 1a66ee9a94d67ae475ae287b280517aa9e3cc318
Parents: e792187
Author: Aleksey Yeschenko
Authored: Wed Dec 19 14:54:46 2012 +0300
Committer: Aleksey Yeschenko
Committed: Wed Dec 19 14:54:46 2012 +0300
--
CHANGES.txt|1 +
.../org/apache/cassandra/db/ColumnFamilyStore.java |7 ++
.../cassandra/db/ColumnFamilyStoreMBean.java |5 +
.../io/compress/CompressedRandomAccessReader.java |2 +-
.../io/compress/CompressionParameters.java | 62 --
5 files changed, 67 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index cc75791..38d9d47 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
1.1.9
* fix multithreaded compaction deadlock (CASSANDRA-4492)
+ * fix specifying and altering crc_check_chance (CASSANDRA-5053)
1.1.8
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 67a883d..8284d38 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -201,6 +201,13 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
metadata.compressionParameters = CompressionParameters.create(opts);
}
+public void setCrcCheckChance(double crcCheckChance) throws
ConfigurationException
+{
+for (SSTableReader sstable : table.getAllSSTables())
+if (sstable.compression)
+
sstable.getCompressionMetadata().parameters.setCrcCheckChance(crcCheckChance);
+}
+
private ColumnFamilyStore(Table table,
String columnFamilyName,
IPartitioner partitioner,
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
index 1d9959e..26da8be 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
@@ -215,6 +215,11 @@ public interface ColumnFamilyStoreMBean
public void setCompressionParameters(Map opts) throws
ConfigurationException;
/**
+ * Set new crc check chance
+ */
+public void setCrcCheckChance(double crcCheckChance) throws
ConfigurationException;
+
+/**
* Disable automatic compaction.
*/
public void disableAutoCompaction();
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
index 5403120..3d3b95b 100644
---
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
+++
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
@@ -90,7 +90,7 @@ public class CompressedRandomAccessReader extends
RandomAccessReader
validBufferBytes = metadata.compressor().uncompress(compressed, 0,
chunk.length, buffer, 0);
-if (metadata.parameters.crcChance >
FBUtilities.threadLocalRandom().nextDouble())
+if (metadata.parameters.getCrcCheckChance() >
FBUtilities.threadLocalRandom().nextDouble())
{
checksum.update(buffer, 0, validBufferBytes);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
b/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
index 9be71f6..05cc707
[1/2] git commit: Merge branch 'cassandra-1.1' into cassandra-1.2.0
Updated Branches: refs/heads/cassandra-1.2.0 08cc800ba -> 620a4b7a4 Merge branch 'cassandra-1.1' into cassandra-1.2.0 Conflicts: CHANGES.txt src/java/org/apache/cassandra/io/compress/CompressionParameters.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/620a4b7a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/620a4b7a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/620a4b7a Branch: refs/heads/cassandra-1.2.0 Commit: 620a4b7a4e86e1f31f9823e12a8d5473980c7f03 Parents: 08cc800 1a66ee9 Author: Aleksey Yeschenko Authored: Wed Dec 19 16:07:39 2012 +0300 Committer: Aleksey Yeschenko Committed: Wed Dec 19 16:07:39 2012 +0300 -- CHANGES.txt|3 +- .../org/apache/cassandra/db/ColumnFamilyStore.java | 14 .../cassandra/db/ColumnFamilyStoreMBean.java |5 ++ .../io/compress/CompressedRandomAccessReader.java |2 +- .../io/compress/CompressionParameters.java | 54 -- .../streaming/compress/CompressedInputStream.java |2 +- 6 files changed, 69 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/620a4b7a/CHANGES.txt -- diff --cc CHANGES.txt index 7a17236,38d9d47..6edf7bb --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,142 -1,29 +1,143 @@@ -1.1.9 +1.2.0-rc2 + * fix nodetool ownership display with vnodes (CASSANDRA-5065) + * cqlsh: add DESCRIBE KEYSPACES command (CASSANDRA-5060) + * Fix potential infinite loop when reloading CFS (CASSANDRA-5064) + * Fix SimpleAuthorizer example (CASSANDRA-5072) + * cqlsh: force CL.ONE for tracing and system.schema* queries (CASSANDRA-5070) + * Includes cassandra-shuffle in the debian package (CASSANDRA-5058) +Merged from 1.1: * fix multithreaded compaction deadlock (CASSANDRA-4492) + * fix temporarily missing schema after upgrade from pre-1.1.5 (CASSANDRA-5061) + * Fix ALTER TABLE overriding compression options with defaults -(CASSANDRA-4996, CASSANDRA-5066) ++ (CASSANDRA-4996, 5066) + * fix specifying and altering crc_check_chance (CASSANDRA-5053) -1.1.8 - * reset getRangeSlice filter after finishing a row for get_paged_slice - (CASSANDRA-4919) - * fix temporarily missing schema after upgrade from pre-1.1.5 (CASSANDRA-5061) +1.2-rc1 + * rename rpc_timeout settings to request_timeout (CASSANDRA-5027) + * add BF with 0.1 FP to LCS by default (CASSANDRA-5029) + * Fix preparing insert queries (CASSANDRA-5016) + * Fix preparing queries with counter increment (CASSANDRA-5022) + * Fix preparing updates with collections (CASSANDRA-5017) + * Don't generate UUID based on other node address (CASSANDRA-5002) + * Fix message when trying to alter a clustering key type (CASSANDRA-5012) + * Update IAuthenticator to match the new IAuthorizer (CASSANDRA-5003) + * Fix inserting only a key in CQL3 (CASSANDRA-5040) + * Fix CQL3 token() function when used with strings (CASSANDRA-5050) +Merged from 1.1: * reduce log spam from invalid counter shards (CASSANDRA-5026) * Improve schema propagation performance (CASSANDRA-5025) - * Fall back to old describe_splits if d_s_ex is not available (CASSANDRA-4803) - * Improve error reporting when streaming ranges fail (CASSANDRA-5009) + * Fix for IndexHelper.IndexFor throws OOB Exception (CASSANDRA-5030) + * cqlsh: make it possible to describe thrift CFs (CASSANDRA-4827) * cqlsh: fix timestamp formatting on some platforms (CASSANDRA-5046) - * Fix ALTER TABLE overriding compression options with defaults (CASSANDRA-4996, 5066) - * Avoid error opening data file on startup (CASSANDRA-4984) - * Fix wrong index_options in cli 'show schema' (CASSANDRA-5008) - * Allow overriding number of available processor (CASSANDRA-4790) -1.1.7 - * cqlsh: improve COPY FROM performance (CASSANDRA-4921) +1.2-beta3 + * make consistency level configurable in cqlsh (CASSANDRA-4829) + * fix cqlsh rendering of blob fields (CASSANDRA-4970) + * fix cqlsh DESCRIBE command (CASSANDRA-4913) + * save truncation position in system table (CASSANDRA-4906) + * Move CompressionMetadata off-heap (CASSANDRA-4937) + * allow CLI to GET cql3 columnfamily data (CASSANDRA-4924) + * Fix rare race condition in getExpireTimeForEndpoint (CASSANDRA-4402) + * acquire references to overlapping sstables during compaction so bloom filter + doesn't get free'd prematurely (CASSANDRA-4934) + * Don't share slice query filter in CQL3 SelectStatement (CASSANDRA-4928) + * Separate tracing from Log4J (CASSANDRA-4861) + * Exclude gcable tombstones from merkle-tree computation (CASSANDRA-4905) + * Better printing of AbstractBounds for tracing (CASSANDRA-4931) + * Optimize mostRecentTombstone che
[jira] [Commented] (CASSANDRA-4894) log number of combined/merged rows during a compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13535925#comment-13535925 ] Yuki Morishita commented on CASSANDRA-4894: --- OK, I will commit this with the change in the log sentence to "Compaction merge counts: [x, x, ...]" then. > log number of combined/merged rows during a compaction > -- > > Key: CASSANDRA-4894 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4894 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Matthew F. Dennis >Assignee: Yuki Morishita >Priority: Minor > Fix For: 1.2.1 > > Attachments: 4894-1.2.txt > > > we already log some details about compactions but it would be useful to know > how many rows were merged (resulting in "useful" work) and how many were > unique (representing "wasted work"). > the simple approach requires two additional counters (one for unique rows, > one for merged rows). As the merge join is progressing if two or more rows > are combined, tick the joined counter. If a row is simply copied tick the > unique counter. > a more complete solution would be to keep a separate count for each number of > merges. This would require number_of_files_being_merged counters. If no > rows were merged, tick counters[0], if two rows were merged tick counters[1] > and so on -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5053) not possible to change crc_check_chance
[ https://issues.apache.org/jira/browse/CASSANDRA-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-5053: - Fix Version/s: 1.2.0 rc2 > not possible to change crc_check_chance > --- > > Key: CASSANDRA-5053 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5053 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.1, 1.2.0 rc1 >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 1.1.9, 1.2.0 rc2 > > Attachments: > 0001-CASSANDRA-5053-make-it-possible-to-change-crc_check_.patch, > 0001-fix-CASSANDRA-5053-not-possible-to-change-crc_check_.patch, 5053-v2.txt > > > It is not possible to change crc_check_chance using a schema modification > after CASSANDRA-4266 > This patch fixes that and moves the setting out into a configuration > parameter instead, you dont want to upgrade/scrub/.. all your sstables to > change the crc_check_chance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5053) not possible to change crc_check_chance
[ https://issues.apache.org/jira/browse/CASSANDRA-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-5053: - Affects Version/s: 1.2.0 rc1 > not possible to change crc_check_chance > --- > > Key: CASSANDRA-5053 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5053 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.1, 1.2.0 rc1 >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 1.1.9 > > Attachments: > 0001-CASSANDRA-5053-make-it-possible-to-change-crc_check_.patch, > 0001-fix-CASSANDRA-5053-not-possible-to-change-crc_check_.patch, 5053-v2.txt > > > It is not possible to change crc_check_chance using a schema modification > after CASSANDRA-4266 > This patch fixes that and moves the setting out into a configuration > parameter instead, you dont want to upgrade/scrub/.. all your sstables to > change the crc_check_chance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5078) save compaction merge counts in a system table
[ https://issues.apache.org/jira/browse/CASSANDRA-5078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew F. Dennis updated CASSANDRA-5078: - Description: we should save the compaction merge stats from CASSANDRA-4894 in the system table and probably expose them via JMX (and nodetool) (was: we should save the compaction merge stats from CASSANDRA-4894 in the system table and probably expose them via JMX) Summary: save compaction merge counts in a system table (was: same compaction merge counts in a system table) > save compaction merge counts in a system table > -- > > Key: CASSANDRA-5078 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5078 > Project: Cassandra > Issue Type: Improvement >Reporter: Matthew F. Dennis >Priority: Minor > > we should save the compaction merge stats from CASSANDRA-4894 in the system > table and probably expose them via JMX (and nodetool) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4894) log number of combined/merged rows during a compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13535899#comment-13535899
]
Matthew F. Dennis commented on CASSANDRA-4894:
--
{quote}
we haven't really concerned ourselves with machine parseability so far
{quote}
that is a mistake. I'm not saying it should be the highest priority or hold up
changes but it should always be considered (on at least a best effort basis).
{quote}
I would in fact be in favor of saving compaction stats in a System table
{quote}
+1
CASSANDRA-5078
even ignoring the machine readable aspect though I want to be able to derive
meaning from the lines at a glance (until CASSANDRA-5078 is done and it's
exposed via nodetool, OPSC, etc) and the compact bracketed format makes that
easier.
> log number of combined/merged rows during a compaction
> --
>
> Key: CASSANDRA-4894
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4894
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Matthew F. Dennis
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 1.2.1
>
> Attachments: 4894-1.2.txt
>
>
> we already log some details about compactions but it would be useful to know
> how many rows were merged (resulting in "useful" work) and how many were
> unique (representing "wasted work").
> the simple approach requires two additional counters (one for unique rows,
> one for merged rows). As the merge join is progressing if two or more rows
> are combined, tick the joined counter. If a row is simply copied tick the
> unique counter.
> a more complete solution would be to keep a separate count for each number of
> merges. This would require number_of_files_being_merged counters. If no
> rows were merged, tick counters[0], if two rows were merged tick counters[1]
> and so on
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5078) same compaction merge counts in a system table
Matthew F. Dennis created CASSANDRA-5078: Summary: same compaction merge counts in a system table Key: CASSANDRA-5078 URL: https://issues.apache.org/jira/browse/CASSANDRA-5078 Project: Cassandra Issue Type: Improvement Reporter: Matthew F. Dennis Priority: Minor we should save the compaction merge stats from CASSANDRA-4894 in the system table and probably expose them via JMX -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: fix specifying and altering crc_check_chance; patch by Marcus Eriksson and Aleksey Yeschenko, reviewed by Aleksey Yeschenko for CASSANDRA-5053
Updated Branches:
refs/heads/cassandra-1.1 e792187bc -> 1a66ee9a9
fix specifying and altering crc_check_chance;
patch by Marcus Eriksson and Aleksey Yeschenko, reviewed by Aleksey
Yeschenko for CASSANDRA-5053
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1a66ee9a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1a66ee9a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1a66ee9a
Branch: refs/heads/cassandra-1.1
Commit: 1a66ee9a94d67ae475ae287b280517aa9e3cc318
Parents: e792187
Author: Aleksey Yeschenko
Authored: Wed Dec 19 14:54:46 2012 +0300
Committer: Aleksey Yeschenko
Committed: Wed Dec 19 14:54:46 2012 +0300
--
CHANGES.txt|1 +
.../org/apache/cassandra/db/ColumnFamilyStore.java |7 ++
.../cassandra/db/ColumnFamilyStoreMBean.java |5 +
.../io/compress/CompressedRandomAccessReader.java |2 +-
.../io/compress/CompressionParameters.java | 62 --
5 files changed, 67 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index cc75791..38d9d47 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
1.1.9
* fix multithreaded compaction deadlock (CASSANDRA-4492)
+ * fix specifying and altering crc_check_chance (CASSANDRA-5053)
1.1.8
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 67a883d..8284d38 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -201,6 +201,13 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
metadata.compressionParameters = CompressionParameters.create(opts);
}
+public void setCrcCheckChance(double crcCheckChance) throws
ConfigurationException
+{
+for (SSTableReader sstable : table.getAllSSTables())
+if (sstable.compression)
+
sstable.getCompressionMetadata().parameters.setCrcCheckChance(crcCheckChance);
+}
+
private ColumnFamilyStore(Table table,
String columnFamilyName,
IPartitioner partitioner,
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
index 1d9959e..26da8be 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStoreMBean.java
@@ -215,6 +215,11 @@ public interface ColumnFamilyStoreMBean
public void setCompressionParameters(Map opts) throws
ConfigurationException;
/**
+ * Set new crc check chance
+ */
+public void setCrcCheckChance(double crcCheckChance) throws
ConfigurationException;
+
+/**
* Disable automatic compaction.
*/
public void disableAutoCompaction();
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
index 5403120..3d3b95b 100644
---
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
+++
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
@@ -90,7 +90,7 @@ public class CompressedRandomAccessReader extends
RandomAccessReader
validBufferBytes = metadata.compressor().uncompress(compressed, 0,
chunk.length, buffer, 0);
-if (metadata.parameters.crcChance >
FBUtilities.threadLocalRandom().nextDouble())
+if (metadata.parameters.getCrcCheckChance() >
FBUtilities.threadLocalRandom().nextDouble())
{
checksum.update(buffer, 0, validBufferBytes);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a66ee9a/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressionParameters.java
b/src/java/org/apache/cass
[jira] [Commented] (CASSANDRA-4894) log number of combined/merged rows during a compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13535852#comment-13535852 ] Sylvain Lebresne commented on CASSANDRA-4894: - I tend to think that the log is more suited to human readable information than machine parseable ones (don't get me wrong, I'm sure lots of people do have script to manipulate the log, I'm just not sure it's the idea tool for that, especially considering we haven't really concerned ourselves with machine parseability so far). I do agree though that compaction stats is probably important enough that you may want to monitor that closely and I would in fact be in favor of saving compaction stats in a System table (much like we plan to save repair stats at some point). That would be imo the good place to be machine friendly. That being said, if you guys prefer the shorter format, and provided that at the time of this ticket the log will be the only place where that information will be available, I'm fine with that. But I do anticipate people asking what those numbers means on irc quite a few time, and I do think having things self-descriptive is a better user experience than having to go find documentation somewhere on the net, even if said doc exists. I could be wrong though and again, not a big deal in any case. > log number of combined/merged rows during a compaction > -- > > Key: CASSANDRA-4894 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4894 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Matthew F. Dennis >Assignee: Yuki Morishita >Priority: Minor > Fix For: 1.2.1 > > Attachments: 4894-1.2.txt > > > we already log some details about compactions but it would be useful to know > how many rows were merged (resulting in "useful" work) and how many were > unique (representing "wasted work"). > the simple approach requires two additional counters (one for unique rows, > one for merged rows). As the merge join is progressing if two or more rows > are combined, tick the joined counter. If a row is simply copied tick the > unique counter. > a more complete solution would be to keep a separate count for each number of > merges. This would require number_of_files_being_merged counters. If no > rows were merged, tick counters[0], if two rows were merged tick counters[1] > and so on -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4894) log number of combined/merged rows during a compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13535835#comment-13535835 ] Matthew F. Dennis commented on CASSANDRA-4894: -- I like the original bracketed format better. The verbose option is hard to parse at a glance. likewise, dropping the zeros would make things more complicated to compare quickly. the bracketed syntax is easy to grep/script/compare/parse/etc. maybe call it "compaction merge counts" instead of "merged row stats" though (so if we had more stats later we don't have to come up with a different name) and then make sure the docs are indexable for "compaction merge counts". Also need to be clear in the docs that counts[0] is just copied, counts[1] is merged two rows, etc ... > log number of combined/merged rows during a compaction > -- > > Key: CASSANDRA-4894 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4894 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Matthew F. Dennis >Assignee: Yuki Morishita >Priority: Minor > Fix For: 1.2.1 > > Attachments: 4894-1.2.txt > > > we already log some details about compactions but it would be useful to know > how many rows were merged (resulting in "useful" work) and how many were > unique (representing "wasted work"). > the simple approach requires two additional counters (one for unique rows, > one for merged rows). As the merge join is progressing if two or more rows > are combined, tick the joined counter. If a row is simply copied tick the > unique counter. > a more complete solution would be to keep a separate count for each number of > merges. This would require number_of_files_being_merged counters. If no > rows were merged, tick counters[0], if two rows were merged tick counters[1] > and so on -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4894) log number of combined/merged rows during a compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13535814#comment-13535814
]
Sylvain Lebresne commented on CASSANDRA-4894:
-
bq. I don't come up with good short description to put in that log line
This is not extremely short, but since the line logged is already pretty long,
why not something like:
{noformat}
... Time: 35ms. Row merged: 1 from 3 sstables, 12 from 2 sstables and 354 from
1 sstable.
{noformat}
And of course we can just skip the 0's in that case.
> log number of combined/merged rows during a compaction
> --
>
> Key: CASSANDRA-4894
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4894
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Matthew F. Dennis
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 1.2.1
>
> Attachments: 4894-1.2.txt
>
>
> we already log some details about compactions but it would be useful to know
> how many rows were merged (resulting in "useful" work) and how many were
> unique (representing "wasted work").
> the simple approach requires two additional counters (one for unique rows,
> one for merged rows). As the merge join is progressing if two or more rows
> are combined, tick the joined counter. If a row is simply copied tick the
> unique counter.
> a more complete solution would be to keep a separate count for each number of
> merges. This would require number_of_files_being_merged counters. If no
> rows were merged, tick counters[0], if two rows were merged tick counters[1]
> and so on
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2307) IndexOutOfBoundsException during compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13535792#comment-13535792 ] Tommi Koivula commented on CASSANDRA-2307: -- We're getting exactly the same error when running a forced major compaction with Cassandra 1.1.0. One of the CFs is producing this error every time. Compaction works for other CFs. There is only one node in this cluster. Cassandra version: 1.1.0 Environment: Ubuntu 12.04, 64-bit Sun JVM 1.6.0_33-b04 Stack trace: java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) at java.util.concurrent.FutureTask.get(FutureTask.java:83) at com.profium.rdfcassandra.CassandraBridge.compact(Unknown Source) at com.profium.rdfcassandra.CassandraBridge.addColumnFamily(Unknown Source) at com.profium.rdfcassandra.CassandraBackedIntegerPersistence.(Unknown Source) at com.profium.sir.ctx.ReasonerModule.createIntegerPersistence(ReasonerModule.java:149) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.profium.ioc.exec.PiocBeanFactory.create(PiocBeanFactory.java:528) at com.profium.ioc.exec.PiocBeanFactory.getOrCreate(PiocBeanFactory.java:456) at com.profium.ioc.exec.PiocBeanDefinitionExecutor.executeCreate(PiocBeanDefinitionExecutor.java:32) at com.profium.ioc.ctx.ApplicationContextFactory.createApplicationContext(ApplicationContextFactory.java:186) at com.profium.ioc.ctx.ApplicationContextFactory.initApplicationContext(ApplicationContextFactory.java:382) at SIR.exec.SirImpl.doChangeMdsRunlevel(SirImpl.java:799) at SIR.exec.SirImpl.raiseToRunlevel(SirImpl.java:685) at com.profium.sir.exec.MdsMonitorThread$1.run(MdsMonitorThread.java:84) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.IndexOutOfBoundsException at java.nio.Buffer.checkIndex(Buffer.java:518) at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:340) at org.apache.cassandra.db.DeletedColumn.getLocalDeletionTime(DeletedColumn.java:50) at org.apache.cassandra.db.Column.isMarkedForDelete(Column.java:110) at org.apache.cassandra.db.Column.reconcile(Column.java:207) at org.apache.cassandra.db.DeletedColumn.reconcile(DeletedColumn.java:58) at org.apache.cassandra.db.ArrayBackedSortedColumns.resolveAgainst(ArrayBackedSortedColumns.java:168) at org.apache.cassandra.db.ArrayBackedSortedColumns.addAllColumns(ArrayBackedSortedColumns.java:232) at org.apache.cassandra.db.AbstractThreadUnsafeSortedColumns.addAll(AbstractThreadUnsafeSortedColumns.java:98) at org.apache.cassandra.db.AbstractColumnContainer.addAll(AbstractColumnContainer.java:92) at org.apache.cassandra.db.AbstractColumnContainer.addAll(AbstractColumnContainer.java:97) at org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:126) at org.apache.cassandra.db.compaction.PrecompactedRow.(PrecompactedRow.java:99) at org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:145) at org.apache.cassandra.db.compaction.CompactionIterable$Reducer.getReduced(CompactionIterable.java:97) at org.apache.cassandra.db.compaction.CompactionIterable$Reducer.getReduced(CompactionIterable.java:82) at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:118) at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:101) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135) at com.google.common.collect.Iterators$7.computeNext(Iterators.java:614) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135) at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:177) at org.apache.cassandra.db.compaction.CompactionManager$7.runMayThrow(CompactionManager.java:360) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concu
[jira] [Comment Edited] (CASSANDRA-5053) not possible to change crc_check_chance
[ https://issues.apache.org/jira/browse/CASSANDRA-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13535778#comment-13535778 ] Aleksey Yeschenko edited comment on CASSANDRA-5053 at 12/19/12 8:25 AM: No idea. But all other setters in that mbean start with 'set'. Are they not exposed either? was (Author: iamaleksey): No idea. But all other setters in that mbean start with 'set'. Are they not exposes either? > not possible to change crc_check_chance > --- > > Key: CASSANDRA-5053 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5053 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.1 >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 1.1.9 > > Attachments: > 0001-CASSANDRA-5053-make-it-possible-to-change-crc_check_.patch, > 0001-fix-CASSANDRA-5053-not-possible-to-change-crc_check_.patch, 5053-v2.txt > > > It is not possible to change crc_check_chance using a schema modification > after CASSANDRA-4266 > This patch fixes that and moves the setting out into a configuration > parameter instead, you dont want to upgrade/scrub/.. all your sstables to > change the crc_check_chance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5053) not possible to change crc_check_chance
[ https://issues.apache.org/jira/browse/CASSANDRA-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13535780#comment-13535780 ] Marcus Eriksson commented on CASSANDRA-5053: ah! they end up under the attributes tab in jconsole > not possible to change crc_check_chance > --- > > Key: CASSANDRA-5053 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5053 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.1 >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 1.1.9 > > Attachments: > 0001-CASSANDRA-5053-make-it-possible-to-change-crc_check_.patch, > 0001-fix-CASSANDRA-5053-not-possible-to-change-crc_check_.patch, 5053-v2.txt > > > It is not possible to change crc_check_chance using a schema modification > after CASSANDRA-4266 > This patch fixes that and moves the setting out into a configuration > parameter instead, you dont want to upgrade/scrub/.. all your sstables to > change the crc_check_chance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5053) not possible to change crc_check_chance
[ https://issues.apache.org/jira/browse/CASSANDRA-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13535778#comment-13535778 ] Aleksey Yeschenko commented on CASSANDRA-5053: -- No idea. But all other setters in that mbean start with 'set'. Are they not exposes either? > not possible to change crc_check_chance > --- > > Key: CASSANDRA-5053 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5053 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.1 >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 1.1.9 > > Attachments: > 0001-CASSANDRA-5053-make-it-possible-to-change-crc_check_.patch, > 0001-fix-CASSANDRA-5053-not-possible-to-change-crc_check_.patch, 5053-v2.txt > > > It is not possible to change crc_check_chance using a schema modification > after CASSANDRA-4266 > This patch fixes that and moves the setting out into a configuration > parameter instead, you dont want to upgrade/scrub/.. all your sstables to > change the crc_check_chance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5053) not possible to change crc_check_chance
[ https://issues.apache.org/jira/browse/CASSANDRA-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13535776#comment-13535776 ] Marcus Eriksson commented on CASSANDRA-5053: uh my mbean/jmx skills fail me here, but seems when naming the method *set*CrcCheckChance it is not exposed via JMX, naming it crcCheckChance does, any clue why? naming convention rules i guess > not possible to change crc_check_chance > --- > > Key: CASSANDRA-5053 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5053 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.1 >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 1.1.9 > > Attachments: > 0001-CASSANDRA-5053-make-it-possible-to-change-crc_check_.patch, > 0001-fix-CASSANDRA-5053-not-possible-to-change-crc_check_.patch, 5053-v2.txt > > > It is not possible to change crc_check_chance using a schema modification > after CASSANDRA-4266 > This patch fixes that and moves the setting out into a configuration > parameter instead, you dont want to upgrade/scrub/.. all your sstables to > change the crc_check_chance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
