[jira] [Commented] (CASSANDRA-1600) Merge get_indexed_slices with get_range_slices
[
https://issues.apache.org/jira/browse/CASSANDRA-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018051#comment-13018051
]
Jeremy Hanna commented on CASSANDRA-1600:
-
Is this going to get in by the April 11th code freeze?
> Merge get_indexed_slices with get_range_slices
> --
>
> Key: CASSANDRA-1600
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1600
> Project: Cassandra
> Issue Type: New Feature
> Components: API
>Reporter: Stu Hood
>Assignee: Jonathan Ellis
> Fix For: 0.8
>
> Attachments:
> 0001-Add-optional-FilterClause-to-KeyRange-and-support-do-v2.patch,
> 0001-Add-optional-FilterClause-to-KeyRange-and-support-doin.txt,
> 0002-allow-get_range_slices-to-apply-filter-to-a-sequenti-v2.patch,
> 0002-allow-get_range_slices-to-apply-filter-to-a-sequential.txt
>
>
> From a comment on 1157:
> {quote}
> IndexClause only has a start key for get_indexed_slices, but it would seem
> that the reasoning behind using 'KeyRange' for get_range_slices applies there
> as well, since if you know the range you care about in the primary index, you
> don't want to continue scanning until you exhaust 'count' (or the cluster).
> Since it would appear that get_indexed_slices would benefit from a KeyRange,
> why not smash get_(range|indexed)_slices together, and make IndexClause an
> optional field on KeyRange?
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1805) refactor and remove contrib/
[ https://issues.apache.org/jira/browse/CASSANDRA-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018054#comment-13018054 ] Jeremy Hanna commented on CASSANDRA-1805: - Jonathan (and Brandon) - re: pig moving to core: did you have something in mind in particular for a build artifact for the Pig loadfunc/storefunc? It would be nice to have it in core to ensure it doesn't get neglected with updates (and rely on the mvn central pig 0.8). It would also be nice to have a cassandra_storage.jar that contains only what is necessary. That or was the intention to have everything in one jar (Cassandra + load/storefunc) since both will be needed in any case when using it. So I guess just a packaging question. > refactor and remove contrib/ > > > Key: CASSANDRA-1805 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1805 > Project: Cassandra > Issue Type: Task >Reporter: Jonathan Ellis >Priority: Minor > Fix For: 0.8 > > Attachments: 1805-sstabledebug.txt > > > Contrib is a mix of examples, tools, and miscellanea that probably doesn't > belong in our source tree. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2371) Removed/Dead Node keeps reappearing
[ https://issues.apache.org/jira/browse/CASSANDRA-2371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018057#comment-13018057 ] Jeremy Hanna commented on CASSANDRA-2371: - I applied the patch and within a few days even with no restarting of any of the Cassandra nodes, the removed token came back. Just FYI. > Removed/Dead Node keeps reappearing > --- > > Key: CASSANDRA-2371 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2371 > Project: Cassandra > Issue Type: Bug > Components: Tools >Affects Versions: 0.7.3, 0.7.4 > Environment: Large Amazon EC2 instances. Ubuntu 10.04.2 >Reporter: techlabs >Assignee: Brandon Williams >Priority: Minor > Fix For: 0.7.5 > > Attachments: 2371.txt > > > The removetoken option does not seem to work. The original node 10.240.50.63 > comes back into the ring, even after the EC2 instance is no longer in > existence. Originally I tried to add a new node 10.214.103.224 with the same > token, but there were some complications with that. I have pasted below all > the INFO log entries found with greping the system log files. > Seems to be a similar issue seen with > http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Ghost-node-showing-up-in-the-ring-td6198180.html > > INFO [GossipStage:1] 2011-03-16 00:54:31,590 StorageService.java (line 745) > Nodes /10.214.103.224 and /10.240.50.63 have the same token > 957044156965139000. /10.214.103.224 is the new owner > INFO [GossipStage:1] 2011-03-16 17:26:51,083 StorageService.java (line 865) > Removing token 957044156965139000 for /10.214.103.224 > INFO [GossipStage:1] 2011-03-19 17:27:24,767 StorageService.java (line 865) > Removing token 957044156965139000 for /10.214.103.224 > INFO [GossipStage:1] 2011-03-19 17:29:30,191 StorageService.java (line 865) > Removing token 957044156965139000 for /10.214.103.224 > INFO [GossipStage:1] 2011-03-19 17:31:35,609 StorageService.java (line 865) > Removing token 957044156965139000 for /10.214.103.224 > INFO [GossipStage:1] 2011-03-19 17:33:39,440 StorageService.java (line 865) > Removing token 957044156965139000 for /10.214.103.224 > INFO [GossipStage:1] 2011-03-23 17:22:55,520 StorageService.java (line 865) > Removing token 957044156965139000 for /10.240.50.63 > INFO [GossipStage:1] 2011-03-10 03:52:37,299 Gossiper.java (line 608) Node > /10.240.50.63 is now part of the cluster > INFO [GossipStage:1] 2011-03-10 03:52:37,545 Gossiper.java (line 600) > InetAddress /10.240.50.63 is now UP > INFO [HintedHandoff:1] 2011-03-10 03:53:36,168 HintedHandOffManager.java > (line 304) Started hinted handoff for endpoint /10.240.50.63 > INFO [HintedHandoff:1] 2011-03-10 03:53:36,169 HintedHandOffManager.java > (line 360) Finished hinted handoff of 0 rows to endpoint /10.240.50.63 > INFO [GossipStage:1] 2011-03-15 23:23:43,770 Gossiper.java (line 623) Node > /10.240.50.63 has restarted, now UP again > INFO [GossipStage:1] 2011-03-15 23:23:43,771 StorageService.java (line 726) > Node /10.240.50.63 state jump to normal > INFO [HintedHandoff:1] 2011-03-15 23:28:48,957 HintedHandOffManager.java > (line 304) Started hinted handoff for endpoint /10.240.50.63 > INFO [HintedHandoff:1] 2011-03-15 23:28:48,958 HintedHandOffManager.java > (line 360) Finished hinted handoff of 0 rows to endpoint /10.240.50.63 > INFO [ScheduledTasks:1] 2011-03-15 23:37:25,071 Gossiper.java (line 226) > InetAddress /10.240.50.63 is now dead. > INFO [GossipStage:1] 2011-03-16 00:54:31,590 StorageService.java (line 745) > Nodes /10.214.103.224 and /10.240.50.63 have the same token > 957044156965139000. /10.214.103.224 is the new owner > WARN [GossipStage:1] 2011-03-16 00:54:31,590 TokenMetadata.java (line 115) > Token 957044156965139000 changing ownership from > /10.240.50.63 to /10.214.103.224 > INFO [GossipStage:1] 2011-03-18 23:37:09,158 Gossiper.java (line 610) Node > /10.240.50.63 is now part of the cluster > INFO [GossipStage:1] 2011-03-21 23:37:10,421 Gossiper.java (line 610) Node > /10.240.50.63 is now part of the cluster > INFO [GossipStage:1] 2011-03-21 23:37:10,421 StorageService.java (line 726) > Node /10.240.50.63 state jump to normal > INFO [GossipStage:1] 2011-03-23 17:22:55,520 StorageService.java (line 865) > Removing token 957044156965139000 for /10.240.50.63 > INFO [ScheduledTasks:1] 2011-03-23 17:22:55,521 HintedHandOffManager.java > (line 210) Deleting any stored hints for 10.240.50.63 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/softwar
[jira] [Commented] (CASSANDRA-2191) Multithread across compaction buckets
[
https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018063#comment-13018063
]
Sylvain Lebresne commented on CASSANDRA-2191:
-
Some comments on the new patches:
* I (really) don't like the stopTheWorld function. I don't like the name and
arguments names and I don't find clear the 'abstraction' it provides. But more
importantly it makes the 'unmarking' of sstable very fragile, since the
markCompacting function *is not* in the 'try .. final' block where the unmark
function is. Let's just inline this stopTheWorld function instead (it won't
really be more code).
* The forcing of major compaction in doCompaction is buggy, because there is no
guarantee we haven't skipped some sstable (because of the skip option) or that
some sstable has been removed from the set by lack of disk space. Forcing a
major compaction in those case is wrong. I know that when you submit a major
compaction you could end up doing a minor one just because a sstable got
flushed between the time you grabbed the set of sstables and the time you
checked this set is still complete. But it is not a new problem (nor a big one,
contrarily to wrongfully removing tombstones) so let's open a separate ticket
if we want to fix that.
* The ksname and cfname field in CompactionController should be replaced by
getters (of cfs.metadata.{ksname|cfname}), if only to make sure we don't make
the renaming of ks/cf more problematic that it already is (it's probably no
problem here but avoiding the duplication of fields would be cool anyway).
* As far as I can tell we still have a problem with cache saving in that you
can have two cache saving happening at the same time (which will be bad). To
answer the proposition of a previous comment, I am not a fan of using the flush
writer for this as it has more important things to do (and we would lose the
reporting through JMX). A specific executor could solve the problem but it is a
bit overkill. Maybe a static atomicBoolean that cache writing tasks would
atomically compare to false and set to true at the beginning. If that succeed
they would do the cache writing and set back to false the boolean, otherwise
the task would just return directly. It means it would discard a cache saving
if one is already running but that's probably fine (that's actually probably
what you want).
* I would still prefer a IdentityHashSet rather than a HashMap whose value is
unused in CompactionExecutor for the sake of the clarity of the code.
* In nodetool printCompactionStats, we should keep the println with plain
English as it was before instead of using the much denser (and much less user
friendly) CompactionInfo.toString().
* In SSTableWriter.Builder, when creating the compactionInfo, putting the
OperationType as it is done will look ugly and I think it will be more
confusing than anything else to the user (the previous message was probably
quite enough).
FYI, my monday morning happens to be quite a few hours before the monday
morning 'freeze'. So if there is fixes for the things above by my monday
morning, I'll look at it in priority.
> Multithread across compaction buckets
> -
>
> Key: CASSANDRA-2191
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2191
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Stu Hood
>Assignee: Stu Hood
>Priority: Critical
> Labels: compaction
> Fix For: 0.8
>
> Attachments: 0001-Add-a-compacting-set-to-DataTracker.txt,
> 0002-Use-the-compacting-set-of-sstables-to-schedule-multith.txt,
> 0003-Expose-multiple-compactions-via-JMX-and-a-concrete-ser.txt,
> 0004-Allow-multithread-compaction-to-be-disabled.txt,
> 0005-Add-a-harness-to-allow-compaction-tasks-that-need-to-a.txt
>
>
> This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and
> reasoning are different enough to open a separate issue.
> The problem with compactions currently is that they compact the set of
> sstables that existed the moment the compaction started. This means that for
> longer running compactions (even when running as fast as possible on the
> hardware), a very large number of new sstables might be created in the
> meantime. We have observed this proliferation of sstables killing performance
> during major/high-bucketed compactions.
> One approach would be to pause compactions in upper buckets (containing
> larger files) when compactions in lower buckets become possible. While this
> would likely solve the problem with read performance, it does not actually
> help us perform compaction any faster, which is a reasonable requirement for
> other situations.
> Instead, we need to be able to perform any compactions that are current
[jira] [Commented] (CASSANDRA-2440) Merge ColumnOrSuperColumn and Counter thrift structure to avoid most of the counter API call duplication
[ https://issues.apache.org/jira/browse/CASSANDRA-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018065#comment-13018065 ] Sylvain Lebresne commented on CASSANDRA-2440: - bq. Could we perhaps also merge CounterSuperColumn with SuperColumn by giving SuperColumn two lists? I could see pros and cons for that. The cons for instance would be that with the patch as is, the COSC structure has a kind of nice symmetry (normal columm and sc/counter column and sc). In particular the distinction for counter/standard is done at the same 'level' (the COSC leve) which could maybe be a little simpler for client. Lastly, I'm more reluctant to make field of SuperColumn optional while Mutation already has all of its field optional. Anyway I guess I don't really care so much one way or another. The main point was to remove all the duplicate API calls. But since I think I have a tiny preference for how it is right now, I'd be willing to be lazy and leave it at that unless there is strong feelings expressed for doing otherwise. > Merge ColumnOrSuperColumn and Counter thrift structure to avoid most of the > counter API call duplication > > > Key: CASSANDRA-2440 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2440 > Project: Cassandra > Issue Type: Improvement > Components: API >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Fix For: 0.8 > > Attachments: cassandra.thrift > > Original Estimate: 8h > Remaining Estimate: 8h > > Having duplicate calls for all counter operation is ugly. Distinguishing a > Column from a CounterColumn is a good idea, but I feel we went too far in > making counters an API on its own. This ticket proposes to merge the Counter > and ColumnOrSuperColumn thrift structure (while keeping CounterColumn and > CounterSuperColumn) and to remove all the duplicate API calls. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2440) Merge ColumnOrSuperColumn and Counter thrift structure to avoid most of the counter API call duplication
[ https://issues.apache.org/jira/browse/CASSANDRA-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2440: Attachment: 0002-Merge-Counter-and-ColumnOrSuperColumn-thrift-structu.patch 0001-Thrift-changes.patch Patch attached (first patch is cassandra.thrift plus the thrift generated file changes, patch two is the rest). > Merge ColumnOrSuperColumn and Counter thrift structure to avoid most of the > counter API call duplication > > > Key: CASSANDRA-2440 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2440 > Project: Cassandra > Issue Type: Improvement > Components: API >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Fix For: 0.8 > > Attachments: 0001-Thrift-changes.patch, > 0002-Merge-Counter-and-ColumnOrSuperColumn-thrift-structu.patch, > cassandra.thrift > > Original Estimate: 8h > Remaining Estimate: 8h > > Having duplicate calls for all counter operation is ugly. Distinguishing a > Column from a CounterColumn is a good idea, but I feel we went too far in > making counters an API on its own. This ticket proposes to merge the Counter > and ColumnOrSuperColumn thrift structure (while keeping CounterColumn and > CounterSuperColumn) and to remove all the duplicate API calls. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2435) auto bootstrap happened on already bootstrapped nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018072#comment-13018072 ] Peter Schuller commented on CASSANDRA-2435: --- FWIW, looks good to me (but I only did visual inspection and some code jumping in the 0.7 branch; haven't tested it). > auto bootstrap happened on already bootstrapped nodes > - > > Key: CASSANDRA-2435 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2435 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.7.0 >Reporter: Peter Schuller >Assignee: Jonathan Ellis >Priority: Critical > Fix For: 0.7.5 > > Attachments: 2435.txt > > > I believe the following was observed on 0.7.2. I meant to dig deeper, but > never had the time, and now I want to at least file this even if I don't have > extremely helpful information. > A piece of background is that we consciously made the decision to have the > default configuration on nodes have auto_bootstrap set to true. The logic was > that if one accidentally were to start a new node, we'd rather have it join > with data than join *without* data and cause bogus read results in the > cluster. > We executed this policy (by way of having the puppet managed config have > auto_bootstrap set to true). > On one of our clusters with 5 nodes, we did some moves. All looked well; the > moves completed. For unrelated reasons, we wanted to restart nodes after they > had been moved. When we did, three of the 5, specifically those 3 that were > *NOT* seed nodes, initiated a bootstrap procedure! Before the moves the > cluster had been running for several days at least. > The logs indicated the automatic token selection, and they joined the ring > under a new automatically selected token. > Presumably, this violated consistency but at the time there was no live > traffic to the cluster and we didn't confirm (put traffic on it after > repair+cleanup). > I did look a little bit at the code in light of this but didn't see anything > obvious, so I don't really know what the likely culprit is. > A potential complication was that seed nodes were moved without using the > correct procedure of de-seeding them first. This was clearly wrong, but it is > not obvious to me that it would cause other nodes to incorrectly bootstrap > since a node should *never* bootstrap more than once if the local system > tables say it's been bootstrapped. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1805) refactor and remove contrib/
[ https://issues.apache.org/jira/browse/CASSANDRA-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018079#comment-13018079 ] Jonathan Ellis commented on CASSANDRA-1805: --- I wouldn't mind seeing a cassandra-hadoop jar but if just shoving it in the main Cassandra jar is easier that is fine. > refactor and remove contrib/ > > > Key: CASSANDRA-1805 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1805 > Project: Cassandra > Issue Type: Task >Reporter: Jonathan Ellis >Priority: Minor > Fix For: 0.8 > > Attachments: 1805-sstabledebug.txt > > > Contrib is a mix of examples, tools, and miscellanea that probably doesn't > belong in our source tree. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1090798 - in /cassandra/trunk: doc/cql/CQL.textile src/java/org/apache/cassandra/cql/Cql.g src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java test/system/test_cql.py
Author: jbellis
Date: Sun Apr 10 13:39:11 2011
New Revision: 1090798
URL: http://svn.apache.org/viewvc?rev=1090798&view=rev
Log:
add 'int' as sql alias for 'bigint'
patch by jbellis
Modified:
cassandra/trunk/doc/cql/CQL.textile
cassandra/trunk/src/java/org/apache/cassandra/cql/Cql.g
cassandra/trunk/src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java
cassandra/trunk/test/system/test_cql.py
Modified: cassandra/trunk/doc/cql/CQL.textile
URL:
http://svn.apache.org/viewvc/cassandra/trunk/doc/cql/CQL.textile?rev=1090798&r1=1090797&r2=1090798&view=diff
==
--- cassandra/trunk/doc/cql/CQL.textile (original)
+++ cassandra/trunk/doc/cql/CQL.textile Sun Apr 10 13:39:11 2011
@@ -207,6 +207,7 @@ It is possible to assign columns a type
|varchar|UTF8 encoded string|
|uuid|Type 1, or type 4 UUID|
|varint|Arbitrary-precision integer|
+|int|8-byte long (same as bigint)|
|bigint|8-byte long|
_Note: In addition to the recognized types listed above, it is also possible
to supply a string containing the name of a class (a sub-class of
@AbstractType@), either fully qualified, or relative to the
@org.apache.cassandra.db.marshal@ package._
@@ -288,6 +289,7 @@ How column name/value terms are interpre
|uuid|The string @now@, to represent a type-1 (time-based) UUID with a
date-time component based on the current time|
|uuid|Numeric value representing milliseconds since epoch|
|uuid|An "iso8601 timestamp":http://en.wikipedia.org/wiki/8601|
+|int|Integer value capable of fitting in 8 bytes (same as bigint)|
|bigint|Integer value capable of fitting in 8 bytes|
|varint|Integer value of arbitrary size|
|bytea|Hex-encoded strings (converted directly to the corresponding bytes)|
Modified: cassandra/trunk/src/java/org/apache/cassandra/cql/Cql.g
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cql/Cql.g?rev=1090798&r1=1090797&r2=1090798&view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/cql/Cql.g (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/cql/Cql.g Sun Apr 10 13:39:11
2011
@@ -342,7 +342,7 @@ dropColumnFamilyStatement returns [Strin
;
comparatorType
-: 'bytea' | 'ascii' | 'text' | 'varchar' | 'varint' | 'bigint' | 'uuid'
+: 'bytea' | 'ascii' | 'text' | 'varchar' | 'int' | 'varint' | 'bigint' |
'uuid'
;
term returns [Term item]
Modified:
cassandra/trunk/src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java?rev=1090798&r1=1090797&r2=1090798&view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java
Sun Apr 10 13:39:11 2011
@@ -68,8 +68,10 @@ public class CreateColumnFamilyStatement
comparators.put("text", "UTF8Type");
comparators.put("varchar", "UTF8Type");
comparators.put("varint", "IntegerType");
+comparators.put("int", "LongType");
comparators.put("bigint", "LongType");
comparators.put("uuid", "UUIDType");
+
keywords.add(KW_COMPARATOR);
keywords.add(KW_COMMENT);
keywords.add(KW_ROWCACHESIZE);
Modified: cassandra/trunk/test/system/test_cql.py
URL:
http://svn.apache.org/viewvc/cassandra/trunk/test/system/test_cql.py?rev=1090798&r1=1090797&r2=1090798&view=diff
==
--- cassandra/trunk/test/system/test_cql.py (original)
+++ cassandra/trunk/test/system/test_cql.py Sun Apr 10 13:39:11 2011
@@ -68,7 +68,7 @@ def load_sample(dbconn):
WITH comparator = ascii AND default_validation = uuid;
""")
dbconn.execute("""
-CREATE COLUMNFAMILY IndexedA (KEY text PRIMARY KEY, birthdate bigint)
+CREATE COLUMNFAMILY IndexedA (KEY text PRIMARY KEY, birthdate int)
WITH comparator = ascii AND default_validation = ascii;
""")
dbconn.execute("CREATE INDEX ON IndexedA (birthdate)")
svn commit: r1090802 - /cassandra/branches/cassandra-0.7/CHANGES.txt
Author: jbellis Date: Sun Apr 10 13:53:22 2011 New Revision: 1090802 URL: http://svn.apache.org/viewvc?rev=1090802&view=rev Log: update CHANGES Modified: cassandra/branches/cassandra-0.7/CHANGES.txt Modified: cassandra/branches/cassandra-0.7/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1090802&r1=1090801&r2=1090802&view=diff == --- cassandra/branches/cassandra-0.7/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.7/CHANGES.txt Sun Apr 10 13:53:22 2011 @@ -24,6 +24,7 @@ * fix race condition that could leave orphaned data files when dropping CF or KS (CASSANDRA-2381) * halve default memtable thresholds (CASSANDRA-2413) + * convert mmap assertion to if/throw so scrub can catch it (CASSANDRA-2417) * Try harder to close files after compaction (CASSANDRA-2431)
[jira] [Commented] (CASSANDRA-2440) Merge ColumnOrSuperColumn and Counter thrift structure to avoid most of the counter API call duplication
[ https://issues.apache.org/jira/browse/CASSANDRA-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018085#comment-13018085 ] Jonathan Ellis commented on CASSANDRA-2440: --- +1 > Merge ColumnOrSuperColumn and Counter thrift structure to avoid most of the > counter API call duplication > > > Key: CASSANDRA-2440 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2440 > Project: Cassandra > Issue Type: Improvement > Components: API >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Fix For: 0.8 > > Attachments: 0001-Thrift-changes.patch, > 0002-Merge-Counter-and-ColumnOrSuperColumn-thrift-structu.patch, > cassandra.thrift > > Original Estimate: 8h > Remaining Estimate: 8h > > Having duplicate calls for all counter operation is ugly. Distinguishing a > Column from a CounterColumn is a good idea, but I feel we went too far in > making counters an API on its own. This ticket proposes to merge the Counter > and ColumnOrSuperColumn thrift structure (while keeping CounterColumn and > CounterSuperColumn) and to remove all the duplicate API calls. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2441) Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10
[ https://issues.apache.org/jira/browse/CASSANDRA-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018096#comment-13018096 ] Jonathan Ellis commented on CASSANDRA-2441: --- Also jrockit: http://www.oracle.com/technetwork/middleware/jrockit/downloads/index.html > Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10 > > > Key: CASSANDRA-2441 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2441 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.8 > Environment: Both servers have identical hardware configuration: > Quad-Core AMD Opteron(tm) Processor 2374 HE, 4 GB RAM (rackspace servers) > Java version "1.6.0_20" > OpenJDK Runtime Environment (IcedTea6 1.9.7) (6b20-1.9.7-0ubuntu1) > OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode) >Reporter: Pavel Yaskevich >Assignee: Jonathan Ellis >Priority: Critical > > Last working commit is c8d1984bf17cab58f40069e522d074c7b0077bc1 (merge from > 0.7), branch: trunk. > What I did is cloned git://git.apache.org/cassandra.git and did git reset > each commit with `ant clean && ant && ./bin/cassandra -f` until I got > cassandra started -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1263) Push replication factor down to the replication strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-1263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018097#comment-13018097 ] Jonathan Ellis commented on CASSANDRA-1263: --- (working on the rebase here, it's pretty straightforward) > Push replication factor down to the replication strategy > > > Key: CASSANDRA-1263 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1263 > Project: Cassandra > Issue Type: Task > Components: Core >Reporter: Jeremy Hanna >Assignee: Jon Hermes >Priority: Minor > Fix For: 0.8 > > Attachments: 1263-2.txt, 1263-3.txt, 1263-4.txt, 1263-5.txt, > 1263-incomplete.txt, 1263.txt > > Original Estimate: 8h > Remaining Estimate: 8h > > Currently the replication factor is in the keyspace metadata. As we've added > the datacenter shard strategy, the replication factor becomes more computed > by the replication strategy. It seems reasonable to therefore push the > replication factor for the keyspace down to the replication strategy so that > it can be handled in one place. > This adds on the work being done in CASSANDRA-1066 since that ticket will > make the replication strategy a member variable of keyspace metadata instead > of just a quasi singleton giving the replication strategy state for each > keyspace. That makes it able to have the replication factor. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1090842 [1/3] - in /cassandra/trunk: interface/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/cli/ src/java/org/apache/cassandra/db/ src/java/org/ap
Author: slebresne
Date: Sun Apr 10 17:31:15 2011
New Revision: 1090842
URL: http://svn.apache.org/viewvc?rev=1090842&view=rev
Log:
Merge COSC and Counter thrift structure
patch by slebresne; reviewed by jbellis for CASSANDRA-2440
Removed:
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Counter.java
Modified:
cassandra/trunk/interface/cassandra.thrift
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/CfDef.java
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/ColumnOrSuperColumn.java
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Mutation.java
cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java
cassandra/trunk/src/java/org/apache/cassandra/db/RowMutation.java
cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraServer.java
cassandra/trunk/src/java/org/apache/cassandra/thrift/ThriftValidation.java
cassandra/trunk/test/system/test_thrift_server.py
cassandra/trunk/tools/stress/src/org/apache/cassandra/stress/operations/CounterAdder.java
cassandra/trunk/tools/stress/src/org/apache/cassandra/stress/operations/CounterGetter.java
Modified: cassandra/trunk/interface/cassandra.thrift
URL:
http://svn.apache.org/viewvc/cassandra/trunk/interface/cassandra.thrift?rev=1090842&r1=1090841&r2=1090842&view=diff
==
--- cassandra/trunk/interface/cassandra.thrift (original)
+++ cassandra/trunk/interface/cassandra.thrift Sun Apr 10 17:31:15 2011
@@ -76,6 +76,16 @@ struct SuperColumn {
2: required list columns,
}
+struct CounterColumn {
+1: required binary name,
+2: required i64 value
+}
+
+struct CounterSuperColumn {
+1: required binary name,
+2: required list columns
+}
+
/**
Methods for fetching rows/records from Cassandra will return either a
single instance of ColumnOrSuperColumn or a list
of ColumnOrSuperColumns (get_slice()). If you're looking up a SuperColumn
(or list of SuperColumns) then the resulting
@@ -83,12 +93,19 @@ struct SuperColumn {
in Columns, those values will be in the attribute column. This change was
made between 0.3 and 0.4 to standardize on
single query methods that may return either a SuperColumn or Column.
+If the query was on a counter column family, you will either get a
counter_column (instead of a column) or a
+counter_super_column (instead of a super_column)
+
@param column. The Column returned by get() or get_slice().
@param super_column. The SuperColumn returned by get() or get_slice().
+@param counter_column. The Counterolumn returned by get() or get_slice().
+@param counter_super_column. The CounterSuperColumn returned by get() or
get_slice().
*/
struct ColumnOrSuperColumn {
1: optional Column column,
2: optional SuperColumn super_column,
+3: optional CounterColumn counter_column,
+4: optional CounterSuperColumn counter_super_column
}
@@ -213,21 +230,6 @@ struct ColumnPath {
5: optional binary column,
}
-struct CounterColumn {
-1: required binary name,
-2: required i64 value
-}
-
-struct CounterSuperColumn {
-1: required binary name,
-2: required list columns
-}
-
-struct Counter {
-1: optional CounterColumn column,
-2: optional CounterSuperColumn super_column
-}
-
/**
A slice range is a structure that stores basic range, ordering and limit
information for a query that will return
multiple columns. It could be thought of as Cassandra's version of LIMIT
and ORDER BY
@@ -331,15 +333,13 @@ struct Deletion {
}
/**
-A Mutation is either an insert (represented by filling
column_or_supercolumn), a deletion (represented by filling the deletion
attribute),
-a counter addition (represented by filling counter), or a counter deletion
(represented by filling counter_deletion).
-@param column_or_supercolumn. An insert to a column or supercolumn
+A Mutation is either an insert (represented by filling
column_or_supercolumn) or a deletion (represented by filling the deletion
attribute).
+@param column_or_supercolumn. An insert to a column or supercolumn
(possibly counter column or supercolumn)
@param deletion. A deletion of a column or supercolumn
*/
struct Mutation {
1: optional ColumnOrSuperColumn column_or_supercolumn,
2: optional Deletion deletion,
-3: optional Counter counter,
}
struct TokenRange {
@@ -390,7 +390,7 @@ struct CfDef {
21: optional i32 memtable_flush_after_mins,
22: optional i32 memtable_throughput_in_mb,
23: optional double memtable_operations_in_millions,
-24: optional bool replicate_on_write=0,
+24: optional bool replicate_on_write,
25: optional double merge_shards_chance,
26: optional string key_validation_class,
27: optiona
[jira] [Commented] (CASSANDRA-2441) Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10
[ https://issues.apache.org/jira/browse/CASSANDRA-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018117#comment-13018117 ] Pavel Yaskevich commented on CASSANDRA-2441: Latest code (branch trunk) works with JRockit (jrockit-jdk1.6.0_22-R28.1.1-4.0.1) but note that JVM does not support following options: -Xmn -XX:ThreadPriorityPolicy -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly > Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10 > > > Key: CASSANDRA-2441 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2441 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.8 > Environment: Both servers have identical hardware configuration: > Quad-Core AMD Opteron(tm) Processor 2374 HE, 4 GB RAM (rackspace servers) > Java version "1.6.0_20" > OpenJDK Runtime Environment (IcedTea6 1.9.7) (6b20-1.9.7-0ubuntu1) > OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode) >Reporter: Pavel Yaskevich >Assignee: Jonathan Ellis >Priority: Critical > > Last working commit is c8d1984bf17cab58f40069e522d074c7b0077bc1 (merge from > 0.7), branch: trunk. > What I did is cloned git://git.apache.org/cassandra.git and did git reset > each commit with `ant clean && ant && ./bin/cassandra -f` until I got > cassandra started -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2441) Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10
[
https://issues.apache.org/jira/browse/CASSANDRA-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018127#comment-13018127
]
Pavel Yaskevich commented on CASSANDRA-2441:
using IBM JDK started without any changes to the conf/cassandra-env.sh and
everything worked fine.
{noformat}
java version "1.6.0"
Java(TM) SE Runtime Environment (build pxi3260sr9fp1-20110208_03(SR9 FP1))
IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32
jvmxi3260sr9-20110203_74623 (JIT enabled, AOT enabled)
J9VM - 20110203_074623
JIT - r9_20101028_17488ifx3
GC - 20101027_AA)
JCL - 20110203_01
{noformat}
> Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10
>
>
> Key: CASSANDRA-2441
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2441
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.8
> Environment: Both servers have identical hardware configuration:
> Quad-Core AMD Opteron(tm) Processor 2374 HE, 4 GB RAM (rackspace servers)
> Java version "1.6.0_20"
> OpenJDK Runtime Environment (IcedTea6 1.9.7) (6b20-1.9.7-0ubuntu1)
> OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)
>Reporter: Pavel Yaskevich
>Assignee: Jonathan Ellis
>Priority: Critical
>
> Last working commit is c8d1984bf17cab58f40069e522d074c7b0077bc1 (merge from
> 0.7), branch: trunk.
> What I did is cloned git://git.apache.org/cassandra.git and did git reset
> each commit with `ant clean && ant && ./bin/cassandra -f` until I got
> cassandra started
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2324) Repair transfers more data than necessary
[
https://issues.apache.org/jira/browse/CASSANDRA-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018132#comment-13018132
]
Hudson commented on CASSANDRA-2324:
---
Integrated in Cassandra #844 (See
[https://hudson.apache.org/hudson/job/Cassandra/844/])
Make repair work on a token range instead of the full ring
patch by slebresne; reviewed by stuhood for CASSANDRA-2324
> Repair transfers more data than necessary
> -
>
> Key: CASSANDRA-2324
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2324
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.6
>Reporter: Brandon Williams
>Assignee: Sylvain Lebresne
> Fix For: 0.8
>
> Attachments:
> 0001-Make-repair-operate-over-a-node-token-range-v2.patch,
> 0001-Make-repair-operate-over-a-node-token-range.patch
>
>
> To repro: 3 node cluster, stress.java 1M rows with -x KEYS and -l 2. The
> index is enough to make some mutations drop (about 20-30k total in my tests).
> Repair afterwards will repair a large amount of ranges the first time.
> However, each subsequent run will repair the same set of small ranges every
> time. INDEXED_RANGE_SLICE in stress never fully works. Counting rows with
> sstablekeys shows there are 2M rows total as expected, however when trying to
> count the indexed keys, I get exceptions like:
> {noformat}
> Exception in thread "main" java.io.IOException: Key out of order!
> DecoratedKey(101571366040797913119296586470838356016,
> 0707ab782c5b5029d28a5e6d508ef72f0222528b5e28da3b7787492679dc51b96f868e0746073e54bc173be927049d0f51e25a6a95b3268213b8969abf40cea7d7)
> > DecoratedKey(12639574763031545147067490818595764132,
> 0bc414be3093348a2ad389ed28f18f0cc9a044b2e98587848a0d289dae13ed0ad479c74654900eeffc6236)
> at
> org.apache.cassandra.tools.SSTableExport.enumeratekeys(SSTableExport.java:206)
> at
> org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:388)
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2440) Merge ColumnOrSuperColumn and Counter thrift structure to avoid most of the counter API call duplication
[ https://issues.apache.org/jira/browse/CASSANDRA-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018131#comment-13018131 ] Hudson commented on CASSANDRA-2440: --- Integrated in Cassandra #844 (See [https://hudson.apache.org/hudson/job/Cassandra/844/]) Merge COSC and Counter thrift structure patch by slebresne; reviewed by jbellis for CASSANDRA-2440 > Merge ColumnOrSuperColumn and Counter thrift structure to avoid most of the > counter API call duplication > > > Key: CASSANDRA-2440 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2440 > Project: Cassandra > Issue Type: Improvement > Components: API >Affects Versions: 0.8 >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Fix For: 0.8 > > Attachments: 0001-Thrift-changes.patch, > 0002-Merge-Counter-and-ColumnOrSuperColumn-thrift-structu.patch, > cassandra.thrift > > Original Estimate: 8h > Remaining Estimate: 8h > > Having duplicate calls for all counter operation is ugly. Distinguishing a > Column from a CounterColumn is a good idea, but I feel we went too far in > making counters an API on its own. This ticket proposes to merge the Counter > and ColumnOrSuperColumn thrift structure (while keeping CounterColumn and > CounterSuperColumn) and to remove all the duplicate API calls. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1090866 - in /cassandra/trunk: CHANGES.txt src/java/org/apache/cassandra/io/CompactionController.java src/java/org/apache/cassandra/io/CompactionIterator.java
Author: slebresne
Date: Sun Apr 10 18:42:30 2011
New Revision: 1090866
URL: http://svn.apache.org/viewvc?rev=1090866&view=rev
Log:
Purge tombstone from row cache
patch by slebresne; reviewed by jbellis for CASSANDRA-2305
Modified:
cassandra/trunk/CHANGES.txt
cassandra/trunk/src/java/org/apache/cassandra/io/CompactionController.java
cassandra/trunk/src/java/org/apache/cassandra/io/CompactionIterator.java
Modified: cassandra/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1090866&r1=1090865&r2=1090866&view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Sun Apr 10 18:42:30 2011
@@ -16,6 +16,7 @@
* Fix clustertool to not throw exception when calling get_endpoints
(CASSANDRA-2437)
* upgrade to thrift 0.6 (CASSANDRA-2412)
* repair works on a token range instead of full ring (CASSANDRA-2324)
+ * purge tombstone from row cache (CASSANDRA-2305)
0.7.5
Modified:
cassandra/trunk/src/java/org/apache/cassandra/io/CompactionController.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/io/CompactionController.java?rev=1090866&r1=1090865&r2=1090866&view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/io/CompactionController.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/io/CompactionController.java
Sun Apr 10 18:42:30 2011
@@ -24,8 +24,9 @@ import java.util.Collections;
import java.util.HashSet;
import java.util.Set;
-import org.apache.cassandra.db.DecoratedKey;
+import org.apache.cassandra.db.ColumnFamily;
import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.DecoratedKey;
import org.apache.cassandra.io.sstable.SSTableReader;
/**
@@ -82,4 +83,15 @@ public class CompactionController
if (cfs != null)
cfs.invalidateCachedRow(key);
}
+
+public void removeDeletedInCache(DecoratedKey key)
+{
+if (cfs != null)
+{
+ColumnFamily cachedRow = cfs.getRawCachedRow(key);
+if (cachedRow != null)
+ColumnFamilyStore.removeDeleted(cachedRow, gcBefore);
+}
+}
+
}
Modified:
cassandra/trunk/src/java/org/apache/cassandra/io/CompactionIterator.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/io/CompactionIterator.java?rev=1090866&r1=1090865&r2=1090866&view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/io/CompactionIterator.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/io/CompactionIterator.java
Sun Apr 10 18:42:30 2011
@@ -34,6 +34,7 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.db.ColumnFamily;
import org.apache.cassandra.db.ColumnFamilyStore;
import org.apache.cassandra.io.sstable.SSTableIdentityIterator;
import org.apache.cassandra.io.sstable.SSTableReader;
@@ -110,10 +111,13 @@ implements Closeable, ICompactionInfo
controller.invalidateCachedRow(compactedRow.key);
return null;
}
-else
-{
-return compactedRow;
-}
+
+// If the raw is cached, we call removeDeleted on it to have/
coherent query returns. However it would look
+// like some deleted columns lived longer than gc_grace +
compaction. This can also free up big amount of
+// memory on long running instances
+controller.removeDeletedInCache(compactedRow.key);
+
+return compactedRow;
}
finally
{
svn commit: r1090867 - /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/CompactionIterator.java
Author: slebresne
Date: Sun Apr 10 18:49:54 2011
New Revision: 1090867
URL: http://svn.apache.org/viewvc?rev=1090867&view=rev
Log:
Purge tombstone from row cache (0.7 version)
patch by slebresne; reviewed by jbellis for CASSANDRA-2305
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/CompactionIterator.java
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/CompactionIterator.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/CompactionIterator.java?rev=1090867&r1=1090866&r2=1090867&view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/CompactionIterator.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/CompactionIterator.java
Sun Apr 10 18:49:54 2011
@@ -32,6 +32,7 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.db.ColumnFamily;
import org.apache.cassandra.db.ColumnFamilyStore;
import org.apache.cassandra.io.sstable.SSTableIdentityIterator;
import org.apache.cassandra.io.sstable.SSTableReader;
@@ -112,10 +113,15 @@ implements Closeable, ICompactionInfo
cfs.invalidateCachedRow(compactedRow.key);
return null;
}
-else
-{
-return compactedRow;
-}
+
+// If the raw is cached, we call removeDeleted on it to have/
coherent query returns. However it would look
+// like some deleted columns lived longer than gc_grace +
compaction. This can also free up big amount of
+// memory on long running instances
+ColumnFamily cachedRow = cfs.getRawCachedRow(compactedRow.key);
+if (cachedRow != null)
+ColumnFamilyStore.removeDeleted(cachedRow, gcBefore);
+
+return compactedRow;
}
finally
{
[jira] [Commented] (CASSANDRA-2435) auto bootstrap happened on already bootstrapped nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018135#comment-13018135 ] Nick Bailey commented on CASSANDRA-2435: +1 > auto bootstrap happened on already bootstrapped nodes > - > > Key: CASSANDRA-2435 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2435 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.7.0 >Reporter: Peter Schuller >Assignee: Jonathan Ellis >Priority: Critical > Fix For: 0.7.5 > > Attachments: 2435.txt > > > I believe the following was observed on 0.7.2. I meant to dig deeper, but > never had the time, and now I want to at least file this even if I don't have > extremely helpful information. > A piece of background is that we consciously made the decision to have the > default configuration on nodes have auto_bootstrap set to true. The logic was > that if one accidentally were to start a new node, we'd rather have it join > with data than join *without* data and cause bogus read results in the > cluster. > We executed this policy (by way of having the puppet managed config have > auto_bootstrap set to true). > On one of our clusters with 5 nodes, we did some moves. All looked well; the > moves completed. For unrelated reasons, we wanted to restart nodes after they > had been moved. When we did, three of the 5, specifically those 3 that were > *NOT* seed nodes, initiated a bootstrap procedure! Before the moves the > cluster had been running for several days at least. > The logs indicated the automatic token selection, and they joined the ring > under a new automatically selected token. > Presumably, this violated consistency but at the time there was no live > traffic to the cluster and we didn't confirm (put traffic on it after > repair+cleanup). > I did look a little bit at the code in light of this but didn't see anything > obvious, so I don't really know what the likely culprit is. > A potential complication was that seed nodes were moved without using the > correct procedure of de-seeding them first. This was clearly wrong, but it is > not obvious to me that it would cause other nodes to incorrectly bootstrap > since a node should *never* bootstrap more than once if the local system > tables say it's been bootstrapped. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2305) Tombstoned rows not purged from cache after gcgraceseconds
[ https://issues.apache.org/jira/browse/CASSANDRA-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018136#comment-13018136 ] Hudson commented on CASSANDRA-2305: --- Integrated in Cassandra-0.7 #429 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/429/]) Purge tombstone from row cache (0.7 version) patch by slebresne; reviewed by jbellis for CASSANDRA-2305 > Tombstoned rows not purged from cache after gcgraceseconds > -- > > Key: CASSANDRA-2305 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2305 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.7.0 >Reporter: Jeffrey Wang >Assignee: Sylvain Lebresne >Priority: Minor > Fix For: 0.7.5 > > Attachments: 0001-Compaction-test.patch, > 0002-Invalidate-row-cache-on-compaction-purge.patch, 2305_2nd_patch.patch > > Original Estimate: 2h > Time Spent: 2h > Remaining Estimate: 0h > > From email to list: > I was wondering if this is the expected behavior of deletes (0.7.0). Let's > say I have a 1-node cluster with a single CF which has gc_grace_seconds = 0. > The following sequence of operations happens (in the given order): > insert row X with timestamp T > delete row X with timestamp T+1 > force flush + compaction > insert row X with timestamp T > My understanding is that the tombstone created by the delete (and row X) will > disappear with the flush + compaction which means the last insertion should > show up. My experimentation, however, suggests otherwise (the last insertion > does not show up). > I believe I have traced this to the fact that the markedForDeleteAt field on > the ColumnFamily does not get reset after a compaction (after > gc_grace_seconds has passed); is this desirable? I think it introduces an > inconsistency in how tombstoned columns work versus tombstoned CFs. Thanks. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1791) Return name of snapshot directory after creating it and allow passing name to clearsnapshot
[ https://issues.apache.org/jira/browse/CASSANDRA-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Bailey updated CASSANDRA-1791: --- Attachment: 0001-Allow-specifying-a-name-for-a-snapshot.patch v3 attached. > Return name of snapshot directory after creating it and allow passing name to > clearsnapshot > --- > > Key: CASSANDRA-1791 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1791 > Project: Cassandra > Issue Type: New Feature > Components: Core > Environment: Debian Squeeze >Reporter: paul cannon >Assignee: Nick Bailey >Priority: Minor > Fix For: 0.8 > > Attachments: 0001-Allow-specifying-a-name-for-a-snapshot.patch, > 1791-v2.txt, 1791.txt > > Original Estimate: 4h > Remaining Estimate: 4h > > When making a snapshot, the new directory is created with a timestamp and, > optionally, a user-supplied tag. For the sake of automated snapshot-creating > tools, it would be helpful to know unequivocally what the new snapshot > directory was named (otherwise, the tool must search for a directory similar > what it expects the name to be, which could be both error-prone and maybe > susceptible to attack). > Recommend making takeSnapshot and takeAllSnapshot return a string, which is > the base component of the new snapshot's directory name. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2252) off-heap memtables
[ https://issues.apache.org/jira/browse/CASSANDRA-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018160#comment-13018160 ] Stu Hood commented on CASSANDRA-2252: - This will need a rebase post-2006. In particular, we'll need to include the bytes allocated by the allocator in the live size, and make sure the shared buffers/slabs are excluded by JAMM. > off-heap memtables > -- > > Key: CASSANDRA-2252 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2252 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis > Fix For: 0.8 > > Attachments: 0001-add-MemtableAllocator.txt, > 0002-add-off-heap-MemtableAllocator-support.txt, 2252-alternate-v2.tgz, > merged-2252.tgz > > Original Estimate: 0.4h > Remaining Estimate: 0.4h > > The memtable design practically actively fights Java's GC design. Todd > Lipcon gave a good explanation over on HBASE-3455. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1902) Migrate cached pages during compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018167#comment-13018167 ] Pavel Yaskevich commented on CASSANDRA-1902: I have been thinking about change in SSTableScanner to use SSTableReader.dfile instead of BRAF which can potentially give us the following benefits: a). don't need to skip a cache because we will be operating on memory mappings b). better performance (not copying data between kernel and user buffers as effect gained from using memory mapped segments, avoiding time operating on the kernel mode (+ time for switching context and read-ahead pressure) which BRAF involves) c). less impact on the live-reads d). less garbage will be created e). less file descriptors opened What do you think? (of course, this should be done in the separate task) > Migrate cached pages during compaction > --- > > Key: CASSANDRA-1902 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1902 > Project: Cassandra > Issue Type: Improvement > Components: Core >Affects Versions: 0.7.1 >Reporter: T Jake Luciani >Assignee: Pavel Yaskevich > Fix For: 0.7.5, 0.8 > > Attachments: > 0001-CASSANDRA-1902-cache-migration-impl-with-config-option.txt, > 1902-BufferedSegmentedFile-logandsleep.txt, 1902-formatted.txt, > 1902-per-column-migration-rebase2.txt, 1902-per-column-migration.txt, > CASSANDRA-1902-v3.patch, CASSANDRA-1902-v4.patch, CASSANDRA-1902-v5.patch, > CASSANDRA-1902-v6.patch, CASSANDRA-1902-v7.patch, CASSANDRA-1902-v8.patch, > CASSANDRA-1902-v9-trunk-with-jmx.patch, CASSANDRA-1902-v9-trunk.patch, > CASSANDRA-1902-v9.patch > > Original Estimate: 32h > Time Spent: 56h > Remaining Estimate: 0h > > Post CASSANDRA-1470 there is an opportunity to migrate cached pages from a > pre-compacted CF during the compaction process. This is now important since > CASSANDRA-1470 caches effectively nothing. > For example an active CF being compacted hurts reads since nothing is cached > in the new SSTable. > The purpose of this ticket then is to make sure SOME data is cached from > active CFs. This can be done my monitoring which Old SSTables are in the page > cache and caching active rows in the New SStable. > A simpler yet similar approach is described here: > http://insights.oetiker.ch/linux/fadvise/ -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1090905 [3/3] - in /cassandra/trunk: ./ conf/ drivers/py/cql/cassandra/ interface/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/avro/ src/java/org/apache/cassandra/cli/ src/
Modified: cassandra/trunk/test/unit/org/apache/cassandra/db/DefsTest.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/test/unit/org/apache/cassandra/db/DefsTest.java?rev=1090905&r1=1090904&r2=1090905&view=diff
==
--- cassandra/trunk/test/unit/org/apache/cassandra/db/DefsTest.java (original)
+++ cassandra/trunk/test/unit/org/apache/cassandra/db/DefsTest.java Sun Apr 10
22:59:57 2011
@@ -400,7 +400,7 @@ public class DefsTest extends CleanupHel
DecoratedKey dk = Util.dk("key0");
CFMetaData newCf = addTestCF("NewKeyspace1", "AddedStandard1", "A new
cf for a new ks");
-KSMetaData newKs = new KSMetaData(newCf.ksName, SimpleStrategy.class,
null, 5, newCf);
+KSMetaData newKs = new KSMetaData(newCf.ksName, SimpleStrategy.class,
KSMetaData.optsWithRF(5), newCf);
new AddKeyspace(newKs).apply();
@@ -579,7 +579,7 @@ public class DefsTest extends CleanupHel
{
assert DatabaseDescriptor.getTableDefinition("EmptyKeyspace") == null;
-KSMetaData newKs = new KSMetaData("EmptyKeyspace",
SimpleStrategy.class, null, 5);
+KSMetaData newKs = new KSMetaData("EmptyKeyspace",
SimpleStrategy.class, KSMetaData.optsWithRF(5));
new AddKeyspace(newKs).apply();
assert DatabaseDescriptor.getTableDefinition("EmptyKeyspace") != null;
@@ -615,7 +615,7 @@ public class DefsTest extends CleanupHel
{
// create a keyspace to serve as existing.
CFMetaData cf = addTestCF("UpdatedKeyspace", "AddedStandard1", "A new
cf for a new ks");
-KSMetaData oldKs = new KSMetaData(cf.ksName, SimpleStrategy.class,
null, 5, cf);
+KSMetaData oldKs = new KSMetaData(cf.ksName, SimpleStrategy.class,
KSMetaData.optsWithRF(5), cf);
new AddKeyspace(oldKs).apply();
@@ -624,7 +624,7 @@ public class DefsTest extends CleanupHel
// anything with cf defs should fail.
CFMetaData cf2 = addTestCF(cf.ksName, "AddedStandard2", "A new cf for
a new ks");
-KSMetaData newBadKs = new KSMetaData(cf.ksName, SimpleStrategy.class,
null, 4, cf2);
+KSMetaData newBadKs = new KSMetaData(cf.ksName, SimpleStrategy.class,
KSMetaData.optsWithRF(4), cf2);
try
{
new UpdateKeyspace(newBadKs).apply();
@@ -636,7 +636,7 @@ public class DefsTest extends CleanupHel
}
// names should match.
-KSMetaData newBadKs2 = new KSMetaData(cf.ksName + "trash",
SimpleStrategy.class, null, 4);
+KSMetaData newBadKs2 = new KSMetaData(cf.ksName + "trash",
SimpleStrategy.class, KSMetaData.optsWithRF(4));
try
{
new UpdateKeyspace(newBadKs2).apply();
@@ -647,12 +647,10 @@ public class DefsTest extends CleanupHel
// expected.
}
-KSMetaData newKs = new KSMetaData(cf.ksName,
OldNetworkTopologyStrategy.class, null, 1);
+KSMetaData newKs = new KSMetaData(cf.ksName,
OldNetworkTopologyStrategy.class, KSMetaData.optsWithRF(1));
new UpdateKeyspace(newKs).apply();
KSMetaData newFetchedKs = DatabaseDescriptor.getKSMetaData(newKs.name);
-assert newFetchedKs.replicationFactor == newKs.replicationFactor;
-assert newFetchedKs.replicationFactor != oldKs.replicationFactor;
assert newFetchedKs.strategyClass.equals(newKs.strategyClass);
assert !newFetchedKs.strategyClass.equals(oldKs.strategyClass);
}
@@ -662,7 +660,7 @@ public class DefsTest extends CleanupHel
{
// create a keyspace with a cf to update.
CFMetaData cf = addTestCF("UpdatedCfKs", "Standard1added", "A new cf
that will be updated");
-KSMetaData ksm = new KSMetaData(cf.ksName, SimpleStrategy.class, null,
1, cf);
+KSMetaData ksm = new KSMetaData(cf.ksName, SimpleStrategy.class,
KSMetaData.optsWithRF(1), cf);
new AddKeyspace(ksm).apply();
assert DatabaseDescriptor.getTableDefinition(cf.ksName) != null;
Modified:
cassandra/trunk/test/unit/org/apache/cassandra/locator/OldNetworkTopologyStrategyTest.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/test/unit/org/apache/cassandra/locator/OldNetworkTopologyStrategyTest.java?rev=1090905&r1=1090904&r2=1090905&view=diff
==
---
cassandra/trunk/test/unit/org/apache/cassandra/locator/OldNetworkTopologyStrategyTest.java
(original)
+++
cassandra/trunk/test/unit/org/apache/cassandra/locator/OldNetworkTopologyStrategyTest.java
Sun Apr 10 22:59:57 2011
@@ -32,6 +32,7 @@ import org.junit.Test;
import static org.junit.Assert.assertEquals;
import org.apache.cassandra.SchemaLoader;
+import org.apache.cassandra.config.KSMetaData;
import org.apache.cassandra.dht.BigIntegerToken;
import org.apache.cassandra.dht.Token;
@@ -61,7 +62,7 @@ public class O
[jira] [Commented] (CASSANDRA-1263) Push replication factor down to the replication strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-1263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018172#comment-13018172 ] Hudson commented on CASSANDRA-1263: --- Integrated in Cassandra #846 (See [https://hudson.apache.org/hudson/job/Cassandra/846/]) push replication_factor into strategy_options patch by jhermes and jbellis; reviewed by jhanna for CASSANDRA-1263 > Push replication factor down to the replication strategy > > > Key: CASSANDRA-1263 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1263 > Project: Cassandra > Issue Type: Task > Components: Core >Reporter: Jeremy Hanna >Assignee: Jon Hermes >Priority: Minor > Fix For: 0.8 > > Attachments: 1263-2.txt, 1263-3.txt, 1263-4.txt, 1263-5.txt, > 1263-incomplete.txt, 1263.txt > > Original Estimate: 8h > Remaining Estimate: 8h > > Currently the replication factor is in the keyspace metadata. As we've added > the datacenter shard strategy, the replication factor becomes more computed > by the replication strategy. It seems reasonable to therefore push the > replication factor for the keyspace down to the replication strategy so that > it can be handled in one place. > This adds on the work being done in CASSANDRA-1066 since that ticket will > make the replication strategy a member variable of keyspace metadata instead > of just a quasi singleton giving the replication strategy state for each > keyspace. That makes it able to have the replication factor. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2305) Tombstoned rows not purged from cache after gcgraceseconds
[ https://issues.apache.org/jira/browse/CASSANDRA-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018171#comment-13018171 ] Hudson commented on CASSANDRA-2305: --- Integrated in Cassandra #846 (See [https://hudson.apache.org/hudson/job/Cassandra/846/]) > Tombstoned rows not purged from cache after gcgraceseconds > -- > > Key: CASSANDRA-2305 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2305 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.7.0 >Reporter: Jeffrey Wang >Assignee: Sylvain Lebresne >Priority: Minor > Fix For: 0.7.5 > > Attachments: 0001-Compaction-test.patch, > 0002-Invalidate-row-cache-on-compaction-purge.patch, 2305_2nd_patch.patch > > Original Estimate: 2h > Time Spent: 2h > Remaining Estimate: 0h > > From email to list: > I was wondering if this is the expected behavior of deletes (0.7.0). Let's > say I have a 1-node cluster with a single CF which has gc_grace_seconds = 0. > The following sequence of operations happens (in the given order): > insert row X with timestamp T > delete row X with timestamp T+1 > force flush + compaction > insert row X with timestamp T > My understanding is that the tombstone created by the delete (and row X) will > disappear with the flush + compaction which means the last insertion should > show up. My experimentation, however, suggests otherwise (the last insertion > does not show up). > I believe I have traced this to the fact that the markedForDeleteAt field on > the ColumnFamily does not get reset after a compaction (after > gc_grace_seconds has passed); is this desirable? I think it introduces an > inconsistency in how tombstoned columns work versus tombstoned CFs. Thanks. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1090924 [2/2] - in /cassandra/trunk/drivers/txpy/txcql/cassandra: Cassandra.py constants.py ttypes.py
Modified: cassandra/trunk/drivers/txpy/txcql/cassandra/ttypes.py
URL:
http://svn.apache.org/viewvc/cassandra/trunk/drivers/txpy/txcql/cassandra/ttypes.py?rev=1090924&r1=1090923&r2=1090924&view=diff
==
--- cassandra/trunk/drivers/txpy/txcql/cassandra/ttypes.py (original)
+++ cassandra/trunk/drivers/txpy/txcql/cassandra/ttypes.py Mon Apr 11 03:04:19
2011
@@ -353,6 +353,165 @@ class SuperColumn:
def __ne__(self, other):
return not (self == other)
+class CounterColumn:
+ """
+ Attributes:
+ - name
+ - value
+ """
+
+ thrift_spec = (
+None, # 0
+(1, TType.STRING, 'name', None, None, ), # 1
+(2, TType.I64, 'value', None, None, ), # 2
+ )
+
+ def __init__(self, name=None, value=None,):
+self.name = name
+self.value = value
+
+ def read(self, iprot):
+if iprot.__class__ == TBinaryProtocol.TBinaryProtocolAccelerated and
isinstance(iprot.trans, TTransport.CReadableTransport) and self.thrift_spec is
not None and fastbinary is not None:
+ fastbinary.decode_binary(self, iprot.trans, (self.__class__,
self.thrift_spec))
+ return
+iprot.readStructBegin()
+while True:
+ (fname, ftype, fid) = iprot.readFieldBegin()
+ if ftype == TType.STOP:
+break
+ if fid == 1:
+if ftype == TType.STRING:
+ self.name = iprot.readString();
+else:
+ iprot.skip(ftype)
+ elif fid == 2:
+if ftype == TType.I64:
+ self.value = iprot.readI64();
+else:
+ iprot.skip(ftype)
+ else:
+iprot.skip(ftype)
+ iprot.readFieldEnd()
+iprot.readStructEnd()
+
+ def write(self, oprot):
+if oprot.__class__ == TBinaryProtocol.TBinaryProtocolAccelerated and
self.thrift_spec is not None and fastbinary is not None:
+ oprot.trans.write(fastbinary.encode_binary(self, (self.__class__,
self.thrift_spec)))
+ return
+oprot.writeStructBegin('CounterColumn')
+if self.name != None:
+ oprot.writeFieldBegin('name', TType.STRING, 1)
+ oprot.writeString(self.name)
+ oprot.writeFieldEnd()
+if self.value != None:
+ oprot.writeFieldBegin('value', TType.I64, 2)
+ oprot.writeI64(self.value)
+ oprot.writeFieldEnd()
+oprot.writeFieldStop()
+oprot.writeStructEnd()
+def validate(self):
+ if self.name is None:
+raise TProtocol.TProtocolException(message='Required field name is
unset!')
+ if self.value is None:
+raise TProtocol.TProtocolException(message='Required field value is
unset!')
+ return
+
+
+ def __repr__(self):
+L = ['%s=%r' % (key, value)
+ for key, value in self.__dict__.iteritems()]
+return '%s(%s)' % (self.__class__.__name__, ', '.join(L))
+
+ def __eq__(self, other):
+return isinstance(other, self.__class__) and self.__dict__ ==
other.__dict__
+
+ def __ne__(self, other):
+return not (self == other)
+
+class CounterSuperColumn:
+ """
+ Attributes:
+ - name
+ - columns
+ """
+
+ thrift_spec = (
+None, # 0
+(1, TType.STRING, 'name', None, None, ), # 1
+(2, TType.LIST, 'columns', (TType.STRUCT,(CounterColumn,
CounterColumn.thrift_spec)), None, ), # 2
+ )
+
+ def __init__(self, name=None, columns=None,):
+self.name = name
+self.columns = columns
+
+ def read(self, iprot):
+if iprot.__class__ == TBinaryProtocol.TBinaryProtocolAccelerated and
isinstance(iprot.trans, TTransport.CReadableTransport) and self.thrift_spec is
not None and fastbinary is not None:
+ fastbinary.decode_binary(self, iprot.trans, (self.__class__,
self.thrift_spec))
+ return
+iprot.readStructBegin()
+while True:
+ (fname, ftype, fid) = iprot.readFieldBegin()
+ if ftype == TType.STOP:
+break
+ if fid == 1:
+if ftype == TType.STRING:
+ self.name = iprot.readString();
+else:
+ iprot.skip(ftype)
+ elif fid == 2:
+if ftype == TType.LIST:
+ self.columns = []
+ (_etype10, _size7) = iprot.readListBegin()
+ for _i11 in xrange(_size7):
+_elem12 = CounterColumn()
+_elem12.read(iprot)
+self.columns.append(_elem12)
+ iprot.readListEnd()
+else:
+ iprot.skip(ftype)
+ else:
+iprot.skip(ftype)
+ iprot.readFieldEnd()
+iprot.readStructEnd()
+
+ def write(self, oprot):
+if oprot.__class__ == TBinaryProtocol.TBinaryProtocolAccelerated and
self.thrift_spec is not None and fastbinary is not None:
+ oprot.trans.write(fastbinary.encode_binary(self, (self.__class__,
self.thrift_spec)))
+ return
+oprot.writeStructBegin('CounterSuperColumn')
+if self.name != None:
+ oprot.writeFieldBegin('name', TType.STRING, 1)
+ oprot.writeString(self.name)
+ oprot.writeFieldEnd()
+if self.columns != None:
+ oprot.writeFieldBegin('columns', TType.LIST, 2)
+ oprot.writeListBegin(TType.
svn commit: r1090925 - in /cassandra/trunk: ./ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/service/ src/java/org/apache/cassandra/tools/ test/unit/org/apache/cassandra/service/
Author: jbellis
Date: Mon Apr 11 03:08:29 2011
New Revision: 1090925
URL: http://svn.apache.org/viewvc?rev=1090925&view=rev
Log:
give snapshots the same name on each node
patch by Nick Bailey; reviewed by jbellis for CASSANDRA-1791
Modified:
cassandra/trunk/CHANGES.txt
cassandra/trunk/src/java/org/apache/cassandra/db/Table.java
cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
cassandra/trunk/src/java/org/apache/cassandra/service/StorageServiceMBean.java
cassandra/trunk/src/java/org/apache/cassandra/tools/ClusterCmd.java
cassandra/trunk/src/java/org/apache/cassandra/tools/NodeCmd.java
cassandra/trunk/src/java/org/apache/cassandra/tools/NodeProbe.java
cassandra/trunk/test/unit/org/apache/cassandra/service/StorageServiceServerTest.java
Modified: cassandra/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1090925&r1=1090924&r2=1090925&view=diff
==
--- cassandra/trunk/CHANGES.txt (original)
+++ cassandra/trunk/CHANGES.txt Mon Apr 11 03:08:29 2011
@@ -18,6 +18,7 @@
* repair works on a token range instead of full ring (CASSANDRA-2324)
* purge tombstones from row cache (CASSANDRA-2305)
* push replication_factor into strategy_options (CASSANDRA-1263)
+ * give snapshots the same name on each node (CASSANDRA-1791)
0.7.5
Modified: cassandra/trunk/src/java/org/apache/cassandra/db/Table.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/Table.java?rev=1090925&r1=1090924&r2=1090925&view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/db/Table.java (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/db/Table.java Mon Apr 11
03:08:29 2011
@@ -190,10 +190,8 @@ public class Table
* @param clientSuppliedName the tag associated with the name of the
snapshot. This
* value can be null.
*/
-public void snapshot(String clientSuppliedName)
+public void snapshot(String snapshotName)
{
-String snapshotName = getTimestampedSnapshotName(clientSuppliedName);
-
for (ColumnFamilyStore cfStore : columnFamilyStores.values())
{
cfStore.snapshot(snapshotName);
@@ -214,15 +212,36 @@ public class Table
return snapshotName;
}
+/**
+ * Clear snapshots for this table. If no tag is given we will clear all
+ * snapshots
+ *
+ * @param snapshotName the user supplied snapshot name
+ * @return true if the snapshot exists
+ */
+public boolean snapshotExists(String snapshotName)
+{
+for (String dataDirPath : DatabaseDescriptor.getAllDataFileLocations())
+{
+String snapshotPath = dataDirPath + File.separator + name +
File.separator + SNAPSHOT_SUBDIR_NAME + File.separator + snapshotName;
+File snapshot = new File(snapshotPath);
+if (snapshot.exists())
+{
+return true;
+}
+}
+return false;
+}
/**
* Clear all the snapshots for a given table.
*/
-public void clearSnapshot() throws IOException
+public void clearSnapshot(String tag) throws IOException
{
for (String dataDirPath : DatabaseDescriptor.getAllDataFileLocations())
{
-String snapshotPath = dataDirPath + File.separator + name +
File.separator + SNAPSHOT_SUBDIR_NAME;
+// If tag is empty we will delete the entire snapshot directory
+String snapshotPath = dataDirPath + File.separator + name +
File.separator + SNAPSHOT_SUBDIR_NAME + File.separator + tag;
File snapshotDir = new File(snapshotPath);
if (snapshotDir.exists())
{
Modified:
cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java?rev=1090925&r1=1090924&r2=1090925&view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
Mon Apr 11 03:08:29 2011
@@ -1348,15 +1348,37 @@ public class StorageService implements I
}
/**
- * Takes the snapshot for a given table.
+ * Takes the snapshot for the given tables. A snapshot name must be
specified.
*
- * @param tableName the name of the table.
- * @param tag the tag given to the snapshot (null is permissible)
+ * @param tag the tag given to the snapshot; may not be null or empty
+ * @param tableNames the name of the tables to snapshot; empty means "all."
*/
-public void takeSnapshot(String tableName, String tag) throws IOException
+publi
svn commit: r1090926 - in /cassandra/branches/cassandra-0.7: CHANGES.txt src/java/org/apache/cassandra/service/StorageService.java
Author: jbellis
Date: Mon Apr 11 03:10:12 2011
New Revision: 1090926
URL: http://svn.apache.org/viewvc?rev=1090926&view=rev
Log:
re-set bootstrapped flag after move finishes
patch by jbellis; reviewed by Peter Schuller and Nick Bailey for CASSANDRA-2435
Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageService.java
Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1090926&r1=1090925&r2=1090926&view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Mon Apr 11 03:10:12 2011
@@ -26,6 +26,7 @@
* halve default memtable thresholds (CASSANDRA-2413)
* convert mmap assertion to if/throw so scrub can catch it (CASSANDRA-2417)
* Try harder to close files after compaction (CASSANDRA-2431)
+ * re-set bootstrapped flag after move finishes (CASSANDRA-2435)
0.7.4
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageService.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageService.java?rev=1090926&r1=1090925&r2=1090926&view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageService.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageService.java
Mon Apr 11 03:10:12 2011
@@ -203,6 +203,7 @@ public class StorageService implements I
public void finishBootstrapping()
{
isBootstrapMode = false;
+SystemTable.setBootstrapped(true);
setToken(getLocalToken());
logger_.info("Bootstrap/move completed! Now serving reads.");
}
@@ -477,10 +478,9 @@ public class StorageService implements I
{
logger_.info("Using saved token " + token);
}
-}
-
-SystemTable.setBootstrapped(true); // first startup is only chance to
bootstrap
-setToken(token);
+SystemTable.setBootstrapped(true); // first startup is only chance
to bootstrap
+setToken(token);
+}
assert tokenMetadata_.sortedTokens().size() > 0;
}
[jira] [Commented] (CASSANDRA-2441) Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10
[ https://issues.apache.org/jira/browse/CASSANDRA-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018183#comment-13018183 ] Jonathan Ellis commented on CASSANDRA-2441: --- So, definitely an OpenJDK-only bug. I'll see if I can give it a no-java-agent mode so we can limp along without it for OpenJDK. > Cassandra crashes with segmentation fault on Debian 5.0 and Ubuntu 10.10 > > > Key: CASSANDRA-2441 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2441 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.8 > Environment: Both servers have identical hardware configuration: > Quad-Core AMD Opteron(tm) Processor 2374 HE, 4 GB RAM (rackspace servers) > Java version "1.6.0_20" > OpenJDK Runtime Environment (IcedTea6 1.9.7) (6b20-1.9.7-0ubuntu1) > OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode) >Reporter: Pavel Yaskevich >Assignee: Jonathan Ellis >Priority: Critical > > Last working commit is c8d1984bf17cab58f40069e522d074c7b0077bc1 (merge from > 0.7), branch: trunk. > What I did is cloned git://git.apache.org/cassandra.git and did git reset > each commit with `ant clean && ant && ./bin/cassandra -f` until I got > cassandra started -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1090927 - in /cassandra/trunk: src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java test/system/test_cql.py
Author: jbellis
Date: Mon Apr 11 03:27:04 2011
New Revision: 1090927
URL: http://svn.apache.org/viewvc?rev=1090927&view=rev
Log:
make text (utf8) the default comparator again (regression from type renames)
patch by jbellis
Modified:
cassandra/trunk/src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java
cassandra/trunk/test/system/test_cql.py
Modified:
cassandra/trunk/src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java?rev=1090927&r1=1090926&r2=1090927&view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java
Mon Apr 11 03:27:04 2011
@@ -233,7 +233,7 @@ public class CreateColumnFamilyStatement
try
{
// RPC uses BytesType as the default validator/comparator but
BytesType expects hex for string terms, (not convenient).
-AbstractType comparator =
DatabaseDescriptor.getComparator(comparators.get(getPropertyString(KW_COMPARATOR,
"utf8")));
+AbstractType comparator =
DatabaseDescriptor.getComparator(comparators.get(getPropertyString(KW_COMPARATOR,
"text")));
String validator = getPropertyString(KW_DEFAULTVALIDATION, "utf8");
newCFMD = new CFMetaData(keyspace,
Modified: cassandra/trunk/test/system/test_cql.py
URL:
http://svn.apache.org/viewvc/cassandra/trunk/test/system/test_cql.py?rev=1090927&r1=1090926&r2=1090927&view=diff
==
--- cassandra/trunk/test/system/test_cql.py (original)
+++ cassandra/trunk/test/system/test_cql.py Mon Apr 11 03:27:04 2011
@@ -369,8 +369,7 @@ class TestCql(ThriftTester):
'age' varint,
'birthdate' bigint,
'id' uuid
-) WITH comparator = text AND comment = 'shiny, new, cf' AND
-default_validation = ascii;
+) WITH comment = 'shiny, new, cf' AND default_validation = ascii;
""")
# TODO: temporary (until this can be done with CQL).
[jira] [Commented] (CASSANDRA-2435) auto bootstrap happened on already bootstrapped nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018184#comment-13018184 ] Hudson commented on CASSANDRA-2435: --- Integrated in Cassandra-0.7 #430 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/430/]) re-set bootstrapped flag after move finishes patch by jbellis; reviewed by Peter Schuller and Nick Bailey for CASSANDRA-2435 > auto bootstrap happened on already bootstrapped nodes > - > > Key: CASSANDRA-2435 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2435 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.7.0 >Reporter: Peter Schuller >Assignee: Jonathan Ellis >Priority: Critical > Fix For: 0.7.5 > > Attachments: 2435.txt > > > I believe the following was observed on 0.7.2. I meant to dig deeper, but > never had the time, and now I want to at least file this even if I don't have > extremely helpful information. > A piece of background is that we consciously made the decision to have the > default configuration on nodes have auto_bootstrap set to true. The logic was > that if one accidentally were to start a new node, we'd rather have it join > with data than join *without* data and cause bogus read results in the > cluster. > We executed this policy (by way of having the puppet managed config have > auto_bootstrap set to true). > On one of our clusters with 5 nodes, we did some moves. All looked well; the > moves completed. For unrelated reasons, we wanted to restart nodes after they > had been moved. When we did, three of the 5, specifically those 3 that were > *NOT* seed nodes, initiated a bootstrap procedure! Before the moves the > cluster had been running for several days at least. > The logs indicated the automatic token selection, and they joined the ring > under a new automatically selected token. > Presumably, this violated consistency but at the time there was no live > traffic to the cluster and we didn't confirm (put traffic on it after > repair+cleanup). > I did look a little bit at the code in light of this but didn't see anything > obvious, so I don't really know what the likely culprit is. > A potential complication was that seed nodes were moved without using the > correct procedure of de-seeding them first. This was clearly wrong, but it is > not obvious to me that it would cause other nodes to incorrectly bootstrap > since a node should *never* bootstrap more than once if the local system > tables say it's been bootstrapped. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1090931 - /cassandra/trunk/src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java
Author: eevans
Date: Mon Apr 11 04:17:14 2011
New Revision: 1090931
URL: http://svn.apache.org/viewvc?rev=1090931&view=rev
Log:
do not default to BytesType when comparator is unknown
Patch by eevans
Modified:
cassandra/trunk/src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java
Modified:
cassandra/trunk/src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java?rev=1090931&r1=1090930&r2=1090931&view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java
Mon Apr 11 04:17:14 2011
@@ -232,9 +232,17 @@ public class CreateColumnFamilyStatement
CFMetaData newCFMD;
try
{
-// RPC uses BytesType as the default validator/comparator but
BytesType expects hex for string terms, (not convenient).
-AbstractType comparator =
DatabaseDescriptor.getComparator(comparators.get(getPropertyString(KW_COMPARATOR,
"text")));
-String validator = getPropertyString(KW_DEFAULTVALIDATION, "utf8");
+/* If not comparator/validator is not specified, default to text
(BytesType is the wrong default for CQL
+ * since it uses hex terms). If the value specified is not found
in the comparators map, assume the user
+ * knows what they are doing (a custom comparator/validator for
example), and pass it on as-is.
+ */
+String comparatorString =
(comparators.get(getPropertyString(KW_COMPARATOR, "text")) != null)
+ ?
comparators.get(getPropertyString(KW_COMPARATOR, "text"))
+ : getPropertyString(KW_COMPARATOR,
"text");
+String validatorString =
(comparators.get(getPropertyString(KW_DEFAULTVALIDATION, "text")) != null)
+ ?
comparators.get(getPropertyString(KW_DEFAULTVALIDATION, "text"))
+ : getPropertyString(KW_DEFAULTVALIDATION,
"text");
+AbstractType comparator =
DatabaseDescriptor.getComparator(comparatorString);
newCFMD = new CFMetaData(keyspace,
name,
@@ -248,7 +256,7 @@ public class CreateColumnFamilyStatement
.readRepairChance(getPropertyDouble(KW_READREPAIRCHANCE,
CFMetaData.DEFAULT_READ_REPAIR_CHANCE))
.replicateOnWrite(getPropertyBoolean(KW_REPLICATEONWRITE,
CFMetaData.DEFAULT_REPLICATE_ON_WRITE))
.gcGraceSeconds(getPropertyInt(KW_GCGRACESECONDS,
CFMetaData.DEFAULT_GC_GRACE_SECONDS))
-
.defaultValidator(DatabaseDescriptor.getComparator(comparators.get(validator)))
+
.defaultValidator(DatabaseDescriptor.getComparator(validatorString))
.minCompactionThreshold(getPropertyInt(KW_MINCOMPACTIONTHRESHOLD,
CFMetaData.DEFAULT_MIN_COMPACTION_THRESHOLD))
.maxCompactionThreshold(getPropertyInt(KW_MAXCOMPACTIONTHRESHOLD,
CFMetaData.DEFAULT_MAX_COMPACTION_THRESHOLD))
.rowCacheSavePeriod(getPropertyInt(KW_ROWCACHESAVEPERIODSECS,
CFMetaData.DEFAULT_ROW_CACHE_SAVE_PERIOD_IN_SECONDS))
[jira] [Commented] (CASSANDRA-1791) Return name of snapshot directory after creating it and allow passing name to clearsnapshot
[ https://issues.apache.org/jira/browse/CASSANDRA-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018193#comment-13018193 ] Hudson commented on CASSANDRA-1791: --- Integrated in Cassandra #847 (See [https://hudson.apache.org/hudson/job/Cassandra/847/]) give snapshots the same name on each node patch by Nick Bailey; reviewed by jbellis for CASSANDRA-1791 > Return name of snapshot directory after creating it and allow passing name to > clearsnapshot > --- > > Key: CASSANDRA-1791 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1791 > Project: Cassandra > Issue Type: New Feature > Components: Core > Environment: Debian Squeeze >Reporter: paul cannon >Assignee: Nick Bailey >Priority: Minor > Fix For: 0.8 > > Attachments: 0001-Allow-specifying-a-name-for-a-snapshot.patch, > 1791-v2.txt, 1791.txt > > Original Estimate: 4h > Remaining Estimate: 4h > > When making a snapshot, the new directory is created with a timestamp and, > optionally, a user-supplied tag. For the sake of automated snapshot-creating > tools, it would be helpful to know unequivocally what the new snapshot > directory was named (otherwise, the tool must search for a directory similar > what it expects the name to be, which could be both error-prone and maybe > susceptible to attack). > Recommend making takeSnapshot and takeAllSnapshot return a string, which is > the base component of the new snapshot's directory name. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1090935 - /cassandra/branches/cassandra-0.8/
Author: eevans Date: Mon Apr 11 05:01:08 2011 New Revision: 1090935 URL: http://svn.apache.org/viewvc?rev=1090935&view=rev Log: branching for 0.8 Added: cassandra/branches/cassandra-0.8/ - copied from r1090933, cassandra/trunk/
svn commit: r1090938 - /cassandra/branches/cassandra-0.8/CHANGES.txt
Author: jbellis Date: Mon Apr 11 05:33:45 2011 New Revision: 1090938 URL: http://svn.apache.org/viewvc?rev=1090938&view=rev Log: update CHANGES Modified: cassandra/branches/cassandra-0.8/CHANGES.txt Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1090938&r1=1090937&r2=1090938&view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Mon Apr 11 05:33:45 2011 @@ -1,16 +1,15 @@ 0.8-dev * remove Avro RPC support (CASSANDRA-926) - * avoid double RowMutation serialization on write path (CASSANDRA-1800) * adds support for columns that act as incr/decr counters - (CASSANDRA-1072, 1937, 1944, 1936, 2101, 2093, 2288, 2105, 2384) + (CASSANDRA-1072, 1937, 1944, 1936, 2101, 2093, 2288, 2105, 2384, 2236) + * CQL (CASSANDRA-1703, 1704, 1705, 1706, 1707, 1708, 1710, 1711, 1940, + 2124, 2302, 2277) + * avoid double RowMutation serialization on write path (CASSANDRA-1800) * make NetworkTopologyStrategy the default (CASSANDRA-1960) - * configurable internode encryption (CASSANDRA-1567) + * configurable internode encryption (CASSANDRA-1567, 2152) * human readable column names in sstable2json output (CASSANDRA-1933) * change default JMX port to 7199 (CASSANDRA-2027) * backwards compatible internal messaging (CASSANDRA-1015) - * check for null encryption in MessagingService (CASSANDRA-2152) - * Fix for Cli to support updating replicate_on_write (CASSANDRA-2236) - * JDBC driver for CQL (CASSANDRA-2124, 2302, 2277) * atomic switch of memtables and sstables (CASSANDRA-2284) * add pluggable SeedProvider (CASSANDRA-1669) * Fix clustertool to not throw exception when calling get_endpoints (CASSANDRA-2437)
[jira] [Issue Comment Edited] (CASSANDRA-2199) system_add_keyspace allows replication_factor of 0
[ https://issues.apache.org/jira/browse/CASSANDRA-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018202#comment-13018202 ] Jonathan Ellis edited comment on CASSANDRA-2199 at 4/11/11 5:48 AM: I believe CASSANDRA-1263 makes strategy_options.replication_factor required (for non-NTS) so if you want 0 you have to explicitly say so. was (Author: jbellis): I believe CASSANDRA-1236 makes strategy_options.replication_factor required (for non-NTS) so if you want 0 you have to explicitly say so. > system_add_keyspace allows replication_factor of 0 > -- > > Key: CASSANDRA-2199 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2199 > Project: Cassandra > Issue Type: Bug >Affects Versions: 0.7.2 >Reporter: Edward Capriolo >Priority: Minor > Attachments: cassandra-2199-1.patch.txt > > > If you define a KsDef the replication_factor will default to 0. Cassandra > will let you create this keyspace but any write to it will fail. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2008) CLI help incorrect in places
[ https://issues.apache.org/jira/browse/CASSANDRA-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018203#comment-13018203 ] Jonathan Ellis commented on CASSANDRA-2008: --- Sorry for the delay. Would you mind rebasing? > CLI help incorrect in places > > > Key: CASSANDRA-2008 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2008 > Project: Cassandra > Issue Type: Improvement > Components: Core >Affects Versions: 0.8 >Reporter: Aaron Morton >Assignee: Aaron Morton >Priority: Trivial > Fix For: 0.8 > > Attachments: 2007.txt, 2008-2.patch, 2008-3.patch > > > Found some errors in the CLI help, such as these for create column family. > - memtable_operations: Flush memtables after this many operations > - memtable_throughput: ... or after this many bytes have been written > - memtable_flush_after: ... or after this many seconds > Should be millions of ops, MB's written and minutes not seconds. Have > confirmed thats how the values are used. Will check all the help. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1090943 - /cassandra/branches/cassandra-0.8/CHANGES.txt
Author: jbellis Date: Mon Apr 11 05:50:26 2011 New Revision: 1090943 URL: http://svn.apache.org/viewvc?rev=1090943&view=rev Log: update CHANGES Modified: cassandra/branches/cassandra-0.8/CHANGES.txt Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1090943&r1=1090942&r2=1090943&view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Mon Apr 11 05:50:26 2011 @@ -18,6 +18,7 @@ * purge tombstones from row cache (CASSANDRA-2305) * push replication_factor into strategy_options (CASSANDRA-1263) * give snapshots the same name on each node (CASSANDRA-1791) + * add key type information (CASSANDRA-2311) 0.7.5
[jira] [Updated] (CASSANDRA-2088) Temp files for failed compactions/streaming not cleaned up
[ https://issues.apache.org/jira/browse/CASSANDRA-2088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Morton updated CASSANDRA-2088: Attachment: 0002-delete-partial-sstable-if-compaction-error.patch 0001-detect-streaming-failures-and-cleanup-temp-files.patch patch 0001 tracks failures during AES streaming, files for failed Stream sessions are cleaned up and repair is allowed to continue. Failed files are logged at the StreamSession, TreeRequest, and RepairSession level. patch 0002 handle exceptions when doing a (normal) compaction and deletes the temp SSTable. The SSTableWriter components are closed before deletion so that windows will delete correctly. > Temp files for failed compactions/streaming not cleaned up > -- > > Key: CASSANDRA-2088 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2088 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Stu Hood >Assignee: Aaron Morton > Fix For: 0.8 > > Attachments: > 0001-detect-streaming-failures-and-cleanup-temp-files.patch, > 0002-delete-partial-sstable-if-compaction-error.patch > > > From separate reports, compaction and repair are currently missing > opportunities to clean up tmp files after failures. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2156) Compaction Throttling
[
https://issues.apache.org/jira/browse/CASSANDRA-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stu Hood updated CASSANDRA-2156:
Attachment: (was:
0006-Throttle-total-compaction-to-a-configurable-throughput.txt)
> Compaction Throttling
> -
>
> Key: CASSANDRA-2156
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2156
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Stu Hood
>Assignee: Stu Hood
> Fix For: 0.8
>
> Attachments:
> for-0.6-0001-Throttle-compaction-to-a-fixed-throughput.txt,
> for-0.6-0002-Make-compaction-throttling-configurable.txt
>
>
> Compaction is currently relatively bursty: we compact as fast as we can, and
> then we wait for the next compaction to be possible ("hurry up and wait").
> Instead, to properly amortize compaction, you'd like to compact exactly as
> fast as you need to to keep the sstable count under control.
> For every new level of compaction, you need to increase the rate that you
> compact at: a rule of thumb that we're testing on our clusters is to
> determine the maximum number of buckets a node can support (aka, if the 15th
> bucket holds 750 GB, we're not going to have more than 15 buckets), and then
> multiply the flush throughput by the number of buckets to get a minimum
> compaction throughput to maintain your sstable count.
> Full explanation: for a min compaction threshold of {{T}}, the bucket at
> level {{N}} can contain {{SsubN = T^N}} 'units' (unit == memtable's worth of
> data on disk). Every time a new unit is added, it has a {{1/SsubN}} chance of
> causing the bucket at level N to fill. If the bucket at level N fills, it
> causes {{SsubN}} units to be compacted. So, for each active level in your
> system you have {{SubN * 1 / SsubN}}, or {{1}} amortized unit to compact any
> time a new unit is added.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2156) Compaction Throttling
[
https://issues.apache.org/jira/browse/CASSANDRA-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stu Hood updated CASSANDRA-2156:
Attachment: 0007-Throttle-total-compaction-to-a-configurable-throughput.txt
> Compaction Throttling
> -
>
> Key: CASSANDRA-2156
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2156
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Stu Hood
>Assignee: Stu Hood
> Fix For: 0.8
>
> Attachments:
> 0007-Throttle-total-compaction-to-a-configurable-throughput.txt,
> for-0.6-0001-Throttle-compaction-to-a-fixed-throughput.txt,
> for-0.6-0002-Make-compaction-throttling-configurable.txt
>
>
> Compaction is currently relatively bursty: we compact as fast as we can, and
> then we wait for the next compaction to be possible ("hurry up and wait").
> Instead, to properly amortize compaction, you'd like to compact exactly as
> fast as you need to to keep the sstable count under control.
> For every new level of compaction, you need to increase the rate that you
> compact at: a rule of thumb that we're testing on our clusters is to
> determine the maximum number of buckets a node can support (aka, if the 15th
> bucket holds 750 GB, we're not going to have more than 15 buckets), and then
> multiply the flush throughput by the number of buckets to get a minimum
> compaction throughput to maintain your sstable count.
> Full explanation: for a min compaction threshold of {{T}}, the bucket at
> level {{N}} can contain {{SsubN = T^N}} 'units' (unit == memtable's worth of
> data on disk). Every time a new unit is added, it has a {{1/SsubN}} chance of
> causing the bucket at level N to fill. If the bucket at level N fills, it
> causes {{SsubN}} units to be compacted. So, for each active level in your
> system you have {{SubN * 1 / SsubN}}, or {{1}} amortized unit to compact any
> time a new unit is added.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2191) Multithread across compaction buckets
[ https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2191: Attachment: (was: 0005-Add-a-harness-to-allow-compaction-tasks-that-need-to-a.txt) > Multithread across compaction buckets > - > > Key: CASSANDRA-2191 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2191 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Stu Hood >Assignee: Stu Hood >Priority: Critical > Labels: compaction > Fix For: 0.8 > > > This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and > reasoning are different enough to open a separate issue. > The problem with compactions currently is that they compact the set of > sstables that existed the moment the compaction started. This means that for > longer running compactions (even when running as fast as possible on the > hardware), a very large number of new sstables might be created in the > meantime. We have observed this proliferation of sstables killing performance > during major/high-bucketed compactions. > One approach would be to pause compactions in upper buckets (containing > larger files) when compactions in lower buckets become possible. While this > would likely solve the problem with read performance, it does not actually > help us perform compaction any faster, which is a reasonable requirement for > other situations. > Instead, we need to be able to perform any compactions that are currently > required in parallel, independent of what bucket they might be in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2191) Multithread across compaction buckets
[ https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2191: Attachment: (was: 0001-Add-a-compacting-set-to-DataTracker.txt) > Multithread across compaction buckets > - > > Key: CASSANDRA-2191 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2191 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Stu Hood >Assignee: Stu Hood >Priority: Critical > Labels: compaction > Fix For: 0.8 > > > This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and > reasoning are different enough to open a separate issue. > The problem with compactions currently is that they compact the set of > sstables that existed the moment the compaction started. This means that for > longer running compactions (even when running as fast as possible on the > hardware), a very large number of new sstables might be created in the > meantime. We have observed this proliferation of sstables killing performance > during major/high-bucketed compactions. > One approach would be to pause compactions in upper buckets (containing > larger files) when compactions in lower buckets become possible. While this > would likely solve the problem with read performance, it does not actually > help us perform compaction any faster, which is a reasonable requirement for > other situations. > Instead, we need to be able to perform any compactions that are currently > required in parallel, independent of what bucket they might be in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2191) Multithread across compaction buckets
[ https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2191: Attachment: (was: 0004-Allow-multithread-compaction-to-be-disabled.txt) > Multithread across compaction buckets > - > > Key: CASSANDRA-2191 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2191 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Stu Hood >Assignee: Stu Hood >Priority: Critical > Labels: compaction > Fix For: 0.8 > > > This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and > reasoning are different enough to open a separate issue. > The problem with compactions currently is that they compact the set of > sstables that existed the moment the compaction started. This means that for > longer running compactions (even when running as fast as possible on the > hardware), a very large number of new sstables might be created in the > meantime. We have observed this proliferation of sstables killing performance > during major/high-bucketed compactions. > One approach would be to pause compactions in upper buckets (containing > larger files) when compactions in lower buckets become possible. While this > would likely solve the problem with read performance, it does not actually > help us perform compaction any faster, which is a reasonable requirement for > other situations. > Instead, we need to be able to perform any compactions that are currently > required in parallel, independent of what bucket they might be in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2191) Multithread across compaction buckets
[ https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2191: Attachment: (was: 0003-Expose-multiple-compactions-via-JMX-and-a-concrete-ser.txt) > Multithread across compaction buckets > - > > Key: CASSANDRA-2191 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2191 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Stu Hood >Assignee: Stu Hood >Priority: Critical > Labels: compaction > Fix For: 0.8 > > > This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and > reasoning are different enough to open a separate issue. > The problem with compactions currently is that they compact the set of > sstables that existed the moment the compaction started. This means that for > longer running compactions (even when running as fast as possible on the > hardware), a very large number of new sstables might be created in the > meantime. We have observed this proliferation of sstables killing performance > during major/high-bucketed compactions. > One approach would be to pause compactions in upper buckets (containing > larger files) when compactions in lower buckets become possible. While this > would likely solve the problem with read performance, it does not actually > help us perform compaction any faster, which is a reasonable requirement for > other situations. > Instead, we need to be able to perform any compactions that are currently > required in parallel, independent of what bucket they might be in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2191) Multithread across compaction buckets
[ https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2191: Attachment: (was: 0002-Use-the-compacting-set-of-sstables-to-schedule-multith.txt) > Multithread across compaction buckets > - > > Key: CASSANDRA-2191 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2191 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Stu Hood >Assignee: Stu Hood >Priority: Critical > Labels: compaction > Fix For: 0.8 > > > This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and > reasoning are different enough to open a separate issue. > The problem with compactions currently is that they compact the set of > sstables that existed the moment the compaction started. This means that for > longer running compactions (even when running as fast as possible on the > hardware), a very large number of new sstables might be created in the > meantime. We have observed this proliferation of sstables killing performance > during major/high-bucketed compactions. > One approach would be to pause compactions in upper buckets (containing > larger files) when compactions in lower buckets become possible. While this > would likely solve the problem with read performance, it does not actually > help us perform compaction any faster, which is a reasonable requirement for > other situations. > Instead, we need to be able to perform any compactions that are currently > required in parallel, independent of what bucket they might be in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2191) Multithread across compaction buckets
[ https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stu Hood updated CASSANDRA-2191: Attachment: 0006-Prevent-cache-saves-from-occuring-concurrently.txt 0005-Acquire-the-writeLock-for-major-cleanup-scrub-in-order.txt 0004-Allow-multithread-compaction-to-be-disabled.txt 0003-Expose-multiple-compactions-via-JMX-and-a-concrete-ser.txt 0002-Use-the-compacting-set-of-sstables-to-schedule-multith.txt 0001-Add-a-compacting-set-to-DataTracker.txt * Inlined stopTheWorld in 0005. Yes, I agree that the name sucked, but whether or not it is possible for a lock acquisition to fail on a server that is not already screwed, and whether an abstraction is in order here is still up for debate * Removed the 'forceMajor' parameter: will open a ticket post-commit to allow for guaranteeing that a manually triggered compaction is major * Moved ksname/cfname into getters. I didn't do this initially because the CFS is sometimes null, but I guess you'd get the NPE in either case * Added an AtomicBoolean to AutoSavingCache in 0006. I reeeally think this should go to the flush stage, since the tasks have almost identical lifetimes, and we don't really need progress for either of them * Wrapped the IdentityHashMap into an IdentityHashSet * Returned printCompactionStats to its former glory * Removed OperationType from SSTableWriter.Builder's task type Thanks! CASSANDRA-2156 has been rebased as well. > Multithread across compaction buckets > - > > Key: CASSANDRA-2191 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2191 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Stu Hood >Assignee: Stu Hood >Priority: Critical > Labels: compaction > Fix For: 0.8 > > Attachments: 0001-Add-a-compacting-set-to-DataTracker.txt, > 0002-Use-the-compacting-set-of-sstables-to-schedule-multith.txt, > 0003-Expose-multiple-compactions-via-JMX-and-a-concrete-ser.txt, > 0004-Allow-multithread-compaction-to-be-disabled.txt, > 0005-Acquire-the-writeLock-for-major-cleanup-scrub-in-order.txt, > 0006-Prevent-cache-saves-from-occuring-concurrently.txt > > > This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and > reasoning are different enough to open a separate issue. > The problem with compactions currently is that they compact the set of > sstables that existed the moment the compaction started. This means that for > longer running compactions (even when running as fast as possible on the > hardware), a very large number of new sstables might be created in the > meantime. We have observed this proliferation of sstables killing performance > during major/high-bucketed compactions. > One approach would be to pause compactions in upper buckets (containing > larger files) when compactions in lower buckets become possible. While this > would likely solve the problem with read performance, it does not actually > help us perform compaction any faster, which is a reasonable requirement for > other situations. > Instead, we need to be able to perform any compactions that are currently > required in parallel, independent of what bucket they might be in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
