[jira] [Commented] (CASSANDRA-13656) Change default start_native_transport configuration option
[ https://issues.apache.org/jira/browse/CASSANDRA-13656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091211#comment-16091211 ] Tomas Repik commented on CASSANDRA-13656: - I agree with you guys, cassandra.yaml file only to set it up. > Change default start_native_transport configuration option > -- > > Key: CASSANDRA-13656 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13656 > Project: Cassandra > Issue Type: Wish > Components: Configuration >Reporter: Tomas Repik >Assignee: Tomas Repik >Priority: Trivial > Fix For: 4.x > > Attachments: update_default_config.patch > > > When you don't specify the start_native_transport option in the > cassandra.yaml config file the default value is set to false. So far I did > not find any good reason for setting it this way so I'm proposing to set it > to true as default. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13418) Allow TWCS to ignore overlaps when dropping fully expired sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091218#comment-16091218 ] Romain GERARD commented on CASSANDRA-13418: --- Yes I am still looking forward to test the proposition of [~krummas] and to add unit tests. I am right now in vacation, so I haven't had the time to play around with the code, but it is defintly in my todo list on my return (begin of August) So I haven't forgotten the patch. > Allow TWCS to ignore overlaps when dropping fully expired sstables > -- > > Key: CASSANDRA-13418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13418 > Project: Cassandra > Issue Type: Improvement > Components: Compaction >Reporter: Corentin Chary > Labels: twcs > Attachments: twcs-cleanup.png > > > http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If > you really want read-repairs you're going to have sstables blocking the > expiration of other fully expired SSTables because they overlap. > You can set unchecked_tombstone_compaction = true or tombstone_threshold to a > very low value and that will purge the blockers of old data that should > already have expired, thus removing the overlaps and allowing the other > SSTables to expire. > The thing is that this is rather CPU intensive and not optimal. If you have > time series, you might not care if all your data doesn't exactly expire at > the right time, or if data re-appears for some time, as long as it gets > deleted as soon as it can. And in this situation I believe it would be really > beneficial to allow users to simply ignore overlapping SSTables when looking > for fully expired ones. > To the question: why would you need read-repairs ? > - Full repairs basically take longer than the TTL of the data on my dataset, > so this isn't really effective. > - Even with a 10% chances of doing a repair, we found out that this would be > enough to greatly reduce entropy of the most used data (and if you have > timeseries, you're likely to have a dashboard doing the same important > queries over and over again). > - LOCAL_QUORUM is too expensive (need >3 replicas), QUORUM is too slow. > I'll try to come up with a patch demonstrating how this would work, try it on > our system and report the effects. > cc: [~adejanovski], [~rgerard] as I know you worked on similar issues already. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12996) update slf4j dependency to 1.7.21
[ https://issues.apache.org/jira/browse/CASSANDRA-12996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091223#comment-16091223 ] Stefan Podkowinski commented on CASSANDRA-12996: Leaving the Fedora specific discussion aside, we still should update to the latest slf4j version for 4.0 as part of CASSANDRA-13361. * [branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-12996-trunk] * [testall|https://circleci.com/gh/spodkowinski/cassandra/78#tests/containers/2] * [dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/128/] [~snazy], wdyt? > update slf4j dependency to 1.7.21 > - > > Key: CASSANDRA-12996 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12996 > Project: Cassandra > Issue Type: Improvement > Components: Libraries >Reporter: Tomas Repik >Assignee: Stefan Podkowinski > Fix For: 4.0 > > Attachments: cassandra-3.11.0-slf4j.patch > > > Cassandra 3.11.0 is about to be included in Fedora. There are some tweaks to > the sources we need to do in order to successfully build it. Cassandra > depends on slf4j 1.7.7, but In Fedora we have the latest upstream version > 1.7.21 It was released some time ago on April 6 2016. I attached a patch > updating Cassandra sources to depend on the newer slf4j sources. The only > actual change is the number of parameters accepted by SubstituteLogger class. > Please consider updating. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13069) Local batchlog for MV may not be correctly written on node movements
[ https://issues.apache.org/jira/browse/CASSANDRA-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091232#comment-16091232 ] Kurt Greaves commented on CASSANDRA-13069: -- [~pauloricardomg] are you still working on this? If you want I can take over. > Local batchlog for MV may not be correctly written on node movements > > > Key: CASSANDRA-13069 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13069 > Project: Cassandra > Issue Type: Bug > Components: Materialized Views >Reporter: Sylvain Lebresne >Assignee: Paulo Motta > > Unless I'm really reading this wrong, I think the code > [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageProxy.java#L829-L843], > which comes from CASSANDRA-10674, isn't working properly. > More precisely, I believe we can have both paired and unpaired mutations, so > that both {{if}} can be taken, but if that's the case, the 2nd write to the > batchlog will basically overwrite (remove) the batchlog write of the 1st > {{if}} and I don't think that's the intention. In practice, this means > "paired" mutation won't be in the batchlog, which mean they won't be replayed > at all if they fail. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12996) update slf4j dependency to 1.7.21
[ https://issues.apache.org/jira/browse/CASSANDRA-12996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091241#comment-16091241 ] Tomas Repik commented on CASSANDRA-12996: - I don't think C* should follow any distributions, the thing I'm suggesting is updating the trunk of C* or the 4.0 branch. Following rather upstream, and updating to the latest upstream release is perfectly fine. I mean if some Linux distribution has adopted newer version of the library I guess it was for a reason. Most likely the newer version provides some important fixes and new features improving the overall quality. I don't see any point in staying with the old versions. > update slf4j dependency to 1.7.21 > - > > Key: CASSANDRA-12996 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12996 > Project: Cassandra > Issue Type: Improvement > Components: Libraries >Reporter: Tomas Repik >Assignee: Stefan Podkowinski > Fix For: 4.0 > > Attachments: cassandra-3.11.0-slf4j.patch > > > Cassandra 3.11.0 is about to be included in Fedora. There are some tweaks to > the sources we need to do in order to successfully build it. Cassandra > depends on slf4j 1.7.7, but In Fedora we have the latest upstream version > 1.7.21 It was released some time ago on April 6 2016. I attached a patch > updating Cassandra sources to depend on the newer slf4j sources. The only > actual change is the number of parameters accepted by SubstituteLogger class. > Please consider updating. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13622) Better config validation/documentation
[ https://issues.apache.org/jira/browse/CASSANDRA-13622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091251#comment-16091251 ] Kurt Greaves commented on CASSANDRA-13622: -- Not sure how keen I am on bounding commitlog_segment_size to 2GB. Obviously that's a ridiculous size for a segment, but I have had to increase it to 1.2GB in the past to allow a write through. Granted it was a side effect from MV's (I think it was streaming related), however the ability to do so helped. Obviously that's not great justification but I think it would be better to give it a much higher limit (store it as a long) and simply advise users (in docs+yaml) that they really shouldn't need to set it above the default. Same goes for max_value_size. I don't see any compelling reason we should stop people experimenting with that if they want to, as it's really just a safety net that the user configures. > Better config validation/documentation > -- > > Key: CASSANDRA-13622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13622 > Project: Cassandra > Issue Type: Bug >Reporter: Kurt Greaves >Assignee: ZhaoYang >Priority: Minor > Labels: lhf > > There are a number of properties in the yaml that are "in_mb", however > resolve to bytes when calculated in {{DatabaseDescriptor.java}}, but are > stored in int's. This means that their maximum values are 2047, as any higher > when converted to bytes overflows the int. > Where possible/reasonable we should convert these to be long's, and stored as > long's. If there is no reason for the value to ever be >2047 we should at > least document that as the max value, or better yet make it error if set > higher than that. Noting that although it's bad practice to increase a lot of > them to such high values, there may be cases where it is necessary and in > which case we should handle it appropriately rather than overflowing and > surprising the user. That is, causing it to break but not in the way the user > expected it to :) > Following are functions that currently could be at risk of the above: > {code:java|title=DatabaseDescriptor.java} > getThriftFramedTransportSize() > getMaxValueSize() > getCompactionLargePartitionWarningThreshold() > getCommitLogSegmentSize() > getNativeTransportMaxFrameSize() > # These are in KB so max value of 2096128 > getBatchSizeWarnThreshold() > getColumnIndexSize() > getColumnIndexCacheSize() > getMaxMutationSize() > {code} > Note we may not actually need to fix all of these, and there may be more. > This was just from a rough scan over the code. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13564) Mismatch Documentation on MATERIALIZE VIEW
[ https://issues.apache.org/jira/browse/CASSANDRA-13564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091255#comment-16091255 ] Kurt Greaves commented on CASSANDRA-13564: -- What is the goal? It sounds like in your view you are just making the partitioning more granular. Is that correct? > Mismatch Documentation on MATERIALIZE VIEW > -- > > Key: CASSANDRA-13564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13564 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website, Materialized Views > Environment: 3.10 >Reporter: Nick Hryhoriev > Labels: doc-impacting > > During create MATERIALIZED VIEW with out cluster key. > I've got exception "No columns are defined for Materialized View other than > primary key" > Does it expected behave? Because i can't find nothing about this in > Documentation. But this check exist in code. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12996) update slf4j dependency to 1.7.21
[ https://issues.apache.org/jira/browse/CASSANDRA-12996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091260#comment-16091260 ] Robert Stupp commented on CASSANDRA-12996: -- I don't object an update of slf4j in trunk. [~spod] Patch LGTM. In the past we already updated some libraries if necessary and if then in trunk. But not just because it's a newer version - that doesn't justify a change in a stable branch IMO. TBH, I would not just assume that a newer patch version just fixes issues - new issues can be introduced and affect systems seriously. > update slf4j dependency to 1.7.21 > - > > Key: CASSANDRA-12996 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12996 > Project: Cassandra > Issue Type: Improvement > Components: Libraries >Reporter: Tomas Repik >Assignee: Stefan Podkowinski > Fix For: 4.0 > > Attachments: cassandra-3.11.0-slf4j.patch > > > Cassandra 3.11.0 is about to be included in Fedora. There are some tweaks to > the sources we need to do in order to successfully build it. Cassandra > depends on slf4j 1.7.7, but In Fedora we have the latest upstream version > 1.7.21 It was released some time ago on April 6 2016. I attached a patch > updating Cassandra sources to depend on the newer slf4j sources. The only > actual change is the number of parameters accepted by SubstituteLogger class. > Please consider updating. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-11500) Obsolete MV entry may not be properly deleted
[ https://issues.apache.org/jira/browse/CASSANDRA-11500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091273#comment-16091273 ] Kurt Greaves commented on CASSANDRA-11500: -- Thanks [~jasonstack], I think that's a wise approach to solving the issues. I'll give it some thought over the next couple days. Worth giving some thought to [~fsander]'s solution as well, however I think I'm in favour of utilising ShadowableTombstones for the same case. Will give it some serious consideration before ruling anything out however. Also, just making a note here that we have to solve the issue raised in CASSANDRA-10965 here as well. Pretty sure you're already addressing this but just saying so it's written down somewhere. > Obsolete MV entry may not be properly deleted > - > > Key: CASSANDRA-11500 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11500 > Project: Cassandra > Issue Type: Bug > Components: Materialized Views >Reporter: Sylvain Lebresne >Assignee: ZhaoYang > > When a Materialized View uses a non-PK base table column in its PK, if an > update changes that column value, we add the new view entry and remove the > old one. When doing that removal, the current code uses the same timestamp > than for the liveness info of the new entry, which is the max timestamp for > any columns participating to the view PK. This is not correct for the > deletion as the old view entry could have other columns with higher timestamp > which won't be deleted as can easily shown by the failing of the following > test: > {noformat} > CREATE TABLE t (k int PRIMARY KEY, a int, b int); > CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS > NOT NULL PRIMARY KEY (k, a); > INSERT INTO t(k, a, b) VALUES (1, 1, 1) USING TIMESTAMP 0; > UPDATE t USING TIMESTAMP 4 SET b = 2 WHERE k = 1; > UPDATE t USING TIMESTAMP 2 SET a = 2 WHERE k = 1; > SELECT * FROM mv WHERE k = 1; // This currently return 2 entries, the old > (invalid) and the new one > {noformat} > So the correct timestamp to use for the deletion is the biggest timestamp in > the old view entry (which we know since we read the pre-existing base row), > and that is what CASSANDRA-11475 does (the test above thus doesn't fail on > that branch). > Unfortunately, even then we can still have problems if further updates > requires us to overide the old entry. Consider the following case: > {noformat} > CREATE TABLE t (k int PRIMARY KEY, a int, b int); > CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS > NOT NULL PRIMARY KEY (k, a); > INSERT INTO t(k, a, b) VALUES (1, 1, 1) USING TIMESTAMP 0; > UPDATE t USING TIMESTAMP 10 SET b = 2 WHERE k = 1; > UPDATE t USING TIMESTAMP 2 SET a = 2 WHERE k = 1; // This will delete the > entry for a=1 with timestamp 10 > UPDATE t USING TIMESTAMP 3 SET a = 1 WHERE k = 1; // This needs to re-insert > an entry for a=1 but shouldn't be deleted by the prior deletion > UPDATE t USING TIMESTAMP 4 SET a = 2 WHERE k = 1; // ... and we can play this > game more than once > UPDATE t USING TIMESTAMP 5 SET a = 1 WHERE k = 1; > ... > {noformat} > In a way, this is saying that the "shadowable" deletion mechanism is not > general enough: we need to be able to re-insert an entry when a prior one had > been deleted before, but we can't rely on timestamps being strictly bigger on > the re-insert. In that sense, this can be though as a similar problem than > CASSANDRA-10965, though the solution there of a single flag is not enough > since we can have to replace more than once. > I think the proper solution would be to ship enough information to always be > able to decide when a view deletion is shadowed. Which means that both > liveness info (for updates) and shadowable deletion would need to ship the > timestamp of any base table column that is part the view PK (so {{a}} in the > example below). It's doable (and not that hard really), but it does require > a change to the sstable and intra-node protocol, which makes this a bit > painful right now. > But I'll also note that as CASSANDRA-1096 shows, the timestamp is not even > enough since on equal timestamp the value can be the deciding factor. So in > theory we'd have to ship the value of those columns (in the case of a > deletion at least since we have it in the view PK for updates). That said, on > that last problem, my preference would be that we start prioritizing > CASSANDRA-6123 seriously so we don't have to care about conflicting timestamp > anymore, which would make this problem go away. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e
[jira] [Resolved] (CASSANDRA-10965) Shadowable tombstones can continue to shadow view results when timestamps match
[ https://issues.apache.org/jira/browse/CASSANDRA-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Greaves resolved CASSANDRA-10965. -- Resolution: Fixed Resolving as duplicate, will solve in CASSANDRA-11500 > Shadowable tombstones can continue to shadow view results when timestamps > match > --- > > Key: CASSANDRA-10965 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10965 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths, Materialized Views >Reporter: Carl Yeksigian > Fix For: 3.0.x > > Attachments: shadow-ts.cql > > > I've attached a script which reproduces the issue. The first time we insert > with {{TIMESTAMP 2}}, we are inserting a new row which has the same timestamp > as the previous shadow tombstone, and it continues to be shadowed by that > tombstone because we shadow values with the same timestamp. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12996) update slf4j dependency to 1.7.21
[ https://issues.apache.org/jira/browse/CASSANDRA-12996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091292#comment-16091292 ] Tomas Repik commented on CASSANDRA-12996: - Agree that new version may introduce new issues, therefore you have the tests, to check it out. Thanks [~spod] for the initiative, good work. > update slf4j dependency to 1.7.21 > - > > Key: CASSANDRA-12996 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12996 > Project: Cassandra > Issue Type: Improvement > Components: Libraries >Reporter: Tomas Repik >Assignee: Stefan Podkowinski > Fix For: 4.0 > > Attachments: cassandra-3.11.0-slf4j.patch > > > Cassandra 3.11.0 is about to be included in Fedora. There are some tweaks to > the sources we need to do in order to successfully build it. Cassandra > depends on slf4j 1.7.7, but In Fedora we have the latest upstream version > 1.7.21 It was released some time ago on April 6 2016. I attached a patch > updating Cassandra sources to depend on the newer slf4j sources. The only > actual change is the number of parameters accepted by SubstituteLogger class. > Please consider updating. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13127) Materialized Views: View row expires too soon
[ https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091304#comment-16091304 ] DOAN DuyHai commented on CASSANDRA-13127: - Hey guys A question about TTLed columns. I want to make sure we do cover this scenario. When an TTLed column expired in the base table, on the next compaction, this TTLed column is replaced by a tombstone. Since all this TTLed -> tombstone mechanism only occurs during compaction, do we have something in the code path that *remove* the corresponding entry in the view ? ping [~jasonstack] [~fsander] > Materialized Views: View row expires too soon > - > > Key: CASSANDRA-13127 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13127 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths, Materialized Views >Reporter: Duarte Nunes >Assignee: ZhaoYang > > Consider the following commands, ran against trunk: > {code} > echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh > echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" | > bin/cqlsh > echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT > NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh > echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh > # wait for row liveness to get closer to expiration > sleep 6; > echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh > echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh > p | c | ttl(v) > ---+---+ > 0 | 0 | 7 > (1 rows) > c | p > ---+--- > 0 | 0 > (1 rows) > # wait for row liveness to expire > sleep 4; > echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh > p | c | ttl(v) > ---+---+ > 0 | 0 | 3 > (1 rows) > c | p > ---+--- > (0 rows) > {code} > Notice how the view row is removed even though the base row is still live. I > would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry > the TTLs are compared instead of the expiration times, but I'm not sure I'm > getting that far ahead in the code when updating a column that's not in the > view. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13127) Materialized Views: View row expires too soon
[ https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091325#comment-16091325 ] ZhaoYang commented on CASSANDRA-13127: -- [~doanduyhai] yes, it's considered. There are 3 cases: TTLed base column as view's PK or view's filter condition will wipe entire view row. TTLed base column as selected in view will work as it is TTLed base column not selected in view, if there is no other live cells or primary key of view, view row is gone. > Materialized Views: View row expires too soon > - > > Key: CASSANDRA-13127 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13127 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths, Materialized Views >Reporter: Duarte Nunes >Assignee: ZhaoYang > > Consider the following commands, ran against trunk: > {code} > echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh > echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" | > bin/cqlsh > echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT > NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh > echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh > # wait for row liveness to get closer to expiration > sleep 6; > echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh > echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh > p | c | ttl(v) > ---+---+ > 0 | 0 | 7 > (1 rows) > c | p > ---+--- > 0 | 0 > (1 rows) > # wait for row liveness to expire > sleep 4; > echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh > p | c | ttl(v) > ---+---+ > 0 | 0 | 3 > (1 rows) > c | p > ---+--- > (0 rows) > {code} > Notice how the view row is removed even though the base row is still live. I > would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry > the TTLs are compared instead of the expiration times, but I'm not sure I'm > getting that far ahead in the code when updating a column that's not in the > view. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[ https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091325#comment-16091325 ] ZhaoYang edited comment on CASSANDRA-13127 at 7/18/17 9:16 AM: --- [~doanduyhai] yes, it's considered. There are 3 cases: TTLed base column as view's PK or view's filter condition will wipe entire view row. TTLed base column as selected in view will work as it is TTLed base column not selected in view, if there is no other live cells or primary key of view, view row is gone. please feel free to point out any cases, I will summarize the solution into CASSANDRA-11500 was (Author: jasonstack): [~doanduyhai] yes, it's considered. There are 3 cases: TTLed base column as view's PK or view's filter condition will wipe entire view row. TTLed base column as selected in view will work as it is TTLed base column not selected in view, if there is no other live cells or primary key of view, view row is gone. > Materialized Views: View row expires too soon > - > > Key: CASSANDRA-13127 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13127 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths, Materialized Views >Reporter: Duarte Nunes >Assignee: ZhaoYang > > Consider the following commands, ran against trunk: > {code} > echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh > echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" | > bin/cqlsh > echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT > NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh > echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh > # wait for row liveness to get closer to expiration > sleep 6; > echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh > echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh > p | c | ttl(v) > ---+---+ > 0 | 0 | 7 > (1 rows) > c | p > ---+--- > 0 | 0 > (1 rows) > # wait for row liveness to expire > sleep 4; > echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh > p | c | ttl(v) > ---+---+ > 0 | 0 | 3 > (1 rows) > c | p > ---+--- > (0 rows) > {code} > Notice how the view row is removed even though the base row is still live. I > would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry > the TTLs are compared instead of the expiration times, but I'm not sure I'm > getting that far ahead in the code when updating a column that's not in the > view. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13564) Mismatch Documentation on MATERIALIZE VIEW
[ https://issues.apache.org/jira/browse/CASSANDRA-13564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091345#comment-16091345 ] Nick Hryhoriev commented on CASSANDRA-13564: Yes, i wanna add some kind of ids, for better join policy. And i don't want to handle two table manually. Actually what i do now, I save same data in two tables, in spark. Some time i have problem with sync this table. So i hope use MV to avoid this effort. > Mismatch Documentation on MATERIALIZE VIEW > -- > > Key: CASSANDRA-13564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13564 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website, Materialized Views > Environment: 3.10 >Reporter: Nick Hryhoriev > Labels: doc-impacting > > During create MATERIALIZED VIEW with out cluster key. > I've got exception "No columns are defined for Materialized View other than > primary key" > Does it expected behave? Because i can't find nothing about this in > Documentation. But this check exist in code. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13564) Mismatch Documentation on MATERIALIZE VIEW
[ https://issues.apache.org/jira/browse/CASSANDRA-13564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091350#comment-16091350 ] Kurt Greaves commented on CASSANDRA-13564: -- Can you give an example of your schema and what you are trying to achieve? I'm having a little trouble following. > Mismatch Documentation on MATERIALIZE VIEW > -- > > Key: CASSANDRA-13564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13564 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website, Materialized Views > Environment: 3.10 >Reporter: Nick Hryhoriev > Labels: doc-impacting > > During create MATERIALIZED VIEW with out cluster key. > I've got exception "No columns are defined for Materialized View other than > primary key" > Does it expected behave? Because i can't find nothing about this in > Documentation. But this check exist in code. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13648) Upgrade metrics to 3.1.5
[ https://issues.apache.org/jira/browse/CASSANDRA-13648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091400#comment-16091400 ] Stefan Podkowinski commented on CASSANDRA-13648: Patch is missing the actual jar files in lib. The build.xml dependencies are just for creating the project pom, if I remember correctly. > Upgrade metrics to 3.1.5 > > > Key: CASSANDRA-13648 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13648 > Project: Cassandra > Issue Type: Bug > Components: Libraries >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa > Fix For: 4.x > > > GH PR #123 indicates that metrics 3.1.5 will fix a reconnect bug: > https://github.com/apache/cassandra/pull/123 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13387) Metrics for repair
[ https://issues.apache.org/jira/browse/CASSANDRA-13387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091411#comment-16091411 ] Stefan Podkowinski commented on CASSANDRA-13387: Please rebase on latest trunk if you're still interested to work on the patch. This will be trunk only, since not a bug fix. > Metrics for repair > -- > > Key: CASSANDRA-13387 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13387 > Project: Cassandra > Issue Type: Improvement >Reporter: Simon Zhou >Assignee: Simon Zhou >Priority: Minor > > We're missing metrics for repair, especially for errors. From what I observed > now, the exception will be caught by UncaughtExceptionHandler set in > CassandraDaemon and is categorized as StorageMetrics.exceptions. This is one > example: > {code} > ERROR [AntiEntropyStage:1] 2017-03-27 18:17:08,385 CassandraDaemon.java:207 - > Exception in thread Thread[AntiEntropyStage:1,5,main] > java.lang.RuntimeException: Parent repair session with id = > 8c85d260-1319-11e7-82a2-25090a89015f has failed. > at > org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:377) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at > org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:392) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at > org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:172) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > ~[na:1.8.0_121] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > ~[na:1.8.0_121] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > ~[na:1.8.0_121] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_121] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13656) Change default start_native_transport configuration option
[ https://issues.apache.org/jira/browse/CASSANDRA-13656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Podkowinski updated CASSANDRA-13656: --- Status: Open (was: Patch Available) Tomas, please go ahead and update your patch with mentioned changes and ping me when done. > Change default start_native_transport configuration option > -- > > Key: CASSANDRA-13656 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13656 > Project: Cassandra > Issue Type: Wish > Components: Configuration >Reporter: Tomas Repik >Assignee: Tomas Repik >Priority: Trivial > Fix For: 4.x > > Attachments: update_default_config.patch > > > When you don't specify the start_native_transport option in the > cassandra.yaml config file the default value is set to false. So far I did > not find any good reason for setting it this way so I'm proposing to set it > to true as default. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13656) Change default start_native_transport configuration option
[ https://issues.apache.org/jira/browse/CASSANDRA-13656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Podkowinski updated CASSANDRA-13656: --- Reviewer: Stefan Podkowinski > Change default start_native_transport configuration option > -- > > Key: CASSANDRA-13656 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13656 > Project: Cassandra > Issue Type: Wish > Components: Configuration >Reporter: Tomas Repik >Assignee: Tomas Repik >Priority: Trivial > Fix For: 4.x > > Attachments: update_default_config.patch > > > When you don't specify the start_native_transport option in the > cassandra.yaml config file the default value is set to false. So far I did > not find any good reason for setting it this way so I'm proposing to set it > to true as default. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13363) java.lang.ArrayIndexOutOfBoundsException: null
[ https://issues.apache.org/jira/browse/CASSANDRA-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091427#comment-16091427 ] Shashikant Kulkarni commented on CASSANDRA-13363: - Hello, I am also facing the similar issue so adding my comment here. I have Apache Cassandra v3.9, with 3 node in cluster. Replication factor 2. Same datacenter. {noformat} ERROR [MessagingService-Incoming-/10.0.0.111] 2017-07-06 14:26:02,506 CassandraDaemon.java:226 - Exception in thread Thread[MessagingService-Incoming-/10.0.0.111,5,main] java.lang.IndexOutOfBoundsException: index (2) must be less than size (2) at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310) ~[guava-18.0.jar:na] at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:292) ~[guava-18.0.jar:na] at com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:65) ~[guava-18.0.jar:na] at org.apache.cassandra.db.ClusteringPrefix$Serializer.deserializeValuesWithoutSize(ClusteringPrefix.java:359) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.ClusteringBoundOrBoundary$Serializer.deserializeValues(ClusteringBoundOrBoundary.java:179) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.ClusteringBoundOrBoundary$Serializer.deserialize(ClusteringBoundOrBoundary.java:161) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.Slice$Serializer.deserialize(Slice.java:322) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.Slices$Serializer.deserialize(Slices.java:336) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.filter.ClusteringIndexSliceFilter$SliceDeserializer.deserialize(ClusteringIndexSliceFilter.java:174) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.filter.AbstractClusteringIndexFilter$FilterSerializer.deserialize(AbstractClusteringIndexFilter.java:77) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.SinglePartitionReadCommand$Deserializer.deserialize(SinglePartitionReadCommand.java:1041) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.ReadCommand$Serializer.deserialize(ReadCommand.java:696) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.ReadCommand$Serializer.deserialize(ReadCommand.java:626) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.io.ForwardingVersionedSerializer.deserialize(ForwardingVersionedSerializer.java:50) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.net.MessageIn.read(MessageIn.java:114) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:190) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92) ~[apache-cassandra-3.9.0.jar:3.9.0] {noformat} After this exception the cluster becomes unresponsive and start throwing errors if we try to query for web application. Thanks > java.lang.ArrayIndexOutOfBoundsException: null > -- > > Key: CASSANDRA-13363 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13363 > Project: Cassandra > Issue Type: Bug > Environment: CentOS 6, Cassandra 3.10 >Reporter: Artem Rokhin >Priority: Critical > > Constantly see this error in the log without any additional information or a > stack trace. > {code} > Exception in thread Thread[MessagingService-Incoming-/10.0.1.26,5,main] > {code} > {code} > java.lang.ArrayIndexOutOfBoundsException: null > {code} > Logger: org.apache.cassandra.service.CassandraDaemon > Thrdead: MessagingService-Incoming-/10.0.1.12 > Method: uncaughtException > File: CassandraDaemon.java > Line: 229 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13363) java.lang.ArrayIndexOutOfBoundsException: null
[ https://issues.apache.org/jira/browse/CASSANDRA-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091427#comment-16091427 ] Shashikant Kulkarni edited comment on CASSANDRA-13363 at 7/18/17 11:24 AM: --- Hello, I am also facing the similar issue so adding my comment here. I have Apache Cassandra v3.9, with 3 node in cluster. Replication factor 2. Same datacenter. CentOS 6.8, Java 8 {noformat} ERROR [MessagingService-Incoming-/10.0.0.111] 2017-07-06 14:26:02,506 CassandraDaemon.java:226 - Exception in thread Thread[MessagingService-Incoming-/10.0.0.111,5,main] java.lang.IndexOutOfBoundsException: index (2) must be less than size (2) at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310) ~[guava-18.0.jar:na] at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:292) ~[guava-18.0.jar:na] at com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:65) ~[guava-18.0.jar:na] at org.apache.cassandra.db.ClusteringPrefix$Serializer.deserializeValuesWithoutSize(ClusteringPrefix.java:359) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.ClusteringBoundOrBoundary$Serializer.deserializeValues(ClusteringBoundOrBoundary.java:179) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.ClusteringBoundOrBoundary$Serializer.deserialize(ClusteringBoundOrBoundary.java:161) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.Slice$Serializer.deserialize(Slice.java:322) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.Slices$Serializer.deserialize(Slices.java:336) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.filter.ClusteringIndexSliceFilter$SliceDeserializer.deserialize(ClusteringIndexSliceFilter.java:174) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.filter.AbstractClusteringIndexFilter$FilterSerializer.deserialize(AbstractClusteringIndexFilter.java:77) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.SinglePartitionReadCommand$Deserializer.deserialize(SinglePartitionReadCommand.java:1041) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.ReadCommand$Serializer.deserialize(ReadCommand.java:696) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.ReadCommand$Serializer.deserialize(ReadCommand.java:626) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.io.ForwardingVersionedSerializer.deserialize(ForwardingVersionedSerializer.java:50) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.net.MessageIn.read(MessageIn.java:114) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:190) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92) ~[apache-cassandra-3.9.0.jar:3.9.0] {noformat} After this exception the cluster becomes unresponsive and start throwing errors if we try to query for web application. Thanks was (Author: shashikantkulkarni): Hello, I am also facing the similar issue so adding my comment here. I have Apache Cassandra v3.9, with 3 node in cluster. Replication factor 2. Same datacenter. {noformat} ERROR [MessagingService-Incoming-/10.0.0.111] 2017-07-06 14:26:02,506 CassandraDaemon.java:226 - Exception in thread Thread[MessagingService-Incoming-/10.0.0.111,5,main] java.lang.IndexOutOfBoundsException: index (2) must be less than size (2) at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310) ~[guava-18.0.jar:na] at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:292) ~[guava-18.0.jar:na] at com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:65) ~[guava-18.0.jar:na] at org.apache.cassandra.db.ClusteringPrefix$Serializer.deserializeValuesWithoutSize(ClusteringPrefix.java:359) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.ClusteringBoundOrBoundary$Serializer.deserializeValues(ClusteringBoundOrBoundary.java:179) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.ClusteringBoundOrBoundary$Serializer.deserialize(ClusteringBoundOrBoundary.java:161) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.Slice$Serializer.deserialize(Slice.java:322) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.Slices$Serializer.deserialize(Slices.java:336) ~[apache-cassandra-3.9.0.jar:3.9.0] at org.apache.cassandra.db.filter.ClusteringIndexS
[jira] [Updated] (CASSANDRA-13656) Change default start_native_transport configuration option
[ https://issues.apache.org/jira/browse/CASSANDRA-13656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomas Repik updated CASSANDRA-13656: Attachment: update_default_config.patch Here is the new patch. > Change default start_native_transport configuration option > -- > > Key: CASSANDRA-13656 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13656 > Project: Cassandra > Issue Type: Wish > Components: Configuration >Reporter: Tomas Repik >Assignee: Tomas Repik >Priority: Trivial > Fix For: 4.x > > Attachments: update_default_config.patch, update_default_config.patch > > > When you don't specify the start_native_transport option in the > cassandra.yaml config file the default value is set to false. So far I did > not find any good reason for setting it this way so I'm proposing to set it > to true as default. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13656) Change default start_native_transport configuration option
[ https://issues.apache.org/jira/browse/CASSANDRA-13656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091539#comment-16091539 ] Stefan Podkowinski commented on CASSANDRA-13656: * [branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-13656] * [testall|https://circleci.com/gh/spodkowinski/cassandra/80] * [dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/132/] > Change default start_native_transport configuration option > -- > > Key: CASSANDRA-13656 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13656 > Project: Cassandra > Issue Type: Wish > Components: Configuration >Reporter: Tomas Repik >Assignee: Tomas Repik >Priority: Trivial > Fix For: 4.x > > Attachments: update_default_config.patch, update_default_config.patch > > > When you don't specify the start_native_transport option in the > cassandra.yaml config file the default value is set to false. So far I did > not find any good reason for setting it this way so I'm proposing to set it > to true as default. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-11500) Obsolete MV entry may not be properly deleted
[ https://issues.apache.org/jira/browse/CASSANDRA-11500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16082241#comment-16082241 ] ZhaoYang edited comment on CASSANDRA-11500 at 7/18/17 3:10 PM: --- h3. Relation: base -> view First of all, I think all of us should agree on what cases view row should exists. IMO, there are two main cases: 1. base pk and view pk are the same (order doesn't matter) and view has no filter conditions or only conditions on base pk. (filter condition is not a concern here, since no previous view data to be cleared) view row exists if any of following is true: * a. base row pk has live livenessInfo(timestamp) and base row pk satifies view's filter conditions if any. * b. or one of base row columns selected in view has live timestamp (via update) and base row pk satifies view's filter conditions if any. this is handled by existing mechanism of liveness and tombstone since all info are included in view row * c. or one of base row columns not selected in view has live timestamp (via update) and base row pk satifies view's filter conditions if any. Those unselected columns' timestamp/ttl/cell-deletion info currently are not stored on view row. 2. base column used in view pk or view has filter conditions on base non-key column which can also lead to entire view row being wiped. view row exists if any of following is true: * a. base row pk has live livenessInfo(timestamp) && base column used in view pk is not null but no timestamp && conditions are satisfied. ( pk having live livenesInfo means it is not deleted by tombstone) * b. or base row column in view pk has timestamp (via update) && conditions are satisfied. eg. if base column used in view pk is TTLed, entire view row should be wiped. Next thing is to model "shadowable tombstone or shadowable liveness" to maintain view data based on above cases. h3. Previous known issues: (I might miss some issues, feel free to ping me..) ttl * view row is not wiped when TTLed on base column used in view pk or TTLed on base non-key column with filter condition * cells with same timestamp, merging ttls are not deterministic. partial update on base columns not selected in view * it results in no view data. because of current update semantics, no view updates are generated * corresponding view row liveness is not depending on liveness of base columns filter conditions or base column used in view pk causes * view row is shadowed after a few modification on base column used in view pk if the base non-key column has TS greater than base pk's ts and view key column's ts. (as mentioned by sylvain: we need to be able to re-insert an entry when a prior one had been deleted need to be careful to hanlde timestamp tie) tombstone merging is not commutative * in current code, shadowable tombstone doesn't co-exist with regular tombstone sstabledump doesn't not support current shadowable tombstone h3. Model (TO BE UPDATED) {{ShadowableTombstone}} : * deletion-time, isShadowable, and "viewKeyTs" aka. base column's ts which is part of view pk(used to reconcile when timestamp tie), if there is no timestamp associated with that column, use base pk timestamp instead. * it's only generated when one base column is a pk in view and this base column value is changed in base row, to mark previous view row as deleted. (original definition of {{shadowable}} in CASSANDRA-10261). in other cases, {{standard tombstone}} is generated for view rows. * if {{ShadowableTombstone}} is superseded by {{LivenessInfo}}, columns shadowed by {{ShadowableTombstone}} will come back alive. (original definition of {{shadowable}} in CASSANDRA-10261) * {{ShadowableTombstone}} should co-exist with {{Standard Tombstone}} if {{shadowable}}'s deletion time supersedes {{standard tombstone}} to avoid bringing columns older than {{standard tombstone}} coming back alive( as in CASSANDRA-13409) {{ShadowableLivenessInfo}}: * timestamp, and "viewKeyTs" * if shadowable and not live, all columns in this row are considered deleted as in CASSANDRA-13657 and CASSANDRA-13127 to solve partial update issues When reconcile {{ShadowableTombstone}} and {{ShadowableLivenessInfo}}: {quote} if deletion-time greater than timestamp, tombstone wins if deletion-time smaller than timestamp, livenessInfo wins when deletion-time ties with timestamp, - if {{ShadowableTombstone}}'s {{viewKeyTs}} >= {{ShadowableLivenessInfo}}'s, then tombstone wins - else livesnessInfo wins. {quote} When inserting to view, always use the greatest timestamp of all base columns in view similar to how view deletion timestamp is computed. h3. *Example* {quote} CREATE TABLE t (k int PRIMARY KEY, a int, b int); CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS NOT NULL PRIMARY KEY (k, a); {{q1}} INSERT INTO t(k, a, b) VALUES (1, 1, 1) USING TIMESTAMP 0; {
[jira] [Commented] (CASSANDRA-13069) Local batchlog for MV may not be correctly written on node movements
[ https://issues.apache.org/jira/browse/CASSANDRA-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091707#comment-16091707 ] Paulo Motta commented on CASSANDRA-13069: - sorry for the delay, got sucked into other things but I am halfway through a patch so will submit this or next week. > Local batchlog for MV may not be correctly written on node movements > > > Key: CASSANDRA-13069 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13069 > Project: Cassandra > Issue Type: Bug > Components: Materialized Views >Reporter: Sylvain Lebresne >Assignee: Paulo Motta > > Unless I'm really reading this wrong, I think the code > [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageProxy.java#L829-L843], > which comes from CASSANDRA-10674, isn't working properly. > More precisely, I believe we can have both paired and unpaired mutations, so > that both {{if}} can be taken, but if that's the case, the 2nd write to the > batchlog will basically overwrite (remove) the batchlog write of the 1st > {{if}} and I don't think that's the intention. In practice, this means > "paired" mutation won't be in the batchlog, which mean they won't be replayed > at all if they fail. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13137) nodetool disablethrift deadlocks if THsHaDisruptorServer is stopped while a request is being processed
[ https://issues.apache.org/jira/browse/CASSANDRA-13137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091819#comment-16091819 ] Sotirios Delimanolis commented on CASSANDRA-13137: -- The PR is merged. Pavel is making a release soon. Can we have it included in 2.2+? > nodetool disablethrift deadlocks if THsHaDisruptorServer is stopped while a > request is being processed > -- > > Key: CASSANDRA-13137 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13137 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: 2.2.9 >Reporter: Sotirios Delimanolis > > We are using Thrift with {{rpc_server_type}} set to {{hsha}}. This creates a > {{THsHaDisruptorServer}} which is a subclass of > [{{TDisruptorServer}}|https://github.com/xedin/disruptor_thrift_server/blob/master/src/main/java/com/thinkaurelius/thrift/TDisruptorServer.java]. > Internally, this spawns {{number_of_cores}} number of selector threads. Each > gets a {{RingBuffer}} and {{rpc_max_threads / cores}} number of worker > threads (the {{RPC-Thread}} threads). As the server starts receiving > requests, each selector thread adds events to its {{RingBuffer}} and the > worker threads process them. > The _events_ are > [{{Message}}|https://github.com/xedin/disruptor_thrift_server/blob/master/src/main/java/com/thinkaurelius/thrift/Message.java] > instances, which have preallocated buffers for eventual IO. > When the thrift server starts up, the corresponding {{ThriftServerThread}} > joins on the selector threads, waiting for them to die. It then iterates > through all the {{SelectorThread}} objects and calls their {{shutdown}} > method which attempts to drain their corresponding {{RingBuffer}}. The [drain > ({{drainAndHalt}})|https://github.com/LMAX-Exchange/disruptor/blob/master/src/main/java/com/lmax/disruptor/WorkerPool.java#L147] > works by letting the worker pool "consumer" threads catch up to the > "producer" index, ie. the selector thread. > When we execute a {{nodetool disablethrift}}, it attempts to {{stop}} the > {{THsHaDisruptorServer}}. That works by setting a {{stopped}} flag to > {{true}}. When the selector threads see that, they break from their > {{select()}} loop, and clean up their resources, ie. the {{Message}} objects > they've created and their buffers. *However*, if one of those {{Message}} > objects is currently being used by a worker pool thread to process a request, > if it calls [this piece of > code|https://github.com/xedin/disruptor_thrift_server/blob/master/src/main/java/com/thinkaurelius/thrift/Message.java#L317], > you'll get the following {{NullPointerException}} > {noformat} > Jan 18, 2017 6:28:50 PM com.lmax.disruptor.FatalExceptionHandler > handleEventException > SEVERE: Exception processing: 633124 > com.thinkaurelius.thrift.Message$Invocation@25c9fbeb > java.lang.NullPointerException > at > com.thinkaurelius.thrift.Message.getInputTransport(Message.java:338) > at com.thinkaurelius.thrift.Message.invoke(Message.java:308) > at > com.thinkaurelius.thrift.Message$Invocation.execute(Message.java:90) > at > com.thinkaurelius.thrift.TDisruptorServer$InvocationHandler.onEvent(TDisruptorServer.java:695) > at > com.thinkaurelius.thrift.TDisruptorServer$InvocationHandler.onEvent(TDisruptorServer.java:689) > at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:112) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {noformat} > That fails because it tries to dereference one of the {{Message}} "cleaned > up", ie. {{null}}, buffers. > Because that call is outside the {{try}} block, the exception escapes and > basically kills the worker pool thread. This has the side effect of > "discarding" one of the consumers of a selector's {{RingBuffer}}. > *That* has the side effect of preventing the {{ThriftServerThread}} from > draining the {{RingBuffer}} (and dying) since the consumers never catch up to > the stopped producer. And that finally has the effect of preventing the > {{nodetool disablethrift}} from proceeding since it's trying to {{join}} the > {{ThriftServerThread}}. Deadlock! > The {{ThriftServerThread}} thread looks like > {noformat} > "Thread-1" #2234 prio=5 os_prio=0 tid=0x7f4ae6ff1000 nid=0x2eb6 runnable > [0x7f4729174000] >java.lang.Thread.State: RUNNABLE > at java.lang.Thread.yield(Native Method) > at com.lmax.disruptor.WorkerPool.drainAndHalt(WorkerPool.java:147) > at > com.thinkaurelius.thrift.TDisruptorServer$
[jira] [Commented] (CASSANDRA-13137) nodetool disablethrift deadlocks if THsHaDisruptorServer is stopped while a request is being processed
[ https://issues.apache.org/jira/browse/CASSANDRA-13137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091832#comment-16091832 ] Jeff Jirsa commented on CASSANDRA-13137: I've been bit by this, personally, at a past employer, so I know exactly how frustrating it is to try to have tooling shut down cassandra only to find out it doesn't shut down cleanly. Changing out the thrift server in a 2.2 tree that's 2 years old ( {{2.2.0}} was tagged July 20 2015) seems like a really invasive change for an annoying bug that doesn't actually impact running servers. However, looking at the changelog for disruptor: https://github.com/xedin/disruptor_thrift_server/commits/master , it looks like it's only really a few changes: [avoid interest changes when processing is still in progress|https://github.com/xedin/disruptor_thrift_server/commit/36d585756b8e5a87f8ab8c309a55c4ba4d5cb730], a [junit fix|https://github.com/xedin/disruptor_thrift_server/commit/ea47bc90cca2362ff93bd786c44d5d078f711d25] and [this patch|https://github.com/xedin/disruptor_thrift_server/commit/6a96b47de29bb4fcd75944f17b724863398b37c4] , so perhaps it's not that scary. [~brandon.williams] - you tend to be as conservative as I am. How do you feel about bumping thrift server in 2.2 ? > nodetool disablethrift deadlocks if THsHaDisruptorServer is stopped while a > request is being processed > -- > > Key: CASSANDRA-13137 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13137 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: 2.2.9 >Reporter: Sotirios Delimanolis > > We are using Thrift with {{rpc_server_type}} set to {{hsha}}. This creates a > {{THsHaDisruptorServer}} which is a subclass of > [{{TDisruptorServer}}|https://github.com/xedin/disruptor_thrift_server/blob/master/src/main/java/com/thinkaurelius/thrift/TDisruptorServer.java]. > Internally, this spawns {{number_of_cores}} number of selector threads. Each > gets a {{RingBuffer}} and {{rpc_max_threads / cores}} number of worker > threads (the {{RPC-Thread}} threads). As the server starts receiving > requests, each selector thread adds events to its {{RingBuffer}} and the > worker threads process them. > The _events_ are > [{{Message}}|https://github.com/xedin/disruptor_thrift_server/blob/master/src/main/java/com/thinkaurelius/thrift/Message.java] > instances, which have preallocated buffers for eventual IO. > When the thrift server starts up, the corresponding {{ThriftServerThread}} > joins on the selector threads, waiting for them to die. It then iterates > through all the {{SelectorThread}} objects and calls their {{shutdown}} > method which attempts to drain their corresponding {{RingBuffer}}. The [drain > ({{drainAndHalt}})|https://github.com/LMAX-Exchange/disruptor/blob/master/src/main/java/com/lmax/disruptor/WorkerPool.java#L147] > works by letting the worker pool "consumer" threads catch up to the > "producer" index, ie. the selector thread. > When we execute a {{nodetool disablethrift}}, it attempts to {{stop}} the > {{THsHaDisruptorServer}}. That works by setting a {{stopped}} flag to > {{true}}. When the selector threads see that, they break from their > {{select()}} loop, and clean up their resources, ie. the {{Message}} objects > they've created and their buffers. *However*, if one of those {{Message}} > objects is currently being used by a worker pool thread to process a request, > if it calls [this piece of > code|https://github.com/xedin/disruptor_thrift_server/blob/master/src/main/java/com/thinkaurelius/thrift/Message.java#L317], > you'll get the following {{NullPointerException}} > {noformat} > Jan 18, 2017 6:28:50 PM com.lmax.disruptor.FatalExceptionHandler > handleEventException > SEVERE: Exception processing: 633124 > com.thinkaurelius.thrift.Message$Invocation@25c9fbeb > java.lang.NullPointerException > at > com.thinkaurelius.thrift.Message.getInputTransport(Message.java:338) > at com.thinkaurelius.thrift.Message.invoke(Message.java:308) > at > com.thinkaurelius.thrift.Message$Invocation.execute(Message.java:90) > at > com.thinkaurelius.thrift.TDisruptorServer$InvocationHandler.onEvent(TDisruptorServer.java:695) > at > com.thinkaurelius.thrift.TDisruptorServer$InvocationHandler.onEvent(TDisruptorServer.java:689) > at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:112) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {noformat} > That fails because it tries to dereference one of the {{Messa
[jira] [Commented] (CASSANDRA-13648) Upgrade metrics to 3.1.5
[ https://issues.apache.org/jira/browse/CASSANDRA-13648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091852#comment-16091852 ] Jeff Jirsa commented on CASSANDRA-13648: Thanks [~spo...@gmail.com] - just rebased and force pushed with new metrics-core, metrics-jvm, and metrics-logback jars (3.1.5) > Upgrade metrics to 3.1.5 > > > Key: CASSANDRA-13648 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13648 > Project: Cassandra > Issue Type: Bug > Components: Libraries >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa > Fix For: 4.x > > > GH PR #123 indicates that metrics 3.1.5 will fix a reconnect bug: > https://github.com/apache/cassandra/pull/123 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13387) Metrics for repair
[ https://issues.apache.org/jira/browse/CASSANDRA-13387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091917#comment-16091917 ] Simon Zhou commented on CASSANDRA-13387: Rebased patch: |4.0 |[patch | https://github.com/szhou1234/cassandra/commit/54ad6690fd306fe2b5a93d73808064ab29f1]| > Metrics for repair > -- > > Key: CASSANDRA-13387 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13387 > Project: Cassandra > Issue Type: Improvement >Reporter: Simon Zhou >Assignee: Simon Zhou >Priority: Minor > > We're missing metrics for repair, especially for errors. From what I observed > now, the exception will be caught by UncaughtExceptionHandler set in > CassandraDaemon and is categorized as StorageMetrics.exceptions. This is one > example: > {code} > ERROR [AntiEntropyStage:1] 2017-03-27 18:17:08,385 CassandraDaemon.java:207 - > Exception in thread Thread[AntiEntropyStage:1,5,main] > java.lang.RuntimeException: Parent repair session with id = > 8c85d260-1319-11e7-82a2-25090a89015f has failed. > at > org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:377) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at > org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:392) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at > org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:172) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) > ~[apache-cassandra-3.0.10.jar:3.0.10] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > ~[na:1.8.0_121] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > ~[na:1.8.0_121] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > ~[na:1.8.0_121] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_121] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13137) nodetool disablethrift deadlocks if THsHaDisruptorServer is stopped while a request is being processed
[ https://issues.apache.org/jira/browse/CASSANDRA-13137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091926#comment-16091926 ] Brandon Williams commented on CASSANDRA-13137: -- If I'm being honest, I have no dog in a fight about thrift, I simply don't care about it anymore after all this time and these battle scars. That said, you make fair points in both directions, so... flip a coin, for me. > nodetool disablethrift deadlocks if THsHaDisruptorServer is stopped while a > request is being processed > -- > > Key: CASSANDRA-13137 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13137 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: 2.2.9 >Reporter: Sotirios Delimanolis > > We are using Thrift with {{rpc_server_type}} set to {{hsha}}. This creates a > {{THsHaDisruptorServer}} which is a subclass of > [{{TDisruptorServer}}|https://github.com/xedin/disruptor_thrift_server/blob/master/src/main/java/com/thinkaurelius/thrift/TDisruptorServer.java]. > Internally, this spawns {{number_of_cores}} number of selector threads. Each > gets a {{RingBuffer}} and {{rpc_max_threads / cores}} number of worker > threads (the {{RPC-Thread}} threads). As the server starts receiving > requests, each selector thread adds events to its {{RingBuffer}} and the > worker threads process them. > The _events_ are > [{{Message}}|https://github.com/xedin/disruptor_thrift_server/blob/master/src/main/java/com/thinkaurelius/thrift/Message.java] > instances, which have preallocated buffers for eventual IO. > When the thrift server starts up, the corresponding {{ThriftServerThread}} > joins on the selector threads, waiting for them to die. It then iterates > through all the {{SelectorThread}} objects and calls their {{shutdown}} > method which attempts to drain their corresponding {{RingBuffer}}. The [drain > ({{drainAndHalt}})|https://github.com/LMAX-Exchange/disruptor/blob/master/src/main/java/com/lmax/disruptor/WorkerPool.java#L147] > works by letting the worker pool "consumer" threads catch up to the > "producer" index, ie. the selector thread. > When we execute a {{nodetool disablethrift}}, it attempts to {{stop}} the > {{THsHaDisruptorServer}}. That works by setting a {{stopped}} flag to > {{true}}. When the selector threads see that, they break from their > {{select()}} loop, and clean up their resources, ie. the {{Message}} objects > they've created and their buffers. *However*, if one of those {{Message}} > objects is currently being used by a worker pool thread to process a request, > if it calls [this piece of > code|https://github.com/xedin/disruptor_thrift_server/blob/master/src/main/java/com/thinkaurelius/thrift/Message.java#L317], > you'll get the following {{NullPointerException}} > {noformat} > Jan 18, 2017 6:28:50 PM com.lmax.disruptor.FatalExceptionHandler > handleEventException > SEVERE: Exception processing: 633124 > com.thinkaurelius.thrift.Message$Invocation@25c9fbeb > java.lang.NullPointerException > at > com.thinkaurelius.thrift.Message.getInputTransport(Message.java:338) > at com.thinkaurelius.thrift.Message.invoke(Message.java:308) > at > com.thinkaurelius.thrift.Message$Invocation.execute(Message.java:90) > at > com.thinkaurelius.thrift.TDisruptorServer$InvocationHandler.onEvent(TDisruptorServer.java:695) > at > com.thinkaurelius.thrift.TDisruptorServer$InvocationHandler.onEvent(TDisruptorServer.java:689) > at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:112) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {noformat} > That fails because it tries to dereference one of the {{Message}} "cleaned > up", ie. {{null}}, buffers. > Because that call is outside the {{try}} block, the exception escapes and > basically kills the worker pool thread. This has the side effect of > "discarding" one of the consumers of a selector's {{RingBuffer}}. > *That* has the side effect of preventing the {{ThriftServerThread}} from > draining the {{RingBuffer}} (and dying) since the consumers never catch up to > the stopped producer. And that finally has the effect of preventing the > {{nodetool disablethrift}} from proceeding since it's trying to {{join}} the > {{ThriftServerThread}}. Deadlock! > The {{ThriftServerThread}} thread looks like > {noformat} > "Thread-1" #2234 prio=5 os_prio=0 tid=0x7f4ae6ff1000 nid=0x2eb6 runnable > [0x7f4729174000] >java.lang.Thread.State: RUNNABLE > at java.lang.Thread.yield(Native Method) >
[jira] [Created] (CASSANDRA-13698) Reinstate or get rid of unit tests with multiple compaction strategies
Paulo Motta created CASSANDRA-13698: --- Summary: Reinstate or get rid of unit tests with multiple compaction strategies Key: CASSANDRA-13698 URL: https://issues.apache.org/jira/browse/CASSANDRA-13698 Project: Cassandra Issue Type: Test Reporter: Paulo Motta Priority: Minor At some point there were (anti-)compaction tests with multiple compaction strategy classes, but now it's only tested with {{STCS}}: * [AnticompactionTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java#L247] * [CompactionsTest|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java#L85] We should either reinstate these tests or decide they are not important and remove the unused parameter. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13648) Upgrade metrics to 3.1.5
[ https://issues.apache.org/jira/browse/CASSANDRA-13648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092271#comment-16092271 ] Jeff Jirsa commented on CASSANDRA-13648: New dtest run started @ https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/134/ CircleCI run green @ https://circleci.com/gh/jeffjirsa/cassandra/250 > Upgrade metrics to 3.1.5 > > > Key: CASSANDRA-13648 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13648 > Project: Cassandra > Issue Type: Bug > Components: Libraries >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa > Fix For: 4.x > > > GH PR #123 indicates that metrics 3.1.5 will fix a reconnect bug: > https://github.com/apache/cassandra/pull/123 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-12014) IndexSummary > 2G causes an assertion error
[ https://issues.apache.org/jira/browse/CASSANDRA-12014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa updated CASSANDRA-12014: --- Reviewer: (was: Jeff Jirsa) > IndexSummary > 2G causes an assertion error > --- > > Key: CASSANDRA-12014 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12014 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Brandon Williams >Assignee: Stefania >Priority: Minor > Fix For: 3.0.x, 3.11.x > > > {noformat} > ERROR [CompactionExecutor:1546280] 2016-06-01 13:21:00,444 > CassandraDaemon.java:229 - Exception in thread > Thread[CompactionExecutor:1546280,1,main] > java.lang.AssertionError: null > at > org.apache.cassandra.io.sstable.IndexSummaryBuilder.maybeAddEntry(IndexSummaryBuilder.java:171) > ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] > at > org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.append(SSTableWriter.java:634) > ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] > at > org.apache.cassandra.io.sstable.SSTableWriter.afterAppend(SSTableWriter.java:179) > ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] > at > org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:205) > ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] > at > org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:126) > ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] > at > org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197) > ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] > at > org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73) > ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] > at > org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) > ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] > at > org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263) > ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > ~[na:1.7.0_51] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_51] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > ~[na:1.7.0_51] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_51] > at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51] > {noformat} > I believe this can be fixed by raising the min_index_interval, but we should > have a better method of coping with this than throwing the AE. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13664) RangeFetchMapCalculator should not try to optimise 'trivial' ranges
[ https://issues.apache.org/jira/browse/CASSANDRA-13664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092496#comment-16092496 ] Marcus Eriksson commented on CASSANDRA-13664: - https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/135/ > RangeFetchMapCalculator should not try to optimise 'trivial' ranges > --- > > Key: CASSANDRA-13664 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13664 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 4.x > > > RangeFetchMapCalculator (CASSANDRA-4650) tries to make the number of streams > out of each node as even as possible. > In a typical multi-dc ring the nodes in the dcs are setup using token + 1, > creating many tiny ranges. If we only try to optimise over the number of > streams, it is likely that the amount of data streamed out of each node is > unbalanced. > We should ignore those trivial ranges and only optimise the big ones, then > share the tiny ones over the nodes. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12173) Materialized View may turn on TRACING
[ https://issues.apache.org/jira/browse/CASSANDRA-12173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092558#comment-16092558 ] Hiroshi Usami commented on CASSANDRA-12173: --- I am sorry for the expectation, we had to give up trying c*3.0.6 cluster at that time because of this, so I cannot try your suggestion with the same cluster now. A person who was working on the client side said to me: - The cluster had very write heavy workload. - He created 2i and MV the day before and observed that it brought 8 times higher writes than usual. - He started additional writes at the day morning, then Cassandra::Errors::WriteTimeoutError started happening. (increasing SSTables in system_traces) I have quite no idea for the bug. Is it possible that someone tried TRACING ON at "2016-07-07T09:04:57.197Z" but only 2 nodes created so many SSTables? (my hypothesis) > Materialized View may turn on TRACING > - > > Key: CASSANDRA-12173 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12173 > Project: Cassandra > Issue Type: Bug > Components: Materialized Views >Reporter: Hiroshi Usami > > We observed this in our test cluster(C*3.0.6), but TRAING was OFF apparently. > After creating Materialized View, the Write count jumped up to 20K from 5K, > and the ViewWrite rose up to 10K. > This is supposed to be done by MV, but some nodes which had 14,000+ SSTables > in the system_traces directory went down in a half day, because of running > out of file descriptors. > {code} > Counting by: find /var/lib/cassandra/data/system_traces/ -name "*-Data.db"|wc > -l > node01: 0 > node02: 3 > node03: 1 > node04: 0 > node05: 0 > node06: 0 > node07: 2 > node08: 0 > node09: 0 > node10: 0 > node11: 2 > node12: 2 > node13: 1 > node14: 7 > node15: 1 > node16: 5 > node17: 0 > node18: 0 > node19: 0 > node20: 0 > node21: 1 > node22: 0 > node23: 2 > node24: 14420 > node25: 0 > node26: 2 > node27: 0 > node28: 1 > node29: 1 > node30: 2 > node31: 1 > node32: 0 > node33: 0 > node34: 0 > node35: 14371 > node36: 0 > node37: 1 > node38: 0 > node39: 0 > node40: 1 > {code} > In node24, the sstabledump of the oldest SSTable in system_traces/events > directory starts with: > {code} > [ > { > "partition" : { > "key" : [ "e07851d0-4421-11e6-abd7-59d7f275ba79" ], > "position" : 0 > }, > "rows" : [ > { > "type" : "row", > "position" : 30, > "clustering" : [ "e07878e0-4421-11e6-abd7-59d7f275ba79" ], > "liveness_info" : { "tstamp" : "2016-07-07T09:04:57.197Z", "ttl" : > 86400, "expires_at" : "2016-07-08T09:04:57Z", "expired" : true }, > "cells" : [ > { "name" : "activity", "value" : "Parsing CREATE MATERIALIZED VIEW > ... > {code} > So this could be the begining of TRACING ON implicitly. In node35, the oldest > one also starts with the "Parsing CREATE MATERIALIZED VIEW". -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13694) sstabledump does not show full precision of timestamp columns
[ https://issues.apache.org/jira/browse/CASSANDRA-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092561#comment-16092561 ] Varun Barala commented on CASSANDRA-13694: -- in DevCenter It shows like this:- ``` 1234,TEST EVENT,1970-01-18 16:16:13+0730,1970-01-18 16:16:13+0730 ``` It depends on `TimestampSerializer` `toString()` method. Solution:- * We can modify the default `toString` method's format but need to check, It'll not break or cause unexpected behavior. Thanks!! > sstabledump does not show full precision of timestamp columns > - > > Key: CASSANDRA-13694 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13694 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: Ubuntu 16.04 LTS >Reporter: Tim Reeves > Fix For: 3.7 > > > Create a table: > CREATE TABLE test_table ( > unit_no bigint, > event_code text, > active_time timestamp, > ack_time timestamp, > PRIMARY KEY ((unit_no, event_code), active_time) > ) WITH CLUSTERING ORDER BY (active_time DESC) > Insert a row: > INSERT INTO test_table (unit_no, event_code, active_time, ack_time) > VALUES (1234, 'TEST EVENT', toTimestamp(now()), > toTimestamp(now())); > Verify that it is in the database with a full timestamp: > cqlsh:pentaho> select * from test_table; > unit_no | event_code | active_time | ack_time > -++-+- > 1234 | TEST EVENT | 2017-07-14 14:52:39.919000+ | 2017-07-14 > 14:52:39.919000+ > (1 rows) > Write file: > nodetool flush > nodetool compact pentaho > Use sstabledump: > treeves@ubuntu:~$ sstabledump > /var/lib/cassandra/data/pentaho/test_table-99ba228068a311e7ac30953b79ac2c3e/mb-2-big-Data.db > [ > { > "partition" : { > "key" : [ "1234", "TEST EVENT" ], > "position" : 0 > }, > "rows" : [ > { > "type" : "row", > "position" : 38, > "clustering" : [ "2017-07-14 15:52+0100" ], > "liveness_info" : { "tstamp" : "2017-07-14T14:52:39.888701Z" }, > "cells" : [ > { "name" : "ack_time", "value" : "2017-07-14 15:52+0100" } > ] > } > ] > } > ] > treeves@ubuntu:~$ > The timestamp in the cluster key, and the regular column, are both truncated > to the minute. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13526) nodetool cleanup on KS with no replicas should remove old data, not silently complete
[ https://issues.apache.org/jira/browse/CASSANDRA-13526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092573#comment-16092573 ] Jeff Jirsa commented on CASSANDRA-13526: Patch looks good to me, dtest looks good as well, with two comments: 1) New dtest repo is https://github.com/apache/cassandra-dtest 2) You should remove dc1 from the replication strategy [here|https://github.com/riptano/cassandra-dtest/commit/15bf712988fb50ae29994da246dec186beff69bd#diff-9d7bd37d410a5598b9700b71476845ebR159] to be very explicit about what we expect to happen. > nodetool cleanup on KS with no replicas should remove old data, not silently > complete > - > > Key: CASSANDRA-13526 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13526 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Jeff Jirsa >Assignee: ZhaoYang > Labels: usability > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x > > > From the user list: > https://lists.apache.org/thread.html/5d49cc6bbc6fd2e5f8b12f2308a3e24212a55afbb441af5cb8cd4167@%3Cuser.cassandra.apache.org%3E > If you have a multi-dc cluster, but some keyspaces not replicated to a given > DC, you'll be unable to run cleanup on those keyspaces in that DC, because > [the cleanup code will see no ranges and exit > early|https://github.com/apache/cassandra/blob/4cfaf85/src/java/org/apache/cassandra/db/compaction/CompactionManager.java#L427-L441] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13526) nodetool cleanup on KS with no replicas should remove old data, not silently complete
[ https://issues.apache.org/jira/browse/CASSANDRA-13526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092573#comment-16092573 ] Jeff Jirsa edited comment on CASSANDRA-13526 at 7/19/17 4:59 AM: - Patch looks good to me, dtest looks good as well, with two comments: 1) New dtest repo is https://github.com/apache/cassandra-dtest 2) You should remove dc1 from the replication strategy [here|https://github.com/riptano/cassandra-dtest/commit/15bf712988fb50ae29994da246dec186beff69bd#diff-9d7bd37d410a5598b9700b71476845ebR159] to be very explicit about what we expect to happen. Would you backport to 3.0 and 3.11 ? was (Author: jjirsa): Patch looks good to me, dtest looks good as well, with two comments: 1) New dtest repo is https://github.com/apache/cassandra-dtest 2) You should remove dc1 from the replication strategy [here|https://github.com/riptano/cassandra-dtest/commit/15bf712988fb50ae29994da246dec186beff69bd#diff-9d7bd37d410a5598b9700b71476845ebR159] to be very explicit about what we expect to happen. > nodetool cleanup on KS with no replicas should remove old data, not silently > complete > - > > Key: CASSANDRA-13526 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13526 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Jeff Jirsa >Assignee: ZhaoYang > Labels: usability > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x > > > From the user list: > https://lists.apache.org/thread.html/5d49cc6bbc6fd2e5f8b12f2308a3e24212a55afbb441af5cb8cd4167@%3Cuser.cassandra.apache.org%3E > If you have a multi-dc cluster, but some keyspaces not replicated to a given > DC, you'll be unable to run cleanup on those keyspaces in that DC, because > [the cleanup code will see no ranges and exit > early|https://github.com/apache/cassandra/blob/4cfaf85/src/java/org/apache/cassandra/db/compaction/CompactionManager.java#L427-L441] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13696) Digest mismatch Exception if hints file has UnknownColumnFamily
[ https://issues.apache.org/jira/browse/CASSANDRA-13696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092595#comment-16092595 ] Jeff Jirsa commented on CASSANDRA-13696: Hey [~jay.zhuang] - took a look and I don't see any issues. The new tests look good. trunk circleCI run failed due to circleci, but run #26 looks green. I'd like to glance at it again tomorrow when I'm more awake, but as of right now, it seems very reasonable and correct to me. > Digest mismatch Exception if hints file has UnknownColumnFamily > --- > > Key: CASSANDRA-13696 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13696 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Jay Zhuang >Assignee: Jay Zhuang >Priority: Critical > > {noformat} > WARN [HintsDispatcher:2] 2017-07-16 22:00:32,579 HintsReader.java:235 - > Failed to read a hint for /127.0.0.2: a2b7daf1-a6a4-4dfc-89de-32d12d2d48b0 - > table with id 3882bbb0-6a71-11e7-9bca-2759083e3964 is unknown in file > a2b7daf1-a6a4-4dfc-89de-32d12d2d48b0-1500242103097-1.hints > ERROR [HintsDispatcher:2] 2017-07-16 22:00:32,580 > HintsDispatchExecutor.java:234 - Failed to dispatch hints file > a2b7daf1-a6a4-4dfc-89de-32d12d2d48b0-1500242103097-1.hints: file is corrupted > ({}) > org.apache.cassandra.io.FSReadError: java.io.IOException: Digest mismatch > exception > at > org.apache.cassandra.hints.HintsReader$HintsIterator.computeNext(HintsReader.java:199) > ~[main/:na] > at > org.apache.cassandra.hints.HintsReader$HintsIterator.computeNext(HintsReader.java:164) > ~[main/:na] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[main/:na] > at > org.apache.cassandra.hints.HintsDispatcher.sendHints(HintsDispatcher.java:157) > ~[main/:na] > at > org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:139) > ~[main/:na] > at > org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:123) > ~[main/:na] > at > org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:95) > ~[main/:na] > at > org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:268) > [main/:na] > at > org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:251) > [main/:na] > at > org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:229) > [main/:na] > at > org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:208) > [main/:na] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_111] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_111] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_111] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_111] > at > org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79) > [main/:na] > at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111] > Caused by: java.io.IOException: Digest mismatch exception > at > org.apache.cassandra.hints.HintsReader$HintsIterator.computeNextInternal(HintsReader.java:216) > ~[main/:na] > at > org.apache.cassandra.hints.HintsReader$HintsIterator.computeNext(HintsReader.java:190) > ~[main/:na] > ... 16 common frames omitted > {noformat} > It causes multiple cassandra nodes stop [by > default|https://github.com/apache/cassandra/blob/cassandra-3.0/conf/cassandra.yaml#L188]. > Here is the reproduce steps on a 3 nodes cluster, RF=3: > 1. stop node1 > 2. send some data with quorum (or one), it will generate hints file on > node2/node3 > 3. drop the table > 4. start node1 > node2/node3 will report "corrupted hints file" and stop. The impact is very > bad for a large cluster, when it happens, almost all the nodes are down at > the same time and we have to remove all the hints files (which contain the > dropped table) to bring the node back. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12173) Materialized View may turn on TRACING
[ https://issues.apache.org/jira/browse/CASSANDRA-12173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092617#comment-16092617 ] Kurt Greaves commented on CASSANDRA-12173: -- system_traces by default has an RF of 2. I suspect you had one tracing session that traced many queries, and thus resulted in only one partition being written (thus only going to 2 nodes). Without more details and a reproducible case it's hard to confirm if there is any bug here. I think this issue can be closed - if it is an actual issue it will undoubtedly be run into again. > Materialized View may turn on TRACING > - > > Key: CASSANDRA-12173 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12173 > Project: Cassandra > Issue Type: Bug > Components: Materialized Views >Reporter: Hiroshi Usami > > We observed this in our test cluster(C*3.0.6), but TRAING was OFF apparently. > After creating Materialized View, the Write count jumped up to 20K from 5K, > and the ViewWrite rose up to 10K. > This is supposed to be done by MV, but some nodes which had 14,000+ SSTables > in the system_traces directory went down in a half day, because of running > out of file descriptors. > {code} > Counting by: find /var/lib/cassandra/data/system_traces/ -name "*-Data.db"|wc > -l > node01: 0 > node02: 3 > node03: 1 > node04: 0 > node05: 0 > node06: 0 > node07: 2 > node08: 0 > node09: 0 > node10: 0 > node11: 2 > node12: 2 > node13: 1 > node14: 7 > node15: 1 > node16: 5 > node17: 0 > node18: 0 > node19: 0 > node20: 0 > node21: 1 > node22: 0 > node23: 2 > node24: 14420 > node25: 0 > node26: 2 > node27: 0 > node28: 1 > node29: 1 > node30: 2 > node31: 1 > node32: 0 > node33: 0 > node34: 0 > node35: 14371 > node36: 0 > node37: 1 > node38: 0 > node39: 0 > node40: 1 > {code} > In node24, the sstabledump of the oldest SSTable in system_traces/events > directory starts with: > {code} > [ > { > "partition" : { > "key" : [ "e07851d0-4421-11e6-abd7-59d7f275ba79" ], > "position" : 0 > }, > "rows" : [ > { > "type" : "row", > "position" : 30, > "clustering" : [ "e07878e0-4421-11e6-abd7-59d7f275ba79" ], > "liveness_info" : { "tstamp" : "2016-07-07T09:04:57.197Z", "ttl" : > 86400, "expires_at" : "2016-07-08T09:04:57Z", "expired" : true }, > "cells" : [ > { "name" : "activity", "value" : "Parsing CREATE MATERIALIZED VIEW > ... > {code} > So this could be the begining of TRACING ON implicitly. In node35, the oldest > one also starts with the "Parsing CREATE MATERIALIZED VIEW". -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12173) Materialized View may turn on TRACING
[ https://issues.apache.org/jira/browse/CASSANDRA-12173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092623#comment-16092623 ] Jeff Jirsa commented on CASSANDRA-12173: The sstables are ordered by decorated key, which is going to be basically random. The fact that a materialized view statement happens to be first is probably just chance. I haven't read any of the MV code, but it seems very unlikely that it turns on tracing (but I suppose nothing is impossible). > Materialized View may turn on TRACING > - > > Key: CASSANDRA-12173 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12173 > Project: Cassandra > Issue Type: Bug > Components: Materialized Views >Reporter: Hiroshi Usami > > We observed this in our test cluster(C*3.0.6), but TRAING was OFF apparently. > After creating Materialized View, the Write count jumped up to 20K from 5K, > and the ViewWrite rose up to 10K. > This is supposed to be done by MV, but some nodes which had 14,000+ SSTables > in the system_traces directory went down in a half day, because of running > out of file descriptors. > {code} > Counting by: find /var/lib/cassandra/data/system_traces/ -name "*-Data.db"|wc > -l > node01: 0 > node02: 3 > node03: 1 > node04: 0 > node05: 0 > node06: 0 > node07: 2 > node08: 0 > node09: 0 > node10: 0 > node11: 2 > node12: 2 > node13: 1 > node14: 7 > node15: 1 > node16: 5 > node17: 0 > node18: 0 > node19: 0 > node20: 0 > node21: 1 > node22: 0 > node23: 2 > node24: 14420 > node25: 0 > node26: 2 > node27: 0 > node28: 1 > node29: 1 > node30: 2 > node31: 1 > node32: 0 > node33: 0 > node34: 0 > node35: 14371 > node36: 0 > node37: 1 > node38: 0 > node39: 0 > node40: 1 > {code} > In node24, the sstabledump of the oldest SSTable in system_traces/events > directory starts with: > {code} > [ > { > "partition" : { > "key" : [ "e07851d0-4421-11e6-abd7-59d7f275ba79" ], > "position" : 0 > }, > "rows" : [ > { > "type" : "row", > "position" : 30, > "clustering" : [ "e07878e0-4421-11e6-abd7-59d7f275ba79" ], > "liveness_info" : { "tstamp" : "2016-07-07T09:04:57.197Z", "ttl" : > 86400, "expires_at" : "2016-07-08T09:04:57Z", "expired" : true }, > "cells" : [ > { "name" : "activity", "value" : "Parsing CREATE MATERIALIZED VIEW > ... > {code} > So this could be the begining of TRACING ON implicitly. In node35, the oldest > one also starts with the "Parsing CREATE MATERIALIZED VIEW". -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13696) Digest mismatch Exception if hints file has UnknownColumnFamily
[ https://issues.apache.org/jira/browse/CASSANDRA-13696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092658#comment-16092658 ] Jay Zhuang commented on CASSANDRA-13696: Thanks [~jjirsa]. I did more investigation today. Seems it's more serious than I thought. Even there's no down node, "drop table" while there's write traffic, it will trigger the problem. Here is reproduce steps: 1. Create a 3 nodes cluster: {{$ ccm create test13696 -v 3.0.14 && ccm populate -n 3 && ccm start}} 2. Send some traffics with cassandra-stress (blogpost.yaml is only in trunk, if you use another yaml file, change the RF=3) {{$ tools/bin/cassandra-stress user profile=test/resources/blogpost.yaml cl=QUORUM truncate=never ops\(insert=1\) duration=30m -rate threads=2 -mode native cql3 -node 127.0.0.1}} 3. While the traffic is running, drop table {{$ cqlsh -e "drop table stresscql.blogposts"}} *All 3 nodes go down because of "Digest mismatch Exception".* The CRC calculation problem has been there for a long time, but only got exposed after CASSANDRA-13004 because of the MessagingService version bump. In the normal case when the versions are the same, HintsDispatcher uses {{[page.buffersIterator()|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsDispatcher.java#L138]}} instead of {{[page.hintsIterator()|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsDispatcher.java#L139]}}. {{buffersIterator()}} doesn't need to decode hints, so it won't have the problem. I think the messagingVersion for the hints file should be updated: https://github.com/apache/cassandra/compare/cassandra-3.0...cooldoger:13696.2-3.0?expand=1 so it could dispatch hints in an optimized way. Not sure if we need to check/bump other {{MessagingService.VERSION_30}}s in the 3.0 branch. cc [~ifesdjeen] > Digest mismatch Exception if hints file has UnknownColumnFamily > --- > > Key: CASSANDRA-13696 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13696 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Jay Zhuang >Assignee: Jay Zhuang >Priority: Critical > > {noformat} > WARN [HintsDispatcher:2] 2017-07-16 22:00:32,579 HintsReader.java:235 - > Failed to read a hint for /127.0.0.2: a2b7daf1-a6a4-4dfc-89de-32d12d2d48b0 - > table with id 3882bbb0-6a71-11e7-9bca-2759083e3964 is unknown in file > a2b7daf1-a6a4-4dfc-89de-32d12d2d48b0-1500242103097-1.hints > ERROR [HintsDispatcher:2] 2017-07-16 22:00:32,580 > HintsDispatchExecutor.java:234 - Failed to dispatch hints file > a2b7daf1-a6a4-4dfc-89de-32d12d2d48b0-1500242103097-1.hints: file is corrupted > ({}) > org.apache.cassandra.io.FSReadError: java.io.IOException: Digest mismatch > exception > at > org.apache.cassandra.hints.HintsReader$HintsIterator.computeNext(HintsReader.java:199) > ~[main/:na] > at > org.apache.cassandra.hints.HintsReader$HintsIterator.computeNext(HintsReader.java:164) > ~[main/:na] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[main/:na] > at > org.apache.cassandra.hints.HintsDispatcher.sendHints(HintsDispatcher.java:157) > ~[main/:na] > at > org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:139) > ~[main/:na] > at > org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:123) > ~[main/:na] > at > org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:95) > ~[main/:na] > at > org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:268) > [main/:na] > at > org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:251) > [main/:na] > at > org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:229) > [main/:na] > at > org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:208) > [main/:na] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_111] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_111] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_111] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_111] > at > org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79) > [main/:na] > at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111] > Caused by: java.io.IOException: Digest mismatch exception > at > org.apache.cassa
[jira] [Comment Edited] (CASSANDRA-13696) Digest mismatch Exception if hints file has UnknownColumnFamily
[ https://issues.apache.org/jira/browse/CASSANDRA-13696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092658#comment-16092658 ] Jay Zhuang edited comment on CASSANDRA-13696 at 7/19/17 6:32 AM: - Thanks [~jjirsa]. I did more investigation today. Seems it's more serious than I thought. Even there's no down node, "drop table" + write traffic, will trigger the problem. Here is reproduce steps: 1. Create a 3 nodes cluster: {{$ ccm create test13696 -v 3.0.14 && ccm populate -n 3 && ccm start}} 2. Send some traffics with cassandra-stress (blogpost.yaml is only in trunk, if you use another yaml file, change the RF=3) {{$ tools/bin/cassandra-stress user profile=test/resources/blogpost.yaml cl=QUORUM truncate=never ops\(insert=1\) duration=30m -rate threads=2 -mode native cql3 -node 127.0.0.1}} 3. While the traffic is running, drop table {{$ cqlsh -e "drop table stresscql.blogposts"}} *All 3 nodes go down because of "Digest mismatch Exception".* The CRC calculation problem has been there for a long time, but only got exposed after CASSANDRA-13004 because of the MessagingService version bump. In the normal case when the versions are the same, HintsDispatcher uses {{[page.buffersIterator()|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsDispatcher.java#L138]}} instead of {{[page.hintsIterator()|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsDispatcher.java#L139]}}. {{buffersIterator()}} doesn't need to decode hints, so it won't have the problem. I think the messagingVersion for the hints file should be updated: https://github.com/apache/cassandra/compare/cassandra-3.0...cooldoger:13696.2-3.0?expand=1 so it could dispatch hints in an optimized way. Not sure if we need to check/bump other {{MessagingService.VERSION_30}}s in the 3.0 branch. cc [~ifesdjeen] was (Author: jay.zhuang): Thanks [~jjirsa]. I did more investigation today. Seems it's more serious than I thought. Even there's no down node, "drop table" while there's write traffic, it will trigger the problem. Here is reproduce steps: 1. Create a 3 nodes cluster: {{$ ccm create test13696 -v 3.0.14 && ccm populate -n 3 && ccm start}} 2. Send some traffics with cassandra-stress (blogpost.yaml is only in trunk, if you use another yaml file, change the RF=3) {{$ tools/bin/cassandra-stress user profile=test/resources/blogpost.yaml cl=QUORUM truncate=never ops\(insert=1\) duration=30m -rate threads=2 -mode native cql3 -node 127.0.0.1}} 3. While the traffic is running, drop table {{$ cqlsh -e "drop table stresscql.blogposts"}} *All 3 nodes go down because of "Digest mismatch Exception".* The CRC calculation problem has been there for a long time, but only got exposed after CASSANDRA-13004 because of the MessagingService version bump. In the normal case when the versions are the same, HintsDispatcher uses {{[page.buffersIterator()|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsDispatcher.java#L138]}} instead of {{[page.hintsIterator()|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsDispatcher.java#L139]}}. {{buffersIterator()}} doesn't need to decode hints, so it won't have the problem. I think the messagingVersion for the hints file should be updated: https://github.com/apache/cassandra/compare/cassandra-3.0...cooldoger:13696.2-3.0?expand=1 so it could dispatch hints in an optimized way. Not sure if we need to check/bump other {{MessagingService.VERSION_30}}s in the 3.0 branch. cc [~ifesdjeen] > Digest mismatch Exception if hints file has UnknownColumnFamily > --- > > Key: CASSANDRA-13696 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13696 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Jay Zhuang >Assignee: Jay Zhuang >Priority: Critical > > {noformat} > WARN [HintsDispatcher:2] 2017-07-16 22:00:32,579 HintsReader.java:235 - > Failed to read a hint for /127.0.0.2: a2b7daf1-a6a4-4dfc-89de-32d12d2d48b0 - > table with id 3882bbb0-6a71-11e7-9bca-2759083e3964 is unknown in file > a2b7daf1-a6a4-4dfc-89de-32d12d2d48b0-1500242103097-1.hints > ERROR [HintsDispatcher:2] 2017-07-16 22:00:32,580 > HintsDispatchExecutor.java:234 - Failed to dispatch hints file > a2b7daf1-a6a4-4dfc-89de-32d12d2d48b0-1500242103097-1.hints: file is corrupted > ({}) > org.apache.cassandra.io.FSReadError: java.io.IOException: Digest mismatch > exception > at > org.apache.cassandra.hints.HintsReader$HintsIterator.computeNext(HintsReader.java:199) > ~[main/:na] > at > org.apache.cassandra.hints.HintsReader$HintsIterator.computeNext(HintsReader.java:1