[jira] [Updated] (CASSANDRA-13477) Support running sstableloader outside of cluster with non default native_transport_port
[
https://issues.apache.org/jira/browse/CASSANDRA-13477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhiyan Shao updated CASSANDRA-13477:
Description:
Currently, hadoop cql3 library does not support setting OutputNativePort. It
always uses the default port 9042. If Cassandra cluster uses a different
native_transport_port, this won't work. There is a setInputNativePort but no
setOutputNativePort.
This ticket is to add this support.
During testing, we found out that storage_port and ssl_storage_port also need
to be set to customized one. Here is our current workaround:
hadoopConfig.set("cassandra.output.native.port", nonDefaultNativePort)
System.setProperty("cassandra.storage_port", nonStoragePort)
System.setProperty("cassandra.ssl_storage_port", nonSSLStoragePort)
was:
Currently, hadoop cql3 library does not support setting OutputNativePort. It
always uses the default port 9042. If Cassandra cluster uses a different
native_transport_port, this won't work. There is a setInputNativePort but no
setOutputNativePort.
This ticket is to add this support.
> Support running sstableloader outside of cluster with non default
> native_transport_port
> ---
>
> Key: CASSANDRA-13477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13477
> Project: Cassandra
> Issue Type: Improvement
> Components: Configuration, Libraries, Tools
>Reporter: Zhiyan Shao
>Assignee: Zhiyan Shao
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.x
>
> Attachments: 13477-3.0.txt
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Currently, hadoop cql3 library does not support setting OutputNativePort. It
> always uses the default port 9042. If Cassandra cluster uses a different
> native_transport_port, this won't work. There is a setInputNativePort but no
> setOutputNativePort.
> This ticket is to add this support.
> During testing, we found out that storage_port and ssl_storage_port also need
> to be set to customized one. Here is our current workaround:
> hadoopConfig.set("cassandra.output.native.port", nonDefaultNativePort)
> System.setProperty("cassandra.storage_port", nonStoragePort)
> System.setProperty("cassandra.ssl_storage_port", nonSSLStoragePort)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13477) Support running sstableloader outside of cluster with non default native_transport_port
[ https://issues.apache.org/jira/browse/CASSANDRA-13477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhiyan Shao updated CASSANDRA-13477: Summary: Support running sstableloader outside of cluster with non default native_transport_port (was: Support running sstableloader outside of cluster with non default native_transport_port, storage_port and ssl_storage_port) > Support running sstableloader outside of cluster with non default > native_transport_port > --- > > Key: CASSANDRA-13477 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13477 > Project: Cassandra > Issue Type: Improvement > Components: Configuration, Libraries, Tools >Reporter: Zhiyan Shao >Assignee: Zhiyan Shao >Priority: Minor > Fix For: 3.0.x, 3.11.x, 4.x > > Attachments: 13477-3.0.txt > > Original Estimate: 1h > Remaining Estimate: 1h > > Currently, hadoop cql3 library does not support setting OutputNativePort. It > always uses the default port 9042. If Cassandra cluster uses a different > native_transport_port, this won't work. There is a setInputNativePort but no > setOutputNativePort. > This ticket is to add this support. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13477) Support running sstableloader outside of cluster with non default native_transport_port, storage_port and ssl_storage_port
[ https://issues.apache.org/jira/browse/CASSANDRA-13477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhiyan Shao updated CASSANDRA-13477: Summary: Support running sstableloader outside of cluster with non default native_transport_port, storage_port and ssl_storage_port (was: Add setOutputNativePort function in org.apache.cassandra.hadoop.cql3.CqlConfigHelper.java) > Support running sstableloader outside of cluster with non default > native_transport_port, storage_port and ssl_storage_port > -- > > Key: CASSANDRA-13477 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13477 > Project: Cassandra > Issue Type: Improvement > Components: Configuration, Libraries, Tools >Reporter: Zhiyan Shao >Assignee: Zhiyan Shao >Priority: Minor > Fix For: 3.0.x, 3.11.x, 4.x > > Attachments: 13477-3.0.txt > > Original Estimate: 1h > Remaining Estimate: 1h > > Currently, hadoop cql3 library does not support setting OutputNativePort. It > always uses the default port 9042. If Cassandra cluster uses a different > native_transport_port, this won't work. There is a setInputNativePort but no > setOutputNativePort. > This ticket is to add this support. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13480) nodetool repair can hang forever if we lose the notification for the repair completing/failing
[ https://issues.apache.org/jira/browse/CASSANDRA-13480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15989556#comment-15989556 ] Matt Byrd commented on CASSANDRA-13480: --- So the patch I have currently also caches the notifications for repairs for a limited time on the co-ordinator, it was initially targeting a release where we didn't yet have the repair history tables. I suppose there is a concern that caching these notifications could under some circumstances cause unwanted extra heap usage. (Similarly to the notifications buffer, although at least here we're only caching a subset that we care more about) So using the repair history tables instead and exposing this information by imx seems like a reasonable alternative. There are perhaps a couple of kinks to work out, but I'll have a go at adapting the patch that I have to work in this way. For one we only have the cmd id int sent back to the nodetool process (rather than the parent session id which the internal table is partition keyed off) We could either keep track of the cmd id int -> parent session uuid in the co-ordinator, either in memory cached to expire or in another internal table, or we could parse the uuid out of the notification sent for the start of the parent repair. Parsing the message is a bit brittle though and not full proof in theory (we could miss that notification also). Ideally I suppose running a repair could return and communicate on the basis of the parent session uuid rather than the int cmd id, but this is a pretty major overhaul and has all sorts of compatibility questions. > nodetool repair can hang forever if we lose the notification for the repair > completing/failing > -- > > Key: CASSANDRA-13480 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13480 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Matt Byrd >Assignee: Matt Byrd >Priority: Minor > Fix For: 4.x > > > When a Jmx lost notification occurs, sometimes the lost notification in > question is the notification which let's RepairRunner know that the repair is > finished (ProgressEventType.COMPLETE or even ERROR for that matter). > This results in nodetool process running the repair hanging forever. > I have a test which reproduces the issue here: > https://github.com/Jollyplum/cassandra-dtest/tree/repair_hang_test > To fix this, If on receiving a notification that notifications have been lost > (JMXConnectionNotification.NOTIFS_LOST), we instead query a new endpoint via > Jmx to receive all the relevant notifications we're interested in, we can > replay those we missed and avoid this scenario. > It's possible also that the JMXConnectionNotification.NOTIFS_LOST itself > might be lost and so for good measure I have made RepairRunner poll > periodically to see if there were any notifications that had been sent but we > didn't receive (scoped just to the particular tag for the given repair). > Users who don't use nodetool but go via jmx directly, can still use this new > endpoint and implement similar behaviour in their clients as desired. > I'm also expiring the notifications which have been kept on the server side. > Please let me know if you've any questions or can think of a different > approach, I also tried setting: > JVM_OPTS="$JVM_OPTS -Djmx.remote.x.notification.buffer.size=5000" > but this didn't fix the test. I suppose it might help under certain scenarios > but in this test we don't even send that many notifications so I'm not > surprised it doesn't fix it. > It seems like getting lost notifications is always a potential problem with > jmx as far as I can tell. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13265) Expiration in OutboundTcpConnection can block the reader Thread
[
https://issues.apache.org/jira/browse/CASSANDRA-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-13265:
---
Attachment: cassandra-13265-trun-dtest_stdout.txt
cassandra-13265-2.2-dtest_stdout.txt
trunk
{noformat}
Test Result (12 failures / +2)
auth_test.TestAuth.system_auth_ks_is_alterable_test
bootstrap_test.TestBootstrap.decommissioned_wiped_node_can_join_test
paging_test.TestPagingWithDeletions.test_ttl_deletions
repair_tests.incremental_repair_test.TestIncRepair.sstable_repairedset_test
repair_tests.incremental_repair_test.TestIncRepair.sstable_repairedset_test
snapshot_test.TestSnapshot.test_snapshot_and_restore_dropping_a_column
repair_tests.incremental_repair_test.TestIncRepair.multiple_full_repairs_lcs_test
repair_tests.incremental_repair_test.TestIncRepair.multiple_full_repairs_lcs_test
sstableutil_test.SSTableUtilTest.compaction_test
topology_test.TestTopology.size_estimates_multidc_test
topology_test.TestTopology.size_estimates_multidc_test
bootstrap_test.TestBootstrap.simultaneous_bootstrap_test
{noformat}
2.2
{noformat}
Test Result (4 failures / -8)
batch_test.TestBatch.logged_batch_throws_uae_test
snapshot_test.TestSnapshot.test_snapshot_and_restore_dropping_a_column
topology_test.TestTopology.size_estimates_multidc_test
bootstrap_test.TestBootstrap.simultaneous_bootstrap_test
{noformat}
> Expiration in OutboundTcpConnection can block the reader Thread
> ---
>
> Key: CASSANDRA-13265
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13265
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra 3.0.9
> Java HotSpot(TM) 64-Bit Server VM version 25.112-b15 (Java version
> 1.8.0_112-b15)
> Linux 3.16
>Reporter: Christian Esken
>Assignee: Christian Esken
> Fix For: 3.0.x
>
> Attachments: cassandra-13265-2.2-dtest_stdout.txt,
> cassandra-13265-trun-dtest_stdout.txt,
> cassandra.pb-cache4-dus.2017-02-17-19-36-26.chist.xz,
> cassandra.pb-cache4-dus.2017-02-17-19-36-26.td.xz
>
>
> I observed that sometimes a single node in a Cassandra cluster fails to
> communicate to the other nodes. This can happen at any time, during peak load
> or low load. Restarting that single node from the cluster fixes the issue.
> Before going in to details, I want to state that I have analyzed the
> situation and am already developing a possible fix. Here is the analysis so
> far:
> - A Threaddump in this situation showed 324 Threads in the
> OutboundTcpConnection class that want to lock the backlog queue for doing
> expiration.
> - A class histogram shows 262508 instances of
> OutboundTcpConnection$QueuedMessage.
> What is the effect of it? As soon as the Cassandra node has reached a certain
> amount of queued messages, it starts thrashing itself to death. Each of the
> Thread fully locks the Queue for reading and writing by calling
> iterator.next(), making the situation worse and worse.
> - Writing: Only after 262508 locking operation it can progress with actually
> writing to the Queue.
> - Reading: Is also blocked, as 324 Threads try to do iterator.next(), and
> fully lock the Queue
> This means: Writing blocks the Queue for reading, and readers might even be
> starved which makes the situation even worse.
> -
> The setup is:
> - 3-node cluster
> - replication factor 2
> - Consistency LOCAL_ONE
> - No remote DC's
> - high write throughput (10 INSERT statements per second and more during
> peak times).
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-8780) cassandra-stress should support multiple table operations
[ https://issues.apache.org/jira/browse/CASSANDRA-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15989473#comment-15989473 ] Ben Slater commented on CASSANDRA-8780: --- Yep, it should be backwards compatible. I'll take a look. Thanks for you help. > cassandra-stress should support multiple table operations > - > > Key: CASSANDRA-8780 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8780 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Benedict >Assignee: Ben Slater > Labels: stress > Fix For: 3.11.x > > Attachments: 8780-trunkv2.txt > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-8780) cassandra-stress should support multiple table operations
[ https://issues.apache.org/jira/browse/CASSANDRA-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-8780: -- Status: Awaiting Feedback (was: Open) > cassandra-stress should support multiple table operations > - > > Key: CASSANDRA-8780 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8780 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Benedict >Assignee: Ben Slater > Labels: stress > Fix For: 3.11.x > > Attachments: 8780-trunkv2.txt > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-8780) cassandra-stress should support multiple table operations
[ https://issues.apache.org/jira/browse/CASSANDRA-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15989258#comment-15989258 ] T Jake Luciani commented on CASSANDRA-8780: --- The dtests seem to have issues running stress, so it looks like this is not backwards compatible. We should support both old/new ways. Can you take a look [~slater_ben]? > cassandra-stress should support multiple table operations > - > > Key: CASSANDRA-8780 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8780 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Benedict >Assignee: Ben Slater > Labels: stress > Fix For: 3.11.x > > Attachments: 8780-trunkv2.txt > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-8780) cassandra-stress should support multiple table operations
[ https://issues.apache.org/jira/browse/CASSANDRA-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-8780: -- Status: Open (was: Patch Available) > cassandra-stress should support multiple table operations > - > > Key: CASSANDRA-8780 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8780 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Benedict >Assignee: Ben Slater > Labels: stress > Fix For: 3.11.x > > Attachments: 8780-trunkv2.txt > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10130) Node failure during 2i update after streaming can have incomplete 2i when restarted
[ https://issues.apache.org/jira/browse/CASSANDRA-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15989138#comment-15989138 ] Andrés de la Peña commented on CASSANDRA-10130: --- The most straightforward solution would be to use the write path, as it is done with materialized views and CDC. As we want to avoid this, we could use [a new table|https://github.com/adelapena/cassandra/blob/10130-trunk/src/java/org/apache/cassandra/db/SystemKeyspace.java#L147] in the system keyspace to keep track of indexes that should be rebuilt before start. So, we could mark the indexes to be rebuilt [right before|https://github.com/adelapena/cassandra/blob/10130-trunk/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java#L231] adding the sstables to the column family and rebuilding the index. If everything is ok, we [unmark|https://github.com/adelapena/cassandra/blob/10130-trunk/src/java/org/apache/cassandra/streaming/StreamReceiveTask.java#L238] the indexes for rebuilding. Otherwise, if the node dies, the next node start would [detect|https://github.com/adelapena/cassandra/blob/10130-trunk/src/java/org/apache/cassandra/index/internal/CassandraIndex.java#L173] the indexes as marked for rebuilding and would throw a full index rebuild. [Here|https://github.com/adelapena/cassandra/commit/cd5c3faab5d59cc75b98a8151f6245034c634e53] is the draft of the approach. > Node failure during 2i update after streaming can have incomplete 2i when > restarted > --- > > Key: CASSANDRA-10130 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10130 > Project: Cassandra > Issue Type: Bug > Components: Coordination >Reporter: Yuki Morishita >Assignee: Andrés de la Peña >Priority: Minor > > Since MV/2i update happens after SSTables are received, node failure during > MV/2i update can leave received SSTables live when restarted while MV/2i are > partially up to date. > We can add some kind of tracking mechanism to automatically rebuild at the > startup, or at least warn user when the node restarts. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-4650) RangeStreamer should be smarter when picking endpoints for streaming in case of N >=3 in each DC.
[ https://issues.apache.org/jira/browse/CASSANDRA-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15989100#comment-15989100 ] sankalp kohli commented on CASSANDRA-4650: -- +1 > RangeStreamer should be smarter when picking endpoints for streaming in case > of N >=3 in each DC. > --- > > Key: CASSANDRA-4650 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4650 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.1.5 >Reporter: sankalp kohli >Assignee: sankalp kohli >Priority: Minor > Labels: streaming > Fix For: 4.x > > Attachments: CASSANDRA-4650_trunk.txt, photo-1.JPG > > Original Estimate: 24h > Remaining Estimate: 24h > > getRangeFetchMap method in RangeStreamer should pick unique nodes to stream > data from when number of replicas in each DC is three or more. > When N>=3 in a DC, there are two options for streaming a range. Consider an > example of 4 nodes in one datacenter and replication factor of 3. > If a node goes down, it needs to recover 3 ranges of data. With current code, > two nodes could get selected as it orders the node by proximity. > We ideally will want to select 3 nodes for streaming the data. We can do this > by selecting unique nodes for each range. > Advantages: > This will increase the performance of bootstrapping a node and will also put > less pressure on nodes serving the data. > Note: This does not affect if N < 3 in each DC as then it streams data from > only 2 nodes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-8780) cassandra-stress should support multiple table operations
[ https://issues.apache.org/jira/browse/CASSANDRA-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985424#comment-15985424 ] T Jake Luciani edited comment on CASSANDRA-8780 at 4/28/17 2:24 PM: Kicked off CI on these [testall|http://cassci.datastax.com/job/tjake-8780-trunk-testall/] [dtest|http://cassci.datastax.com/job/tjake-8780-trunk-dtest/] was (Author: tjake): Kicked off CI on these [testall|https://circleci.com/gh/tjake/cassandra/2] [dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/33/] > cassandra-stress should support multiple table operations > - > > Key: CASSANDRA-8780 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8780 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Benedict >Assignee: Ben Slater > Labels: stress > Fix For: 3.11.x > > Attachments: 8780-trunkv2.txt > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13346) Failed unregistering mbean during drop keyspace
[
https://issues.apache.org/jira/browse/CASSANDRA-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988859#comment-15988859
]
Chris Lohfink commented on CASSANDRA-13346:
---
the "all" map is actually for metrics that have a global representation thats
aggregated. For example, theres a global sstables per read metric that
aggregates all the individual table and keyspace metrics so you can see the
entire nodes sstables per read to see if any are high instead of walking
through entire table set to find out.
> Failed unregistering mbean during drop keyspace
> ---
>
> Key: CASSANDRA-13346
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13346
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
> Environment: Cassandra 3.9
>Reporter: Gábor Auth
>Assignee: Lerh Chuan Low
>Priority: Minor
> Labels: lhf
> Fix For: 3.0.x, 3.11.x
>
> Attachments: 13346-3.0.X.txt, 13346-3.X.txt
>
>
> All node throw exceptions about materialized views during drop keyspace:
> {code}
> WARN [MigrationStage:1] 2017-03-16 16:54:25,016 ColumnFamilyStore.java:535 -
> Failed unregistering mbean:
> org.apache.cassandra.db:type=Tables,keyspace=test20160810,table=unit_by_account
> java.lang.NullPointerException: null
> at
> java.util.concurrent.ConcurrentHashMap.replaceNode(ConcurrentHashMap.java:1106)
> ~[na:1.8.0_121]
> at
> java.util.concurrent.ConcurrentHashMap.remove(ConcurrentHashMap.java:1097)
> ~[na:1.8.0_121]
> at
> java.util.concurrent.ConcurrentHashMap$KeySetView.remove(ConcurrentHashMap.java:4569)
> ~[na:1.8.0_121]
> at
> org.apache.cassandra.metrics.TableMetrics.release(TableMetrics.java:712)
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at
> org.apache.cassandra.db.ColumnFamilyStore.unregisterMBean(ColumnFamilyStore.java:570)
> [apache-cassandra-3.9.0.jar:3.9.0]
> at
> org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:527)
> [apache-cassandra-3.9.0.jar:3.9.0]
> at
> org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:517)
> [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.db.Keyspace.unloadCf(Keyspace.java:365)
> [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.db.Keyspace.dropCf(Keyspace.java:358)
> [apache-cassandra-3.9.0.jar:3.9.0]
> at org.apache.cassandra.config.Schema.dropView(Schema.java:744)
> [apache-cassandra-3.9.0.jar:3.9.0]
> at
> org.apache.cassandra.schema.SchemaKeyspace.lambda$mergeSchema$373(SchemaKeyspace.java:1287)
> [apache-cassandra-3.9.0.jar:3.9.0]
> at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_121]
> at
> org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1287)
> [apache-cassandra-3.9.0.jar:3.9.0]
> at
> org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1256)
> [apache-cassandra-3.9.0.jar:3.9.0]
> at
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:51)
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_121]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13418) Allow TWCS to ignore overlaps when dropping fully expired sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988846#comment-15988846
]
Jeff Jirsa commented on CASSANDRA-13418:
I think the path forward is this:
Let's create a new compaction option, that ISNT in any of the autocomplete
logic, and has a scary name {{unsafe_aggressive_sstable_expiration}} . We could
also add a system property {{-Dcassandra.unsafe.aggressive_sstable_expiration}}
to make sure nobody ever enables this by mistake and gets unsafe behavior. The
same patch should add a section to the documentation explaining exactly why
this option is unsafe.
Finally, The approach in the patch on this ticket is different from the
approach on the fork [~intjonathan] linked - the reviewer (I can do it, or
perhaps [~krummas] ) should definitely evaluate which approach is going to be
better.
> Allow TWCS to ignore overlaps when dropping fully expired sstables
> --
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
> Issue Type: Improvement
> Components: Compaction
>Reporter: Corentin Chary
> Labels: twcs
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If
> you really want read-repairs you're going to have sstables blocking the
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a
> very low value and that will purge the blockers of old data that should
> already have expired, thus removing the overlaps and allowing the other
> SSTables to expire.
> The thing is that this is rather CPU intensive and not optimal. If you have
> time series, you might not care if all your data doesn't exactly expire at
> the right time, or if data re-appears for some time, as long as it gets
> deleted as soon as it can. And in this situation I believe it would be really
> beneficial to allow users to simply ignore overlapping SSTables when looking
> for fully expired ones.
> To the question: why would you need read-repairs ?
> - Full repairs basically take longer than the TTL of the data on my dataset,
> so this isn't really effective.
> - Even with a 10% chances of doing a repair, we found out that this would be
> enough to greatly reduce entropy of the most used data (and if you have
> timeseries, you're likely to have a dashboard doing the same important
> queries over and over again).
> - LOCAL_QUORUM is too expensive (need >3 replicas), QUORUM is too slow.
> I'll try to come up with a patch demonstrating how this would work, try it on
> our system and report the effects.
> cc: [~adejanovski], [~rgerard] as I know you worked on similar issues already.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13418) Allow TWCS to ignore overlaps when dropping fully expired sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988836#comment-15988836 ] Nate McCall commented on CASSANDRA-13418: - I would like to see an append-only optimization as [~iksaif] and others have described: off by default and scary warnings in the docs. This is a common enough use case (particularly in that folks like [~intjonathan] and crew having a patched version deployed already) that we should provide an optimization for it. > Allow TWCS to ignore overlaps when dropping fully expired sstables > -- > > Key: CASSANDRA-13418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13418 > Project: Cassandra > Issue Type: Improvement > Components: Compaction >Reporter: Corentin Chary > Labels: twcs > > http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If > you really want read-repairs you're going to have sstables blocking the > expiration of other fully expired SSTables because they overlap. > You can set unchecked_tombstone_compaction = true or tombstone_threshold to a > very low value and that will purge the blockers of old data that should > already have expired, thus removing the overlaps and allowing the other > SSTables to expire. > The thing is that this is rather CPU intensive and not optimal. If you have > time series, you might not care if all your data doesn't exactly expire at > the right time, or if data re-appears for some time, as long as it gets > deleted as soon as it can. And in this situation I believe it would be really > beneficial to allow users to simply ignore overlapping SSTables when looking > for fully expired ones. > To the question: why would you need read-repairs ? > - Full repairs basically take longer than the TTL of the data on my dataset, > so this isn't really effective. > - Even with a 10% chances of doing a repair, we found out that this would be > enough to greatly reduce entropy of the most used data (and if you have > timeseries, you're likely to have a dashboard doing the same important > queries over and over again). > - LOCAL_QUORUM is too expensive (need >3 replicas), QUORUM is too slow. > I'll try to come up with a patch demonstrating how this would work, try it on > our system and report the effects. > cc: [~adejanovski], [~rgerard] as I know you worked on similar issues already. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-13483) test failure in snapshot_test.TestSnapshot.test_snapshot_and_restore_dropping_a_column
Michael Shuler created CASSANDRA-13483:
--
Summary: test failure in
snapshot_test.TestSnapshot.test_snapshot_and_restore_dropping_a_column
Key: CASSANDRA-13483
URL: https://issues.apache.org/jira/browse/CASSANDRA-13483
Project: Cassandra
Issue Type: Bug
Reporter: Michael Shuler
Attachments: node1_debug.log, node1_gc.log, node1.log
example failure:
http://cassci.datastax.com/job/cassandra-2.2_offheap_dtest/488/testReport/snapshot_test/TestSnapshot/test_snapshot_and_restore_dropping_a_column
{code}
Error Message
Subprocess ['nodetool', '-h', 'localhost', '-p', '7100', ['refresh', 'ks',
'cf']] exited with non-zero status; exit status: 1;
stdout: nodetool: Unknown keyspace/cf pair (ks.cf)
See 'nodetool help' or 'nodetool help '.
{code}
{code}
Stacktrace
File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
File "/home/automaton/cassandra-dtest/snapshot_test.py", line 145, in
test_snapshot_and_restore_dropping_a_column
node1.nodetool('refresh ks cf')
File "/home/automaton/venv/local/lib/python2.7/site-packages/ccmlib/node.py",
line 789, in nodetool
return handle_external_tool_process(p, ['nodetool', '-h', 'localhost',
'-p', str(self.jmx_port), cmd.split()])
File "/home/automaton/venv/local/lib/python2.7/site-packages/ccmlib/node.py",
line 2002, in handle_external_tool_process
raise ToolError(cmd_args, rc, out, err)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-13478) SASIndex has a time to live issue in Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-13478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov reassigned CASSANDRA-13478: --- Assignee: Alex Petrov > SASIndex has a time to live issue in Cassandra > -- > > Key: CASSANDRA-13478 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13478 > Project: Cassandra > Issue Type: Bug > Components: sasi > Environment: cqlsh 5.0.1 | Cassandra 3.10 | CQL spec 3.4.4 | Native > protocol v4 | ubuntu 14.04 >Reporter: jack chen >Assignee: Alex Petrov >Priority: Minor > Attachments: schema > > > I have a table, the schema can be seen in attach file > I would like to search the data using the timestamp data type with lt gt eq > as a query condition, > Ex: > Select * from userlist where lastposttime> '2017-04-01 16:00:00+'; > There are 2 conditions : > If I insert the data and then select it, the result will be correct > But in case I insert data and then the next day I restart Cassandra, and > after that select the data, there will be no data selected > The difference is that there is no Service restart on th next day in the > first manner. Actually, the data are still living in Cassandra, but TimeStamp > can’t be used as the query condition -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-4650) RangeStreamer should be smarter when picking endpoints for streaming in case of N >=3 in each DC.
[ https://issues.apache.org/jira/browse/CASSANDRA-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-4650: -- Reviewer: Marcus Eriksson (was: T Jake Luciani) > RangeStreamer should be smarter when picking endpoints for streaming in case > of N >=3 in each DC. > --- > > Key: CASSANDRA-4650 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4650 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.1.5 >Reporter: sankalp kohli >Assignee: sankalp kohli >Priority: Minor > Labels: streaming > Fix For: 4.x > > Attachments: CASSANDRA-4650_trunk.txt, photo-1.JPG > > Original Estimate: 24h > Remaining Estimate: 24h > > getRangeFetchMap method in RangeStreamer should pick unique nodes to stream > data from when number of replicas in each DC is three or more. > When N>=3 in a DC, there are two options for streaming a range. Consider an > example of 4 nodes in one datacenter and replication factor of 3. > If a node goes down, it needs to recover 3 ranges of data. With current code, > two nodes could get selected as it orders the node by proximity. > We ideally will want to select 3 nodes for streaming the data. We can do this > by selecting unique nodes for each range. > Advantages: > This will increase the performance of bootstrapping a node and will also put > less pressure on nodes serving the data. > Note: This does not affect if N < 3 in each DC as then it streams data from > only 2 nodes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13471) [PerDiskMemtableFlushWriter_0:1312] 2017-04-25 09:48:14,818 CassandraDaemon.java:226 - Exception in thread Thread[PerDiskMemtableFlushWriter_0:1312,5,main]
[
https://issues.apache.org/jira/browse/CASSANDRA-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988762#comment-15988762
]
Alex Petrov commented on CASSANDRA-13471:
-
>From the quick glance this doesn't look like a SASI issue, as the exception is
>happening on the flush, just on SASI thread.
> [PerDiskMemtableFlushWriter_0:1312] 2017-04-25 09:48:14,818
> CassandraDaemon.java:226 - Exception in thread
> Thread[PerDiskMemtableFlushWriter_0:1312,5,main]
> ---
>
> Key: CASSANDRA-13471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13471
> Project: Cassandra
> Issue Type: Bug
> Components: sasi
> Environment: Centos7.3 x86_64, cassandra 3.9
>Reporter: Chernishev Aleksandr
> Fix For: 3.9
>
>
> with sasi index in table test.object:
> {code}
> CREATE TABLE test.object (
> bname text,
> name text,
> acl text,
> checksum text,
> chunksize bigint,
> contenttype text,
> creationdate timestamp,
> inode uuid,
> lastmodified timestamp,
> metadata map,
> parts map,
> size bigint,
> storageclass text,
> version uuid,
> PRIMARY KEY (bname, name)
> ) WITH CLUSTERING ORDER BY (name ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '128', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 0.5
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> CREATE INDEX inode_index ON test.object (inode);
> {code}
> im get error(on random servers in cluster in random times) :
> {code}
> ERROR [PerDiskMemtableFlushWriter_0:1312] 2017-04-25 09:48:14,818
> CassandraDaemon.java:226 - Exception in thread
> Thread[PerDiskMemtableFlushWriter_0:1312,5,main]
> java.lang.RuntimeException: Last written key
> DecoratedKey(d3f60675-e56e-4551-b468-d4e31c8ee82b,
> d3f60675e56e4551b468d4e31c8ee82b) >= current key
> DecoratedKey(6a473364-2f43-3876-574d-693461546d73,
> d3f6355a5bd8415290668f987a15594c) writing into
> /data3/test/object-f40120c028d111e78e26c57aefc93bac/.inode_index/mc-26-big-Data.db
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.beforeAppend(BigTableWriter.java:122)
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:161)
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:48)
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:458)
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:493)
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:380)
> ~[apache-cassandra-3.9.0.jar:3.9.0]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> {code}
> After that in debug log repeated messages:
> {code}
> DEBUG [MemtablePostFlush:405] 2017-04-25 09:48:15,944
> ColumnFamilyStore.java:936 - forceFlush requested but everything is clean in
> object
> {code}
> Commitlog not truncate and grows.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13468) Improve incremental repair logging
[ https://issues.apache.org/jira/browse/CASSANDRA-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988706#comment-15988706 ] Marcus Eriksson commented on CASSANDRA-13468: - +1 > Improve incremental repair logging > -- > > Key: CASSANDRA-13468 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13468 > Project: Cassandra > Issue Type: Improvement >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Minor > > The logging for incremental repair should be improved a bit. In it's current > state, it's pretty conservative and doesn't really give much insight into > whats going on. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13468) Improve incremental repair logging
[ https://issues.apache.org/jira/browse/CASSANDRA-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-13468: Status: Ready to Commit (was: Patch Available) > Improve incremental repair logging > -- > > Key: CASSANDRA-13468 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13468 > Project: Cassandra > Issue Type: Improvement >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Minor > > The logging for incremental repair should be improved a bit. In it's current > state, it's pretty conservative and doesn't really give much insight into > whats going on. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-12386) Exception in thread Thread[MemtableFlushWriter:33,5,main]
[
https://issues.apache.org/jira/browse/CASSANDRA-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-12386:
Description:
In logs i got an exception which looks like :-
{code}
ERROR [MemtableFlushWriter:33] 2016-08-04 19:40:17,712 CassandraDaemon.java:201
- Exception in thread Thread[MemtableFlushWriter:33,5,main]
java.lang.RuntimeException: Last written key DecoratedKey(076710082,
303736373130303832) >= current key DecoratedKey(07671^@^@^@^@^@^@,
3037363731313232333534) writing into
/data/cassandra/data//-01ad9750723e11e4bfe0d3887930a87c/./mb-23389-big-Data.db
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.beforeAppend(BigTableWriter.java:106)
~[apache-cassandra-3.0.7.jar:3.0.7]
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:145)
~[apache-cassandra-3.0.7.jar:3.0.7]
at
org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
~[apache-cassandra-3.0.7.jar:3.0.7]
at
org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
~[apache-cassandra-3.0.7.jar:3.0.7]
at
org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:379)
~[apache-cassandra-3.0.7.jar:3.0.7]
at org.apache.cassandra.db.Memtable.flush(Memtable.java:326)
~[apache-cassandra-3.0.7.jar:3.0.7]
at
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1071)
~[apache-cassandra-3.0.7.jar:3.0.7]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[na:1.8.0_91]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_91]
{code}
It seems similar to https://issues.apache.org/jira/browse/CASSANDRA-9126 but
seems a bit different.
Please review. Thanks.
was:
In logs i got an exception which looks like :-
ERROR [MemtableFlushWriter:33] 2016-08-04 19:40:17,712 CassandraDaemon.java:201
- Exception in thread Thread[MemtableFlushWriter:33,5,main]
java.lang.RuntimeException: Last written key DecoratedKey(076710082,
303736373130303832) >= current key DecoratedKey(07671^@^@^@^@^@^@,
3037363731313232333534) writing into
/data/cassandra/data//-01ad9750723e11e4bfe0d3887930a87c/./mb-23389-big-Data.db
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.beforeAppend(BigTableWriter.java:106)
~[apache-cassandra-3.0.7.jar:3.0.7]
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:145)
~[apache-cassandra-3.0.7.jar:3.0.7]
at
org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
~[apache-cassandra-3.0.7.jar:3.0.7]
at
org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
~[apache-cassandra-3.0.7.jar:3.0.7]
at
org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:379)
~[apache-cassandra-3.0.7.jar:3.0.7]
at org.apache.cassandra.db.Memtable.flush(Memtable.java:326)
~[apache-cassandra-3.0.7.jar:3.0.7]
at
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1071)
~[apache-cassandra-3.0.7.jar:3.0.7]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[na:1.8.0_91]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_91]
It seems similar to https://issues.apache.org/jira/browse/CASSANDRA-9126 but
seems a bit different.
Please review. Thanks.
> Exception in thread Thread[MemtableFlushWriter:33,5,main]
> -
>
> Key: CASSANDRA-12386
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12386
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: CentOS 6.8 x86_64, Cassandra 3.0.7
>Reporter: vin01
>Priority: Minor
>
> In logs i got an exception which looks like :-
> {code}
> ERROR [MemtableFlushWriter:33] 2016-08-04 19:40:17,712
> CassandraDaemon.java:201 - Exception in thread
> Thread[MemtableFlushWriter:33,5,main]
> java.lang.RuntimeException: Last written key DecoratedKey(076710082,
> 303736373130303832) >= current key DecoratedKey(07671^@^@^@^@^@^@,
> 3037363731313232333534) writing into
> /data/cassandra/data//-01ad9750723e11e4bfe0d3887930a87c/./mb-23389-big-Data.db
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.beforeAppend(BigTableWriter.java:106)
> ~[apache-cassandra-3.0.7.jar:3.0.7]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:145)
> ~[apache-cassandra-3.0.7.jar:3.0.7]
>
[jira] [Updated] (CASSANDRA-13471) [PerDiskMemtableFlushWriter_0:1312] 2017-04-25 09:48:14,818 CassandraDaemon.java:226 - Exception in thread Thread[PerDiskMemtableFlushWriter_0:1312,5,main]
[
https://issues.apache.org/jira/browse/CASSANDRA-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-13471:
Description:
with sasi index in table test.object:
{code}
CREATE TABLE test.object (
bname text,
name text,
acl text,
checksum text,
chunksize bigint,
contenttype text,
creationdate timestamp,
inode uuid,
lastmodified timestamp,
metadata map,
parts map,
size bigint,
storageclass text,
version uuid,
PRIMARY KEY (bname, name)
) WITH CLUSTERING ORDER BY (name ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '128', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 0.5
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
CREATE INDEX inode_index ON test.object (inode);
{code}
im get error(on random servers in cluster in random times) :
{code}
ERROR [PerDiskMemtableFlushWriter_0:1312] 2017-04-25 09:48:14,818
CassandraDaemon.java:226 - Exception in thread
Thread[PerDiskMemtableFlushWriter_0:1312,5,main]
java.lang.RuntimeException: Last written key
DecoratedKey(d3f60675-e56e-4551-b468-d4e31c8ee82b,
d3f60675e56e4551b468d4e31c8ee82b) >= current key
DecoratedKey(6a473364-2f43-3876-574d-693461546d73,
d3f6355a5bd8415290668f987a15594c) writing into
/data3/test/object-f40120c028d111e78e26c57aefc93bac/.inode_index/mc-26-big-Data.db
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.beforeAppend(BigTableWriter.java:122)
~[apache-cassandra-3.9.0.jar:3.9.0]
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:161)
~[apache-cassandra-3.9.0.jar:3.9.0]
at
org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:48)
~[apache-cassandra-3.9.0.jar:3.9.0]
at
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:458)
~[apache-cassandra-3.9.0.jar:3.9.0]
at
org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:493)
~[apache-cassandra-3.9.0.jar:3.9.0]
at
org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:380)
~[apache-cassandra-3.9.0.jar:3.9.0]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_121]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_121]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_121]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
{code}
After that in debug log repeated messages:
{code}
DEBUG [MemtablePostFlush:405] 2017-04-25 09:48:15,944
ColumnFamilyStore.java:936 - forceFlush requested but everything is clean in
object
{code}
Commitlog not truncate and grows.
was:
with sasi index in table test.object:
{code}
CREATE TABLE test.object (
bname text,
name text,
acl text,
checksum text,
chunksize bigint,
contenttype text,
creationdate timestamp,
inode uuid,
lastmodified timestamp,
metadata map,
parts map,
size bigint,
storageclass text,
version uuid,
PRIMARY KEY (bname, name)
) WITH CLUSTERING ORDER BY (name ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '128', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 0.5
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
CREATE INDEX inode_index ON test.object (inode);
{code}
im get error(on random servers in cluster in random times) :
{code}
ERROR [PerDiskMemtableFlushWriter_0:1312] 2017-04-25 09:48:14,818
CassandraDaemon.java:226 - Exception in thread
Thread[PerDiskMemtableFlushWriter_0:1312,5,main]
java.lang.RuntimeException: Last written key
DecoratedKey(d3f60675-e56e-4551-b468-d4e31c8ee82b,
d3f60675e56e4551b468d4e31c8ee82b) >= current key
[jira] [Assigned] (CASSANDRA-13482) NPE on non-existing row read when row cache is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-13482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov reassigned CASSANDRA-13482:
---
Assignee: Alex Petrov
> NPE on non-existing row read when row cache is enabled
> --
>
> Key: CASSANDRA-13482
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13482
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>
> The problem is reproducible on 3.0 with:
> {code}
> -# row_cache_class_name: org.apache.cassandra.cache.OHCProvider
> +row_cache_class_name: org.apache.cassandra.cache.OHCProvider
> -row_cache_size_in_mb: 0
> +row_cache_size_in_mb: 100
> {code}
> Table setup:
> {code}
> CREATE TABLE cache_tables (pk int, v1 int, v2 int, v3 int, primary key (pk,
> v1)) WITH CACHING = { 'keys': 'ALL', 'rows_per_partition': '1' } ;
> {code}
> No data is required, only a head query (or any pk/ck query but with full
> partitions cached).
> {code}
> select * from cross_page_queries where pk = 1 ;
> {code}
> {code}
> java.lang.AssertionError: null
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterators.concat(UnfilteredRowIterators.java:193)
> ~[main/:na]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.getThroughCache(SinglePartitionReadCommand.java:461)
> ~[main/:na]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryStorage(SinglePartitionReadCommand.java:358)
> ~[main/:na]
> at
> org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:395)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1794)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2472)
> ~[main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_121]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
> ~[main/:na]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
> [main/:na]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-13482) NPE on non-existing row read when row cache is enabled
Alex Petrov created CASSANDRA-13482:
---
Summary: NPE on non-existing row read when row cache is enabled
Key: CASSANDRA-13482
URL: https://issues.apache.org/jira/browse/CASSANDRA-13482
Project: Cassandra
Issue Type: Bug
Reporter: Alex Petrov
The problem is reproducible on 3.0 with:
{code}
-# row_cache_class_name: org.apache.cassandra.cache.OHCProvider
+row_cache_class_name: org.apache.cassandra.cache.OHCProvider
-row_cache_size_in_mb: 0
+row_cache_size_in_mb: 100
{code}
Table setup:
{code}
CREATE TABLE cache_tables (pk int, v1 int, v2 int, v3 int, primary key (pk,
v1)) WITH CACHING = { 'keys': 'ALL', 'rows_per_partition': '1' } ;
{code}
No data is required, only a head query (or any pk/ck query but with full
partitions cached).
{code}
select * from cross_page_queries where pk = 1 ;
{code}
{code}
java.lang.AssertionError: null
at
org.apache.cassandra.db.rows.UnfilteredRowIterators.concat(UnfilteredRowIterators.java:193)
~[main/:na]
at
org.apache.cassandra.db.SinglePartitionReadCommand.getThroughCache(SinglePartitionReadCommand.java:461)
~[main/:na]
at
org.apache.cassandra.db.SinglePartitionReadCommand.queryStorage(SinglePartitionReadCommand.java:358)
~[main/:na]
at
org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:395)
~[main/:na]
at
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1794)
~[main/:na]
at
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2472)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_121]
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
~[main/:na]
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
[main/:na]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[main/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13418) Allow TWCS to ignore overlaps when dropping fully expired sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988490#comment-15988490 ] Corentin Chary commented on CASSANDRA-13418: I agree that fixing CASSANDRA-13418 would be a better solution, but this is likely to take more time, and I'm unsure we will get to the point where it really solve our issues in all the cases. I'll would be inclined to add the option too, with appropriate documentation. > Allow TWCS to ignore overlaps when dropping fully expired sstables > -- > > Key: CASSANDRA-13418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13418 > Project: Cassandra > Issue Type: Improvement > Components: Compaction >Reporter: Corentin Chary > Labels: twcs > > http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If > you really want read-repairs you're going to have sstables blocking the > expiration of other fully expired SSTables because they overlap. > You can set unchecked_tombstone_compaction = true or tombstone_threshold to a > very low value and that will purge the blockers of old data that should > already have expired, thus removing the overlaps and allowing the other > SSTables to expire. > The thing is that this is rather CPU intensive and not optimal. If you have > time series, you might not care if all your data doesn't exactly expire at > the right time, or if data re-appears for some time, as long as it gets > deleted as soon as it can. And in this situation I believe it would be really > beneficial to allow users to simply ignore overlapping SSTables when looking > for fully expired ones. > To the question: why would you need read-repairs ? > - Full repairs basically take longer than the TTL of the data on my dataset, > so this isn't really effective. > - Even with a 10% chances of doing a repair, we found out that this would be > enough to greatly reduce entropy of the most used data (and if you have > timeseries, you're likely to have a dashboard doing the same important > queries over and over again). > - LOCAL_QUORUM is too expensive (need >3 replicas), QUORUM is too slow. > I'll try to come up with a patch demonstrating how this would work, try it on > our system and report the effects. > cc: [~adejanovski], [~rgerard] as I know you worked on similar issues already. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10496) Make DTCS/TWCS split partitions based on time during compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-10496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988485#comment-15988485
]
Corentin Chary commented on CASSANDRA-10496:
Inspecting each timestamp on each cell is surely more correct, but in the first
version I'll be looking only at the minTimestamp of the partition (as long as
you have short living partitions).
With the current writer mechanism I didn't find a way to switch the writer in
the middle of a partition anyway..
> Make DTCS/TWCS split partitions based on time during compaction
> ---
>
> Key: CASSANDRA-10496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10496
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Marcus Eriksson
> Labels: dtcs
> Fix For: 3.11.x
>
>
> To avoid getting old data in new time windows with DTCS (or related, like
> [TWCS|CASSANDRA-9666]), we need to split out old data into its own sstable
> during compaction.
> My initial idea is to just create two sstables, when we create the compaction
> task we state the start and end times for the window, and any data older than
> the window will be put in its own sstable.
> By creating a single sstable with old data, we will incrementally get the
> windows correct - say we have an sstable with these timestamps:
> {{[100, 99, 98, 97, 75, 50, 10]}}
> and we are compacting in window {{[100, 80]}} - we would create two sstables:
> {{[100, 99, 98, 97]}}, {{[75, 50, 10]}}, and the first window is now
> 'correct'. The next compaction would compact in window {{[80, 60]}} and
> create sstables {{[75]}}, {{[50, 10]}} etc.
> We will probably also want to base the windows on the newest data in the
> sstables so that we actually have older data than the window.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10496) Make DTCS/TWCS split partitions based on time during compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-10496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988457#comment-15988457
]
Marcus Eriksson commented on CASSANDRA-10496:
-
you need to inspect the timestamp on each cell in each partition and split
based on that
> Make DTCS/TWCS split partitions based on time during compaction
> ---
>
> Key: CASSANDRA-10496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10496
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Marcus Eriksson
> Labels: dtcs
> Fix For: 3.11.x
>
>
> To avoid getting old data in new time windows with DTCS (or related, like
> [TWCS|CASSANDRA-9666]), we need to split out old data into its own sstable
> during compaction.
> My initial idea is to just create two sstables, when we create the compaction
> task we state the start and end times for the window, and any data older than
> the window will be put in its own sstable.
> By creating a single sstable with old data, we will incrementally get the
> windows correct - say we have an sstable with these timestamps:
> {{[100, 99, 98, 97, 75, 50, 10]}}
> and we are compacting in window {{[100, 80]}} - we would create two sstables:
> {{[100, 99, 98, 97]}}, {{[75, 50, 10]}}, and the first window is now
> 'correct'. The next compaction would compact in window {{[80, 60]}} and
> create sstables {{[75]}}, {{[50, 10]}} etc.
> We will probably also want to base the windows on the newest data in the
> sstables so that we actually have older data than the window.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10496) Make DTCS/TWCS split partitions based on time during compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-10496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988455#comment-15988455
]
Corentin Chary commented on CASSANDRA-10496:
I wanted to give it a shot for TWCS because of CASSANDRA-13418, I was thinking
about using a custom CompactionAwareWriter to seggregate data by timestamp in
the first window (and also make --split-output work). Currently I'm planning to
use partition.stats().minTimestamp, but I'm not sure how it is affect by
read-repairs. It may be a better idea to group data by deletion time instead ..
> Make DTCS/TWCS split partitions based on time during compaction
> ---
>
> Key: CASSANDRA-10496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10496
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Marcus Eriksson
> Labels: dtcs
> Fix For: 3.11.x
>
>
> To avoid getting old data in new time windows with DTCS (or related, like
> [TWCS|CASSANDRA-9666]), we need to split out old data into its own sstable
> during compaction.
> My initial idea is to just create two sstables, when we create the compaction
> task we state the start and end times for the window, and any data older than
> the window will be put in its own sstable.
> By creating a single sstable with old data, we will incrementally get the
> windows correct - say we have an sstable with these timestamps:
> {{[100, 99, 98, 97, 75, 50, 10]}}
> and we are compacting in window {{[100, 80]}} - we would create two sstables:
> {{[100, 99, 98, 97]}}, {{[75, 50, 10]}}, and the first window is now
> 'correct'. The next compaction would compact in window {{[80, 60]}} and
> create sstables {{[75]}}, {{[50, 10]}} etc.
> We will probably also want to base the windows on the newest data in the
> sstables so that we actually have older data than the window.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-4650) RangeStreamer should be smarter when picking endpoints for streaming in case of N >=3 in each DC.
[ https://issues.apache.org/jira/browse/CASSANDRA-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-4650: --- Fix Version/s: 4.x > RangeStreamer should be smarter when picking endpoints for streaming in case > of N >=3 in each DC. > --- > > Key: CASSANDRA-4650 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4650 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.1.5 >Reporter: sankalp kohli >Assignee: sankalp kohli >Priority: Minor > Labels: streaming > Fix For: 4.x > > Attachments: CASSANDRA-4650_trunk.txt, photo-1.JPG > > Original Estimate: 24h > Remaining Estimate: 24h > > getRangeFetchMap method in RangeStreamer should pick unique nodes to stream > data from when number of replicas in each DC is three or more. > When N>=3 in a DC, there are two options for streaming a range. Consider an > example of 4 nodes in one datacenter and replication factor of 3. > If a node goes down, it needs to recover 3 ranges of data. With current code, > two nodes could get selected as it orders the node by proximity. > We ideally will want to select 3 nodes for streaming the data. We can do this > by selecting unique nodes for each range. > Advantages: > This will increase the performance of bootstrapping a node and will also put > less pressure on nodes serving the data. > Note: This does not affect if N < 3 in each DC as then it streams data from > only 2 nodes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-4650) RangeStreamer should be smarter when picking endpoints for streaming in case of N >=3 in each DC.
[ https://issues.apache.org/jira/browse/CASSANDRA-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-4650: --- Status: Patch Available (was: Awaiting Feedback) > RangeStreamer should be smarter when picking endpoints for streaming in case > of N >=3 in each DC. > --- > > Key: CASSANDRA-4650 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4650 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.1.5 >Reporter: sankalp kohli >Assignee: sankalp kohli >Priority: Minor > Labels: streaming > Fix For: 4.x > > Attachments: CASSANDRA-4650_trunk.txt, photo-1.JPG > > Original Estimate: 24h > Remaining Estimate: 24h > > getRangeFetchMap method in RangeStreamer should pick unique nodes to stream > data from when number of replicas in each DC is three or more. > When N>=3 in a DC, there are two options for streaming a range. Consider an > example of 4 nodes in one datacenter and replication factor of 3. > If a node goes down, it needs to recover 3 ranges of data. With current code, > two nodes could get selected as it orders the node by proximity. > We ideally will want to select 3 nodes for streaming the data. We can do this > by selecting unique nodes for each range. > Advantages: > This will increase the performance of bootstrapping a node and will also put > less pressure on nodes serving the data. > Note: This does not affect if N < 3 in each DC as then it streams data from > only 2 nodes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-4650) RangeStreamer should be smarter when picking endpoints for streaming in case of N >=3 in each DC.
[ https://issues.apache.org/jira/browse/CASSANDRA-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988398#comment-15988398 ] Marcus Eriksson commented on CASSANDRA-4650: I have rebased it on trunk here: https://github.com/krummas/cassandra/commits/marcuse/4650-squashed Also added a comment about how the graph is modelled to hopefully make that clear I ran the dtests locally and they look good > RangeStreamer should be smarter when picking endpoints for streaming in case > of N >=3 in each DC. > --- > > Key: CASSANDRA-4650 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4650 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.1.5 >Reporter: sankalp kohli >Assignee: sankalp kohli >Priority: Minor > Labels: streaming > Fix For: 4.x > > Attachments: CASSANDRA-4650_trunk.txt, photo-1.JPG > > Original Estimate: 24h > Remaining Estimate: 24h > > getRangeFetchMap method in RangeStreamer should pick unique nodes to stream > data from when number of replicas in each DC is three or more. > When N>=3 in a DC, there are two options for streaming a range. Consider an > example of 4 nodes in one datacenter and replication factor of 3. > If a node goes down, it needs to recover 3 ranges of data. With current code, > two nodes could get selected as it orders the node by proximity. > We ideally will want to select 3 nodes for streaming the data. We can do this > by selecting unique nodes for each range. > Advantages: > This will increase the performance of bootstrapping a node and will also put > less pressure on nodes serving the data. > Note: This does not affect if N < 3 in each DC as then it streams data from > only 2 nodes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
