[jira] [Commented] (CASSANDRA-15225) FileUtils.close() does not handle non-IOException
[
https://issues.apache.org/jira/browse/CASSANDRA-15225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16899886#comment-16899886
]
Liudmila Kornilova commented on CASSANDRA-15225:
{quote}2) Log statement has only one '{}'. I don't think it will print
exception. I prefer caller to handle the logging part.
{quote}
I checked this. An exception is extracted here
ch.qos.logback.classic.spi.LoggingEvent:49 and it is printed
{quote}unless you want to make any changes in light of [~n.v.harikrishna]'s
feedback?
{quote}
I decided to change the code such that new exception is created.
I see 2 advantages of this version:
1. {{Throwable}} will not be lost if it appears before {{IOException}} or
{{IOException}} does not appear at all. {{Throwable}} will be rethrown as
IOException with cause.
2. Catch clauses become identical and they can be collapsed to one
{{Throwable}} clause
I pushed a separate commit so you can see the difference. I can squash commits
if you wish
> FileUtils.close() does not handle non-IOException
> -
>
> Key: CASSANDRA-15225
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15225
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Benedict
>Assignee: Liudmila Kornilova
>Priority: Normal
> Labels: pull-request-available
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This can lead to {{close}} not being invoked on remaining items
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15258) Cassandra JDK11 Windows not working
[ https://issues.apache.org/jira/browse/CASSANDRA-15258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RamyaK updated CASSANDRA-15258: --- Fix Version/s: 4.0.x > Cassandra JDK11 Windows not working > --- > > Key: CASSANDRA-15258 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15258 > Project: Cassandra > Issue Type: Bug > Components: Build, Local/Startup and Shutdown >Reporter: RamyaK >Priority: Normal > Fix For: 4.0.x > > > I'm trying to setup Cassandra 4.0 trunk with OpenJDK11, but getting below > error on start up. > > + $content = Get-Content "$env:CASSANDRA_CONF\jvm.options" > + ~ > + CategoryInfo : ObjectNotFound: > (D:\Stuff\save\C...onf\jvm.options:String) [Get-Content], > ItemNotFoundException > + FullyQualifiedErrorId : > PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand > Also JVM_VERSION is 11, still its showing as > Cassandra 4.0 requires either Java 8 (update 151 or newer) or Java 11 (or > newer). Java 11 is not supported. > > Please suggest. > -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15258) Cassandra JDK11 Windows not working
[ https://issues.apache.org/jira/browse/CASSANDRA-15258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RamyaK updated CASSANDRA-15258: --- Platform: Java11,OpenJDK,Windows (was: All) > Cassandra JDK11 Windows not working > --- > > Key: CASSANDRA-15258 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15258 > Project: Cassandra > Issue Type: Bug > Components: Build, Local/Startup and Shutdown >Reporter: RamyaK >Priority: Normal > Fix For: 4.0.x > > > I'm trying to setup Cassandra 4.0 trunk with OpenJDK11, but getting below > error on start up. > > + $content = Get-Content "$env:CASSANDRA_CONF\jvm.options" > + ~ > + CategoryInfo : ObjectNotFound: > (D:\Stuff\save\C...onf\jvm.options:String) [Get-Content], > ItemNotFoundException > + FullyQualifiedErrorId : > PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand > Also JVM_VERSION is 11, still its showing as > Cassandra 4.0 requires either Java 8 (update 151 or newer) or Java 11 (or > newer). Java 11 is not supported. > > Please suggest. > -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15258) Cassandra JDK11 Windows not working
RamyaK created CASSANDRA-15258: -- Summary: Cassandra JDK11 Windows not working Key: CASSANDRA-15258 URL: https://issues.apache.org/jira/browse/CASSANDRA-15258 Project: Cassandra Issue Type: Bug Components: Build, Local/Startup and Shutdown Reporter: RamyaK I'm trying to setup Cassandra 4.0 trunk with OpenJDK11, but getting below error on start up. + $content = Get-Content "$env:CASSANDRA_CONF\jvm.options" + ~ + CategoryInfo : ObjectNotFound: (D:\Stuff\save\C...onf\jvm.options:String) [Get-Content], ItemNotFoundException + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand Also JVM_VERSION is 11, still its showing as Cassandra 4.0 requires either Java 8 (update 151 or newer) or Java 11 (or newer). Java 11 is not supported. Please suggest. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15258) Cassandra JDK11 Windows not working
[ https://issues.apache.org/jira/browse/CASSANDRA-15258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RamyaK updated CASSANDRA-15258: --- Severity: Critical Since Version: 4.0.x Labels: windows (was: ) Fix Version/s: (was: 4.0.x) > Cassandra JDK11 Windows not working > --- > > Key: CASSANDRA-15258 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15258 > Project: Cassandra > Issue Type: Bug > Components: Build, Local/Startup and Shutdown >Reporter: RamyaK >Priority: Urgent > Labels: windows > > I'm trying to setup Cassandra 4.0 trunk with OpenJDK11, but getting below > error on start up. > > + $content = Get-Content "$env:CASSANDRA_CONF\jvm.options" > + ~ > + CategoryInfo : ObjectNotFound: > (D:\Stuff\save\C...onf\jvm.options:String) [Get-Content], > ItemNotFoundException > + FullyQualifiedErrorId : > PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand > Also JVM_VERSION is 11, still its showing as > Cassandra 4.0 requires either Java 8 (update 151 or newer) or Java 11 (or > newer). Java 11 is not supported. > > Please suggest. > -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14952) NPE when using allocate_tokens_for_keyspace and add new DC
[
https://issues.apache.org/jira/browse/CASSANDRA-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16899686#comment-16899686
]
mck edited comment on CASSANDRA-14952 at 8/5/19 2:34 PM:
-
> Do we want to treat the first node added in a new datacenter as a unique
> unit, which is what we get with rack = 1?
It seems to make sense to treat such a node as its own unique unit (as it's the
first in any eventuating unit group). Although seeds (non-autobootstrapping)
and non-existant dc names (CASSANDRA-12681) can also prevent that from
happening.
A slightly modified version of your fix [~chovatia.jayd...@gmail.com]
|| branch || circleci || asf jenkins testall || asf jenkins dtests ||
|
[CASSANDRA-14952|https://github.com/thelastpickle/cassandra/commit/3a72a51f9cb06ac85a4c78f3719a598a3a754909]
|
[circleci|https://circleci.com/workflow-run/b1f8b919-f889-47c5-9019-22a3468a428d]
|
[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/40//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/40/]
|
[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/675//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/675/]
|
was (Author: michaelsembwever):
> Do we want to treat the first node added in a new datacenter as a unique
> unit, which is what we get with rack = 1?
It seems to make sense to treat such a node as its own unique unit (as it's the
first in any eventuating unit group). Although seeds (non-autobootstrapping)
and non-existant dc names (CASSANDRA-12681) can also prevent that from
happening.
A slightly modified version of your fix [~chovatia.jayd...@gmail.com]
|| branch || circleci || asf jenkins testall || asf jenkins dtests ||
|
[CASSANDRA-14952|https://github.com/thelastpickle/cassandra/commit/3a72a51f9cb06ac85a4c78f3719a598a3a754909]
|
[circleci|https://circleci.com/workflow-run/b1f8b919-f889-47c5-9019-22a3468a428d]
|
[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/40//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/40/]
|
[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/675//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/675/]
| |
> NPE when using allocate_tokens_for_keyspace and add new DC
> --
>
> Key: CASSANDRA-14952
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14952
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Low
> Fix For: 3.0.x
>
>
> Received following NPE while bootstrapping very first node in the new
> datacenter with {{allocate_tokens_for_keyspace}} yaml option
> {code:java}
> INFO 21:44:13 JOINING: getting bootstrap token
> Exception (java.lang.NullPointerException) encountered during startup: null
> java.lang.NullPointerException
> at
> org.apache.cassandra.dht.tokenallocator.TokenAllocation.getStrategy(TokenAllocation.java:208)
> at
> org.apache.cassandra.dht.tokenallocator.TokenAllocation.getStrategy(TokenAllocation.java:170)
> at
> org.apache.cassandra.dht.tokenallocator.TokenAllocation.allocateTokens(TokenAllocation.java:55)
> at
> org.apache.cassandra.dht.BootStrapper.allocateTokens(BootStrapper.java:206)
> at
> org.apache.cassandra.dht.BootStrapper.getBootstrapTokens(BootStrapper.java:173)
> at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:854)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:666)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:579)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:351)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:586)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:714)
> {code}
> Please find reproducible steps here:
> 1. Set {{allocate_tokens_for_keyspace}} property with
> {{Networktopologystrategy}} say Networktopologystrategy, 'dc1' : 1, 'dc2'
> : 1
> 2. Start first node in {{dc1}}
> 3. Now bootstrap second node in {{dc2,}} it will throw above exception.
> RCA:
>
> [doAddEndpoint|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/locator/TokenMetadata.java#L1325]
> is invoked from the
> [bootstrap|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/service/StorageService.java#L1254]
>
[jira] [Created] (CASSANDRA-15259) Selecting Index by Lowest Mean Column Count Selects Random Index
Jordan West created CASSANDRA-15259:
---
Summary: Selecting Index by Lowest Mean Column Count Selects
Random Index
Key: CASSANDRA-15259
URL: https://issues.apache.org/jira/browse/CASSANDRA-15259
Project: Cassandra
Issue Type: Bug
Components: Feature/2i Index
Reporter: Jordan West
Assignee: Jordan West
{{CassandraIndex}} uses
{{[ColumnFamilyStore#getMeanColumns|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/index/internal/CassandraIndex.java#L273],
average columns per partition, which always returns the same answer for index
CFs because they contain no regular columns and clustering columns aren't
included in the count in Cassandra 3.0+.}}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15259) Selecting Index by Lowest Mean Column Count Selects Random Index
[
https://issues.apache.org/jira/browse/CASSANDRA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-15259:
Description:
{{CassandraIndex}} uses
[{{ColumnFamilyStore#getMeanColumns}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/index/internal/CassandraIndex.java#L273],
average columns per partition, which always returns the same answer for index
CFs because they contain no regular columns and clustering columns aren't
included in the count in Cassandra 3.0+.
was:
{{CassandraIndex}} uses
{{[ColumnFamilyStore#getMeanColumns|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/index/internal/CassandraIndex.java#L273],
average columns per partition, which always returns the same answer for index
CFs because they contain no regular columns and clustering columns aren't
included in the count in Cassandra 3.0+.}}
> Selecting Index by Lowest Mean Column Count Selects Random Index
>
>
> Key: CASSANDRA-15259
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15259
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> {{CassandraIndex}} uses
> [{{ColumnFamilyStore#getMeanColumns}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/index/internal/CassandraIndex.java#L273],
> average columns per partition, which always returns the same answer for
> index CFs because they contain no regular columns and clustering columns
> aren't included in the count in Cassandra 3.0+.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-8838) Resumable bootstrap streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900234#comment-16900234 ] Jeff Jirsa commented on CASSANDRA-8838: --- For the record, this patch almost certainly causes us to violate consistency/correctness for the reasons discussed in the 4 comments above (missing writes while restarting / may not store hints / may not deliver hints before timeout), and that it's enabled by default and can't be disabled is really unfortunate for people who want to disable this feature. We need to do be more aware of new features and correctness in the future. > Resumable bootstrap streaming > - > > Key: CASSANDRA-8838 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8838 > Project: Cassandra > Issue Type: Sub-task > Components: Legacy/Streaming and Messaging >Reporter: Yuki Morishita >Assignee: Yuki Morishita >Priority: Low > Labels: dense-storage > Fix For: 2.2.0 beta 1 > > > This allows the bootstrapping node not to be streamed already received data. > The bootstrapping node records received keyspace/ranges as one stream session > completes. When some sessions with other nodes fail, bootstrapping fails > also, though next time it re-bootstraps, already received keyspace/ranges are > skipped to be streamed. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15260) Add `allocate_tokens_for_dc_rf` yaml option for token allocation
mck created CASSANDRA-15260: --- Summary: Add `allocate_tokens_for_dc_rf` yaml option for token allocation Key: CASSANDRA-15260 URL: https://issues.apache.org/jira/browse/CASSANDRA-15260 Project: Cassandra Issue Type: Improvement Components: Local/Config Reporter: mck Assignee: mck Similar to option in DSE `allocate_tokens_for_local_replication_factor` Currently the [ReplicationAwareTokenAllocator|https://www.datastax.com/dev/blog/token-allocation-algorithm] requires a defined keyspace and a replica factor specified in the current datacenter. This is problematic in a number of ways. Come version 4.0 the replica factor can not be defined in new datacenters before those datacenters are up and running. Previously even real keyspaces could not be used as a new datacenter has to, in practice, have all its nodes up and running before it has the capacity to replicate data into it. New datacenters, or lift-and-shifting a cluster via datacenter migration, can be done using a dummy keyspace that duplicates the replication strategy and factor of the real keyspace. This issues are reduced by avoiding the keyspace definition and lookup, and presuming the replica strategy is by datacenter, ie NTS, with the introduction of an `allocate_tokens_for_dc_rf` option. It may also be of value considering whether `allocate_tokens_for_dc_rf=3` is the default, as this is the replication factor for the vast majority of datacenters in production. I suspect this would be a good improvement over the existing randomly generated tokens algorithm. Initial patch is available in https://github.com/thelastpickle/cassandra/commit/fc4865b0399570e58f11215565ba17dc4a53da97 -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15260) Add `allocate_tokens_for_dc_rf` yaml option for token allocation
[
https://issues.apache.org/jira/browse/CASSANDRA-15260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mck updated CASSANDRA-15260:
Description:
Similar to DSE's option: {{allocate_tokens_for_local_replication_factor}}
Currently the
[ReplicationAwareTokenAllocator|https://www.datastax.com/dev/blog/token-allocation-algorithm]
requires a defined keyspace and a replica factor specified in the current
datacenter.
This is problematic in a number of ways. The real keyspace can not be used when
adding new datacenters as, in practice, all its nodes need to be up and running
before it has the capacity to replicate data into it. New datacenters (or
lift-and-shifting a cluster via datacenter migration) therefore has to be done
using a dummy keyspace that duplicates the replication strategy+factor of the
real keyspace. This gets even more difficult come version 4.0, as the replica
factor can not even be defined in new datacenters before those datacenters are
up and running.
These issues are removed by avoiding the keyspace definition and lookup, and
presuming the replica strategy is by datacenter, ie NTS, with the introduction
of an {{allocate_tokens_for_dc_rf}} option.
It may also be of value considering whether {{allocate_tokens_for_dc_rf=3}}
becomes the default, as this is the replication factor for the vast majority of
datacenters in production. I suspect this would be a good improvement over the
existing randomly generated tokens algorithm.
Initial patch is available in
[https://github.com/thelastpickle/cassandra/commit/fc4865b0399570e58f11215565ba17dc4a53da97]
The patch does not remove the existing {{allocate_tokens_for_keyspace}} option,
as it still provides the codebase for handling different replication strategies.
was:
Similar to option in DSE `allocate_tokens_for_local_replication_factor`
Currently the
[ReplicationAwareTokenAllocator|https://www.datastax.com/dev/blog/token-allocation-algorithm]
requires a defined keyspace and a replica factor specified in the current
datacenter.
This is problematic in a number of ways. Come version 4.0 the replica factor
can not be defined in new datacenters before those datacenters are up and
running. Previously even real keyspaces could not be used as a new datacenter
has to, in practice, have all its nodes up and running before it has the
capacity to replicate data into it. New datacenters, or lift-and-shifting a
cluster via datacenter migration, can be done using a dummy keyspace that
duplicates the replication strategy and factor of the real keyspace.
This issues are reduced by avoiding the keyspace definition and lookup, and
presuming the replica strategy is by datacenter, ie NTS, with the introduction
of an `allocate_tokens_for_dc_rf` option.
It may also be of value considering whether `allocate_tokens_for_dc_rf=3` is
the default, as this is the replication factor for the vast majority of
datacenters in production. I suspect this would be a good improvement over the
existing randomly generated tokens algorithm.
Initial patch is available in
https://github.com/thelastpickle/cassandra/commit/fc4865b0399570e58f11215565ba17dc4a53da97
> Add `allocate_tokens_for_dc_rf` yaml option for token allocation
>
>
> Key: CASSANDRA-15260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15260
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: mck
>Assignee: mck
>Priority: Normal
>
> Similar to DSE's option: {{allocate_tokens_for_local_replication_factor}}
> Currently the
> [ReplicationAwareTokenAllocator|https://www.datastax.com/dev/blog/token-allocation-algorithm]
> requires a defined keyspace and a replica factor specified in the current
> datacenter.
> This is problematic in a number of ways. The real keyspace can not be used
> when adding new datacenters as, in practice, all its nodes need to be up and
> running before it has the capacity to replicate data into it. New datacenters
> (or lift-and-shifting a cluster via datacenter migration) therefore has to be
> done using a dummy keyspace that duplicates the replication strategy+factor
> of the real keyspace. This gets even more difficult come version 4.0, as the
> replica factor can not even be defined in new datacenters before those
> datacenters are up and running.
> These issues are removed by avoiding the keyspace definition and lookup, and
> presuming the replica strategy is by datacenter, ie NTS, with the
> introduction of an {{allocate_tokens_for_dc_rf}} option.
> It may also be of value considering whether {{allocate_tokens_for_dc_rf=3}}
> becomes the default, as this is the replication factor for the vast majority
> of datacenters in production. I suspect this would be a good improvement over
[jira] [Updated] (CASSANDRA-15260) Add `allocate_tokens_for_dc_rf` yaml option for token allocation
[
https://issues.apache.org/jira/browse/CASSANDRA-15260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mck updated CASSANDRA-15260:
Fix Version/s: 4.x
> Add `allocate_tokens_for_dc_rf` yaml option for token allocation
>
>
> Key: CASSANDRA-15260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15260
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: mck
>Assignee: mck
>Priority: Normal
> Fix For: 4.x
>
>
> Similar to DSE's option: {{allocate_tokens_for_local_replication_factor}}
> Currently the
> [ReplicationAwareTokenAllocator|https://www.datastax.com/dev/blog/token-allocation-algorithm]
> requires a defined keyspace and a replica factor specified in the current
> datacenter.
> This is problematic in a number of ways. The real keyspace can not be used
> when adding new datacenters as, in practice, all its nodes need to be up and
> running before it has the capacity to replicate data into it. New datacenters
> (or lift-and-shifting a cluster via datacenter migration) therefore has to be
> done using a dummy keyspace that duplicates the replication strategy+factor
> of the real keyspace. This gets even more difficult come version 4.0, as the
> replica factor can not even be defined in new datacenters before those
> datacenters are up and running.
> These issues are removed by avoiding the keyspace definition and lookup, and
> presuming the replica strategy is by datacenter, ie NTS, with the
> introduction of an {{allocate_tokens_for_dc_rf}} option.
> It may also be of value considering whether {{allocate_tokens_for_dc_rf=3}}
> becomes the default, as this is the replication factor for the vast majority
> of datacenters in production. I suspect this would be a good improvement over
> the existing randomly generated tokens algorithm.
> Initial patch is available in
> [https://github.com/thelastpickle/cassandra/commit/fc4865b0399570e58f11215565ba17dc4a53da97]
> The patch does not remove the existing {{allocate_tokens_for_keyspace}}
> option, as it still provides the codebase for handling different replication
> strategies.
>
> fyi [~blambov] [~jay.zhuang] [~chovatia.jayd...@gmail.com] [~alokamvenki]
> [~alexchueshev]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15260) Add `allocate_tokens_for_dc_rf` yaml option for token allocation
[
https://issues.apache.org/jira/browse/CASSANDRA-15260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mck updated CASSANDRA-15260:
Description:
Similar to DSE's option: {{allocate_tokens_for_local_replication_factor}}
Currently the
[ReplicationAwareTokenAllocator|https://www.datastax.com/dev/blog/token-allocation-algorithm]
requires a defined keyspace and a replica factor specified in the current
datacenter.
This is problematic in a number of ways. The real keyspace can not be used when
adding new datacenters as, in practice, all its nodes need to be up and running
before it has the capacity to replicate data into it. New datacenters (or
lift-and-shifting a cluster via datacenter migration) therefore has to be done
using a dummy keyspace that duplicates the replication strategy+factor of the
real keyspace. This gets even more difficult come version 4.0, as the replica
factor can not even be defined in new datacenters before those datacenters are
up and running.
These issues are removed by avoiding the keyspace definition and lookup, and
presuming the replica strategy is by datacenter, ie NTS, with the introduction
of an {{allocate_tokens_for_dc_rf}} option.
It may also be of value considering whether {{allocate_tokens_for_dc_rf=3}}
becomes the default, as this is the replication factor for the vast majority of
datacenters in production. I suspect this would be a good improvement over the
existing randomly generated tokens algorithm.
Initial patch is available in
[https://github.com/thelastpickle/cassandra/commit/fc4865b0399570e58f11215565ba17dc4a53da97]
The patch does not remove the existing {{allocate_tokens_for_keyspace}} option,
as it still provides the codebase for handling different replication strategies.
fyi [~blambov] [~jay.zhuang] [~chovatia.jayd...@gmail.com] [~alokamvenki]
[~alexchueshev]
was:
Similar to DSE's option: {{allocate_tokens_for_local_replication_factor}}
Currently the
[ReplicationAwareTokenAllocator|https://www.datastax.com/dev/blog/token-allocation-algorithm]
requires a defined keyspace and a replica factor specified in the current
datacenter.
This is problematic in a number of ways. The real keyspace can not be used when
adding new datacenters as, in practice, all its nodes need to be up and running
before it has the capacity to replicate data into it. New datacenters (or
lift-and-shifting a cluster via datacenter migration) therefore has to be done
using a dummy keyspace that duplicates the replication strategy+factor of the
real keyspace. This gets even more difficult come version 4.0, as the replica
factor can not even be defined in new datacenters before those datacenters are
up and running.
These issues are removed by avoiding the keyspace definition and lookup, and
presuming the replica strategy is by datacenter, ie NTS, with the introduction
of an {{allocate_tokens_for_dc_rf}} option.
It may also be of value considering whether {{allocate_tokens_for_dc_rf=3}}
becomes the default, as this is the replication factor for the vast majority of
datacenters in production. I suspect this would be a good improvement over the
existing randomly generated tokens algorithm.
Initial patch is available in
[https://github.com/thelastpickle/cassandra/commit/fc4865b0399570e58f11215565ba17dc4a53da97]
The patch does not remove the existing {{allocate_tokens_for_keyspace}} option,
as it still provides the codebase for handling different replication strategies.
> Add `allocate_tokens_for_dc_rf` yaml option for token allocation
>
>
> Key: CASSANDRA-15260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15260
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: mck
>Assignee: mck
>Priority: Normal
>
> Similar to DSE's option: {{allocate_tokens_for_local_replication_factor}}
> Currently the
> [ReplicationAwareTokenAllocator|https://www.datastax.com/dev/blog/token-allocation-algorithm]
> requires a defined keyspace and a replica factor specified in the current
> datacenter.
> This is problematic in a number of ways. The real keyspace can not be used
> when adding new datacenters as, in practice, all its nodes need to be up and
> running before it has the capacity to replicate data into it. New datacenters
> (or lift-and-shifting a cluster via datacenter migration) therefore has to be
> done using a dummy keyspace that duplicates the replication strategy+factor
> of the real keyspace. This gets even more difficult come version 4.0, as the
> replica factor can not even be defined in new datacenters before those
> datacenters are up and running.
> These issues are removed by avoiding the keyspace definition and lookup, and
> presuming the replica strategy is by datacenter,
[jira] [Updated] (CASSANDRA-15259) Selecting Index by Lowest Mean Column Count Selects Random Index
[
https://issues.apache.org/jira/browse/CASSANDRA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-15259:
Severity: Critical
Complexity: Normal
Discovered By: User Report
Bug Category: Parent values: Degradation(12984)Level 1 values: Performance
Bug/Regression(12997)
Since Version: 3.0.0
Fix Version/s: 3.11.x
4.0
3.0.19
> Selecting Index by Lowest Mean Column Count Selects Random Index
>
>
> Key: CASSANDRA-15259
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15259
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Urgent
> Fix For: 3.0.19, 4.0, 3.11.x
>
>
> {{CassandraIndex}} uses
> [{{ColumnFamilyStore#getMeanColumns}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/index/internal/CassandraIndex.java#L273],
> average columns per partition, which always returns the same answer for
> index CFs because they contain no regular columns and clustering columns
> aren't included in the count in Cassandra 3.0+.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15259) Selecting Index by Lowest Mean Column Count Selects Random Index
[
https://issues.apache.org/jira/browse/CASSANDRA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-15259:
Status: Open (was: Triage Needed)
> Selecting Index by Lowest Mean Column Count Selects Random Index
>
>
> Key: CASSANDRA-15259
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15259
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Urgent
> Fix For: 3.0.19, 4.0, 3.11.x
>
>
> {{CassandraIndex}} uses
> [{{ColumnFamilyStore#getMeanColumns}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/index/internal/CassandraIndex.java#L273],
> average columns per partition, which always returns the same answer for
> index CFs because they contain no regular columns and clustering columns
> aren't included in the count in Cassandra 3.0+.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15259) Selecting Index by Lowest Mean Column Count Selects Random Index
[
https://issues.apache.org/jira/browse/CASSANDRA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-15259:
Test and Documentation Plan: Regression test added as part of patch
Status: Patch Available (was: In Progress)
||Branch||Tests||
|[trunk|https://github.com/apache/cassandra/compare/trunk...jrwest:jwest/15259-trunk]|[cci|https://circleci.com/workflow-run/fd4a72ce-d1fd-4c15-8b2a-a20544b658c8]|
|[3.11|https://github.com/apache/cassandra/compare/trunk...jrwest:jwest/15259-3.11]|[cci|https://circleci.com/workflow-run/69faf98e-7cc8-4f68-9af9-d6317e29923d]|
|[3.0|https://github.com/apache/cassandra/compare/trunk...jrwest:jwest/15259-3.0]|[cci|https://circleci.com/workflow-run/b958fc7e-91f1-4b30-a7df-89ea7941ee60]|
> Selecting Index by Lowest Mean Column Count Selects Random Index
>
>
> Key: CASSANDRA-15259
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15259
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Urgent
> Fix For: 3.0.19, 4.0, 3.11.x
>
>
> {{CassandraIndex}} uses
> [{{ColumnFamilyStore#getMeanColumns}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/index/internal/CassandraIndex.java#L273],
> average columns per partition, which always returns the same answer for
> index CFs because they contain no regular columns and clustering columns
> aren't included in the count in Cassandra 3.0+.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15194) Improve readability of Table metrics Virtual tables units
[
https://issues.apache.org/jira/browse/CASSANDRA-15194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900329#comment-16900329
]
Jon Haddad commented on CASSANDRA-15194:
Patch looks great, thanks for those fixes. +1 from me.
> Improve readability of Table metrics Virtual tables units
> -
>
> Key: CASSANDRA-15194
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15194
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Virtual Tables
>Reporter: Jon Haddad
>Assignee: Chris Lohfink
>Priority: Normal
> Fix For: 4.0
>
>
> I just noticed this strange output in the coordinator_reads output::
> {code}
> cqlsh:system_views> select * from coordinator_reads ;
> count | keyspace_name | table_name | 99th | max |
> median | per_second
> ---+++--+-++
> 7573 | tlp_stress | keyvalue |0 | 0 |
>0 | 2.2375e-16
> 6076 | tlp_stress | random_access |0 | 0 |
>0 | 7.4126e-12
>390 | tlp_stress |sensor_data_udt |0 | 0 |
>0 | 1.7721e-64
> 30 | system | local |0 | 0 |
>0 | 0.006406
> 11 | system_schema |columns |0 | 0 |
>0 | 1.1192e-16
> 11 | system_schema |indexes |0 | 0 |
>0 | 1.1192e-16
> 11 | system_schema | tables |0 | 0 |
>0 | 1.1192e-16
> 11 | system_schema | views |0 | 0 |
>0 | 1.1192e-16
> {code}
> cc [~cnlwsu]
> btw I realize the output is technically correct, but it's not very readable.
> For practical purposes this should just say 0.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15214) OOMs caught and not rethrown
[ https://issues.apache.org/jira/browse/CASSANDRA-15214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900379#comment-16900379 ] Dinesh Joshi commented on CASSANDRA-15214: -- I think this issue might be related to https://bugs.openjdk.java.net/browse/JDK-8027434. Other projects that use the JVM have run into a similar issue and the usual solution is to use [jvmkill|https://github.com/airlift/jvmkill]. The issue at hand is when a JVM has run out of memory (heap or otherwise), it enters an undefined state. In this situation, I would not expect the handlers to work as expected either. I think we should either use jvmkill or [jvmquake|https://github.com/Netflix-Skunkworks/jvmquake] to solve this issue as it has proven to be reliable and Netflix, Facebook and other large JVM users are actively using it. > OOMs caught and not rethrown > > > Key: CASSANDRA-15214 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15214 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client, Messaging/Internode >Reporter: Benedict >Priority: Normal > Fix For: 4.0 > > > Netty (at least, and perhaps elsewhere in Executors) catches all exceptions, > so presently there is no way to ensure that an OOM reaches the JVM handler to > trigger a crash/heapdump. > It may be that the simplest most consistent way to do this would be to have a > single thread spawned at startup that waits for any exceptions we must > propagate to the Runtime. > We could probably submit a patch upstream to Netty, but for a guaranteed > future proof approach, it may be worth paying the cost of a single thread. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15214) OOMs caught and not rethrown
[ https://issues.apache.org/jira/browse/CASSANDRA-15214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900379#comment-16900379 ] Dinesh Joshi edited comment on CASSANDRA-15214 at 8/5/19 8:07 PM: -- I think this issue might be related to https://bugs.openjdk.java.net/browse/JDK-8027434. Other projects that use the JVM have run into a similar issue and the usual solution is to use [jvmkill|https://github.com/airlift/jvmkill]. The issue at hand is that when a JVM runs out of memory (heap or otherwise), it enters an undefined state. In this situation, I would not expect the handlers to work as expected. I think we should either use jvmkill or [jvmquake|https://github.com/Netflix-Skunkworks/jvmquake] to solve this issue as it has proven to be reliable and Netflix, Facebook and other large JVM users are actively using it. was (Author: djoshi3): I think this issue might be related to https://bugs.openjdk.java.net/browse/JDK-8027434. Other projects that use the JVM have run into a similar issue and the usual solution is to use [jvmkill|https://github.com/airlift/jvmkill]. The issue at hand is when a JVM has run out of memory (heap or otherwise), it enters an undefined state. In this situation, I would not expect the handlers to work as expected either. I think we should either use jvmkill or [jvmquake|https://github.com/Netflix-Skunkworks/jvmquake] to solve this issue as it has proven to be reliable and Netflix, Facebook and other large JVM users are actively using it. > OOMs caught and not rethrown > > > Key: CASSANDRA-15214 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15214 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client, Messaging/Internode >Reporter: Benedict >Priority: Normal > Fix For: 4.0 > > > Netty (at least, and perhaps elsewhere in Executors) catches all exceptions, > so presently there is no way to ensure that an OOM reaches the JVM handler to > trigger a crash/heapdump. > It may be that the simplest most consistent way to do this would be to have a > single thread spawned at startup that waits for any exceptions we must > propagate to the Runtime. > We could probably submit a patch upstream to Netty, but for a guaranteed > future proof approach, it may be worth paying the cost of a single thread. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15260) Add `allocate_tokens_for_dc_rf` yaml option for token allocation
[
https://issues.apache.org/jira/browse/CASSANDRA-15260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mck updated CASSANDRA-15260:
Description:
Similar to DSE's option: {{allocate_tokens_for_local_replication_factor}}
Currently the
[ReplicationAwareTokenAllocator|https://www.datastax.com/dev/blog/token-allocation-algorithm]
requires a defined keyspace and a replica factor specified in the current
datacenter.
This is problematic in a number of ways. The real keyspace can not be used when
adding new datacenters as, in practice, all its nodes need to be up and running
before it has the capacity to replicate data into it. New datacenters (or
lift-and-shifting a cluster via datacenter migration) therefore has to be done
using a dummy keyspace that duplicates the replication strategy+factor of the
real keyspace. This gets even more difficult come version 4.0, as the replica
factor can not even be defined in new datacenters before those datacenters are
up and running.
These issues are removed by avoiding the keyspace definition and lookup, and
presuming the replica strategy is by datacenter, ie NTS, with the introduction
of an {{allocate_tokens_for_dc_rf}} option.
It may also be of value considering whether {{allocate_tokens_for_dc_rf=3}}
becomes the default, as this is the replication factor for the vast majority of
datacenters in production. I suspect this would be a good improvement over the
existing randomly generated tokens algorithm.
Initial patch is available in
[https://github.com/thelastpickle/cassandra/commit/fc4865b0399570e58f11215565ba17dc4a53da97]
The patch does not remove the existing {{allocate_tokens_for_keyspace}} option,
as that provides the codebase for handling different replication strategies.
fyi [~blambov] [~jay.zhuang] [~chovatia.jayd...@gmail.com] [~alokamvenki]
[~alexchueshev]
was:
Similar to DSE's option: {{allocate_tokens_for_local_replication_factor}}
Currently the
[ReplicationAwareTokenAllocator|https://www.datastax.com/dev/blog/token-allocation-algorithm]
requires a defined keyspace and a replica factor specified in the current
datacenter.
This is problematic in a number of ways. The real keyspace can not be used when
adding new datacenters as, in practice, all its nodes need to be up and running
before it has the capacity to replicate data into it. New datacenters (or
lift-and-shifting a cluster via datacenter migration) therefore has to be done
using a dummy keyspace that duplicates the replication strategy+factor of the
real keyspace. This gets even more difficult come version 4.0, as the replica
factor can not even be defined in new datacenters before those datacenters are
up and running.
These issues are removed by avoiding the keyspace definition and lookup, and
presuming the replica strategy is by datacenter, ie NTS, with the introduction
of an {{allocate_tokens_for_dc_rf}} option.
It may also be of value considering whether {{allocate_tokens_for_dc_rf=3}}
becomes the default, as this is the replication factor for the vast majority of
datacenters in production. I suspect this would be a good improvement over the
existing randomly generated tokens algorithm.
Initial patch is available in
[https://github.com/thelastpickle/cassandra/commit/fc4865b0399570e58f11215565ba17dc4a53da97]
The patch does not remove the existing {{allocate_tokens_for_keyspace}} option,
as it still provides the codebase for handling different replication strategies.
fyi [~blambov] [~jay.zhuang] [~chovatia.jayd...@gmail.com] [~alokamvenki]
[~alexchueshev]
> Add `allocate_tokens_for_dc_rf` yaml option for token allocation
>
>
> Key: CASSANDRA-15260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15260
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: mck
>Assignee: mck
>Priority: Normal
> Fix For: 4.x
>
>
> Similar to DSE's option: {{allocate_tokens_for_local_replication_factor}}
> Currently the
> [ReplicationAwareTokenAllocator|https://www.datastax.com/dev/blog/token-allocation-algorithm]
> requires a defined keyspace and a replica factor specified in the current
> datacenter.
> This is problematic in a number of ways. The real keyspace can not be used
> when adding new datacenters as, in practice, all its nodes need to be up and
> running before it has the capacity to replicate data into it. New datacenters
> (or lift-and-shifting a cluster via datacenter migration) therefore has to be
> done using a dummy keyspace that duplicates the replication strategy+factor
> of the real keyspace. This gets even more difficult come version 4.0, as the
> replica factor can not even be defined in new datacenters before those
> datacenters are up and running.
> These iss
[jira] [Updated] (CASSANDRA-15260) Add `allocate_tokens_for_dc_rf` yaml option for token allocation
[
https://issues.apache.org/jira/browse/CASSANDRA-15260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mck updated CASSANDRA-15260:
Impacts: (was: None)
> Add `allocate_tokens_for_dc_rf` yaml option for token allocation
>
>
> Key: CASSANDRA-15260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15260
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: mck
>Assignee: mck
>Priority: Normal
> Fix For: 4.x
>
>
> Similar to DSE's option: {{allocate_tokens_for_local_replication_factor}}
> Currently the
> [ReplicationAwareTokenAllocator|https://www.datastax.com/dev/blog/token-allocation-algorithm]
> requires a defined keyspace and a replica factor specified in the current
> datacenter.
> This is problematic in a number of ways. The real keyspace can not be used
> when adding new datacenters as, in practice, all its nodes need to be up and
> running before it has the capacity to replicate data into it. New datacenters
> (or lift-and-shifting a cluster via datacenter migration) therefore has to be
> done using a dummy keyspace that duplicates the replication strategy+factor
> of the real keyspace. This gets even more difficult come version 4.0, as the
> replica factor can not even be defined in new datacenters before those
> datacenters are up and running.
> These issues are removed by avoiding the keyspace definition and lookup, and
> presuming the replica strategy is by datacenter, ie NTS, with the
> introduction of an {{allocate_tokens_for_dc_rf}} option.
> It may also be of value considering whether {{allocate_tokens_for_dc_rf=3}}
> becomes the default, as this is the replication factor for the vast majority
> of datacenters in production. I suspect this would be a good improvement over
> the existing randomly generated tokens algorithm.
> Initial patch is available in
> [https://github.com/thelastpickle/cassandra/commit/fc4865b0399570e58f11215565ba17dc4a53da97]
> The patch does not remove the existing {{allocate_tokens_for_keyspace}}
> option, as it still provides the codebase for handling different replication
> strategies.
>
> fyi [~blambov] [~jay.zhuang] [~chovatia.jayd...@gmail.com] [~alokamvenki]
> [~alexchueshev]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15260) Add `allocate_tokens_for_dc_rf` yaml option for token allocation
[
https://issues.apache.org/jira/browse/CASSANDRA-15260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mck updated CASSANDRA-15260:
Description:
Similar to DSE's option: {{allocate_tokens_for_local_replication_factor}}
Currently the
[ReplicationAwareTokenAllocator|https://www.datastax.com/dev/blog/token-allocation-algorithm]
requires a defined keyspace and a replica factor specified in the current
datacenter.
This is problematic in a number of ways. The real keyspace can not be used when
adding new datacenters as, in practice, all its nodes need to be up and running
before it has the capacity to replicate data into it. New datacenters (or
lift-and-shifting a cluster via datacenter migration) therefore has to be done
using a dummy keyspace that duplicates the replication strategy+factor of the
real keyspace. This gets even more difficult come version 4.0, as the replica
factor can not even be defined in new datacenters before those datacenters are
up and running.
These issues are removed by avoiding the keyspace definition and lookup, and
presuming the replica strategy is by datacenter, ie NTS. This can be done with
the use of an {{allocate_tokens_for_dc_rf}} option.
It may also be of value considering whether {{allocate_tokens_for_dc_rf=3}}
becomes the default? as this is the replication factor for the vast majority of
datacenters in production. I suspect this would be a good improvement over the
existing randomly generated tokens algorithm.
Initial patch is available in
[https://github.com/thelastpickle/cassandra/commit/fc4865b0399570e58f11215565ba17dc4a53da97]
The patch does not remove the existing {{allocate_tokens_for_keyspace}} option,
as that provides the codebase for handling different replication strategies.
fyi [~blambov] [~jay.zhuang] [~chovatia.jayd...@gmail.com] [~alokamvenki]
[~alexchueshev]
was:
Similar to DSE's option: {{allocate_tokens_for_local_replication_factor}}
Currently the
[ReplicationAwareTokenAllocator|https://www.datastax.com/dev/blog/token-allocation-algorithm]
requires a defined keyspace and a replica factor specified in the current
datacenter.
This is problematic in a number of ways. The real keyspace can not be used when
adding new datacenters as, in practice, all its nodes need to be up and running
before it has the capacity to replicate data into it. New datacenters (or
lift-and-shifting a cluster via datacenter migration) therefore has to be done
using a dummy keyspace that duplicates the replication strategy+factor of the
real keyspace. This gets even more difficult come version 4.0, as the replica
factor can not even be defined in new datacenters before those datacenters are
up and running.
These issues are removed by avoiding the keyspace definition and lookup, and
presuming the replica strategy is by datacenter, ie NTS, with the introduction
of an {{allocate_tokens_for_dc_rf}} option.
It may also be of value considering whether {{allocate_tokens_for_dc_rf=3}}
becomes the default, as this is the replication factor for the vast majority of
datacenters in production. I suspect this would be a good improvement over the
existing randomly generated tokens algorithm.
Initial patch is available in
[https://github.com/thelastpickle/cassandra/commit/fc4865b0399570e58f11215565ba17dc4a53da97]
The patch does not remove the existing {{allocate_tokens_for_keyspace}} option,
as that provides the codebase for handling different replication strategies.
fyi [~blambov] [~jay.zhuang] [~chovatia.jayd...@gmail.com] [~alokamvenki]
[~alexchueshev]
> Add `allocate_tokens_for_dc_rf` yaml option for token allocation
>
>
> Key: CASSANDRA-15260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15260
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: mck
>Assignee: mck
>Priority: Normal
> Fix For: 4.x
>
>
> Similar to DSE's option: {{allocate_tokens_for_local_replication_factor}}
> Currently the
> [ReplicationAwareTokenAllocator|https://www.datastax.com/dev/blog/token-allocation-algorithm]
> requires a defined keyspace and a replica factor specified in the current
> datacenter.
> This is problematic in a number of ways. The real keyspace can not be used
> when adding new datacenters as, in practice, all its nodes need to be up and
> running before it has the capacity to replicate data into it. New datacenters
> (or lift-and-shifting a cluster via datacenter migration) therefore has to be
> done using a dummy keyspace that duplicates the replication strategy+factor
> of the real keyspace. This gets even more difficult come version 4.0, as the
> replica factor can not even be defined in new datacenters before those
> datacenters are up and running.
> These
[jira] [Updated] (CASSANDRA-15259) Selecting Index by Lowest Mean Column Count Selects Random Index
[
https://issues.apache.org/jira/browse/CASSANDRA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Blake Eggleston updated CASSANDRA-15259:
Reviewers: Blake Eggleston
> Selecting Index by Lowest Mean Column Count Selects Random Index
>
>
> Key: CASSANDRA-15259
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15259
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Urgent
> Fix For: 3.0.19, 4.0, 3.11.x
>
>
> {{CassandraIndex}} uses
> [{{ColumnFamilyStore#getMeanColumns}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/index/internal/CassandraIndex.java#L273],
> average columns per partition, which always returns the same answer for
> index CFs because they contain no regular columns and clustering columns
> aren't included in the count in Cassandra 3.0+.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15259) Selecting Index by Lowest Mean Column Count Selects Random Index
[
https://issues.apache.org/jira/browse/CASSANDRA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Blake Eggleston updated CASSANDRA-15259:
Status: Review In Progress (was: Patch Available)
> Selecting Index by Lowest Mean Column Count Selects Random Index
>
>
> Key: CASSANDRA-15259
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15259
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Urgent
> Fix For: 3.0.19, 4.0, 3.11.x
>
>
> {{CassandraIndex}} uses
> [{{ColumnFamilyStore#getMeanColumns}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/index/internal/CassandraIndex.java#L273],
> average columns per partition, which always returns the same answer for
> index CFs because they contain no regular columns and clustering columns
> aren't included in the count in Cassandra 3.0+.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15214) OOMs caught and not rethrown
[ https://issues.apache.org/jira/browse/CASSANDRA-15214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900394#comment-16900394 ] Benedict commented on CASSANDRA-15214: -- Sorry, I completely forgot to respond to this ticket so thanks for bumping it [~djoshi3] >From my POV, including a C JVMTI agent is absolutely fine. We'd have to take >a closer look at jvmkill and jvmquake, and do our own brief audit of the >version we include to ensure it seems to behave reasonably. But I don't see >any problem with utilising non-Java functionality. > OOMs caught and not rethrown > > > Key: CASSANDRA-15214 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15214 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client, Messaging/Internode >Reporter: Benedict >Priority: Normal > Fix For: 4.0 > > > Netty (at least, and perhaps elsewhere in Executors) catches all exceptions, > so presently there is no way to ensure that an OOM reaches the JVM handler to > trigger a crash/heapdump. > It may be that the simplest most consistent way to do this would be to have a > single thread spawned at startup that waits for any exceptions we must > propagate to the Runtime. > We could probably submit a patch upstream to Netty, but for a guaranteed > future proof approach, it may be worth paying the cost of a single thread. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15214) OOMs caught and not rethrown
[ https://issues.apache.org/jira/browse/CASSANDRA-15214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900394#comment-16900394 ] Benedict edited comment on CASSANDRA-15214 at 8/5/19 8:45 PM: -- Sorry, I completely forgot to respond to this ticket so thanks for bumping it [~djoshi3] >From my POV, including a C JVMTI agent is absolutely fine, [~jolynch]. We'd >have to take a closer look at jvmkill and jvmquake, and do our own brief audit >of the version we include to ensure it seems to behave reasonably. But I >don't see any problem with utilising non-Java functionality. was (Author: benedict): Sorry, I completely forgot to respond to this ticket so thanks for bumping it [~djoshi3] >From my POV, including a C JVMTI agent is absolutely fine. We'd have to take >a closer look at jvmkill and jvmquake, and do our own brief audit of the >version we include to ensure it seems to behave reasonably. But I don't see >any problem with utilising non-Java functionality. > OOMs caught and not rethrown > > > Key: CASSANDRA-15214 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15214 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client, Messaging/Internode >Reporter: Benedict >Priority: Normal > Fix For: 4.0 > > > Netty (at least, and perhaps elsewhere in Executors) catches all exceptions, > so presently there is no way to ensure that an OOM reaches the JVM handler to > trigger a crash/heapdump. > It may be that the simplest most consistent way to do this would be to have a > single thread spawned at startup that waits for any exceptions we must > propagate to the Runtime. > We could probably submit a patch upstream to Netty, but for a guaranteed > future proof approach, it may be worth paying the cost of a single thread. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15259) Selecting Index by Lowest Mean Column Count Selects Random Index
[
https://issues.apache.org/jira/browse/CASSANDRA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Blake Eggleston updated CASSANDRA-15259:
Status: Changes Suggested (was: Review In Progress)
The {{totalRows}} metadata element was added to the sstable format in 3.0, so
we’ll still need to use the old method for 2.x sstables. Looks good otherwise.
> Selecting Index by Lowest Mean Column Count Selects Random Index
>
>
> Key: CASSANDRA-15259
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15259
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Urgent
> Fix For: 3.0.19, 4.0, 3.11.x
>
>
> {{CassandraIndex}} uses
> [{{ColumnFamilyStore#getMeanColumns}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/index/internal/CassandraIndex.java#L273],
> average columns per partition, which always returns the same answer for
> index CFs because they contain no regular columns and clustering columns
> aren't included in the count in Cassandra 3.0+.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15214) OOMs caught and not rethrown
[ https://issues.apache.org/jira/browse/CASSANDRA-15214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900410#comment-16900410 ] Dinesh Joshi commented on CASSANDRA-15214: -- Sounds great. [~benedict] who would be able to take up the audit? Is this something I can help with? > OOMs caught and not rethrown > > > Key: CASSANDRA-15214 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15214 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client, Messaging/Internode >Reporter: Benedict >Priority: Normal > Fix For: 4.0 > > > Netty (at least, and perhaps elsewhere in Executors) catches all exceptions, > so presently there is no way to ensure that an OOM reaches the JVM handler to > trigger a crash/heapdump. > It may be that the simplest most consistent way to do this would be to have a > single thread spawned at startup that waits for any exceptions we must > propagate to the Runtime. > We could probably submit a patch upstream to Netty, but for a guaranteed > future proof approach, it may be worth paying the cost of a single thread. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15259) Selecting Index by Lowest Mean Column Count Selects Random Index
[
https://issues.apache.org/jira/browse/CASSANDRA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900575#comment-16900575
]
Jordan West commented on CASSANDRA-15259:
-
[~bdeggleston] good catch re: 2.1 sstables. I see two ways to handle that off
the top of my head – besides not including the legacy sstables in the
calculation which is broken.
I think I prefer {{getMeanRowCount2}} (average of the row count and column
count) because in the case of 100% legacy sstables or 100% new sstables it
degrades to {{getMeanColumns}} or the original {{getMeanRowCount.}} Neither
implementation is ideal since we have to handle it at the per sstable level and
what that means for an average is ambiguous.
Also, I wonder if the method name should change and/or if the logic should be
moved to somewhere index specific like {{CassandraIndex}}, now that what its
doing is a bit more specialized and less clear. WDYT?
{code:java}
public int getMeanRowCount()
{
long totalRows = 0;
long totalPartitions = 0;
for (SSTableReader sstable : getSSTables(SSTableSet.CANONICAL))
{
if (sstable.descriptor.version.storeRows())
{
totalPartitions += sstable.getEstimatedPartitionSize().count();
totalRows += sstable.getTotalRows();
} else
{
long colCount = sstable.getEstimatedColumnCount().count();
totalPartitions += colCount;
totalRows += sstable.getEstimatedColumnCount().mean() * colCount;
}
}
return totalPartitions > 0 ? (int) (totalRows / totalPartitions) : 0;
}
public int getMeanRowCount2()
{
long totalRows = 0;
long totalPartitions = 0;
long legacyCols = 0;
long legacyTotal = 0;
for (SSTableReader sstable : getSSTables(SSTableSet.CANONICAL))
{
if (sstable.descriptor.version.storeRows())
{
totalPartitions += sstable.getEstimatedPartitionSize().count();
totalRows += sstable.getTotalRows();
} else
{
long colCount = sstable.getEstimatedColumnCount().count();
legacyCols += sstable.getEstimatedColumnCount().mean() * colCount;
legacyTotal += colCount;
}
}
int rowMean = totalPartitions > 0 ? (int) (totalRows / totalPartitions) : 0;
int legacyMean = legacyTotal > 0 ? (int) (legacyCols / legacyTotal) : 0;
return (int) (((rowMean * totalPartitions) + (legacyMean * legacyTotal)) /
(totalPartitions + legacyTotal));
}
{code}
> Selecting Index by Lowest Mean Column Count Selects Random Index
>
>
> Key: CASSANDRA-15259
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15259
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Urgent
> Fix For: 3.0.19, 4.0, 3.11.x
>
>
> {{CassandraIndex}} uses
> [{{ColumnFamilyStore#getMeanColumns}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/index/internal/CassandraIndex.java#L273],
> average columns per partition, which always returns the same answer for
> index CFs because they contain no regular columns and clustering columns
> aren't included in the count in Cassandra 3.0+.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15172) AbstractLocalAwareExecutorService Exception During Upgrade to 3.11.4
[
https://issues.apache.org/jira/browse/CASSANDRA-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900581#comment-16900581
]
feroz shaik commented on CASSANDRA-15172:
-
We have also hit this problem today while upgrading from 2.1.16 to 3.11.4/
we encountered this as soon as node started up with 3.11.4
ERROR [Native-Transport-Requests-32] 2019-08-06 02:14:20,353
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
and the below errors continued in the logfile as long as the process was up.
ERROR [Native-Transport-Requests-12] 2019-08-06 03:00:47,135
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-8] 2019-08-06 03:00:48,778
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-13] 2019-08-06 03:00:57,454
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-11] 2019-08-06 03:00:57,482
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-2] 2019-08-06 03:00:58,543
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-8] 2019-08-06 03:00:58,899
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-17] 2019-08-06 03:00:59,074
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-12] 2019-08-06 03:01:08,123
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-17] 2019-08-06 03:01:19,055
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-4] 2019-08-06 03:01:20,880
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
WARN [ReadStage-13] 2019-08-06 03:01:29,983
AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
Thread[ReadStage-13,5,main]: {}
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-2] 2019-08-06 03:01:31,119
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-6] 2019-08-06 03:01:46,262
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-15] 2019-08-06 03:01:46,520
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
WARN [ReadStage-2] 2019-08-06 03:01:48,842
AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
Thread[ReadStage-2,5,main]: {}
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-1] 2019-08-06 03:01:50,351
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-5] 2019-08-06 03:02:06,061
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
WARN [ReadStage-8] 2019-08-06 03:02:07,616
AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
Thread[ReadStage-8,5,main]: {}
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-17] 2019-08-06 03:02:08,384
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-5] 2019-08-06 03:02:10,244
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
The nodetool version says 3.11.4 and the no of connections on 9042 was similar
to other nodes. The exceptions were scary that we had to call off the change.
Any help and insights to this problem from the community is appreciated.
> AbstractLocalAwareExecutorService Exception During Upgrade to 3.11.4
>
>
> Key: CASSANDRA-15172
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15172
> Project: Cassandra
> Issue Type: Bug
>Reporter: Shalom
>Priority: Normal
>
> Hi All,
> This is the first time I open an issue, so apologies if I'm not following the
> rules properly.
>
> After upgrading a node from version 2.1.21 to 3.11.4, we've started seeing a
> lot of AbstractLocalAwareExecutorService exceptions. This happened right
> after the node successfully started up with the new 3.11.4 binaries.
> INFO [main] 2019-06-05 04:41:37,730 Gossiper.java:1715 - No gossip backlog;
> proceeding
> INFO [main] 2019-06-05 04:41:38,036 NativeTr
[jira] [Comment Edited] (CASSANDRA-15172) AbstractLocalAwareExecutorService Exception During Upgrade to 3.11.4
[
https://issues.apache.org/jira/browse/CASSANDRA-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900581#comment-16900581
]
feroz shaik edited comment on CASSANDRA-15172 at 8/6/19 3:37 AM:
-
We have also hit this problem today while upgrading from 2.1.16 to 3.11.4/
we encountered this as soon as node started up with 3.11.4
WARN [ReadStage-4] 2019-08-06 02:57:57,408
AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
Thread[ReadStage-4,5,main]: {}
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-32] 2019-08-06 02:14:20,353
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
and the below errors continued in the logfile as long as the process was up.
ERROR [Native-Transport-Requests-12] 2019-08-06 03:00:47,135
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-8] 2019-08-06 03:00:48,778
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-13] 2019-08-06 03:00:57,454
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-11] 2019-08-06 03:00:57,482
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-2] 2019-08-06 03:00:58,543
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-8] 2019-08-06 03:00:58,899
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-17] 2019-08-06 03:00:59,074
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-12] 2019-08-06 03:01:08,123
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-17] 2019-08-06 03:01:19,055
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-4] 2019-08-06 03:01:20,880
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
WARN [ReadStage-13] 2019-08-06 03:01:29,983
AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
Thread[ReadStage-13,5,main]: {}
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-2] 2019-08-06 03:01:31,119
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-6] 2019-08-06 03:01:46,262
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-15] 2019-08-06 03:01:46,520
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
WARN [ReadStage-2] 2019-08-06 03:01:48,842
AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
Thread[ReadStage-2,5,main]: {}
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-1] 2019-08-06 03:01:50,351
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-5] 2019-08-06 03:02:06,061
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
WARN [ReadStage-8] 2019-08-06 03:02:07,616
AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
Thread[ReadStage-8,5,main]: {}
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-17] 2019-08-06 03:02:08,384
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [Native-Transport-Requests-5] 2019-08-06 03:02:10,244
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
The nodetool version says 3.11.4 and the no of connections on 9042 was similar
to other nodes. The exceptions were scary that we had to call off the change.
Any help and insights to this problem from the community is appreciated.
was (Author: ferozshaik...@gmail.com):
We have also hit this problem today while upgrading from 2.1.16 to 3.11.4/
we encountered this as soon as node started up with 3.11.4
ERROR [Native-Transport-Requests-32] 2019-08-06 02:14:20,353
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
and the below errors continued in the logfile as long as the process was up.
ERROR [Native-Transport-Requests-12] 2019-08-06 03:00:47,135
ErrorMessage.java:384 - Unexpected exception during request
java.lang.NullPointerException: null
ERROR [
