[jira] [Updated] (CASSANDRA-15437) Decommission fails with "Unable to stream hints since no live endpoints seen" even if no hints need to be sent
[
https://issues.apache.org/jira/browse/CASSANDRA-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-15437:
--
Bug Category: Parent values: Correctness(12982)
Complexity: Normal
Discovered By: Adhoc Test
Severity: Low
Assignee: Stefan Miklosovic
Status: Open (was: Triage Needed)
> Decommission fails with "Unable to stream hints since no live endpoints seen"
> even if no hints need to be sent
> --
>
> Key: CASSANDRA-15437
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15437
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Membership
>Reporter: YCozy
>Assignee: Stefan Miklosovic
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Dear Cassandra developers, I was applying fault-injection to test Cassandra
> and noticed the following behavior. I think this may be a bug. Please let me
> know if I'm missing something.
>
> Step to reproduce:
> # Start a two node cluster (node1 & node2) using {{ccm}}.
> # Add another node to the cluster (node3).
> # Partition node3 from the other two nodes.
> # Try to decommission node3 using {{nodetool decommission}}.
> # Notice that the decommission failed with the following error log:
>
> {code:java}
> ERROR [RMI TCP Connection(4)-127.0.0.1] 2019-11-25 22:45:27,716
> StorageService.java:4198 - Error while decommissioning node
> java.lang.RuntimeException: Unable to stream hints since no live endpoints
> seen
> at
> org.apache.cassandra.service.StorageService.getPreferredHintsStreamTarget(StorageService.java:4281)
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$TransferHintsTask.run(HintsDispatchExecutor.java:156)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748){code}
>
> Since I didn't write any data, there is no hint to be sent. In this case,
> shouldn't the decommission continue?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15437) Decommission fails with "Unable to stream hints since no live endpoints seen" even if no hints need to be sent
[
https://issues.apache.org/jira/browse/CASSANDRA-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CASSANDRA-15437:
---
Labels: pull-request-available (was: )
> Decommission fails with "Unable to stream hints since no live endpoints seen"
> even if no hints need to be sent
> --
>
> Key: CASSANDRA-15437
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15437
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Membership
>Reporter: YCozy
>Priority: Normal
> Labels: pull-request-available
>
> Dear Cassandra developers, I was applying fault-injection to test Cassandra
> and noticed the following behavior. I think this may be a bug. Please let me
> know if I'm missing something.
>
> Step to reproduce:
> # Start a two node cluster (node1 & node2) using {{ccm}}.
> # Add another node to the cluster (node3).
> # Partition node3 from the other two nodes.
> # Try to decommission node3 using {{nodetool decommission}}.
> # Notice that the decommission failed with the following error log:
>
> {code:java}
> ERROR [RMI TCP Connection(4)-127.0.0.1] 2019-11-25 22:45:27,716
> StorageService.java:4198 - Error while decommissioning node
> java.lang.RuntimeException: Unable to stream hints since no live endpoints
> seen
> at
> org.apache.cassandra.service.StorageService.getPreferredHintsStreamTarget(StorageService.java:4281)
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$TransferHintsTask.run(HintsDispatchExecutor.java:156)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748){code}
>
> Since I didn't write any data, there is no hint to be sent. In this case,
> shouldn't the decommission continue?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15437) Decommission fails with "Unable to stream hints since no live endpoints seen" even if no hints need to be sent
[
https://issues.apache.org/jira/browse/CASSANDRA-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
YCozy updated CASSANDRA-15437:
--
Description:
Dear Cassandra developers, I was applying fault-injection to test Cassandra and
noticed the following behavior. I think this may be a bug. Please let me know
if I'm missing something.
Step to reproduce:
# Start a two node cluster (node1 & node2) using {{ccm}}.
# Add another node to the cluster (node3).
# Partition node3 from the other two nodes.
# Try to decommission node3 using {{nodetool decommission}}.
# Notice that the decommission failed with the following error log:
{code:java}
ERROR [RMI TCP Connection(4)-127.0.0.1] 2019-11-25 22:45:27,716
StorageService.java:4198 - Error while decommissioning node
java.lang.RuntimeException: Unable to stream hints since no live endpoints seen
at
org.apache.cassandra.service.StorageService.getPreferredHintsStreamTarget(StorageService.java:4281)
at
org.apache.cassandra.hints.HintsDispatchExecutor$TransferHintsTask.run(HintsDispatchExecutor.java:156)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748){code}
Since I didn't write any data, there is no hint to be sent. In this case,
shouldn't the decommission continue?
was:
Dear Cassandra developers, I was applying fault-injection to test Cassandra and
noticed the following behavior. I think this may be a bug. Please let me know
if I'm missing something.
Step to reproduce:
# Start a two node cluster (node1 & node2) using {{ccm}}.
# Add another node to the cluster (node3).
# Partition node3 from the other two nodes.
# Try to decommission node3 using {{nodetool decommission}}.
# Notice that the decommission failed with the following error log:
{code:java}
ERROR [RMI TCP Connection(4)-127.0.0.1] 2019-11-25 22:45:27,716
StorageService.java:4198 - Error while decommissioning node
java.lang.RuntimeException: Unable to stream hints since no live endpoints seen
at
org.apache.cassandra.service.StorageService.getPreferredHintsStreamTarget(StorageService.java:4281)
at
org.apache.cassandra.hints.HintsDispatchExecutor$TransferHintsTask.run(HintsDispatchExecutor.java:156)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748){code}
Since I didn't write any data, there is no hint to be sent. In this case,
should the decommission continue?
> Decommission fails with "Unable to stream hints since no live endpoints seen"
> even if no hints need to be sent
> --
>
> Key: CASSANDRA-15437
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15437
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Membership
>Reporter: YCozy
>Priority: Normal
>
> Dear Cassandra developers, I was applying fault-injection to test Cassandra
> and noticed the following behavior. I think this may be a bug. Please let me
> know if I'm missing something.
>
> Step to reproduce:
> # Start a two node cluster (node1 & node2) using {{ccm}}.
> # Add another node to the cluster (node3).
> # Partition node3 from the other two nodes.
> # Try to decommission node3 using {{nodetool decommission}}.
> # Notice that the decommission failed with the following error log:
>
> {code:java}
> ERROR [RMI TCP Connection(4)-127.0.0.1] 2019-11-25 22:45:27,716
> StorageService.java:4198 - Error while decommissioning node
> java.lang.RuntimeException: Unable to stream hints since no live endpoints
> seen
> at
> org.apache.cassandra.service.StorageService.getPreferredHintsStreamTarget(StorageService.java:4281)
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$TransferHintsTask.run(HintsDispatchExecutor.java:156)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624
[jira] [Updated] (CASSANDRA-15437) Decommission fails with "Unable to stream hints since no live endpoints seen" even if no hints need to be sent
[
https://issues.apache.org/jira/browse/CASSANDRA-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
YCozy updated CASSANDRA-15437:
--
Description:
Dear Cassandra developers, I was applying fault-injection to test Cassandra and
noticed the following behavior. I think this may be a bug. Please let me know
if I'm missing something.
Step to reproduce:
# Start a two node cluster (node1 & node2) using {{ccm}}.
# Add another node to the cluster (node3).
# Partition node3 from the other two nodes.
# Try to decommission node3 using {{nodetool decommission}}.
# Notice that the decommission failed with the following error log:
{code:java}
ERROR [RMI TCP Connection(4)-127.0.0.1] 2019-11-25 22:45:27,716
StorageService.java:4198 - Error while decommissioning node
java.lang.RuntimeException: Unable to stream hints since no live endpoints seen
at
org.apache.cassandra.service.StorageService.getPreferredHintsStreamTarget(StorageService.java:4281)
at
org.apache.cassandra.hints.HintsDispatchExecutor$TransferHintsTask.run(HintsDispatchExecutor.java:156)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748){code}
Since I didn't write any data, there is no hint to be sent. In this case,
should the decommission continue?
was:
Dear Cassandra developers, I was applying fault-injection to test Cassandra and
noticed the following behavior. I think this may be a bug. Please let me know
if I'm missing something.
Step to reproduce:
# Start a two node cluster (node1 & node2) using {{ccm}}.
# Add another node to the cluster (node3).
# Partition node3 from the other two nodes.
# Try to decommission node3 using {{nodetool decommission}}.
# Notice that the decommission failed with the following error log:
```
ERROR [RMI TCP Connection(4)-127.0.0.1] 2019-11-25 22:45:27,716
StorageService.java:4198 - Error while decommissioning node
{{java.lang.RuntimeException: Unable to stream hints since no live endpoints
seen}}
at
org.apache.cassandra.service.StorageService.getPreferredHintsStreamTarget(StorageService.java:4281)
at
org.apache.cassandra.hints.HintsDispatchExecutor$TransferHintsTask.run(HintsDispatchExecutor.java:156)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
```
Since I didn't write any data, there is no hint to be sent. In this case,
should the decommission continue?
> Decommission fails with "Unable to stream hints since no live endpoints seen"
> even if no hints need to be sent
> --
>
> Key: CASSANDRA-15437
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15437
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Membership
>Reporter: YCozy
>Priority: Normal
>
> Dear Cassandra developers, I was applying fault-injection to test Cassandra
> and noticed the following behavior. I think this may be a bug. Please let me
> know if I'm missing something.
>
> Step to reproduce:
> # Start a two node cluster (node1 & node2) using {{ccm}}.
> # Add another node to the cluster (node3).
> # Partition node3 from the other two nodes.
> # Try to decommission node3 using {{nodetool decommission}}.
> # Notice that the decommission failed with the following error log:
>
> {code:java}
> ERROR [RMI TCP Connection(4)-127.0.0.1] 2019-11-25 22:45:27,716
> StorageService.java:4198 - Error while decommissioning node
> java.lang.RuntimeException: Unable to stream hints since no live endpoints
> seen
> at
> org.apache.cassandra.service.StorageService.getPreferredHintsStreamTarget(StorageService.java:4281)
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$TransferHintsTask.run(HintsDispatchExecutor.java:156)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> io.net
[jira] [Updated] (CASSANDRA-15437) Decommission fails with "Unable to stream hints since no live endpoints seen" even if no hints need to be sent
[
https://issues.apache.org/jira/browse/CASSANDRA-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
YCozy updated CASSANDRA-15437:
--
Description:
Dear Cassandra developers, I was applying fault-injection to test Cassandra and
noticed the following behavior. I think this may be a bug. Please let me know
if I'm missing something.
Step to reproduce:
# Start a two node cluster (node1 & node2) using {{ccm}}.
# Add another node to the cluster (node3).
# Partition node3 from the other two nodes.
# Try to decommission node3 using {{nodetool decommission}}.
# Notice that the decommission failed with the following error log:
```
ERROR [RMI TCP Connection(4)-127.0.0.1] 2019-11-25 22:45:27,716
StorageService.java:4198 - Error while decommissioning node
{{java.lang.RuntimeException: Unable to stream hints since no live endpoints
seen}}
at
org.apache.cassandra.service.StorageService.getPreferredHintsStreamTarget(StorageService.java:4281)
at
org.apache.cassandra.hints.HintsDispatchExecutor$TransferHintsTask.run(HintsDispatchExecutor.java:156)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
```
Since I didn't write any data, there is no hint to be sent. In this case,
should the decommission continue?
was:
Dear Cassandra developers, I was applying fault-injection to test Cassandra and
noticed the following behavior. I think this may be a bug. Please let me know
if I'm missing something.
Step to reproduce:
# Start a two node cluster (node1 & node2) using {{ccm}}.
# Add another node to the cluster (node3).
# Partition node3 from the other two nodes.
# Try to decommission node3 using {{nodetool decommission}}.
# Notice that the decommission failed with the following error log:
{{ERROR [RMI TCP Connection(4)-127.0.0.1] 2019-11-25 22:45:27,716
StorageService.java:4198 - Error while decommissioning node }}
{{java.lang.RuntimeException: Unable to stream hints since no live endpoints
seen}}
{{ at
org.apache.cassandra.service.StorageService.getPreferredHintsStreamTarget(StorageService.java:4281)}}
{{ at
org.apache.cassandra.hints.HintsDispatchExecutor$TransferHintsTask.run(HintsDispatchExecutor.java:156)}}
{{ at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
{{ at java.util.concurrent.FutureTask.run(FutureTask.java:266)}}
{{ at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
{{ at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
{{ at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)}}
{{ at java.lang.Thread.run(Thread.java:748)}}
Since I didn't write any data, there is no hint to be sent. In this case,
should the decommission continue?
> Decommission fails with "Unable to stream hints since no live endpoints seen"
> even if no hints need to be sent
> --
>
> Key: CASSANDRA-15437
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15437
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Membership
>Reporter: YCozy
>Priority: Normal
>
> Dear Cassandra developers, I was applying fault-injection to test Cassandra
> and noticed the following behavior. I think this may be a bug. Please let me
> know if I'm missing something.
>
> Step to reproduce:
> # Start a two node cluster (node1 & node2) using {{ccm}}.
> # Add another node to the cluster (node3).
> # Partition node3 from the other two nodes.
> # Try to decommission node3 using {{nodetool decommission}}.
> # Notice that the decommission failed with the following error log:
> ```
> ERROR [RMI TCP Connection(4)-127.0.0.1] 2019-11-25 22:45:27,716
> StorageService.java:4198 - Error while decommissioning node
> {{java.lang.RuntimeException: Unable to stream hints since no live endpoints
> seen}}
> at
> org.apache.cassandra.service.StorageService.getPreferredHintsStreamTarget(StorageService.java:4281)
> at
> org.apache.cassandra.hints.HintsDispatchExecutor$TransferHintsTask.run(HintsDispatchExecutor.java:156)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> io.netty
