Gaurav Sharma created CASSANDRA-11827:
-----------------------------------------
Summary: Node is not getting back to UN
Key: CASSANDRA-11827
URL: https://issues.apache.org/jira/browse/CASSANDRA-11827
Project: Cassandra
Issue Type: Task
Components: Configuration, Observability
Environment: Ubuntu 12.04.5 LTS
Reporter: Gaurav Sharma
Fix For: 2.1.x
Hi ,
One of node went down due to high CPU/Memory utilization , now when we are
trying to start it again , it is not joining cluster . On other nodes in
system.log we can see below error ::
INFO [HANDSHAKE-/172.16.1.71] 2016-05-18 15:21:27,011
OutboundTcpConnection.java:485 - Handshaking version with /172.16.1.71
DEBUG [MessagingService-Outgoing-/172.16.1.71] 2016-05-18 15:21:32,011
OutboundTcpConnection.java:412 - Target max version is -2147483648; no version
information yet, will retry
INFO [MessagingService-Outgoing-/172.16.1.71] 2016-05-18 15:21:32,011
YamlConfigurationLoader.java:92 - Loading settings from
file:/etc/cassandra/cassandra.yaml
INFO [MessagingService-Outgoing-/172.16.1.71] 2016-05-18 15:21:32,013
YamlConfigurationLoader.java:135 - Node
configuration:[authenticator=AllowAllAuthenticator; authorizer=AllowAl
lAuthorizer; auto_snapshot=false; batch_size_warn_threshold_in_kb=5;
batchlog_replay_throttle_in_kb=1024; broadcast_rpc_address=172.16.1.73;
cas_contention_timeout_in_ms=5000; clien
t_encryption_options=<REDACTED>; cluster_name=wynkframework;
column_index_size_in_kb=64; commit_failure_policy=stop;
commitlog_directory=/mnt/cassandra/commitlog; commitlog_segment_
size_in_mb=32; commitlog_sync=periodic; commitlog_sync_period_in_ms=10000;
compaction_large_partition_warning_threshold_mb=100;
compaction_throughput_mb_per_sec=32; concurrent_compa
ctors=4; concurrent_counter_writes=48; concurrent_reads=48;
concurrent_writes=32; counter_cache_save_period=7200;
counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=
10000; cross_node_timeout=true; data_file_directories=[/var/lib/cassandra/data,
/data/cassandra, /data1/cassandra]; disk_failure_policy=stop;
dynamic_snitch_badness_threshold=0.1; d
ynamic_snitch_reset_interval_in_ms=600000;
dynamic_snitch_update_interval_in_ms=100; endpoint_snitch=SimpleSnitch;
hinted_handoff_enabled=false; hinted_handoff_throttle_in_kb=1024;
incremental_backups=false; index_summary_capacity_in_mb=null;
index_summary_resize_interval_in_minutes=60; inter_dc_tcp_nodelay=false;
internode_compression=all; key_cache_save_peri
od=14400; key_cache_size_in_mb=1024; listen_address=172.16.1.73;
max_hint_window_in_ms=10800000; max_hints_delivery_threads=4;
memory_allocator=JEMallocAllocator; memtable_allocatio
n_type=offheap_objects; memtable_cleanup_threshold=0.11;
memtable_flush_writers=4; native_transport_max_concurrent_connections=100;
native_transport_port=9042; num_tokens=256; parti
tioner=org.apache.cassandra.dht.Murmur3Partitioner;
permissions_validity_in_ms=2000; range_request_timeout_in_ms=30000;
read_request_timeout_in_ms=20000; request_scheduler=org.apach
e.cassandra.scheduler.NoScheduler; request_timeout_in_ms=30000;
row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=172.16.1.73;
rpc_keepalive=true; rpc_max_threads=2048; r
pc_min_threads=16; rpc_port=9160; rpc_server_type=sync;
saved_caches_directory=/mnt/cassandra/saved_caches;
seed_provider=[{class_name=org.apache.cassandra.locator.SimpleSeedProvide
r, parameters=[{seeds=172.16.1.103,172.16.1.73}]}];
server_encryption_options=<REDACTED>; snapshot_before_compaction=false;
ssl_storage_port=7001; sstable_preemptive_open_interval_i
n_mb=50; start_native_transport=true; start_rpc=true; storage_port=7000;
thrift_framed_transport_size_in_mb=15; tombstone_failure_threshold=100000;
tombstone_warn_threshold=1000; tr
ickle_fsync=false; trickle_fsync_interval_in_kb=10240;
truncate_request_timeout_in_ms=60000; write_request_timeout_in_ms=30000]
TRACE [HANDSHAKE-/172.16.1.71] 2016-05-18 15:21:32,015
OutboundTcpConnection.java:492 - Cannot handshake version with /172.16.1.71
java.nio.channels.AsynchronousCloseException: null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
