Can you post the output of nodetool status and your Astyanax connection
settings?
On Fri, Jul 11, 2014 at 9:06 PM, Ruchir Jha <ruchir....@gmail.com> wrote:
> This is how we create our keyspace. We just ran this command once through
> a cqlsh session on one of the nodes, so don't quite understand what you
> mean by "check that your DC names match up"
>
> CREATE KEYSPACE prod WITH replication = {
> 'class': 'NetworkTopologyStrategy',
> 'datacenter1': '3'
> };
>
>
>
> On Fri, Jul 11, 2014 at 3:48 PM, Chris Lohfink <clohf...@blackbirdit.com>
> wrote:
>
>> What replication strategy are you using? if using NetworkTopolgyStrategy
>> double check that your DC names match up (case sensitive)
>>
>> Chris
>>
>> On Jul 11, 2014, at 9:38 AM, Ruchir Jha <ruchir....@gmail.com> wrote:
>>
>> Here's the complete stack trace:
>>
>> com.netflix.astyanax.connectionpool.exceptions.TokenRangeOfflineException:
>> TokenRangeOfflineException:
>> [host=ny4lpcas5.fusionts.corp(10.10.20.47):9160, latency=22784(42874),
>> attempts=3]UnavailableException()
>> at
>> com.netflix.astyanax.thrift.ThriftConverter.ToConnectionPoolException(ThriftConverter.java:165)
>> at
>> com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:65)
>> at
>> com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:28)
>> at
>> com.netflix.astyanax.thrift.ThriftSyncConnectionFactoryImpl$ThriftConnection.execute(ThriftSyncConnectionFactoryImpl.java:151)
>> at
>> com.netflix.astyanax.connectionpool.impl.AbstractExecuteWithFailoverImpl.tryOperation(AbstractExecuteWithFailoverImpl.java:69)
>> at
>> com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:256)
>> at
>> com.netflix.astyanax.thrift.ThriftKeyspaceImpl.executeOperation(ThriftKeyspaceImpl.java:485)
>> at
>> com.netflix.astyanax.thrift.ThriftKeyspaceImpl.access$000(ThriftKeyspaceImpl.java:79)
>> at
>> com.netflix.astyanax.thrift.ThriftKeyspaceImpl$1.execute(ThriftKeyspaceImpl.java:123)
>> Caused by: UnavailableException()
>> at
>> org.apache.cassandra.thrift.Cassandra$batch_mutate_result.read(Cassandra.java:20841)
>> at
>> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
>> at
>> org.apache.cassandra.thrift.Cassandra$Client.recv_batch_mutate(Cassandra.java:964)
>> at
>> org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:950)
>> at
>> com.netflix.astyanax.thrift.ThriftKeyspaceImpl$1$1.internalExecute(ThriftKeyspaceImpl.java:129)
>> at
>> com.netflix.astyanax.thrift.ThriftKeyspaceImpl$1$1.internalExecute(ThriftKeyspaceImpl.java:126)
>> at
>> com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:60)
>> ... 12 more
>>
>>
>>
>> On Fri, Jul 11, 2014 at 9:11 AM, Prem Yadav <ipremya...@gmail.com> wrote:
>>
>>> Please post the full exception.
>>>
>>>
>>> On Fri, Jul 11, 2014 at 1:50 PM, Ruchir Jha <ruchir....@gmail.com>
>>> wrote:
>>>
>>>> We have a 12 node cluster and we are consistently seeing this exception
>>>> being thrown during peak write traffic. We have a replication factor of 3
>>>> and a write consistency level of QUORUM. Also note there is no unusual Or
>>>> Full GC activity during this time. Appreciate any help.
>>>>
>>>> Sent from my iPhone
>>>
>>>
>>>
>>
>>
>
