Re: Understanding Messages in the Debug.log

2017-09-22 Thread Alex Kotelnikov
I see a lot of that on one of our clusters. There is pretty much no load on it. Clocks are quite well synchronized. date +start:%H:%M:%S.%N ; for d in 2 3 4 5 6 8 9; do ssh 10.3.0.17$d date +%H:%M:%S.%N & done;date +end:%H:%M:%S.%N start:01:22:45.478726356 end:01:22:45.480993049 01:22:45.570313

Re: Understanding Messages in the Debug.log

2017-09-21 Thread Nate McCall
> > > The message in the debug log is > > DEBUG [GossipStage:1] 2017-09-21 09:19:52,627 FailureDetector.java:456 - > Ignoring interval time of 2000275419 > > > Did you truncate the log message? There should be and "for [endpoint]" on the end which should help you narrow things down to a set of prob

RE: Understanding Messages in the Debug.log

2017-09-21 Thread Jonathan Baynes
Messages in the Debug.log Could also be a network hiccup, an NTP problem, a disk pause, or anything that causes a long gc pause including misconfigured heap. Are they all on the same machine? Or all instances in the cluster? If there are so many you need to root cause it, you probably want to look for

Re: Understanding Messages in the Debug.log

2017-09-21 Thread Jeff Jirsa
n it, only system databases, what on earth could > be causing load on the cluster to cause the error message?? > > > > From: Jacob Shadix [mailto:jacobsha...@gmail.com] > Sent: 21 September 2017 12:34 > To: user@cassandra.apache.org > Subject: Re: Understanding Me

Re: Understanding Messages in the Debug.log

2017-09-21 Thread Milan Milosevic
no data, a Keyspace with a > table designed but no data in it, only system databases, what on earth > could be causing load on the cluster to cause the error message?? > > > > > > > > *From:* Jacob Shadix [mailto:jacobsha...@gmail.com] > *Sent:* 21 September 2017 1

RE: Understanding Messages in the Debug.log

2017-09-21 Thread Jonathan Baynes
, what on earth could be causing load on the cluster to cause the error message?? From: Jacob Shadix [mailto:jacobsha...@gmail.com] Sent: 21 September 2017 12:34 To: user@cassandra.apache.org Subject: Re: Understanding Messages in the Debug.log check out this thread - https://www.mail

Re: Understanding Messages in the Debug.log

2017-09-21 Thread Jacob Shadix
check out this thread - https://www.mail-archive.com/user@cassandra.apache.org/msg52313.html -- Jacob Shadix On Thu, Sep 21, 2017 at 4:56 AM, Jonathan Baynes < jonathan.bay...@tradeweb.com> wrote: > Good Morning Community, > > > > Can anyone help me understand what these messages (and I have a l

Understanding Messages in the Debug.log

2017-09-21 Thread Jonathan Baynes
Good Morning Community, Can anyone help me understand what these messages (and I have a lot of them in the log) mean, they must be a symptom of something but I am unsure how to starting working this out. I'm using C* 3.0.11, CQL 5.0.1. I have 6 node ring over 2 DC's (3 in each DC) each of the