[jira] [Commented] (KUDU-1521) Flakiness in TestAsyncKuduSession
[
https://issues.apache.org/jira/browse/KUDU-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16854210#comment-16854210
]
Alexey Serbin commented on KUDU-1521:
-
{noformat}
There was 1 failure:
1) test(org.apache.kudu.client.TestAsyncKuduSession)
java.lang.AssertionError: Expected PleaseThrottleException
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.kudu.client.TestAsyncKuduSession.test(TestAsyncKuduSession.java:386)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
{noformat}
> Flakiness in TestAsyncKuduSession
> -
>
> Key: KUDU-1521
> URL: https://issues.apache.org/jira/browse/KUDU-1521
> Project: Kudu
> Issue Type: Bug
> Components: client
>Affects Versions: 0.9.1
>Reporter: Adar Dembo
>Assignee: Will Berkeley
>Priority: Major
> Attachments:
> org.apache.kudu.client.TestAsyncKuduSession-TableIsDeleted-output.txt,
> org.apache.kudu.client.TestAsyncKuduSession-output.txt,
> org.apache.kudu.client.TestAsyncKuduSession.test.log.xz
>
>
> I've been trying to parse the various failures in
> http://104.196.14.100/job/kudu-gerrit/2270/BUILD_TYPE=RELEASE. Here's what I
> see in the test:
> The way test() tests AUTO_FLUSH_BACKGROUND is inherently flaky; a delay while
> running test code will give the background flush task a chance to fire when
> the test code doesn't expect it. I've seen this cause lead to no
> PleaseThrottleException, but I suspect the first block of test code dealing
> with background flushes is flaky too (since it's testing elapsed time).
> There's also some test failures that I can't figure out. I've pasted them
> below for posterity:
> {noformat}
> 03:52:14
> testGetTableLocationsErrorCauseSessionStuck(org.kududb.client.TestAsyncKuduSession)
> Time elapsed: 100.009 sec <<< ERROR!
> 03:52:14 java.lang.Exception: test timed out after 10 milliseconds
> 03:52:14 at java.lang.Object.wait(Native Method)
> 03:52:14 at java.lang.Object.wait(Object.java:503)
> 03:52:14 at com.stumbleupon.async.Deferred.doJoin(Deferred.java:1136)
> 03:52:14 at com.stumbleupon.async.Deferred.join(Deferred.java:1019)
> 03:52:14 at
> org.kududb.client.TestAsyncKuduSession.testGetTableLocationsErrorCauseSessionStuck(TestAsyncKuduSession.java:133)
> 03:52:14
> 03:52:14
> testBatchErrorCauseSessionStuck(org.kududb.client.TestAsyncKuduSession) Time
> elapsed: 0.199 sec <<< ERROR!
> 03:52:14 org.kududb.client.MasterErrorException: Server[Kudu Master -
> 127.13.215.1:64030] NOT_FOUND[code 1]: The table was deleted: Table deleted
> at 2016-07-09 03:50:24 UTC
> 03:52:14 at
> org.kududb.client.TabletClient.dispatchMasterErrorOrReturnException(TabletClient.java:533)
> 03:52:14 at org.kududb.client.TabletClient.decode(TabletClient.java:463)
> 03:52:14 at org.kududb.client.TabletClient.decode(TabletClient.java:83)
> 03:52:14 at
> org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500)
> 03:52:14 at
> org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
> 03:52:14 at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> 03:52:14 at
> org.kududb.client.TabletClient.handleUpstream(TabletClient.java:638)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> 03:52:14 at
> org.jboss.netty.handler.timeout.ReadTimeoutHandler.messageReceived(ReadTimeoutHandler.java:184)
> 03:52:14 at
> org.jboss.netty.channel.SimpleChannelUps
[jira] [Commented] (KUDU-1521) Flakiness in TestAsyncKuduSession
[
https://issues.apache.org/jira/browse/KUDU-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16627870#comment-16627870
]
Mike Percy commented on KUDU-1521:
--
I also observed a case where it failed the part of the test where it expected
the PleaseThrottleException but it never appeared:
{code:java}
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.kudu.client.TestAsyncKuduSession.test(TestAsyncKuduSession.java:452)
{code}
Sounds like the same issue.
> Flakiness in TestAsyncKuduSession
> -
>
> Key: KUDU-1521
> URL: https://issues.apache.org/jira/browse/KUDU-1521
> Project: Kudu
> Issue Type: Bug
> Components: client
>Affects Versions: 0.9.1
>Reporter: Adar Dembo
>Assignee: Todd Lipcon
>Priority: Major
> Attachments:
> org.apache.kudu.client.TestAsyncKuduSession-TableIsDeleted-output.txt,
> org.apache.kudu.client.TestAsyncKuduSession-output.txt,
> org.apache.kudu.client.TestAsyncKuduSession.test.log.xz
>
>
> I've been trying to parse the various failures in
> http://104.196.14.100/job/kudu-gerrit/2270/BUILD_TYPE=RELEASE. Here's what I
> see in the test:
> The way test() tests AUTO_FLUSH_BACKGROUND is inherently flaky; a delay while
> running test code will give the background flush task a chance to fire when
> the test code doesn't expect it. I've seen this cause lead to no
> PleaseThrottleException, but I suspect the first block of test code dealing
> with background flushes is flaky too (since it's testing elapsed time).
> There's also some test failures that I can't figure out. I've pasted them
> below for posterity:
> {noformat}
> 03:52:14
> testGetTableLocationsErrorCauseSessionStuck(org.kududb.client.TestAsyncKuduSession)
> Time elapsed: 100.009 sec <<< ERROR!
> 03:52:14 java.lang.Exception: test timed out after 10 milliseconds
> 03:52:14 at java.lang.Object.wait(Native Method)
> 03:52:14 at java.lang.Object.wait(Object.java:503)
> 03:52:14 at com.stumbleupon.async.Deferred.doJoin(Deferred.java:1136)
> 03:52:14 at com.stumbleupon.async.Deferred.join(Deferred.java:1019)
> 03:52:14 at
> org.kududb.client.TestAsyncKuduSession.testGetTableLocationsErrorCauseSessionStuck(TestAsyncKuduSession.java:133)
> 03:52:14
> 03:52:14
> testBatchErrorCauseSessionStuck(org.kududb.client.TestAsyncKuduSession) Time
> elapsed: 0.199 sec <<< ERROR!
> 03:52:14 org.kududb.client.MasterErrorException: Server[Kudu Master -
> 127.13.215.1:64030] NOT_FOUND[code 1]: The table was deleted: Table deleted
> at 2016-07-09 03:50:24 UTC
> 03:52:14 at
> org.kududb.client.TabletClient.dispatchMasterErrorOrReturnException(TabletClient.java:533)
> 03:52:14 at org.kududb.client.TabletClient.decode(TabletClient.java:463)
> 03:52:14 at org.kududb.client.TabletClient.decode(TabletClient.java:83)
> 03:52:14 at
> org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500)
> 03:52:14 at
> org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
> 03:52:14 at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> 03:52:14 at
> org.kududb.client.TabletClient.handleUpstream(TabletClient.java:638)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> 03:52:14 at
> org.jboss.netty.handler.timeout.ReadTimeoutHandler.messageReceived(ReadTimeoutHandler.java:184)
> 03:52:14 at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> 03:52:14 at
> org.kududb.client.AsyncKuduClient$TabletClientPipeline.sendUpstream(AsyncKuduClient.java:1877)
> 03:52:14 at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> 03:52:14 at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
> 03:52:14 at
> org.jboss.
[jira] [Commented] (KUDU-1521) Flakiness in TestAsyncKuduSession
[
https://issues.apache.org/jira/browse/KUDU-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16626763#comment-16626763
]
Alexey Serbin commented on KUDU-1521:
-
Still flaky, with 3 attempts failed in a row. The first attempt failed with:
{noformat}
com.stumbleupon.async.TimeoutException: Timed out after 5ms when joining
Deferred@147093588(state=PENDING, result=null, callback=wakeup thread
Time-limited test, errback=wakeup thread Time-limited test)
at com.stumbleupon.async.Deferred.doJoin(Deferred.java:1183)
at com.stumbleupon.async.Deferred.join(Deferred.java:1042)
at
org.apache.kudu.client.TestAsyncKuduSession.test(TestAsyncKuduSession.java:299)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:745)
[^org.apache.kudu.client.TestAsyncKuduSession.test.log.xz]
{noformat}
I attached full log of the failed scenario (the first failed run out of three
failed).
> Flakiness in TestAsyncKuduSession
> -
>
> Key: KUDU-1521
> URL: https://issues.apache.org/jira/browse/KUDU-1521
> Project: Kudu
> Issue Type: Bug
> Components: client
>Affects Versions: 0.9.1
>Reporter: Adar Dembo
>Assignee: Todd Lipcon
>Priority: Major
> Attachments:
> org.apache.kudu.client.TestAsyncKuduSession-TableIsDeleted-output.txt,
> org.apache.kudu.client.TestAsyncKuduSession-output.txt,
> org.apache.kudu.client.TestAsyncKuduSession.test.log.xz
>
>
> I've been trying to parse the various failures in
> http://104.196.14.100/job/kudu-gerrit/2270/BUILD_TYPE=RELEASE. Here's what I
> see in the test:
> The way test() tests AUTO_FLUSH_BACKGROUND is inherently flaky; a delay while
> running test code will give the background flush task a chance to fire when
> the test code doesn't expect it. I've seen this cause lead to no
> PleaseThrottleException, but I suspect the first block of test code dealing
> with background flushes is flaky too (since it's testing elapsed time).
> There's also some test failures that I can't figure out. I've pasted them
> below for posterity:
> {noformat}
> 03:52:14
> testGetTableLocationsErrorCauseSessionStuck(org.kududb.client.TestAsyncKuduSession)
> Time elapsed: 100.009 sec <<< ERROR!
> 03:52:14 java.lang.Exception: test timed out after 10 milliseconds
> 03:52:14 at java.lang.Object.wait(Native Method)
> 03:52:14 at java.lang.Object.wait(Object.java:503)
> 03:52:14 at com.stumbleupon.async.Deferred.doJoin(Deferred.java:1136)
> 03:52:14 at com.stumbleupon.async.Deferred.join(Deferred.java:1019)
> 03:52:14 at
> org.kududb.client.TestAsyncKuduSession.testGetTableLocationsErrorCauseSessionStuck(TestAsyncKuduSession.java:133)
> 03:52:14
> 03:52:14
> testBatchErrorCauseSessionStuck(org.kududb.client.TestAsyncKuduSession) Time
> elapsed: 0.199 sec <<< ERROR!
> 03:52:14 org.kududb.client.MasterErrorException: Server[Kudu Master -
> 127.13.215.1:64030] NOT_FOUND[code 1]: The table was deleted: Table deleted
> at 2016-07-09 03:50:24 UTC
> 03:52:14 at
> org.kududb.client.TabletClient.dispatchMasterErrorOrReturnException(TabletClient.java:533)
> 03:52:14 at org.kududb.client.TabletClient.decode(TabletClient.java:463)
> 03:52:14 at org.kududb.client.TabletClient.decode(TabletClient.java:83)
> 03:52:14 at
> org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500)
> 03:52:14 at
> org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
> 03:52:14 at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> 03:52:14 at
> org.kududb.client.TabletClient.handleUpstream(TabletClient.java:638)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline
[jira] [Commented] (KUDU-1521) Flakiness in TestAsyncKuduSession
[
https://issues.apache.org/jira/browse/KUDU-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417989#comment-16417989
]
Todd Lipcon commented on KUDU-1521:
---
Seems like the "The table was deleted: Table deleted" case is due to the rerun
not working properly, and the root cause is the "expected:
PleaseThrottleException" that Adar mentioned.
> Flakiness in TestAsyncKuduSession
> -
>
> Key: KUDU-1521
> URL: https://issues.apache.org/jira/browse/KUDU-1521
> Project: Kudu
> Issue Type: Bug
> Components: client
>Affects Versions: 0.9.1
>Reporter: Adar Dembo
>Assignee: Andrew Wong
>Priority: Major
> Attachments:
> org.apache.kudu.client.TestAsyncKuduSession-TableIsDeleted-output.txt,
> org.apache.kudu.client.TestAsyncKuduSession-output.txt
>
>
> I've been trying to parse the various failures in
> http://104.196.14.100/job/kudu-gerrit/2270/BUILD_TYPE=RELEASE. Here's what I
> see in the test:
> The way test() tests AUTO_FLUSH_BACKGROUND is inherently flaky; a delay while
> running test code will give the background flush task a chance to fire when
> the test code doesn't expect it. I've seen this cause lead to no
> PleaseThrottleException, but I suspect the first block of test code dealing
> with background flushes is flaky too (since it's testing elapsed time).
> There's also some test failures that I can't figure out. I've pasted them
> below for posterity:
> {noformat}
> 03:52:14
> testGetTableLocationsErrorCauseSessionStuck(org.kududb.client.TestAsyncKuduSession)
> Time elapsed: 100.009 sec <<< ERROR!
> 03:52:14 java.lang.Exception: test timed out after 10 milliseconds
> 03:52:14 at java.lang.Object.wait(Native Method)
> 03:52:14 at java.lang.Object.wait(Object.java:503)
> 03:52:14 at com.stumbleupon.async.Deferred.doJoin(Deferred.java:1136)
> 03:52:14 at com.stumbleupon.async.Deferred.join(Deferred.java:1019)
> 03:52:14 at
> org.kududb.client.TestAsyncKuduSession.testGetTableLocationsErrorCauseSessionStuck(TestAsyncKuduSession.java:133)
> 03:52:14
> 03:52:14
> testBatchErrorCauseSessionStuck(org.kududb.client.TestAsyncKuduSession) Time
> elapsed: 0.199 sec <<< ERROR!
> 03:52:14 org.kududb.client.MasterErrorException: Server[Kudu Master -
> 127.13.215.1:64030] NOT_FOUND[code 1]: The table was deleted: Table deleted
> at 2016-07-09 03:50:24 UTC
> 03:52:14 at
> org.kududb.client.TabletClient.dispatchMasterErrorOrReturnException(TabletClient.java:533)
> 03:52:14 at org.kududb.client.TabletClient.decode(TabletClient.java:463)
> 03:52:14 at org.kududb.client.TabletClient.decode(TabletClient.java:83)
> 03:52:14 at
> org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500)
> 03:52:14 at
> org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
> 03:52:14 at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> 03:52:14 at
> org.kududb.client.TabletClient.handleUpstream(TabletClient.java:638)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> 03:52:14 at
> org.jboss.netty.handler.timeout.ReadTimeoutHandler.messageReceived(ReadTimeoutHandler.java:184)
> 03:52:14 at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> 03:52:14 at
> org.kududb.client.AsyncKuduClient$TabletClientPipeline.sendUpstream(AsyncKuduClient.java:1877)
> 03:52:14 at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> 03:52:14 at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> 03:52:14 at
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> 03:52:14 at
> org.jboss.netty.util.internal.DeadLoc
[jira] [Commented] (KUDU-1521) Flakiness in TestAsyncKuduSession
[
https://issues.apache.org/jira/browse/KUDU-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417894#comment-16417894
]
Todd Lipcon commented on KUDU-1521:
---
Still flaky as of today
> Flakiness in TestAsyncKuduSession
> -
>
> Key: KUDU-1521
> URL: https://issues.apache.org/jira/browse/KUDU-1521
> Project: Kudu
> Issue Type: Bug
> Components: client
>Affects Versions: 0.9.1
>Reporter: Adar Dembo
>Assignee: Andrew Wong
>Priority: Major
> Attachments:
> org.apache.kudu.client.TestAsyncKuduSession-TableIsDeleted-output.txt,
> org.apache.kudu.client.TestAsyncKuduSession-output.txt
>
>
> I've been trying to parse the various failures in
> http://104.196.14.100/job/kudu-gerrit/2270/BUILD_TYPE=RELEASE. Here's what I
> see in the test:
> The way test() tests AUTO_FLUSH_BACKGROUND is inherently flaky; a delay while
> running test code will give the background flush task a chance to fire when
> the test code doesn't expect it. I've seen this cause lead to no
> PleaseThrottleException, but I suspect the first block of test code dealing
> with background flushes is flaky too (since it's testing elapsed time).
> There's also some test failures that I can't figure out. I've pasted them
> below for posterity:
> {noformat}
> 03:52:14
> testGetTableLocationsErrorCauseSessionStuck(org.kududb.client.TestAsyncKuduSession)
> Time elapsed: 100.009 sec <<< ERROR!
> 03:52:14 java.lang.Exception: test timed out after 10 milliseconds
> 03:52:14 at java.lang.Object.wait(Native Method)
> 03:52:14 at java.lang.Object.wait(Object.java:503)
> 03:52:14 at com.stumbleupon.async.Deferred.doJoin(Deferred.java:1136)
> 03:52:14 at com.stumbleupon.async.Deferred.join(Deferred.java:1019)
> 03:52:14 at
> org.kududb.client.TestAsyncKuduSession.testGetTableLocationsErrorCauseSessionStuck(TestAsyncKuduSession.java:133)
> 03:52:14
> 03:52:14
> testBatchErrorCauseSessionStuck(org.kududb.client.TestAsyncKuduSession) Time
> elapsed: 0.199 sec <<< ERROR!
> 03:52:14 org.kududb.client.MasterErrorException: Server[Kudu Master -
> 127.13.215.1:64030] NOT_FOUND[code 1]: The table was deleted: Table deleted
> at 2016-07-09 03:50:24 UTC
> 03:52:14 at
> org.kududb.client.TabletClient.dispatchMasterErrorOrReturnException(TabletClient.java:533)
> 03:52:14 at org.kududb.client.TabletClient.decode(TabletClient.java:463)
> 03:52:14 at org.kududb.client.TabletClient.decode(TabletClient.java:83)
> 03:52:14 at
> org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500)
> 03:52:14 at
> org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
> 03:52:14 at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> 03:52:14 at
> org.kududb.client.TabletClient.handleUpstream(TabletClient.java:638)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> 03:52:14 at
> org.jboss.netty.handler.timeout.ReadTimeoutHandler.messageReceived(ReadTimeoutHandler.java:184)
> 03:52:14 at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> 03:52:14 at
> org.kududb.client.AsyncKuduClient$TabletClientPipeline.sendUpstream(AsyncKuduClient.java:1877)
> 03:52:14 at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> 03:52:14 at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> 03:52:14 at
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> 03:52:14 at
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> 03:52:14 at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 03:52:14
[jira] [Commented] (KUDU-1521) Flakiness in TestAsyncKuduSession
[
https://issues.apache.org/jira/browse/KUDU-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244616#comment-16244616
]
Todd Lipcon commented on KUDU-1521:
---
Still flaky as of 11/8
> Flakiness in TestAsyncKuduSession
> -
>
> Key: KUDU-1521
> URL: https://issues.apache.org/jira/browse/KUDU-1521
> Project: Kudu
> Issue Type: Bug
> Components: client
>Affects Versions: 0.9.1
>Reporter: Adar Dembo
> Attachments:
> org.apache.kudu.client.TestAsyncKuduSession-TableIsDeleted-output.txt,
> org.apache.kudu.client.TestAsyncKuduSession-output.txt
>
>
> I've been trying to parse the various failures in
> http://104.196.14.100/job/kudu-gerrit/2270/BUILD_TYPE=RELEASE. Here's what I
> see in the test:
> The way test() tests AUTO_FLUSH_BACKGROUND is inherently flaky; a delay while
> running test code will give the background flush task a chance to fire when
> the test code doesn't expect it. I've seen this cause lead to no
> PleaseThrottleException, but I suspect the first block of test code dealing
> with background flushes is flaky too (since it's testing elapsed time).
> There's also some test failures that I can't figure out. I've pasted them
> below for posterity:
> {noformat}
> 03:52:14
> testGetTableLocationsErrorCauseSessionStuck(org.kududb.client.TestAsyncKuduSession)
> Time elapsed: 100.009 sec <<< ERROR!
> 03:52:14 java.lang.Exception: test timed out after 10 milliseconds
> 03:52:14 at java.lang.Object.wait(Native Method)
> 03:52:14 at java.lang.Object.wait(Object.java:503)
> 03:52:14 at com.stumbleupon.async.Deferred.doJoin(Deferred.java:1136)
> 03:52:14 at com.stumbleupon.async.Deferred.join(Deferred.java:1019)
> 03:52:14 at
> org.kududb.client.TestAsyncKuduSession.testGetTableLocationsErrorCauseSessionStuck(TestAsyncKuduSession.java:133)
> 03:52:14
> 03:52:14
> testBatchErrorCauseSessionStuck(org.kududb.client.TestAsyncKuduSession) Time
> elapsed: 0.199 sec <<< ERROR!
> 03:52:14 org.kududb.client.MasterErrorException: Server[Kudu Master -
> 127.13.215.1:64030] NOT_FOUND[code 1]: The table was deleted: Table deleted
> at 2016-07-09 03:50:24 UTC
> 03:52:14 at
> org.kududb.client.TabletClient.dispatchMasterErrorOrReturnException(TabletClient.java:533)
> 03:52:14 at org.kududb.client.TabletClient.decode(TabletClient.java:463)
> 03:52:14 at org.kududb.client.TabletClient.decode(TabletClient.java:83)
> 03:52:14 at
> org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500)
> 03:52:14 at
> org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
> 03:52:14 at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> 03:52:14 at
> org.kududb.client.TabletClient.handleUpstream(TabletClient.java:638)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> 03:52:14 at
> org.jboss.netty.handler.timeout.ReadTimeoutHandler.messageReceived(ReadTimeoutHandler.java:184)
> 03:52:14 at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> 03:52:14 at
> org.kududb.client.AsyncKuduClient$TabletClientPipeline.sendUpstream(AsyncKuduClient.java:1877)
> 03:52:14 at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> 03:52:14 at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> 03:52:14 at
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> 03:52:14 at
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> 03:52:14 at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 03:52:14 at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(Thre
[jira] [Commented] (KUDU-1521) Flakiness in TestAsyncKuduSession
[
https://issues.apache.org/jira/browse/KUDU-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173712#comment-16173712
]
Alexey Serbin commented on KUDU-1521:
-
Just another error path is outlined below:
{noformat}
18:16:39.348 [DEBUG - New I/O worker #69] (AsyncKuduClient.java:950) Notify RPC
Batch{operations=1, tablet="5ceb1b36c63b4045be78db7318693bc3" [, ),
ignoreAllDuplicateRows=false, rpc=KuduRpc(method=Write, tablet=null, attempt=1,
DeadlineTracker(timeout=3, elapsed=2), Traces: [0ms] querying master, [0ms]
Sub rpc: GetTableLocations sending RPC to server , [2ms] Sub rpc:
GetTableLocations received from server response Not found: The table was
deleted: Table deleted at 2017-09-20 18:16:38 UTC (error 0),
Deferred@509465434(state=PENDING, result=null, callback=apply batch response,
errback=apply batch error response))} after lookup exception
org.apache.kudu.client.NonRecoverableException: The table was deleted: Table
deleted at 2017-09-20 18:16:38 UTC
at
org.apache.kudu.client.RpcProxy.dispatchMasterError(RpcProxy.java:360)
at org.apache.kudu.client.RpcProxy.responseReceived(RpcProxy.java:253)
at org.apache.kudu.client.RpcProxy.access$000(RpcProxy.java:60)
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:132)
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:128)
at
org.apache.kudu.client.Connection.messageReceived(Connection.java:375)
at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.apache.kudu.client.Connection.handleUpstream(Connection.java:239)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.jboss.netty.handler.timeout.ReadTimeoutHandler.messageReceived(ReadTimeoutHandler.java:184)
at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at
org.jboss.netty.util.ThreadRena
[jira] [Commented] (KUDU-1521) Flakiness in TestAsyncKuduSession
[
https://issues.apache.org/jira/browse/KUDU-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16142382#comment-16142382
]
Adar Dembo commented on KUDU-1521:
--
bq. Adar Dembo wanna try looping dist test and see if this is still an issue?
The issue I originally wrote about (in test()) still exists. I wasn't able to
figure out the other issues when I first reported this bug.
> Flakiness in TestAsyncKuduSession
> -
>
> Key: KUDU-1521
> URL: https://issues.apache.org/jira/browse/KUDU-1521
> Project: Kudu
> Issue Type: Bug
> Components: client
>Affects Versions: 0.9.1
>Reporter: Adar Dembo
>
> I've been trying to parse the various failures in
> http://104.196.14.100/job/kudu-gerrit/2270/BUILD_TYPE=RELEASE. Here's what I
> see in the test:
> The way test() tests AUTO_FLUSH_BACKGROUND is inherently flaky; a delay while
> running test code will give the background flush task a chance to fire when
> the test code doesn't expect it. I've seen this cause lead to no
> PleaseThrottleException, but I suspect the first block of test code dealing
> with background flushes is flaky too (since it's testing elapsed time).
> There's also some test failures that I can't figure out. I've pasted them
> below for posterity:
> {noformat}
> 03:52:14
> testGetTableLocationsErrorCauseSessionStuck(org.kududb.client.TestAsyncKuduSession)
> Time elapsed: 100.009 sec <<< ERROR!
> 03:52:14 java.lang.Exception: test timed out after 10 milliseconds
> 03:52:14 at java.lang.Object.wait(Native Method)
> 03:52:14 at java.lang.Object.wait(Object.java:503)
> 03:52:14 at com.stumbleupon.async.Deferred.doJoin(Deferred.java:1136)
> 03:52:14 at com.stumbleupon.async.Deferred.join(Deferred.java:1019)
> 03:52:14 at
> org.kududb.client.TestAsyncKuduSession.testGetTableLocationsErrorCauseSessionStuck(TestAsyncKuduSession.java:133)
> 03:52:14
> 03:52:14
> testBatchErrorCauseSessionStuck(org.kududb.client.TestAsyncKuduSession) Time
> elapsed: 0.199 sec <<< ERROR!
> 03:52:14 org.kududb.client.MasterErrorException: Server[Kudu Master -
> 127.13.215.1:64030] NOT_FOUND[code 1]: The table was deleted: Table deleted
> at 2016-07-09 03:50:24 UTC
> 03:52:14 at
> org.kududb.client.TabletClient.dispatchMasterErrorOrReturnException(TabletClient.java:533)
> 03:52:14 at org.kududb.client.TabletClient.decode(TabletClient.java:463)
> 03:52:14 at org.kududb.client.TabletClient.decode(TabletClient.java:83)
> 03:52:14 at
> org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500)
> 03:52:14 at
> org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
> 03:52:14 at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> 03:52:14 at
> org.kududb.client.TabletClient.handleUpstream(TabletClient.java:638)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> 03:52:14 at
> org.jboss.netty.handler.timeout.ReadTimeoutHandler.messageReceived(ReadTimeoutHandler.java:184)
> 03:52:14 at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> 03:52:14 at
> org.kududb.client.AsyncKuduClient$TabletClientPipeline.sendUpstream(AsyncKuduClient.java:1877)
> 03:52:14 at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> 03:52:14 at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> 03:52:14 at
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> 03:52:14 at
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> 03:52:14 at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 03:52:14 at
> java.util.concurren
[jira] [Commented] (KUDU-1521) Flakiness in TestAsyncKuduSession
[
https://issues.apache.org/jira/browse/KUDU-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16142261#comment-16142261
]
Jean-Daniel Cryans commented on KUDU-1521:
--
[~adar] wanna try looping dist test and see if this is still an issue?
> Flakiness in TestAsyncKuduSession
> -
>
> Key: KUDU-1521
> URL: https://issues.apache.org/jira/browse/KUDU-1521
> Project: Kudu
> Issue Type: Bug
> Components: client
>Affects Versions: 0.9.1
>Reporter: Adar Dembo
>
> I've been trying to parse the various failures in
> http://104.196.14.100/job/kudu-gerrit/2270/BUILD_TYPE=RELEASE. Here's what I
> see in the test:
> The way test() tests AUTO_FLUSH_BACKGROUND is inherently flaky; a delay while
> running test code will give the background flush task a chance to fire when
> the test code doesn't expect it. I've seen this cause lead to no
> PleaseThrottleException, but I suspect the first block of test code dealing
> with background flushes is flaky too (since it's testing elapsed time).
> There's also some test failures that I can't figure out. I've pasted them
> below for posterity:
> {noformat}
> 03:52:14
> testGetTableLocationsErrorCauseSessionStuck(org.kududb.client.TestAsyncKuduSession)
> Time elapsed: 100.009 sec <<< ERROR!
> 03:52:14 java.lang.Exception: test timed out after 10 milliseconds
> 03:52:14 at java.lang.Object.wait(Native Method)
> 03:52:14 at java.lang.Object.wait(Object.java:503)
> 03:52:14 at com.stumbleupon.async.Deferred.doJoin(Deferred.java:1136)
> 03:52:14 at com.stumbleupon.async.Deferred.join(Deferred.java:1019)
> 03:52:14 at
> org.kududb.client.TestAsyncKuduSession.testGetTableLocationsErrorCauseSessionStuck(TestAsyncKuduSession.java:133)
> 03:52:14
> 03:52:14
> testBatchErrorCauseSessionStuck(org.kududb.client.TestAsyncKuduSession) Time
> elapsed: 0.199 sec <<< ERROR!
> 03:52:14 org.kududb.client.MasterErrorException: Server[Kudu Master -
> 127.13.215.1:64030] NOT_FOUND[code 1]: The table was deleted: Table deleted
> at 2016-07-09 03:50:24 UTC
> 03:52:14 at
> org.kududb.client.TabletClient.dispatchMasterErrorOrReturnException(TabletClient.java:533)
> 03:52:14 at org.kududb.client.TabletClient.decode(TabletClient.java:463)
> 03:52:14 at org.kududb.client.TabletClient.decode(TabletClient.java:83)
> 03:52:14 at
> org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500)
> 03:52:14 at
> org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
> 03:52:14 at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> 03:52:14 at
> org.kududb.client.TabletClient.handleUpstream(TabletClient.java:638)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> 03:52:14 at
> org.jboss.netty.handler.timeout.ReadTimeoutHandler.messageReceived(ReadTimeoutHandler.java:184)
> 03:52:14 at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> 03:52:14 at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> 03:52:14 at
> org.kududb.client.AsyncKuduClient$TabletClientPipeline.sendUpstream(AsyncKuduClient.java:1877)
> 03:52:14 at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> 03:52:14 at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
> 03:52:14 at
> org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> 03:52:14 at
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> 03:52:14 at
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> 03:52:14 at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 03:52:14 at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 03:52:14 at java.lang.Thread.run(Thread.java:745)
> 03:52:14
>
