[jira] [Commented] (KUDU-2599) Timeout in DefaultSourceTest.testSocketReadTimeoutPropagation

2018-11-08 Thread Adar Dembo (JIRA) 


[ 
https://issues.apache.org/jira/browse/KUDU-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680907#comment-16680907
 ] 

Adar Dembo commented on KUDU-2599:
--

The fix is shockingly stupid. Stay tuned.

> Timeout in DefaultSourceTest.testSocketReadTimeoutPropagation
> -
>
> Key: KUDU-2599
> URL: https://issues.apache.org/jira/browse/KUDU-2599
> Project: Kudu
>  Issue Type: Bug
>  Components: spark
>Affects Versions: 1.8.0
>Reporter: Will Berkeley
>Assignee: Adar Dembo
>Priority: Major
> Attachments: DefaultSourceTestFailure-snippet.txt, 
> TEST-org.apache.kudu.spark.kudu.DefaultSourceTest.xml
>
>
> Log attached
> Here is the relevant stack trace:
> {code:java}
>  classname="org.apache.kudu.spark.kudu.DefaultSourceTest" time="19.927">
>  type="java.security.PrivilegedActionException">java.security.PrivilegedActionException:
>  org.apache.kudu.client.NoLeaderFoundException: Master config 
> (127.23.212.125:33436,127.23.212.126:35133,127.23.212.124:41615) has no 
> leader. Exceptions received: org.apache.kudu.client.RecoverableException: 
> [peer master-127.23.212.125:33436(127.23.212.125:33436)] encountered a read 
> timeout; closing the channel,org.apache.kudu.client.RecoverableException: 
> [peer master-127.23.212.126:35133(127.23.212.126:35133)] encountered a read 
> timeout; closing the channel,org.apache.kudu.client.RecoverableException: 
> [peer master-127.23.212.124:41615(127.23.212.124:41615)] encountered a read 
> timeout; closing the channel
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:360)
> at org.apache.kudu.spark.kudu.KuduContext.init(KuduContext.scala:122)
> at 
> org.apache.kudu.spark.kudu.KuduRelation.init(DefaultSource.scala:212)
> at 
> org.apache.kudu.spark.kudu.DefaultSource.createRelation(DefaultSource.scala:101)
> at 
> org.apache.kudu.spark.kudu.DefaultSource.createRelation(DefaultSource.scala:76)
> at 
> org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:341)
> at 
> org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:239)
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:227)
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:164)
> at 
> org.apache.kudu.spark.kudu.package$KuduDataFrameReader.kudu(package.scala:30)
> at 
> org.apache.kudu.spark.kudu.DefaultSourceTest.testSocketReadTimeoutPropagation(DefaultSourceTest.scala:924)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.apache.kudu.junit.RetryRule$RetryStatement.evaluate(RetryRule.java:72)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at 
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106)
> at 
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
> at 
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
> at 
> org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66)
> at 
>

[jira] [Commented] (KUDU-2599) Timeout in DefaultSourceTest.testSocketReadTimeoutPropagation

2018-11-08 Thread Grant Henke (JIRA) 


[ 
https://issues.apache.org/jira/browse/KUDU-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680369#comment-16680369
 ] 

Grant Henke commented on KUDU-2599:
---

I modified the tests to output VLOG level 1 messages and inline the failing 
exception. Attached is a snippet of the failing log. It's not immediately clear 
why the following exception is periodically occurring:
{code:java}
java.security.PrivilegedActionException: 
org.apache.kudu.client.NoLeaderFoundException: Master config 
(127.14.104.60:38345,127.14.104.61:42699,127.14.104.62:45145) has no 
leader.20:04:13.043{code}
[^DefaultSourceTestFailure-snippet.txt]

> Timeout in DefaultSourceTest.testSocketReadTimeoutPropagation
> -
>
> Key: KUDU-2599
> URL: https://issues.apache.org/jira/browse/KUDU-2599
> Project: Kudu
>  Issue Type: Bug
>  Components: spark
>Affects Versions: 1.8.0
>Reporter: Will Berkeley
>Priority: Major
> Attachments: DefaultSourceTestFailure-snippet.txt, 
> TEST-org.apache.kudu.spark.kudu.DefaultSourceTest.xml
>
>
> Log attached
> Here is the relevant stack trace:
> {code:java}
>  classname="org.apache.kudu.spark.kudu.DefaultSourceTest" time="19.927">
>  type="java.security.PrivilegedActionException">java.security.PrivilegedActionException:
>  org.apache.kudu.client.NoLeaderFoundException: Master config 
> (127.23.212.125:33436,127.23.212.126:35133,127.23.212.124:41615) has no 
> leader. Exceptions received: org.apache.kudu.client.RecoverableException: 
> [peer master-127.23.212.125:33436(127.23.212.125:33436)] encountered a read 
> timeout; closing the channel,org.apache.kudu.client.RecoverableException: 
> [peer master-127.23.212.126:35133(127.23.212.126:35133)] encountered a read 
> timeout; closing the channel,org.apache.kudu.client.RecoverableException: 
> [peer master-127.23.212.124:41615(127.23.212.124:41615)] encountered a read 
> timeout; closing the channel
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:360)
> at org.apache.kudu.spark.kudu.KuduContext.init(KuduContext.scala:122)
> at 
> org.apache.kudu.spark.kudu.KuduRelation.init(DefaultSource.scala:212)
> at 
> org.apache.kudu.spark.kudu.DefaultSource.createRelation(DefaultSource.scala:101)
> at 
> org.apache.kudu.spark.kudu.DefaultSource.createRelation(DefaultSource.scala:76)
> at 
> org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:341)
> at 
> org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:239)
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:227)
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:164)
> at 
> org.apache.kudu.spark.kudu.package$KuduDataFrameReader.kudu(package.scala:30)
> at 
> org.apache.kudu.spark.kudu.DefaultSourceTest.testSocketReadTimeoutPropagation(DefaultSourceTest.scala:924)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.apache.kudu.junit.RetryRule$RetryStatement.evaluate(RetryRule.java:72)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at 
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106)
> at 
>

[jira] [Commented] (KUDU-2599) Timeout in DefaultSourceTest.testSocketReadTimeoutPropagation

2018-10-29 Thread Grant Henke (JIRA) 


[ 
https://issues.apache.org/jira/browse/KUDU-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16667206#comment-16667206
 ] 

Grant Henke commented on KUDU-2599:
---

It seams there are a few DefaultSourceTest cases that can be susceptible to 
this.

> Timeout in DefaultSourceTest.testSocketReadTimeoutPropagation
> -
>
> Key: KUDU-2599
> URL: https://issues.apache.org/jira/browse/KUDU-2599
> Project: Kudu
>  Issue Type: Bug
>  Components: spark
>Affects Versions: 1.8.0
>Reporter: Will Berkeley
>Priority: Major
> Attachments: TEST-org.apache.kudu.spark.kudu.DefaultSourceTest.xml
>
>
> Log attached



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KUDU-2599) Timeout in DefaultSourceTest.testSocketReadTimeoutPropagation

2018-10-09 Thread Hao Hao (JIRA) 


[ 
https://issues.apache.org/jira/browse/KUDU-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643941#comment-16643941
 ] 

Hao Hao commented on KUDU-2599:
---

This also happens to 
DefaultSourceTest.testTableScanWithProjectionAndPredicateDecimal128.

> Timeout in DefaultSourceTest.testSocketReadTimeoutPropagation
> -
>
> Key: KUDU-2599
> URL: https://issues.apache.org/jira/browse/KUDU-2599
> Project: Kudu
>  Issue Type: Bug
>  Components: spark
>Affects Versions: 1.8.0
>Reporter: Will Berkeley
>Priority: Major
> Attachments: TEST-org.apache.kudu.spark.kudu.DefaultSourceTest.xml
>
>
> Log attached



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)