[jira] [Assigned] (SPARK-20282) Flaky test: org.apache.spark.sql.streaming/StreamingQuerySuite/OneTime_trigger__commit_log__and_exception

2017-04-10 Thread Apache Spark (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SPARK-20282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apache Spark reassigned SPARK-20282:


Assignee: (was: Apache Spark)

> Flaky test: 
> org.apache.spark.sql.streaming/StreamingQuerySuite/OneTime_trigger__commit_log__and_exception
> -
>
> Key: SPARK-20282
> URL: https://issues.apache.org/jira/browse/SPARK-20282
> Project: Spark
>  Issue Type: Test
>  Components: Structured Streaming, Tests
>Affects Versions: 2.2.0
>Reporter: Shixiong Zhu
>Priority: Minor
>  Labels: flaky-test
>
> I saw the following failure several times:
> {code}
> sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException: 
> Assert on query failed: 
> == Progress ==
>AssertOnQuery(, )
>StopStream
>AddData to MemoryStream[value#30891]: 1,2
>
> StartStream(OneTimeTrigger,org.apache.spark.util.SystemClock@35cdc93a,Map())
>CheckAnswer: [6],[3]
>StopStream
> => AssertOnQuery(, )
>AssertOnQuery(, )
>StartStream(OneTimeTrigger,org.apache.spark.util.SystemClock@cdb247d,Map())
>CheckAnswer: [6],[3]
>StopStream
>AddData to MemoryStream[value#30891]: 3
>
> StartStream(OneTimeTrigger,org.apache.spark.util.SystemClock@55394e4d,Map())
>CheckLastBatch: [2]
>StopStream
>AddData to MemoryStream[value#30891]: 0
>
> StartStream(OneTimeTrigger,org.apache.spark.util.SystemClock@749aa997,Map())
>ExpectFailure[org.apache.spark.SparkException, isFatalError: false]
>AssertOnQuery(, )
>AssertOnQuery(, incorrect start offset or end offset on 
> exception)
> == Stream ==
> Output Mode: Append
> Stream state: not started
> Thread state: dead
> == Sink ==
> 0: [6] [3]
> == Plan ==
>  
>  
>   at 
> org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:495)
>   at 
> org.scalatest.FunSuite.newAssertionFailedException(FunSuite.scala:1555)
>   at org.scalatest.Assertions$class.fail(Assertions.scala:1328)
>   at org.scalatest.FunSuite.fail(FunSuite.scala:1555)
>   at 
> org.apache.spark.sql.streaming.StreamTest$class.failTest$1(StreamTest.scala:347)
>   at 
> org.apache.spark.sql.streaming.StreamTest$class.verify$1(StreamTest.scala:318)
>   at 
> org.apache.spark.sql.streaming.StreamTest$$anonfun$liftedTree1$1$1.apply(StreamTest.scala:483)
>   at 
> org.apache.spark.sql.streaming.StreamTest$$anonfun$liftedTree1$1$1.apply(StreamTest.scala:357)
>   at 
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
>   at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
>   at 
> org.apache.spark.sql.streaming.StreamTest$class.liftedTree1$1(StreamTest.scala:357)
>   at 
> org.apache.spark.sql.streaming.StreamTest$class.testStream(StreamTest.scala:356)
>   at 
> org.apache.spark.sql.streaming.StreamingQuerySuite.testStream(StreamingQuerySuite.scala:41)
>   at 
> org.apache.spark.sql.streaming.StreamingQuerySuite$$anonfun$6.apply$mcV$sp(StreamingQuerySuite.scala:166)
>   at 
> org.apache.spark.sql.streaming.StreamingQuerySuite$$anonfun$6.apply(StreamingQuerySuite.scala:161)
>   at 
> org.apache.spark.sql.streaming.StreamingQuerySuite$$anonfun$6.apply(StreamingQuerySuite.scala:161)
>   at org.apache.spark.sql.catalyst.util.package$.quietly(package.scala:42)
>   at 
> org.apache.spark.sql.test.SQLTestUtils$$anonfun$testQuietly$1.apply$mcV$sp(SQLTestUtils.scala:268)
>   at 
> org.apache.spark.sql.test.SQLTestUtils$$anonfun$testQuietly$1.apply(SQLTestUtils.scala:268)
>   at 
> org.apache.spark.sql.test.SQLTestUtils$$anonfun$testQuietly$1.apply(SQLTestUtils.scala:268)
>   at 
> org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
>   at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
>   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
>   at org.scalatest.Transformer.apply(Transformer.scala:22)
>   at org.scalatest.Transformer.apply(Transformer.scala:20)
>   at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:166)
>   at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:68)
>   at 
> org.scalatest.FunSuiteLike$class.invokeWithFixture$1(FunSuiteLike.scala:163)
>   at 
> org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
>   at 
> org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
>   at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
>   at org.scalatest.FunSuiteLike$class.runTest(FunSuiteLike.scala:175)
>   at 
> org.apache.spark.sql.streaming.StreamingQuerySuite.org$scalatest$BeforeAndAfterEach$$super$runTest(StreamingQuerySuite.scala:41)
>

[jira] [Assigned] (SPARK-20282) Flaky test: org.apache.spark.sql.streaming/StreamingQuerySuite/OneTime_trigger__commit_log__and_exception

2017-04-10 Thread Apache Spark (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SPARK-20282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apache Spark reassigned SPARK-20282:


Assignee: Apache Spark

> Flaky test: 
> org.apache.spark.sql.streaming/StreamingQuerySuite/OneTime_trigger__commit_log__and_exception
> -
>
> Key: SPARK-20282
> URL: https://issues.apache.org/jira/browse/SPARK-20282
> Project: Spark
>  Issue Type: Test
>  Components: Structured Streaming, Tests
>Affects Versions: 2.2.0
>Reporter: Shixiong Zhu
>Assignee: Apache Spark
>Priority: Minor
>  Labels: flaky-test
>
> I saw the following failure several times:
> {code}
> sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException: 
> Assert on query failed: 
> == Progress ==
>AssertOnQuery(, )
>StopStream
>AddData to MemoryStream[value#30891]: 1,2
>
> StartStream(OneTimeTrigger,org.apache.spark.util.SystemClock@35cdc93a,Map())
>CheckAnswer: [6],[3]
>StopStream
> => AssertOnQuery(, )
>AssertOnQuery(, )
>StartStream(OneTimeTrigger,org.apache.spark.util.SystemClock@cdb247d,Map())
>CheckAnswer: [6],[3]
>StopStream
>AddData to MemoryStream[value#30891]: 3
>
> StartStream(OneTimeTrigger,org.apache.spark.util.SystemClock@55394e4d,Map())
>CheckLastBatch: [2]
>StopStream
>AddData to MemoryStream[value#30891]: 0
>
> StartStream(OneTimeTrigger,org.apache.spark.util.SystemClock@749aa997,Map())
>ExpectFailure[org.apache.spark.SparkException, isFatalError: false]
>AssertOnQuery(, )
>AssertOnQuery(, incorrect start offset or end offset on 
> exception)
> == Stream ==
> Output Mode: Append
> Stream state: not started
> Thread state: dead
> == Sink ==
> 0: [6] [3]
> == Plan ==
>  
>  
>   at 
> org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:495)
>   at 
> org.scalatest.FunSuite.newAssertionFailedException(FunSuite.scala:1555)
>   at org.scalatest.Assertions$class.fail(Assertions.scala:1328)
>   at org.scalatest.FunSuite.fail(FunSuite.scala:1555)
>   at 
> org.apache.spark.sql.streaming.StreamTest$class.failTest$1(StreamTest.scala:347)
>   at 
> org.apache.spark.sql.streaming.StreamTest$class.verify$1(StreamTest.scala:318)
>   at 
> org.apache.spark.sql.streaming.StreamTest$$anonfun$liftedTree1$1$1.apply(StreamTest.scala:483)
>   at 
> org.apache.spark.sql.streaming.StreamTest$$anonfun$liftedTree1$1$1.apply(StreamTest.scala:357)
>   at 
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
>   at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
>   at 
> org.apache.spark.sql.streaming.StreamTest$class.liftedTree1$1(StreamTest.scala:357)
>   at 
> org.apache.spark.sql.streaming.StreamTest$class.testStream(StreamTest.scala:356)
>   at 
> org.apache.spark.sql.streaming.StreamingQuerySuite.testStream(StreamingQuerySuite.scala:41)
>   at 
> org.apache.spark.sql.streaming.StreamingQuerySuite$$anonfun$6.apply$mcV$sp(StreamingQuerySuite.scala:166)
>   at 
> org.apache.spark.sql.streaming.StreamingQuerySuite$$anonfun$6.apply(StreamingQuerySuite.scala:161)
>   at 
> org.apache.spark.sql.streaming.StreamingQuerySuite$$anonfun$6.apply(StreamingQuerySuite.scala:161)
>   at org.apache.spark.sql.catalyst.util.package$.quietly(package.scala:42)
>   at 
> org.apache.spark.sql.test.SQLTestUtils$$anonfun$testQuietly$1.apply$mcV$sp(SQLTestUtils.scala:268)
>   at 
> org.apache.spark.sql.test.SQLTestUtils$$anonfun$testQuietly$1.apply(SQLTestUtils.scala:268)
>   at 
> org.apache.spark.sql.test.SQLTestUtils$$anonfun$testQuietly$1.apply(SQLTestUtils.scala:268)
>   at 
> org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
>   at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
>   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
>   at org.scalatest.Transformer.apply(Transformer.scala:22)
>   at org.scalatest.Transformer.apply(Transformer.scala:20)
>   at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:166)
>   at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:68)
>   at 
> org.scalatest.FunSuiteLike$class.invokeWithFixture$1(FunSuiteLike.scala:163)
>   at 
> org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
>   at 
> org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
>   at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
>   at org.scalatest.FunSuiteLike$class.runTest(FunSuiteLike.scala:175)
>   at 
>