Hello ,I am a new guy on scala &spark, yestday i compile spark from 1.0.0
source code and run test,there is and testcase failed:
For example run command in shell : sbt/sbt "testOnly
org.apache.spark.streaming.InputStreamsSuite"
the testcase: test("socket input stream") would failed , test result like :
===============test result===================================
[info] InputStreamsSuite:
[info] - socket input stream *** FAILED *** (20 seconds, 547 milliseconds)
[info] 0 did not equal 5 (InputStreamsSuite.scala:96)
[info] org.scalatest.exceptions.TestFailedException:
[info] at
org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:318)
[info] at
org.apache.spark.streaming.InputStreamsSuite.newAssertionFailedException(InputStreamsSuite.scala:44)
[info] at org.scalatest.Assertions$class.assert(Assertions.scala:401)
[info] at
org.apache.spark.streaming.InputStreamsSuite.assert(InputStreamsSuite.scala:44)
[info] at
org.apache.spark.streaming.InputStreamsSuite$$anonfun$1.apply$mcV$sp(InputStreamsSuite.scala:96)
[info] at
org.apache.spark.streaming.InputStreamsSuite$$anonfun$1.apply(InputStreamsSuite.scala:46)
[info] at
org.apache.spark.streaming.InputStreamsSuite$$anonfun$1.apply(InputStreamsSuite.scala:46)
[info] at org.scalatest.FunSuite$$anon$1.apply(FunSuite.scala:1265)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[info] at
org.apache.spark.streaming.InputStreamsSuite.withFixture(InputStreamsSuite.scala:44)
[info] at
org.scalatest.FunSuite$class.invokeWithFixture$1(FunSuite.scala:1262)
[info] at org.scalatest.FunSuite$$anonfun$runTest$1.apply(FunSuite.scala:1271)
[info] at org.scalatest.FunSuite$$anonfun$runTest$1.apply(FunSuite.scala:1271)
[info] at org.scalatest.SuperEngine.runTestImpl(Engine.scala:198)
[info] at org.scalatest.FunSuite$class.runTest(FunSuite.scala:1271)
[info] at
org.apache.spark.streaming.InputStreamsSuite.org$scalatest$BeforeAndAfter$$super$runTest(InputStreamsSuite.scala:44)
[info] at org.scalatest.BeforeAndAfter$class.runTest(BeforeAndAfter.scala:171)
[info] at
org.apache.spark.streaming.InputStreamsSuite.runTest(InputStreamsSuite.scala:44)
[info] at
org.scalatest.FunSuite$$anonfun$runTests$1.apply(FunSuite.scala:1304)
[info] at
org.scalatest.FunSuite$$anonfun$runTests$1.apply(FunSuite.scala:1304)
[info] at
org.scalatest.SuperEngine$$anonfun$org$scalatest$SuperEngine$$runTestsInBranch$1.apply(Engine.scala:260)
[info] at
org.scalatest.SuperEngine$$anonfun$org$scalatest$SuperEngine$$runTestsInBranch$1.apply(Engine.scala:249)
[info] at scala.collection.immutable.List.foreach(List.scala:318)
[info] at
org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:249)
[info] at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:326)
[info] at org.scalatest.FunSuite$class.runTests(FunSuite.scala:1304)
[info] at
org.apache.spark.streaming.InputStreamsSuite.runTests(InputStreamsSuite.scala:44)
[info] at org.scalatest.Suite$class.run(Suite.scala:2303)
[info] at
org.apache.spark.streaming.InputStreamsSuite.org$scalatest$FunSuite$$super$run(InputStreamsSuite.scala:44)
[info] at org.scalatest.FunSuite$$anonfun$run$1.apply(FunSuite.scala:1310)
[info] at org.scalatest.FunSuite$$anonfun$run$1.apply(FunSuite.scala:1310)
[info] at org.scalatest.SuperEngine.runImpl(Engine.scala:362)
[info] at org.scalatest.FunSuite$class.run(FunSuite.scala:1310)
[info] at
org.apache.spark.streaming.InputStreamsSuite.org$scalatest$BeforeAndAfter$$super$run(InputStreamsSuite.scala:44)
[info] at org.scalatest.BeforeAndAfter$class.run(BeforeAndAfter.scala:208)
[info] at
org.apache.spark.streaming.InputStreamsSuite.run(InputStreamsSuite.scala:44)
[info] at
org.scalatest.tools.ScalaTestFramework$ScalaTestRunner.run(ScalaTestFramework.scala:214)
[info] at sbt.RunnerWrapper$1.runRunner2(FrameworkWrapper.java:223)
[info] at sbt.RunnerWrapper$1.execute(FrameworkWrapper.java:236)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:294)
[info] at sbt.ForkMain$Run$2.call(ForkMain.java:284)
[info] at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:138)
[info] at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[info] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
[info] at java.lang.Thread.run(Thread.java:662)
I check the souce code, it seem's this assert would fail (line number is 96):
assert(output.size === expectedOutput.size)
so i print the output & outputBuffer & expectedOutput value ：
// Verify whether data received was as expected
85 println("--------------------------------")
86 println("output.size = " + output.size)
87
88 output.foreach(x => println("[" + x.mkString(",") + "]"))
89 println("outputBuffer.size="+outputBuffer.size)
90 outputBuffer.foreach(x => println("[" + x.mkString(",") + "]"))
91 println("expected expectedOutput.size = " + expectedOutput.size)
92 expectedOutput.foreach(x => println("[" + x.mkString(",") + "]"))
93 println("--------------------------------")
the test result :
--------------------------------
output.size = 0
outputBuffer.size=5
[]
[]
[]
[]
[]
expected expectedOutput.size = 5
[1]
[2]
[3]
[4]
[5]
--------------------------------
Anybody can help me why the testcase failed ? the variable output.size is a
wrong value ?
my runtime :
java -version:
java version "1.6.0_32"
Java(TM) SE Runtime Environment (build 1.6.0_32-b05)
uname -a
Linux ML_221_121 2.6.18-308.4.1.el5 #1 SMP Wed Mar 28 01:54:56 EDT 2012 x86_64
x86_64 x86_64 GNU/Linux
thanks
crazymb
2014.6.22
