Andrew Or created SPARK-13344:
---------------------------------
Summary: SaveLoadSuite has many accumulator exceptions
Key: SPARK-13344
URL: https://issues.apache.org/jira/browse/SPARK-13344
Project: Spark
Issue Type: Bug
Components: SQL, Tests
Affects Versions: 2.0.0
Reporter: Andrew Or
Assignee: Andrew Or
This is because SparkFunSuite clears all accumulators after every single test.
This suite reuses a DF and all of its associated internal accumulators across
many tests.
This is likely caused by SPARK-10620.
{code}
10:52:38.967 WARN org.apache.spark.executor.TaskMetrics: encountered
unregistered accumulator 253 when reconstructing task metrics.
10:52:38.967 ERROR org.apache.spark.scheduler.DAGScheduler: Failed to update
accumulators for task 0
org.apache.spark.SparkException: attempted to access non-existent accumulator
253
at
org.apache.spark.scheduler.DAGScheduler$$anonfun$updateAccumulators$1.apply(DAGScheduler.scala:1099)
at
org.apache.spark.scheduler.DAGScheduler$$anonfun$updateAccumulators$1.apply(DAGScheduler.scala:1091)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
