[
https://issues.apache.org/jira/browse/TINKERPOP-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stephen mallette updated TINKERPOP-1043:
----------------------------------------
Component/s: hadoop
> Spark test InputRDDTest failing on shouldReadFromArbitraryRDD
> -------------------------------------------------------------
>
> Key: TINKERPOP-1043
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1043
> Project: TinkerPop
> Issue Type: Bug
> Components: hadoop, test-suite
> Affects Versions: 3.1.1-incubating
> Environment: Debian GNU/Linux 8, Java 8
> Reporter: Jonathan Ellithorpe
>
> Getting the following when running InputRDDTest.shouldReadFromArbitraryRDD:
> {code}
> java.lang.IllegalStateException:
> org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
> /home/jdellit/NetBeansProjects/incubator-tinkerpop/spark-gremlin/target/test-case-data/InputRDDTest/shouldReadFromArbitraryRDD/~reducing
> already exists
> at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
> at
> org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.ComputerResultStep.processNextStart(ComputerResultStep.java:80)
> at
> org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:126)
> at
> org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:37)
> at
> org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.next(DefaultTraversal.java:158)
> at
> org.apache.tinkerpop.gremlin.spark.structure.io.InputRDDTest.shouldReadFromArbitraryRDD(InputRDDTest.java:56)
> Caused by: org.apache.hadoop.mapred.FileAlreadyExistsException: Output
> directory
> /home/jdellit/NetBeansProjects/incubator-tinkerpop/spark-gremlin/target/test-case-data/InputRDDTest/shouldReadFromArbitraryRDD/~reducing
> already exists
> at
> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:146)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1011)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:998)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:998)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:306)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopDataset(PairRDDFunctions.scala:998)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply$mcV$sp(PairRDDFunctions.scala:938)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:930)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:930)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:306)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopFile(PairRDDFunctions.scala:930)
> at
> org.apache.spark.api.java.JavaPairRDD.saveAsNewAPIHadoopFile(JavaPairRDD.scala:809)
> at
> org.apache.tinkerpop.gremlin.spark.process.computer.SparkExecutor.saveMapReduceRDD(SparkExecutor.java:203)
> at
> org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer.lambda$submitWithExecutor$22(SparkGraphComputer.java:227)
> at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
