[ https://issues.apache.org/jira/browse/TINKERPOP-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15289591#comment-15289591 ]
Marko A. Rodriguez commented on TINKERPOP-1304: ----------------------------------------------- This isn't a bug. Your output format is {{ScriptOutputFormat}} so the next in the chain assumes {{ScriptInputFormat}} and since there is no parser script provided, NPE. There is really no way around this. You should use {{GryoOutputFormat}} as your {{graphWriter}}. After chatting with [~dkuppitz] in HipChat, I realized that we can't even have "intermediate formats" as there is no way for the OLAP job to know its part of a chain --- except at the Traversal level, but thats way above {{HadoopGraph}} stuff. :/ ... > Input format for OLAP jobs is changed during job execution > ---------------------------------------------------------- > > Key: TINKERPOP-1304 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1304 > Project: TinkerPop > Issue Type: Bug > Components: process > Affects Versions: 3.2.0-incubating > Reporter: Daniel Kuppitz > Assignee: Marko A. Rodriguez > > To reproduce the error, create the following config file: > {noformat} > # hadoop-script-output.properties > gremlin.graph=org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph > gremlin.hadoop.graphReader=org.apache.tinkerpop.gremlin.hadoop.structure.io.gryo.GryoInputFormat > gremlin.hadoop.graphWriter=org.apache.tinkerpop.gremlin.hadoop.structure.io.script.ScriptOutputFormat > gremlin.hadoop.jarsInDistributedCache=true > gremlin.hadoop.defaultGraphComputer=org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer > gremlin.hadoop.inputLocation=tinkerpop-modern.kryo > gremlin.hadoop.outputLocation=output > gremlin.hadoop.scriptOutputFormat.script=script-output.groovy > spark.master=local[*] > spark.executor.memory=1g > spark.serializer=org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer > {noformat} > ... and this Groovy file: > {code} > // script-output.groovy > def stringify(vertex) { > return "foo" > } > {code} > Finally execute the following code in the Gremlin console: > {noformat} > :install org.apache.tinkerpop hadoop-gremlin 3.2.1-SNAPSHOT > :install org.apache.tinkerpop spark-gremlin 3.2.1-SNAPSHOT > :q > {noformat} > {noformat} > :plugin use tinkerpop.hadoop > :plugin use tinkerpop.spark > hdfs.copyFromLocal("data/tinkerpop-modern.kryo", "tinkerpop-modern.kryo") > hdfs.copyFromLocal("/tmp/script-output.groovy", "script-output.groovy") > graph = GraphFactory.open("/tmp/hadoop-script-output.properties") > g = graph.traversal().withComputer() > g.V().hasLabel("person").program(BulkDumperVertexProgram.build().create(graph)) > {noformat} > It will fail with the following exception: > {noformat} > ERROR org.apache.spark.executor.Executor - Exception in task 0.0 in stage > 0.0 (TID 0) > java.lang.IllegalArgumentException: Can not create a Path from a null string > at org.apache.hadoop.fs.Path.checkPathArg(Path.java:122) > at org.apache.hadoop.fs.Path.<init>(Path.java:134) > at > org.apache.tinkerpop.gremlin.hadoop.structure.io.script.ScriptRecordReader.initialize(ScriptRecordReader.java:88) > at > org.apache.tinkerpop.gremlin.hadoop.structure.io.script.ScriptInputFormat.createRecordReader(ScriptInputFormat.java:39) > at > org.apache.spark.rdd.NewHadoopRDD$$anon$1.<init>(NewHadoopRDD.scala:156) > at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:129) > at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:64) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:270) > at > org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:270) > at > org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73) > at > org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41) > at org.apache.spark.scheduler.Task.run(Task.scala:89) > at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {noformat} > Other traversals and {{graph.compute().program(...).submit().get()}} work > fine, only traversals using {{program()}} seem to cause this problem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)