Josh Wills created CRUNCH-670:
---------------------------------
Summary: Make the AvroPathPerKeyTarget work with the SparkRuntime
Key: CRUNCH-670
URL: https://issues.apache.org/jira/browse/CRUNCH-670
Project: Crunch
Issue Type: Improvement
Reporter: Josh Wills
Assignee: Josh Wills
There is an issue where the AvroPathPerKeyTarget won't properly copy the output
of a Spark pipeline from the temp directory to the target directory because it
assumes it will always get a valid Crunch output index (0, 1, 2, ...) and the
SparkRuntime passes -1 to the Target's output handler method (to signal that
it's the only output for the job.) I _think_ the right move is to have
AvroPathPerKeyTarget rewrite a -1 index to 0 so as not to break any other
implementations that depend on the SparkRuntime's behavior.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
