Josh Wills created CRUNCH-412:
---------------------------------
Summary: MemPipeline mode for simulating MapReduce quirks
Key: CRUNCH-412
URL: https://issues.apache.org/jira/browse/CRUNCH-412
Project: Crunch
Issue Type: Improvement
Components: Core
Reporter: Josh Wills
Assignee: Josh Wills
>From a discussion on the mailing list, we'd like to have a MemPipeline mode
>that simulates a couple of the quirks of MapReduce/MRPipeline for more
>reliable testing, namely:
1) Shuffle code that re-uses reduce-side objects so we can detect bugs caused
by object modification, and
2) Serializes/deserializes DoFns before running them in order to test for any
non-serializable code that sneaks into a pipeline.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
