[
https://issues.apache.org/jira/browse/CRUNCH-470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130996#comment-14130996
]
Micah Whitacre commented on CRUNCH-470:
---------------------------------------
Note here's a link for doing testing with the minicluster[1].
Something that'd help clarify is what problem/unnecessary burden you are
experiencing. This way we could focus on helping to make that easier instead
of just jumping to an implementation of something.
[1] - https://wiki.apache.org/hadoop/HowToDevelopUnitTests
> Add hdfs/yarn minicluster crunch pipeline
> -----------------------------------------
>
> Key: CRUNCH-470
> URL: https://issues.apache.org/jira/browse/CRUNCH-470
> Project: Crunch
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.8.3
> Reporter: Rafal Wojdyla
> Assignee: Josh Wills
> Priority: Minor
>
> Crunch currently has two pipelines:
> * MemPipeline
> * MRPipeline
> MemPipeline is in-memory pipelines based on local in-memory mapreduce mode.
> MRPipeline is distributed pipeline based on distributed MapReduce.
> Using HDFS/YARN Minicluster it's possible to better emulate Hadoop cluster,
> and it could be a 'final test' before running on the cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
