[
https://issues.apache.org/jira/browse/MAHOUT-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986981#comment-13986981
]
Dmitriy Lyubimov commented on MAHOUT-1529:
------------------------------------------
first idea is that we introduce and require an explicit computational action
operator, something like drmExecute(implicit ctx) that would force computation
in stratosphere, and in case of spark, be ignored.
Similarly, cache instructions would just be ignored for Stratosphere.
Generally, coarse-iterative algorithms (<50 iterations -- ALS, ssvd) could just
ignore drmExecute() api altogether, leaving it for the caller to execute.
> Finalize abstraction of distributed logical plans from backend operations
> -------------------------------------------------------------------------
>
> Key: MAHOUT-1529
> URL: https://issues.apache.org/jira/browse/MAHOUT-1529
> Project: Mahout
> Issue Type: Improvement
> Reporter: Dmitriy Lyubimov
> Fix For: 1.0
>
>
> We have a few situations when algorithm-facing API has Spark dependencies
> creeping in.
> In particular, we know of the following cases:
> (1) checkpoint() accepts Spark constant StorageLevel directly;
> (2) certain things in CheckpointedDRM;
> (3) drmParallelize etc. routines in the "drm" and "sparkbindings" package.
> (5) drmBroadcast returns a Spark-specific Broadcast object
--
This message was sent by Atlassian JIRA
(v6.2#6252)
