[
https://issues.apache.org/jira/browse/MAHOUT-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13988088#comment-13988088
]
Dmitriy Lyubimov commented on MAHOUT-1529:
------------------------------------------
[~avati] the use patterns of optimizer checkpoints are discussed at length in
my talk. Two basics use cases are explicit management of cache policies and
common computational path.
[~ssc]
bq. Why would we need that explicit execute operator for Stratosphere?
Correct me if i am reading Stratosphere wrong. (I still haven't run a single
program on it, please forgive me being a bit superfluous here). Stratosphere
programming api implies that we may define more than 1 sink in the graph (i.e.
writeDRM() calls) without triggering computational action. How would we
trigger it if sink definitions such as writeDRM don't trigger it anymore?
Also not clear with collect() stuff, i guess it doesn't have a direct mapping
either until Stephan finishes his promised piece on it.
-d
> Finalize abstraction of distributed logical plans from backend operations
> -------------------------------------------------------------------------
>
> Key: MAHOUT-1529
> URL: https://issues.apache.org/jira/browse/MAHOUT-1529
> Project: Mahout
> Issue Type: Improvement
> Reporter: Dmitriy Lyubimov
> Fix For: 1.0
>
>
> We have a few situations when algorithm-facing API has Spark dependencies
> creeping in.
> In particular, we know of the following cases:
> (1) checkpoint() accepts Spark constant StorageLevel directly;
> (2) certain things in CheckpointedDRM;
> (3) drmParallelize etc. routines in the "drm" and "sparkbindings" package.
> (5) drmBroadcast returns a Spark-specific Broadcast object
--
This message was sent by Atlassian JIRA
(v6.2#6252)
