[
https://issues.apache.org/jira/browse/MAHOUT-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986964#comment-13986964
]
Dmitriy Lyubimov commented on MAHOUT-1529:
------------------------------------------
yes i figured as much. Multiple sinks are defined, and then the optimizer there
just goes over what we were trying to do with collapsing common paths or
something like that.
This is actually pretty cool. However, it does pose a conundrum of reconciling
with current semantics.
> Finalize abstraction of distributed logical plans from backend operations
> -------------------------------------------------------------------------
>
> Key: MAHOUT-1529
> URL: https://issues.apache.org/jira/browse/MAHOUT-1529
> Project: Mahout
> Issue Type: Improvement
> Reporter: Dmitriy Lyubimov
> Fix For: 1.0
>
>
> We have a few situations when algorithm-facing API has Spark dependencies
> creeping in.
> In particular, we know of the following cases:
> (1) checkpoint() accepts Spark constant StorageLevel directly;
> (2) certain things in CheckpointedDRM;
> (3) drmParallelize etc. routines in the "drm" and "sparkbindings" package.
> (5) drmBroadcast returns a Spark-specific Broadcast object
--
This message was sent by Atlassian JIRA
(v6.2#6252)
