[
https://issues.apache.org/jira/browse/MAHOUT-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14002651#comment-14002651
]
Dmitriy Lyubimov commented on MAHOUT-1529:
------------------------------------------
ok i started nudging this a bit forward, did a couple of fairly drastical
refactoring, moving api parts to math-scala. math-scala should compile .
Decompositions are moved too.
Things left include moving package-level routines requiring implicit context;
fixing spark and spark-shell modules; moving tests where appropriate.
With tests. a little conundrum is such that we don't have a "local" engine --
we would use "Spark local" for that, i.e. some concrete engine. So even though
decomposition code now completely lives in math-scala with no spark
dependencies, it looks like its tests will still have to live in spark module,
where unit-testing in local spark mode is defined. It kinda makes sense, since
we probably will want to run MathSuite separately for each engine we add; but
is a bit weird since it keeps something like ssvd() and its engine-specific
tests apart.
> Finalize abstraction of distributed logical plans from backend operations
> -------------------------------------------------------------------------
>
> Key: MAHOUT-1529
> URL: https://issues.apache.org/jira/browse/MAHOUT-1529
> Project: Mahout
> Issue Type: Improvement
> Reporter: Dmitriy Lyubimov
> Fix For: 1.0
>
>
> We have a few situations when algorithm-facing API has Spark dependencies
> creeping in.
> In particular, we know of the following cases:
> -(1) checkpoint() accepts Spark constant StorageLevel directly;-
> (2) certain things in CheckpointedDRM;
> (3) drmParallelize etc. routines in the "drm" and "sparkbindings" package.
> (5) drmBroadcast returns a Spark-specific Broadcast object
> *Current tracker:*
> https://github.com/dlyubimov/mahout-commits/tree/MAHOUT-1529.
> *Pull requests are welcome*.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
