[
https://issues.apache.org/jira/browse/MAHOUT-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037628#comment-14037628
]
ASF GitHub Bot commented on MAHOUT-1541:
----------------------------------------
GitHub user pferrel opened a pull request:
https://github.com/apache/mahout/pull/22
MAHOUT-1541
Not ready for merge, just looking for a checkpoint review. The code works
and I'm reasonable happy with it. It needs review for:
* Design: will it be flexible, are there better ways to organize the code?
* Scala usage: I'm pretty new to Scala.
* Package structure: no clue here so its all in "drivers"
Please don't take the tests too seriously and some things are not
implemented. I'd like a once over before I do a bunch of test cases and finish
polishing up.
This will close three tickets when complete. MAHOUT-1541, MAHOUT-1568, and
MAHOUT-1569
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pferrel/mahout mahout-1541
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/mahout/pull/22.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22
----
commit 107a0ba9605241653a85b113661a8fa5c055529f
Author: pferrel <[email protected]>
Date: 2014-06-04T19:54:22Z
added Sebastian's CooccurrenceAnalysis patch updated it to use current
Mahout-DSL
commit 74b9921c4c9bd8903585bbd74d9e66298ea8b7a0
Author: pferrel <[email protected]>
Date: 2014-06-04T20:09:07Z
Adding stuff for itemsimilarity driver for Spark
commit a59265931ed3a51ba81e1a0a7171ebb102be4fa4
Author: pferrel <[email protected]>
Date: 2014-06-04T20:13:13Z
added scopt to pom deps
commit 16c03f7fa73c156859d1dba3a333ef9e8bf922b0
Author: pferrel <[email protected]>
Date: 2014-06-04T21:32:18Z
added Sebastian's MurmurHash changes
Signed-off-by: pferrel <[email protected]>
commit 8a4b4347ddb7b9ac97590aa20189d89d8a07a80a
Author: pferrel <[email protected]>
Date: 2014-06-04T21:33:11Z
Merge branch 'mahout-1464' into mahout-1541
commit 2f87f5433f90fa2c49ef386ca245943e1fc73beb
Author: pferrel <[email protected]>
Date: 2014-06-05T01:44:16Z
MAHOUT-1541 still working on this, some refactoring in the DSL for
abstracting away Spark has moved access to rddsno Jira is closed yet
commit c6adaa44c80bba99d41600e260bbb1ad5c972e69
Author: pferrel <[email protected]>
Date: 2014-06-05T16:52:23Z
MAHOUT-1464 import cleanup, minor changes to examples for running on Spark
Cluster
commit 2caceab31703ed214c1e66d5fc63b8bdb05d37a3
Author: pferrel <[email protected]>
Date: 2014-06-05T16:55:09Z
Merge branch 'mahout-1464' into mahout-1541
commit 6df6a54e3ff174d39bd817caf7d16c2d362be3f8
Author: pferrel <[email protected]>
Date: 2014-06-07T20:39:25Z
Merge branch 'master' into mahout-1541
commit a2f84dea3f32d3df3e98c61f085bc1fabd453551
Author: pferrel <[email protected]>
Date: 2014-06-07T21:27:06Z
drmWrap seems to be the answer to the changed DrmLike interface. Code works
again but more to do.
commit d3a2ba5027436d0abef67a1a5e82557064f4ba49
Author: pferrel <[email protected]>
Date: 2014-06-17T16:00:38Z
merged master, got new cooccurrence code
commit 4b2fb07b21a8ac2d532ee51b65b27d1482293cb0
Author: pferrel <[email protected]>
Date: 2014-06-19T17:08:02Z
for high level review, not ready for merge
commit 996ccfb82a8ed3ff90f51968e661b2449f3c4759
Author: pferrel <[email protected]>
Date: 2014-06-19T17:46:23Z
for high level review, not ready for merge. changed to dot notation
commit f62ab071869ee205ad398a3e094d871138e11a9e
Author: pferrel <[email protected]>
Date: 2014-06-19T18:13:44Z
for high level review, not ready for merge. fixed a couple scaladoc refs
----
> Create CLI Driver for Spark Cooccurrence Analysis
> -------------------------------------------------
>
> Key: MAHOUT-1541
> URL: https://issues.apache.org/jira/browse/MAHOUT-1541
> Project: Mahout
> Issue Type: Bug
> Components: CLI
> Reporter: Pat Ferrel
> Assignee: Pat Ferrel
>
> Create a CLI driver to import data in a flexible manner, create an
> IndexedDataset with BiMap ID translation dictionaries, call the Spark
> CooccurrenceAnalysis with the appropriate params, then write output with
> external IDs optionally reattached.
> Ultimately it should be able to read input as the legacy mr does but will
> support reading externally defined IDs and flexible formats. Output will be
> of the legacy format or text files of the user's specification with
> reattached Item IDs.
> Support for legacy formats is a question, users can always use the legacy
> code if they want this. Internal to the IndexedDataset is a Spark DRM so
> pipelining can be accomplished without any writing to an actual file so the
> legacy sequence file output may not be needed.
> Opinions?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
