[
https://issues.apache.org/jira/browse/MAHOUT-944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208376#comment-13208376
]
Grant Ingersoll commented on MAHOUT-944:
----------------------------------------
Looks reasonable at first blush, with a few comments:
# Why the need to get the scorer, etc.? I wonder if it would be more efficient
to just have a simple Collector that did the work and we skipped scoring, etc.
# What's the benefit of the Field/Extra Fields thing? Would it make sense to
just have List<String> fields? If there is more than one, let's concat,
otherwise...
# LuceneIndexToSequenceFilesConfiguration -> LISFConfig? Let's shorten that
sucker up as the verbosity doesn't really get us anything
# In the Driver, please switch the input args processing to the AbstractJob
model. See KMeansDriver as an example
# Even if we don't have a M/R job, it would be nice if we could take in, via
the driver, multiple indexes. You could imagine piling all of your shards
together and then converting them all.
# Have you tested this with numeric (trie) fields?
# The integration pom.xml inherits from the parent, which has Lucene defined in
it, so no need to mod the integration one, I think. We should upgrade the
parent one to 3.5.0.
A better name for all of this is probably LuceneStorageTo... as it implies that
the fields must have storage. I could see us having another implementation
that works on the posting list itself
> LuceneIndexToSequenceFiles (lucene2seq) utility
> -----------------------------------------------
>
> Key: MAHOUT-944
> URL: https://issues.apache.org/jira/browse/MAHOUT-944
> Project: Mahout
> Issue Type: New Feature
> Components: Integration
> Affects Versions: 0.5
> Reporter: Frank Scholten
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 0.7
>
> Attachments: MAHOUT-944.patch, MAHOUT-944.patch, MAHOUT-944.patch,
> MAHOUT-944.patch
>
>
> Here is a lucene2seq tool I used in a project. It creates sequence files
> based on the stored fields of a lucene index.
> The output from this tool can be then fed into seq2sparse and from there you
> can do text clustering.
> Comes with Java bean configuration.
> Let me know what you think. Some CLI code can be added later on. I used this
> for a small-scale project +- 100.000 docs. Is a MR version useful or is that
> overkill?
> See https://github.com/frankscholten/mahout/tree/lucene2seq for commits and
> review comments from Simon Willnauer (Thanks Simon!)
> or the attached patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
