[
https://issues.apache.org/jira/browse/LUCENE-6536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14578664#comment-14578664
]
Uwe Schindler commented on LUCENE-6536:
---------------------------------------
bq. Personally, I think if someone wants to do this, a better integration point
is to make it a java 7 filesystem provider. That is really how such a
filesystem should work anyway.
I agree. This is how it should be. Once HDFS provides a Java 7
FileSystemProvider SPI (see,
http://docs.oracle.com/javase/7/docs/api/java/nio/file/spi/FileSystemProvider.html),
you just need to plug your HDFS JAR file into the classpath, then you would be
able to create a standard FSDirectory (NIO, Simple, mmap) using Paths.get(URI)
and you are done. No single line of Lucene code needed. I have no idea why
Hadoop does not yet provide a FileSystem implementation for Java 7 (maybe
because they are still on Java 6).
I would suggest that you talk with the Hadoop people about doing this
(including the block cache - I don't want to also take over responsibility for
the block cache in Lucene). Or you start your own project implementing the
FSProvider.
> Migrate HDFSDirectory from solr to lucene-hadoop
> ------------------------------------------------
>
> Key: LUCENE-6536
> URL: https://issues.apache.org/jira/browse/LUCENE-6536
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Greg Bowyer
> Labels: hadoop, hdfs, lucene, solr
> Attachments: LUCENE-6536.patch
>
>
> I am currently working on a search engine that is throughput orientated and
> works entirely in apache-spark.
> As part of this, I need a directory implementation that can operate on HDFS
> directly. This got me thinking, can I take the one that was worked on so hard
> for solr hadoop.
> As such I migrated the HDFS and blockcache directories out to a lucene-hadoop
> module.
> Having done this work, I am not sure if it is actually a good change, it
> feels a bit messy, and I dont like how the Metrics class gets extended and
> abused.
> Thoughts anyone
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
