[
https://issues.apache.org/jira/browse/STANBOL-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715209#comment-13715209
]
Rupert Westenthaler commented on STANBOL-1128:
----------------------------------------------
Update of the current development status:
* The engine requires FST structures:
* FST structures are read from within the SolrCore directory under
"fst/{name}.{lang}.fst"
* Runtime creation is optionally supported, but will require considerable
CPU, memory and time
* The engine requires the SolrCore to be registered and available
* NOTE that NO Solr queries are issued as linking in done in-memory!
* However selected fields of the Lucene documents for linked Entities need
to be read to obtain the labels, types and entity rankings
* Linking process:
* I contributed the ability to only link selected tokens to the
SolrTextTagger [1] project. This allows to have the same TextProcessing
configurations as currently supported by the EntityLinkingEngine.
* Detection of Entities with wrong Token order (e.g. {Family} {Given} for
persons) can not be supported. Users will need to create those Label
combinations at indexing time!
* Linking is done based on the Solr Field configuration. This means that
all available Solr Analyzers can be used for linking (even phonetic ...)
> Implement a Lucene FST based Entity Linking Engine (based on OpenSextant /
> SolrTextTagger)
> ------------------------------------------------------------------------------------------
>
> Key: STANBOL-1128
> URL: https://issues.apache.org/jira/browse/STANBOL-1128
> Project: Stanbol
> Issue Type: New Feature
> Components: Enhancement Engines
> Reporter: Rupert Westenthaler
> Assignee: Rupert Westenthaler
>
> This will implement an in-memory EntityLinking EnhancementEngine based on
> Lucenes FST (Finite State Transducer) technology.
> This engine could make direct use of the Classes contained in the OpenSextant
> / SolrTextTagger code [1]. This uses a two layered FST
> (1) to represent words and
> (2) to map those words to phrases
> With this it is possible to efficiently hold big vocabularies in memory (>
> 300MByte for geonames.org). See the presentation at [2] for more details.
> While the license is fully compatible (ASL 2.0) the library is currently not
> available on Maven Central. We need to contact the author regarding this.
> [1] https://github.com/OpenSextant/SolrTextTagger
> [2]
> http://www.lucenerevolution.org/2013/Text-Tagging-with-Finite-State-Transducers
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
