[
https://issues.apache.org/jira/browse/STANBOL-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125187#comment-13125187
]
Olivier Grisel commented on STANBOL-321:
----------------------------------------
We should have a look at how solr.HyphenatedWordsFilterFactory works and
whether we can reuse it directly for our purpose.
> Named Entity detection engine should better deal with hyphenated text
> ---------------------------------------------------------------------
>
> Key: STANBOL-321
> URL: https://issues.apache.org/jira/browse/STANBOL-321
> Project: Stanbol
> Issue Type: Bug
> Reporter: Olivier Grisel
> Assignee: Olivier Grisel
>
> We need some pre-processing to make it easier for OpenNLP to deal with
> hyphens, for instance this is an example of a real PDF:
> Sparse RBMs and sparse auto-encoders
> (RBM, SAE): In some of our experiments, we
> train sparse RBMs (Hinton et al., 2006) and
> sparse auto-encoders (Ranzato et al., 2007; Ben-
> gio et al., 2006), both using a logistic sigmoid non-
> linearity g(W x + b). These algorithms yield a set
> of weights W and biases b. To obtain the dictio-
> nary, D, we simply discard the biases and take
> D = W ⊤ , then normalize the columns of D.
> The current implementation return a TextAnnotation with entity-type = Person
> for the mention "Ben -" (which is then matched to "Ben Stiller" :P).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
