[jira] [Commented] (OPENNLP-1479) Write better tests for pattern verification (tokenizers)

[ASF GitHub Bot (Jira)]

    [ 
https://issues.apache.org/jira/browse/OPENNLP-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17795267#comment-17795267
 ] 

ASF GitHub Bot commented on OPENNLP-1479:
-----------------------------------------

kinow commented on PR #563:
URL: https://github.com/apache/opennlp/pull/563#issuecomment-1849666153

   Have also submitted a suggestion to Google Translate to fix is on their end 
too (cc @franra9)  :bear: 
   
   ![Screenshot from 2023-12-11 
10-40-18](https://github.com/apache/opennlp/assets/304786/ae836ebb-8f34-4558-ba7f-690242f1a8ea)
   
   




> Write better tests for pattern verification (tokenizers)
> --------------------------------------------------------
>
>                 Key: OPENNLP-1479
>                 URL: https://issues.apache.org/jira/browse/OPENNLP-1479
>             Project: OpenNLP
>          Issue Type: Improvement
>          Components: Tokenizer
>    Affects Versions: 2.1.1
>            Reporter: Bruno P. Kinoshita
>            Assignee: Lara Marinov
>            Priority: Major
>             Fix For: 2.3.2
>
>
> From [https://github.com/apache/opennlp/pull/516#issuecomment-1455015772]
> At the moment our tests verify that the tokenizer objects are created 
> correctly (i.e. tests getters and setters, constructor, etc.), without 
> verifying the actual behavior when used in conjunction with other classes 
> (factory, tokenizer, trainers, etc).
> It would be best to test the patterns used in the factories for different 
> languages with some interesting sample data (maybe something from project 
> gutenberg, open source news sites, etc.).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)