mawiesne commented on code in PR #38:
URL: https://github.com/apache/opennlp-sandbox/pull/38#discussion_r1688023847
##########
tf-ner-poc/src/main/java/org/apache/opennlp/namefinder/WordIndexer.java:
##########
@@ -25,23 +27,74 @@
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
+import java.util.Properties;
import java.util.regex.Pattern;
-import opennlp.tools.util.StringUtil;
-
public class WordIndexer {
private final Map<Character, Integer> char2idx;
private final Map<String, Integer> word2idx;
- public static final String UNK = "$UNK$";
- public static String NUM = "$NUM$";
+ public static final String UNK = "__UNK__";
+ public static final String NUM = "__NUM__";
- private final boolean lowerCase = false;
- private final boolean allowUnk = true;
+ private boolean lowerCase = false;
+ private boolean allowUnk = true;
+ private boolean allowNum = false;
private final Pattern digitPattern =
Pattern.compile("\\d+(,\\d+)*(\\.\\d+)?");
Review Comment:
This pattern can't be declared final, as there is a setter for this, see
below.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
