mawiesne commented on code in PR #829:
URL: https://github.com/apache/opennlp/pull/829#discussion_r2248142968
##########
opennlp-core/opennlp-runtime/src/main/java/opennlp/tools/sentdetect/SentenceDetectorME.java:
##########
@@ -340,15 +340,27 @@ protected boolean isAcceptableBreak(CharSequence s, int
fromIndex, int candidate
return true;
for (StringList abb : abbDict) {
- String token = abb.getToken(0);
+ final String token = abb.getToken(0);
+ final int tokenPosition = s.toString().indexOf(token, fromIndex);
+ if (tokenPosition == -1) {
+ continue; // skip fast
+ }
+ final char prevChar = s.charAt(tokenPosition - 1);
int tokenLength = token.length();
- int tokenPosition = s.toString().indexOf(token, fromIndex);
- if (tokenPosition + tokenLength < candidateIndex || tokenPosition >
candidateIndex)
- continue;
+ if (tokenPosition + tokenLength < candidateIndex || tokenPosition >
candidateIndex ||
+ /*
+ * Note:
+ * Skip abbreviation candidate if regular characters exist directly
before it,
+ * That is, any letter or digit except: a whitespace, an apostrophe,
or an opening round bracket.
Review Comment:
I don't get "\.". There is no check for an escaped dot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
