Richard Zowalla created OPENNLP-1950:
----------------------------------------
Summary: ADChunkSampleStream fails with NPE on the FlorestaVirgem
corpus
Key: OPENNLP-1950
URL: https://issues.apache.org/jira/browse/OPENNLP-1950
Project: OpenNLP
Issue Type: Bug
Reporter: Richard Zowalla
Reading {{FlorestaVirgem\_CF\_3.0\_ad.txt}} \(Linguateca, ISO\-8859\-1\) with
{{ADChunkSampleStream}} fails after 4,641 samples:
{noformat}
java.lang.NullPointerException
at java.base/java.util.Objects.requireNonNull\(Objects.java:220\)
at java.base/java.util.List.copyOf\(List.java:1191\)
at opennlp.tools.chunker.ChunkSample.\(ChunkSample.java:68\)
at
opennlp.tools.formats.ad.ADChunkSampleStream.read\(ADChunkSampleStream.java:116\)
{noformat}
One of the token, tag or chunk lists passed to {{ChunkSample}} contains a
{{null}} element. {{ArvoresDeitadasEval}} only uses Bosque for chunking, so the
evals don't catch it.
_Fix:_ find the construct that yields the null tag, handle it or skip the
sample with a warning, and add a test with that construct.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
