Revision: 19443
http://sourceforge.net/p/gate/code/19443
Author: ian_roberts
Date: 2016-06-28 12:42:13 +0000 (Tue, 28 Jun 2016)
Log Message:
-----------
A term processor that completely ignores null and empty string terms, useful
when you want to index token features that may not be present on every token.
Added Paths:
-----------
mimir/trunk/mimir-core/src/gate/mimir/util/IgnoreEmptiesTermProcessor.java
Added:
mimir/trunk/mimir-core/src/gate/mimir/util/IgnoreEmptiesTermProcessor.java
===================================================================
--- mimir/trunk/mimir-core/src/gate/mimir/util/IgnoreEmptiesTermProcessor.java
(rev 0)
+++ mimir/trunk/mimir-core/src/gate/mimir/util/IgnoreEmptiesTermProcessor.java
2016-06-28 12:42:13 UTC (rev 19443)
@@ -0,0 +1,60 @@
+/*
+ * IgnoreEmptiesTermProcessor.java
+ *
+ * Copyright (c) 2012, The University of Sheffield.
+ *
+ * This file is part of GATE MÃmir (see http://gate.ac.uk/family/mimir.html),
+ * and is free software, licenced under the GNU Lesser General Public License,
+ * Version 3, June 2007 (also included with this distribution as file
+ * LICENCE-LGPL3.html).
+ *
+ * Mark A. Greenwood, 5 January 2012
+ *
+ * $Id$
+ */
+package gate.mimir.util;
+
+import it.unimi.dsi.lang.MutableString;
+import it.unimi.di.big.mg4j.index.TermProcessor;
+
+/**
+ * Term processor that completely ignores null or empty-string terms rather
+ * than indexing them as the empty string. This is useful when you want to
+ * index token features that are not present on every token.
+ */
+public class IgnoreEmptiesTermProcessor implements TermProcessor
+{
+ private final static IgnoreEmptiesTermProcessor INSTANCE = new
IgnoreEmptiesTermProcessor();
+
+ public final static TermProcessor getInstance() {
+ return INSTANCE;
+ }
+
+ private IgnoreEmptiesTermProcessor() {
+ //nothing to do but this method forces people to use the getInstance()
method;
+ }
+
+ @Override
+ public boolean processTerm(final MutableString term) {
+ return (term != null && term.length() > 0);
+ }
+
+ @Override
+ public boolean processPrefix(final MutableString prefix) {
+ return processTerm(prefix);
+ }
+
+ private Object readResolve() {
+ return INSTANCE;
+ }
+
+ @Override
+ public String toString() {
+ return this.getClass().getName();
+ }
+
+ @Override
+ public IgnoreEmptiesTermProcessor copy() {
+ return this;
+ }
+}
Property changes on:
mimir/trunk/mimir-core/src/gate/mimir/util/IgnoreEmptiesTermProcessor.java
___________________________________________________________________
Added: svn:keywords
## -0,0 +1 ##
+Id
\ No newline at end of property
Added: svn:eol-style
## -0,0 +1 ##
+native
\ No newline at end of property
This was sent by the SourceForge.net collaborative development platform, the
world's largest Open Source development site.
------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
GATE-cvs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/gate-cvs
