Sorry, I accidentally hit send somehow.
Here is my ant build.xml for building the CustomPlugins jar file: <?xml version="1.0" encoding="UTF-8" standalone="no"?> <project basedir="." default="jar" name="CustomPlugins"> <property environment="env"/> <property name="debuglevel" value="source,lines,vars"/> <property name="target" value="1.6"/> <property name="source" value="1.6"/> <path id="CustomPlugins.classpath"> <pathelement location="bin"/> <pathelement location="lib/apache-solr-core-4.0.0.jar"/> <pathelement location="lib/lucene-core-4.0.0.jar"/> <pathelement location="lib/lucene-queries-4.0.0.jar"/> <pathelement location="lib/apache-solr-solrj-4.0.0.jar"/> </path> <target name="init"> <mkdir dir="bin"/> <copy includeemptydirs="false" todir="bin"> <fileset dir="src"> <exclude name="**/*.launch"/> <exclude name="**/*.java"/> </fileset> </copy> </target> <target name="clean"> <delete dir="bin"/> <delete dir="dist"/> </target> <target depends="clean" name="cleanall"/> <target depends="build-subprojects,build-project" name="build"/> <target name="build-subprojects"/> <target depends="init" name="build-project"> <echo message="building project ${ant.project.name}: ${ant.file}"/> <javac debug="true" debuglevel="DEBUG" destdir="bin" source="${source}" target="${target}"> <src path="src"/> <classpath refid="CustomPlugins.classpath"/> </javac> </target> <target depends="build" name="jar"> <echo message="${ant.project.name}: ${ant.file}"/> <mkdir dir="dist"/> <jar jarfile="dist/CustomPlugins.jar" basedir="bin" includes="**/*.class"> </jar> </target> </project> Here is the code for the GeneticLocation.java file. It is not complete, and might have errors in it. I used PointType as my starting point, and trimmed out what I didn't think I needed. I want to verify that I can load it now before I muck with it any further. package org.jax.mgi.fe.solrplugin; import java.io.IOException; import java.util.ArrayList; import java.util.List; import java.util.Map; import org.apache.lucene.document.FieldType; import org.apache.lucene.index.IndexableField; import org.apache.lucene.search.BooleanClause; import org.apache.lucene.search.BooleanQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.SortField; import org.apache.solr.common.SolrException; import org.apache.solr.common.params.MapSolrParams; import org.apache.solr.common.params.SolrParams; import org.apache.solr.response.TextResponseWriter; import org.apache.solr.schema.AbstractSubTypeFieldType; import org.apache.solr.schema.IndexSchema; import org.apache.solr.schema.SchemaField; import org.apache.solr.search.QParser; import org.apache.lucene.queries.function.ValueSource; import org.apache.lucene.queries.function.valuesource.VectorValueSource; /** * Custom solr field type to support querying against genetic location data. */ public class GeneticLocation extends AbstractSubTypeFieldType { @Override protected void init(IndexSchema schema, Map<String, String> args) { SolrParams p = new MapSolrParams(args); this.schema = schema; super.init(schema, args); // cache suffixes createSuffixCache(2); } @Override public boolean isPolyField() { return true; // really only true if the field is indexed } @Override public IndexableField[] createFields(SchemaField field, Object value, float boost) { String externalVal = value.toString(); String[] coords = externalVal.split("-"); if(coords.length!=2) { throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,"Invalid coordinate format for "+externalVal); } List<IndexableField> f = new ArrayList<IndexableField>(); if (field.indexed()) { SchemaField coordField1 = subField(field,0); SchemaField coordField2 = subField(field,1); f.add(coordField1.createField(coords[0],coordField1.indexed() && !coordField1.omitNorms() ? boost : 1f)); f.add(coordField2.createField(coords[1],coordField2.indexed() && !coordField2.omitNorms() ? boost : 1f)); } if (field.stored()) { String storedVal = externalVal; // normalize or not? FieldType customType = new FieldType(); customType.setStored(true); f.add(createField(field.getName(), storedVal, customType, 1f)); } return (IndexableField[]) f.toArray(); } @Override public ValueSource getValueSource(SchemaField field, QParser parser) { ArrayList<ValueSource> vs = new ArrayList<ValueSource>(); SchemaField coord1 = subField(field, 0); vs.add(coord1.getType().getValueSource(coord1, parser)); SchemaField coord2 = subField(field, 1); vs.add(coord2.getType().getValueSource(coord2, parser)); return new CoordTypeValueSource(field, vs); } /** * It never makes sense to create a single field, so make it impossible to happen by * throwing UnsupportedOperationException * */ @Override public IndexableField createField(SchemaField field, Object value, float boost) { throw new UnsupportedOperationException("PointType uses multiple fields. field=" + field.getName()); } @Override public void write(TextResponseWriter writer, String name, IndexableField f) throws IOException { writer.writeStr(name, f.stringValue(), true); } @Override public SortField getSortField(SchemaField field, boolean top) { throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Sorting not supported on PointType " + field.getName()); } @Override public Query getFieldQuery(QParser parser, SchemaField field, String externalVal) { String[] coords = externalVal.split("-"); if(coords.length!=2) { throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,"Invalid coordinate format for "+externalVal); } BooleanQuery bq = new BooleanQuery(true); SchemaField coord1 = subField(field, 0); Query tq1 = coord1.getType().getFieldQuery(parser, coord1, coords[0]); bq.add(tq1, BooleanClause.Occur.MUST); SchemaField coord2 = subField(field, 1); Query tq2 = coord2.getType().getFieldQuery(parser, coord1, coords[1]); bq.add(tq2, BooleanClause.Occur.MUST); return bq; } } class CoordTypeValueSource extends VectorValueSource { private final SchemaField sf; public CoordTypeValueSource(SchemaField sf, List<ValueSource> sources) { super(sources); this.sf = sf; } @Override public String name() { return "point"; } @Override public String description() { return name()+"("+sf.getName()+")"; } } ________________________________________ From: Kevin Stone Sent: Saturday, July 20, 2013 8:24 AM To: solr-user@lucene.apache.org Subject: RE: custom field type plugin Thank you for the links, they really helped me understand. I see how the spatial solution works now. I think this could work as a good backup if I cannot get the custom field type working. The custom field would ideally be a bit more robust than what I mentioned before, because a region really means four pieces, a chromosome (e.g. 1-22), a start base pair, an end base pair, and the direction (forward or reverse). But if need be, the chomosome and direction can be multiplied into the base pairs to get it down to two translated numbers. As for the upper bounds, I do have an idea, but it would be a large number, say between 1 and 10 billion depending on how I translate the values. I'll just have to try it out I guess. Ok, now back to the custom field problem. From here on I'll spam source code and stack traces. I started fresh, removing all places where I may have had my jar file, and popped in a fresh solr.war. I define the plugin class in my schema like this: <fieldType name="geneticLocation" class="org.jax.mgi.fe.solrplugin.GeneticLocation" omitNorms="true"/> and use it here: <field name="coordinate" type="geneticLocation" indexed="true" stored="true" /> Ok, when I start solr, I get this error saying it can't find the plugin class that is defined in my schema. org.apache.solr.common.SolrException: Plugin init failure for [schema.xml] fieldType "geneticLocation": Error loading class 'org.jax.mgi.fe.solrplugin.GeneticLocation' at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:177) at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:369) at org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:113) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:846) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:534) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:356) ...etc... Caused by: org.apache.solr.common.SolrException: Error loading class 'org.jax.mgi.fe.solrplugin.GeneticLocation' at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:436) at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:457) at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:453) ...etc... So, that's all fine. In my solr.xml, I define this sharedLib folder: <solr persistent="true" sharedLib="../lib"> I shut the server down, drop in my CustomPlugins.jar file and start the server back up. And... I got a different error! It said I was missing the subFieldType or subFieldSuffix in my fieldType definition. So I added one 'subFieldSuffix="_gl". Then I restart the server thinking that I'm making progress, and I get the old error again. I pulled out the jar, did the above test again to verify that it couldn't find my plugin. Then I re-add it and restart. Nope, still this error about AbstractSubTypeFieldType. Here is the full stack trace: SEVERE: null:java.lang.NoClassDefFoundError: org/apache/solr/schema/AbstractSubTypeFieldType at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:791) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:401) at java.lang.ClassLoader.loadClass(ClassLoader.java:410) at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789) at java.lang.ClassLoader.loadClass(ClassLoader.java:410) at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:266) at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:420) at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:457) at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:453) at org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:81) at org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:43) at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:151) at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:369) at org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:113) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:846) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:534) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:356) at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:308) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:107) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713) at org.mortbay.jetty.servlet.Context.startContext(Context.java:140) at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130) at org.mortbay.jetty.Server.doStart(Server.java:224) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:985) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.mortbay.start.Main.invokeMain(Main.java:194) at org.mortbay.start.Main.start(Main.java:534) at org.mortbay.start.Main.start(Main.java:441) at org.mortbay.start.Main.main(Main.java:119) Caused by: java.lang.ClassNotFoundException: org.apache.solr.schema.AbstractSubTypeFieldType at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) ... 57 more Here is my ant build.xml for building the CustomPlugins jar file: ________________________________________ From: Chris Hostetter [hossman_luc...@fucit.org] Sent: Friday, July 19, 2013 10:08 PM To: solr-user@lucene.apache.org Subject: RE: custom field type plugin : I can try again this weekend to get a clean environment. However, the : order I did things in was the reverse of what you suggest. I got the Hmmm... then i'm kind of at a loss to explain what you're describing. need to see more details of the configs, dir structure, jar structure, etc... : The spatial feature looks intriguing, although I have no idea if it : could fit my use case. It looks fairly complex a concept, but maybe it : is all the different shapes and geometry that is confusing me. If I : thought of my problem in terms of geometry, I would say a chromosome : region is like a segment of a line. I would need to define multiple line : segments and be able to query by a single point and only return : documents that have a line segment that the single point falls on. Does : that make sense? Is that at all doable with a spatial query? The tricky thing about leveraging the spatial stuff for this type of problem is that it's frequently better to *not* let yourself think in terms of the a straightforward mapping between your problem space and geometry. Instead of modeling your data as documents containing multiple line segments and trying to search for a document containing a line segment that contains your 1D point, imagine modeling your data as documents containing multiple 2D points, one point per "range", where the X coordinate is the lower bound of your range, and the Y axis is the upper bound of the range... https://people.apache.org/~hossman/spatial-for-non-spatial-meetup-20130117/#slide8 ...and to find all documents containing a range that contains a specified input value V, you then query for all documents containing points inside of a specially crafted bounding box based on V... https://people.apache.org/~hossman/spatial-for-non-spatial-meetup-20130117/#slide11 ..the big caveat to this approach that i failed to mention before is that it presumes there is an absolute min/max definable for the overall range of values you are dealing with so that you can define the bounding boxes appropriates -- otherwise the geometery won't work. In anycase .. it's an interesting idea i wanted to through out there for you to consider i case it worked for you before you jumped through a tone of hoops trying to get a new custom FieldType to work. -Hoss The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.