NullPointerException for ExternalFileField when key field has no terms
Hi, I use various ID fields as the keys for various ExternalFileField fields, and I have noticed that I will sometimes get the following error: ERROR org.apache.solr.servlet.SolrDispatchFilter û null:java.lang.NullPointerException at org.apache.solr.search.function.FileFloatSource.getFloats(FileFloatSource.java:273) at org.apache.solr.search.function.FileFloatSource.access$000(FileFloatSource.java:51) at org.apache.solr.search.function.FileFloatSource$2.createValue(FileFloatSource.java:147) at org.apache.solr.search.function.FileFloatSource$Cache.get(FileFloatSource.java:190) at org.apache.solr.search.function.FileFloatSource.getCachedFloats(FileFloatSource.java:141) at org.apache.solr.search.function.FileFloatSource.getValues(FileFloatSource.java:84) at org.apache.solr.response.transform.ValueSourceAugmenter.transform(ValueSourceAugmenter.java:95) at org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:252) at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:170) at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:184) at org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:300) at org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:96) at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:61) at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:765) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:426) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:368) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Unknown Source) The source code referenced in the error is below (FileFloatSource.java:273): TermsEnum termsEnum = MultiFields.getTerms(reader, idName).iterator(null); So if there are no terms in the index for the key field, then getTerms will return null, and of course trying to call iterator on null will cause the exception. For my use-case, it makes sense that the key field may have no terms (initially) because there are various types of documents sharing the index, and they will not all exist at the onset. The default value for the EFF would suffice in those cases. Is this worthy of a JIRA? I have gone through whatever documentation I can find for ExternalFileField and I can't seem to find anything about requiring key terms first. It seems that this error is not encountered often because users generally set the unique key field as the external file key field, so it always exists. The workaround is to ensure at least
RE: NullPointerException for ExternalFileField when key field has no terms
Hi - yes it is worth a ticket as the javadoc says it is ok: http://lucene.apache.org/solr/4_10_1/solr-core/org/apache/solr/schema/ExternalFileField.html -Original message- From:Matthew Nigl matthew.n...@gmail.com Sent: Wednesday 8th October 2014 14:48 To: solr-user@lucene.apache.org Subject: NullPointerException for ExternalFileField when key field has no terms Hi, I use various ID fields as the keys for various ExternalFileField fields, and I have noticed that I will sometimes get the following error: ERROR org.apache.solr.servlet.SolrDispatchFilter û null:java.lang.NullPointerException at org.apache.solr.search.function.FileFloatSource.getFloats(FileFloatSource.java:273) at org.apache.solr.search.function.FileFloatSource.access$000(FileFloatSource.java:51) at org.apache.solr.search.function.FileFloatSource$2.createValue(FileFloatSource.java:147) at org.apache.solr.search.function.FileFloatSource$Cache.get(FileFloatSource.java:190) at org.apache.solr.search.function.FileFloatSource.getCachedFloats(FileFloatSource.java:141) at org.apache.solr.search.function.FileFloatSource.getValues(FileFloatSource.java:84) at org.apache.solr.response.transform.ValueSourceAugmenter.transform(ValueSourceAugmenter.java:95) at org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:252) at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:170) at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:184) at org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:300) at org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:96) at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:61) at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:765) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:426) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:368) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Unknown Source) The source code referenced in the error is below (FileFloatSource.java:273): TermsEnum termsEnum = MultiFields.getTerms(reader, idName).iterator(null); So if there are no terms in the index for the key field, then getTerms will return null, and of course trying to call iterator on null will cause the exception. For my use-case, it makes sense that the key field may have no terms (initially) because there are various types of documents
Re: NullPointerException for ExternalFileField when key field has no terms
Thanks Markus. I initially interpreted the line It's OK to have a keyField value that can't be found in the index as meaning that the key field value in the external file does not have to exist as a term in the index. On 8 October 2014 23:56, Markus Jelsma markus.jel...@openindex.io wrote: Hi - yes it is worth a ticket as the javadoc says it is ok: http://lucene.apache.org/solr/4_10_1/solr-core/org/apache/solr/schema/ExternalFileField.html -Original message- From:Matthew Nigl matthew.n...@gmail.com Sent: Wednesday 8th October 2014 14:48 To: solr-user@lucene.apache.org Subject: NullPointerException for ExternalFileField when key field has no terms Hi, I use various ID fields as the keys for various ExternalFileField fields, and I have noticed that I will sometimes get the following error: ERROR org.apache.solr.servlet.SolrDispatchFilter û null:java.lang.NullPointerException at org.apache.solr.search.function.FileFloatSource.getFloats(FileFloatSource.java:273) at org.apache.solr.search.function.FileFloatSource.access$000(FileFloatSource.java:51) at org.apache.solr.search.function.FileFloatSource$2.createValue(FileFloatSource.java:147) at org.apache.solr.search.function.FileFloatSource$Cache.get(FileFloatSource.java:190) at org.apache.solr.search.function.FileFloatSource.getCachedFloats(FileFloatSource.java:141) at org.apache.solr.search.function.FileFloatSource.getValues(FileFloatSource.java:84) at org.apache.solr.response.transform.ValueSourceAugmenter.transform(ValueSourceAugmenter.java:95) at org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:252) at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:170) at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:184) at org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:300) at org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:96) at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:61) at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:765) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:426) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:368) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Unknown Source) The source code referenced
