Hi folks,
Continuing down the path of working with the solr datastore I have run into
another issue related to how the data store goes about looking at
attributes/fields when creating feature types. Basically the data store
ends up doing a query against the solr server for each field that is
returned from the luke admin endpoint. The reason being to determine if the
field actually shows up in any documents.
The code that does this is here:
https://github.com/geotools/geotools/blob/master/modules/unsupported/solr/src/main/java/org/geotools/data/solr/SolrDataStore.java#L149
As you can imagine this becomes a problem for indexes with many fields. I
am working with one that has ~700 fields and suffice it to say that loading
the feature types from the datastore takes quite some time.
A colleague of mine who is a solr developer pointed out that the
LukeResponse.FieldInfo class has a document count on it:
https://github.com/apache/lucene-solr/blob/trunk/solr/solrj/src/java/org/apache/solr/client/solrj/response/LukeResponse.java#L170
The caveat with it is that it doesn't account for deleted documents in the
index, which perhaps is why it wasn't used in the first place. That said if
the point is to determine "field usage" it will probably suffice in many
cases.
So... I would like to utilize this property in some shape or form to avoid
the extensive lookup process. Either by exposing it as an option or by just
changing the behaviour. Either way is fine by me, whatever Andrea and the
other maintianers prefer will work.
Thanks!
-Justin
------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
