Thanks, Toke.
I’m still surveying the code; do you know of a place in the code that might be
more problematic?
We’d be mainly concerned about searching, sorting and (simple, low-cardinality)
faceting working for us.
Some features like grouping are not currently used by us, so in a pinch a
custom build might be a partial patch. We’ll just have to see.
- Ronald S. Wood
On 8/25/16, 06:50, "Toke Eskildsen" <t...@statsbiblioteket.dk> wrote:
Ronald Wood <rw...@smarsh.com> wrote:
> Did you find you had to do a full conversion all at once because simply
turning on
> docvalues in the schema caused issues?
Yes.
> I ask because my presupposition has been that we could turn it on without
any
> harm as we incrementally converted our indexes.
If you don't use the field for any queries until all the values has been
re-build, I guess that would work. The I-am-not-so-sure-part is how the merger
handles the case of a field having DocValues in one segment and not in another.
But mixing docValued & non-docValued segments with a schema that says
DocValues will make DocValue-using queries fail, as you seem to have
encountered:
> But this won’t work if enabling docvalues in the schema will lead to
errors when
> fields don’t have docvalues actually populated. I.e.. the
“IllegalStateException:
> unexpected docvalues type NONE for field 'id' (expected=SORTED)” error I
see.
> I’m still trying to get to the bottom of whether that error means I
cannot safely do
> an incremental conversion in-place.
When you enable docValues in the Solr schema, Solr also uses that
information when reading the data from the segments, so when the code detects
the missing docValues in the segments themselves, it is already too far down
the execution path to change strategy. Basically the contract (schema) is
broken, so all bets are off.
Your gradual enabling would work, at least for faceting, if it was possible
to force the selection code to only use the indexed values, but the current
code does not have an option for forcing the use of the indexed value. You
could add it as a (small) hack, if you are comfortable with that. I don't know
how easy or hard it would to hack grouping.
- Toke Eskildsen
