Re: Multi-valued fields and TokenStream

2014-11-06 Thread Robert Muir
On Thu, Nov 6, 2014 at 3:41 PM, david.w.smi...@gmail.com wrote: > On Thu, Nov 6, 2014 at 3:19 PM, Robert Muir wrote: >> >> Do the concatenation yourself with your own TokenStream. You can index >> a field with a tokenstream for expert cases (the individual stored >> values can be added separately

Re: Multi-valued fields and TokenStream

2014-11-06 Thread Steve Rowe
> On Nov 6, 2014, at 3:13 PM, david.w.smi...@gmail.com wrote: > > Are you suggesting that DefaultIndexingChain.PerField.invert(boolean > firstValue) would, prior to calling reset(), call > setPositionIncrement(Integer.MAX_VALUE), but only when ‘firstValue’ is false? > H. I guess that wou

Re: Multi-valued fields and TokenStream

2014-11-06 Thread david.w.smi...@gmail.com
On Thu, Nov 6, 2014 at 3:19 PM, Robert Muir wrote: > Do the concatenation yourself with your own TokenStream. You can index > a field with a tokenstream for expert cases (the individual stored > values can be added separately) > Yes, but that’s quite awkward and a fair amount of surrounding code

Re: Multi-valued fields and TokenStream

2014-11-06 Thread Robert Muir
Do the concatenation yourself with your own TokenStream. You can index a field with a tokenstream for expert cases (the individual stored values can be added separately) No need to make the tokenstream API more complicated: its already very complicated. On Thu, Nov 6, 2014 at 3:13 PM, david.w.smi

Re: Multi-valued fields and TokenStream

2014-11-06 Thread david.w.smi...@gmail.com
Are you suggesting that DefaultIndexingChain.PerField.invert(boolean firstValue) would, prior to calling reset(), call setPositionIncrement(Integer.MAX_VALUE), but only when ‘firstValue’ is false? H. I guess that would work, although it seems a bit hacky and it’s tying this to a specific attr

Re: Multi-valued fields and TokenStream

2014-11-06 Thread Steve Rowe
Maybe the position increment gap would be useful? If set to a value larger than likely max position for any individual value, it could be used to infer (non-)first-value-ness. > On Nov 5, 2014, at 1:03 PM, david.w.smi...@gmail.com wrote: > > Several times now, I’ve had to come up with work-aro

Multi-valued fields and TokenStream

2014-11-05 Thread david.w.smi...@gmail.com
Several times now, I’ve had to come up with work-arounds for a TokenStream not knowing it’s processing the first value or a subsequent-value of a multi-valued field. Two of these times, the use-case was ensuring the first position of each value started at a multiple of 1000 (or some other configur