I would like to specify a particular payload for all tokens emitted from a tokenizer, but don't see a clear way to do this. Ideally I could specify that something like the DelimitedPayloadTokenFilter be run on the entire field and then standard analysis be done on the rest of the field, so in the case that I had the following text
this is a test\Foo I would like to create tokens "this", "is", "a", "test" each with a payload of Foo. From what I'm seeing though only test get's the payload. Is there anyway to accomplish this or will I need to implement a custom tokenizer?
