Re: Vectorization, dictionary size, OpenObjectIntHashMap and OOM

[Sean Owen]

It's a trie? Yeah that could be a big win. It gets tricky with Unicode, but
imagine there is a lot of gain even so.
"Bigrams over 11M terms" jumped out too as a place to start.
(I don't see any particular backwards compatibility issue with Lucene 3 to
even worry about.)