indeed, I did look at Parquet and had the same feeling as Otis, some striking similarity with terminology used around stored fields.
If I got it right, parquet chunk stores sets of documents in chunks, just like lucene does but each chunk is column stride. Maybe possible to apply this idea to compressing stored fields (chunks in column stride fashion)? On Sun, Sep 15, 2013 at 11:17 PM, Otis Gospodnetic < [email protected]> wrote: > Hi, > > I was reading the Parquet announcement from July: > > https://blog.twitter.com/2013/announcing-parquet-10-columnar-storage-for-hadoop > > And a few things caught my attention - Dictionary encoding and > (dynamic) bit packing. This smells like something Adrien likes to eat > for breakfast. > > Over in the Hadoop ecosystem Parquet interest has picked up: > http://search-hadoop.com/?q=parquet > > I thought I'd point it out as I haven't seen anyone bring this up. I > imagine there are ideas to be borrowed there. > > Otis > -- > Solr & ElasticSearch Support -- http://sematext.com/ > Performance Monitoring -- http://sematext.com/spm > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
