Okay, but what exactly does Parquet have to offer to a search engine?
I mean, is it simply an alternate form of codec?
Would it merely reduce I/O and mass storage requirements?
Would it impact "search" performance at all?
Would it add a significant search start-up "warming" overhead? Or, does it
offer some magic that would in fact dramatically reduce the time to do the
first query?
Or, is it merely an alternative format for ingestion of an input stream?
Like, say, better than JavaBin? Or, maybe for more efficient internode
transfers of documents for SolrCloud?
-- Jack Krupansky
-----Original Message-----
From: Otis Gospodnetic
Sent: Sunday, September 15, 2013 5:17 PM
To: dev@lucene.apache.org
Subject: Parquet dictionary encoding & bit packing
Hi,
I was reading the Parquet announcement from July:
https://blog.twitter.com/2013/announcing-parquet-10-columnar-storage-for-hadoop
And a few things caught my attention - Dictionary encoding and
(dynamic) bit packing. This smells like something Adrien likes to eat
for breakfast.
Over in the Hadoop ecosystem Parquet interest has picked up:
http://search-hadoop.com/?q=parquet
I thought I'd point it out as I haven't seen anyone bring this up. I
imagine there are ideas to be borrowed there.
Otis
--
Solr & ElasticSearch Support -- http://sematext.com/
Performance Monitoring -- http://sematext.com/spm
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
