Added PR https://github.com/apache/spark/pull/4139 <https://github.com/apache/spark/pull/4139> - I think tests have been re-arranged so merge necessary
Mick > On 19 Jan 2015, at 18:31, Reynold Xin <r...@databricks.com> wrote: > > Definitely go for a pull request! > > > On Mon, Jan 19, 2015 at 10:10 AM, Mick Davies <michael.belldav...@gmail.com > <mailto:michael.belldav...@gmail.com>> wrote: > > Looking at Parquet code - it looks like hooks are already in place to > support this. > > In particular PrimitiveConverter has methods hasDictionarySupport and > addValueFromDictionary for this purpose. These are not used by > CatalystPrimitiveConverter. > > I think that it would be pretty straightforward to add this. Has anyone > considered this? Shall I get a pull request together for it. > > Mick > > > > -- > View this message in context: > http://apache-spark-developers-list.1001551.n3.nabble.com/Optimize-encoding-decoding-strings-when-using-Parquet-tp10141p10195.html > > <http://apache-spark-developers-list.1001551.n3.nabble.com/Optimize-encoding-decoding-strings-when-using-Parquet-tp10141p10195.html> > Sent from the Apache Spark Developers List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > <mailto:dev-unsubscr...@spark.apache.org> > For additional commands, e-mail: dev-h...@spark.apache.org > <mailto:dev-h...@spark.apache.org> > >
