[ https://issues.apache.org/jira/browse/SPARK-12307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15057067#comment-15057067 ]
Hyukjin Kwon edited comment on SPARK-12307 at 12/15/15 12:40 AM: ----------------------------------------------------------------- Just my personal thought. I felt in the same way with this issue before but I ended up with not creating this issue because options would be too many. I mean, there are many options omitted, for example, the options for ORC, Parquet writer version, Parquet metadata and etc. I agree that it is a good idea to add some options like this but we might need to choose some of important options to expose by a certain condition to decide to add them. was (Author: hyukjin.kwon): Just my personal though. I felt in the same way with this issue before but I ended up with not creating this issue because options would be too many. I mean, there are many options omitted, for example, the options for ORC, Parquet writer version, Parquet metadata and etc. I agree that it is a good idea to add some options like this but we might need to choose some of important options to expose by a certain condition to decide to add them. > ParquetFormat options should be exposed through the DataFrameReader/Writer > options API > -------------------------------------------------------------------------------------- > > Key: SPARK-12307 > URL: https://issues.apache.org/jira/browse/SPARK-12307 > Project: Spark > Issue Type: Improvement > Components: SQL > Reporter: holdenk > Priority: Trivial > > Currently many options for loading/saving Parquet need to be set globally on > the SparkContext. It would be useful to also provide support for setting > these options through the DataFrameReader/DataFrameWriter. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org