[ https://issues.apache.org/jira/browse/SPARK-8121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng Lian updated SPARK-8121: ------------------------------ Summary: When using with Hadoop 1.x, "spark.sql.parquet.output.committer.class" is overriden by "spark.sql.sources.outputCommitterClass" (was: "spark.sql.parquet.output.committer.class" is overriden by "spark.sql.sources.outputCommitterClass") > When using with Hadoop 1.x, "spark.sql.parquet.output.committer.class" is > overriden by "spark.sql.sources.outputCommitterClass" > ------------------------------------------------------------------------------------------------------------------------------- > > Key: SPARK-8121 > URL: https://issues.apache.org/jira/browse/SPARK-8121 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.4.0 > Reporter: Cheng Lian > Assignee: Cheng Lian > > When {{spark.sql.sources.outputCommitterClass}} is configured, > {{spark.sql.parquet.output.committer.class}} will be overriden. > For example, if {{spark.sql.parquet.output.committer.class}} is set to > {{FileOutputCommitter}}, while {{spark.sql.sources.outputCommitterClass}} is > set to {{DirectParquetOutputCommitter}}, neither {{_metadata}} nor > {{_common_metadata}} will be written because {{FileOutputCommitter}} > overrides {{DirectParquetOutputCommitter}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org