[
https://issues.apache.org/jira/browse/SPARK-16221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cheng Lian updated SPARK-16221:
-------------------------------
Assignee: Dongjoon Hyun
> Redirect Parquet JUL logger via SLF4J for WRITE operations
> ----------------------------------------------------------
>
> Key: SPARK-16221
> URL: https://issues.apache.org/jira/browse/SPARK-16221
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Dongjoon Hyun
> Assignee: Dongjoon Hyun
> Priority: Minor
>
> SPARK-8118 implements redirecting Parquet JUL logger via SLF4J, but it is
> currently applied only when READ operations occurs. If users use only WRITE
> operations, there occurs many Parquet logs.
> This issue makes the redirection work on WRITE operations, too.
> **Before**
> {code}
> scala>
> spark.range(10).write.format("parquet").mode("overwrite").save("/tmp/p")
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> Jun 26, 2016 9:04:38 PM INFO: org.apache.parquet.hadoop.codec.CodecConfig:
> Compression: SNAPPY
> ... about 70 lines Parquet Log ...
> scala>
> spark.range(10).write.format("parquet").mode("overwrite").save("/tmp/p")
> ... about 70 lines Parquet Log ...
> {code}
> **After**
> {code}
> scala>
> spark.range(10).write.format("parquet").mode("overwrite").save("/tmp/p")
> [Stage 0:> (0 + 8) /
> 8]
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
>
> scala>
> spark.range(10).write.format("parquet").mode("overwrite").save("/tmp/p")
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
