[jira] [Created] (SPARK-16221) Redirect Parquet JUL logger via SLF4J for WRITE operations

[Dongjoon Hyun (JIRA)]

Dongjoon Hyun created SPARK-16221:
-------------------------------------

             Summary: Redirect Parquet JUL logger via SLF4J for WRITE operations
                 Key: SPARK-16221
                 URL: https://issues.apache.org/jira/browse/SPARK-16221
             Project: Spark
          Issue Type: Improvement
          Components: SQL
            Reporter: Dongjoon Hyun
            Priority: Minor


SPARK-8118 implements redirecting Parquet JUL logger via SLF4J, but it is 
currently applied only when READ operations occurs. If users use only WRITE 
operations, there occurs many Parquet logs.

This issue makes the redirection work on WRITE operations, too.

**Before**
{code}
scala> spark.range(10).write.format("parquet").mode("overwrite").save("/tmp/p")
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further 
details.
Jun 26, 2016 9:04:38 PM INFO: org.apache.parquet.hadoop.codec.CodecConfig: 
Compression: SNAPPY
... about 70 lines Parquet Log ...
scala> spark.range(10).write.format("parquet").mode("overwrite").save("/tmp/p")
... about 70 lines Parquet Log ...
{code}

**After**
{code}
scala> spark.range(10).write.format("parquet").mode("overwrite").save("/tmp/p")
[Stage 0:>                                                          (0 + 8) / 8]
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further 
details.                                                                        
        
scala> spark.range(10).write.format("parquet").mode("overwrite").save("/tmp/p")
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org