subject:"Can we disable parquet logs in Spark\?"

RE: Can we disable parquet logs in Spark?

2016-10-21 Thread Yu, Yucai

21, 2016 2:50 PM To: user@spark.apache.org Subject: Can we disable parquet logs in Spark? Hi, I see lots of parquet logs in container logs(YARN mode), like below: stdout: Oct 21, 2016 2:27:30 PM INFO: org.apache.parquet.hadoop.ColumnChunkPageWriteStore: written 8,448B for [ss_promo_sk] INT32:

Can we disable parquet logs in Spark?

2016-10-20 Thread Yu, Yucai

Hi, I see lots of parquet logs in container logs(YARN mode), like below: stdout: Oct 21, 2016 2:27:30 PM INFO: org.apache.parquet.hadoop.ColumnChunkPageWriteStore: written 8,448B for [ss_promo_sk] INT32: 5,996 values, 8,513B raw, 8,409B comp, 1 pages, encodings: [PLAIN_DICTIONARY, BIT_PACKED,