Hi, I see lots of parquet logs in container logs(YARN mode), like below:
stdout: Oct 21, 2016 2:27:30 PM INFO: org.apache.parquet.hadoop.ColumnChunkPageWriteStore: written 8,448B for [ss_promo_sk] INT32: 5,996 values, 8,513B raw, 8,409B comp, 1 pages, encodings: [PLAIN_DICTIONARY, BIT_PACKED, RLE], dic { 1,475 entries, 5,900B raw, 1,475B comp} Oct 21, 2016 2:27:30 PM INFO: org.apache.parquet.hadoop.ColumnChunkPageWriteStore: written 1,376B for [ss_ticket_number] INT32: 5,996 values, 1,730B raw, 1,340B comp, 1 pages, encodings: [PLAIN_DICTIONARY, BIT_PACKED, RLE], dic { 524 entries, 2,096B raw, 524B comp} Oct 21, 2016 2:27:30 PM INFO: org.apache.parquet.hadoop.ColumnChunkPageWriteStore: written 5,516B for [ss_quantity] INT32: 5,996 values, 5,567B raw, 5,479B comp, 1 pages, encodings: [PLAIN_DICTIONARY, BIT_PACKED, RLE], dic { 100 entries, 400B raw, 100B comp} Oct 21, 2016 2:27:30 PM INFO: org.apache.parquet.hadoop.ColumnChunkPageWriteStore: written 14,385B for [ss_wholesale_cost] INT32: 5,996 values, 23,931B raw, 14,346B comp, 1 pages, encodings: [BIT_PACKED, PLAIN, RLE] Oct 21, 2016 2:27:30 PM INFO: org.apache.parquet.hadoop.ColumnChunkPageWriteStore: written 15,043B for [ss_list_price] INT32: 5,996 values, 23,871B raw, 15,004B comp, 1 pages, encodings: [BIT_PACKED, PLAIN, RLE] Oct 21, 2016 2:27:30 PM INFO: org.apache.parquet.hadoop.ColumnChunkPageWriteStore: written 14,442B for [ss_sales_price] INT32: 5,996 values, 23,896B raw, 14,403B comp, 1 pages, encodings: [BIT_PACKED, PLAIN, RLE] Oct 21, 2016 2:27:30 PM INFO: org.apache.parquet.hadoop.ColumnChunkPageWriteStore: written 3,538B for [ss_ext_discount_amt] INT32: 5,996 values, 7,317B raw, 3,501B comp, 1 pages, encodings: [PLAIN_DICTIONARY, BIT_PACKED, RLE], dic { 1,139 entries, 4,556B raw, 1,139B comp} Oct 21, 2016 2:27:30 PM INFO: org.apache.parquet.hadoop.ColumnChunkPageWriteStore: written 18,052B for [ss_ext_sales_price] INT32: 5,996 values, 23,907B raw, 18,013B comp, 1 pages, encodings: [BIT_PACKED, PLAIN, RLE] Oc I tried below in log4j.properties, but not work. log4j.logger.org.apache.parquet=ERROR log4j.logger.parquet=ERROR Is there a way to disable them? Thanks a lot! Yucai