[ https://issues.apache.org/jira/browse/HIVE-21455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796888#comment-16796888 ]
Hive QA commented on HIVE-21455: -------------------------------- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12963013/HIVE-21455.2.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 15832 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/16584/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16584/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16584/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12963013 - PreCommit-HIVE-Build > Too verbose logging in AvroGenericRecordReader > ---------------------------------------------- > > Key: HIVE-21455 > URL: https://issues.apache.org/jira/browse/HIVE-21455 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 > Affects Versions: 1.2.0, 1.1.0, 2.0.0, 2.1.0, 3.0.0, 3.1.0 > Reporter: Miklos Szurap > Assignee: Miklos Szurap > Priority: Minor > Attachments: HIVE-21455.2.patch, HIVE-21455.patch > > > {{AvroGenericRecordReader}} logs the Avro schema for each datafile. It is too > verbose, likely we don't need to log that on INFO level. > For example a table: > {noformat} > create table avro_tbl (c1 string, c2 int, c3 float) stored as avro; > {noformat} > and querying it with a select star - with 3 datafiles HiveServer2 logs the > following: > {noformat} > 2019-03-15 09:18:35,999 INFO org.apache.hadoop.mapred.FileInputFormat: > [HiveServer2-Handler-Pool: Thread-64]: Total input paths to process : 3 > 2019-03-15 09:18:35,999 INFO > org.apache.hadoop.hive.ql.io.avro.AvroGenericRecordReader: > [HiveServer2-Handler-Pool: Thread-64]: Found the avro schema in the job: > {"type":"record","name":"avro_tbl","namespace":"test","fields":[{"name":"c1","type":["null","string"],"default":null},{"name":"c2","type":["null","int"],"default":null},{"name":"c3","type":["null","float"],"default":null}]} > 2019-03-15 09:18:36,004 INFO > org.apache.hadoop.hive.ql.io.avro.AvroGenericRecordReader: > [HiveServer2-Handler-Pool: Thread-64]: Found the avro schema in the job: > {"type":"record","name":"avro_tbl","namespace":"test","fields":[{"name":"c1","type":["null","string"],"default":null},{"name":"c2","type":["null","int"],"default":null},{"name":"c3","type":["null","float"],"default":null}]} > 2019-03-15 09:18:36,010 INFO > org.apache.hadoop.hive.ql.io.avro.AvroGenericRecordReader: > [HiveServer2-Handler-Pool: Thread-64]: Found the avro schema in the job: > {"type":"record","name":"avro_tbl","namespace":"test","fields":[{"name":"c1","type":["null","string"],"default":null},{"name":"c2","type":["null","int"],"default":null},{"name":"c3","type":["null","float"],"default":null}]} > {noformat} > This has a huge performance and storage penalty on a table with big schema > and thousands of datafiles. -- This message was sent by Atlassian JIRA (v7.6.3#76005)