[ https://issues.apache.org/jira/browse/HBASE-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276341#comment-13276341 ]
jirapos...@reviews.apache.org commented on HBASE-6004: ------------------------------------------------------ bq. On 2012-05-15 23:11:04, Andrew Purtell wrote: bq. > src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java, line 164 bq. > <https://reviews.apache.org/r/5131/diff/1/?file=109074#file109074line164> bq. > bq. > Maybe the client should just log this? Client sometimes just gets a RetriesExhaustedException. bq. On 2012-05-15 23:11:04, Andrew Purtell wrote: bq. > src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java, line 51 bq. > <https://reviews.apache.org/r/5131/diff/1/?file=109076#file109076line51> bq. > bq. > We have other configuration variables that start with 'hbase.mapreduce.*' Will change to mapreduce. bq. On 2012-05-15 23:11:04, Andrew Purtell wrote: bq. > src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java, line 54 bq. > <https://reviews.apache.org/r/5131/diff/1/?file=109074#file109074line54> bq. > bq. > We control logging via log4j.properties files. Here also new configuration in another file to switch some additional logging on and off. Would it make more sense to create a logger class e.g. ScannerCallable.ActivityLog that logs at TRACE level and update log4j.properties with log4j.logger.org.apache.hadoop.hbase.client.ScannerCallable.ActivityLog = TRACE (default is INFO, i.e. disabled). Just a thought. We did something like this for security audit logging. It is a good idea. Because of other parameters introduced, it's better to put them in the same place for now. - Jimmy ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5131/#review7918 ----------------------------------------------------------- On 2012-05-15 22:24:40, Jimmy Xiang wrote: bq. bq. ----------------------------------------------------------- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/5131/ bq. ----------------------------------------------------------- bq. bq. (Updated 2012-05-15 22:24:40) bq. bq. bq. Review request for hbase. bq. bq. bq. Summary bq. ------- bq. bq. Added some logging for MR debugging in case scanner times out. The logging is disabled by default. bq. It will be helpful to know how much time spent in the scanner and, how much in the mapper task. bq. In case scanner issue, it is helpful to know the region server id, last successful rows and so on. bq. bq. bq. This addresses bug HBASE-6004. bq. https://issues.apache.org/jira/browse/HBASE-6004 bq. bq. bq. Diffs bq. ----- bq. bq. src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 46b1c56 bq. src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java 42569fb bq. src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java 1c8a393 bq. bq. Diff: https://reviews.apache.org/r/5131/diff bq. bq. bq. Testing bq. ------- bq. bq. bq. Thanks, bq. bq. Jimmy bq. bq. > Adding more logging to help debugging MR job > -------------------------------------------- > > Key: HBASE-6004 > URL: https://issues.apache.org/jira/browse/HBASE-6004 > Project: HBase > Issue Type: Improvement > Affects Versions: 0.94.0, 0.96.0 > Reporter: Jimmy Xiang > Assignee: Jimmy Xiang > Priority: Minor > Fix For: 0.94.0, 0.96.0 > > Attachments: hbase-6004.patch > > > MR job sometime fails because scanner expired. In this case, it will be > helpful to know the last successful row, the ip of the region sever, and so > on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira