[jira] Created: (NUTCH-163) LogFormatter design
LogFormatter design --- Key: NUTCH-163 URL: http://issues.apache.org/jira/browse/NUTCH-163 Project: Nutch Type: Improvement Environment: All platforms Reporter: Daniel Feinstein In Nutch project LogFormatter has duplicated functionality: 1) Logger records format and 2) Severe error handler The first usage is standard and usually could be overwritten by a user of the package by modifying logging.properties file. The second usage is much more problematic because it affects the behavior of the whole application (not only Nutch package). To support the error handling LogFormatter enforce usage of the formatter class by all classes of the whole application which uses Nutch package. This is done by overwriting all the system handlers (class java.util.logging.Handler). This operation prevents the application to use its own log formatter. Also this cause LogFormatter.hasLoggedSevere() to be sensitive to all severe records in the big system but not only to relevant. More than that this flag, LogFormatter.loggedSevere is never cleaned what means if an application had one, even unrelated severe record, tools like Fetcher will never run until the application will be restarted. I would like to suggest the following solutions: 1) To separate the functionality of log formatting and error handling or 2) Change LogFormatter class to be affected only by nutch package functions For my opinion the first solution is much better especially if error handling will be encapsulated for each task. I have found the following usages of LogFormatter.hasLoggedSevere(): - Fetcher - URLFilterChecker - ParseSegment Unfortunately I'm not familiar enough with the usages above to implement this solution that why I suggest the second one. I have rewritten my own implementation of LogFormatter class which is used for more than a year in www.rawsugar.com application. I could provide the file but do not know how to attach it to the issue. I hope this change will be accepted by the community. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (NUTCH-163) LogFormatter design
[ http://issues.apache.org/jira/browse/NUTCH-163?page=all ] Daniel Feinstein updated NUTCH-163: --- Attachment: LogFormatter.java Here is the solution we have in RawSugar.com project > LogFormatter design > --- > > Key: NUTCH-163 > URL: http://issues.apache.org/jira/browse/NUTCH-163 > Project: Nutch > Type: Improvement > Environment: All platforms > Reporter: Daniel Feinstein > Attachments: LogFormatter.java > > In Nutch project LogFormatter has duplicated functionality: > 1) Logger records format and > 2) Severe error handler > > The first usage is standard and usually could be overwritten by a user of the > package by modifying logging.properties file. > The second usage is much more problematic because it affects the behavior of > the whole application (not only Nutch package). To support the error handling > LogFormatter enforce usage of the formatter class by all classes of the whole > application which uses Nutch package. This is done by overwriting all the > system handlers (class java.util.logging.Handler). This operation prevents > the application to use its own log formatter. Also this cause > LogFormatter.hasLoggedSevere() to be sensitive to all severe records in the > big system but not only to relevant. More than that this flag, > LogFormatter.loggedSevere is never cleaned what means if an application had > one, even unrelated severe record, tools like Fetcher will never run until > the application will be restarted. > > I would like to suggest the following solutions: > 1) To separate the functionality of log formatting and error handling or > 2) Change LogFormatter class to be affected only by nutch package functions > > For my opinion the first solution is much better especially if error handling > will be encapsulated for each task. I have found the following usages of > LogFormatter.hasLoggedSevere(): > - Fetcher > - URLFilterChecker > - ParseSegment > Unfortunately I'm not familiar enough with the usages above to implement this > solution that why I suggest the second one. > I have rewritten my own implementation of LogFormatter class which is used > for more than a year in www.rawsugar.com application. > I could provide the file but do not know how to attach it to the issue. I > hope this change will be accepted by the community. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-163) LogFormatter design
[ http://issues.apache.org/jira/browse/NUTCH-163?page=comments#action_12361823 ] Daniel Feinstein commented on NUTCH-163: You just need to replace the file. I have checked it on Nutch 0.7 release but I belive that there are no changes in the latest version. What the code basicaly does: it wraps an existing logging handler formatter with Nutch log formatter instead of replacing it. Another change is that SEVERE flag is set only if the error has been received from Nutch package call. I also have added a clear flag method (clearLoggedSevere) for future use. > LogFormatter design > --- > > Key: NUTCH-163 > URL: http://issues.apache.org/jira/browse/NUTCH-163 > Project: Nutch > Type: Improvement > Environment: All platforms > Reporter: Daniel Feinstein > Attachments: LogFormatter.java > > In Nutch project LogFormatter has duplicated functionality: > 1) Logger records format and > 2) Severe error handler > > The first usage is standard and usually could be overwritten by a user of the > package by modifying logging.properties file. > The second usage is much more problematic because it affects the behavior of > the whole application (not only Nutch package). To support the error handling > LogFormatter enforce usage of the formatter class by all classes of the whole > application which uses Nutch package. This is done by overwriting all the > system handlers (class java.util.logging.Handler). This operation prevents > the application to use its own log formatter. Also this cause > LogFormatter.hasLoggedSevere() to be sensitive to all severe records in the > big system but not only to relevant. More than that this flag, > LogFormatter.loggedSevere is never cleaned what means if an application had > one, even unrelated severe record, tools like Fetcher will never run until > the application will be restarted. > > I would like to suggest the following solutions: > 1) To separate the functionality of log formatting and error handling or > 2) Change LogFormatter class to be affected only by nutch package functions > > For my opinion the first solution is much better especially if error handling > will be encapsulated for each task. I have found the following usages of > LogFormatter.hasLoggedSevere(): > - Fetcher > - URLFilterChecker > - ParseSegment > Unfortunately I'm not familiar enough with the usages above to implement this > solution that why I suggest the second one. > I have rewritten my own implementation of LogFormatter class which is used > for more than a year in www.rawsugar.com application. > I could provide the file but do not know how to attach it to the issue. I > hope this change will be accepted by the community. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
