----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30098/ -----------------------------------------------------------
Review request for oozie. Bugs: OOZIE-1894 https://issues.apache.org/jira/browse/OOZIE-1894 Repository: oozie-git Description ------- Ooize debuggability is not that easy. We noticed most of user has difficulty finding proper error message. They will create ticket to solution-team/dev to help. Though we have log streaming, but it flooded with other log messages. It's very difficult for naive user to find out failure reason. Here is my solution, 1. Add a new appender to log only error message. Oozie appender will still have all the log message ( including error). 2. Have a new UI tab ( cli/web-service as well) to show error message. When user ask for error message, parse it from error logs. It will be faster. This has multiple advantage 1. Error log streaming will be faster. 2. User will see less and precise log, will help him to solve issue by his own. 3. It will help ops/dev to find out system issues. New appender. +log4j.appender.oozieError=org.apache.log4j.rolling.RollingFileAppender +log4j.appender.oozieError.RollingPolicy=org.apache.oozie.util.OozieRollingPolicy +log4j.appender.oozieError.File=${oozie.log.dir}/oozie-error.log +log4j.appender.oozieError.Append=true +log4j.appender.oozieError.layout=org.apache.log4j.PatternLayout +log4j.appender.oozieError.layout.ConversionPattern=%d{ISO8601} %5p %c{1}:%L - SERVER[${oozie.instance.id}] %m%n +# The FileNamePattern must end with "-%d{yyyy-MM-dd-HH}.gz" or "-%d{yyyy-MM-dd-HH}" and also start with the +# value of log4j.appender.oozie.File +log4j.appender.oozieError.RollingPolicy.FileNamePattern=${log4j.appender.oozieError.File}-%d{yyyy-MM-dd-HH} +# The MaxHistory controls how many log files will be retained (720 hours / 24 hours per day = 30 days); -1 to disable +log4j.appender.oozieError.RollingPolicy.MaxHistory=720 +log4j.appender.oozieError.filter.1 = org.apache.log4j.varia.LevelMatchFilter +log4j.appender.oozieError.filter.1.levelToMatch = WARN +log4j.appender.oozieError.filter.2 = org.apache.log4j.varia.LevelMatchFilter +log4j.appender.oozieError.filter.2.levelToMatch = ERROR +log4j.appender.oozieError.filter.3 = org.apache.log4j.varia.LevelMatchFilter +log4j.appender.oozieError.filter.3.levelToMatch = FATAL +log4j.appender.oozieError.filter.4 = org.apache.log4j.varia.DenyAllFilter -log4j.logger.org.apache.oozie=INFO, oozie +log4j.logger.org.apache.oozie=ALL, oozie, oozieError Diffs ----- client/src/main/java/org/apache/oozie/cli/OozieCLI.java bc8fa50 client/src/main/java/org/apache/oozie/client/OozieClient.java 800e871 client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 85efecf core/src/main/conf/oozie-log4j.properties b91bfa4 core/src/main/java/org/apache/oozie/BaseEngine.java a982ad4 core/src/main/java/org/apache/oozie/BundleEngine.java 27afe73 core/src/main/java/org/apache/oozie/CoordinatorEngine.java 3406c6f core/src/main/java/org/apache/oozie/DagEngine.java 70ddd44 core/src/main/java/org/apache/oozie/service/XLogService.java e97c38a core/src/main/java/org/apache/oozie/service/XLogStreamingService.java 9a42f2f core/src/main/java/org/apache/oozie/service/XLogUtil.java e69de29 core/src/main/java/org/apache/oozie/service/ZKXLogStreamingService.java 94d8cb3 core/src/main/java/org/apache/oozie/servlet/BaseJobServlet.java c94d1e2 core/src/main/java/org/apache/oozie/servlet/V0JobServlet.java b160b46 core/src/main/java/org/apache/oozie/servlet/V1JobServlet.java eed7ca1 core/src/main/java/org/apache/oozie/servlet/V2JobServlet.java da81b49 core/src/main/java/org/apache/oozie/util/JobUtils.java e0a9ce6 core/src/main/java/org/apache/oozie/util/XLogUserFilterParam.java af9b979 core/src/test/java/org/apache/oozie/service/TestXLogStreamingService.java 5ad6dea core/src/test/java/org/apache/oozie/service/TestZKXLogStreamingService.java 2fd08b3 core/src/test/resources/test-no-dash-log4j.properties 36b7b18 docs/src/site/twiki/DG_CommandLineTool.twiki 4ac73f9 docs/src/site/twiki/WebServicesAPI.twiki fc51934 webapp/src/main/webapp/oozie-console.js 8f2098d Diff: https://reviews.apache.org/r/30098/diff/ Testing ------- UTC and manul testing Thanks, Purshotam Shah