[jira] Commented: (PIG-1215) Make Hadoop jobId more prominent in the client log
[ https://issues.apache.org/jira/browse/PIG-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835558#action_12835558 ] Olga Natkovich commented on PIG-1215: - Changes look good and I also ran a few tests. The only thing that needs to be changes is that when loglevel is specified we still want to print to screen in addition to printing to the file. > Make Hadoop jobId more prominent in the client log > -- > > Key: PIG-1215 > URL: https://issues.apache.org/jira/browse/PIG-1215 > Project: Pig > Issue Type: Improvement >Reporter: Olga Natkovich >Assignee: Ashutosh Chauhan > Fix For: 0.7.0 > > Attachments: pig-1215.patch, pig-1215.patch, pig-1215_1.patch, > pig-1215_3.patch > > > This is a request from applications that want to be able to programmatically > parse client logs to find hadoop Ids. > The woould like to see each job id on a separate line in the following format: > hadoopJobId: job_123456789 > They would also like to see the jobs in the order they are executed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1215) Make Hadoop jobId more prominent in the client log
[ https://issues.apache.org/jira/browse/PIG-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834458#action_12834458 ] Olga Natkovich commented on PIG-1215: - The log part of the change is not quite right. I think we need to have a property that controls the level of output to the log file. We don't want new file - we want to re-use the existing log file and just put more infomration there. I am still looking at the second half of the patch. > Make Hadoop jobId more prominent in the client log > -- > > Key: PIG-1215 > URL: https://issues.apache.org/jira/browse/PIG-1215 > Project: Pig > Issue Type: Improvement >Reporter: Olga Natkovich >Assignee: Ashutosh Chauhan > Fix For: 0.7.0 > > Attachments: pig-1215.patch, pig-1215.patch, pig-1215_1.patch > > > This is a request from applications that want to be able to programmatically > parse client logs to find hadoop Ids. > The woould like to see each job id on a separate line in the following format: > hadoopJobId: job_123456789 > They would also like to see the jobs in the order they are executed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1215) Make Hadoop jobId more prominent in the client log
[ https://issues.apache.org/jira/browse/PIG-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1285#action_1285 ] Hadoop QA commented on PIG-1215: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12435733/pig-1215_1.patch against trunk revision 909584. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 213 javac compiler warnings (more than the trunk's current 212 warnings). +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/204/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/204/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/204/console This message is automatically generated. > Make Hadoop jobId more prominent in the client log > -- > > Key: PIG-1215 > URL: https://issues.apache.org/jira/browse/PIG-1215 > Project: Pig > Issue Type: Improvement >Reporter: Olga Natkovich >Assignee: Ashutosh Chauhan > Fix For: 0.7.0 > > Attachments: pig-1215.patch, pig-1215.patch, pig-1215_1.patch > > > This is a request from applications that want to be able to programmatically > parse client logs to find hadoop Ids. > The woould like to see each job id on a separate line in the following format: > hadoopJobId: job_123456789 > They would also like to see the jobs in the order they are executed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1215) Make Hadoop jobId more prominent in the client log
[
https://issues.apache.org/jira/browse/PIG-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832775#action_12832775
]
Ashutosh Chauhan commented on PIG-1215:
---
Pig already has extensive support for logging via log4j. Applications wanting
to mine pig logs can configure the pig logging using -4 option of Pig. for
example if you provide "pig -4 logger.conf myscript.pig" log4j properties
specified in file logger.conf will be picked up. So, application can configure
log4j as they want. One such config which I tried is following:
{noformat}
$ cat logger.conf
log4j.rootLogger=INFO, R
log4j.logger.org.apache.pig=INFO
log4j.appender.R=org.apache.log4j.RollingFileAppender
# file name in which log messages will be printed.
log4j.appender.R.File=piglogs.log
log4j.appender.R.MaxFileSize=100KB
# Keep one backup file
log4j.appender.R.MaxBackupIndex=1
log4j.appender.R.layout=org.apache.log4j.PatternLayout
# Pattern to output the caller's file name and line number.
log4j.appender.R.layout.ConversionPattern=%p %t %c - %m%n
{noformat}
Above log4j configuration prints all the info level messages in piglogs.log
file. All the other properties of log4j (like log rotation, format of log
messages etc) is configurable through this file. More info at:
http://logging.apache.org/log4j/1.2/manual.html
Thus no new property or any other change in Pig is required to redirect log
messages to file.
Unit test failure is one of those temporary "port out of range" failures
unrelated to patch. Javac warning is deprecation warning due to use of a
deprecated Hadoop api. These deprecation warnings will be addressed when we
move to new Hadoop api.
This patch is ready for review.
> Make Hadoop jobId more prominent in the client log
> --
>
> Key: PIG-1215
> URL: https://issues.apache.org/jira/browse/PIG-1215
> Project: Pig
> Issue Type: Improvement
>Reporter: Olga Natkovich
>Assignee: Ashutosh Chauhan
> Fix For: 0.7.0
>
> Attachments: pig-1215.patch, pig-1215.patch
>
>
> This is a request from applications that want to be able to programmatically
> parse client logs to find hadoop Ids.
> The woould like to see each job id on a separate line in the following format:
> hadoopJobId: job_123456789
> They would also like to see the jobs in the order they are executed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1215) Make Hadoop jobId more prominent in the client log
[ https://issues.apache.org/jira/browse/PIG-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832743#action_12832743 ] Hadoop QA commented on PIG-1215: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12435554/pig-1215.patch against trunk revision 908324. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 213 javac compiler warnings (more than the trunk's current 212 warnings). +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/201/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/201/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/201/console This message is automatically generated. > Make Hadoop jobId more prominent in the client log > -- > > Key: PIG-1215 > URL: https://issues.apache.org/jira/browse/PIG-1215 > Project: Pig > Issue Type: Improvement >Reporter: Olga Natkovich >Assignee: Ashutosh Chauhan > Fix For: 0.7.0 > > Attachments: pig-1215.patch, pig-1215.patch > > > This is a request from applications that want to be able to programmatically > parse client logs to find hadoop Ids. > The woould like to see each job id on a separate line in the following format: > hadoopJobId: job_123456789 > They would also like to see the jobs in the order they are executed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1215) Make Hadoop jobId more prominent in the client log
[ https://issues.apache.org/jira/browse/PIG-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831817#action_12831817 ] Olga Natkovich commented on PIG-1215: - can we also make the value NOT_AVAILABLE rather than NOT AVAILABLE to make it easier for tools to parse > Make Hadoop jobId more prominent in the client log > -- > > Key: PIG-1215 > URL: https://issues.apache.org/jira/browse/PIG-1215 > Project: Pig > Issue Type: Improvement >Reporter: Olga Natkovich >Assignee: Ashutosh Chauhan > Fix For: 0.7.0 > > Attachments: pig-1215.patch > > > This is a request from applications that want to be able to programmatically > parse client logs to find hadoop Ids. > The woould like to see each job id on a separate line in the following format: > hadoopJobId: job_123456789 > They would also like to see the jobs in the order they are executed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1215) Make Hadoop jobId more prominent in the client log
[ https://issues.apache.org/jira/browse/PIG-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831812#action_12831812 ] Olga Natkovich commented on PIG-1215: - I would like to request an additional change to make sure that we can write HadoopId information to the client side log file not just stdout. This would happen only if special property is used. So the additional ask is to implement handling of this new property and when it is present to make sure that all messages at the level of INFO are written to the log file. This can be accomplished by changing the log listener for the log file so it picks up INFO level log events. We don't want to do this by default because it would drastically increase the number of log files created by Pig since now we only create the file when there is a real problem executing it. > Make Hadoop jobId more prominent in the client log > -- > > Key: PIG-1215 > URL: https://issues.apache.org/jira/browse/PIG-1215 > Project: Pig > Issue Type: Improvement >Reporter: Olga Natkovich >Assignee: Ashutosh Chauhan > Fix For: 0.7.0 > > Attachments: pig-1215.patch > > > This is a request from applications that want to be able to programmatically > parse client logs to find hadoop Ids. > The woould like to see each job id on a separate line in the following format: > hadoopJobId: job_123456789 > They would also like to see the jobs in the order they are executed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1215) Make Hadoop jobId more prominent in the client log
[ https://issues.apache.org/jira/browse/PIG-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831316#action_12831316 ] Hadoop QA commented on PIG-1215: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12435211/pig-1215.patch against trunk revision 907760. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/194/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/194/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/194/console This message is automatically generated. > Make Hadoop jobId more prominent in the client log > -- > > Key: PIG-1215 > URL: https://issues.apache.org/jira/browse/PIG-1215 > Project: Pig > Issue Type: Improvement >Reporter: Olga Natkovich >Assignee: Ashutosh Chauhan > Fix For: 0.7.0 > > Attachments: pig-1215.patch > > > This is a request from applications that want to be able to programmatically > parse client logs to find hadoop Ids. > The woould like to see each job id on a separate line in the following format: > hadoopJobId: job_123456789 > They would also like to see the jobs in the order they are executed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1215) Make Hadoop jobId more prominent in the client log
[ https://issues.apache.org/jira/browse/PIG-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831309#action_12831309 ] Hadoop QA commented on PIG-1215: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12435211/pig-1215.patch against trunk revision 907760. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/205/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/205/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/205/console This message is automatically generated. > Make Hadoop jobId more prominent in the client log > -- > > Key: PIG-1215 > URL: https://issues.apache.org/jira/browse/PIG-1215 > Project: Pig > Issue Type: Improvement >Reporter: Olga Natkovich >Assignee: Ashutosh Chauhan > Fix For: 0.7.0 > > Attachments: pig-1215.patch > > > This is a request from applications that want to be able to programmatically > parse client logs to find hadoop Ids. > The woould like to see each job id on a separate line in the following format: > hadoopJobId: job_123456789 > They would also like to see the jobs in the order they are executed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
