[
https://issues.apache.org/jira/browse/OOZIE-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065780#comment-14065780
]
Purshotam Shah commented on OOZIE-1716:
---------------------------------------
Yes, recent option of OOZIE-1737 has this fix.
Below example will fetch recent 1 hour logs for job.
$ ./oozie job -log 0000003-140319184715726-oozie-puru-C -logfilter recent=1h
-oozie http://localhost:11000/oozie/
{code}
2014-03-20 09:59:50,329 INFO CoordActionInputCheckXCommand:539 - SERVER[ ]
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-140319184715726-oozie-puru-C]
ACTION[0000003-140319184715726-oozie-puru-C@1]
[0000003-140319184715726-oozie-puru-C@1]::CoordActionInputCheck:: Missing
deps:hdfs://localhost:9000/user/purushah/examples/input-data/rawLogs/
2014-03-20 09:59:50,330 INFO CoordActionInputCheckXCommand:539 - SERVER[ ]
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-140319184715726-oozie-puru-C]
ACTION[0000003-140319184715726-oozie-puru-C@1]
[0000003-140319184715726-oozie-puru-C@1]::ActionInputCheck:: In
checkListOfPaths:
hdfs://localhost:9000/user/purushah/examples/input-data/rawLogs/ is Missing.
2014-03-20 10:02:19,087 INFO CoordActionInputCheckXCommand:539 - SERVER[ ]
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-140319184715726-oozie-puru-C]
ACTION[0000003-140319184715726-oozie-puru-C@2]
[0000003-140319184715726-oozie-puru-C@2]::CoordActionInputCheck:: Missing
deps:hdfs://localhost:9000/user/purushah/examples/input-data/rawLogs/
2014-03-20 10:02:19,088 INFO CoordActionInputCheckXCommand:539 - SERVER[ ]
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-140319184715726-oozie-puru-C]
ACTION[0000003-140319184715726-oozie-puru-C@2]
[0000003-140319184715726-oozie-puru-C@2]::ActionInputCheck:: In
checkListOfPaths:
hdfs://localhost:9000/user/purushah/examples/input-data/rawLogs/ is Missing.
$
{code}
> Add a way to stream the logs for only the last x hours
> ------------------------------------------------------
>
> Key: OOZIE-1716
> URL: https://issues.apache.org/jira/browse/OOZIE-1716
> Project: Oozie
> Issue Type: New Feature
> Affects Versions: trunk
> Reporter: Robert Kanter
> Assignee: Purshotam Shah
>
> When a user has a busy cluster and long running coordinator jobs, streaming
> the logs can be really slow. In this situation, the user is typically only
> interested in the latest logs and doesn't care about logs from a while ago.
> So, we can speed this up by adding a parameter or something to the log
> streaming API to only fetch the logs from the last x hours. The log files
> roll over every hour, so this should be pretty straightforward. We can also
> prepend a message to the logs streamed back mentioning that only the last x
> hours of logs are included. The existing way of getting all of the logs
> should remain unaffected and still work.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
