[
https://issues.apache.org/jira/browse/OOZIE-1899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512035#comment-14512035
]
Rohini Palaniswamy commented on OOZIE-1899:
-------------------------------------------
Also to note that even for a 300K files listing, it only takes 50MB according
to HDFS-985 and I doubt it will cause much memory issues with Oozie server
unless there are extreme cases of ~million listings with very long path names.
And in those cases as I mentioned before, you will hit OOM in Oozie before you
get to throw an error that the limit is exceeded which does not solve the
problem. Even in HDFS-985, they do not worry about the issue of memory pressure
on NN, but holding locks and RPC payload sizes.
> Improve the documentation for the FS action's glob feature
> ----------------------------------------------------------
>
> Key: OOZIE-1899
> URL: https://issues.apache.org/jira/browse/OOZIE-1899
> Project: Oozie
> Issue Type: Sub-task
> Components: docs
> Affects Versions: trunk, 4.0.0
> Reporter: Robert Kanter
> Fix For: trunk
>
>
> We should add some more detail to the documentation on the FS action's glob
> feature. It currently just says this:
> {quote}
> In case of move , delete , chmod and chgrp commands, a glob pattern can also
> be specified instead of an absolute path. For move , glob pattern can only be
> specified for source path and not the target.
> {quote}
> The user has no idea how to specify a glob pattern after reading this. An
> example would be nice too.
> Also, {{oozie.action.fs.glob.max}} in oozie-site/default is supposed to let
> you specify how many files can be matched against the glob pattern. This
> isn't mentioned anywhere in the documentation and isn't even in oozie-site or
> oozie-default. It should be added to oozie-default.xml and mentioned in the
> docs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
