[
https://issues.apache.org/jira/browse/OOZIE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995844#comment-13995844
]
Robert Kanter commented on OOZIE-1740:
--------------------------------------
Sorry for not realizing this before, but after looking at your latest patch, I
just realized that your function has no way of knowing which Hadoop conf to
load from HadoopAccessorService, so you're just picking the first one. Perhaps
instead of having null or "" load the default, the function should take a
"{{host:port}}" pair, in which case it simply calls
{{Services.get().get(HadoopAccessorService.class).createJobConf(hostPort)}}
which will load and return the correct configurations for you. If that
{{hostPort}} doesn't exist, it returns whatever {{*=/path}} is set to.
With that in mind, I'm not sure it makes sense for the function to accept an
arbitrary path anymore. The idea is to allow you to modify the configs in your
cluster, which should be available to the HadoopAccessorService, and would now
be more easily passed to the EL function with {{host:port}} instead of a local
path. Remember that you can also load hadoop configuration formatted files
(from HDFS) with the {{<job-xml>}} element.
To clarify, the function would look like this:
{noformat}
${hadoop:conf(HOST:PORT, 'mapreduce.map.java.opts')}
{noformat}
where {{HOST:PORT}} are either the JT/RM or NN. And because users typically
parameterize those host:ports anyway, they can do something like this:
{noformat}
${hadoop:conf(jobTracker, 'mapreduce.map.java.opts')}
{noformat}
which I think is much easier than dealing with a local path. What do you
think? Again, sorry for not seeing this sooner.
{quote}Btw, I also agree the 3rd comment: '3. It might also be helpful to make
this EL function available in Coordinators too'. However I suggest to implement
this feature in another jira for coordinator only and this jira will focus on
workflow as its title. What's your opinion?{quote}
This is actually something trivial to do; you just simply add it to the
appropriate property for coordinator EL functions in oozie-default.xml, like
you already did for workflows.
> Add a new function hadoop:conf() that can be invoked from the workflow.xml
> and will return a hadoop configuration option
> ------------------------------------------------------------------------------------------------------------------------
>
> Key: OOZIE-1740
> URL: https://issues.apache.org/jira/browse/OOZIE-1740
> Project: Oozie
> Issue Type: Improvement
> Components: workflow
> Affects Versions: trunk, 3.3.2
> Reporter: sam liu
> Assignee: sam liu
> Attachments: OOZIE-1740.patch, OOZIE-1740.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
