[
https://issues.apache.org/jira/browse/OOZIE-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14014763#comment-14014763
]
Purshotam Shah commented on OOZIE-1724:
---------------------------------------
Looks good.
Few minor comment.
1. We should support multiple files, This may not be needed now. But in future
if hcat, hbase etc. support multiple files. Just changing configuration might
work. You can support dir as well.
2. We can move loadHCatConf ( renaming to loadConf) to some util class, so that
later on hbase can also use it. Function can take conf path and return
configuration.
{code}
+ if (path.startsWith("hdfs")) {
+ Path p = new Path(path);
+ HadoopAccessorService has =
services.get(HadoopAccessorService.class);
+ try {
+ FileSystem fs = has.createFileSystem(
+ System.getProperty("user.name"), p.toUri(),
has.createJobConf(p.toUri().getAuthority()));
+ if (fs.exists(p)) {
+ FSDataInputStream is = null;
+ try {
+ is = fs.open(p);
+ hcatConf = new XConfiguration(is);
+ } finally {
+ if (is != null) {
+ is.close();
+ }
+ }
+ } else {
+ LOG.warn("HCat Configuration could not be found at ["
+ path + "]");
+ }
+ } catch (HadoopAccessorException hae) {
+ throw new IOException(hae);
+ }
{code}
Add log statement to say that configuration is loaded from HDFS path.
You are already doing it for files.
We can support configuration as part of sharelib. We can create different JIRA
for this.
Sharelib can contain jar as well as conf.
Ex.
tmp_<timestamp>/hcat/conf/hive-site.xml.
With new function in sharelib like "Configuration getConf(String tag)" will
return conf for tag.
With current approach if we modify/add conf file, we need to restart server.
With sharelib supporting conf. We can just issue sharelib update command to
update conf, no restart needed. It also takes away server start dependency with
hdfs.
> Make it easier to specify the HCat hive-site.xml for the Oozie Server
> ---------------------------------------------------------------------
>
> Key: OOZIE-1724
> URL: https://issues.apache.org/jira/browse/OOZIE-1724
> Project: Oozie
> Issue Type: Improvement
> Affects Versions: 4.0.0
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Attachments: OOZIE-1724.patch, OOZIE-1724.patch, OOZIE-1724.patch
>
>
> Currently, to specify the hive-site.xml to configure Oozie to talk to the
> HCatalog server (see
> http://oozie.apache.org/docs/4.0.0/AG_Install.html#HCatalog_Configuration) it
> requires you to put the hive-site.xml in the classpath, which means you'd put
> it in the libext and run the oozie-setup.sh.
> It would be much easier if we added an oozie-site.xml config property to
> specify this file, similar to how we specify the core/mapred/yarn/etc-site
> files for Hadoop (see
> http://oozie.apache.org/docs/4.0.0/AG_HadoopConfiguration.html).
> Something like:
> {code:xml}
> <property>
> <name>oozie.service.HCatAccessorService.hcat.configuration</name>
> <value>path/to/hive-site.xml</value>
> </property>
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
