[
https://issues.apache.org/jira/browse/OOZIE-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13991227#comment-13991227
]
Purshotam Shah commented on OOZIE-1825:
---------------------------------------
No. We will store only lib jars path to protoconf, so that we don't have call
DFS for every action.
For each action execution, we will load protoconf and merge with hadoop
properties.
If protoconf already have hadoop properties, merging will produce same result.
> Optimize wf_jobs protoconf storage
> ----------------------------------
>
> Key: OOZIE-1825
> URL: https://issues.apache.org/jira/browse/OOZIE-1825
> Project: Oozie
> Issue Type: Bug
> Reporter: Purshotam Shah
>
> Currently protoconf stores all Hadoop configuration properties + libpath + ...
> Hadoop configuration properties are lot and take huge space, I don't think we
> need to store Hadoop configuration properties in protoconf.
> We can load it every time wf action runs.
> Some statistics on table storage.
> Table size(gb)
> WFJOBS_PROTO_ACTION_CONF 55.54
> WFJOBS_WF_INSTANCE 11.89
> WFJOBS_CONF 2.43
> If we can remove Hadoop configuration form proto, size will be < 1gb.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
