[
https://issues.apache.org/jira/browse/FLUME-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257324#comment-13257324
]
[email protected] commented on FLUME-1052:
------------------------------------------------------
bq. On 2012-04-19 07:02:57, Juhani Connolly wrote:
bq. > pom.xml, line 41
bq. >
<https://reviews.apache.org/r/4502/diff/4-6/?file=101779#file101779line41>
bq. >
bq. > These changes don't look like they belong here... Maybe your patch
was based against the wrong version?
Hi Juhani,
Thanks for the feedback. This appears in the diff because I rebased the patch
on trunk (sorry, I may have forgotten to mention this). Please see :
https://github.com/apache/flume/blob/trunk/pom.xml(lines 41 to 46). Thats the
reason you see it as added when you diff between patch versions 4 and 6. This
patch should be applied to trunk.
- Hari
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4502/#review7018
-----------------------------------------------------------
On 2012-04-19 01:33:31, Hari Shreedharan wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4502/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-19 01:33:31)
bq.
bq.
bq. Review request for Flume.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Main config component.
bq.
bq.
bq. This addresses bug FLUME-1052.
bq. https://issues.apache.org/jira/browse/FLUME-1052
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. flume-ng-configuration/pom.xml PRE-CREATION
bq. flume-ng-configuration/src/main/java/org/apache/flume/Context.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/ComponentConfiguration.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/ComponentConfigurationFactory.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/ConfigurationException.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/FlumeConfiguration.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/FlumeConfigurationError.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/FlumeConfigurationErrorType.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/channel/ChannelConfiguration.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/channel/ChannelSelectorConfiguration.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/channel/ChannelSelectorType.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/channel/ChannelType.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkConfiguration.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkGroupConfiguration.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkProcessorConfiguration.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkProcessorType.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkType.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceConfiguration.java
PRE-CREATION
bq.
flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceType.java
PRE-CREATION
bq. flume-ng-core/pom.xml b9f1e12
bq. flume-ng-core/src/main/java/org/apache/flume/ChannelSelector.java
fba2dcb
bq. flume-ng-core/src/main/java/org/apache/flume/Context.java 5294e31
bq. flume-ng-core/src/main/java/org/apache/flume/SinkProcessor.java d863ed0
bq. flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java
be1891b
bq.
flume-ng-core/src/main/java/org/apache/flume/channel/ChannelSelectorFactory.java
800f471
bq.
flume-ng-core/src/main/java/org/apache/flume/channel/ChannelSelectorType.java
511fc65
bq. flume-ng-core/src/main/java/org/apache/flume/channel/ChannelType.java
d8419e8
bq.
flume-ng-core/src/main/java/org/apache/flume/channel/DefaultChannelFactory.java
963a6a3
bq.
flume-ng-core/src/main/java/org/apache/flume/conf/ConfigurableComponent.java
PRE-CREATION
bq. flume-ng-core/src/main/java/org/apache/flume/conf/Configurables.java
84492e5
bq.
flume-ng-core/src/main/java/org/apache/flume/sink/DefaultSinkFactory.java
b89dfa0
bq.
flume-ng-core/src/main/java/org/apache/flume/sink/DefaultSinkProcessor.java
257bab3
bq. flume-ng-core/src/main/java/org/apache/flume/sink/SinkGroup.java 0dffd69
bq.
flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java
6160a17
bq. flume-ng-core/src/main/java/org/apache/flume/sink/SinkType.java 6b08c09
bq.
flume-ng-core/src/main/java/org/apache/flume/source/DefaultSourceFactory.java
a610e6f
bq. flume-ng-core/src/main/java/org/apache/flume/source/SourceType.java
cd8991e
bq. flume-ng-dist/pom.xml 642e681
bq. flume-ng-dist/src/main/assembly/dist.xml 917ac02
bq. flume-ng-dist/src/main/assembly/src.xml 3b24b39
bq.
flume-ng-node/src/main/java/org/apache/flume/conf/properties/FlumeConfiguration.java
d66f6d1
bq.
flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java
1f0e8c6
bq. pom.xml ed8092d
bq.
bq. Diff: https://reviews.apache.org/r/4502/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Functional testing done.
bq.
bq.
bq. Thanks,
bq.
bq. Hari
bq.
bq.
> Core configuration component
> ----------------------------
>
> Key: FLUME-1052
> URL: https://issues.apache.org/jira/browse/FLUME-1052
> Project: Flume
> Issue Type: Sub-task
> Affects Versions: v1.0.0
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Fix For: v1.2.0
>
>
> Currently the configuration provider implementation encompasses all the
> syntactic and structural validation rules for loading the configuration.
> Externalizing this functionality to a library will allow external tools to
> easily operate on flume configuration files and be able to help parse and
> validate these files.
> Currently the configuration of each component sits deep inside the component
> themselves. There is no way to verify if a configuration is valid before run
> time. In most systems like Flume, it is likely that these confs will be
> deployed from one single host, on to the machines where flume agents are
> running. Only when each agent starts, invalid confs are identified because
> the Agents would terminate by throwing exceptions. This is a first attempt to
> make a component-aware configuration system which is independent of the
> Flume, and does not require the Flume jar to be installed. Each component
> needs a configuration manager, which configures the components.
> Provide abstract Configuration stubs for each component type, sources,
> channels, sinks, selectors etc, which are in the new package, independent on
> ng-core. Now for each of the channels extend the abstract class and check the
> config properties for each of the required parameters for that component, for
> example: MultiplexingChannelSelectorConfigurator would look for default
> channel etc. If a particular component does not have a configuration class
> then let the current mechanism continue.
> This will require implementation for each component, but it should not be too
> complex. One additional advantage we get from this is that we can separate
> out the config validation from the components into these stubs, but we will
> still need to read the values out of the Context as we do currently(else we
> end up making the configuration dependent on flume-ng-core itself which we
> wanted to avoid).
> A problem with this is making a change to the configuration would require
> changes in the configuration classes and in the components also(where the
> configuration is read and the component is actually configured). I could not
> figure out a way of avoiding this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
