Updated Branches: refs/heads/flume-1.4 5876271c0 -> 7d3e26e2a
FLUME-1845. Document plugin.d directory structure (Mike Percy via Hari Shreedharan) Project: http://git-wip-us.apache.org/repos/asf/flume/repo Commit: http://git-wip-us.apache.org/repos/asf/flume/commit/7d3e26e2 Tree: http://git-wip-us.apache.org/repos/asf/flume/tree/7d3e26e2 Diff: http://git-wip-us.apache.org/repos/asf/flume/diff/7d3e26e2 Branch: refs/heads/flume-1.4 Commit: 7d3e26e2af8693c44bb068950ac052a1b2f4f51f Parents: 5876271 Author: Hari Shreedharan <[email protected]> Authored: Fri Jan 25 19:41:03 2013 -0800 Committer: Hari Shreedharan <[email protected]> Committed: Fri Jan 25 19:41:30 2013 -0800 ---------------------------------------------------------------------- flume-ng-doc/sphinx/FlumeUserGuide.rst | 45 +++++++++++++++++++++++++++ 1 files changed, 45 insertions(+), 0 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/flume/blob/7d3e26e2/flume-ng-doc/sphinx/FlumeUserGuide.rst ---------------------------------------------------------------------- diff --git a/flume-ng-doc/sphinx/FlumeUserGuide.rst b/flume-ng-doc/sphinx/FlumeUserGuide.rst index 898de23..8eb3734 100644 --- a/flume-ng-doc/sphinx/FlumeUserGuide.rst +++ b/flume-ng-doc/sphinx/FlumeUserGuide.rst @@ -215,6 +215,51 @@ The original Flume terminal will output the event in a log message. Congratulations - you've successfully configured and deployed a Flume agent! Subsequent sections cover agent configuration in much more detail. +Installing third-party plugins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Flume has a fully plugin-based architecture. While Flume ships with many +out-of-the-box sources, channels, sinks, serializers, and the like, many +implementations exist which ship separately from Flume. + +While it has always been possible to include custom Flume components by +adding their jars to the FLUME_CLASSPATH variable in the flume-env.sh file, +Flume now supports a special directory called ``plugins.d`` which automatically +picks up plugins that are packaged in a specific format. This allows for easier +management of plugin packaging issues as well as simpler debugging and +troubleshooting of several classes of issues, especially library dependency +conflicts. + +The plugins.d directory +''''''''''''''''''''''' + +The ``plugins.d`` directory is located at ``$FLUME_HOME/plugins.d``. At startup +time, the ``flume-ng`` start script looks in the ``plugins.d`` directory for +plugins that conform to the below format and includes them in proper paths when +starting up ``java``. + +Directory layout for plugins +'''''''''''''''''''''''''''' + +Each plugin (subdirectory) within ``plugins.d`` can have up to three +sub-directories: + +#. lib - the plugin's jar(s) +#. libext - the plugin's dependency jar(s) +#. native - any required native libraries, such as ``.so`` files + +Example of two plugins within the plugins.d directory: + +.. code-block:: none + + plugins.d/ + plugins.d/custom-source-1/ + plugins.d/custom-source-1/lib/my-source.jar + plugins.d/custom-source-1/libext/spring-core-2.5.6.jar + plugins.d/custom-source-2/ + plugins.d/custom-source-2/lib/custom.jar + plugins.d/custom-source-2/native/gettext.so + Data ingestion --------------
