GitHub user ottobackwards reopened a pull request:
https://github.com/apache/metron/pull/865
METRON-1212 The bundle System and Maven Plugin (Feature Branch)
This PR contains the Bundle system and Maven Plugin.
The bundle system and the plugin are adapted from the Apache Nifi project.
## bundles-maven-plugin
The bundles-maven-plugin is an adapted version of the jar dependency plugin
whose function is to bundle a jar of jars based on the dependencies for a
project. It also creates metadata attributes.
A project's jar, and it's non-provided dependency jars are place in a /lib
entry in the bundle, with the bundle itself being in jar format.
## bundles-lib
The bundles-lib contains the functionality required to:
- discover bundles
- inspect bundles for exposed extension types
- load the bundles
- create special class loaders for bundles
- deliver instances of extension types for use
NAR exposed the bundles through many classes. I have created the
BundleSystem interface to expose a more usable, simplified api for our use
cases.
### From the original PR for METRON-777:
Metron Bundle Plugin
- adaptation of the nifi plugin
- more configurable wrt file extension/dependency and metadata naming
bundle-lib
- adaptation of nifi-nar-utils to be used outside of the nifi project
- rudimentary extensibility to allow configuration and injection of service
types and other things that were hard coded to nifi
- refactored from File based to VFS based
- rebranding to Bundle from Nar ( although the lib and the plugin allow
that to be configured now )
- added capability to the properties class to write to stream, adapted to
uri from paths
- added integration tests for hdfs
- changed to be ClassIndex based instead of ServiceLoader. Service loader
is slower, and Casey's ClassIndex work is great. This also removes the NAR's
required manual maintenance of the service file.
- refactored to use VFS to load the bundle/nar into the classloader AND to
use VFS to load the dependency jars -> VFS as a composite filesystem. Thus
going from NAR's 'working directory', exploded NARS to just loading the
bundle/nar.
## Previous Review
Please see [@mattf_apache's
review](https://github.com/apache/metron/pull/530/files/c5f8c34e4de8e6d456b97edd6f8a0d33b4819d69)
## changes from that review
I have changed the InitContext operations to have explicit builders, and
made it so that creating a context can be done separately from initialization.
Two contexts can then be 'merged'. This is to allow for the addition of new
bundles after initialization.
In preparing this PR I have:
- made checkstyle fixes
- fixed several types
- added a requested set of tests loading and executing simple
interface/implementation from bundle beyond what is already in the bundle-lib
tests
## Testing
*` cd bundles-maven-plugin && mvn -q install && cd .. ` must be run once to
install the maven plugin
* This review is code review and test code review and running only
* [Test Project](https://github.com/ottobackwards/test-bundles-plugin) can
be examined as a simple example of how to create bundles.
* The README.md has getting started and quickstart sections with some
overview of creating by hand
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to
be created at [Metron
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [x] Have you ensured that the full suite of tests and checks have been
executed in the root metron \
- [x] Have you written or updated unit tests and or integration tests to
verify your changes?
### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in
which it is rendered by building and verifying the site-book?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ottobackwards/metron fifth_bundles
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/865.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #865
----
commit 207d367e360074ad53d6f7ece4e1f1febc5a64e1
Author: Otto Fowler <ottobackwards@...>
Date: 2017-09-27T10:29:42Z
The bundle=lib and maven plugin for bundles.
as compared to the feature branch the differences are:
- Added some modules to test loading a bundle, and executing a function
- checkstyle changes
- some minor typo fixes
commit 63134bb7af555fe0896152e7f3a8ba5841a8c1e0
Author: Otto Fowler <ottobackwards@...>
Date: 2017-09-27T11:48:50Z
update readme based on new build options around delay loading
commit 1971262db7f68e9501bf3de1c6b6a10032ace0fc
Author: Otto Fowler <ottobackwards@...>
Date: 2017-10-25T18:43:28Z
fix versions
commit 4f618979038f4a0ab8685ca22de0d2857df71444
Author: Otto Fowler <ottobackwards@...>
Date: 2017-10-27T16:14:21Z
expanded the readme with some getting started and quickstart information
commit 98f5e183e1bda7e589772e300094dfa21a436dd9
Author: Otto Fowler <ottobackwards@...>
Date: 2017-10-27T16:17:58Z
further refinement
commit 773011f65067097cfe3d5750f7397c046621f7cd
Author: Otto Fowler <ottobackwards@...>
Date: 2017-10-27T19:59:44Z
add some doc around this test project
commit 6a592267295ab2223fd9a628d75f3afb6232fd6f
Author: Otto Fowler <ottobackwards@...>
Date: 2017-10-27T20:04:36Z
clarify that the code is not built
commit ea802f2ed8822c35931f0a26955837f6c1ba9de9
Author: Otto Fowler <ottobackwards@...>
Date: 2017-10-27T20:07:34Z
more fixes
commit 1d18bd2582a62fbbf9edc18ae73ec2e1d93e5f88
Author: Otto Fowler <ottobackwards@...>
Date: 2017-11-08T15:25:41Z
add in the instructions for building metron with the plugin
commit 7c3296a8ab8d0f241d513e7935641368d7dce533
Author: Otto Fowler <ottobackwards@...>
Date: 2017-11-08T15:34:49Z
plugin build note
commit 4c8363a58841c56512696bbe8b899cd76e899bc7
Author: Otto Fowler <ottobackwards@...>
Date: 2017-11-09T16:10:13Z
spelling fix, and note on @IndexSubclasses
commit 5a46767cff88999760c82fc1f3fefc3be483e1da
Author: Otto Fowler <ottobackwards@...>
Date: 2017-11-09T16:26:14Z
note on ClassIndex
commit cd1e574d611c1fe520fcfadfa70244edce4c6e2f
Author: Otto Fowler <ottobackwards@...>
Date: 2017-11-15T16:55:56Z
Added a usage guide the describes how to create a bundle and use it
Clean up the bundles-testing
commit 3c1545f1b24d1662bfd7f159ceb419c1c0f67192
Author: Otto Fowler <ottobackwards@...>
Date: 2017-11-15T16:59:31Z
fix name
commit 567cdeaf4de00cb3ec32b1c5794d958f9f39d894
Author: Otto Fowler <ottobackwards@...>
Date: 2017-11-22T14:15:59Z
refactored to metron-bundles directory
commit 2bbd3f7244ad77a76ca4984dde2c183b3af96fc5
Author: Otto Fowler <ottobackwards@...>
Date: 2017-11-22T14:52:47Z
fix parent
commit e4d9ac279c4949cdbbbd6f76c1741f6bdaeb9211
Author: Otto Fowler <ottobackwards@...>
Date: 2017-12-12T20:00:32Z
remove ./ pathing in tests
commit d13b13012fb8badbcb41003ec82c6de844a64624
Author: Otto Fowler <ottobackwards@...>
Date: 2017-12-12T20:15:22Z
change tests to try to track down issue in travis
commit 37587689e9feadeb54465aa7319f114a4e11255b
Author: cstella <cestella@...>
Date: 2017-12-12T20:24:07Z
METRON-1306: When index template install fails, we should fail the install
closes apache/incubator-metron#834
commit 633ccf0a9a73b5ae408cb30fb3a804b301e7dbed
Author: Otto Fowler <ottobackwards@...>
Date: 2017-12-12T20:31:51Z
trying to go to older trusty
commit 6ad350adb5b3b4bf040464da8fab4b0385444102
Author: Otto Fowler <ottobackwards@...>
Date: 2017-12-12T20:54:27Z
revert travis version, do not put stuff into target
commit 4d9262dae01336b477ec7a1c92ad7c3bef0974b4
Author: Otto Fowler <ottobackwards@...>
Date: 2017-12-12T21:17:34Z
try to not use target in these tests
commit 59fec256855ba21b9f4b4981979e340afde06312
Author: Otto Fowler <ottobackwards@...>
Date: 2017-12-13T11:53:13Z
fixup copies
commit 6ca08b9f598b649d60c6b6468164a374b7f6f555
Author: MohanDV <mohan.dv@...>
Date: 2017-12-13T15:55:04Z
METRON-1343 Swagger UI for User Controller needs request method (MohanDV
via ottobackwards) closes apache/metron#862
commit d9ed1bad1f5b2e2471fbea11353f2947f7f52e13
Author: nickwallen <nick@...>
Date: 2017-12-14T15:59:27Z
METRON-1349 Full Dev Builds Metron Twice (nickwallen) closes
apache/metron#866
commit c4cee6af64eda4db4da3eff86abab7d4ae4ec56a
Author: mmiklavc <michael.miklavcic@...>
Date: 2017-12-14T20:29:49Z
METRON-1345: Update EC2 README for custom Ansible (mmiklavc via mmiklavc)
closes apache/metron#859
commit 7d16af7868e22acacd7cfa44dccc7a5fe053e08c
Author: Otto Fowler <ottobackwards@...>
Date: 2017-12-15T13:47:33Z
Merge remote-tracking branch 'apache/master' into fifth_bundles
commit 558df79961ef9295784fe0cd246aac80e009387d
Author: Otto Fowler <ottobackwards@...>
Date: 2017-12-15T14:54:40Z
format fixes based on review
commit ad3d46b681702d4cf76d1b68d322a1046bd65174
Author: Otto Fowler <ottobackwards@...>
Date: 2017-12-15T22:27:27Z
fix typo per review
commit 306ef833067f8e4dc146ab3f5a98dd78590fb713
Author: Otto Fowler <ottobackwards@...>
Date: 2017-12-30T14:35:22Z
add headers to markdown files
----
---
