GitHub user wardbekker opened a pull request:
https://github.com/apache/metron/pull/619
METRON-939 Elasticsearch ES5 with Xshield client support
## Contributor Comments
This is a work-in-progress fix for Elasticsearch ES5 with Xshield client
support. I'm opening a pull request to work through the remaining issues with
the team. To be clear, the aim is discussion only for now, no for actually
pulling in the code.
Open issues in for this pull request:
- While the metron-elasticsearch unit and integration tests are working,
the ElasticsearchDataPrunerTest from metron-datamanagement are not. It can't
seem to initialize ElasticSearch classes (e.g. java.lang.NoClassDefFoundError:
Could not initialize class org.elasticsearch.common.ParseField). It seems to be
related with this issue: https://github.com/elastic/elasticsearch/issues/21627
. I could use some help with resolving this.
- The ES server used in the integration test is not using Xpack
authentication. I could not find any public reference on this. Using
XPackPlugin.class for the test node in place of the netty4 plugin doesn't seem
to work. Xpack authentication will likely work for the writer bolt, but no
integration test can be added until this issue is resolved.
- I've updated the index template for yaf as used in the integration tests
to be ES5.x compatible. Other index templates need the same treatment
- ES & Kibana Mpacks are not updated.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wardbekker/metron elasticsearch
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/619.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #619
----
commit d69d86c32434145d3e489490c784e3d4543e624a
Author: Ward Bekker <[email protected]>
Date: 2017-06-14T00:40:58Z
Upped the ES version and made it compile
commit ba80aabd37f471fb34492dd8b79dbb79bead58c8
Author: Ward Bekker <[email protected]>
Date: 2017-06-14T15:03:27Z
fixed jackson dependency
commit d529af6064f16711fde6b447ced112a3d513e8c5
Author: Ward Bekker <[email protected]>
Date: 2017-06-14T21:12:29Z
Using netty4 transport type
commit e29514cd72ffa829f1f51286b3dcf3ab4b56a38a
Author: Ward Bekker <[email protected]>
Date: 2017-06-15T16:48:43Z
Changed transport client to Xpack transport client
commit e8412e07ca42a8f6c66d14150f7a03054be85d4a
Author: Ward Bekker <[email protected]>
Date: 2017-06-16T13:46:02Z
PreBuiltXPackTransportClient working
commit ea29f157a9f9492c4b4aeacebfd9a14dd5dc8b07
Author: Ward Bekker <[email protected]>
Date: 2017-06-16T14:02:12Z
remove uncommented code
commit bfade63b992f4366a8acb5dd61f4274f8e153c8c
Author: Ward Bekker <[email protected]>
Date: 2017-06-17T14:52:06Z
Green on all elasticsearch integration tests
commit 34b5b8e93083f871f628959614f66e16ad5b3056
Author: Ward Bekker <[email protected]>
Date: 2017-06-17T15:10:56Z
changed template to ES 5.0 schema
commit 3c2c5e94235cc4fdb1d0df0c891925c46f597da5
Author: Ward Bekker <[email protected]>
Date: 2017-06-19T08:05:46Z
Integration test cleanup
commit 21a1a71ec4b5b580d176f6fee85e6b742c37d1ff
Author: Ward Bekker <[email protected]>
Date: 2017-06-19T09:10:31Z
using the shaded jar in data-management project
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
