GitHub user nickwallen opened a pull request:
https://github.com/apache/metron/pull/903
METRON-1370 Create Full Dev Equivalent for Ubuntu
This PR adds a single node, VM based deployment mechanism based on Ubuntu
Trusty. This has feature parity with the existing CentOS single node VM. This
will help us continue to improve and support the Ubuntu-based DEB packages that
were added in #868.
### Changes
* The Metron, Elasticsearch, and Kibana MPacks were updated to support
installation on Ubuntu. This included making the following changes.
* Adding repo definitions for Ubuntu
* Changing the default location of the `/etc/sysconfig/elasticsearch`
configuration to work across platforms
* Adding an additional configuration in the MPack to allow
Elasticsearch to perform a memory lock on Ubuntu
* Support configuration of Elasticsearch on a platform running Systemd;
like Ubuntu
* The Elasticsearch MPack was enhanced to correctly perform a service
check. Previously the hostname and port were hard coded and the service check
was not functional in most environments.
* The Ubuntu DEBs were enhanced to allow for the addition of "maintainer
scripts". This allows the Management UI and Alerts UI service scripts to be
installed with the DEBs. This was needed to reach feature parity with the RPMs.
* The name `full-dev-platform` no longer makes sense IMHO. We do not have
a `quick-dev-platform` any longer to distinguish it from. Now the main
distinguisher is the underlying operating system. I renamed our "Full Dev"
development environment to `metron-on-centos` and `metron-on-ubuntu`. I am
completely open to community suggestions on what these should be named.
```
metron-deployment/vagrant/
âââ README.md
âââ fastcapa
âââ metron-on-centos
âââ metron-on-ubuntu
```
* The organization of `metron-deployment` had to change so that assets
could be reused across both the CentOS and Ubuntu deployments. All shared
Ansible assets were moved to `metron-deployment/ansible`. This includes the
Ansible modules, playbooks and roles that are all used across multiple
environments.
```
metron-deployment/ansible/
âââ extra_modules
âââ playbooks
âââ roles
```
* The following Ansible roles had to be updated to support both CentOS and
Ubuntu based on small differences in package names and conventions.
* `ambari-common`
* `ambari-config`
* `ambari-master`
* `ambari-slave`
* `libselinux-python`
* `ntp`
* A new role was added to enable swap space; `enable-swap`. This is
required for the Ubuntu deployment as the underlying Ubuntu Trusty image does
not have swap space enabled by default.
* The `metron-builder` role was changed to selectively build either the
DEBs or the RPMs as needed.
* The `metron-rpms` role was renamed to `metron-packages` and was also
enhanced to create the local repository on a Ubuntu host. This also
selectively copies either RPMs or DEBs to the VM as needed.
* Monit is no longer needed and so has been removed from all VM
deployments. Monit was added prior to the existence of our Mpack installer.
It is no longer needed
* Removed the `metron-streaming` role which is no longer applicable since
these functions are now performed by the MPack.
### Testing
I have performed the following testing based on these changes.
- [ ] Run Metron on CentOS.
```
cd metron-deployment/vagrant/metron-on-centos
vagrant up
```
* Validate that alerts are visible within the Alerts UI.
* Validate that the Metron Service Check completes successfully.
* Validate that the Elasticsearch service check completes successfully.
Be sure to shutdown and halt or destroy this VM before moving on to the
next step.
- [ ] Run Metron on Ubuntu.
```
cd metron-deployment/vagrant/metron-on-centos
vagrant up
```
* Validate that alerts are visible within the Alerts UI.
* Validate that the Metron Service Check completes successfully.
* Validate that the Elasticsearch service check completes successfully.
Be sure to shutdown and halt or destroy this VM before moving on to the
next step.
- [ ] Run one of the Fastcapa test environments.
```
cd metron-deployment/vagrant/fastcapa/centos-7.1/
vagrant up
```
* If the process fails at the task "fastcapa : Restart for modified
kernel params" simply run `vagrant provision` again.
* Ensure that the deployment process reports success. Automated
validation is performed as part of the deployment process.
- [ ] Deploy Metron to AWS EC2.
```
cd metron-deployment/amazon-ec2
./run.sh
```
* Validate that alerts are visible within the Alerts UI.
* Validate that the Metron Service Check completes successfully.
* Validate that the Elasticsearch service check completes successfully.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nickwallen/metron METRON-1370-PR
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/903.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #903
----
commit 5cb44aadece20b185d552c9e869fec5e01eff77b
Author: Nick Allen <nick@...>
Date: 2018-01-03T17:54:04Z
METRON-1370 Create Full Dev Equivalent for Ubuntu
----
---
