[GitHub] metron issue #943: METRON-1462: Separate ES and Kibana from Metron Mpack

2018-02-28 Thread anandsubbu
Github user anandsubbu commented on the issue:

https://github.com/apache/metron/pull/943
  
+1

Works as advertised. I used both mpacks to deploy a 12-node CentOS 7 
cluster (Ambari 2.6.0.0 and HDP 2.6.3.0). Was able to kerberize the cluster and 
get bro indices into ES.


---


[GitHub] metron issue #940: METRON-1460: Create a complementary non-split-join enrich...

2018-02-28 Thread mraliagha
Github user mraliagha commented on the issue:

https://github.com/apache/metron/pull/940
  
@cestella Thanks, Casey. Wouldn't be still hard to tune this solution? 
Still, thread pool tuning and probably the race condition between these threads 
and normal Strom workers makes the tuning hard for a production platform with 
tons of feeds/topologies. Storm resource management is very basic at this stage 
to absorb spikes, and having a separate thread pool transfers the complexity 
from one place to another place. 


---


[GitHub] metron pull request #945: METRON-1464: Convert schemas to be compatible with...

2018-02-28 Thread merrimanr
GitHub user merrimanr opened a pull request:

https://github.com/apache/metron/pull/945

METRON-1464: Convert schemas to be compatible with Solr 5.5.2

## Contributor Comments
This PR makes the Solr schema compatible with Solr 5.5.2.  The client code 
worked as is (compiled with version 6.6.2), no changes necessary.

To test this in full dev:

1. Spin up full dev
2. Install HDP Search ("Chapter 3:  Installing HDP Search Using Ambari" in 
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_solr-search-installation/bk_solr-search-installation.pdf)
3. Create the bro and snort collections manually:
```
/opt/lucidworks-hdpsearch/solr/bin/solr create -c bro -d 
/usr/metron/0.4.3/config/schema/bro
/opt/lucidworks-hdpsearch/solr/bin/solr create -c snort -d 
/usr/metron/0.4.3/config/schema/snort
```
4. Create a file named `metron_solr_template_installed_flag_file` in 
$METRON_HOME/config
5. Navigate to the Metron service in Ambari and change the "Solr Zookeeper 
Urls" setting in the Index Settings tab to `node1:2181/solr`
6. Change the "Random Access Search Engine" setting in the Indexing tab to 
`Solr`
7. Restart the Indexing component in Ambari

Navigate to node1:8983 and inspect the bro and snort collections.  They 
should contain data.  There will likely be additional Ambari work should we 
decide to move to HDP Search.  This PR still assumes Solr 6.6.2 will be 
installed manually.

## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [ ] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
  ```

- [] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [x] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] Have you verified the basic functionality of the build by building 
and running locally with Vagrant full-dev environment or the equivalent?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered by building and verifying the site-book? If not then run 
the following commands and the verify changes via 
`site-book/target/site/index.html`:

  ```
  cd site-book
  mvn site
  ```

 Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up 
for your personal repository such that your branches are built there before 
submitting a pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/merrimanr/incubator-metron METRON-1464

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/metron/pull/945.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #945


commit ac1de170af8fa769683f72783f6258cf12663f94
Author: merrimanr 
Date:   2018-02-28T22:49:37Z

initial commit




---


[GitHub] metron issue #936: METRON-1450:Add rest endpoint documentation for splitting...

2018-02-28 Thread MohanDV
Github user MohanDV commented on the issue:

https://github.com/apache/metron/pull/936
  
@nickwallen / @JonZeolla can you please merge this request. 


---


[GitHub] metron issue #940: METRON-1460: Create a complementary non-split-join enrich...

2018-02-28 Thread cestella
Github user cestella commented on the issue:

https://github.com/apache/metron/pull/940
  
@nickwallen Sounds good.  When scale tests are done, can we make sure that 
we also include #944 ?


---