[GitHub] incubator-metron pull request: Metron 142 Simplify Parser configur...
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/120#discussion_r63255049 --- Diff: metron-deployment/roles/metron_streaming/tasks/metron_topology.yml --- @@ -16,7 +16,7 @@ # --- - name: Submit Metron Parser topologies --- End diff -- Based on recent changes, we no longer submit the topologies as part of the deployment process. The topologies (and most other services) are started at the end of the deployment only if the user has configured them to start. Based on the new arguments, we'll need updates in the monit space. I'm thinking at least these will be impacted. - `metron-deployment/roles/monit/templates/monit/parsers.monit` - `metron-deployment/roles/monit/templates/scripts/start_topology.sh` Let me know if you'd like help tackling that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-metron pull request: Added vagrant install hostmanager s...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-metron/pull/121 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-metron pull request: METRON-132 Monit Integration
Github user cestella commented on the pull request: https://github.com/apache/incubator-metron/pull/118#issuecomment-219158724 +1, looks great! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-metron pull request: [METRON-150] Adding WebSphere parse...
Github user DomenicPuzio commented on the pull request: https://github.com/apache/incubator-metron/pull/115#issuecomment-219095558 @merrimanr and @cestella, thank you SO MUCH for the help! I really appreciate it! I will certainly follow the steps to test on vagrant for future parsers. I'm very excited to be a contributor, and I'm already working on my next parser! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-metron pull request: Added vagrant install hostmanager s...
Github user cestella commented on the pull request: https://github.com/apache/incubator-metron/pull/121#issuecomment-219068764 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-metron pull request: METRON-155 Added query filtering ca...
Github user nickwallen commented on the pull request: https://github.com/apache/incubator-metron/pull/119#issuecomment-219059773 I think that makes sense, especially since this creates the 'hook' for later contributions of BPF. The only downside is that we don't want to confuse users by having 8 ways to do things. But I don't think that will be a problem as long as we eventually get the BPF functionality. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-metron pull request: METRON-155 Added query filtering ca...
Github user cestella commented on the pull request: https://github.com/apache/incubator-metron/pull/119#issuecomment-219049457 @nickwallen Agreed we should support BPF. This PR makes the filter pluggable and we already have the query language. We can have a follow-on PR for BPF support IMO. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-metron pull request: [METRON-150] Adding WebSphere parse...
Github user cestella commented on the pull request:
https://github.com/apache/incubator-metron/pull/115#issuecomment-219048736
Ok, tested this. Sorry it took so long (vagrant drama).
Ok, so this is what I did to test this.
Spin up the full-dev-vagrant:
1. Do a build by running `mvn clean integration-test` from the
`metron-platform` directory
2. From the `metron-deployment/vagrant/full-dev-platform` directory run
`vagrant up`
* Note: If you already have a vagrant machine running and you just want
to
redeploy code, you should be able to do `vagrant provision --tags
enrichment`
Log into the full-dev-vagrant and do the following:
* Optionally, stop some things to give myself enough memory headroom:
* Kill every running parser topology via the [storm
UI](http://node1:8744/index.html) to proceed (that VM is pretty packed atm).
* Stop pcap replay via `/etc/init.d/pcap-replay stop`
* Kill bro via `/usr/local/bin/broctl` and type `stop` and then `exit`
at the broctl prompt
* Marvel at the lack of fans running on your laptop
* Create the kafka topic for the sensor:
`/usr/hdp/current/kafka-broker/bin//kafka-topics.sh --zookeeper localhost:2181
--create --topic websphere --partitions 1 --replication-factor 1`
* Create text file with sample data in it called `WebsphereOutput.txt`.
This is essentially just
[WebsphereOutput.txt](https://raw.githubusercontent.com/DomenicPuzio/incubator-metron/af50623dcb764fda0281b3657c0f40c993c958f3/metron-platform/metron-integration-test/src/main/resources/sample/data/SampleInput/WebsphereOutput.txt)
* Start the parser topology using
`/usr/metron/0.1BETA/bin/start_parser_topology.sh`, in this case
`/usr/metron/0.1BETA/bin/start_parser_topology.sh websphere`
* Add your data to the kafka queue you created earlier via `cat
WebsphereOutput.txt |
/usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list
node1:6667 --topic websphere`
* Wait a minute and then check the elasticsearch head plugin (if you
haven't installed it, you can via `/usr/share/elasticsearch/bin/plugin -install
mobz/elasticsearch-head`).
* Browse to the `Browser` tab and click on the index named
`${sensor_name}_index_${DATE}` (in my case: `websphere_index_2016.05.13.13`
* You should see some messages. Click on each one and inspect the JSON
to ensure it looks right.
For future parsers, this is a good template for acceptance testing. You
shouldn't have to go through a full maven provisioning each time if you
keep the VM around and just run `vagrant provision -tags enrichment`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
