[GitHub] metron issue #831: METRON-1302: Split up Indexing Topology into batch and ra...

[mmiklavc]

Github user mmiklavc commented on the issue:

    https://github.com/apache/metron/pull/831
  
    Unless there are other things I should test, I think it looks pretty good. 
Data is coming through to ES + Kibana (with the new ES 5.6.2 code, no less!), 
the topologies appear as expected and without error in Storm, data is being 
written to HDFS, and the menu options appear correct in Ambari. +1 home skillet.
    
    
![image](https://user-images.githubusercontent.com/658443/34692410-4fb37a36-f47d-11e7-841f-2ff820a17b0d.png)
    
    
![image](https://user-images.githubusercontent.com/658443/34692421-5d39f40a-f47d-11e7-9cce-b29b49e1dbe9.png)
    
    And data in HDFS
    
    ```
    [root@node1 indexing]# hdfs dfs -ls -R /apps/metron/indexing/indexed
    drwxrwxr-x   - storm hadoop          0 2018-01-08 21:00 
/apps/metron/indexing/indexed/bro
    -rw-r--r--   1 storm hadoop    2984217 2018-01-08 21:00 
/apps/metron/indexing/indexed/bro/enrichment-hdfsIndexingBolt-2-0-1515445254997.json
    drwxrwxr-x   - storm hadoop          0 2018-01-08 21:10 
/apps/metron/indexing/indexed/error
    -rw-r--r--   1 storm hadoop    6670648 2018-01-08 21:10 
/apps/metron/indexing/indexed/error/enrichment-hdfsIndexingBolt-2-0-1515445818471.json
    drwxrwxr-x   - storm hadoop          0 2018-01-08 21:01 
/apps/metron/indexing/indexed/snort
    -rw-r--r--   1 storm hadoop    2374289 2018-01-08 21:01 
/apps/metron/indexing/indexed/snort/enrichment-hdfsIndexingBolt-2-0-1515445279421.json
    ```


---