GitHub user iraghumitra opened a pull request: https://github.com/apache/metron/pull/710
Metron-1083: Add filters using faceted search capabilities of metron-rest-api ## Contributor Comments This PR is on top of [METRON-1068](https://github.com/apache/metron/pull/699). This PR adds faceted search capabilites to the UI. - A new facet pane is added to GUI that groups data based on the following 'hard coded' fields ('source:type', 'ip_src_addr', 'ip_dst_addr', 'host', 'enrichments:geo:ip_dst_addr:country') - The count of unique values for each facet and count for individual values in facet are displayed in the UI - User can expand collapse each facet - Clicking on one of the facet value would add the facet value to search and the facet values are updated in accordance with the new search criteria - All the functionality in the search like hover delete, clear is intact - If you rename a column using the rename option in column settings the renamed field name would appear in the Facet filter as well as in the search ![image](https://user-images.githubusercontent.com/15019012/29522018-755cdbd8-86a5-11e7-87e1-5bff6db36ba7.png) ![image](https://user-images.githubusercontent.com/15019012/29521892-ef9ff19c-86a4-11e7-9917-6a79f6ede10b.png) ## Testing If you run the metron-alerts UI using any one of the ways mentioned below you would notice that UI has a new facet pane on the left of the page and all the functionality described above should be working. There are through E2E tests written to test the faceted seacrh functionality. **Dev:** From 'metron/metron-interface/metron-alerts' you can run `./scripts/start-dev.sh` and open the GUI at `localhost:4200` **E2E:** From 'metron/metron-interface/metron-alerts' you can run `./scripts/start-server-for-e2e.sh` in one terminal and run `npm run e2e` in a seperate terminal **Deployment:** Follow the steps mentioned in [Readme](https://github.com/apache/metron/tree/master/metron-interface/metron-alerts#installing-on-an-existing-cluster) and you should see a login page and data being fetched from rest-api's ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron. Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions. Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [x] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [x] Have you included steps or a guide to how the change may be verified and tested manually? - [x] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: ``` mvn -q clean integration-test install && build_utils/verify_licenses.sh ``` - [x] Have you written or updated unit tests and or integration tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book mvn site ``` #### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/iraghumitra/incubator-metron METRON-1083 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/710.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #710 ---- commit e22a82ec19d3e48b1dca629eaea4afcefa22acf1 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-05-23T12:40:41Z Initial commit for alerts commit 38c3abb7222251a5c97c4936ab16eec9ddf5eca2 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-05-23T12:43:26Z Formatted README commit b5e1e10091c6d77d447334069d17997a33d50a80 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-05-23T12:50:54Z Added TOC and all sections i wanted commit ad34ab1377889c8b47a70b9e65d1e760f20ffd1e Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-05-24T12:50:44Z Added script to deploy on cluster updated readme for the same Integrated pom.xml to build metron alerts commit ca7d317735b152ec7d6fa50b2791261a63c84829 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-01T11:26:20Z - Added displayQuery to show renamed cols in search bar - Added translate pipe to rename col names - Added option to rename col's commit e96d7b9b0947b50c56e0a557051da094abe0c3b9 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-01T11:27:17Z Added missed file commit c0b5ffbbf2d000892cc74e50492a7ec5be1e2fec Author: RaghuMitra <raghumitra....@gmail.com> Date: 2017-06-01T11:28:33Z Merge pull request #1 from iraghumitra/rename-cols Rename cols commit 5a5cb1876c72924c39426ac9e019b445db538e23 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-01T12:03:50Z Added missing pipe commit 9a15b3d7d1327032ee6be91200434a10e9cbdfc9 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-01T14:06:56Z - Indent space in search bar add 10 to 15 pixels of indent on the left and right side of the textbox - Take out trashcan for recent - Change recent to 10 by default but store 25, Iâll do a design for how that needs to look - when the search box overflows, the save search button should be centered vertically with the expanded textbook - Search button should be square commit 8962211a3a557d5d371e63da19a580c3ef0926e8 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-08T07:24:52Z 1. Removed the filters from list page 2. Corrected the mapping for alert id 3. Fixed issue with severity colour coding, the color coding was disappearing if we hover in and hover out of the severity cell 4. Moved alert-severity.directive to directives folder 5. Removed bootstrap from angular-cli as this is already included by ng build commit 17b1e3a507fcf3260e461bb9dd4451301e5420e1 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-19T07:39:01Z Merge branch 'master' into METRON-988 commit 9dd171a73aaa4af126d68e351e2b5cda03fb7676 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-19T13:30:07Z Added e2e test cases Added license header to all files commit 3d34df625fd9cd6d0f449732a1ba65bbec9fbe94 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-19T14:50:00Z Added licence header to all files Removed failing text commit 8375ce94feb71993aa90ad788cfcb99719c91391 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-19T14:58:34Z Updated Readme for e2e commit 979469c851b1b060379b2117cdaceacf5ada8e92 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-19T15:52:09Z Update version in package.json Renamed mock folder to mock-data commit 4d7109d2e7d3038a11e06356ed411e214247ef26 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-19T16:06:05Z Made lint happy commit 5d24357de83b110195a9e78566fabfac23e342fd Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-19T17:12:34Z Fixed brokenlink in readme commit 28ce6f60af63763be4093315c397f31a236a6cf5 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-19T17:14:10Z Fixed brokenlink in readme commit 8b2c78836045be67cb8118c4c724e80e89760374 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-21T13:06:18Z Added e2e test case for deleting the first search when we have more than one search request Escaping the values in search request Removed the unused variable in AlertService commit 377e5bebc38326bc3e125b0a9c16b1e3bb9beafa Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-22T01:32:08Z Changed score fields from _score to Added filter support for score 'threat:triage:score' Mapping _id field to _uid for sorting commit 6da19e288bdf800961cafff1caf77057e71c6b1f Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-23T13:14:14Z Merge remote-tracking branch 'incubator-metron/master' commit 533703649b96ace8984fb6b8d4d9b523d8e3cb59 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-06-30T13:04:22Z Added extra spaces to all the missing files Added ColumnMetaData type in return types Corrected the application name in expressjs server Removed aggs in QueryBuilder as it is not used yet commit 91b41ae924f76d2f95dbe4748cfd5040f5db9328 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-07-03T09:16:19Z - Fixed the key used to display the score in details pane - Fixed the fields in Alerts - Renamed MetadataUtil to more appropriate ElasticsearchUtils - Added AlertsSearchResponse as a obj to hold the search responses - Added abstract class 'DataSource' that holds all the api requests needed by GUI - Added new class 'ElasticSearchLocalstorageImpl' that provides solr and local storage implementation for DataSource commit 23f1c2039a195e3e58ddf1d1a1e67029454f19c7 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-07-03T09:23:08Z Made lint happy commit 42a78c7273a6e90d43b4fa78d75d020dce2d0ff0 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-07-03T09:54:25Z Checking for datasource before injecting ES Impl commit 210c0996958168e8f3e9f2336b0cf521896dbbe2 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-07-04T05:16:40Z Renamed MetadataUtils to ElasticsearchUtils Abstracted search-request from query-builder commit 52dee65d9b1606a6c4366f2d98296468ebcad81b Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-07-04T05:45:39Z Moved query builder to alert-list commit ec3733c96df27ba5165a632b6058ec69435754ba Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-07-09T11:03:14Z Fetching only the display fields in the query Search data to be fired after getting all the columns for display commit 8bae3ced9e0583e4d74bda05b459fbbef1dc8f7c Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-07-09T11:03:52Z Merge remote-tracking branch 'incubator-metron/master' commit b5b09cab6e1f3bf9995e6965b3e16381790b4b71 Author: iraghumitra <raghumitra....@gmail.com> Date: 2017-07-09T11:04:26Z Merge branch 'master' into METRON-988 ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---