[jira] [Commented] (METRON-1809) Support Column Oriented Input with Batch Profiler

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16640348#comment-16640348
 ] 

ASF GitHub Bot commented on METRON-1809:


GitHub user nickwallen opened a pull request:

https://github.com/apache/metron/pull/1229

METRON-1809 Support Column Oriented Input with Batch Profiler

The Batch Profiler currently only accepts input formats that can be 
directly serialized to JSON.  This PR enhances that support to accept a wider 
variety of input formats, most notably columnar formats  like ORC and Parquet.

My first attempt at doing this in #1191 didn't go far enough.  The tests I 
had written only tested reading a single text column from ORC, which is not how 
someone would store the data in ORC.  Each field of the message would be stored 
in a separate column.  The test I had written failed to test this difference.

## Changes

* This introduces the `TelemetryReader` interface that handles differences 
between input formats. 

* This provides two different `TelemetryReader` implementations.  One that 
handles textual input that can be directly serialized; like JSON or CSV.  
Another that handles column oriented input like ORC and Parquet.

* A new property was introduced to make this configurable to allow the user 
to define the `TelemetryReader` implementation.

* The README was updated to define the new property and provide examples 
for common formats.

* Unit and integration tests were added or updated to test the new 
functionality.

## Test Plan

1. Stand-up a development environment.

```
cd metron-deployment/development/centos6
vagrant up
vagrant ssh
sudo su -
```

1. Validate the environment by ensuring alerts are visible within the 
Alerts UI and that the Metron Service Check in Ambari passes.

1. Allow some telemetry to be archived in HDFS.
```
[root@node1 ~]# hdfs dfs -cat /apps/metron/indexing/indexed/*/* | wc -l
6916
```

1. Shutdown Metron topologies, Storm, Elasticsearch, Kibana, MapReduce2 to 
free up some resources on the VM.

1. Use Ambari to install Spark (version 2.3+).  Actions > Add Service > 
Spark2

1. Make sure Spark can talk to HBase.

```
SPARK_HOME=/usr/hdp/current/spark2-client
cp  /usr/hdp/current/hbase-client/conf/hbase-site.xml $SPARK_HOME/conf/
```

1. Follow the Getting Started section of the README to seed a basic profile 
using the text/json telemetry that is archived in HDFS.

1. Create the Profile.

```
[root@node1 ~]# source /etc/default/metron
[root@node1 ~]# cat $METRON_HOME/config/zookeeper/profiler.json
{
  "profiles": [
{
  "profile": "hello-world",
  "foreach": "'global'",
  "init":{ "count": "0" },
  "update":  { "count": "count + 1" },
  "result":  "count"
}
  ],
  "timestampField": "timestamp"
}
```

1. Edit the Batch Profiler properties. to point it at the correct input 
path (changed localhost:9000 to localhost:8020).
```
[root@node1 ~]# cat 
/usr/metron/0.5.1/config/batch-profiler.properties

spark.app.name=Batch Profiler
spark.master=local
spark.sql.shuffle.partitions=8


profiler.batch.input.path=hdfs://localhost:8020/apps/metron/indexing/indexed/*/*
profiler.batch.input.format=text

profiler.period.duration=15
profiler.period.duration.units=MINUTES
```

1. Edit logging as you see fit.  For example, set Spark logging to WARN 
and Profiler logging to DEBUG. This is described in the README.

1. Run the Batch Profiler.
```
$METRON_HOME/bin/start_batch_profiler.sh
```

1. Launch the Stellar REPL and retrieve the profile data.  Save this result 
as it will be used for validation in subsequent steps.
```
[root@node1 ~]# $METRON_HOME/bin/stellar -z $ZOOKEEPER
...
Stellar, Go!
Functions are loading lazily in the background and will be unavailable 
until loaded fully.
...
[Stellar]>>> window := PROFILE_FIXED(2, "HOURS")
[ProfilePeriod{period=1707332, durationMillis=90}, 
ProfilePeriod{period=1707333, durationMillis=90}, 
ProfilePeriod{period=1707334, durationMillis=90}, 
ProfilePeriod{period=1707335, durationMillis=90}, 
ProfilePeriod{period=1707336, durationMillis=90}, 
ProfilePeriod{period=1707337, durationMillis=90}, 
ProfilePeriod{period=1707338, 

[GitHub] metron pull request #1229: METRON-1809 Support Column Oriented Input with Ba...

[nickwallen]

GitHub user nickwallen opened a pull request:

https://github.com/apache/metron/pull/1229

METRON-1809 Support Column Oriented Input with Batch Profiler

The Batch Profiler currently only accepts input formats that can be 
directly serialized to JSON.  This PR enhances that support to accept a wider 
variety of input formats, most notably columnar formats  like ORC and Parquet.

My first attempt at doing this in #1191 didn't go far enough.  The tests I 
had written only tested reading a single text column from ORC, which is not how 
someone would store the data in ORC.  Each field of the message would be stored 
in a separate column.  The test I had written failed to test this difference.

## Changes

* This introduces the `TelemetryReader` interface that handles differences 
between input formats. 

* This provides two different `TelemetryReader` implementations.  One that 
handles textual input that can be directly serialized; like JSON or CSV.  
Another that handles column oriented input like ORC and Parquet.

* A new property was introduced to make this configurable to allow the user 
to define the `TelemetryReader` implementation.

* The README was updated to define the new property and provide examples 
for common formats.

* Unit and integration tests were added or updated to test the new 
functionality.

## Test Plan

1. Stand-up a development environment.

```
cd metron-deployment/development/centos6
vagrant up
vagrant ssh
sudo su -
```

1. Validate the environment by ensuring alerts are visible within the 
Alerts UI and that the Metron Service Check in Ambari passes.

1. Allow some telemetry to be archived in HDFS.
```
[root@node1 ~]# hdfs dfs -cat /apps/metron/indexing/indexed/*/* | wc -l
6916
```

1. Shutdown Metron topologies, Storm, Elasticsearch, Kibana, MapReduce2 to 
free up some resources on the VM.

1. Use Ambari to install Spark (version 2.3+).  Actions > Add Service > 
Spark2

1. Make sure Spark can talk to HBase.

```
SPARK_HOME=/usr/hdp/current/spark2-client
cp  /usr/hdp/current/hbase-client/conf/hbase-site.xml $SPARK_HOME/conf/
```

1. Follow the Getting Started section of the README to seed a basic profile 
using the text/json telemetry that is archived in HDFS.

1. Create the Profile.

```
[root@node1 ~]# source /etc/default/metron
[root@node1 ~]# cat $METRON_HOME/config/zookeeper/profiler.json
{
  "profiles": [
{
  "profile": "hello-world",
  "foreach": "'global'",
  "init":{ "count": "0" },
  "update":  { "count": "count + 1" },
  "result":  "count"
}
  ],
  "timestampField": "timestamp"
}
```

1. Edit the Batch Profiler properties. to point it at the correct input 
path (changed localhost:9000 to localhost:8020).
```
[root@node1 ~]# cat 
/usr/metron/0.5.1/config/batch-profiler.properties

spark.app.name=Batch Profiler
spark.master=local
spark.sql.shuffle.partitions=8


profiler.batch.input.path=hdfs://localhost:8020/apps/metron/indexing/indexed/*/*
profiler.batch.input.format=text

profiler.period.duration=15
profiler.period.duration.units=MINUTES
```

1. Edit logging as you see fit.  For example, set Spark logging to WARN 
and Profiler logging to DEBUG. This is described in the README.

1. Run the Batch Profiler.
```
$METRON_HOME/bin/start_batch_profiler.sh
```

1. Launch the Stellar REPL and retrieve the profile data.  Save this result 
as it will be used for validation in subsequent steps.
```
[root@node1 ~]# $METRON_HOME/bin/stellar -z $ZOOKEEPER
...
Stellar, Go!
Functions are loading lazily in the background and will be unavailable 
until loaded fully.
...
[Stellar]>>> window := PROFILE_FIXED(2, "HOURS")
[ProfilePeriod{period=1707332, durationMillis=90}, 
ProfilePeriod{period=1707333, durationMillis=90}, 
ProfilePeriod{period=1707334, durationMillis=90}, 
ProfilePeriod{period=1707335, durationMillis=90}, 
ProfilePeriod{period=1707336, durationMillis=90}, 
ProfilePeriod{period=1707337, durationMillis=90}, 
ProfilePeriod{period=1707338, durationMillis=90}, 
ProfilePeriod{period=1707339, durationMillis=90}, 
ProfilePeriod{period=1707340, durationMillis=90}]

[Stellar]>>> PROFILE_GET("hello-world","global", window)
[1020, 5066, 830]
```

1. Delete the 

[jira] [Created] (METRON-1809) Support Column Oriented Input with Batch Profiler

[Nick Allen (JIRA)]

Nick Allen created METRON-1809:
--

 Summary: Support Column Oriented Input with Batch Profiler
 Key: METRON-1809
 URL: https://issues.apache.org/jira/browse/METRON-1809
 Project: Metron
  Issue Type: Bug
Reporter: Nick Allen
Assignee: Nick Allen


The Batch Profiler currently only accepts input formats that can be directly 
serialized to JSON.  This should be enhanced to accept a wider variety of input 
formats.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (METRON-1808) Add Ansible created pyc to gitignore

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16640300#comment-16640300
 ] 

ASF GitHub Bot commented on METRON-1808:


Github user asfgit closed the pull request at:

https://github.com/apache/metron/pull/1228


> Add Ansible created pyc to gitignore
> 
>
> Key: METRON-1808
> URL: https://issues.apache.org/jira/browse/METRON-1808
> Project: Metron
>  Issue Type: Improvement
>Reporter: Justin Leet
>Assignee: Justin Leet
>Priority: Major
>
> Want to get rid of 
> {code}
> {14:06}~/Documents/workspace/metron:master ✗ ➭ git status
> On branch master
> Your branch is up to date with 'origin/master'.
> Untracked files:
>   (use "git add ..." to include in what will be committed)
>   metron-deployment/ansible/callback_plugins/0_minimum_ansible_version.pyc
> nothing added to commit but untracked files present (use "git add" to track)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron pull request #1228: METRON-1808: Add Ansible created pyc to gitignore

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/metron/pull/1228


---

[jira] [Commented] (METRON-1795) General Purpose Regex Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16640281#comment-16640281
 ] 

ASF GitHub Bot commented on METRON-1795:


Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/1214
  
@jagdeepsingh2  - you *could* try this - 
https://stackoverflow.com/questions/134882/undoing-a-git-rebase, but at this 
point it might be better to just open a new PR bc pushing up to github is going 
to cause some additional drama as well. You'll want to keep the default 
checklist that's populated in the description when you open the PR. Please note 
the comments from @nickwallen and myself regarding what should also be included 
in your description.

In general, once you've pushed a branch to the public it's better to just 
git merge, otherwise you can get into trouble like this. We flatten PR's once 
they're committed to master anyhow.


> General Purpose Regex Parser
> 
>
> Key: METRON-1795
> URL: https://issues.apache.org/jira/browse/METRON-1795
> Project: Metron
>  Issue Type: New Feature
>Reporter: Jagdeep Singh
>Priority: Minor
>
> We have implemented a general purpose regex parser for Metron that we are 
> interested in contributing back to the community.
>  
> While the Metron Grok parser provides some regex based capability today, the 
> intention of this general purpose regex parser is to:
>  # Allow for more advanced parsing scenarios (specifically, dealing with 
> multiple regex lines for devices that contain several log formats within them)
>  # Give users and developers of Metron additional options for parsing
>  # With the new parser chaining and regex routing feature available in 
> Metron, this gives some additional flexibility to logically separate a flow 
> by:
>  # Regex routing to segregate logs at a device level and handle envelope 
> unwrapping
>  # This general purpose regex parser to parse an entire device type that 
> contains multiple log formats within the single device (for example, RHEL 
> logs)
> At the high-level control flow is like this:
>  # Identify the record type if incoming raw message.
>  # Find and apply the regular expression of corresponding record type to 
> extract the fields (using named groups). 
>  # Apply the message header regex to extract the fields in the header part of 
> the message (using named groups).
>  
> The parser config uses the following structure:
>   
> {code:java}
> "recordTypeRegex": "(?(?<=\\s)\\b(kernel|syslog)\\b(?=\\[|:))"  
>  "messageHeaderRegex": 
> "(?(?<=^<)\\d{1,4}(?=>)).*?(?(?<=>)[A-Za-z]{3}\\s{1,2}\\d{1,2}\\s\\d{1,2}:\\d{1,2}:\\d{1,2}(?=\\s)).*?(?(?<=\\s).*?(?=\\s))",
>    "fields": [
>   {
>     "recordType": "kernel",
>     "regex": ".*(?(?<=\\]|\\w\\:).*?(?=$))"
>   },
>   {
>     "recordType": "syslog",
>     "regex": 
> ".*(?(?<=PID\\s=\\s).*?(?=\\sLine)).*(?(?<=64\\s)\/([A-Za-z0-9_-]+\/)+(?=\\w))(?.*?(?=\")).*(?(?<=\").*?(?=$))"
>   }
> ]
> {code}
>  
> Where:
>  * *recordTypeRegex* is used to distinctly identify a record type. It inputs 
> a valid regular expression and may also have named groups, which would be 
> extracted into fields.
>  * *messageHeaderRegex* is used to specify a regular expression to extract 
> fields from a message part which is common across all the messages (i.e, 
> syslog fields, standard headers)
>  * *fields*: json list of objects containing recordType and regex. The 
> expression that is evaluated is based on the output of the recordTypeRegex
>  * Note: *recordTypeRegex* and *messageHeaderRegex* could be specified as 
> lists also (as a JSON array), where the list will be evaluated in order until 
> a matching regular expression is found.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron issue #1214: METRON-1795 Initial commit for a general purpose regular...

[mmiklavc]

Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/1214
  
@jagdeepsingh2  - you *could* try this - 
https://stackoverflow.com/questions/134882/undoing-a-git-rebase, but at this 
point it might be better to just open a new PR bc pushing up to github is going 
to cause some additional drama as well. You'll want to keep the default 
checklist that's populated in the description when you open the PR. Please note 
the comments from @nickwallen and myself regarding what should also be included 
in your description.

In general, once you've pushed a branch to the public it's better to just 
git merge, otherwise you can get into trouble like this. We flatten PR's once 
they're committed to master anyhow.


---

[jira] [Commented] (METRON-1805) Provide a default value for the Storm topology.max.spout.pending setting

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16640265#comment-16640265
 ] 

ASF GitHub Bot commented on METRON-1805:


Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/1221
  
This seems reasonable to me all the way around. +1 by inspection.


> Provide a default value for the Storm topology.max.spout.pending setting
> 
>
> Key: METRON-1805
> URL: https://issues.apache.org/jira/browse/METRON-1805
> Project: Metron
>  Issue Type: Improvement
>Reporter: Ryan Merriman
>Priority: Major
>
> Should we provide a default value for this property?  Which topologies should 
> that apply to?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron issue #1221: METRON-1805: Provide a default value for the Storm topol...

[mmiklavc]

Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/1221
  
This seems reasonable to me all the way around. +1 by inspection.


---

[jira] [Commented] (METRON-1808) Add Ansible created pyc to gitignore

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16640212#comment-16640212
 ] 

ASF GitHub Bot commented on METRON-1808:


Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/1228
  
+1 I noticed this too.  Thanks for the PR


> Add Ansible created pyc to gitignore
> 
>
> Key: METRON-1808
> URL: https://issues.apache.org/jira/browse/METRON-1808
> Project: Metron
>  Issue Type: Improvement
>Reporter: Justin Leet
>Assignee: Justin Leet
>Priority: Major
>
> Want to get rid of 
> {code}
> {14:06}~/Documents/workspace/metron:master ✗ ➭ git status
> On branch master
> Your branch is up to date with 'origin/master'.
> Untracked files:
>   (use "git add ..." to include in what will be committed)
>   metron-deployment/ansible/callback_plugins/0_minimum_ansible_version.pyc
> nothing added to commit but untracked files present (use "git add" to track)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron issue #1228: METRON-1808: Add Ansible created pyc to gitignore

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/1228
  
+1 I noticed this too.  Thanks for the PR


---

[jira] [Updated] (METRON-1808) Add Ansible created pyc to gitignore

[Justin Leet (JIRA)]

 [ 
https://issues.apache.org/jira/browse/METRON-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Justin Leet updated METRON-1808:

Issue Type: Improvement  (was: Task)

> Add Ansible created pyc to gitignore
> 
>
> Key: METRON-1808
> URL: https://issues.apache.org/jira/browse/METRON-1808
> Project: Metron
>  Issue Type: Improvement
>Reporter: Justin Leet
>Assignee: Justin Leet
>Priority: Major
>
> Want to get rid of 
> {code}
> {14:06}~/Documents/workspace/metron:master ✗ ➭ git status
> On branch master
> Your branch is up to date with 'origin/master'.
> Untracked files:
>   (use "git add ..." to include in what will be committed)
>   metron-deployment/ansible/callback_plugins/0_minimum_ansible_version.pyc
> nothing added to commit but untracked files present (use "git add" to track)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron pull request #1228: METRON-1808: Add Ansible created pyc to gitignore

[justinleet]

GitHub user justinleet opened a pull request:

https://github.com/apache/metron/pull/1228

METRON-1808: Add Ansible created pyc to gitignore

## Contributor Comments
Added *.pyc to gitignore for metron-development.  We don't need to commit 
pyc files.

Ansible creates a pyc file (0_minimum_ansible_version.pyc) that isn't 
filtered by gitignore. I added the general pyc ignore, because I don't we care 
about committing pyc files at all.

After running up full dev to create the file., this file should no longer 
be picked up by git.

## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [x] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
  ```

- [x] Have you verified the basic functionality of the build by building 
and running locally with Vagrant full-dev environment or the equivalent?

 Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up 
for your personal repository such that your branches are built there before 
submitting a pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/justinleet/metron gitignore

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/metron/pull/1228.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1228


commit 3eba1ecced27ab4b4c302e77ba2878a2bd9d8bdb
Author: justinjleet 
Date:   2018-10-05T18:06:48Z

Adding ignore pyc




---

[jira] [Commented] (METRON-1808) Add Ansible created pyc to gitignore

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16640169#comment-16640169
 ] 

ASF GitHub Bot commented on METRON-1808:


GitHub user justinleet opened a pull request:

https://github.com/apache/metron/pull/1228

METRON-1808: Add Ansible created pyc to gitignore

## Contributor Comments
Added *.pyc to gitignore for metron-development.  We don't need to commit 
pyc files.

Ansible creates a pyc file (0_minimum_ansible_version.pyc) that isn't 
filtered by gitignore. I added the general pyc ignore, because I don't we care 
about committing pyc files at all.

After running up full dev to create the file., this file should no longer 
be picked up by git.

## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [x] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
  ```

- [x] Have you verified the basic functionality of the build by building 
and running locally with Vagrant full-dev environment or the equivalent?

 Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up 
for your personal repository such that your branches are built there before 
submitting a pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/justinleet/metron gitignore

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/metron/pull/1228.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1228


commit 3eba1ecced27ab4b4c302e77ba2878a2bd9d8bdb
Author: justinjleet 
Date:   2018-10-05T18:06:48Z

Adding ignore pyc




> Add Ansible created pyc to gitignore
> 
>
> Key: METRON-1808
> URL: https://issues.apache.org/jira/browse/METRON-1808
> Project: Metron
>  Issue Type: Improvement
>Reporter: Justin Leet
>Assignee: Justin Leet
>Priority: Major
>
> Want to get rid of 
> {code}
> {14:06}~/Documents/workspace/metron:master ✗ ➭ git status
> On branch master
> Your branch is up to date with 'origin/master'.
> Untracked files:
>   (use "git add ..." to include in what will be committed)
>   metron-deployment/ansible/callback_plugins/0_minimum_ansible_version.pyc
> nothing added to commit but untracked files present (use "git add" to track)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (METRON-1808) Add Ansible created pyc to gitignore

[Justin Leet (JIRA)]

Justin Leet created METRON-1808:
---

 Summary: Add Ansible created pyc to gitignore
 Key: METRON-1808
 URL: https://issues.apache.org/jira/browse/METRON-1808
 Project: Metron
  Issue Type: Task
Reporter: Justin Leet
Assignee: Justin Leet


Want to get rid of 

{code}
{14:06}~/Documents/workspace/metron:master ✗ ➭ git status
On branch master
Your branch is up to date with 'origin/master'.

Untracked files:
  (use "git add ..." to include in what will be committed)

metron-deployment/ansible/callback_plugins/0_minimum_ansible_version.pyc

nothing added to commit but untracked files present (use "git add" to track)
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (METRON-1807) Auto populate the recommended values to some of the metron config parameters

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16640122#comment-16640122
 ] 

ASF GitHub Bot commented on METRON-1807:


GitHub user MohanDV opened a pull request:

https://github.com/apache/metron/pull/1227

METRON-1807: Auto populate the recommended values to some of the metron 
config parameters


## Contributor Comments
Auto populate the recommended values to below metron config parameters 

Storm Rest Server Address 
Zeppelin Server Address 
Solr Zookeeper Urls

### Testing Done : 
Spun-up a multinode cluster and verified that the above property values got 
auto populated with recommended values , here are the screenshots for the same 

https://user-images.githubusercontent.com/12934693/46550498-e8500800-c8f2-11e8-8a02-ac35903fbdbb.png;>
https://user-images.githubusercontent.com/12934693/46550514-f2720680-c8f2-11e8-8746-ed344221c4d7.png;>


## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [ ] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
  ```

- [ ] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] Have you verified the basic functionality of the build by building 
and running locally with Vagrant full-dev environment or the equivalent?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered by building and verifying the site-book? If not then run 
the following commands and the verify changes via 
`site-book/target/site/index.html`:

  ```
  cd site-book
  mvn site
  ```

 Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up 
for your personal repository such that your branches are built there before 
submitting a pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/MohanDV/metron METRON-1807

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/metron/pull/1227.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1227


commit 389552606146bedda5e670c6554919c3e75ce384
Author: Mohan Venkateshaiah 
Date:   2018-10-05T17:27:30Z

METRON-1807: Auto populate the recommended values to some of the metron 
config parameters




> Auto populate the recommended values to some of the metron config parameters 
> -
>
> Key: METRON-1807
> URL: https://issues.apache.org/jira/browse/METRON-1807
> Project: Metron
>  Issue Type: Bug
>Reporter: Mohan
>Assignee: Mohan
>Priority: Major
>
> Auto populate the recommended values to below metron config parameters 
> Storm Rest Server Address 
> Zeppelin Server Address 
> Solr Zookeeper Urls



--

[GitHub] metron pull request #1227: METRON-1807: Auto populate the recommended values...

[MohanDV]

GitHub user MohanDV opened a pull request:

https://github.com/apache/metron/pull/1227

METRON-1807: Auto populate the recommended values to some of the metron 
config parameters


## Contributor Comments
Auto populate the recommended values to below metron config parameters 

Storm Rest Server Address 
Zeppelin Server Address 
Solr Zookeeper Urls

### Testing Done : 
Spun-up a multinode cluster and verified that the above property values got 
auto populated with recommended values , here are the screenshots for the same 

https://user-images.githubusercontent.com/12934693/46550498-e8500800-c8f2-11e8-8a02-ac35903fbdbb.png;>
https://user-images.githubusercontent.com/12934693/46550514-f2720680-c8f2-11e8-8746-ed344221c4d7.png;>


## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [ ] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
  ```

- [ ] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] Have you verified the basic functionality of the build by building 
and running locally with Vagrant full-dev environment or the equivalent?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered by building and verifying the site-book? If not then run 
the following commands and the verify changes via 
`site-book/target/site/index.html`:

  ```
  cd site-book
  mvn site
  ```

 Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up 
for your personal repository such that your branches are built there before 
submitting a pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/MohanDV/metron METRON-1807

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/metron/pull/1227.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1227


commit 389552606146bedda5e670c6554919c3e75ce384
Author: Mohan Venkateshaiah 
Date:   2018-10-05T17:27:30Z

METRON-1807: Auto populate the recommended values to some of the metron 
config parameters




---

[jira] [Created] (METRON-1807) Auto populate the recommended values to some of the metron config parameters

[Mohan (JIRA)]

Mohan created METRON-1807:
-

 Summary: Auto populate the recommended values to some of the 
metron config parameters 
 Key: METRON-1807
 URL: https://issues.apache.org/jira/browse/METRON-1807
 Project: Metron
  Issue Type: Bug
Reporter: Mohan
Assignee: Mohan


Auto populate the recommended values to below metron config parameters 

Storm Rest Server Address 
Zeppelin Server Address 
Solr Zookeeper Urls



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (METRON-1805) Provide a default value for the Storm topology.max.spout.pending setting

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639994#comment-16639994
 ] 

ASF GitHub Bot commented on METRON-1805:


Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/1221
  
> @merrimanr: Which topologies should we provide defaults for?

To follow-up on the other topologies, we do not set a [default for the 
Profiler](https://github.com/apache/metron/blob/2ee6cc7e0b448d8d27f56f873e2c15a603c53917/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-profiler-env.xml#L168),
 which we probably should.

I am not sure about Parsers.


> Provide a default value for the Storm topology.max.spout.pending setting
> 
>
> Key: METRON-1805
> URL: https://issues.apache.org/jira/browse/METRON-1805
> Project: Metron
>  Issue Type: Improvement
>Reporter: Ryan Merriman
>Priority: Major
>
> Should we provide a default value for this property?  Which topologies should 
> that apply to?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron issue #1221: METRON-1805: Provide a default value for the Storm topol...

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/1221
  
> @merrimanr: Which topologies should we provide defaults for?

To follow-up on the other topologies, we do not set a [default for the 
Profiler](https://github.com/apache/metron/blob/2ee6cc7e0b448d8d27f56f873e2c15a603c53917/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-profiler-env.xml#L168),
 which we probably should.

I am not sure about Parsers.


---

[jira] [Commented] (METRON-1805) Provide a default value for the Storm topology.max.spout.pending setting

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639985#comment-16639985
 ] 

ASF GitHub Bot commented on METRON-1805:


Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/1221
  
For context, I set the default for Enrichments to 500.  There was some 
[discussion 
here](https://github.com/apache/metron/pull/984#discussion_r180150453).

I would echo my comments from that discussion here.  I think our defaults 
should focus on day 1 user experience.  Defining a default in the 300 - 700 
range tends to be reasonable on most platforms for day 1.  What we don't want 
is to start Metron up and the topology to immediately die with an OOM because 
there is no back pressure.


> Provide a default value for the Storm topology.max.spout.pending setting
> 
>
> Key: METRON-1805
> URL: https://issues.apache.org/jira/browse/METRON-1805
> Project: Metron
>  Issue Type: Improvement
>Reporter: Ryan Merriman
>Priority: Major
>
> Should we provide a default value for this property?  Which topologies should 
> that apply to?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron issue #1221: METRON-1805: Provide a default value for the Storm topol...

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/1221
  
For context, I set the default for Enrichments to 500.  There was some 
[discussion 
here](https://github.com/apache/metron/pull/984#discussion_r180150453).

I would echo my comments from that discussion here.  I think our defaults 
should focus on day 1 user experience.  Defining a default in the 300 - 700 
range tends to be reasonable on most platforms for day 1.  What we don't want 
is to start Metron up and the topology to immediately die with an OOM because 
there is no back pressure.


---

[jira] [Commented] (METRON-1796) [UI] Migrate off moment.js

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639950#comment-16639950
 ] 

ASF GitHub Bot commented on METRON-1796:


Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/1219#discussion_r223045213
  
--- Diff: metron-interface/metron-alerts/package.json ---
@@ -22,17 +22,17 @@
 "@angular/platform-browser": "^6.1.6",
 "@angular/platform-browser-dynamic": "^6.1.6",
 "@angular/router": "^6.1.6",
+"@ruffle1986/pikaday-time": "^1.6.1",
 "@types/bootstrap": "^4.1.1",
 "@types/jquery": "^3.3.4",
 "ace-builds": "^1.2.6",
 "ajv": "^6.5.1",
 "angular-confirmation-popover": "^4.2.0",
 "bootstrap": "4.0.0-alpha.6",
 "core-js": "^2.4.1",
+"date-fns": "^1.29.0",
 "font-awesome": "^4.7.0",
-"moment": "^2.22.2",
 "ng2-dragula": "^1.5.0",
-"pikaday-time": "^1.6.1",
--- End diff --

Is this a fair statement of the facts?  I want to make sure I understand.

* The [@owenmean/pikeday](https://github.com/owenmead/pikaday) dependency 
is not a new dependency for us
* [@owenmean/pikeday](https://github.com/owenmead/pikaday) is a fork of a 
popular library called [pikaday](https://github.com/dbushell/Pikaday) that has 
some enhancements we use
* [@owenmean/pikeday](https://github.com/owenmead/pikaday) is a deprecated 
library that is no longer maintained
* The only reason we had to touch the 
[@owenmean/pikeday](https://github.com/owenmead/pikaday) dependency is because 
it pulls in MomentJS
* There are only two ways to exclude MomentJS from being pulled in by 
[@owenmean/pikeday](https://github.com/owenmead/pikaday) 
   * Option 1: Find a long-term replacement for 
[@owenmean/pikeday](https://github.com/owenmead/pikaday) that has broader 
community support
   * Option 2: Directly forking it to 
[@ruffle1986/pikaday-time](https://github.com/ruffle1986/pikaday) and excluding 
MomentJS
* As part of this PR you chose to go with option 2, favoring expediency.
* You think we should do option 1 (find a long-term replacement for 
[@owenmean/pikeday](https://github.com/owenmead/pikaday)) as a separate, 
follow-on PR.



> [UI] Migrate off moment.js
> --
>
> Key: METRON-1796
> URL: https://issues.apache.org/jira/browse/METRON-1796
> Project: Metron
>  Issue Type: Improvement
>Reporter: Tamas Fodor
>Assignee: Tamas Fodor
>Priority: Minor
>
> Remove Moment.js and replace with another smaller library.
> Moment.js requires us to import the entire library vs. a few necessary 
> modules.
> Moment.js can prevent bundlers from supporting tree-shaking.
> By removing Moment.js, we can decrease our overall bundle size and prevent 
> issues with tree-shaking in the future.
> Here you can find the discussion on the mailing list:
> https://lists.apache.org/thread.html/2e4fafa4256ce14ebcd4433420974e24962884204418ade51f0e3bfb@%3Cdev.metron.apache.org%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron pull request #1219: METRON-1796: [UI] Migrate off moment.js

[nickwallen]

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/1219#discussion_r223045213
  
--- Diff: metron-interface/metron-alerts/package.json ---
@@ -22,17 +22,17 @@
 "@angular/platform-browser": "^6.1.6",
 "@angular/platform-browser-dynamic": "^6.1.6",
 "@angular/router": "^6.1.6",
+"@ruffle1986/pikaday-time": "^1.6.1",
 "@types/bootstrap": "^4.1.1",
 "@types/jquery": "^3.3.4",
 "ace-builds": "^1.2.6",
 "ajv": "^6.5.1",
 "angular-confirmation-popover": "^4.2.0",
 "bootstrap": "4.0.0-alpha.6",
 "core-js": "^2.4.1",
+"date-fns": "^1.29.0",
 "font-awesome": "^4.7.0",
-"moment": "^2.22.2",
 "ng2-dragula": "^1.5.0",
-"pikaday-time": "^1.6.1",
--- End diff --

Is this a fair statement of the facts?  I want to make sure I understand.

* The [@owenmean/pikeday](https://github.com/owenmead/pikaday) dependency 
is not a new dependency for us
* [@owenmean/pikeday](https://github.com/owenmead/pikaday) is a fork of a 
popular library called [pikaday](https://github.com/dbushell/Pikaday) that has 
some enhancements we use
* [@owenmean/pikeday](https://github.com/owenmead/pikaday) is a deprecated 
library that is no longer maintained
* The only reason we had to touch the 
[@owenmean/pikeday](https://github.com/owenmead/pikaday) dependency is because 
it pulls in MomentJS
* There are only two ways to exclude MomentJS from being pulled in by 
[@owenmean/pikeday](https://github.com/owenmead/pikaday) 
   * Option 1: Find a long-term replacement for 
[@owenmean/pikeday](https://github.com/owenmead/pikaday) that has broader 
community support
   * Option 2: Directly forking it to 
[@ruffle1986/pikaday-time](https://github.com/ruffle1986/pikaday) and excluding 
MomentJS
* As part of this PR you chose to go with option 2, favoring expediency.
* You think we should do option 1 (find a long-term replacement for 
[@owenmean/pikeday](https://github.com/owenmead/pikaday)) as a separate, 
follow-on PR.



---

[jira] [Commented] (METRON-1805) Provide a default value for the Storm topology.max.spout.pending setting

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639902#comment-16639902
 ] 

ASF GitHub Bot commented on METRON-1805:


Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/1221
  
As far as I know there was not an issue with other topologies.  Writing 
data tends to cause back pressure so that's why these changes were suggested 
for indexing topologies.


> Provide a default value for the Storm topology.max.spout.pending setting
> 
>
> Key: METRON-1805
> URL: https://issues.apache.org/jira/browse/METRON-1805
> Project: Metron
>  Issue Type: Improvement
>Reporter: Ryan Merriman
>Priority: Major
>
> Should we provide a default value for this property?  Which topologies should 
> that apply to?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron issue #1221: METRON-1805: Provide a default value for the Storm topol...

[merrimanr]

Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/1221
  
As far as I know there was not an issue with other topologies.  Writing 
data tends to cause back pressure so that's why these changes were suggested 
for indexing topologies.


---

[jira] [Commented] (METRON-1805) Provide a default value for the Storm topology.max.spout.pending setting

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639898#comment-16639898
 ] 

ASF GitHub Bot commented on METRON-1805:


Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/1221
  
I think this is fine. This is just the backpressure setting and having it 
set to the default means that the spigot is fully open and will potentially 
flood the topology with a high velocity of data. We recommend tuning all 
topologies, so starting from a default that doesn't crush the system seems 
reasonable to me. Since this change is only being made to indexing, can you 
clarify if there is/was any observable issues in the other topologies?


> Provide a default value for the Storm topology.max.spout.pending setting
> 
>
> Key: METRON-1805
> URL: https://issues.apache.org/jira/browse/METRON-1805
> Project: Metron
>  Issue Type: Improvement
>Reporter: Ryan Merriman
>Priority: Major
>
> Should we provide a default value for this property?  Which topologies should 
> that apply to?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron issue #1221: METRON-1805: Provide a default value for the Storm topol...

[mmiklavc]

Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/1221
  
I think this is fine. This is just the backpressure setting and having it 
set to the default means that the spigot is fully open and will potentially 
flood the topology with a high velocity of data. We recommend tuning all 
topologies, so starting from a default that doesn't crush the system seems 
reasonable to me. Since this change is only being made to indexing, can you 
clarify if there is/was any observable issues in the other topologies?


---

[jira] [Commented] (METRON-1796) [UI] Migrate off moment.js

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639896#comment-16639896
 ] 

ASF GitHub Bot commented on METRON-1796:


Github user ruffle1986 commented on a diff in the pull request:

https://github.com/apache/metron/pull/1219#discussion_r223030359
  
--- Diff: metron-interface/metron-alerts/package.json ---
@@ -22,17 +22,17 @@
 "@angular/platform-browser": "^6.1.6",
 "@angular/platform-browser-dynamic": "^6.1.6",
 "@angular/router": "^6.1.6",
+"@ruffle1986/pikaday-time": "^1.6.1",
 "@types/bootstrap": "^4.1.1",
 "@types/jquery": "^3.3.4",
 "ace-builds": "^1.2.6",
 "ajv": "^6.5.1",
 "angular-confirmation-popover": "^4.2.0",
 "bootstrap": "4.0.0-alpha.6",
 "core-js": "^2.4.1",
+"date-fns": "^1.29.0",
 "font-awesome": "^4.7.0",
-"moment": "^2.22.2",
 "ng2-dragula": "^1.5.0",
-"pikaday-time": "^1.6.1",
--- End diff --

> Why is pikaday-time suddenly an issue?

I agree with you to not use smaller individual projects like 
`pikaday-time`. `pikaday-time` is introduced in the project by @iraghumitra 
almost a year ago.
The goal wasn't replacing `pikaday-time` with a better well-supported 
library but eliminating `moment.js` because of the aforementioned reasons in 
the description.

> I am concerned about this pikaday dependency. I would rather see us 
depending on larger, community supported projects like https://momentjs.com/, 
rather than smaller, individual supported projects like @owenmean/pikaday (or 
even your own fork @ruffle1986/pikaday-time).

`moment.js` is a very popular and great library to manipulate date and 
time. 

`date-fns` has the exact same purpose.

`pikaday` is a standalone calendar user interface component.

`pikaday-time` is an extension of `pikaday` with additional fields to 
select time, not just date.

`moment.js` is an optional dependency of `pikaday` (! not `pikaday-time`). 
it works perfectly fine without `moment.js`. But if it is convenient to use 
`pikaday` with `moment`, it's fine then, you can use it by calling `pikaday`'s 
`getMoment` method if needed.
`pikaday-time` is just a fork of `pikaday` and the author of `pikaday-time` 
made a mistake by setting `moment.js` as a dependency of `pikaday-time` and 
every time we install `pikaday-time` we install `moment.js` too. If `moment.js` 
is installed in the `node_modules` folder `pikaday` will use it therefore it 
will be part of the bundle.


> [UI] Migrate off moment.js
> --
>
> Key: METRON-1796
> URL: https://issues.apache.org/jira/browse/METRON-1796
> Project: Metron
>  Issue Type: Improvement
>Reporter: Tamas Fodor
>Assignee: Tamas Fodor
>Priority: Minor
>
> Remove Moment.js and replace with another smaller library.
> Moment.js requires us to import the entire library vs. a few necessary 
> modules.
> Moment.js can prevent bundlers from supporting tree-shaking.
> By removing Moment.js, we can decrease our overall bundle size and prevent 
> issues with tree-shaking in the future.
> Here you can find the discussion on the mailing list:
> https://lists.apache.org/thread.html/2e4fafa4256ce14ebcd4433420974e24962884204418ade51f0e3bfb@%3Cdev.metron.apache.org%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron pull request #1219: METRON-1796: [UI] Migrate off moment.js

[ruffle1986]

Github user ruffle1986 commented on a diff in the pull request:

https://github.com/apache/metron/pull/1219#discussion_r223030359
  
--- Diff: metron-interface/metron-alerts/package.json ---
@@ -22,17 +22,17 @@
 "@angular/platform-browser": "^6.1.6",
 "@angular/platform-browser-dynamic": "^6.1.6",
 "@angular/router": "^6.1.6",
+"@ruffle1986/pikaday-time": "^1.6.1",
 "@types/bootstrap": "^4.1.1",
 "@types/jquery": "^3.3.4",
 "ace-builds": "^1.2.6",
 "ajv": "^6.5.1",
 "angular-confirmation-popover": "^4.2.0",
 "bootstrap": "4.0.0-alpha.6",
 "core-js": "^2.4.1",
+"date-fns": "^1.29.0",
 "font-awesome": "^4.7.0",
-"moment": "^2.22.2",
 "ng2-dragula": "^1.5.0",
-"pikaday-time": "^1.6.1",
--- End diff --

> Why is pikaday-time suddenly an issue?

I agree with you to not use smaller individual projects like 
`pikaday-time`. `pikaday-time` is introduced in the project by @iraghumitra 
almost a year ago.
The goal wasn't replacing `pikaday-time` with a better well-supported 
library but eliminating `moment.js` because of the aforementioned reasons in 
the description.

> I am concerned about this pikaday dependency. I would rather see us 
depending on larger, community supported projects like https://momentjs.com/, 
rather than smaller, individual supported projects like @owenmean/pikaday (or 
even your own fork @ruffle1986/pikaday-time).

`moment.js` is a very popular and great library to manipulate date and 
time. 

`date-fns` has the exact same purpose.

`pikaday` is a standalone calendar user interface component.

`pikaday-time` is an extension of `pikaday` with additional fields to 
select time, not just date.

`moment.js` is an optional dependency of `pikaday` (! not `pikaday-time`). 
it works perfectly fine without `moment.js`. But if it is convenient to use 
`pikaday` with `moment`, it's fine then, you can use it by calling `pikaday`'s 
`getMoment` method if needed.
`pikaday-time` is just a fork of `pikaday` and the author of `pikaday-time` 
made a mistake by setting `moment.js` as a dependency of `pikaday-time` and 
every time we install `pikaday-time` we install `moment.js` too. If `moment.js` 
is installed in the `node_modules` folder `pikaday` will use it therefore it 
will be part of the bundle.


---

[jira] [Commented] (METRON-1796) [UI] Migrate off moment.js

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639842#comment-16639842
 ] 

ASF GitHub Bot commented on METRON-1796:


Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/1219#discussion_r223008966
  
--- Diff: metron-interface/metron-alerts/package.json ---
@@ -22,17 +22,17 @@
 "@angular/platform-browser": "^6.1.6",
 "@angular/platform-browser-dynamic": "^6.1.6",
 "@angular/router": "^6.1.6",
+"@ruffle1986/pikaday-time": "^1.6.1",
 "@types/bootstrap": "^4.1.1",
 "@types/jquery": "^3.3.4",
 "ace-builds": "^1.2.6",
 "ajv": "^6.5.1",
 "angular-confirmation-popover": "^4.2.0",
 "bootstrap": "4.0.0-alpha.6",
 "core-js": "^2.4.1",
+"date-fns": "^1.29.0",
 "font-awesome": "^4.7.0",
-"moment": "^2.22.2",
 "ng2-dragula": "^1.5.0",
-"pikaday-time": "^1.6.1",
--- End diff --

The core of this change is to move from moment.js to date-fns.  Why is 
pikaday-time suddenly an issue?

I am concerned about this pikaday dependency.  I would rather see us 
depending on larger, community supported projects like https://momentjs.com/, 
rather than smaller, individual supported projects like @owenmean/pikaday (or 
even your own fork @ruffle1986/pikaday-time).

Not only for continued support from obsolescence, but also because security 
vulnerabilities are all too common and our UI is a large attack surface.  
Larger communities means vulnerabilities are more likely to be uncovered and 
patched. 

I get the technical motivation here.  We want to decrease the load time.  
At the same time, we need to consider the organizations behind our dependencies 
to ensure their long-term viability and support.  

 Is there not another way we can tackle the technical challenge here?



> [UI] Migrate off moment.js
> --
>
> Key: METRON-1796
> URL: https://issues.apache.org/jira/browse/METRON-1796
> Project: Metron
>  Issue Type: Improvement
>Reporter: Tamas Fodor
>Assignee: Tamas Fodor
>Priority: Minor
>
> Remove Moment.js and replace with another smaller library.
> Moment.js requires us to import the entire library vs. a few necessary 
> modules.
> Moment.js can prevent bundlers from supporting tree-shaking.
> By removing Moment.js, we can decrease our overall bundle size and prevent 
> issues with tree-shaking in the future.
> Here you can find the discussion on the mailing list:
> https://lists.apache.org/thread.html/2e4fafa4256ce14ebcd4433420974e24962884204418ade51f0e3bfb@%3Cdev.metron.apache.org%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron issue #1219: METRON-1796: [UI] Migrate off moment.js

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/1219
  
@ruffle1986 I just want to thank you for all the detail you put into your 
PR description and commentary to help ensure the community understands your 
work.  Well done and it should serve as an example for all of us.


---

[jira] [Commented] (METRON-1796) [UI] Migrate off moment.js

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639843#comment-16639843
 ] 

ASF GitHub Bot commented on METRON-1796:


Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/1219
  
@ruffle1986 I just want to thank you for all the detail you put into your 
PR description and commentary to help ensure the community understands your 
work.  Well done and it should serve as an example for all of us.


> [UI] Migrate off moment.js
> --
>
> Key: METRON-1796
> URL: https://issues.apache.org/jira/browse/METRON-1796
> Project: Metron
>  Issue Type: Improvement
>Reporter: Tamas Fodor
>Assignee: Tamas Fodor
>Priority: Minor
>
> Remove Moment.js and replace with another smaller library.
> Moment.js requires us to import the entire library vs. a few necessary 
> modules.
> Moment.js can prevent bundlers from supporting tree-shaking.
> By removing Moment.js, we can decrease our overall bundle size and prevent 
> issues with tree-shaking in the future.
> Here you can find the discussion on the mailing list:
> https://lists.apache.org/thread.html/2e4fafa4256ce14ebcd4433420974e24962884204418ade51f0e3bfb@%3Cdev.metron.apache.org%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron pull request #1219: METRON-1796: [UI] Migrate off moment.js

[nickwallen]

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/1219#discussion_r223008966
  
--- Diff: metron-interface/metron-alerts/package.json ---
@@ -22,17 +22,17 @@
 "@angular/platform-browser": "^6.1.6",
 "@angular/platform-browser-dynamic": "^6.1.6",
 "@angular/router": "^6.1.6",
+"@ruffle1986/pikaday-time": "^1.6.1",
 "@types/bootstrap": "^4.1.1",
 "@types/jquery": "^3.3.4",
 "ace-builds": "^1.2.6",
 "ajv": "^6.5.1",
 "angular-confirmation-popover": "^4.2.0",
 "bootstrap": "4.0.0-alpha.6",
 "core-js": "^2.4.1",
+"date-fns": "^1.29.0",
 "font-awesome": "^4.7.0",
-"moment": "^2.22.2",
 "ng2-dragula": "^1.5.0",
-"pikaday-time": "^1.6.1",
--- End diff --

The core of this change is to move from moment.js to date-fns.  Why is 
pikaday-time suddenly an issue?

I am concerned about this pikaday dependency.  I would rather see us 
depending on larger, community supported projects like https://momentjs.com/, 
rather than smaller, individual supported projects like @owenmean/pikaday (or 
even your own fork @ruffle1986/pikaday-time).

Not only for continued support from obsolescence, but also because security 
vulnerabilities are all too common and our UI is a large attack surface.  
Larger communities means vulnerabilities are more likely to be uncovered and 
patched. 

I get the technical motivation here.  We want to decrease the load time.  
At the same time, we need to consider the organizations behind our dependencies 
to ensure their long-term viability and support.  

 Is there not another way we can tackle the technical challenge here?



---

[jira] [Commented] (METRON-1749) Update Angular to latest release in Management UI

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639834#comment-16639834
 ] 

ASF GitHub Bot commented on METRON-1749:


Github user tiborm commented on the issue:

https://github.com/apache/metron/pull/1217
  
Great improvement! Thanks, Shane! +1


> Update Angular to latest release in Management UI
> -
>
> Key: METRON-1749
> URL: https://issues.apache.org/jira/browse/METRON-1749
> Project: Metron
>  Issue Type: Improvement
>Reporter: Shane Ardell
>Assignee: Shane Ardell
>Priority: Major
>
> Currently, the Management UI is on Angular v2. Not only has there been many 
> updates and improvements to Angular, that release is no longer supported by 
> the Angular team. This means critical fixes and security patches are no 
> longer being done.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron issue #1217: METRON-1749: Update Angular to latest release in Managem...

[tiborm]

Github user tiborm commented on the issue:

https://github.com/apache/metron/pull/1217
  
Great improvement! Thanks, Shane! +1


---

[jira] [Commented] (METRON-1796) [UI] Migrate off moment.js

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639830#comment-16639830
 ] 

ASF GitHub Bot commented on METRON-1796:


Github user tiborm commented on the issue:

https://github.com/apache/metron/pull/1219
  
Thanks, Tamas! Reviewed and tested. +1


> [UI] Migrate off moment.js
> --
>
> Key: METRON-1796
> URL: https://issues.apache.org/jira/browse/METRON-1796
> Project: Metron
>  Issue Type: Improvement
>Reporter: Tamas Fodor
>Assignee: Tamas Fodor
>Priority: Minor
>
> Remove Moment.js and replace with another smaller library.
> Moment.js requires us to import the entire library vs. a few necessary 
> modules.
> Moment.js can prevent bundlers from supporting tree-shaking.
> By removing Moment.js, we can decrease our overall bundle size and prevent 
> issues with tree-shaking in the future.
> Here you can find the discussion on the mailing list:
> https://lists.apache.org/thread.html/2e4fafa4256ce14ebcd4433420974e24962884204418ade51f0e3bfb@%3Cdev.metron.apache.org%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron issue #1219: METRON-1796: [UI] Migrate off moment.js

[tiborm]

Github user tiborm commented on the issue:

https://github.com/apache/metron/pull/1219
  
Thanks, Tamas! Reviewed and tested. +1


---

[jira] [Commented] (METRON-1803) Integrate Cypress with Travis

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639690#comment-16639690
 ] 

ASF GitHub Bot commented on METRON-1803:


Github user tiborm closed the pull request at:

https://github.com/apache/metron/pull/1225


> Integrate Cypress with Travis
> -
>
> Key: METRON-1803
> URL: https://issues.apache.org/jira/browse/METRON-1803
> Project: Metron
>  Issue Type: Improvement
>Reporter: Tibor Meller
>Assignee: Tibor Meller
>Priority: Major
>
> [DISCUSS] thread on the dev mailing list:
> [https://lists.apache.org/thread.html/b6a0272c7809c05e8b7aff20171720e8ec76f8a0e9481169c37a4a4a@%3Cdev.metron.apache.org%3E]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (METRON-1803) Integrate Cypress with Travis

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639691#comment-16639691
 ] 

ASF GitHub Bot commented on METRON-1803:


GitHub user tiborm opened a pull request:

https://github.com/apache/metron/pull/1226

METRON-1803: Integrate Cypress with Travis

## Contributor Comments

[DISCUSS] thread on the dev mailing list: 

https://lists.apache.org/thread.html/b6a0272c7809c05e8b7aff20171720e8ec76f8a0e9481169c37a4a4a@%3Cdev.metron.apache.org%3E

JIRA Ticket: https://issues.apache.org/jira/browse/METRON-1803

This PR adds Cypress.io to our project. It makes us able to run E2E tests, 
UI integration tests. Also higher level UI tests with mocked services. 
As part of this PR I added tests to cover the following functionalities of 
PCAP UI panel:

- checking running jobs on navigating to PCAP tab
- submitting PCAP job request
- requesting job status
- process status in percentage
- getting PCAP JSON
- rendering PCAP table
- showing PCAP details
- navigating across pages
- downloading PDML
- cancelli PCAP query job
- showing filter validation messages
- showing date validation messages

Please consider while reviewing that these tests are not meant to cover all 
the details of the functionalities above. The aim here is to extend the test 
coverage build up by a large number of Unit Tests with higher level UI tests.

### New folders
Cypress files sit in cypress folder. Some of them might look like an empty 
placeholder but cypress actively uses all files in this PR.

 Integration folder
That contains the actual Cypress tests. The folder name comes from the 
Cypress naming convention, however, no real integration tests implemented here 
yet (at least no other than the integration of UI components to a whole 
application). We could change the name of the folder anytime we like to, but 
only one folder applicable.

 Fixture folder
The tests are running on mocked data. This makes us able to integrate them 
with Travis without the need to having PCAP related services available from our 
CI.
Each file in the Fixture folder represents a response to a particular 
request. This makes it fairly easy to create and maintain.

### How to run tests on your local machine
To run Cypress tests locally please do the following:
1. pull this branch to your workspace
2. move to metron-interface/metron-alerts folder
3. run: npm ci
4. run: scripts/start-server-for-e2e.sh
5. run: node_modules/cypress/bin/cypress run
or run: node_modules/cypress/bin/cypress open
if you like to use the dashboard.

### Travis integration
As part of this PR I modified the package.json in metron-alert to make 
Cypress running in Travis. If you like to see the integration working please 
check to logs of build step #3.
Failing Cypress tests brakes the build.

Example of build broken by failing Cypress test: 
https://travis-ci.com/tiborm/metron/builds/86869264


## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [x] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
  ```

- [x] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [x] If adding new dependencies to the code, are these 

[jira] [Updated] (METRON-1803) Integrate Cypress with Travis

[Tibor Meller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/METRON-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tibor Meller updated METRON-1803:
-
Description: 
[DISCUSS] thread on the dev mailing list:
[https://lists.apache.org/thread.html/b6a0272c7809c05e8b7aff20171720e8ec76f8a0e9481169c37a4a4a@%3Cdev.metron.apache.org%3E]

JIRA Ticket: https://issues.apache.org/jira/browse/METRON-1803

This PR adds Cypress.io to our project. It makes us able to run E2E tests, UI 
integration tests. Also higher level UI tests with mocked services.
As part of this PR I added tests to cover the following functionalities of PCAP 
UI panel:
 * checking running jobs on navigating to PCAP tab
 * submitting PCAP job request
 * requesting job status
 * process status in percentage
 * getting PCAP JSON
 * rendering PCAP table
 * showing PCAP details
 * navigating across pages
 * downloading PDML
 * cancelli PCAP query job
 * showing filter validation messages
 * showing date validation messages

Please consider while reviewing that these tests are not meant to cover all the 
details of the functionalities above. The aim here is to extend the test 
coverage build up by a large number of Unit Tests with higher level UI tests.
h3. New folders

Cypress files sit in cypress folder. Some of them might look like an empty 
placeholder but cypress actively uses all files in this PR.
h4. Integration folder

That contains the actual Cypress tests. The folder name comes from the Cypress 
naming convention, however, no real integration tests implemented here yet (at 
least no other than the integration of UI components to a whole application). 
We could change the name of the folder anytime we like to, but only one folder 
applicable.
h4. Fixture folder

The tests are running on mocked data. This makes us able to integrate them with 
Travis without the need to having PCAP related services available from our CI.
Each file in the Fixture folder represents a response to a particular request. 
This makes it fairly easy to create and maintain.
h3. How to run tests on your local machine

To run Cypress tests locally please do the following:
 # pull this branch to your workspace
 # move to metron-interface/metron-alerts folder
 # run: npm ci
 # run: scripts/start-server-for-e2e.sh
 # run: node_modules/cypress/bin/cypress run
or run: node_modules/cypress/bin/cypress open
if you like to use the dashboard.

h3. Travis integration

As part of this PR I modified the package.json in metron-alert to make Cypress 
running in Travis. If you like to see the integration working please check to 
logs of build step [#3|https://github.com/apache/metron/pull/3].
Failing Cypress tests brakes the build.

Example of build broken by failing Cypress test:
[https://travis-ci.com/tiborm/metron/builds/86869264]

  was:
[DISCUSS] thread on the dev mailing list:
[https://lists.apache.org/thread.html/b6a0272c7809c05e8b7aff20171720e8ec76f8a0e9481169c37a4a4a@%3Cdev.metron.apache.org%3E]


> Integrate Cypress with Travis
> -
>
> Key: METRON-1803
> URL: https://issues.apache.org/jira/browse/METRON-1803
> Project: Metron
>  Issue Type: Improvement
>Reporter: Tibor Meller
>Assignee: Tibor Meller
>Priority: Major
>
> [DISCUSS] thread on the dev mailing list:
> [https://lists.apache.org/thread.html/b6a0272c7809c05e8b7aff20171720e8ec76f8a0e9481169c37a4a4a@%3Cdev.metron.apache.org%3E]
> JIRA Ticket: https://issues.apache.org/jira/browse/METRON-1803
> This PR adds Cypress.io to our project. It makes us able to run E2E tests, UI 
> integration tests. Also higher level UI tests with mocked services.
> As part of this PR I added tests to cover the following functionalities of 
> PCAP UI panel:
>  * checking running jobs on navigating to PCAP tab
>  * submitting PCAP job request
>  * requesting job status
>  * process status in percentage
>  * getting PCAP JSON
>  * rendering PCAP table
>  * showing PCAP details
>  * navigating across pages
>  * downloading PDML
>  * cancelli PCAP query job
>  * showing filter validation messages
>  * showing date validation messages
> Please consider while reviewing that these tests are not meant to cover all 
> the details of the functionalities above. The aim here is to extend the test 
> coverage build up by a large number of Unit Tests with higher level UI tests.
> h3. New folders
> Cypress files sit in cypress folder. Some of them might look like an empty 
> placeholder but cypress actively uses all files in this PR.
> h4. Integration folder
> That contains the actual Cypress tests. The folder name comes from the 
> Cypress naming convention, however, no real integration tests implemented 
> here yet (at least no other than the integration of UI components to a whole 
> application). We could change the name of the folder anytime we like to, but 
> only one folder applicable.
> h4. 

[GitHub] metron pull request #1226: METRON-1803: Integrate Cypress with Travis

[tiborm]

GitHub user tiborm opened a pull request:

https://github.com/apache/metron/pull/1226

METRON-1803: Integrate Cypress with Travis

## Contributor Comments

[DISCUSS] thread on the dev mailing list: 

https://lists.apache.org/thread.html/b6a0272c7809c05e8b7aff20171720e8ec76f8a0e9481169c37a4a4a@%3Cdev.metron.apache.org%3E

JIRA Ticket: https://issues.apache.org/jira/browse/METRON-1803

This PR adds Cypress.io to our project. It makes us able to run E2E tests, 
UI integration tests. Also higher level UI tests with mocked services. 
As part of this PR I added tests to cover the following functionalities of 
PCAP UI panel:

- checking running jobs on navigating to PCAP tab
- submitting PCAP job request
- requesting job status
- process status in percentage
- getting PCAP JSON
- rendering PCAP table
- showing PCAP details
- navigating across pages
- downloading PDML
- cancelli PCAP query job
- showing filter validation messages
- showing date validation messages

Please consider while reviewing that these tests are not meant to cover all 
the details of the functionalities above. The aim here is to extend the test 
coverage build up by a large number of Unit Tests with higher level UI tests.

### New folders
Cypress files sit in cypress folder. Some of them might look like an empty 
placeholder but cypress actively uses all files in this PR.

 Integration folder
That contains the actual Cypress tests. The folder name comes from the 
Cypress naming convention, however, no real integration tests implemented here 
yet (at least no other than the integration of UI components to a whole 
application). We could change the name of the folder anytime we like to, but 
only one folder applicable.

 Fixture folder
The tests are running on mocked data. This makes us able to integrate them 
with Travis without the need to having PCAP related services available from our 
CI.
Each file in the Fixture folder represents a response to a particular 
request. This makes it fairly easy to create and maintain.

### How to run tests on your local machine
To run Cypress tests locally please do the following:
1. pull this branch to your workspace
2. move to metron-interface/metron-alerts folder
3. run: npm ci
4. run: scripts/start-server-for-e2e.sh
5. run: node_modules/cypress/bin/cypress run
or run: node_modules/cypress/bin/cypress open
if you like to use the dashboard.

### Travis integration
As part of this PR I modified the package.json in metron-alert to make 
Cypress running in Travis. If you like to see the integration working please 
check to logs of build step #3.
Failing Cypress tests brakes the build.

Example of build broken by failing Cypress test: 
https://travis-ci.com/tiborm/metron/builds/86869264


## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [x] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
  ```

- [x] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [x] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] Have you verified the basic functionality of the build by building 
and running locally with Vagrant 

[GitHub] metron pull request #1225: METRON-1803: Integrate Cypress with Travis

[tiborm]

Github user tiborm closed the pull request at:

https://github.com/apache/metron/pull/1225


---

[GitHub] metron pull request #1217: METRON-1749: Update Angular to latest release in ...

[sardell]

Github user sardell commented on a diff in the pull request:

https://github.com/apache/metron/pull/1217#discussion_r222949278
  
--- Diff: metron-interface/metron-config/package.json ---
@@ -7,10 +7,11 @@
   },
   "angular-cli": {},
   "scripts": {
-"build": "./node_modules/angular-cli/bin/ng build -prod",
+"build": "./node_modules/@angular/cli/bin/ng build --prod",
--- End diff --

Thanks for pointing this out! I think the original path may have been 
leftover from the original project generation with the old version of angular 
cli. I have since removed it.


---

[jira] [Commented] (METRON-1749) Update Angular to latest release in Management UI

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639562#comment-16639562
 ] 

ASF GitHub Bot commented on METRON-1749:


Github user sardell commented on a diff in the pull request:

https://github.com/apache/metron/pull/1217#discussion_r222949278
  
--- Diff: metron-interface/metron-config/package.json ---
@@ -7,10 +7,11 @@
   },
   "angular-cli": {},
   "scripts": {
-"build": "./node_modules/angular-cli/bin/ng build -prod",
+"build": "./node_modules/@angular/cli/bin/ng build --prod",
--- End diff --

Thanks for pointing this out! I think the original path may have been 
leftover from the original project generation with the old version of angular 
cli. I have since removed it.


> Update Angular to latest release in Management UI
> -
>
> Key: METRON-1749
> URL: https://issues.apache.org/jira/browse/METRON-1749
> Project: Metron
>  Issue Type: Improvement
>Reporter: Shane Ardell
>Assignee: Shane Ardell
>Priority: Major
>
> Currently, the Management UI is on Angular v2. Not only has there been many 
> updates and improvements to Angular, that release is no longer supported by 
> the Angular team. This means critical fixes and security patches are no 
> longer being done.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (METRON-1803) Integrate Cypress with Travis

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639544#comment-16639544
 ] 

ASF GitHub Bot commented on METRON-1803:


GitHub user tiborm opened a pull request:

https://github.com/apache/metron/pull/1225

METRON-1803: Integrate Cypress with Travis

## Contributor Comments

[DISCUSS] thread on the dev mailing list: 

https://lists.apache.org/thread.html/b6a0272c7809c05e8b7aff20171720e8ec76f8a0e9481169c37a4a4a@%3Cdev.metron.apache.org%3E

JIRA Ticket: https://issues.apache.org/jira/browse/METRON-1803

This PR adds Cypress.io to our project. It makes us able to run E2E tests, 
UI integration tests. Also higher level UI tests with mocked services. 
As part of this PR I also added tests to cover the following 
functionalities of PCAP UI panel:

- checking running jobs on navigating to PCAP tab
- submitting PCAP job request
- requesting job status
- process status in percentage
- getting PCAP JSON
- rendering PCAP table
- showing PCAP details
- navigating across pages
- downloading PDML
- cancelli PCAP query job
- showing filter validation messages
- showing date validation messages

Please consider while reviewing that these tests are not meant to cover all 
the details of the functionalities above. The aim here is to extend the test 
coverage build up by a large number of Unit Tests with higher level UI tests.

### New folders
 Integration folder
That contains the actual Cypress tests. The folder name comes from the 
Cypress naming convention, however, no real integration tests implemented here 
yet (at least no other than the integration of UI components to a whole 
application). We could change the name of the folder anytime we like to, but 
only one folder applicable.

 Fixture folder
The tests are running on mocked data. This makes us able to integrate them 
with Travis without the need to having PCAP related services available from our 
CI.
Each file in the Fixture folder represents a response to a particular 
request. This makes it fairly easy to create and maintain.

### How to run tests on your local machine
To run Cypress tests locally please do the following:
1. pull this branch to your workspace
2. move to metron-interface/metron-alerts folder
3. run: npm ci
4. run: scripts/start-server-for-e2e.sh
5. run: node_modules/cypress/bin/cypress run
or run: node_modules/cypress/bin/cypress open
if you like to use the dashboard.

### Travis integration
As part of this PR modified the package.json script block in metron-alert 
to make Cypress tests running in Travis. If you like to see the integration 
working please check to logs of build step #3 (mvn test --projects 
metron-interface/metron-config,metron-interface/metron-alerts)
Failing Cypress tests brakes the build.

Example of build broken by failing Cypress test: 
https://travis-ci.com/tiborm/metron/builds/86869264


## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [ ] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [ ] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
  ```

- [ ] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible 

[GitHub] metron pull request #1225: METRON-1803: Integrate Cypress with Travis

[tiborm]

GitHub user tiborm opened a pull request:

https://github.com/apache/metron/pull/1225

METRON-1803: Integrate Cypress with Travis

## Contributor Comments

[DISCUSS] thread on the dev mailing list: 

https://lists.apache.org/thread.html/b6a0272c7809c05e8b7aff20171720e8ec76f8a0e9481169c37a4a4a@%3Cdev.metron.apache.org%3E

JIRA Ticket: https://issues.apache.org/jira/browse/METRON-1803

This PR adds Cypress.io to our project. It makes us able to run E2E tests, 
UI integration tests. Also higher level UI tests with mocked services. 
As part of this PR I also added tests to cover the following 
functionalities of PCAP UI panel:

- checking running jobs on navigating to PCAP tab
- submitting PCAP job request
- requesting job status
- process status in percentage
- getting PCAP JSON
- rendering PCAP table
- showing PCAP details
- navigating across pages
- downloading PDML
- cancelli PCAP query job
- showing filter validation messages
- showing date validation messages

Please consider while reviewing that these tests are not meant to cover all 
the details of the functionalities above. The aim here is to extend the test 
coverage build up by a large number of Unit Tests with higher level UI tests.

### New folders
 Integration folder
That contains the actual Cypress tests. The folder name comes from the 
Cypress naming convention, however, no real integration tests implemented here 
yet (at least no other than the integration of UI components to a whole 
application). We could change the name of the folder anytime we like to, but 
only one folder applicable.

 Fixture folder
The tests are running on mocked data. This makes us able to integrate them 
with Travis without the need to having PCAP related services available from our 
CI.
Each file in the Fixture folder represents a response to a particular 
request. This makes it fairly easy to create and maintain.

### How to run tests on your local machine
To run Cypress tests locally please do the following:
1. pull this branch to your workspace
2. move to metron-interface/metron-alerts folder
3. run: npm ci
4. run: scripts/start-server-for-e2e.sh
5. run: node_modules/cypress/bin/cypress run
or run: node_modules/cypress/bin/cypress open
if you like to use the dashboard.

### Travis integration
As part of this PR modified the package.json script block in metron-alert 
to make Cypress tests running in Travis. If you like to see the integration 
working please check to logs of build step #3 (mvn test --projects 
metron-interface/metron-config,metron-interface/metron-alerts)
Failing Cypress tests brakes the build.

Example of build broken by failing Cypress test: 
https://travis-ci.com/tiborm/metron/builds/86869264


## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [ ] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [ ] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
  ```

- [ ] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] Have you verified the basic functionality of the build by building 
and running locally with Vagrant full-dev environment or the equivalent?

 

[jira] [Commented] (METRON-1797) Add PCAP UI test implemented in Cypress.js

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639535#comment-16639535
 ] 

ASF GitHub Bot commented on METRON-1797:


Github user tiborm closed the pull request at:

https://github.com/apache/metron/pull/1223


> Add PCAP UI test implemented in Cypress.js
> --
>
> Key: METRON-1797
> URL: https://issues.apache.org/jira/browse/METRON-1797
> Project: Metron
>  Issue Type: Improvement
>Reporter: Tibor Meller
>Assignee: Tibor Meller
>Priority: Major
>
> [DISCUSS] thread on the dev mailing list:
> [https://lists.apache.org/thread.html/b6a0272c7809c05e8b7aff20171720e8ec76f8a0e9481169c37a4a4a@%3Cdev.metron.apache.org%3E]
> This PR adds Cypress.io to our project. It makes us able to run E2E tests, UI 
> integration tests. Also higher level UI tests with mocked services.
> As part of this PR I also added tests to cover the following functionalities 
> of PCAP UI panel:
>  * checking running jobs on navigating to PCAP tab
>  * submitting PCAP job request
>  * requesting job status
>  * process status in percentage
>  * getting PCAP JSON
>  * rendering PCAP table
>  * showing PCAP details
>  * navigating across pages
>  * downloading PDML
>  * cancelli PCAP query job
>  * showing filter validation messages
>  * showing date validation messages
> Please consider while reviewing that these tests are not meant to cover all 
> the details of the functionalities above. The aim here is to extend the test 
> coverage build up by a large number of Unit Tests with higher level UI tests.
> h2. Integration folder
> That contains the actual Cypress tests. The folder name comes from the 
> Cypress naming convention, however, no real integration tests implemented 
> here yet (at least no other than the integration of UI components to a whole 
> application). We could change the name of the folder anytime we like to, but 
> only one folder applicable.
> h2. Fixture folder
> The tests are running on mocked data. This makes us able to integrate them 
> with Travis without the need to having PCAP related services available from 
> our CI.
> Each file in the Fixture folder represents a response to a particular 
> request. This makes it fairly easy to create and maintain.
> h1. How to run
> To run Cypress tests locally please do the following:
>  # pull this branch to your workspace
>  # move to metron-interface/metron-alerts folder
>  # run: npm ci
>  # run: scripts/start-server-for-e2e.sh
>  # run: node_modules/cypress/bin/cypress run
> or run: node_modules/cypress/bin/cypress open
> if you like to use the dashboard.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron pull request #1223: METRON-1797: Add PCAP UI test implemented in Cypr...

[tiborm]

Github user tiborm closed the pull request at:

https://github.com/apache/metron/pull/1223


---

[jira] [Updated] (METRON-1803) Integrate Cypress with Travis

[Tibor Meller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/METRON-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tibor Meller updated METRON-1803:
-
Summary: Integrate Cypress with Travis  (was: Integrate Cypress tests with 
Travis)

> Integrate Cypress with Travis
> -
>
> Key: METRON-1803
> URL: https://issues.apache.org/jira/browse/METRON-1803
> Project: Metron
>  Issue Type: Improvement
>Reporter: Tibor Meller
>Assignee: Tibor Meller
>Priority: Major
>
> [DISCUSS] thread on the dev mailing list:
> [https://lists.apache.org/thread.html/b6a0272c7809c05e8b7aff20171720e8ec76f8a0e9481169c37a4a4a@%3Cdev.metron.apache.org%3E]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (METRON-1797) Add PCAP UI test implemented in Cypress.js

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639501#comment-16639501
 ] 

ASF GitHub Bot commented on METRON-1797:


GitHub user tiborm reopened a pull request:

https://github.com/apache/metron/pull/1223

METRON-1797: Add PCAP UI test implemented in Cypress.js

## Contributor Comments

[DISCUSS] thread on the dev mailing list: 

https://lists.apache.org/thread.html/b6a0272c7809c05e8b7aff20171720e8ec76f8a0e9481169c37a4a4a@%3Cdev.metron.apache.org%3E

JIRA Ticket: https://issues.apache.org/jira/browse/METRON-1797

This PR adds Cypress.io to our project. It makes us able to run E2E tests, 
UI integration tests. Also higher level UI tests with mocked services. 
As part of this PR I also added tests to cover the following 
functionalities of PCAP UI panel:

- checking running jobs on navigating to PCAP tab
- submitting PCAP job request
- requesting job status
- process status in percentage
- getting PCAP JSON
- rendering PCAP table
- showing PCAP details
- navigating across pages
- downloading PDML
- cancelli PCAP query job
- showing filter validation messages
- showing date validation messages

Please consider while reviewing that these tests are not meant to cover all 
the details of the functionalities above. The aim here is to extend the test 
coverage build up by a large number of Unit Tests with higher level UI tests.

## Integration folder
That contains the actual Cypress tests. The folder name comes from the 
Cypress naming convention, however, no real integration tests implemented here 
yet (at least no other than the integration of UI components to a whole 
application). We could change the name of the folder anytime we like to, but 
only one folder applicable.

## Fixture folder
The tests are running on mocked data. This makes us able to integrate them 
with Travis without the need to having PCAP related services available from our 
CI.
Each file in the Fixture folder represents a response to a particular 
request. This makes it fairly easy to create and maintain.

# How to run
To run Cypress tests locally please do the following:
1. pull this branch to your workspace
2. move to metron-interface/metron-alerts folder
3. run: npm ci
4. run: scripts/start-server-for-e2e.sh
5. run: node_modules/cypress/bin/cypress run
or run: node_modules/cypress/bin/cypress open
if you like to use the dashboard.


## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [x] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
  ```

- [x] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [x] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] Have you verified the basic functionality of the build by building 
and running locally with Vagrant full-dev environment or the equivalent?

### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in 
which it is rendered by building and verifying the site-book? If not then run 
the following commands and the verify changes via 

[GitHub] metron pull request #1223: METRON-1797: Add PCAP UI test implemented in Cypr...

[tiborm]

GitHub user tiborm reopened a pull request:

https://github.com/apache/metron/pull/1223

METRON-1797: Add PCAP UI test implemented in Cypress.js

## Contributor Comments

[DISCUSS] thread on the dev mailing list: 

https://lists.apache.org/thread.html/b6a0272c7809c05e8b7aff20171720e8ec76f8a0e9481169c37a4a4a@%3Cdev.metron.apache.org%3E

JIRA Ticket: https://issues.apache.org/jira/browse/METRON-1797

This PR adds Cypress.io to our project. It makes us able to run E2E tests, 
UI integration tests. Also higher level UI tests with mocked services. 
As part of this PR I also added tests to cover the following 
functionalities of PCAP UI panel:

- checking running jobs on navigating to PCAP tab
- submitting PCAP job request
- requesting job status
- process status in percentage
- getting PCAP JSON
- rendering PCAP table
- showing PCAP details
- navigating across pages
- downloading PDML
- cancelli PCAP query job
- showing filter validation messages
- showing date validation messages

Please consider while reviewing that these tests are not meant to cover all 
the details of the functionalities above. The aim here is to extend the test 
coverage build up by a large number of Unit Tests with higher level UI tests.

## Integration folder
That contains the actual Cypress tests. The folder name comes from the 
Cypress naming convention, however, no real integration tests implemented here 
yet (at least no other than the integration of UI components to a whole 
application). We could change the name of the folder anytime we like to, but 
only one folder applicable.

## Fixture folder
The tests are running on mocked data. This makes us able to integrate them 
with Travis without the need to having PCAP related services available from our 
CI.
Each file in the Fixture folder represents a response to a particular 
request. This makes it fairly easy to create and maintain.

# How to run
To run Cypress tests locally please do the following:
1. pull this branch to your workspace
2. move to metron-interface/metron-alerts folder
3. run: npm ci
4. run: scripts/start-server-for-e2e.sh
5. run: node_modules/cypress/bin/cypress run
or run: node_modules/cypress/bin/cypress open
if you like to use the dashboard.


## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [x] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
  ```

- [x] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [x] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] Have you verified the basic functionality of the build by building 
and running locally with Vagrant full-dev environment or the equivalent?

### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in 
which it is rendered by building and verifying the site-book? If not then run 
the following commands and the verify changes via 
`site-book/target/site/index.html`:

  ```
  cd site-book
  mvn site
  ```

 Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
It is 

[jira] [Commented] (METRON-1771) Update REST endpoints to support eventually consistent UI updates

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639338#comment-16639338
 ] 

ASF GitHub Bot commented on METRON-1771:


Github user asfgit closed the pull request at:

https://github.com/apache/metron/pull/1190


> Update REST endpoints to support eventually consistent UI updates
> -
>
> Key: METRON-1771
> URL: https://issues.apache.org/jira/browse/METRON-1771
> Project: Metron
>  Issue Type: Improvement
>Reporter: Ryan Merriman
>Priority: Major
>
> Currently the REST endpoints that perform document updates either return 
> true/false or nothing.  This puts the responsibility of retrieving the 
> updated state of the object on the client in a separate call or 
> optimistically applying the changes and reverting when an update fails.  This 
> can be problematic if a client attempts to get the current state immediately 
> after an update and the change isn't visible yet in the back end.
> Ideally they should return the updated state of the object, eliminating the 
> need to look up the updated state in a separate call.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (METRON-1695) Expose pcap properties through Ambari

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639339#comment-16639339
 ] 

ASF GitHub Bot commented on METRON-1695:


Github user asfgit closed the pull request at:

https://github.com/apache/metron/pull/1207


> Expose pcap properties through Ambari
> -
>
> Key: METRON-1695
> URL: https://issues.apache.org/jira/browse/METRON-1695
> Project: Metron
>  Issue Type: Bug
>Reporter: Anand Subramanian
>Assignee: Anand Subramanian
>Priority: Major
>
> Currently, the $METRON_HOME/config/pcap.properties file is hardcoded with the 
> defaults. One has to hand edit the file before deploying the PCAP topology. 
> These properties should be configurable via Ambari.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron pull request #1207: METRON-1695: Expose pcap properties through Ambar...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/metron/pull/1207


---

[GitHub] metron pull request #1190: METRON-1771: Update REST endpoints to support eve...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/metron/pull/1190


---

[jira] [Commented] (METRON-1695) Expose pcap properties through Ambari

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639333#comment-16639333
 ] 

ASF GitHub Bot commented on METRON-1695:


Github user anandsubbu commented on the issue:

https://github.com/apache/metron/pull/1207
  
Thanks @MohanDV !


> Expose pcap properties through Ambari
> -
>
> Key: METRON-1695
> URL: https://issues.apache.org/jira/browse/METRON-1695
> Project: Metron
>  Issue Type: Bug
>Reporter: Anand Subramanian
>Assignee: Anand Subramanian
>Priority: Major
>
> Currently, the $METRON_HOME/config/pcap.properties file is hardcoded with the 
> defaults. One has to hand edit the file before deploying the PCAP topology. 
> These properties should be configurable via Ambari.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] metron issue #1207: METRON-1695: Expose pcap properties through Ambari

[anandsubbu]

Github user anandsubbu commented on the issue:

https://github.com/apache/metron/pull/1207
  
Thanks @MohanDV !


---

[jira] [Commented] (METRON-1695) Expose pcap properties through Ambari

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639304#comment-16639304
 ] 

ASF GitHub Bot commented on METRON-1695:


Github user MohanDV commented on the issue:

https://github.com/apache/metron/pull/1207
  
> Cool, thanks for your opinion @nickwallen . I will wait for @MohanDV 's 
thoughts as well before merging.

+1 Thanks @anandsubbu 


> Expose pcap properties through Ambari
> -
>
> Key: METRON-1695
> URL: https://issues.apache.org/jira/browse/METRON-1695
> Project: Metron
>  Issue Type: Bug
>Reporter: Anand Subramanian
>Assignee: Anand Subramanian
>Priority: Major
>
> Currently, the $METRON_HOME/config/pcap.properties file is hardcoded with the 
> defaults. One has to hand edit the file before deploying the PCAP topology. 
> These properties should be configurable via Ambari.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)