[jira] [Commented] (METRON-1650) Packaging docker containers are too large

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560526#comment-16560526
 ] 

ASF GitHub Bot commented on METRON-1650:


Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/1091
  
thanks again @jameslamb!


> Packaging docker containers are too large
> -
>
> Key: METRON-1650
> URL: https://issues.apache.org/jira/browse/METRON-1650
> Project: Metron
>  Issue Type: Improvement
>Reporter: James Lamb
>Priority: Minor
>
> I have been looking through the project source code, and found that the 
> packaging docker containers are bigger than they need to be. 
> The containers' size could be cut substantially by:
>  * Grouping as many commands as possible into as few RUN stages as possible
>  * Taking advantage of `yum clean` and `apt-get clean`
>  * Removing artifacts pulled with `wget`
>  * Removing unused libraries or libraries only needed at build time (like 
> `tar` and `wget`)
> I am creating this issue but will soon submit a PR to address it.
> Thank you for considering my proposal!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron issue #1091: METRON-1650: Cut size of packaging docker containers

2018-07-27 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/1091
  
thanks again @jameslamb!


---


[jira] [Commented] (METRON-1671) Create PCAP UI

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560271#comment-16560271
 ] 

ASF GitHub Bot commented on METRON-1671:


Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/1103
  
Given that unit tests will be addressed in a forthcoming PR, this is good 
enough for me as an initial UI.  +1 pending other commenters are satisfied.


> Create PCAP UI
> --
>
> Key: METRON-1671
> URL: https://issues.apache.org/jira/browse/METRON-1671
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> As a user, I like to submit PCAP query requests from the Metron UI.
> As a user, I like to be able to narrow down the result set of the query by 
> the following filters:
>   - IP Source Address
>   - IP Source Port
>   - IP Dest Address
>   - IP Dest Port
>   - Protocol
>   - Include Reverse Traffic
>   - Free text filtering
> As a user, I like to get feedback from the progress of the ongoing request by 
> a progress bar.
> As a user, I like to see the result of the query in a grid. Each line of the 
> grid should show and extendable packet record.
> As a user, when I clicking on a row of the result grid I would like to see 
> additional information about that particular packet.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron issue #1103: METRON-1671: Initial PCAP UI

2018-07-27 Thread merrimanr
Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/1103
  
Given that unit tests will be addressed in a forthcoming PR, this is good 
enough for me as an initial UI.  +1 pending other commenters are satisfied.


---


[jira] [Commented] (METRON-1650) Packaging docker containers are too large

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560263#comment-16560263
 ] 

ASF GitHub Bot commented on METRON-1650:


Github user asfgit closed the pull request at:

https://github.com/apache/metron/pull/1091


> Packaging docker containers are too large
> -
>
> Key: METRON-1650
> URL: https://issues.apache.org/jira/browse/METRON-1650
> Project: Metron
>  Issue Type: Improvement
>Reporter: James Lamb
>Priority: Minor
>
> I have been looking through the project source code, and found that the 
> packaging docker containers are bigger than they need to be. 
> The containers' size could be cut substantially by:
>  * Grouping as many commands as possible into as few RUN stages as possible
>  * Taking advantage of `yum clean` and `apt-get clean`
>  * Removing artifacts pulled with `wget`
>  * Removing unused libraries or libraries only needed at build time (like 
> `tar` and `wget`)
> I am creating this issue but will soon submit a PR to address it.
> Thank you for considering my proposal!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron pull request #1091: METRON-1650: Cut size of packaging docker contain...

2018-07-27 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/metron/pull/1091


---


[jira] [Commented] (METRON-1683) PCAP UI - Fix the download progress bar

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560178#comment-16560178
 ] 

ASF GitHub Bot commented on METRON-1683:


Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/1122
  
I tested this in full dev and I think it's working well from a functional 
perspective.

I do believe the tests are still lacking though.  Here are the issues I 
found:

- When I run `npm test` I see this:
```
Executed 4 of 63 (skipped 59)
```
Why are tests being skipped?  Is it something with my environment?

- It looks like `pcap.service.spec.ts` hasn't been updated in a while
- As far as I can tell, `pcap-panel.component.spec.ts` isn't testing the 
onSearch function
- I think we're still missing tests in `pcap-packet.component.spec.ts`
- I see sample data declared as `fakePacket` in some tests.  Can we make 
this smaller?  It seems like this test data is bigger than it needs to be and 
redundant.  Is there a way to share this across tests?


> PCAP UI - Fix the download progress bar
> ---
>
> Key: METRON-1683
> URL: https://issues.apache.org/jira/browse/METRON-1683
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Shane Ardell
>Priority: Major
>
> The download progress bar currently does not appear when a PCAP search is 
> submitted. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron issue #1122: METRON-1683: Fix the download progress bar in PCAP UI

2018-07-27 Thread merrimanr
Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/1122
  
I tested this in full dev and I think it's working well from a functional 
perspective.

I do believe the tests are still lacking though.  Here are the issues I 
found:

- When I run `npm test` I see this:
```
Executed 4 of 63 (skipped 59)
```
Why are tests being skipped?  Is it something with my environment?

- It looks like `pcap.service.spec.ts` hasn't been updated in a while
- As far as I can tell, `pcap-panel.component.spec.ts` isn't testing the 
onSearch function
- I think we're still missing tests in `pcap-packet.component.spec.ts`
- I see sample data declared as `fakePacket` in some tests.  Can we make 
this smaller?  It seems like this test data is bigger than it needs to be and 
redundant.  Is there a way to share this across tests?


---


[jira] [Commented] (METRON-1700) Create REST endpoint to get job configuration

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559946#comment-16559946
 ] 

ASF GitHub Bot commented on METRON-1700:


GitHub user merrimanr opened a pull request:

https://github.com/apache/metron/pull/1135

METRON-1700: Create REST endpoint to get job configuration

## Contributor Comments
This PR adds a REST endpoint that returns the job configuration for a 
submitted job.  The endpoint returns a static list of properties that can be 
serialized and are useful for frontend applications.  Changes outside of REST 
were trivial because the PcapJob class already stores the configuration.

### Changes Included
- Added a method to the Statusable interface that returns a configuration 
map
- Added a method to the PcapService that filters the Statusable 
configuration and returns a subset of properties
- Update unit and integration tests

### Testing
This has been tested in full dev.  Testing instructions are as follows:

1. Spin up full dev and put pcap data in the `/apps/metron/pcap/input` HDFS 
directory
2. Submit a pcap query:
```
curl -X POST --header 'Content-Type: application/json' --header 'Accept: 
application/json' -d '{}' 'http://node1:8082/api/v1/pcap/fixed'
```
A job status should be returned as usual.

3. Now the configuration can be retrieved with the get configuration 
endpoint:
```
curl -X GET --header 'Accept: application/json' 
'http://node1:8082/api/v1/pcap/job_1532356159290_0052/configuration'
```
A subset of properties, including those in the pcap query:
```
{
  "finalOutputPath": "/apps/metron/pcap/output",
  "ipSrcPort": "",
  "includeReverse": false,
  "startTimeMs": 1449010818000,
  "endTimeMs": 1532642418000,
  "protocol": "",
  "ipDstAddr": "",
  "basePath": "/apps/metron/pcap/input",
  "packetFilter": "",
  "ipSrcAddr": "",
  "ipDstPort": "",
  "numReducers": 10,
}
```

## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [x] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
  ```

- [x] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [x] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] Have you verified the basic functionality of the build by building 
and running locally with Vagrant full-dev environment or the equivalent?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered by building and verifying the site-book? If not then run 
the following commands and the verify changes via 
`site-book/target/site/index.html`:

  ```
  cd site-book
  mvn site
  ```

 Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up 
for your personal repository such that your branches are built there before 
submitting a pull request.


You can merge this pull request into a Git repository by running:

$ git pull 

[GitHub] metron pull request #1135: METRON-1700: Create REST endpoint to get job conf...

2018-07-27 Thread merrimanr
GitHub user merrimanr opened a pull request:

https://github.com/apache/metron/pull/1135

METRON-1700: Create REST endpoint to get job configuration

## Contributor Comments
This PR adds a REST endpoint that returns the job configuration for a 
submitted job.  The endpoint returns a static list of properties that can be 
serialized and are useful for frontend applications.  Changes outside of REST 
were trivial because the PcapJob class already stores the configuration.

### Changes Included
- Added a method to the Statusable interface that returns a configuration 
map
- Added a method to the PcapService that filters the Statusable 
configuration and returns a subset of properties
- Update unit and integration tests

### Testing
This has been tested in full dev.  Testing instructions are as follows:

1. Spin up full dev and put pcap data in the `/apps/metron/pcap/input` HDFS 
directory
2. Submit a pcap query:
```
curl -X POST --header 'Content-Type: application/json' --header 'Accept: 
application/json' -d '{}' 'http://node1:8082/api/v1/pcap/fixed'
```
A job status should be returned as usual.

3. Now the configuration can be retrieved with the get configuration 
endpoint:
```
curl -X GET --header 'Accept: application/json' 
'http://node1:8082/api/v1/pcap/job_1532356159290_0052/configuration'
```
A subset of properties, including those in the pcap query:
```
{
  "finalOutputPath": "/apps/metron/pcap/output",
  "ipSrcPort": "",
  "includeReverse": false,
  "startTimeMs": 1449010818000,
  "endTimeMs": 1532642418000,
  "protocol": "",
  "ipDstAddr": "",
  "basePath": "/apps/metron/pcap/input",
  "packetFilter": "",
  "ipSrcAddr": "",
  "ipDstPort": "",
  "numReducers": 10,
}
```

## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [x] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
  ```

- [x] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [x] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] Have you verified the basic functionality of the build by building 
and running locally with Vagrant full-dev environment or the equivalent?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered by building and verifying the site-book? If not then run 
the following commands and the verify changes via 
`site-book/target/site/index.html`:

  ```
  cd site-book
  mvn site
  ```

 Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up 
for your personal repository such that your branches are built there before 
submitting a pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/merrimanr/incubator-metron METRON-1700

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/metron/pull/1135.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in 

[jira] [Created] (METRON-1700) Create REST endpoint to get job configuration

2018-07-27 Thread Ryan Merriman (JIRA)
Ryan Merriman created METRON-1700:
-

 Summary: Create REST endpoint to get job configuration
 Key: METRON-1700
 URL: https://issues.apache.org/jira/browse/METRON-1700
 Project: Metron
  Issue Type: Sub-task
Reporter: Ryan Merriman


We need a REST endpoint that will allow us to retrieve the job configuration 
properties.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (METRON-1699) Create Batch Profiler

2018-07-27 Thread Nick Allen (JIRA)


 [ 
https://issues.apache.org/jira/browse/METRON-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Allen updated METRON-1699:
---
Description: 
Create a Batch Profiler that satisfies the following use cases.
h3. Use Cases
 * As a Security Data Scientist, I want to understand the historical behaviors 
and trends of a profile that I have created so that I can determine if I have 
created a feature set that has predictive value for model building.

 * As a Security Data Scientist, I want to understand the historical behaviors 
and trends of a profile that I have created so that I can determine if I have 
defined the profile correctly and created a feature set that matches reality.

 * As a Security Platform Engineer, I want to generate a profile using archived 
telemetry when I deploy a new model to production so that models depending on 
that profile can function on day 1.

h3. Goal
 * Currently, a profile can only be generated from the telemetry consumed 
*after* the profile was created.
 * The goal would be to enable “profile seeding” which allows profiles to be 
populated from a time *before* the profile was created.
 * A profile would be seeded using the telemetry that has been archived by 
Metron in HDFS.
 * A profile consumer should not be able to distinguish the “seeded” portion of 
a profile.

!Screen Shot 2018-07-27 at 10.55.27 AM.png!
h3. Current State
 * There are currently two ports of the Profiler; the Streaming Profiler that 
handles streaming data in Storm and the other that runs in the REPL and allows 
a user to manually build, test, and debug profiles.
 * These ports largely share a common code base in 
metron-analytics/metron-profiler-common.
 * A smaller set of “orchestration” logic is required to maintain each port; 
one for Storm, another for the REPL.
 * Both Profiler ports supports both system time and event time processing.

!Screen Shot 2018-07-27 at 11.07.33 AM.png!
h3. Approach
 * Create a third port of the Profiler; the Batch Profiler.
 * The Batch Profiler will be built to run in Spark so that the telemetry can 
be consumed in batch.
 * Allows a user to seed profiles using the JSON telemetry that is archived in 
HDFS by Metron Indexing.
 * Only generates the profile data stored in HBase, not the messages that are 
produced for Threat Triage and Kafka.
 * Any number of profiles can be generated at once, but no dependencies between 
the profiles are supported. A dependency is where one profile is a consumer of 
the profile generated by another.
 * The Batch Profiler must use the timestamps contained within the telemetry; 
it runs on event time. Luckily the Profiler already supports event time.
 * Enable a pluggable mechanism so that telemetry stored in different formats 
can be consumed by the Batch Profiler. For example, the Profiler should be able 
to consume telemetry stored as raw JSON or in other formats like ORC or 
Parquet.  

!Screen Shot 2018-07-27 at 11.10.16 AM.png!

  was:
Create a Batch Profiler that satisfies the following use cases.

h3. Use Cases

* As a Security Data Scientist, I want to understand the historical behaviors 
and trends of a profile that I have created so that I can determine if I have 
created a feature set that has predictive value for model building.

* As a Security Data Scientist, I want to understand the historical behaviors 
and trends of a profile that I have created so that I can determine if I have 
defined the profile correctly and created a feature set that matches reality.

* As a Security Platform Engineer, I want to generate a profile using archived 
telemetry when I deploy a new model to production so that models depending on 
that profile can function on day 1.

h3. Goal

* Currently, a profile can only be generated from the telemetry consumed 
*after* the profile was created. 
* The goal would be to enable “profile seeding” which allows profiles to be 
populated from a time *before* the profile was created.
* A profile would be seeded using the telemetry that has been archived by 
Metron in HDFS.
* A profile consumer should not be able to distinguish the “seeded” portion of 
a profile.  

 !Screen Shot 2018-07-27 at 10.55.27 AM.png! 

h3. Current State

* There are currently two ports of the Profiler; the Streaming Profiler that 
handles streaming data in Storm and the other that runs in the REPL and allows 
a user to manually build, test, and debug profiles.
* These ports largely share a common code base in 
metron-analytics/metron-profiler-common.
* A smaller set of “orchestration” logic is required to maintain each port; one 
for Storm, another for the REPL.
* Both Profiler ports supports both system time and event time processing.

 !Screen Shot 2018-07-27 at 11.07.33 AM.png! 



> Create Batch Profiler
> -
>
> Key: METRON-1699
> URL: https://issues.apache.org/jira/browse/METRON-1699
> Project: 

[jira] [Commented] (METRON-1650) Packaging docker containers are too large

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559881#comment-16559881
 ] 

ASF GitHub Bot commented on METRON-1650:


Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/1091
  
can one of you ( @cestella or @merrimanr ) merge?  I can't right now


> Packaging docker containers are too large
> -
>
> Key: METRON-1650
> URL: https://issues.apache.org/jira/browse/METRON-1650
> Project: Metron
>  Issue Type: Improvement
>Reporter: James Lamb
>Priority: Minor
>
> I have been looking through the project source code, and found that the 
> packaging docker containers are bigger than they need to be. 
> The containers' size could be cut substantially by:
>  * Grouping as many commands as possible into as few RUN stages as possible
>  * Taking advantage of `yum clean` and `apt-get clean`
>  * Removing artifacts pulled with `wget`
>  * Removing unused libraries or libraries only needed at build time (like 
> `tar` and `wget`)
> I am creating this issue but will soon submit a PR to address it.
> Thank you for considering my proposal!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron issue #1091: METRON-1650: Cut size of packaging docker containers

2018-07-27 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/1091
  
can one of you ( @cestella or @merrimanr ) merge?  I can't right now


---


[jira] [Updated] (METRON-1699) Create Batch Profiler

2018-07-27 Thread Nick Allen (JIRA)


 [ 
https://issues.apache.org/jira/browse/METRON-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Allen updated METRON-1699:
---
Attachment: Screen Shot 2018-07-27 at 11.10.16 AM.png

> Create Batch Profiler
> -
>
> Key: METRON-1699
> URL: https://issues.apache.org/jira/browse/METRON-1699
> Project: Metron
>  Issue Type: Improvement
>Reporter: Nick Allen
>Assignee: Nick Allen
>Priority: Major
> Attachments: Screen Shot 2018-07-27 at 10.55.27 AM.png, Screen Shot 
> 2018-07-27 at 11.07.33 AM.png, Screen Shot 2018-07-27 at 11.10.16 AM.png
>
>
> Create a Batch Profiler that satisfies the following use cases.
> h3. Use Cases
> * As a Security Data Scientist, I want to understand the historical behaviors 
> and trends of a profile that I have created so that I can determine if I have 
> created a feature set that has predictive value for model building.
> * As a Security Data Scientist, I want to understand the historical behaviors 
> and trends of a profile that I have created so that I can determine if I have 
> defined the profile correctly and created a feature set that matches reality.
> * As a Security Platform Engineer, I want to generate a profile using 
> archived telemetry when I deploy a new model to production so that models 
> depending on that profile can function on day 1.
> h3. Goal
> * Currently, a profile can only be generated from the telemetry consumed 
> *after* the profile was created. 
> * The goal would be to enable “profile seeding” which allows profiles to be 
> populated from a time *before* the profile was created.
> * A profile would be seeded using the telemetry that has been archived by 
> Metron in HDFS.
> * A profile consumer should not be able to distinguish the “seeded” portion 
> of a profile.  
>  !Screen Shot 2018-07-27 at 10.55.27 AM.png! 
> h3. Current State
> * There are currently two ports of the Profiler; the Streaming Profiler that 
> handles streaming data in Storm and the other that runs in the REPL and 
> allows a user to manually build, test, and debug profiles.
> * These ports largely share a common code base in 
> metron-analytics/metron-profiler-common.
> * A smaller set of “orchestration” logic is required to maintain each port; 
> one for Storm, another for the REPL.
> * Both Profiler ports supports both system time and event time processing.
>  !Screen Shot 2018-07-27 at 11.07.33 AM.png! 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (METRON-1699) Create Batch Profiler

2018-07-27 Thread Nick Allen (JIRA)


 [ 
https://issues.apache.org/jira/browse/METRON-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Allen updated METRON-1699:
---
Description: 
Create a Batch Profiler that satisfies the following use cases.

h3. Use Cases

* As a Security Data Scientist, I want to understand the historical behaviors 
and trends of a profile that I have created so that I can determine if I have 
created a feature set that has predictive value for model building.

* As a Security Data Scientist, I want to understand the historical behaviors 
and trends of a profile that I have created so that I can determine if I have 
defined the profile correctly and created a feature set that matches reality.

* As a Security Platform Engineer, I want to generate a profile using archived 
telemetry when I deploy a new model to production so that models depending on 
that profile can function on day 1.

h3. Goal

* Currently, a profile can only be generated from the telemetry consumed 
*after* the profile was created. 
* The goal would be to enable “profile seeding” which allows profiles to be 
populated from a time *before* the profile was created.
* A profile would be seeded using the telemetry that has been archived by 
Metron in HDFS.
* A profile consumer should not be able to distinguish the “seeded” portion of 
a profile.  

 !Screen Shot 2018-07-27 at 10.55.27 AM.png! 

h3. Current State

* There are currently two ports of the Profiler; the Streaming Profiler that 
handles streaming data in Storm and the other that runs in the REPL and allows 
a user to manually build, test, and debug profiles.
* These ports largely share a common code base in 
metron-analytics/metron-profiler-common.
* A smaller set of “orchestration” logic is required to maintain each port; one 
for Storm, another for the REPL.
* Both Profiler ports supports both system time and event time processing.

 !Screen Shot 2018-07-27 at 11.07.33 AM.png! 


  was:
Create a Batch Profiler that satisfies the following use cases.

h3. Use Cases

* As a Security Data Scientist, I want to understand the historical behaviors 
and trends of a profile that I have created so that I can determine if I have 
created a feature set that has predictive value for model building.

* As a Security Data Scientist, I want to understand the historical behaviors 
and trends of a profile that I have created so that I can determine if I have 
defined the profile correctly and created a feature set that matches reality.

* As a Security Platform Engineer, I want to generate a profile using archived 
telemetry when I deploy a new model to production so that models depending on 
that profile can function on day 1.

h3. Goal

* Currently, a profile can only be generated from the telemetry consumed 
*after* the profile was created. 
* The goal would be to enable “profile seeding” which allows profiles to be 
populated from a time *before* the profile was created.
* A profile would be seeded using the telemetry that has been archived by 
Metron in HDFS.
* A profile consumer should not be able to distinguish the “seeded” portion of 
a profile.  

 !Screen Shot 2018-07-27 at 10.55.27 AM.png! 





> Create Batch Profiler
> -
>
> Key: METRON-1699
> URL: https://issues.apache.org/jira/browse/METRON-1699
> Project: Metron
>  Issue Type: Improvement
>Reporter: Nick Allen
>Assignee: Nick Allen
>Priority: Major
> Attachments: Screen Shot 2018-07-27 at 10.55.27 AM.png, Screen Shot 
> 2018-07-27 at 11.07.33 AM.png
>
>
> Create a Batch Profiler that satisfies the following use cases.
> h3. Use Cases
> * As a Security Data Scientist, I want to understand the historical behaviors 
> and trends of a profile that I have created so that I can determine if I have 
> created a feature set that has predictive value for model building.
> * As a Security Data Scientist, I want to understand the historical behaviors 
> and trends of a profile that I have created so that I can determine if I have 
> defined the profile correctly and created a feature set that matches reality.
> * As a Security Platform Engineer, I want to generate a profile using 
> archived telemetry when I deploy a new model to production so that models 
> depending on that profile can function on day 1.
> h3. Goal
> * Currently, a profile can only be generated from the telemetry consumed 
> *after* the profile was created. 
> * The goal would be to enable “profile seeding” which allows profiles to be 
> populated from a time *before* the profile was created.
> * A profile would be seeded using the telemetry that has been archived by 
> Metron in HDFS.
> * A profile consumer should not be able to distinguish the “seeded” portion 
> of a profile.  
>  !Screen Shot 2018-07-27 at 10.55.27 AM.png! 
> h3. Current State
> * There are currently two ports of the Profiler; the 

[jira] [Updated] (METRON-1699) Create Batch Profiler

2018-07-27 Thread Nick Allen (JIRA)


 [ 
https://issues.apache.org/jira/browse/METRON-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Allen updated METRON-1699:
---
Attachment: Screen Shot 2018-07-27 at 11.07.33 AM.png

> Create Batch Profiler
> -
>
> Key: METRON-1699
> URL: https://issues.apache.org/jira/browse/METRON-1699
> Project: Metron
>  Issue Type: Improvement
>Reporter: Nick Allen
>Assignee: Nick Allen
>Priority: Major
> Attachments: Screen Shot 2018-07-27 at 10.55.27 AM.png, Screen Shot 
> 2018-07-27 at 11.07.33 AM.png
>
>
> Create a Batch Profiler that satisfies the following use cases.
> h3. Use Cases
> * As a Security Data Scientist, I want to understand the historical behaviors 
> and trends of a profile that I have created so that I can determine if I have 
> created a feature set that has predictive value for model building.
> * As a Security Data Scientist, I want to understand the historical behaviors 
> and trends of a profile that I have created so that I can determine if I have 
> defined the profile correctly and created a feature set that matches reality.
> * As a Security Platform Engineer, I want to generate a profile using 
> archived telemetry when I deploy a new model to production so that models 
> depending on that profile can function on day 1.
> h3. Goal
> * Currently, a profile can only be generated from the telemetry consumed 
> *after* the profile was created. 
> * The goal would be to enable “profile seeding” which allows profiles to be 
> populated from a time *before* the profile was created.
> * A profile would be seeded using the telemetry that has been archived by 
> Metron in HDFS.
> * A profile consumer should not be able to distinguish the “seeded” portion 
> of a profile.  
>  !Screen Shot 2018-07-27 at 10.55.27 AM.png! 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (METRON-1699) Create Batch Profiler

2018-07-27 Thread Nick Allen (JIRA)


 [ 
https://issues.apache.org/jira/browse/METRON-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Allen updated METRON-1699:
---
Description: 
Create a Batch Profiler that satisfies the following use cases.

h3. Use Cases

* As a Security Data Scientist, I want to understand the historical behaviors 
and trends of a profile that I have created so that I can determine if I have 
created a feature set that has predictive value for model building.

* As a Security Data Scientist, I want to understand the historical behaviors 
and trends of a profile that I have created so that I can determine if I have 
defined the profile correctly and created a feature set that matches reality.

* As a Security Platform Engineer, I want to generate a profile using archived 
telemetry when I deploy a new model to production so that models depending on 
that profile can function on day 1.

h3. Goal

* Currently, a profile can only be generated from the telemetry consumed 
*after* the profile was created. 
* The goal would be to enable “profile seeding” which allows profiles to be 
populated from a time *before* the profile was created.
* A profile would be seeded using the telemetry that has been archived by 
Metron in HDFS.
* A profile consumer should not be able to distinguish the “seeded” portion of 
a profile.  

 !Screen Shot 2018-07-27 at 10.55.27 AM.png! 




  was:
Create a Batch Profiler that satisfies the following use cases.

h3. Use Cases

* As a Security Data Scientist, I want to understand the historical behaviors 
and trends of a profile that I have created so that I can determine if I have 
created a feature set that has predictive value for model building.

* As a Security Data Scientist, I want to understand the historical behaviors 
and trends of a profile that I have created so that I can determine if I have 
defined the profile correctly and created a feature set that matches reality.

* As a Security Platform Engineer, I want to generate a profile using archived 
telemetry when I deploy a new model to production so that models depending on 
that profile can function on day 1.





> Create Batch Profiler
> -
>
> Key: METRON-1699
> URL: https://issues.apache.org/jira/browse/METRON-1699
> Project: Metron
>  Issue Type: Improvement
>Reporter: Nick Allen
>Assignee: Nick Allen
>Priority: Major
> Attachments: Screen Shot 2018-07-27 at 10.55.27 AM.png
>
>
> Create a Batch Profiler that satisfies the following use cases.
> h3. Use Cases
> * As a Security Data Scientist, I want to understand the historical behaviors 
> and trends of a profile that I have created so that I can determine if I have 
> created a feature set that has predictive value for model building.
> * As a Security Data Scientist, I want to understand the historical behaviors 
> and trends of a profile that I have created so that I can determine if I have 
> defined the profile correctly and created a feature set that matches reality.
> * As a Security Platform Engineer, I want to generate a profile using 
> archived telemetry when I deploy a new model to production so that models 
> depending on that profile can function on day 1.
> h3. Goal
> * Currently, a profile can only be generated from the telemetry consumed 
> *after* the profile was created. 
> * The goal would be to enable “profile seeding” which allows profiles to be 
> populated from a time *before* the profile was created.
> * A profile would be seeded using the telemetry that has been archived by 
> Metron in HDFS.
> * A profile consumer should not be able to distinguish the “seeded” portion 
> of a profile.  
>  !Screen Shot 2018-07-27 at 10.55.27 AM.png! 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron issue #1091: METRON-1650: Cut size of packaging docker containers

2018-07-27 Thread merrimanr
Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/1091
  
+1 from me.  Thank you so much @jameslamb for sticking with this.  Nice 
work!


---


[jira] [Updated] (METRON-1699) Create Batch Profiler

2018-07-27 Thread Nick Allen (JIRA)


 [ 
https://issues.apache.org/jira/browse/METRON-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Allen updated METRON-1699:
---
Attachment: Screen Shot 2018-07-27 at 10.55.27 AM.png

> Create Batch Profiler
> -
>
> Key: METRON-1699
> URL: https://issues.apache.org/jira/browse/METRON-1699
> Project: Metron
>  Issue Type: Improvement
>Reporter: Nick Allen
>Assignee: Nick Allen
>Priority: Major
> Attachments: Screen Shot 2018-07-27 at 10.55.27 AM.png
>
>
> Create a Batch Profiler that satisfies the following use cases.
> h3. Use Cases
> * As a Security Data Scientist, I want to understand the historical behaviors 
> and trends of a profile that I have created so that I can determine if I have 
> created a feature set that has predictive value for model building.
> * As a Security Data Scientist, I want to understand the historical behaviors 
> and trends of a profile that I have created so that I can determine if I have 
> defined the profile correctly and created a feature set that matches reality.
> * As a Security Platform Engineer, I want to generate a profile using 
> archived telemetry when I deploy a new model to production so that models 
> depending on that profile can function on day 1.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (METRON-1650) Packaging docker containers are too large

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559866#comment-16559866
 ] 

ASF GitHub Bot commented on METRON-1650:


Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/1091
  
+1 from me.  Thank you so much @jameslamb for sticking with this.  Nice 
work!


> Packaging docker containers are too large
> -
>
> Key: METRON-1650
> URL: https://issues.apache.org/jira/browse/METRON-1650
> Project: Metron
>  Issue Type: Improvement
>Reporter: James Lamb
>Priority: Minor
>
> I have been looking through the project source code, and found that the 
> packaging docker containers are bigger than they need to be. 
> The containers' size could be cut substantially by:
>  * Grouping as many commands as possible into as few RUN stages as possible
>  * Taking advantage of `yum clean` and `apt-get clean`
>  * Removing artifacts pulled with `wget`
>  * Removing unused libraries or libraries only needed at build time (like 
> `tar` and `wget`)
> I am creating this issue but will soon submit a PR to address it.
> Thank you for considering my proposal!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (METRON-1699) Create Batch Profiler

2018-07-27 Thread Nick Allen (JIRA)
Nick Allen created METRON-1699:
--

 Summary: Create Batch Profiler
 Key: METRON-1699
 URL: https://issues.apache.org/jira/browse/METRON-1699
 Project: Metron
  Issue Type: Improvement
Reporter: Nick Allen
Assignee: Nick Allen


Create a Batch Profiler that satisfies the following use cases.

h3. Use Cases

* As a Security Data Scientist, I want to understand the historical behaviors 
and trends of a profile that I have created so that I can determine if I have 
created a feature set that has predictive value for model building.

* As a Security Data Scientist, I want to understand the historical behaviors 
and trends of a profile that I have created so that I can determine if I have 
defined the profile correctly and created a feature set that matches reality.

* As a Security Platform Engineer, I want to generate a profile using archived 
telemetry when I deploy a new model to production so that models depending on 
that profile can function on day 1.






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (METRON-1650) Packaging docker containers are too large

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559849#comment-16559849
 ] 

ASF GitHub Bot commented on METRON-1650:


Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/1091
  
@merrimanr I'd like to get your sign off on this, now that @cestella and I 
have given a +1


> Packaging docker containers are too large
> -
>
> Key: METRON-1650
> URL: https://issues.apache.org/jira/browse/METRON-1650
> Project: Metron
>  Issue Type: Improvement
>Reporter: James Lamb
>Priority: Minor
>
> I have been looking through the project source code, and found that the 
> packaging docker containers are bigger than they need to be. 
> The containers' size could be cut substantially by:
>  * Grouping as many commands as possible into as few RUN stages as possible
>  * Taking advantage of `yum clean` and `apt-get clean`
>  * Removing artifacts pulled with `wget`
>  * Removing unused libraries or libraries only needed at build time (like 
> `tar` and `wget`)
> I am creating this issue but will soon submit a PR to address it.
> Thank you for considering my proposal!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron issue #1091: METRON-1650: Cut size of packaging docker containers

2018-07-27 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/1091
  
@merrimanr I'd like to get your sign off on this, now that @cestella and I 
have given a +1


---


[jira] [Commented] (METRON-1650) Packaging docker containers are too large

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559775#comment-16559775
 ] 

ASF GitHub Bot commented on METRON-1650:


Github user cestella commented on the issue:

https://github.com/apache/metron/pull/1091
  
This gets a +1 from me too, great job!


> Packaging docker containers are too large
> -
>
> Key: METRON-1650
> URL: https://issues.apache.org/jira/browse/METRON-1650
> Project: Metron
>  Issue Type: Improvement
>Reporter: James Lamb
>Priority: Minor
>
> I have been looking through the project source code, and found that the 
> packaging docker containers are bigger than they need to be. 
> The containers' size could be cut substantially by:
>  * Grouping as many commands as possible into as few RUN stages as possible
>  * Taking advantage of `yum clean` and `apt-get clean`
>  * Removing artifacts pulled with `wget`
>  * Removing unused libraries or libraries only needed at build time (like 
> `tar` and `wget`)
> I am creating this issue but will soon submit a PR to address it.
> Thank you for considering my proposal!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron issue #1091: METRON-1650: Cut size of packaging docker containers

2018-07-27 Thread cestella
Github user cestella commented on the issue:

https://github.com/apache/metron/pull/1091
  
This gets a +1 from me too, great job!


---


[jira] [Commented] (METRON-1604) Add power pc to OS family for the HCP management pack repo info

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559772#comment-16559772
 ] 

ASF GitHub Bot commented on METRON-1604:


Github user asfgit closed the pull request at:

https://github.com/apache/metron/pull/1052


> Add power pc to OS family for the HCP management pack repo info 
> 
>
> Key: METRON-1604
> URL: https://issues.apache.org/jira/browse/METRON-1604
> Project: Metron
>  Issue Type: Bug
>Reporter: Mohan
>Assignee: Mohan
>Priority: Minor
>
> Add power pc to OS family for the HCP management pack repo info otherwise we 
> don't get "redhat-ppc" under stack selection while installing



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron pull request #1052: METRON-1604 : Add RHEL 7 power pc to OS family fo...

2018-07-27 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/metron/pull/1052


---


[jira] [Commented] (METRON-1604) Add power pc to OS family for the HCP management pack repo info

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559765#comment-16559765
 ] 

ASF GitHub Bot commented on METRON-1604:


Github user cestella commented on the issue:

https://github.com/apache/metron/pull/1052
  
+1 by inspection


> Add power pc to OS family for the HCP management pack repo info 
> 
>
> Key: METRON-1604
> URL: https://issues.apache.org/jira/browse/METRON-1604
> Project: Metron
>  Issue Type: Bug
>Reporter: Mohan
>Assignee: Mohan
>Priority: Minor
>
> Add power pc to OS family for the HCP management pack repo info otherwise we 
> don't get "redhat-ppc" under stack selection while installing



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron issue #1052: METRON-1604 : Add RHEL 7 power pc to OS family for the H...

2018-07-27 Thread cestella
Github user cestella commented on the issue:

https://github.com/apache/metron/pull/1052
  
+1 by inspection


---


[jira] [Commented] (METRON-1671) Create PCAP UI

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559736#comment-16559736
 ] 

ASF GitHub Bot commented on METRON-1671:


Github user sardell commented on the issue:

https://github.com/apache/metron/pull/1103
  
@justinleet In response to your UI-related questions:

> Better input validation would be nice. I can currently enter negative 
ports (which don't match anything) or ports beyond max int (which match 
everything!). Ideally this would also happen in the REST layer, because it 
kicks off jobs that have nonsense parameters.

It looks like @ruffle1986 added some validation [in this 
branch](https://github.com/ruffle1986/metron/commit/1c4bc69eb55d1ba96ce6d101516099d2a788ecf7).
 Are you fine with this being a follow up PR?

> The UI doesn't appear to allow users to kill jobs (although the REST API 
exists for it). Is it correct that this isn't supported?

In the current iteration of the UI, a button to kill jobs is not available. 
I agree that we should add this to the UI, but if it's okay with you I think 
this could be a follow on PR. 

> The parameters differ if you've never entered a field vs. entered then 
deleted it. For example, for ports (and possibly just the two port fields), 
it'll be empty string if never entered and null if entered then deleted. This 
appears to be benign, although surprising.

Nice catch. I think we should open a ticket to investigate this further.

> Kicking off a job, then navigating to the alerts view, then navigating 
back allows a user to kick off a second job. I'm not sure what happens when as 
these jobs finish in potentially either order (or any order if you repeat this 
and kick off many jobs).

If a job is now running, an error is returned from the API. The UI work to 
handle this is in [this 
commit](https://github.com/apache/metron/pull/1122/commits/60f5d76d62f9b23dd44d9890b05c0101ae2b2080).

> It's odd that the port fields have increment / decrement buttons. I would 
expect a user to manually enter that every time.

It looks like this was resolved by @ruffle1986 [in this 
branch](https://github.com/ruffle1986/metron/commit/1c4bc69eb55d1ba96ce6d101516099d2a788ecf7)
 by removing the type attribute on the input.





> Create PCAP UI
> --
>
> Key: METRON-1671
> URL: https://issues.apache.org/jira/browse/METRON-1671
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> As a user, I like to submit PCAP query requests from the Metron UI.
> As a user, I like to be able to narrow down the result set of the query by 
> the following filters:
>   - IP Source Address
>   - IP Source Port
>   - IP Dest Address
>   - IP Dest Port
>   - Protocol
>   - Include Reverse Traffic
>   - Free text filtering
> As a user, I like to get feedback from the progress of the ongoing request by 
> a progress bar.
> As a user, I like to see the result of the query in a grid. Each line of the 
> grid should show and extendable packet record.
> As a user, when I clicking on a row of the result grid I would like to see 
> additional information about that particular packet.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron issue #1103: METRON-1671: Initial PCAP UI

2018-07-27 Thread sardell
Github user sardell commented on the issue:

https://github.com/apache/metron/pull/1103
  
@justinleet In response to your UI-related questions:

> Better input validation would be nice. I can currently enter negative 
ports (which don't match anything) or ports beyond max int (which match 
everything!). Ideally this would also happen in the REST layer, because it 
kicks off jobs that have nonsense parameters.

It looks like @ruffle1986 added some validation [in this 
branch](https://github.com/ruffle1986/metron/commit/1c4bc69eb55d1ba96ce6d101516099d2a788ecf7).
 Are you fine with this being a follow up PR?

> The UI doesn't appear to allow users to kill jobs (although the REST API 
exists for it). Is it correct that this isn't supported?

In the current iteration of the UI, a button to kill jobs is not available. 
I agree that we should add this to the UI, but if it's okay with you I think 
this could be a follow on PR. 

> The parameters differ if you've never entered a field vs. entered then 
deleted it. For example, for ports (and possibly just the two port fields), 
it'll be empty string if never entered and null if entered then deleted. This 
appears to be benign, although surprising.

Nice catch. I think we should open a ticket to investigate this further.

> Kicking off a job, then navigating to the alerts view, then navigating 
back allows a user to kick off a second job. I'm not sure what happens when as 
these jobs finish in potentially either order (or any order if you repeat this 
and kick off many jobs).

If a job is now running, an error is returned from the API. The UI work to 
handle this is in [this 
commit](https://github.com/apache/metron/pull/1122/commits/60f5d76d62f9b23dd44d9890b05c0101ae2b2080).

> It's odd that the port fields have increment / decrement buttons. I would 
expect a user to manually enter that every time.

It looks like this was resolved by @ruffle1986 [in this 
branch](https://github.com/ruffle1986/metron/commit/1c4bc69eb55d1ba96ce6d101516099d2a788ecf7)
 by removing the type attribute on the input.





---


[jira] [Commented] (METRON-1671) Create PCAP UI

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559614#comment-16559614
 ] 

ASF GitHub Bot commented on METRON-1671:


Github user sardell commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205746588
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-packet-line/pcap-packet-line.component.html
 ---
@@ -0,0 +1,19 @@
+
+{{ip.timestamp.value}} {{ip.timestamp.show}}
--- End diff --

@justinleet [I created a 
ticket](https://issues.apache.org/jira/browse/METRON-1698) to get some feedback 
on whether we should be displaying both field values as separate fields in the 
UI or if we should just choose one. I'm going to hold off on making changes to 
this code until we get some feedback from the community.


> Create PCAP UI
> --
>
> Key: METRON-1671
> URL: https://issues.apache.org/jira/browse/METRON-1671
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> As a user, I like to submit PCAP query requests from the Metron UI.
> As a user, I like to be able to narrow down the result set of the query by 
> the following filters:
>   - IP Source Address
>   - IP Source Port
>   - IP Dest Address
>   - IP Dest Port
>   - Protocol
>   - Include Reverse Traffic
>   - Free text filtering
> As a user, I like to get feedback from the progress of the ongoing request by 
> a progress bar.
> As a user, I like to see the result of the query in a grid. Each line of the 
> grid should show and extendable packet record.
> As a user, when I clicking on a row of the result grid I would like to see 
> additional information about that particular packet.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron pull request #1103: METRON-1671: Initial PCAP UI

2018-07-27 Thread sardell
Github user sardell commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205746588
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-packet-line/pcap-packet-line.component.html
 ---
@@ -0,0 +1,19 @@
+
+{{ip.timestamp.value}} {{ip.timestamp.show}}
--- End diff --

@justinleet [I created a 
ticket](https://issues.apache.org/jira/browse/METRON-1698) to get some feedback 
on whether we should be displaying both field values as separate fields in the 
UI or if we should just choose one. I'm going to hold off on making changes to 
this code until we get some feedback from the community.


---


[jira] [Created] (METRON-1698) PCAP UI - Timestamp field contains both the epoch timestamp and the date

2018-07-27 Thread Shane Ardell (JIRA)
Shane Ardell created METRON-1698:


 Summary: PCAP UI - Timestamp field contains both the epoch 
timestamp and the date
 Key: METRON-1698
 URL: https://issues.apache.org/jira/browse/METRON-1698
 Project: Metron
  Issue Type: Improvement
Reporter: Shane Ardell


As pointed out in the initial PCAP UI work, [the timestamp field in the pcap 
results contains both the epoch timestamp and the date 
|https://github.com/apache/metron/pull/1103]. It seems like we either need to 
separate these values into two separate fields or choose one to display.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (METRON-1671) Create PCAP UI

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559570#comment-16559570
 ] 

ASF GitHub Bot commented on METRON-1671:


Github user tiborm commented on the issue:

https://github.com/apache/metron/pull/1103
  
As part of the latest commits I removed commented code blocks, and fixed 
the variable naming issues in pcap-packet-line.component.ts.
This PR and the followup ones are updated by the latest changes from the 
base branch.


> Create PCAP UI
> --
>
> Key: METRON-1671
> URL: https://issues.apache.org/jira/browse/METRON-1671
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> As a user, I like to submit PCAP query requests from the Metron UI.
> As a user, I like to be able to narrow down the result set of the query by 
> the following filters:
>   - IP Source Address
>   - IP Source Port
>   - IP Dest Address
>   - IP Dest Port
>   - Protocol
>   - Include Reverse Traffic
>   - Free text filtering
> As a user, I like to get feedback from the progress of the ongoing request by 
> a progress bar.
> As a user, I like to see the result of the query in a grid. Each line of the 
> grid should show and extendable packet record.
> As a user, when I clicking on a row of the result grid I would like to see 
> additional information about that particular packet.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron issue #1103: METRON-1671: Initial PCAP UI

2018-07-27 Thread tiborm
Github user tiborm commented on the issue:

https://github.com/apache/metron/pull/1103
  
As part of the latest commits I removed commented code blocks, and fixed 
the variable naming issues in pcap-packet-line.component.ts.
This PR and the followup ones are updated by the latest changes from the 
base branch.


---


[jira] [Commented] (METRON-1671) Create PCAP UI

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559567#comment-16559567
 ] 

ASF GitHub Bot commented on METRON-1671:


Github user tiborm commented on the issue:

https://github.com/apache/metron/pull/1103
  
@mmiklavc Thanks for the comment! I extended the JIRA ticket with user 
story like test scenarios.
Also added a short description of to the PR description about how to spin 
up a full dev with pcap.


> Create PCAP UI
> --
>
> Key: METRON-1671
> URL: https://issues.apache.org/jira/browse/METRON-1671
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> As a user, I like to submit PCAP query requests from the Metron UI.
> As a user, I like to be able to narrow down the result set of the query by 
> the following filters:
>   - IP Source Address
>   - IP Source Port
>   - IP Dest Address
>   - IP Dest Port
>   - Protocol
>   - Include Reverse Traffic
>   - Free text filtering
> As a user, I like to get feedback from the progress of the ongoing request by 
> a progress bar.
> As a user, I like to see the result of the query in a grid. Each line of the 
> grid should show and extendable packet record.
> As a user, when I clicking on a row of the result grid I would like to see 
> additional information about that particular packet.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron issue #1103: METRON-1671: Initial PCAP UI

2018-07-27 Thread tiborm
Github user tiborm commented on the issue:

https://github.com/apache/metron/pull/1103
  
@mmiklavc Thanks for the comment! I extended the JIRA ticket with user 
story like test scenarios.
Also added a short description of to the PR description about how to spin 
up a full dev with pcap.


---


[jira] [Updated] (METRON-1671) Create PCAP UI

2018-07-27 Thread Tibor Meller (JIRA)


 [ 
https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tibor Meller updated METRON-1671:
-
Description: 
As a user, I like to submit PCAP query requests from the Metron UI.
As a user, I like to be able to narrow down the result set of the query by the 
following filters:

  - IP Source Address
  - IP Source Port
  - IP Dest Address
  - IP Dest Port
  - Protocol
  - Include Reverse Traffic
  - Free text filtering

As a user, I like to get feedback from the progress of the ongoing request by a 
progress bar.

As a user, I like to see the result of the query in a grid. Each line of the 
grid should show and extendable packet record.

As a user, when I clicking on a row of the result grid I would like to see 
additional information about that particular packet.

  was:
The initial feature set of PCAP UI is the follwing:
 - Filtering by
  - IP Source Address
  - IP Source Port
  - IP Dest Address
  - IP Dest Port
  - Protocol
  - Include Reverse Traffic
  - Free text filtering
 - Showing PDML result


> Create PCAP UI
> --
>
> Key: METRON-1671
> URL: https://issues.apache.org/jira/browse/METRON-1671
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> As a user, I like to submit PCAP query requests from the Metron UI.
> As a user, I like to be able to narrow down the result set of the query by 
> the following filters:
>   - IP Source Address
>   - IP Source Port
>   - IP Dest Address
>   - IP Dest Port
>   - Protocol
>   - Include Reverse Traffic
>   - Free text filtering
> As a user, I like to get feedback from the progress of the ongoing request by 
> a progress bar.
> As a user, I like to see the result of the query in a grid. Each line of the 
> grid should show and extendable packet record.
> As a user, when I clicking on a row of the result grid I would like to see 
> additional information about that particular packet.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron pull request #1103: METRON-1671: Initial PCAP UI

2018-07-27 Thread tiborm
Github user tiborm commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205732629
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-packet-line/pcap-packet-line.component.ts
 ---
@@ -0,0 +1,55 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+import { Component, OnInit, Input } from '@angular/core';
+import { PdmlPacket, PdmlProto, PdmlField } from '../model/pdml'
+
+@Component({
+  selector: '[app-pcap-packet-line]',
+  templateUrl: './pcap-packet-line.component.html',
+  styleUrls: ['./pcap-packet-line.component.scss']
+})
+export class PcapPacketLineComponent implements OnInit {
+
+  @Input() packet: PdmlPacket
+
+  ip: {
+timestamp: PdmlField,
+ip_src_addr: PdmlField, ip_src_port: PdmlField,
+ip_dest_addr: PdmlField, ip_dest_port: PdmlField,
+protocol: PdmlField
+  }
+
+  constructor() { }
+
+  ngOnInit() {
+let gen_proto: PdmlProto = this.packet.protos.filter(p => p.name == 
"geninfo")[0]
--- End diff --

Thanks @merrimanr! I addressed this in a new commit.


---


[jira] [Commented] (METRON-1671) Create PCAP UI

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559542#comment-16559542
 ] 

ASF GitHub Bot commented on METRON-1671:


Github user tiborm commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205732629
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-packet-line/pcap-packet-line.component.ts
 ---
@@ -0,0 +1,55 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+import { Component, OnInit, Input } from '@angular/core';
+import { PdmlPacket, PdmlProto, PdmlField } from '../model/pdml'
+
+@Component({
+  selector: '[app-pcap-packet-line]',
+  templateUrl: './pcap-packet-line.component.html',
+  styleUrls: ['./pcap-packet-line.component.scss']
+})
+export class PcapPacketLineComponent implements OnInit {
+
+  @Input() packet: PdmlPacket
+
+  ip: {
+timestamp: PdmlField,
+ip_src_addr: PdmlField, ip_src_port: PdmlField,
+ip_dest_addr: PdmlField, ip_dest_port: PdmlField,
+protocol: PdmlField
+  }
+
+  constructor() { }
+
+  ngOnInit() {
+let gen_proto: PdmlProto = this.packet.protos.filter(p => p.name == 
"geninfo")[0]
--- End diff --

Thanks @merrimanr! I addressed this in a new commit.


> Create PCAP UI
> --
>
> Key: METRON-1671
> URL: https://issues.apache.org/jira/browse/METRON-1671
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> The initial feature set of PCAP UI is the follwing:
>  - Filtering by
>   - IP Source Address
>   - IP Source Port
>   - IP Dest Address
>   - IP Dest Port
>   - Protocol
>   - Include Reverse Traffic
>   - Free text filtering
>  - Showing PDML result



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (METRON-1671) Create PCAP UI

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559506#comment-16559506
 ] 

ASF GitHub Bot commented on METRON-1671:


Github user tiborm commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205723353
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-panel/pcap-panel.component.ts 
---
@@ -0,0 +1,78 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+import { Component, OnInit, Input } from '@angular/core';
+
+import { PcapService, PcapStatusResponse } from '../service/pcap.service'
+import { PcapRequest } from '../model/pcap.request'
+import { Pdml } from '../model/pdml'
+import {Subscription} from "rxjs/Rx";
+
+@Component({
+  selector: 'app-pcap-panel',
+  templateUrl: './pcap-panel.component.html',
+  styleUrls: ['./pcap-panel.component.scss']
+})
+export class PcapPanelComponent implements OnInit {
+
+  @Input() pdml: Pdml = null;
+
+  @Input() pcapRequest: PcapRequest;
+
+  statusSubscription: Subscription;
+  queryRunning: boolean = false;
+  progressWidth: number = 0;
+  selectedPage: number = 1;
+
+  constructor(private pcapService: PcapService ) { }
+
+  ngOnInit() {
+  }
+
+  onSearch(pcapRequest) {
+console.log(pcapRequest);
+this.pdml = null;
+this.progressWidth = 0;
+this.pcapService.submitRequest(pcapRequest).subscribe(id => {
+  this.queryRunning = true;
+  this.statusSubscription = 
this.pcapService.pollStatus(id).subscribe((statusResponse: PcapStatusResponse) 
=> {
+if ('SUCCEEDED' === statusResponse.jobStatus) {
--- End diff --

FAILED status handling added. KILLED status is out of scope of the current 
PR.


> Create PCAP UI
> --
>
> Key: METRON-1671
> URL: https://issues.apache.org/jira/browse/METRON-1671
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> The initial feature set of PCAP UI is the follwing:
>  - Filtering by
>   - IP Source Address
>   - IP Source Port
>   - IP Dest Address
>   - IP Dest Port
>   - Protocol
>   - Include Reverse Traffic
>   - Free text filtering
>  - Showing PDML result



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (METRON-1671) Create PCAP UI

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559505#comment-16559505
 ] 

ASF GitHub Bot commented on METRON-1671:


Github user tiborm commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205723106
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-panel/pcap-panel.component.ts 
---
@@ -0,0 +1,78 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+import { Component, OnInit, Input } from '@angular/core';
+
+import { PcapService, PcapStatusResponse } from '../service/pcap.service'
+import { PcapRequest } from '../model/pcap.request'
+import { Pdml } from '../model/pdml'
+import {Subscription} from "rxjs/Rx";
+
+@Component({
+  selector: 'app-pcap-panel',
+  templateUrl: './pcap-panel.component.html',
+  styleUrls: ['./pcap-panel.component.scss']
+})
+export class PcapPanelComponent implements OnInit {
+
+  @Input() pdml: Pdml = null;
+
+  @Input() pcapRequest: PcapRequest;
+
+  statusSubscription: Subscription;
+  queryRunning: boolean = false;
+  progressWidth: number = 0;
+  selectedPage: number = 1;
+
+  constructor(private pcapService: PcapService ) { }
+
+  ngOnInit() {
+  }
+
+  onSearch(pcapRequest) {
+console.log(pcapRequest);
+this.pdml = null;
+this.progressWidth = 0;
+this.pcapService.submitRequest(pcapRequest).subscribe(id => {
+  this.queryRunning = true;
+  this.statusSubscription = 
this.pcapService.pollStatus(id).subscribe((statusResponse: PcapStatusResponse) 
=> {
+if ('SUCCEEDED' === statusResponse.jobStatus) {
+  this.statusSubscription.unsubscribe();
+  this.queryRunning = false;
+  this.pcapService.getPackets(id, 
this.selectedPage).toPromise().then(pdml => {
+this.pdml = pdml;
+  });
+} else if (this.progressWidth < 100) {
+  this.progressWidth = Math.trunc(statusResponse.percentComplete);
+}
+  });
+});
+
+// 
this.pcapService.getTestPackets(this.pcapRequest).subscribe(response => {
--- End diff --

Commented code removed.


> Create PCAP UI
> --
>
> Key: METRON-1671
> URL: https://issues.apache.org/jira/browse/METRON-1671
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> The initial feature set of PCAP UI is the follwing:
>  - Filtering by
>   - IP Source Address
>   - IP Source Port
>   - IP Dest Address
>   - IP Dest Port
>   - Protocol
>   - Include Reverse Traffic
>   - Free text filtering
>  - Showing PDML result



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (METRON-1671) Create PCAP UI

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559497#comment-16559497
 ] 

ASF GitHub Bot commented on METRON-1671:


Github user tiborm commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205721981
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-panel/pcap-panel.component.spec.ts
 ---
@@ -0,0 +1,51 @@
+import { async, ComponentFixture, TestBed } from '@angular/core/testing';
--- End diff --

Thanks, we added license headers.


> Create PCAP UI
> --
>
> Key: METRON-1671
> URL: https://issues.apache.org/jira/browse/METRON-1671
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> The initial feature set of PCAP UI is the follwing:
>  - Filtering by
>   - IP Source Address
>   - IP Source Port
>   - IP Dest Address
>   - IP Dest Port
>   - Protocol
>   - Include Reverse Traffic
>   - Free text filtering
>  - Showing PDML result



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (METRON-1671) Create PCAP UI

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559498#comment-16559498
 ] 

ASF GitHub Bot commented on METRON-1671:


Github user tiborm commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205722021
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-panel/pcap-panel.component.ts 
---
@@ -0,0 +1,65 @@
+import { Component, OnInit, Input } from '@angular/core';
--- End diff --

Thanks, we added license headers.


> Create PCAP UI
> --
>
> Key: METRON-1671
> URL: https://issues.apache.org/jira/browse/METRON-1671
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> The initial feature set of PCAP UI is the follwing:
>  - Filtering by
>   - IP Source Address
>   - IP Source Port
>   - IP Dest Address
>   - IP Dest Port
>   - Protocol
>   - Include Reverse Traffic
>   - Free text filtering
>  - Showing PDML result



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron pull request #1103: METRON-1671: Initial PCAP UI

2018-07-27 Thread tiborm
Github user tiborm commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205722021
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-panel/pcap-panel.component.ts 
---
@@ -0,0 +1,65 @@
+import { Component, OnInit, Input } from '@angular/core';
--- End diff --

Thanks, we added license headers.


---


[GitHub] metron pull request #1103: METRON-1671: Initial PCAP UI

2018-07-27 Thread tiborm
Github user tiborm commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205721981
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-panel/pcap-panel.component.spec.ts
 ---
@@ -0,0 +1,51 @@
+import { async, ComponentFixture, TestBed } from '@angular/core/testing';
--- End diff --

Thanks, we added license headers.


---


[jira] [Commented] (METRON-1671) Create PCAP UI

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559493#comment-16559493
 ] 

ASF GitHub Bot commented on METRON-1671:


Github user tiborm commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205721365
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-packet/pcap-packet.component.ts
 ---
@@ -0,0 +1,22 @@
+import { Component, OnInit, Input } from '@angular/core';
--- End diff --

Thanks, we added license headers.


> Create PCAP UI
> --
>
> Key: METRON-1671
> URL: https://issues.apache.org/jira/browse/METRON-1671
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> The initial feature set of PCAP UI is the follwing:
>  - Filtering by
>   - IP Source Address
>   - IP Source Port
>   - IP Dest Address
>   - IP Dest Port
>   - Protocol
>   - Include Reverse Traffic
>   - Free text filtering
>  - Showing PDML result



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron pull request #1103: METRON-1671: Initial PCAP UI

2018-07-27 Thread tiborm
Github user tiborm commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205721365
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-packet/pcap-packet.component.ts
 ---
@@ -0,0 +1,22 @@
+import { Component, OnInit, Input } from '@angular/core';
--- End diff --

Thanks, we added license headers.


---


[jira] [Commented] (METRON-1671) Create PCAP UI

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559464#comment-16559464
 ] 

ASF GitHub Bot commented on METRON-1671:


Github user tiborm commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205714876
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-filters/pcap-filters.component.spec.ts
 ---
@@ -0,0 +1,29 @@
+import { async, ComponentFixture, TestBed } from '@angular/core/testing';
--- End diff --

Thanks, we added license headers.


> Create PCAP UI
> --
>
> Key: METRON-1671
> URL: https://issues.apache.org/jira/browse/METRON-1671
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> The initial feature set of PCAP UI is the follwing:
>  - Filtering by
>   - IP Source Address
>   - IP Source Port
>   - IP Dest Address
>   - IP Dest Port
>   - Protocol
>   - Include Reverse Traffic
>   - Free text filtering
>  - Showing PDML result



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (METRON-1671) Create PCAP UI

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559465#comment-16559465
 ] 

ASF GitHub Bot commented on METRON-1671:


Github user tiborm commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205714935
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-filters/pcap-filters.component.ts
 ---
@@ -0,0 +1,24 @@
+import { Component, OnInit, Input, Output, EventEmitter } from 
'@angular/core';
--- End diff --

Thanks, we added license headers.


> Create PCAP UI
> --
>
> Key: METRON-1671
> URL: https://issues.apache.org/jira/browse/METRON-1671
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> The initial feature set of PCAP UI is the follwing:
>  - Filtering by
>   - IP Source Address
>   - IP Source Port
>   - IP Dest Address
>   - IP Dest Port
>   - Protocol
>   - Include Reverse Traffic
>   - Free text filtering
>  - Showing PDML result



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron pull request #1103: METRON-1671: Initial PCAP UI

2018-07-27 Thread tiborm
Github user tiborm commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205715003
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-list/pcap-list.component.spec.ts
 ---
@@ -0,0 +1,53 @@
+import { async, ComponentFixture, TestBed } from '@angular/core/testing';
--- End diff --

Thanks, we added license headers.


---


[jira] [Commented] (METRON-1671) Create PCAP UI

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559466#comment-16559466
 ] 

ASF GitHub Bot commented on METRON-1671:


Github user tiborm commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205715003
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-list/pcap-list.component.spec.ts
 ---
@@ -0,0 +1,53 @@
+import { async, ComponentFixture, TestBed } from '@angular/core/testing';
--- End diff --

Thanks, we added license headers.


> Create PCAP UI
> --
>
> Key: METRON-1671
> URL: https://issues.apache.org/jira/browse/METRON-1671
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> The initial feature set of PCAP UI is the follwing:
>  - Filtering by
>   - IP Source Address
>   - IP Source Port
>   - IP Dest Address
>   - IP Dest Port
>   - Protocol
>   - Include Reverse Traffic
>   - Free text filtering
>  - Showing PDML result



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron pull request #1103: METRON-1671: Initial PCAP UI

2018-07-27 Thread tiborm
Github user tiborm commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205714935
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-filters/pcap-filters.component.ts
 ---
@@ -0,0 +1,24 @@
+import { Component, OnInit, Input, Output, EventEmitter } from 
'@angular/core';
--- End diff --

Thanks, we added license headers.


---


[GitHub] metron pull request #1103: METRON-1671: Initial PCAP UI

2018-07-27 Thread tiborm
Github user tiborm commented on a diff in the pull request:

https://github.com/apache/metron/pull/1103#discussion_r205714876
  
--- Diff: 
metron-interface/metron-alerts/src/app/pcap/pcap-filters/pcap-filters.component.spec.ts
 ---
@@ -0,0 +1,29 @@
+import { async, ComponentFixture, TestBed } from '@angular/core/testing';
--- End diff --

Thanks, we added license headers.


---


[jira] [Commented] (METRON-1662) PCAP UI - Downloading PCAP page files

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559451#comment-16559451
 ] 

ASF GitHub Bot commented on METRON-1662:


Github user tiborm commented on the issue:

https://github.com/apache/metron/pull/1118
  
@mmiklavc I updated the JIRA ticket with instructions for testing.
https://issues.apache.org/jira/browse/METRON-1662



> PCAP UI - Downloading PCAP page files
> -
>
> Key: METRON-1662
> URL: https://issues.apache.org/jira/browse/METRON-1662
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> As a user, I like to be able to download the filtered PCAP query output in 
> PCAP binary format.
>  As a user, when I click the download button below the result grid, I like to 
> get the result page selected by the paginator component.
>  
> *Acceptance criteria*
> Given
>  * the user is logged in
>  * and navigated to the PCAP panel
>  * submitted a PCAP query
>  * the valid result appeared on the UI
> When
>  * the user clicking the download button below the result grid
> Then
>  * the selected page of the result should download in PCAP binary format



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron issue #1118: METRON-1662: Adding download button

2018-07-27 Thread tiborm
Github user tiborm commented on the issue:

https://github.com/apache/metron/pull/1118
  
@mmiklavc I updated the JIRA ticket with instructions for testing.
https://issues.apache.org/jira/browse/METRON-1662



---


[jira] [Updated] (METRON-1662) PCAP UI - Downloading PCAP page files

2018-07-27 Thread Tibor Meller (JIRA)


 [ 
https://issues.apache.org/jira/browse/METRON-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tibor Meller updated METRON-1662:
-
Description: 
As a user, I like to be able to download the filtered PCAP query output in PCAP 
binary format.
 As a user, when I click the download button below the result grid, I like to 
get the result page selected by the paginator component.

 

*Acceptance criteria*

Given
 * the user is logged in
 * and navigated to the PCAP panel
 * submitted a PCAP query
 * the valid result appeared on the UI

When
 * the user clicking the download button below the result grid

Then
 * the selected page of the result should download in PCAP binary format

  was:
As a user, I like to be able to download the filtered PCAP query output in PCAP 
binary format.
As a user, when I click the download button below the result grid, I like to 
get the result page selected by the paginator component.

 

*Acceptance criteria*

Given
 * the user is logged in
 * and navigated to the PCAP panel
 * submitted a PCAP query
 * the valid result appeared on the UI

When
 * the user clicking the download button

Then
 * the selected page of the result should download in PCAP binary format


> PCAP UI - Downloading PCAP page files
> -
>
> Key: METRON-1662
> URL: https://issues.apache.org/jira/browse/METRON-1662
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> As a user, I like to be able to download the filtered PCAP query output in 
> PCAP binary format.
>  As a user, when I click the download button below the result grid, I like to 
> get the result page selected by the paginator component.
>  
> *Acceptance criteria*
> Given
>  * the user is logged in
>  * and navigated to the PCAP panel
>  * submitted a PCAP query
>  * the valid result appeared on the UI
> When
>  * the user clicking the download button below the result grid
> Then
>  * the selected page of the result should download in PCAP binary format



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (METRON-1662) PCAP UI - Downloading PCAP page files

2018-07-27 Thread Tibor Meller (JIRA)


 [ 
https://issues.apache.org/jira/browse/METRON-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tibor Meller updated METRON-1662:
-
Description: 
As a user, I like to be able to download the filtered PCAP query output in PCAP 
binary format.
As a user, when I click the download button below the result grid, I like to 
get the result page selected by the paginator component.

 

*Acceptance criteria*

Given
 * the user is logged in
 * and navigated to the PCAP panel
 * submitted a PCAP query
 * the valid result appeared on the UI

When
 * the user clicking the download button

Then
 * the selected page of the result should download in PCAP binary format

  was:
As a user, I like to be able to download the filtered PCAP query output in PCAP 
binary format.
As a user, when I click the download button below the result grid, I like to 
get the result page selected by the paginator component.

 

*Acceptance* criterias

Given
 * the user is logged in
 * and navigated to the PCAP panel
 * submitted a PCAP query
 * the valid result appeared on the UI

When
 * the user clicking the download button

Then
 * the selected page of the result should download in PCAP binary format


> PCAP UI - Downloading PCAP page files
> -
>
> Key: METRON-1662
> URL: https://issues.apache.org/jira/browse/METRON-1662
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> As a user, I like to be able to download the filtered PCAP query output in 
> PCAP binary format.
> As a user, when I click the download button below the result grid, I like to 
> get the result page selected by the paginator component.
>  
> *Acceptance criteria*
> Given
>  * the user is logged in
>  * and navigated to the PCAP panel
>  * submitted a PCAP query
>  * the valid result appeared on the UI
> When
>  * the user clicking the download button
> Then
>  * the selected page of the result should download in PCAP binary format



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (METRON-1662) PCAP UI - Downloading PCAP page files

2018-07-27 Thread Tibor Meller (JIRA)


 [ 
https://issues.apache.org/jira/browse/METRON-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tibor Meller updated METRON-1662:
-
Description: 
As a user, I like to be able to download the filtered PCAP query output in PCAP 
binary format.
As a user, when I click the download button below the result grid, I like to 
get the result page selected by the paginator component.

 

*Acceptance* criterias

Given
 * the user is logged in
 * and navigated to the PCAP panel
 * submitted a PCAP query
 * the valid result appeared on the UI

When
 * the user clicking the download button

Then
 * the selected page of the result should download in PCAP binary format

  was:
As a user, I like to be able to download the filtered PCAP query output in PCAP 
binary format.
As a user, when I click the download button below the result grid, I like to 
get the result page selected by the paginator component.

 

Acceptance criterias

Given
 * the user is logged in
 * and navigated to the PCAP panel
 * submitted a PCAP query
 * response of the PCAP query


> PCAP UI - Downloading PCAP page files
> -
>
> Key: METRON-1662
> URL: https://issues.apache.org/jira/browse/METRON-1662
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> As a user, I like to be able to download the filtered PCAP query output in 
> PCAP binary format.
> As a user, when I click the download button below the result grid, I like to 
> get the result page selected by the paginator component.
>  
> *Acceptance* criterias
> Given
>  * the user is logged in
>  * and navigated to the PCAP panel
>  * submitted a PCAP query
>  * the valid result appeared on the UI
> When
>  * the user clicking the download button
> Then
>  * the selected page of the result should download in PCAP binary format



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (METRON-1662) PCAP UI - Downloading PCAP page files

2018-07-27 Thread Tibor Meller (JIRA)


 [ 
https://issues.apache.org/jira/browse/METRON-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tibor Meller updated METRON-1662:
-
Description: 
As a user, I like to be able to download the filtered PCAP query output in PCAP 
binary format.
As a user, when I click the download button below the result grid, I like to 
get the result page selected by the paginator component.

 

Acceptance criterias

Given
 * the user is logged in
 * and navigated to the PCAP panel
 * submitted a PCAP query
 * response of the PCAP query

  was:
As a user, I like to be able to download the filtered PCAP query output in PCAP 
binary format.
As a user, when I click the download button below the result grid, I like to 
get the result page selected by the paginator component.

 

Acceptance criterias


> PCAP UI - Downloading PCAP page files
> -
>
> Key: METRON-1662
> URL: https://issues.apache.org/jira/browse/METRON-1662
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> As a user, I like to be able to download the filtered PCAP query output in 
> PCAP binary format.
> As a user, when I click the download button below the result grid, I like to 
> get the result page selected by the paginator component.
>  
> Acceptance criterias
> Given
>  * the user is logged in
>  * and navigated to the PCAP panel
>  * submitted a PCAP query
>  * response of the PCAP query



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (METRON-1662) PCAP UI - Downloading PCAP page files

2018-07-27 Thread Tibor Meller (JIRA)


 [ 
https://issues.apache.org/jira/browse/METRON-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tibor Meller updated METRON-1662:
-
Description: 
As a user, I like to be able to download the filtered PCAP query output in PCAP 
binary format.
As a user, when I click the download button below the result grid, I like to 
get the result page selected by the paginator component.

 

Acceptance criterias

> PCAP UI - Downloading PCAP page files
> -
>
> Key: METRON-1662
> URL: https://issues.apache.org/jira/browse/METRON-1662
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>
> As a user, I like to be able to download the filtered PCAP query output in 
> PCAP binary format.
> As a user, when I click the download button below the result grid, I like to 
> get the result page selected by the paginator component.
>  
> Acceptance criterias



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (METRON-1662) PCAP UI - Downloading PCAP page files

2018-07-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559430#comment-16559430
 ] 

ASF GitHub Bot commented on METRON-1662:


Github user tiborm commented on the issue:

https://github.com/apache/metron/pull/1118
  
@cestella license headers added.


> PCAP UI - Downloading PCAP page files
> -
>
> Key: METRON-1662
> URL: https://issues.apache.org/jira/browse/METRON-1662
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Tibor Meller
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] metron issue #1118: METRON-1662: Adding download button

2018-07-27 Thread tiborm
Github user tiborm commented on the issue:

https://github.com/apache/metron/pull/1118
  
@cestella license headers added.


---