[jira] [Created] (METRON-1710) api/v1/sensor/indexing/config/list/indices/{writer} will return sensor lists all indices for invalid writer
Mohan created METRON-1710:
-
Summary: api/v1/sensor/indexing/config/list/indices/{writer} will
return sensor lists all indices for invalid writer
Key: METRON-1710
URL: https://issues.apache.org/jira/browse/METRON-1710
Project: Metron
Issue Type: Bug
Reporter: Mohan
Assignee: Mohan
api/v1/sensor/indexing/config/list/indices/{writer} will return sensor lists
all indices for invalid writer retrieves all indices for any invalid writer
names .
{code:java}
curl -X GET --header 'Accept: application/json'
'http://nat-r7-zots-metron-1.openstacklocal:8082/api/v1/sensor/indexing/config/list/indices/1'
returns response code : 200
and response body :
[
"websphere",
"jsonMap",
"jsonMapQuery",
"squid",
"error",
"testsensor",
"jsonMapWrappedQuery",
"snort",
"asa",
"bro",
"newsensor",
"yaf"
]
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (METRON-1710) api/v1/sensor/indexing/config/list/indices/{writer} will return sensor lists all indices for invalid writer
[
https://issues.apache.org/jira/browse/METRON-1710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohan updated METRON-1710:
--
Description:
api/v1/sensor/indexing/config/list/indices/{writer} will return sensor lists
all indices for invalid writer retrieves all indices for any invalid writer
names .
{code:java}
curl -X GET --header 'Accept: application/json'
'http://node1:8082/api/v1/sensor/indexing/config/list/indices/1'
returns response code : 200
and response body :
[
"websphere",
"jsonMap",
"jsonMapQuery",
"squid",
"error",
"testsensor",
"jsonMapWrappedQuery",
"snort",
"asa",
"bro",
"newsensor",
"yaf"
]
{code}
was:
api/v1/sensor/indexing/config/list/indices/{writer} will return sensor lists
all indices for invalid writer retrieves all indices for any invalid writer
names .
{code:java}
curl -X GET --header 'Accept: application/json'
'http://nat-r7-zots-metron-1.openstacklocal:8082/api/v1/sensor/indexing/config/list/indices/1'
returns response code : 200
and response body :
[
"websphere",
"jsonMap",
"jsonMapQuery",
"squid",
"error",
"testsensor",
"jsonMapWrappedQuery",
"snort",
"asa",
"bro",
"newsensor",
"yaf"
]
{code}
> api/v1/sensor/indexing/config/list/indices/{writer} will return sensor lists
> all indices for invalid writer
>
>
> Key: METRON-1710
> URL: https://issues.apache.org/jira/browse/METRON-1710
> Project: Metron
> Issue Type: Bug
>Reporter: Mohan
>Assignee: Mohan
>Priority: Major
>
> api/v1/sensor/indexing/config/list/indices/{writer} will return sensor lists
> all indices for invalid writer retrieves all indices for any invalid writer
> names .
> {code:java}
> curl -X GET --header 'Accept: application/json'
> 'http://node1:8082/api/v1/sensor/indexing/config/list/indices/1'
>
> returns response code : 200
> and response body :
> [
> "websphere",
> "jsonMap",
> "jsonMapQuery",
> "squid",
> "error",
> "testsensor",
> "jsonMapWrappedQuery",
> "snort",
> "asa",
> "bro",
> "newsensor",
> "yaf"
> ]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] metron issue #1132: METRON-1695: Expose pcap properties through Ambari
Github user simonellistonball commented on the issue: https://github.com/apache/metron/pull/1132 +1 to that, PCAP should definitely get its own service, but agreed with @anandsubbu that should probably be a follow on item. ---
[jira] [Commented] (METRON-1695) Expose pcap properties through Ambari
[ https://issues.apache.org/jira/browse/METRON-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565153#comment-16565153 ] ASF GitHub Bot commented on METRON-1695: Github user simonellistonball commented on the issue: https://github.com/apache/metron/pull/1132 +1 to that, PCAP should definitely get its own service, but agreed with @anandsubbu that should probably be a follow on item. > Expose pcap properties through Ambari > - > > Key: METRON-1695 > URL: https://issues.apache.org/jira/browse/METRON-1695 > Project: Metron > Issue Type: Bug >Reporter: Anand Subramanian >Assignee: Anand Subramanian >Priority: Major > > Currently, the $METRON_HOME/config/pcap.properties file is hardcoded with the > defaults. One has to hand edit the file before deploying the PCAP topology. > These properties should be configurable via Ambari. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (METRON-1711) PCAP UI: Reconnecting UI to an ongoing query
Tibor Meller created METRON-1711: Summary: PCAP UI: Reconnecting UI to an ongoing query Key: METRON-1711 URL: https://issues.apache.org/jira/browse/METRON-1711 Project: Metron Issue Type: Sub-task Reporter: Tibor Meller As a user, after I login/reload and navigating to PCAP panel, I would like to see my possibly existing (currently one) query on the UI. If the query completed I would like to see the result of the job. If the query is in progress, I would like to see the progress/state of the job in the same bar when I submitted the request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (METRON-1711) PCAP UI: Reconnecting UI to an ongoing query
[ https://issues.apache.org/jira/browse/METRON-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tibor Meller updated METRON-1711: - Description: As a user, after I login/reload and navigating to PCAP panel, I would like to see my possibly existing (currently one) query on the UI. If the query completed I would like to see the result of the job. If the query is in progress, I would like to see the progress/state of the job in the same bar when I submitted the request. Backend ticket: https://issues.apache.org/jira/browse/METRON-1702 was: As a user, after I login/reload and navigating to PCAP panel, I would like to see my possibly existing (currently one) query on the UI. If the query completed I would like to see the result of the job. If the query is in progress, I would like to see the progress/state of the job in the same bar when I submitted the request. > PCAP UI: Reconnecting UI to an ongoing query > > > Key: METRON-1711 > URL: https://issues.apache.org/jira/browse/METRON-1711 > Project: Metron > Issue Type: Sub-task >Reporter: Tibor Meller >Priority: Major > > As a user, after I login/reload and navigating to PCAP panel, I would like to > see my possibly existing (currently one) query on the UI. > If the query completed I would like to see the result of the job. > If the query is in progress, I would like to see the progress/state of the > job in the same bar when I submitted the request. > > Backend ticket: https://issues.apache.org/jira/browse/METRON-1702 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron pull request #1136: METRON-1701: Update steps to run the pycapa on ke...
Github user MohanDV commented on a diff in the pull request: https://github.com/apache/metron/pull/1136#discussion_r206862766 --- Diff: metron-sensors/pycapa/README.md --- @@ -239,25 +239,47 @@ The probe can be used in a Kerberized environment. Follow these additional step * The keytab used is located at `/etc/security/keytabs/metron.headless.keytab` * The service principal is `met...@example.com` +If it is not, ensure that you have `libsasl` or `libsasl2` installed. On CentOS, this can be installed with the following command. --- End diff -- done ---
[GitHub] metron pull request #1136: METRON-1701: Update steps to run the pycapa on ke...
Github user MohanDV commented on a diff in the pull request: https://github.com/apache/metron/pull/1136#discussion_r206863111 --- Diff: metron-sensors/pycapa/README.md --- @@ -239,25 +239,47 @@ The probe can be used in a Kerberized environment. Follow these additional step * The keytab used is located at `/etc/security/keytabs/metron.headless.keytab` * The service principal is `met...@example.com` +If it is not, ensure that you have `libsasl` or `libsasl2` installed. On CentOS, this can be installed with the following command. +``` +yum install -y cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi +``` + + 1. Use the non-binary wheel to install confluent-kafka-python. --- End diff -- Moved these instruction to the top as description . ---
[jira] [Commented] (METRON-1701) Update General notes on the installation of Pycapa on Kerberized cluster
[ https://issues.apache.org/jira/browse/METRON-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565241#comment-16565241 ] ASF GitHub Bot commented on METRON-1701: Github user MohanDV commented on a diff in the pull request: https://github.com/apache/metron/pull/1136#discussion_r206862766 --- Diff: metron-sensors/pycapa/README.md --- @@ -239,25 +239,47 @@ The probe can be used in a Kerberized environment. Follow these additional step * The keytab used is located at `/etc/security/keytabs/metron.headless.keytab` * The service principal is `met...@example.com` +If it is not, ensure that you have `libsasl` or `libsasl2` installed. On CentOS, this can be installed with the following command. --- End diff -- done > Update General notes on the installation of Pycapa on Kerberized cluster > > > Key: METRON-1701 > URL: https://issues.apache.org/jira/browse/METRON-1701 > Project: Metron > Issue Type: Bug >Reporter: Mohan >Assignee: Mohan >Priority: Minor > > Update the steps to use Pycapa with Kerberos -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1701) Update General notes on the installation of Pycapa on Kerberized cluster
[ https://issues.apache.org/jira/browse/METRON-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565244#comment-16565244 ] ASF GitHub Bot commented on METRON-1701: Github user MohanDV commented on a diff in the pull request: https://github.com/apache/metron/pull/1136#discussion_r206863111 --- Diff: metron-sensors/pycapa/README.md --- @@ -239,25 +239,47 @@ The probe can be used in a Kerberized environment. Follow these additional step * The keytab used is located at `/etc/security/keytabs/metron.headless.keytab` * The service principal is `met...@example.com` +If it is not, ensure that you have `libsasl` or `libsasl2` installed. On CentOS, this can be installed with the following command. +``` +yum install -y cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi +``` + + 1. Use the non-binary wheel to install confluent-kafka-python. --- End diff -- Moved these instruction to the top as description . > Update General notes on the installation of Pycapa on Kerberized cluster > > > Key: METRON-1701 > URL: https://issues.apache.org/jira/browse/METRON-1701 > Project: Metron > Issue Type: Bug >Reporter: Mohan >Assignee: Mohan >Priority: Minor > > Update the steps to use Pycapa with Kerberos -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron pull request #1136: METRON-1701: Update steps to run the pycapa on ke...
Github user MohanDV commented on a diff in the pull request: https://github.com/apache/metron/pull/1136#discussion_r206863799 --- Diff: metron-sensors/pycapa/README.md --- @@ -239,25 +239,47 @@ The probe can be used in a Kerberized environment. Follow these additional step * The keytab used is located at `/etc/security/keytabs/metron.headless.keytab` * The service principal is `met...@example.com` +If it is not, ensure that you have `libsasl` or `libsasl2` installed. On CentOS, this can be installed with the following command. +``` +yum install -y cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi +``` + + 1. Use the non-binary wheel to install confluent-kafka-python. + +The Python client README (https://github.com/confluentinc/confluent-kafka-python) has an important note for Kerberos case: +``` +**NOTE:** The pre-built Linux wheels do NOT contain SASL Kerberos support. + If you need SASL Kerberos support you must install librdkafka and + its dependencies using the above repositories and then build + confluent-kafka from source. + ``` --- End diff -- done ---
[jira] [Commented] (METRON-1701) Update General notes on the installation of Pycapa on Kerberized cluster
[ https://issues.apache.org/jira/browse/METRON-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565246#comment-16565246 ] ASF GitHub Bot commented on METRON-1701: Github user MohanDV commented on a diff in the pull request: https://github.com/apache/metron/pull/1136#discussion_r206863799 --- Diff: metron-sensors/pycapa/README.md --- @@ -239,25 +239,47 @@ The probe can be used in a Kerberized environment. Follow these additional step * The keytab used is located at `/etc/security/keytabs/metron.headless.keytab` * The service principal is `met...@example.com` +If it is not, ensure that you have `libsasl` or `libsasl2` installed. On CentOS, this can be installed with the following command. +``` +yum install -y cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi +``` + + 1. Use the non-binary wheel to install confluent-kafka-python. + +The Python client README (https://github.com/confluentinc/confluent-kafka-python) has an important note for Kerberos case: +``` +**NOTE:** The pre-built Linux wheels do NOT contain SASL Kerberos support. + If you need SASL Kerberos support you must install librdkafka and + its dependencies using the above repositories and then build + confluent-kafka from source. + ``` --- End diff -- done > Update General notes on the installation of Pycapa on Kerberized cluster > > > Key: METRON-1701 > URL: https://issues.apache.org/jira/browse/METRON-1701 > Project: Metron > Issue Type: Bug >Reporter: Mohan >Assignee: Mohan >Priority: Minor > > Update the steps to use Pycapa with Kerberos -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron pull request #1136: METRON-1701: Update steps to run the pycapa on ke...
Github user MohanDV commented on a diff in the pull request: https://github.com/apache/metron/pull/1136#discussion_r206864051 --- Diff: metron-sensors/pycapa/README.md --- @@ -54,8 +54,8 @@ General notes on the installation of Pycapa. ``` export PREFIX=/usr -wget https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz -O - | tar -xz -cd librdkafka-0.9.4/ +wget https://github.com/edenhill/librdkafka/archive/v0.11.5.tar.gz -O - | tar -xz +cd librdkafka-0.11.5/ --- End diff -- Yes it works for unsecured deployment as well. ---
[jira] [Commented] (METRON-1701) Update General notes on the installation of Pycapa on Kerberized cluster
[ https://issues.apache.org/jira/browse/METRON-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565247#comment-16565247 ] ASF GitHub Bot commented on METRON-1701: Github user MohanDV commented on a diff in the pull request: https://github.com/apache/metron/pull/1136#discussion_r206864051 --- Diff: metron-sensors/pycapa/README.md --- @@ -54,8 +54,8 @@ General notes on the installation of Pycapa. ``` export PREFIX=/usr -wget https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz -O - | tar -xz -cd librdkafka-0.9.4/ +wget https://github.com/edenhill/librdkafka/archive/v0.11.5.tar.gz -O - | tar -xz +cd librdkafka-0.11.5/ --- End diff -- Yes it works for unsecured deployment as well. > Update General notes on the installation of Pycapa on Kerberized cluster > > > Key: METRON-1701 > URL: https://issues.apache.org/jira/browse/METRON-1701 > Project: Metron > Issue Type: Bug >Reporter: Mohan >Assignee: Mohan >Priority: Minor > > Update the steps to use Pycapa with Kerberos -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1136: METRON-1701: Update steps to run the pycapa on kerberize...
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1136 +1 This looks great. Thanks for making these updates. Very useful! ---
[jira] [Commented] (METRON-1701) Update General notes on the installation of Pycapa on Kerberized cluster
[ https://issues.apache.org/jira/browse/METRON-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565279#comment-16565279 ] ASF GitHub Bot commented on METRON-1701: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1136 +1 This looks great. Thanks for making these updates. Very useful! > Update General notes on the installation of Pycapa on Kerberized cluster > > > Key: METRON-1701 > URL: https://issues.apache.org/jira/browse/METRON-1701 > Project: Metron > Issue Type: Bug >Reporter: Mohan >Assignee: Mohan >Priority: Minor > > Update the steps to use Pycapa with Kerberos -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1683) PCAP UI - Fix the download progress bar
[ https://issues.apache.org/jira/browse/METRON-1683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565307#comment-16565307 ] ASF GitHub Bot commented on METRON-1683: Github user sardell commented on the issue: https://github.com/apache/metron/pull/1122 @merrimanr I've increased test coverage in the pcap service to cover all functions there. I also removed that large set of test pdml data and replaced it with mock data. > PCAP UI - Fix the download progress bar > --- > > Key: METRON-1683 > URL: https://issues.apache.org/jira/browse/METRON-1683 > Project: Metron > Issue Type: Sub-task >Reporter: Shane Ardell >Priority: Major > > The download progress bar currently does not appear when a PCAP search is > submitted. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1122: METRON-1683: Fix the download progress bar in PCAP UI
Github user sardell commented on the issue: https://github.com/apache/metron/pull/1122 @merrimanr I've increased test coverage in the pcap service to cover all functions there. I also removed that large set of test pdml data and replaced it with mock data. ---
[jira] [Deleted] (METRON-1711) PCAP UI: Reconnecting UI to an ongoing query
[ https://issues.apache.org/jira/browse/METRON-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Merriman deleted METRON-1711: -- > PCAP UI: Reconnecting UI to an ongoing query > > > Key: METRON-1711 > URL: https://issues.apache.org/jira/browse/METRON-1711 > Project: Metron > Issue Type: Sub-task >Reporter: Tibor Meller >Priority: Major > > As a user, after I login/reload and navigating to PCAP panel, I would like to > see my possibly existing (currently one) query on the UI. > If the query completed I would like to see the result of the job. > If the query is in progress, I would like to see the progress/state of the > job in the same bar when I submitted the request. > > Backend ticket: https://issues.apache.org/jira/browse/METRON-1702 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (METRON-1712) PCAP UI - Input validation
Shane Ardell created METRON-1712: Summary: PCAP UI - Input validation Key: METRON-1712 URL: https://issues.apache.org/jira/browse/METRON-1712 Project: Metron Issue Type: Sub-task Reporter: Shane Ardell The PCAP panel currently has input validation on the date, ip address and port number fields. What validation requirements are left for the remaining fields? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (METRON-1713) PCAP UI - Add a way to kill a pcap job
Shane Ardell created METRON-1713: Summary: PCAP UI - Add a way to kill a pcap job Key: METRON-1713 URL: https://issues.apache.org/jira/browse/METRON-1713 Project: Metron Issue Type: Sub-task Reporter: Shane Ardell Attachments: Screen Shot 2018-08-01 at 4.04.58 PM.png It looks like there is an endpoint to kill a running pcap query job, but we don't have a way in the UI to do so. Can we add a button to kill a running job to the UI? Attached image is from Swagger after building full-dev from the feature branch from METRON-1554. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (METRON-1715) Create DEB Packaging for Batch Profiler
Nick Allen created METRON-1715: -- Summary: Create DEB Packaging for Batch Profiler Key: METRON-1715 URL: https://issues.apache.org/jira/browse/METRON-1715 Project: Metron Issue Type: Sub-task Reporter: Nick Allen Create DEB packaging for the Batch Profiler -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (METRON-1714) Create RPM Packaging for the Batch Profiler
Nick Allen created METRON-1714: -- Summary: Create RPM Packaging for the Batch Profiler Key: METRON-1714 URL: https://issues.apache.org/jira/browse/METRON-1714 Project: Metron Issue Type: Sub-task Reporter: Nick Allen Create RPM packaging for the Batch Profiler -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (METRON-1716) MPack Support for the Batch Profiler
Nick Allen created METRON-1716: -- Summary: MPack Support for the Batch Profiler Key: METRON-1716 URL: https://issues.apache.org/jira/browse/METRON-1716 Project: Metron Issue Type: Sub-task Reporter: Nick Allen -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (METRON-1717) Relocate Storm Profiler Code
Nick Allen created METRON-1717: -- Summary: Relocate Storm Profiler Code Key: METRON-1717 URL: https://issues.apache.org/jira/browse/METRON-1717 Project: Metron Issue Type: Sub-task Reporter: Nick Allen The Storm port of the Profiler currently lives in metron-analytics/metron-profiler. This should be moved to metron-analytics/metron-profiler-storm. This would mirror the project name for the Spark port; matron-profiler-spark. The package name for the Storm port of the Profiler should be changed to org.apache.metron.profiler.storm. This would mimic the package name used for Spark; org.apache.metron.profiler.spark. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (METRON-1718) Add tests to handle different input parameter values
Shane Ardell created METRON-1718: Summary: Add tests to handle different input parameter values Key: METRON-1718 URL: https://issues.apache.org/jira/browse/METRON-1718 Project: Metron Issue Type: Sub-task Reporter: Shane Ardell The parameters of a pcap request differ if you've never entered a field vs. entered then deleted it. For example, for ports (and possibly just the two port fields), it'll be empty string if never entered and null if entered then deleted. We should have tests to ensure being provided either works as expected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1103: METRON-1671: Initial PCAP UI
Github user sardell commented on the issue: https://github.com/apache/metron/pull/1103 @justinleet Here are the tickets for the follow-on items: * Input validation: https://issues.apache.org/jira/browse/METRON-1712 * Kill button for in progress jobs: https://issues.apache.org/jira/browse/METRON-1713 * Testing different parameter values: https://issues.apache.org/jira/browse/METRON-1718 I believe that's all the tickets that need to be created stemming from the discussion above. ---
[jira] [Commented] (METRON-1671) Create PCAP UI
[ https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565400#comment-16565400 ] ASF GitHub Bot commented on METRON-1671: Github user sardell commented on the issue: https://github.com/apache/metron/pull/1103 @justinleet Here are the tickets for the follow-on items: * Input validation: https://issues.apache.org/jira/browse/METRON-1712 * Kill button for in progress jobs: https://issues.apache.org/jira/browse/METRON-1713 * Testing different parameter values: https://issues.apache.org/jira/browse/METRON-1718 I believe that's all the tickets that need to be created stemming from the discussion above. > Create PCAP UI > -- > > Key: METRON-1671 > URL: https://issues.apache.org/jira/browse/METRON-1671 > Project: Metron > Issue Type: Sub-task >Reporter: Tibor Meller >Priority: Major > > As a user, I like to submit PCAP query requests from the Metron UI. > As a user, I like to be able to narrow down the result set of the query by > the following filters: > - IP Source Address > - IP Source Port > - IP Dest Address > - IP Dest Port > - Protocol > - Include Reverse Traffic > - Free text filtering > As a user, I like to get feedback from the progress of the ongoing request by > a progress bar. > As a user, I like to see the result of the query in a grid. Each line of the > grid should show and extendable packet record. > As a user, when I clicking on a row of the result grid I would like to see > additional information about that particular packet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1718) Add tests to handle different input parameter values
[ https://issues.apache.org/jira/browse/METRON-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565404#comment-16565404 ] Justin Leet commented on METRON-1718: - Both of these should be acceptable inputs from the front-end. The backend should make sure functionality is the same (and as far as I've seen it is), so this is to make sure we don't break that down the line. > Add tests to handle different input parameter values > > > Key: METRON-1718 > URL: https://issues.apache.org/jira/browse/METRON-1718 > Project: Metron > Issue Type: Sub-task >Reporter: Shane Ardell >Priority: Major > > The parameters of a pcap request differ if you've never entered a field vs. > entered then deleted it. For example, for ports (and possibly just the two > port fields), it'll be empty string if never entered and null if entered then > deleted. > We should have tests to ensure being provided either works as expected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1103: METRON-1671: Initial PCAP UI
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1103 @sardell Thanks for adding the tickets. +1, @tiborm Thanks for your patience during the whole review process. I'm looking forward to getting the ball rolling on the rest of the contributions! ---
[jira] [Commented] (METRON-1671) Create PCAP UI
[ https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565408#comment-16565408 ] ASF GitHub Bot commented on METRON-1671: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1103 @sardell Thanks for adding the tickets. +1, @tiborm Thanks for your patience during the whole review process. I'm looking forward to getting the ball rolling on the rest of the contributions! > Create PCAP UI > -- > > Key: METRON-1671 > URL: https://issues.apache.org/jira/browse/METRON-1671 > Project: Metron > Issue Type: Sub-task >Reporter: Tibor Meller >Priority: Major > > As a user, I like to submit PCAP query requests from the Metron UI. > As a user, I like to be able to narrow down the result set of the query by > the following filters: > - IP Source Address > - IP Source Port > - IP Dest Address > - IP Dest Port > - Protocol > - Include Reverse Traffic > - Free text filtering > As a user, I like to get feedback from the progress of the ongoing request by > a progress bar. > As a user, I like to see the result of the query in a grid. Each line of the > grid should show and extendable packet record. > As a user, when I clicking on a row of the result grid I would like to see > additional information about that particular packet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron pull request #1136: METRON-1701: Update steps to run the pycapa on ke...
Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/1136 ---
[jira] [Commented] (METRON-1701) Update General notes on the installation of Pycapa on Kerberized cluster
[ https://issues.apache.org/jira/browse/METRON-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565569#comment-16565569 ] ASF GitHub Bot commented on METRON-1701: Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/1136 > Update General notes on the installation of Pycapa on Kerberized cluster > > > Key: METRON-1701 > URL: https://issues.apache.org/jira/browse/METRON-1701 > Project: Metron > Issue Type: Bug >Reporter: Mohan >Assignee: Mohan >Priority: Minor > > Update the steps to use Pycapa with Kerberos -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1718) Add tests to handle different input parameter values
[ https://issues.apache.org/jira/browse/METRON-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565581#comment-16565581 ] Michael Miklavcic commented on METRON-1718: --- Would the plan be to normalize the user's input? Backend should/does handle it either way, but it probably makes sense to have the UI choose either empty or null and handle it consistently. > Add tests to handle different input parameter values > > > Key: METRON-1718 > URL: https://issues.apache.org/jira/browse/METRON-1718 > Project: Metron > Issue Type: Sub-task >Reporter: Shane Ardell >Priority: Major > > The parameters of a pcap request differ if you've never entered a field vs. > entered then deleted it. For example, for ports (and possibly just the two > port fields), it'll be empty string if never entered and null if entered then > deleted. > We should have tests to ensure being provided either works as expected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1103: METRON-1671: Initial PCAP UI
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1103 Looks like all comments have been addressed, reviewers have been able to test successfully, and any remaining follow-on work has been added to the feature branch Jira/Epic. I'm +1 by inspection. Nice work gentlemen. ---
[jira] [Commented] (METRON-1671) Create PCAP UI
[ https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565585#comment-16565585 ] ASF GitHub Bot commented on METRON-1671: Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1103 Looks like all comments have been addressed, reviewers have been able to test successfully, and any remaining follow-on work has been added to the feature branch Jira/Epic. I'm +1 by inspection. Nice work gentlemen. > Create PCAP UI > -- > > Key: METRON-1671 > URL: https://issues.apache.org/jira/browse/METRON-1671 > Project: Metron > Issue Type: Sub-task >Reporter: Tibor Meller >Priority: Major > > As a user, I like to submit PCAP query requests from the Metron UI. > As a user, I like to be able to narrow down the result set of the query by > the following filters: > - IP Source Address > - IP Source Port > - IP Dest Address > - IP Dest Port > - Protocol > - Include Reverse Traffic > - Free text filtering > As a user, I like to get feedback from the progress of the ongoing request by > a progress bar. > As a user, I like to see the result of the query in a grid. Each line of the > grid should show and extendable packet record. > As a user, when I clicking on a row of the result grid I would like to see > additional information about that particular packet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1671) Create PCAP UI
[ https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565611#comment-16565611 ] ASF GitHub Bot commented on METRON-1671: Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/1103 This has been merged into the feature branch. Can you close it @tiborm? > Create PCAP UI > -- > > Key: METRON-1671 > URL: https://issues.apache.org/jira/browse/METRON-1671 > Project: Metron > Issue Type: Sub-task >Reporter: Tibor Meller >Priority: Major > > As a user, I like to submit PCAP query requests from the Metron UI. > As a user, I like to be able to narrow down the result set of the query by > the following filters: > - IP Source Address > - IP Source Port > - IP Dest Address > - IP Dest Port > - Protocol > - Include Reverse Traffic > - Free text filtering > As a user, I like to get feedback from the progress of the ongoing request by > a progress bar. > As a user, I like to see the result of the query in a grid. Each line of the > grid should show and extendable packet record. > As a user, when I clicking on a row of the result grid I would like to see > additional information about that particular packet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1103: METRON-1671: Initial PCAP UI
Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/1103 This has been merged into the feature branch. Can you close it @tiborm? ---
[jira] [Commented] (METRON-1718) Add tests to handle different input parameter values
[ https://issues.apache.org/jira/browse/METRON-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565636#comment-16565636 ] Justin Leet commented on METRON-1718: - Ideally, I think the UI handles it consistently, but as long as the backend handles both reasonably, I personally consider it fairly low priority. Having said that if someone wants to pick it up, I'm not going to complain at all. > Add tests to handle different input parameter values > > > Key: METRON-1718 > URL: https://issues.apache.org/jira/browse/METRON-1718 > Project: Metron > Issue Type: Sub-task >Reporter: Shane Ardell >Priority: Major > > The parameters of a pcap request differ if you've never entered a field vs. > entered then deleted it. For example, for ports (and possibly just the two > port fields), it'll be empty string if never entered and null if entered then > deleted. > We should have tests to ensure being provided either works as expected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1132: METRON-1695: Expose pcap properties through Ambari
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1132 I think I'm ok with this as a first step. Please document the lifecycle for how pcap.properties is updated in the README. It should be clear to users that they will need to restart parsers for the changes to take effect after the initial install is performed. Can you confirm, is that the way the properties are/will be deployed with this PR @anandsubbu? https://github.com/apache/metron/pull/1134 should be included in the refactoring once we make PCAP a proper component. ---
[jira] [Commented] (METRON-1695) Expose pcap properties through Ambari
[ https://issues.apache.org/jira/browse/METRON-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565683#comment-16565683 ] ASF GitHub Bot commented on METRON-1695: Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1132 I think I'm ok with this as a first step. Please document the lifecycle for how pcap.properties is updated in the README. It should be clear to users that they will need to restart parsers for the changes to take effect after the initial install is performed. Can you confirm, is that the way the properties are/will be deployed with this PR @anandsubbu? https://github.com/apache/metron/pull/1134 should be included in the refactoring once we make PCAP a proper component. > Expose pcap properties through Ambari > - > > Key: METRON-1695 > URL: https://issues.apache.org/jira/browse/METRON-1695 > Project: Metron > Issue Type: Bug >Reporter: Anand Subramanian >Assignee: Anand Subramanian >Priority: Major > > Currently, the $METRON_HOME/config/pcap.properties file is hardcoded with the > defaults. One has to hand edit the file before deploying the PCAP topology. > These properties should be configurable via Ambari. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1134: METRON-1696: Create the HDFS directory for pcap sequence...
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1134 See comment here - https://github.com/apache/metron/pull/1132#issuecomment-409656807 I'm +1 via inspection pending a follow-on Jira being created, having it linked to this Jira, and also linked for reference in the comments on this PR. ---
[jira] [Commented] (METRON-1696) Pcap parser fails to write pacap sequence file to hdfs on kerberized cluster
[
https://issues.apache.org/jira/browse/METRON-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565687#comment-16565687
]
ASF GitHub Bot commented on METRON-1696:
Github user mmiklavc commented on the issue:
https://github.com/apache/metron/pull/1134
See comment here -
https://github.com/apache/metron/pull/1132#issuecomment-409656807
I'm +1 via inspection pending a follow-on Jira being created, having it
linked to this Jira, and also linked for reference in the comments on this PR.
> Pcap parser fails to write pacap sequence file to hdfs on kerberized cluster
> -
>
> Key: METRON-1696
> URL: https://issues.apache.org/jira/browse/METRON-1696
> Project: Metron
> Issue Type: Bug
>Reporter: Mohan
>Assignee: Mohan
>Priority: Major
>
> pcap parser fails to write the pcap sequence files to hdfs directory due to
> insufficient privileges to hdfs folder for 'metron' user
> {code:java}
> 2018-07-25 10:15:50.035 o.a.m.s.p.HDFSWriterCallback
> Thread-9-kafkaSpout-executor[3 3] [ERROR] Permission denied: user=metron,
> access=WRITE,
> inode="/apps/metron/pcap/pcap_pcap_1532513746365022000_0_pcap-20-1532414055":hdfs:hdfs:drwxr-xr-x
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:353)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:325)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:246)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1950)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1934)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1917)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:2767)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2702)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2586)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:736)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:409)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2347)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] metron issue #1132: METRON-1695: Expose pcap properties through Ambari
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1132 I'm +1 via inspection pending the requested documentation. Also, please create the follow-on Jira, have it linked to this Jira, and also link it here for reference in the comments on this PR. Both this Jira/PR and https://github.com/apache/metron/pull/1134#issuecomment-409658527 should link up. ---
[jira] [Commented] (METRON-1695) Expose pcap properties through Ambari
[ https://issues.apache.org/jira/browse/METRON-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565693#comment-16565693 ] ASF GitHub Bot commented on METRON-1695: Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1132 I'm +1 via inspection pending the requested documentation. Also, please create the follow-on Jira, have it linked to this Jira, and also link it here for reference in the comments on this PR. Both this Jira/PR and https://github.com/apache/metron/pull/1134#issuecomment-409658527 should link up. > Expose pcap properties through Ambari > - > > Key: METRON-1695 > URL: https://issues.apache.org/jira/browse/METRON-1695 > Project: Metron > Issue Type: Bug >Reporter: Anand Subramanian >Assignee: Anand Subramanian >Priority: Major > > Currently, the $METRON_HOME/config/pcap.properties file is hardcoded with the > defaults. One has to hand edit the file before deploying the PCAP topology. > These properties should be configurable via Ambari. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1132: METRON-1695: Expose pcap properties through Ambari
Github user anandsubbu commented on the issue: https://github.com/apache/metron/pull/1132 > Please document the lifecycle for how pcap.properties is updated in the README. It should be clear to users that they will need to restart parsers for the changes to take effect after the initial install is performed. Can you confirm, is that the way the properties are/will be deployed with this PR @anandsubbu? Hi @mmiklavc - there is no change in behavior from earlier to now. Nothing changes from a usage perspective. Let me clarify more with an example for a multi-node deployment... **Earlier** * User deploys metron * `pcap.properties` file created with defaults under `$METRON_HOME/config` * User hand edits the properties file (sets ZK quorum and other parameters as required) * Starts the topology per the instructions [here](https://github.com/apache/metron/tree/master/metron-platform/metron-pcap-backend#starting-the-topology). **Now** * During deploy time, user is presented with a separate config tab in Ambari to configure/reconfigure PCAP properties. If user chooses to leave them untouched, the pcap.properties is initialized with appropriate defaults. ZK quorum is auto-populated as well. * `pcap.properties` file is created during metron-parsers startup step. * Starts the topology per the instructions [here](https://github.com/apache/metron/tree/master/metron-platform/metron-pcap-backend#starting-the-topology). The steps in the [README](https://github.com/apache/metron/tree/master/metron-platform/metron-pcap-backend) still holds. Now, we can add a note in the README to indicate that the PCAP properties can be configured via the 'PCAP' tab in Ambari Metron config. But I noticed this was not explicitly mentioned for other components (e.g. [REST](https://github.com/apache/metron/tree/master/metron-interface/metron-rest#configuration), [elasticsearch](https://github.com/apache/metron/tree/master/metron-platform/metron-elasticsearch#properties)). Let me know if you prefer to have this added. ---
[jira] [Commented] (METRON-1695) Expose pcap properties through Ambari
[ https://issues.apache.org/jira/browse/METRON-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565739#comment-16565739 ] ASF GitHub Bot commented on METRON-1695: Github user anandsubbu commented on the issue: https://github.com/apache/metron/pull/1132 > Please document the lifecycle for how pcap.properties is updated in the README. It should be clear to users that they will need to restart parsers for the changes to take effect after the initial install is performed. Can you confirm, is that the way the properties are/will be deployed with this PR @anandsubbu? Hi @mmiklavc - there is no change in behavior from earlier to now. Nothing changes from a usage perspective. Let me clarify more with an example for a multi-node deployment... **Earlier** * User deploys metron * `pcap.properties` file created with defaults under `$METRON_HOME/config` * User hand edits the properties file (sets ZK quorum and other parameters as required) * Starts the topology per the instructions [here](https://github.com/apache/metron/tree/master/metron-platform/metron-pcap-backend#starting-the-topology). **Now** * During deploy time, user is presented with a separate config tab in Ambari to configure/reconfigure PCAP properties. If user chooses to leave them untouched, the pcap.properties is initialized with appropriate defaults. ZK quorum is auto-populated as well. * `pcap.properties` file is created during metron-parsers startup step. * Starts the topology per the instructions [here](https://github.com/apache/metron/tree/master/metron-platform/metron-pcap-backend#starting-the-topology). The steps in the [README](https://github.com/apache/metron/tree/master/metron-platform/metron-pcap-backend) still holds. Now, we can add a note in the README to indicate that the PCAP properties can be configured via the 'PCAP' tab in Ambari Metron config. But I noticed this was not explicitly mentioned for other components (e.g. [REST](https://github.com/apache/metron/tree/master/metron-interface/metron-rest#configuration), [elasticsearch](https://github.com/apache/metron/tree/master/metron-platform/metron-elasticsearch#properties)). Let me know if you prefer to have this added. > Expose pcap properties through Ambari > - > > Key: METRON-1695 > URL: https://issues.apache.org/jira/browse/METRON-1695 > Project: Metron > Issue Type: Bug >Reporter: Anand Subramanian >Assignee: Anand Subramanian >Priority: Major > > Currently, the $METRON_HOME/config/pcap.properties file is hardcoded with the > defaults. One has to hand edit the file before deploying the PCAP topology. > These properties should be configurable via Ambari. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron pull request #1103: METRON-1671: Initial PCAP UI
Github user tiborm closed the pull request at: https://github.com/apache/metron/pull/1103 ---
[GitHub] metron issue #1103: METRON-1671: Initial PCAP UI
Github user tiborm commented on the issue: https://github.com/apache/metron/pull/1103 Thanks to everyone for the feedback and comments! ---
[jira] [Commented] (METRON-1671) Create PCAP UI
[ https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565764#comment-16565764 ] ASF GitHub Bot commented on METRON-1671: Github user tiborm commented on the issue: https://github.com/apache/metron/pull/1103 Thanks to everyone for the feedback and comments! > Create PCAP UI > -- > > Key: METRON-1671 > URL: https://issues.apache.org/jira/browse/METRON-1671 > Project: Metron > Issue Type: Sub-task >Reporter: Tibor Meller >Priority: Major > > As a user, I like to submit PCAP query requests from the Metron UI. > As a user, I like to be able to narrow down the result set of the query by > the following filters: > - IP Source Address > - IP Source Port > - IP Dest Address > - IP Dest Port > - Protocol > - Include Reverse Traffic > - Free text filtering > As a user, I like to get feedback from the progress of the ongoing request by > a progress bar. > As a user, I like to see the result of the query in a grid. Each line of the > grid should show and extendable packet record. > As a user, when I clicking on a row of the result grid I would like to see > additional information about that particular packet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1671) Create PCAP UI
[ https://issues.apache.org/jira/browse/METRON-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565766#comment-16565766 ] ASF GitHub Bot commented on METRON-1671: Github user tiborm closed the pull request at: https://github.com/apache/metron/pull/1103 > Create PCAP UI > -- > > Key: METRON-1671 > URL: https://issues.apache.org/jira/browse/METRON-1671 > Project: Metron > Issue Type: Sub-task >Reporter: Tibor Meller >Priority: Major > > As a user, I like to submit PCAP query requests from the Metron UI. > As a user, I like to be able to narrow down the result set of the query by > the following filters: > - IP Source Address > - IP Source Port > - IP Dest Address > - IP Dest Port > - Protocol > - Include Reverse Traffic > - Free text filtering > As a user, I like to get feedback from the progress of the ongoing request by > a progress bar. > As a user, I like to see the result of the query in a grid. Each line of the > grid should show and extendable packet record. > As a user, when I clicking on a row of the result grid I would like to see > additional information about that particular packet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (METRON-1719) PCAP sensor should have its own service startup scripts
Anand Subramanian created METRON-1719: - Summary: PCAP sensor should have its own service startup scripts Key: METRON-1719 URL: https://issues.apache.org/jira/browse/METRON-1719 Project: Metron Issue Type: Bug Reporter: Anand Subramanian Presently, the PCAP sensor needs to be started from the command like from $METRON_HOME/bin/start_pcap_topology.sh, unlike the other parsers which use a different startup script. PCAP sensor needs to have its own startup and service monitoring scripts, similar to the other parsers. It needs to be able to be started/stopped from Ambari and the Management UI like the other parsers. IMHO, the stock startup status of the PCAP sensor on full-dev environment should be set to 'Stopped' state, since keeping it turned ON would only burden (the already burdened) full-dev deployment further. This point is however open for discussion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1132: METRON-1695: Expose pcap properties through Ambari
Github user anandsubbu commented on the issue: https://github.com/apache/metron/pull/1132 Btw, I created [METRON-1719](https://issues.apache.org/jira/browse/METRON-1719) to track PCAP sensor being an independent service entity. ---
[jira] [Commented] (METRON-1695) Expose pcap properties through Ambari
[ https://issues.apache.org/jira/browse/METRON-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565803#comment-16565803 ] ASF GitHub Bot commented on METRON-1695: Github user anandsubbu commented on the issue: https://github.com/apache/metron/pull/1132 Btw, I created [METRON-1719](https://issues.apache.org/jira/browse/METRON-1719) to track PCAP sensor being an independent service entity. > Expose pcap properties through Ambari > - > > Key: METRON-1695 > URL: https://issues.apache.org/jira/browse/METRON-1695 > Project: Metron > Issue Type: Bug >Reporter: Anand Subramanian >Assignee: Anand Subramanian >Priority: Major > > Currently, the $METRON_HOME/config/pcap.properties file is hardcoded with the > defaults. One has to hand edit the file before deploying the PCAP topology. > These properties should be configurable via Ambari. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1132: METRON-1695: Expose pcap properties through Ambari
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1132 @anandsubbu, considering the changes in this PR: 1. Let's say I manually make changes to `pcap.properties`. I restart the parser topologies. Later, I manually restart pcap with the shell command. What happens to my `pcap.properties` changes? 2. Similarly, I make changes via Ambari's new PCAP panel. Now I go start pcap using `$METRON_HOME/bin/start_pcap_topology.sh`. What does pcap.properties look like? In 1, your changes are overwritten by the parsers' restart. In 2, you've made changes in Ambari but they don't get materialized in the properties file because you haven't restarted the parsers - I may be mistaken, but I'm pretty sure clicking "save" in Ambari does not deploy the new properties, and this is the reason I ask for a doc change. Previously, configuring pcap was all manual steps. Now it will be a combo of manual steps for start/stop, automated steps for the config updates. And more specifically for the config updates, you will *only* see them if you make them in Ambari and restart the parsers. Also, if you choose not to use Ambari for this config handling and instead make your config changes to pcap.properties locally, you will have to remember that restarting the parsers will overwrite your pcap config - something you're only likely to realize the next time you start/restart pcap. This is the behavior that I think we should document if we accept this. Does that make s ense? ---
[jira] [Commented] (METRON-1695) Expose pcap properties through Ambari
[ https://issues.apache.org/jira/browse/METRON-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565898#comment-16565898 ] ASF GitHub Bot commented on METRON-1695: Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1132 @anandsubbu, considering the changes in this PR: 1. Let's say I manually make changes to `pcap.properties`. I restart the parser topologies. Later, I manually restart pcap with the shell command. What happens to my `pcap.properties` changes? 2. Similarly, I make changes via Ambari's new PCAP panel. Now I go start pcap using `$METRON_HOME/bin/start_pcap_topology.sh`. What does pcap.properties look like? In 1, your changes are overwritten by the parsers' restart. In 2, you've made changes in Ambari but they don't get materialized in the properties file because you haven't restarted the parsers - I may be mistaken, but I'm pretty sure clicking "save" in Ambari does not deploy the new properties, and this is the reason I ask for a doc change. Previously, configuring pcap was all manual steps. Now it will be a combo of manual steps for start/stop, automated steps for the config updates. And more specifically for the config updates, you will *only* see them if you make them in Ambari and restart the parsers. Also, if you choose not to use Ambari for this config handling and instead make your config changes to pcap.properties locally, you will have to remember that restarting the parsers will overwrite your pcap config - something you're only likely to realize the next time you start/restart pcap. This is the behavior that I think we should document if we accept this. Does that make sense? > Expose pcap properties through Ambari > - > > Key: METRON-1695 > URL: https://issues.apache.org/jira/browse/METRON-1695 > Project: Metron > Issue Type: Bug >Reporter: Anand Subramanian >Assignee: Anand Subramanian >Priority: Major > > Currently, the $METRON_HOME/config/pcap.properties file is hardcoded with the > defaults. One has to hand edit the file before deploying the PCAP topology. > These properties should be configurable via Ambari. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (METRON-1719) PCAP sensor should have its own service startup scripts
[ https://issues.apache.org/jira/browse/METRON-1719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohan reassigned METRON-1719: - Assignee: Mohan > PCAP sensor should have its own service startup scripts > --- > > Key: METRON-1719 > URL: https://issues.apache.org/jira/browse/METRON-1719 > Project: Metron > Issue Type: Bug >Reporter: Anand Subramanian >Assignee: Mohan >Priority: Major > > Presently, the PCAP sensor needs to be started from the command like from > $METRON_HOME/bin/start_pcap_topology.sh, unlike the other parsers which use a > different startup script. > PCAP sensor needs to have its own startup and service monitoring scripts, > similar to the other parsers. It needs to be able to be started/stopped from > Ambari and the Management UI like the other parsers. > IMHO, the stock startup status of the PCAP sensor on full-dev environment > should be set to 'Stopped' state, since keeping it turned ON would only > burden (the already burdened) full-dev deployment further. This point is > however open for discussion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1719) PCAP sensor should have its own service startup scripts
[ https://issues.apache.org/jira/browse/METRON-1719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16566359#comment-16566359 ] Anand Subramanian commented on METRON-1719: --- Dupe of METRON-1709 > PCAP sensor should have its own service startup scripts > --- > > Key: METRON-1719 > URL: https://issues.apache.org/jira/browse/METRON-1719 > Project: Metron > Issue Type: Bug >Reporter: Anand Subramanian >Assignee: Mohan >Priority: Major > > Presently, the PCAP sensor needs to be started from the command like from > $METRON_HOME/bin/start_pcap_topology.sh, unlike the other parsers which use a > different startup script. > PCAP sensor needs to have its own startup and service monitoring scripts, > similar to the other parsers. It needs to be able to be started/stopped from > Ambari and the Management UI like the other parsers. > IMHO, the stock startup status of the PCAP sensor on full-dev environment > should be set to 'Stopped' state, since keeping it turned ON would only > burden (the already burdened) full-dev deployment further. This point is > however open for discussion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
