[jira] (METRON-680) GeoLiteDatabase incorrectly using country geoname_id instead of city
Title: Message Title James Sirota commented on METRON-680 Re: GeoLiteDatabase incorrectly using country geoname_id instead of city The kibana dashboards feed off the lat/lon coordinates. This field is used primarily for log correlation. We include the locID for convenience, but it's not a hard requirement to have it. We definitely don't want use city else use country. We should be consistent. If city doesn't exist then we should just not have anything there Add Comment This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d)
[jira] (METRON-680) GeoLiteDatabase incorrectly using country geoname_id instead of city
Title: Message Title James Sirota updated an issue Metron / METRON-680 GeoLiteDatabase incorrectly using country geoname_id instead of city Change By: James Sirota Priority: Major Minor Add Comment This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d)
[jira] [Updated] (METRON-600) Fix Metron Website
[ https://issues.apache.org/jira/browse/METRON-600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-600: Assignee: Ryan Merriman (was: James Sirota) > Fix Metron Website > -- > > Key: METRON-600 > URL: https://issues.apache.org/jira/browse/METRON-600 > Project: Metron > Issue Type: Improvement >Reporter: James Sirota >Assignee: Ryan Merriman > > Issue 1 > Podling web sites MUST include a clear disclaimer on their website and in all > documentation (including releases) stating that they are in incubation. > Podlings SHOULD use the following text for all disclaimers (replace the > underlined phrases as appropriate): > Apache Podling-Name is an effort undergoing incubation at The Apache Software > Foundation (ASF), sponsored by the name of Apache TLP sponsor. Incubation is > required of all newly accepted projects until a further review indicates that > the infrastructure, communications, and decision making process have > stabilized in a manner consistent with other successful ASF projects. While > incubation status is not necessarily a reflection of the completeness or > stability of the code, it does indicate that the project has yet to be fully > endorsed by the ASF. > Issue 2: > Podlings websites SHOULD contain the Apache Incubator Project logo as sign of > affiliation > Apache Project Web Sites typically include several standard pages. Each page > is formatted with a navigation bar on the left and a project standard header > that includes the Incubator graphic. > [We need to make the Logo more prominent and move towards the top of the page > rather than having it on the bottom like we do] > Issue 3: > The sources for every podling site sources should be maintained in the > podling's site SVN or git directory > [A downloads page needs to be created with links per release. The link to > the artifact needs to be using the mirror site for apache. For example, the > 0.3.0 release would be > http://www.apache.org/dyn/closer.lua/incubator/metron/0.3.0/apache-metron-0.3.0-incubating.tar.gz. > The MD5, SHA and Signature can be from the apache release site > Look at the storm page as an example: > http://storm.apache.org/downloads.html] > Issue 4 > [Lets try to conform as much as possible to the following suggested template] > Project Home Page: the primary entry point to the site; contains project > description, news, invitation to join the project. > [We have this, great] > License Page: usually, the Apache License 2.0 > [We don't have this, we should probably put it under the about page] > Downloads: many projects in incubation will release code, and this page > describes them and has links to the download pages that redirect to Apache > Mirror sites. > [We have this, great] > Documentation: this page describes the project documentation, including > javadoc for Java projects; guides, tutorials, and links to external > documentation. > [We should probably just link to the wiki so we don't have to maintain this > in two places] > Committers: a list of current committers on the project. > [We need to update this from our status page that can be found here. Need to > make sure both are consistent. > http://incubator.apache.org/projects/metron.html > ] > Mailing Lists: there are several mailing lists that the community might be > interested in, and this page contains mailto: links that allow easy > subscription (and unsubscription) to any of them. > [We should probably put this under our community page and also link to the > apache status page to make sure it's consistent] > FAQ: frequently asked questions are answered here. > [We probably want to link to the wiki for this, as they would be easier to > update this way] > Road Map: if the project has a vision of future community or development > activities, the road map is published here. > [We should probably link to our Jira collection for the next release] > Source Code: links to the browsable source repository and svn commands to > check out the sources. > [We have this, great] > Coding Standards: the coding standards for submitted code by the community, > along with a description of how strict the project intends to be. > Issue Tracking: links to the JIRA or other issue tracking tool, possibly > including frequently used filters for issue lists. > [This exists on our wiki. We should link to that] > Dependencies: other projects that this project depends on. > [We can link to the wiki for this] > favicon: the project's icon in a format suitable for a browser's address bar. > If absent, an Apache Feather will be displayed. > [we don't have a feather nor a metron logo. not sure if this is important, > but we should probably have something] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (METRON-600) Fix Metron Website
[ https://issues.apache.org/jira/browse/METRON-600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota reassigned METRON-600: --- Assignee: James Sirota > Fix Metron Website > -- > > Key: METRON-600 > URL: https://issues.apache.org/jira/browse/METRON-600 > Project: Metron > Issue Type: Improvement >Reporter: James Sirota >Assignee: James Sirota > > Issue 1 > Podling web sites MUST include a clear disclaimer on their website and in all > documentation (including releases) stating that they are in incubation. > Podlings SHOULD use the following text for all disclaimers (replace the > underlined phrases as appropriate): > Apache Podling-Name is an effort undergoing incubation at The Apache Software > Foundation (ASF), sponsored by the name of Apache TLP sponsor. Incubation is > required of all newly accepted projects until a further review indicates that > the infrastructure, communications, and decision making process have > stabilized in a manner consistent with other successful ASF projects. While > incubation status is not necessarily a reflection of the completeness or > stability of the code, it does indicate that the project has yet to be fully > endorsed by the ASF. > Issue 2: > Podlings websites SHOULD contain the Apache Incubator Project logo as sign of > affiliation > Apache Project Web Sites typically include several standard pages. Each page > is formatted with a navigation bar on the left and a project standard header > that includes the Incubator graphic. > [We need to make the Logo more prominent and move towards the top of the page > rather than having it on the bottom like we do] > Issue 3: > The sources for every podling site sources should be maintained in the > podling's site SVN or git directory > [A downloads page needs to be created with links per release. The link to > the artifact needs to be using the mirror site for apache. For example, the > 0.3.0 release would be > http://www.apache.org/dyn/closer.lua/incubator/metron/0.3.0/apache-metron-0.3.0-incubating.tar.gz. > The MD5, SHA and Signature can be from the apache release site > Look at the storm page as an example: > http://storm.apache.org/downloads.html] > Issue 4 > [Lets try to conform as much as possible to the following suggested template] > Project Home Page: the primary entry point to the site; contains project > description, news, invitation to join the project. > [We have this, great] > License Page: usually, the Apache License 2.0 > [We don't have this, we should probably put it under the about page] > Downloads: many projects in incubation will release code, and this page > describes them and has links to the download pages that redirect to Apache > Mirror sites. > [We have this, great] > Documentation: this page describes the project documentation, including > javadoc for Java projects; guides, tutorials, and links to external > documentation. > [We should probably just link to the wiki so we don't have to maintain this > in two places] > Committers: a list of current committers on the project. > [We need to update this from our status page that can be found here. Need to > make sure both are consistent. > http://incubator.apache.org/projects/metron.html > ] > Mailing Lists: there are several mailing lists that the community might be > interested in, and this page contains mailto: links that allow easy > subscription (and unsubscription) to any of them. > [We should probably put this under our community page and also link to the > apache status page to make sure it's consistent] > FAQ: frequently asked questions are answered here. > [We probably want to link to the wiki for this, as they would be easier to > update this way] > Road Map: if the project has a vision of future community or development > activities, the road map is published here. > [We should probably link to our Jira collection for the next release] > Source Code: links to the browsable source repository and svn commands to > check out the sources. > [We have this, great] > Coding Standards: the coding standards for submitted code by the community, > along with a description of how strict the project intends to be. > Issue Tracking: links to the JIRA or other issue tracking tool, possibly > including frequently used filters for issue lists. > [This exists on our wiki. We should link to that] > Dependencies: other projects that this project depends on. > [We can link to the wiki for this] > favicon: the project's icon in a format suitable for a browser's address bar. > If absent, an Apache Feather will be displayed. > [we don't have a feather nor a metron logo. not sure if this is important, > but we should probably have something] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-600) Fix Metron Website
James Sirota created METRON-600: --- Summary: Fix Metron Website Key: METRON-600 URL: https://issues.apache.org/jira/browse/METRON-600 Project: Metron Issue Type: Improvement Reporter: James Sirota Issue 1 Podling web sites MUST include a clear disclaimer on their website and in all documentation (including releases) stating that they are in incubation. Podlings SHOULD use the following text for all disclaimers (replace the underlined phrases as appropriate): Apache Podling-Name is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the name of Apache TLP sponsor. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. Issue 2: Podlings websites SHOULD contain the Apache Incubator Project logo as sign of affiliation Apache Project Web Sites typically include several standard pages. Each page is formatted with a navigation bar on the left and a project standard header that includes the Incubator graphic. [We need to make the Logo more prominent and move towards the top of the page rather than having it on the bottom like we do] Issue 3: The sources for every podling site sources should be maintained in the podling's site SVN or git directory [A downloads page needs to be created with links per release. The link to the artifact needs to be using the mirror site for apache. For example, the 0.3.0 release would be http://www.apache.org/dyn/closer.lua/incubator/metron/0.3.0/apache-metron-0.3.0-incubating.tar.gz. The MD5, SHA and Signature can be from the apache release site Look at the storm page as an example: http://storm.apache.org/downloads.html] Issue 4 [Lets try to conform as much as possible to the following suggested template] Project Home Page: the primary entry point to the site; contains project description, news, invitation to join the project. [We have this, great] License Page: usually, the Apache License 2.0 [We don't have this, we should probably put it under the about page] Downloads: many projects in incubation will release code, and this page describes them and has links to the download pages that redirect to Apache Mirror sites. [We have this, great] Documentation: this page describes the project documentation, including javadoc for Java projects; guides, tutorials, and links to external documentation. [We should probably just link to the wiki so we don't have to maintain this in two places] Committers: a list of current committers on the project. [We need to update this from our status page that can be found here. Need to make sure both are consistent. http://incubator.apache.org/projects/metron.html ] Mailing Lists: there are several mailing lists that the community might be interested in, and this page contains mailto: links that allow easy subscription (and unsubscription) to any of them. [We should probably put this under our community page and also link to the apache status page to make sure it's consistent] FAQ: frequently asked questions are answered here. [We probably want to link to the wiki for this, as they would be easier to update this way] Road Map: if the project has a vision of future community or development activities, the road map is published here. [We should probably link to our Jira collection for the next release] Source Code: links to the browsable source repository and svn commands to check out the sources. [We have this, great] Coding Standards: the coding standards for submitted code by the community, along with a description of how strict the project intends to be. Issue Tracking: links to the JIRA or other issue tracking tool, possibly including frequently used filters for issue lists. [This exists on our wiki. We should link to that] Dependencies: other projects that this project depends on. [We can link to the wiki for this] favicon: the project's icon in a format suitable for a browser's address bar. If absent, an Apache Feather will be displayed. [we don't have a feather nor a metron logo. not sure if this is important, but we should probably have something] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (METRON-589) Dist area should only contain latest release
[ https://issues.apache.org/jira/browse/METRON-589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota reassigned METRON-589: --- Assignee: James Sirota > Dist area should only contain latest release > > > Key: METRON-589 > URL: https://issues.apache.org/jira/browse/METRON-589 > Project: Metron > Issue Type: Bug >Reporter: John D. Ament >Assignee: James Sirota > > The dist area https://dist.apache.org/repos/dist/release/incubator/metron/ > should only contain the latest release. Older releases are found in the > archives. > Most projects don't directly link to this area, but instead provide the dyn > style links. See Geode for example: > http://geode.incubator.apache.org/releases/ - this works for older releases > as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (METRON-567) Usernames as numerics strings attempted to be parsed and compared as numbers
[ https://issues.apache.org/jira/browse/METRON-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668023#comment-15668023 ] James Sirota commented on METRON-567: - can you please provide your grok statement and a stack trace? > Usernames as numerics strings attempted to be parsed and compared as numbers > > > Key: METRON-567 > URL: https://issues.apache.org/jira/browse/METRON-567 > Project: Metron > Issue Type: Bug >Affects Versions: 0.2.1BETA > Environment: Linux CentOS 6.5 > 252GB RAM > HDP 2.5 > 16TB HDD >Reporter: ed de >Priority: Minor > > 1. Windows logs are being ingested through Nifi, most usernames are number > (ex: 423191384) > 2. Windows parser Grok pattern for element "usrName" has been modified to and > from : GREEDYDATA, NUMBER, WORD, USERNAME. > 3. An enrichment has been flatline loaded into Hbase containing department, > manager, firstname, lastname, etc. > 4. The enrichment works if the usrName is characters (ex: DONALDDUCK) > 5. The consistent error message is "cannot cast java.lang.Long to > java.lang.String". This is readily apparent in the enrichment log under > /var/log/storm/enrichment* > To recreate, build a parser that looks for a username, then build a simple > enrichment, then feed a sample of numeric and non-numeric username logs > through the system and see which one parses and enriches. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (METRON-567) Usernames as numerics strings attempted to be parsed and compared as numbers
[ https://issues.apache.org/jira/browse/METRON-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15664694#comment-15664694 ] James Sirota commented on METRON-567: - are you pretty sure your message is actually being parsed correctly? > Usernames as numerics strings attempted to be parsed and compared as numbers > > > Key: METRON-567 > URL: https://issues.apache.org/jira/browse/METRON-567 > Project: Metron > Issue Type: Bug >Affects Versions: 0.2.1BETA > Environment: Linux CentOS 6.5 > 252GB RAM > HDP 2.5 > 16TB HDD >Reporter: ed de >Priority: Minor > > 1. Windows logs are being ingested through Nifi, most usernames are number > (ex: 423191384) > 2. Windows parser Grok pattern for element "usrName" has been modified to and > from : GREEDYDATA, NUMBER, WORD, USERNAME. > 3. An enrichment has been flatline loaded into Hbase containing department, > manager, firstname, lastname, etc. > 4. The enrichment works if the usrName is characters (ex: DONALDDUCK) > 5. The consistent error message is "cannot cast java.lang.Long to > java.lang.String". This is readily apparent in the enrichment log under > /var/log/storm/enrichment* > To recreate, build a parser that looks for a username, then build a simple > enrichment, then feed a sample of numeric and non-numeric username logs > through the system and see which one parses and enriches. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-295) Script parsing bolt
[ https://issues.apache.org/jira/browse/METRON-295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-295: Fix Version/s: (was: 0.2.2BETA) > Script parsing bolt > > > Key: METRON-295 > URL: https://issues.apache.org/jira/browse/METRON-295 > Project: Metron > Issue Type: New Feature >Affects Versions: 0.2.2BETA >Reporter: James Sirota >Assignee: Karthik Narayanan >Priority: Minor > Labels: newbie, platform > > In addition to having a Grok parsing bolt we need a bolt that can execute a > script in order to parse a telemetry. This way you can still script the > parsing for telemetries for which Grok expressions are too complex, but still > don't have to define a java parser -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-295) Script parsing bolt
[ https://issues.apache.org/jira/browse/METRON-295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-295: Affects Version/s: (was: 0.2.1BETA) 0.2.2BETA > Script parsing bolt > > > Key: METRON-295 > URL: https://issues.apache.org/jira/browse/METRON-295 > Project: Metron > Issue Type: New Feature >Affects Versions: 0.2.2BETA >Reporter: James Sirota >Assignee: Karthik Narayanan >Priority: Minor > Labels: newbie, platform > > In addition to having a Grok parsing bolt we need a bolt that can execute a > script in order to parse a telemetry. This way you can still script the > parsing for telemetries for which Grok expressions are too complex, but still > don't have to define a java parser -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-551) MAAS to check for file permissions
James Sirota created METRON-551: --- Summary: MAAS to check for file permissions Key: METRON-551 URL: https://issues.apache.org/jira/browse/METRON-551 Project: Metron Issue Type: Improvement Reporter: James Sirota Priority: Minor If you send a model or a launching script to MAAS that has the wrong permissions it will not deploy correctly and fail silently -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-550) MAAS Error Checking
James Sirota created METRON-550: --- Summary: MAAS Error Checking Key: METRON-550 URL: https://issues.apache.org/jira/browse/METRON-550 Project: Metron Issue Type: Improvement Reporter: James Sirota Priority: Minor We need to make sure that we check user input, especially input to yarn being a shell script that executes the model. It's very easy to push the actual script in by accident. Also the launcher fails silently and doesn't let you know you did something wrong. You don't find out until you actually try to look for your yarn app -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-363) Fix Cisco ASA Parser
[ https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-363: Assignee: Kyle Richardson (was: Otto Fowler) > Fix Cisco ASA Parser > > > Key: METRON-363 > URL: https://issues.apache.org/jira/browse/METRON-363 > Project: Metron > Issue Type: Improvement >Reporter: Kyle Richardson >Assignee: Kyle Richardson >Priority: Minor > > The current ASA parser is broken. This effort is to rework the current parser > to support the variety of syslog messages produced by Cisco ASA devices as > well as provide the necessary support files/configs for easier deployment of > the Storm topology. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-295) Script parsing bolt
[ https://issues.apache.org/jira/browse/METRON-295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-295: Assignee: (was: James Sirota) > Script parsing bolt > > > Key: METRON-295 > URL: https://issues.apache.org/jira/browse/METRON-295 > Project: Metron > Issue Type: New Feature >Affects Versions: 0.2.1BETA >Reporter: James Sirota >Priority: Minor > Labels: newbie, platform > Fix For: 0.2.1BETA > > > In addition to having a Grok parsing bolt we need a bolt that can execute a > script in order to parse a telemetry. This way you can still script the > parsing for telemetries for which Grok expressions are too complex, but still > don't have to define a java parser -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (METRON-227) Add Time-Based Flushing to Writer Bolt
[ https://issues.apache.org/jira/browse/METRON-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota reassigned METRON-227: --- Assignee: James Sirota (was: Ajay Yadav) > Add Time-Based Flushing to Writer Bolt > -- > > Key: METRON-227 > URL: https://issues.apache.org/jira/browse/METRON-227 > Project: Metron > Issue Type: Bug >Reporter: Domenic Puzio >Assignee: James Sirota > > We need to change the BulkMessageWriterBolt and BulkWriterComponent to use > time-based flushing when writing data to Elasticsearch or Solr. > Currently, we set a batch size, and the Writer waits for that number of > tuples to build up; however, Storm has a timeout value that prevents it from > waiting for too long. If the Writer does not get the batch size before the > timeout, then it recycles the tuples through the topology. In addition, Storm > only allows so many pending messages that have not been acked - if too many > messages are waiting for the bulk Writer, then it will recycle them through > the topology. This is not desired behavior and directly impacts the > performance of this Writer. We would like to be able to specify a unit of > time for which the topology would flush, writing the data it's currently > holding to Elasticsearch or Solr even if the batch size is not met. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-227) Add Time-Based Flushing to Writer Bolt
[ https://issues.apache.org/jira/browse/METRON-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-227: Labels: (was: 0.2.1BETA) > Add Time-Based Flushing to Writer Bolt > -- > > Key: METRON-227 > URL: https://issues.apache.org/jira/browse/METRON-227 > Project: Metron > Issue Type: Bug >Reporter: Domenic Puzio >Assignee: Ajay Yadav > > We need to change the BulkMessageWriterBolt and BulkWriterComponent to use > time-based flushing when writing data to Elasticsearch or Solr. > Currently, we set a batch size, and the Writer waits for that number of > tuples to build up; however, Storm has a timeout value that prevents it from > waiting for too long. If the Writer does not get the batch size before the > timeout, then it recycles the tuples through the topology. In addition, Storm > only allows so many pending messages that have not been acked - if too many > messages are waiting for the bulk Writer, then it will recycle them through > the topology. This is not desired behavior and directly impacts the > performance of this Writer. We would like to be able to specify a unit of > time for which the topology would flush, writing the data it's currently > holding to Elasticsearch or Solr even if the batch size is not met. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-227) Add Time-Based Flushing to Writer Bolt
[ https://issues.apache.org/jira/browse/METRON-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-227: Fix Version/s: (was: 0.2.1BETA) > Add Time-Based Flushing to Writer Bolt > -- > > Key: METRON-227 > URL: https://issues.apache.org/jira/browse/METRON-227 > Project: Metron > Issue Type: Bug >Reporter: Domenic Puzio >Assignee: Ajay Yadav > > We need to change the BulkMessageWriterBolt and BulkWriterComponent to use > time-based flushing when writing data to Elasticsearch or Solr. > Currently, we set a batch size, and the Writer waits for that number of > tuples to build up; however, Storm has a timeout value that prevents it from > waiting for too long. If the Writer does not get the batch size before the > timeout, then it recycles the tuples through the topology. In addition, Storm > only allows so many pending messages that have not been acked - if too many > messages are waiting for the bulk Writer, then it will recycle them through > the topology. This is not desired behavior and directly impacts the > performance of this Writer. We would like to be able to specify a unit of > time for which the topology would flush, writing the data it's currently > holding to Elasticsearch or Solr even if the batch size is not met. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-322) Global Batching & flushing
[ https://issues.apache.org/jira/browse/METRON-322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-322: Assignee: Matt Foley > Global Batching & flushing > -- > > Key: METRON-322 > URL: https://issues.apache.org/jira/browse/METRON-322 > Project: Metron > Issue Type: Improvement >Reporter: Ajay Yadav >Assignee: Matt Foley > > Flushing individual telemetries with disparate traffic are not only difficult > to tune in single topology but also creates lot of failed message overhead as > topology level configurations like “timeout, max.spout.pending” etc can’t be > changed for every telemetry. Instead of batching individual telemetries in > enrichment we should batch & flush them together. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-322) Global Batching & flushing
[ https://issues.apache.org/jira/browse/METRON-322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-322: Assignee: (was: Ajay Yadav) > Global Batching & flushing > -- > > Key: METRON-322 > URL: https://issues.apache.org/jira/browse/METRON-322 > Project: Metron > Issue Type: Improvement >Reporter: Ajay Yadav > > Flushing individual telemetries with disparate traffic are not only difficult > to tune in single topology but also creates lot of failed message overhead as > topology level configurations like “timeout, max.spout.pending” etc can’t be > changed for every telemetry. Instead of batching individual telemetries in > enrichment we should batch & flush them together. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-382) Fix special characters for config
James Sirota created METRON-382: --- Summary: Fix special characters for config Key: METRON-382 URL: https://issues.apache.org/jira/browse/METRON-382 Project: Metron Issue Type: Bug Affects Versions: 0.2.1BETA Reporter: James Sirota Assignee: Casey Stella Fix For: 0.2.1BETA We check our configs for proper JSON formatting, but we don't check for special characters. It would be nice to check and automatically filter them out. The command in Linux that works for this is: tr -cd '\11\12\40-\176' < myfile1 > myfile2 We need to execute this as a part of our script to filter them out automatically. I can see a lot of people being tripped up by this. Every time you validate a JSON against a linter and then paste it back into VI these characters come in somehow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (METRON-193) Metron - PCAP Support for Windows
[ https://issues.apache.org/jira/browse/METRON-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417991#comment-15417991 ] James Sirota commented on METRON-193: - If this is something you wanted to contribute back we can walk you through the Apache process and try to test it with Metron in our environment > Metron - PCAP Support for Windows > - > > Key: METRON-193 > URL: https://issues.apache.org/jira/browse/METRON-193 > Project: Metron > Issue Type: Wish > Environment: Dev >Reporter: Tom James >Priority: Trivial > Fix For: 0.3.0BETA > > > Metron uses DPDK for packet capture. It seems DPDK does not have official > support for Windows. If we can enable Metron PCAP module to extract packets > from Windows environment as well, that would be a nice addition. > Right now, I'm thinking of a few possible solution, > 1. WinPcap based pacpy > 2. Dedicated native pcap client based on WinPcap libraries > 3. NDIS/WPF based driver/client for dedicated pcap support. > Please come forward with any suggestions, queries and support. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (METRON-193) Metron - PCAP Support for Windows
[ https://issues.apache.org/jira/browse/METRON-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417893#comment-15417893 ] James Sirota commented on METRON-193: - Thats great. Is this something you would like to contribute? > Metron - PCAP Support for Windows > - > > Key: METRON-193 > URL: https://issues.apache.org/jira/browse/METRON-193 > Project: Metron > Issue Type: Wish > Environment: Dev >Reporter: Tom James >Priority: Trivial > Fix For: 0.3.0BETA > > > Metron uses DPDK for packet capture. It seems DPDK does not have official > support for Windows. If we can enable Metron PCAP module to extract packets > from Windows environment as well, that would be a nice addition. > Right now, I'm thinking of a few possible solution, > 1. WinPcap based pacpy > 2. Dedicated native pcap client based on WinPcap libraries > 3. NDIS/WPF based driver/client for dedicated pcap support. > Please come forward with any suggestions, queries and support. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (METRON-193) Metron - PCAP Support for Windows
[ https://issues.apache.org/jira/browse/METRON-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417890#comment-15417890 ] James Sirota commented on METRON-193: - Hi Tom, What kind of hardware are you running this on? > Metron - PCAP Support for Windows > - > > Key: METRON-193 > URL: https://issues.apache.org/jira/browse/METRON-193 > Project: Metron > Issue Type: Wish > Environment: Dev >Reporter: Tom James >Priority: Trivial > Fix For: 0.3.0BETA > > > Metron uses DPDK for packet capture. It seems DPDK does not have official > support for Windows. If we can enable Metron PCAP module to extract packets > from Windows environment as well, that would be a nice addition. > Right now, I'm thinking of a few possible solution, > 1. WinPcap based pacpy > 2. Dedicated native pcap client based on WinPcap libraries > 3. NDIS/WPF based driver/client for dedicated pcap support. > Please come forward with any suggestions, queries and support. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-361) Metron Model Exchange
James Sirota created METRON-361: --- Summary: Metron Model Exchange Key: METRON-361 URL: https://issues.apache.org/jira/browse/METRON-361 Project: Metron Issue Type: New Feature Affects Versions: 0.3.0BETA Reporter: James Sirota I want to be able to run a model trained on x instance of Metron to also run on Y instance of Metron -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-318) Metron Assessment Tool
James Sirota created METRON-318: --- Summary: Metron Assessment Tool Key: METRON-318 URL: https://issues.apache.org/jira/browse/METRON-318 Project: Metron Issue Type: New Feature Reporter: James Sirota We need a tool to profile Metron telemetries prior to metron being deployed in someone's environment. This jira is to solicit architecture ideas and feedback on the design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-313) PCAP Service page capability
James Sirota created METRON-313: --- Summary: PCAP Service page capability Key: METRON-313 URL: https://issues.apache.org/jira/browse/METRON-313 Project: Metron Issue Type: Improvement Reporter: James Sirota Priority: Trivial Fix For: 0.3.0BETA Add a capability to page through a PCAP file if the file is too large. Files from a PCAP service should be delivered to HDFS and then delivered to user in segments -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-311) Benchmark PCAP probe on hardware cards
James Sirota created METRON-311: --- Summary: Benchmark PCAP probe on hardware cards Key: METRON-311 URL: https://issues.apache.org/jira/browse/METRON-311 Project: Metron Issue Type: Wish Affects Versions: 0.3.0BETA Reporter: James Sirota Priority: Trivial We want to be able to test our probes on bare metal cards to know what throughput a single probe can produce and how the ingest scales with multiple probes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-309) Create a normalcy profiler
James Sirota created METRON-309: --- Summary: Create a normalcy profiler Key: METRON-309 URL: https://issues.apache.org/jira/browse/METRON-309 Project: Metron Issue Type: New Feature Reporter: James Sirota Fix For: 0.2.1BETA We need to create a telemetry that can take the streaming data and build a entity-specific behavioral profile (most likely statistical summaries in Hbase) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-308) PCAP Replay
James Sirota created METRON-308: --- Summary: PCAP Replay Key: METRON-308 URL: https://issues.apache.org/jira/browse/METRON-308 Project: Metron Issue Type: New Feature Affects Versions: 0.3.0BETA Reporter: James Sirota I want to use the existing PCAP service to replay recovered PCAP through all of my available PCAP sensors and re-ingest that stale data back into the system, but in such a way that land that telemetry in a separate location from live events -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-307) Error Topology
James Sirota created METRON-307: --- Summary: Error Topology Key: METRON-307 URL: https://issues.apache.org/jira/browse/METRON-307 Project: Metron Issue Type: Improvement Affects Versions: 0.3.0BETA Reporter: James Sirota Priority: Minor We need an extensible topology that provides a capability to pull data from deadletterq and failed validation topics and pushing them into an index, files on hdfs, etc., or to a UI. We also need to provide a capability (either an editor or a UI) where they can be edited and pushed back onto the RAW topic -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-296) patternsCommonDir is hard coded in Grok parser
[ https://issues.apache.org/jira/browse/METRON-296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-296: Summary: patternsCommonDir is hard coded in Grok parser (was: scriptsCommonDir is hard coded in Grok parser) > patternsCommonDir is hard coded in Grok parser > -- > > Key: METRON-296 > URL: https://issues.apache.org/jira/browse/METRON-296 > Project: Metron > Issue Type: Improvement >Affects Versions: 0.2.1BETA >Reporter: James Sirota >Priority: Minor > Labels: newbie > Fix For: 0.2.1BETA > > > This setting needs to be pulled out into Zookeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-296) scriptsCommonDir is hard coded in Grok parser
James Sirota created METRON-296: --- Summary: scriptsCommonDir is hard coded in Grok parser Key: METRON-296 URL: https://issues.apache.org/jira/browse/METRON-296 Project: Metron Issue Type: Improvement Affects Versions: 0.2.1BETA Reporter: James Sirota Priority: Minor This setting needs to be pulled out into Zookeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (METRON-295) Script parsing bolt
[ https://issues.apache.org/jira/browse/METRON-295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota reassigned METRON-295: --- Assignee: James Sirota > Script parsing bolt > > > Key: METRON-295 > URL: https://issues.apache.org/jira/browse/METRON-295 > Project: Metron > Issue Type: New Feature >Affects Versions: 0.2.1BETA >Reporter: James Sirota >Assignee: James Sirota >Priority: Minor > Labels: newbie > Fix For: 0.2.1BETA > > > In addition to having a Grok parsing bolt we need a bolt that can execute a > script in order to parse a telemetry. This way you can still script the > parsing for telemetries for which Grok expressions are too complex, but still > don't have to define a java parser -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-295) Script parsing bolt
James Sirota created METRON-295: --- Summary: Script parsing bolt Key: METRON-295 URL: https://issues.apache.org/jira/browse/METRON-295 Project: Metron Issue Type: New Feature Affects Versions: 0.2.1BETA Reporter: James Sirota Priority: Minor Fix For: 0.2.1BETA In addition to having a Grok parsing bolt we need a bolt that can execute a script in order to parse a telemetry. This way you can still script the parsing for telemetries for which Grok expressions are too complex, but still don't have to define a java parser -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-290) Add ontologies to Stellar for building knowledge graphs
James Sirota created METRON-290:
---
Summary: Add ontologies to Stellar for building knowledge graphs
Key: METRON-290
URL: https://issues.apache.org/jira/browse/METRON-290
Project: Metron
Issue Type: New Feature
Affects Versions: 0.2.1BETA
Reporter: James Sirota
Assignee: Casey Stella
Fix For: 0.2.1BETA
I want to be able to define, extract, and build ontologies/relations from
Metron telemetry. Here is an example. If my message is as follows:
{ip1:someIP1, ip2:someIp2, protocol:TCP,userAgent:A1}
I want the following config defined:
ontology1: ip1 connectsTo ip2
ontology2: ip1 hasProtocol protocol
ontology3: ip1 hasUserAgent userAgent
...
ontology n
And from that config I want the following be created with Stellar:
someIP1 : connectsTo : someIP2
someIP1 : hasProtocol : TCP
someIP1: hasUserAgent: A1
And then be pushed to a graph database with a TTL value. By applying these to
multiple telemetries we create a knowledge graph. This is one of the
fundamental capabilities that Metron needs for doing advanced analytics
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (METRON-289) Uptake pre-parsed Metron telemetry
James Sirota created METRON-289: --- Summary: Uptake pre-parsed Metron telemetry Key: METRON-289 URL: https://issues.apache.org/jira/browse/METRON-289 Project: Metron Issue Type: New Feature Affects Versions: 0.2.2BETA Reporter: James Sirota Priority: Minor Fix For: 0.2.2BETA If Metron message has been parsed upstream by another tool I want to have a capability to bypass the parsing function in the parsing topology and go straight into Stellar -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-287) Ability to route alerts to a kafka queue
James Sirota created METRON-287: --- Summary: Ability to route alerts to a kafka queue Key: METRON-287 URL: https://issues.apache.org/jira/browse/METRON-287 Project: Metron Issue Type: New Feature Reporter: James Sirota Priority: Minor Fix For: 0.2.2BETA I want to be able to route Metron alerts to a kafka topic so that I can consume them from an external system -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-280) bro parsing issue
[
https://issues.apache.org/jira/browse/METRON-280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Sirota updated METRON-280:
Priority: Minor (was: Major)
> bro parsing issue
> -
>
> Key: METRON-280
> URL: https://issues.apache.org/jira/browse/METRON-280
> Project: Metron
> Issue Type: Bug
>Affects Versions: 0.2.1BETA
>Reporter: Neha Sinha
>Priority: Minor
> Fix For: 0.2.1BETA
>
> Attachments: bro_parser_stacktrace.rtf
>
>
> Hi,
> The bro parser fails to parse the following event in my metron environment :-
> {"http":
> {"ts":1467657279.0,"uid":"CMYLzP3PKiwZAgBa51","id.orig_h":"192.168.138.158","id.orig_p":49206,"id.resp_h":"95.163.121.204",
> "id.resp_p":80,"trans_depth":2,"method":"GET","host":"7oqnsnzwwnm6zb7y.gigapaysun.com","uri":"/img/flags/it.png","referrer":"http://7oqnsnzwwnm6zb7y.gigapaysun.com/11iQmfg","user_agent":"Mozilla/4.0
> (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR
> 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC
> 6.0)","request_body_len":0,"response_body_len":552,"status_code":200,"status_msg":"OK","tags":[],"resp_fuids":["F3m7vB2RjUe4n01aqj"],"resp_mime_types":["image/png"]}}
> When I looked up the stack trace it complains of the following statement in
> BasicBroparser.java file :-
> convertedTimestamp=convertedTimestamp.substring(0,13);
> Since the "ts" field in the respective bro events is not 13 chars long the
> parser threw the exception.we need to fix the bro parser to accomodate
> parsing of such events.
> Please find attached the parser exception message .
> Regards,
> Neha
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (METRON-285) Metron parsers to support IPV6
James Sirota created METRON-285: --- Summary: Metron parsers to support IPV6 Key: METRON-285 URL: https://issues.apache.org/jira/browse/METRON-285 Project: Metron Issue Type: Improvement Reporter: James Sirota Priority: Minor Fix For: 0.3.0BETA We need to be able to support environments with mixed IPV4 and IPV6 addresses -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-283) Migrate Geo Enrichment outside of MySQL
[ https://issues.apache.org/jira/browse/METRON-283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-283: Description: We need to migrate our enrichment SQL store from MySQL to Phoenix or some other SQL on Hbase library. Or alternatively come up with a way to do this without using SQL. This way we don't have a dependency on MySQL and there is one less thing that we need to install on our platform (was: We need to migrate our enrichment SQL store from MySQL to Phoenix. This way we don't have a dependency on MySQL and there is one less thing that we need to install on our platform ) > Migrate Geo Enrichment outside of MySQL > --- > > Key: METRON-283 > URL: https://issues.apache.org/jira/browse/METRON-283 > Project: Metron > Issue Type: Improvement >Reporter: James Sirota >Priority: Minor > Fix For: 0.3.0BETA > > > We need to migrate our enrichment SQL store from MySQL to Phoenix or some > other SQL on Hbase library. Or alternatively come up with a way to do this > without using SQL. This way we don't have a dependency on MySQL and there is > one less thing that we need to install on our platform -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-283) Migrate Geo Enrichment outside of MySQL
[ https://issues.apache.org/jira/browse/METRON-283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-283: Summary: Migrate Geo Enrichment outside of MySQL (was: Migrate MySQL to Phoenix) > Migrate Geo Enrichment outside of MySQL > --- > > Key: METRON-283 > URL: https://issues.apache.org/jira/browse/METRON-283 > Project: Metron > Issue Type: Improvement >Reporter: James Sirota >Priority: Minor > Fix For: 0.3.0BETA > > > We need to migrate our enrichment SQL store from MySQL to Phoenix. This way > we don't have a dependency on MySQL and there is one less thing that we need > to install on our platform -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (METRON-283) Migrate MySQL to Phoenix
[ https://issues.apache.org/jira/browse/METRON-283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15364800#comment-15364800 ] James Sirota commented on METRON-283: - It's a suggested improvement to get SQL functionality out of MySQL and into a SQL solution that runs on top of Hbase. Phoenix is the only one i know of. Introducing SQL on Hbase gets us out of needing a separate SQL store. > Migrate MySQL to Phoenix > > > Key: METRON-283 > URL: https://issues.apache.org/jira/browse/METRON-283 > Project: Metron > Issue Type: Improvement >Reporter: James Sirota >Priority: Minor > Fix For: 0.3.0BETA > > > We need to migrate our enrichment SQL store from MySQL to Phoenix. This way > we don't have a dependency on MySQL and there is one less thing that we need > to install on our platform -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-282) Scan pull requests for sensitive data
James Sirota created METRON-282: --- Summary: Scan pull requests for sensitive data Key: METRON-282 URL: https://issues.apache.org/jira/browse/METRON-282 Project: Metron Issue Type: Improvement Reporter: James Sirota Priority: Minor Fix For: 0.2.2BETA We need to come up with a way to scan each pull requests to make sure it doesn't have sensitive information in it such as ips, usernames, ssns, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-278) Add debug statements
James Sirota created METRON-278: --- Summary: Add debug statements Key: METRON-278 URL: https://issues.apache.org/jira/browse/METRON-278 Project: Metron Issue Type: Improvement Reporter: James Sirota Priority: Minor There needs to be more logging in the code to enable debugging. We need more debug and trace statements so when we turn up logging the code is easier to debug -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-271) Add graph DB to the platform
[ https://issues.apache.org/jira/browse/METRON-271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-271: Issue Type: New Feature (was: Bug) > Add graph DB to the platform > > > Key: METRON-271 > URL: https://issues.apache.org/jira/browse/METRON-271 > Project: Metron > Issue Type: New Feature >Reporter: James Sirota > Labels: 0.2.2BETA, METRON_ML > Fix For: 0.2.1BETA > > > I propose adding a graph database (Titan or others) so we can use graph > mining as feature inputs to some of our models and anomaly detectors -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-270) Add Zeppelin to the platform
[ https://issues.apache.org/jira/browse/METRON-270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-270: Issue Type: New Feature (was: Bug) > Add Zeppelin to the platform > > > Key: METRON-270 > URL: https://issues.apache.org/jira/browse/METRON-270 > Project: Metron > Issue Type: New Feature >Reporter: James Sirota > Labels: 0.2.2BETA, METRON_ML > Fix For: 0.2.1BETA > > > I propose adding Zeppelin to the platform to aid in interactive dashboarding > and data visualizations -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-268) Add Jupyter to the platform
[ https://issues.apache.org/jira/browse/METRON-268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-268: Priority: Minor (was: Major) Issue Type: New Feature (was: Bug) > Add Jupyter to the platform > > > Key: METRON-268 > URL: https://issues.apache.org/jira/browse/METRON-268 > Project: Metron > Issue Type: New Feature >Reporter: James Sirota >Priority: Minor > Labels: 0.2.1BETA, METRON_ML > Fix For: 0.2.1BETA > > > We need an analytics workbench for visualizing data and creating ML models. > I propose having a Jupyter interface with R-Spark and Py-Spark enabled -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-269) Integrate Spark into the platform
[ https://issues.apache.org/jira/browse/METRON-269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-269: Issue Type: New Feature (was: Bug) > Integrate Spark into the platform > -- > > Key: METRON-269 > URL: https://issues.apache.org/jira/browse/METRON-269 > Project: Metron > Issue Type: New Feature >Reporter: James Sirota > Labels: 0.2.1BETA, METRON_ML > Fix For: 0.2.1BETA > > > I propose adding Spark to the project to aid in batch analytics and modeling -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-261) Storm Supervisors Fail to Start
[ https://issues.apache.org/jira/browse/METRON-261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-261: Priority: Minor (was: Major) > Storm Supervisors Fail to Start > --- > > Key: METRON-261 > URL: https://issues.apache.org/jira/browse/METRON-261 > Project: Metron > Issue Type: Bug >Reporter: Nick Allen >Priority: Minor > Fix For: 0.2.1BETA > > > After deployment completes, the Storm Supervisors often fail to start > correctly. This prevents any data from being ingested until the Supervisors > are manually started. > It appears that the Supervisors fail to communicate with Zookeeper and they > timeout and die. Zookeeper may just not be ready in time. Not sure if this > is something we can fix or if this is an Ambari issue. > 2016-06-25 12:48:16.448 o.a.s.z.ClientCnxn [WARN] Session 0x0 for server > null, unexpected error, closing socket connection and attempting reconnect > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > ~[?:1.8.0_40] > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) > ~[?:1.8.0_40] > at > org.apache.storm.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) > ~[storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at > org.apache.storm.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > 2016-06-25 12:48:17.154 o.a.s.c.ConnectionState [ERROR] Connection timed out > for connection string (ec2-52-41-178-50.us-west-2.compute.amazonaws.com:2181) > and timeout (15000) / elapsed (15053) > org.apache.storm.curator.CuratorConnectionLossException: KeeperErrorCode = > ConnectionLoss > at > org.apache.storm.curator.ConnectionState.checkTimeouts(ConnectionState.java:195) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at > org.apache.storm.curator.ConnectionState.getZooKeeper(ConnectionState.java:87) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at > org.apache.storm.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:115) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at > org.apache.storm.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:487) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at > org.apache.storm.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:226) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at > org.apache.storm.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:215) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at > org.apache.storm.curator.RetryLoop.callWithRetry(RetryLoop.java:107) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at > org.apache.storm.curator.framework.imps.ExistsBuilderImpl.pathInForegroundStandard(ExistsBuilderImpl.java:212) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at > org.apache.storm.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:205) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at > org.apache.storm.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:168) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at > org.apache.storm.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:39) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at > backtype.storm.zookeeper$exists_node_QMARK_$fn__3211.invoke(zookeeper.clj:107) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at > backtype.storm.zookeeper$exists_node_QMARK_.invoke(zookeeper.clj:104) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at backtype.storm.zookeeper$mkdirs.invoke(zookeeper.clj:120) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at > backtype.storm.cluster$mk_distributed_cluster_state.doInvoke(cluster.clj:60) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at clojure.lang.RestFn.invoke(RestFn.java:486) [clojure-1.6.0.jar:?] > at > backtype.storm.cluster$mk_storm_cluster_state.doInvoke(cluster.clj:314) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at clojure.lang.RestFn.invoke(RestFn.java:439) [clojure-1.6.0.jar:?] > at > backtype.storm.daemon.supervisor$supervisor_data.invoke(supervisor.clj:296) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at > backtype.storm.daemon.supervisor$fn__8449$exec_fn__3614__auto8450.invoke(supervisor.clj:504) > [storm-core-0.10.0.2.3.4.7-4.jar:0.10.0.2.3.4.7-4] > at clojure.lang.AFn.applyToHelper(AFn.java:160) [clojure-1.6.0.jar:?] >
[jira] [Updated] (METRON-267) Add Third Installer Option to Apache Metron Web Page
[ https://issues.apache.org/jira/browse/METRON-267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-267: Priority: Trivial (was: Major) > Add Third Installer Option to Apache Metron Web Page > > > Key: METRON-267 > URL: https://issues.apache.org/jira/browse/METRON-267 > Project: Metron > Issue Type: Improvement >Reporter: George Vetticaden >Assignee: Ryan Merriman >Priority: Trivial > Labels: 0.2.1BETA, METRON_UI > Fix For: 0.2.1BETA > > > With Metron .2 Release we have now officially support an third install > option. Installing Metron on an existing HDP managed cluster. > Hence we need to update the metron home page: > https://metron.incubator.apache.org/documentation/ to include the third > install option that links to the following: > https://cwiki.apache.org/confluence/display/METRON/Metron+Installation+on+a+Ambari+Managed+Cluster > Right now the home page only describes 2 install options. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-259) ERROR! ERROR! 'dict object' has no attribute u'ansible_tap0'
[
https://issues.apache.org/jira/browse/METRON-259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Sirota updated METRON-259:
Priority: Minor (was: Major)
> ERROR! ERROR! 'dict object' has no attribute u'ansible_tap0'
>
>
> Key: METRON-259
> URL: https://issues.apache.org/jira/browse/METRON-259
> Project: Metron
> Issue Type: Bug
>Reporter: Nick Allen
>Assignee: Nick Allen
>Priority: Minor
> Fix For: 0.2.1BETA
>
> Attachments: ansible (2).log
>
>
> 2016-06-24 11:10:53,994 p=66991 u=xxx | TASK [snort : Configure home
> network] **
> 2016-06-24 11:10:54,277 p=66991 u=xxx | ^[[0;31mfatal:
> [ec2-xxx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com]: FAILED! => {"failed":
> true, "msg": "ERROR! ERROR! 'dict object' has no attribute
> u'ansible_tap0'"}^[[0m
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (METRON-257) Allow pcap result pagination from the Pcap CLI
[ https://issues.apache.org/jira/browse/METRON-257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-257: Priority: Minor (was: Major) > Allow pcap result pagination from the Pcap CLI > -- > > Key: METRON-257 > URL: https://issues.apache.org/jira/browse/METRON-257 > Project: Metron > Issue Type: Improvement >Reporter: Casey Stella >Priority: Minor > Fix For: 0.2.2BETA > > > Right now we are returning the whole result set as part of the PCap CLI. We > should allow for pagination of results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-196) Deployment Fails Without Ansible 2.0.0.2
[
https://issues.apache.org/jira/browse/METRON-196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Sirota updated METRON-196:
Priority: Minor (was: Major)
Issue Type: Improvement (was: Bug)
> Deployment Fails Without Ansible 2.0.0.2
>
>
> Key: METRON-196
> URL: https://issues.apache.org/jira/browse/METRON-196
> Project: Metron
> Issue Type: Improvement
>Reporter: Nick Allen
>Priority: Minor
> Labels: 0.2.2BETA
> Fix For: 0.3.0BETA
>
>
> The following error occurs when deploying Metron with versions other than
> 2.0.0.2; particularly version 2.0.1. The current work around is to ask users
> to downgrade Ansible version per
> https://cwiki.apache.org/confluence/display/METRON/Downgrade+Ansible.
> ASK [elasticsearch : Add Elasticsearch templates for topologies]
> **
> failed: [node1] (item={u'sensor': u'bro', u'file': {'mappings': {'bro_doc':
> {'_timestamp': {'enabled': True}, 'properties':
> {'enrichments:geo:ip_dst_addr:location_point': {'type': 'geo_point'},
> 'timestamp': {'type': 'date', 'format': 'epoch_millis', 'template':
> 'bro_index*'}}) => {"content": "", "content_length": "450", "content_type":
> "application/json; charset=UTF-8", "failed": true, "item": {"file":
> {"mappings": {"bro_doc": {"_timestamp": {"enabled": true}, "properties":
> {"enrichments:geo:ip_dst_addr:location_point": {"type": "geo_point"},
> "timestamp": {"format": "epoch_millis", "type": "date", "template":
> "bro_index*"}, "sensor": "bro"}, "msg": "Status code was not [200]: HTTP
> Error 400: Bad Request", "redirected": false, "status": 400, "url":
> "http://node1:9200/_template/template_bro"}
> failed: [node1] (item={u'sensor': u'yaf', u'file': {'mappings': {'yaf_doc':
> {'_timestamp': {'enabled': True}, 'properties': {'uflags': {'type':
> 'string'}, 'pkt': {'type': 'string'}, 'app': {'type': 'string'}, 'rtt':
> {'type': 'string'}, 'tag': {'type': 'string'}, 'duration': {'type':
> 'string'}, 'riflags': {'type': 'string'}, 'sip': {'type': 'string'}, 'proto':
> {'type': 'string'}, 'rtag': {'type': 'string'}, 'oct': {'type': 'string'},
> 'risn': {'type': 'string'}, 'end-time': {'type': 'string'}, 'end-reason':
> {'type': 'string'}, 'timestamp': {'type': 'date', 'format': 'epoch_millis'},
> 'dp': {'type': 'string'}, 'enrichments:geo:ip_dst_addr:location_point':
> {'type': 'geo_point'}, 'roct': {'type': 'string'}, 'sp': {'type': 'string'},
> 'iflags': {'type': 'string'}, 'isn': {'type': 'string'}, 'ruflags': {'type':
> 'string'}, 'rpkt': {'type': 'string'}, 'dip': {'type': 'string',
> 'template': 'yaf_index*'}}) => {"content": "", "content_length": "450",
> "content_type": "application/json; charset=UTF-8", "failed": true, "item":
> {"file": {"mappings": {"yaf_doc": {"_timestamp": {"enabled": true},
> "properties": {"app": {"type": "string"}, "dip": {"type": "string"}, "dp":
> {"type": "string"}, "duration": {"type": "string"}, "end-reason": {"type":
> "string"}, "end-time": {"type": "string"},
> "enrichments:geo:ip_dst_addr:location_point": {"type": "geo_point"},
> "iflags": {"type": "string"}, "isn": {"type": "string"}, "oct": {"type":
> "string"}, "pkt": {"type": "string"}, "proto": {"type": "string"}, "riflags":
> {"type": "string"}, "risn": {"type": "string"}, "roct": {"type": "string"},
> "rpkt": {"type": "string"}, "rtag": {"type": "string"}, "rtt": {"type":
> "string"}, "ruflags": {"type": "string"}, "sip": {"type": "string"}, "sp":
> {"type": "string"}, "tag": {"type": "string"}, "timestamp": {"format":
> "epoch_millis", "type": "date"}, "uflags": {"type": "string", "template":
> "yaf_index*"}, "sensor": "yaf"}, "msg": "Status code was not [200]: HTTP
> Error 400: Bad Request", "redirected": false, "status": 400, "url":
> "http://node1:9200/_template/template_yaf"}
> failed: [node1] (item={u'sensor': u'snort', u'file': {'mappings':
> {'snort_doc': {'_timestamp': {'enabled': True}, 'properties':
> {'enrichments:geo:ip_dst_addr:location_point': {'type': 'geo_point'},
> 'timestamp': {'type': 'date', 'format': 'epoch_millis', 'template':
> 'snort_index*'}}) => {"content": "", "content_length": "450", "content_type":
> "application/json; charset=UTF-8", "failed": true, "item": {"file":
> {"mappings": {"snort_doc": {"_timestamp": {"enabled": true}, "properties":
> {"enrichments:geo:ip_dst_addr:location_point": {"type": "geo_point"},
> "timestamp": {"format": "epoch_millis", "type": "date", "template":
> "snort_index*"}, "sensor": "snort"}, "msg": "Status code was not [200]: HTTP
> Error 400: Bad Request", "redirected": false, "status": 400, "url":
> "http://node1:9200/_template/template_snort"}
> to retry, use: --limit @../../playbooks/metron_full_install.retry
> PLAY RECAP
>
[jira] [Updated] (METRON-192) Metron Platform Extension
[ https://issues.apache.org/jira/browse/METRON-192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-192: Priority: Minor (was: Major) > Metron Platform Extension > - > > Key: METRON-192 > URL: https://issues.apache.org/jira/browse/METRON-192 > Project: Metron > Issue Type: Wish >Reporter: James Sirota >Priority: Minor > Labels: ForwardLookingEpic > Fix For: 0.3.0BETA > > > I envision for Metron-Forensics to be a package that utilizes Metron's PCAP > capture and replay utilities to bring a new set of forensic capabilities to > Metron. I see forensics to be subdivided into the following sets of > capabilities: > Passive Network Analysis (PNA) > POF: http://lcamtuf.coredump.cx/p0f3/ > Passive Asset Detection System: http://passive.sourceforge.net/ > NMap https://nmap.org/ > Network Miner: http://www.netresec.com/?page=NetworkMiner > Tenable Passive Vulnerability Scanner > http://www.tenable.com/products/passive-vulnerability-scanner > PCAP Search, Reconstruction, and Forensics: > ChaosLoader: http://chaosreader.sourceforge.net/ > TCP Extract: http://tcpxtract.sourceforge.net/ > TCP ICK: http://tcpick.sourceforge.net/ > NSM Console: http://writequit.org/projects/nsm-console/ > Moloch: https://github.com/aol/moloch > Berkeley Packet Filter: http://www.freebsd.org/cgi/man.cgi?bpf > Scapy: http://www.secdev.org/projects/scapy/ > xPlico http://www.xplico.org/ > Wireshark https://www.wireshark.org/ > Malware Forensics: > IDA Pro: https://www.hex-rays.com/products/ida/ > YARA: https://plusvic.github.io/yara/ > Data Loss Prevention > OpelDLP https://code.google.com/archive/p/opendlp/ > OpenNLP https://opennlp.apache.org/ > Stanford NER http://nlp.stanford.edu/software/CRF-NER.shtml > Netflow > Silk: https://tools.netsa.cert.org/silk/download.html > Sandboxing: > Cuckoo Sandbox: https://www.cuckoosandbox.org/ > Visualization: > Maltego https://www.paterva.com/web7/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-187) Support Deployment of Metron on Isolated Networks
[ https://issues.apache.org/jira/browse/METRON-187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-187: Issue Type: Improvement (was: Bug) > Support Deployment of Metron on Isolated Networks > - > > Key: METRON-187 > URL: https://issues.apache.org/jira/browse/METRON-187 > Project: Metron > Issue Type: Improvement >Reporter: Nick Allen >Assignee: Nick Allen > Fix For: 0.2.1BETA > > > h2. Problem > In many cases Metron will need to be deployed on a network that does not have > direct access to the public interwebs. The current deployment scheme requires > access to the public interwebs to download artifacts like RPMs, tarballs, > rule sets, etc. > h2. Assumptions > There exists a machine that will orchestrate the deployment that meets the > following requirements. > - The machine can deploy "Full Dev Platform" or "Quick Dev Platform". In > short, this machine must run either Linux or OSX and have Ansible, Vagrant, > Maven, Java, among the other dependencies installed. > - The machine must be able to connect to both the public internet and the > private, isolated network. This does not need to occur at the same time. > For example, the machine can connect to the public internet, then disconnect > from the public internet, then connect to the private, isolated network. > This scheme also meets the requirement. > h2. Solution > The following high-level approach can be taken. > - Extract: Extract artifacts from public internet and store on local > deployment machine. > - Transfer: Move deployment machine, along with extracted artifacts, to > private, isolated network. > - Reuse: Deploy Metron using the artifacts stored on the deployment machine. > The following details steps implement the high-level approach of extract, > transfer, and reuse. > - Connect the deployment host to the public internet. > - Run a customized Vagrant installation of Metron on the deployment host. > - After the normal "Quick Dev Platform" deployment completes, the > customization ensures that all required artifacts that were downloaded from > the public internet are persisted locally on the deployment host. > - Validate that the Vagrant installation worked correctly. > - Disconnect the deployment host from the public internet. > - Connect the deployment host to the private, isolated network. > - Prior to the normal Metron deployment, the locally persisted artifacts will > be deployed to a designated repository server. The repository server will > host the artifacts by whatever means are required for the artifact. For > example, for RPMs an RPM Repository will be created. > - All properties that refer to resources on the public internet will be > updated to point to the repository server. > - Run the normal Metron deployment process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-168) EC2 deployment fails intermittently on check hosts task
[
https://issues.apache.org/jira/browse/METRON-168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Sirota updated METRON-168:
Priority: Minor (was: Major)
> EC2 deployment fails intermittently on check hosts task
>
>
> Key: METRON-168
> URL: https://issues.apache.org/jira/browse/METRON-168
> Project: Metron
> Issue Type: Bug
>Reporter: Ryan Merriman
>Priority: Minor
> Fix For: 0.2.1BETA
>
>
> When deploying Metron on EC2, the process fails with the following message
> during the check-hosts task:
> fatal: [ec2-54-186-185-186.us-west-2.compute.amazonaws.com]: UNREACHABLE! =>
> {"changed": false, "msg": "Failed to connect to the host via ssh.",
> "unreachable": true}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (METRON-161) Create AD Parser
[
https://issues.apache.org/jira/browse/METRON-161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Sirota updated METRON-161:
Priority: Minor (was: Major)
> Create AD Parser
>
>
> Key: METRON-161
> URL: https://issues.apache.org/jira/browse/METRON-161
> Project: Metron
> Issue Type: New Feature
>Reporter: Deeptaanshu Kumar
>Assignee: James Sirota
>Priority: Minor
> Labels: ParserExtension
> Fix For: 0.2.2BETA
>
>
> Create a parser for the Active Directory telemetry source. This data source
> has 3 formats that should be parsed as specified below:
> Required Active Directory fields:
> dcName
> admonEventType
> description
> distinguishedName
> DC
> CN
> whenChanged
> whenCreated
> memberOf
> userAccountControl
> Sample Active Directory log message:
> 04/11/2016 17:00:03.182
> dcName=wewewew.google.com
> admonEventType=Update
> Names:
> objectCategory=CN=ms-DS-Az-Role,CN=Schema,CN=Configuration,DC=google,DC=com
> name=CRA3
> distinguishedName=CN=CRA,CN=AzRoleObjectContainer-f2c06b86-f897-4ca4-ac5e-2762c25c5da4,CN=f2c06b86-f897-4ca4-ac5e-2762c25c5da4,CN=636cb236-cdb1-443b-bfb3-7683dd85b2f4,CN=Authorization,CN=Corporate,OU=Zones,OU=UNIX,DC=google,DC=com
> cn=CRA
> Object Details:
> objectGUID=dd4fb895-3672-4f0c-bd73-f41f05205f37
> whenChanged=05:00.03 PM, Mon 04/11/2016
> whenCreated=04:59.49 PM, Mon 04/11/2016
> objectClass=top|msDS-AzRole
> Event Details:
> uSNChanged=1645647639
> uSNCreated=1645647635
> instanceType=4
> Additional Details:
> msDS-AzApplicationData=ptype=g
> msDS-TasksForAzRole=CN=role-Unix
> Sysadmin,CN=AzTaskObjectContainer-636cb236-cdb1-443b-bfb3-7683dd85b2f4,CN=636cb236-cdb1-443b-bfb3-7683dd85b2f4,CN=Authorization,CN=Corporate,OU=Zones,OU=UNIX,DC=google,DC=com
> msDS-MembersForAzRole=CN=PAWS_ENVPR_DDEPROD_ADM,OU=Bigdata,OU=Groups,DC=google,DC=com
> dSCorePropagationData=1601010100.0Z
> showInAdvancedViewOnly=TRUE
> Data after parsing:
> { "timestamp": "April 11th 2016 17:00:03 (NOTE: Timezone unknown. Solve for
> this)", "hostname": "wewewew", "dcName": "wewewew.google.com",
> "admonEventType": "Update", "names.objectCategory":
> "CN=ms-DS-Az-Role,CN=Schema,CN=Configuration,DC=google,DC=com", "names.name":
> "CRA", "names.distinguishedName":
> "CN=CRA,CN=AzRoleObjectContainer-f2c06b86-f897-4ca4-ac5e-2762c25c5da4,CN=f2c06b86-f897-4ca4-ac5e-2762c25c5da4,CN=636cb236-cdb1-443b-bfb3-7683dd85b2f4,CN=Authorization,CN=Corporate,OU=Zones,OU=UNIX,DC=google,DC=com",
> "names.cn": "CRA", "object.objectGUID":
> "dd4fb895-3672-4f0c-bd73-f41f05205f37", "object.whenChanged": "05:00.03 PM,
> Mon 04/11/2016", "object.whenCreated": "04:59.49 PM, Mon 04/11/2016",
> "object.objectClass": "top|msDS-AzRole", "event.uSNChanged": "1645647639",
> "event.uSNCreated": "1645647635", event.instanceType": "4",
> "additional.msDS-AzApplicationData": "ptype=g",
> "additional.msDS-TasksForAzRole": "CN=role-Unix
> Sysadmin,CN=AzTaskObjectContainer-636cb236-cdb1-443b-bfb3-7683dd85b2f4,CN=636cb236-cdb1-443b-bfb3-7683dd85b2f4,CN=Authorization,CN=Corporate,OU=Zones,OU=UNIX,DC=google,DC=com",
> "additional.msDS-MembersForAzRole":
> "CN=PAWS_ENVPR_DDEPROD_ADM,OU=Bigdata,OU=Groups,DC=google,DC=com",
> "additional.dSCorePropagationData": "1601010100.0Z",
> "additional.showInAdvancedViewOnly": "TRUE" }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (METRON-165) Create Windows Syslog Parser
[
https://issues.apache.org/jira/browse/METRON-165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Sirota updated METRON-165:
Priority: Minor (was: Major)
> Create Windows Syslog Parser
>
>
> Key: METRON-165
> URL: https://issues.apache.org/jira/browse/METRON-165
> Project: Metron
> Issue Type: New Feature
>Reporter: Deeptaanshu Kumar
>Assignee: James Sirota
>Priority: Minor
> Labels: ParserExtension
> Fix For: 0.2.2BETA
>
>
> Create a parser for Windows Sylog.
> Below are sample messages and their expected parsed output:
> <13> ABC 02/05/2016 09:54:39 AM
> LogName=Security
> SourceName=Microsoft Windows security auditing.
> EventCode=4624
> EventType=0
> Type=Information
> ComputerName=ABC.google.com
> TaskCategory=Logon
> OpCode=Info
> RecordNumber=112720121
> Keywords=Audit Success
> Message=An account was successfully logged on.
> Subject:
> Security ID:NULL SID
> Account Name: -
> Account Domain: -
> Logon ID: 0x0
> Logon Type: 3
> New Logon:
> Security ID:ABC
> Account Name: ABC
> Account Domain: ABC
> Logon ID: 0x4e149e04
> Logon GUID: {89C4AB77-51D6-D17B-3EAD-BC8676D1A4D2}
> Process Information:
> Process ID: 0x0
> Process Name: -
> Network Information:
> Workstation Name:
> Source Network Address: 10.0.0.0
> Source Port:64340
> Detailed Authentication Information:
> Logon Process: Kerberos
> Authentication Package: Kerberos
> Transited Services: -
> Package Name (NTLM only): -
> Key Length: 0
> This event is generated when a logon session is created. It is generated on
> the computer that was accessed.
> The subject fields indicate the account on the local system which requested
> the logon. This is most commonly a service such as the Server service, or a
> local process such as Winlogon.exe or Services.exe.
> The logon type field indicates the kind of logon that occurred. The most
> common types are 2 (interactive) and 3 (network).
> The New Logon fields indicate the account for whom the new logon was created,
> i.e. the account that was logged on.
> The network fields indicate where a remote logon request originated.
> Workstation name is not always available and may be left blank in some cases.
> The authentication information fields provide detailed information about this
> specific logon request.
> - Logon GUID is a unique identifier that can be used to correlate this
> event with a KDC event.
> - Transited services indicate which intermediate services have
> participated in this logon request.
> - Package name indicates which sub-protocol was used among the NTLM
> protocols.
> - Key length indicates the length of the generated session key. This
> will be 0 if no session key was requested.
> Here is the sample output:
> {"computer_name":"ABC.google.com","keywords":"Audit
> Success","log_name":"Security","record_number":"112720121","device_generated_timestamp":1454666079000,"source_type":"Windows
> Syslog","message":"An account was successfully logged
> on.\nSubject:\n\tSecurity ID:\t\tNULL SID\n\tAccount Name:\t\t-\n\tAccount
> Domain:\t\t-\n\tLogon ID:\t\t0x0\nLogon Type:\t\t\t3\nNew Logon:\n\tSecurity
> ID:\t\tABC\\ABC\n\tAccount Name:\t\tABC\n\tAccount Domain:\t\tABC\n\tLogon
> ID:\t\t0x4e149e04\n\tLogon
> GUID:\t\t{89C4AB77-51D6-D17B-3EAD-BC8676D1A4D2}\nProcess
> Information:\n\tProcess ID:\t\t0x0\n\tProcess Name:\t\t-\nNetwork
> Information:\n\tWorkstation Name:\t\n\tSource Network
> Address:\t10.0.0.0\n\tSource Port:\t\t64340\nDetailed Authentication
> Information:\n\tLogon Process:\t\tKerberos\n\tAuthentication
> Package:\tKerberos\n\tTransited Services:\t-\n\tPackage Name (NTLM
> only):\t-\n\tKey Length:\t\t0\nThis event is generated when a logon session
> is created. It is generated on the computer that was accessed.\nThe subject
> fields indicate the account on the local system which requested the logon.
> This is most commonly a service such as the Server service, or a local
> process such as Winlogon.exe or Services.exe.\nThe logon type field indicates
> the kind of logon that occurred. The most common types are 2 (interactive)
> and 3 (network).\nThe New Logon fields indicate the account for whom the new
> logon was created, i.e. the account that was logged on.\nThe network fields
> indicate where a remote logon request originated. Workstation name is not
> always available and may be left blank in some cases.\nThe authentication
> information fields provide detailed information about this specific logon
>
[jira] [Updated] (METRON-154) Decouple enrichment and indexing
[ https://issues.apache.org/jira/browse/METRON-154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-154: Assignee: Nick Allen (was: Ryan Merriman) Priority: Minor (was: Major) > Decouple enrichment and indexing > > > Key: METRON-154 > URL: https://issues.apache.org/jira/browse/METRON-154 > Project: Metron > Issue Type: New Feature >Reporter: Ryan Merriman >Assignee: Nick Allen >Priority: Minor > Fix For: 0.2.2BETA > > > This task involves adding another layer of abstraction between enrichment and > indexing through the use of Kafka topics. The primary driver is the use case > where a sensor is parsed in a parser topology but doesn't necessarily need to > be enriched. This would allow parsed sensor messages to be indexed directly > without putting unnecessary load on the enrichment topology. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-153) Add support for Centos 7
[ https://issues.apache.org/jira/browse/METRON-153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-153: Priority: Minor (was: Major) Fix Version/s: (was: 0.2.2BETA) > Add support for Centos 7 > > > Key: METRON-153 > URL: https://issues.apache.org/jira/browse/METRON-153 > Project: Metron > Issue Type: Improvement >Reporter: David M. Lyle >Priority: Minor > Labels: 0.2.2BETA > Fix For: 0.2.2BETA > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-144) Transient failure in intergration testing
[ https://issues.apache.org/jira/browse/METRON-144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-144: Priority: Minor (was: Major) > Transient failure in intergration testing > - > > Key: METRON-144 > URL: https://issues.apache.org/jira/browse/METRON-144 > Project: Metron > Issue Type: Bug >Reporter: David M. Lyle >Priority: Minor > Fix For: 0.2.1BETA > > > Integration test sometimes fails with: > Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 36.011 sec > <<< FAILURE! - in > org.apache.metron.pcap.integration.PcapTopologyIntegrationTest > testTimestampInPacket(org.apache.metron.pcap.integration.PcapTopologyIntegrationTest) > Time elapsed: 1.727 sec <<< ERROR! > java.lang.NullPointerException > at > org.apache.metron.integration.components.FluxTopologyComponent.stop(FluxTopologyComponent.java:100) > at > org.apache.metron.integration.ComponentRunner.stop(ComponentRunner.java:120) > at > org.apache.metron.pcap.integration.PcapTopologyIntegrationTest.testTopology(PcapTopologyIntegrationTest.java:352) > at > org.apache.metron.pcap.integration.PcapTopologyIntegrationTest.testTimestampInPacket(PcapTopologyIntegrationTest.java:128) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:483) > at org.junit.internal.runners.TestMethod.invoke(TestMethod.java:59) > at > org.junit.internal.runners.MethodRoadie.runTestMethod(MethodRoadie.java:98) > at org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie.java:79) > at > org.junit.internal.runners.MethodRoadie.runBeforesThenTestThenAfters(MethodRoadie.java:87) > at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie.java:77) > at org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:42) > at > org.junit.internal.runners.JUnit4ClassRunner.invokeTestMethod(JUnit4ClassRunner.java:88) > at > org.junit.internal.runners.JUnit4ClassRunner.runMethods(JUnit4ClassRunner.java:51) > at > org.junit.internal.runners.JUnit4ClassRunner$1.run(JUnit4ClassRunner.java:44) > at > org.junit.internal.runners.ClassRoadie.runUnprotected(ClassRoadie.java:27) > at > org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:37) > at > org.junit.internal.runners.JUnit4ClassRunner.run(JUnit4ClassRunner.java:42) > at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153) > at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128) > at > org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203) > at > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155) > at > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) > Results : > Tests in error: > PcapTopologyIntegrationTest.testTimestampInPacket:128->testTopology:352 » > NullPointer -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-99) Make separate config.properties for sensors as well as enrichment
[ https://issues.apache.org/jira/browse/METRON-99?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-99: --- Priority: Minor (was: Major) > Make separate config.properties for sensors as well as enrichment > - > > Key: METRON-99 > URL: https://issues.apache.org/jira/browse/METRON-99 > Project: Metron > Issue Type: Improvement >Reporter: Casey Stella >Priority: Minor > Fix For: 0.2.1BETA > > > Right now we have one config.properties for the enrichment topology as well > as the various sensor topologies. This is confusing as it's unclear which > properties are relevant and which are not. We should split this up. There > are a couple ways to do it, either have one sensor properties that is used > across all the sensors or have one sensor properties per sensor. I tend to > favor the latter as I suspect each sensor may have its own configs. > Part of this task should be to ensure that the appropriate sensor properties > are parameterized in vagrant as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-134) EC2 Deployment Will Continue Even if All Hosts Are Not Ready
[
https://issues.apache.org/jira/browse/METRON-134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Sirota updated METRON-134:
Priority: Minor (was: Major)
> EC2 Deployment Will Continue Even if All Hosts Are Not Ready
>
>
> Key: METRON-134
> URL: https://issues.apache.org/jira/browse/METRON-134
> Project: Metron
> Issue Type: Bug
> Environment: amazon-ec2
>Reporter: Nick Allen
>Priority: Minor
> Fix For: 0.2.1BETA
>
>
> The deployment process should stop immediately if all hosts needed for Metron
> are not ready to continue the deployment process. Otherwise, known as the
> "Jimmy Lin" issues. :)
> TASK [setup]
> ***
> fatal: [ec2-54-200-158-35.us-west-2.compute.amazonaws.com]: UNREACHABLE! =>
> {"changed": false, "msg": "Failed to connect to the host via ssh.",
> "unreachable": true}
> fatal: [ec2-54-186-101-72.us-west-2.compute.amazonaws.com]: UNREACHABLE! =>
> {"changed": false, "msg": "Failed to connect to the host via ssh.",
> "unreachable": true}
> fatal: [ec2-54-186-227-146.us-west-2.compute.amazonaws.com]: UNREACHABLE! =>
> {"changed": false, "msg": "Failed to connect to the host via ssh.",
> "unreachable": true}
> fatal: [ec2-54-191-198-134.us-west-2.compute.amazonaws.com]: UNREACHABLE! =>
> {"changed": false, "msg": "Failed to connect to the host via ssh.",
> "unreachable": true}
> fatal: [ec2-54-187-163-186.us-west-2.compute.amazonaws.com]: UNREACHABLE! =>
> {"changed": false, "msg": "Failed to connect to the host via ssh.",
> "unreachable": true}
> fatal: [ec2-54-186-225-238.us-west-2.compute.amazonaws.com]: UNREACHABLE! =>
> {"changed": false, "msg": "Failed to connect to the host via ssh.",
> "unreachable": true}
> fatal: [ec2-54-200-145-97.us-west-2.compute.amazonaws.com]: UNREACHABLE! =>
> {"changed": false, "msg": "Failed to connect to the host via ssh.",
> "unreachable": true}
> fatal: [ec2-54-187-172-159.us-west-2.compute.amazonaws.com]: UNREACHABLE! =>
> {"changed": false, "msg": "Failed to connect to the host via ssh.",
> "unreachable": true}
> ok: [ec2-54-187-25-6.us-west-2.compute.amazonaws.com]
> ok: [ec2-54-186-18-33.us-west-2.compute.amazonaws.com]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (METRON-98) Travis does not fail a build if the integration tests fail
[ https://issues.apache.org/jira/browse/METRON-98?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-98: --- Priority: Minor (was: Major) > Travis does not fail a build if the integration tests fail > -- > > Key: METRON-98 > URL: https://issues.apache.org/jira/browse/METRON-98 > Project: Metron > Issue Type: Bug >Reporter: Ryan Merriman >Assignee: Ryan Merriman >Priority: Minor > Fix For: 0.2.1BETA > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-75) Expand Volume Should Only Run Once
[ https://issues.apache.org/jira/browse/METRON-75?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-75: --- Priority: Minor (was: Major) > Expand Volume Should Only Run Once > --- > > Key: METRON-75 > URL: https://issues.apache.org/jira/browse/METRON-75 > Project: Metron > Issue Type: Improvement >Reporter: David M. Lyle >Priority: Minor > Labels: deployment, easyfix, newbie > Fix For: 0.2.1BETA > > > Current expand-volume would be re-run if the xvda_vol_size variable is > increased between runs. It should only be run during initial provisioning of > the VMs. Put a guard in to make sure it does not run after initial > provisioning. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-277) Zookeeper config access control management
[ https://issues.apache.org/jira/browse/METRON-277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-277: Description: I need to have access control and audit trail around who can access, upload, modify, and view Metron's zookeeper configs Summary: Zookeeper config access control management (was: I need to have access control and audit trail around who can access, upload, modify, and view Metron's zookeeper configs) > Zookeeper config access control management > --- > > Key: METRON-277 > URL: https://issues.apache.org/jira/browse/METRON-277 > Project: Metron > Issue Type: New Feature >Reporter: James Sirota > Fix For: 0.3.0BETA > > > I need to have access control and audit trail around who can access, upload, > modify, and view Metron's zookeeper configs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-277) I need to have access control and audit trail around who can access, upload, modify, and view Metron's zookeeper configs
James Sirota created METRON-277: --- Summary: I need to have access control and audit trail around who can access, upload, modify, and view Metron's zookeeper configs Key: METRON-277 URL: https://issues.apache.org/jira/browse/METRON-277 Project: Metron Issue Type: New Feature Reporter: James Sirota -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-276) I want to be able to run Metron on a pre-built Kerberized cluster
James Sirota created METRON-276: --- Summary: I want to be able to run Metron on a pre-built Kerberized cluster Key: METRON-276 URL: https://issues.apache.org/jira/browse/METRON-276 Project: Metron Issue Type: New Feature Reporter: James Sirota -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-274) Sign original string
James Sirota created METRON-274: --- Summary: Sign original string Key: METRON-274 URL: https://issues.apache.org/jira/browse/METRON-274 Project: Metron Issue Type: New Feature Reporter: James Sirota Priority: Minor I want to be able to sign the original string and have that signature incorporated into the Metron message body -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-75) Expand Volume Should Only Run Once
[ https://issues.apache.org/jira/browse/METRON-75?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-75: --- This is a newbie issue designed to introduce people from the community to the Metron project and get people started on the road to Metron committer and PPMC member. If you are interested in working on this issue please reach out to us on the Metron boards and existing metron committers and PPMC members will help you setup your environment and work on this issue. Thanks, and we look forward to having you as a part of the growing Metron community > Expand Volume Should Only Run Once > --- > > Key: METRON-75 > URL: https://issues.apache.org/jira/browse/METRON-75 > Project: Metron > Issue Type: Improvement >Reporter: David M. Lyle > Labels: deployment, easyfix, newbie > Fix For: 0.2.1BETA > > > Current expand-volume would be re-run if the xvda_vol_size variable is > increased between runs. It should only be run during initial provisioning of > the VMs. Put a guard in to make sure it does not run after initial > provisioning. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-139) Intermittent Test Failures
[ https://issues.apache.org/jira/browse/METRON-139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-139: This is a newbie issue designed to introduce people from the community to the Metron project and get people started on the road to Metron committer and PPMC member. If you are interested in working on this issue please reach out to us on the Metron boards and existing metron committers and PPMC members will help you setup your environment and work on this issue. Thanks, and we look forward to having you as a part of the growing Metron community > Intermittent Test Failures > -- > > Key: METRON-139 > URL: https://issues.apache.org/jira/browse/METRON-139 > Project: Metron > Issue Type: Bug > Environment: Travis CI Tests >Reporter: Nick Allen >Priority: Minor > Labels: newbie > Fix For: 0.2.1BETA > > > The automated tests run as part of the Travis CI build seem to be failing > intermittently, although rarely. > (1) > testTimestampInPacket(org.apache.metron.pcap.integration.PcapTopologyIntegrationTest) > Time elapsed: 23.05 sec <<< ERROR! > java.lang.NullPointerException > at > org.apache.metron.integration.components.FluxTopologyComponent.stop(FluxTopologyComponent.java:100) > at > org.apache.metron.integration.ComponentRunner.stop(ComponentRunner.java:120) > at > org.apache.metron.pcap.integration.PcapTopologyIntegrationTest.testTopology(PcapTopologyIntegrationTest.java:352) > at > org.apache.metron.pcap.integration.PcapTopologyIntegrationTest.testTimestampInPacket(PcapTopologyIntegrationTest.java:128) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:483) > at org.junit.internal.runners.TestMethod.invoke(TestMethod.java:59) > at > org.junit.internal.runners.MethodRoadie.runTestMethod(MethodRoadie.java:98) > at org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie.java:79) > at > org.junit.internal.runners.MethodRoadie.runBeforesThenTestThenAfters(MethodRoadie.java:87) > at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie.java:77) > at org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:42) > at > org.junit.internal.runners.JUnit4ClassRunner.invokeTestMethod(JUnit4ClassRunner.java:88) > at > org.junit.internal.runners.JUnit4ClassRunner.runMethods(JUnit4ClassRunner.java:51) > at > org.junit.internal.runners.JUnit4ClassRunner$1.run(JUnit4ClassRunner.java:44) > at > org.junit.internal.runners.ClassRoadie.runUnprotected(ClassRoadie.java:27) > at > org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:37) > at > org.junit.internal.runners.JUnit4ClassRunner.run(JUnit4ClassRunner.java:42) > at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153) > at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128) > at > org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203) > at > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155) > at > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) > Results : > Tests in error: > PcapTopologyIntegrationTest.testTimestampInPacket:128->testTopology:352 » > NullPointer > Tests run: 2, Failures: 0, Errors: 1, Skipped: 0 > (2) > test(org.apache.metron.elasticsearch.integration.ElasticsearchEnrichmentIntegrationTest) > Time elapsed: 120.086 sec <<< ERROR! > java.lang.RuntimeException: Too many retries: 11 > at > org.apache.metron.integration.ComponentRunner.process(ComponentRunner.java:140) > at > org.apache.metron.integration.EnrichmentIntegrationTest.test(EnrichmentIntegrationTest.java:208) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:483) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at >
[jira] [Updated] (METRON-87) Ansible Roles defining hdp_repo_def could duplicate repo definitions.
[
https://issues.apache.org/jira/browse/METRON-87?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Sirota updated METRON-87:
---
This is a newbie issue designed to introduce people from the community to the
Metron project and get people started on the road to Metron committer and PPMC
member. If you are interested in working on this issue please reach out to us
on the Metron boards and existing metron committers and PPMC members will help
you setup your environment and work on this issue. Thanks, and we look forward
to having you as a part of the growing Metron community
> Ansible Roles defining hdp_repo_def could duplicate repo definitions.
> ---
>
> Key: METRON-87
> URL: https://issues.apache.org/jira/browse/METRON-87
> Project: Metron
> Issue Type: Bug
>Reporter: David M. Lyle
> Labels: deployment, easyfix, newbie
> Fix For: 0.2.1BETA
>
>
> When roles contain
> - name: Retrieve HDP repository definition
> get_url:
> url: "{{ hdp_repo_def }}"
> dest: /etc/yum.repos.d/hdp.repo
> mode: 0644
> are assigned to hosts which are also hadoop_slave hosts, they will duplicate
> the HDP and HDP-UTILS repo definitions making yum unhappy.
> Proposed fix:
> Change the destination to /etc/yum.repos.d/HDP.repo.
> Move all definitions of hdp_repo_def to {role}/defaults/main.yml
> Change default location of repo file to:
> http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.4.0/hdp.repo
> Remove unused variable from the yaf role.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (METRON-257) Allow pcap result pagination from the Pcap CLI
[ https://issues.apache.org/jira/browse/METRON-257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-257: Labels: (was: 0.2.1BETA) > Allow pcap result pagination from the Pcap CLI > -- > > Key: METRON-257 > URL: https://issues.apache.org/jira/browse/METRON-257 > Project: Metron > Issue Type: Improvement >Reporter: Casey Stella > Fix For: 0.2.2BETA > > > Right now we are returning the whole result set as part of the PCap CLI. We > should allow for pagination of results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-206) Integrate with Ambari
[ https://issues.apache.org/jira/browse/METRON-206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-206: Labels: ForwardLookingEpic (was: 0.2.2BETA ForwardLookingEpic) > Integrate with Ambari > - > > Key: METRON-206 > URL: https://issues.apache.org/jira/browse/METRON-206 > Project: Metron > Issue Type: Wish >Reporter: James Sirota > Labels: ForwardLookingEpic > Fix For: 0.3.0BETA > > > Create a set of Ambari services for Metron so that we can rely on Ambari to > lay down the Hadoop cluster for us. This way we can provision Metron as an > Ambari application without having to worry about the underlying dependencies > of supporting different operating systems or versions of Hadoop. This would > significantly simplify our Ansible deployment scripts -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-267) Add Third Installer Option to Apache Metron Web Page
[ https://issues.apache.org/jira/browse/METRON-267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-267: Assignee: Ryan Merriman > Add Third Installer Option to Apache Metron Web Page > > > Key: METRON-267 > URL: https://issues.apache.org/jira/browse/METRON-267 > Project: Metron > Issue Type: Improvement >Reporter: George Vetticaden >Assignee: Ryan Merriman > Labels: 0.2.1BETA, METRON_UI > > With Metron .2 Release we have now officially support an third install > option. Installing Metron on an existing HDP managed cluster. > Hence we need to update the metron home page: > https://metron.incubator.apache.org/documentation/ to include the third > install option that links to the following: > https://cwiki.apache.org/confluence/display/METRON/Metron+Installation+on+a+Ambari+Managed+Cluster > Right now the home page only describes 2 install options. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-267) Add Third Installer Option to Apache Metron Web Page
[ https://issues.apache.org/jira/browse/METRON-267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-267: Labels: 0.2.1BETA METRON_UI (was: ) > Add Third Installer Option to Apache Metron Web Page > > > Key: METRON-267 > URL: https://issues.apache.org/jira/browse/METRON-267 > Project: Metron > Issue Type: Improvement >Reporter: George Vetticaden > Labels: 0.2.1BETA, METRON_UI > > With Metron .2 Release we have now officially support an third install > option. Installing Metron on an existing HDP managed cluster. > Hence we need to update the metron home page: > https://metron.incubator.apache.org/documentation/ to include the third > install option that links to the following: > https://cwiki.apache.org/confluence/display/METRON/Metron+Installation+on+a+Ambari+Managed+Cluster > Right now the home page only describes 2 install options. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-266) Remove duplicate entries of sniff_interface from metron_example inventory
[ https://issues.apache.org/jira/browse/METRON-266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-266: Assignee: Nick Allen > Remove duplicate entries of sniff_interface from metron_example inventory > - > > Key: METRON-266 > URL: https://issues.apache.org/jira/browse/METRON-266 > Project: Metron > Issue Type: Bug >Reporter: George Vetticaden >Assignee: Nick Allen >Priority: Minor > Labels: 0.2.1BETA > > There are duplicate entries of sniff_interface in the sample inventory file: > https://github.com/apache/incubator-metron/blob/master/metron-deployment/inventory/metron_example/group_vars/all. > > Remove the duplicates as you run into issues with them when you do the metron > install on an existing Ambari managed cluster based on these instructions: > https://cwiki.apache.org/confluence/display/METRON/Metron+Installation+on+a+Ambari+Managed+Cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-266) Remove duplicate entries of sniff_interface from metron_example inventory
[ https://issues.apache.org/jira/browse/METRON-266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-266: Labels: 0.2.1BETA (was: ) > Remove duplicate entries of sniff_interface from metron_example inventory > - > > Key: METRON-266 > URL: https://issues.apache.org/jira/browse/METRON-266 > Project: Metron > Issue Type: Bug >Reporter: George Vetticaden >Priority: Minor > Labels: 0.2.1BETA > > There are duplicate entries of sniff_interface in the sample inventory file: > https://github.com/apache/incubator-metron/blob/master/metron-deployment/inventory/metron_example/group_vars/all. > > Remove the duplicates as you run into issues with them when you do the metron > install on an existing Ambari managed cluster based on these instructions: > https://cwiki.apache.org/confluence/display/METRON/Metron+Installation+on+a+Ambari+Managed+Cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-272) ML_PLATFORM
James Sirota created METRON-272: --- Summary: ML_PLATFORM Key: METRON-272 URL: https://issues.apache.org/jira/browse/METRON-272 Project: Metron Issue Type: New Feature Reporter: James Sirota Assignee: James Sirota -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-271) Add graph DB to the platform
James Sirota created METRON-271: --- Summary: Add graph DB to the platform Key: METRON-271 URL: https://issues.apache.org/jira/browse/METRON-271 Project: Metron Issue Type: Bug Reporter: James Sirota I propose adding a graph database (Titan or others) so we can use graph mining as feature inputs to some of our models and anomaly detectors -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-270) Add Zeppelin to the platform
James Sirota created METRON-270: --- Summary: Add Zeppelin to the platform Key: METRON-270 URL: https://issues.apache.org/jira/browse/METRON-270 Project: Metron Issue Type: Bug Reporter: James Sirota I propose adding Zeppelin to the platform to aid in interactive dashboarding and data visualizations -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-269) Integrate Spark into the platform
James Sirota created METRON-269: --- Summary: Integrate Spark into the platform Key: METRON-269 URL: https://issues.apache.org/jira/browse/METRON-269 Project: Metron Issue Type: Bug Reporter: James Sirota I propose adding Spark to the project to aid in batch analytics and modeling -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (METRON-268) Add Jupyter to the platform
James Sirota created METRON-268: --- Summary: Add Jupyter to the platform Key: METRON-268 URL: https://issues.apache.org/jira/browse/METRON-268 Project: Metron Issue Type: Bug Reporter: James Sirota We need an analytics workbench for visualizing data and creating ML models. I propose having a Jupyter interface with R-Spark and Py-Spark enabled -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-153) Add support for Centos 7
[ https://issues.apache.org/jira/browse/METRON-153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-153: Labels: 0.2.2BETA (was: ) > Add support for Centos 7 > > > Key: METRON-153 > URL: https://issues.apache.org/jira/browse/METRON-153 > Project: Metron > Issue Type: Improvement >Reporter: David M. Lyle > Labels: 0.2.2BETA > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-191) mysql-password.yml ignores errors when setting up Mysql Password
[ https://issues.apache.org/jira/browse/METRON-191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-191: Labels: 0.2.1BETA deployment (was: deployment) > mysql-password.yml ignores errors when setting up Mysql Password > > > Key: METRON-191 > URL: https://issues.apache.org/jira/browse/METRON-191 > Project: Metron > Issue Type: Bug >Reporter: David M. Lyle > Labels: 0.2.1BETA, deployment > > mysql-password.yml ignores errors when setting up Mysql Password. If this > fails, the deployment will fail when initializing the GeoIP database in the > enrichment setup. This should fail on legitimate errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-206) Integrate with Ambari
[ https://issues.apache.org/jira/browse/METRON-206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-206: Labels: 0.2.2BETA ForwardLookingEpic (was: ForwardLookingEpic) > Integrate with Ambari > - > > Key: METRON-206 > URL: https://issues.apache.org/jira/browse/METRON-206 > Project: Metron > Issue Type: Wish >Reporter: James Sirota > Labels: 0.2.2BETA, ForwardLookingEpic > > Create a set of Ambari services for Metron so that we can rely on Ambari to > lay down the Hadoop cluster for us. This way we can provision Metron as an > Ambari application without having to worry about the underlying dependencies > of supporting different operating systems or versions of Hadoop. This would > significantly simplify our Ansible deployment scripts -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-205) Integrate with Cloudbreak
[ https://issues.apache.org/jira/browse/METRON-205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-205: Labels: 0.2.2BETA ForwardLookingEpic (was: ForwardLookingEpic) > Integrate with Cloudbreak > -- > > Key: METRON-205 > URL: https://issues.apache.org/jira/browse/METRON-205 > Project: Metron > Issue Type: Wish >Reporter: James Sirota > Labels: 0.2.2BETA, ForwardLookingEpic > Attachments: IMG_1108.JPG > > > I would like to integrate our deployment scripts with Cloudbreak so that we > can leverage it for performing cloud and bare metal installs. To do so we > would need to make two major modifications/additions to Cloudbreak. Just to > make sure everyone is on the same page I am attaching the architecture of the > Cloudbreak tool. See attached file. What we have to do is: > - Modify the deployer docker image and install a version of Ansible on there > in addition to the Salt installer that it already has > - Add Metron artifacts to the deployer docker image so that it can perform > disconnected installs via Ansible > - Have Cloudbreak lay down the cluster for us via Salt and then switch to > Ansible and install Metron on top of the cluster provisioned by Cloudbreak > This effectively gets us out of the cluster provisioning game and > significantly reduces what we need to do in Ansible -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-225) Amazon-ec2 run script continues running on Maven build error
[ https://issues.apache.org/jira/browse/METRON-225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-225: Labels: 0.2.1BETA (was: ) > Amazon-ec2 run script continues running on Maven build error > > > Key: METRON-225 > URL: https://issues.apache.org/jira/browse/METRON-225 > Project: Metron > Issue Type: Bug >Reporter: David M. Lyle > Labels: 0.2.1BETA > > metron-deployment/amazon-ec2/run.sh will continue to execute after the build > fails. The script should halt if Metron cannot be built. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-232) Improve Acking strategy in the topologies
[ https://issues.apache.org/jira/browse/METRON-232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-232: Labels: 0.2.1BETA (was: ) > Improve Acking strategy in the topologies > - > > Key: METRON-232 > URL: https://issues.apache.org/jira/browse/METRON-232 > Project: Metron > Issue Type: Improvement >Reporter: David M. Lyle > Labels: 0.2.1BETA > > Currently, we employ 2 acking strategies: > Parsers - Ack everything except bulk writer errors. > Enrichment - Ack nothing. > This should be improved to enforce guaranteed delivery with sensible replays. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-242) remove Squid pattern
[
https://issues.apache.org/jira/browse/METRON-242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Sirota updated METRON-242:
Labels: 0.2.1BETA (was: )
> remove Squid pattern
>
>
> Key: METRON-242
> URL: https://issues.apache.org/jira/browse/METRON-242
> Project: Metron
> Issue Type: Improvement
>Reporter: George Vetticaden
>Priority: Minor
> Labels: 0.2.1BETA
>
> when deploying metron on AWS, I noticed the following patterns created by
> default..
> -rw-r--r-- 3 hdfs hadoop 13427 2016-06-20 01:52
> /apps/metron/patterns/asa
> -rw-r--r-- 3 hdfs hadoop 5203 2016-06-20 01:52
> /apps/metron/patterns/common
> -rw-r--r-- 3 hdfs hadoop524 2016-06-20 01:52
> /apps/metron/patterns/fireeye
> -rw-r--r-- 3 hdfs hadoop 2552 2016-06-20 01:52
> /apps/metron/patterns/sourcefire
> -rw-r--r-- 3 hdfs hadoop242 2016-06-20 21:04
> /apps/metron/patterns/squid
> -rw-r--r-- 3 hdfs hadoop 2221 2016-06-20 01:52
> /apps/metron/patterns/websphere
> -rw-r--r-- 3 hdfs hadoop879 2016-06-20 01:52
> /apps/metron/patterns/yaf
> We need to remove the Squid patterns since that is only for code exercnise.
> If we are going to keep it, then it needs to be updated to the be the
> following:
> SQUID_DELIMITED %{NUMBER:timestamp} %{SPACE:UNWANTED} %{INT:elapsed}
> %{IPV4:ip_src_addr} %{WORD:action}/%{NUMBER:code} %{NUMBER:bytes}
> %{WORD:method} %{NOTSPACE:url} - %{WORD:UNWANTED}\/%{IPV4:ip_dst_addr}
> %{WORD:UNWANTED}\/%{WORD:UNWANTED}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (METRON-171) Add .class files to gitignore
[ https://issues.apache.org/jira/browse/METRON-171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-171: Labels: 0.2.1BETA (was: ) > Add .class files to gitignore > - > > Key: METRON-171 > URL: https://issues.apache.org/jira/browse/METRON-171 > Project: Metron > Issue Type: Improvement >Reporter: Phil Austin >Priority: Trivial > Labels: 0.2.1BETA > > Currently .class files are not ignored. They should be. Simply adding *.class > to gitignore -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-249) Field Transformation functions fail to handle invalid user inputs
[
https://issues.apache.org/jira/browse/METRON-249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Sirota updated METRON-249:
Labels: 0.2.1BETA (was: )
> Field Transformation functions fail to handle invalid user inputs
> --
>
> Key: METRON-249
> URL: https://issues.apache.org/jira/browse/METRON-249
> Project: Metron
> Issue Type: Bug
>Reporter: Neha Sinha
> Labels: 0.2.1BETA
> Attachments: LogException.rtf
>
>
> Hi,
> The field transformation functions fail to handle invalid user input .On
> providing invalid inputs the parser throws exceptions and fails to create the
> required indices in elasticsearch.
> ==
> Steps to Reproduce
> ==
> Edit the squid.json file and provide the following definition to it:-(Note-we
> are giving an invalid input :-123 to the URL_TO_HOST function)
> ---
> {
> "parserClassName": "org.apache.metron.parsers.GrokParser",
> "sensorTopic": "squid",
> "parserConfig": {
> "grokPath": "/patterns/squid",
> "patternLabel": "SQUID_DELIMITED",
> "timestampField": "timestamp"
> },
> "fieldTransformations" : [
> {
> "transformation" : "MTL"
> ,"output" : [ "full_hostname", "domain_without_subdomains" ]
> ,"config" : {
> "full_hostname" : “URL_TO_HOST(123)"
> ,"domain_without_subdomains" : "DOMAIN_REMOVE_SUBDOMAINS(full_hostname)"
> }
> }
>]
> }
>
> Replay Squid events/logs and monitor the logs in storm for squid topology.
> Attached exception log would be seen and no indexes would be created
> respective to the logs.
> Expected Behaviour :-
> 1.The error should be more clean.
> 2.Since we cannot validate the inputs the invalid inputs should be ignored
> and the indices should get created anyway based on the Grok parser output
> Regards,
> Neha
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (METRON-207) Integrate Metron with Ambari Metrics Service
[ https://issues.apache.org/jira/browse/METRON-207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-207: Labels: 0.2.1BETA (was: Beta0.2) > Integrate Metron with Ambari Metrics Service > - > > Key: METRON-207 > URL: https://issues.apache.org/jira/browse/METRON-207 > Project: Metron > Issue Type: Wish >Reporter: James Sirota > Labels: 0.2.1BETA > > I want to be able to visualize Metron metrics via Ambari by using the Ambari > Metrics Service -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (METRON-171) Add .class files to gitignore
[ https://issues.apache.org/jira/browse/METRON-171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347179#comment-15347179 ] James Sirota commented on METRON-171: - Hi Phil, what is the status on this? > Add .class files to gitignore > - > > Key: METRON-171 > URL: https://issues.apache.org/jira/browse/METRON-171 > Project: Metron > Issue Type: Improvement >Reporter: Phil Austin >Priority: Trivial > > Currently .class files are not ignored. They should be. Simply adding *.class > to gitignore -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-214) Build binary rpm as secondary artifacts of Maven build
[ https://issues.apache.org/jira/browse/METRON-214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-214: Labels: 0.2.1BETA (was: ) > Build binary rpm as secondary artifacts of Maven build > -- > > Key: METRON-214 > URL: https://issues.apache.org/jira/browse/METRON-214 > Project: Metron > Issue Type: Sub-task >Reporter: David M. Lyle > Labels: 0.2.1BETA > > In order to allow yum install of core Metron functionality, create a RPM to > package the build artifacts required to install "Metron Core". > Parser Topologies > Enrichment Topology > Writer Topologies > Metron Helper Scripts (initd/systemd, zk loaders, etc) > RPM install scripts will handle configuration and start up of deployed > components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-227) Add Time-Based Flushing to Writer Bolt
[ https://issues.apache.org/jira/browse/METRON-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-227: Labels: 0.2.1BETA (was: ) > Add Time-Based Flushing to Writer Bolt > -- > > Key: METRON-227 > URL: https://issues.apache.org/jira/browse/METRON-227 > Project: Metron > Issue Type: Bug >Reporter: Domenic Puzio >Assignee: Ajay Yadav > Labels: 0.2.1BETA > > We need to change the BulkMessageWriterBolt and BulkWriterComponent to use > time-based flushing when writing data to Elasticsearch or Solr. > Currently, we set a batch size, and the Writer waits for that number of > tuples to build up; however, Storm has a timeout value that prevents it from > waiting for too long. If the Writer does not get the batch size before the > timeout, then it recycles the tuples through the topology. In addition, Storm > only allows so many pending messages that have not been acked - if too many > messages are waiting for the bulk Writer, then it will recycle them through > the topology. This is not desired behavior and directly impacts the > performance of this Writer. We would like to be able to specify a unit of > time for which the topology would flush, writing the data it's currently > holding to Elasticsearch or Solr even if the batch size is not met. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-226) Field transformation utility for time zone conversion
[
https://issues.apache.org/jira/browse/METRON-226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Sirota updated METRON-226:
Labels: 0.2.1BETA (was: )
> Field transformation utility for time zone conversion
> -
>
> Key: METRON-226
> URL: https://issues.apache.org/jira/browse/METRON-226
> Project: Metron
> Issue Type: New Feature
>Reporter: Sunny Kumar
>Priority: Minor
> Labels: 0.2.1BETA
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> The user would be able to provide input of the timezone for a stream in json
> for the specific parser. The field transformation will convert the time stamp
> to UTC and will also take care of the Daylight Saving offset. The code in the
> JSON will look like:
> "fieldTransformations" : [
>{
> "input" : "timestamp",
> "transformation": "TRANSFORM_TO_UTC_EPOCH",
> "config":
> {
> "timeZone":"America/New_York"
> }
> }
> ],
> The timezone should be valid as per the list at:
> https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
> which are supported by the java.util.TimeZone package.
> An invalid timezone would be defaulted to UTC.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (METRON-240) Indexing Prioritization by Data Type
[ https://issues.apache.org/jira/browse/METRON-240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-240: Labels: 0.2.2BETA (was: ) > Indexing Prioritization by Data Type > > > Key: METRON-240 > URL: https://issues.apache.org/jira/browse/METRON-240 > Project: Metron > Issue Type: Improvement >Reporter: Domenic Puzio > Labels: 0.2.2BETA > > All data sources go through the Enrichment topology, which means that sources > should be indexed as they come into the Kafka queue. However, sometimes (for > example, during a DDOS attack) one sensor could double or triple in volume > for a period of time, swamping the Enrichment topology during that time and > preventing data from other sensors from being indexed. > We would like to be able to prioritize data from particular sensors so that > even if one sensor is extra volume-heavy, the high-priority sources are still > indexed. So if we are running 3 parser topologies, we would like to give them > a prioritization so that one gets indexed before the others in the case that > the Enrichment topology cannot keep up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-257) Allow pcap result pagination from the Pcap CLI
[ https://issues.apache.org/jira/browse/METRON-257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-257: Labels: 0.2.1BETA (was: ) > Allow pcap result pagination from the Pcap CLI > -- > > Key: METRON-257 > URL: https://issues.apache.org/jira/browse/METRON-257 > Project: Metron > Issue Type: Improvement >Reporter: Casey Stella > Labels: 0.2.1BETA > > Right now we are returning the whole result set as part of the PCap CLI. We > should allow for pagination of results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (METRON-258) Allow sideloading of parsers
[ https://issues.apache.org/jira/browse/METRON-258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Sirota updated METRON-258: Labels: 0.2.1BETA (was: ) > Allow sideloading of parsers > > > Key: METRON-258 > URL: https://issues.apache.org/jira/browse/METRON-258 > Project: Metron > Issue Type: Improvement >Reporter: Casey Stella > Labels: 0.2.1BETA > > Right now custom parsers must be implemented within metron's metron-parsers > project. We should allow side-loading of jars when submitting parsers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
