[jira] [Commented] (METRON-363) Fix Cisco ASA Parser
[
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15581043#comment-15581043
]
ASF GitHub Bot commented on METRON-363:
---
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/276#discussion_r83572123
--- Diff:
metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/utils/SyslogUtilsTest.java
---
@@ -0,0 +1,61 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import org.apache.metron.parsers.ParseException;
+import org.junit.Test;
+
+import java.time.ZoneOffset;
+
+import static org.junit.Assert.*;
+
+public class SyslogUtilsTest {
--- End diff --
Bummer. Sorry, I have no advice. Any Mockito experts out there?
> Fix Cisco ASA Parser
>
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
> Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser
> to support the variety of syslog messages produced by Cisco ASA devices as
> well as provide the necessary support files/configs for easier deployment of
> the Storm topology.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (METRON-505) Add environment variable and system property functions to the Stellar language
[
https://issues.apache.org/jira/browse/METRON-505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15580893#comment-15580893
]
ASF GitHub Bot commented on METRON-505:
---
GitHub user mmiklavc opened a pull request:
https://github.com/apache/incubator-metron/pull/312
METRON-505: Add environment variable and system property functions to the
Stellar language
Tested in quick-dev from the Stellar REPL.
Build the platform, as normal. The new functions will get picked up
automatically on the classpath via the annotations.
**Examples:**
```
SYSTEM_ENV_GET for exposing environment variables
[root@node1 bin]# export MIKE_VAR="blah blah blah"
[root@node1 bin]# echo $MIKE_VAR
blah blah blah
[Stellar]>>> mikevar := SYSTEM_ENV_GET('MIKE_VAR')
[Stellar]>>> mikevar
blah blah blah
SYSTEM_PROPERTY_GET for exposing system properties
[Stellar]>>> prop := SYSTEM_PROPERTY_GET( 'os.name' )
[Stellar]>>> prop
Linux
[Stellar]>>> prop := SYSTEM_PROPERTY_GET( 'user.name' )
[Stellar]>>> prop
root
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mmiklavc/incubator-metron add-sysfuncs-stellar
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/312.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #312
commit 9ce57ee69e852c8f4816aa5822eb6b21ab9bbdca
Author: Michael Miklavcic
Date: 2016-10-17T01:26:47Z
METRON-505: Add environment variable and system property functions to the
Stellar language
> Add environment variable and system property functions to the Stellar language
> --
>
> Key: METRON-505
> URL: https://issues.apache.org/jira/browse/METRON-505
> Project: Metron
> Issue Type: Improvement
>Reporter: Michael Miklavcic
>Assignee: Michael Miklavcic
>
> Stellar and the Stellar REPL allow you assign variables and compose functions
> for use in parsing, enrichment, and threat triage. There is currently no
> mechanism for extracting values from environment variables and Java system
> properties. This Jira tracks adding functions for exposing this functionality
> in Stellar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (METRON-505) Add environment variable and system property functions to the Stellar language
Michael Miklavcic created METRON-505: Summary: Add environment variable and system property functions to the Stellar language Key: METRON-505 URL: https://issues.apache.org/jira/browse/METRON-505 Project: Metron Issue Type: Improvement Reporter: Michael Miklavcic Assignee: Michael Miklavcic Stellar and the Stellar REPL allow you assign variables and compose functions for use in parsing, enrichment, and threat triage. There is currently no mechanism for extracting values from environment variables and Java system properties. This Jira tracks adding functions for exposing this functionality in Stellar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (METRON-363) Fix Cisco ASA Parser
[
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15580412#comment-15580412
]
ASF GitHub Bot commented on METRON-363:
---
Github user kylerichardson commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/276#discussion_r83558126
--- Diff:
metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/utils/SyslogUtilsTest.java
---
@@ -0,0 +1,61 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import org.apache.metron.parsers.ParseException;
+import org.junit.Test;
+
+import java.time.ZoneOffset;
+
+import static org.junit.Assert.*;
+
+public class SyslogUtilsTest {
--- End diff --
Yes, in theory that is.
I tried doing just that with PowerMock and Mockito; unfortunately, it seems
there is a bug in the underlying `javassist` library that doesn't play well
with the new `java.time` classes.
The issue is reportedly fixed in Javassist 3.20.0-GA; however, it doesn't
appear that PowerMock has updated to this version.
Reference: [JASSIST-246](https://issues.jboss.org/browse/JASSIST-246) and
https://github.com/jayway/powermock/issues/557
> Fix Cisco ASA Parser
>
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
> Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser
> to support the variety of syslog messages produced by Cisco ASA devices as
> well as provide the necessary support files/configs for easier deployment of
> the Storm topology.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (METRON-502) Make the ParserIntegrationTest more clear on errors
[ https://issues.apache.org/jira/browse/METRON-502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15580251#comment-15580251 ] ASF GitHub Bot commented on METRON-502: --- GitHub user ottobackwards opened a pull request: https://github.com/apache/incubator-metron/pull/311 METRON-502 Make the ParserIntegrationTest more clear on errors This is an attempt to enhance the Integration Test framework to support exposing processor errors and invalid messages. There are two goals here: 1. Expose errors that are in present but not exposed because the kafka queues or other error sources are not check or exposed through the Processor run 2. Allow the integration tests to fail quicker as opposed to timing out ( if for example there are invalid messages or errors in the kafka queue such that the message in never == message out the test will keep re-trying until timeout ). A new type ProcessorResult has been added to the system as the return from running a Processor. This type encapsulates the results, errors and invalids from any given run. All integration tests with master run. As always any feedback is appreciated. Specifically in two areas: - Is there a better way, or a way more consistent with the system than introducing the ProcessorResult type? - The integration project does not have any tests, should it ( unit tests?) - What is the best way to test this? Should each individual test have a 'failure mode' test as well? You can merge this pull request into a Git repository by running: $ git pull https://github.com/ottobackwards/incubator-metron METRON-502 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-metron/pull/311.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #311 commit 5b993beb9017ad8554ed72f5d315169ba2b67c56 Author: Otto Fowler Date: 2016-10-15T13:19:58Z first pass at refactoring to support error and invalid messages. Squid Test is failing validation, but that is message content( json/json ) around domain without subdomains, which may be addressed in a different PR - I need to check commit befe5f0408d8cf738df2c319e922240549771263 Author: Otto Fowler Date: 2016-10-16T16:24:04Z update other tests - mvn integration-test now running completely commit 964940067ce0cfac0eeb177dc76a80b3931e4385 Author: Otto Fowler Date: 2016-10-16T16:25:12Z Merge remote-tracking branch 'apache/master' into METRON-502 > Make the ParserIntegrationTest more clear on errors > --- > > Key: METRON-502 > URL: https://issues.apache.org/jira/browse/METRON-502 > Project: Metron > Issue Type: Improvement >Affects Versions: 0.2.1BETA >Reporter: Casey Stella > > Right now the ParserIntegrationTest fails if it does not receive the expected > number of messages after a given amount of time. In situations where > messages get redirected to the error or invalid queues, this results in the > test waiting and failing at the end of a timeout. The developer cannot > determine if the cause is a timeout or actual errors. > We should, instead, dump out the error and invalid queue as soon as it > becomes populated and fail the test with more specific reason. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
