[GitHub] metron pull request #481: METRON-322 Global Batching and Flushing
Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/481 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] metron issue #481: METRON-322 Global Batching and Flushing
Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/481 Updated and re-ran unit and integration tests. @ottobackwards , metron-solr integration tests ran fine. I did see a problem with metron-rest integration test org.apache.metron.rest.controller.UpdateControllerIntegrationTest , but it happens in current master branch without this PR too. @dlyle65535 , proceeding with commit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] metron issue #481: METRON-322 Global Batching and Flushing
Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/481 @dlyle65535 yes, I'll clean this up. I'll run integration tests one more time to make sure I don't see @ottobackwards 's problem, then commit on the strength of his and your +1. Let me know if you disagree. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] metron issue #689: METRON-1102: Add support for ingesting cybox URI observab...
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/689 +1 by inspection. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] metron issue #689: METRON-1102: Add support for ingesting cybox URI observab...
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/689 I would like to see at least the version of Stix and Cybox that is supported documented. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 Hey @ottobackwards. I see no benefit to getting this committed prior to or separate from shoring up the docs. I'm going to work on something to get the ball rolling as I work through some use case testing on this, and I'm working on it now. I feel pretty strongly about making sure the documentation is clear for this PR. We need to be careful about making this accessible to people considering the breadth of architectural change this makes for our classloading and deployment model. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] metron issue #689: METRON-1102: Add support for ingesting cybox URI observab...
Github user cestella commented on the issue: https://github.com/apache/metron/pull/689 Yeah, it seems to me that we might need something like a location for discussion and description of architectural decisions made. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] metron issue #689: METRON-1102: Add support for ingesting cybox URI observab...
Github user cestella commented on the issue: https://github.com/apache/metron/pull/689 Ah, for the moment we only have the StixExtractor. We could MAKE a cybox extractor and it could reuse the handler logic, but we haven't done that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] metron issue #689: METRON-1102: Add support for ingesting cybox URI observab...
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/689 As for where... over in METRON-777 @mmiklavc is talking about a better documentation area... this may fall into that --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] metron issue #689: METRON-1102: Add support for ingesting cybox URI observab...
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/689 > Can we handle just Cybox What I mean is, can we import a file that is **ONLY** cybox observables, without Stix. Stix supports cybox. But they are separate things. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] metron issue #689: METRON-1102: Add support for ingesting cybox URI observab...
Github user cestella commented on the issue: https://github.com/apache/metron/pull/689 Ok, so good questions @ottobackwards . I'll do my best to answer them, but the answer to some of these expands past this PR and to the history of Taxii support for Metron (which was one of the first things we added and thus at a period of time where documentation was scarcer than it is even now ;) ) * `Where is the documentation for the version of Stix and the Version of Cybox metron supports?` We do not currently document that, the answer is, however, that our support for Stix, cybox and taxii is entirely bound up in the mitre java-stix library. We use the most current version [released](https://github.com/STIXProject/java-stix/tree/v1.2.0.2), which is 1.2.0.2. * `How is the extractor factored to handle support for other versions?` The extractor is handled to support other versions only insomuch as the java-stix library can support multiple versions. As this is officially supported by the stix project, I think that it's backwards compatible, but there may be nuance here that I'm missing. * `How is the extractor factored to handle JSON if / when stix and cybox change over?` The extractor abstraction works at the level of the object model that the java-stix library provides us rather than doing actual parsing (i.e. we implement support for new types by providing a handler that looks for objects of that type as the output of the parse). If Stix moves to JSON, presumably the library will handle that transparently *or* we'll need another approach. * `Can we handle just Cybox? Should this be factored to support them separately?` We can create handlers for anything the java-stix library can parse, but cybox seems to be common and officially supported by the stix project. * `Where is the documentation for the support in this PR?` I added the new URI type to the README.md in metron-data-management. Since that's the scope of this PR, not to document better our taxii/stix/etc support. Ok, so it's apparent that some of the design decisions around taxii never made it into documentation. A couple of questions for you: * Where should that documentation live? * Are we unhappy enough about having our abstraction bound to the (from what I can tell only) java library provided by the stix project that you'd like to start a discuss thread about developing a better approach to taxii? Just a note on the second, we chose that because it was the only game in town other than parsing the XML ourselves and it was the officially supported library. I even looked into that and decided against it as the XML format is extremely complex with lots of referential links that need to get coalesced to handle the blocks of stix that come across. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] metron pull request #689: METRON-1102: Add support for ingesting cybox URI o...
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/689#discussion_r132481074
--- Diff:
metron-platform/metron-data-management/src/main/java/org/apache/metron/dataloads/extractor/stix/StixExtractor.java
---
@@ -38,6 +39,7 @@
import java.util.Map;
public class StixExtractor implements Extractor {
+private static final Logger LOG =
Logger.getLogger(StixExtractor.class);
--- End diff --
Yeah, this ended up starting live as a patch against an older branch that I
just made current. I changed the logging to use slf4j now. Good eye,
@simonellistonball
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/620 Great. This is good to go. Going to merge now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] metron pull request #683: METRON-1084: Management UI web server license shou...
Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/683 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] metron issue #688: METRON-1094: MaaS will not start due to classpath error r...
Github user anandsubbu commented on the issue: https://github.com/apache/metron/pull/688 I'm +1 (non-binding) Fired up a full dev and was able to test that `maas_service.sh` starts up fine. No errors seen in the YARN application logs. A few observations though: a) I am not sure if its my setup, but the maas uber jar, maas_deploy and maas_service scripts were missing from `$METRON_HOME/bin` folder on my quick dev. I had to SCP `metron-maas-service-0.4.1-archive.tar.gz` and untar to get these binaries. Is this expected? b) The `maas_service.sh` script can only be started as *hdfs* user. Please let me know if this needs to be updated in the README, and I can add this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] metron issue #531: METRON-854 create dhcp dump parser
Github user basvdl commented on the issue: https://github.com/apache/metron/pull/531 @simonellistonball after some testing we concluded that Bro is not giving the output we want (source: https://bro-tracker.atlassian.net/browse/BIT-1630). The output doesn't contain hostnames, so the relation IP / Hostname can't be made. I still agree on modifying the source, DHCPDump, is not the preferred way to go. Can you assist in how to ship and parse multi-line log events, so I can adjust the parser accordingly without messing with the source? Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
