[GitHub] incubator-metron issue #413: METRON-654 Create RPM Installer for Profiler

[mattf-horton]

Github user mattf-horton commented on the issue:

https://github.com/apache/incubator-metron/pull/413
  
+1 Used these RPMs to add Profiler, after installing a single-node Centos7 
vm with the Ambari mpack, using OracleJDK8, HDP-2.5.3, and Ambari-2.4.2.  
Performed `rpm -ivh metron-profiler.rpm`, validated as above, then did 
`bin/start_profiler_topology.sh`, and used it to test PR-414, during which Bro 
data was pumped through the usual sequence of topologies, and was observed in 
Storm UI to propagate through all into indexer and profiler. Elasticsearch and 
HBase were queried from outside Metron to confirm presence and format of data, 
and PROFILE_GET was used in Stellar REPL to read the profile data back out as 
expected.

Looking forward to the mpack additions!  Thanks, @nickwallen .


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #414: METRON-532 Define Profile Period When Calling P...

[mattf-horton]

Github user mattf-horton commented on the issue:

https://github.com/apache/incubator-metron/pull/414
  
@cestella @mmiklavc @nickwallen , thank you for the review comments.  
Please see latest for improvements to comments per your suggestions.  No code 
was changed.  Also fixed a doc bug regarding profiler "init" syntax in example 
code.

This code has been validated on a single-node 16GB Centos7 vm with 
OracleJDK8, HDP-2.5.3, and Ambari-2.4.2.  The cluster was installed via my 
metron-mpack-singlenode, then Profiler was added with @nickwallen 's RPMs.  Bro 
data was pumped through the usual sequence of topologies, and was observed in 
Storm UI to propagate through all into indexer and profiler.  Elasticsearch and 
HBase were queried from outside Metron to confirm presence and format of data, 
and PROFILE_GET was used in Stellar REPL to read the profile data back out as 
expected.  Use of the config_overrides map argument in PROFILE_GET was also 
observed to be effective.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request #414: METRON-532 Define Profile Period When Ca...

[nickwallen]

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/414#discussion_r95707005
  
--- Diff: 
metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/GetProfile.java
 ---
@@ -132,36 +157,26 @@
 
   private static final Logger LOG = 
LoggerFactory.getLogger(GetProfile.class);
 
-  /**
-   * A client that can retrieve profile values.
-   */
+  // Cached client that can retrieve profile values.
--- End diff --

You can leave it however you like.  I was just curious for my own future 
reference.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Kibana Tile Map not available

[Dima Kovalyov]

Hello,

I have attached screenshot of how my Tile Map looks like right now
(kibana_tilemap.png).
Basically there is no geo map, only dots that represent location on the map.

It appears that geo map images are hosted on external CDN which is now
not available for me. I feel like it is not available for you either.
Is that so?
Please advise.

Example URLs:
https://otile3-s.mqcdn.com/tiles/1.0.0/map/2/1/1.jpeg
https://otile3-s.mqcdn.com/tiles/1.0.0/map/2/0/2.jpeg

- Dima

[GitHub] incubator-metron issue #316: METRON-503: Metron REST API

[merrimanr]

Github user merrimanr commented on the issue:

https://github.com/apache/incubator-metron/pull/316
  
Just pushed out a commit to address Response Behavior.  I defined an error 
response type using the structure you proposed and added a RestExceptionHandler 
as described in the link to the Spring Boot docs above.  I think it's a good 
solution.  One question though, do we need the "responseCode" field?  Won't it 
always match the Http status code?

Also added the metron-rest-client module.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #414: METRON-532 Define Profile Period When Calling P...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/414
  
@mattf-horton hah fair point, I was being overwrought, but we might 
consider it if we ever have a bundle of Stellar changes to make.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #414: METRON-532 Define Profile Period When Calling P...

[mattf-horton]

Github user mattf-horton commented on the issue:

https://github.com/apache/incubator-metron/pull/414
  
@cestella , Everlasting? Really? :-)  Considering that Java doesn't, and 
even most interpreted languages that do support them added them as a 2nd or 3rd 
generation feature, I'd say you're in very good company!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #414: METRON-532 Define Profile Period When Calling P...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/414
  
@mattf-horton  Thanks for pointing out the precedent.  It is to my 
everlasting shame that I did not think to do named arguments initially for 
Stellar.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #414: METRON-532 Define Profile Period When Calling P...

[mattf-horton]

Github user mattf-horton commented on the issue:

https://github.com/apache/incubator-metron/pull/414
  
A further comment on multiple optional parameters:  C++ allows multiple 
optional (defaulted) arguments, and does not support named arguments.  The rule 
"if a later optional arg is provided, all preceding optional args must also be 
provided, even if with trivial values", is exactly what C++ does. It is also 
consistent with what Python does when multiple defaulted arguments are provided 
positionally instead of as "named arguments" -- altho of course in Python one 
probably would use the named args instead.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #414: METRON-532 Define Profile Period When Calling P...

[mattf-horton]

Github user mattf-horton commented on the issue:

https://github.com/apache/incubator-metron/pull/414
  
@nickwallen and @cestella, thanks for the comments.  Regarding multiple 
optional args, I followed the rule that if the second optional arg is provided, 
then the first "optional" arg isn't optional; or more generally, if a later 
optional arg is provided, all preceding optional args must also be provided, 
even if with trivial values.  No tricky polymorphisms with object types, just a 
simple argument count. I think that's the best we can do without named 
arguments.

BTW, @cestella , my previous PR-345 did use object type checking to allow 
either or both to be truly optional, but it felt very hacky, was obviously not 
generalizable, and I was happy to change to this.

Hopefully I made this clear for users in both the comments and documents.  
Let me know if you feel more needs to be said.  Thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #414: METRON-532 Define Profile Period When Calling P...

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/414
  
Sounds good.  +1 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #414: METRON-532 Define Profile Period When Calling P...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/414
  
@nickwallen Yeah, I noticed that.  I THINK I've come around to it in this 
case.  They are differing types so *should* be able to be differentiated 
relatively easily.  Thanks for bringing attention to it, though.  It won't hurt 
to ensure nobody else is offended by it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request #414: METRON-532 Define Profile Period When Ca...

[nickwallen]

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/414#discussion_r95610064
  
--- Diff: 
metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/GetProfile.java
 ---
@@ -132,36 +157,26 @@
 
   private static final Logger LOG = 
LoggerFactory.getLogger(GetProfile.class);
 
-  /**
-   * A client that can retrieve profile values.
-   */
+  // Cached client that can retrieve profile values.
--- End diff --

Just curious, why change the comment style?  When should I use 
javadoc-style comments versus traditional `//`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request #414: METRON-532 Define Profile Period When Ca...

[nickwallen]

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/414#discussion_r95608748
  
--- Diff: 
metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/GetProfile.java
 ---
@@ -132,36 +157,26 @@
 
   private static final Logger LOG = 
LoggerFactory.getLogger(GetProfile.class);
 
-  /**
-   * A client that can retrieve profile values.
-   */
+  // Cached client that can retrieve profile values.
--- End diff --

Just curious, why switch the comment style here?  When should I use javadoc 
comments versus regular comments?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request #414: METRON-532 Define Profile Period When Ca...

[nickwallen]

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/414#discussion_r95609517
  
--- Diff: 
metron-analytics/metron-profiler-client/src/test/java/org/apache/metron/profiler/client/GetProfileTest.java
 ---
@@ -116,6 +119,33 @@ public void setup() {
 .build());
   }
 
+  private Context setup2() {
--- End diff --

What does `setup2` do differently than `setup`?  Could we comment or rename 
the function to be more descriptive?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #414: METRON-532 Define Profile Period When Calling P...

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/414
  
The function has two optional arguments, no?  I thought [per the discussion 
in 
METRON-590](https://github.com/apache/incubator-metron/pull/395#discussion_r93141706)
 that was not well liked?  Is this different somehow?




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[DISCUSS][MENTORS] Hosting Kraken maven artifacts in incubator-metron git repo

[Michael Miklavcic]

Restarting this thread w/new subject

This is a question primarily for the mentors.

*Background*
metron-common is currently depending on the openSOC github repo for hosting
kraken artifacts. The original reason for this was that these jars are not
hosted in Maven Central, and they were not reliably available in the Kraken
repo. https://issues.apache.org/jira/browse/METRON-650 is tracking work
around copying these artifacts to the Metron repo.

Kraken source on openSOC - https://github.com/OpenSOC/kraken
Krake maven repo on openSOC -
https://github.com/OpenSOC/kraken/tree/mvn-repo

*Ask*
Create a new branch in incubator-metron to host any necessary maven
artifacts. This branch would simply be incubator-metron/mvn-repo. This is
similar to how we've hosted the asf-site.

*Concerns/Questions*

   1. Can we host these jars/artifacts in this manner?
   2. Concerns regarding licensing?
   3. Do we need to also grab and host the source code?

Re: [DISCUSS] Hosting Kraken maven artifacts in incubator-metron git repo

[Michael Miklavcic]

Ok, that was funny

On Wed, Jan 11, 2017 at 7:43 AM, Casey Stella  wrote:

> I'd recommend restarting this discuss thread with the subject
> "[DISCUSS][MENTORS] Hosting Kraken maven artifacts in incubator-metron git
> repo".  That's the Mentors bat-symbol. The other option is to repeat
> "mentors" 3 times and either the mentors will appear or you will be eaten
> by a grue. ;)
>
> On Wed, Jan 11, 2017 at 9:32 AM, Michael Miklavcic <
> michael.miklav...@gmail.com> wrote:
>
> > Any comment from the mentors on this?
> >
> > On Mon, Jan 9, 2017 at 2:00 PM, Michael Miklavcic <
> > michael.miklav...@gmail.com> wrote:
> >
> > > This is a question primarily for the mentors.
> > >
> > > *Background*
> > > metron-common is currently depending on the openSOC github repo for
> > > hosting kraken artifacts. The original reason for this was that these
> > jars
> > > are not hosted in Maven Central, and they were not reliably available
> in
> > > the Kraken repo. https://issues.apache.org/jira/browse/METRON-650 is
> > > tracking work around copying these artifacts to the Metron repo.
> > >
> > > Kraken source on openSOC - https://github.com/OpenSOC/kraken
> > > Krake maven repo on openSOC - https://github.com/OpenSOC/
> > > kraken/tree/mvn-repo
> > >
> > > *Ask*
> > > Create a new branch in incubator-metron to host any necessary maven
> > > artifacts. This branch would simply be incubator-metron/mvn-repo. This
> is
> > > similar to how we've hosted the asf-site.
> > >
> > > *Concerns/Questions*
> > >
> > >1. Can we host these jars/artifacts in this manner?
> > >2. Concerns regarding licensing?
> > >3. Do we need to also grab and host the source code?
> > >
> > >
> >
>

[GitHub] incubator-metron issue #414: METRON-532 Define Profile Period When Calling P...

[mmiklavc]

Github user mmiklavc commented on the issue:

https://github.com/apache/incubator-metron/pull/414
  
Clean, well-documented, tested. Nice work @mattf-horton! +1 from me as well.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [DISCUSS] Hosting Kraken maven artifacts in incubator-metron git repo

[Casey Stella]

I'd recommend restarting this discuss thread with the subject
"[DISCUSS][MENTORS] Hosting Kraken maven artifacts in incubator-metron git
repo".  That's the Mentors bat-symbol. The other option is to repeat
"mentors" 3 times and either the mentors will appear or you will be eaten
by a grue. ;)

On Wed, Jan 11, 2017 at 9:32 AM, Michael Miklavcic <
michael.miklav...@gmail.com> wrote:

> Any comment from the mentors on this?
>
> On Mon, Jan 9, 2017 at 2:00 PM, Michael Miklavcic <
> michael.miklav...@gmail.com> wrote:
>
> > This is a question primarily for the mentors.
> >
> > *Background*
> > metron-common is currently depending on the openSOC github repo for
> > hosting kraken artifacts. The original reason for this was that these
> jars
> > are not hosted in Maven Central, and they were not reliably available in
> > the Kraken repo. https://issues.apache.org/jira/browse/METRON-650 is
> > tracking work around copying these artifacts to the Metron repo.
> >
> > Kraken source on openSOC - https://github.com/OpenSOC/kraken
> > Krake maven repo on openSOC - https://github.com/OpenSOC/
> > kraken/tree/mvn-repo
> >
> > *Ask*
> > Create a new branch in incubator-metron to host any necessary maven
> > artifacts. This branch would simply be incubator-metron/mvn-repo. This is
> > similar to how we've hosted the asf-site.
> >
> > *Concerns/Questions*
> >
> >1. Can we host these jars/artifacts in this manner?
> >2. Concerns regarding licensing?
> >3. Do we need to also grab and host the source code?
> >
> >
>

Re: [DISCUSS] Hosting Kraken maven artifacts in incubator-metron git repo

[Michael Miklavcic]

Any comment from the mentors on this?

On Mon, Jan 9, 2017 at 2:00 PM, Michael Miklavcic <
michael.miklav...@gmail.com> wrote:

> This is a question primarily for the mentors.
>
> *Background*
> metron-common is currently depending on the openSOC github repo for
> hosting kraken artifacts. The original reason for this was that these jars
> are not hosted in Maven Central, and they were not reliably available in
> the Kraken repo. https://issues.apache.org/jira/browse/METRON-650 is
> tracking work around copying these artifacts to the Metron repo.
>
> Kraken source on openSOC - https://github.com/OpenSOC/kraken
> Krake maven repo on openSOC - https://github.com/OpenSOC/
> kraken/tree/mvn-repo
>
> *Ask*
> Create a new branch in incubator-metron to host any necessary maven
> artifacts. This branch would simply be incubator-metron/mvn-repo. This is
> similar to how we've hosted the asf-site.
>
> *Concerns/Questions*
>
>1. Can we host these jars/artifacts in this manner?
>2. Concerns regarding licensing?
>3. Do we need to also grab and host the source code?
>
>

[GitHub] incubator-metron issue #404: METRON-624: Updated Comparison/Equality Evaluat...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/404
  
Yeah, this is good work.  I looked at it again after my vacation and I like 
it.  I give it a +1.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #414: METRON-532 Define Profile Period When Calling P...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/414
  
Ok, I like this a lot.  I didn't see anything that I really have an issue 
with at all, in fact.  Great job @mattf-horton !  You get my +1 pending your 
tests :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request #403: METRON-640: Add a Stellar function to co...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-metron/pull/403


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request #401: METRON-637: Add a STATS_BIN function to ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-metron/pull/401


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #399: METRON-600: Fix Metron Website

[iraghumitra]

Github user iraghumitra commented on the issue:

https://github.com/apache/incubator-metron/pull/399
  
@mattf-horton My bad missed the point about checksums.

@ottobackwards The original idea was "Community Resources" would help 
people to follow metron activities and understanding metron in depth, "Get 
Involved" would help users to setup Metron and get it running. The current 
links might not be doing justice to this thought. 
I looked at storm, atlas and ranger websites none of them seem to be having 
the mailing list on the home page. The mailing list  is either on  "Getting 
Help" or on "Project Information" pages. We can move the mailing list to home 
page if we think it is useful there.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---