Re: [DISCUSS] Metron Logo Options

[Houshang Livian]

Dave Hirko and James,

What did you want to communicate with the logo?

Houshang Livian
Senior User Experience Designer
Hortonworks

www.hortonworks.com


Mobile: (831) 521-4176
hliv...@hortonworks.com


On Apr 19, 2016, at 4:58 PM, Debo Dutta (dedutta) 
> wrote:

Agree w Charles around the generic notion of real time streams of info. Metron 
could be much more generic.

Sent from my iPhone

On Apr 19, 2016, at 4:50 PM, Charles Porter 
> wrote:

This is the logo we agreed on in january:
https://drive.google.com/folderview?id=0ByxWQZe48HUHeVRmaWVYOHF1X3c#

I personally like not using a shield. It is a little overused in this
context.
The logo from the January  vote suggests the mechanized watching of flows
of vital information, which is really the fundamental capability of Metron.
That is, its utility is really not limited to security.

My .02

-Charles

On Tue, Apr 19, 2016 at 7:42 PM, Houshang Livian 
>
wrote:

Thank you Sheetal,

You designed the shield last time. Do you have any comments on the new
ideas?




On 4/19/16, 2:04 PM, "Sheetal Dolas" 
> wrote:

Hoshang, these were the options proposed last time

http://i.imgur.com/QS0IITo.png

On Tue, Apr 19, 2016 at 9:11 AM, Rita McKissick <
rmckiss...@hortonworks.com>
wrote:

I agree that #1 looks very similar to Metasploit's logo. Although the
castle wall is a good way to imply defense. I would prefer something a
bit
more modern. My favorite remains #3. Maybe it would be helpful if the
object above the M looks a bit more like a shield.

Thanks,

Rita

Rita McKissick ! Sr. Technical Writer









On 4/18/16, 12:13 PM, "Houshang Livian" 
>
wrote:

Apologies for the flood of emails. Here is a dropbox link for those
that
could not see the options:
https://www.dropbox.com/s/6hxz76aet8gzt0a/metron-logo-options.png?dl=0


From: default 
mailto:dev@metron.incubator.apache.org>>" 

mailto:dev@metron.incubator.apache.org>>>
Date: Monday, April 18, 2016 at 12:07 PM
To: 
"dev@metron.incubator.apache.orgmailto:dev@metron.incubator.apache.org>>" 

mailto:dev@metron.incubator.apache.org>>>
Subject: Re: [DISCUSS] Metron Logo Options

Trying again with the options as an attachment.

From: default 
mailto:dev@metron.incubator.apache.org>>" 

mailto:dev@metron.incubator.apache.org>>>
Date: Monday, April 18, 2016 at 12:05 PM
To: 
"dev@metron.incubator.apache.orgmailto:dev@metron.incubator.apache.org>>" 

mailto:dev@metron.incubator.apache.org>>>
Subject: [DISCUSS] Metron Logo Options

Hey Guys,

As you know, Metron is based on a DC Comic Book character.

Here are 5 different logo options for Metron. Let me know your thoughts
and your votes.
[cid:FC8299C6-AAAB-4CC2-9129-3C19479F1D8F]

Houshang Livian
Senior User Experience Designer
Hortonworks

www.hortonworks.com


Mobile: (831) 521-4176
hliv...@hortonworks.com

Re: [DISCUSS] Metron Logo Options

[Debo Dutta (dedutta)]

Agree w Charles around the generic notion of real time streams of info. Metron 
could be much more generic. 

Sent from my iPhone

> On Apr 19, 2016, at 4:50 PM, Charles Porter  
> wrote:
> 
> This is the logo we agreed on in january:
> https://drive.google.com/folderview?id=0ByxWQZe48HUHeVRmaWVYOHF1X3c#
> 
> I personally like not using a shield. It is a little overused in this
> context.
> The logo from the January  vote suggests the mechanized watching of flows
> of vital information, which is really the fundamental capability of Metron.
> That is, its utility is really not limited to security.
> 
> My .02
> 
> -Charles
> 
> On Tue, Apr 19, 2016 at 7:42 PM, Houshang Livian 
> wrote:
> 
>> Thank you Sheetal,
>> 
>> You designed the shield last time. Do you have any comments on the new
>> ideas?
>> 
>> 
>> 
>> 
>>> On 4/19/16, 2:04 PM, "Sheetal Dolas"  wrote:
>>> 
>>> Hoshang, these were the options proposed last time
>>> 
>>> http://i.imgur.com/QS0IITo.png
>>> 
>>> On Tue, Apr 19, 2016 at 9:11 AM, Rita McKissick <
>> rmckiss...@hortonworks.com>
>>> wrote:
>>> 
 I agree that #1 looks very similar to Metasploit’s logo. Although the
 castle wall is a good way to imply defense. I would prefer something a
>> bit
 more modern. My favorite remains #3. Maybe it would be helpful if the
 object above the M looks a bit more like a shield.
 
 Thanks,
 
 Rita
 
 Rita McKissick ! Sr. Technical Writer
 
 
 
 
 
 
 
 
 
 On 4/18/16, 12:13 PM, "Houshang Livian" 
>> wrote:
 
> Apologies for the flood of emails. Here is a dropbox link for those
>> that
 could not see the options:
> https://www.dropbox.com/s/6hxz76aet8gzt0a/metron-logo-options.png?dl=0
> 
> 
> From: default  Reply-To: "dev@metron.incubator.apache.org>>> dev@metron.incubator.apache.org>" > >>> dev@metron.incubator.apache.org>>
> Date: Monday, April 18, 2016 at 12:07 PM
> To: "dev@metron.incubator.apache.org>>> dev@metron.incubator.apache.org>" > >>> dev@metron.incubator.apache.org>>
> Subject: Re: [DISCUSS] Metron Logo Options
> 
> Trying again with the options as an attachment.
> 
> From: default  Reply-To: "dev@metron.incubator.apache.org>>> dev@metron.incubator.apache.org>" > >>> dev@metron.incubator.apache.org>>
> Date: Monday, April 18, 2016 at 12:05 PM
> To: "dev@metron.incubator.apache.org>>> dev@metron.incubator.apache.org>" > >>> dev@metron.incubator.apache.org>>
> Subject: [DISCUSS] Metron Logo Options
> 
> Hey Guys,
> 
> As you know, Metron is based on a DC Comic Book character.
> 
> Here are 5 different logo options for Metron. Let me know your thoughts
 and your votes.
> [cid:FC8299C6-AAAB-4CC2-9129-3C19479F1D8F]
> 
> Houshang Livian
> Senior User Experience Designer
> Hortonworks
> 
> www.hortonworks.com
> 
> 
> Mobile: (831) 521-4176
> hliv...@hortonworks.com
> 
>> 

Re: [DISCUSS] Metron Logo Options

[Charles Porter]

This is the logo we agreed on in january:
https://drive.google.com/folderview?id=0ByxWQZe48HUHeVRmaWVYOHF1X3c#

I personally like not using a shield. It is a little overused in this
context.
The logo from the January  vote suggests the mechanized watching of flows
of vital information, which is really the fundamental capability of Metron.
That is, its utility is really not limited to security.

My .02

-Charles

On Tue, Apr 19, 2016 at 7:42 PM, Houshang Livian 
wrote:

> Thank you Sheetal,
>
> You designed the shield last time. Do you have any comments on the new
> ideas?
>
>
>
>
> On 4/19/16, 2:04 PM, "Sheetal Dolas"  wrote:
>
> >Hoshang, these were the options proposed last time
> >
> >http://i.imgur.com/QS0IITo.png
> >
> >On Tue, Apr 19, 2016 at 9:11 AM, Rita McKissick <
> rmckiss...@hortonworks.com>
> >wrote:
> >
> >> I agree that #1 looks very similar to Metasploit’s logo. Although the
> >> castle wall is a good way to imply defense. I would prefer something a
> bit
> >> more modern. My favorite remains #3. Maybe it would be helpful if the
> >> object above the M looks a bit more like a shield.
> >>
> >> Thanks,
> >>
> >> Rita
> >>
> >> Rita McKissick ! Sr. Technical Writer
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> On 4/18/16, 12:13 PM, "Houshang Livian" 
> wrote:
> >>
> >> >Apologies for the flood of emails. Here is a dropbox link for those
> that
> >> could not see the options:
> >> >https://www.dropbox.com/s/6hxz76aet8gzt0a/metron-logo-options.png?dl=0
> >> >
> >> >
> >> >From: default  >>
> >> >Reply-To: "dev@metron.incubator.apache.org >> dev@metron.incubator.apache.org>"   >> dev@metron.incubator.apache.org>>
> >> >Date: Monday, April 18, 2016 at 12:07 PM
> >> >To: "dev@metron.incubator.apache.org >> dev@metron.incubator.apache.org>"   >> dev@metron.incubator.apache.org>>
> >> >Subject: Re: [DISCUSS] Metron Logo Options
> >> >
> >> >Trying again with the options as an attachment.
> >> >
> >> >From: default  >>
> >> >Reply-To: "dev@metron.incubator.apache.org >> dev@metron.incubator.apache.org>"   >> dev@metron.incubator.apache.org>>
> >> >Date: Monday, April 18, 2016 at 12:05 PM
> >> >To: "dev@metron.incubator.apache.org >> dev@metron.incubator.apache.org>"   >> dev@metron.incubator.apache.org>>
> >> >Subject: [DISCUSS] Metron Logo Options
> >> >
> >> >Hey Guys,
> >> >
> >> >As you know, Metron is based on a DC Comic Book character.
> >> >
> >> >Here are 5 different logo options for Metron. Let me know your thoughts
> >> and your votes.
> >> > [cid:FC8299C6-AAAB-4CC2-9129-3C19479F1D8F]
> >> >
> >> >Houshang Livian
> >> >Senior User Experience Designer
> >> >Hortonworks
> >> >
> >> >www.hortonworks.com
> >> >
> >> >
> >> >Mobile: (831) 521-4176
> >> >hliv...@hortonworks.com
> >> >
> >> >
> >>
>

Re: [DISCUSS] Tagging Jira's for newbies

[Sheetal Dolas]

+1 for the idea of tagging.

I would however prefer the tag names that focus on Jira than a person. As
simple as traditional : Simple, Medium,  Complex (or Basic, Intermediate,
Advanced)

On Sat, Apr 16, 2016 at 1:48 PM, George Vetticaden <
gvettica...@hortonworks.com> wrote:

> + 1.
> We should get this documented on the wiki.
>
> > On Apr 16, 2016, at 2:08 PM, James Sirota 
> wrote:
> >
> > Very cool.  I like the idea.  So we’ll standardize on tagging Jiras with
> newbie and newbie++
> >
> > Would anyone have a problem with us doing this?  Or can propose an
> alternate label scheme?
> >
> > Thanks,
> > James
> >
> >
> >
> >
> > On 4/15/16, 1:16 PM, "David Lyle"  wrote:
> >
> >> I've been using:
> >>
> >> newbie for low complexity introductory Jiras:
> >>
> https://issues.apache.org/jira/browse/METRON-87?jql=project%20%3D%20METRON%20AND%20labels%20%3D%20newbie
> >> newbie++ for medium complexity introductory Jiras:
> >>
> https://issues.apache.org/jira/browse/METRON-91?jql=project%20%3D%20METRON%20AND%20labels%20%3D%20%22newbie%2B%2B%22
> >>
> >> In my experience, that's pretty standard. Anything more complex than
> >> newbie++ is just untagged wrt complexity.
> >>
> >> -D...
> >>
> >>
> >>
> >> On Fri, Apr 15, 2016 at 3:19 PM, James Sirota 
> >> wrote:
> >>
> >>> Hi Guys,
> >>>
> >>> As we are picking up more community members I think we need to start
> >>> tagging Jira’s with “complexity labels” and building up a pool of Jiras
> >>> that newbies can work on.  I wanted to open this up to the community
> to see
> >>> how we wanted to handle that.  What should these labels be and how many
> >>> label types would we want [newbie/internmediate/advanced]?  How do
> other
> >>> Apache projects make it easy to onboard new community members?
> >>>
> >>> Thanks,
> >>> James
> >>>
>
>

[GitHub] incubator-metron pull request: Dataload testing

[franklu1]

GitHub user franklu1 opened a pull request:

https://github.com/apache/incubator-metron/pull/83

Dataload testing

Add Dataloads unit testing.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/franklu1/incubator-metron dataload-testing

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-metron/pull/83.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #83


commit 0c1ddc181c4c17f5616ee69a8a1f7163ee92d356
Author: rmerriman 
Date:   2016-04-07T19:45:38Z

Unit tests for GrokParser

commit 4ebc97a226c1f4c8fa9f03b102fb92f49a36af11
Author: rmerriman 
Date:   2016-04-12T15:12:48Z

Created separate Metron-TestingUtilities project and moved MockHBaseTable 
class

commit 002ed83a70af5e9f924892918080847285acbe09
Author: Frank Lu 
Date:   2016-04-13T13:11:54Z

Unit test for enrichment adapters

commit 3e951cda08f8cfae6ad94f86fdb7c42d6e27e185
Author: Frank Lu 
Date:   2016-04-18T16:11:30Z

Merge remote-tracking branch 'upstream/master'

commit 119c70c8cca12e3723c8b3ce9e9c9af3b915b6a0
Author: Frank Lu 
Date:   2016-04-19T01:13:39Z

Unit testing for DataLoads




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request: METRON-97: Create data purging scri...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-metron/pull/82


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request: METRON-96: Create data purging scri...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-metron/pull/79


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [DISCUSS] Metron Logo Options

[Rita McKissick]

I agree that #1 looks very similar to Metasploit’s logo. Although the castle 
wall is a good way to imply defense. I would prefer something a bit more 
modern. My favorite remains #3. Maybe it would be helpful if the object above 
the M looks a bit more like a shield.

Thanks,

Rita

Rita McKissick ! Sr. Technical Writer









On 4/18/16, 12:13 PM, "Houshang Livian"  wrote:

>Apologies for the flood of emails. Here is a dropbox link for those that could 
>not see the options:
>https://www.dropbox.com/s/6hxz76aet8gzt0a/metron-logo-options.png?dl=0
>
>
>From: default >
>Reply-To: 
>"dev@metron.incubator.apache.org" 
>>
>Date: Monday, April 18, 2016 at 12:07 PM
>To: "dev@metron.incubator.apache.org" 
>>
>Subject: Re: [DISCUSS] Metron Logo Options
>
>Trying again with the options as an attachment.
>
>From: default >
>Reply-To: 
>"dev@metron.incubator.apache.org" 
>>
>Date: Monday, April 18, 2016 at 12:05 PM
>To: "dev@metron.incubator.apache.org" 
>>
>Subject: [DISCUSS] Metron Logo Options
>
>Hey Guys,
>
>As you know, Metron is based on a DC Comic Book character.
>
>Here are 5 different logo options for Metron. Let me know your thoughts and 
>your votes.
> [cid:FC8299C6-AAAB-4CC2-9129-3C19479F1D8F]
>
>Houshang Livian
>Senior User Experience Designer
>Hortonworks
>
>www.hortonworks.com
>
>
>Mobile: (831) 521-4176
>hliv...@hortonworks.com
>
>

[GitHub] incubator-metron pull request: METRON-103: Unit Tests for storm bo...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-metron/pull/81


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request: METRON-96: Create data purging scri...

[dlyle65535]

Github user dlyle65535 commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/79#discussion_r60252183
  
--- Diff: metron-streaming/Metron-DataLoads/pom.xml ---
@@ -167,56 +225,14 @@
 
 
 
-
 org.apache.maven.plugins
 maven-surefire-plugin
-2.12.4
 
-
--Xmx2048m -XX:MaxPermSize=256m 
-Djava.util.logging.config.file=logging.properties
+-Xmx2048m -XX:MaxPermSize=256m 
-XX:-UseSplitVerifier
 true
-
 false
-
 
-
--- End diff --

:) 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request: METRON-96: Create data purging scri...

[dlyle65535]

Github user dlyle65535 commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/79#discussion_r60252124
  
--- Diff: metron-streaming/Metron-DataLoads/pom.xml ---
@@ -167,56 +225,14 @@
 
 
 
-
 org.apache.maven.plugins
 maven-surefire-plugin
-2.12.4
 
-
--Xmx2048m -XX:MaxPermSize=256m 
-Djava.util.logging.config.file=logging.properties
+-Xmx2048m -XX:MaxPermSize=256m 
-XX:-UseSplitVerifier
 true
-
 false
-
 
-
--- End diff --

I don't mean it to. The effective pom gets the executions info from the top 
level pom.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request: METRON-97: Create data purging scri...

[merrimanr]

Github user merrimanr commented on the pull request:

https://github.com/apache/incubator-metron/pull/82#issuecomment-211976602
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request: METRON-103: Unit Tests for storm bo...

[cestella]

Github user cestella commented on the pull request:

https://github.com/apache/incubator-metron/pull/81#issuecomment-211975900
  
+1, looks good


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request: METRON-103: Unit Tests for storm bo...

[cestella]

Github user cestella commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/81#discussion_r60251529
  
--- Diff: 
metron-streaming/Metron-EnrichmentAdapters/src/test/java/org/apache/metron/enrichment/bolt/EnrichmentJoinBoltTest.java
 ---
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.enrichment.bolt;
+
+import org.adrianwalker.multilinestring.Multiline;
+import org.apache.metron.bolt.BaseEnrichmentBoltTest;
+import org.json.simple.JSONObject;
+import org.json.simple.parser.JSONParser;
+import org.json.simple.parser.ParseException;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+
+public class EnrichmentJoinBoltTest extends BaseEnrichmentBoltTest {
+
+  /**
+   * {
--- End diff --

Nevermind, they don't.  retracted


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request: METRON-103: Unit Tests for storm bo...

[cestella]

Github user cestella commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/81#discussion_r60251394
  
--- Diff: 
metron-streaming/Metron-EnrichmentAdapters/src/test/java/org/apache/metron/enrichment/bolt/EnrichmentJoinBoltTest.java
 ---
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.enrichment.bolt;
+
+import org.adrianwalker.multilinestring.Multiline;
+import org.apache.metron.bolt.BaseEnrichmentBoltTest;
+import org.json.simple.JSONObject;
+import org.json.simple.parser.JSONParser;
+import org.json.simple.parser.ParseException;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+
+public class EnrichmentJoinBoltTest extends BaseEnrichmentBoltTest {
+
+  /**
+   * {
--- End diff --

I think those *'s are going to show up in your string, is that what you 
intended?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request: METRON-96: Create data purging scri...

[cestella]

Github user cestella commented on the pull request:

https://github.com/apache/incubator-metron/pull/79#issuecomment-211974974
  
+1, looks great.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request: METRON-103: Unit Tests for storm bo...

[cestella]

Github user cestella commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/81#discussion_r60249308
  
--- Diff: 
metron-streaming/Metron-Common/src/test/java/org/apache/metron/bolt/BulkMessageWriterTest.java
 ---
@@ -0,0 +1,147 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.bolt;
+
+import backtype.storm.tuple.Tuple;
+import backtype.storm.tuple.Values;
+import org.adrianwalker.multilinestring.Multiline;
+import org.apache.metron.Constants;
+import org.apache.metron.domain.Configurations;
+import org.apache.metron.writer.interfaces.BulkMessageWriter;
+import org.hamcrest.Description;
+import org.json.simple.JSONObject;
+import org.json.simple.parser.JSONParser;
+import org.json.simple.parser.ParseException;
+import org.junit.Before;
+import org.junit.Test;
+import org.mockito.ArgumentMatcher;
+import org.mockito.Matchers;
+import org.mockito.Mock;
+
+import java.io.FileInputStream;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import static org.junit.Assert.fail;
+import static org.mockito.Matchers.any;
+import static org.mockito.Matchers.argThat;
+import static org.mockito.Matchers.eq;
+import static org.mockito.Mockito.doThrow;
+import static org.mockito.Mockito.reset;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+public class BulkMessageWriterTest extends BaseEnrichmentBoltTest {
+
+  protected class MessageListMatcher extends 
ArgumentMatcher {
+
+private List expectedMessageList;
+
+public MessageListMatcher(List expectedMessageList) {
+  this.expectedMessageList = expectedMessageList;
+}
+
+@Override
+public boolean matches(Object o) {
+  List actualMessageList = (List) o;
+  for(JSONObject message: actualMessageList) 
removeTimingFields(message);
+  return expectedMessageList.equals(actualMessageList);
+}
+
+@Override
+public void describeTo(Description description) {
+  description.appendText(String.format("[%s]", expectedMessageList));
+}
+
+  }
+
+  /**
+   * {
+   * "field": "value",
+   * "source.type": "yaf"
+   * }
+   */
+  @Multiline
+  private String sampleMessageString;
+
+  private JSONObject sampleMessage;
+  private List messageList;
+  private List tupleList;
+
+  @Before
+  public void parseMessages() throws ParseException {
+JSONParser parser = new JSONParser();
+sampleMessage = (JSONObject) parser.parse(sampleMessageString);
+sampleMessage.put("field", "value1");
+messageList = new ArrayList<>();
+messageList.add(((JSONObject) sampleMessage.clone()));
+sampleMessage.put("field", "value2");
+messageList.add(((JSONObject) sampleMessage.clone()));
+sampleMessage.put("field", "value3");
+messageList.add(((JSONObject) sampleMessage.clone()));
+sampleMessage.put("field", "value4");
+messageList.add(((JSONObject) sampleMessage.clone()));
+sampleMessage.put("field", "value5");
+messageList.add(((JSONObject) sampleMessage.clone()));
+  }
+
+  @Mock
+  private BulkMessageWriter bulkMessageWriter;
+
+  @Test
+  public void test() throws Exception {
+BulkMessageWriterBolt bulkMessageWriterBolt = new 
BulkMessageWriterBolt("zookeeperUrl").withBulkMessageWriter(bulkMessageWriter);
+bulkMessageWriterBolt.setCuratorFramework(client);
+bulkMessageWriterBolt.setTreeCache(cache);
+
bulkMessageWriterBolt.configurations.updateSensorEnrichmentConfig(sensorType, 
new FileInputStream(sampleSensorEnrichmentConfigPath));
+bulkMessageWriterBolt.declareOutputFields(declarer);
+verify(declarer, times(1)).declareStream(eq("error"), 

[GitHub] incubator-metron pull request: METRON-103: Unit Tests for storm bo...

[merrimanr]

Github user merrimanr commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/81#discussion_r60250609
  
--- Diff: 
metron-streaming/Metron-Common/src/test/java/org/apache/metron/bolt/BulkMessageWriterTest.java
 ---
@@ -0,0 +1,147 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.bolt;
+
+import backtype.storm.tuple.Tuple;
+import backtype.storm.tuple.Values;
+import org.adrianwalker.multilinestring.Multiline;
+import org.apache.metron.Constants;
+import org.apache.metron.domain.Configurations;
+import org.apache.metron.writer.interfaces.BulkMessageWriter;
+import org.hamcrest.Description;
+import org.json.simple.JSONObject;
+import org.json.simple.parser.JSONParser;
+import org.json.simple.parser.ParseException;
+import org.junit.Before;
+import org.junit.Test;
+import org.mockito.ArgumentMatcher;
+import org.mockito.Matchers;
+import org.mockito.Mock;
+
+import java.io.FileInputStream;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import static org.junit.Assert.fail;
+import static org.mockito.Matchers.any;
+import static org.mockito.Matchers.argThat;
+import static org.mockito.Matchers.eq;
+import static org.mockito.Mockito.doThrow;
+import static org.mockito.Mockito.reset;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+public class BulkMessageWriterTest extends BaseEnrichmentBoltTest {
+
+  protected class MessageListMatcher extends 
ArgumentMatcher {
+
+private List expectedMessageList;
+
+public MessageListMatcher(List expectedMessageList) {
+  this.expectedMessageList = expectedMessageList;
+}
+
+@Override
+public boolean matches(Object o) {
+  List actualMessageList = (List) o;
+  for(JSONObject message: actualMessageList) 
removeTimingFields(message);
+  return expectedMessageList.equals(actualMessageList);
+}
+
+@Override
+public void describeTo(Description description) {
+  description.appendText(String.format("[%s]", expectedMessageList));
+}
+
+  }
+
+  /**
+   * {
+   * "field": "value",
+   * "source.type": "yaf"
+   * }
+   */
+  @Multiline
+  private String sampleMessageString;
+
+  private JSONObject sampleMessage;
+  private List messageList;
+  private List tupleList;
+
+  @Before
+  public void parseMessages() throws ParseException {
+JSONParser parser = new JSONParser();
+sampleMessage = (JSONObject) parser.parse(sampleMessageString);
+sampleMessage.put("field", "value1");
+messageList = new ArrayList<>();
+messageList.add(((JSONObject) sampleMessage.clone()));
+sampleMessage.put("field", "value2");
+messageList.add(((JSONObject) sampleMessage.clone()));
+sampleMessage.put("field", "value3");
+messageList.add(((JSONObject) sampleMessage.clone()));
+sampleMessage.put("field", "value4");
+messageList.add(((JSONObject) sampleMessage.clone()));
+sampleMessage.put("field", "value5");
+messageList.add(((JSONObject) sampleMessage.clone()));
+  }
+
+  @Mock
+  private BulkMessageWriter bulkMessageWriter;
+
+  @Test
+  public void test() throws Exception {
+BulkMessageWriterBolt bulkMessageWriterBolt = new 
BulkMessageWriterBolt("zookeeperUrl").withBulkMessageWriter(bulkMessageWriter);
+bulkMessageWriterBolt.setCuratorFramework(client);
+bulkMessageWriterBolt.setTreeCache(cache);
+
bulkMessageWriterBolt.configurations.updateSensorEnrichmentConfig(sensorType, 
new FileInputStream(sampleSensorEnrichmentConfigPath));
+bulkMessageWriterBolt.declareOutputFields(declarer);
+verify(declarer, times(1)).declareStream(eq("error"), 

[GitHub] incubator-metron pull request: METRON-96: Create data purging scri...

[cestella]

Github user cestella commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/79#discussion_r60250470
  
--- Diff: metron-streaming/Metron-DataLoads/pom.xml ---
@@ -167,56 +225,14 @@
 
 
 
-
 org.apache.maven.plugins
 maven-surefire-plugin
-2.12.4
 
-
--Xmx2048m -XX:MaxPermSize=256m 
-Djava.util.logging.config.file=logging.properties
+-Xmx2048m -XX:MaxPermSize=256m 
-XX:-UseSplitVerifier
 true
-
 false
-
 
-
--- End diff --

So, I don't mind this, but this change will make the "integration tests" 
run as part of the unit test cycle and people have gotten touchy in the past 
due to the length that it adds to mvn package.  I'm generally for it as it 
ensures we get the full suite, but I wanted to make sure it was intended and 
we're ok with it in general.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request: METRON-96: Create data purging scri...

[merrimanr]

Github user merrimanr commented on the pull request:

https://github.com/apache/incubator-metron/pull/79#issuecomment-211972084
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request: METRON-97: Create data purging scri...

[cestella]

Github user cestella commented on the pull request:

https://github.com/apache/incubator-metron/pull/82#issuecomment-211970028
  
+1, looks good


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request: METRON-103: Unit Tests for storm bo...

[dlyle65535]

Github user dlyle65535 commented on the pull request:

https://github.com/apache/incubator-metron/pull/81#issuecomment-211967271
  
+1 Great stuff, ship it! :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---