Re: [VOTE] Move Apache Metron to the Apache Attic and Dissolve PMC

+1 On Mon, Nov 16, 2020 at 11:41 AM zeo...@gmail.com wrote: > +1 > > -- > Jon Zeolla > @jonzeolla > > PittSec | BSidesPGH | SteelCityInfoSec > > On Mon, Nov 16, 2020, 11:33 AM Casey Stella wrote: > > > +1 > > > > On Mon, Nov 16, 2020 at 09:01 Justin Leet wrote: > > > > > Hi all, > > > > > > Th

Re: Development Activity has dropped to effectively 0, what should we do?

t; docker image that is designed to connect with other dockerized applications > such as Storm, Kafka, etc..? > > --Tom. > > On 2020-04-17, 11:27 AM, "Nick Allen" wrote: > > This is a good discussion and one that I haven't fully grappled with > in my >

Re: Development Activity has dropped to effectively 0, what should we do?

This is a good discussion and one that I haven't fully grappled with in my own mind yet. I'll have more to add, but I just want to chime in on the topic of Ambari at this point. ### Ambari and the Paywall The problem with Ambari is that its installation mechanism requires a repository of compiled

Re: Centos6 and Centos7 instructions

Hi Tom - The source for https://metron.apache.org/current-book/metron-deployment/development/centos6/index.html is contained in the README at `metron/metron-deployment/development/centos6/README.md`. When a release happens we have a script that generates the site book from the various documentati

Re: Possible approach to solve GeoIP update?

That seems like a viable solution to me. On Thu, Apr 9, 2020 at 7:58 PM Yerex, Tom wrote: > Good afternoon, > > Reviewing hxxps:// > issues.apache.org/jira/projects/METRON/issues/METRON-2340 I'm attempting > to sketch out a rough solution and I would like guidance from more > experienced minds

Re: [DISCUSS] Next Release - Life After 0.7.1

really solid > >> release. > >> > > > >> > > https://github.com/apache/metron/pull/1568 should be in before > >> release. > >> > It > >> > > addresses an issue with our validation of dependencies_with_url.csv > >>

Re: JUnit 5 PR merged into master

Thanks for the hard work on that upgrade and this very useful highlight reel. On Sat, Dec 7, 2019, 10:17 AM Justin Leet wrote: > Hi all, > > The JUnit 5 migration PR has been merged to master. From this point > forward, please use the newer interfaces and methods. There are plenty of > examples

[DISCUSS] Next Release - Life After 0.7.1

Hello Metron'ers - I would like to make the case that it is time for us to cut the next Apache Metron release. - Our last release was 0.7.1 on May 15th . It has b

Re: [DISCUSS] deprecate misleading install methods and docs?

FYI - See the following which removes the automated AWS deployment mechanism. https://issues.apache.org/jira/browse/METRON-2321 https://github.com/apache/metron/pull/1565 On Tue, Oct 29, 2019 at 10:23 AM Nick Allen wrote: > +1 On the remove option. I think we should *completely remove*

Branch Cleanup

Heads up... I accidentally pushed a feature branch to Apache called METRON-2223. This was my mistake. I have just deleted the branch. My apologies

Re: [DISCUSS] deprecate misleading install methods and docs?

+1 On the remove option. I think we should *completely remove* the automated AWS deployment mechanism because it has been too difficult to maintain, deploys an unsecure cluster by default, and is not the preferred installation path for AWS. If a user wants to deploy to AWS, they should launch th

Re: [DISCUSS] Curator client upgrade

+1 to making the change on the feature branch. We don't really know how this might affect master which is still building against HDP 2.6, nor is it strictly needed there. Going to Curator 4.0.0 is only needed due to the HDP 3.1 upgrade. This is also likely to get more focused testing cycles in

Re: [DISCUSS] Deprecate Least Recently Used Pruner

PM Otto Fowler wrote: > Can you summarize what it does? Is it from OpenSOC? > > > > > On August 13, 2019 at 17:53:52, Nick Allen (n...@nickallen.org) wrote: > > As part of https://github.com/apache/metron/pull/1470, I found it > difficult > to update the "Least

[DISCUSS] Deprecate Least Recently Used Pruner

As part of https://github.com/apache/metron/pull/1470, I found it difficult to update the "Least Recently Used Pruner" to work with HBase 2.0.2. I am sure that given more time and effort, I could make it work, but is it worth it? This is a feature that I myself am not familiar with. I do not know

Re: [DISCUSS] Alerts UI: Loading state while fetching data

Yes! I think this is sorely needed. Would this also include indicating when an error has occurred in the backend call? That might also be helpful and somewhat related to METRON-2190. On Wed, Jul 24, 2019 at 9:27 AM Tibor Meller wrote: > Hi all, > > I think it would great to have a loading sta

Re: Test failure in master

Thanks for fixing that. On Fri, Jun 28, 2019 at 6:40 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Just an fyi, if you run into the FileFilterUtilTest failing, pull latest > from master. There was a time-sensitive unit test condition that has now > been fixed in https://issues.apac

Re: Travis failing due to same Maven issue

I am not sure if this will actually solve the issue, but here is another attempt. https://github.com/apache/metron/pull/1442 On Wed, Jun 12, 2019 at 9:31 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > I've seen this now with 2 back-to-back commits to master that end with > Travis

Re: Storm 1.0.x eol

As part of the upgrade to HDP 3.1, we should consider ending all Metron support for Storm 1.0.x. We have a few build options and workarounds in the code base (see storm-kafka-override, Maven profile HDP-1.5.0.0, etc) that would need cleaned up. On Mon, Jun 3, 2019 at 5:39 PM Otto Fowler wrote:

Re: [DISCUSS] Metron RPM spec file changelog

I am also in favor of not maintaining a separate change log; that is what git is for. We just need to ensure that the rpmlint check can be satisfied in some way if we do not maintain the change log. On Fri, May 31, 2019 at 1:56 PM Ryan Merriman wrote: > I vote we get rid of them. It's easy eno

Re: Master build - failed due to Maven download fail

https://github.com/apache/metron/pull/1433 On Fri, May 24, 2019 at 12:07 PM Nick Allen wrote: > FYI, I just created this to track the issue. > > https://issues.apache.org/jira/browse/METRON-2143 > > On Mon, May 20, 2019 at 6:39 PM Justin Leet wrote: > >> I saw

Re: Master build - failed due to Maven download fail

FYI, I just created this to track the issue. https://issues.apache.org/jira/browse/METRON-2143 On Mon, May 20, 2019 at 6:39 PM Justin Leet wrote: > I saw the same thing on https://github.com/apache/metron/pull/1407, but > bouncing Travis fixed it. Not sure if it's a connection issue or what, bu

Re: [DISCUSS] Travis CI Build Time Limits

Docker changes, I think we should be extra vigilant on getting >those build/test times *down*, rather than enabling them to increase. > Sure, >there are other options for solving that problem, but Travis is a simple >equalizer because it's impartial to whatever local hardwa

Re: [DISCUSS] Build RPM/DEBs in Travis?

be > building them, because nobody is going to run both unless they specifically > done something they'd expect to affect both. > > On Wed, May 22, 2019 at 9:30 AM Nick Allen wrote: > > > In light of issues like this https://github.com/apache/metron/pull/1419, > > has anyo

Re: [DISCUSS] Travis CI Build Time Limits

FYI - Here is a POC build of this concept running. This only runs the Parser and Enrichment integration tests, but as separate jobs in Travis. https://travis-ci.org/nickwallen/metron/builds/535791047 On Wed, May 22, 2019 at 9:20 AM Nick Allen wrote: > > > Justin Leet said

[DISCUSS] Build RPM/DEBs in Travis?

In light of issues like this https://github.com/apache/metron/pull/1419, has anyone looked into building our RPMs and DEBs in Travis? This is a very common and easy mistake to make and our CI builds really should be able to catch this.

[DISCUSS] Travis CI Build Time Limits

> Justin Leet said [1]: Looking at our build times, I'm actually concerned that if we kill the caches our builds won't complete. The integration tests take >45 minutes and it's very possible redownloading everything goes over our r

Re: [DISCUSS] JsonMapParser original string functionality

On Fri, May 10, 2019 at 5:53 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > I think that's an excellent idea. Can anyone think of a situation where we > wouldn't want to add this the same way for all parsers? I suppose we could > always allow this to be overr

Re: [DISCUSS] JsonMapParser original string functionality

I think maintaining the integrity of the original data makes a lot of sense for any parser. And ideally the original string should be what came out of Kafka with only the minimally necessary processing. With that in mind, we could solve this one level up. Instead of relying on each parser to do t

Re: [VOTE] Metron Release Candidate 0.7.1-RC2

I really enjoyed the retro, 3-digit vibe on that one. On Fri, May 10, 2019 at 4:38 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > "METRON-685" - wow, that one was a long time coming. > > On Thu, May 9, 2019 at 5:54 PM Nick Allen wrote: > > >

Re: [VOTE] Metron Release Candidate 0.7.1-RC2

+1 binding I validated the release tarball, ran the full test suite and validated the CentOS 6 development environment. Everything looks solid. Let's ship it. On Wed, May 8, 2019 at 6:50 PM Justin Leet wrote: > This is a call to vote on releasing Apache Metron 0.7.1 > > Full list of changes i

Re: [DISCUSS] Parser Aggregation in Management UI

Have you considered creating a feature branch for the effort? This would allow you to break the effort into chunks, where the result of each PR may not be a fully working "master-ready" result. I am sure you guys tackled the work in chunks when developing it, so consider just replaying those chunk

Re: [DISCUSS] Full-dev role in PR testign

I'm exploring the use of TestContainers right now as part of the HDP 3.1 effort. Still exploring feasibility, but it is looking promising. On Fri, May 3, 2019 at 10:46 AM Justin Leet wrote: > I think everything Casey mentioned is a good call-out as things start to > build into specifics. I defi

Re: [DISCUSS] Metron Release - 0.7.1 next steps

I think any open source project needs to strive to cut releases regularly. This is healthy for the project and community. It gets new features and functionality out to the community so we can get feedback, find what is working and what is not, iterate and improve. You probably agree with this. W

Re: [DISCUSS] Metron Release - 0.7.1 next steps

To echo Justin's comments, I am in favor of #2, which provides a clear, well-defined path to a release. - Why hold back a release, especially a point release containing 89 improvements, for one issue that will not affect most users? - It is one thing to stall a release to address a bug

Re: [VOTE] Metron Release Candidate 0.7.1-RC1

:29, Anand Subramanian ( > > > asubraman...@hortonworks.com) wrote: > > > > > > +1 (non-binding) > > > > > > * Built RPMs and mpacks. > > > * Brought up Metron stack on 12-node CentOS 7 openstack cluster. > >

Re: [VOTE] Metron Release Candidate 0.7.1-RC1

No, not AWS. On Thu, Apr 25, 2019, 8:49 PM Michael Miklavcic wrote: > Just curious, did you also do AWS? I haven't run that in a while. Wondered > if it worked ok. > > On Thu, Apr 25, 2019, 5:48 PM Nick Allen wrote: > > > +1 Verified release with all documented

Re: [VOTE] Metron Release Candidate 0.7.1-RC1

r 25, 2019 at 3:45 PM Nick Allen wrote: > > > No voting required. Those are just docs. Whoever is willing to correct > > and has access, should be able to. Good catch. > > > > On Thu, Apr 25, 2019 at 4:32 PM Michael Miklavcic < > > michael.miklav...@gmail.

Re: [VOTE] Metron Release Candidate 0.7.1-RC1

No voting required. Those are just docs. Whoever is willing to correct and has access, should be able to. Good catch. On Thu, Apr 25, 2019 at 4:32 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > We're also not "incubator-metron" any longer. Do we require any kind of > voting or +1

Re: [DISCUSS] Next Release

0.7.1 Anand Subramanian METRON-2038Done 0.7.1 Nick Allen METRON-2035Done 0.7.1 Nick Allen METRON-2041Done 0.7.1 Michael Miklavcic METRON-2036

Re: [DISCUSS] Next Release

ab some time, I'll try > to get a list of tickets to update to the appropriate contributors. > > Thanks, > Justin > > > > On Tue, Apr 23, 2019, 09:54 Nick Allen wrote: > > > Justin - Can you kick-off the release process? > > > > On Wed, Apr 17, 2019 a

Re: [DISCUSS] Upgrade to HDP 3.1.0

ort, though there is >some possible risk if Ambari and the individual components introduce >changes here) > > Fortunately, Zookeeper appears to have stayed the same across versions. It > might be worthwhile to get a chart of the versions for each platform added > to the epic Jir

Re: [DISCUSS] Next Release

ddress a problem first. Having reviewed > the test-failures list, I'm in favor of moving forward with the release. > > On Wed, Apr 17, 2019 at 9:29 AM Nick Allen wrote: > > > Oh, and that failure from 21 days ago was caused by a downstream issue > > brought in by PR #1364,

[DISCUSS] Upgrade to HDP 3.1.0

We currently support running Metron on an HDP 2.6.5 cluster. I'd like to get Metron updated to run in an HDP 3.1.0

Re: [DISCUSS] Next Release

/cad679a5948ce0ee9866e61bbf75b1f5f255682c On Wed, Apr 17, 2019 at 11:25 AM Nick Allen wrote: > Are you suggesting the release should wait on this? Personally, I don't > feel that we have any *persistently intermittent* test failures that would > block a release. The last build failure on master

Re: [DISCUSS] Next Release

d still consider it important to complete prior to a > >> release > >> > and I agree with Justin's comments in > >> > > >> > > >> > > >> > https://lists.apache.org/thread.html/50b89b919bd8bef3f7fcdef167cbd7e489fa74a1e2da3e4fd

Re: Problems with Dev deployment.

That script (metron-deployment/scripts/platform-info.sh) should be solid. I was going to suggest that you run that on your computer and send us the output. That has often helped us debug issues like this before. If it is not finding Node/NPM, then that is a problem that needs addressed. On Wed

Re: [DISCUSS] Reverting METRON-2023 (PR #1364)

+1 Thanks for digging into this. On Wed, Apr 3, 2019, 6:07 AM Shane Ardell wrote: > Hello everyone, > > I'm sending out this email to notify the community of my intention to > revert the commit for METRON-2023 (PR #1364 > ). This commit introduced a >

Re: [DISCUSS] Next Release

pache.org/jira/browse/METRON-2036. Even though it's > a > > > > non-prod issue, this is a core part of our infrastructure/development > > > > lifecycle that is currently broken and fits with our previous > > agreements > > > of > > > > holdin

[DISCUSS] Next Release

I would like to open a discussion in regards to the next release. Our last 0.7.0 release was on Dec 11th. I believe we have a significant number of bug fixes and performance improvements that would make a worthy point release; 0.7.1. Although, we should review the change log and see if there are

Re: [DISCUSS] Upgrading HBase and Kafka support

+1 for option 3. I am in favor of using Docker for the integration tests for all the reasons that you mentioned. On Fri, Mar 8, 2019 at 9:47 AM Ryan Merriman wrote: > I have been researching the effort involved to upgrade to HDP 3. Along the > way I've found a couple challenging issues that we

Re: [DISCUSS] Architecture documentation

And just to be clear, this is just a continuation of the discussion in this thread. This is not in any way a blocker for Ryan's PR, of course. On Tue, Feb 26, 2019 at 1:44 PM Nick Allen wrote: > > We could also enhance > "metron/dev-utilities/release-utils/validate-jira-for-

Re: [DISCUSS] Architecture documentation

ested. I > think > > we > > > should have an "architecture.md" in metron root that replaces this - > > > > > > > > > > > https://github.com/apache/metron/blob/d7d4fd9afb19e2bd2e66babb7e1514a19eae07d0/README.md#navigating-the-

Re: [DISCUSS] Architecture documentation

I don't think we should hold up this work to document something that wasn't previously documented. A follow-on is sufficient. On Mon, Feb 25, 2019 at 8:50 AM Ryan Merriman wrote: > Recently I submitted a PR > that > introduces a large number of chang

Re: [DISCUSS] Architecture documentation

> Procedurally, do we have a way to ensure that any follow-on documentation > happens prior to a release (anything in the wiki, etc.)? If someone thinks the code base needs X before the next release, then they can bring up X during the release discussion. We don't need additional procedure aroun

Re: [VOTE] Metron Release Candidate 0.7.0-RC1

+1 binding - All of the tarballs, checksums, and signatures are correct - All of the tests and integration tests ran successfully. - The release also spun-up correctly in the development environment. FYI - I had to slightly modify the metron-rc-check script for this to work. See the pa

Re: [VOTE] Metron Release Candidate 0.7.0-RC1

with a more general solution (e.g. having a > script to publish just the KEYS file?) . Given that this causes problems > with the RC check, it seems like we need to update something one way or the > other. It might just be updating the RC check script in the short term. > > >

Re: [VOTE] Metron Release Candidate 0.7.0-RC1

Should there be a KEYS file with the release candidate at https://dist.apache.org/repos/dist/dev/metron/0.7.0-RC1/KEYS? Or was that changed to https://dist.apache.org/repos/dist/release/metron/KEYS ? ``` $ ~/Development/metron/dev-utilities/release-utils/metron-rc-check --version=0.7.0 --candidate

Re: [DISCUSS] Mandatory relocation of Apache git repositories on git-wip-us.apache.org

> > METRON-1931 is created for updating the scripts with the new GitBox > > location. I'll start with this once the repositories are moved. > > > > - Roy > > > > Op ma 10 dec. 2018 om 15:52 schreef Nick Allen : > > > >> +1 Thanks for the hea

Re: [DISCUSS] Mandatory relocation of Apache git repositories on git-wip-us.apache.org

+1 Thanks for the heads up. We will do whatever we need to to help with the transition. On Sun, Dec 9, 2018 at 11:03 AM Otto Fowler wrote: > +1 > > We will need jiras and PR’s for updating our scripts post move however. > > > On December 9, 2018 at 06:32:44, Roy Lenferink (rlenfer...@apache.or

Re: [DISCUSS] Remove `/api/v1/update/replace` endpoint

Heads up - I have not received a response from anyone for about 3 days. I am going to take that as no one has a problem with this change. I will likely merge the PR today. On Tue, Dec 4, 2018 at 4:05 PM Nick Allen wrote: > PR #1284 completely removes the `/api/v1/update/replace` endpo

Re: Metron Release 0.6.1 and/or Plugin release 0.3.0?

#x27;d like taken care of that's relatively close to going in? > > >> > > >> If the plugin gets fixed before we're set to move forward with a core > > >> release (or we choose not to fix it, given the current version is > > >> affected), I'm ha

[DISCUSS] Remove `/api/v1/update/replace` endpoint

PR #1284 completely removes the `/api/v1/update/replace` endpoint. This endpoint is not being used by any other services in Metron currently. As part of the work I am doing to allow Elasticsearch to auto-generate the document ID, I would have had to make code changes to this endpoint to continue

Re: Metron Release 0.6.1 and/or Plugin release 0.3.0?

OK, well either way, I see no need to hold up Metron 0.6.1. On Mon, Dec 3, 2018 at 3:51 PM zeo...@gmail.com wrote: > I believe that 0.2.0 is impacted by the bug. > > Jon > > On Mon, Dec 3, 2018 at 3:50 PM Nick Allen wrote: > > > In light of this comment [1], I propose t

Re: Metron Release 0.6.1 and/or Plugin release 0.3.0?

tron-bro-plugin-kafka 0.3 release is good to go from my side. Thanks > > for all of the reviews Nick > > > > On Wed, Nov 21, 2018 at 11:16 AM Nick Allen wrote: > > > > > Ha. Yes, that definitely counts and makes a ton of sense. Thanks! > > > > > > On We

Re: Unzipping Cypress

Yes, I have noticed that too. If not a way to reduce the time, we should not be logging the unzipping process percentile-by-percentile in the Travis CI builds. On Sat, Nov 24, 2018 at 9:49 AM Otto Fowler wrote: > Anyone else seeing a lot of time taken downloading and unzipping Cypress on > buil

Re: Metron Release 0.6.1 and/or Plugin release 0.3.0?

E statements can cause parse errors in Stellar > (justinleet) closes apache/metron#1268 > METRON-1872 Move rat plugin away from snapshot version (justinleet) > closes apache/metron#1264 > > On Wed, Nov 21, 2018 at 10:59 AM Nick Allen wrote: > > > Also, I'd like to get thi

Re: Metron Release 0.6.1 and/or Plugin release 0.3.0?

>> > > > >> > > Jon > >> > > > >> > > On Wed, Oct 17, 2018 at 10:37 AM Michael Miklavcic < > >> > > michael.miklav...@gmail.com> wrote: > >> > > > >> > >> And I do think we will be ready to roll another Metr

Re: Metron Release 0.6.1 and/or Plugin release 0.3.0?

n-bro-plugin-kafka#2 and > >> apache/metron-bro-plugin-kafka#13? > >> > > > >> > > Jon > >> > > > >> > > On Wed, Oct 17, 2018 at 10:37 AM Michael Miklavcic < > >> > > michael.miklav...@gmail.com> wrote: > >

Re: [DISCUSS] Slack Channel Use

lat > > > as > > > compared to last quarter. That's not to say that we couldn't stand > more > > > discussion on the lists, but a lot of the dev discussion happens on > > github > > > and JIRA and I'm happy to see an uptick in user traffic. > &

Re: [DISCUSS] Deprecate split-join enrichment topology in favor of unified enrichment topology

+1 On Thu, Nov 1, 2018, 6:27 PM Justin Leet wrote: > +1, I haven't seen any case where the split-join topology isn't made > obsolete by the unified topology. > > On Thu, Nov 1, 2018 at 6:17 PM Michael Miklavcic < > michael.miklav...@gmail.com> wrote: > > > Fellow Metronians, > > > > We've had th

Fwd: [DISCUSS] Day 1 User Experience - Getting Metron Running

hen. > > > On October 26, 2018 at 14:52:27, Nick Allen (n...@nickallen.org) wrote: > > From what I can tell, Katakoda functions mainly through hosting Docker > containers. So if I were to create Katakoda demo like "Introduction to > Stellar REPL", I would need to create

Re: [DISCUSS] Day 1 User Experience - Getting Metron Running

nd hosts your container for each user session. That is my assumption from looking through some of the demos that currently exist. On Fri, Oct 26, 2018 at 2:42 PM Otto Fowler wrote: > What is the metron on docker part? > > > On October 26, 2018 at 14:37:48, Nick Allen (n...@nickallen

Re: [DISCUSS] Day 1 User Experience - Getting Metron Running

n the user experience for > > first timers to Metron. Katakoda seems very interesting - simple and > > straight-forward. I loved the way you can provide instructions, commands > > (that can be directly clicked!), links, explanation and so on. > > > > Regards, > >

[DISCUSS] Day 1 User Experience - Getting Metron Running

We all know spinning up the development environment is a pain. Unfortunately, it is the only way for a new user to get a feel for Metron. We need a better way to introduce new users to Metron. I am hoping we can brainstorm ways to improve that experience. Here are a few thoughts that might help s

Re: metron-elasticsearch integration tests failing after merging in master

I usually create a JIRA when I see an intermittent failure like this. But I do not see a record of this particular issue before. https://jira.apache.org/jira/issues/?jql=text%20~%20%22Intermittent%22%20AND%20project%20%3D%20Metron%20AND%20resolution%20%3D%20Unresolved On Wed, Oct 24, 2018 at 12

Re: [DISCUSS] Slack Channel Use

:23 AM Casey Stella wrote: > > > Agreed, the benefit of the mailing list is that it’s searchable by > ponymail > > and the major search engines. > > On Mon, Oct 22, 2018 at 17:18 Nick Allen wrote: > > > > > I don't know that it is the same kind of s

Re: Revert PR #1218

ne, no? > > Simon > > On Tue, 23 Oct 2018 at 19:58, Nick Allen wrote: > > > Hi Guys - > > > > @rmerriman tracked down some problems that were introduced with my PR > > #1218. Thanks to him for finding this. The change was intended to > improve &g

Revert PR #1218

Hi Guys - @rmerriman tracked down some problems that were introduced with my PR #1218. Thanks to him for finding this. The change was intended to improve Elasticsearch write performance by allowing Elasticsearch to set its own document ID. The problem is that if you then go to the Alerts UI and

Re: [DISCUSS] Slack Channel Use

t; > preference that we use the mailing lists for the reasons you stated > (and > >> > also because not everyone has access to slack generally). On the other > >> > hand, I think an interactive medium like Slack has a lot of advantages > >> in > >> >

Re: [DISCUSS] Slack Channel Use

is that > there are more than Jon and I answering now. > > > On October 22, 2018 at 12:18:08, Nick Allen (n...@nickallen.org) wrote: > > It seems that we are seeing a lot of Metron usage and support questions on > the Slack Channel. > These are questions that previously woul

[DISCUSS] Slack Channel Use

It seems that we are seeing a lot of Metron usage and support questions on the Slack Channel. These are questions that previously would have been directed to the User or Dev mailing lists. Since this is occurring in the Slack Channel, the conversations are not archived. In my opinion, this is not

[DISCUSS] Recurrent Large Indexing Error Messages

I want to discuss solutions for the problem that I have described in METRON-1832; Recurrent Large Indexing Error Messages. I feel this is a very easy trap to fall into when using the default settings that currently come with Metron. ## Problem https://issues.apache.org/jira/browse/METRON-1832

Re: Metron Release 0.6.1 and/or Plugin release 0.3.0?

I am in favor of a release for both. There are a lot of really useful bug fixes, management of pcap through Ambari, more flexibility for configuring JAAS in Ambari, increased Elasticsearch performance, the Syslog parser, and the Batch Profiler, among others. I would be happy with calling it a 0.6.

Re: [DISCUSS] Switching to a better alternative of Pikaday.js

> Before making a decision on what's next, I'd to ask you a question. Is it > really a priority and is it really worth the effort to touch our currently used date picker component to get ~15% reduction in the bundle size by removing moment? As an aside, I think there is a greater benefit here too

Re: [DISCUSS] Add e2e step to PR checklist

feedback, Nick. I agree, we need them to be reliable > > enough to not be a source of constant false positives prior to putting > them > > into the checklist. > > On Thu, Oct 4, 2018 at 15:34 Nick Allen wrote: > > > > > I think we still have an issue of reliability

Re: [DISCUSS] Add e2e step to PR checklist

I think we still have an issue of reliability. I can never reliably get them all to pass. I have no idea which failures are real. Am I the only one that experiences this? We need a reliable pass/fail on these before we talk about adding them to the checklist. For example, I just tried to run t

Re: Metron dev environments moving to require Ansible 2.4+

I have been able to spin-up a development environment today without a problem. On Mon, Oct 1, 2018 at 3:58 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Anyone run latest master today with full dev? I'm seeing > NoClassDefFoundError exceptions on starting enrichments. I upgraded to

Re: [DISCUSS] Batch Profiler Feature Branch

gt; >> Thanks a lot for the contribution, this is great stuff! > >> > >> On Wed, Sep 26, 2018 at 6:26 PM Nick Allen wrote: > >> > >> > Or support to be offered for merging this feature branch into master? > >> > > >>

Re: [DISCUSS] Batch Profiler Feature Branch

Or support to be offered for merging this feature branch into master? On Wed, Sep 26, 2018 at 6:20 PM Nick Allen wrote: > Thanks for the review. With https://github.com/apache/metron/pull/1209 > complete, > I think the feature branch is ready to be merged. Sounds like I have

Re: [DISCUSS] Batch Profiler Feature Branch

mail.com> wrote: > I just made a couple minor comments on that PR, and I am in agreement about > the readiness for merging with master. Good stuff Nick. > > On Fri, Sep 21, 2018 at 12:37 PM Nick Allen wrote: > > > Here is a PR that adds the input time constraints to the

Re: [DISCUSS] Batch Profiler Feature Branch

branch has been merged. Let me know if you disagree. Thanks On Thu, Sep 20, 2018 at 1:55 PM Nick Allen wrote: > Yeah, agreed. Per use case 3, when deploying to production there really > wouldn't be a huge overlap like 3 months of already profiled data. Its day > 1, the profile was

Re: [DISCUSS] Batch Profiler Feature Branch

batch job > over all the data. It might also make it easier to deal with remediation, > ie an error doesn't force you to re-run over the entire history. Same goes > for testing out the profile seeing batch job in the first place. > > On Thu, Sep 20, 2018 at 11:23 AM Nick Allen wr

Re: [DISCUSS] Batch Profiler Feature Branch

), in its current > form the batch job runs over all 9 months? > > On Thu, Sep 20, 2018 at 11:13 AM Nick Allen wrote: > > > > How do we establish "tm" from 1.1 above? Any concerns about overlap or > > gaps after the seeding is performed? > > > > Good

Re: [DISCUSS] Batch Profiler Feature Branch

nt > >2. I've already done #1 above. Time goes by and now I want to add a > new > >profile. > > 1. Same first step above > > 2. I would run the batch loader with *only* that new profile > > definition to seed? > > > > Forgive m

Re: [DISCUSS] Batch Profiler Feature Branch

loader with *only* that new profile > definition to seed? > > Forgive me if I missed this in PR's and discussion in the FB, but how do we > establish "tm" from 1.1 above? Any concerns about overlap or gaps after the > seeding is performed? > > On Th

Re: [DISCUSS] Batch Profiler Feature Branch

gt; > The profile not being able to read from ZK feels like a fairly > substantial, > > if subtle, set of potential problems. I'd like to see that in either > > before merging or at least pretty soon after merging. Is it a lot of > work > > to add that functionality

Re: [DISCUSS] Batch Profiler Feature Branch

> I think what you have outlined above is a good initial stab at the > > feature. Manual install of spark is not a big deal. Configuring via > > command line while we mature this feature is ok as well. Doesn't look > like > > configuration steps are too hard. I thin

[DISCUSS] Batch Profiler Feature Branch

I would like to open a discussion to get the Batch Profiler feature branch merged into master as part of METRON-1699 [1] Create Batch Profiler. All of the work that I had in mind for our first draft of the Batch Profiler has been completed. Please take a look through what I have and let me know i

Re: [DISCUSS] PCAP data for testing and development

I would just be worried about resource constraints on the VM. But Simon's idea of 'do a loop and stop' might be a good solution. We could probably orchestrate that with Ansible tags actually. If you pass the tag, it will 'do a loop and stop', but by default it keeps the current behavior. On

Re: [RESULT][VOTE] Metron Release Candidate 0.6.0-RC1

Woop woop! On Wed, Sep 12, 2018 at 12:15 PM Justin Leet wrote: > The vote has passed. Including my +1, the voting was: > 3 binding +1’s > 1 non-binding +1’s > no 0’s > no -1’s. >

  1   2   3   4   >