I just registered 2 channels under my nick (you need to be registered with freenode to create a channel) …. I am on a mac now and textual5 works for me. These are open channels.
13:10:20] -ChanServ- #apache-metron is now registered to ddutta. [13:10:20] -ChanServ- [13:10:20] -ChanServ- Channel guidelines can be found on the freenode website [13:10:20] -ChanServ- (http://freenode.net/channel_guidelines.shtml). [13:10:20] -ChanServ- This is a primary namespace channel as per [13:10:20] -ChanServ- http://freenode.net/policy.shtml#primarychannels [13:10:20] -ChanServ- If you do not own this name, please consider [13:10:20] -ChanServ- dropping #apache-metron and using ##apache-metron instead. [13:11:19] <ddutta> register #apache-metron-dev [13:11:19] -ChanServ- #apache-metron-dev is now registered to ddutta. [13:11:19] -ChanServ- [13:11:19] -ChanServ- Channel guidelines can be found on the freenode website [13:11:19] -ChanServ- (http://freenode.net/channel_guidelines.shtml). [13:11:19] -ChanServ- This is a primary namespace channel as per [13:11:19] -ChanServ- http://freenode.net/policy.shtml#primarychannels [13:11:19] -ChanServ- If you do not own this name, please consider [13:11:19] -ChanServ- dropping #apache-metron-dev and using ##apache-metron-dev instead. On 4/11/16, 12:06 PM, "James Sirota" <jsir...@hortonworks.com> wrote: >Great, thanks, Debo. Where can I find instructions on how to get to it? > >Thanks, >James > > > > >On 4/11/16, 9:41 AM, "Debo Dutta (dedutta)" <dedu...@cisco.com> wrote: > >>Hi James >> >>Ok set it up and ack ….. >> >>Thx >> >> >> >> >> >>On 4/10/16, 6:31 PM, "James Sirota" <jsir...@hortonworks.com> wrote: >> >>>Hi Debo, >>> >>>I think it would be great if you set it up >>> >>>Thanks, >>>James >>> >>> >>> >>> >>>On 4/10/16, 6:25 PM, "Debojyoti Dutta" <ddu...@gmail.com> wrote: >>> >>>>I have set it up for another open source effort in the past and it was not >>>>very hard. Am happy to volunteer if needed. >>>> >>>>Thx >>>>Debo >>>> >>>>Sent from my iPhone >>>> >>>>> On Apr 10, 2016, at 5:53 PM, James Sirota <jsir...@hortonworks.com> wrote: >>>>> >>>>> I’d be open to an IRC channel. Does anyone know if Apache allows this? >>>>> If yes, does anyone know how to set one up? >>>>> >>>>> Thanks, >>>>> James >>>>> >>>>> >>>>> >>>>> >>>>>> On 4/10/16, 4:52 PM, "Debojyoti Dutta" <ddu...@gmail.com> wrote: >>>>>> >>>>>> Hi Nick >>>>>> >>>>>> I like your suggestions. For the enrichment layer do you think it would >>>>>> also include any advanced analytics. Else we might want to have an >>>>>> analytics layer. >>>>>> >>>>>> It would be good to have an arch which could be extended for new >>>>>> functionality. >>>>>> >>>>>> However Ryan's suggestion of the ui API and deployer also makes sense. >>>>>> >>>>>> Should we have an IRC channel to discuss this or maybe etherpad? >>>>>> >>>>>> Debo >>>>>> >>>>>> Sent from my iPhone >>>>>> >>>>>>> On Apr 10, 2016, at 4:36 PM, Nick Allen <n...@nickallen.org> wrote: >>>>>>> >>>>>>> It might help to think of our code base as four separate types of >>>>>>> functionality. This is primarily meant to give us a framework to think >>>>>>> about the organization of Metron (and drive more discussion), rather >>>>>>> than >>>>>>> my proposal for a specific structure. >>>>>>> >>>>>>> - Sensor - Anything that captures external, non-streaming data and >>>>>>> presents it in a form ready for stream processing. >>>>>>> - Input - Responsible for preparing streaming data for enrichment. The >>>>>>> existing "parsers" fit neatly into this space. >>>>>>> - Enrichment - Responsible for enriching an incoming data feed like >>>>>>> geoip, asset enrichment, threat intel lookups, etc. >>>>>>> - Output - Responsible for persisting data that has been processed by >>>>>>> Metron which obviously means search indexers or data stores. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Apr 8, 2016 at 4:46 PM, Ryan Merriman >>>>>>> <rmerri...@hortonworks.com> >>>>>>> wrote: >>>>>>> >>>>>>>> All, >>>>>>>> >>>>>>>> I would like to propose a review and refactor of the current project >>>>>>>> organization within Metron. Much of the way the legacy code was >>>>>>>> organized >>>>>>>> does not make sense anymore and could be designed so that it is easier >>>>>>>> to >>>>>>>> navigate and understand. Our test coverage has increased >>>>>>>> substantially so >>>>>>>> I believe we can do this with confidence. >>>>>>>> >>>>>>>> First off, I think we should agree on a naming convention. I see some >>>>>>>> projects (YARN and Storm for example) that prepend the sub-project >>>>>>>> with the >>>>>>>> name of the top-level project (storm-core for example). Metron also >>>>>>>> currently does this (Metron-Common). I think that's fine, although in >>>>>>>> the >>>>>>>> case of Metron, I feel like having "Metron" prepended is redundant. >>>>>>>> Regardless of whether we decide to stick with that approach, I propose >>>>>>>> that >>>>>>>> project names be uniform and lowercase. For example, under these >>>>>>>> assumptions "Metron-Common" would change to "common". >>>>>>>> >>>>>>>> The first level of organization makes sense to me. Only change I would >>>>>>>> make would be to project names: >>>>>>>> >>>>>>>> * deployment >>>>>>>> * streaming >>>>>>>> * ui >>>>>>>> >>>>>>>> Or if we want to keep metron in project names: >>>>>>>> >>>>>>>> * metron-deployment >>>>>>>> * metron-streaming >>>>>>>> * metron-ui >>>>>>>> >>>>>>>> For now I don't see any changes necessary in deployment or ui >>>>>>>> organization. I see the streaming project structure primarily driven >>>>>>>> by 2 >>>>>>>> things: the Maven dependency tree and deployment targets. For >>>>>>>> example, >>>>>>>> solr and elasticsearch code should be separated (because their >>>>>>>> dependency >>>>>>>> on lucene conflicts) but both will depend on common enrichment code. >>>>>>>> Also, >>>>>>>> now that parser, enrichment and pcap topologies are separate, code for >>>>>>>> those topologies will be deployed as separate jars. No reason to >>>>>>>> include >>>>>>>> parser code in enrichment topologies and vice-versa. Any other >>>>>>>> considerations I'm missing? >>>>>>>> >>>>>>>> With that being said, here is my initial proposal: >>>>>>>> >>>>>>>> * common - Any common code that all topologies depend on >>>>>>>> (configuration classes, generic writers for example). No dependencies >>>>>>>> on >>>>>>>> other Metron projects. >>>>>>>> * test - Contains utilities for writing unit tests, sample configs >>>>>>>> and >>>>>>>> sample data. Will depend on common. >>>>>>>> * integration-test - Contains utilities and classes needed to run our >>>>>>>> integration tests (in memory components for example). Will depend on >>>>>>>> common and test. >>>>>>>> * dataload - Contains all code related to data loading. Will also >>>>>>>> include any property files needed and integration tests. Will depend >>>>>>>> on >>>>>>>> common, test (test scope), and integration-test (test scope). >>>>>>>> * parser - All code specific to the parser topologies. Would also >>>>>>>> include scripts, property files, flux files and parser topology >>>>>>>> integration >>>>>>>> tests. This project will depend on common, test (test scope), and >>>>>>>> integration-testing (test scope). >>>>>>>> * enrichment - All code specific to the enrichment topologies (except >>>>>>>> solr and elasticsearch). Would also include scripts, property files, >>>>>>>> flux >>>>>>>> files and enrichment topology integration tests. This project will >>>>>>>> depend >>>>>>>> on common, test (test scope), and integration-test (test scope). >>>>>>>> * elasticsearch - All Elasticsearch related code. Will depend on >>>>>>>> enrichment. >>>>>>>> * solr - All Solr related code. Will depend on enrichment. >>>>>>>> * pcap - All code specific to the topology dedicated to pcap. Would >>>>>>>> also include scripts, property files, flux files and pcap integration >>>>>>>> test. This project will depend on common, test (test scope) and >>>>>>>> integration-test (test scope). >>>>>>>> * api - This will serve as a generic replacement for >>>>>>>> Metron-Pcap_Service. Will contain all code to build a Metron web >>>>>>>> service >>>>>>>> middle layer that can expose APIs through REST or other client >>>>>>>> protocols. >>>>>>>> Could possibly depend on all other projects or separated further if >>>>>>>> version >>>>>>>> conflicts arise (separate api projects for solr and elasticsearch for >>>>>>>> example). >>>>>>>> >>>>>>>> Looking forward to hearing everyone's feedback and great ideas. >>>>>>>> >>>>>>>> Ryan Merriman >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Nick Allen <n...@nickallen.org> >>>>>> >>>>
