I'm a bit worried about copying and pasting from the NiFi project their nar infrastructure. That seems..unclean to me and since we're not using the classloader part of nar for this, does it make more sense to just use jar?
On Fri, Mar 10, 2017 at 3:50 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > Compared to how much time vagrant up takes now, you won’t even notice it ;) > > That is definitely an option. I guess what I want to work out is if we > are going to want to > go to NAR, why not just go to NAR. > > In the end, the customer for this - like Jon Zeolla, isn’t going to care > about the intermediate step, > he wants the archetype that builds the ‘metron parser plugin’. > > Which is why I hesitate to put out an archetype that is going to obsolete > so soon. > > Does that make sense? > > On March 10, 2017 at 14:50:55, Casey Stella (ceste...@gmail.com) wrote: > > I'm a little concerned about this increasing the size and length of the > build due to the repeated shading. Should we figure out a way to deploy > jars with provided dependencies on metron-parser-common as suggested in > the > previous JIRAs first? > > On Fri, Mar 10, 2017 at 2:31 PM, Matt Foley <mfo...@hortonworks.com> > wrote: > > > It sounds like: > > - This is a self-contained chunk of work, that can be tested, reviewed, > > and committed on its own, then the other ideas you propose can follow > it. > > - It crosses a lot of lines, and restructures a lot of code, so will > “rot” > > fairly quickly as other people make commits, so if possible you should > get > > a PR out there and we should work through it as soon as possible. > > Are those both true? > > > > How do other people feel about grouping a given sensor’s parser, > enricher, > > indexing logic all together? It seems to have multiple advantages are > > there also disadvantages? > > > > On 3/10/17, 6:31 AM, "Otto Fowler" <ottobackwa...@gmail.com> wrote: > > > > As previously discussed here, I have been working on side loading of > > parsers. The goals of this work are: > > * Make it possible of developers to create, maintain and deploy parsers > > outside of the Metron code tree and not have to fork > > * Create maven archetype support for developers of parsers > > * Introduce a parser ‘lifecycle’ to support multiple instances and > > configurations, states of being installed, under configuration, and > > deployed > > etc. > > > > I would like to have some discussion based on where I am after rebasing > > onto METRON-671 which revamps deployment to be totally ambari based. > > > > > > Parsers as components: > > > > I have all the parsers broken out into individual packages/rpms/jars. > > What I have done is taken metron-parsers and broken it out to: > > > > * metron-parsers-common > > * This has all the base classes and interfaces, common testing > > components > > etc > > * metron-parser-base > > * This has the Grok, CSV, and JsonMap parsers and support > > * metron-parser-X > > * A module per parser type which we currently have in the system > > * Each parser has all the indexing, enrichment and parser > > configurations > > for that parser in its package > > > > I will go into packaging and deployment issues in another email. > > > > I have this all working: > > * the parsers are built > > * the parsers are tested > > * the parsers are integrated into the deployment build such that > > vagrant up > > just works as previously in full and quick dev > > * maven component of rpm docker > > * the metron.spec file > > * ambari installation > > * zookeeper configuration deployment > > * the ambari parser service code > > * the Rest interface works > > * see all installed parser configurations etc > > > > > > So this part of the work, is I think ready for a PR and review/next > > steps > > on it’s own. > > > > I think that it sets up the components and is a base for building out > > the > > rest of the functionality we want. > > > > > > > >
