Just wanted to update this thread: I've been diligently working to the plan we discussed above:
*1) Refactor existing Ansible deployment to use the Ambari MPack to install metron-common, metron-enrichments and metron-parsers. * *2) Regenerate quick-dev to leverage the change.* 3) Create rpm packages for all deployed components that don't currently have them. - Sensor probes - Sensor stubs 4) Create MPack service defs for the RPMs in (2). 5) Refactor existing Ansible deployment to use the Ambari MPack to install all services. 6) Regenerate quick-dev to leverage the change. 7) Plan iteration 2 to see if there are other opportunities to reduce our use of Ansible. I've completed #1 and will have #2 completed shortly, that will close out METRON-671. If we're all still good with that direction, once I finish 671, I'd like to cut JIRAs for #3 and #4. Thoughts? -D... On Thu, Jan 19, 2017 at 9:00 AM, David Lyle <dlyle65...@gmail.com> wrote: > For the first increment, I am planning on using the Ambari Metron topology > service that currently exists in the MPack. Handling the side loading isn't > in the scope of this proposal. We'll need to design that separately. > > -D... > > > On Thu, Jan 19, 2017 at 8:48 AM, Otto Fowler <ottobackwa...@gmail.com> > wrote: > >> So - my prototype was adding a new parser type and completely integrating >> it with the install, all the way down to monit templates. >> All of that work was configuration work in ansible. >> >> I think I’m asking about changes to something that wasn’t documented >> anyways, as I think you are pointing out by reference, so I’ll just say >> that this change has an effect on side loading. You will be building an >> Ambari Metron Topology service, probably from some template - hopefully >> using maven archetypes or something the whole way. >> >> >> On January 19, 2017 at 05:56:42, David Lyle (dlyle65...@gmail.com) wrote: >> >> Looks like my replies were only going to Otto, sorry about that- I'll >> gather them here: >> >> "What does this do for Monit?" >> >> Monit would be deprecated for components under Ambari management. >> >> "How would this effect deploying new parsers or parsers not shipped? >> When I prototyped this I added a monit entry." >> >> Hard to say having not seen Otto's prototype, but I suspect no effect. >> >> Fwiw, there is a jira about sideloading that is meant for deploying >> custom >> parsers. Last I looked, the management was still up for design. >> >> I"ve opened https://issues.apache.org/jira/browse/METRON-667 to track >> this >> work. I'd like to get started. Thoughts? >> >> -D... >> >> >> On Wed, Jan 18, 2017 at 1:28 PM, David Lyle <dlyle65...@gmail.com> >> wrote: >> >> > Hard to say having not seen your prototype, but I suspect no effect. >> > >> > Fwiw, there is a jira about sideloading that is meant for deploying >> custom >> > parsers. Last I looked, the management was still up for design. >> > >> > -D... >> > >> > On Wed, Jan 18, 2017 at 13:07 Otto Fowler <ottobackwa...@gmail.com> >> wrote: >> > >> >> How would this effect deploying new parsers or parsers not shipped? >> >> When I prototyped this I added a monit entry. >> >> >> >> >> >> On January 17, 2017 at 10:34:32, David Lyle (dlyle65...@gmail.com) >> wrote: >> >> >> >> In our "Dev Guide and Committer Review Guide additions" discussion, we >> had >> >> >> >> >> >> a bit of a side discussion about reducing reliance (perhaps to zero) >> on >> >> >> >> >> >> Ansible for our installation. >> >> >> >> >> >> >> >> >> >> >> >> It seemed there was consensus around that idea (if not, please let me >> >> >> >> >> >> know), so I propose the following steps to get there: >> >> >> >> >> >> >> >> >> >> >> >> 1) Refactor existing Ansible deployment to use the Ambari MPack to >> install >> >> >> >> >> >> metron-common, metron-enrichments and metron-parsers. >> >> >> >> >> >> 2) Regenerate quick-dev to leverage the change. >> >> >> >> >> >> 3) Create rpm packages for all deployed components that don't >> currently >> >> >> >> >> >> have them. >> >> >> >> >> >> - Sensor probes >> >> >> >> >> >> - Sensor stubs >> >> >> >> >> >> 4) Create MPack service defs for the RPMs in (2). >> >> >> >> >> >> 5) Refactor existing Ansible deployment to use the Ambari MPack to >> install >> >> >> >> >> >> all services. >> >> >> >> >> >> 6) Regenerate quick-dev to leverage the change. >> >> >> >> >> >> 7) Plan iteration 2 to see if there are other opportunities to reduce >> our >> >> >> >> >> >> use of Ansible. >> >> >> >> >> >> >> >> >> >> >> >> One note: if we decide to go this direction, it'd be helpful if, >> during >> >> the >> >> >> >> >> >> transition, we stopped adding additional Ansible deployment code. >> >> >> >> >> >> >> >> >> >> >> >> Thoughts? >> >> >> >> >> >> >> >> >> >> >> >> Thanks, >> >> >> >> >> >> >> >> >> >> >> >> -David... >> >> >> >> >> >> >> >> >