That's correct. Quick dev will be built using the MPack, but after that, Quick Dev will remove the Metron components and reinstall them using REST calls to Ambari, so pieces of the MPack that was used to create the Quick Dev image would be executed. RPMs are executed in all cases.
The instructions (that will need a bit of modification because of a name-change) are here: https://github.com/apache/incubator-metron/blob/master/metron-deployment/packer-build/README.md -D... On Thu, Mar 2, 2017 at 3:02 PM, Casey Stella <ceste...@gmail.com> wrote: > Just to clarify, your 1 and 2, which you're working on, will give us the > ability with full-dev (not quick-dev) to exercise the RPMs and management > pack on the non-sensor code (i.e. the current state of the management > pack). As far as I'm concerned, this is huge. This ensures we have an > easy vehicle to ensure PRs work with the management pack. I think a couple > things should be ensured going forward: > > - People whose PR have changes that affect the management pack, should > - notify the reviewers that it was tested on full-dev > - They should regenerate quickdev to ensure things aren't broken. > Dave, can you remind us all where the instructions are for that? > > > On Thu, Mar 2, 2017 at 9:47 AM, David Lyle <dlyle65...@gmail.com> wrote: > > > Just wanted to update this thread: > > > > I've been diligently working to the plan we discussed above: > > > > *1) Refactor existing Ansible deployment to use the Ambari MPack to > install > > metron-common, metron-enrichments and metron-parsers. * > > *2) Regenerate quick-dev to leverage the change.* > > 3) Create rpm packages for all deployed components that don't currently > > have them. > > - Sensor probes > > - Sensor stubs > > 4) Create MPack service defs for the RPMs in (2). > > 5) Refactor existing Ansible deployment to use the Ambari MPack to > install > > all services. > > 6) Regenerate quick-dev to leverage the change. > > 7) Plan iteration 2 to see if there are other opportunities to reduce our > > use of Ansible. > > > > I've completed #1 and will have #2 completed shortly, that will close out > > METRON-671. If we're all still good with that direction, once I finish > > 671, I'd like to cut JIRAs for #3 and #4. > > > > Thoughts? > > > > -D... > > > > > > On Thu, Jan 19, 2017 at 9:00 AM, David Lyle <dlyle65...@gmail.com> > wrote: > > > > > For the first increment, I am planning on using the Ambari Metron > > topology > > > service that currently exists in the MPack. Handling the side loading > > isn't > > > in the scope of this proposal. We'll need to design that separately. > > > > > > -D... > > > > > > > > > On Thu, Jan 19, 2017 at 8:48 AM, Otto Fowler <ottobackwa...@gmail.com> > > > wrote: > > > > > >> So - my prototype was adding a new parser type and completely > > integrating > > >> it with the install, all the way down to monit templates. > > >> All of that work was configuration work in ansible. > > >> > > >> I think I’m asking about changes to something that wasn’t documented > > >> anyways, as I think you are pointing out by reference, so I’ll just > say > > >> that this change has an effect on side loading. You will be building > > an > > >> Ambari Metron Topology service, probably from some template - > hopefully > > >> using maven archetypes or something the whole way. > > >> > > >> > > >> On January 19, 2017 at 05:56:42, David Lyle (dlyle65...@gmail.com) > > wrote: > > >> > > >> Looks like my replies were only going to Otto, sorry about that- I'll > > >> gather them here: > > >> > > >> "What does this do for Monit?" > > >> > > >> Monit would be deprecated for components under Ambari management. > > >> > > >> "How would this effect deploying new parsers or parsers not shipped? > > >> When I prototyped this I added a monit entry." > > >> > > >> Hard to say having not seen Otto's prototype, but I suspect no effect. > > >> > > >> Fwiw, there is a jira about sideloading that is meant for deploying > > >> custom > > >> parsers. Last I looked, the management was still up for design. > > >> > > >> I"ve opened https://issues.apache.org/jira/browse/METRON-667 to track > > >> this > > >> work. I'd like to get started. Thoughts? > > >> > > >> -D... > > >> > > >> > > >> On Wed, Jan 18, 2017 at 1:28 PM, David Lyle <dlyle65...@gmail.com> > > >> wrote: > > >> > > >> > Hard to say having not seen your prototype, but I suspect no effect. > > >> > > > >> > Fwiw, there is a jira about sideloading that is meant for deploying > > >> custom > > >> > parsers. Last I looked, the management was still up for design. > > >> > > > >> > -D... > > >> > > > >> > On Wed, Jan 18, 2017 at 13:07 Otto Fowler <ottobackwa...@gmail.com> > > >> wrote: > > >> > > > >> >> How would this effect deploying new parsers or parsers not shipped? > > >> >> When I prototyped this I added a monit entry. > > >> >> > > >> >> > > >> >> On January 17, 2017 at 10:34:32, David Lyle (dlyle65...@gmail.com) > > >> wrote: > > >> >> > > >> >> In our "Dev Guide and Committer Review Guide additions" discussion, > > we > > >> had > > >> >> > > >> >> > > >> >> a bit of a side discussion about reducing reliance (perhaps to > zero) > > >> on > > >> >> > > >> >> > > >> >> Ansible for our installation. > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> It seemed there was consensus around that idea (if not, please let > me > > >> >> > > >> >> > > >> >> know), so I propose the following steps to get there: > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> 1) Refactor existing Ansible deployment to use the Ambari MPack to > > >> install > > >> >> > > >> >> > > >> >> metron-common, metron-enrichments and metron-parsers. > > >> >> > > >> >> > > >> >> 2) Regenerate quick-dev to leverage the change. > > >> >> > > >> >> > > >> >> 3) Create rpm packages for all deployed components that don't > > >> currently > > >> >> > > >> >> > > >> >> have them. > > >> >> > > >> >> > > >> >> - Sensor probes > > >> >> > > >> >> > > >> >> - Sensor stubs > > >> >> > > >> >> > > >> >> 4) Create MPack service defs for the RPMs in (2). > > >> >> > > >> >> > > >> >> 5) Refactor existing Ansible deployment to use the Ambari MPack to > > >> install > > >> >> > > >> >> > > >> >> all services. > > >> >> > > >> >> > > >> >> 6) Regenerate quick-dev to leverage the change. > > >> >> > > >> >> > > >> >> 7) Plan iteration 2 to see if there are other opportunities to > reduce > > >> our > > >> >> > > >> >> > > >> >> use of Ansible. > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> One note: if we decide to go this direction, it'd be helpful if, > > >> during > > >> >> the > > >> >> > > >> >> > > >> >> transition, we stopped adding additional Ansible deployment code. > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> Thoughts? > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> Thanks, > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> -David... > > >> >> > > >> >> > > >> >> > > >> > > >> > > > > > >
