Just wanted to update this thread:
I've been diligently working to the plan we discussed above:
*1) Refactor existing Ansible deployment to use the Ambari MPack to install
metron-common, metron-enrichments and metron-parsers. *
*2) Regenerate quick-dev to leverage the change.*
3) Create rpm packages for all deployed components that don't currently
have them.
- Sensor probes
- Sensor stubs
4) Create MPack service defs for the RPMs in (2).
5) Refactor existing Ansible deployment to use the Ambari MPack to install
all services.
6) Regenerate quick-dev to leverage the change.
7) Plan iteration 2 to see if there are other opportunities to reduce our
use of Ansible.
I've completed #1 and will have #2 completed shortly, that will close out
METRON-671. If we're all still good with that direction, once I finish
671, I'd like to cut JIRAs for #3 and #4.
Thoughts?
-D...
On Thu, Jan 19, 2017 at 9:00 AM, David Lyle <dlyle65...@gmail.com> wrote:
> For the first increment, I am planning on using the Ambari Metron topology
> service that currently exists in the MPack. Handling the side loading isn't
> in the scope of this proposal. We'll need to design that separately.
>
> -D...
>
>
> On Thu, Jan 19, 2017 at 8:48 AM, Otto Fowler <ottobackwa...@gmail.com>
> wrote:
>
>> So - my prototype was adding a new parser type and completely integrating
>> it with the install, all the way down to monit templates.
>> All of that work was configuration work in ansible.
>>
>> I think I’m asking about changes to something that wasn’t documented
>> anyways, as I think you are pointing out by reference, so I’ll just say
>> that this change has an effect on side loading. You will be building an
>> Ambari Metron Topology service, probably from some template - hopefully
>> using maven archetypes or something the whole way.
>>
>>
>> On January 19, 2017 at 05:56:42, David Lyle (dlyle65...@gmail.com) wrote:
>>
>> Looks like my replies were only going to Otto, sorry about that- I'll
>> gather them here:
>>
>> "What does this do for Monit?"
>>
>> Monit would be deprecated for components under Ambari management.
>>
>> "How would this effect deploying new parsers or parsers not shipped?
>> When I prototyped this I added a monit entry."
>>
>> Hard to say having not seen Otto's prototype, but I suspect no effect.
>>
>> Fwiw, there is a jira about sideloading that is meant for deploying
>> custom
>> parsers. Last I looked, the management was still up for design.
>>
>> I"ve opened https://issues.apache.org/jira/browse/METRON-667 to track
>> this
>> work. I'd like to get started. Thoughts?
>>
>> -D...
>>
>>
>> On Wed, Jan 18, 2017 at 1:28 PM, David Lyle <dlyle65...@gmail.com>
>> wrote:
>>
>> > Hard to say having not seen your prototype, but I suspect no effect.
>> >
>> > Fwiw, there is a jira about sideloading that is meant for deploying
>> custom
>> > parsers. Last I looked, the management was still up for design.
>> >
>> > -D...
>> >
>> > On Wed, Jan 18, 2017 at 13:07 Otto Fowler <ottobackwa...@gmail.com>
>> wrote:
>> >
>> >> How would this effect deploying new parsers or parsers not shipped?
>> >> When I prototyped this I added a monit entry.
>> >>
>> >>
>> >> On January 17, 2017 at 10:34:32, David Lyle (dlyle65...@gmail.com)
>> wrote:
>> >>
>> >> In our "Dev Guide and Committer Review Guide additions" discussion, we
>> had
>> >>
>> >>
>> >> a bit of a side discussion about reducing reliance (perhaps to zero)
>> on
>> >>
>> >>
>> >> Ansible for our installation.
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> It seemed there was consensus around that idea (if not, please let me
>> >>
>> >>
>> >> know), so I propose the following steps to get there:
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> 1) Refactor existing Ansible deployment to use the Ambari MPack to
>> install
>> >>
>> >>
>> >> metron-common, metron-enrichments and metron-parsers.
>> >>
>> >>
>> >> 2) Regenerate quick-dev to leverage the change.
>> >>
>> >>
>> >> 3) Create rpm packages for all deployed components that don't
>> currently
>> >>
>> >>
>> >> have them.
>> >>
>> >>
>> >> - Sensor probes
>> >>
>> >>
>> >> - Sensor stubs
>> >>
>> >>
>> >> 4) Create MPack service defs for the RPMs in (2).
>> >>
>> >>
>> >> 5) Refactor existing Ansible deployment to use the Ambari MPack to
>> install
>> >>
>> >>
>> >> all services.
>> >>
>> >>
>> >> 6) Regenerate quick-dev to leverage the change.
>> >>
>> >>
>> >> 7) Plan iteration 2 to see if there are other opportunities to reduce
>> our
>> >>
>> >>
>> >> use of Ansible.
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> One note: if we decide to go this direction, it'd be helpful if,
>> during
>> >> the
>> >>
>> >>
>> >> transition, we stopped adding additional Ansible deployment code.
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> Thoughts?
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> Thanks,
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> -David...
>> >>
>> >>
>> >>
>>
>>
>
