That's good feedback, Jon. I think that puts us at:
- Expand ambari to manage the remaining sensor-specific configs
- Refactor the push calls to zookeeper (in ConfigurationUtils, I think)
to push to ambari and take an Ambari user/pw and (optionally) reason
- We shall retain current functionality wrt live configuration changes.
Suggestion- ConfigurationUtils will push to both zookeeper and Ambari in an
atomic operation. (I suspect we can make ambari do this as well)
- Refactor the middleware that Ryan submitted to have the API calls take
an Ambari user/pw and (optionally) reason
- Refactor the management UI to pass in an Ambari user/pw and (optionally)
reason
- Refactor the Stellar Management functions CONFIG_PUT to accept an Ambari
user/pw and (optionally) reason
-D...
On Fri, Jan 13, 2017 at 11:17 AM, Nick Allen <n...@nickallen.org> wrote:
> +1 I strongly agree with Jon's view. Requiring a restart would be a big
> step backwards.
>
> I think the power of the platform is that the user can act on live
> streaming data in a quick, iterative fashion. Adding enrichments, creating
> triage rules, adjusting profiles are all operational activities that can be
> performed at any time in response to active threats.
>
>
>
>
> On Fri, Jan 13, 2017 at 10:59 AM, zeo...@gmail.com <zeo...@gmail.com>
> wrote:
>
> > Right, good conversation to bring up for sure.
> >
> > Just to comment on production generally only being updated during
> > maintenance windows - I can tell you that my plans are to make my dev,
> > test, and prod Metron a very dynamic and frequently changing environment
> > which will have coordinated but frequent modifications and I strongly
> > prefer not having to restart anywhere that I can. Of course it will
> > happen, but keeping it to a minimum is key.
> >
> > Jon
> >
> > On Fri, Jan 13, 2017 at 10:53 AM Nick Allen <n...@nickallen.org> wrote:
> >
> > > Makes sense, Dave. I am totally clear on the proposal. I just wanted
> to
> > > ask the stupid question to bring the conversation full circle, leave no
> > > stone unturned, insert favorite idiom here.
> > >
> > > On Fri, Jan 13, 2017 at 10:46 AM, David Lyle <dlyle65...@gmail.com>
> > wrote:
> > >
> > > > To be clear- NOBODY is suggesting replacing Zookeeper with Ambari.
> > > >
> > > > So, as a bit of a reset- here's what's being proposed:
> > > >
> > > > - Expand ambari to manage the remaining sensor-specific configs
> > > > - Refactor the push calls to zookeeper (in ConfigurationUtils, I
> > think)
> > > > to push to ambari and take an Ambari user/pw and (optionally)
> reason
> > > > - (Ambari can push to zookeeper, but it requires a service restart,
> so
> > > for
> > > > "live changes" you may
> > > > want do both a rest call and zookeeper update from
> > > ConfigurationUtils)
> > > > WAS
> > > > Question remains about whether ambari can do the push to
> zookeeper
> > > > or whetheror whether ConfigurationUtils has to push to zookeeper
> as
> > > > well as update
> > > > ambari.
> > > > - Refactor the middleware that Ryan submitted to have the API calls
> > > take
> > > > an Ambari user/pw and (optionally) reason
> > > > - Refactor the management UI to pass in an Ambari user/pw and
> > > > (optionally) reason
> > > > - Refactor the Stellar Management functions CONFIG_PUT to accept an
> > > > Ambari user/pw and (optionally) reason
> > > >
> > > > -D...
> > > >
> > > >
> > > >
> > > > On Fri, Jan 13, 2017 at 10:42 AM, Ryan Merriman <merrim...@gmail.com
> >
> > > > wrote:
> > > >
> > > > > The driver for using Zookeeper is that it is asynchronous and
> accepts
> > > > > callbacks. Ambari would need to have that capability, otherwise we
> > > have
> > > > to
> > > > > poll which is a deal breaker in my opinion.
> > > > >
> > > > > On Fri, Jan 13, 2017 at 9:28 AM, Casey Stella <ceste...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > No, it was good to bring up, Nick. I might have it wrong re:
> > Ambari.
> > > > > >
> > > > > > Casey
> > > > > >
> > > > > > On Fri, Jan 13, 2017 at 10:27 AM, Nick Allen <n...@nickallen.org
> >
> > > > wrote:
> > > > > >
> > > > > > > That makes sense. I wasn't sure based on Matt's original
> > > > > > > suggestion/description of Ambari, whether that was something
> that
> > > > > Ambari
> > > > > > > had also designed for or not.
> > > > > > >
> > > > > > > On Fri, Jan 13, 2017 at 10:14 AM, Casey Stella <
> > ceste...@gmail.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > Polling the Ambari server via REST (or their API if they have
> > > one),
> > > > > > would
> > > > > > > > entail all workers hitting one server and create a single
> point
> > > of
> > > > > > > failure
> > > > > > > > (the ambari server is what serves up REST). Zookeeper's
> intent
> > > is
> > > > to
> > > > > > not
> > > > > > > > have a single point of failure like this and (one of its
> main)
> > > > > > use-cases
> > > > > > > is
> > > > > > > > to serve up configs in a distributed environment.
> > > > > > > >
> > > > > > > > Casey
> > > > > > > >
> > > > > > > > On Fri, Jan 13, 2017 at 9:55 AM, Nick Allen <
> > n...@nickallen.org>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Let me ask a stupid question. What does Zookeeper do for
> us
> > > that
> > > > > > > Ambari
> > > > > > > > > cannot? Why keep Zookeeper in the mix?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, Jan 13, 2017 at 9:28 AM, David Lyle <
> > > > dlyle65...@gmail.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > In the main yes- I've made some changes:
> > > > > > > > > >
> > > > > > > > > > - Expand ambari to manage the remaining sensor-specific
> > > > configs
> > > > > > > > > > - Refactor the push calls to zookeeper (in
> > > > ConfigurationUtils, I
> > > > > > > > think)
> > > > > > > > > > to push to ambari and take an Ambari user/pw and
> > > > (optionally)
> > > > > > > reason
> > > > > > > > > > - (Ambari can push to zookeeper, but it requires a
> service
> > > > > > restart,
> > > > > > > so
> > > > > > > > > for
> > > > > > > > > > "live changes" you may
> > > > > > > > > > want do both a rest call and zookeeper update from
> > > > > > > > > ConfigurationUtils)
> > > > > > > > > > WAS
> > > > > > > > > > Question remains about whether ambari can do the push
> > to
> > > > > > > zookeeper
> > > > > > > > > > or whetheror whether ConfigurationUtils has to push
> to
> > > > > > zookeeper
> > > > > > > as
> > > > > > > > > > well as update
> > > > > > > > > > ambari.
> > > > > > > > > > - Refactor the middleware that Ryan submitted to have
> the
> > > API
> > > > > > calls
> > > > > > > > > take
> > > > > > > > > > an Ambari user/pw and (optionally) reason
> > > > > > > > > > - Refactor the management UI to pass in an Ambari
> user/pw
> > > and
> > > > > > > > > > (optionally) reason
> > > > > > > > > > - Refactor the Stellar Management functions CONFIG_PUT
> to
> > > > > accept
> > > > > > an
> > > > > > > > > > Ambari user/pw and (optionally) reason
> > > > > > > > > >
> > > > > > > > > > I think we'd need to do some detailed design around how
> to
> > > > handle
> > > > > > > what
> > > > > > > > we
> > > > > > > > > > expect to be dynamic configs, but the main principle
> should
> > > > (imo)
> > > > > > be
> > > > > > > to
> > > > > > > > > > always know who and why and make sure that Ambari is
> aware
> > > and
> > > > is
> > > > > > the
> > > > > > > > > > static backing store for Zookeeper.
> > > > > > > > > >
> > > > > > > > > > -D...
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Fri, Jan 13, 2017 at 9:19 AM, Casey Stella <
> > > > > ceste...@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > So, basically, your proposed changes, broken into
> > tangible
> > > > > > gobbets
> > > > > > > of
> > > > > > > > > > work:
> > > > > > > > > > >
> > > > > > > > > > > - Expand ambari to manage the remaining
> > sensor-specific
> > > > > > configs
> > > > > > > > > > > - Refactor the push calls to zookeeper (in
> > > > > > ConfigurationUtils, I
> > > > > > > > > > think)
> > > > > > > > > > > to push to ambari and take a reason
> > > > > > > > > > > - Question remains about whether ambari can do
> the
> > > push
> > > > > to
> > > > > > > > > > zookeeper
> > > > > > > > > > > or whether ConfigurationUtils has to push to
> > > zookeeper
> > > > as
> > > > > > > well
> > > > > > > > as
> > > > > > > > > > > update
> > > > > > > > > > > ambari.
> > > > > > > > > > > - Refactor the middleware that Ryan submitted to
> have
> > > the
> > > > > API
> > > > > > > > calls
> > > > > > > > > > take
> > > > > > > > > > > a reason
> > > > > > > > > > > - Refactor the management UI to pass in a reason
> > > > > > > > > > > - Refactor the Stellar Management functions
> CONFIG_PUT
> > > to
> > > > > > > accept a
> > > > > > > > > > > reason
> > > > > > > > > > >
> > > > > > > > > > > Just so we can evaluate it and I can ensure I haven't
> > > > > overlooked
> > > > > > > some
> > > > > > > > > > > important point. Please tell me if Ambari cannot do
> the
> > > > things
> > > > > > > we're
> > > > > > > > > > > suggesting it can do.
> > > > > > > > > > >
> > > > > > > > > > > Casey
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Jan 13, 2017 at 9:15 AM, David Lyle <
> > > > > > dlyle65...@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > That's exactly correct, Casey. Basically, an
> expansion
> > of
> > > > > what
> > > > > > > > we're
> > > > > > > > > > > > currently doing with global.json,
> enrichment.properties
> > > and
> > > > > > > > > > > > elasticsearch.properties.
> > > > > > > > > > > >
> > > > > > > > > > > > -D...
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Fri, Jan 13, 2017 at 9:12 AM, Casey Stella <
> > > > > > > ceste...@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > I would suggest not having Ambari replace
> > zookeeper. I
> > > > > think
> > > > > > > the
> > > > > > > > > > > > proposal
> > > > > > > > > > > > > is to have Ambari replace the editable store (like
> > the
> > > > JSON
> > > > > > > files
> > > > > > > > > on
> > > > > > > > > > > > > disk). Zookeeper woudl be the source of truth for
> > the
> > > > > > running
> > > > > > > > > > > topologies
> > > > > > > > > > > > > and ambari would be sync'd to it.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Correct if I misspeak, dave or matt.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Casey
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Fri, Jan 13, 2017 at 9:09 AM, Nick Allen <
> > > > > > > n...@nickallen.org>
> > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Ambari seems like a logical choice.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > *>> It doesn’t natively integrate Zookeeper
> storage
> > > of
> > > > > > > configs,
> > > > > > > > > but
> > > > > > > > > > > > there
> > > > > > > > > > > > > > is a natural place to specify copy to/from
> > Zookeeper
> > > > for
> > > > > > the
> > > > > > > > > files
> > > > > > > > > > > > > > desired.*
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > How would Ambari interact with Zookeeper in this
> > > > > scenario?
> > > > > > > > Would
> > > > > > > > > > > > Ambari
> > > > > > > > > > > > > > replace Zookeeper completely? Or would Zookeeper
> > act
> > > as
> > > > > the
> > > > > > > > > > > persistence
> > > > > > > > > > > > > > tier under Ambari?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Thu, Jan 12, 2017 at 9:24 PM, Matt Foley <
> > > > > > > ma...@apache.org>
> > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Mike, could you try again on the image, please,
> > > > making
> > > > > > sure
> > > > > > > > it
> > > > > > > > > > is a
> > > > > > > > > > > > > > simple
> > > > > > > > > > > > > > > format (gif, png, or jpeg)? It got munched, at
> > > least
> > > > > in
> > > > > > my
> > > > > > > > > > viewer.
> > > > > > > > > > > > > > Thanks.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Casey, responding to some of the questions you
> > > > raised:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I’m going to make a rather strong statement:
> We
> > > > > already
> > > > > > > > have a
> > > > > > > > > > > > service
> > > > > > > > > > > > > > > “to intermediate and handle config
> > > update/retrieval”.
> > > > > > > > > > > > > > > Furthermore, it:
> > > > > > > > > > > > > > > - Correctly handles the problems of distributed
> > > > > services
> > > > > > > > > running
> > > > > > > > > > on
> > > > > > > > > > > > > > > multi-node clusters. (That’s a HARD problem,
> > > people,
> > > > > and
> > > > > > > we
> > > > > > > > > > > > shouldn’t
> > > > > > > > > > > > > > try
> > > > > > > > > > > > > > > to reinvent the wheel.)
> > > > > > > > > > > > > > > - Correctly handles Kerberos security. (That’s
> > > kinda
> > > > > hard
> > > > > > > > too,
> > > > > > > > > or
> > > > > > > > > > > at
> > > > > > > > > > > > > > least
> > > > > > > > > > > > > > > a lot of work.)
> > > > > > > > > > > > > > > - It does automatic versioning of
> configurations,
> > > and
> > > > > > > allows
> > > > > > > > > > > viewing,
> > > > > > > > > > > > > > > comparing, and reverting historical configs
> > > > > > > > > > > > > > > - It has a capable REST API for all those
> things.
> > > > > > > > > > > > > > > It doesn’t natively integrate Zookeeper storage
> > of
> > > > > > configs,
> > > > > > > > but
> > > > > > > > > > > there
> > > > > > > > > > > > > is
> > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > natural place to specify copy to/from Zookeeper
> > for
> > > > the
> > > > > > > files
> > > > > > > > > > > > desired.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > It is Ambari. And we should commit to it,
> rather
> > > > than
> > > > > > try
> > > > > > > to
> > > > > > > > > > > > re-create
> > > > > > > > > > > > > > > such features.
> > > > > > > > > > > > > > > Because it has a good REST API, it is perfectly
> > > > > feasible
> > > > > > to
> > > > > > > > > > > implement
> > > > > > > > > > > > > > > Stellar functions that call it.
> > > > > > > > > > > > > > > GUI configuration tools can also use the Ambari
> > > APIs,
> > > > > or
> > > > > > > > better
> > > > > > > > > > yet
> > > > > > > > > > > > be
> > > > > > > > > > > > > > > integrated in an “Ambari View”. (Eg, see the
> > “Yarn
> > > > > > Capacity
> > > > > > > > > > > Scheduler
> > > > > > > > > > > > > > > Configuration Tool” example in the Ambari
> > > > > documentation,
> > > > > > > > under
> > > > > > > > > > > “Using
> > > > > > > > > > > > > > > Ambari Views”.)
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Arguments are: Parsimony, Sufficiency, Not
> > > > reinventing
> > > > > > the
> > > > > > > > > wheel,
> > > > > > > > > > > and
> > > > > > > > > > > > > Not
> > > > > > > > > > > > > > > spending weeks and weeks of developer time over
> > the
> > > > > next
> > > > > > > year
> > > > > > > > > > > > > reinventing
> > > > > > > > > > > > > > > the wheel while getting details wrong multiple
> > > times…
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Okay, off soapbox.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Casey asked what the config update behavior of
> > > Ambari
> > > > > is,
> > > > > > > and
> > > > > > > > > how
> > > > > > > > > > > it
> > > > > > > > > > > > > will
> > > > > > > > > > > > > > > interact with changes made from outside Ambari.
> > > > > > > > > > > > > > > The following is from my experience working
> with
> > > the
> > > > > > Ambari
> > > > > > > > > Mpack
> > > > > > > > > > > for
> > > > > > > > > > > > > > > Metron. I am not otherwise an Ambari expert,
> so
> > > > > tomorrow
> > > > > > > > I’ll
> > > > > > > > > > get
> > > > > > > > > > > it
> > > > > > > > > > > > > > > reviewed by an Ambari development engineer.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Ambari-server runs on one node, and
> Ambari-agent
> > > runs
> > > > > on
> > > > > > > each
> > > > > > > > > of
> > > > > > > > > > > all
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > nodes.
> > > > > > > > > > > > > > > Ambari-server has a private set of py, xml, and
> > > > > template
> > > > > > > > files,
> > > > > > > > > > > which
> > > > > > > > > > > > > > > together are used both to generate the Ambari
> > > > > > configuration
> > > > > > > > > GUI,
> > > > > > > > > > > with
> > > > > > > > > > > > > > > defaults, and to generate configuration files
> (of
> > > any
> > > > > > > needed
> > > > > > > > > > > > filetype)
> > > > > > > > > > > > > > for
> > > > > > > > > > > > > > > the various Stack components.
> > > > > > > > > > > > > > > Ambari-server also has a database where it
> stores
> > > the
> > > > > > > schema
> > > > > > > > > > > related
> > > > > > > > > > > > to
> > > > > > > > > > > > > > > these files, so even if you reach in and edit
> > > > Ambari’s
> > > > > > > files,
> > > > > > > > > it
> > > > > > > > > > > will
> > > > > > > > > > > > > > Error
> > > > > > > > > > > > > > > out if the set of parameters or parameter names
> > > > > changes.
> > > > > > > The
> > > > > > > > > > > > > historical
> > > > > > > > > > > > > > > information about configuration changes is also
> > > > stored
> > > > > in
> > > > > > > the
> > > > > > > > > db.
> > > > > > > > > > > > > > > For each component (and in the case of Metron,
> > for
> > > > each
> > > > > > > > > > topology),
> > > > > > > > > > > > > there
> > > > > > > > > > > > > > > is a python file which controls the logic for
> > these
> > > > > > > actions,
> > > > > > > > > > among
> > > > > > > > > > > > > > others:
> > > > > > > > > > > > > > > - Install
> > > > > > > > > > > > > > > - Start / stop / restart / status
> > > > > > > > > > > > > > > - Configure
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > It is actually up to this python code (which we
> > > wrote
> > > > > for
> > > > > > > the
> > > > > > > > > > > Metron
> > > > > > > > > > > > > > > Mpack) what happens in each of these API calls.
> > > But
> > > > > the
> > > > > > > > > current
> > > > > > > > > > > > code,
> > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > I believe this is typical of Ambari-managed
> > > > components,
> > > > > > > > > performs
> > > > > > > > > > a
> > > > > > > > > > > > > > > “Configure” action whenever you press the
> “Save”
> > > > button
> > > > > > > after
> > > > > > > > > > > > changing
> > > > > > > > > > > > > a
> > > > > > > > > > > > > > > component config in Ambari, and also on each
> > > Install
> > > > > and
> > > > > > > > Start
> > > > > > > > > or
> > > > > > > > > > > > > > Restart.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > The Configure action consists of approximately
> > the
> > > > > > > following
> > > > > > > > > > > sequence
> > > > > > > > > > > > > > (see
> > > > > > > > > > > > > > > disclaimer above :-)
> > > > > > > > > > > > > > > - Recreate the generated config files, using
> the
> > > > > template
> > > > > > > > files
> > > > > > > > > > and
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > actual configuration most recently set in
> Ambari
> > > > > > > > > > > > > > > o Note this is also under the control of python
> > > code
> > > > > that
> > > > > > > we
> > > > > > > > > > wrote,
> > > > > > > > > > > > and
> > > > > > > > > > > > > > > this is the appropriate place to push to ZK if
> > > > desired.
> > > > > > > > > > > > > > > - Propagate those config files to each
> > > Ambari-agent,
> > > > > > with a
> > > > > > > > > > command
> > > > > > > > > > > > to
> > > > > > > > > > > > > > set
> > > > > > > > > > > > > > > them locally
> > > > > > > > > > > > > > > - The ambari-agents on each node receive the
> > files
> > > > and
> > > > > > > write
> > > > > > > > > them
> > > > > > > > > > > to
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > specified locations on local storage
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Ambari-server then whines that the updated
> > services
> > > > > > should
> > > > > > > be
> > > > > > > > > > > > > restarted,
> > > > > > > > > > > > > > > but does not initiate that action itself
> (unless
> > of
> > > > > > course
> > > > > > > > the
> > > > > > > > > > > > > initiating
> > > > > > > > > > > > > > > action was a Start command from the
> > administrator).
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Make sense? It’s all quite straightforward in
> > > > concept,
> > > > > > > > there’s
> > > > > > > > > > > just
> > > > > > > > > > > > an
> > > > > > > > > > > > > > > awful lot of stuff wrapped around that to make
> it
> > > all
> > > > > go
> > > > > > > > > smoothly
> > > > > > > > > > > and
> > > > > > > > > > > > > > > handle the problems when it doesn’t.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > There’s additional complexity in that the
> > > > Ambari-agent
> > > > > > also
> > > > > > > > > > caches
> > > > > > > > > > > > (on
> > > > > > > > > > > > > > > each node) both the template files and COMPILED
> > > forms
> > > > > of
> > > > > > > the
> > > > > > > > > > python
> > > > > > > > > > > > > files
> > > > > > > > > > > > > > > (.pyc) involved in transforming them. The pyc
> > > files
> > > > > > > > > incorporate
> > > > > > > > > > > some
> > > > > > > > > > > > > > > amount of additional info regarding parameter
> > > values,
> > > > > but
> > > > > > > I’m
> > > > > > > > > not
> > > > > > > > > > > > sure
> > > > > > > > > > > > > of
> > > > > > > > > > > > > > > the form. I don’t think that changes the above
> > in
> > > > any
> > > > > > > > > practical
> > > > > > > > > > > way
> > > > > > > > > > > > > > unless
> > > > > > > > > > > > > > > you’re trying to cheat Ambari by reaching in
> and
> > > > > editing
> > > > > > > its
> > > > > > > > > > files
> > > > > > > > > > > > > > > directly. In that case, you also need to whack
> > the
> > > > pyc
> > > > > > > files
> > > > > > > > > (on
> > > > > > > > > > > > each
> > > > > > > > > > > > > > > node) to force the data to be reloaded from
> > > > > > Ambari-server.
> > > > > > > > > Best
> > > > > > > > > > > > > solution
> > > > > > > > > > > > > > > is don’t cheat.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Also, there may be circumstances under which
> the
> > > > > > > Ambari-agent
> > > > > > > > > > will
> > > > > > > > > > > > > detect
> > > > > > > > > > > > > > > changes and re-write the latest version it
> knows
> > of
> > > > the
> > > > > > > > config
> > > > > > > > > > > files,
> > > > > > > > > > > > > > even
> > > > > > > > > > > > > > > without a Save or Start action at the
> > > Ambari-server.
> > > > > I’m
> > > > > > > not
> > > > > > > > > > sure
> > > > > > > > > > > of
> > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > and need to check with Ambari developers. It
> may
> > > no
> > > > > > longer
> > > > > > > > > > happen,
> > > > > > > > > > > > > altho
> > > > > > > > > > > > > > > I’m pretty sure change detection/reversion was
> a
> > > > > feature
> > > > > > of
> > > > > > > > > early
> > > > > > > > > > > > > > versions
> > > > > > > > > > > > > > > of Ambari.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Hope this helps,
> > > > > > > > > > > > > > > --Matt
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > ==============================
> ==================
> > > > > > > > > > > > > > > From: Michael Miklavcic <
> > > michael.miklav...@gmail.com
> > > > >
> > > > > > > > > > > > > > > Reply-To: "dev@metron.incubator.apache.org"
> > > > > > > > > > > > > > <dev@metron.incubator.apache.
> > > > > > > > > > > > > > > org>
> > > > > > > > > > > > > > > Date: Thursday, January 12, 2017 at 3:59 PM
> > > > > > > > > > > > > > > To: "dev@metron.incubator.apache.org"
> > > > > > > > > > > <dev@metron.incubator.apache.
> > > > > > > > > > > > org
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Subject: Re: [DISCUSS] Ambari Metron
> > Configuration
> > > > > > > Management
> > > > > > > > > > > > > > consequences
> > > > > > > > > > > > > > > and call to action
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Hi Casey,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thanks for starting this thread. I believe you
> > are
> > > > > > correct
> > > > > > > in
> > > > > > > > > > your
> > > > > > > > > > > > > > > assessment of the 4 options for updating
> configs
> > in
> > > > > > Metron.
> > > > > > > > > When
> > > > > > > > > > > > using
> > > > > > > > > > > > > > more
> > > > > > > > > > > > > > > than one of these options we can get into a
> > > > split-brain
> > > > > > > > > > scenario. A
> > > > > > > > > > > > > basic
> > > > > > > > > > > > > > > example is updating the global config on disk
> and
> > > > using
> > > > > > the
> > > > > > > > > > > > > > > zk_load_configs.sh. Later, if a user decides to
> > > > restart
> > > > > > > > Ambari,
> > > > > > > > > > the
> > > > > > > > > > > > > > cached
> > > > > > > > > > > > > > > version stored by Ambari (it's in the MySQL or
> > > other
> > > > > > > database
> > > > > > > > > > > backing
> > > > > > > > > > > > > > > Ambari) will be written out to disk in the
> > defined
> > > > > config
> > > > > > > > > > > directory,
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > > subsequently loaded using the
> zk_load_configs.sh
> > > > under
> > > > > > the
> > > > > > > > > hood.
> > > > > > > > > > > Any
> > > > > > > > > > > > > > global
> > > > > > > > > > > > > > > configuration modified outside of Ambari will
> be
> > > lost
> > > > > at
> > > > > > > this
> > > > > > > > > > > point.
> > > > > > > > > > > > > This
> > > > > > > > > > > > > > > is obviously undesirable, but I also like the
> > > purpose
> > > > > and
> > > > > > > > > utility
> > > > > > > > > > > > > exposed
> > > > > > > > > > > > > > > by the multiple config management interfaces we
> > > > > currently
> > > > > > > > have
> > > > > > > > > > > > > > available. I
> > > > > > > > > > > > > > > also agree that a service would be best.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > For reference, here's my understanding of the
> > > current
> > > > > > > > > > configuration
> > > > > > > > > > > > > > > loading mechanisms and their deps.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > <image>
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Mike
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Thu, Jan 12, 2017 at 3:08 PM, Casey Stella <
> > > > > > > > > > ceste...@gmail.com>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > In the course of discussion on the PR for
> > > METRON-652
> > > > > > > > > > > > > > > <
> > > https://github.com/apache/incubator-metron/pull/415
> > > > >
> > > > > > > > > something
> > > > > > > > > > > > that I
> > > > > > > > > > > > > > > should definitely have understood better came
> to
> > > > light
> > > > > > and
> > > > > > > I
> > > > > > > > > > > thought
> > > > > > > > > > > > > that
> > > > > > > > > > > > > > > it was worth bringing to the attention of the
> > > > community
> > > > > > to
> > > > > > > > get
> > > > > > > > > > > > > > > clarification/discuss is just how we manage
> > > configs.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Currently (assuming the management UI that Ryan
> > > > > Merriman
> > > > > > > > > > submitted)
> > > > > > > > > > > > > > configs
> > > > > > > > > > > > > > > are managed/adjusted via a couple of different
> > > > > mechanism.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > - zk_load_utils.sh: pushed and pulled from
> > disk
> > > to
> > > > > > > > zookeeper
> > > > > > > > > > > > > > > - Stellar REPL: pushed and pulled via the
> > > > > > > > > > CONFIG_GET/CONFIG_PUT
> > > > > > > > > > > > > > > functions
> > > > > > > > > > > > > > > - Ambari: initialized via the zk_load_utils
> > > script
> > > > > and
> > > > > > > > then
> > > > > > > > > > some
> > > > > > > > > > > > of
> > > > > > > > > > > > > > them
> > > > > > > > > > > > > > > are managed directly (global config) and
> some
> > > > > > indirectly
> > > > > > > > > > > > > > > (sensor-specific
> > > > > > > > > > > > > > > configs).
> > > > > > > > > > > > > > > - NOTE: Upon service restart, it may or
> may
> > > not
> > > > > > > > overwrite
> > > > > > > > > > > > changes
> > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > disk or on zookeeper. *Can someone more
> > > > > > > knowledgeable
> > > > > > > > > than
> > > > > > > > > > > me
> > > > > > > > > > > > > > about
> > > > > > > > > > > > > > > this describe precisely the semantics
> that
> > we
> > > > can
> > > > > > > > expect
> > > > > > > > > on
> > > > > > > > > > > > > > > service restart
> > > > > > > > > > > > > > > for Ambari? What gets overwritten on disk
> > and
> > > > > what
> > > > > > > gets
> > > > > > > > > > > updated
> > > > > > > > > > > > > > > in ambari?*
> > > > > > > > > > > > > > > - The Management UI: manages some of the
> > > configs.
> > > > > > *RYAN:
> > > > > > > > > Which
> > > > > > > > > > > > > configs
> > > > > > > > > > > > > > > do we support here and which don't we
> support
> > > > here?*
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > As you can see, we have a mishmash of
> mechanisms
> > to
> > > > > > update
> > > > > > > > and
> > > > > > > > > > > manage
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > configuration for Metron in zookeeper. In the
> > > > > beginning
> > > > > > > the
> > > > > > > > > > > approach
> > > > > > > > > > > > > was
> > > > > > > > > > > > > > > just to edit configs on disk and push/pull them
> > via
> > > > > > > > > > zk_load_utils.
> > > > > > > > > > > > > > Configs
> > > > > > > > > > > > > > > could be historically managed using source
> > control,
> > > > > etc.
> > > > > > > As
> > > > > > > > we
> > > > > > > > > > got
> > > > > > > > > > > > > more
> > > > > > > > > > > > > > > and more components managing the configs, we
> > > haven't
> > > > > > taken
> > > > > > > > care
> > > > > > > > > > > that
> > > > > > > > > > > > > they
> > > > > > > > > > > > > > > they all work with each other in an expected
> way
> > (I
> > > > > > believe
> > > > > > > > > these
> > > > > > > > > > > are
> > > > > > > > > > > > > > > true..correct me if I'm wrong):
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > - If configs are modified in the management
> UI
> > > or
> > > > > the
> > > > > > > > > Stellar
> > > > > > > > > > > REPL
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > > someone forgets to pull the configs from
> > > zookeeper
> > > > > to
> > > > > > > > disk,
> > > > > > > > > > > before
> > > > > > > > > > > > > > they
> > > > > > > > > > > > > > > do
> > > > > > > > > > > > > > > a push via zk_load_utils, they will clobber
> > the
> > > > > > configs
> > > > > > > in
> > > > > > > > > > > > zookeeper
> > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > old configs.
> > > > > > > > > > > > > > > - If the global config is changed on disk
> and
> > > the
> > > > > > ambari
> > > > > > > > > > service
> > > > > > > > > > > > > > > restarts, it'll get reset with the original
> > > global
> > > > > > > config.
> > > > > > > > > > > > > > > - *Ryan, in the management UI, if someone
> > > changes
> > > > > the
> > > > > > > > > > zookeeper
> > > > > > > > > > > > > > configs
> > > > > > > > > > > > > > > from outside, are those configs reflected
> > > > > immediately
> > > > > > in
> > > > > > > > the
> > > > > > > > > > > UI?*
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > It seems to me that we have a couple of options
> > > here:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > - A service to intermediate and handle
> config
> > > > > > > > > update/retrieval
> > > > > > > > > > > and
> > > > > > > > > > > > > > > tracking historical changes so these
> different
> > > > > > > mechanisms
> > > > > > > > > can
> > > > > > > > > > > use
> > > > > > > > > > > > a
> > > > > > > > > > > > > > > common
> > > > > > > > > > > > > > > component for config management/tracking and
> > > > > refactor
> > > > > > > the
> > > > > > > > > > > existing
> > > > > > > > > > > > > > > mechanisms to use that service
> > > > > > > > > > > > > > > - Standardize on exactly one component to
> > manage
> > > > the
> > > > > > > > configs
> > > > > > > > > > and
> > > > > > > > > > > > > > regress
> > > > > > > > > > > > > > > the others (that's a verb, right? nicer
> than
> > > > > > delete.)
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I happen to like the service approach, myself,
> > but
> > > I
> > > > > > wanted
> > > > > > > > to
> > > > > > > > > > put
> > > > > > > > > > > it
> > > > > > > > > > > > > up
> > > > > > > > > > > > > > > for discussion and hopefully someone will
> > volunteer
> > > > to
> > > > > > > design
> > > > > > > > > > such
> > > > > > > > > > > a
> > > > > > > > > > > > > > thing.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > To frame the debate, I want us to keep in mind
> a
> > > > couple
> > > > > > of
> > > > > > > > > things
> > > > > > > > > > > > that
> > > > > > > > > > > > > > may
> > > > > > > > > > > > > > > or may not be relevant to the discussion:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > - We will eventually be moving to support
> > > kerberos
> > > > > so
> > > > > > > > there
> > > > > > > > > > > should
> > > > > > > > > > > > > at
> > > > > > > > > > > > > > > least be a path to use kerberos for any
> > solution
> > > > IMO
> > > > > > > > > > > > > > > - There is value in each of the different
> > > > mechanisms
> > > > > > in
> > > > > > > > > place
> > > > > > > > > > > now.
> > > > > > > > > > > > > If
> > > > > > > > > > > > > > > there weren't, then they wouldn't have been
> > > > created.
> > > > > > > > Before
> > > > > > > > > > we
> > > > > > > > > > > > try
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > > make
> > > > > > > > > > > > > > > this a "there can be only one" argument, I'd
> > > like
> > > > to
> > > > > > > hear
> > > > > > > > > very
> > > > > > > > > > > > good
> > > > > > > > > > > > > > > arguments.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Finally, I'd appreciate if some people might
> > answer
> > > > the
> > > > > > > > > > questions I
> > > > > > > > > > > > > have
> > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > bold there. Hopefully this discussion, if
> > nothing
> > > > else
> > > > > > > > > happens,
> > > > > > > > > > > will
> > > > > > > > > > > > > > > result in fodder for proper documentation of
> the
> > > ins
> > > > > and
> > > > > > > outs
> > > > > > > > > of
> > > > > > > > > > > each
> > > > > > > > > > > > > of
> > > > > > > > > > > > > > > the components bulleted above.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Best,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Casey
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > > Nick Allen <n...@nickallen.org>
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Nick Allen <n...@nickallen.org>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Nick Allen <n...@nickallen.org>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Nick Allen <n...@nickallen.org>
> > >
> > --
> >
> > Jon
> >
> > Sent from my mobile device
> >
>
>
>
> --
> Nick Allen <n...@nickallen.org>
>
