Have I misunderstood the Ranger integration work? When a GRANT/REVOKE is executed in HAWQ it will be replicated to Ranger and when a GRANT/REVOKE is executed in Ranger, it will be replicated to HAWQ. Right? If yes, then this is the model that I'm suggesting for Ambari.
Jon Roberts Principal Engineer | jrobe...@pivotal.io | 615-426-8661 On Thu, Mar 2, 2017 at 7:27 PM, Vineet Goel <vvin...@apache.org> wrote: > Having worked with Ambari and knowing the complexities of this desired > integration, I agree with Alex on his comments below. Users need some time > to get used to exclusive HAWQ CLI or Ambari usage model, and avoid mix and > match despite the fact that HAWQ CLI and Ambari have their > advantages/disadvantages. > > -Vineet > > > On Thu, Mar 2, 2017 at 5:17 PM Alexander Denissov <adenis...@pivotal.io> > wrote: > > > Ambari is designed as a tool to sit on top of individual service CLI > tools > > and provide consistent user experience. Ambari can also have wizards and > > custom actions that go beyond service lifecycle management. It is assumed > > by Ambari design that once Ambari is used to manage configurations, CLI > > tools should not be used, otherwise changes will be out-of-sync or will > be > > overwritten by Ambari upon service restart. > > > > Having HAWQ CLI tools communicate back to Ambari is not desirable > because: > > - knowledge of Ambari REST APIs needs to be built into the tools > > - Ambari APIs change outside of service release lifecycle > > - complexity in error handling (what do you do if Ambari call failed ?) > > - introduction of feedback loop and potential infinite cycle (Ambari > calls > > the tool, the tool calls Ambari back, etc) > > > > If customers want to do some extra automation they can decide how to tie > > tools and Ambari, if they like, but I will say we should not have HAWQ > CLI > > tools call Ambari. > > > > -- > > Thanks, > > Alex Denissov > > > > On Thu, Mar 2, 2017 at 4:44 PM, Alex (Oleksandr) Diachenko < > > odiache...@pivotal.io> wrote: > > > > > Sure, > > > > > > Users can automate configuration changes via Ambari API, which provides > > > unified access > > > to all services in a cluster. It's very likely they need to configure > > HAWQ, > > > HDFS and YARN > > > when working with configuration changes, so with Ambari API it would be > > > easier to call one API > > > rather than using all different CLI's. > > > > > > Regards, Alex. > > > > > > On Thu, Mar 2, 2017 at 4:39 PM, Lei Chang <lei_ch...@apache.org> > wrote: > > > > > > > I think there are some use cases we need What Jon proposed. > > > > > > > > For example, users installed hawq via Ambari, but they want to > automate > > > the > > > > configuration changes, it would be convenient to have hawq CLI change > > the > > > > configurations. > > > > > > > > Cheers > > > > Lei > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Mar 3, 2017 at 8:36 AM, Alex (Oleksandr) Diachenko < > > > > odiache...@pivotal.io> wrote: > > > > > > > > > I see, that makes sense. > > > > > But is there any action users cannot do via Ambari? > > > > > > > > > > Ranger is also a good example, there we are making assumption, > > > > > users either use Ranger or HAWQ's authorization engine. > > > > > > > > > > The same logic might be extrapolated to HAWQ/Ambari - users might > use > > > > > either Ambari or HAWQ CLI, but not both at the same time. > > > > > In that way, we can keep things simple. > > > > > > > > > > Regards, Alex. > > > > > > > > > > > > > > > > > > > > On Thu, Mar 2, 2017 at 4:28 PM, Jon Roberts <jrobe...@pivotal.io> > > > wrote: > > > > > > > > > > > Right. Just like HAWQ will be operational without Ranger. > > > > > > > > > > > > We have the hawq CLI and will obviously continue to have it. > Some > > > > people > > > > > > use Ambari while others don't. So just like with Ranger support, > > > > > integrate > > > > > > when possible but don't require it. > > > > > > > > > > > > > > > > > > Jon > > > > > > > > > > > > On Thu, Mar 2, 2017 at 6:26 PM, Alex (Oleksandr) Diachenko < > > > > > > odiache...@pivotal.io> wrote: > > > > > > > > > > > > > Not really, because HAWQ should be operational even without > > > Ambari(if > > > > > > > that's the case). > > > > > > > > > > > > > > On Thu, Mar 2, 2017 at 4:21 PM, Jon Roberts < > jrobe...@pivotal.io > > > > > > > > wrote: > > > > > > > > > > > > > > > If that is the case, should we remove the "hawq" CLI? > > > > > > > > > > > > > > > > Jon > > > > > > > > > > > > > > > > On Thu, Mar 2, 2017 at 6:12 PM, Alex (Oleksandr) Diachenko < > > > > > > > > odiache...@pivotal.io> wrote: > > > > > > > > > > > > > > > > > Hi Jon, > > > > > > > > > > > > > > > > > > I think it was designed that Ambari is supposed to be only > > one > > > > > source > > > > > > > of > > > > > > > > > true. > > > > > > > > > The whole purpose of integration id to provide a > > user-friendly > > > > > > > interface > > > > > > > > > and avoid manually editing/distributing config files > > > > > > > > > or running CLI commands. > > > > > > > > > The idea of coupling HAWQ master with Ambari doesn't seem > to > > be > > > > > > clean. > > > > > > > > > > > > > > > > > > Regards, Alex. > > > > > > > > > > > > > > > > > > On Thu, Mar 2, 2017 at 4:05 PM, Jon Roberts < > > > jrobe...@pivotal.io > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > It would be handy if the "hawq config" also updated > > Ambari's > > > > > > database > > > > > > > > so > > > > > > > > > > that changes could be made in either place are retained > > when > > > > > > changes > > > > > > > > are > > > > > > > > > > made in either place. > > > > > > > > > > > > > > > > > > > > Register Ambari: > > > > > > > > > > hawq ambari -u admin -w admin -h myhost -p 8080 > > > > > > > > > > > > > > > > > > > > "hawq config" could then raise INFO/WARN messages about > > > > updating > > > > > > > > Ambari. > > > > > > > > > > > > > > > > > > > > Example: > > > > > > > > > > hawq config -c hawq_rm_stmt_vseg_memory -v 16gb > > > > > > > > > > INFO: Updated Ambari with hawq_rm_stmt_vseg_memory=16gb > > > > > > > > > > or > > > > > > > > > > hawq config -c hawq_rm_stmt_vseg_memory -v 16gb > > > > > > > > > > WARN: Failed to update Ambari with > > > > hawq_rm_stmt_vseg_memory=16gb. > > > > > > > > Please > > > > > > > > > > update Ambari credentials manually to retain this > > > configuration > > > > > > > change > > > > > > > > > > after a restart. > > > > > > > > > > > > > > > > > > > > The implementation would require interacting with the > > Ambari > > > > APIs > > > > > > and > > > > > > > > > also > > > > > > > > > > storing the credentials in an encrypted file on the HAWQ > > > > Master. > > > > > > > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Jon Roberts > > > > > > > > > > Principal Engineer | jrobe...@pivotal.io | 615-426-8661 > > <(615)%20426-8661> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
