Ranger community advises against sync between Ranger policies and native authorization metastores. It complicates design and users haven't shown a real need to switch between auth models back and forth.
-Vineet On Thu, Mar 2, 2017 at 6:33 PM Alexander Denissov <adenis...@pivotal.io> wrote: > Not quite. > > For the first phase, there will be no integration between GRANT/REVOKE and > Ranger policies -- once Ranger is turned on, GRANT/REVOKE from psql will no > longer work. > > In a later stage, we will likely provide integration HAWQ --> Ranger, such > that when GRANT/REVOKE is issued from psql, an appropriate policy is > created / deleted from Ranger. > > I don't think Ranger --> HAWQ integration is planned or is even possible, > as admin action in Ranger UI would need to be somehow detected and > translated to HAWQ grants which are not going to be used anyways as > authorization will be performed in Ranger only. > > -- > Thanks, > Alex > > On Thu, Mar 2, 2017 at 6:20 PM, Jon Roberts <jrobe...@pivotal.io> wrote: > > > Have I misunderstood the Ranger integration work? When a GRANT/REVOKE is > > executed in HAWQ it will be replicated to Ranger and when a GRANT/REVOKE > is > > executed in Ranger, it will be replicated to HAWQ. Right? If yes, then > > this is the model that I'm suggesting for Ambari. > > > > Jon Roberts > > Principal Engineer | jrobe...@pivotal.io | 615-426-8661 > <(615)%20426-8661> > > > > On Thu, Mar 2, 2017 at 7:27 PM, Vineet Goel <vvin...@apache.org> wrote: > > > > > Having worked with Ambari and knowing the complexities of this desired > > > integration, I agree with Alex on his comments below. Users need some > > time > > > to get used to exclusive HAWQ CLI or Ambari usage model, and avoid mix > > and > > > match despite the fact that HAWQ CLI and Ambari have their > > > advantages/disadvantages. > > > > > > -Vineet > > > > > > > > > On Thu, Mar 2, 2017 at 5:17 PM Alexander Denissov < > adenis...@pivotal.io> > > > wrote: > > > > > > > Ambari is designed as a tool to sit on top of individual service CLI > > > tools > > > > and provide consistent user experience. Ambari can also have wizards > > and > > > > custom actions that go beyond service lifecycle management. It is > > assumed > > > > by Ambari design that once Ambari is used to manage configurations, > CLI > > > > tools should not be used, otherwise changes will be out-of-sync or > will > > > be > > > > overwritten by Ambari upon service restart. > > > > > > > > Having HAWQ CLI tools communicate back to Ambari is not desirable > > > because: > > > > - knowledge of Ambari REST APIs needs to be built into the tools > > > > - Ambari APIs change outside of service release lifecycle > > > > - complexity in error handling (what do you do if Ambari call failed > ?) > > > > - introduction of feedback loop and potential infinite cycle (Ambari > > > calls > > > > the tool, the tool calls Ambari back, etc) > > > > > > > > If customers want to do some extra automation they can decide how to > > tie > > > > tools and Ambari, if they like, but I will say we should not have > HAWQ > > > CLI > > > > tools call Ambari. > > > > > > > > -- > > > > Thanks, > > > > Alex Denissov > > > > > > > > On Thu, Mar 2, 2017 at 4:44 PM, Alex (Oleksandr) Diachenko < > > > > odiache...@pivotal.io> wrote: > > > > > > > > > Sure, > > > > > > > > > > Users can automate configuration changes via Ambari API, which > > provides > > > > > unified access > > > > > to all services in a cluster. It's very likely they need to > configure > > > > HAWQ, > > > > > HDFS and YARN > > > > > when working with configuration changes, so with Ambari API it > would > > be > > > > > easier to call one API > > > > > rather than using all different CLI's. > > > > > > > > > > Regards, Alex. > > > > > > > > > > On Thu, Mar 2, 2017 at 4:39 PM, Lei Chang <lei_ch...@apache.org> > > > wrote: > > > > > > > > > > > I think there are some use cases we need What Jon proposed. > > > > > > > > > > > > For example, users installed hawq via Ambari, but they want to > > > automate > > > > > the > > > > > > configuration changes, it would be convenient to have hawq CLI > > change > > > > the > > > > > > configurations. > > > > > > > > > > > > Cheers > > > > > > Lei > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Mar 3, 2017 at 8:36 AM, Alex (Oleksandr) Diachenko < > > > > > > odiache...@pivotal.io> wrote: > > > > > > > > > > > > > I see, that makes sense. > > > > > > > But is there any action users cannot do via Ambari? > > > > > > > > > > > > > > Ranger is also a good example, there we are making assumption, > > > > > > > users either use Ranger or HAWQ's authorization engine. > > > > > > > > > > > > > > The same logic might be extrapolated to HAWQ/Ambari - users > might > > > use > > > > > > > either Ambari or HAWQ CLI, but not both at the same time. > > > > > > > In that way, we can keep things simple. > > > > > > > > > > > > > > Regards, Alex. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Mar 2, 2017 at 4:28 PM, Jon Roberts < > jrobe...@pivotal.io > > > > > > > > wrote: > > > > > > > > > > > > > > > Right. Just like HAWQ will be operational without Ranger. > > > > > > > > > > > > > > > > We have the hawq CLI and will obviously continue to have it. > > > Some > > > > > > people > > > > > > > > use Ambari while others don't. So just like with Ranger > > support, > > > > > > > integrate > > > > > > > > when possible but don't require it. > > > > > > > > > > > > > > > > > > > > > > > > Jon > > > > > > > > > > > > > > > > On Thu, Mar 2, 2017 at 6:26 PM, Alex (Oleksandr) Diachenko < > > > > > > > > odiache...@pivotal.io> wrote: > > > > > > > > > > > > > > > > > Not really, because HAWQ should be operational even without > > > > > Ambari(if > > > > > > > > > that's the case). > > > > > > > > > > > > > > > > > > On Thu, Mar 2, 2017 at 4:21 PM, Jon Roberts < > > > jrobe...@pivotal.io > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > If that is the case, should we remove the "hawq" CLI? > > > > > > > > > > > > > > > > > > > > Jon > > > > > > > > > > > > > > > > > > > > On Thu, Mar 2, 2017 at 6:12 PM, Alex (Oleksandr) > Diachenko > > < > > > > > > > > > > odiache...@pivotal.io> wrote: > > > > > > > > > > > > > > > > > > > > > Hi Jon, > > > > > > > > > > > > > > > > > > > > > > I think it was designed that Ambari is supposed to be > > only > > > > one > > > > > > > source > > > > > > > > > of > > > > > > > > > > > true. > > > > > > > > > > > The whole purpose of integration id to provide a > > > > user-friendly > > > > > > > > > interface > > > > > > > > > > > and avoid manually editing/distributing config files > > > > > > > > > > > or running CLI commands. > > > > > > > > > > > The idea of coupling HAWQ master with Ambari doesn't > seem > > > to > > > > be > > > > > > > > clean. > > > > > > > > > > > > > > > > > > > > > > Regards, Alex. > > > > > > > > > > > > > > > > > > > > > > On Thu, Mar 2, 2017 at 4:05 PM, Jon Roberts < > > > > > jrobe...@pivotal.io > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > It would be handy if the "hawq config" also updated > > > > Ambari's > > > > > > > > database > > > > > > > > > > so > > > > > > > > > > > > that changes could be made in either place are > retained > > > > when > > > > > > > > changes > > > > > > > > > > are > > > > > > > > > > > > made in either place. > > > > > > > > > > > > > > > > > > > > > > > > Register Ambari: > > > > > > > > > > > > hawq ambari -u admin -w admin -h myhost -p 8080 > > > > > > > > > > > > > > > > > > > > > > > > "hawq config" could then raise INFO/WARN messages > about > > > > > > updating > > > > > > > > > > Ambari. > > > > > > > > > > > > > > > > > > > > > > > > Example: > > > > > > > > > > > > hawq config -c hawq_rm_stmt_vseg_memory -v 16gb > > > > > > > > > > > > INFO: Updated Ambari with > hawq_rm_stmt_vseg_memory=16gb > > > > > > > > > > > > or > > > > > > > > > > > > hawq config -c hawq_rm_stmt_vseg_memory -v 16gb > > > > > > > > > > > > WARN: Failed to update Ambari with > > > > > > hawq_rm_stmt_vseg_memory=16gb. > > > > > > > > > > Please > > > > > > > > > > > > update Ambari credentials manually to retain this > > > > > configuration > > > > > > > > > change > > > > > > > > > > > > after a restart. > > > > > > > > > > > > > > > > > > > > > > > > The implementation would require interacting with the > > > > Ambari > > > > > > APIs > > > > > > > > and > > > > > > > > > > > also > > > > > > > > > > > > storing the credentials in an encrypted file on the > > HAWQ > > > > > > Master. > > > > > > > > > > > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Jon Roberts > > > > > > > > > > > > Principal Engineer | jrobe...@pivotal.io | > > 615-426-8661 <(615)%20426-8661> > > > > <(615)%20426-8661> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
