+1, thanks for all of the details.
On Fri, Aug 9, 2019 at 3:21 PM Grant Henke <ghe...@cloudera.com> wrote: > > Hello Kudu developers, > > Recently I have started work on upgrading Kudu to use Apache Hive 3.x. > Given this is a major upgrade it does come with some challenges. As of Kudu > 1.10.0 we use Hive in the HMS synchronization feature. This feature > includes a Kudu server side notification listener and HMS client. It also > includes a Java side HMS plugin to enforce Kudu imperatives within the HMS. > That feature is useful on its own in many ways, but is also required for > fine grained authorization via Apache Sentry. > > The primary challenge is that Apache Sentry currently does not support Hive > 3 and it will likely take a large effort to enable support. It is also > unclear if there is anyone in the Sentry community that want's to > contribute and release such support. > > I have started preliminary efforts to support Hive 3 in Kudu and the HMS > synchronization feature. This includes 3 patches. The first patch > <https://gerrit.cloudera.org/#/c/14018/> is changes that work in both Hive > 2 and Hive 3 that minimize the work needed when we upgrade in the future. > This can be committed to master when reviewed and ready. The second patch > <https://gerrit.cloudera.org/#/c/14006> disables the sentry integration so > I can test the changes required to support HMS synchronization on its own. > Those changes and testing are the third patch > <https://gerrit.cloudera.org/#/c/13256/>. > > Given fine grained authorization is a critical feature for many users, we > can't remove Sentry support without providing an alternative authorization > implementation. At the same time we have started work on authorization via > Apache Ranger. Once that implementation exists and has been > contributed/released we can make a decision about how to move forward. > > Given what we know today and the current situation here is my suggested > plan: > > 1. Commit the Hive 3 preparation patch to simplify upgrading in the > future > 2. Verify the feasibility of upgrading with the mentioned POC patches, > but do not commit them. > - This means we will remain on Hive 2 until step 4 or 5 below. > 3. Start work on an Apache Ranger integration for Kudu. > 4. If Hive 3 support is added in Sentry, consider upgrading to Hive 3 > then. > 5. When Ranger support is complete, consider removing Sentry support in > favor of Ranger and upgrade to Hive 3. > - This may require a migration path from Sentry to Ranger. > > Please let me know if you have any thoughts or feedback on the above plan. > > Thank you, > Grant > -- > Grant Henke > Software Engineer | Cloudera > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
