zhangbutao commented on PR #6507: URL: https://github.com/apache/hive/pull/6507#issuecomment-4709101116
@ayushtkn Great! Thanks your effort for the feature!!! If I remember correctly, I know there is the Hive LLAP Workload Management feature (https://issues.apache.org/jira/browse/HIVE-17481) for resource isolation. However, this resource isolation feature makes it very difficult to achieve absolute resource isolation among multiple tenants within a single LLAP cluster. LLAP cannot simply and effectively limit resource usage for certain users or tasks using queues like Tez on YARN. But with LLAP on Kubernetes, I think LLAP can achieve better resource isolation. Specifically, user1 can start their own LLAP compute group, for example `llap-cluster1` (comprising, say, llap0, llap1, llap2), and user2 can start their own LLAP compute group, `llap-cluster2` (llap3, llap4, llap5). Both LLAP compute groups could be recognized by the same HS2/Tez AM. Users might only need to pass the LLAP compute group name in Beeline, for example: `beeline -u "jdbc:hive2://hive-hiveserver2:10001/;transportMode=http;httpPath=cliservice;llap=llap-cluster1" user1` This way, user1 and user2 can use their own LLAP instances for queries, and their query workloads will not interfere with each other at all. In this Kubernetes-based LLAP deployment model, multi-tenant resource isolation is effectively achieved — simpler and more effective than the Workload Management approach. Haha, this is just a multi-tenant idea I came up with after seeing the elastic deployment capabilities of LLAP on Kubernetes. I think it’s worth exploring further. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
