Hi everyone, Here is what we have: We have deployed the hive metastore on k8s. We use Spark and Presto for DDL and DQL. Both also run on k8s. As the backend store we use Minio. For rule based authorization we use Ranger. We have connected everything successfully.
Here is what we need: Spark does today only offer client side authorization. In a container based environment, we cannot rely on it. So we would like to use Hive Metastore integrated storage based authorization. We like to implement a new provider that queries S3 API for access. Here is our challenge: Right now Storage based authorization is based on a user field. We do not have enabled the concept of a user in our spark applications. However we have S3 credentials (access and secret key) to help us do authorization. Here is what we need guidance with: How can we get arbitrary variables from the client into the metastore (access and secret key), so that we then can use them in the new authorization provider? I realize that this also is a question I need to ask to the hive developers. I will also do that. Regards This email may contain information which is privileged or protected against unauthorized disclosure or communication. If you are not the intended recipient, please notify the sender and delete this message and any attachments from your system without producing, distributing or retaining copies thereof or disclosing its contents to any other person. Telia Company processes emails and other files that may contain personal data in accordance with Telia Company's Privacy Policy<https://www.teliacompany.com/en/about-the-company/privacy/>.
