----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74022/ -----------------------------------------------------------
Review request for ranger, bhavik patel and madhan. Repository: ranger Description ------- When using [kyuubi-spark-authz plugin](https://github.com/apache/incubator-kyuubi/tree/master/extensions/spark/kyuubi-spark-authz) (which extends RangerBasePlugin) to save audit log to ElasticSearch in Spark application, I found that if Spark application was submitted in local or client mode, it hanged forever even after main thread exited. But if I turn off xasecure.audit.destination.elasticsearch, Spark application exits normally. The reason is that ThreadFactory is not set when building RestHighLevelClient and Executors.defaultThreadFactory() is used to create http connection threads, which are non-daemon threads. Therefore, I suggest to set a ThreadFactory which creates daemon threads when building RestHighLevelClient. Diffs ----- agents-audit/src/main/java/org/apache/ranger/audit/destination/ElasticSearchAuditDestination.java 343f3c7cf8 Diff: https://reviews.apache.org/r/74022/diff/1/ Testing ------- Manual tested following the steps in https://issues.apache.org/jira/browse/RANGER-3787 Build kyuubi-spark-authz plugin: ``` git clone g...@github.com:zhouyifan279/incubator-kyuubi.git cd incubator-kyuubi ./build/mvn clean package -pl :kyuubi-spark-authz_2.12 -Pspark-3.1-ranger-3.0 -DskipTests ``` Setup plugin in Spark: ``` Dowload and untar spark distribution: https://www.apache.org/dyn/closer.lua/spark/spark-3.1.3/spark-3.1.3-bin-hadoop3.2.tgz Put `incubator-kyuubi/extensions/spark/kyuubi-spark-authz/target/kyuubi-spark-authz_2.12-1.6.0-SNAPSHOT-1-with-ranger-3.0.jar` under `spark-3.1.3-bin-hadoop3.2/jars` Put `ranger-spark-security.xml` and `ranger-spark-audit.xml` under `spark-3.1.3-bin-hadoop3.2/conf` ``` Run Spark application: ``` ./spark-3.1.3-bin-hadoop3.2/bin/spark-submit --master local --class org.apache.spark.examples.sql.hive.SparkHiveExample --conf spark.sql.extensions=spark.sql.extensions org.apache.kyuubi.plugin.spark.authz.ranger.RangerSparkExtension examples/jars/spark-examples_2.12-3.1.3.jar ``` Expected result: Spark application exits when job finishes. Thanks, zhouyifan279
