[
https://issues.apache.org/jira/browse/RANGER-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255860#comment-17255860
]
Md Mehrab Alam commented on RANGER-3133:
----------------------------------------
Yes, the Ranger HDFS plugin is working.
Hadoop Version: 3.2.1
PrestoSql version: 345
Presto Ranger plugin: 2.1.0 (compile with Java 11 because PrestoSQL-345 not
support java 8)
The only difference is Presto is compile with Java 11 but we should different
Exception
Presto ranger install.properties
{code:java}
POLICY_MGR_URL=http://ranger-admin.hadoop:6080
REPOSITORY_NAME=prod-presto
COMPONENT_INSTALL_DIR_NAME=/usr/share/presto
PRESTO_CONF_DIR=/etc/presto
XAAUDIT.SUMMARY.ENABLE=true
XAAUDIT.SOLR.ENABLE=false
XAAUDIT.SOLR.URL=NONE
XAAUDIT.SOLR.USER=NONE
XAAUDIT.SOLR.PASSWORD=NONE
XAAUDIT.SOLR.ZOOKEEPER=NONE
XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hadoop/hdfs/audit/solr/spool
# Enable audit logs to ElasticSearch
#Example
#XAAUDIT.ELASTICSEARCH.ENABLE=true
#XAAUDIT.ELASTICSEARCH.URL=localhost
#XAAUDIT.ELASTICSEARCH.INDEX=audit
XAAUDIT.ELASTICSEARCH.ENABLE=false
XAAUDIT.ELASTICSEARCH.URL=elasticsearch.hadoop
XAAUDIT.ELASTICSEARCH.USER=elastic
XAAUDIT.ELASTICSEARCH.PASSWORD=elastic-password
XAAUDIT.ELASTICSEARCH.INDEX=audit
XAAUDIT.ELASTICSEARCH.PORT=9200
XAAUDIT.ELASTICSEARCH.PROTOCOL=http
XAAUDIT.HDFS.ENABLE=true
XAAUDIT.HDFS.HDFS_DIR=hdfs://hadoop-ns:9000/ranger/audit
XAAUDIT.HDFS.FILE_SPOOL_DIR=/var/log/presto/logs/hdfs/audit/hdfs/spool
XAAUDIT.HDFS.AZURE_ACCOUNTNAME=__REPLACE_AZURE_ACCOUNT_NAME
XAAUDIT.HDFS.AZURE_ACCOUNTKEY=__REPLACE_AZURE_ACCOUNT_KEY
XAAUDIT.HDFS.AZURE_SHELL_KEY_PROVIDER=__REPLACE_AZURE_SHELL_KEY_PROVIDER
XAAUDIT.HDFS.AZURE_ACCOUNTKEY_PROVIDER=__REPLACE_AZURE_ACCOUNT_KEY_PROVIDER
#Log4j Audit Provider
XAAUDIT.LOG4J.ENABLE=true
XAAUDIT.LOG4J.IS_ASYNC=false
XAAUDIT.LOG4J.ASYNC.MAX.QUEUE.SIZE=10240
XAAUDIT.LOG4J.ASYNC.MAX.FLUSH.INTERVAL.MS=30000
XAAUDIT.LOG4J.DESTINATION.LOG4J=true
XAAUDIT.LOG4J.DESTINATION.LOG4J.LOGGER=xaaudit
XAAUDIT.HDFS.IS_ENABLED=true
XAAUDIT.HDFS.DESTINATION_DIRECTORY=hdfs://hadoop-ns:9000/ranger/audit/%app-type%/%time:yyyyMMdd%
XAAUDIT.HDFS.LOCAL_BUFFER_DIRECTORY=/var/log/presto/logs/hadoop/%app-type%/audit
XAAUDIT.HDFS.LOCAL_ARCHIVE_DIRECTORY=/var/log/presto/logs/hadoop/%app-type%/audit/archive
XAAUDIT.HDFS.DESTINTATION_FILE=%hostname%-audit.log
XAAUDIT.HDFS.DESTINTATION_FLUSH_INTERVAL_SECONDS=900
XAAUDIT.HDFS.DESTINTATION_ROLLOVER_INTERVAL_SECONDS=86400
XAAUDIT.HDFS.DESTINTATION_OPEN_RETRY_INTERVAL_SECONDS=60
XAAUDIT.HDFS.LOCAL_BUFFER_FILE=%time:yyyyMMdd-HHmm.ss%.log
XAAUDIT.HDFS.LOCAL_BUFFER_FLUSH_INTERVAL_SECONDS=123
XAAUDIT.HDFS.LOCAL_BUFFER_ROLLOVER_INTERVAL_SECONDS=112
XAAUDIT.HDFS.LOCAL_ARCHIVE_MAX_FILE_COUNT=20
#Solr Audit Provider
XAAUDIT.SOLR.IS_ENABLED=false
XAAUDIT.SOLR.MAX_QUEUE_SIZE=1
XAAUDIT.SOLR.MAX_FLUSH_INTERVAL_MS=1000
XAAUDIT.SOLR.SOLR_URL=http://localhost:6083/solr/ranger_audits
SSL_KEYSTORE_FILE_PATH=/etc/hadoop/conf/ranger-plugin-keystore.jks
SSL_KEYSTORE_PASSWORD=myKeyFilePassword
SSL_TRUSTSTORE_FILE_PATH=/etc/hadoop/conf/ranger-plugin-truststore.jks
SSL_TRUSTSTORE_PASSWORD=changeit
CUSTOM_USER=presto
CUSTOM_GROUP=presto
{code}
> Ranger plugin audits to hdfs fail for presto
> --------------------------------------------
>
> Key: RANGER-3133
> URL: https://issues.apache.org/jira/browse/RANGER-3133
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
> Affects Versions: 2.1.0
> Reporter: Md Mehrab Alam
> Priority: Major
>
> 2020-12-25T01:30:30.057+0530 ERROR
> org.apache.ranger.audit.queue.AuditBatchQueue0
> org.apache.ranger.audit.provider.BaseAuditHandler Error writing to log
> file.2020-12-25T01:30:30.057+0530 ERROR
> org.apache.ranger.audit.queue.AuditBatchQueue0
> org.apache.ranger.audit.provider.BaseAuditHandler Error writing to log
> file.org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for
> scheme "hdfs" at
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3332) at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3352) at
> org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124) at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3403) at
> org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3371) at
> org.apache.hadoop.fs.FileSystem.get(FileSystem.java:477) at
> org.apache.ranger.audit.destination.HDFSAuditDestination.getLogFileStream(HDFSAuditDestination.java:277)
> at
> org.apache.ranger.audit.destination.HDFSAuditDestination$1.run(HDFSAuditDestination.java:157)
> at
> org.apache.ranger.audit.destination.HDFSAuditDestination$1.run(HDFSAuditDestination.java:154)
> at java.base/java.security.AccessController.doPrivileged(Native Method) at
> java.base/javax.security.auth.Subject.doAs(Subject.java:423) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at
> org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:529)
> at
> org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:154)
> at
> org.apache.ranger.audit.destination.HDFSAuditDestination.log(HDFSAuditDestination.java:227)
> at
> org.apache.ranger.audit.queue.AuditBatchQueue.runLogAudit(AuditBatchQueue.java:309)
> at
> org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:215)
> at java.base/java.lang.Thread.run(Thread.java:834)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
