Hi! I'm trying to implement Apache Ranger authorization for Hadoop. I use Ranger 2.1.0. Policy server itself works, at least I see Hadoop cluster and can create policies within Ranger Admin Console. For some reason I cannot implement ranger-usersync - it installs but does not show any activity. I only see in logs that it loads and starts synchronization with Unix. I see no attempts to sync with LDAP (AD) neither in its' logs nor in tcpdump report. Ranger Admin UI does not show usersync plugin in the plugins list either. Testing LDAP connection from ranger-admin machine with ldapsearch shows no problem with AD connection and retrieving information.
I use CentOS 8 both for Hadoop and Ranger Policy Server machines. In CentOS 7 situation is the same. Java version is 1.8.0_265. I do not use Ambari as I haven't managed to build it. Below are installation.properties contents for ranger-usersync and ranger-admin components. Comments removed to save space. ---- ranger-usersync: ---- ranger_base_dir = /etc/ranger POLICY_MGR_URL = http://172.25.32.225:6080 SYNC_SOURCE = ldap MIN_UNIX_USER_ID_TO_SYNC = 0 MIN_UNIX_GROUP_ID_TO_SYNC = 0 SYNC_INTERVAL = 5 unix_user=ranger unix_group=ranger rangerUsersync_password=P@ssw0rd usersync_principal= usersync_keytab= hadoop_conf=/etc/hadoop/conf CRED_KEYSTORE_FILENAME=/etc/ranger/usersync/conf/rangerusersync.jceks AUTH_SSL_ENABLED=false AUTH_SSL_KEYSTORE_FILE=/etc/ranger/usersync/conf/cert/unixauthservice.jks AUTH_SSL_KEYSTORE_PASSWORD=UnIx529p AUTH_SSL_TRUSTSTORE_FILE= AUTH_SSL_TRUSTSTORE_PASSWORD= ROLE_ASSIGNMENT_LIST_DELIMITER = & USERS_GROUPS_ASSIGNMENT_LIST_DELIMITER = : USERNAME_GROUPNAME_ASSIGNMENT_LIST_DELIMITER = , GROUP_BASED_ROLE_ASSIGNMENT_RULES=ROLE_SYS_ADMIN:u:administrator&ROLE_SYS_ADMIN:g:domain admins,administrators&ROLE_KEY_ADMIN:u:administrator&ROLE_KEY_ADMIN:g:domain admins&ROLE_ADMIN_AUDITOR:u:administrator&ROLE_KEY_ADMIN_AUDITOR:u:administrator&ROLE_KEY_ADMIN_AUDITOR:g:administrator&ROLE_ADMIN_AUDITOR:g:domain admins SYNC_LDAP_URL = ldap://172.25.32.193 SYNC_LDAP_BIND_DN = cn=administrator,cn=users,dc=open,dc=ru SYNC_LDAP_BIND_PASSWORD = P@ssw0rd SYNC_LDAP_DELTASYNC = false SYNC_LDAP_SEARCH_BASE = ou=Test OU,dc=open,dc=ru SYNC_LDAP_USER_SEARCH_BASE = ou=Test OU,dc=open,dc=ru SYNC_LDAP_USER_SEARCH_SCOPE = sub SYNC_LDAP_USER_OBJECT_CLASS = user SYNC_LDAP_USER_SEARCH_FILTER = (memberof=CN=ranger-admins,OU=Test OU,DC=open,DC=ru|memberof=CN=hadoop-users,OU=Test OU,DC=open,DC=ru) SYNC_LDAP_USER_NAME_ATTRIBUTE = cn SYNC_LDAP_USER_GROUP_NAME_ATTRIBUTE = memberof SYNC_LDAP_USERNAME_CASE_CONVERSION=lower SYNC_LDAP_GROUPNAME_CASE_CONVERSION=lower logdir=/var/log/ranger/usersync USERSYNC_PID_DIR_PATH=/home/ranger/pid SYNC_GROUP_SEARCH_ENABLED=true SYNC_GROUP_USER_MAP_SYNC_ENABLED=true SYNC_GROUP_SEARCH_BASE=ou=Test OU,dc=open,dc=ru SYNC_GROUP_SEARCH_SCOPE=sub SYNC_GROUP_OBJECT_CLASS=group SYNC_LDAP_GROUP_SEARCH_FILTER= SYNC_GROUP_NAME_ATTRIBUTE=cn SYNC_GROUP_MEMBER_ATTRIBUTE_NAME=member SYNC_PAGED_RESULTS_ENABLED= SYNC_PAGED_RESULTS_SIZE= SYNC_LDAP_REFERRAL = follow JVM_METRICS_ENABLED= JVM_METRICS_FILENAME= JVM_METRICS_FILEPATH= JVM_METRICS_FREQUENCY_TIME_IN_MILLIS= ---- ranger-admin: ---- PYTHON_COMMAND_INVOKER=python DB_FLAVOR=MYSQL SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar db_root_user=root db_root_password= db_host=localhost:3306 db_ssl_enabled=false db_ssl_required=false db_ssl_verifyServerCertificate=false db_ssl_auth_type=2-way javax_net_ssl_keyStore= javax_net_ssl_keyStorePassword= javax_net_ssl_trustStore= javax_net_ssl_trustStorePassword= db_name=ranger db_user=ranger db_password=P@ssw0rd rangerAdmin_password=P@ssw0rd rangerTagsync_password=P@ssw0rd rangerUsersync_password=P@ssw0rd keyadmin_password=P@ssw0rd audit_store=solr audit_elasticsearch_urls= audit_elasticsearch_port= audit_elasticsearch_protocol= audit_elasticsearch_user= audit_elasticsearch_password= audit_elasticsearch_index= audit_elasticsearch_bootstrap_enabled=true audit_solr_urls=http://172.25.32.225:6083/solr/ranger_audits audit_solr_user=ranger audit_solr_password=P@ssw0rd audit_solr_zookeepers= audit_solr_collection_name=ranger_audits audit_solr_config_name=ranger_audits audit_solr_no_shards=1 audit_solr_no_replica=1 audit_solr_max_shards_per_node=1 audit_solr_acl_user_list_sasl=solr,infra-solr audit_solr_bootstrap_enabled=true policymgr_external_url=http://localhost:6080 policymgr_http_enabled=true policymgr_https_keystore_file= policymgr_https_keystore_keyalias=rangeradmin policymgr_https_keystore_password= policymgr_supportedcomponents=hdfs unix_user=ranger unix_user_pwd=ranger unix_group=ranger authentication_method=ACTIVE_DIRECTORY remoteLoginEnabled=true authServiceHostName=localhost authServicePort=5151 ranger_unixauth_keystore=keystore.jks ranger_unixauth_keystore_password=password ranger_unixauth_truststore=cacerts ranger_unixauth_truststore_password=changeit xa_ldap_url= xa_ldap_userDNpattern= xa_ldap_groupSearchBase= xa_ldap_groupSearchFilter= xa_ldap_groupRoleAttribute= xa_ldap_base_dn= xa_ldap_bind_dn= xa_ldap_bind_password= xa_ldap_referral= xa_ldap_userSearchFilter= xa_ldap_ad_domain=open.ru xa_ldap_ad_url=ldap://172.25.32.193 xa_ldap_ad_base_dn=dc=open,dc=ru xa_ldap_ad_bind_dn=cn=administrator,cn=users,dc=open,dc=ru xa_ldap_ad_bind_password=P@ssw0rd xa_ldap_ad_referral=follow xa_ldap_ad_userSearchFilter=(sAMAccountName={0}) spnego_principal= spnego_keytab= token_valid=30 cookie_domain= cookie_path=/ admin_principal= admin_keytab= lookup_principal= lookup_keytab= hadoop_conf=/etc/hadoop/conf sso_enabled=false sso_providerurl=https://127.0.0.1:8443/gateway/knoxsso/api/v1/websso sso_publickey= RANGER_ADMIN_LOG_DIR=$PWD RANGER_PID_DIR_PATH=/home/ranger/pid XAPOLICYMGR_DIR=$PWD app_home=$PWD/ews/webapp TMPFILE=$PWD/.fi_tmp LOGFILE=$PWD/logfile LOGFILES="$LOGFILE" JAVA_BIN='java' JAVA_VERSION_REQUIRED='1.8' JAVA_ORACLE='Java(TM) SE Runtime Environment' ranger_admin_max_heap_size=1g PATCH_RETRY_INTERVAL=120 STALE_PATCH_ENTRY_HOLD_TIME=10 mysql_core_file=db/mysql/optimized/current/ranger_core_db_mysql.sql mysql_audit_file=db/mysql/xa_audit_db.sql oracle_core_file=db/oracle/optimized/current/ranger_core_db_oracle.sql oracle_audit_file=db/oracle/xa_audit_db_oracle.sql postgres_core_file=db/postgres/optimized/current/ranger_core_db_postgres.sql postgres_audit_file=db/postgres/xa_audit_db_postgres.sql sqlserver_core_file=db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql sqlserver_audit_file=db/sqlserver/xa_audit_db_sqlserver.sql sqlanywhere_core_file=db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql sqlanywhere_audit_file=db/sqlanywhere/xa_audit_db_sqlanywhere.sql cred_keystore_filename=$app_home/WEB-INF/classes/conf/.jceks/rangeradmin.jceks --------- /var/log/ranger/usersync/usersync-x5-rngr-ps-ranger.log --------- 27 Nov 2020 13:17:31 INFO UnixAuthenticationService [main] - Starting User Sync Service! 27 Nov 2020 13:17:31 INFO UnixAuthenticationService [main] - Start : startUnixUserGroupSyncProcess 27 Nov 2020 13:17:31 INFO UnixAuthenticationService [main] - UnixUserSyncThread started 27 Nov 2020 13:17:31 INFO UnixAuthenticationService [main] - creating UserSyncMetricsProducer thread with default metrics location : /var/log/ranger/usersync 27 Nov 2020 13:17:31 INFO UnixAuthenticationService [main] - Ranger userSync metrics is not enabled 27 Nov 2020 13:17:31 INFO AbstractMapper [UnixUserSyncThread] - Initializing for ranger.usersync.mapping.username.regex 27 Nov 2020 13:17:31 INFO AbstractMapper [UnixUserSyncThread] - Initializing for ranger.usersync.mapping.groupname.regex 27 Nov 2020 13:17:31 INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - LdapDeltaUserGroupBuilder created 27 Nov 2020 13:17:31 INFO UserGroupSyncConfig [UnixUserSyncThread] - Sleep Time Between Cycle can not be lower than [3600000] millisec. resetting to min value. 27 Nov 2020 13:17:31 INFO UserGroupSync [UnixUserSyncThread] - initializing sink: org.apache.ranger.ldapusersync.process.LdapPolicyMgrUserGroupBuilder 27 Nov 2020 13:17:33 INFO AbstractMapper [UnixUserSyncThread] - Initializing for ranger.usersync.mapping.username.regex 27 Nov 2020 13:17:33 INFO AbstractMapper [UnixUserSyncThread] - Initializing for ranger.usersync.mapping.groupname.regex 27 Nov 2020 13:17:33 INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - LdapDeltaUserGroupBuilder created 27 Nov 2020 13:17:33 INFO UserGroupSync [UnixUserSyncThread] - initializing source: org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder 27 Nov 2020 13:17:33 INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - LdapDeltaUserGroupBuilder initialization started 27 Nov 2020 13:17:34 INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - LdapDeltaUserGroupBuilder initialization completed with -- ldapUrl: ldap:// 172.25.32.193:389, ldapBindDn: cn=administrator,cn=users,dc=open,dc=ru, ldapBindPassword: ***** , ldapAuthenticationMechanism: simple, searchBase: ou=Test OU,dc=open,dc=ru, userSearchBase: [ou=Test OU,dc=open,dc=ru], userSearchScope: 2, userObjectClass: user, userSearchFilter: (memberof=CN=ranger-admins,OU=Test OU,DC=open,DC=ru|memberof=CN=hadoop-users,OU=Test OU,DC=open,DC=ru), extendedUserSearchFilter: null, userNameAttribute: cn, userSearchAttributes: [uSNChanged, cn, modifytimestamp, objectid, userurincipaluame], userGroupNameAttributeSet: null, otherUserAttributes: [userurincipaluame], pagedResultsEnabled: true, pagedResultsSize: 500, groupSearchEnabled: true, groupSearchBase: [ou=Test OU,dc=open,dc=ru], groupSearchScope: 2, groupObjectClass: group, groupSearchFilter: , extendedGroupSearchFilter: (&null(|(member={0})(member={1}))), extendedAllGroupsSearchFilter: null, groupMemberAttributeName: member, groupNameAttribute: cn, groupSearchAttributes: [uSNChanged, displayname, member, cn, modifytimestamp, objectid], groupSearchFirstEnabled: false, userSearchEnabled: true, ldapReferral: follow 27 Nov 2020 13:17:34 INFO UserGroupSync [UnixUserSyncThread] - Begin: initial load of user/group from source==>sink 27 Nov 2020 13:17:34 INFO UserGroupSync [UnixUserSyncThread] - End: initial load of user/group from source==>sink 27 Nov 2020 13:17:34 INFO UserGroupSync [UnixUserSyncThread] - Done initializing user/group source and sink 27 Nov 2020 13:17:36 INFO UnixAuthenticationService [main] - Enabling Unix Auth Service! 27 Nov 2020 13:17:37 INFO UnixAuthenticationService [main] - Enabling Protocol: [SSLv2Hello] 27 Nov 2020 13:17:37 INFO UnixAuthenticationService [main] - Enabling Protocol: [TLSv1.2] --------- Unfortunately I have not much experience with Ranger/Hadoop, any help is appreciated. Regards, Dmitry
