[
https://issues.apache.org/jira/browse/RANGER-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ognjenit updated RANGER-4792:
-----------------------------
Attachment: image-2024-06-27-21-30-33-630.png
> Fix issue with creating index and import data in ElasticSearch as Audit
> database
> --------------------------------------------------------------------------------
>
> Key: RANGER-4792
> URL: https://issues.apache.org/jira/browse/RANGER-4792
> Project: Ranger
> Issue Type: Bug
> Components: admin, audit
> Affects Versions: 2.4.0
> Environment: Container:
> - Linux: Debian buster
> - Java: openjdk- 11
> - Tested on kubernetes and openshift on AWS/Azure and on-prem
> Reporter: ognjenit
> Priority: Major
> Attachments: image-2024-06-27-21-30-33-630.png
>
>
> Hi all,
> I apologize in advance if I haven't adjusted this issue properly.
> Short description:
> I have deployed Trino with ranger-trino-plugin and I wanted to use
> ElasticSearch (7.10.2) as a place where I want to store the audit. When I
> configured ranger-admin to use elasticsearch (audit_store=elasticsearch and
> all other parameters audit_elasticsearch_*) I started getting errors in the
> catalina.out: java.lang.NoSuchFieldError: LUCENE_8_5_1. As I increased the
> version of Lucena, it was written in the logs that an even higher version was
> needed. So in the end, I moved it to 8.11.3 and 8.4.0 for lucene-spatial
> since it is the latest.
> After it was fixed, I tried to use https for elasticsearch protocol
> (audit_elasticsearch_protocol) however, it always showed that ranger-admin
> use http instead of https. I show in code that audit_elasticsearch_protocol
> is not configured well.
> As soon as it done, ranger admin successfully created ES index. However, I
> need to move from MiscUtil.toDate to MiscUtil.toLocalDate for evtTime
> "column" since I was getting error: Error converting value to date. Value =
> 2024-05-13T13:08:47.905Z
> As soon as I fixed it, I found an error in Trino that the plugin couldn't
> insert data into elasticsearch. After I upgraded httpcomponents bug-fix
> version, it's started inserting data.
> I opened PR with the fix 2.4.0 version, do I need to do the same on the
> master branch?
> PR: https://github.com/apache/ranger/pull/314/files
> h4. 1. Lucene version - fixed problem with writing data to ElasticSearch
> {*}Error{*}: java.lang.NoSuchFieldError: LUCENE_8_5_1
> I tried to change minor version one by one, but only latest version fit for
> me.
> Changes:
> * agents-audit/pom.xml: 311
> * pom.xml: 241
> h4. 2. Elastic search protocol - fixed problem with changing protocol
> Even though I changed ranger.audit.elasticsearch.protocol from http to https,
> audit plugin still using http protocol.
> Changes:
> * security-admin/scripts/ranger-admin-site-template.xml: 167-170
> * security-admin/scripts/setup.sh: 79, 794-797
> * security-admin/scripts/upgrade_admin.py: 116
> * security-admin/src/main/resources/conf.dist/ranger-admin-site.xml: 53-57
> *
> security-admin/src/test/java/org/apache/ranger/elasticsearch/ElasticSearchAccessAuditsServiceTest.java:
> 56
> h4. 3. Audit plugin - cannot write audit to ES
> {*}Error{*}: bootstrap method initialization exception
> After changing the version of httpcomponents I started seeing audit
> Changes:
> * pom.xml: 137, 138, 140
> h4. 4. Ranger admin console - Audit show 1-1-1970
> {*}Erro{*}: Error converting value to date. Value = 2024-05-13T13:08:47.905Z
> Even though evtTime was ok in ElasticSearch, ranger couldn't show it on GUI.
> Changes:
> *
> security-admin/src/main/java/org/apache/ranger/elasticsearch/ElasticSearchAccessAuditsService.java:
> 260
> *
> security-admin/src/main/java/org/apache/ranger/solr/SolrAccessAuditsService.java:
> 239
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
