Madhan Neethiraj created RANGER-5151:
----------------------------------------
Summary: Audit to HDFS fails with error: Stream encountered errors
while writing audits to HDFS!
Key: RANGER-5151
URL: https://issues.apache.org/jira/browse/RANGER-5151
Project: Ranger
Issue Type: Bug
Components: audit
Affects Versions: 2.6.0, 2.5.0, 2.4.0, 2.3.0
Reporter: Madhan Neethiraj
RANGER-3603 introduced an option to periodically check for rollover of the HDFS
file currently open to write audit logs. This check closes the file if it finds
that the time for rollover is already past. However, this can result in
following error if another thread is in the middle of writting to the file:
{noformat}
2025-02-19T01:54:57,711 INFO [AuditFilePeriodicRollOverTask]
utils.AbstractRangerAuditWriter: Closing file. Rolling over. name = hdfs,
fileName =
hdfs://ranger-hadoop:9000/ranger/audit/hiveServer2/20250219/hiveServer2_ranger_audit_ranger-hive.example.com.6.log
2025-02-19T01:54:58,116 ERROR [org.apache.ranger.audit.queue.AuditBatchQueue1]
utils.RangerJSONAuditWriter: Stream encountered errors while writing audits to
HDFS!
2025-02-19T01:54:58,116 WARN [org.apache.ranger.audit.queue.AuditBatchQueue1]
provider.BaseAuditHandler: failed to log audit event:
{"repoType":3,"repo":"dev_hive","reqUser":"madhan","evtTime":"2025-02-19
01:54:54.981","access":"SELECT","resource":"test_madhan/test_tbl1/dob,id,name","resType":"@column","action":"select","result":1,"agent":"hiveServer2","policy":7,"reason":null,"enforcer":"ranger-acl","sess":"adbab684-f28a-4f75-bc65-9dd213b7fa58","cliType":"HIVESERVER2","cliIP":"172.18.0.13","reqData":"select
* from
test_madhan.test_tbl1","agentHost":"ranger-hive.example.com","logType":"RangerAudit","id":"22008707-e40a-4f37-8741-767703aee89f-0","seq_num":1,"event_count":1,"event_dur_ms":0,"tags":[],"datasets":null,"projects":null,"additional_info":"{\"forwarded-ip-addresses\":\"[]\",\"remote-ip-address\":\"172.18.0.13\"}","cluster_name":"","zone_name":null,"policy_version":1}
, errorMessage=
{noformat}
To prevent this error, closing of the file must be performed in the same thread
that writes to the file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
