Sravani Gadey created HADOOP-18257: -------------------------------------- Summary: Analyzing S3A Audit Logs Key: HADOOP-18257 URL: https://issues.apache.org/jira/browse/HADOOP-18257 Project: Hadoop Common Issue Type: Task Components: fs/s3 Reporter: Sravani Gadey
The main aim is to analyze S3A Audit logs to give better insights in Hive and Spark jobs. Steps involved are: * Merging audit log files containing huge number of audit logs collected from a job containing various S3 requests. * Parsing audit logs using regular expressions i.e., dividing them into key value pairs. * Converting the key value pairs into CSV file and AVRO file formats. * Querying on data which would give better insights for different jobs. * Visualizing the audit logs on Zeppelin or Jupyter notebook with graphs. -- This message was sent by Atlassian Jira (v8.20.7#820007) --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org