[
https://issues.apache.org/jira/browse/HADOOP-18258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691436#comment-17691436
]
ASF GitHub Bot commented on HADOOP-18258:
-----------------------------------------
sravanigadey commented on code in PR #4383:
URL: https://github.com/apache/hadoop/pull/4383#discussion_r1112667084
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/audit/TestAuditTool.java:
##########
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.s3a.audit;
+
+import java.io.File;
+import java.io.FileWriter;
+import java.nio.file.Files;
+
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.avro.file.DataFileReader;
+import org.apache.avro.io.DatumReader;
+import org.apache.avro.specific.SpecificDatumReader;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.LocatedFileStatus;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.RemoteIterator;
+import org.apache.hadoop.fs.s3a.AbstractS3ATestBase;
+
+import static
org.apache.hadoop.fs.s3a.audit.TestS3AAuditLogMergerAndParser.SAMPLE_LOG_ENTRY;
+
+/**
+ * This will implement tests on AuditTool class.
+ */
+public class TestAuditTool extends AbstractS3ATestBase {
+
+ private static final Logger LOG =
+ LoggerFactory.getLogger(TestAuditTool.class);
+
+ private final AuditTool auditTool = new AuditTool();
+
+ /**
+ * Sample directories and files to test.
+ */
+ private File sampleFile;
+ private File sampleDir;
+ private File sampleDestDir;
+
+ /**
+ * Testing run method in AuditTool class by passing source and destination
+ * paths.
+ */
+ @Test
+ public void testRun() throws Exception {
+ sampleDir = Files.createTempDirectory("sampleDir").toFile();
+ sampleFile = File.createTempFile("sampleFile", ".txt", sampleDir);
+ try (FileWriter fw = new FileWriter(sampleFile)) {
+ fw.write(SAMPLE_LOG_ENTRY);
+ fw.flush();
+ }
+ sampleDestDir = Files.createTempDirectory("sampleDestDir").toFile();
+ Path logsPath = new Path(sampleDir.toURI());
+ Path destPath = new Path(sampleDestDir.toURI());
+ String[] args = {destPath.toString(), logsPath.toString()};
+ auditTool.run(args);
+ FileSystem fileSystem = destPath.getFileSystem(getConfiguration());
+ RemoteIterator<LocatedFileStatus> listOfDestFiles =
+ fileSystem.listFiles(destPath, true);
+ while (listOfDestFiles.hasNext()) {
Review Comment:
As suggested in the above comment, I added the expected path in code and
opened the file using `fileSystem.open(expectedPath)` .
The test will fail at open if `AvroData.avro` is not there or created and if
file is there then test will pass.
> Merging of S3A Audit Logs
> -------------------------
>
> Key: HADOOP-18258
> URL: https://issues.apache.org/jira/browse/HADOOP-18258
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Sravani Gadey
> Assignee: Sravani Gadey
> Priority: Major
> Labels: pull-request-available
> Time Spent: 12.5h
> Remaining Estimate: 0h
>
> Merging audit log files containing huge number of audit logs collected from a
> job like Hive or Spark job containing various S3 requests like list, head,
> get and put requests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
