Andrey Yarovoy created HDDS-17857:
-------------------------------------

             Summary: Tests write audit.log and system_audit.log into the 
module directory instead of target
                 Key: HDDS-17857
                 URL: https://issues.apache.org/jira/browse/HDDS-17857
             Project: Apache Ozone
          Issue Type: Bug
          Components: test
            Reporter: Andrey Yarovoy
            Assignee: Andrey Yarovoy


h3. Problem

Several tests turn on audit logging by setting 
\{{log4j.configurationFile=auditlog.properties}}. Every 
\{{auditlog.properties}} under \{{src/test/resources}} names its file with a 
relative path:

{noformat}
appender.audit.fileName=audit.log
appender.systemaudit.fileName=system_audit.log (integration-test only)
{noformat}

Log4j resolves these paths against the working directory of the surefire JVM, 
which is the module directory, so the files land next to \{{pom.xml}} and 
\{{src/}}, not under \{{target/}}. They show up as untracked files in \{{git 
status}} (\{{.gitignore}} doesn't cover them), and \{{mvn clean}} doesn't 
remove them, so they pile up between runs. A careless \{{git add -A}} can 
commit them.

Example: after running \{{TestSecureOzoneRpcClient}}, \{{git status}} shows

{noformat}
?? hadoop-ozone/integration-test/audit.log
?? hadoop-ozone/integration-test/system_audit.log
{noformat}

{\{OzoneRpcClientTests}} (the base of \{{TestOzoneRpcClient}} and 
\{{TestSecureOzoneRpcClient}}) calls \{{AuditLogTestUtils.enableAuditLog()}} 
but never deletes the files. Other tests clean up only partly: \{{TestOmAcls}} 
and \{{TestOMRatisSnapshotTransfer}} delete them, and \{{TestOzoneAuditLogger}} 
deletes \{{audit.log}} in \{{@AfterAll}}. That cleanup doesn't run if the fork 
is killed or times out.

Affected configs:
* \{{hadoop-ozone/integration-test/src/test/resources/auditlog.properties}} 
(\{{audit.log}}, \{{system_audit.log}})
* \{{hadoop-hdds/framework/src/test/resources/auditlog.properties}}
* \{{hadoop-hdds/rocks-native/src/test/resources/auditlog.properties}}
* \{{hadoop-ozone/s3gateway/src/test/resources/auditlog.properties}}

The readers use the same relative names: \{{AuditLogTestUtils}} 
(\{{AUDITLOG_FILENAME}}, \{{SYSTEM_AUDITLOG_FILENAME}}), 
\{{TestOzoneAuditLogger}}, \{{TestS3GatewayAuditLog}} and 
\{{TestOzoneRpcClientForAclAuditLog}}.

h3. Proposed fix

Write the files under the module's \{{target/}} directory. Surefire already 
passes \{{hadoop.log.dir=$\{project.build.directory\}/log}} to every test JVM 
(root \{{pom.xml}}, \{{systemPropertyVariables}}), so:

# In each test \{{auditlog.properties}}, use a log4j2 system property lookup 
with a fallback for IDE runs, e.g.
{noformat}
appender.audit.fileName=${sys:hadoop.log.dir:-target/log}/audit.log
appender.systemaudit.fileName=${sys:hadoop.log.dir:-target/log}/system_audit.log
{noformat}
# Resolve the same location in \{{AuditLogTestUtils}} (e.g. 
\{{Paths.get(System.getProperty("hadoop.log.dir", "target/log"), 
"audit.log")}}) and use it in the tests that open the file directly 
(\{{TestOzoneAuditLogger}}, \{{TestS3GatewayAuditLog}}, 
\{{TestOzoneRpcClientForAclAuditLog}}), so that writers and readers agree.
# Optionally, add \{{audit.log}} and \{{system_audit.log}} to \{{.gitignore}}, 
so that files left by older runs or branches don't show up.

h3. Compatibility / side effects

Test-only. The production audit configs in 
\{{hadoop-ozone/dist/src/shell/conf/*-audit-log4j2.properties}} don't change. 
\{{mvn clean}} will now remove the files.

h3. Out of scope

Tests that run in parallel forks of the same module still share one file, 
because they share one \{{target/log}}. Separating them (e.g. with 
\{{$\{surefire.forkNumber\}}}) would be a separate change.

h3. Tests

* Run the audit tests listed above and check that they still pass.
* After the run, \{{git status}} shows no \{{audit.log}} or 
\{{system_audit.log}}, and the files are under \{{<module>/target/log/}}.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to