Andrey Yarovoy created HDDS-17857:
-------------------------------------
Summary: Tests write audit.log and system_audit.log into the
module directory instead of target
Key: HDDS-17857
URL: https://issues.apache.org/jira/browse/HDDS-17857
Project: Apache Ozone
Issue Type: Bug
Components: test
Reporter: Andrey Yarovoy
Assignee: Andrey Yarovoy
h3. Problem
Several tests turn on audit logging by setting
\{{log4j.configurationFile=auditlog.properties}}. Every
\{{auditlog.properties}} under \{{src/test/resources}} names its file with a
relative path:
{noformat}
appender.audit.fileName=audit.log
appender.systemaudit.fileName=system_audit.log (integration-test only)
{noformat}
Log4j resolves these paths against the working directory of the surefire JVM,
which is the module directory, so the files land next to \{{pom.xml}} and
\{{src/}}, not under \{{target/}}. They show up as untracked files in \{{git
status}} (\{{.gitignore}} doesn't cover them), and \{{mvn clean}} doesn't
remove them, so they pile up between runs. A careless \{{git add -A}} can
commit them.
Example: after running \{{TestSecureOzoneRpcClient}}, \{{git status}} shows
{noformat}
?? hadoop-ozone/integration-test/audit.log
?? hadoop-ozone/integration-test/system_audit.log
{noformat}
{\{OzoneRpcClientTests}} (the base of \{{TestOzoneRpcClient}} and
\{{TestSecureOzoneRpcClient}}) calls \{{AuditLogTestUtils.enableAuditLog()}}
but never deletes the files. Other tests clean up only partly: \{{TestOmAcls}}
and \{{TestOMRatisSnapshotTransfer}} delete them, and \{{TestOzoneAuditLogger}}
deletes \{{audit.log}} in \{{@AfterAll}}. That cleanup doesn't run if the fork
is killed or times out.
Affected configs:
* \{{hadoop-ozone/integration-test/src/test/resources/auditlog.properties}}
(\{{audit.log}}, \{{system_audit.log}})
* \{{hadoop-hdds/framework/src/test/resources/auditlog.properties}}
* \{{hadoop-hdds/rocks-native/src/test/resources/auditlog.properties}}
* \{{hadoop-ozone/s3gateway/src/test/resources/auditlog.properties}}
The readers use the same relative names: \{{AuditLogTestUtils}}
(\{{AUDITLOG_FILENAME}}, \{{SYSTEM_AUDITLOG_FILENAME}}),
\{{TestOzoneAuditLogger}}, \{{TestS3GatewayAuditLog}} and
\{{TestOzoneRpcClientForAclAuditLog}}.
h3. Proposed fix
Write the files under the module's \{{target/}} directory. Surefire already
passes \{{hadoop.log.dir=$\{project.build.directory\}/log}} to every test JVM
(root \{{pom.xml}}, \{{systemPropertyVariables}}), so:
# In each test \{{auditlog.properties}}, use a log4j2 system property lookup
with a fallback for IDE runs, e.g.
{noformat}
appender.audit.fileName=${sys:hadoop.log.dir:-target/log}/audit.log
appender.systemaudit.fileName=${sys:hadoop.log.dir:-target/log}/system_audit.log
{noformat}
# Resolve the same location in \{{AuditLogTestUtils}} (e.g.
\{{Paths.get(System.getProperty("hadoop.log.dir", "target/log"),
"audit.log")}}) and use it in the tests that open the file directly
(\{{TestOzoneAuditLogger}}, \{{TestS3GatewayAuditLog}},
\{{TestOzoneRpcClientForAclAuditLog}}), so that writers and readers agree.
# Optionally, add \{{audit.log}} and \{{system_audit.log}} to \{{.gitignore}},
so that files left by older runs or branches don't show up.
h3. Compatibility / side effects
Test-only. The production audit configs in
\{{hadoop-ozone/dist/src/shell/conf/*-audit-log4j2.properties}} don't change.
\{{mvn clean}} will now remove the files.
h3. Out of scope
Tests that run in parallel forks of the same module still share one file,
because they share one \{{target/log}}. Separating them (e.g. with
\{{$\{surefire.forkNumber\}}}) would be a separate change.
h3. Tests
* Run the audit tests listed above and check that they still pass.
* After the run, \{{git status}} shows no \{{audit.log}} or
\{{system_audit.log}}, and the files are under \{{<module>/target/log/}}.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]