jerryshao commented on code in PR #13275: URL: https://github.com/apache/gravitino/pull/13275#discussion_r4035559297
########## docs/how-to-install.md: ########## @@ -75,6 +75,31 @@ The Gravitino server configuration file is `conf/gravitino.conf`. Configure the The Gravitino server log configuration file is `conf/log4j2.properties`. Gravitino uses Log4j2 as the logging system. Refer to the [Log4j2 configuration guide](https://logging.apache.org/log4j/2.x/) to do the log configuration. +##### Log rotation and retention + +A log rotates when its current file reaches the roll size, and at the end of each day. Every rotation compresses the file into its own archive, `<log name>_<yyyyMMdd>.<index>.log.gz`, so no archive replaces another. Review Comment: Good point, fixed: it now says a log rotates when its file reaches the roll size or when the day ends. ########## docs/how-to-install.md: ########## @@ -75,6 +75,31 @@ The Gravitino server configuration file is `conf/gravitino.conf`. Configure the The Gravitino server log configuration file is `conf/log4j2.properties`. Gravitino uses Log4j2 as the logging system. Refer to the [Log4j2 configuration guide](https://logging.apache.org/log4j/2.x/) to do the log configuration. +##### Log rotation and retention + +A log rotates when its current file reaches the roll size, and at the end of each day. Every rotation compresses the file into its own archive, `<log name>_<yyyyMMdd>.<index>.log.gz`, so no archive replaces another. + +When a log rotates, an archive of that log is deleted if it is older than `logMaxAge`, or if the archives of the log exceed its total size cap. The oldest archives are deleted first. Deletion only looks at the archives of the log itself, directly in the log directory. The cap counts archives only, not the file being written. + +| Log file | Configuration file | Roll size | Kept for | Total size cap | +|-------------------------------------|-------------------------------------------------|-----------|----------|---------------------------------| +| `gravitino-server.log` | `conf/log4j2.properties` | 100MB | 30 days | 2GB (`serverLogMaxTotalSize`) | +| `gravitino_audit.log` | `conf/log4j2.properties` | 256MB | 30 days | 10GB (`auditLogMaxTotalSize`) | +| `gravitino_lineage.log` | `conf/log4j2.properties` | 100MB | 30 days | 1GB (`lineageLogMaxTotalSize`) | +| `gravitino-iceberg-rest-server.log` | `conf/gravitino-iceberg-rest-log4j2.properties` | 100MB | 30 days | 2GB (`serverLogMaxTotalSize`) | +| `gravitino-lance-rest-server.log` | `conf/gravitino-lance-rest-log4j2.properties` | 100MB | 30 days | 1GB (`serverLogMaxTotalSize`) | + +The retention properties are at the top of each configuration file. For example, to keep 90 days of logs with up to 5GB of server log archives: + +```properties +property.logMaxAge = 90d +property.serverLogMaxTotalSize = 5GB +``` + +`logMaxAge` applies to every log in the same file. To change one log only, set the `ifFileName.ifAny.ifLastModified.age` of its appender. If you change the archive name in a `filePattern`, change the `ifFileName.glob` of the same appender to match, or the archives of that log are never deleted. Review Comment: `ifFileName`, `ifAny`, and `ifLastModified` are the component names of Log4j2 Delete action conditions (`IfFileName`, `IfAny`, `IfLastModified`, `IfAccumulatedFileSize`), so the key reads as nested conditions: if the file name matches, and if any of the age or size conditions holds, delete the archive. I added this to the doc with a link to the Log4j2 manual and spelled out the full keys. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
