leangjonathan commented on code in PR #969:
URL: https://github.com/apache/polaris/pull/969#discussion_r1949712370
##########
polaris-core/src/main/resources/schemas/policies/system/orphan-file-removal/2025-02-03.json:
##########
@@ -0,0 +1,47 @@
+{
+ "license": "Licensed under the Apache License, Version 2.0
(http://www.apache.org/licenses/LICENSE-2.0)",
+ "$id":
"https://polaris.apache.org/schemas/policies/system/orphan-file-removal/2025-02-03.json";,
+ "title": "Orphan File Removal Policy",
+ "description": "Inheritable Polaris policy schema for Iceberg table orphan
file removal.",
+ "type": "object",
+ "properties": {
+ "version": {
+ "type": "string",
+ "const": "2025-02-03",
+ "description": "Schema version."
+ },
+ "enable": {
+ "type": "boolean",
+ "description": "Enable or disable orphan file removal."
+ },
+ "older_than": {
+ "type": "number",
+ "description": "A Unix timestamp. Remove orphan files created before
this timestamp."
+ },
+ "location": {
+ "type": "string",
+ "description": "Customized directory other than table location to look
for files in."
Review Comment:
I think this should have a warning that if you specify locations other than
the table base location for example `s3://my-bucket` instead of
`s3://my-bucket/my-table-location`, all files not referenced by the table will
purged including potentially other table files if those files are stored in the
specified path. I think this aligns with best practice notes that one shouldn't
store tables under the same location
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
