amogh-jahagirdar commented on code in PR #4443:
URL: https://github.com/apache/iceberg/pull/4443#discussion_r848551718
##########
docs/integrations/aws.md:
##########
@@ -431,7 +445,24 @@ spark-sql --conf
spark.sql.catalog.my_catalog=org.apache.iceberg.spark.SparkCata
--conf spark.sql.catalog.my_catalog.s3.write.tags.my_key1=my_val1 \
--conf spark.sql.catalog.my_catalog.s3.write.tags.my_key2=my_val2
```
-For the above example, the objects in S3 will be saved with tags:
`my_key1=my_val1` and `my_key2=my_val2`.
+For the above example, the objects in S3 will be saved with tags:
`my_key1=my_val1` and `my_key2=my_val2`. Do note that the specified write tags
will be saved only while object creation.
+
+We can add tags before deleting the objects as well. For example, to add S3
delete tags with Spark 3.0, you can start the Spark SQL shell with:
+
+```
+sh spark-sql --conf
spark.sql.catalog.my_catalog=org.apache.iceberg.spark.SparkCatalog \
+ --conf
spark.sql.catalog.my_catalog.warehouse=s3://iceberg-warehouse/s3-tagging \
+ --conf
spark.sql.catalog.my_catalog.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog
\
+ --conf
spark.sql.catalog.my_catalog.io-impl=org.apache.iceberg.aws.s3.S3FileIO \
+ --conf spark.sql.catalog.my_catalog.s3.delete.tags.my_key3=my_val3
+```
+
+Users are expected to set the delete tags with `s3.delete.tags` and manage the
deleted files through S3 lifecycle policy.
+With the `s3.delete.tags` config, objects are tagged with the configured
key-value pairs before deletion. This is considered a soft-delete, because
users can configure tag-based object lifecycle policy at bucket level to
transition objects to different tiers.
Review Comment:
Ah good point, you are right. I forgot about the behavior we discussed later
on in that PR. Cool, I think then "before deletion" is what we want
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
