kevinjqliu commented on code in PR #1023:
URL: https://github.com/apache/iceberg-python/pull/1023#discussion_r1709774974
##########
pyiceberg/table/__init__.py:
##########
@@ -593,7 +593,9 @@ def delete(self, delete_filter: Union[str,
BooleanExpression], snapshot_properti
filtered_df = df.filter(preserve_row_filter)
# Only rewrite if there are records being deleted
- if len(df) != len(filtered_df):
+ if len(filtered_df) == 0:
+ replaced_files.append((original_file.file, []))
+ elif len(df) != len(filtered_df):
Review Comment:
nit: is it more readable if inlined?
```
if filtered_df and len(df) != len(filtered_df):
```
##########
tests/integration/test_writes/test_writes.py:
##########
@@ -1309,3 +1309,27 @@ def
test_table_v1_with_null_nested_namespace(session_catalog: Catalog, arrow_tab
# We expect no error here
session_catalog.drop_table(identifier)
+
+
[email protected]
+def test_overwrite_all_data_with_filter(session_catalog: Catalog) -> None:
+ schema = Schema(
+ NestedField(1, "id", StringType(), required=True),
+ NestedField(2, "name", StringType(), required=False),
+ identifier_field_ids=[1],
+ )
+
+ data = pa.Table.from_pylist(
+ [
+ {"id": "1", "name": "Amsterdam"},
+ {"id": "2", "name": "San Francisco"},
+ {"id": "3", "name": "Drachten"},
+ ],
+ schema=schema.as_arrow(),
+ )
+
+ identifier = "default.test_overwrite_all_data_with_filter"
+ tbl = _create_table(session_catalog, identifier, data=[data],
schema=schema)
+ tbl.overwrite(data, In("id", ["1", "2", "3"]))
+
+ assert len(tbl.scan().to_arrow()) == 3
Review Comment:
nit: since all data match the filter, the `overwrite` operation is a no-op,
right? if so, can we assert that in the test? maybe show that the files are the
same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
