Re: [PR] Support partial deletes [iceberg-python]

2024-07-09 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1670138135 ## pyiceberg/table/__init__.py: ## @@ -539,6 +578,81 @@ def overwrite( for data_file in data_files:

Re: [PR] Support partial deletes [iceberg-python]

2024-07-09 Thread via GitHub
Fokko merged PR #569: URL: https://github.com/apache/iceberg-python/pull/569 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Support partial deletes [iceberg-python]

2024-07-08 Thread via GitHub
HonahX commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1669726783 ## pyiceberg/table/__init__.py: ## @@ -539,6 +578,81 @@ def overwrite( for data_file in data_files:

Re: [PR] Support partial deletes [iceberg-python]

2024-07-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1666905217 ## pyiceberg/table/__init__.py: ## @@ -498,7 +524,10 @@ def append(self, df: pa.Table, snapshot_properties: Dict[str, str] = EMPTY_DICT)

Re: [PR] Support partial deletes [iceberg-python]

2024-07-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1666901857 ## pyiceberg/table/__init__.py: ## @@ -538,6 +566,74 @@ def overwrite( for data_file in data_files:

Re: [PR] Support partial deletes [iceberg-python]

2024-07-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1666900390 ## pyiceberg/table/__init__.py: ## @@ -517,9 +546,6 @@ def overwrite( if not isinstance(df, pa.Table): raise ValueError(f"Expected PyArrow

Re: [PR] Support partial deletes [iceberg-python]

2024-07-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1666880488 ## pyiceberg/table/__init__.py: ## @@ -3882,7 +4161,7 @@ def _get_table_partitions( return table_partitions -def _determine_partitions(spec: PartitionSpec,

Re: [PR] Support partial deletes [iceberg-python]

2024-07-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1666876690 ## tests/integration/test_deletes.py: ## @@ -0,0 +1,368 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

Re: [PR] Support partial deletes [iceberg-python]

2024-07-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1666875275 ## pyiceberg/table/__init__.py: ## @@ -237,6 +245,10 @@ class TableProperties: WRITE_PARTITION_SUMMARY_LIMIT = "write.summary.partition-limit"

Re: [PR] Support partial deletes [iceberg-python]

2024-07-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1666868594 ## pyiceberg/table/__init__.py: ## @@ -454,6 +482,74 @@ def overwrite( for data_file in data_files:

Re: [PR] Support partial deletes [iceberg-python]

2024-06-12 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1637353561 ## pyiceberg/table/__init__.py: ## @@ -2931,14 +3161,52 @@ def _deleted_entries(self) -> List[ManifestEntry]: return [] -class

Re: [PR] Support partial deletes [iceberg-python]

2024-06-12 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1637353561 ## pyiceberg/table/__init__.py: ## @@ -2931,14 +3161,52 @@ def _deleted_entries(self) -> List[ManifestEntry]: return [] -class

Re: [PR] Support partial deletes [iceberg-python]

2024-06-12 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1637347402 ## pyiceberg/table/__init__.py: ## @@ -2931,14 +3161,52 @@ def _deleted_entries(self) -> List[ManifestEntry]: return [] -class

Re: [PR] Support partial deletes [iceberg-python]

2024-06-12 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1637279841 ## pyiceberg/table/__init__.py: ## @@ -454,6 +482,74 @@ def overwrite( for data_file in data_files:

Re: [PR] Support partial deletes [iceberg-python]

2024-06-12 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1637279841 ## pyiceberg/table/__init__.py: ## @@ -454,6 +482,74 @@ def overwrite( for data_file in data_files:

Re: [PR] Support partial deletes [iceberg-python]

2024-06-12 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1637279841 ## pyiceberg/table/__init__.py: ## @@ -454,6 +482,74 @@ def overwrite( for data_file in data_files:

Re: [PR] Support partial deletes [iceberg-python]

2024-06-12 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1637279841 ## pyiceberg/table/__init__.py: ## @@ -454,6 +482,74 @@ def overwrite( for data_file in data_files:

Re: [PR] Support partial deletes [iceberg-python]

2024-06-09 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1632276771 ## pyiceberg/table/__init__.py: ## @@ -434,6 +456,9 @@ def overwrite( if table_arrow_schema != df.schema: df = df.cast(table_arrow_schema)

Re: [PR] Support partial deletes [iceberg-python]

2024-05-27 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1615942072 ## pyiceberg/table/__init__.py: ## @@ -2897,12 +2987,152 @@ def _commit(self) -> UpdatesAndRequirements: ), (

Re: [PR] Support partial deletes [iceberg-python]

2024-05-27 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1615924919 ## pyiceberg/table/__init__.py: ## @@ -2931,14 +3161,52 @@ def _deleted_entries(self) -> List[ManifestEntry]: return [] -class

Re: [PR] Support partial deletes [iceberg-python]

2024-05-27 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1615924919 ## pyiceberg/table/__init__.py: ## @@ -2931,14 +3161,52 @@ def _deleted_entries(self) -> List[ManifestEntry]: return [] -class

Re: [PR] Support partial deletes [iceberg-python]

2024-05-27 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1615912422 ## pyiceberg/table/__init__.py: ## @@ -443,6 +471,74 @@ def overwrite( for data_file in data_files:

Re: [PR] Support partial deletes [iceberg-python]

2024-05-13 Thread via GitHub
jqin61 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1599183250 ## pyiceberg/table/__init__.py: ## @@ -2897,12 +2987,152 @@ def _commit(self) -> UpdatesAndRequirements: ), (

Re: [PR] Support partial deletes [iceberg-python]

2024-05-13 Thread via GitHub
jqin61 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1599183250 ## pyiceberg/table/__init__.py: ## @@ -2897,12 +2987,152 @@ def _commit(self) -> UpdatesAndRequirements: ), (

Re: [PR] Support partial deletes [iceberg-python]

2024-05-13 Thread via GitHub
jqin61 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1599183250 ## pyiceberg/table/__init__.py: ## @@ -2897,12 +2987,152 @@ def _commit(self) -> UpdatesAndRequirements: ), (

Re: [PR] Support partial deletes [iceberg-python]

2024-05-13 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1598823358 ## pyiceberg/table/__init__.py: ## @@ -2931,14 +3161,52 @@ def _deleted_entries(self) -> List[ManifestEntry]: return [] -class

Re: [PR] Support partial deletes [iceberg-python]

2024-05-13 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1598626280 ## pyiceberg/table/__init__.py: ## @@ -443,6 +471,74 @@ def overwrite( for data_file in data_files:

Re: [PR] Support partial deletes [iceberg-python]

2024-05-10 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1597144084 ## pyiceberg/table/__init__.py: ## @@ -434,6 +460,8 @@ def overwrite( if table_arrow_schema != df.schema: df = df.cast(table_arrow_schema) +

Re: [PR] Support partial deletes [iceberg-python]

2024-05-10 Thread via GitHub
jqin61 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1597063514 ## pyiceberg/table/__init__.py: ## @@ -434,6 +460,8 @@ def overwrite( if table_arrow_schema != df.schema: df = df.cast(table_arrow_schema)

Re: [PR] Support partial deletes [iceberg-python]

2024-05-08 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1594704486 ## pyiceberg/table/__init__.py: ## @@ -434,6 +458,9 @@ def overwrite( if table_arrow_schema != df.schema: df = df.cast(table_arrow_schema) +

Re: [PR] Support partial deletes [iceberg-python]

2024-05-08 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1594703680 ## pyiceberg/table/__init__.py: ## @@ -434,6 +458,9 @@ def overwrite( if table_arrow_schema != df.schema: df = df.cast(table_arrow_schema) +

Re: [PR] Support partial deletes [iceberg-python]

2024-05-08 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1594058803 ## tests/integration/test_deletes.py: ## @@ -0,0 +1,257 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

Re: [PR] Support partial deletes [iceberg-python]

2024-05-08 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1594057885 ## pyiceberg/table/__init__.py: ## @@ -2897,12 +2959,161 @@ def _commit(self) -> UpdatesAndRequirements: ), (

Re: [PR] Support partial deletes [iceberg-python]

2024-05-08 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1594055639 ## pyiceberg/table/__init__.py: ## @@ -443,6 +468,54 @@ def overwrite( for data_file in data_files:

Re: [PR] Support partial deletes [iceberg-python]

2024-05-08 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1594053750 ## pyiceberg/table/__init__.py: ## @@ -443,6 +468,54 @@ def overwrite( for data_file in data_files:

Re: [PR] Support partial deletes [iceberg-python]

2024-05-08 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1594048917 ## pyiceberg/table/__init__.py: ## @@ -292,7 +303,13 @@ def _apply(self, updates: Tuple[TableUpdate, ...], requirements: Tuple[TableRequ

Re: [PR] Support partial deletes [iceberg-python]

2024-05-04 Thread via GitHub
HonahX commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1584249768 ## pyiceberg/table/__init__.py: ## @@ -443,6 +468,54 @@ def overwrite( for data_file in data_files:

Re: [PR] Support partial deletes [iceberg-python]

2024-05-02 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1588290938 ## pyiceberg/table/__init__.py: ## @@ -434,6 +456,9 @@ def overwrite( if table_arrow_schema != df.schema: df = df.cast(table_arrow_schema) +

Re: [PR] Support partial deletes [iceberg-python]

2024-05-02 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1588286537 ## pyiceberg/table/__init__.py: ## @@ -292,7 +303,13 @@ def _apply(self, updates: Tuple[TableUpdate, ...], requirements: Tuple[TableRequ

Re: [PR] Support partial deletes [iceberg-python]

2024-05-02 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1588266506 ## pyiceberg/table/__init__.py: ## @@ -235,6 +242,10 @@ class TableProperties: WRITE_PARTITION_SUMMARY_LIMIT = "write.summary.partition-limit"

Re: [PR] Support partial deletes [iceberg-python]

2024-04-26 Thread via GitHub
kevinjqliu commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1581278513 ## pyiceberg/table/__init__.py: ## @@ -434,6 +456,9 @@ def overwrite( if table_arrow_schema != df.schema: df =

Re: [PR] Support partial deletes [iceberg-python]

2024-04-25 Thread via GitHub
MehulBatra commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1580084812 ## pyiceberg/table/__init__.py: ## @@ -235,6 +242,10 @@ class TableProperties: WRITE_PARTITION_SUMMARY_LIMIT = "write.summary.partition-limit"

Re: [PR] Support partial deletes [iceberg-python]

2024-04-25 Thread via GitHub
MehulBatra commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1580084812 ## pyiceberg/table/__init__.py: ## @@ -235,6 +242,10 @@ class TableProperties: WRITE_PARTITION_SUMMARY_LIMIT = "write.summary.partition-limit"

Re: [PR] Support partial deletes [iceberg-python]

2024-04-25 Thread via GitHub
MehulBatra commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1580084812 ## pyiceberg/table/__init__.py: ## @@ -235,6 +242,10 @@ class TableProperties: WRITE_PARTITION_SUMMARY_LIMIT = "write.summary.partition-limit"

Re: [PR] Support partial deletes [iceberg-python]

2024-04-16 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1566870388 ## pyiceberg/io/pyarrow.py: ## @@ -1912,3 +1920,55 @@ def _get_parquet_writer_kwargs(table_properties: Properties) -> Dict[str, Any]: