Re: [PR] Incremental Append Scan [iceberg-python]

glesperance commented on PR #533: URL: https://github.com/apache/iceberg-python/pull/533#issuecomment-2248379289 I managed to get a poor mans append-scan with this https://github.com/apache/iceberg-python/issues/240#issuecomment-2248323987 Looking at this PR wouldn't it be simpler to

Re: [PR] Incremental Append Scan [iceberg-python]

chinmay-bhat commented on code in PR #533: URL: https://github.com/apache/iceberg-python/pull/533#discussion_r1628643760 ## pyiceberg/table/__init__.py: ## @@ -1754,6 +1788,134 @@ def to_arrow(self) -> pa.Table: def to_pandas(self, **kwargs: Any) -> pd.DataFrame:

Re: [PR] Incremental Append Scan [iceberg-python]

chinmay-bhat commented on code in PR #533: URL: https://github.com/apache/iceberg-python/pull/533#discussion_r1628636085 ## pyiceberg/table/__init__.py: ## @@ -1754,6 +1788,134 @@ def to_arrow(self) -> pa.Table: def to_pandas(self, **kwargs: Any) -> pd.DataFrame:

Re: [PR] Incremental Append Scan [iceberg-python]

hililiwei commented on code in PR #533: URL: https://github.com/apache/iceberg-python/pull/533#discussion_r1626843860 ## pyiceberg/table/__init__.py: ## @@ -1754,6 +1788,134 @@ def to_arrow(self) -> pa.Table: def to_pandas(self, **kwargs: Any) -> pd.DataFrame:

Re: [PR] Incremental Append Scan [iceberg-python]

hililiwei commented on code in PR #533: URL: https://github.com/apache/iceberg-python/pull/533#discussion_r1626816089 ## pyiceberg/table/__init__.py: ## @@ -1754,6 +1788,134 @@ def to_arrow(self) -> pa.Table: def to_pandas(self, **kwargs: Any) -> pd.DataFrame:

Re: [PR] Incremental Append Scan [iceberg-python]

hililiwei commented on code in PR #533: URL: https://github.com/apache/iceberg-python/pull/533#discussion_r1626799873 ## pyiceberg/table/__init__.py: ## @@ -1754,6 +1788,134 @@ def to_arrow(self) -> pa.Table: def to_pandas(self, **kwargs: Any) -> pd.DataFrame:

Re: [PR] Incremental Append Scan [iceberg-python]

chinmay-bhat commented on code in PR #533: URL: https://github.com/apache/iceberg-python/pull/533#discussion_r1623258053 ## pyiceberg/table/__init__.py: ## @@ -1754,6 +1788,134 @@ def to_arrow(self) -> pa.Table: def to_pandas(self, **kwargs: Any) -> pd.DataFrame:

Re: [PR] Incremental Append Scan [iceberg-python]

chinmay-bhat commented on code in PR #533: URL: https://github.com/apache/iceberg-python/pull/533#discussion_r1623258053 ## pyiceberg/table/__init__.py: ## @@ -1754,6 +1788,134 @@ def to_arrow(self) -> pa.Table: def to_pandas(self, **kwargs: Any) -> pd.DataFrame:

Re: [PR] Incremental Append Scan [iceberg-python]

chinmay-bhat commented on code in PR #533: URL: https://github.com/apache/iceberg-python/pull/533#discussion_r1623258053 ## pyiceberg/table/__init__.py: ## @@ -1754,6 +1788,134 @@ def to_arrow(self) -> pa.Table: def to_pandas(self, **kwargs: Any) -> pd.DataFrame:

Re: [PR] Incremental Append Scan [iceberg-python]

chinmay-bhat commented on code in PR #533: URL: https://github.com/apache/iceberg-python/pull/533#discussion_r1620353853 ## pyiceberg/table/__init__.py: ## @@ -1754,6 +1788,134 @@ def to_arrow(self) -> pa.Table: def to_pandas(self, **kwargs: Any) -> pd.DataFrame:

Re: [PR] Incremental Append Scan [iceberg-python]

chinmay-bhat commented on code in PR #533: URL: https://github.com/apache/iceberg-python/pull/533#discussion_r1620357567 ## pyiceberg/table/__init__.py: ## @@ -1754,6 +1788,134 @@ def to_arrow(self) -> pa.Table: def to_pandas(self, **kwargs: Any) -> pd.DataFrame:

Re: [PR] Incremental Append Scan [iceberg-python]

chinmay-bhat commented on code in PR #533: URL: https://github.com/apache/iceberg-python/pull/533#discussion_r1620353853 ## pyiceberg/table/__init__.py: ## @@ -1754,6 +1788,134 @@ def to_arrow(self) -> pa.Table: def to_pandas(self, **kwargs: Any) -> pd.DataFrame:

Re: [PR] Incremental Append Scan [iceberg-python]

Fokko commented on PR #533: URL: https://github.com/apache/iceberg-python/pull/533#issuecomment-2126662798 @hililiwei I'm sorry, this also fell off my radar. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Incremental Append Scan [iceberg-python]

hililiwei commented on PR #533: URL: https://github.com/apache/iceberg-python/pull/533#issuecomment-2084982599 Sorry for the late correction. I've adjusted the code based on the latest comments. Could you please take a look? -- This is an automated message from the Apache Git Service.

Re: [PR] Incremental Append Scan [iceberg-python]

hililiwei commented on code in PR #533: URL: https://github.com/apache/iceberg-python/pull/533#discussion_r1584569269 ## pyiceberg/table/__init__.py: ## @@ -1594,6 +1617,197 @@ def to_ray(self) -> ray.data.dataset.Dataset: return ray.data.from_arrow(self.to_arrow())

Re: [PR] Incremental Append Scan [iceberg-python]

Fokko commented on code in PR #533: URL: https://github.com/apache/iceberg-python/pull/533#discussion_r1568698784 ## pyiceberg/table/__init__.py: ## @@ -1594,6 +1617,197 @@ def to_ray(self) -> ray.data.dataset.Dataset: return ray.data.from_arrow(self.to_arrow())

Re: [PR] Incremental Append Scan [iceberg-python]

hililiwei commented on code in PR #533: URL: https://github.com/apache/iceberg-python/pull/533#discussion_r1530185804 ## pyiceberg/table/__init__.py: ## @@ -3014,3 +3145,35 @@ def _new_field_id(self) -> int: def _is_duplicate_partition(self, transform: Transform[Any,

Re: [PR] Incremental Append Scan [iceberg-python]

hililiwei commented on code in PR #533: URL: https://github.com/apache/iceberg-python/pull/533#discussion_r1530184252 ## pyiceberg/table/__init__.py: ## @@ -1578,6 +1595,120 @@ def to_ray(self) -> ray.data.dataset.Dataset: return ray.data.from_arrow(self.to_arrow())

[PR] Incremental Append Scan [iceberg-python]

hililiwei opened a new pull request, #533: URL: https://github.com/apache/iceberg-python/pull/533 Hi，@Fokko， long time no see. I have written some preliminary code for incremental reading, which still has many areas for improvement. However, I would like to discuss it with you at an early