maslennikov commented on issue #2152:
URL:
https://github.com/apache/iceberg-python/issues/2152#issuecomment-3745393760
> You can currently achieve this by something like...
>
> for batch in my_batch_iterator:
> # would need to check schema here
> table.append(pa.Table.from_batches([batch]))
>
> Not saying that this isnt a feature that would be nice but the ability to
make it happen is already possible
Still would be nice to have a proper streaming / batch reader support out of
the box with added benefit of correct transactionality and overwrite semantics.
```py
with my_table.transaction() as tx:
# imitate overwrite logic by truncating the table first
tx.delete(delete_filter=AlwaysTrue())
for batch in batch_reader:
tx.append(pa.Table.from_batches([batch]))
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
