[
https://issues.apache.org/jira/browse/ARROW-16761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Antoine Pitrou resolved ARROW-16761.
------------------------------------
Resolution: Fixed
Issue resolved by pull request 13338
[https://github.com/apache/arrow/pull/13338]
> [C++][Python] Track bytes_written on FileWriter / WrittenFile
> -------------------------------------------------------------
>
> Key: ARROW-16761
> URL: https://issues.apache.org/jira/browse/ARROW-16761
> Project: Apache Arrow
> Issue Type: New Feature
> Components: C++, Python
> Affects Versions: 8.0.0
> Reporter: Will Jones
> Assignee: Will Jones
> Priority: Major
> Labels: pull-request-available
> Fix For: 9.0.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> For Apache Iceberg and Delta Lake tables, we need to be able to get the size
> of the files written in bytes. In Iceberg, this is the required field
> {{file_size_in_bytes}} ([docs|https://iceberg.apache.org/spec/#manifests]).
> In Delta, this is the required field {{size}} as part of the Add action.
> I think this could be exposed on
> [FileWriter|https://github.com/apache/arrow/blob/8c63788ff7d52812599a546989b7df10887cb01e/cpp/src/arrow/dataset/file_base.h#L305]
> and then through that
> [WrittenFile|https://github.com/apache/arrow/blob/8c63788ff7d52812599a546989b7df10887cb01e/python/pyarrow/_dataset.pyx#L766-L769].
> But lower-level than that I'm not yet sure. {{FileWriter}} owns its
> {{OutputStream}}; would {{OutputStream::Tell()}} give the correct count?
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
