jqin61 opened a new issue, #398:
URL: https://github.com/apache/iceberg-python/issues/398
### Question
Hi I was trying to understand whether there is any special design initiative
for:
1. The Python representation of partition of datetime-related type is
integer. For example:
```
iceberg_table.current_snapshot().manifests(iceberg_table.io)[0].fetch_manifest_entry(iceberg_table.io)[0].data_file.partition
```
would show DataFile instance having partition as
```
Record[SNAPSHOT_DATE=18628]
```
Since DataFile class is a Python object, would it make more sense to
represent it using native Python date/datetime?
2. As a result, conversion between date/timestamp(tz) and integer stays
outside of avro reader/writer. For example, we have the source code of :
```
@dataclass(frozen=True)
class DateWriter(Writer):
def write(self, encoder: BinaryEncoder, val: int) -> None:
encoder.write_int(val)
```
which can only take int as input.
I asked because I was trying to implement partition write support and during
writing the manifest list, I instantiated the DataFile using partition =
Record(field=raw date-typed Python object) which breaks the avro writer for
writing the manifest file. So I wonder whether I should 1. do the conversion
from date -> into before instantiating the DataFile's partition field or maybe
2.change the avro DateWriter to handle the date-integer conversion (and do this
for avro reader as well).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
