deniskuzZ commented on PR #11216:
URL: https://github.com/apache/iceberg/pull/11216#issuecomment-2579758121
hi @ajantha-bhat, could you please check below:
it seems that Date, time, timestamp partition values are not properly
serialized
see Type.TypeID
````
DATE(Integer.class),
TIME(Long.class),
TIMESTAMP(Long.class),
````
PartitionSpec.partitionToPath(PartitionStatsRecord.unwrap().partition())
thows an exception
````
Caused by: java.lang.IllegalStateException: Not an instance of
java.lang.Integer: 1999-12-12
at org.apache.iceberg.data.GenericRecord.get(GenericRecord.java:123)
at org.apache.iceberg.PartitionSpec.get(PartitionSpec.java:195)
at
org.apache.iceberg.PartitionSpec.partitionToPath(PartitionSpec.java:213)
````
I think, instead of Record(1999-12-12) it should be Record{10956}
full code snippet
````
Types.StructType partitionType = Partitioning.partitionType(table);
Schema schema = PartitionStatsHandler.schema(partitionType);
CloseableIterable<PartitionStatsRecord> partitionStatsRecords =
PartitionStatsHandler.readPartitionStatsFile(
schema, table.io().newInputFile(statsFile.path()));
try (Closeable toClose = partitionStatsRecords) {
PartitionStats partitionStats = Iterables.tryFind(partitionStatsRecords,
stats -> {
PartitionSpec spec = table.specs().get(stats.unwrap().specId());
return
spec.partitionToPath(stats.unwrap().partition()).equals(partish.getPartition().getName());
})
.transform(PartitionStatsRecord::unwrap)
.orNull();
if (partitionStats != null) {
Map<String, String> stats = ImmutableMap.of(
TOTAL_DATA_FILES_PROP,
String.valueOf(partitionStats.dataFileCount()),
TOTAL_RECORDS_PROP, String.valueOf(partitionStats.dataRecordCount()),
TOTAL_EQ_DELETES_PROP,
String.valueOf(partitionStats.equalityDeleteRecordCount()),
TOTAL_POS_DELETES_PROP,
String.valueOf(partitionStats.positionDeleteRecordCount()),
TOTAL_FILE_SIZE_PROP,
String.valueOf(partitionStats.totalDataFileSizeInBytes())
);
return stats;
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
