alexandreyc commented on PR #13310: URL: https://github.com/apache/arrow/pull/13310#issuecomment-1148146775
Yes, it doesn't look like there is an equivalent Parquet type. Nonetheless, it seems that decision to convert Arrow's `Null` to Parquet's `Int32` has already been made in the code base. See [here](https://github.com/apache/arrow/blob/8c63788ff7d52812599a546989b7df10887cb01e/go/parquet/pqarrow/schema.go#L259). The use case is quite simple: in real world databases there are columns which are full of `NULL` values and I guess that's why Arrow has `Null`. Some might say that's a bad practice but it exists... I don't see any reason why we should not allow converting record batches with `Null` arrays to Parquet. What do you think? In the mean time I will look into others implementations to see what's being done here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
