I'm writing a PostgreSQL dump file to parquet converter as a hobby project.
PostgreSQL dump files are simply plain-text files containing, as the man page
puts it "the SQL commands required to reconstruct the database to the state it
was in at the time it was saved." Null SQL values are represented by "\N".
I have implemented a fairly efficient line-wise dump file reader which includes
a rudimentary table typer. However I'm not sure how to represent nulls in my go
code so that I can pass these onto parquet encoders.
Is a struct something like the following a good idea for each type?
type pgInt64 struct {
val int64
isNull bool
}
(This somewhat mirrors the sort of sidecar "validity" design of Arrow and
definition levels of Parquet as described at
https://arrow.apache.org/blog/2022/10/05/arrow-parquet-encoding-part-1/).
Simple tests encoding parquet format files with github.com/xitongsys/parquet-go
and github.com/fraugster/parquet-go work ok. However I'm equally unsure how to
pass these nulls.
Thanks for any insights,
Rory
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/Y82i7BLna9/1PJHL%40campbell-lange.net.
