Hey Gus! I missed this when you first sent it, but I think it's a great idea. I'm currently wrapping up at a conference right now, but I'm gonna take a look at your attempt at implementing when I get a chance and respond with some feedback early next week.
Take care and sorry for the delayed response! --Matt On Thu, Mar 23, 2023 at 1:00 PM Gus Minto-Cowcher <[email protected]> wrote: > Hi All, > > I took a stab at implementing something like this here: > https://gist.github.com/gmintoco/3e65aa7b47ae37b0685db88b2755933f > > I would appreciate any feedback or better approaches people might have :) > > Thanks! > Gus > > On Thu, Mar 16, 2023 at 6:13 PM Gus Minto-Cowcher <[email protected]> > wrote: > >> Hi all, >> >> Hoping for some advice here. If I have a very large struct (50+fields) >> that I would like to represent using Arrow and then write out as a Parquet >> file. Currently, I would use schema.NewSchemaFromStruct ( >> github.com/apache/arrow/go/v11/parquet/schema) to automatically generate >> a schema from the Struct and then build a record manually using the record >> builder. >> like: >> >> builder := array.NewRecordBuilder(pool, schema) >> >> for _, struct := range structs { >> builder.Field(0).(*array.BinaryBuilder).AppendString("test") >> builder.Field(1).(*array.BinaryBuilder).AppendString("test") >> builder.Field(2).(*array.BinaryBuilder).AppendString("test") >> builder.Field(..).(*array.BinaryBuilder).AppendString("test") >> } >> >> is there instead some way of writing a struct directly to a record or I >> am missing something? It stands to reason if the schema can be inferred it >> shouldn't be a problem to also create a record directly from the struct, >> instead of building it manually. >> >> like: >> >> builder.AppendStructs(structs) >> >> It seems very tedious to have to write out the entire struct essentially >> with the type assertions etc. I am hoping someone has come up with an >> easier way to do this :) >> >> Thanks for any insights people might have, >> Gus >> >
