GitHub user vinothchandar added a comment to the discussion: Native Protobuf Record Payload
@gudladona So you want to shuffle proto bytes all the way to storage? >My thinking being we could accomplish the same with proto bytes If we are thinking Spark-centric, then it may be better to convert to `Row` upfront and let Spark optimize it. (for e.g may be down the line, it gets columnar shuffle) > We could have only 1 pass of Proto to avro conversion That approach will have 1 pass to Row. like today, where we convert 1 pass to avro. GitHub link: https://github.com/apache/hudi/discussions/13867#discussioncomment-14408497 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
