[ https://issues.apache.org/jira/browse/HUDI-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhaojing Yu updated HUDI-4036: ------------------------------ Fix Version/s: 0.13.0 (was: 0.12.1) > Investigate whether meta fields could be omitted completely > ----------------------------------------------------------- > > Key: HUDI-4036 > URL: https://issues.apache.org/jira/browse/HUDI-4036 > Project: Apache Hudi > Issue Type: Task > Components: writer-core > Reporter: Alexey Kudinkin > Assignee: Alexey Kudinkin > Priority: Critical > Fix For: 0.13.0 > > > Currently, even when meta fields are not populated, we still insert > empty-string columns to adhere to the expected schema. > This has a non-trivial overhead of ~20% (relative to just writing dataset as > is), since Spark had to essentially "re-write" the original row with > prepended new fields. > We should investigate whether it's feasible to avoid adding empty-string > columns completely if meta-fields are disabled. -- This message was sent by Atlassian Jira (v8.20.10#820010)