[ https://issues.apache.org/jira/browse/HUDI-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
sivabalan narayanan updated HUDI-4036: -------------------------------------- Sprint: 2022/09/05 (was: 2022/08/22) > Investigate whether meta fields could be omitted completely > ----------------------------------------------------------- > > Key: HUDI-4036 > URL: https://issues.apache.org/jira/browse/HUDI-4036 > Project: Apache Hudi > Issue Type: Task > Components: writer-core > Reporter: Alexey Kudinkin > Assignee: Alexey Kudinkin > Priority: Critical > Fix For: 0.12.1 > > > Currently, even when meta fields are not populated, we still insert > empty-string columns to adhere to the expected schema. > This has a non-trivial overhead of ~20% (relative to just writing dataset as > is), since Spark had to essentially "re-write" the original row with > prepended new fields. > We should investigate whether it's feasible to avoid adding empty-string > columns completely if meta-fields are disabled. -- This message was sent by Atlassian Jira (v8.20.10#820010)