Hey Team, I have created a PR for RFC claim for DataFrame implementation for HUDI write path.
Background: Today, HUDI write path is majorly implemented using RDD APIs for Apache Spark and RDD APIs are not supported by accelerators like Spark-Rapids, Velox-Gluten etc. Having a DataFrame implementation will also speed-up overall execution compared to RDD computations. This RFC will help in designing new writer paths which will rely on DataFrame APIs rather than using RDDs. -- Thanks and Regards, Jalpesh Borad +91 8140500542
