[
https://issues.apache.org/jira/browse/HUDI-1745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan resolved HUDI-1745.
---------------------------------------
Resolution: Fixed
> Hudi compilation fails w/ spark version < 2.4.4 due to usage of unavailable
> spark api
> -------------------------------------------------------------------------------------
>
> Key: HUDI-1745
> URL: https://issues.apache.org/jira/browse/HUDI-1745
> Project: Apache Hudi
> Issue Type: Bug
> Components: Spark Integration
> Affects Versions: 0.8.0
> Reporter: sivabalan narayanan
> Priority: Major
> Labels: sev:critical
>
> [https://github.com/apache/hudi/issues/2748]
>
> PR thats of interest: [https://github.com/apache/hudi/pull/2431]
>
> I see we have three options. Let me know if we have more.
> Option1:
> Similar to SparkRowSerDe, we might have to introduce an interface for
> translateSqlOptions and override based on spark versions. But already we have
> two sub modules for spark2 and spark3. and now we might have to add more such
> modules for diff spark2 versions which might need more thought to do it
> elegantly.
> Option2:
> Since this feature is added only w/ 0.8.0, and not like a more sought after
> feature, we could revert this commit and unblock ourselves for 0.8.0. Once
> release is complete, we can decide how to do about doing this and get this
> feature in for next release.
> Option3:
> we say that hudi does not support spark version < 2.4.4 w/ 0.8.0. Don't think
> we can go this route. But just listing it out.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
