umehrot2 commented on issue #1290: [HUDI-584] Relocate spark-avro dependency by maven-shade-plugin URL: https://github.com/apache/incubator-hudi/pull/1290#issuecomment-582650497 As @vinothchandar mentioned we did discuss the possibility of doing this, but were thrown back because of the reason mentioned above by Vinoth. But just thinking out aloud, now that Hudi has been migrated to `spark 2.4.4` is it really recommended for user's using older or newer versions of spark version to be simply dropping the pre-built hudi jars with spark 2.4.4 ? Shouldn't the recommendation for such users be to actually build their own hudi jars, with the spark version they use. This would help them catch any possible compatibility issues during compile time. I am not sure how safe it is for us to claim that user's use for our pre-built jars regardless of spark versions. And if building own jars is the recommendation, then i think this change can be pulled in.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
