[GitHub] [hudi] garyli1019 commented on pull request #2243: HUDI-1392 lose partition info when using spark parameter basePath

2020-11-24 Thread GitBox 


garyli1019 commented on pull request #2243:
URL: https://github.com/apache/hudi/pull/2243#issuecomment-733445977


   @yui2010 merging. Please assign the Jira ticket to yourself and close it. If 
you don't have contributor access yet, please send an email with your Jira ID 
to the dev mailing list and someone will add you to the project. Thanks!



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hudi] garyli1019 commented on pull request #2243: HUDI-1392 lose partition info when using spark parameter basePath

2020-11-23 Thread GitBox 


garyli1019 commented on pull request #2243:
URL: https://github.com/apache/hudi/pull/2243#issuecomment-732175563


   Hi @yui2010 , sorry I am not quite sure if I understand correctly.
   So the problem here is Spark with version lower than 2.4.7 will use 
Map.get("basePath") but the actual key in the Map has been converted to 
"basepath". Therefore we need to somehow force Spark to use 
CaseInsensitiveMap.get("basePath"), and then we will get the correct result. Is 
that correct?
   If so, how can we do that on the Hudi side?
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org