databricks-david-lewis commented on PR #39488: URL: https://github.com/apache/spark/pull/39488#issuecomment-1383440531
Thanks for the comment @JoshRosen! You put it very well. One of the other places I wanted to use `SparkPath` was in the streaming interfaces (`FileStreamSink`, `FileStreamSource`, `SinkFileStatus`, and maybe more). I also wanted to standardize on the currently expected behavior of users providing hadoop-path-strings in SQL statements (table locations, SQLOnFiles, DataFrame-Reader/-Writer, etc). Right now they all (I think) end up in a new Path(_), but could benefit from type-safety. @mridulm , this PR is ready for review I think. Please take a look. I think the sooner we can get it into a Spark minor version the better. Even if that means we are incrementally changing the internal interfaces in subsequent minor versions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
