[GitHub] [hudi] zherenyu831 commented on issue #1798: Question reading partition path with less level is more faster than what document mentioned

2020-07-12 Thread GitBox
zherenyu831 commented on issue #1798: URL: https://github.com/apache/hudi/issues/1798#issuecomment-657193426 @umehrot2 @vinothchandar Sorry for lately reply. Here is my snapshot of spark ui. First query I used, files processed by resolveRelation was 950 ``` spark.read.fo

[GitHub] [hudi] zherenyu831 commented on issue #1798: Question reading partition path with less level is more faster than what document mentioned

2020-07-06 Thread GitBox
zherenyu831 commented on issue #1798: URL: https://github.com/apache/hudi/issues/1798#issuecomment-654575639 @bhasudha It is very simple query ``` val df = spark.read.format("org.apache.hudi").load("s3://test/data/*/*/*") val updatedDf = df.filter("_hoodie_commit_time between