GitHub user bersprockets opened a pull request:
https://github.com/apache/spark/pull/22865
[DOC] Fix doc for spark.sql.parquet.recordLevelFilter.enabled
## What changes were proposed in this pull request?
Updated the doc string value for
spark.sql.parquet.recordLevelFilter.enabled to indicate that
spark.sql.parquet.enableVectorizedReader must be disabled.
The code in ParquetFileFormat uses
spark.sql.parquet.recordLevelFilter.enabled only after falling back to
parquet-mr (see else for this if statement):
https://github.com/apache/spark/blob/d5573c578a1eea9ee04886d9df37c7178e67bb30/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala#L412
https://github.com/apache/spark/blob/d5573c578a1eea9ee04886d9df37c7178e67bb30/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala#L427-L430
Tests also bear this out.
## How was this patch tested?
This is just a doc string fix: I built Spark and ran a single test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bersprockets/spark confdocfix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22865.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22865
----
commit af8a85ae4a1e477801bf104af6d4909cd822ba01
Author: Bruce Robbins <bersprockets@...>
Date: 2018-10-27T21:47:50Z
update doc string for spark.sql.parquet.recordLevelFilter.enabled
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
