maropu commented on a change in pull request #28672: URL: https://github.com/apache/spark/pull/28672#discussion_r432315161
########## File path: docs/sql-ref-syntax-qry-select-hints.md ########## @@ -21,14 +21,69 @@ license: | ### Description -Join Hints allow users to suggest the join strategy that Spark should use. Prior to Spark 3.0, only the `BROADCAST` Join Hint was supported. `MERGE`, `SHUFFLE_HASH` and `SHUFFLE_REPLICATE_NL` Joint Hints support was added in 3.0. When different join strategy hints are specified on both sides of a join, Spark prioritizes hints in the following order: `BROADCAST` over `MERGE` over `SHUFFLE_HASH` over `SHUFFLE_REPLICATE_NL`. When both sides are specified with the `BROADCAST` hint or the `SHUFFLE_HASH` hint, Spark will pick the build side based on the join type and the sizes of the relations. Since a given strategy may not support all join types, Spark is not guaranteed to use the join strategy suggested by the hint. +Hints give users a way to suggest how Spark SQL to use specific approaches to generate its execution plan. ### Syntax ```sql -/*+ join_hint [ , ... ] */ +/*+ hint [ , ... ] */ ``` +### Partitioning Hints + +`COALESCE`/`REPARTITION`/`REPARTITION_BY_RANGE` hints have functionalities equivalent to those of the +`Dataset` `coalesce`/`repartition`/`repartitionByRange` APIs. The `COALESCE` hint can be used to reduce +the number of partitions to the specified number of partitions. The `REPARTITION`/`REPARTITION_BY_RANGE` +hint can be used to repartition to the specified number of partitions using the specified partitioning expressions. +The `COALESCE` hint takes a partition number as a +parameter. The `REPARTITION` hint takes a partition number, column names, or both as parameters. +The `REPARTITION_BY_RANGE` hint takes column names and an optional partition number as parameters. +These hints give users a way to tune performance and control the number of output files in Spark SQL. + +### Examples +```sql +SELECT /*+ COALESCE(3) */ * FROM t; + +EXPLAIN SELECT /*+ COALESCE(3) */ * FROM t; +== Physical Plan == +Coalesce 3 ++- *(1) ColumnarToRow + +- FileScan parquet default.t[name#5,c#6] Batched: true, DataFilters: [], Format: Parquet, + Location: CatalogFileIndex[file:/spark/spark-warehouse/t], PartitionFilters: [], + PushedFilters: [], ReadSchema: struct<name:string> + +SELECT /*+ REPARTITION(3) */ * FROM t; Review comment: One more comment; probably, the join hint section should have the same format for the examples. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org