nchammas opened a new pull request, #44755:
URL: https://github.com/apache/spark/pull/44755
### What changes were proposed in this pull request?
Enable Spark configs to be assigned to documentation groups. These groups
will be used to automatically build config tables for display in our
documentation.
Instead of having to maintain [large blocks of HTML tables][1] throughout
our documentation, config tables can simply be included as follows:
```liquid
{% include_api_gen _generated/config_tables/sql-tuning-caching-data.html %}
```
This is a stripped down version of #44300 to facilitate review of the core
idea.
[1]:
https://github.com/apache/spark/blob/7db85642600b1e3b39ca11e41d4e3e0bf1c8962b/docs/sql-performance-tuning.md?plain=1#L37-L56
### Why are the changes needed?
Using this approach we can accomplish several goals at once:
- Eliminate thousands of lines of manually maintained HTML tables of Spark
configs.
- Ensure that internal configs are not accidentally documented publicly.
(e.g. `spark.sql.files.openCostInBytes`)
- Ensure that configs are documented publicly exactly as they are in the
code. (e.g. `spark.sql.autoBroadcastJoinThreshold`)
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
No testing beyond what was already done in #44300, pending review.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
