andygrove opened a new issue, #55: URL: https://github.com/apache/datafusion-spark/issues/55
**Problem** docs.rs for `datafusion-spark` lists functions with little or no description. Users can't tell how each function differs from the DataFusion built-in of the same name, which is the main reason to use the crate. Today there are 87 UDF implementations. Many have a struct doc comment that links to the Spark docs, but only 1 implements `documentation()`. So there are no per-function descriptions, argument lists, or examples, and very few places describe how Spark's behavior differs from DataFusion's. **Goal** Every UDF documents, in its rustdoc and via `documentation()`: 1. A one-line description and the Spark function it emulates, with a link to the Spark docs. 2. Syntax, arguments, and return type. 3. **Spark-specific behavior**: NULL handling, overflow, ANSI-mode differences, indexing, rounding, supported input types, and known deviations from Spark. 4. At least one SQL example. **Plan** - [ ] Define a doc template and a reference example (e.g. `sha2`, `substring`) in the contributing guidelines - [ ] Add a CI check (a test that every registered UDF has `documentation()`) - [ ] Per-package sub-issues (good first issues): aggregate, array, bitmap, bitwise, collection, conditional, conversion, datetime, generator, hash, json, map, math, misc, string, url - [ ] Decide whether to generate a function reference page from `documentation()`, as DataFusion core does **Related** - https://github.com/apache/datafusion/issues/15914 (the `datafusion-spark` epic) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
