viirya commented on PR #53:
URL:
https://github.com/apache/spark-connect-rust/pull/53#issuecomment-5414601137
Reviewing this after the fact since it's already merged — flagging two
broken examples and the CI gap that let them through. Both verified against
merged `master`.
## Two Rust examples don't compile
I extracted all 56 `rust` code blocks from the 16 pages and compiled each
against the merged `apache-spark-connect` crate. Two fail for reasons
independent of surrounding context:
**`docs/quickstart.md:39`** — `lit` is used but not imported:
```rust
use spark_connect::functions as f;
let df = spark.range(100)?;
df
.filter(f::col("id").gt(lit(50))) // error[E0425]: cannot find
function `lit`
```
`lit` is re-exported at `spark_connect::lit` (`lib.rs:35`), so the fix is
`use spark_connect::{functions as f, lit};` — which is exactly what the *next*
block on the same page already does. Worth prioritizing because this is the
second snippet in the quickstart: the first thing a new user copies after
hello-world.
**`docs/configuration.md:75`** — `SparkSessionBuilder` isn't in scope:
```rust
let spark = SparkSessionBuilder::default() // error[E0433]: cannot find
type
```
Needs `use spark_connect::SparkSessionBuilder;`.
(Other compile failures I saw were artifacts of my extraction harness —
fragments referencing a `df` bound in an earlier block — so I'm not reporting
those.)
## The docs workflow can't catch this
`docs.yml` runs only `mkdocs gh-deploy --force --no-history`. There's no
`mkdocs build --strict` in CI (the PR notes it was run locally) and no
compilation of any Rust snippet, so both errors deployed to a public site with
CI green.
Two things make a fix cheap here, both already in place:
- **`pymdownx.snippets` is already enabled** (`mkdocs.yml:81`).
- **`examples` is already a workspace member** that compiles.
So the examples could live as real files under `examples/` and be included
into the Markdown via snippets, at which point `cargo build -p examples` guards
them and the docs can't drift from the API. Failing that, a small
extract-and-`cargo check` step in `docs.yml` would catch the same class. At
minimum, adding `mkdocs build --strict` to CI would catch nav/link regressions
even if snippets stay inline.
For what it's worth, the rest held up well: zero broken internal links
across all 16 pages, and the package naming is consistent with `pyproject.toml`
(`pyspark-client-rust`).
I checked the Rust examples, internal links, and naming. I did not build the
MkDocs site, so I can't speak to the rendered output, the Mermaid diagrams in
`architecture.md`, or the custom light/dark palette.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
