dongjoon-hyun opened a new pull request, #542:
URL: https://github.com/apache/spark-connect-swift/pull/542
### What changes were proposed in this pull request?
This PR rebuilds the DocC catalog's Topics index so that the library's
public API is
discoverable in the generated documentation. No source code is changed.
**SQL functions.** None of the module-level functions were curated. All 753
public top-level
functions fell into DocC's automatically-generated `Functions` section on
the module landing
page as one flat, unsectioned list. This PR adds 13 article pages:
- `Functions.md`, a hub page linking to the family pages below.
- `ColumnFunctions.md` (21), `AggregateFunctions.md` (130),
`WindowFunctions.md` (13),
`MathFunctions.md` (85), `StringFunctions.md` (124),
`DateTimeFunctions.md` (115),
`CollectionFunctions.md` (81), `SemiStructuredFunctions.md` (53),
`SketchFunctions.md` (52),
`GeospatialFunctions.md` (10), `ConditionalFunctions.md` (21),
`MiscFunctions.md` (48).
Each page has an Overview with a runnable example and splits its Topics into
3-14 semantic
sections. Pages are organized by source-file family, so adding a function to
`XmlFunctions.swift` maps to one obvious place in the docs. The single
exception is
`count`/`sum`/`avg`/`mean`/`min`/`max` from `Functions.swift`, curated on
the aggregate page
where readers will look for them.
All 753 declarations are curated exactly once. The 82 overloaded names use
DocC's type
signature disambiguation, for example:
```
- ``lit(_:)-(Bool)``
- ``parse_url(_:_:)-(_,Column)``
```
**Missing public types.** `SparkConnect.md` gains `Column`, `SparkLiteral`,
`Window`,
`WindowSpec`, `Trigger`, `StreamingQueryException`, `StreamingQueryStatus`,
`DataFrameWriterV2`, `WhenMatched`, `WhenNotMatched`,
`WhenNotMatchedBySource`,
`DataFrameNaFunctions`, `DataFrameStatFunctions`, `Observation`, `RowSchema`,
`SparkConnectError`, `~=(_:_:)`, `RuntimeConf`, and the six catalog value
types
(`CatalogMetadata`, `CatalogColumn`, `Database`, `SparkTable`, `Function`,
`TablePartition`),
across new `Expressions`, `Window Frames`, and `Error Handling` sections.
`LocalTime` and
`TimestampNanos` move from `DataFrames` to `Data Types`.
`SparkConnectClient`, `CaseInsensitiveDictionary`, `ErrorUtils`,
`ProtoUtils`,
`SparkFileUtils`, `CRC32`, and `SHA256` are curated under `Low-Level and
Utility APIs`. These
are `public`, so DocC lists them whether or not they are curated; the `///
@nodoc` marker used
by the vendored Arrow sources is a jazzy/SPI convention that DocC does not
act on, and actually
hiding them would need the underscored `@_documentation(visibility:)`
attribute. Grouping them
under an explicitly named section is honest about what they are without
touching any source.
**`Column.md`.** A new symbol page with an Overview and 15 topic sections
covering all 84
public members. Operator overloads are separated into Arithmetic,
Comparison, and Logical
sections instead of appearing as repeated identical titles in the automatic
member list.
**Bundle metadata.** `Info.plist` drops the `CFBundleVersion` key and updates
`NSHumanReadableCopyright` to `© 2025 and onwards, The Apache Software
Foundation`, matching
the "and onwards" form used in this project's `NOTICE` so it needs no yearly
edit. The version
key was set to `0.1.0` and had never been updated across the `0.2.0` through
`0.7.0` releases;
it is removed rather than bumped because DocC does not consume it. The
`INFO.PLIST FALLBACKS` section of `docc convert --help` lists only
`CFBundleDisplayName`,
`CFBundleIdentifier`, a default module kind, and a default code-listing
language; the version
never reaches the generated site, and a stale value produces no diagnostic,
which is why it
went unnoticed. Keeping it would mean either shipping a wrong value again
after the next
release or adding a release step to maintain something nothing reads.
### Why are the changes needed?
The published documentation on Swift Package Index does not currently make
this library
navigable. 259 SQL functions were added recently and the catalog index had
never been updated,
so the largest part of the public API was reachable only through one
undifferentiated
auto-generated list, and core types such as `Column` appeared only in DocC's
automatic
`Structures`/`Classes` groups mixed in with the vendored Arrow
implementation.
### Does this PR introduce _any_ user-facing change?
No. This is a documentation-only change; the generated DocC site is
reorganized.
### How was this patch tested?
Pass the CI. In addition, DocC was built locally and compared against the
pre-change baseline.
The repository has no DocC plugin dependency, so symbol graphs were
extracted from a release
build and `docc` was invoked directly rather than adding a dependency to
`Package.swift`:
```
swift build -c release \
-Xswiftc -emit-symbol-graph -Xswiftc -emit-symbol-graph-dir -Xswiftc
/tmp/sg
```
```
xcrun docc convert Sources/SparkConnect/Documentation.docc \
--fallback-display-name SparkConnect \
--fallback-bundle-identifier org.apache.spark.connect.swift.SparkConnect \
--additional-symbol-graph-dir /tmp/sg --output-path /tmp/docs
```
Results:
- 61 warnings before this change, 61 after; `diff` of the two warning lists
is empty. All 61
are pre-existing warnings from doc comments in `.swift` sources and are
untouched here, so
`--warnings-as-errors` is not usable on this catalog yet.
- Inspecting the rendered JSON confirms 765 topic links across the 13
function pages, all
unique, with no duplicate curation and no link resolving to an unintended
symbol.
`column.json` shows 84 unique links, every one a `Column` member.
- The auto-generated `Functions` section (753 entries) is gone from the
module landing page,
which now has 11 curated sections holding 60 entries. The only remaining
uncurated top-level
symbols are vendored Arrow and FlatBuffers types.
- Every symbol name written in Topics was cross-checked against the sources;
no missing name.
- `plutil -lint` reports the plist as valid. Converting with and without
`CFBundleVersion`
produces 3,256 rendered JSON files that are all semantically identical,
confirming the key
contributes nothing to the site.
- `markdownlint --config .markdownlint.yaml --ignore-path
.markdownlintignore` is clean.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
