yihua opened a new pull request, #19408:
URL: https://github.com/apache/hudi/pull/19408
### Describe the issue this Pull Request addresses
The six per-version extended SQL AST builders
(`HoodieSpark{3.3,3.4,3.5,4.0,4.1,4.2}ExtendedSqlAstBuilder.scala`) were only
about a quarter line-covered. The extended parser is reached only when a
statement contains a Hudi-specific ` blob` or ` vector` token
(`isHoodieCommand`), and such a statement is then parsed entirely by this
builder. After the recent prune of the forked parser to Hudi-only statements
(#19132), the remaining CREATE TABLE clause visitors (partition transforms,
literal arguments, bucket spec, file and row format, interval and other data
types, table properties and options) are all reachable and required, but were
largely unexercised because the existing tests used a bare `blob`/`vector`
column with no other clauses.
### Summary and Changelog
Enhances the existing blob/vector SQL DDL and index syntax test suites to
exercise the extended AST builder's CREATE TABLE clause visitors. This is a
test-only, parser-coverage change. Because the builder is not dead code (every
remaining path is reachable through a blob/vector statement), no further prune
is warranted.
New cases route through the extended builder by keeping a BLOB or VECTOR
column in every statement, and assert the parsed logical plan
(`CreateTable.tableSchema` and `CreateTable.partitioning`, both stable across
Spark 3.3 through 4.2) via `spark.sessionState.sqlParser.parsePlan`, or assert
the exact parse error for the negative cases. Parse-level assertions exercise
the builder without needing table creation, so they also cover clauses Hudi
does not support at execution time (transform partitioning, STORED AS / ROW
FORMAT, interval columns).
Suites enhanced (no new suite added):
- `TestBlobDataType`: primitive, complex and interval data-type arms;
partition transforms (identity, years/months/days/hours, generic apply, and
bucket with int/long/short counts); typed transform-argument literals (string,
boolean, null, integer, timestamp constructor, multi-unit interval); their
error twins (invalid bucket count, non-column reference, too many arguments,
reversed interval endpoints, unknown type); and file-format / row-format
clauses (STORED AS generic and INPUTFORMAT/OUTPUTFORMAT, ROW FORMAT SERDE /
DELIMITED, plus the incompatible-combination, STORED BY, and USING-with-serde
errors).
- `TestCreateTable` (vector): partition transforms; typed transform-argument
literals (adds long, exponent/double, and unit-to-unit interval); CLUSTERED BY
bucket spec (plain, SORTED BY ASC, and the SORTED BY DESC rejection); and
LOCATION / COMMENT / OPTIONS / TBLPROPERTIES with string, integer and boolean
values, plus the path-collision and reserved-property (provider/location/owner)
errors.
- `TestIndexSyntax`: folds the uncovered index branches into the existing
fixture-backed test: CREATE INDEX without USING (default empty index type),
DROP INDEX without IF EXISTS, and SHOW INDEXES with the IN keyword.
Finding: the extended parser's grammar mandates at least one transform
argument, so `getSingleFieldReference`'s empty-arguments branch ("Not enough
arguments for transform") is unreachable from SQL and is therefore not asserted.
### Impact
None. Test-only change; no production code or public API is modified.
### Risk Level
none
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
