tokoko opened a new pull request, #54360: URL: https://github.com/apache/arrow/pull/54360
### Rationale for this change Flight SQL can already report whether a server accepts Substrait plans and which versions it accepts (`FLIGHT_SQL_SERVER_SUBSTRAIT`, `_MIN_VERSION`, `_MAX_VERSION`). It can't report *which part* of Substrait the server accepts. In practice no consumer implements the whole spec, so a producer only learns that a plan is unsupported when execution fails. Substrait now has a standard format for this: [dialect files](https://github.com/substrait-io/substrait/blob/main/text/dialect_schema.yaml). A dialect is a YAML document that lists the types, relations, expressions and function signatures a system supports. substrait-java already publishes one for Spark ([spark_dialect.yaml](https://github.com/substrait-io/substrait-java/blob/main/spark/spark_dialect.yaml)). ### What changes are included in this PR? Two new `SqlInfo` codes in `FlightSql.proto`, mirrored in the C++ `SqlInfoOptions::SqlInfo` enum: - `SQL_SUBSTRAIT_DIALECT = 577`: a UTF-8 string holding a Substrait dialect YAML document. Should only be returned if `FLIGHT_SQL_SERVER_SUBSTRAIT` is true. If it isn't returned, the server doesn't describe which parts of Substrait it accepts; that doesn't mean it accepts all of Substrait. - `SQL_SUBSTRAIT_DIALECT_HASH = 578`: an opaque string identifying the dialect's content, for client-side caching. It must change whenever the dialect changes (e.g. a hex-encoded SHA-256 digest of the dialect). Clients compare it for equality only. The hash code exists because of size. A dialect covering every function in the standard Substrait extensions plus all types, relations and expressions is about 34 KiB of YAML (about 3.6 KiB gzipped). Real dialects are subsets of that. This is fine to fetch once per connection, but chatty clients can fetch the hash first and only refetch the dialect when the hash changes. Both codes are in the [500, 1000) range with the other SQL syntax codes, since they describe the syntax the server accepts. Because ADBC reuses Flight SQL's info codes, ADBC gets the same codes without a separate change. ### Are these changes tested? No. The change only adds enum values and documentation; no behavior changes. ### Are there any user-facing changes? Yes. Two new `SqlInfo` codes are available to Flight SQL servers and clients. The change is backwards-compatible: no existing codes or types change. * GitHub Issue: #53872 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
