The Substrait specification is still evolving. Is it possible that a consumer would want to say "give me version X" of the dialect and the server might reply "I don't have version X but here's version Y"?
On Sun, Oct 18, 2026 at 4:53 AM Tornike Gurgenidze <[email protected]> wrote: > > Hi all, > > I'd like to propose adding two new SqlInfo codes to FlightSql.proto so that > a server can advertise which part of Substrait it accepts: > > - SQL_SUBSTRAIT_DIALECT (577): a UTF-8 string holding a Substrait dialect, > i.e. a YAML document listing the types, relations, expressions and function > signatures the server accepts in a plan > - SQL_SUBSTRAIT_DIALECT_HASH (578): an opaque string identifying the > dialect's content, for client-side caching > > Motivation. Flight SQL can already say whether a server accepts Substrait > (FLIGHT_SQL_SERVER_SUBSTRAIT) and which versions (_MIN_VERSION / > _MAX_VERSION), but not which part of the spec. In practice no consumer > implements all of Substrait, so a producer only finds out a plan is > unsupported when execution fails. Substrait now has a standard format for > describing this: dialect files ( > https://github.com/substrait-io/substrait/blob/main/text/dialect_schema.yaml). > substrait-java already publishes one for its Spark integration ( > https://github.com/substrait-io/substrait-java/blob/main/spark/spark_dialect.yaml > ). > > Why a separate hash code. Dialects aren't tiny. One listing every function > impl in the standard Substrait extensions (207 functions, 507 impls) plus > all types, relations and expressions is about 34 KiB / ~1,900 lines of > YAML. Real dialects are subsets of that. That's fine to return once per > connection, but a chatty client can fetch only the hash and refetch the > dialect when it changes. The hash is opaque and compared for equality only; > the server must return a new value whenever the dialect changes (a > hex-encoded SHA-256 of the dialect is suggested, not required). > > Semantics. The dialect should only be returned if > FLIGHT_SQL_SERVER_SUBSTRAIT is true. If a server doesn't return it, the > server doesn't describe which parts of Substrait it accepts; that doesn't > mean it accepts all of Substrait. > > Placement. I've put both codes in the [500, 1000) range next to the other > SQL syntax codes, since they describe the syntax a server accepts. That > also means the change applies to ADBC, which reuses Flight SQL's SqlInfo > codes. If people would rather see them next to the > FLIGHT_SQL_SERVER_SUBSTRAIT* codes in the server info range, I can move > them, but ADBC would then need a separate change. > > Changes are fully backwards-compatible: no existing codes or types change. > > - Issue: https://github.com/apache/arrow/issues/53872 > - PR: https://github.com/apache/arrow/pull/54360 > > Thanks, > Tornike
