jiangxt2 commented on PR #12902: URL: https://github.com/apache/gravitino/pull/12902#issuecomment-5536623144
### Supplemental functional Docker evidence I ran additional functional characterization against the current PR head (`d2b7c692248333dd3aed6786116dcf4bd4c6192e`) using Spark 3.5.9 / Scala 2.12 and the repository-managed Doris 4.0.6 Docker fixture. These checks intentionally compare behavior rather than timing or throughput. | Area | Existing generic JDBC path | Governed Doris adapter | Verified conclusion | |---|---|---|---| | Default selection | `jdbc-doris` uses the generic JDBC catalog when specialized support is disabled | Selected only when `spark.sql.gravitino.enableDorisSupport=true` | The existing default and other JDBC providers remain unchanged | | Eligible detail-read plan | `JDBCScan` | Official `DorisScanV2` tablet reader | The hybrid router selects the intended physical reader | | Detail-read parallelism | One JDBC partition without generic JDBC partition properties | Eight Doris tablet partitions with `doris.request.tablet.size=1` in the focused check; the earlier standalone run used both executors | Native tablet parallelism is functional rather than only declared | | Catalog-controlled SQL parallelism | The Doris SQL-lane properties are not mapped by the generic converter | Four configured JDBC ranges, with one test row observed in every range | The specialized SQL lane uses all configured partitions when bounds match the data domain | | Aggregates, Top-N, limit, and offset | Executed through Spark JDBC | Routed through the governed JDBC semantics lane | Results matched; this lane is a correctness and controlled-parallelism feature, not a universal acceleration claim | | Doris `LARGEINT` and `JSON` | Their Gravitino `ExternalType` representation causes the generic Spark type converter to throw `UnsupportedOperationException` | Accepted through explicit SQL projection and exposed as Spark String | This is String normalization, not native non-String type support | | Doris `DATETIME(p)` | Generic conversion does not establish the specialized precision-specific contract | Exposed through the documented precision-specific String representation and validated on writes | This is a Doris semantic contract, not recovery from a generic `UnsupportedOperationException` | | Standard non-String types | Converter-level characterization succeeds for Boolean, integral, floating, Decimal, Date, Binary, no-time-zone Timestamp, and List | The same converter-level Spark types are produced | There is no newly supported non-String `UnsupportedOperationException` case | | Small multi-partition append | Four-partition Spark JDBC append completed with the expected Doris count and checksum | Four-partition append through the official Doris writer completed with the same count and checksum | Both paths can write in parallel; the specialized value is the Doris-native writer and governance contract | | Governed write protocol | Spark JDBC writer semantics | Stream Load mode with forced 2PC, strict mode, zero filter tolerance, schemaless mode off, and automatic redirect off | The adapter adds a protected Doris-specific write contract rather than merely exposing more write operations | | Authorization and credentials | Existing generic JDBC behavior | Vended JDBC credentials are required, and read/write authorization completes before specialized physical access | The stricter behavior is intentionally scoped to the opt-in adapter | | Capability surface | Generic JDBC capabilities | `BATCH_READ` by default; batch append and optional truncate are explicitly enabled, while streaming, row-level operations, predicate/dynamic overwrite, and Spark catalog DDL remain rejected | The adapter follows a minimum-capability model | The exact answer to the `UnsupportedOperationException` question is: **no non-String case was found**. Generic JDBC throws for Gravitino `ExternalType`; every such case accepted by the specialized converter is represented as Spark String. Standard non-String types already convert successfully through generic JDBC, so the PR does not claim them as new exception compatibility. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
