cloud-fan commented on PR #56919: URL: https://github.com/apache/spark/pull/56919#issuecomment-4851448466
Note: I force-pushed a reworked version of this PR. The original commit only introduced the `typeSoFar` scaffolding in the base class (where it was always `NullType`, so effectively a no-op). This version wires it through the RDD driver so inference is genuinely incremental, adds the `spark.sql.xml.schemaInference.incrementalTypeCasting.enabled` kill switch (default true), and adds a flag-on == flag-off end-to-end equivalence test confirming inferred schemas are unchanged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
