LuciferYang commented on PR #58409:
URL: https://github.com/apache/spark/pull/58409#issuecomment-5469590047
Thanks @peter-toth, and thanks for re-running it. All five taken.
**1.** Added `select("y", "z")`. You are right that the other two are
prefixes and pass on base, so the description now says "each one-column and
two-column projection whose values the fix changes" and names why the rest are
left out, rather than claiming every one.
**2.** Added to the user-facing section: a read that used to land on a
type-compatible neighbouring field now pairs with its own field and fails when
the types do not match, so a query that returned values can return an error
instead. It reads better there than in a test comment, and it is the shape most
likely to come back as a report.
**3.** The description now names both, #58340 on the V2 path and #58411 on
the V1 path, says that this makes the read projection-independent rather than
only correct under pruning, and says this is the one to land first so the two
gates can come off together.
**4.** You are right, and the example settles it: `test.avro` has twelve
Avro fields and the SPARK-34365 test reads two Catalyst fields out of it
correctly, so trailing extras are simply ignored. The sentence now says the
fields after the gap shift by one whatever the projection is.
**5.** Default dropped, and the nested call passes `Array.empty` explicitly.
Locally: the ten `AvroSuite` cases on both paths, `AvroSchemaHelperSuite`,
`AvroSerdeSuite` and `AvroCatalystDataConversionSuite`, 64 tests;
`avro/scalastyle`, `avro/Test/scalastyle` and `sql/scalastyle`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
