rangareddy commented on issue #15814: URL: https://github.com/apache/hudi/issues/15814#issuecomment-4911760295
This issue was reviewed as part of the JIRA-migrated backlog triage, and I tried to reproduce it on current code. **Setup:** Hudi `1.2.0` (`hudi-spark3.4-bundle`) on Spark `3.4.4`. I reproduced the exact flagged path from `SourceFormatAdapter` - `spark.read().schema(dataType).json(rdd)` followed by materializing the rows (`collectAsList` + reading each field) - using a schema with the three field types called out in the report: `bytes`, `int`+date logicalType, and decimal-as-`fixed`. **Result: the reported `NullPointerException` does not reproduce.** All rows materialize without error. The stack trace in the report is from Spark 2.x internals (`Dataset.scala:3391`, `SpecificSafeProjection`); on the Spark 3.4 line current Hudi uses, that JSON->Row path no longer throws. What remains is ordinary Spark JSON behavior under an explicit schema: - `bytes`/BinaryType: read from a base64 string; a non-base64 string yields null. - date-logicalType/DateType: read from an ISO date string; a raw int yields null. - decimal-as-fixed/DecimalType: read correctly. i.e. values convert when encoded the way Spark's JSON reader expects, and become null (not a crash) otherwise. If you can still reproduce the NPE on a current Hudi/Spark version, please reopen with your Hudi + Spark versions and a sample JSON record. Closing as not reproducible on current versions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
