[GitHub] [hudi] ad1happy2go commented on issue #8904: [SUPPORT] spark-sql hudi table Caused by: org.apache.avro.AvroTypeException: Found string, expecting union
ad1happy2go commented on issue #8904: URL: https://github.com/apache/hudi/issues/8904#issuecomment-1598347082 @zyclove This config allows hudi to preserve existing spark behaviour. In existing spark, if we read a table using the parent directory, an extra column will be added to the table by default if it contains the subdirectories in hive style partitioning. Hudi by default writes the partition column in the data itself which was not the case with default spark behaviour(example if using parquet). So if this property is enabled, the partition values will be read from the path and not from the data. It more related to read hudi table and not write. Hudi always writes partition column with data. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] ad1happy2go commented on issue #8904: [SUPPORT] spark-sql hudi table Caused by: org.apache.avro.AvroTypeException: Found string, expecting union
ad1happy2go commented on issue #8904: URL: https://github.com/apache/hudi/issues/8904#issuecomment-1587367913 @zyclove Can you create a share the code or reproducible script, so we can triage it better by reproducing it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] ad1happy2go commented on issue #8904: [SUPPORT] spark-sql hudi table Caused by: org.apache.avro.AvroTypeException: Found string, expecting union
ad1happy2go commented on issue #8904: URL: https://github.com/apache/hudi/issues/8904#issuecomment-1583978910 @zyclove This is known issue with hudi 0.11.1. This was fixed with this commit - https://github.com/apache/hudi/pull/6358 Can you try out this and let us know. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
