cxzl25 commented on PR #36772: URL: https://github.com/apache/spark/pull/36772#issuecomment-1147811164
> can we have a test case for this I see that the UT of HIVE-25190 has adjusted the xmx to 3g. The current xmx of spark's uni test is 4g. I'm not sure if this scenario can be tested using the orc writer api, I can continue to investigate. > upgraded the storage api to 2.8.1 at [ORC-867](https://issues.apache.org/jira/browse/ORC-867) at Apache ORC 1.7.0+ Because I see [HIVE-25190](https://issues.apache.org/jira/browse/HIVE-25190) resolves Overflow of newLength, [HIVE-25400](https://issues.apache.org/jira/browse/HIVE-25400) optimizes HIVE-25190 some changed behavior, both of these are in 2.7.3 release. Then I saw that the orc and hive storage versions were not aligned, so I used this minor version 2.7.3. I read the description of ORC-867 because 2.8.0 accidentally introduced guava dependencies, so 2.8.1 was used, while 2.7.3 did not introduce guava dependencies. Should I raise another PR to upgrade to 2.8.1? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
