wankunde commented on code in PR #2371:
URL: https://github.com/apache/orc/pull/2371#discussion_r2324590406
##########
java/core/src/java/org/apache/orc/OrcConf.java:
##########
@@ -182,6 +187,9 @@ public enum OrcConf {
"added to all of the writers. Valid range is [1,10000] and is primarily
meant for" +
"testing. Setting this too low may negatively affect performance."
+ " Use orc.stripe.row.count instead if the value larger than
orc.stripe.row.count."),
+ STRIPE_SIZE_CHECK("orc.stripe.size.check",
"hive.exec.orc.default.stripe.size.check",
+ 128L * 1024 * 1024,
Review Comment:
The default stripe size is 64MB, so I think 128MB is large enough to flush
this strip.
```
STRIPE_SIZE("orc.stripe.size", "hive.exec.orc.default.stripe.size",
64L * 1024 * 1024,
"Define the default ORC stripe size, in bytes."),
```
Test with our production jobs, our spark jobs could run with 6GB executors
if STRIPE_SIZE_CHECK = 128MB, and need 8GB executors if STRIPE_SIZE_CHECK =
256MB
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
