(orc) branch main updated: ORC-1930: Improve `GenerateVariants` to accept ORC configs via system properties

[dongjoon]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/orc.git


The following commit(s) were added to refs/heads/main by this push:
     new f01382a99 ORC-1930: Improve `GenerateVariants` to accept ORC configs 
via system properties
f01382a99 is described below

commit f01382a993d6a2e47b935f3261a44efa335a6599
Author: Dongjoon Hyun <dongj...@apache.org>
AuthorDate: Wed Jun 18 16:06:00 2025 -0700

    ORC-1930: Improve `GenerateVariants` to accept ORC configs via system 
properties
    
    ### What changes were proposed in this pull request?
    
    This PR aims to improve `GenerateVariants` to accept ORC configs via system 
properties.
    
    ### Why are the changes needed?
    
    This is helpful to run benchmarks with adjusted configurations.
    
    ### How was this patch tested?
    
    Manual tests.
    
    **BUILD**
    
    ```
    $ mvn clean package -Pbenchmark -DskipTests
    $ cd bench
    ```
    
    **BEFORE**
    
    No error because the configuration is not applied.
    
    ```
    $ java -Dorc.compression.zstd.level=99 -jar 
core/target/orc-benchmarks-core-*-uber.jar generate data -d sales -f orc -c 
zstd -s 2000000
    Processing sales [orc]
    [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load 
native-hadoop library for your platform... using builtin-java classes where 
applicable
    ...
    ```
    
    **AFTER**
    
    Error occurs because the configuration is applied.
    
    ```
    $ java -Dorc.compression.zstd.level=99 -jar 
core/target/orc-benchmarks-core-*-uber.jar generate data -d sales -f orc -c 
zstd -s 2000000
    Processing sales [orc]
    [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load 
native-hadoop library for your platform... using builtin-java classes where 
applicable
    Exception in thread "main" java.lang.IllegalArgumentException: Zstd 
compression level should be in the range -131072 to 22
    ```
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No.
    
    Closes #2288 from dongjoon-hyun/ORC-1930.
    
    Authored-by: Dongjoon Hyun <dongj...@apache.org>
    Signed-off-by: Dongjoon Hyun <dongj...@apache.org>
---
 .../src/java/org/apache/orc/bench/core/convert/GenerateVariants.java  | 4 ++++
 1 file changed, 4 insertions(+)

diff --git 
a/java/bench/core/src/java/org/apache/orc/bench/core/convert/GenerateVariants.java
 
b/java/bench/core/src/java/org/apache/orc/bench/core/convert/GenerateVariants.java
index 0450088d5..0f1e1965c 100644
--- 
a/java/bench/core/src/java/org/apache/orc/bench/core/convert/GenerateVariants.java
+++ 
b/java/bench/core/src/java/org/apache/orc/bench/core/convert/GenerateVariants.java
@@ -122,6 +122,10 @@ public class GenerateVariants implements OrcBenchmark {
     Configuration conf = new Configuration();
     // Disable Hadoop checksums
     conf.set("fs.file.impl", "org.apache.hadoop.fs.RawLocalFileSystem");
+    for (String key: System.getProperties().stringPropertyNames()) {
+      if (!key.startsWith("orc.")) continue;
+      conf.set(key, System.getProperty(key));
+    }
     Path root = new Path(cli.getArgs()[0]);
 
     for (final String data: dataList) {