[ https://issues.apache.org/jira/browse/SPARK-42372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-42372: ------------------------------------ Assignee: Apache Spark > Improve performance of HiveGenericUDTF by making inputProjection instantiate > once > --------------------------------------------------------------------------------- > > Key: SPARK-42372 > URL: https://issues.apache.org/jira/browse/SPARK-42372 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 3.4.0 > Reporter: Kent Yao > Assignee: Apache Spark > Priority: Major > > {code:java} > +++ b/sql/hive/benchmarks/HiveUDFBenchmark-per-row-results.txt > @@ -0,0 +1,7 @@ > +OpenJDK 64-Bit Server VM 1.8.0_352-bre_2022_12_13_23_06-b00 on Mac OS X 13.1 > +Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz > +Hive UDTF benchmark: Best Time(ms) Avg Time(ms) > Stdev(ms) Rate(M/s) Per Row(ns) Relative > +------------------------------------------------------------------------------------------------------------------------ > +Hive UDTF dup 2 1574 1680 > 118 0.7 1501.1 1.0X > +Hive UDTF dup 4 2642 3076 > 588 0.4 2519.9 0.6X > + > diff --git a/sql/hive/benchmarks/HiveUDFBenchmark-results.txt > b/sql/hive/benchmarks/HiveUDFBenchmark-results.txt > new file mode 100644 > index 0000000000..8af8b6582c > --- /dev/null > +++ b/sql/hive/benchmarks/HiveUDFBenchmark-results.txt > @@ -0,0 +1,7 @@ > +OpenJDK 64-Bit Server VM 1.8.0_352-bre_2022_12_13_23_06-b00 on Mac OS X 13.1 > +Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz > +Hive UDTF benchmark: Best Time(ms) Avg Time(ms) > Stdev(ms) Rate(M/s) Per Row(ns) Relative > +------------------------------------------------------------------------------------------------------------------------ > +Hive UDTF dup 2 712 789 > 101 1.5 678.7 1.0X > +Hive UDTF dup 4 1212 1294 > 78 0.9 1156.0 0.6X > + {code} > over 2x performance gain via a benchmarking -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org