Yicong-Huang opened a new pull request, #59444: URL: https://github.com/apache/spark/pull/59444
### What changes were proposed in this pull request? Add `TableUDFTimeBench` and `TableUDFPeakmemBench` to `python/benchmarks/bench_eval_type.py` for `SQL_TABLE_UDF` (pickled Python UDTF, `useArrow=False`). - `MockProtocolWriter.write_pickled_data_payload`: pickles row tuples in batches of 100 (matching `BatchEvalPythonExec.getInputIterator` on the JVM side) and ends with `END_OF_DATA_SECTION`. - `MockDataFactory.make_rows`: builds Python row tuples from the existing type pools. - The UDTF payload reuses `write_arrow_udtf_payload` (same UDTF wire protocol). - UDTFs: identity, explode (x3), count (emits only from `terminate`), stringify. Every input column is passed to `eval` as an argument; scenarios cover few/many args with int, string and mixed columns. ### Why are the changes needed? Part of SPARK-59415. `SQL_TABLE_UDF` is the only UDTF eval type without an ASV benchmark; it is needed to measure the upcoming migration of `SQL_TABLE_UDF` to the `EvalTypeHandler` pipeline (SPARK-60218). ### Does this PR introduce _any_ user-facing change? No, benchmark only. ### How was this patch tested? `asv run --python=same --bench "TableUDF(Time|Peakmem)Bench" -a "repeat=(3,5,5.0)"`; also decoded the worker output for every scenario to check row counts and values. | scenario | identity_udtf | explode_udtf | count_udtf | stringify_udtf | |---|---|---|---|---| | few_args_ints | 155+-1ms | 194+-3ms | 104+-4ms | 173+-3ms | | many_args_ints | 88.5+-0.4ms | 107+-1ms | 64.2+-0.5ms | 127+-2ms | | few_args_strings | 151+-1ms | 190+-4ms | 101+-2ms | 168+-3ms | | many_args_strings | 93.9+-0.9ms | 108+-1ms | 56.6+-1ms | 107+-3ms | | mixed_types | 66.0+-1ms | 81.4+-0.6ms | 42.5+-0.2ms | 79.5+-0.6ms | ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5.5 This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
