Chirag Singh created SPARK-54383:
------------------------------------
Summary: InternalRowComparableWrapper schema cache lookups slowing
planning time
Key: SPARK-54383
URL: https://issues.apache.org/jira/browse/SPARK-54383
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 4.0.1
Reporter: Chirag Singh
For KeyGroupedPartitioned scans, the InternalRowComparableWrapper does a cache
lookup of the current data types to create a schema and ordering for the given
types. When there are a large number of partitions, this cache lookup (which
occurs in many places once per partition) can become a bottleneck on physical
planning.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
