[ 
https://issues.apache.org/jira/browse/SPARK-56076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated SPARK-56076:
-----------------------------------
    Labels: pull-request-available  (was: )

> Types Framework - Phase 1b - Generic Row Access
> -----------------------------------------------
>
>                 Key: SPARK-56076
>                 URL: https://issues.apache.org/jira/browse/SPARK-56076
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>    Affects Versions: 4.2.0
>            Reporter: David Milicevic
>            Priority: Major
>              Labels: pull-request-available
>
> *Description:*
> {*}{*}Add generic row read/write support so that framework types can be 
> stored in and retrieved from UnsafeRow without per-type edits to row 
> interfaces.
> *What* *this* *includes:*
>  * {*}{*}Two new TypeOps methods: getValue(input: SpecializedGetters, 
> ordinal: Int): Any and unsafeWrite(writer: UnsafeWriter, ordinal: Int, value: 
> Any): Unit
>  * PhysicalFrameworkDelegatedType(ops: TypeOps) — a general-purpose delegated 
> physical type. Dispatchers match on it and call ops.getValue() / 
> ops.unsafeWrite() without knowing the storage format (struct, binary, etc.)
>  * TypeOps.getOrNull(dt) helper for clean dispatch in Java files 
> (ColumnarRow, ColumnarBatchRow)
>  * Integration in ~10 files: TypeOps.scala, TimeTypeOps.scala, 
> PhysicalDataType.scala, SpecializedGettersReader.java, 
> InterpretedUnsafeProjection.scala, ColumnarRow.java, ColumnarBatchRow.java, 
> InternalRow.scala, literals.scala
>  * TimeType reference implementation (Long-backed, simple case)
> *Note:*
> {*}{*}unsafeRead was dropped — it's redundant because UnsafeRow implements 
> SpecializedGetters, so getValue(unsafeRow, ordinal) already covers the read 
> path. PhysicalFrameworkDelegatedType replaces the earlier PhysicalOpaqueType 
> concept from the design doc — the name conveys both framework ownership and 
> delegated behavior. It takes no structFields parameter; the TypeOps handles 
> storage format internally, making it work for struct-backed types, 
> binary-backed types, or anything else.
> Struct-writing utility methods in InterpretedUnsafeProjection are widened to 
> private[catalyst] so that future TypeOps implementations can reuse the same 
> struct-writing machinery.
> *Design doc:*
> Linked in the parent work item.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to