[
https://issues.apache.org/jira/browse/SPARK-45649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17788306#comment-17788306
]
Wenchen Fan commented on SPARK-45649:
-------------------------------------
This has a bug. I've uploaded the test data. You can reproduce the bug via
{code:java}
spark.read.parquet("/tmp/test_table.parquet").createOrReplaceTempView("test_table")
spark.sql("SELECT nullable_struct_col, id, lag(long_mixedparts, 5) OVER
(PARTITION BY nullable_struct_col ORDER BY id ) FROM test_table").collect {code}
> Unify the prepare framework for `OffsetWindowFunctionFrame`
> -----------------------------------------------------------
>
> Key: SPARK-45649
> URL: https://issues.apache.org/jira/browse/SPARK-45649
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: Jiaan Geng
> Assignee: Jiaan Geng
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: test_table.parquet.zip
>
>
> Currently, the implementation the `prepare` of all the
> `OffsetWindowFunctionFrame` have the same code logic show below.
> ```
> override def prepare(rows: ExternalAppendOnlyUnsafeRowArray): Unit = {
> if (offset > rows.length) {
> fillDefaultValue(EmptyRow)
> } else {
> resetStates(rows)
> if (ignoreNulls) {
> ...
> } else {
> ...
> }
> }
> }
> ```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
