Re: [PR] [VL ] Not fallback for function spark_partition_id and monotonically_increasing_id [incubator-gluten]

[via GitHub]

gaoyangxiaozhu commented on PR #5046:
URL: 
https://github.com/apache/incubator-gluten/pull/5046#issuecomment-2009403778

   @zhli1142015 current `monotonically_increasing_id` implement causes 
inconsistent behavior vs vinalla spark.
   
   looks in velox we always use one function instance when query has duplicate 
same function expr call
   
   
![image](https://github.com/apache/incubator-gluten/assets/9858245/8e7438d6-fa6b-450d-8be2-8e8c3e4799b7)
   
   simple reproduce:
   ```
   val df = sc.parallelize(Seq[Int](), 2).mapPartitions { _ => 
Iterator(Tuple1(1), Tuple1(2)) }.toDF("a")
   val df2 = df.select(monotonically_increasing_id(), 
expr("monotonically_increasing_id()"))
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@gluten.apache.org
For additional commands, e-mail: commits-h...@gluten.apache.org