Gluten is still failing in some cases against Spark 3.5

2024-03-07 Thread Zhang Hongze
Hi dev, It seems that Gluten is still not working correctly with Spark 3.5 in some cases. (After applying some recent fixes from Ke and Xiduo) I'll create an issue ticket for that. https://github.com/apache/incubator-gluten/issues/4899 . If you are a Spark 3.5 and Gluten user, please be aware of

Gluten advanced CBO

2024-03-20 Thread Zhang Hongze
Hi dev, I filed https://github.com/apache/incubator-gluten/issues/5057 as the proposal to introduce search-based query optimization techniques into Gluten (named advanced CBO at the moment). Please feel free to comment. Thanks, Hongze

Re: About Branch Protection rule

2024-03-27 Thread Zhang Hongze
Thanks Weiting. > is set to *require at least one review approval* If I understand correctly, we are going to add this restriction back to Apache Gluten repo, is that correct? I think for some trivial PRs committers may want to keep some flexibility to merge by themselves as long as they are con

Gluten RFC for columnar rule enhancement

2024-03-29 Thread Zhang Hongze
Hi all, I've created a renewed document for columnar rule enhancement. And I've defined the scope of the work so that we don't lose our initial purpose during doing this. Please review and feel free to comment. Thanks. https://docs.google.com/document/d/1-LwU6367Tz3uuDu_qDXwQ_PbBU51Iio8KVm5ePwq5

About GHA dependency cache for Velox static build

2024-04-01 Thread Zhang Hongze
Hi all, If you are now experiencing unintended caching of Velox static build (the cache was added by https://github.com/apache/incubator-gluten/pull/5145), please go to page https://github.com/apache/incubator-gluten/actions/caches to manually remove the cache from your branch. I'll work on impro

Re: FW: [Gluten] Feedback collection for Gluten Native Diagnose Experience Enhancement

2024-04-02 Thread Zhang Hongze
+1 Can't wait to see this. So far profiling Gluten's code is non-trivial and the features in proposal will ease that a lot. About showing flame graphs on UI (which seems to be super helpful, indeed): Do you have some initial designs for that? Like it's turned on/off by configuration or an UI butto