James, As we have both discovered, the function sets of other DBMSs are confusing and often contradictory. In some cases, DBMS vendors add functions for compatibility with other DBMSs, we attempt to emulate those clones, and discover differences with the originals.
In the past few weeks, Tanner, Oliver and I have done our best to add a few functions based on BigQuery's specifications without breaking compatibility (or conflicting with functions that have been added to Drill but not upstreamed). But we will have inevitably made a few mistakes. The only way I know to inoculate against this kind of problem in future is for projects like Drill to upstream tests. Julian On Wed, Mar 8, 2023 at 8:26 PM James Turton <dz...@apache.org> wrote: > > Much appreciated. I only have so much know-how in this area but > CALCITE-5469 looks completely normal to me, just something that we need > to accommodate due to Drill having at some point settled on a > conflicting definition of DATE_DIFF that seems to resemble Hive and > MySQL's DATEDIFFs (no underscore) more than anything else. > > We'll start a new thread if we can't simply take care of it ourselves. > > On 2023/03/08 19:28, Tanner Clary wrote: > > Hello, > > > > With regards to the unrelated issue with DATE_DIFF, I authored CALCITE-5469 > > so perhaps if you want to open a new thread or post a comment on the case > > itself, I would be happy to take a look. > > > > Best, > > Tanner > > > > On Wed, Mar 8, 2023 at 8:42 AM James Turton <dz...@apache.org> wrote: > > > >> Hi > >> > >> All of Drill's DATE_TRUNC unit tests pass when Drill uses > >> calcite-1.34.0-SNAPSHOT (and once we accommodate the new QUALIFY > >> clause). While we do now have an unrelated issue with DATE_DIFF which I > >> believe has resulted from the introduction of a three parameter > >> DATE_DIFF function in CALCITE-5469, I'm quite sure that we can resolve > >> this in Drill. > >> > >> In summary I'm a +1 for this Calcite snapshot becoming an RC. > >> > >> Thanks > >> James Turton > >> > >> > >> On 2023/03/07 00:11, Stamatis Zampetakis wrote: > >>> Hey Charles, > >>> > >>> Please test Drill with the latest calcite-1.34.0-SNAPSHOT [1] and if all > >> is > >>> good on your end I will prepare an RC for vote. > >>> > >>> Best, > >>> Stamatis > >>> > >>> [1] > >>> > >> https://repository.apache.org/content/groups/snapshots/org/apache/calcite/calcite-core/1.34.0-SNAPSHOT/ > >>> On Sun, Mar 5, 2023 at 7:16 PM Charles Givre <cgi...@gmail.com> wrote: > >>> > >>>> Julian, > >>>> Now that Drill is on main Calcite instead of the fork, I'll commit that > >>>> the Drill community will do our best to try Drill with the RC > >> candidates to > >>>> see if we can catch issues during the release cycle. > >>>> Thanks, > >>>> -- C > >>>> > >>>> > >>>>> On Mar 5, 2023, at 12:20 PM, Julian Hyde <jhyde.apa...@gmail.com> > >> wrote: > >>>>> It was indeed a regression, but it didn’t break any of Calcite’s tests > >>>> and no one spoke up during the release vote. Mistakes are expensive to > >> fix > >>>> after a release, cheaper during the release vote, and cheapest of all if > >>>> found by the test suite. > >>>>>> On Mar 5, 2023, at 6:33 AM, Charles Givre <cgi...@gmail.com> wrote: > >>>>>> > >>>>>> That would be great! Again I’m only asking because this was a > >>>> regression. I really do appreciate it. Thanks! > >>>>>> Sent from my iPhone > >>>>>> > >>>>>>> On Mar 4, 2023, at 13:59, Stamatis Zampetakis <zabe...@gmail.com> > >>>> wrote: > >>>>>>> If we get the 1.34.0 out a bit sooner than usual I guess this will > >> be > >>>> good > >>>>>>> enough for Drill. If the others agree I can try to prepare an RC > >> during > >>>>>>> next week. WDYT ? > >>>>>>> > >>>>>>> Best, > >>>>>>> Stamatis > >>>>>>> > >>>>>>> > >>>>>>>> On Sat, Mar 4, 2023, 6:13 PM Alessandro Solimando < > >>>>>>>> alessandro.solima...@gmail.com> wrote: > >>>>>>>> > >>>>>>>> The second option Benchao mentions is what Hive currently does as > >>>> well. > >>>>>>>> Best regards, > >>>>>>>> Alessandro > >>>>>>>> > >>>>>>>>>> On Sat 4 Mar 2023, 13:19 Benchao Li, <libenc...@apache.org> > >> wrote: > >>>>>>>>>> Hi Charles, > >>>>>>>>>> > >>>>>>>>>> Thank for reaching out! > >>>>>>>>>> > >>>>>>>>>> IIRC, the idea of releasing bugfix version has been brought up in > >>>> the > >>>>>>>> past, > >>>>>>>>> but I couldn't find the discussion (in Jira and dev ML). > >>>>>>>>> > >>>>>>>>> I'd like to share my understanding why we chose not to release bug > >>>> fix > >>>>>>>>> versions, please correct me if I'm wrong, > >>>>>>>>> - Calcite has many bug fixes that span multi versions (even more > >>>> that 10 > >>>>>>>>> versions), then only keeping several (such as 3) bug fix releases > >>>> does > >>>>>>>> not > >>>>>>>>> solve all these problems. > >>>>>>>>> - Actually we usually do not distinguish too much between "bugfix" > >>>> and > >>>>>>>> "new > >>>>>>>>> feature", so maintaining bug fix releases is not that easy. > >>>>>>>>> - Calcite lacks reviewers and also release managers, only keeping > >>>> linear > >>>>>>>>> releasing in rhythm could save us some efforts. > >>>>>>>>> > >>>>>>>>> For regressions, I agree that this hurts downstream projects. For > >>>> such > >>>>>>>>> cases, there are two approaches come into my mind: > >>>>>>>>> - We can release a new version quickly than usual. > >>>>>>>>> - The projects that need the fix/feature before our next scheduled > >>>>>>>> release, > >>>>>>>>> they could copy these files into their projects, as we already did > >> in > >>>>>>>>> Flink[1]. They could remove these files once they adopt the new > >>>> release > >>>>>>>> of > >>>>>>>>> Calcite. > >>>>>>>>> > >>>>>>>>> I hope this helps. > >>>>>>>>> > >>>>>>>>> [1] > >>>>>>>>> > >>>>>>>>> > >> https://github.com/apache/flink/tree/master/flink-table/flink-table-planner/src/main/java/org/apache/calcite > >>>>>>>>> Charles Givre <cgi...@gmail.com> 于2023年3月2日周四 06:22写道： > >>>>>>>>> > >>>>>>>>>> Hello Calcite Devs, > >>>>>>>>>> I wanted to thank everyone for the recent release of Calcite 1.33. > >>>> I > >>>>>>>> am > >>>>>>>>>> the PMC Chair for Apache Drill and we just released Drill 1.21[0] > >>>> which > >>>>>>>>> is > >>>>>>>>>> now using the latest version of Calcite instead of our 2-3 year > >> old > >>>>>>>> fork! > >>>>>>>>>> However, we encountered a small issue with Calcite 1.33 that does > >>>> not > >>>>>>>>>> affect just Drill. Specifically, there was a regression which was > >>>>>>>> caused > >>>>>>>>>> by CALCITE-5447[1] which effectively broke the DATE_TRUNC > >> function. > >>>>>>>> The > >>>>>>>>>> bugfix has been fixed and merged in CALCITE-5522[2]. > >>>>>>>>>> > >>>>>>>>>> In any event, given that this function is fairly important and the > >>>>>>>>> lengthy > >>>>>>>>>> release schedules of both Drill and Calcite, I wanted to ask > >> whether > >>>>>>>> the > >>>>>>>>>> Calcite might consider doing a quick bugfix release with this and > >>>> any > >>>>>>>>> other > >>>>>>>>>> regressions that may have popped up in 1.33 and have since been > >>>> fixed. > >>>>>>>>>> Thank you very much for all your work! > >>>>>>>>>> Best, > >>>>>>>>>> -- Charles > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> [0]: > >>>>>>>>>> > >> https://github.com/apache/drill-site/blob/master/blog/_posts/en/2023-02-21-drill-1.21.0-released.md > >>>>>>>>>> [1]: https://issues.apache.org/jira/browse/CALCITE-5447 > >>>>>>>>>> [2]: https://issues.apache.org/jira/browse/CALCITE-5522 > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> > >>>>>>>>> Best, > >>>>>>>>> Benchao Li > >>>>>>>>> > >> >
