Hi, I think we should also include DATAFU-180 in the upcoming release, WDYT?
Ohad. On Sun, 9 Feb 2025 at 10:50, Eyal Allweil <e...@apache.org> wrote: > Hi everyone, > > All three issues have been completed and merged - anything else worth > waiting for or can we start a release? > > Cheers, > Eyal > > On 2024/12/03 15:41:00 Eyal Allweil wrote: > > Hi all, > > > > We've had a relatively inactive year, but now we have three issues/pull > > requests that need to be reviewed. Once we merge them I think we can > > release a new version, which will support Spark all the way to Spark > 3.4.x > > and bring us up to date with their releases. I'm writing a short > > description here - anyone who can, please review them. If you don't have > > time for an in-depth code review, checking the interface/documentation of > > the two new methods is also important. > > > > The issues are: > > > > DATAFU-176 <https://issues.apache.org/jira/browse/DATAFU-176> - do > > dedupTopN with combiner. This is like our dedupTopN, but uses the > combiner > > to deal with extreme skew efficiently. A use case that came up at PayPal. > > > > DATAFU-177 <https://issues.apache.org/jira/browse/DATAFU-177>- Add > > dedupByAllExcept. This method is for deduplicating otherwise identical > rows > > with differing ids. Also a use case that came up at PayPal. > > > > DATAFU-179 <https://issues.apache.org/jira/browse/DATAFU-179> - support > > Spark 3.3.x and 3.4.x. Self-explanatory. I did this one, and it didn't > take > > much, but I'd be glad for some good double-checking. > > > > A note to interested non-committers - *we welcome your review and > comments*! > > Feel free to write in either the Jira issues or the Github PRs. > > > > Cheers, > > Eyal > > >
