Some fixes/improvements were made to the codebase since the last release, and sadly an official release is needed to pick up those changes. Ray asked the community more than a month ago. More recently, other people have been asking too on the user mailing list.
Like I said, it might be okay to change the scope but what I'm asking is a little help/transparency here because it looks like I'm chasing a moving target. If we can clarify which new issues have to be part of the release and why (depending on the severity), and how long we think it will take, I'd hope we can have some constructive discussion. As for the dependencies change: - I actually wrote a pull request to address CVEs in both Hadoop and Jetty - The Guava change will not address the most recent CVE. To address the CVE, code must be changed, and it doesn't require a Guava update. The change made to the Guava library was to deprecate the unsecure method... So imho updating dependencies to address CVE without looking at the CVE itself does not make things safer. So to address specifically the CVE, I opened a new ticket (DRILL-7936 <https://issues.apache.org/jira/browse/DRILL-7936>) and a pull request (https://github.com/apache/drill/pull/2240) On Thu, May 27, 2021 at 9:30 AM Charles Givre <cgi...@gmail.com> wrote: > Hi Laurent, > I’m not sure what the rush is to get a release out. I would much rather > do a quality release than just get something out the door for the sake of > getting something out the door. > > In reference to Drill-7934 (Parquet), DRILL-7919 I am personally not in > favor of putting out a release with known bugs, especially when these bugs > affect parts of Drill that are in active use, we don’t do releases that > frequently, and there is a PR that is awaiting merge. > > I’m also not in favor of a release that has known issues with > dependencies, especially again when there are pending PRs that address > these CVEs. If we did more frequent releases (which we have discussed and > hope to do going forward), then fine, but we’ve been averaging 2 a year and > I’d hate for users to have to wait 6 months for these fixes. > > — C > > > > > On May 27, 2021, at 12:19 PM, Laurent Goujon <laur...@dremio.com> wrote: > > > > Since I'm also a reviewer and that I see that the past comments I've been > > addressed, and since I do not see another committer opposing the patch, > > wouldn't I be able to give my +1 and that would clear that bar? > > > > As for the parquet issues, when we started the release discussion a month > > ago, we agreed on a scope, and the parquet issues were not part of it. I > > understand that scope can change but can we discuss it in this thread > about > > why this release should include it vs wait on the next release? We need > to > > draw a line somewhere. > > > > Laurent > > > > On Thu, May 27, 2021 at 8:05 AM Charles Givre <cgi...@gmail.com> wrote: > > > >> Laurent, > >> Per Apache policy, you need a +1 from a reviewer to merge a PR. Unless > >> there is one, please do not merge. I'll reach out to Vitalii to see > what > >> the current status is. Also there are a few bug fixes for the Parquet > >> which Vova submitted which looks like we should include as well. > >> Best, > >> -- C > >> > >>> On May 27, 2021, at 11:01 AM, Laurent Goujon <laur...@dremio.com> > wrote: > >>> > >>> Sadly, I haven't heard from people regarding the patches. At the same > >> time, > >>> I think we held the window open for merging the changes for a very long > >>> time. Unless there's objection, I'm planning to merge the Guava and > >>> Jetty/Hadoop pull requests later today, and doing the first RC for > Drill > >>> 1.19.0 > >>> > >>> Here are the pull request links: > >>> * https://github.com/apache/drill/pull/2202 > >>> * https://github.com/apache/drill/pull/2236 > >>> > >>> Laurent > >>> > >>> > >>> On Wed, May 26, 2021 at 11:59 AM Laurent Goujon <laur...@dremio.com> > >> wrote: > >>> > >>>> After several retries, the Guava checks successfully passed: > >>>> https://github.com/apache/drill/pull/2202 > >>>> > >>>> Charles, can we proceed on merging your change? > >>>> > >>>> Laurent > >>>> > >>>> On Tue, May 25, 2021 at 10:24 PM Laurent Goujon <laur...@dremio.com> > >>>> wrote: > >>>> > >>>>> Just an update. There's a patch for updating both Jetty and Hadoop > (at > >>>>> the same time) as those changes are co-dependent: > >>>>> https://github.com/apache/drill/pull/2236 > >>>>> > >>>>> As for the Guava patch, I'd be happy to help, but I'm not sure what's > >>>>> left. As far as I can tell the shaded version of Guava has been > >> updated, > >>>>> but the build is failing. The security vulnerabilities for Guava are > >>>>> moderate (and actually it seems a fix for CVE-2020-8908 would > require a > >>>>> code change instead of a Guava update. > >>>>> > >>>>> Since this has been almost a month since we started this release > >> process, > >>>>> I wonder if we still want to wait on this patch, or if we should move > >> it to > >>>>> the next release. > >>>>> > >>>>> Let me know what people think, > >>>>> > >>>>> On Tue, May 25, 2021 at 8:24 AM Laurent Goujon <laur...@dremio.com> > >>>>> wrote: > >>>>> > >>>>>> Anything I can help with? > >>>>>> > >>>>>> On Tue, May 25, 2021 at 7:02 AM Charles Givre <cgi...@gmail.com> > >> wrote: > >>>>>> > >>>>>>> HI Laurent, > >>>>>>> My apologies. I said Junit, when I was meaning to say to the Guava > >> PR ( > >>>>>>> https://github.com/apache/drill/pull/2202 < > >>>>>>> https://github.com/apache/drill/pull/2202>). I think this one is > >>>>>>> almost done as well. > >>>>>>> -- C > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>> On May 24, 2021, at 5:29 PM, Laurent Goujon <laur...@dremio.com> > >>>>>>> wrote: > >>>>>>>> > >>>>>>>> Ok, I was hoping that some of the PRs could be merged, but if we > are > >>>>>>> in > >>>>>>>> agreement, let's start the work :) > >>>>>>>> > >>>>>>>> On Sun, May 23, 2021 at 6:52 PM luoc <l...@apache.org> wrote: > >>>>>>>> > >>>>>>>>> Hi Charles, > >>>>>>>>> All right, we'll be expecting the update. > >>>>>>>>> > >>>>>>>>>> 2021年5月24日 上午12:13，Charles Givre <cgi...@gmail.com> 写道： > >>>>>>>>>> > >>>>>>>>>> Hi Luoc, > >>>>>>>>>> We still have a few PRs pending that we really should get into > >> Drill > >>>>>>>>> 1.19. The main one is the junit upgrade. There are a few > critical > >>>>>>> CVEs > >>>>>>>>> associated with that, so I do think it is important to get that > one > >>>>>>>>> merged. I think Vitalii will have that one done in short order. > >>>>>>>>>> Best, > >>>>>>>>>> -- C > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> On May 22, 2021, at 5:16 AM, luoc <l...@apache.org> wrote: > >>>>>>>>>>> > >>>>>>>>>>> Hi Laurent, > >>>>>>>>>>> It’s time to do a release with 1.19.0. > >>>>>>>>>>> > >>>>>>>>>>>> 2021年5月19日 上午2:20，Vitalii Diravka <vita...@apache.org> 写道： > >>>>>>>>>>>> > >>>>>>>>>>>> Hi Laurent, > >>>>>>>>>>>> DRILL-7871 requires additional time to be introduced and it is > >>>>>>> better > >>>>>>>>> to > >>>>>>>>>>>> include it for the next release. > >>>>>>>>>>>> DRILL-7904 is updated, I think it will be merged in a few > days. > >>>>>>> But it > >>>>>>>>>>>> doesn't matter whether it is included in this release or in > the > >>>>>>> next > >>>>>>>>> one. > >>>>>>>>>>>> > >>>>>>>>>>>> So we can plan to start the release process > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> Kind regards > >>>>>>>>>>>> Vitalii > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On Tue, May 11, 2021 at 7:52 PM Laurent Goujon < > >>>>>>> laur...@dremio.com> > >>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Thanks Vitalii > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Tue, May 11, 2021 at 9:29 AM Vitalii Diravka < > >>>>>>> vita...@apache.org> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Hi Luoc! > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> They are almost ready. I plan to update PR for them today. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Kind regards > >>>>>>>>>>>>>> Vitalii > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Sat, May 8, 2021 at 5:26 PM luoc <l...@apache.org> > wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hi Vitalii, > >>>>>>>>>>>>>>> Would you mind sharing that... Is DRILL-7904 ready to > review > >>>>>>> again? > >>>>>>>>>>>>>> And what’s > >>>>>>>>>>>>>>> the status on the DRILL-7871? thanks > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> 2021年5月4日 下午1:10，Ted Dunning <ted.dunn...@gmail.com> 写道： > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Laurent, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I don't have a stake here, so can't really comment about > >>>>>>> specifics, > >>>>>>>>> but > >>>>>>>>>>>>>> the > >>>>>>>>>>>>>>> process is looking good. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Mon, May 3, 2021 at 9:23 PM Laurent Goujon < > >>>>>>> laur...@dremio.com> > >>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks for all the answers > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> So the issues I found based on the feedback are: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> - DRILL-7878: Fix LGTM Alerts > >>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/DRILL-7878> > >>>>>>>>>>>>>>> - DRILL-7871: StoragePluginStore instances for different > >> users > >>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/DRILL-7871> > >>>>>>>>>>>>>>> - DRILL-7908: Fix GitHub Actions CI > >>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/DRILL-7908> > >>>>>>>>>>>>>>> - DRILL-7904: Update to 30-jre Guava version > >>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/DRILL-7904> > >>>>>>>>>>>>>>> - DRILL-7826: Merge Pcap and Pcapng format plugin based on > >> EVF > >>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/DRILL-7826> > >>>>>>>>>>>>>>> - DRILL-7828: Refactor Pcap and Pcapng format plugin > >>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/DRILL-7828> > >>>>>>>>>>>>>>> - DRILL-7910: Bumps commons-io from 2.4 to 2.7 > >>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/DRILL-7910> > >>>>>>>>>>>>>>> - DRILL-7901: Bump junit from 4.12 to 4.13.1 > >>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/DRILL-7901> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I wanted to propose Monday May 10th to do the first release > >>>>>>>>> candidate, > >>>>>>>>>>>>>> but > >>>>>>>>>>>>>>> I have some concerns about some of the changes which may > not > >> be > >>>>>>>>> ready > >>>>>>>>>>>>> by > >>>>>>>>>>>>>>> then considering they seem to involve some level of effort > >> and > >>>>>>> are > >>>>>>>>> in > >>>>>>>>>>>>>> very > >>>>>>>>>>>>>>> early stage: The LGTM alert changes and the > >> StoragePluginStore > >>>>>>> model > >>>>>>>>>>>>>>> change. JUnit version update might also become quite a > large > >>>>>>> change > >>>>>>>>> if > >>>>>>>>>>>>>>> instead of moving to 4.13.1, Drill is switching to JUnit5. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> What do people think? > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Sat, Apr 24, 2021 at 1:00 PM Vitalii Diravka < > >>>>>>> vita...@apache.org > >>>>>>>>>> > >>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hi Laurent, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I want to include: > >>>>>>>>>>>>>>> DRILL-7871 < > https://issues.apache.org/jira/browse/DRILL-7871 > >>> > >>>>>>>>>>>>> (preparing > >>>>>>>>>>>>>>> PR) > >>>>>>>>>>>>>>> DRILL-7908 < > https://issues.apache.org/jira/browse/DRILL-7908 > >>> > >>>>>>>>>>>>> (preparing > >>>>>>>>>>>>>>> PR) > >>>>>>>>>>>>>>> DRILL-7904 < > https://issues.apache.org/jira/browse/DRILL-7904 > >>> > >>>>>>> (PR > >>>>>>>>> is > >>>>>>>>>>>>>>> opened, in review) > >>>>>>>>>>>>>>> DRILL-7828 < > https://issues.apache.org/jira/browse/DRILL-7828 > >>> > >>>>>>> (PR > >>>>>>>>> is > >>>>>>>>>>>>>>> opened, review is almost completed) > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> All these tasks are expected to be completed in a week > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Kind regards > >>>>>>>>>>>>>>> Vitalii > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Fri, Apr 23, 2021 at 9:25 PM Charles Givre < > >>>>>>> cgi...@gmail.com> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hi Laurent, > >>>>>>>>>>>>>>> We have a few PRs pending which I'd like to see in the next > >>>>>>> version > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> which > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> are: > >>>>>>>>>>>>>>> 1. The update(s) and bug fixes to the Mongo plugin. > >>>>>>>>>>>>>>> 2. There is an extended PR for bug fixes which clean up a > >> lot > >>>>>>> of > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> alerts > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> generated by LGTM > >>>>>>>>>>>>>>> 3. There are a few other library updates which are > pending. > >>>>>>>>>>>>>>> 4. We have some work which changes the access model around > >>>>>>> storage > >>>>>>>>>>>>>>> plugins which would be good for this release > >>>>>>>>>>>>>>> 5. The PCAP/PCAP-NG consolidation is awaiting review. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I think that's it. > >>>>>>>>>>>>>>> -- C > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Apr 22, 2021, at 12:33 PM, Laurent Goujon < > >>>>>>> laur...@dremio.com> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hello everyone, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> It has been more than 6 months since the last release, and > I > >>>>>>> believe > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> this > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> would be a good time to discuss the next one. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> As mentioned in a previous email thread, I am volunteering > to > >>>>>>> be the > >>>>>>>>>>>>>>> release manager, and I'm looking forward working with the > >>>>>>> whole > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> community > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> to make another great release. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> We have around 80 changes in master since the last release, > >> and > >>>>>>>>> there > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> are > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> several changes open for review too. It would be nice if > >> people > >>>>>>>>> could > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> reply > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> to this email and share issues which should be part of that > >>>>>>> release, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> so > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> we > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> can decide on an initial cut-off date. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks in advance, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Laurent > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>>> > >> > >> > >
