When I disabled doclint and there were around 322 java doc warnings. I'm not sure if we can fix all of them and I'm attaching a link to the log file that contains all the warnings. Please let me know your thoughts on this.
https://drive.google.com/file/d/1GQ6xORr0HkoEmorzvTcl0XZS4NeLXu83/view?usp=sharing On Thu, Mar 8, 2018 at 10:39 AM, Jyothsna Reddy <[email protected]> wrote: > I tried to disable it using -Dadditionalparam=-Xdoclint:none. Somewhere I > have read that -DXdoclint:none is for older versions of maven. But I'm not > sure of it. > > > > > On Thu, Mar 8, 2018 at 10:18 AM, Jyothsna Reddy <[email protected]> > wrote: > >> Following branch has changed related to disabling doclint in pom.xml. >> Please check it out. >> >> https://github.com/dvjyothsna/drill.git DRILL-4547 >> >> >> >> >> >> >> On Thu, Mar 8, 2018 at 4:19 AM, Parth Chandra <[email protected]> wrote: >> >>> The issue with the failing test in TestDrillbitResilience. >>> cancelAfterAllResultsProduced is similar to DRILL-3967 ( >>> TestDrillbitResilience.cancelAfterEverythingIsCompleted failure). >>> >>> In both this test case, and DRILL-3967, the query is paused (in different >>> places) and a cancel is sent. The query is then resumed and the resulting >>> state is checked. The problem is that the tests have a race condition >>> between the cancellation and the resuming of the query. Sometimes the >>> resume reaches first and sometimes the cancel reaches first. The failure >>> described by Volodymyr is caused by this race condition. I don't know why >>> the test was done like this, but this is an existing problem and >>> shouldn't >>> hold up the release. >>> >>> However, I also see a failure where we encounter an illegal state >>> transition (the query state is CANCELLATION_REQUESTED and the Foreman >>> tries >>> to move to an ENQUEUED state). This happens once in about twenty-five >>> executions. The Foreman only requests ENQUEUE once, when the query is >>> about >>> to start, so this means the cancellation request reached before the query >>> start request. How this happens in the unit test I have not been able to >>> determine yet (because it really shouldn't be possible). >>> >>> To recreate the problem I simply added a repeat rule in the class and >>> set a >>> repeat count of 1000. The problem occurs easily if the test is run from >>> the >>> command line. When running in debug, I was unable to see the problem. >>> >>> I'll spend some more time on this, but just in case someone wants to >>> investigate further, feel free ... >>> >>> >>> On Thu, Mar 8, 2018 at 10:59 AM, Parth Chandra <[email protected]> >>> wrote: >>> >>> > Not sure if that would work. The release build does not allow >>> uncommitted >>> > files, so I have to commit pom.xml changes to at least the local repo, >>> > which will get pushed to my public repo when the release is done. Not >>> > committing this to Apache master would be cheating would leave us with >>> a >>> > build that does not match any source in Apache master? Javadoc >>> generated is >>> > never committed to any repo. It is part of the src release jars, AFAIK. >>> > >>> > Also, I'm not sure where in the pom Jyothsna made the change; I added >>> the >>> > -Dxoclint:none to the build section of the apache-release profile and >>> java >>> > exec still gives over 100 javadoc errors. >>> > >>> > We have to fix these one of these days. Might as well do it now. >>> Knowing >>> > how it works, if we don't fix these now, someone will be scrambling to >>> fix >>> > these just before the 1.14.0 release :( >>> > >>> > >>> > >>> > On Thu, Mar 8, 2018 at 10:06 AM, Aman Sinha <[email protected]> >>> wrote: >>> > >>> >> Parth, would it work if you made the pom.xml changes locally in your >>> >> branch, generated the javadoc but only commit the javadoc jar files >>> to the >>> >> release branch, not the pom.xml changes ? >>> >> Anyone downloading Drill source code to build should not run into this >>> >> since typically they won't be building javadoc. >>> >> >>> >> -Aman >>> >> >>> >> On Wed, Mar 7, 2018 at 6:37 PM, Parth Chandra <[email protected]> >>> wrote: >>> >> >>> >> > Unfortunately, we cannot do that since we also want to be able to >>> build >>> >> > with JDK 7 for at least a couple of releases to allow for a >>> reasonable >>> >> > transition time. doclint was introduced in JDK 8 so JDK 7 fails >>> >> because it >>> >> > doesn't recognize the parameter. >>> >> > >>> >> > >>> >> > >>> >> > On Thu, Mar 8, 2018 at 7:03 AM, Jyothsna Reddy < >>> [email protected]> >>> >> > wrote: >>> >> > >>> >> > > Regarding DRILL-4547, I used Vladimir's branch(DRILL-1491) and >>> added >>> >> > > following lines to pom.xml to disable doc lint. The javadoc >>> doesn't >>> >> throw >>> >> > > any errors and the build is successful. >>> >> > > >>> >> > > <activation> >>> >> > > >>> >> > > <jdk>[1.8,)</jdk> >>> >> > > >>> >> > > </activation> >>> >> > > >>> >> > > <properties> >>> >> > > >>> >> > > <additionalparam>-Xdoclint:none</additionalparam> >>> >> > > >>> >> > > </properties> >>> >> > > >>> >> > > >>> >> > > >>> >> > > >>> >> > > >>> >> > > On Wed, Mar 7, 2018 at 3:08 PM, Hanumath Rao Maduri < >>> >> [email protected]> >>> >> > > wrote: >>> >> > > >>> >> > > > On my machine I couldn't repro the issue related to >>> >> > > TestDrillbitResilience. >>> >> > > > cancelAfterAllResultsProduced. >>> >> > > > I used the vladimir's branch (i.e DRILL-1491). >>> >> > > > Used the maven test command for testing it. >>> >> > > > >>> >> > > > output of the test run. >>> >> > > > ... 4 common frames omitted >>> >> > > > Tests run: 20, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: >>> >> > 124.187 >>> >> > > > sec - in org.apache.drill.exec.server.TestDrillbitResilience >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > > > On Wed, Mar 7, 2018 at 11:00 AM, Parth Chandra < >>> [email protected]> >>> >> > > wrote: >>> >> > > > >>> >> > > > > Yes I agree. JDBC would be a new feature that we can defer to >>> >> 1.14.0. >>> >> > > > > I'm hoping we can resolve the other three in the next few >>> days. >>> >> > Target >>> >> > > > date >>> >> > > > > for starting release process - Friday Mar 9th >>> >> > > > > >>> >> > > > > Once these are resolved, I will create a branch for the >>> release so >>> >> > that >>> >> > > > > Apache master remains open for commits. If any issues are >>> found in >>> >> > the >>> >> > > > > release branch, we will fix them in master and I will >>> cherry-pick >>> >> the >>> >> > > > into >>> >> > > > > the release branch. Once the release is finalized I will add a >>> >> > release >>> >> > > > tag >>> >> > > > > and remove the branch. >>> >> > > > > >>> >> > > > > Also note if QA folks want to get started on testing the >>> release, >>> >> the >>> >> > > > > current head of Apache master is close to final. Javadoc >>> >> generation >>> >> > is >>> >> > > > only >>> >> > > > > a release build issue, and the other issues are localized to >>> >> > specific >>> >> > > > > cases. >>> >> > > > > >>> >> > > > > Note: to reproduce the javadoc issues: >>> >> > > > > # set JAVA_HOME to JDK 8 >>> >> > > > > mvn javadoc:javadoc -Papache-release >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > > On Wed, Mar 7, 2018 at 11:23 PM, Aman Sinha < >>> [email protected] >>> >> > >>> >> > > > wrote: >>> >> > > > > >>> >> > > > > > It seems to me the main blockers are: >>> >> > > > > > >>> >> > > > > > 1. DRILL-4547 Javadoc fails with Java8 <-- Can we >>> split up >>> >> the >>> >> > > > work >>> >> > > > > > among few people to resolve these ? >>> >> > > > > > 2. DRILL-6216 Metadata mismatch.. <-- Agreement >>> was >>> >> to >>> >> > > > revert >>> >> > > > > > one small piece of code and it appears Sorabh is looking >>> into it >>> >> > > > > > 3. TestDrillbitResilience.cancelAfterAllResultsProduced >>> <-- >>> >> need >>> >> > > > > someone >>> >> > > > > > to look into this >>> >> > > > > > >>> >> > > > > > Regarding the JDBC issues that Parth mentioned, looking at >>> the >>> >> > JIRAs, >>> >> > > > it >>> >> > > > > > seems they are not showstoppers...Parth do you agree ? >>> >> > > > > > >>> >> > > > > > Since we are close to the finish line for JDK 8, IMO we >>> should >>> >> try >>> >> > > and >>> >> > > > > see >>> >> > > > > > if in another day or two we can get over these hurdles. >>> >> > > > > > >>> >> > > > > > -Aman >>> >> > > > > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > > > On Wed, Mar 7, 2018 at 7:17 AM, Pritesh Maker < >>> [email protected]> >>> >> > > wrote: >>> >> > > > > > >>> >> > > > > > > The JDK 8 issues will likely require more time to harden >>> for >>> >> it >>> >> > to >>> >> > > be >>> >> > > > > > > included in the 1.13 release. My recommendation would be >>> to >>> >> move >>> >> > > > ahead >>> >> > > > > > with >>> >> > > > > > > the 1.13 release now and address these issues right. >>> >> > > > > > > >>> >> > > > > > > Pritesh >>> >> > > > > > > >>> >> > > > > > > -----Original Message----- >>> >> > > > > > > From: Parth Chandra <[email protected]> >>> >> > > > > > > Sent: March 7, 2018 3:34 AM >>> >> > > > > > > To: dev <[email protected]> >>> >> > > > > > > Subject: Re: [DISCUSS] 1.13.0 release >>> >> > > > > > > >>> >> > > > > > > My mistake Volodymyr. >>> >> > > > > > > >>> >> > > > > > > Found some other JDK 8 issues in JIRA not tracked in >>> >> DRILL-1491 >>> >> > > > > > > >>> >> > > > > > > DRILL-4547 Javadoc fails with Java8 >>> >> > > > > > > DRILL-6163 Switch Travis To Java 8 >>> >> > > > > > > >>> >> > > > > > > The following are tracked in DRILL-1491, but it doesn't >>> look >>> >> like >>> >> > > > we're >>> >> > > > > > > addressing these. Are we? >>> >> > > > > > > >>> >> > > > > > > DRILL-4329 13 Unit tests are failing with JDK 8 >>> >> > > > > > > DRILL-4333 DRILL-4329 tests in >>> >> > > > > > > Drill2489CallsAfterCloseThrowExceptionsTest fail in Java >>> 8 >>> >> > > > > > > DRILL-5120 Upgrade JDBC Driver for new Java 8 methods >>> >> > > > > > > DRILL-5680 BasicPhysicalOpUnitTest can't run in >>> Eclipse >>> >> with >>> >> > > > Java >>> >> > > > > 8 >>> >> > > > > > > >>> >> > > > > > > >>> >> > > > > > > *DRILL-4547 is a showstopper*. The release build >>> >> > (-Papache-release) >>> >> > > > > fails >>> >> > > > > > > with far too many Javadoc errors even with doc lint turned >>> >> off. >>> >> > > > > > > >>> >> > > > > > > DRILL-4333, DRILL-4329, DRILL-5120 are JDBC related which >>> is a >>> >> > > > project >>> >> > > > > by >>> >> > > > > > > itself. >>> >> > > > > > > >>> >> > > > > > > Note that fixing JDBC related issues and adding the >>> command >>> >> line >>> >> > > > option >>> >> > > > > > to >>> >> > > > > > > turn doc lint off will likely break Java 7 builds. >>> >> > > > > > > >>> >> > > > > > > >>> >> > > > > > > Folks who voted to get JDK 8 into this release, what is >>> the >>> >> > > consensus >>> >> > > > > on >>> >> > > > > > > JDBC/Java8 ? >>> >> > > > > > > Also, any volunteers on helping debug >>> >> > > > > > > TestDrillbitResilience.cancelAfterAllResultsProduced >>> >> > > > > > > ? >>> >> > > > > > > >>> >> > > > > > > >>> >> > > > > > > >>> >> > > > > > > On Wed, Mar 7, 2018 at 3:20 PM, Volodymyr Tkach < >>> >> > > > [email protected] >>> >> > > > > > >>> >> > > > > > > wrote: >>> >> > > > > > > >>> >> > > > > > > > Addition to my last message: >>> >> > > > > > > > The link with PR for DRILL-1491 >>> >> https://urldefense.proofpoint >>> >> > . >>> >> > > > > > > com/v2/url?u=https-3A__github. >>> com_apache_drill_pull_1143&d= >>> >> > > DwIBaQ&c= >>> >> > > > > > > cskdkSMqhcnjZxdQVpwTXg&r=zySISmkmM4WNViCKijENtQ&m= >>> >> > > oTnKwfjj5hFBosMrq_ >>> >> > > > > > > WWhazhGeoC2nGSKeMOPxU2_cM&s=p3 >>> uialdRhgnf3XRY22R4SWXGZIq66a >>> >> > > > > > Pijuy-Ms0J_-4&e= >>> >> > > > > > > > on which the we can see TestDrillbitResilience. >>> >> > > > > > > > cancelAfterAllResultsProduced >>> >> > > > > > > > failure. >>> >> > > > > > > > >>> >> > > > > > > > 2018-03-07 11:45 GMT+02:00 Volodymyr Tkach < >>> >> > > [email protected] >>> >> > > > >: >>> >> > > > > > > > >>> >> > > > > > > > > *To Parth:* >>> >> > > > > > > > > The failure can only be seen if run on DRILL-1491 >>> branch, >>> >> > > because >>> >> > > > > it >>> >> > > > > > > uses >>> >> > > > > > > > > jdk 1.8 in pom.xml >>> >> > > > > > > > > >>> >> > > > > > > > > <source>1.8</source> >>> >> > > > > > > > > <target>1.8</target> >>> >> > > > > > > > > >>> >> > > > > > > > > 2018-03-07 6:03 GMT+02:00 Sorabh Hamirwasia < >>> >> > > > [email protected] >>> >> > > > > >: >>> >> > > > > > > > > >>> >> > > > > > > > >> Just sent an email on RCA of DRILL-6216 to discuss >>> next >>> >> > steps. >>> >> > > > > > > > >> >>> >> > > > > > > > >> >>> >> > > > > > > > >> Thanks, >>> >> > > > > > > > >> Sorabh >>> >> > > > > > > > >> >>> >> > > > > > > > >> ________________________________ >>> >> > > > > > > > >> From: Parth Chandra <[email protected]> >>> >> > > > > > > > >> Sent: Tuesday, March 6, 2018 6:48:21 PM >>> >> > > > > > > > >> To: dev >>> >> > > > > > > > >> Subject: Re: [DISCUSS] 1.13.0 release >>> >> > > > > > > > >> >>> >> > > > > > > > >> We have two items remaining - >>> >> > > > > > > > >> >>> >> > > > > > > > >> DRILL-1491 - Ideally, I would like to make sure that >>> >> CANCEL >>> >> > is >>> >> > > > > > handled >>> >> > > > > > > > >> correctly with JDK 8. If the failure of the unit >>> test is >>> >> > > because >>> >> > > > > the >>> >> > > > > > > > >> cancel >>> >> > > > > > > > >> is received after the query is completed, then the >>> issue >>> >> is >>> >> > > less >>> >> > > > > > > severe, >>> >> > > > > > > > >> but I would like to be sure that this is the case. >>> >> > > > > > > > >> Are there others who see the DrillbitResilience tests >>> >> > failing >>> >> > > > for >>> >> > > > > > > them? >>> >> > > > > > > > >> Can >>> >> > > > > > > > >> we try to assist Volodymyr? I don't see the failures >>> >> myself. >>> >> > > > > > > > >> >>> >> > > > > > > > >> DRILL-6216 - this is a showstopper. >>> >> > > > > > > > >> >>> >> > > > > > > > >> >>> >> > > > > > > > >> On Wed, Mar 7, 2018 at 5:27 AM, Kunal Khatua < >>> >> > > > > [email protected] >>> >> > > > > > > >>> >> > > > > > > > >> wrote: >>> >> > > > > > > > >> >>> >> > > > > > > > >> > Hi Parth >>> >> > > > > > > > >> > >>> >> > > > > > > > >> > DRILL-6216 is a release blocker that is being >>> currently >>> >> > > looked >>> >> > > > > > into. >>> >> > > > > > > > >> > >>> >> > > > > > > > >> > Ref: >>> >> > > > > > > > >> > DRILL-6216: Metadata mismatch when connecting to a >>> >> Drill >>> >> > > > 1.12.0 >>> >> > > > > > > with a >>> >> > > > > > > > >> > Drill-1.13.0-SNAPSHOT driver >>> >> > > > > > > > >> > https://urldefense.proofpoint. >>> >> > com/v2/url?u=https-3A__issues >>> >> > > . >>> >> > > > > > > > >> apache.org_jira_browse_DRILL-2D6216&d=DwIBaQ&c= >>> >> > cskdkSMqhcnjZ >>> >> > > > > > > > >> xdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOt >>> >> > hF2jlYxvhTlQg&m=xu >>> >> > > > > > > > >> Rz02Sbprxvbtw1OrBuDvlRbp2lh9mz >>> >> > 3sxpP5-wHPs&s=txeKaKzF67flAi48 >>> >> > > > > > > > >> DUNLgMWbxje1GXWxfFpG6BEPXk0&e= >>> >> > > > > > > > >> > >>> >> > > > > > > > >> > Please add it to the list of required commits as >>> well. >>> >> > > > > > > > >> > >>> >> > > > > > > > >> > Thanks >>> >> > > > > > > > >> > ~ Kunal >>> >> > > > > > > > >> > On 3/6/2018 9:53:06 AM, Volodymyr Tkach < >>> >> > > > [email protected]> >>> >> > > > > > > > wrote: >>> >> > > > > > > > >> > Right now i haven't found the reason of >>> >> > > > > > > > >> > TestDrillbitResilience.cancelA >>> fterAllResultsProduced >>> >> > > failure, >>> >> > > > > > most >>> >> > > > > > > > >> likely >>> >> > > > > > > > >> > the cause of the failure is that the query is able >>> to >>> >> have >>> >> > > > been >>> >> > > > > > > > >> completed >>> >> > > > > > > > >> > before cancellation request is processed. >>> >> > > > > > > > >> > This test not only the case, there is one more >>> ignored >>> >> > test >>> >> > > > > > > > >> > TestDrillbitResilience.cancelA >>> >> fterEverythingIsCompleted >>> >> > and >>> >> > > > > jira >>> >> > > > > > > > >> > DRILL-3967 >>> >> > > > > > > > >> > created, although the environment is AWS. >>> >> > > > > > > > >> > >>> >> > > > > > > > >> > Maybe it makes sense to ignore this test to >>> unblock the >>> >> > > > release >>> >> > > > > > and >>> >> > > > > > > > >> merge >>> >> > > > > > > > >> > JDK8 changes? >>> >> > > > > > > > >> > >>> >> > > > > > > > >> >>> >> > > > > > > > > >>> >> > > > > > > > > >>> >> > > > > > > > >>> >> > > > > > > >>> >> > > > > > >>> >> > > > > >>> >> > > > >>> >> > > >>> >> > >>> >> >>> > >>> > >>> >> >> >
