I'll now kick off the RC creation process. On Sun, May 14, 2017 at 7:14 PM, Stephan Ewen <se...@apache.org> wrote:
> +1 for another RC once the shading fix is merged. > > Even if RC1 would have pending issues, users that want to help testing need > a new RC, because (as Robert said), RC0's shading issues make it virtually > unusable... > > On Fri, May 12, 2017 at 3:45 PM, Robert Metzger <rmetz...@apache.org> > wrote: > > > I'm considering to do another non voting RC1 on Monday with the > incremental > > checkpointing and maven fixes in (and of course as many other things as > > possible). Once the other critical fixes are in (ideally Tuesday or > > Wednesday), I'll create the first voting one. > > > > The current RC0 is almost unusable because of the incorrect shading. > > > > On Fri, May 12, 2017 at 2:09 PM, Greg Hogan <c...@greghogan.com> wrote: > > > > > +1 for sticking to the code freeze deadline and building a new release > > > candidate but since the release is still two weeks off (5/26) I think > it > > > better to delay voting to give time for additional bug fixes. > > > > > > > > > > On May 11, 2017, at 10:19 AM, Robert Metzger <rmetz...@apache.org> > > > wrote: > > > > > > > > It seems that we found quite a large number of critical issues in the > > > first > > > > RC. > > > > > > > > - FLINK-6537 Umbrella issue for fixes to incremental snapshots > (Stefan > > > has > > > > a PR open to fix the critical ones) > > > > - FLINK-6531 Deserialize checkpoint hooks with user classloader (has > a > > > > pending PR) > > > > - FLINK-6515 KafkaConsumer checkpointing fails because of ClassLoader > > > > issues (status unknown) > > > > - FLINK-6514 Cannot start Flink Cluster in standalone mode (Stephan > is > > on > > > > it) > > > > - FLINK-6508 Include license files of packaged dependencies (Stephan > is > > > on > > > > it) + FLINK-6501 Make sure NOTICE files are bundled into shaded JAR > > files > > > > - FLINK-6284 Incorrect sorting of completed checkpoints in > > > > ZooKeeperCompletedCheckpointStore (unknown) > > > > > > > > I would like to get these issues fixed by end of this week (Sunday), > so > > > > that I can create the first voting RC on Monday morning. > > > > Please reject if you think we will not manage to get the stuff fixed > > > until > > > > then. > > > > > > > > > > > > > > > > On Thu, May 11, 2017 at 10:54 AM, Till Rohrmann < > trohrm...@apache.org> > > > > wrote: > > > > > > > >> Unfortunately, it won't be fully functional in 1.3. > > > >> > > > >> Cheers, > > > >> Till > > > >> > > > >> On Thu, May 11, 2017 at 10:45 AM, Renjie Liu < > liurenjie2...@gmail.com > > > > > > >> wrote: > > > >> > > > >>> @Rohrmann Will FLIP 6 be fully functional in 1.3 release? > > > >>> > > > >>> On Thu, May 11, 2017 at 4:12 PM Gyula Fóra <gyula.f...@gmail.com> > > > wrote: > > > >>> > > > >>>> Thanks Stefan! > > > >>>> Gyula > > > >>>> > > > >>>> Stefan Richter <s.rich...@data-artisans.com> ezt írta (időpont: > > 2017. > > > >>> máj. > > > >>>> 11., Cs, 10:04): > > > >>>> > > > >>>>> > > > >>>>> Hi, > > > >>>>> > > > >>>>> Thanks for reporting this. I found a couple of issues yesterday > > and I > > > >>> am > > > >>>>> currently working on a bundle of fixes. I will take a look at > this > > > >>>> problem, > > > >>>>> and if it is already covered. > > > >>>>> > > > >>>>> Best, > > > >>>>> Stefan > > > >>>>> > > > >>>>>> Am 11.05.2017 um 09:47 schrieb Gyula Fóra <gyula.f...@gmail.com > >: > > > >>>>>> > > > >>>>>> Hi, > > > >>>>>> I am not sure if this belong to this thread, but while trying to > > > >> run > > > >>> a > > > >>>>> job > > > >>>>>> with rocks incremental backend I ran into 2 issues: > > > >>>>>> > > > >>>>>> One with savepoints, I can't figure out because I can't make > sense > > > >> of > > > >>>> the > > > >>>>>> error or how it happenned: > > > >>>>>> The error stack trace is here: > > > >>>>>> https://gist.github.com/gyfora/2f7bb387bbd9f455f9702908cde0b239 > > > >>>>>> This happens on every savepoint attempt and seems to be related > to > > > >>> the > > > >>>>>> kafka source. Interestingly other tasks succeed in writing data > to > > > >>>> hdfs. > > > >>>>>> > > > >>>>>> The other one is covered by > > > >>>>> https://issues.apache.org/jira/browse/FLINK-6531 I > > > >>>>>> guess. I am not sure if the first one is related though. > > > >>>>>> > > > >>>>>> Thank you! > > > >>>>>> Gyula > > > >>>>>> > > > >>>>>> Till Rohrmann <trohrm...@apache.org> ezt írta (időpont: 2017. > > máj. > > > >>>> 11., > > > >>>>> Cs, > > > >>>>>> 9:14): > > > >>>>>> > > > >>>>>>> Hi Renjie, > > > >>>>>>> > > > >>>>>>> 1.3 already contains some Flip-6 code. However, it is not yet > > > >> fully > > > >>>>>>> functional. What you already can do is to run local jobs on the > > > >>> Flip-6 > > > >>>>> code > > > >>>>>>> base by instantiating a MiniCluster and then using the > > > >>>>>>> Flip6LocalStreamEnvironment. > > > >>>>>>> > > > >>>>>>> Cheers, > > > >>>>>>> Till > > > >>>>>>> > > > >>>>>>> On Thu, May 11, 2017 at 6:00 AM, Renjie Liu < > > > >>> liurenjie2...@gmail.com> > > > >>>>>>> wrote: > > > >>>>>>> > > > >>>>>>>> Hi, all: > > > >>>>>>>> Will the FLIP 6 be included in release 1.3? > > > >>>>>>>> > > > >>>>>>>> On Wed, May 10, 2017 at 9:48 PM Gyula Fóra < > > gyula.f...@gmail.com > > > >>> > > > >>>>> wrote: > > > >>>>>>>> > > > >>>>>>>>> Thanks you! :) > > > >>>>>>>>> > > > >>>>>>>>> Chesnay Schepler <ches...@apache.org> ezt írta (időpont: > 2017. > > > >>> máj. > > > >>>>>>> 10., > > > >>>>>>>>> Sze, 15:44): > > > >>>>>>>>> > > > >>>>>>>>>> I guess it is related to this one > > > >>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6514 ? > > > >>>>>>>>>> > > > >>>>>>>>>> On 10.05.2017 15:34, Gyula Fóra wrote: > > > >>>>>>>>>>> Hi, > > > >>>>>>>>>>> > > > >>>>>>>>>>> I tried to run an application on 1.3 but I keep getting the > > > >>>>>>> following > > > >>>>>>>>>> error: > > > >>>>>>>>>>> java.lang.NoClassDefFoundError: Could not initialize class > > > >>>>>>>>>>> org.apache.hadoop.security.UserGroupInformation > > > >>>>>>>>>>> at > > > >>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>> org.apache.flink.runtime.security.modules.HadoopModule. > > > >>>>>>>> install(HadoopModule.java:45) > > > >>>>>>>>>>> at > > > >>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>> org.apache.flink.runtime.security.SecurityUtils. > > > >>>>>>>> install(SecurityUtils.java:78) > > > >>>>>>>>>>> at org.apache.flink.client.CliFrontend.main(CliFrontend. > > > >>> java:1128) > > > >>>>>>>>>>> > > > >>>>>>>>>>> Even after adding hadoop-common to the lib manually (which > I > > > >>> guess > > > >>>>>>>>> should > > > >>>>>>>>>>> not be necessary). > > > >>>>>>>>>>> > > > >>>>>>>>>>> Any idea what might cause this? > > > >>>>>>>>>>> > > > >>>>>>>>>>> Thanks, > > > >>>>>>>>>>> Gyula > > > >>>>>>>>>>> > > > >>>>>>>>>>> Chesnay Schepler <ches...@apache.org> ezt írta (időpont: > > > >> 2017. > > > >>>>>>> máj. > > > >>>>>>>>> 9., > > > >>>>>>>>>> K, > > > >>>>>>>>>>> 13:27): > > > >>>>>>>>>>> > > > >>>>>>>>>>>> I looked into AppVeyor and managed to create a working > setup > > > >>> that > > > >>>>>>>>>>>> replicates the problems i encounter locally. > > > >>>>>>>>>>>> https://ci.appveyor.com/project/zentol/flink/build/1.0.12 > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> The integration should be as simple as placing an > > > >> appveyor.yml > > > >>>>>>>>> (similar > > > >>>>>>>>>>>> to the travis.yml) into the repository and asking INFRA to > > > >>>> enabled > > > >>>>>>>> it > > > >>>>>>>>>>>> for Flink. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> I've opened a JIRA to track this initiative: > > > >>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6510 > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> On 09.05.2017 10:17, Robert Metzger wrote: > > > >>>>>>>>>>>>> Chesnay, I would propose to look into a CI solution for > > > >>> Windows. > > > >>>>>>>> I'll > > > >>>>>>>>>> try > > > >>>>>>>>>>>>> out appveyor.com for my Flink fork and report back how > > well > > > >>> it > > > >>>>>>>> works > > > >>>>>>>>>>>> (let > > > >>>>>>>>>>>>> me know if you want to look into it). > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> On Tue, May 9, 2017 at 10:05 AM, Stephan Ewen < > > > >>> se...@apache.org > > > >>>>> > > > >>>>>>>>>> wrote: > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>>> @Renjie What functionality are you looking for? Most > stuff > > > >>>>>>> should > > > >>>>>>>>> work > > > >>>>>>>>>>>> in > > > >>>>>>>>>>>>>> the normal master branch. > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> On Tue, May 9, 2017 at 10:02 AM, Renjie Liu < > > > >>>>>>>>> liurenjie2...@gmail.com> > > > >>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> Stephan: > > > >>>>>>>>>>>>>>> This means that we can use FLIP 6 for basic > > functionality > > > >>>> now? > > > >>>>>>>> Any > > > >>>>>>>>>>>>>>> document for setting up that? Will FLIP 6 be finished > in > > > >>>>>>> release > > > >>>>>>>>> 1.3? > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> On Tue, May 9, 2017 at 3:43 PM Stephan Ewen < > > > >>> se...@apache.org > > > >>>>> > > > >>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> @Renjie Liu: A good part of the FLIP-6 code is in > there, > > > >>> but > > > >>>>>>> not > > > >>>>>>>>>> all. > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> It does run well on Yarn, Mesos, Docker, etc. > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> We need to finish the FLIP-6 work mainly for full > > > >>> elasticity. > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> On Tue, May 9, 2017 at 5:24 AM, Renjie Liu < > > > >>>>>>>>> liurenjie2...@gmail.com > > > >>>>>>>>>>> > > > >>>>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> Hi, does this include the FLIP6? > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> On Tue, May 9, 2017 at 2:29 AM Stephan Ewen < > > > >>>>>>> se...@apache.org> > > > >>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>>>> Did a quick test: Simply adding the > > > >>>>>>> "org.apache.maven.plugins. > > > >>>>>>>>>>>>>>>>>> shade.resource.ApacheNoticeResourceTransformer" > helps > > > >>> with > > > >>>>>>>>> NOTICE > > > >>>>>>>>>>>>>>>>>> files, > > > >>>>>>>>>>>>>>>>>> but does not add the required BSD licence copies. > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> On Mon, May 8, 2017 at 8:25 PM, Stephan Ewen < > > > >>>>>>>> se...@apache.org> > > > >>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>>>>> I did the first pass for the legal check. > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> - Source LICENSE and NOTICE are okay > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> - In the shaded JAR files, we are not bundling > the > > > >>>>>>>> license > > > >>>>>>>>>> and > > > >>>>>>>>>>>>>>>>>>> notice files of the dependencies we include in the > > > >>> shaded > > > >>>>>>>> jars. > > > >>>>>>>>>>>>>>>>>>> => Not a problem for Guava (Apache Licensed) > > > >>>>>>>>>>>>>>>>>>> => I think is a problem for ASM > (redistribution > > > >> in > > > >>>>>>>>> binary > > > >>>>>>>>>>>> form, > > > >>>>>>>>>>>>>>>>>>> hence needs a notice of the copy) > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> - The Table API / SQL module needs more entries > for > > > >>>>>>>> Janino > > > >>>>>>>>> / > > > >>>>>>>>>>>>>>>>>>> Reflections (both BSD licensed) > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> So that is definitely a blocker. > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> On Mon, May 8, 2017 at 12:14 PM, Robert Metzger < > > > >>>>>>>>>>>> rmetz...@apache.org > > > >>>>>>>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> Hi Devs, > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> I've created a first non-voting release candidate > > for > > > >>>>>>> Flink > > > >>>>>>>>>> 1.3.0. > > > >>>>>>>>>>>>>>>>>>>> Please use this RC to test as much as you can and > > > >>> provide > > > >>>>>>>>>> feedback > > > >>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>>>>>> the Flink community. The more we find and fix now, > > > >> the > > > >>>>>>>> better > > > >>>>>>>>>>>> Flink > > > >>>>>>>>>>>>>> 1.3.0 > > > >>>>>>>>>>>>>>>>>>>> wil be :) > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> I've CC'ed the user@ mailing list to get more > > people > > > >>> to > > > >>>>>>>> test > > > >>>>>>>>>> it. > > > >>>>>>>>>>>> DO > > > >>>>>>>>>>>>>>>>>>>> NOT USE THIS RELEASE CANDIDATE IN PRODUCTION. > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> I will prepare a google document to synchronize > the > > > >>>>>>> testing > > > >>>>>>>>>>>> effort a > > > >>>>>>>>>>>>>>>>>>>> bit more. > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> Depending on the number of issues we identify, I > > hope > > > >>>> that > > > >>>>>>>> we > > > >>>>>>>>>> can > > > >>>>>>>>>>>> do > > > >>>>>>>>>>>>>>>>>>>> the first VOTEing RC early next week. > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> ------------------------------ > > > >>>>>>>> ------------------------------ > > > >>>>>>>>>>>>>> --------- > > > >>>>>>>>>>>>>>>>>>>> The release commit is > > > >>>>>>>>> f94c002991dcce9f1104f8e61b43efb2f8247cb4, > > > >>>>>>>>>>>>>>>>>>>> located here: > > > >>>>>>> http://git-wip-us.apache.org/repos/asf/flink/ > > > >>>>>>>>>>>>>>>>>>>> commit/f94c0029 > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> The artifacts are located here: > > http://people.apache > > > >> . > > > >>>>>>>>>>>>>>>>>>>> org/~rmetzger/flink-1.3.0-rc0/ > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> The maven staging repository is here: > > > >>> https://repository > > > >>>> . > > > >>>>>>>>>>>>>>>>>>>> apache.org/content/repositorie > s/orgapacheflink-1118 > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> ------------------------------ > > > >>>>>>>> ------------------------------ > > > >>>>>>>>>>>>>> --------- > > > >>>>>>>>>>>>>>>>>>>> Happy testing! > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> Regards, > > > >>>>>>>>>>>>>>>>>>>> Robert > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> -- > > > >>>>>>>>>>>>>>>>> Liu, Renjie > > > >>>>>>>>>>>>>>>>> Software Engineer, MVAD > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> -- > > > >>>>>>>>>>>>>>> Liu, Renjie > > > >>>>>>>>>>>>>>> Software Engineer, MVAD > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>> -- > > > >>>>>>>> Liu, Renjie > > > >>>>>>>> Software Engineer, MVAD > > > >>>>>>>> > > > >>>>>>> > > > >>>>> > > > >>>>> > > > >>>> > > > >>> -- > > > >>> Liu, Renjie > > > >>> Software Engineer, MVAD > > > >>> > > > >> > > > > > > > > >
