I just deleted the Record API to check what would break. Doesn't look too scary, just a few tests that need to be adapted. I'm right in the middle of that. Hope to open a PR soon.
2015-10-22 16:42 GMT+02:00 Chesnay Schepler <ches...@apache.org>: > @RecordAPI: Yes, i was curious where we are at regarding the removal of > the Record API. > If there are still tests left to port (or other related things) I'd be > more than happy to do it (got a /lot/ of free time on my hands). > The related JIRA issues weren't particularly helpful though in figuring > out what still needs to be done. > > @Project Restructuring: I prefer doing it now. > > > On 22.10.2015 15:54, Till Rohrmann wrote: > >> This reminded me that at some point we wanted to remove the old record >> API ( >> https://issues.apache.org/jira/browse/FLINK-1681). I think that Chesnay >> checked with Henry on this topic in JIRA. >> >> On Thu, Oct 22, 2015 at 3:38 PM, Fabian Hueske <fhue...@gmail.com> wrote: >> >> I'd like to bring up Vasia's question on the project structure. >>> >>> Stephan started the discussion and proposed a new project structure about >>> three weeks ago [1]. >>> The proposal was refined a bit and eventually backed by many +1s. >>> >>> Do we want to make this happen in 0.10 or do we postpone it after the >>> release? >>> >>> Cheers, >>> Fabian >>> >>> [1][ >>> >>> >>> http://mail-archives.apache.org/mod_mbox/flink-dev/201510.mbox/%3CCANC1h_u6qtEsF1WCcoU1d38JGd%2BXTAQWmvp9Stx4vfe68BOjBw%40mail.gmail.com%3E >>> >>> >>> 2015-10-22 15:10 GMT+02:00 Suneel Marthi <smar...@apache.org>: >>> >>> We r actually targeting Flink 0.10, since 0.10 would be out by the time >>>> >>> we >>> >>>> have Flink-Mahout integration in place. >>>> >>>> On Thu, Oct 22, 2015 at 9:02 AM, Till Rohrmann <trohrm...@apache.org> >>>> wrote: >>>> >>>> Forget my last mail. Just found out that the Mahout guys are still >>>>> >>>> running >>>> >>>>> on 0.9-SNAPSHOT. >>>>> >>>>> >>>>> On Thu, Oct 22, 2015 at 2:53 PM, Till Rohrmann <trohrm...@apache.org> >>>>> wrote: >>>>> >>>>> I found another issue (FLINK-2894 >>>>>> <https://issues.apache.org/jira/browse/FLINK-2894>) while helping >>>>>> >>>>> the >>> >>>> mahout guys with the flink bindings for Samsara. Currently we don't >>>>>> >>>>> allow >>>> >>>>> to register default serializer for Kryo. This means it is not >>>>>> >>>>> possible >>> >>>> to >>>> >>>>> specify a serializer for a base class and all its subclasses. In >>>>>> >>>>> order >>> >>>> to >>>> >>>>> push the flink integration with Samsara, I would like to include a >>>>>> >>>>> fix >>> >>>> for >>>>> >>>>>> it in 0.10. >>>>>> >>>>>> On Thu, Oct 22, 2015 at 2:19 PM, Sachin Goel < >>>>>> >>>>> sachingoel0...@gmail.com >>> >>>> wrote: >>>>>> >>>>>> On Oct 22, 2015 3:01 PM, "Maximilian Michels" <m...@apache.org> >>>>>>> >>>>>> wrote: >>> >>>> @Stephan: That's right, the detached mode is very useful for >>>>>>>> >>>>>>> streaming >>>> >>>>> programs. Let's see if we can merge Sachin's pull request to give >>>>>>>> >>>>>>> more >>>> >>>>> meaningful exceptions in case of user programs which are not >>>>>>>> compatible with the detached execution mode. >>>>>>>> >>>>>>> I'll leave it up to you guys to make a decision on it then. But >>>>>>> >>>>>> something >>>>> >>>>>> needs to be done about it. :') >>>>>>> >>>>>>> @Vasia: That's a feature :) You can adjust the number of old jobs >>>>>>>> >>>>>>> to >>> >>>> be kept by setting 'jobmanager.web.history' (default is 5). >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Oct 22, 2015 at 11:28 AM, Vasiliki Kalavri >>>>>>>> <vasilikikala...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I found an issue with the web interface. It only shows the last >>>>>>>>> >>>>>>>> 5 >>> >>>> finished >>>>>>> >>>>>>>> jobs and the "Jobs Finished" counter also goes up to 5. >>>>>>>>> I found FLINK-2206, but it seems that this was fixed for the old >>>>>>>>> >>>>>>>> interface >>>>>>> >>>>>>>> and broken again in the new one? >>>>>>>>> Shall I open another issue or is there something I need to >>>>>>>>> >>>>>>>> configure >>>> >>>>> in >>>>>>> the >>>>>>> >>>>>>>> new web interface and I haven't? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> -V. >>>>>>>>> >>>>>>>>> On 22 October 2015 at 11:20, Stephan Ewen <se...@apache.org> >>>>>>>>> >>>>>>>> wrote: >>>> >>>>> I am onto FLINK-2800 and FLINK-2888 >>>>>>>>>> >>>>>>>>>> I would not disable YARN detached mode, it is used quite a bit >>>>>>>>>> >>>>>>>>> by >>> >>>> streaming >>>>>>> >>>>>>>> users and makes perfect sense for streaming jobs, which are >>>>>>>>>> >>>>>>>>> always >>>> >>>>> one-shot >>>>>>> >>>>>>>> currently. >>>>>>>>>> >>>>>>>>>> On Thu, Oct 22, 2015 at 11:03 AM, Maximilian Michels < >>>>>>>>>> >>>>>>>>> m...@apache.org >>>>> >>>>>> wrote: >>>>>>>>>> >>>>>>>>>> I had a gut feeling this wouldn't be the last RC :) >>>>>>>>>>> >>>>>>>>>>> I second Stephan, Ufuk, and Fabian. It's a good idea to start >>>>>>>>>>> >>>>>>>>>> testing the >>>>>>> >>>>>>>> release candidate. We might discover more issues on the way. >>>>>>>>>>> >>>>>>>>>> In >>> >>>> the >>>>> >>>>>> meantime, let's fix FLINK-2763 and FLINK-2800 and push those >>>>>>>>>>> >>>>>>>>>> to >>> >>>> the >>>>> >>>>>> release-0.10 branch. >>>>>>>>>>> >>>>>>>>>>> Eager execution calls are not supported by the detached YARN >>>>>>>>>>> >>>>>>>>>> execution >>>>>>> >>>>>>>> mode. The Flink standalone cluster mode doesn't have a >>>>>>>>>>> >>>>>>>>>> detached >>> >>>> mode. >>>>>>> >>>>>>>> Sachin was working on throwing a proper exception for >>>>>>>>>>> >>>>>>>>>> programs >>> >>>> that >>>>> >>>>>> try >>>>>>> >>>>>>>> to >>>>>>>>>> >>>>>>>>>>> submit a job which contains eager execution calls. In the >>>>>>>>>>> >>>>>>>>>> course >>>> >>>>> of >>>>> >>>>>> throwing a proper exception, he also added detached execution >>>>>>>>>>> >>>>>>>>>> mode >>>>> >>>>>> support >>>>>>>>>> >>>>>>>>>>> for the standalone cluster mode. The reason why eager >>>>>>>>>>> >>>>>>>>>> execution >>> >>>> doesn't >>>>>>> >>>>>>>> work with detached programs is that the client submits the >>>>>>>>>>> >>>>>>>>>> first >>>> >>>>> Flink >>>>>>> >>>>>>>> job >>>>>>>>>> >>>>>>>>>>> (call to execute/count/collect/print) to the cluster and then >>>>>>>>>>> >>>>>>>>>> returns an >>>>>>> >>>>>>>> empty ExecutionResult. This leads to error if the user >>>>>>>>>>> >>>>>>>>>> program >>> >>>> tries >>>>>>> to >>>>>>> >>>>>>>> access the ExecutionResult. >>>>>>>>>>> >>>>>>>>>>> Sachin's pull request hasn't been merged in time for the >>>>>>>>>>> >>>>>>>>>> release. I >>>>> >>>>>> would >>>>>>> >>>>>>>> like Flink to support detached execution mode but I suggest >>>>>>>>>>> >>>>>>>>>> to >>> >>>> disable >>>>>>> >>>>>>>> detached execution mode for YARN in this release. We can >>>>>>>>>>> >>>>>>>>>> include >>>> >>>>> a >>>>> >>>>>> proper >>>>>>> >>>>>>>> support for the next release. >>>>>>>>>>> >>>>>>>>>>> On Thu, Oct 22, 2015 at 9:50 AM, Sachin Goel < >>>>>>>>>>> >>>>>>>>>> sachingoel0...@gmail.com> >>>>>>> >>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Not sure if it's a blocker, but the yarn detached mode is >>>>>>>>>>>> >>>>>>>>>>> faulty >>>>> >>>>>> for >>>>>>> >>>>>>>> interactive programs with eager execution calls. The most >>>>>>>>>>>> >>>>>>>>>>> basic >>>> >>>>> starting >>>>>>>>>> >>>>>>>>>>> point for yarn, i.e. *bin/flink run -m yarn-cluster -yd -n >>>>>>>>>>>> >>>>>>>>>>> <> >>> >>>> examples/Wordcount.jar* fails in a bad way. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- Sachin Goel >>>>>>>>>>>> Computer Science, IIT Delhi >>>>>>>>>>>> m. +91-9871457685 >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Oct 22, 2015 at 3:57 AM, fhueske < >>>>>>>>>>>> >>>>>>>>>>> fhue...@gmail.com> >>> >>>> wrote: >>>>>>> >>>>>>>> +1 to that, Stephan. >>>>>>>>>>>>> >>>>>>>>>>>>> I can help with FLINK-2763 or FLINK-2800. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> From: Stephan Ewen >>>>>>>>>>>>> Sent: Thursday, October 22, 2015 0:02 >>>>>>>>>>>>> To: dev@flink.apache.org >>>>>>>>>>>>> Subject: Re: [VOTE] Release Apache Flink 0.10.0 >>>>>>>>>>>>> >>>>>>>>>>>> (release-0.10.0-rc0) >>>>>>> >>>>>>>> >>>>>>>>>>>>> From my side 2888 is a valid blocker. Aljoscha also found >>>>>>>>>>>>> >>>>>>>>>>>> another >>>>>>> >>>>>>>> blocker >>>>>>>>>>> >>>>>>>>>>>> bug, so this RC will need a few patches. >>>>>>>>>>>>> >>>>>>>>>>>>> I think for 2824 there was no consensus to what would >>>>>>>>>>>>> >>>>>>>>>>>> actually >>>>> >>>>>> be the >>>>>>> >>>>>>>> desired behavior, which makes it a bad candidate for a >>>>>>>>>>>>> >>>>>>>>>>>> release >>>>> >>>>>> blocker. >>>>>>>>>> >>>>>>>>>>> I would try and fix FLINK-2763 and FLINK-2800 if >>>>>>>>>>>>> >>>>>>>>>>>> possible, >>> >>>> but >>>>> >>>>>> not >>>>>>> >>>>>>>> block >>>>>>>>>>> >>>>>>>>>>>> the release on that. They seem to be very corner case. >>>>>>>>>>>>> >>>>>>>>>>>> Good >>> >>>> to >>>>> >>>>>> fix >>>>>>> >>>>>>>> them, >>>>>>>>>>> >>>>>>>>>>>> but not blockers. Too many people are on the 0.10 >>>>>>>>>>>>> >>>>>>>>>>>> SNAPSHOT >>> >>>> right >>>>>>> now >>>>>>> >>>>>>>> and >>>>>>>>>>> >>>>>>>>>>>> too many urgent fixes are in that people wait to be >>>>>>>>>>>>> >>>>>>>>>>>> available >>>> >>>>> in >>>>>>> a >>>>>>> >>>>>>>> release. >>>>>>>>>>>> >>>>>>>>>>>>> How about we start testing anyways, because I would >>>>>>>>>>>>> >>>>>>>>>>>> expect >>> >>>> us >>>> >>>>> to >>>>>>> find >>>>>>> >>>>>>>> more >>>>>>>>>>>> >>>>>>>>>>>>> issues, and we save time if we do not create a new >>>>>>>>>>>>> >>>>>>>>>>>> release >>> >>>> candidate >>>>>>> >>>>>>>> for >>>>>>>>>>> >>>>>>>>>>>> each patch. >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Oct 21, 2015 at 11:44 PM, Flavio Pompermaier < >>>>>>>>>>>>> >>>>>>>>>>>> pomperma...@okkam.it >>>>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> I would also point out that Flink-2763 and Flink-2800 >>>>>>>>>>>>>> >>>>>>>>>>>>> could >>>> >>>>> be >>>>>>> >>>>>>>> worth >>>>>>>>>> >>>>>>>>>>> of >>>>>>>>>>> >>>>>>>>>>>> further investigations before this release >>>>>>>>>>>>>> >>>>>>>>>>>>>> Best, >>>>>>>>>>>>>> Flavio >>>>>>>>>>>>>> On 21 Oct 2015 23:33, "Gyula Fóra" <gyf...@apache.org> >>>>>>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>> >>>>>>>> Thanks Max for the effort, this is going to be huge >>>>>>>>>>>>>>> >>>>>>>>>>>>>> :) >>> >>>> Unfortunately I have to say -1 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> FLINK-2888 and FLINK-2824 are blockers from my point >>>>>>>>>>>>>>> >>>>>>>>>>>>>> of >>> >>>> view. >>>>>>> >>>>>>>> Cheers, >>>>>>>>>>>>>>> Gyula >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Vasiliki Kalavri <vasilikikala...@gmail.com> ezt >>>>>>>>>>>>>>> >>>>>>>>>>>>>> írta >>> >>>> (időpont: >>>>>>> >>>>>>>> 2015. >>>>>>>>>>>> >>>>>>>>>>>>> okt. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> 21., Sze, 20:07): >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Awesome! Thanks Max :)) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I have a couple of questions: >>>>>>>>>>>>>>>> - what about the blocker issue (according to the >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> wiki) >>>> >>>>> FLINK-2747? >>>>>>>>>>> >>>>>>>>>>>> - weren't we going to get rid of staging >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> altogether? >>> >>>> Cheers, >>>>>>>>>>>>>>>> -V. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 21 October 2015 at 19:54, Stephan Ewen < >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> se...@apache.org >>>>>>> >>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Super, thanks Max! >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> We should also bump the master to the next >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> version >>> >>>> then, >>>>>>> to >>>>>>> >>>>>>>> separate >>>>>>>>>>>>> >>>>>>>>>>>>>> what >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> goes into release fixes and what goes into the >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> next >>> >>>> version... >>>>>>>>>> >>>>>>>>>>> Is that going to be 1.0-SNAPSHOT? ;-) That is a >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> separate >>>>>>> >>>>>>>> thread, >>>>>>>>>>> >>>>>>>>>>>> I >>>>>>>>>>>> >>>>>>>>>>>>> guess... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Wed, Oct 21, 2015 at 7:12 PM, Maximilian >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Michels < >>>> >>>>> m...@apache.org> >>>>>>>>>>>>> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Release candidates have to be tested >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> thoroughly. >>> >>>> Therefore, I >>>>>>>>>> >>>>>>>>>>> would >>>>>>>>>>>>> >>>>>>>>>>>>>> like >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> everybody to take a look at the release page in >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> the >>>> >>>>> wiki: >>>>>>> >>>>>> https://cwiki.apache.org/confluence/display/FLINK/0.10+Release >>> >>>> I've compiled the checks into a document. I >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> would >>> >>>> like >>>>>>> >>>>>>>> everyone >>>>>>>>>>> >>>>>>>>>>>> to >>>>>>>>>>>>> >>>>>>>>>>>>>> assign >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> one of the checks in the documents to test the >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> release >>>>>>> >>>>>>>> candidate: >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>> >>> https://docs.google.com/document/d/1TWCFj55xTyJjGYe8x9YEqmICgSvcexDPlbgP4CnLpLY/edit?usp=sharing >>> >>>> On Wed, Oct 21, 2015 at 7:10 PM, Maximilian >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Michels < >>>>> >>>>>> m...@apache.org> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Dear community, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The past months we have been working very >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> hard >>> >>>> to >>>> >>>>> push >>>>>>> >>>>>>>> towards >>>>>>>>>>>> >>>>>>>>>>>>> 0.10. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> would like to propose the first release >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> candidate. >>>>> >>>>>> =================================== >>>>>>>>>>>>>>>>>>> Please vote on releasing the following >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> candidate >>>> >>>>> as >>>>> >>>>>> Apache >>>>>>>>>> >>>>>>>>>>> Flink >>>>>>>>>>>>> >>>>>>>>>>>>>> version >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 0.10.0: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The commit to be voted on: >>>>>>>>>>>>>>>>>>> b697064b71b97e51703caae13660038949d41631 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Branch: >>>>>>>>>>>>>>>>>>> release-0.10.0-rc0 (see >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>> https://git1-us-west.apache.org/repos/asf/flink/?p=flink.git >>>>>>>>>>> >>>>>>>>>>>> ) >>>>>>>>>>>> >>>>>>>>>>>>> The release artifacts to be voted on can be >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> found >>>> >>>>> at: >>>>>>> >>>>>> http://people.apache.org/~mxm/flink-0.10.0-rc0/ >>>> >>>>> Release artifacts are signed with the key >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> with >>> >>>> fingerprint >>>>>>>>>> >>>>>>>>>>> C2909CBF: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The staging repository for this release can >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> be >>> >>>> found >>>>>>> at: >>>>>>> >>>>>> https://repository.apache.org/content/repositories/orgapacheflink-1047 >>>> >>>>> ------------------------------------------------------------- >>>>>>>>>>> >>>>>>>>>>>> Please vote on releasing this package as >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Apache >>> >>>> Flink >>>>>>> >>>>>>>> 0.10.0. >>>>>>>>>>> >>>>>>>>>>>> The vote is open for the next 72 hours and >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> passes >>>> >>>>> if >>>>>>> a >>>>>>> >>>>>>>> majority >>>>>>>>>>>> >>>>>>>>>>>>> of >>>>>>>>>>>>>> >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> least three +1 PMC votes are cast. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The vote ends on Monday October 26, 2015. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> [ ] +1 Release this package as Apache Flink >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 0.10.0 >>>>> >>>>>> [ ] -1 Do not release this package because >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ... >>> >>>> =================================== >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>> >
