Re: [NOTICE] Release guide updated for updating japicmp configuration

2020-06-02 Thread Till Rohrmann
Thanks for activating the checks and providing the tooling Chesnay. Cheers, Till On Fri, May 29, 2020 at 6:51 PM Yu Li wrote: > Thanks Chesnay for the efforts! > > Best Regards, > Yu > > > On Fri, 29 May 2020 at 18:03, Piotr Nowojski wrote: > > > Thanks Chesney for adding those scripts and

Re: [DISCUSS] Stability guarantees for @PublicEvolving classes

2020-06-02 Thread Till Rohrmann
said, how and when will a @PublicEvolving > >> become @Public, and I'm not sure if we can have a technical solution to > >> keep such a rule. (In my opinion, check such things -- change > >> @PublicEvolving to @Public -- manually may not so easy) > >> > >

Re: [VOTE] Release flink-shaded 11.0, release candidate #1

2020-05-28 Thread Till Rohrmann
+1 (binding) - verified checksums and signature - mvn clean verify passes on source release - verified licenses - checked pom.xml changes Cheers, Till On Thu, May 28, 2020 at 1:05 PM Congxian Qiu wrote: > +1 (non-binding) > > checked > - mvn clean verify, ok > - gpg & sha512, ok > - all pom

Re: [DISCUSS] FLINK-17989 - java.lang.NoClassDefFoundError org.apache.flink.fs.azure.common.hadoop.HadoopRecoverableWriter

2020-05-28 Thread Till Rohrmann
l attempt to connect with Klou today to collaborate to see what the > level of effort is to add this support. > > Thanks. > > > > On Thu, May 28, 2020 at 11:54 AM Till Rohrmann > wrote: > >> Hi Israel, >> >> thanks for reaching out to the Flink community

Re: [DISCUSS] FLINK-17989 - java.lang.NoClassDefFoundError org.apache.flink.fs.azure.common.hadoop.HadoopRecoverableWriter

2020-05-28 Thread Till Rohrmann
Hi Israel, thanks for reaching out to the Flink community. As Guowei said, the StreamingFileSink can currently only recover from faults if it writes to HDFS or S3. Other file systems are currently not supported if you need fault tolerance. Maybe Klou can tell you more about the background and

[jira] [Created] (FLINK-18012) Deactivate slot timeout if TaskSlotTable.tryMarkSlotActive is called

2020-05-28 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-18012: - Summary: Deactivate slot timeout if TaskSlotTable.tryMarkSlotActive is called Key: FLINK-18012 URL: https://issues.apache.org/jira/browse/FLINK-18012 Project

[jira] [Created] (FLINK-17978) Test Hadoop dependency change

2020-05-27 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17978: - Summary: Test Hadoop dependency change Key: FLINK-17978 URL: https://issues.apache.org/jira/browse/FLINK-17978 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-17977) Check log sanity

2020-05-27 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17977: - Summary: Check log sanity Key: FLINK-17977 URL: https://issues.apache.org/jira/browse/FLINK-17977 Project: Flink Issue Type: Task Components

[jira] [Created] (FLINK-17976) Test native K8s integration

2020-05-27 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17976: - Summary: Test native K8s integration Key: FLINK-17976 URL: https://issues.apache.org/jira/browse/FLINK-17976 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-17975) Test ZooKeeper 3.5 support

2020-05-27 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17975: - Summary: Test ZooKeeper 3.5 support Key: FLINK-17975 URL: https://issues.apache.org/jira/browse/FLINK-17975 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-17974) Test new Flink Docker image

2020-05-27 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17974: - Summary: Test new Flink Docker image Key: FLINK-17974 URL: https://issues.apache.org/jira/browse/FLINK-17974 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-17973) Test memory configuration of Flink cluster

2020-05-27 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17973: - Summary: Test memory configuration of Flink cluster Key: FLINK-17973 URL: https://issues.apache.org/jira/browse/FLINK-17973 Project: Flink Issue Type

[jira] [Created] (FLINK-17970) Increase default value of IO pool executor to 4 * #cores

2020-05-27 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17970: - Summary: Increase default value of IO pool executor to 4 * #cores Key: FLINK-17970 URL: https://issues.apache.org/jira/browse/FLINK-17970 Project: Flink

Re: [DISCUSS] Semantics of our JIRA fields

2020-05-26 Thread Till Rohrmann
>>>>>> > > >>>>>> > > >>>>>> Flavio Pompermaier 于2020年5月24日周日 下午1:23写道: > > >>>>>> > > >>>>>>> In my experience it's quite complicated for a normal reporter to > > &

[jira] [Created] (FLINK-17947) Retry REST requests if RpcEndpoint died before responding to request

2020-05-26 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17947: - Summary: Retry REST requests if RpcEndpoint died before responding to request Key: FLINK-17947 URL: https://issues.apache.org/jira/browse/FLINK-17947 Project

[jira] [Created] (FLINK-17938) Cannot run mvn clean verify flink-yarn-tests

2020-05-26 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17938: - Summary: Cannot run mvn clean verify flink-yarn-tests Key: FLINK-17938 URL: https://issues.apache.org/jira/browse/FLINK-17938 Project: Flink Issue Type

Re: [DISCUSS] Release flink-shaded 11.0 (and include it in 1.11.0)

2020-05-25 Thread Till Rohrmann
+1 for the new flink-shaded release. Cheers, Till On Mon, May 25, 2020 at 9:06 AM Chesnay Schepler wrote: > Hello, > > I would like to do another flink-shaded release for 1.11.0, to include a > zookeeper 3.4 security fix and resolve a shading issue when working with > Gradle. > > >

Re: Please update the Requirements of PyPI

2020-05-25 Thread Till Rohrmann
Thanks for the pointer Ray. I've pulled in Jincheng who works on Flink's Python integration. Cheers, Till On Mon, May 25, 2020 at 8:41 AM Ray Chen wrote: > Hi PyFlink Developers, > > > I got some errors from the pyarrow depended on apache-flink v1.10.1. > > It seems updating pyarrow version

Re: Shall we avoid binding "unsafe ports" during random port selection?

2020-05-25 Thread Till Rohrmann
Hi Weike, would it be good enough if the user did not include unsafe ranges when specifying `rest.bind-port`? My concern with excluding unsafe ports is that it adds some invisible magic which can be hard to understand for the user. I think over the past couple of years it has proven that auto

Re: [DISCUSS] Semantics of our JIRA fields

2020-05-22 Thread Till Rohrmann
Thanks for drafting this proposal Robert. +1 for the proposal. Cheers, Till On Fri, May 22, 2020 at 5:39 PM Leonard Xu wrote: > Thanks bringing up this topic @Robert, +1 to the proposal. > > It clarifies the JIRA fields well and should be a rule to follow. > > Best, > Leonard Xu > > 在

Re: Issues using StreamingFileSink (Cannot locate configuration: tried hadoop-metrics2-s3a-file-system.properties, hadoop-metrics2.properties)

2020-05-22 Thread Till Rohrmann
Hi, for the first message I think this might be related to BEANUTILS-477 [1] and should not be a problem. The other messages look to me like the metric system initialization of the Hadoop components. I don't think that there is something wrong with them. [1]

[RESULT] [VOTE] Guarantee that @PublicEvolving classes are API and binary compatible across bug fix releases (x.y.u and x.y.v)

2020-05-20 Thread Till Rohrmann
-binding) Congxian Qiu (non-binding) Dian Fu (non-binding) Jeff Zhang (non-binding) Yu Li (non-binding) Piotr Nowojski (non-binding) Benchao Li (non-binding) Yun Tang (non-binding) Zhu Zhu (non-binding) Leonard Xu (non-binding) Xintong Song (non-binding) Tzu-Li Tai (binding) Till Rohrmann (binding

[jira] [Created] (FLINK-17844) Activate japicmp-maven-plugin checks for @PublicEvolving between bug fix releases (x.y.u -> x.y.v)

2020-05-20 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17844: - Summary: Activate japicmp-maven-plugin checks for @PublicEvolving between bug fix releases (x.y.u -> x.y.v) Key: FLINK-17844 URL: https://issues.apache.org/jira/browse/FL

Re: [VOTE] Guarantee that @PublicEvolving classes are API and binary compatible across bug fix releases (x.y.u and x.y.v)

2020-05-20 Thread Till Rohrmann
Thanks everyone for voting. I will post the result of the vote in a separate email. Cheers, Till On Mon, May 18, 2020 at 1:03 PM Piotr Nowojski wrote: > +1 > > > On 18 May 2020, at 10:43, Canbin Zheng wrote: > > > > +1 (non-binding) > > > > Regards, >

Re: [VOTE] Guarantee that @PublicEvolving classes are API and binary compatible across bug fix releases (x.y.u and x.y.v)

2020-05-18 Thread Till Rohrmann
t;>>> Best, > > > >>>>>>> Congxian > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> Yangze Guo 于2020年5月16日周六 上午12:51写道: > > > >>>>>>> > > > >>&

[jira] [Created] (FLINK-17765) Verbose client error messages

2020-05-16 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17765: - Summary: Verbose client error messages Key: FLINK-17765 URL: https://issues.apache.org/jira/browse/FLINK-17765 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-17738) Remove legacy scheduler option

2020-05-15 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17738: - Summary: Remove legacy scheduler option Key: FLINK-17738 URL: https://issues.apache.org/jira/browse/FLINK-17738 Project: Flink Issue Type: Task

Re: [DISCUSS] Stability guarantees for @PublicEvolving classes

2020-05-15 Thread Till Rohrmann
The vote thread can be found here https://lists.apache.org/thread.html/rc58099fb0e31d0eac951a7bbf7f8bda8b7b65c9ed0c04622f5333745%40%3Cdev.flink.apache.org%3E . Cheers, Till On Fri, May 15, 2020 at 3:03 PM Till Rohrmann wrote: > I completely agree that there are many other aspect of

[VOTE] Guarantee that @PublicEvolving classes are API and binary compatible across bug fix releases (x.y.u and x.y.v)

2020-05-15 Thread Till Rohrmann
Dear community, with reference to the dev ML thread about guaranteeing API and binary compatibility for @PublicEvolving classes across bug fix releases [1] I would like to start a vote about it. The proposal is that the Flink community starts to guarantee that @PublicEvolving classes will be API

Re: [DISCUSS] Stability guarantees for @PublicEvolving classes

2020-05-15 Thread Till Rohrmann
> > On Thu, May 14, 2020 at 8:22 PM Stephan Ewen wrote: > > > > > I just want to throw in that we also need to rethink our policy towards > > > using @PublicEvolving. > > > > > > We often introduce this easily (for every new feature) an

Re: [DISCUSS] Exact feature freeze date

2020-05-15 Thread Till Rohrmann
+1 from my side extend the feature freeze until Monday morning. Cheers, Till On Fri, May 15, 2020 at 2:04 PM Robert Metzger wrote: > I'm okay, but I would suggest to agree on a time of day. What about Monday > morning in Europe? > > On Fri, May 15, 2020 at 1:43 PM Piotr Nowojski > wrote: > >

Re: [DISCUSS] Rework History Server into Global Dashboard

2020-05-15 Thread Till Rohrmann
release anyways. We will test run it > there and gather production feedback for the Flink community and can make a > better decision afterwards when we see the real value. > > Cheers, > Gyula > > > > On Thu, May 14, 2020 at 3:36 PM Till Rohrmann > wrote: > &g

Re: [ANNOUNCE] Apache Flink 1.10.1 released

2020-05-15 Thread Till Rohrmann
Thanks Yu for being our release manager and everyone else who made the release possible! Cheers, Till On Fri, May 15, 2020 at 9:15 AM Congxian Qiu wrote: > Thanks a lot for the release and your great job, Yu! > Also thanks to everyone who made this release possible! > > Best, > Congxian > > >

Re: [DISCUSS] increase "state.backend.fs.memory-threshold" from 1K to 100K

2020-05-14 Thread Till Rohrmann
I cannot say much about the concrete value but if our users have problems with the existing default values, then it makes sense to me to change it. One thing to check could be whether it is possible to provide a meaningful exception in case that the state size exceeds the frame size. At the

Re: [DISCUSS] Rework History Server into Global Dashboard

2020-05-14 Thread Till Rohrmann
Hi Gyula, thanks for proposing this extension. I can see that such a feature could be helpful. However, I wouldn't consider the management of multiple clusters core to Flink. Managing a single cluster is already complex enough and given the available community capacity I would rather concentrate

Re: [DISCUSS] Stability guarantees for @PublicEvolving classes

2020-05-14 Thread Till Rohrmann
release, we should expose it > > in > > > dev mail list for discussing or a FLIP? > > > > > > BTW, public api can be changed by major releases? In annotation > comments: > > > "Only major releases (1.0, 2.0, 3.0) can break interfaces with this &

Re: [NOTICE] Azure Pipelines Status

2020-05-14 Thread Till Rohrmann
Thanks for the update Robert. One idea to make the e2e also run on the Alibaba infrastructure would be to ensure that e2e tests clean up after they have run. Do we know which e2e tests don't do this properly? Cheers, Till On Thu, May 14, 2020 at 8:38 AM Robert Metzger wrote: > Hi all, > >

Re: [NOTICE] Deletion of older releases from dist.apache.org

2020-05-14 Thread Till Rohrmann
Ack'ed Thanks for updating the release guide Chesnay. Cheers, Till On Wed, May 13, 2020 at 3:42 PM Chesnay Schepler wrote: > Hello, > > I have just amended the release guide > < > https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release> > > to specify that older releases

Re: [Discuss] Breaking API Change in 1.10.1

2020-05-14 Thread Till Rohrmann
gt; >> > >>>>> IIRC @PublicEvolving had no /real/ guarantees, from the 1.0.0 >> > >>>> announcement: >> > >>>>> /"Flink 1.0.0 introduces interface stability annotations for API >> > >> classes >> >

[DISCUSS] Stability guarantees for @PublicEvolving classes

2020-05-14 Thread Till Rohrmann
Dear community, in the latest 1.10.1 bug fix release I introduced a binary incompatible change to a class which is annotated with @PublicEvolving [1]. While this change is technically ok since we only provide API and binary compatibility for @Public classes across releases, it raised the question

Re: [Discuss] Breaking API Change in 1.10.1

2020-05-14 Thread Till Rohrmann
ain > > >>>>> stable across all releases of the 1.x series. The > > >>>>> //|@PublicEvolving|//annotation marks API features that may be > > subject > > >>>>> to change in future versions."/ > > >>>>> >

Re: [Discuss] Breaking API Change in 1.10.1

2020-05-13 Thread Till Rohrmann
would like users to be > able to upgrade to the latest patch release without hesitation. > > On Wed, May 13, 2020 at 11:45 AM Till Rohrmann > wrote: > > > Thanks for reporting this issue Seth. This is indeed a big problem. > > > > I looked into the problem and it seem

Re: [Discuss] Breaking API Change in 1.10.1

2020-05-13 Thread Till Rohrmann
Thanks for reporting this issue Seth. This is indeed a big problem. I looked into the problem and it seems we have the following situation: # 1.9.x --> 1.10.0 There is an API breaking change between 1.9.x and 1.10 due FLINK-13864 because it introduced another generic parameter. I expected only

Re: What's the best practice to determine whether a job has finished or not?

2020-05-12 Thread Till Rohrmann
For that attached YARN per-job mode, the cluster should wait until the result of the job has been fetched. Can't this be used to report the final status of the job? Cheers, Till On Tue, May 12, 2020 at 9:52 AM Aljoscha Krettek wrote: > Hi, > > The problem is that the JobClient is talking to

Re: [DISCUSS] Add a material web page under "https://flink.apache.org/"

2020-05-12 Thread Till Rohrmann
I think this is a very good idea. With a team committed to keep the site up to date the main concern should also be resolved. Cheers, Till On Mon, May 11, 2020 at 5:25 PM Yuan Mei wrote: > Yep, Marta's concern is definitely a great point in terms of maintenance > costs. > > What I would like

Re: [PROPOSAL] Google Season of Docs 2020.

2020-05-12 Thread Till Rohrmann
t;1. Improving the Table API/SQL documentation. > > > > > >> > 2. Improving the documentation about Deployments. > > > > > >> >3. Restructuring and standardizing the documentation about > > > > > >> Connectors. > > > > > >>

Re: [DISCUSS] Align the behavior of internal return result of MapState#entries, keys, values and iterator.

2020-05-12 Thread Till Rohrmann
A bit late but also +1 for the proposal to return an empty iterator/collection. Cheers, Till On Mon, May 11, 2020 at 11:17 AM SteNicholas wrote: > Hi Tang Yun, > I have already created new issue for FLINK-17610 to align the behavior > of result of internal map state to return empty

[jira] [Created] (FLINK-17554) Add release hooks for user code class loader

2020-05-07 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17554: - Summary: Add release hooks for user code class loader Key: FLINK-17554 URL: https://issues.apache.org/jira/browse/FLINK-17554 Project: Flink Issue Type

Re: [DISCUSS] Send issue and pull request notifications for flink-web and flink-shaded to iss...@flink.apache.org

2020-05-07 Thread Till Rohrmann
> Zhu Zhu > > Congxian Qiu 于2020年5月6日周三 下午12:02写道: > > > +1 for this > > > > Best, > > Congxian > > > > > > Benchao Li 于2020年5月5日周二 下午5:22写道: > > > > > belated +1 > > > > > > Till Rohrmann 于2020年

Re: [VOTE] Release 1.10.1, release candidate #2

2020-05-07 Thread Till Rohrmann
s under 1.10.2, so will start to prepare RC3. > > Best Regards, > Yu > > > On Wed, 6 May 2020 at 17:36, Till Rohrmann wrote: > > > I've merged the fix for FLINK-17514. > > > > Cheers, > > Till > > > > On Wed, May 6, 2020 at 10:53 AM Dawid Wysak

Re: [VOTE] Release 1.10.1, release candidate #2

2020-05-06 Thread Till Rohrmann
the Kinesis fix - it would be nice to include > if > >> there is another RC: > >> > >> https://github.com/apache/flink/pull/11998 > >> > >> > >> On Tue, May 5, 2020 at 4:50 AM Till Rohrmann > wrote: > >> > >>> I've op

Re: What is the RocksDB local directory in flink checkpointing?

2020-05-06 Thread Till Rohrmann
Hi LakeShen, `state.backend.rocksdb.localdir` defines the directory in which RocksDB will store its local files. Local files are RocksDB's SST and metadata files for example. This directory does not need to be persisted. If the config option is not configured, then it will use the nodes temporary

Re: [DISCUSS] Update our Roadmap

2020-05-06 Thread Till Rohrmann
; I agree with Till — this would make more sense somewhere during the > kick-off of the release cycle. > > > On Tue, May 5, 2020 at 4:44 PM Till Rohrmann wrote: > > > I don't think that a post-release step makes sense since the release > > manager of release X won't neces

Re: [DISCUSS] Have foreground processes also create log files

2020-05-06 Thread Till Rohrmann
nce with Flink. > > > > > > > > > > On Tue, May 5, 2020 at 10:49 AM David Anderson < > > da...@alpinegizmo.com> > > > > > wrote: > > > > > > > > > >> I like this idea because it should improve

Re: [DISCUSS] Update our Roadmap

2020-05-05 Thread Till Rohrmann
;> > > > >> On Sun, Aug 18, 2019 at 4:46 PM Stephan Ewen > > wrote: > > > >> > > > >> > I could help with that. > > > >> > > > > >> > On Fri, Aug 16, 2019 at 2:36 PM Robert Metzger < > rmetz...@apac

[jira] [Created] (FLINK-17527) kubernetes-session.sh uses log4j-console.properties

2020-05-05 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17527: - Summary: kubernetes-session.sh uses log4j-console.properties Key: FLINK-17527 URL: https://issues.apache.org/jira/browse/FLINK-17527 Project: Flink Issue

Re: [VOTE] Release 1.10.1, release candidate #2

2020-05-05 Thread Till Rohrmann
; > > On Tue, 5 May 2020 at 17:28, Till Rohrmann wrote: > > > I agree with Aljoscha. This is something we should fix since this is very > > important for Flink's stability. I will prepare a fix for the problem. > > > > Cheers, > > Till > > > > On Tue

Re: [VOTE] Release 1.10.1, release candidate #2

2020-05-05 Thread Till Rohrmann
I agree with Aljoscha. This is something we should fix since this is very important for Flink's stability. I will prepare a fix for the problem. Cheers, Till On Tue, May 5, 2020 at 10:30 AM Congxian Qiu wrote: > +1 (no-binding) > - sha and gpg, ok > - all pom files point to same version, ok >

Re: [DISCUSS] Releasing "fat" and "slim" Flink distributions

2020-05-05 Thread Till Rohrmann
> connectors/formats, which will keep the size of the distribution the > same, or a bit smaller. > > Best, > Aljoscha > > On 04.05.20 18:59, Till Rohrmann wrote: > > Thanks everyone for this lively discussion and all your thoughts. > > > > Let me t

[jira] [Created] (FLINK-17513) Send flink-shaded issues and pull request notifications to iss...@flink.apache.org

2020-05-05 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17513: - Summary: Send flink-shaded issues and pull request notifications to iss...@flink.apache.org Key: FLINK-17513 URL: https://issues.apache.org/jira/browse/FLINK-17513

[jira] [Created] (FLINK-17512) Send flink-web issues and pull request notifications to iss...@flink.apache.org

2020-05-05 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17512: - Summary: Send flink-web issues and pull request notifications to iss...@flink.apache.org Key: FLINK-17512 URL: https://issues.apache.org/jira/browse/FLINK-17512

Re: The Questions about Submitting PR

2020-05-05 Thread Till Rohrmann
Hi Roc, afaik it can take a bit of time until the status of the PR is updated in accordance with the AZP results. As of a couple of days ago, we no longer use Travis for running CI. That's why you don't see it anymore. Cheers, Till On Tue, May 5, 2020 at 1:59 AM Roc Marshal wrote: > Hi,all.

Re: [DISCUSS] Send issue and pull request notifications for flink-web and flink-shaded to iss...@flink.apache.org

2020-05-05 Thread Till Rohrmann
;> On Mon, May 4, 2020 at 5:50 PM Dawid Wysakowicz < > > >>> dwysakow...@apache.org> > > >>>>> wrote: > > >>>>> > > >>>>>> +1 > > >>>>>> > > >>>>>> Yes, please. I

Re: [DISCUSS] Releasing "fat" and "slim" Flink distributions

2020-05-04 Thread Till Rohrmann
t;>>>>> opt > >>>>>>>> to some directory (lib/plugins) is less error-prone than just > >>>> selecting > >>>>>>> the > >>>>>>>> feature and having the tool handle the rest. > >>>>>>>

Re: [DISCUSS] Have foreground processes also create log files

2020-05-04 Thread Till Rohrmann
Hi everyone, thanks for starting this discussion Chesnay. I think it would be nice if we also displayed the logs when starting the process in the foreground. The repercussions could be mitigated if the default logger configurations would contain file rolling with a max log file size. @Yang I

Re: [DISCUSS] Send issue and pull request notifications for flink-web and flink-shaded to iss...@flink.apache.org

2020-05-04 Thread Till Rohrmann
For future reference here is the link describing how to configure the notifications: https://cwiki.apache.org/confluence/display/INFRA/.asf.yaml+features+for+git+repositories#id-.asf.yamlfeaturesforgitrepositories-Notificationsettingsforrepositories On Mon, May 4, 2020 at 5:43 PM Till Rohrmann

[DISCUSS] Send issue and pull request notifications for flink-web and flink-shaded to iss...@flink.apache.org

2020-05-04 Thread Till Rohrmann
Hi everyone, due to some changes on the ASF side, we are now seeing issue and pull request notifications for the flink-web [1] and flink-shaded [2] repo on dev@flink.apache.org. I think this is not ideal since the dev ML is much more noisy now. I would propose to send these notifications to

Re: [DISCUSS] Hierarchies in ConfigOption

2020-05-04 Thread Till Rohrmann
Hi everyone, I like Timo's proposal to organize our configuration more hierarchical since this is what the coding guide specifies. The benefit I see is that config options belonging to the same concept will be found in the same nested object. Moreover, it will be possible to split the

Re: Presentation - Real World Architectural Patterns with Apache Pulsar and Flink

2020-05-04 Thread Till Rohrmann
Thanks for sharing your talk with the Flink community. Cheers, Till On Tue, Apr 28, 2020 at 10:00 PM Devin Bost wrote: > You can also access the slide deck here: > https://www.slideshare.net/DevinBost/realworld-pulsar-architectural-patterns > [image: image.png] > Devin G. Bost > > > On Tue,

[jira] [Created] (FLINK-17503) Make memory configuration logging more user-friendly

2020-05-04 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17503: - Summary: Make memory configuration logging more user-friendly Key: FLINK-17503 URL: https://issues.apache.org/jira/browse/FLINK-17503 Project: Flink Issue

Re: "[VOTE] FLIP-108: edit the Public API"

2020-05-01 Thread Till Rohrmann
Thanks for updating the FLIP Yangze. +1 (binding) for the update. Cheers, Till On Thu, Apr 30, 2020 at 4:34 PM Yangze Guo wrote: > Hi, there. > > The "FLIP-108: Add GPU support in Flink"[1] is now working in > progress. However, we met problems regarding class loader and > dependency. For

Re: [DISCUSS][FLIP-108] Problems regarding the class loader and dependency

2020-04-30 Thread Till Rohrmann
rom? > > > > > > The component could be either the UDF/operator itself, or some AI > > libraries > > > used by the operator. For 1.11, we do not have plan for introducing new > > GPU > > > aware operators in Flink. So all the usages of the GPU resource

Re: [DISCUSS][FLIP-108] Problems regarding the class loader and dependency

2020-04-29 Thread Till Rohrmann
Thanks for bringing this up Yangze and Xintong. I see the problem. Help me to understand how the ExternalResourceInfo is intended to be used by the user. Will she ask for some properties and then pass them to another component? Where does this component come from? Cheers, Till On Wed, Apr 29,

Re: Issue

2020-04-28 Thread Till Rohrmann
Hi Pavan, please post these kind of questions to the user ML. I've cross linked it now. Image attachments will be filtered out. Consequently, we cannot see what you have posted. Moreover, it would be good if you could provide the community with a bit more details what the custom way is and what

Re: [VOTE] FLIP-103: Better TM/JM Log Display

2020-04-28 Thread Till Rohrmann
g, a temporary workaround could > be: > > show the main log and stdout file by default and make the log list a > > separate page. > > > > This workaround would allow us to roll out the feature while not changing > > existing behavior. > > > > What

Re: [DISCUSS] Move docker development into versioned branches

2020-04-27 Thread Till Rohrmann
Thanks for starting this discussion Chesnay. Your proposal sounds good to me. I can see how the current setup makes the development of version specific features impractical. Hence, +1 for the proposed changes. Cheers, Till On Mon, Apr 27, 2020 at 12:19 PM David Anderson wrote: > Makes sense to

Re: [ANNOUNCE] Apache Flink 1.9.3 released

2020-04-27 Thread Till Rohrmann
Thanks Dian for being our release manager and thanks to everyone who helped making this release possible. Cheers, Till On Mon, Apr 27, 2020 at 3:26 AM Jingsong Li wrote: > Thanks Dian for managing this release! > > Best, > Jingsong Lee > > On Sun, Apr 26, 2020 at 7:17 PM Jark Wu wrote: > >>

Re: [ANNOUNCE] Development progress of Apache Flink 1.11

2020-04-24 Thread Till Rohrmann
Thanks for the update Piotr. Cheers, Till On Fri, Apr 24, 2020 at 4:42 PM Piotr Nowojski wrote: > Hi community, > > It has been more than 6 weeks since the previous announcement and as we are > approaching the expected feature freeze we would like to share the Flink > 1.11 status update with

Re: Tags for beginners

2020-04-24 Thread Till Rohrmann
Hi Manish, the label the Flink community uses is "starter". Be aware that it has not been used very consistently in the past, though. Cheers, Till On Fri, Apr 24, 2020 at 4:12 PM Manish G wrote: > Are there tags for jira issues which can help in identifying issues > which beginners can start

Re: New contributor

2020-04-24 Thread Till Rohrmann
Hi Etienne, welcome to the Flink community. Looking forward to working with you on Flink :-) Cheers, Till On Fri, Apr 24, 2020 at 11:20 AM Etienne Chauchot wrote: > Hi everyone, > > Let me introduce myself, I'm Etienne Chauchot, I'm an Apache Beam > committer and a PMC member and I would like

[jira] [Created] (FLINK-17372) SlotManager should expose total required resources

2020-04-24 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17372: - Summary: SlotManager should expose total required resources Key: FLINK-17372 URL: https://issues.apache.org/jira/browse/FLINK-17372 Project: Flink Issue

Re: [DISCUSS] Exact feature freeze date

2020-04-23 Thread Till Rohrmann
+1 for extending the feature freeze until May 15th. Cheers, Till On Thu, Apr 23, 2020 at 1:00 PM Piotr Nowojski wrote: > Hi Stephan, > > As release manager I’ve seen that quite a bit of features could use of the > extra couple of weeks. This also includes some features that I’m involved >

Re: [DISCUSS] Adding support for Hadoop 3 and removing flink-shaded-hadoop

2020-04-22 Thread Till Rohrmann
Hi Robert, I think it would be a helpful simplification of Flink's build setup if we can get rid of flink-shaded-hadoop. Moreover relying only on the vanilla Hadoop dependencies for the modules which interact with Hadoop/Yarn sounds like a good idea to me. Adding support for Hadoop 3 would also

Re: Apply to contribute to Flink

2020-04-21 Thread Till Rohrmann
Here is some information about how to contribute to Flink [1]. [1] https://flink.apache.org/contributing/contribute-code.html Cheers, Till On Tue, Apr 21, 2020 at 10:16 AM Till Rohrmann wrote: > Hi Lee, > > there are no special permissions required to contribute to Flink. The way &

Re: Apply to contribute to Flink

2020-04-21 Thread Till Rohrmann
Hi Lee, there are no special permissions required to contribute to Flink. The way it works is that you write on a JIRA issue for which you would like to contribute. Once a committer sees it you can discuss with him your solution approach. Once she signs it off she will assign you to the ticket

[jira] [Created] (FLINK-17268) SourceReaderTestBase.testAddSplitToExistingFetcher deadlocks on Travis

2020-04-20 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17268: - Summary: SourceReaderTestBase.testAddSplitToExistingFetcher deadlocks on Travis Key: FLINK-17268 URL: https://issues.apache.org/jira/browse/FLINK-17268 Project

Re: [ANNOUNCE] New Apache Flink PMC Member - Hequn Chen

2020-04-17 Thread Till Rohrmann
Congratulations Hequn! Cheers, Till On Fri, Apr 17, 2020 at 2:49 PM Shuo Cheng wrote: > Congratulations, Hequn > > Best, > Shuo > > On 4/17/20, hufeih...@mails.ucas.ac.cn wrote: > > Congratulations , Hequn > > > > Best wish > > > > > > hufeih...@mails.ucas.ac.cn > > Congratulations, Hequn! >

Re: [PROPOSAL] Google Season of Docs 2020.

2020-04-17 Thread Till Rohrmann
Thanks for driving this effort Marta. I'd be up for mentoring improvements for the deployment section as described in FLIP-42. Cheers, Till On Fri, Apr 17, 2020 at 11:20 AM Aljoscha Krettek wrote: > Hi, > > first, excellent that you're driving this, Marta! > > By now I have made quite some

Re: [VOTE] FLIP-118: Improve Flink’s ID system

2020-04-16 Thread Till Rohrmann
Thanks for creating this FLIP, Yangze. +1 (binding). Cheers, Till On Thu, Apr 16, 2020 at 10:51 AM Yangze Guo wrote: > Hi everyone, > > I'd like to start the vote of FLIP-118 [1], which improves the log > readability by adding more information to Flink’s IDs. This FLIP is > discussed in the

Re: [DISCUSS] Releasing "fat" and "slim" Flink distributions

2020-04-16 Thread Till Rohrmann
I think what Chesnay and Dawid proposed would be the ideal solution. Ideally, we would also have a nice web tool for the website which generates the corresponding distribution for download. To get things started we could start with only supporting to download/creating the "fat" version with the

Re: [DISCUSS] Generating java code?

2020-04-15 Thread Till Rohrmann
rkill to me. It also introduces some inconsistency in > what one needs to do before working on stuff which I'm apprehensive about. > > On 15/04/2020 16:31, Till Rohrmann wrote: > > Hi everyone, > > > > thanks for starting this discussion Niels. > > > > I like

Re: [DISCUSS] Generating java code?

2020-04-15 Thread Till Rohrmann
Hi everyone, thanks for starting this discussion Niels. I like the idea of getting rid of parsing a Properties instance. On the other hand, I also understand that people are concerned about disrupting the workflows of our devs. Maybe we can find a compromise between both approaches. For

Re: [DISCUSS] FLIP-118: Improve Flink’s ID system

2020-04-15 Thread Till Rohrmann
> > the "PartitionRequest" and "TaskEventRequest" by 8 bytes(subtaskIndex > > > > and attemptNumber). But it may not infect the TDD as much as > > > > IntermediateResultPartitionID, since there is only one > > > > ExecutionAttemptID per TDD. > &

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-15 Thread Till Rohrmann
ceType" param, what if there are > > multiple subtypes in the ExternalResourceInfos set of one external > > resource? It seems user has to set the T to ExternalResourceInfo and > > the mechanism is useless at this case. > > > > Best, > > Yangze Guo >

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-15 Thread Till Rohrmann
external > resource? It seems user has to set the T to ExternalResourceInfo and > the mechanism is useless at this case. > > Best, > Yangze Guo > > On Wed, Apr 15, 2020 at 3:57 PM Till Rohrmann > wrote: > > > > Ok, if there can be multiple resources

Re: [DISCUSS] Releasing Flink 1.10.1

2020-04-15 Thread Till Rohrmann
roblem with > >> >> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory > >> >> > >> >> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to > >> >> minimise its OutOfMemoryError > >> >> > >> >>

Re: [PROPOSAL] Contribute training materials to Apache Flink

2020-04-15 Thread Till Rohrmann
Hi David, making the training materials available on flink.apache.org would increase the reach and improve its visibility. Since this is very helpful material for our users +1 for contributing the training material. If we decide not maintain different versions, then we might be able to highlight

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-15 Thread Till Rohrmann
traction is done by Operator or Flink core. > This interface would not gain benefits for type safety. > > Best, > Yangze Guo > > On Wed, Apr 15, 2020 at 1:38 AM Till Rohrmann > wrote: > > > > Thanks for updating the FLIP, Yangze. > > > > If ExternalResou

Re: [DISCUSS] Releasing Flink 1.9.3

2020-04-15 Thread Till Rohrmann
Hi Dian, creating a new 1.9 bug fix release is a very good idea. +1 for creating it soon. Also thanks for volunteering as our release manager. Cheers, Till On Fri, Apr 10, 2020 at 7:27 AM Dian Fu wrote: > Hi Jincheng, > > Thanks a lot for offering help. It would be very helpful. Thanks again!

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-14 Thread Till Rohrmann
> > > > > > getExternalResourceInfo(ResourceSpec resourceSpec); > > > > > > > To: Map> > > > getExternalResourceInfo(); > > > > > > > > > > > > > > Best, > > > > > > > Yangze Guo > > &g

<    4   5   6   7   8   9   10   11   12   13   >