回复：[DISCUSS] Improve broadcast serialization

Hi Piotr 1. I agree with we should discuss higher level first and focus on implementation on jira/pr. As long as RecordSerializer does not maintain the BufferBuilder, it can become stateless, then it can get BufferBuilderfrom the RecordWriter at any time. And I think it is the precondition to

[jira] [Created] (FLINK-9891) Flink cluster is not shutdown in YARN mode when Flink client is stopped

Sergey Krasovskiy created FLINK-9891: Summary: Flink cluster is not shutdown in YARN mode when Flink client is stopped Key: FLINK-9891 URL: https://issues.apache.org/jira/browse/FLINK-9891

[jira] [Created] (FLINK-9890) Remove obsolete Class ResourceManagerConfiguration

Gary Yao created FLINK-9890: --- Summary: Remove obsolete Class ResourceManagerConfiguration Key: FLINK-9890 URL: https://issues.apache.org/jira/browse/FLINK-9890 Project: Flink Issue Type:

[jira] [Created] (FLINK-9889) .bat script to start Flink task manager

Pavel Shvetsov created FLINK-9889: - Summary: .bat script to start Flink task manager Key: FLINK-9889 URL: https://issues.apache.org/jira/browse/FLINK-9889 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] Improve broadcast serialization

Hi > 1. I want to define a new AbstractRecordWriter as base class which defines > some abstract methods and utility codes. The current RecordWriter used for > other partitioner and new BroadcastRecordWriter used only for > BroadcastPartitioner will both extend AbstractRecordWriter. The

[jira] [Created] (FLINK-9888) Remove unsafe defaults from release scripts

Chesnay Schepler created FLINK-9888: --- Summary: Remove unsafe defaults from release scripts Key: FLINK-9888 URL: https://issues.apache.org/jira/browse/FLINK-9888 Project: Flink Issue Type:

[jira] [Created] (FLINK-9887) Ensure serializer upgrades work with timer service remake

Stefan Richter created FLINK-9887: - Summary: Ensure serializer upgrades work with timer service remake Key: FLINK-9887 URL: https://issues.apache.org/jira/browse/FLINK-9887 Project: Flink

回复：[DISCUSS] Improve broadcast serialization

Hi Piotr, Your thoughts bring me more inspirations and possibilities. 1. I want to define a new AbstractRecordWriter as base class which defines some abstract methods and utility codes. The current RecordWriter used for other partitioner and new BroadcastRecordWriter used only for

[jira] [Created] (FLINK-9886) Build SQL jars with every build

Timo Walther created FLINK-9886: --- Summary: Build SQL jars with every build Key: FLINK-9886 URL: https://issues.apache.org/jira/browse/FLINK-9886 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-9885) End-to-end test: Elasticsearch 6.x connector

Tzu-Li (Gordon) Tai created FLINK-9885: -- Summary: End-to-end test: Elasticsearch 6.x connector Key: FLINK-9885 URL: https://issues.apache.org/jira/browse/FLINK-9885 Project: Flink Issue

Re: What is the way to prove performance improvement?

Hi Piotrek, Thanks for your reply! These improvements are inspired by some batch table tasks outside flink/master. Let me take a look at that you link to and see if I could define proper benchmark. Best, tison. Piotr Nowojski 于2018年7月18日周三 下午4:45写道： > Hi, > > There is a some small set of

Re: What is the way to prove performance improvement?

Hi, There is a some small set of benchmarks defined in https://github.com/dataArtisans/flink-benchmarks , however their scope is limited and after briefly looking at your PRs, I wouldn’t expect them to cover your cases. However if you could

Re: [DISCUSS] Improve broadcast serialization

Hi, Couple of more thoughts a) I’m not sure if you would have to modify current RecordWriter at all. You could extract interface from current RecordWriter and just provide two implementations: current one and BroadcastRecordWriter. I’m not sure, but it doesn’t seem like they would

Re: Flink CLI properties with HA

Vino, I'm not getting any error but my suspicion was that if I dont specify this `high-availability.storageDir` property in flink CLI side then the CLI will not be able to submit job to flink cluster(HA enabled). But if provide this property in CLI side the job submission will be successful even

[jira] [Created] (FLINK-9884) Slot request may not be removed when it has already be assigned in slot manager

shuai.xu created FLINK-9884: --- Summary: Slot request may not be removed when it has already be assigned in slot manager Key: FLINK-9884 URL: https://issues.apache.org/jira/browse/FLINK-9884 Project: Flink

Re: Flink 1.5 job distribution over cluster nodes

Hi Shachar, with Flink 1.5 we added resource elasticity. This means that Flink is now able to allocate new containers on a cluster management framework like Yarn or Mesos. Due to these changes (which also apply to the standalone mode), Flink no longer reasons about a fixed set of TaskManagers

[jira] [Created] (FLINK-9883) Filter the target window to calculate when deal with slidewindow

aitozi created FLINK-9883: - Summary: Filter the target window to calculate when deal with slidewindow Key: FLINK-9883 URL: https://issues.apache.org/jira/browse/FLINK-9883 Project: Flink Issue

Re: What is the way to prove performance improvement?

Hi Till, Thanks for your reply! I will try to add ones later. Best, tison. Till Rohrmann 于2018年7月18日周三 下午2:48写道： > Hi Tison, > > at the moment there is formal way to verify performance improvements. What > you can do is to provide your measurements by adding the graphs to the PR > thread and

Re: What is the way to prove performance improvement?

Hi Tison, at the moment there is formal way to verify performance improvements. What you can do is to provide your measurements by adding the graphs to the PR thread and specify the setup. Then others could try to verify these numbers by running their own benchmark. Cheers, Till On Wed, Jul 18,