[jira] [Created] (FLINK-14767) Remove TaskManagerOptions#EXIT_ON_FATAL_AKKA_ERROR

vinoyang created FLINK-14767: Summary: Remove TaskManagerOptions#EXIT_ON_FATAL_AKKA_ERROR Key: FLINK-14767 URL: https://issues.apache.org/jira/browse/FLINK-14767 Project: Flink Issue Type:

Re: Flink on Yarn resource arrangement

Hi Alex, Which Flink version are you using? AFAIK, since Flink 1.8+, the config option: "-yn" for Flink on YARN job cluster mode does not take effect(always 1 and would be overridden). So, the config option "-ys" and "-p" will decide the number of TM. The first example: -p(20)/-ys(3) should be

Flink on Yarn resource arrangement

Hi all, Could you list details how Flink job on Yarn resources managed ? I used command “-p 20 -yn 5 -ys 3 -yjm 2048m -ytm 2048m” to run flink job. I got containers vcores 8 22 Task Managers 7 Total Task Slots 21 I used command “-p 20 -yn 7 -ys 4 -yjm 2048m -ytm 2048m” to

[jira] [Created] (FLINK-14766) Remove volatile variable in ExecutionVertex

Jiayi Liao created FLINK-14766: -- Summary: Remove volatile variable in ExecutionVertex Key: FLINK-14766 URL: https://issues.apache.org/jira/browse/FLINK-14766 Project: Flink Issue Type:

[jira] [Created] (FLINK-14765) Remove STATE_UPDATER in Execution

vinoyang created FLINK-14765: Summary: Remove STATE_UPDATER in Execution Key: FLINK-14765 URL: https://issues.apache.org/jira/browse/FLINK-14765 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] Expose or setup a secur...@flink.apache.org mailing list for security report and discussion

Hi Dian, Good idea and +1 to setup security mailing list. Security vulnerabilities should not be publicly disclosed (e.g. via dev ML or JIRA) until the project has responded. However, AFAIK, Flink doesn't have an official process to report vulnerabilities. It would be nice to have one to protect

Re: [DISCUSS] Expose or setup a secur...@flink.apache.org mailing list for security report and discussion

Hi Dian, Good idea! +1 to have a security mailing list. It is nice for Flink to have an official procedure to handle security problems, e.g., reporting, addressing and publishing. Best, Hequn On Thu, Nov 14, 2019 at 1:20 PM Jeff Zhang wrote: > Thanks Dian Fu for this proposal. +1 for creating

Re: [DISCUSS] Expose or setup a secur...@flink.apache.org mailing list for security report and discussion

Thanks Dian Fu for this proposal. +1 for creating security mail list. To be noticed, security mail list is private mail list, could not be subscribed publicly. FYI, apache member can create mail list using this self service tool https://selfserve.apache.org/ jincheng sun 于2019年11月14日周四

Re: Error:java: 无效的标记: --add-exports=java.base/sun.net.util=ALL-UNNAMED

Hi, I have encountered the same issue when setting up a dev environment. It seems that the my Intellij (2019.2.1) unexpectedly activates java11 profile of maven. It doesn't match the Java compiler (JDK8). I'm not sure why it happened silently. So for me, the solution is "Intellij" -> "View" ->

[jira] [Created] (FLINK-14764) The final Explicit/Implicit conversion matrix we should support in our planner

Zhenghua Gao created FLINK-14764: Summary: The final Explicit/Implicit conversion matrix we should support in our planner Key: FLINK-14764 URL: https://issues.apache.org/jira/browse/FLINK-14764

Re: [DISCUSS] Expose or setup a secur...@flink.apache.org mailing list for security report and discussion

Hi Dian, Thanks a lot for bringing up this discussion. This is very important for Flink community! I think setup a security mailing list for Flink is pretty nice although ` secur...@apache.org` can be used and the report will be forwarded to Flink private mailing list if there is no project

[DISCUSS] Expose or setup a secur...@flink.apache.org mailing list for security report and discussion

Hi all, I'm reaching out to see if there is an existing security specific mailing list in Flink. If there is, we should expose it in the offcial web site of Flink [1] to guide people to report security issues to this mailing list. If it still doesn't exist, I'm here to propose to setup a

Re: [DISCUSS] FLIP-86: Improve Connector Properties

Hi Dawid, Konstantin, This is an interesting topic. I will list the prons and cons of these 2 options. 1) under a single key: `connector.properties` Cons: - Users can pass in arbitrary properties defined in kafka docs Prons: - The property value is collapsed together and will be long and not

[jira] [Created] (FLINK-14763) cep parallelism error

richt richt created FLINK-14763: --- Summary: cep parallelism error Key: FLINK-14763 URL: https://issues.apache.org/jira/browse/FLINK-14763 Project: Flink Issue Type: Bug Components:

[jira] [Created] (FLINK-14762) Implement ClusterClientJobClientAdapter

Zili Chen created FLINK-14762: - Summary: Implement ClusterClientJobClientAdapter Key: FLINK-14762 URL: https://issues.apache.org/jira/browse/FLINK-14762 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] Releasing Flink 1.8.3

Hi Chesnay, Jincheng Sure, I think it's good to have these fixes. Thanks a lot for providing the information about the security vulnerabilities! @Chesnay Best, Hequn On Thu, Nov 14, 2019 at 10:07 AM jincheng sun wrote: > +1 for try to eliminate the security vulnerabilities. Great thanks for >

Re: [DISCUSS] Releasing Flink 1.8.3

+1 for try to eliminate the security vulnerabilities. Great thanks for doing this important work, Chesnay! What do you think Hequn ? Best, Jincheng Chesnay Schepler 于2019年11月13日周三 下午5:17写道： > It would be great if you could give me a day or 2 to check how easy it > would be to bump the various

[jira] [Created] (FLINK-14761) ProgramDeployer#deployJobOnNewCluster potentially leak cluster

Zili Chen created FLINK-14761: - Summary: ProgramDeployer#deployJobOnNewCluster potentially leak cluster Key: FLINK-14761 URL: https://issues.apache.org/jira/browse/FLINK-14761 Project: Flink

[jira] [Created] (FLINK-14760) Documentation links check failed on travis

Dian Fu created FLINK-14760: --- Summary: Documentation links check failed on travis Key: FLINK-14760 URL: https://issues.apache.org/jira/browse/FLINK-14760 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-14759) Remove unused class TaskManagerCliOptions

Zili Chen created FLINK-14759: - Summary: Remove unused class TaskManagerCliOptions Key: FLINK-14759 URL: https://issues.apache.org/jira/browse/FLINK-14759 Project: Flink Issue Type: Sub-task

flink checkpoint state data corruption

Hi all, I'm using protobufs as keys of a Flink stream using code copied from this pull request , but deserialization is failing after checkpoint restore due to missing data. I'm using HDFS and the RocksDB backend. I tried providing the path to a

[jira] [Created] (FLINK-14758) Add Executor-related interfaces and utilities

Kostas Kloudas created FLINK-14758: -- Summary: Add Executor-related interfaces and utilities Key: FLINK-14758 URL: https://issues.apache.org/jira/browse/FLINK-14758 Project: Flink Issue

[jira] [Created] (FLINK-14757) scala.StreamExecutionEnvironment Doesn't set return type upon adding source

Tim Bess created FLINK-14757: Summary: scala.StreamExecutionEnvironment Doesn't set return type upon adding source Key: FLINK-14757 URL: https://issues.apache.org/jira/browse/FLINK-14757 Project: Flink

Re: [DISCUSS] FLIP-86: Improve Connector Properties

Hi Dawid, Hi Jark, in my experience it is very important to be able to forward arbitrary properties to the underlying KafkaClient. This seems to be possible in both cases. I am leaning towards Jark's original suggestion. Flink's documentation would only need to state that it forwards everything

[jira] [Created] (FLINK-14756) Migrate duration and memory size ConfigOptions in HeartbeatManagerOptions

Dawid Wysakowicz created FLINK-14756: Summary: Migrate duration and memory size ConfigOptions in HeartbeatManagerOptions Key: FLINK-14756 URL: https://issues.apache.org/jira/browse/FLINK-14756

[jira] [Created] (FLINK-14755) Migrate duration and memory size ConfigOptions in BlobServerOptions

Dawid Wysakowicz created FLINK-14755: Summary: Migrate duration and memory size ConfigOptions in BlobServerOptions Key: FLINK-14755 URL: https://issues.apache.org/jira/browse/FLINK-14755 Project:

[jira] [Created] (FLINK-14754) Migrate duration and memory size ConfigOptions in RestOptions

Dawid Wysakowicz created FLINK-14754: Summary: Migrate duration and memory size ConfigOptions in RestOptions Key: FLINK-14754 URL: https://issues.apache.org/jira/browse/FLINK-14754 Project: Flink

[jira] [Created] (FLINK-14752) Migrate duration and memory size ConfigOptions in ClusterOptions

Dawid Wysakowicz created FLINK-14752: Summary: Migrate duration and memory size ConfigOptions in ClusterOptions Key: FLINK-14752 URL: https://issues.apache.org/jira/browse/FLINK-14752 Project:

[jira] [Created] (FLINK-14753) Migrate duration and memory size ConfigOptions in AkkaOptions

Dawid Wysakowicz created FLINK-14753: Summary: Migrate duration and memory size ConfigOptions in AkkaOptions Key: FLINK-14753 URL: https://issues.apache.org/jira/browse/FLINK-14753 Project: Flink

[jira] [Created] (FLINK-14751) Migrate duration and memory size ConfigOptions in TaskManagerOptions

Dawid Wysakowicz created FLINK-14751: Summary: Migrate duration and memory size ConfigOptions in TaskManagerOptions Key: FLINK-14751 URL: https://issues.apache.org/jira/browse/FLINK-14751

[jira] [Created] (FLINK-14748) Migrate duration and memory size related ConfigOptions to typed options

Dawid Wysakowicz created FLINK-14748: Summary: Migrate duration and memory size related ConfigOptions to typed options Key: FLINK-14748 URL: https://issues.apache.org/jira/browse/FLINK-14748

[jira] [Created] (FLINK-14750) Migrate duration and memory size ConfigOptions in RestartStrategyOptions

Dawid Wysakowicz created FLINK-14750: Summary: Migrate duration and memory size ConfigOptions in RestartStrategyOptions Key: FLINK-14750 URL: https://issues.apache.org/jira/browse/FLINK-14750

[jira] [Created] (FLINK-14749) Migrate duration and memory size ConfigOptions in JobManagerOptions

Dawid Wysakowicz created FLINK-14749: Summary: Migrate duration and memory size ConfigOptions in JobManagerOptions Key: FLINK-14749 URL: https://issues.apache.org/jira/browse/FLINK-14749 Project:

[jira] [Created] (FLINK-14747) Performance regression with latest changes to Mailbox

Arvid Heise created FLINK-14747: --- Summary: Performance regression with latest changes to Mailbox Key: FLINK-14747 URL: https://issues.apache.org/jira/browse/FLINK-14747 Project: Flink Issue

[jira] [Created] (FLINK-14746) History server does not handle uncaught exceptions in archive fetcher.

David Moravek created FLINK-14746: - Summary: History server does not handle uncaught exceptions in archive fetcher. Key: FLINK-14746 URL: https://issues.apache.org/jira/browse/FLINK-14746 Project:

Re: [DISCUSS] FLIP-86: Improve Connector Properties

Hi Jark, Majority of the changes make sense to me. I think they will be helpful for the users. I have a slight concern about the kafka's connector.properties property :) .  I wonder if we should have them under a single key: connector.properties:

[jira] [Created] (FLINK-14744) Neighboring vertex should be chained if ship strategy is FORWARD or NONE

Jiayi Liao created FLINK-14744: -- Summary: Neighboring vertex should be chained if ship strategy is FORWARD or NONE Key: FLINK-14744 URL: https://issues.apache.org/jira/browse/FLINK-14744 Project: Flink

[DISCUSS] FLIP-86: Improve Connector Properties

Hi everyone, Connector properties is a very basic component which is used to construct a connector table via YAML, descriptor API, or DDL. It is also the serialization representation when persisting into catalog. However, we have encountered some problems when using connector properties,

[jira] [Created] (FLINK-14743) Optimize BaseRowSerializer to use Projection instead of switch

Jingsong Lee created FLINK-14743: Summary: Optimize BaseRowSerializer to use Projection instead of switch Key: FLINK-14743 URL: https://issues.apache.org/jira/browse/FLINK-14743 Project: Flink

[jira] [Created] (FLINK-14742) Unstable tests TaskExecutorTest#testSubmitTaskBeforeAcceptSlot

Zili Chen created FLINK-14742: - Summary: Unstable tests TaskExecutorTest#testSubmitTaskBeforeAcceptSlot Key: FLINK-14742 URL: https://issues.apache.org/jira/browse/FLINK-14742 Project: Flink

[jira] [Created] (FLINK-14741) Document pluggable sources and sinks in the production code for testing

Sendoh created FLINK-14741: -- Summary: Document pluggable sources and sinks in the production code for testing Key: FLINK-14741 URL: https://issues.apache.org/jira/browse/FLINK-14741 Project: Flink

[jira] [Created] (FLINK-14740) Create OperatorID for OperatorMetricGroup which in batch job

lining created FLINK-14740: -- Summary: Create OperatorID for OperatorMetricGroup which in batch job Key: FLINK-14740 URL: https://issues.apache.org/jira/browse/FLINK-14740 Project: Flink Issue

[jira] [Created] (FLINK-14739) add failOnCastException Configuration to Json FormatDescriptor

Jun Zhang created FLINK-14739: - Summary: add failOnCastException Configuration to Json FormatDescriptor Key: FLINK-14739 URL: https://issues.apache.org/jira/browse/FLINK-14739 Project: Flink

Re: [DISCUSS] Releasing Flink 1.8.3

It would be great if you could give me a day or 2 to check how easy it would be to bump the various jackson dependencies to eliminate a few security vulnerabilities. On 09/11/2019 05:10, jincheng sun wrote: Hi Flink devs, It has been more than 2 months since the 1.8.2 released. So, What do

[jira] [Created] (FLINK-14738) Job which status is finished, get operator metric

lining created FLINK-14738: -- Summary: Job which status is finished, get operator metric Key: FLINK-14738 URL: https://issues.apache.org/jira/browse/FLINK-14738 Project: Flink Issue Type: Wish

[jira] [Created] (FLINK-14737) HiveTypeUtil.toFlinkPrimitiveType returns wrong Timestamp(6) instead of Timestamp(9)

Zhenghua Gao created FLINK-14737: Summary: HiveTypeUtil.toFlinkPrimitiveType returns wrong Timestamp(6) instead of Timestamp(9) Key: FLINK-14737 URL: https://issues.apache.org/jira/browse/FLINK-14737

[jira] [Created] (FLINK-14736) Fail to connect Hive metastore

Jiayi Liao created FLINK-14736: -- Summary: Fail to connect Hive metastore Key: FLINK-14736 URL: https://issues.apache.org/jira/browse/FLINK-14736 Project: Flink Issue Type: Bug