[jira] [Created] (FLINK-16739) PrestoS3FileSystemITCase#testSimpleFileWriteAndRead fails with no such key

2020-03-23 Thread Zhijiang (Jira)
Zhijiang created FLINK-16739: Summary: PrestoS3FileSystemITCase#testSimpleFileWriteAndRead fails with no such key Key: FLINK-16739 URL: https://issues.apache.org/jira/browse/FLINK-16739 Project: Flink

[jira] [Created] (FLINK-16738) Add convenience Kinesis generic egress record builder to the Python SDK

2020-03-23 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-16738: --- Summary: Add convenience Kinesis generic egress record builder to the Python SDK Key: FLINK-16738 URL: https://issues.apache.org/jira/browse/FLINK-16738

[jira] [Created] (FLINK-16737) Remove KubernetesUtils#getContentFromFile

2020-03-23 Thread Canbin Zheng (Jira)
Canbin Zheng created FLINK-16737: Summary: Remove KubernetesUtils#getContentFromFile Key: FLINK-16737 URL: https://issues.apache.org/jira/browse/FLINK-16737 Project: Flink Issue Type:

[jira] [Created] (FLINK-16736) Expose Elasticsearch sink metric

2020-03-23 Thread YangLee (Jira)
YangLee created FLINK-16736: --- Summary: Expose Elasticsearch sink metric Key: FLINK-16736 URL: https://issues.apache.org/jira/browse/FLINK-16736 Project: Flink Issue Type: Improvement

Re: The question about the FLIP-45

2020-03-23 Thread Yu Li
Hi LakeShen, Sorry for the late response. For the first question, literally, the stop command should be used if one means to stop the job instead of canceling it. For the second one, since FLIP-45 is still under discussion [1] [2] (although a little bit stalled due to priority), we still don't

[jira] [Created] (FLINK-16735) FlinkKafkaProducer should check that it is not null before sending a record

2020-03-23 Thread Shangwen Tang (Jira)
Shangwen Tang created FLINK-16735: - Summary: FlinkKafkaProducer should check that it is not null before sending a record Key: FLINK-16735 URL: https://issues.apache.org/jira/browse/FLINK-16735

Re: [DISCUSS] Introduce TableFactory for StatefulSequenceSource

2020-03-23 Thread Jingsong Li
Hi Benchao, > do you think we need to add more columns with various types? I didn't list all types, but we should support primitive types, varchar, Decimal, Timestamp and etc... This can be done continuously. Hi Benchao, Jark, About console and blackhole, yes, they can have no schema, the

[jira] [Created] (FLINK-16733) Refactor YarnClusterDescriptor

2020-03-23 Thread Xintong Song (Jira)
Xintong Song created FLINK-16733: Summary: Refactor YarnClusterDescriptor Key: FLINK-16733 URL: https://issues.apache.org/jira/browse/FLINK-16733 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-16732) Failed to call Hive UDF with constant return value

2020-03-23 Thread Rui Li (Jira)
Rui Li created FLINK-16732: -- Summary: Failed to call Hive UDF with constant return value Key: FLINK-16732 URL: https://issues.apache.org/jira/browse/FLINK-16732 Project: Flink Issue Type: Bug

Re: [DISCUSS] FLIP-95: New TableSource and TableSink interfaces

2020-03-23 Thread Kurt Young
Thanks Timo for the design doc. In general I'm +1 to this, with a minor comment. Since we introduced dozens interfaces all at once, I'm not sure if it's good to annotate them with @PublicEnvolving already. I can imagine these interfaces would only be stable after 1 or 2 major release. Given the

Re: [DISCUSS] FLIP-95: New TableSource and TableSink interfaces

2020-03-23 Thread Becket Qin
Hi Timo, Thanks for the proposal. I completely agree that the current Table connectors could be simplified quite a bit. I haven't finished reading everything, but here are some quick thoughts. Actually to me the biggest question is why should there be two different connector systems for

[jira] [Created] (FLINK-16731) Support show partitions table command in sql client

2020-03-23 Thread Jun Zhang (Jira)
Jun Zhang created FLINK-16731: - Summary: Support show partitions table command in sql client Key: FLINK-16731 URL: https://issues.apache.org/jira/browse/FLINK-16731 Project: Flink Issue Type:

[jira] [Created] (FLINK-16730) Python SDK Getting Started

2020-03-23 Thread Seth Wiesman (Jira)
Seth Wiesman created FLINK-16730: Summary: Python SDK Getting Started Key: FLINK-16730 URL: https://issues.apache.org/jira/browse/FLINK-16730 Project: Flink Issue Type: Improvement

Re: [Dev Blog] Migrating Flink's CI Infrastructure from Travis CI to Azure Pipelines

2020-03-23 Thread Arvid Heise
Thank you Robert! (also thanks for incorporating my feedback so swiftly) On Mon, Mar 23, 2020 at 8:54 PM Seth Wiesman wrote: > Very interesting! No questions but thank you for taking the initiative to > put out the first dev blog. > > Seth > > > On Mar 23, 2020, at 5:14 AM, Robert Metzger

Re: [Dev Blog] Migrating Flink's CI Infrastructure from Travis CI to Azure Pipelines

2020-03-23 Thread Seth Wiesman
Very interesting! No questions but thank you for taking the initiative to put out the first dev blog. Seth > On Mar 23, 2020, at 5:14 AM, Robert Metzger wrote: > > Hi all, > > I have just published the first post to the dev blog: >

Re: [DISCUSS] FLIP-95: New TableSource and TableSink interfaces

2020-03-23 Thread Dawid Wysakowicz
Hi Timo, Thank you for the proposal. I think it is an important improvement that will benefit many parts of the Table API. The proposal looks really good to me and personally I would be comfortable with voting on the current state. Best, Dawid On 23/03/2020 18:53, Timo Walther wrote: > Hi

Dynamic Flink SQL

2020-03-23 Thread Krzysztof Zarzycki
Dear Flink community! In our company we have implemented a system that realize the dynamic business rules pattern. We spoke about it during Flink Forward 2019 https://www.youtube.com/watch?v=CyrQ5B0exqU. The system is a great success and we would like to improve it. Let me shortly mention what

Re: [DISCUSS] FLIP-95: New TableSource and TableSink interfaces

2020-03-23 Thread Timo Walther
Hi everyone, I received some questions around how the new interfaces play together with formats and their factories. Furthermore, for MySQL or Postgres CDC logs, the format should be able to return a `ChangelogMode`. Also, I incorporated the feedback around the factory design in general.

[jira] [Created] (FLINK-16729) Offer an out-of-the-box Set serializer

2020-03-23 Thread Nico Kruber (Jira)
Nico Kruber created FLINK-16729: --- Summary: Offer an out-of-the-box Set serializer Key: FLINK-16729 URL: https://issues.apache.org/jira/browse/FLINK-16729 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-16728) Taskmanager dies after job got stuck and canceling fails

2020-03-23 Thread Leonid Ilyevsky (Jira)
Issue Type: Bug Affects Versions: 1.10.0 Reporter: Leonid Ilyevsky Attachments: taskmanager.log.20200323.gz At some point I noticed that a few jobs got stuck (they basically stopped processing the messages, I could detect this watching the expected output), so I tried

Re: [DISCUSS] JMX remote monitoring integration with Flink

2020-03-23 Thread Rong Rong
Thanks @Till for sharing the JIRA information. I thought as well that this should not be an isolated case to our situation. We would continue to follow up on the JIRA ticket. Best, Rong On Fri, Mar 20, 2020 at 7:30 AM Till Rohrmann wrote: > Hi Rong Rong, > > you are right that it JMX is quite

[DISCUSS] Switch to Azure Pipelines as the primary CI tool / switch off Travis

2020-03-23 Thread Robert Metzger
Hey devs, I would like to discuss whether it makes sense to fully switch to Azure Pipelines and phase out our Travis integration. More information on our Azure integration can be found here:

[jira] [Created] (FLINK-16727) cannot cast 2020-11-12 as class java.time.LocalDate

2020-03-23 Thread Matrix42 (Jira)
Matrix42 created FLINK-16727: Summary: cannot cast 2020-11-12 as class java.time.LocalDate Key: FLINK-16727 URL: https://issues.apache.org/jira/browse/FLINK-16727 Project: Flink Issue Type: Bug

Re: TIME/TIMESTAMP parse in Flink TABLE/SQL API

2020-03-23 Thread Jark Wu
Hi Jingsong, Dawid, I created https://issues.apache.org/jira/browse/FLINK-16725 to track this issue. We can continue discussion there. Best, Jark On Thu, 27 Feb 2020 at 10:32, Jingsong Li wrote: > Hi Jark, > > The matrix I see is SQL cast. If we need bring another conversion matrix > that is

[jira] [Created] (FLINK-16726) ScalarFunction throws Given parameters of function 'func' do not match any signature.

2020-03-23 Thread Matrix42 (Jira)
Matrix42 created FLINK-16726: Summary: ScalarFunction throws Given parameters of function 'func' do not match any signature. Key: FLINK-16726 URL: https://issues.apache.org/jira/browse/FLINK-16726

[jira] [Created] (FLINK-16725) Support to parse various timestamp data format in JSON

2020-03-23 Thread Jark Wu (Jira)
Jark Wu created FLINK-16725: --- Summary: Support to parse various timestamp data format in JSON Key: FLINK-16725 URL: https://issues.apache.org/jira/browse/FLINK-16725 Project: Flink Issue Type: New

[jira] [Created] (FLINK-16724) ListSerializer cannot serialize list which containers null

2020-03-23 Thread Chongchen Chen (Jira)
Chongchen Chen created FLINK-16724: -- Summary: ListSerializer cannot serialize list which containers null Key: FLINK-16724 URL: https://issues.apache.org/jira/browse/FLINK-16724 Project: Flink

[jira] [Created] (FLINK-16723) Move Python SDK example out of statefun-python-sdk

2020-03-23 Thread Igal Shilman (Jira)
Igal Shilman created FLINK-16723: Summary: Move Python SDK example out of statefun-python-sdk Key: FLINK-16723 URL: https://issues.apache.org/jira/browse/FLINK-16723 Project: Flink Issue

[jira] [Created] (FLINK-16722) Add Python SDK walkthrough

2020-03-23 Thread Igal Shilman (Jira)
Igal Shilman created FLINK-16722: Summary: Add Python SDK walkthrough Key: FLINK-16722 URL: https://issues.apache.org/jira/browse/FLINK-16722 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-16721) getProcessCpuLoad seems to report a wrong cpu load

2020-03-23 Thread xiaogang zhou (Jira)
xiaogang zhou created FLINK-16721: - Summary: getProcessCpuLoad seems to report a wrong cpu load Key: FLINK-16721 URL: https://issues.apache.org/jira/browse/FLINK-16721 Project: Flink Issue

[jira] [Created] (FLINK-16720) Maven gets stuck downloading artifacts on Azure

2020-03-23 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-16720: -- Summary: Maven gets stuck downloading artifacts on Azure Key: FLINK-16720 URL: https://issues.apache.org/jira/browse/FLINK-16720 Project: Flink Issue

[jira] [Created] (FLINK-16719) flink-runtime-web install failure

2020-03-23 Thread jackray wang (Jira)
jackray wang created FLINK-16719: Summary: flink-runtime-web install failure Key: FLINK-16719 URL: https://issues.apache.org/jira/browse/FLINK-16719 Project: Flink Issue Type: Bug

[Dev Blog] Migrating Flink's CI Infrastructure from Travis CI to Azure Pipelines

2020-03-23 Thread Robert Metzger
Hi all, I have just published the first post to the dev blog: https://cwiki.apache.org/confluence/display/FLINK/2020/03/22/Migrating+Flink%27s+CI+Infrastructure+from+Travis+CI+to+Azure+Pipelines . I'm looking forward to your feedback and questions on the article :) Best, Robert

[jira] [Created] (FLINK-16718) KvStateServerHandlerTest leaks Netty ByteBufs

2020-03-23 Thread Gary Yao (Jira)
Gary Yao created FLINK-16718: Summary: KvStateServerHandlerTest leaks Netty ByteBufs Key: FLINK-16718 URL: https://issues.apache.org/jira/browse/FLINK-16718 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-16717) Use headless service for rpc and blob port when flink on K8S

2020-03-23 Thread vanderliang (Jira)
vanderliang created FLINK-16717: --- Summary: Use headless service for rpc and blob port when flink on K8S Key: FLINK-16717 URL: https://issues.apache.org/jira/browse/FLINK-16717 Project: Flink

[ANNOUNCE] Feature freeze for Apache Flink Stateful Functions 2.0.0

2020-03-23 Thread Tzu-Li (Gordon) Tai
Hi devs, Following the consensus [1] to release Stateful Functions soon with new version as 2.0.0, I've proceeded to cut the feature branch `release-2.0` [2] for the project. I'll aim to create the first voting release candidate by the end of this week (Mar. 23). The release candidate will also

Re: [DISCUSS] Introduce TableFactory for StatefulSequenceSource

2020-03-23 Thread Jark Wu
Hi Jingsong, Regarding (2) and (3), I was thinking to ignore manually DDL work, so users can use them directly: # this will log results to `.out` files INSERT INTO console SELECT ... # this will drop all received records INSERT INTO blackhole SELECT ... Here `console` and `blackhole` are

[jira] [Created] (FLINK-16716) Update Roadmap after Flink 1.10 release

2020-03-23 Thread Fabian Hueske (Jira)
Fabian Hueske created FLINK-16716: - Summary: Update Roadmap after Flink 1.10 release Key: FLINK-16716 URL: https://issues.apache.org/jira/browse/FLINK-16716 Project: Flink Issue Type: Bug

Re: [DISCUSS] Introduce TableFactory for StatefulSequenceSource

2020-03-23 Thread Benchao Li
Hi Jingsong, Thanks for bring this up. Generally, it's a very good proposal. About data gen source, do you think we need to add more columns with various types? About print sink, do we need to specify the schema? Jingsong Li 于2020年3月23日周一 下午1:51写道: > Thanks Bowen, Jark and Dian for your

[jira] [Created] (FLINK-16715) Always use the configuration argument in YarnClusterDescriptor#startAppMaster to make it more self-contained

2020-03-23 Thread Canbin Zheng (Jira)
Canbin Zheng created FLINK-16715: Summary: Always use the configuration argument in YarnClusterDescriptor#startAppMaster to make it more self-contained Key: FLINK-16715 URL:

[jira] [Created] (FLINK-16714) Kafka committedOffsets value <0

2020-03-23 Thread pingle wang (Jira)
pingle wang created FLINK-16714: --- Summary: Kafka committedOffsets value <0 Key: FLINK-16714 URL: https://issues.apache.org/jira/browse/FLINK-16714 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-16713) Flink can not support the type of source table to query data from es

2020-03-23 Thread jackray wang (Jira)
jackray wang created FLINK-16713: Summary: Flink can not support the type of source table to query data from es Key: FLINK-16713 URL: https://issues.apache.org/jira/browse/FLINK-16713 Project: Flink