Re: [DISCUSS] Table API Enhancement Outline

Hi Fabian/Xiaowei, I am very sorry for my late reply! Glad to see your reply, and sounds pretty good！ I agree that the approach with append() which can clearly defined the result schema is better which Fabian mentioned. In addition and append() and also contains non-time attributes, e.g.:

[jira] [Created] (FLINK-10873) Remove tableEnv in DataSetConversions#toTable and DataStreamConversions#toTable

Jeff Zhang created FLINK-10873: -- Summary: Remove tableEnv in DataSetConversions#toTable and DataStreamConversions#toTable Key: FLINK-10873 URL: https://issues.apache.org/jira/browse/FLINK-10873 Project:

Re: How to use RocksDBStateBackend predefined options

Hi all We already found the programmatic way to configure RocksDB was not so friendly in our Alibaba's production environment, and refactored it by wrapping customer configurations for RocksDB. We pre-wrapped configurations such as block cache size, whether to cache index into block cache

[jira] [Created] (FLINK-10872) Extend SQL client end-to-end to test KafkaTableSink for kafka connector 0.11

vinoyang created FLINK-10872: Summary: Extend SQL client end-to-end to test KafkaTableSink for kafka connector 0.11 Key: FLINK-10872 URL: https://issues.apache.org/jira/browse/FLINK-10872 Project: Flink

[jira] [Created] (FLINK-10871) ExecutionEnvironment & TableEnvironment should be singleton

Jeff Zhang created FLINK-10871: -- Summary: ExecutionEnvironment & TableEnvironment should be singleton Key: FLINK-10871 URL: https://issues.apache.org/jira/browse/FLINK-10871 Project: Flink

回复：Memory size for network buffers

1. The max value is actually used in current calculation. If the value based on fraction * totalMem is more than max value, we take the max value as the final result. 2. I think there are many things to do behind this, such as the elastic allocation you mentioned. The network buffer is already

[jira] [Created] (FLINK-10870) Reconstruct Flink Dashboard Overview Module to Angular 7.0

Yadong Xie created FLINK-10870: -- Summary: Reconstruct Flink Dashboard Overview Module to Angular 7.0 Key: FLINK-10870 URL: https://issues.apache.org/jira/browse/FLINK-10870 Project: Flink Issue

Re: Flink sql joined with dimtable from mysql

Thanks for your guidance. > 在 2018年11月13日，下午9:49，Xiaowei Jiang 写道： > > It was not super clean on what you did. But from your description, the join > was not correct initially because existing MySQL data was not seen by Flink > yet. Later when updates are seen by Flink, the result will be

Re: [DISCUSS] Enhancing the functionality and productivity of Table API

Hi Piotrek, Fabian: I am very glad to see your reply. Thank you very much Piotrek for asking very good questions. I will share my opinion: - The Enhancing TableAPI that I proposed is proposed for user friendliness. After enhancement, it will maintain the characteristics of TableAPI,

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

Hi Xuefu, Currently the new design doc is on “view only" mode, and people cannot leave comments. Can you please change it to "can comment" or "can edit" mode? Thanks, Bowen On Mon, Nov 12, 2018 at 9:51 PM

Re: Memory size for network buffers

Hi Zhijiang, Thanks for taking a look. Please note that the configuration isn't real, it was just for black box testing of these configuration parameters. Since the max parameter is never used, so why does it even exist? Currently memory size is always fixed, either based on min or fraction.

[jira] [Created] (FLINK-10869) Update S3 testing settings

Stephan Ewen created FLINK-10869: Summary: Update S3 testing settings Key: FLINK-10869 URL: https://issues.apache.org/jira/browse/FLINK-10869 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-10868) Flink's Yarn ResourceManager doesn't use yarn.maximum-failed-containers as limit of resource acquirement

Zhenqiu Huang created FLINK-10868: - Summary: Flink's Yarn ResourceManager doesn't use yarn.maximum-failed-containers as limit of resource acquirement Key: FLINK-10868 URL:

[jira] [Created] (FLINK-10867) Add a DataSet-based CacheOperator to reuse results between jobs

Miguel E. Coimbra created FLINK-10867: - Summary: Add a DataSet-based CacheOperator to reuse results between jobs Key: FLINK-10867 URL: https://issues.apache.org/jira/browse/FLINK-10867 Project:

[jira] [Created] (FLINK-10866) Queryable state can prevent cluster from starting

Till Rohrmann created FLINK-10866: - Summary: Queryable state can prevent cluster from starting Key: FLINK-10866 URL: https://issues.apache.org/jira/browse/FLINK-10866 Project: Flink Issue

Re: [DISCUSS] FLIP-27: Refactor Source Interface

Hi, Thanks again for the detailed answer :) Sorry for responding with a delay. > Completely agree that in pattern 2, having a callback is necessary for that > single thread outside of the connectors. And the connectors MUST have > internal threads. Yes, this thread will have to exists

Re: Flink sql joined with dimtable from mysql

It was not super clean on what you did. But from your description, the join was not correct initially because existing MySQL data was not seen by Flink yet. Later when updates are seen by Flink, the result will be correct. A better place for such question is probably on the user mailing list.

Re: [DISCUSS] Enhancing the functionality and productivity of Table API

Yes, that is my understanding as well. Manual time management would be another difference. Something still to be discussed would be whether (or to what extent) it would be possible to define the physical execution plan with hints or methods like partitionByHash and sortPartition. Best, Fabian

Re: Apply for Contributor Permission

Welcome to the Flink community! I gave you contributor permissions. Regards, Timo Am 13.11.18 um 14:09 schrieb Shuiqiang Chen: Hi guys: Could somebody give me contributor permissions? my jira username is : csq. Thanks.

Apply for Contributor Permission

Hi guys: Could somebody give me contributor permissions? my jira username is : csq. Thanks.

Re: [DISCUSS] Enhancing the functionality and productivity of Table API

Hi, > This thread is meant to enhancing the functionalities of TableAPI. I don't > think that anyone is suggesting either reducing the effort in SQL or > DataStream. So let's focus on how we can enhance TableAPI. I wasn’t thinking about that. As I said before, I was rising a question, what

[jira] [Created] (FLINK-10865) Implement Flink's own Aliyun OSS filesystem

wujinhu created FLINK-10865: --- Summary: Implement Flink's own Aliyun OSS filesystem Key: FLINK-10865 URL: https://issues.apache.org/jira/browse/FLINK-10865 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-10864) Support multiple Main classes per jar

Flavio Pompermaier created FLINK-10864: -- Summary: Support multiple Main classes per jar Key: FLINK-10864 URL: https://issues.apache.org/jira/browse/FLINK-10864 Project: Flink Issue

[jira] [Created] (FLINK-10863) Assign uids to all operators

Stefan Richter created FLINK-10863: -- Summary: Assign uids to all operators Key: FLINK-10863 URL: https://issues.apache.org/jira/browse/FLINK-10863 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-10862) REST API does not show program descriptions of "simple" ProgramDescription

Flavio Pompermaier created FLINK-10862: -- Summary: REST API does not show program descriptions of "simple" ProgramDescription Key: FLINK-10862 URL: https://issues.apache.org/jira/browse/FLINK-10862

[jira] [Created] (FLINK-10861) Kafka e2e test fails during tearing down

Piotr Nowojski created FLINK-10861: -- Summary: Kafka e2e test fails during tearing down Key: FLINK-10861 URL: https://issues.apache.org/jira/browse/FLINK-10861 Project: Flink Issue Type: Bug

Flink sql joined with dimtable from mysql

hi, I want to use flink sql to left join static dimension table from mysql currently, so I converted the mysql table into data stream to join with DataStream which has converted to flink table. While I found that the real-time stream data is not joined correctly with mysql data at the

[jira] [Created] (FLINK-10860) java.lang.ArrayIndexOutOfBoundsException Error reading state

JC created FLINK-10860: -- Summary: java.lang.ArrayIndexOutOfBoundsException Error reading state Key: FLINK-10860 URL: https://issues.apache.org/jira/browse/FLINK-10860 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-10859) Queryable state (rocksdb) end-to-end test fails on Travis

Timo Walther created FLINK-10859: Summary: Queryable state (rocksdb) end-to-end test fails on Travis Key: FLINK-10859 URL: https://issues.apache.org/jira/browse/FLINK-10859 Project: Flink

[jira] [Created] (FLINK-10858) Create Angular Scaffold in Flink Dashboard

Yadong Xie created FLINK-10858: -- Summary: Create Angular Scaffold in Flink Dashboard Key: FLINK-10858 URL: https://issues.apache.org/jira/browse/FLINK-10858 Project: Flink Issue Type: Sub-task