Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

2018-11-14 Thread Zhang, Xuefu
Thanks, Bowen, for catching the error. I have granted comment permission with the link. I also updated the doc with the latest class definitions. Everyone is encouraged to review and comment. Thanks, Xuefu -- Sender:Bowen Li

[jira] [Created] (FLINK-10891) Upgrade Kafka client version to 2.0.1

2018-11-14 Thread vinoyang (JIRA)
vinoyang created FLINK-10891: Summary: Upgrade Kafka client version to 2.0.1 Key: FLINK-10891 URL: https://issues.apache.org/jira/browse/FLINK-10891 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-10890) CLONE - Add DataStream HBase Sink

2018-11-14 Thread jocean....@gamil.com (JIRA)
jocean@gamil.com created FLINK-10890: Summary: CLONE - Add DataStream HBase Sink Key: FLINK-10890 URL: https://issues.apache.org/jira/browse/FLINK-10890 Project: Flink Issue

Re: [DISCUSS] FLIP-27: Refactor Source Interface

2018-11-14 Thread Becket Qin
Hi Piotrek, Thanks a lot for the detailed reply. All makes sense to me. WRT the confusion between advance() / getCurrent(), do you think it would help if we combine them and have something like: CompletableFuture getNext(); long getWatermark(); long getCurrentTimestamp(); Cheers, Jiangjie

Need help with Kinesis related PRs

2018-11-14 Thread Thomas Weise
Hi, The following two PRs need review from committers: https://github.com/apache/flink/pull/6980 https://github.com/apache/flink/pull/6968 Any help greatly appreciated! Thanks, Thomas

[jira] [Created] (FLINK-10889) Semantic inconsistency between DataSet#print and DataStream#print

2018-11-14 Thread Jeff Zhang (JIRA)
Jeff Zhang created FLINK-10889: -- Summary: Semantic inconsistency between DataSet#print and DataStream#print Key: FLINK-10889 URL: https://issues.apache.org/jira/browse/FLINK-10889 Project: Flink

Re: Sharing state between subtasks

2018-11-14 Thread Jamie Grier
Hey all, I think all we need for this on the state sharing side is pretty simple. I opened a JIRA to track this work and submitted a PR for the state sharing bit. https://issues.apache.org/jira/browse/FLINK-10886 https://github.com/apache/flink/pull/7099 Please provide feedback :) -Jamie On

[jira] [Created] (FLINK-10888) Expose new global watermark RPC to sources

2018-11-14 Thread Jamie Grier (JIRA)
Jamie Grier created FLINK-10888: --- Summary: Expose new global watermark RPC to sources Key: FLINK-10888 URL: https://issues.apache.org/jira/browse/FLINK-10888 Project: Flink Issue Type:

[jira] [Created] (FLINK-10887) Add source watermarking tracking to the JobMaster

2018-11-14 Thread Jamie Grier (JIRA)
Jamie Grier created FLINK-10887: --- Summary: Add source watermarking tracking to the JobMaster Key: FLINK-10887 URL: https://issues.apache.org/jira/browse/FLINK-10887 Project: Flink Issue Type:

[jira] [Created] (FLINK-10886) Event time synchronization across sources

2018-11-14 Thread Jamie Grier (JIRA)
Jamie Grier created FLINK-10886: --- Summary: Event time synchronization across sources Key: FLINK-10886 URL: https://issues.apache.org/jira/browse/FLINK-10886 Project: Flink Issue Type:

Re: support/docs for compression in StreamingFileSink

2018-11-14 Thread Addison Higham
Just noticed one detail about using the BulkWriter interface, you no longer can assign a rolling policy. That makes sense for formats like orc/parquet, but perhaps not for simple text compression. On Wed, Nov 14, 2018 at 1:43 PM Addison Higham wrote: > HI all, > > I am moving some code to use

support/docs for compression in StreamingFileSink

2018-11-14 Thread Addison Higham
HI all, I am moving some code to use the StreamingFileSink. Currently, it doesn't look like there is any native support for compression (gzip or otherwise) built into flink when using the StreamingFileSink. It seems like this is a really common need that as far as I could tell, wasn't represented

[jira] [Created] (FLINK-10885) Avro Confluent Schema Registry E2E test failed on Travis

2018-11-14 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-10885: Summary: Avro Confluent Schema Registry E2E test failed on Travis Key: FLINK-10885 URL: https://issues.apache.org/jira/browse/FLINK-10885 Project: Flink

[jira] [Created] (FLINK-10884) Flink on yarn TM container will be killed by nodemanager because of the exceeded physical memory.

2018-11-14 Thread wgcn (JIRA)
wgcn created FLINK-10884: Summary: Flink on yarn TM container will be killed by nodemanager because of the exceeded physical memory. Key: FLINK-10884 URL: https://issues.apache.org/jira/browse/FLINK-10884

[jira] [Created] (FLINK-10883) Submitting a jobs without enough slots times out due to a unspecified timeout

2018-11-14 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-10883: Summary: Submitting a jobs without enough slots times out due to a unspecified timeout Key: FLINK-10883 URL: https://issues.apache.org/jira/browse/FLINK-10883

[jira] [Created] (FLINK-10882) Misleading job/task state for scheduled jobs

2018-11-14 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-10882: Summary: Misleading job/task state for scheduled jobs Key: FLINK-10882 URL: https://issues.apache.org/jira/browse/FLINK-10882 Project: Flink Issue

[jira] [Created] (FLINK-10881) SavepointITCase deadlocks on travis

2018-11-14 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-10881: Summary: SavepointITCase deadlocks on travis Key: FLINK-10881 URL: https://issues.apache.org/jira/browse/FLINK-10881 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-10880) Failover strategies should not be applied to Batch Execution

2018-11-14 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-10880: Summary: Failover strategies should not be applied to Batch Execution Key: FLINK-10880 URL: https://issues.apache.org/jira/browse/FLINK-10880 Project: Flink

Re: REST job submission

2018-11-14 Thread Flavio Pompermaier
Done: https://issues.apache.org/jira/browse/FLINK-10879 On Wed, Nov 14, 2018 at 10:14 AM Chesnay Schepler wrote: > I wouldn't consider it a _bug_ in that sense, but agree that the current > behavior isn't ideal. Running a job via the CLI or WebUI should behave > the same way, as such please

[jira] [Created] (FLINK-10879) Align Flink clients on env.execute()

2018-11-14 Thread Flavio Pompermaier (JIRA)
Flavio Pompermaier created FLINK-10879: -- Summary: Align Flink clients on env.execute() Key: FLINK-10879 URL: https://issues.apache.org/jira/browse/FLINK-10879 Project: Flink Issue Type:

[jira] [Created] (FLINK-10878) State evolution E2E test failed on travis

2018-11-14 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-10878: Summary: State evolution E2E test failed on travis Key: FLINK-10878 URL: https://issues.apache.org/jira/browse/FLINK-10878 Project: Flink Issue

[jira] [Created] (FLINK-10877) Remove duplicate dependency entries in kafka connector pom

2018-11-14 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-10877: - Summary: Remove duplicate dependency entries in kafka connector pom Key: FLINK-10877 URL: https://issues.apache.org/jira/browse/FLINK-10877 Project: Flink

[jira] [Created] (FLINK-10876) Deadlock if closing firstly pending transactions in FlinkKafkaProducer(011).close()

2018-11-14 Thread Andrey Zagrebin (JIRA)
Andrey Zagrebin created FLINK-10876: --- Summary: Deadlock if closing firstly pending transactions in FlinkKafkaProducer(011).close() Key: FLINK-10876 URL: https://issues.apache.org/jira/browse/FLINK-10876

Re: REST job submission

2018-11-14 Thread Chesnay Schepler
I wouldn't consider it a _bug_ in that sense, but agree that the current behavior isn't ideal. Running a job via the CLI or WebUI should behave the same way, as such please open a JIRA. On 12.11.2018 12:50, Flavio Pompermaier wrote: Hi to all, in our ETL we need to call an external (REST)

[jira] [Created] (FLINK-10875) Add `toTableWithTimestamp` method in `DataStreamConversions`

2018-11-14 Thread sunjincheng (JIRA)
sunjincheng created FLINK-10875: --- Summary: Add `toTableWithTimestamp` method in `DataStreamConversions` Key: FLINK-10875 URL: https://issues.apache.org/jira/browse/FLINK-10875 Project: Flink

[jira] [Created] (FLINK-10874) Kafka 2.0 connector testMigrateFromAtLeastOnceToExactlyOnce failure

2018-11-14 Thread Piotr Nowojski (JIRA)
Piotr Nowojski created FLINK-10874: -- Summary: Kafka 2.0 connector testMigrateFromAtLeastOnceToExactlyOnce failure Key: FLINK-10874 URL: https://issues.apache.org/jira/browse/FLINK-10874 Project: