[jira] [Created] (FLINK-20760) Broken doc link in Apache Flink Code Style and Quality Guide

[Shengkai Fang (Jira)]

Shengkai Fang created FLINK-20760:
-

 Summary: Broken doc link in Apache Flink Code Style and Quality 
Guide
 Key: FLINK-20760
 URL: https://issues.apache.org/jira/browse/FLINK-20760
 Project: Flink
  Issue Type: Bug
  Components: Documentation
Affects Versions: 1.12.0
Reporter: Shengkai Fang


The 
[page|https://flink.apache.org/contributing/code-style-and-quality-java.md#java-optional]
 should be linked to 
[here|https://flink.apache.org/contributing/code-style-and-quality-java.html#java-optional].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

来自李奇的邮件

[李奇]

Re: [VOTE] Apache Flink Stateful Functions 2.2.2, release candidate #2

[Xingbo Huang]

+1 (non-binding)

- Verify checksums and GPG files
- Verify that the source archives do not contains any binaries
- Build the source with Maven to ensure all source files have Apache
headers (JDK8)
Command: mvn clean install -Papache-release
- Run e2e tests (JDK8)
- Check that all POM files, Dockerfiles, examples point to the same
version. That includes the quickstart artifact POM files.
- pip install apache_flink_statefun-2.2.2-py3-none-any.whl
- Verified NOTICE files in statefun-flink-datastream,
statefun-flink-distribution and statefun-ridesharing-example-simulator

Best,
Xingbo

[jira] [Created] (FLINK-20759) The separator causes job submission to fail through the flink command line

[Ruguo Yu (Jira)]

Ruguo Yu created FLINK-20759:


 Summary: The separator causes job submission to fail through the 
flink command line
 Key: FLINK-20759
 URL: https://issues.apache.org/jira/browse/FLINK-20759
 Project: Flink
  Issue Type: Bug
  Components: Client / Job Submission
Affects Versions: 1.12.0
Reporter: Ruguo Yu
 Fix For: 1.13.0


Flink has some configuration options use semicolon to split multiple value 
content such as yarn.provided.lib.dirs, yarn.ship-archives, yarn.ship-files, 
etc. for example: yarn.ship-files=/aaa/bbb/xxx;/aaa/bbb/yyy;/aaa/bbb/zzz

If this kind configuration options (a semicolon-separated list) is set in the 
flink-conf.yaml file and it is ok to submit a job via a script command 
line(such as bin/flink run-application -t yarn-application ..), however it 
is wrong if set use -D in script command line(suh as bin/flink run-application 
-t yarn-application -Dyarn.ship-files=/aaa/bbb/xxx;/aaa/bbb/yyy;/aaa/bbb/zzz 
..), because the semicolon will cause the script to end and submit job fail.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-20758) Use region file mechanism for shuffle data reading before we switch to managed memory

[Yingjie Cao (Jira)]

Yingjie Cao created FLINK-20758:
---

 Summary: Use region file mechanism for shuffle data reading before 
we switch to managed memory
 Key: FLINK-20758
 URL: https://issues.apache.org/jira/browse/FLINK-20758
 Project: Flink
  Issue Type: Sub-task
  Components: Runtime / Network
Affects Versions: 1.12.0
Reporter: Yingjie Cao
 Fix For: 1.12.2


FLINK-15981 implemented region file based data reader to solve the direct 
memory OOM issue introduced by usage of unmanaged direct memory, however only 
for BoundedBlockingResultPartition. We can introduce it to sort-merge based 
blocking shuffle to avoid the similar direct memory OOM problem which can 
improve the usability a lot.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-20757) Optimize data broadcast for sort-merge shuffle

[Yingjie Cao (Jira)]

Yingjie Cao created FLINK-20757:
---

 Summary: Optimize data broadcast for sort-merge shuffle
 Key: FLINK-20757
 URL: https://issues.apache.org/jira/browse/FLINK-20757
 Project: Flink
  Issue Type: Sub-task
  Components: Runtime / Network
Affects Versions: 1.12.0
Reporter: Yingjie Cao
 Fix For: 1.13.0


For data broadcast, we can only copy the record once when writing data into 
SortBuffer. Besides, we can write only one copy of data when spilling data into 
disk. These optimizations can improve the performance of data broadcast.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-20756) PythonCalcSplitConditionRule is not working as expected

[Wei Zhong (Jira)]

Wei Zhong created FLINK-20756:
-

 Summary: PythonCalcSplitConditionRule is not working as expected
 Key: FLINK-20756
 URL: https://issues.apache.org/jira/browse/FLINK-20756
 Project: Flink
  Issue Type: Bug
Reporter: Wei Zhong


Currently if users write such a SQL:

`SELECT pyFunc5(f0, f1) FROM (SELECT e.f0, e.f1 FROM (SELECT pyFunc5(a) as e 
FROM MyTable) where e.f0 is NULL)`

It will be optimized to:

`FlinkLogicalCalc(select=[pyFunc5(pyFunc5(a)) AS f0])
+- FlinkLogicalCalc(select=[a], where=[IS NULL(pyFunc5(a).f0)])
 +- FlinkLogicalLegacyTableSourceScan(table=[[default_catalog, 
default_database, MyTable, source: [TestTableSource(a, b, c, d)]]], fields=[a, 
b, c, d])`

The optimized plan is not runnable, we need to fix this.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-20755) proctime 在yaml配置中启动sql-clinet报错

[yunsky (Jira)]

yunsky created FLINK-20755:
--

 Summary: proctime 在yaml配置中启动sql-clinet报错
 Key: FLINK-20755
 URL: https://issues.apache.org/jira/browse/FLINK-20755
 Project: Flink
  Issue Type: Bug
  Components: Table SQL / Client
Affects Versions: 1.12.0
Reporter: yunsky


Exception in thread "main" org.apache.flink.table.client.SqlClientException: 
Unexpected exception. This is a bug. Please consider filing an issue.Exception 
in thread "main" org.apache.flink.table.client.SqlClientException: Unexpected 
exception. This is a bug. Please consider filing an issue. at 
org.apache.flink.table.client.SqlClient.main(SqlClient.java:208)Caused by: 
org.apache.flink.table.client.gateway.SqlExecutionException: Could not create 
execution context. at 
org.apache.flink.table.client.gateway.local.ExecutionContext$Builder.build(ExecutionContext.java:878)
 at 
org.apache.flink.table.client.gateway.local.LocalExecutor.openSession(LocalExecutor.java:226)
 at org.apache.flink.table.client.SqlClient.start(SqlClient.java:108) at 
org.apache.flink.table.client.SqlClient.main(SqlClient.java:196)Caused by: 
org.apache.flink.table.api.NoMatchingTableFactoryException: Could not find a 
suitable table factory for 
'org.apache.flink.table.factories.TableSourceFactory' inthe classpath.
Reason: No factory supports all properties.
The matching 
candidates:org.apache.flink.connector.jdbc.table.JdbcTableSourceSinkFactoryUnsupported
 property keys:schema.#.proctime

 

 

 

在sql create语句中可以使用，在yaml报错

schema:
 - name: id
 data-type: VARCHAR
 - name: relation_id
 data-type: VARCHAR
 - name: send_to
 data-type: VARCHAR 
 - name: status
 data-type: VARCHAR
 - name: create_by
 data-type: VARCHAR
 - name: send_to_name
 data-type: VARCHAR 
 - name: proc_time
 data-type: TIMESTAMP(3)
 proctime: true

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-20754) Flink 1.12 docker images not present on Dockerhub

[David Pichler (Jira)]

David Pichler created FLINK-20754:
-

 Summary: Flink 1.12 docker images not present on Dockerhub
 Key: FLINK-20754
 URL: https://issues.apache.org/jira/browse/FLINK-20754
 Project: Flink
  Issue Type: Bug
  Components: Deployment / Kubernetes
Affects Versions: 1.12.0
 Environment: To reproduce:
{code:java}
$ echo 'FROM flink:1.12.0-scala_2.11' > Dockerfile
$ docker build .
Sending build context to Docker daemon  2.048kB
Step 1/1 : FROM flink:1.12.0-scala_2.11
manifest for flink:1.12.0-scala_2.11 not found: manifest unknown: manifest 
unknown{code}
Reporter: David Pichler


As far as I can tell, the flink docker images for 1.12.0 aren't present on 
Dockerhub: 
[https://hub.docker.com/_/flink?tab=tags=1=last_updated=1.12]

Which is breaking the deployment tutorial referencing those images here: 
[https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/resource-providers/standalone/kubernetes.html]

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-20753) Duplicates With Exactly-once Kafka -> Kakfa Producer

[Nazar Volynets (Jira)]

Nazar Volynets created FLINK-20753:
--

 Summary: Duplicates With Exactly-once Kafka -> Kakfa Producer
 Key: FLINK-20753
 URL: https://issues.apache.org/jira/browse/FLINK-20753
 Project: Flink
  Issue Type: Bug
  Components: Connectors / Kafka, Runtime / Checkpointing
Affects Versions: 1.12.0
 Environment: Java 11
Flink stated within IDE
Reporter: Nazar Volynets


*Introduction*

Based on as follows statements from Flink's docs:

1. 
[https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/kafka.html]
{quote}Flink provides an [Apache Kafka|https://kafka.apache.org/] connector for 
reading data from and writing data to Kafka topics with exactly-once guarantees.
{quote}
2. 
[https://ci.apache.org/projects/flink/flink-docs-release-1.11/learn-flink/fault_tolerance.html#exactly-once-end-to-end]
{quote}To achieve exactly once end-to-end, so that every event from the sources 
affects the sinks exactly once, the following must be true:
 # your sources must be replayable, and
 # your sinks must be transactional (or idempotent){quote}
3. 
[https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/kafka.html#caveats]
{quote}{{Semantic.EXACTLY_ONCE}} mode relies on the ability to commit 
transactions that were started before taking a checkpoint, after recovering 
from the said checkpoint. If the time between Flink application crash and 
completed restart is larger than Kafka's transaction timeout there will be data 
loss (Kafka will automatically abort transactions that exceeded timeout time)
{quote}
4. [https://issues.apache.org/jira/browse/FLINK-7210]
There is references/mentions about two-phase commit mechanic used in old Flink 
Kafka connector. So it is expected that latest one version of connector has the 
same functionality.

it is indirectly expectation of EXACTLY_ONCE Kafka->Kafka end-to-end delivery 
guarantees.

Moreover it is emphasised to tune Kafka cluster transaction timeout (make it 
from 15 mins to 1 hour) to omit data loss.

Moving forward, all these three statements are met by `Kafka Source` -> `Kafka 
Sink` app:
 * regarding first-one -> you are reading from & to Kafka
 * about second-one -> `Kafka Source` is replayable & `Kafka Sink` is 
transactional
 * last one -> `Kafka Sink` is transactional & consequently in case of 
EXACTLY_ONCE this operator has a state; so it expected that transaction will be 
rolled back.

But in fact there is no possibility to achieve EXACTLY_ONCE for simple Flink 
`Kafka Source` -> `Kafka Sink` application. Duplicates still exists as result 
EXACTLY_ONCE semantics is violated.

*Details*

+STRs:+
 # Create simple Flink's `Kafka Source` -> `Kafka Sink` app
 ## Stream execution env:
 ### Parallelism -> 1
 ### Enable checkpointing -> 1 ms (do it so big intentionally)
 ### State backend -> RocksDB
 ### Checkpointing mode -> EXACTLY_ONCE
 ### Min pause between checkpoints -> 500 ms
 ### Max concurrent checkpoints -> 1
 ## Flink Kafka consumer
 ### Nothing valuable
 ## Flink Kafka producer
 ### Props:
  ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, "true"
  ProducerConfig.ACKS_CONFIG, "all"
  ProducerConfig.MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION, "1"
 ### EXACTLY_ONCE Semantic
 # Deploy `Kafka Source` Cluster
 ## Cretae `topic-1` with 3 patitions
 # Deploy `Kafka Sink` Cluster
 ## Cretae `topic-1` with 3 patitions
 # Spin up some Kafka client to generate data into `Kafka Source`:`topic-1` 
(e.g. Confluent `kafka-console-producer`)
 # Spin up +transactional+ Kafka consumer to drain data from `Kafka 
Sink`:`topic-1` (e.g. Confluent `kafka-console-consumer`)
 # Use Flink's app described in step #1 to ship data from `Kafka Source` -> 
`Kafka Sink` Kafka cluster.
 # Wait until Flink app will create a first checkpoint. 
 # Brutally kill Flink's app (SIGKILL)
 # Wait 10 secs
 # Start Flink app again.
 # Check on duplications in +transactional+ Kafka consumer (described in step 
#5)

+Actual+

Duplication are exist in +transactional+ Kafka consumer output.

+Expected+
 * Kafka transaction should be rolled back by Flink Kafka producer with 
EXACTLY_ONCE Semantic
 * Flink should automatically replay the data from `Kafka Source` based on 
offsets persisted in latest checkpoint

*Example*

+App+
{code:java|title=build.gradle (dependencies)}
...
ext {
  ...
  javaVersion = '11'
  flinkVersion = '1.12.0'
  scalaBinaryVersion = '2.11'
  ...
}

dependencies {
  ...
  implementation 
"org.apache.flink:flink-streaming-java_${scalaBinaryVersion}:${flinkVersion}"
  implementation 
"org.apache.flink:flink-clients_${scalaBinaryVersion}:${flinkVersion}"
  implementation 
"org.apache.flink:flink-statebackend-rocksdb_${scalaBinaryVersion}:${flinkVersion}"
  ...
}
{code}
{code:java|title=App}
public static void main(String[] args) {
  ...
  StreamExecutionEnvironment env = 
StreamExecutionEnvironment.getExecutionEnvironment();

  

[jira] [Created] (FLINK-20752) FailureRateRestartBackoffTimeStrategy allows one less restart than configured

[Chesnay Schepler (Jira)]

Chesnay Schepler created FLINK-20752:


 Summary: FailureRateRestartBackoffTimeStrategy allows one less 
restart than configured
 Key: FLINK-20752
 URL: https://issues.apache.org/jira/browse/FLINK-20752
 Project: Flink
  Issue Type: Bug
  Components: Runtime / Coordination
Affects Versions: 1.10.0
Reporter: Chesnay Schepler
Assignee: Chesnay Schepler
 Fix For: 1.13.0


The {{FailureRateRestartBackoffTimeStrategy}} maintains a list of failure 
timestamps, keeping N timestamps where N is the configured of failures per 
interval.

The timestamp is added when #notifyFailure() is called, and later evaluated 
within #canRestart().

To determine whether a restart should be allowed we first check whether we are 
already storing N timestamps, and if so check whether the earliest failure 
still falls within the current interval. If it does, we reject the restart.

The problem is that we check whether we have already stored exactly N 
timestamps. If we have exactly N timestamps, and we allow N failures per 
interval, then we should be checking whether we have N+1 timestamps have been 
stored instead.

For example, let's say we allow 2 exceptions, and 2 have occurred so far. 
Regardless of what the timestamps are, we should still allow a restart in this 
case.
Only once a third exception occurs should we be looking at the timestamps, and 
we should furthermore only look at the exception exceeding the allowed failure 
count; in this example it is the very first exception.

I don't know why this went unnoticed for so long; the relevant tests fail 
rather reliably for me locally. ({{FailureRateRestartBackoffTimeStrategyTest}}, 
{{SimpleRecoveryFailureRateStrategyITBase}})

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-20751) Port batch python aggregate nodes to Java

[godfrey he (Jira)]

godfrey he created FLINK-20751:
--

 Summary: Port batch python aggregate nodes to Java
 Key: FLINK-20751
 URL: https://issues.apache.org/jira/browse/FLINK-20751
 Project: Flink
  Issue Type: Sub-task
  Components: Table SQL / Planner
Reporter: godfrey he
 Fix For: 1.13.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-20750) Port stream python aggregate nodes to Java

[godfrey he (Jira)]

godfrey he created FLINK-20750:
--

 Summary: Port stream python aggregate nodes to Java
 Key: FLINK-20750
 URL: https://issues.apache.org/jira/browse/FLINK-20750
 Project: Flink
  Issue Type: Sub-task
  Components: Table SQL / Planner
Reporter: godfrey he
 Fix For: 1.13.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-20749) Allow to register new slots listener at DeclarativeSlotPool

[Till Rohrmann (Jira)]

Till Rohrmann created FLINK-20749:
-

 Summary: Allow to register new slots listener at 
DeclarativeSlotPool
 Key: FLINK-20749
 URL: https://issues.apache.org/jira/browse/FLINK-20749
 Project: Flink
  Issue Type: Improvement
  Components: Runtime / Coordination
Affects Versions: 1.13.0
Reporter: Till Rohrmann
Assignee: Till Rohrmann
 Fix For: 1.13.0


At the moment it is not possible to register a new slots listener at the 
{{DeclarativeSlotPool}} after its creation. This is problematic because the 
listener might not be known at the time of instantiating the 
{{DeclarativeSlotPool}}. I propose to add a {{registerNewSlotListener}} method 
to the {{DeclarativeSlotPool}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-20748) Remap excess slots to outstanding requirements

[Chesnay Schepler (Jira)]

Chesnay Schepler created FLINK-20748:


 Summary: Remap excess slots to outstanding requirements
 Key: FLINK-20748
 URL: https://issues.apache.org/jira/browse/FLINK-20748
 Project: Flink
  Issue Type: Improvement
  Components: Runtime / Coordination
Affects Versions: 1.13.0
Reporter: Chesnay Schepler
 Fix For: 1.13.0


The {{DeclarativeSlotPool}} maintains a mapping of slots to requirements that 
they fulfill.   This mapping is currently determined when the slot is offered 
to the pool, and is only modified if the scheduler reserves this slot for a 
different requirement.

This currently implies 2 things:
 * a slot can be prematurely released, although it could fulfill a different 
outstanding requirement
 * a new slot could be requested, despite there being another slot around that 
could fulfill the requirement

We should remedy this by checking whether excess slots can be re-mapped to a 
different requirement, whenever the requirements change or a slot is freed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-20747) max 函数冲突报ClassCastException异常

[zengjinbobo (Jira)]

zengjinbobo created FLINK-20747:
---

 Summary: max 函数冲突报ClassCastException异常
 Key: FLINK-20747
 URL: https://issues.apache.org/jira/browse/FLINK-20747
 Project: Flink
  Issue Type: Bug
  Components: Connectors / Kafka
Affects Versions: 1.12.0
Reporter: zengjinbobo
 Fix For: 1.12.0
 Attachments: image-2020-12-23-18-04-21-079.png

由于新版1.12.0发布，sql改造过程中，发现1.11.1中能正常执行sql，报异常

java.lang.ClassCastException: java.lang.Integer cannot be cast to 
org.apache.flink.table.data.StringDatajava.lang.ClassCastException: 
java.lang.Integer cannot be cast to org.apache.flink.table.data.StringData at 
org$apache$flink$table$planner$functions$aggfunctions$MaxWithRetractAggFunction$MaxWithRetractAccumulator$Converter.toInternal(Unknown
 Source) at 
org.apache.flink.table.data.conversion.StructuredObjectConverter.toInternal(StructuredObjectConverter.java:92)
 at 
org.apache.flink.table.data.conversion.StructuredObjectConverter.toInternal(StructuredObjectConverter.java:47)
 at 
org.apache.flink.table.data.conversion.DataStructureConverter.toInternalOrNull(DataStructureConverter.java:59)
 at GroupAggsHandler$875.getAccumulators(Unknown Source) at 
org.apache.flink.table.runtime.operators.aggregate.GroupAggFunction.processElement(GroupAggFunction.java:175)
 at 
org.apache.flink.table.runtime.operators.aggregate.GroupAggFunction.processElement(GroupAggFunction.java:45)
 at 
org.apache.flink.streaming.api.operators.KeyedProcessOperator.processElement(KeyedProcessOperator.java:85)
 at 
org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:193)
 at 
org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.processElement(StreamTaskNetworkInput.java:179)
 at 
org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.emitNext(StreamTaskNetworkInput.java:152)
 at 
org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:67)
 at 
org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:372)
 at 
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:186)
 at 
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:575)
 at 
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:539) 
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:722) at 
org.apache.flink.runtime.taskmanager.Task.run(Task.java:547)

 

 

!image-2020-12-23-18-04-21-079.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Releasing Apache Flink 1.12.1

[Xintong Song]

Hi devs,

Updates on the progress of release.

In the past week, more than 20 issues were resolved for release 1.12.1.
Thanks for the efforts.

We still have 3 unresolved release blockers at the moment.

   - [FLINK-20648] Unable to restore from savepoints with Kubernetes HA.
   Consensus has been reached on the solution. @Yang Wang is working on a
   PR.
   - [FLINK-20654] Unaligned checkpoint recovery may lead to corrupted data
   stream.
   @Roman Khachatryan is still investigating the problem.
   - [FLINK-20664] Support setting service account for TaskManager pod.
   Boris Lublinsky has opened a PR, which is already reviewed and close to
   mergeable.

Since we are targeting a swift release, I'm not intended to further delay
the release for other non-blocker issues, unless there's a good reason.
If there's anything that you believe is absolutely necessary for release
1.12.1, please reach out to me.
Otherwise, the voting process will be started as soon as the above blockers
are addressed.

Thank you~

Xintong Song



On Mon, Dec 21, 2020 at 10:05 AM Xingbo Huang  wrote:

> Hi Xintong,
>
> Thanks a lot for driving this.
>
> I'd like to bring one more issue to your attention:
> https://issues.apache.org/jira/browse/FLINK-20389.
> This issue occurs quite frequently. Arvid and Kezhu have done some
> investigations of this issue and it may indicate a bug of the new Source
> API. It would be great to figure out the root cause of this issue.
>
> Best,
> Xingbo
>
> Xintong Song  于2020年12月18日周五 下午7:49写道：
>
> > Thanks for the replies so far.
> >
> > I've been reaching out to the owners of the reported issues. It seems
> most
> > of the blockers are likely resolved in the next few days.
> >
> > Since some of the issues are quite critical, I'd like to aim for a
> *feature
> > freeze on Dec. 23rd*, and start the release voting process by the end of
> > this week.
> >
> > If there's anything you might need more time for, please reach out to me.
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> >
> > On Fri, Dec 18, 2020 at 3:19 PM Tzu-Li (Gordon) Tai  >
> > wrote:
> >
> > > Thanks Xintong for driving this.
> > >
> > > I'd like to make two more issues related to the Kinesis connector
> changes
> > > in 1.12.0 a blocker for 1.12.1:
> > > https://issues.apache.org/jira/browse/FLINK-20630
> > > https://issues.apache.org/jira/browse/FLINK-20629
> > >
> > > There are already PRs for these issues from @Cranmer, Danny
> > > , will try to merge these very soon.
> > >
> > > Cheers,
> > > Gordon
> > >
> > > On Fri, Dec 18, 2020 at 1:19 PM Guowei Ma 
> wrote:
> > >
> > >> Thanks for driving this release Xintong.
> > >> I think https://issues.apache.org/jira/browse/FLINK-20652 should be
> > >> addressed.
> > >>
> > >> Best,
> > >> Guowei
> > >>
> > >>
> > >> On Fri, Dec 18, 2020 at 11:53 AM Jingsong Li 
> > >> wrote:
> > >>
> > >> > Thanks for volunteering as our release manager Xintong. +1 for
> > releasing
> > >> > Flink 1.12.1 soon.
> > >> >
> > >> > I think https://issues.apache.org/jira/browse/FLINK-20665 should be
> > >> > addressed, I marked it as a Blocker.
> > >> >
> > >> > Best,
> > >> > Jingsong
> > >> >
> > >> > On Fri, Dec 18, 2020 at 11:16 AM Yang Wang 
> > >> wrote:
> > >> >
> > >> > > Hi David,
> > >> > >
> > >> > > I will take a look this ticket FLINK-20648 and try to get it
> > resolved
> > >> in
> > >> > > this release cycle.
> > >> > >
> > >> > > @Xintong Song 
> > >> > > One more Kubernetes HA related issue. We need to support setting
> > >> service
> > >> > > account for TaskManager pod[1]. Even though we have a work around
> > for
> > >> > this
> > >> > > issue, but it is not acceptable to always let the default service
> > >> account
> > >> > > with enough permissions.
> > >> > >
> > >> > > [1]. https://issues.apache.org/jira/browse/FLINK-20664
> > >> > >
> > >> > > Best,
> > >> > > Yang
> > >> > >
> > >> > >
> > >> > > David Morávek  于2020年12月18日周五 上午12:47写道：
> > >> > >
> > >> > > > Hi, I think https://issues.apache.org/jira/browse/FLINK-20648
> > >> should
> > >> > be
> > >> > > > addressed, as Kubernetes HA was one of the main selling points
> of
> > >> this
> > >> > > > release. WDYT?
> > >> > > >
> > >> > > > D.
> > >> > > >
> > >> > > > Sent from my iPhone
> > >> > > >
> > >> > > > > On 17. 12. 2020, at 13:54, Yun Tang  wrote:
> > >> > > > >
> > >> > > > > Thanks for driving this quick-fix release.
> > >> > > > > +1 for fixing the bug of RocksDB state-backend with reduce
> > >> operators.
> > >> > > > >
> > >> > > > > Best
> > >> > > > > Yun Tang
> > >> > > > > 
> > >> > > > > From: Till Rohrmann 
> > >> > > > > Sent: Thursday, December 17, 2020 20:51
> > >> > > > > To: dev 
> > >> > > > > Subject: Re: [DISCUSS] Releasing Apache Flink 1.12.1
> > >> > > > >
> > >> > > > > Thanks for volunteering as our release manager Xintong. +1
> for a
> > >> > swift
> > >> > > > bug
> > >> > > > > fix release.
> > >> > > > >
> > >> > > > > Cheers,
> > >> > > > > Till
> > >> > > > >

[jira] [Created] (FLINK-20746) Support ignore-first-line option for CSV format

[Leonard Xu (Jira)]

Leonard Xu created FLINK-20746:
--

 Summary: Support ignore-first-line option for CSV format
 Key: FLINK-20746
 URL: https://issues.apache.org/jira/browse/FLINK-20746
 Project: Flink
  Issue Type: Improvement
  Components: Connectors / FileSystem, Formats (JSON, Avro, Parquet, 
ORC, SequenceFile)
Affects Versions: 1.13.0
Reporter: Leonard Xu


ignore-first-line option is a useful feature for CSV format in filesystem 
connector, and I found there're users  consulting the feature in 
stackoverflow[1]. 

 

[1]https://stackoverflow.com/questions/65359382/apache-flink-sql-reference-guide-for-table-properties



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[DISCUSS][FLINK-20726] Introduce Pulsar connector

[赵 建云]

We are now done with the Flink Pulsar connector. It is ready to be merged.

Now our plan is

  1.  merge Source and Sink based on Function, because it has been verified by 
users and is reliable.
  2.  Next is to merge the DynamicTable feature. table implementation depends 
on Step 1.
  3.  merge Pulsar Catalog
  4.  merge the PulsarSource based on FLIP-27: Refactor Source 
Interface
 of PulsarSource
  5.  merge PulsarSink based on FLIP-143: Unified Sink 
API
  6.  merge upsert-pulsar

About wiki, I need to update the FLIP-72: Introduce Pulsar 
Connector,
 but I don't have access to it.

StreamNative/Pulsar Flink Connector: 
https://github.com/streamnative/pulsar-flink


Looking forward to your feedback~

Jianyun Zhao

[jira] [Created] (FLINK-20745) Clean useless codes: Never push calcProgram to correlate

[Jingsong Lee (Jira)]

Jingsong Lee created FLINK-20745:


 Summary: Clean useless codes: Never push calcProgram to correlate
 Key: FLINK-20745
 URL: https://issues.apache.org/jira/browse/FLINK-20745
 Project: Flink
  Issue Type: Improvement
  Components: Table SQL / Planner
Reporter: Jingsong Lee
Assignee: Jingsong Lee






--
This message was sent by Atlassian Jira
(v8.3.4#803005)