[jira] [Created] (FLINK-21148) YARNSessionFIFOSecuredITCase cannot connect to BlobServer

2021-01-25 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-21148: Summary: YARNSessionFIFOSecuredITCase cannot connect to BlobServer Key: FLINK-21148 URL: https://issues.apache.org/jira/browse/FLINK-21148 Project: Flink

[jira] [Created] (FLINK-21147) Resuming Savepoint (file, async, no parallelism change) fails with UnknownHostException

2021-01-25 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-21147: Summary: Resuming Savepoint (file, async, no parallelism change) fails with UnknownHostException Key: FLINK-21147 URL: https://issues.apache.org/jira/browse/FLINK-21147

[jira] [Created] (FLINK-21146) 【SQL】Flink SQL Client not support specify the queue to submit the job

2021-01-25 Thread zhisheng (Jira)
zhisheng created FLINK-21146: Summary: 【SQL】Flink SQL Client not support specify the queue to submit the job Key: FLINK-21146 URL: https://issues.apache.org/jira/browse/FLINK-21146 Project: Flink

[jira] [Created] (FLINK-21145) Flink Temporal Join Hive optimization

2021-01-25 Thread HideOnBush (Jira)
HideOnBush created FLINK-21145: -- Summary: Flink Temporal Join Hive optimization Key: FLINK-21145 URL: https://issues.apache.org/jira/browse/FLINK-21145 Project: Flink Issue Type: Wish

[jira] [Created] (FLINK-21144) KubernetesResourceManagerDriver#tryResetPodCreationCoolDown causes fatal error

2021-01-25 Thread Yang Wang (Jira)
Yang Wang created FLINK-21144: - Summary: KubernetesResourceManagerDriver#tryResetPodCreationCoolDown causes fatal error Key: FLINK-21144 URL: https://issues.apache.org/jira/browse/FLINK-21144 Project:

[jira] [Created] (FLINK-21143) 【runtime】flink job use the lib jars instead of the `yarn.provided.lib.dirs` config jars

2021-01-25 Thread zhisheng (Jira)
zhisheng created FLINK-21143: Summary: 【runtime】flink job use the lib jars instead of the `yarn.provided.lib.dirs` config jars Key: FLINK-21143 URL: https://issues.apache.org/jira/browse/FLINK-21143

[jira] [Created] (FLINK-21142) Flink guava Dependence problem

2021-01-25 Thread YUJIANBO (Jira)
YUJIANBO created FLINK-21142: Summary: Flink guava Dependence problem Key: FLINK-21142 URL: https://issues.apache.org/jira/browse/FLINK-21142 Project: Flink Issue Type: Bug Components:

Re: [DISCUSS] FLIP-161: Configuration through environment variables

2021-01-25 Thread Thomas Weise
It appears that the discussion in FLIP-161 is primarily focussed on setting individual configuration entries? On the other hand, $FLINK_PROPERTIES (and envsubst) were introduced to set a fragment or the complete flink-conf.yaml. It's used by at least 2 of the Flink k8s operators. In these cases

Re: Apache Pinot Sink

2021-01-25 Thread Yupeng Fu
Hi Mats and Jakob, +1 to what Till said about non-deterministic behavior. Also I suggest you look at only Pinot's offline segment creation from Flink. Pinot provides an inbuilt lambda architecture and has the real-time and offline segments per table (architecture diagram

Re: [DISCUSS] FLIP-156: Runtime Interfaces for Fine-Grained Resource Requirements

2021-01-25 Thread Xintong Song
Thanks for the summary, Yangze. The changes and follow-up issues LGTM. Let's wait for responses from the others before starting a vote. Thank you~ Xintong Song On Tue, Jan 26, 2021 at 11:08 AM Yangze Guo wrote: > Thanks everyone for the lively discussion. I'd like to try to > summarize the

[jira] [Created] (FLINK-21141) Timeout bug in data stream connect operation

2021-01-25 Thread HUANG Hong (Jira)
HUANG Hong created FLINK-21141: -- Summary: Timeout bug in data stream connect operation Key: FLINK-21141 URL: https://issues.apache.org/jira/browse/FLINK-21141 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-21140) Extract zip file dependencies before adding to PYTHONPATH

2021-01-25 Thread Dian Fu (Jira)
Dian Fu created FLINK-21140: --- Summary: Extract zip file dependencies before adding to PYTHONPATH Key: FLINK-21140 URL: https://issues.apache.org/jira/browse/FLINK-21140 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-156: Runtime Interfaces for Fine-Grained Resource Requirements

2021-01-25 Thread Yangze Guo
Thanks everyone for the lively discussion. I'd like to try to summarize the current convergence in the discussion. Please let me know if I got things wrong or missed something crucial here. Change of this FLIP: - Treat the SSG resource requirements as a hint instead of a restriction for the

Re: [DISCUSS] FLIP-159: Reactive Mode

2021-01-25 Thread Xintong Song
## configuration option I see your point that autoscaling mode might be more suitable for session clusters. It doesn't change that `execution-mode` could be a job-level configuration. But I'm good with keeping it cluster-level and marking it experimental at the moment, so we can change it later

[jira] [Created] (FLINK-21139) java.lang.AssertionError: Expected: a numeric value within <1.0E-6> of <40.0> but: <0.0> differed by <39.999999>

2021-01-25 Thread Ivan Serdyuk (Jira)
Ivan Serdyuk created FLINK-21139: Summary: java.lang.AssertionError: Expected: a numeric value within <1.0E-6> of <40.0> but: <0.0> differed by <39.99> Key: FLINK-21139 URL:

[jira] [Created] (FLINK-21138) KvStateServerHandler is not invoked with user code classloader

2021-01-25 Thread Maciej Prochniak (Jira)
Maciej Prochniak created FLINK-21138: Summary: KvStateServerHandler is not invoked with user code classloader Key: FLINK-21138 URL: https://issues.apache.org/jira/browse/FLINK-21138 Project:

Re: [DISCUSS] FLIP-161: Configuration through environment variables

2021-01-25 Thread Khachatryan Roman
I agree with Till about $FLINK_PROPERTIES being only supported by a Flink buildfile. Besides that, 1. having different ways of configuring different applications doesn't seem convenient to me. For example, Kafka and ZK configured via usual properties and Flink via concatenated one. 2. It could

[jira] [Created] (FLINK-21137) Maven plugin surefire failed due to test failure

2021-01-25 Thread Linyu Yao (Jira)
Linyu Yao created FLINK-21137: - Summary: Maven plugin surefire failed due to test failure Key: FLINK-21137 URL: https://issues.apache.org/jira/browse/FLINK-21137 Project: Flink Issue Type: New

Re: [DISCUSS] FLIP-161: Configuration through environment variables

2021-01-25 Thread Till Rohrmann
The problem I see with $FLINK_PROPERTIES is that this is only supported by Flink's Docker entrypoint.sh. In fact this variable was introduced as a temporary bridge to allow Docker users to change Flink's configuration dynamically. If we had had a proper way of configuring Flink processes via env

Re: Apache Pinot Sink

2021-01-25 Thread Till Rohrmann
Hi Mats and Jakob, In the general case, I don't think that elements from upstream Flink tasks always arrive at the same subtask of the sink. One problem is that user computations can be non-deterministic. Moreover, a rebalance operation can distribute the events of a task A among several

Re: [ANNOUNCE] New formatting rules are now in effect

2021-01-25 Thread Aljoscha Krettek
I've always been using the most recent IntelliJ plugin version and it's fine for all of my code so far and it was never a problem when I worked on the initial reformatting. For the rare case where more recent versions of the plugin would produce formatting that is incompatible with 1.7.5 our

Re: [DISCUSS] FLIP-161: Configuration through environment variables

2021-01-25 Thread Ingo Bürk
Hi Chesnay, thanks for your input! After some thought I think your proposal makes a lot of sense, i.e. if we have one single $FLINK_DYNAMIC_PROPERTIES environment variable, in a Kubernetes environment we could do something like ``` env: - name: S3_KEY valueFrom: # get from secret - name:

[jira] [Created] (FLINK-21136) Reactive Mode: Adjust timeout behavior in declarative scheduler

2021-01-25 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-21136: -- Summary: Reactive Mode: Adjust timeout behavior in declarative scheduler Key: FLINK-21136 URL: https://issues.apache.org/jira/browse/FLINK-21136 Project: Flink

[jira] [Created] (FLINK-21135) Reactive Mode: Change Declarative Scheduler to set infinite parallelism in JobGraph

2021-01-25 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-21135: -- Summary: Reactive Mode: Change Declarative Scheduler to set infinite parallelism in JobGraph Key: FLINK-21135 URL: https://issues.apache.org/jira/browse/FLINK-21135

Re: Apache Pinot Sink

2021-01-25 Thread Poerschke, Mats
Hi all, We want to give you a short update on the Pinot Sink since we started implementing a PoC. As described earlier, we aim to use batch-uploading of segments to Pinot in combination with caching elements in the Flink sink. Our current implementation works like this: Besides the pinot

[jira] [Created] (FLINK-21134) Introduce execution mode configuration key and check for supported ClusterEntrypoint type

2021-01-25 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-21134: -- Summary: Introduce execution mode configuration key and check for supported ClusterEntrypoint type Key: FLINK-21134 URL: https://issues.apache.org/jira/browse/FLINK-21134

[jira] [Created] (FLINK-21133) FLIP-27 Source does not work with synchronous savepoint

2021-01-25 Thread Kezhu Wang (Jira)
Kezhu Wang created FLINK-21133: -- Summary: FLIP-27 Source does not work with synchronous savepoint Key: FLINK-21133 URL: https://issues.apache.org/jira/browse/FLINK-21133 Project: Flink Issue

[jira] [Created] (FLINK-21132) BoundedOneInput.endInput is called when taking synchronous savepoint

2021-01-25 Thread Kezhu Wang (Jira)
Kezhu Wang created FLINK-21132: -- Summary: BoundedOneInput.endInput is called when taking synchronous savepoint Key: FLINK-21132 URL: https://issues.apache.org/jira/browse/FLINK-21132 Project: Flink

Re: [VOTE] Release 1.10.3, release candidate #1

2021-01-25 Thread Robert Metzger
+1 (binding) - checked the diff between 1.10.2 and 1.10.3: https://github.com/apache/flink/compare/release-1.10.2...release-1.10.3-rc1 - I didn't spot any dependency changes - the ci builds for the 1.10 branch seem to be pretty stable: https://travis-ci.com/github/apache/flink/builds - the jars

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

2021-01-25 Thread Kurt Young
Thanks Leonard for the detailed response and also the bad case about option 1, these all make sense to me. Also nice catch about conversion support of LocalZonedTimestampType, I think it actually makes sense to support java.sql.Timestamp as well as java.time.LocalDateTime. It also has a slight

[jira] [Created] (FLINK-21131) Show alignment timeout in checkpoint configuration (web UI)

2021-01-25 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-21131: - Summary: Show alignment timeout in checkpoint configuration (web UI) Key: FLINK-21131 URL: https://issues.apache.org/jira/browse/FLINK-21131 Project: Flink

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

2021-01-25 Thread Leonard Xu
Hi, All Thanks for your comments. I think all of the thread have agreed that: (1) The return values of CURRENT_TIME/CURRENT_TIMESTAMP/NOW()/PROCTIME() are wrong. (2) The LOCALTIME/LOCALTIMESTAMP and CURRENT_TIME/CURRENT_TIMESTAMP should be different whether from SQL standard’s perspective or

[jira] [Created] (FLINK-21130) Add bin/ directory to PATH in docker image

2021-01-25 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-21130: Summary: Add bin/ directory to PATH in docker image Key: FLINK-21130 URL: https://issues.apache.org/jira/browse/FLINK-21130 Project: Flink Issue

[jira] [Created] (FLINK-21129) Long metric names cause Graphite to not store them

2021-01-25 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-21129: - Summary: Long metric names cause Graphite to not store them Key: FLINK-21129 URL: https://issues.apache.org/jira/browse/FLINK-21129 Project: Flink Issue

[jira] [Created] (FLINK-21128) Rethink how Kubernetes interfaces with our docker image

2021-01-25 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-21128: Summary: Rethink how Kubernetes interfaces with our docker image Key: FLINK-21128 URL: https://issues.apache.org/jira/browse/FLINK-21128 Project: Flink

[jira] [Created] (FLINK-21127) Stores finished status for fully finished operators

2021-01-25 Thread Yun Gao (Jira)
Yun Gao created FLINK-21127: --- Summary: Stores finished status for fully finished operators Key: FLINK-21127 URL: https://issues.apache.org/jira/browse/FLINK-21127 Project: Flink Issue Type:

[jira] [Created] (FLINK-21126) Reconsider FLINK_PROPERTIES

2021-01-25 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-21126: Summary: Reconsider FLINK_PROPERTIES Key: FLINK-21126 URL: https://issues.apache.org/jira/browse/FLINK-21126 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-21125) Sum or Sum0 overflow quietly

2021-01-25 Thread Sebastian Liu (Jira)
Sebastian Liu created FLINK-21125: - Summary: Sum or Sum0 overflow quietly Key: FLINK-21125 URL: https://issues.apache.org/jira/browse/FLINK-21125 Project: Flink Issue Type: Bug

Re: [VOTE] Release 1.10.3, release candidate #1

2021-01-25 Thread Yang Wang
+1 (non-binding) - checked/verified signatures and checksums - build Flink from the attached source code - started a Yarn per-job/session cluster - check webUI is normal and no suspicious logs and exceptions Best, Yang Leonard Xu 于2021年1月25日周一 上午10:31写道: > +1 (non-binding) > > -

Re: about flink calcite implement

2021-01-25 Thread Kurt Young
Flink will convert all projections and filters to calc before converting to logical nodes. You can check out FlinkStreamRuleSets.LOGICAL_RULES Best, Kurt On Mon, Jan 25, 2021 at 11:03 AM laughing.sh...@qq.com < laughing.sh...@qq.com> wrote: > > In flink table planner module, I can not find the

Re: [ANNOUNCE] New formatting rules are now in effect

2021-01-25 Thread Till Rohrmann
Thanks for looking into this problem Matthias. The patched version also works for me. +1 for what Jark proposed. Cheers, Till On Sun, Jan 24, 2021 at 3:50 PM Matthias Pohl wrote: > Maybe, it's worth a try: I created issue #560 [1] for this. Let's see what > happens. > > [1]

[jira] [Created] (FLINK-21124) Checkpoint abort RPC can be ignored in some cases

2021-01-25 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-21124: - Summary: Checkpoint abort RPC can be ignored in some cases Key: FLINK-21124 URL: https://issues.apache.org/jira/browse/FLINK-21124 Project: Flink

[jira] [Created] (FLINK-21123) Upgrade Beanutils 1.9.x to 1.9.4

2021-01-25 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-21123: Summary: Upgrade Beanutils 1.9.x to 1.9.4 Key: FLINK-21123 URL: https://issues.apache.org/jira/browse/FLINK-21123 Project: Flink Issue Type:

[jira] [Created] (FLINK-21122) Update docs/ops/monitoring/checkpoint_monitoring.zh.md

2021-01-25 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-21122: - Summary: Update docs/ops/monitoring/checkpoint_monitoring.zh.md Key: FLINK-21122 URL: https://issues.apache.org/jira/browse/FLINK-21122 Project: Flink

[jira] [Created] (FLINK-21121) TaggedOperatorSubtaskState is missing when creating a new savepoint using state processor api

2021-01-25 Thread binguo (Jira)
binguo created FLINK-21121: -- Summary: TaggedOperatorSubtaskState is missing when creating a new savepoint using state processor api Key: FLINK-21121 URL: https://issues.apache.org/jira/browse/FLINK-21121

[jira] [Created] (FLINK-21120) Add support for ignore abnormal watermark

2021-01-25 Thread zlzhang0122 (Jira)
zlzhang0122 created FLINK-21120: --- Summary: Add support for ignore abnormal watermark Key: FLINK-21120 URL: https://issues.apache.org/jira/browse/FLINK-21120 Project: Flink Issue Type:

[jira] [Created] (FLINK-21119) Building Flink Azure FS Hadoop hangs during python CI

2021-01-25 Thread Yun Tang (Jira)
Yun Tang created FLINK-21119: Summary: Building Flink Azure FS Hadoop hangs during python CI Key: FLINK-21119 URL: https://issues.apache.org/jira/browse/FLINK-21119 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-159: Reactive Mode

2021-01-25 Thread Robert Metzger
Thank you very much for the comments so far. @Steven: No fixed parallelism for any of the operators > > Regarding this limitation, can the scheduler only adjust the default > parallelism? if some operators set parallelism explicitly (like always 1), > just leave them unchanged. We will respect

[jira] [Created] (FLINK-21118) TPC-DS end-to-end test (Blink planner) fails with "no more allocated slots"

2021-01-25 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-21118: Summary: TPC-DS end-to-end test (Blink planner) fails with "no more allocated slots" Key: FLINK-21118 URL: https://issues.apache.org/jira/browse/FLINK-21118