[GitHub] [flink] wenlong88 commented on a change in pull request #15271: [FLINK-21813][table-planner-blink] Support json ser/de for StreamExecOverAggregate

[GitBox]

wenlong88 commented on a change in pull request #15271:
URL: https://github.com/apache/flink/pull/15271#discussion_r597445044



##
File path: 
flink-table/flink-table-planner-blink/src/main/java/org/apache/flink/table/planner/plan/nodes/exec/serde/RexLiteralJsonDeserializer.java
##
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.plan.nodes.exec.serde;
+
+import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParser;
+import 
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonProcessingException;
+import 
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.DeserializationContext;
+import 
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.std.StdDeserializer;
+
+import org.apache.calcite.rex.RexLiteral;
+import org.apache.calcite.rex.RexNode;
+
+import java.io.IOException;
+
+/**
+ * JSON deserializer for {@link RexLiteral}. refer to {@link 
RexNodeJsonSerializer} for serializer.
+ */
+public class RexLiteralJsonDeserializer extends StdDeserializer {

Review comment:
   I am afraid not, this class is a wrapper of RexNodeJsonDeserializer, 
which is used to deserialize List




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] xintongsong commented on a change in pull request #15273: [FLINK-21800][core] Guard MemorySegment against concurrent frees.

[GitBox]

xintongsong commented on a change in pull request #15273:
URL: https://github.com/apache/flink/pull/15273#discussion_r597442899



##
File path: 
flink-core/src/test/java/org/apache/flink/core/memory/OffHeapUnsafeMemorySegmentTest.java
##
@@ -58,4 +61,31 @@ public void testCallCleanerOnFree() {
 .free();
 assertTrue(cleanerFuture.isDone());
 }
+
+@Test
+public void testCallCleanerOnceOnConcurrentFree() throws 
InterruptedException {
+final AtomicInteger counter = new AtomicInteger(0);

Review comment:
   By "throwing path", do you mean the path that `checkMultipleFree` is 
activated? I think that should be covered by the current test case as long as 
the env is set, which I've verified manually.
   
   Not sure if we need two different test cases with `checkMultipleFree` set 
differently. The two cases would have to be executed in different processes, 
because `checkMultipleFree` is determined only when the class `MemorySegment` 
is first loaded.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] godfreyhe commented on a change in pull request #15276: [FLINK-21837][flink-table-planner-blink] support StreamExecIntervalJoin serialization/deserialization

[GitBox]

godfreyhe commented on a change in pull request #15276:
URL: https://github.com/apache/flink/pull/15276#discussion_r597437534



##
File path: 
flink-table/flink-table-planner-blink/src/test/java/org/apache/flink/table/planner/plan/nodes/exec/serde/IntervalJoinSpecJsonSerdeTest.java
##
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.plan.nodes.exec.serde;
+
+import org.apache.flink.table.planner.plan.nodes.exec.spec.IntervalJoinSpec;
+import org.apache.flink.table.planner.plan.nodes.exec.spec.JoinSpec;
+import org.apache.flink.table.runtime.operators.join.FlinkJoinType;
+
+import 
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper;
+import 
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.module.SimpleModule;
+
+import org.apache.calcite.rex.RexNode;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.io.IOException;
+
+import static org.junit.Assert.assertEquals;
+
+/** Tests for {@link JoinSpec} serialization and deserialization. */

Review comment:
   JoinSpec -> IntervalJoinSpec




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] wuchong merged pull request #15275: [hotfix][docs]add 'IF NOT EXISTS' to create table statement

[GitBox]

wuchong merged pull request #15275:
URL: https://github.com/apache/flink/pull/15275


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] godfreyhe commented on a change in pull request #15271: [FLINK-21813][table-planner-blink] Support json ser/de for StreamExecOverAggregate

[GitBox]

godfreyhe commented on a change in pull request #15271:
URL: https://github.com/apache/flink/pull/15271#discussion_r597419527



##
File path: 
flink-table/flink-table-planner-blink/src/main/java/org/apache/flink/table/planner/plan/nodes/exec/serde/RexLiteralJsonDeserializer.java
##
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.plan.nodes.exec.serde;
+
+import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParser;
+import 
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonProcessingException;
+import 
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.DeserializationContext;
+import 
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.std.StdDeserializer;
+
+import org.apache.calcite.rex.RexLiteral;
+import org.apache.calcite.rex.RexNode;
+
+import java.io.IOException;
+
+/**
+ * JSON deserializer for {@link RexLiteral}. refer to {@link 
RexNodeJsonSerializer} for serializer.
+ */
+public class RexLiteralJsonDeserializer extends StdDeserializer {

Review comment:
   can we use RexNodeJsonDeserializer to replace this?

##
File path: 
flink-table/flink-table-planner-blink/src/main/java/org/apache/flink/table/planner/plan/nodes/exec/serde/RexWindowBoundJsonDeserializer.java
##
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.plan.nodes.exec.serde;
+
+import org.apache.flink.table.api.TableException;
+
+import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParser;
+import 
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.DeserializationContext;
+import 
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonNode;
+import 
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.std.StdDeserializer;
+
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.rex.RexWindowBound;
+import org.apache.calcite.rex.RexWindowBounds;
+
+import java.io.IOException;
+
+import static 
org.apache.flink.table.planner.plan.nodes.exec.serde.RexWindowBoundJsonSerializer.FIELD_NAME_IS_FOLLOWING;
+import static 
org.apache.flink.table.planner.plan.nodes.exec.serde.RexWindowBoundJsonSerializer.FIELD_NAME_IS_PRECEDING;
+import static 
org.apache.flink.table.planner.plan.nodes.exec.serde.RexWindowBoundJsonSerializer.FIELD_NAME_KIND;
+import static 
org.apache.flink.table.planner.plan.nodes.exec.serde.RexWindowBoundJsonSerializer.FIELD_NAME_OFFSET;
+import static 
org.apache.flink.table.planner.plan.nodes.exec.serde.RexWindowBoundJsonSerializer.KIND_BOUNDED_WINDOW;
+import static 
org.apache.flink.table.planner.plan.nodes.exec.serde.RexWindowBoundJsonSerializer.KIND_CURRENT_ROW;
+import static 
org.apache.flink.table.planner.plan.nodes.exec.serde.RexWindowBoundJsonSerializer.KIND_UNBOUNDED_FOLLOWING;
+import static 
org.apache.flink.table.planner.plan.nodes.exec.serde.RexWindowBoundJsonSerializer.KIND_UNBOUNDED_PRECEDING;
+
+/**
+ * JSON deserializer for {@link RexWindowBound}. refer to {@link 
RexWindowBoundJsonSerializer} for
+ * serializer.
+ */
+public class RexWindowBoundJsonDeserializer extends 
StdDeserializer {

Review comment:
   nit: add serialVersionUID

##
File path: 
flink-table/flink-table-planner-blink/src/main/java/org/apache/flink/table/planner/plan/nodes/exec/spec/OverSpec.java
##
@@ -83,24 +108,

[GitHub] [flink] fsk119 commented on a change in pull request #15255: [FLINK-21466][table] Add SQL Client default mode value embedded

[GitBox]

fsk119 commented on a change in pull request #15255:
URL: https://github.com/apache/flink/pull/15255#discussion_r597428918



##
File path: 
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/SqlClient.java
##
@@ -125,15 +125,21 @@ private void openCli(String sessionId, Executor executor) 
{
 // 

 
 public static void main(String[] args) {
-if (args.length < 1) {
-CliOptionsParser.printHelpClient();
-return;
+final String model;

Review comment:
   rename `model` to `mode`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15273: [FLINK-21800][core] Guard MemorySegment against concurrent frees.

[GitBox]

flinkbot edited a comment on pull request #15273:
URL: https://github.com/apache/flink/pull/15273#issuecomment-802508808


   
   ## CI report:
   
   * 99f0b12ad7cd87ca9dfd5d9d3e97030eb41135f9 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15001)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15272: [FLINK-21816][table-planner-blink] Suport json ser/de for StreamExecMatch

[GitBox]

flinkbot edited a comment on pull request #15272:
URL: https://github.com/apache/flink/pull/15272#issuecomment-802067367


   
   ## CI report:
   
   * e96eaf020bcec88f7e5364264f15ac4bf6ed34b4 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14987)
 
   * 2542929c02c33edd9817274c8c329949ee27a505 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15007)
 
   * 826d695c88ff0250a44d950a3494ade0bfd63bbb Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15015)
 
   * 2be101d2ed755846ad22218f45624a1da53f4d1c UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15270: [FLINK-21843][table-planner-blink] Support json ser/de for StreamExecGroupWindowAggregate

[GitBox]

flinkbot edited a comment on pull request #15270:
URL: https://github.com/apache/flink/pull/15270#issuecomment-802066772


   
   ## CI report:
   
   * 6204d0655db1e1ebbaf3e38db42093fd07351fb8 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14999)
 
   * 32b0acb81732ffe1acee685854c0ecb915d7 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15005)
 
   * 5a8dc8cb90d08b1677d54283a3c8713fdb4b4d7c Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15014)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15236: [FLINK-21818][table] Refactor SlicingWindowAggOperatorBuilder to accept serializer instead of LogicalType

[GitBox]

flinkbot edited a comment on pull request #15236:
URL: https://github.com/apache/flink/pull/15236#issuecomment-800154343


   
   ## CI report:
   
   * a4b23175ae924c5ea608ecdd5d3c6f3751d1b252 UNKNOWN
   * d2404ab56c12185a521eb2c512df610c01680cf5 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14856)
 
   * 69ba58df5004bfc9614c41de349248d661f80a1f UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15149: [FLINK-21294][python] Support state access API for the map/flat_map operation of Python ConnectedStreams

[GitBox]

flinkbot edited a comment on pull request #15149:
URL: https://github.com/apache/flink/pull/15149#issuecomment-796635889


   
   ## CI report:
   
   * 8b47e45a3f59187897cace4c120b59a9dc5b7f16 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14870)
 
   * bd03d15579abca299380c1aa8c8e16b5f6ddc200 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15013)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] zhuzhurk commented on a change in pull request #15199: [FLINK-20740][network] Introduce a separated buffer pool and a separated thread pool for sort-merge blocking shuffle

[GitBox]

zhuzhurk commented on a change in pull request #15199:
URL: https://github.com/apache/flink/pull/15199#discussion_r597382774



##
File path: 
flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
##
@@ -513,6 +513,30 @@
 + " the configured min/max size, the 
min/max size will be used. The exact size of Network Memory can be"
 + " explicitly specified by setting the 
min/max size to the same value.");
 
+/**
+ * Memory used by blocking shuffle for shuffle data read (currently only 
used by sort-merge
+ * shuffle). The minimum valid value can be configured is 32M.

Review comment:
   Not sure why we must require it to be at least 32m? Even if for 
FRAMEWORK_OFF_HEAP_MEMORY it is not required to be larger than 32m.
   Or maybe the smallest allowed value should be 
BatchShuffleReadBufferPool.NUM_BYTES_PER_REQUEST which is 8m?

##
File path: 
flink-runtime/src/main/java/org/apache/flink/runtime/io/disk/BatchShuffleReadIOExecutor.java
##
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.disk;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.runtime.util.ExecutorThreadFactory;
+import org.apache.flink.runtime.util.Hardware;
+
+import javax.annotation.Nonnull;
+import javax.annotation.concurrent.GuardedBy;
+
+import java.util.concurrent.Executor;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ThreadFactory;
+
+import static org.apache.flink.util.Preconditions.checkArgument;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * A fixed-size {@link Executor} pool used by batch shuffle for shuffle data 
read (currently only
+ * used by sort-merge blocking shuffle.
+ */
+@Internal
+public class BatchShuffleReadIOExecutor implements Executor {
+
+/** Minimum valid number of executor threads. */
+public static final int MIN_NUM_THREADS = 4;

Review comment:
   is it necessary to add a lower-bound limit to the threads?
   IIUC, this further results in a 32m requirement of the shuffle read memory 
size. maybe 32m as the default values of NETWORK_BATCH_SHUFFLE_READ_MEMORY is 
enough for out of box experience?

##
File path: 
flink-runtime/src/main/java/org/apache/flink/runtime/io/disk/BatchShuffleReadBufferPool.java
##
@@ -0,0 +1,353 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.disk;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.configuration.TaskManagerOptions;
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.core.memory.MemorySegmentFactory;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.annotation.concurrent.GuardedBy;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Queue;
+import java.util.concurrent.TimeoutException;
+
+import static org.apache.flink.util.Preconditions.checkArgument;
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * A fixed-size {@link MemorySegment} pool used by batch shuffle for shuffle 
data re

[GitHub] [flink] flinkbot edited a comment on pull request #15275: [hotfix][docs]add 'IF NOT EXISTS' to create table statement

[GitBox]

flinkbot edited a comment on pull request #15275:
URL: https://github.com/apache/flink/pull/15275#issuecomment-802508892


   
   ## CI report:
   
   * 4d815720a65c9884045069e85b196be6dc112a11 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15002)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15272: [FLINK-21816][table-planner-blink] Suport json ser/de for StreamExecMatch

[GitBox]

flinkbot edited a comment on pull request #15272:
URL: https://github.com/apache/flink/pull/15272#issuecomment-802067367


   
   ## CI report:
   
   * e96eaf020bcec88f7e5364264f15ac4bf6ed34b4 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14987)
 
   * 2542929c02c33edd9817274c8c329949ee27a505 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15007)
 
   * 826d695c88ff0250a44d950a3494ade0bfd63bbb UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15270: [FLINK-21843][table-planner-blink] Support json ser/de for StreamExecGroupWindowAggregate

[GitBox]

flinkbot edited a comment on pull request #15270:
URL: https://github.com/apache/flink/pull/15270#issuecomment-802066772


   
   ## CI report:
   
   * bff3961bf9c86186d08014044f28843a4dd72397 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14985)
 
   * 6204d0655db1e1ebbaf3e38db42093fd07351fb8 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14999)
 
   * 32b0acb81732ffe1acee685854c0ecb915d7 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15005)
 
   * 5a8dc8cb90d08b1677d54283a3c8713fdb4b4d7c UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15185: [FLINK-21768][clients]Optimize system.exit() logic of CliFrontend

[GitBox]

flinkbot edited a comment on pull request #15185:
URL: https://github.com/apache/flink/pull/15185#issuecomment-79889


   
   ## CI report:
   
   * bbfe60eb6db082a292800187af1734c687136b32 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14997)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15149: [FLINK-21294][python] Support state access API for the map/flat_map operation of Python ConnectedStreams

[GitBox]

flinkbot edited a comment on pull request #15149:
URL: https://github.com/apache/flink/pull/15149#issuecomment-796635889


   
   ## CI report:
   
   * 8b47e45a3f59187897cace4c120b59a9dc5b7f16 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14870)
 
   * bd03d15579abca299380c1aa8c8e16b5f6ddc200 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] JingsongLi commented on a change in pull request #15236: [FLINK-21818][table] Refactor SlicingWindowAggOperatorBuilder to accept serializer instead of LogicalType

[GitBox]

JingsongLi commented on a change in pull request #15236:
URL: https://github.com/apache/flink/pull/15236#discussion_r597424374



##
File path: 
flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/typeutils/WindowKeySerializer.java
##
@@ -109,10 +106,9 @@ public void copy(DataInputView source, DataOutputView 
target) throws IOException
 @Override
 public int serializeToPages(WindowKey record, AbstractPagedOutputView 
target)
 throws IOException {
-int windowSkip = checkSkipWriteForWindowPart(target);
 target.writeLong(record.getWindow());
-int keySkip = keySerializer.serializeToPages(record.getKey(), target);
-return windowSkip + keySkip;
+keySerializer.serializeToPages(record.getKey(), target);
+return 0;

Review comment:
   Added comments.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] JingsongLi commented on a change in pull request #15236: [FLINK-21818][table] Refactor SlicingWindowAggOperatorBuilder to accept serializer instead of LogicalType

[GitBox]

JingsongLi commented on a change in pull request #15236:
URL: https://github.com/apache/flink/pull/15236#discussion_r597420746



##
File path: 
flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/typeutils/WindowKeySerializer.java
##
@@ -109,10 +106,9 @@ public void copy(DataInputView source, DataOutputView 
target) throws IOException
 @Override
 public int serializeToPages(WindowKey record, AbstractPagedOutputView 
target)
 throws IOException {
-int windowSkip = checkSkipWriteForWindowPart(target);
 target.writeLong(record.getWindow());
-int keySkip = keySerializer.serializeToPages(record.getKey(), target);
-return windowSkip + keySkip;
+keySerializer.serializeToPages(record.getKey(), target);
+return 0;

Review comment:
   Actually, the return value is just for save 
`checkSkipReadForFixLengthPart` in the `mapFromPages`, the cost is very small.

##
File path: 
flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/typeutils/WindowKeySerializer.java
##
@@ -109,10 +106,9 @@ public void copy(DataInputView source, DataOutputView 
target) throws IOException
 @Override
 public int serializeToPages(WindowKey record, AbstractPagedOutputView 
target)
 throws IOException {
-int windowSkip = checkSkipWriteForWindowPart(target);
 target.writeLong(record.getWindow());
-int keySkip = keySerializer.serializeToPages(record.getKey(), target);
-return windowSkip + keySkip;
+keySerializer.serializeToPages(record.getKey(), target);
+return 0;

Review comment:
   Actually, the return value is just for saving 
`checkSkipReadForFixLengthPart` in the `mapFromPages`, the cost is very small.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] jiamo commented on a change in pull request #15157: [FLINK-21661][kinesis] Fix fetch interval for polling consumer

[GitBox]

jiamo commented on a change in pull request #15157:
URL: https://github.com/apache/flink/pull/15157#discussion_r597419965



##
File path: 
flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/internals/publisher/polling/PollingRecordPublisher.java
##
@@ -168,8 +177,8 @@ private GetRecordsResult getRecords(String shardItr, int 
maxNumberOfRecords)
 
 // sleep for the fetch interval before the next getRecords 
attempt with the
 // refreshed iterator
-if (expiredIteratorBackoffMillis != 0) {
-Thread.sleep(expiredIteratorBackoffMillis);
+if (fetchIntervalMillis != 0) {
+Thread.sleep(fetchIntervalMillis);

Review comment:
   I got it. Thanks




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] JingsongLi commented on a change in pull request #15236: [FLINK-21818][table] Refactor SlicingWindowAggOperatorBuilder to accept serializer instead of LogicalType

[GitBox]

JingsongLi commented on a change in pull request #15236:
URL: https://github.com/apache/flink/pull/15236#discussion_r597419792



##
File path: 
flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/typeutils/WindowKeySerializer.java
##
@@ -109,10 +106,9 @@ public void copy(DataInputView source, DataOutputView 
target) throws IOException
 @Override
 public int serializeToPages(WindowKey record, AbstractPagedOutputView 
target)
 throws IOException {
-int windowSkip = checkSkipWriteForWindowPart(target);
 target.writeLong(record.getWindow());
-int keySkip = keySerializer.serializeToPages(record.getKey(), target);
-return windowSkip + keySkip;
+keySerializer.serializeToPages(record.getKey(), target);
+return 0;

Review comment:
   No, we cannot, The return value is to help better relocate the start 
offset where the data is located, and the offset we need here is the offset 
that we started to write. Consider this case:
   |---First segment|Second Segment| 
   |Left 10 bytes|-|
   In fact, we will write 8 bytes in the first segment and skip the next two 
bytes. At this time, its offset should also be 0




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] wenlong88 commented on a change in pull request #15272: [FLINK-21816][table-planner-blink] Suport json ser/de for StreamExecMatch

[GitBox]

wenlong88 commented on a change in pull request #15272:
URL: https://github.com/apache/flink/pull/15272#discussion_r597419515



##
File path: 
flink-table/flink-table-planner-blink/src/main/java/org/apache/flink/table/planner/plan/nodes/exec/serde/RexNodeJsonDeserializer.java
##
@@ -334,6 +351,21 @@ private SqlOperator toOperator(JsonNode jsonNode, 
SerdeContext ctx) throws IOExc
 }
 }
 
+// try to find operator from std operator table.
+SqlStdOperatorTable.instance()

Review comment:
   no, that is why I add this code, FlinkSqlOperatorTable dosen't not cover 
all of the operator, and I found that we use some operator like PATTERN_CONCAT 
directly.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (FLINK-21865) Add a Docker Compose greeter example to StateFun playgrounds

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-21865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-21865:
---
Labels: pull-request-available  (was: )

> Add a Docker Compose greeter example to StateFun playgrounds
> 
>
> Key: FLINK-21865
> URL: https://issues.apache.org/jira/browse/FLINK-21865
> Project: Flink
>  Issue Type: Task
>  Components: Stateful Functions
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Major
>  Labels: pull-request-available
> Fix For: statefun-3.0.0
>
>
> This example is intended as a follow-up after completion of the Java SDK 
> Showcase Tutorial (FLINK-21862).
> If users are already familiar with the Java SDK fundamentals and would like 
> to get a better understanding of how a realistic StateFun application looks 
> like, then this would be the example they start with. Otherwise, we would 
> recommend users to take a look at the Showcase tutorial first.
> This example works with Docker Compose, and runs a few services that build up 
> an end-to-end StateFun application:
> - Functions service that runs functions and expose them through an HTTP 
> endpoint.
> - StateFun runtime processes (a manager plus workers) that will handle 
> ingress, egress, and inter-function messages as well as function state 
> storage in a consistent and fault-tolerant manner.
> - Apache Kafka broker for the application ingress and egress.
> To motivate this example, we'll implement a simple user greeter application, 
> which has two functions - a {{UserFn}} that expects {{UserLogin}} JSON events 
> from an ingress and keeps in state storage information about users, and a 
> {{GreetingsFn}} that accepts user information to generate personalized 
> greeting messages that are sent to users via an egress.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink-statefun-playground] tzulitai opened a new pull request #4: [FLINK-21865] Add a Docker Compose greeter example

[GitBox]

tzulitai opened a new pull request #4:
URL: https://github.com/apache/flink-statefun-playground/pull/4


   This example is intended as a follow-up after completion of the Java SDK 
Showcase Tutorial.
   
   If users are already familiar with the Java SDK fundamentals and would like 
to get a better understanding of how a realistic StateFun application looks 
like, then this would be the example they ideally start with.
   Otherwise, for completely new users, we would recommend users to take a look 
at the Showcase tutorial first.
   
   This example works with Docker Compose (the only environment requirement to 
get this example running), and runs a few services that build up an end-to-end 
StateFun application.
   
   Please see the README.md for a full description of the example, including 
explanations of the directory structure, how to run the example, and how users 
would interact with the example.
   
   ## Notable remarks
   
   See the functions service Dockerfile: 
https://github.com/apache/flink-statefun-playground/compare/dev...tzulitai:FLINK-21865?expand=1#diff-2f2d3a56094ea5cf1343d6b70c642e908224e8ba05f6ba7f5be8b8d9efbc71ecR20.
   
   These lines and the Java SDK jar should be removed before we release this 
example to `main` branch.
   This is only required right now because we don't have the latest SDK 
artifacts published to Maven central.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (FLINK-21865) Add a Docker Compose greeter example to StateFun playgrounds

[Tzu-Li (Gordon) Tai (Jira)]

Tzu-Li (Gordon) Tai created FLINK-21865:
---

 Summary: Add a Docker Compose greeter example to StateFun 
playgrounds
 Key: FLINK-21865
 URL: https://issues.apache.org/jira/browse/FLINK-21865
 Project: Flink
  Issue Type: Task
  Components: Stateful Functions
Reporter: Tzu-Li (Gordon) Tai
Assignee: Tzu-Li (Gordon) Tai
 Fix For: statefun-3.0.0


This example is intended as a follow-up after completion of the Java SDK 
Showcase Tutorial (FLINK-21862).

If users are already familiar with the Java SDK fundamentals and would like to 
get a better understanding of how a realistic StateFun application looks like, 
then this would be the example they start with. Otherwise, we would recommend 
users to take a look at the Showcase tutorial first.

This example works with Docker Compose, and runs a few services that build up 
an end-to-end StateFun application:
- Functions service that runs functions and expose them through an HTTP 
endpoint.
- StateFun runtime processes (a manager plus workers) that will handle ingress, 
egress, and inter-function messages as well as function state storage in a 
consistent and fault-tolerant manner.
- Apache Kafka broker for the application ingress and egress.

To motivate this example, we'll implement a simple user greeter application, 
which has two functions - a {{UserFn}} that expects {{UserLogin}} JSON events 
from an ingress and keeps in state storage information about users, and a 
{{GreetingsFn}} that accepts user information to generate personalized greeting 
messages that are sent to users via an egress.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-21747) Encounter an exception that contains "max key length exceeded ..." when reporting metrics to influxdb

[tim yu (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-21747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304643#comment-17304643
 ] 

tim yu commented on FLINK-21747:


Hello [~jark] and [~chesnay]

Through a review of metrics system code, I find that only the operator name is 
truncated if it exceeds 80 characters length.

I encounter this issue, because this simple rule is not applied to job and task 
name.

Should we truncate it if the name of job or task is too long ?

> Encounter an exception that contains "max key length exceeded ..."  when 
> reporting metrics to influxdb
> --
>
> Key: FLINK-21747
> URL: https://issues.apache.org/jira/browse/FLINK-21747
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Metrics
>Affects Versions: 1.10.0
>Reporter: tim yu
>Priority: Major
>
> I run a streaming job with insert statement whose size is too large, it 
> reports metrics to influxdb. I find many influxdb exceptions that contains 
> "max key length exceeded ..." in the log file of the job manager . The job 
> could not write any metrics to the influxdb, because "task_name" is too long.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] flinkbot edited a comment on pull request #15276: [FLINK-21837][flink-table-planner-blink] support StreamExecIntervalJoin serialization/deserialization

[GitBox]

flinkbot edited a comment on pull request #15276:
URL: https://github.com/apache/flink/pull/15276#issuecomment-802542650


   
   ## CI report:
   
   * 0d5e735e81bfe061bbec689dda74e152553ee19d UNKNOWN
   * 865591e0c6465384ca646fd723760c77fa819f1f Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15011)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15256: [FLINK-21851][table-runtime] Refactor BinaryRowDataKeySelector in testing

[GitBox]

flinkbot edited a comment on pull request #15256:
URL: https://github.com/apache/flink/pull/15256#issuecomment-801601761


   
   ## CI report:
   
   * 31a064e494baca38cf8185fe7d84c2b3579cf28f Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14934)
 
   * 5197def439c2ecbf323f7ca2ba6631e63c492f5f Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15012)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15276: [FLINK-21837][flink-table-planner-blink] support StreamExecIntervalJoin serialization/deserialization

[GitBox]

flinkbot edited a comment on pull request #15276:
URL: https://github.com/apache/flink/pull/15276#issuecomment-802542650


   
   ## CI report:
   
   * 0d5e735e81bfe061bbec689dda74e152553ee19d UNKNOWN
   * 865591e0c6465384ca646fd723760c77fa819f1f UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15256: [FLINK-21851][table-runtime] Refactor BinaryRowDataKeySelector in testing

[GitBox]

flinkbot edited a comment on pull request #15256:
URL: https://github.com/apache/flink/pull/15256#issuecomment-801601761


   
   ## CI report:
   
   * 31a064e494baca38cf8185fe7d84c2b3579cf28f Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14934)
 
   * 5197def439c2ecbf323f7ca2ba6631e63c492f5f UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] wuchong commented on a change in pull request #15236: [FLINK-21818][table] Refactor SlicingWindowAggOperatorBuilder to accept serializer instead of LogicalType

[GitBox]

wuchong commented on a change in pull request #15236:
URL: https://github.com/apache/flink/pull/15236#discussion_r597401185



##
File path: 
flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/typeutils/WindowKeySerializer.java
##
@@ -109,10 +106,9 @@ public void copy(DataInputView source, DataOutputView 
target) throws IOException
 @Override
 public int serializeToPages(WindowKey record, AbstractPagedOutputView 
target)
 throws IOException {
-int windowSkip = checkSkipWriteForWindowPart(target);
 target.writeLong(record.getWindow());
-int keySkip = keySerializer.serializeToPages(record.getKey(), target);
-return windowSkip + keySkip;
+keySerializer.serializeToPages(record.getKey(), target);
+return 0;

Review comment:
   Return the num of bytes skipped by `keySerializer`?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (FLINK-21864) Support StreamExecTemporalJoin json serialization/deserialization

[Terry Wang (Jira)]

Terry Wang created FLINK-21864:
--

 Summary: Support StreamExecTemporalJoin json 
serialization/deserialization
 Key: FLINK-21864
 URL: https://issues.apache.org/jira/browse/FLINK-21864
 Project: Flink
  Issue Type: Sub-task
  Components: Table SQL / Planner
Reporter: Terry Wang
 Fix For: 1.13.0


Support StreamExecTemporalJoin json serialization/deserialization



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] godfreyhe commented on a change in pull request #15272: [FLINK-21816][table-planner-blink] Suport json ser/de for StreamExecMatch

[GitBox]

godfreyhe commented on a change in pull request #15272:
URL: https://github.com/apache/flink/pull/15272#discussion_r597400191



##
File path: 
flink-table/flink-table-planner-blink/src/test/java/org/apache/flink/table/planner/runtime/stream/jsonplan/MatchRecognizeJsonPlanITCase.java
##
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.runtime.stream.jsonplan;
+
+import org.apache.flink.table.planner.factories.TestValuesTableFactory;
+import org.apache.flink.table.planner.utils.JsonPlanTestBase;
+import org.apache.flink.types.Row;
+
+import org.junit.Test;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.List;
+import java.util.concurrent.ExecutionException;
+
+/** Test json deserialization for match recognize. */
+public class MatchRecognizeJsonPlanITCase extends JsonPlanTestBase {
+@Test
+public void testSimpleMatch() throws ExecutionException, 
InterruptedException, IOException {

Review comment:
   nit: `throws Exception` is enough

##
File path: 
flink-table/flink-table-planner-blink/src/main/java/org/apache/flink/table/planner/plan/nodes/exec/serde/RexNodeJsonSerializer.java
##
@@ -113,6 +117,9 @@ public void serialize(
 case CORREL_VARIABLE:
 serialize((RexCorrelVariable) rexNode, jsonGenerator);
 break;
+case PATTERN_INPUT_REF:

Review comment:
   please add some tests in `RexNodeSerdeTest`

##
File path: 
flink-table/flink-table-planner-blink/src/main/java/org/apache/flink/table/planner/plan/nodes/exec/serde/RexNodeJsonDeserializer.java
##
@@ -334,6 +351,21 @@ private SqlOperator toOperator(JsonNode jsonNode, 
SerdeContext ctx) throws IOExc
 }
 }
 
+// try to find operator from std operator table.
+SqlStdOperatorTable.instance()

Review comment:
   does `ctx.getOperatorTable()` contain all operators used in Flink ?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #15276: [FLINK-21837][flink-table-planner-blink] support StreamExecIntervalJoin serialization/deserialization

[GitBox]

flinkbot commented on pull request #15276:
URL: https://github.com/apache/flink/pull/15276#issuecomment-802542650


   
   ## CI report:
   
   * 0d5e735e81bfe061bbec689dda74e152553ee19d UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (FLINK-21701) Support "RESET key" statement in the SQL Client

[Shengkai Fang (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-21701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304628#comment-17304628
 ] 

Shengkai Fang commented on FLINK-21701:
---

[~xiangtao] Please make a try.

>From my side, my suggestions as follow.
 - What value to reset?
 -- If the key is defined in the YAML file, e.g. sql-client.default.yaml and 
flink-conf.yaml, reset the key the value in the YAML file. We have already 
stores this key value in the {{DefaultContext}};
 -- If the key is not defined in the YAML , please remove it from the 
configuration;

 - Compatibility.
 -- We have already deprecated the YAML file and introduced a util named 
{{YamlConfigUtils}}. Please use this if {{setSessionProperty}} doesn't 
satfisify your requirement.

> Support "RESET key" statement in the SQL Client
> ---
>
> Key: FLINK-21701
> URL: https://issues.apache.org/jira/browse/FLINK-21701
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table SQL / Client
>Affects Versions: 1.13.0
>Reporter: Shengkai Fang
>Priority: Major
>  Labels: starter
>
> Allow to reset the specified key by input 
> {code:java}
> // code placeholder
> RESET table.planner.type;{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] flinkbot commented on pull request #15276: [FLINK-21837][flink-table-planner-blink] support StreamExecIntervalJoin serialization/deserialization

[GitBox]

flinkbot commented on pull request #15276:
URL: https://github.com/apache/flink/pull/15276#issuecomment-802536047


   Thanks a lot for your contribution to the Apache Flink project. I'm the 
@flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress 
of the review.
   
   
   ## Automated Checks
   Last check on commit 0d5e735e81bfe061bbec689dda74e152553ee19d (Fri Mar 19 
04:05:30 UTC 2021)
   
   **Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up 
to date!
   
   
   Mention the bot in a comment to re-run the automated checks.
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review 
Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full 
explanation of the review process.
The Bot is tracking the review progress through labels. Labels are applied 
according to the order of the review items. For consensus, approval by a Flink 
committer of PMC member is required Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot approve description` to approve one or more aspects (aspects: 
`description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until 
`architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's 
attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (FLINK-21837) Support StreamExecIntervalJoin json ser/de

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-21837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-21837:
---
Labels: pull-request-available  (was: )

> Support StreamExecIntervalJoin json ser/de
> --
>
> Key: FLINK-21837
> URL: https://issues.apache.org/jira/browse/FLINK-21837
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table SQL / Planner
>Reporter: Terry Wang
>Assignee: Terry Wang
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.13.0
>
>
> Support StreamExecIntervalJoin json ser/des



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] zjuwangg opened a new pull request #15276: [FLINK-21837][flink-table-planner-blink] support StreamExecIntervalJoin serialization/deserialization

[GitBox]

zjuwangg opened a new pull request #15276:
URL: https://github.com/apache/flink/pull/15276


   ## What is the purpose of the change
   
   *support StreamExecIntervalJoin serialization/deserialization*
   
   
   ## Brief change log
   
 - 0d5e735 *support StreamExecIntervalJoin serialization/deserialization*
   
   
   ## Verifying this change
   
   This change added tests and can be verified as follows:
 - *Added integration tests `IntervalJoinJsonPlanITCase` to verify exec 
plan can work e2e normaly*
 - *Added `IntervalJoinJsonPlanTest` to verify exec plan is as expected *
   
   ## Does this pull request potentially affect one of the following parts:
   
 - Dependencies (does it add or upgrade a dependency): (no)
 - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
 - The serializers: (no)
 - The runtime per-record code paths (performance sensitive): ( no)
 - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (no)
 - The S3 file system connector: (no)
   
   ## Documentation
   
 - Does this pull request introduce a new feature? (no)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15271: [FLINK-21813][table-planner-blink] Support json ser/de for StreamExecOverAggregate

[GitBox]

flinkbot edited a comment on pull request #15271:
URL: https://github.com/apache/flink/pull/15271#issuecomment-802067101


   
   ## CI report:
   
   * 9800b92650659d48d1abfe6974b4fe1b9cb5257f Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14986)
 
   * bef4e943848053624fa307a294e0476ffab58652 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15006)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15272: [FLINK-21816][table-planner-blink] Suport json ser/de for StreamExecMatch

[GitBox]

flinkbot edited a comment on pull request #15272:
URL: https://github.com/apache/flink/pull/15272#issuecomment-802067367


   
   ## CI report:
   
   * e96eaf020bcec88f7e5364264f15ac4bf6ed34b4 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14987)
 
   * 2542929c02c33edd9817274c8c329949ee27a505 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15007)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15274: [FLINK-21829][Hive Connect]Fix to throw exception when custom hadoop conf path does not exist

[GitBox]

flinkbot edited a comment on pull request #15274:
URL: https://github.com/apache/flink/pull/15274#issuecomment-802519234


   
   ## CI report:
   
   * 4dd5e4e8e3f0a1a89d1e05a1547217939c9ff703 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15008)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15270: [FLINK-21843][table-planner-blink] Support json ser/de for StreamExecGroupWindowAggregate

[GitBox]

flinkbot edited a comment on pull request #15270:
URL: https://github.com/apache/flink/pull/15270#issuecomment-802066772


   
   ## CI report:
   
   * bff3961bf9c86186d08014044f28843a4dd72397 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14985)
 
   * 6204d0655db1e1ebbaf3e38db42093fd07351fb8 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14999)
 
   * 32b0acb81732ffe1acee685854c0ecb915d7 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15005)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] godfreyhe commented on a change in pull request #15270: [FLINK-21843][table-planner-blink] Support json ser/de for StreamExecGroupWindowAggregate

[GitBox]

godfreyhe commented on a change in pull request #15270:
URL: https://github.com/apache/flink/pull/15270#discussion_r597386827



##
File path: 
flink-table/flink-table-planner-blink/src/main/java/org/apache/flink/table/planner/plan/nodes/exec/stream/StreamExecGroupWindowAggregate.java
##
@@ -89,34 +97,74 @@
  * * other is from the legacy GROUP WINDOW FUNCTION syntax. In the long 
future, {@link
  * StreamExecGroupWindowAggregate} will be dropped.
  */
+@JsonIgnoreProperties(ignoreUnknown = true)
 public class StreamExecGroupWindowAggregate extends ExecNodeBase

Review comment:
   please rebase master and let StreamExecGroupWindowAggregate extends from 
StreamExecAggregateBase, which has define some common field names 

##
File path: 
flink-table/flink-table-planner-blink/src/main/java/org/apache/flink/table/planner/plan/nodes/exec/serde/LogicalWindowJsonSerializer.java
##
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.plan.nodes.exec.serde;
+
+import org.apache.flink.table.api.TableException;
+import org.apache.flink.table.expressions.FieldReferenceExpression;
+import org.apache.flink.table.expressions.ValueLiteralExpression;
+import org.apache.flink.table.planner.plan.logical.LogicalWindow;
+import org.apache.flink.table.planner.plan.logical.SessionGroupWindow;
+import org.apache.flink.table.planner.plan.logical.SlidingGroupWindow;
+import org.apache.flink.table.planner.plan.logical.TumblingGroupWindow;
+import org.apache.flink.table.types.AtomicDataType;
+
+import 
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonGenerator;
+import 
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.SerializerProvider;
+import 
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ser.std.StdSerializer;
+
+import java.io.IOException;
+import java.time.Duration;
+
+import static 
org.apache.flink.table.planner.plan.utils.AggregateUtil.hasTimeIntervalType;
+import static 
org.apache.flink.table.planner.plan.utils.AggregateUtil.toDuration;
+import static org.apache.flink.table.planner.plan.utils.AggregateUtil.toLong;
+
+/** JSON serializer for {@link LogicalWindow}. */
+public class LogicalWindowJsonSerializer extends StdSerializer {
+
+public static final String FIELD_NAME_KIND = "kind";
+public static final String KIND_TUMBLING = "TUMBLING";
+public static final String KIND_SLIDING = "SLIDING";
+public static final String KIND_SESSION = "SESSION";
+
+public static final String FIELD_NAME_ALIAS = "alias";
+public static final String FIELD_NAME_TIME_FIELD = "timeField";
+public static final String FIELD_NAME_FIELD_NAME = "fieldName";
+public static final String FIELD_NAME_FIELD_INDEX = "fieldIndex";
+public static final String FIELD_NAME_INPUT_INDEX = "inputIndex";
+public static final String FIELD_NAME_FIELD_TYPE = "fieldType";
+
+public static final String FIELD_NAME_SIZE = "size";
+public static final String FIELD_NAME_IS_TIME_WINDOW = "isTimeWindow";
+
+public static final String FIELD_NAME_SLIDE = "slide";
+public static final String FIELD_NAME_GAP = "gap";
+
+public LogicalWindowJsonSerializer() {
+super(LogicalWindow.class);
+}
+
+@Override
+public void serialize(
+LogicalWindow logicalWindow, JsonGenerator gen, SerializerProvider 
serializerProvider)
+throws IOException {
+gen.writeStartObject();
+if (logicalWindow instanceof TumblingGroupWindow) {
+TumblingGroupWindow window = (TumblingGroupWindow) logicalWindow;
+gen.writeStringField(FIELD_NAME_KIND, KIND_TUMBLING);
+gen.writeObjectField(FIELD_NAME_ALIAS, 
logicalWindow.aliasAttribute());
+FieldReferenceExpression timeField = logicalWindow.timeAttribute();
+serializeFieldReferenceExpression(timeField, gen);
+ValueLiteralExpression size = window.size();
+if (hasTimeIntervalType(size)) {
+Duration duration = toDuration(size);
+gen.writeBooleanField(FIELD_NAME_IS_TIME_WINDOW, true);
+gen.writeObjectField(FIELD_NAME_SIZE, dura

[jira] [Closed] (FLINK-21852) Introduce SupportsAnyNull to BinaryRowData

[Jingsong Lee (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-21852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jingsong Lee closed FLINK-21852.

Resolution: Fixed

master (release-3.0): d84bf8064c2d0b66d1e2cd7e668376c171316530

> Introduce SupportsAnyNull to BinaryRowData
> --
>
> Key: FLINK-21852
> URL: https://issues.apache.org/jira/browse/FLINK-21852
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / Runtime
>Reporter: Jingsong Lee
>Assignee: Jingsong Lee
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.13.0
>
>
> We should avoid force casting to implementation. It is better to rely on 
> interface.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] JingsongLi merged pull request #15257: [FLINK-21852][table] Introduce SupportsAnyNull to BinaryRowData

[GitBox]

JingsongLi merged pull request #15257:
URL: https://github.com/apache/flink/pull/15257


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (FLINK-21747) Encounter an exception that contains "max key length exceeded ..." when reporting metrics to influxdb

[tim yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-21747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

tim yu updated FLINK-21747:
---
Description: I run a streaming job with insert statement whose size is too 
large, it reports metrics to influxdb. I find many influxdb exceptions that 
contains "max key length exceeded ..." in the log file of the job manager . The 
job could not write any metrics to the influxdb, because "task_name" is too 
long.  (was: I run a stream job with insert statement whose size is too long, 
it report metrics to influxdb. I find many influxdb exceptions that contains 
"max key length exceeded ..." in the log file of job manager . The job could 
not write any metrics to the influxdb, because "task_name" and "operator_name" 
is too long.)

> Encounter an exception that contains "max key length exceeded ..."  when 
> reporting metrics to influxdb
> --
>
> Key: FLINK-21747
> URL: https://issues.apache.org/jira/browse/FLINK-21747
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Metrics
>Affects Versions: 1.10.0
>Reporter: tim yu
>Priority: Major
>
> I run a streaming job with insert statement whose size is too large, it 
> reports metrics to influxdb. I find many influxdb exceptions that contains 
> "max key length exceeded ..." in the log file of the job manager . The job 
> could not write any metrics to the influxdb, because "task_name" is too long.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Closed] (FLINK-21861) Support StreamExecIncrementalGroupAggregate json serialization/deserialization

[godfrey he (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-21861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

godfrey he closed FLINK-21861.
--
Resolution: Fixed

Fixed in 1.13.0: 8f08001bb2ebf53a332d07257dc6d42c3b6205aa

> Support StreamExecIncrementalGroupAggregate json serialization/deserialization
> --
>
> Key: FLINK-21861
> URL: https://issues.apache.org/jira/browse/FLINK-21861
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table SQL / Planner
>Reporter: godfrey he
>Assignee: godfrey he
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.13.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] godfreyhe closed pull request #15269: [FLINK-21861][table-planner-blink] Support StreamExecIncrementalGroupAggregate json serialization/deserialization

[GitBox]

godfreyhe closed pull request #15269:
URL: https://github.com/apache/flink/pull/15269


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (FLINK-21837) Support StreamExecIntervalJoin json ser/de

[Terry Wang (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-21837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Terry Wang updated FLINK-21837:
---
Description: Support StreamExecIntervalJoin json ser/des  (was: Support 
StreamExecIntervalJoin/StreamExecLookupJoin/StreamExecTemporalJoin json ser/des)

> Support StreamExecIntervalJoin json ser/de
> --
>
> Key: FLINK-21837
> URL: https://issues.apache.org/jira/browse/FLINK-21837
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table SQL / Planner
>Reporter: Terry Wang
>Assignee: Terry Wang
>Priority: Major
> Fix For: 1.13.0
>
>
> Support StreamExecIntervalJoin json ser/des



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (FLINK-21837) Support StreamExecIntervalJoin json ser/de

[Terry Wang (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-21837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Terry Wang updated FLINK-21837:
---
Summary: Support StreamExecIntervalJoin json ser/de  (was: Support 
StreamExecIntervalJoin/StreamExecLookupJoin/StreamExecTemporalJoin json ser/de)

> Support StreamExecIntervalJoin json ser/de
> --
>
> Key: FLINK-21837
> URL: https://issues.apache.org/jira/browse/FLINK-21837
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table SQL / Planner
>Reporter: Terry Wang
>Assignee: Terry Wang
>Priority: Major
> Fix For: 1.13.0
>
>
> Support StreamExecIntervalJoin/StreamExecLookupJoin/StreamExecTemporalJoin 
> json ser/des



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] flinkbot edited a comment on pull request #15275: [hotfix][docs]add 'IF NOT EXISTS' to create table statement

[GitBox]

flinkbot edited a comment on pull request #15275:
URL: https://github.com/apache/flink/pull/15275#issuecomment-802508892


   
   ## CI report:
   
   * 4d815720a65c9884045069e85b196be6dc112a11 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15002)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #15274: [FLINK-21829][Hive Connect]Fix to throw exception when custom hadoop conf path does not exist

[GitBox]

flinkbot commented on pull request #15274:
URL: https://github.com/apache/flink/pull/15274#issuecomment-802519234


   
   ## CI report:
   
   * 4dd5e4e8e3f0a1a89d1e05a1547217939c9ff703 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] godfreyhe commented on pull request #15269: [FLINK-21861][table-planner-blink] Support StreamExecIncrementalGroupAggregate json serialization/deserialization

[GitBox]

godfreyhe commented on pull request #15269:
URL: https://github.com/apache/flink/pull/15269#issuecomment-802519191


   Thanks for the review @wenlong88 , I resolve the conflict in my local and 
merge this pr


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15273: [FLINK-21800][core] Guard MemorySegment against concurrent frees.

[GitBox]

flinkbot edited a comment on pull request #15273:
URL: https://github.com/apache/flink/pull/15273#issuecomment-802508808


   
   ## CI report:
   
   * 99f0b12ad7cd87ca9dfd5d9d3e97030eb41135f9 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15001)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15272: [FLINK-21816][table-planner-blink] Suport json ser/de for StreamExecMatch

[GitBox]

flinkbot edited a comment on pull request #15272:
URL: https://github.com/apache/flink/pull/15272#issuecomment-802067367


   
   ## CI report:
   
   * e96eaf020bcec88f7e5364264f15ac4bf6ed34b4 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14987)
 
   * 2542929c02c33edd9817274c8c329949ee27a505 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15271: [FLINK-21813][table-planner-blink] Support json ser/de for StreamExecOverAggregate

[GitBox]

flinkbot edited a comment on pull request #15271:
URL: https://github.com/apache/flink/pull/15271#issuecomment-802067101


   
   ## CI report:
   
   * 9800b92650659d48d1abfe6974b4fe1b9cb5257f Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14986)
 
   * bef4e943848053624fa307a294e0476ffab58652 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15270: [FLINK-21843][table-planner-blink] Support json ser/de for StreamExecGroupWindowAggregate

[GitBox]

flinkbot edited a comment on pull request #15270:
URL: https://github.com/apache/flink/pull/15270#issuecomment-802066772


   
   ## CI report:
   
   * bff3961bf9c86186d08014044f28843a4dd72397 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14985)
 
   * 6204d0655db1e1ebbaf3e38db42093fd07351fb8 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14999)
 
   * 32b0acb81732ffe1acee685854c0ecb915d7 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15257: [FLINK-21852][table] Introduce SupportsAnyNull to BinaryRowData

[GitBox]

flinkbot edited a comment on pull request #15257:
URL: https://github.com/apache/flink/pull/15257#issuecomment-801601825


   
   ## CI report:
   
   * c75184bb0b59c6bd548ba481d96525991a2bac07 UNKNOWN
   * e98801c410ec95737e91892661e393752f5e4468 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14935)
 
   * e7adcc84dfcccb2a5e47735e21726727f86c8363 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14998)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Closed] (FLINK-21860) Support StreamExecExpand json serialization/deserialization

[godfrey he (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-21860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

godfrey he closed FLINK-21860.
--
Resolution: Fixed

Fixed in 1.13.0: f026dd6ec81eee2cc64dd9d0ff0a2faf964b6a10

> Support StreamExecExpand json serialization/deserialization
> ---
>
> Key: FLINK-21860
> URL: https://issues.apache.org/jira/browse/FLINK-21860
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table SQL / Planner
>Reporter: godfrey he
>Assignee: godfrey he
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.13.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] godfreyhe closed pull request #15268: [FLINK-21860][table-planner-blink] Support StreamExecExpand json serialization/deserialization

[GitBox]

godfreyhe closed pull request #15268:
URL: https://github.com/apache/flink/pull/15268


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] godfreyhe commented on pull request #15268: [FLINK-21860][table-planner-blink] Support StreamExecExpand json serialization/deserialization

[GitBox]

godfreyhe commented on pull request #15268:
URL: https://github.com/apache/flink/pull/15268#issuecomment-802516630


   Thanks for the review @wenlong88 , I will resolve the conflict in my local 
and merge the pr


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Closed] (FLINK-21791) Support StreamExecLocalGroupAggregate and StreamExecGlobalGroupAggregate json serialization/deserialization

[godfrey he (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-21791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

godfrey he closed FLINK-21791.
--
Resolution: Fixed

Fixed in 1.13.0: fdcadc4684e82d3cf69a78d87ec5033ac534e728

> Support StreamExecLocalGroupAggregate and StreamExecGlobalGroupAggregate json 
> serialization/deserialization
> ---
>
> Key: FLINK-21791
> URL: https://issues.apache.org/jira/browse/FLINK-21791
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table SQL / Planner
>Reporter: godfrey he
>Assignee: godfrey he
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.13.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] godfreyhe merged pull request #15264: [FLINK-21791][table-planner-blink] Support StreamExecLocalGroupAggregate and StreamExecGlobalGroupAggregate json serialization/deserialization

[GitBox]

godfreyhe merged pull request #15264:
URL: https://github.com/apache/flink/pull/15264


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] kezhuw commented on a change in pull request #15273: [FLINK-21800][core] Guard MemorySegment against concurrent frees.

[GitBox]

kezhuw commented on a change in pull request #15273:
URL: https://github.com/apache/flink/pull/15273#discussion_r597375675



##
File path: 
flink-core/src/main/java/org/apache/flink/core/memory/MemorySegment.java
##
@@ -217,10 +222,14 @@ public int size() {
 /**
  * Checks whether the memory segment was freed.
  *
+ * This method internally involves cross-thread synchronization. Do not 
use for performance
+ * sensitive code paths.
+ *
  * @return true, if the memory segment has been freed, 
false otherwise.
  */
 public boolean isFreed() {
-return address > addressLimit;
+// in performance sensitive cases, use 'address > addressLimit' instead
+return isFreedAtomic.get();

Review comment:
   I think we could promote this atomic handling to 
`MemoryUtils.createMemoryCleaner` or even `MemoryUtils.allocateUnsafe`(with 
enriched struct). Then we could remove this caveat. I think we are only 
guarding unsafe part ? If yes, I recommend to move this concurrent guarding to 
unsafe`cleaner`.

##
File path: 
flink-core/src/test/java/org/apache/flink/core/memory/OffHeapUnsafeMemorySegmentTest.java
##
@@ -58,4 +61,31 @@ public void testCallCleanerOnFree() {
 .free();
 assertTrue(cleanerFuture.isDone());
 }
+
+@Test
+public void testCallCleanerOnceOnConcurrentFree() throws 
InterruptedException {
+final AtomicInteger counter = new AtomicInteger(0);

Review comment:
   I think we should also test throwing path, in that path the final 
`counter` should also be one.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] godfreyhe commented on pull request #15264: [FLINK-21791][table-planner-blink] Support StreamExecLocalGroupAggregate and StreamExecGlobalGroupAggregate json serialization/deserializat

[GitBox]

godfreyhe commented on pull request #15264:
URL: https://github.com/apache/flink/pull/15264#issuecomment-802512285


   My private azure is green, 
https://dev.azure.com/godfreyhe/godfreyhe-flink/_build/results?buildId=354&view=results


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Closed] (FLINK-21841) Can not find kafka-connect with sql-kafka-connector

[JieFang.He (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-21841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

JieFang.He closed FLINK-21841.
--
Resolution: Not A Problem

> Can not find kafka-connect with sql-kafka-connector
> ---
>
> Key: FLINK-21841
> URL: https://issues.apache.org/jira/browse/FLINK-21841
> Project: Flink
>  Issue Type: Bug
>  Components: Connectors / Kafka, Table SQL / Ecosystem
>Affects Versions: 1.11.1
>Reporter: JieFang.He
>Priority: Major
>
>  
> When use sql-kafka with fat-jar（make flink-sql-connector-kafka_2.11 in user 
> jar） with flink 1.11.1  like
> {code:java}
> CREATE TABLE user_behavior (
>  user_id INT,
>  action STRING,
>  province INT,
>  ts TIMESTAMP(3)
> ) WITH (
>  'connector' = 'kafka',
>  'topic' = 'intopic',
>  'properties.bootstrap.servers' = 'kafkaserver:9092',
>  'properties.group.id' = 'testGroup',
>  'format' = 'csv',
>  'scan.startup.mode' = 'earliest-offset'
> )
> {code}
>  I get a exception
> {code:java}
> Caused by: org.apache.flink.table.api.ValidationException: Cannot discover a 
> connector using option ''connector'='kafka''.
> at 
> org.apache.flink.table.factories.FactoryUtil.getDynamicTableFactory(FactoryUtil.java:329)
> at 
> org.apache.flink.table.factories.FactoryUtil.createTableSource(FactoryUtil.java:118)
> ... 35 more
> Caused by: org.apache.flink.table.api.ValidationException: Could not find any 
> factory for identifier 'kafka' that implements 
> 'org.apache.flink.table.factories.DynamicTableSourceFactory' in the 
> classpath.Available factory identifiers are:datagen
> {code}
> It looks like the issue 
> [FLINK-18076|https://issues.apache.org/jira/browse/FLINK-18076] is not deal 
> with all exceptions
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] flinkbot commented on pull request #15275: [hotfix][docs]add 'IF NOT EXISTS' to create table statement

[GitBox]

flinkbot commented on pull request #15275:
URL: https://github.com/apache/flink/pull/15275#issuecomment-802508892


   
   ## CI report:
   
   * 4d815720a65c9884045069e85b196be6dc112a11 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #15273: [FLINK-21800][core] Guard MemorySegment against concurrent frees.

[GitBox]

flinkbot commented on pull request #15273:
URL: https://github.com/apache/flink/pull/15273#issuecomment-802508808


   
   ## CI report:
   
   * 99f0b12ad7cd87ca9dfd5d9d3e97030eb41135f9 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15270: [FLINK-21843][table-planner-blink] Support json ser/de for StreamExecGroupWindowAggregate

[GitBox]

flinkbot edited a comment on pull request #15270:
URL: https://github.com/apache/flink/pull/15270#issuecomment-802066772


   
   ## CI report:
   
   * bff3961bf9c86186d08014044f28843a4dd72397 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14985)
 
   * 6204d0655db1e1ebbaf3e38db42093fd07351fb8 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14999)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15257: [FLINK-21852][table] Introduce SupportsAnyNull to BinaryRowData

[GitBox]

flinkbot edited a comment on pull request #15257:
URL: https://github.com/apache/flink/pull/15257#issuecomment-801601825


   
   ## CI report:
   
   * c75184bb0b59c6bd548ba481d96525991a2bac07 UNKNOWN
   * e98801c410ec95737e91892661e393752f5e4468 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14935)
 
   * e7adcc84dfcccb2a5e47735e21726727f86c8363 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15185: [FLINK-21768][clients]Optimize system.exit() logic of CliFrontend

[GitBox]

flinkbot edited a comment on pull request #15185:
URL: https://github.com/apache/flink/pull/15185#issuecomment-79889


   
   ## CI report:
   
   * 95bf51ed50d7706aeca72db222e5862a1c78eb79 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14657)
 
   * bbfe60eb6db082a292800187af1734c687136b32 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14997)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (FLINK-21841) Can not find kafka-connect with sql-kafka-connector

[JieFang.He (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-21841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304609#comment-17304609
 ] 

JieFang.He commented on FLINK-21841:


[~jark] Thank you for your answers. It works. I used the wrong Maven plugin

> Can not find kafka-connect with sql-kafka-connector
> ---
>
> Key: FLINK-21841
> URL: https://issues.apache.org/jira/browse/FLINK-21841
> Project: Flink
>  Issue Type: Bug
>  Components: Connectors / Kafka, Table SQL / Ecosystem
>Affects Versions: 1.11.1
>Reporter: JieFang.He
>Priority: Major
>
>  
> When use sql-kafka with fat-jar（make flink-sql-connector-kafka_2.11 in user 
> jar） with flink 1.11.1  like
> {code:java}
> CREATE TABLE user_behavior (
>  user_id INT,
>  action STRING,
>  province INT,
>  ts TIMESTAMP(3)
> ) WITH (
>  'connector' = 'kafka',
>  'topic' = 'intopic',
>  'properties.bootstrap.servers' = 'kafkaserver:9092',
>  'properties.group.id' = 'testGroup',
>  'format' = 'csv',
>  'scan.startup.mode' = 'earliest-offset'
> )
> {code}
>  I get a exception
> {code:java}
> Caused by: org.apache.flink.table.api.ValidationException: Cannot discover a 
> connector using option ''connector'='kafka''.
> at 
> org.apache.flink.table.factories.FactoryUtil.getDynamicTableFactory(FactoryUtil.java:329)
> at 
> org.apache.flink.table.factories.FactoryUtil.createTableSource(FactoryUtil.java:118)
> ... 35 more
> Caused by: org.apache.flink.table.api.ValidationException: Could not find any 
> factory for identifier 'kafka' that implements 
> 'org.apache.flink.table.factories.DynamicTableSourceFactory' in the 
> classpath.Available factory identifiers are:datagen
> {code}
> It looks like the issue 
> [FLINK-18076|https://issues.apache.org/jira/browse/FLINK-18076] is not deal 
> with all exceptions
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] hehuiyuan commented on pull request #15274: [FLINK-21829][Hive Connect]Fix to throw exception when custom hadoop conf path does not exist

[GitBox]

hehuiyuan commented on pull request #15274:
URL: https://github.com/apache/flink/pull/15274#issuecomment-802500849


   @flinkbot re-run
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] hehuiyuan closed pull request #15274: [FLINK-21829][Hive Connect]Fix to throw exception when custom hadoop conf path does not exist

[GitBox]

hehuiyuan closed pull request #15274:
URL: https://github.com/apache/flink/pull/15274


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #15275: [hotfix][docs]add 'IF NOT EXISTS' to create table statement

[GitBox]

flinkbot commented on pull request #15275:
URL: https://github.com/apache/flink/pull/15275#issuecomment-802498311


   Thanks a lot for your contribution to the Apache Flink project. I'm the 
@flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress 
of the review.
   
   
   ## Automated Checks
   Last check on commit 4d815720a65c9884045069e85b196be6dc112a11 (Fri Mar 19 
02:44:53 UTC 2021)
   
✅no warnings
   
   Mention the bot in a comment to re-run the automated checks.
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review 
Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full 
explanation of the review process.
The Bot is tracking the review progress through labels. Labels are applied 
according to the order of the review items. For consensus, approval by a Flink 
committer of PMC member is required Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot approve description` to approve one or more aspects (aspects: 
`description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until 
`architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's 
attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Closed] (FLINK-21805) Support StreamExecRank and StreamExecSortLimit and StreamExecLimit Json ser/de

[godfrey he (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-21805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

godfrey he closed FLINK-21805.
--
Resolution: Fixed

Fixed in 1.13.0: ca80fc9e63c923893efd3d61b816bb731d5400b1

> Support StreamExecRank and StreamExecSortLimit and StreamExecLimit Json ser/de
> --
>
> Key: FLINK-21805
> URL: https://issues.apache.org/jira/browse/FLINK-21805
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table SQL / Planner
>Reporter: Wenlong Lyu
>Assignee: Wenlong Lyu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.13.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (FLINK-21805) Support StreamExecRank and StreamExecSortLimit and StreamExecLimit Json ser/de

[godfrey he (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-21805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

godfrey he updated FLINK-21805:
---
Fix Version/s: 1.13.0

> Support StreamExecRank and StreamExecSortLimit and StreamExecLimit Json ser/de
> --
>
> Key: FLINK-21805
> URL: https://issues.apache.org/jira/browse/FLINK-21805
> Project: Flink
>  Issue Type: Sub-task
>Reporter: Wenlong Lyu
>Assignee: Wenlong Lyu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.13.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (FLINK-21805) Support StreamExecRank and StreamExecSortLimit and StreamExecLimit Json ser/de

[godfrey he (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-21805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

godfrey he updated FLINK-21805:
---
Component/s: Table SQL / Planner

> Support StreamExecRank and StreamExecSortLimit and StreamExecLimit Json ser/de
> --
>
> Key: FLINK-21805
> URL: https://issues.apache.org/jira/browse/FLINK-21805
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table SQL / Planner
>Reporter: Wenlong Lyu
>Assignee: Wenlong Lyu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.13.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] godfreyhe closed pull request #15231: [FLINK-21805][table-planner-blink] Support json ser/de for StreamExecRank, StreamExecLimit and StreamExecSortLimit

[GitBox]

godfreyhe closed pull request #15231:
URL: https://github.com/apache/flink/pull/15231


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] zhuxiaoshang opened a new pull request #15275: [hotfix][docs]add 'IF NOT EXISTS' to create table statement

[GitBox]

zhuxiaoshang opened a new pull request #15275:
URL: https://github.com/apache/flink/pull/15275


   
   
   ## What is the purpose of the change
   
   add 'IF NOT EXISTS' to create table statement docs
   
   ## Brief change log
   
   add 'IF NOT EXISTS' to create table statement docs
   
   ## Verifying this change
   
   This change is a trivial rework / code cleanup without any test coverage.
   
   ## Does this pull request potentially affect one of the following parts:
   
 - Dependencies (does it add or upgrade a dependency): (yes / no)no
 - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (yes / no)no
 - The serializers: (yes / no / don't know)no
 - The runtime per-record code paths (performance sensitive): (yes / no / 
don't know)no
 - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't 
know)no
 - The S3 file system connector: (yes / no / don't know)no
   
   ## Documentation
   
 - Does this pull request introduce a new feature? (yes / no)no
 - If yes, how is the feature documented? (not applicable / docs / JavaDocs 
/ not documented)no
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15270: [FLINK-21843][table-planner-blink] Support json ser/de for StreamExecGroupWindowAggregate

[GitBox]

flinkbot edited a comment on pull request #15270:
URL: https://github.com/apache/flink/pull/15270#issuecomment-802066772


   
   ## CI report:
   
   * bff3961bf9c86186d08014044f28843a4dd72397 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14985)
 
   * 6204d0655db1e1ebbaf3e38db42093fd07351fb8 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #15185: [FLINK-21768][clients]Optimize system.exit() logic of CliFrontend

[GitBox]

flinkbot edited a comment on pull request #15185:
URL: https://github.com/apache/flink/pull/15185#issuecomment-79889


   
   ## CI report:
   
   * 95bf51ed50d7706aeca72db222e5862a1c78eb79 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14657)
 
   * bbfe60eb6db082a292800187af1734c687136b32 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] becketqin commented on pull request #14784: [FLINK-21160][connector/kafka] Fix bug of referencing uninitialized deserializer when using KafkaRecordDeserializer#valueOnly

[GitBox]

becketqin commented on pull request #14784:
URL: https://github.com/apache/flink/pull/14784#issuecomment-802492952


   Thanks for the patch. LGTM.
   
   Merged to master:
   844601b9c1468e241325bcff9695a52035192661
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] godfreyhe commented on pull request #15231: [FLINK-21805][table-planner-blink] Support json ser/de for StreamExecRank, StreamExecLimit and StreamExecSortLimit

[GitBox]

godfreyhe commented on pull request #15231:
URL: https://github.com/apache/flink/pull/15231#issuecomment-802492993


   The failed test has nothing with this pr, I will merge this pr.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #15274: [FLINK-21829][Hive Connect]Fix to throw exception when custom hadoop conf path does not exist

[GitBox]

flinkbot commented on pull request #15274:
URL: https://github.com/apache/flink/pull/15274#issuecomment-802492492


   Thanks a lot for your contribution to the Apache Flink project. I'm the 
@flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress 
of the review.
   
   
   ## Automated Checks
   Last check on commit 4dd5e4e8e3f0a1a89d1e05a1547217939c9ff703 (Fri Mar 19 
02:33:55 UTC 2021)
   
   **Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up 
to date!
* **This pull request references an unassigned [Jira 
ticket](https://issues.apache.org/jira/browse/FLINK-21829).** According to the 
[code contribution 
guide](https://flink.apache.org/contributing/contribute-code.html), tickets 
need to be assigned before starting with the implementation work.
   
   
   Mention the bot in a comment to re-run the automated checks.
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review 
Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full 
explanation of the review process.
The Bot is tracking the review progress through labels. Labels are applied 
according to the order of the review items. For consensus, approval by a Flink 
committer of PMC member is required Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot approve description` to approve one or more aspects (aspects: 
`description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until 
`architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's 
attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] becketqin merged pull request #14784: [FLINK-21160][connector/kafka] Fix bug of referencing uninitialized deserializer when using KafkaRecordDeserializer#valueOnly

[GitBox]

becketqin merged pull request #14784:
URL: https://github.com/apache/flink/pull/14784


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #15273: [FLINK-21800][core] Guard MemorySegment against concurrent frees.

[GitBox]

flinkbot commented on pull request #15273:
URL: https://github.com/apache/flink/pull/15273#issuecomment-802491454


   Thanks a lot for your contribution to the Apache Flink project. I'm the 
@flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress 
of the review.
   
   
   ## Automated Checks
   Last check on commit 99f0b12ad7cd87ca9dfd5d9d3e97030eb41135f9 (Fri Mar 19 
02:32:02 UTC 2021)
   
   **Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up 
to date!
   
   
   Mention the bot in a comment to re-run the automated checks.
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review 
Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full 
explanation of the review process.
The Bot is tracking the review progress through labels. Labels are applied 
according to the order of the review items. For consensus, approval by a Flink 
committer of PMC member is required Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot approve description` to approve one or more aspects (aspects: 
`description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until 
`architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's 
attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] hehuiyuan opened a new pull request #15274: [FLINK-21829][Hive Connect]Fix to throw exception when custom hadoop conf path does not exist

[GitBox]

hehuiyuan opened a new pull request #15274:
URL: https://github.com/apache/flink/pull/15274


   ## What is the purpose of the change
   There is no prompt when the the path to hadoop conf configured is wrong 
unintentional.
   It is better to  load hadoop conf from possiable  hadoop path when the path 
is wrong.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] xintongsong commented on pull request #15273: [FLINK-21800][core] Guard MemorySegment against concurrent frees.

[GitBox]

xintongsong commented on pull request #15273:
URL: https://github.com/apache/flink/pull/15273#issuecomment-802490464


   cc @tillrohrmann @kezhuw 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (FLINK-21800) Guard MemorySegment against concurrent frees.

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-21800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-21800:
---
Labels: pull-request-available  (was: )

> Guard MemorySegment against concurrent frees.
> -
>
> Key: FLINK-21800
> URL: https://issues.apache.org/jira/browse/FLINK-21800
> Project: Flink
>  Issue Type: Sub-task
>  Components: Runtime / Coordination
>Reporter: Xintong Song
>Assignee: Xintong Song
>Priority: Critical
>  Labels: pull-request-available
> Fix For: 1.13.0
>
>
> This ticket tracks the effort that:
> Guard memory segment against multiple frees, ensuring the underlying memory 
> is only released once.
> Introduce a configuration option to explicitly fail on multiple-frees, which 
> helps detects the cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] xintongsong opened a new pull request #15273: [FLINK-21800][core] Guard MemorySegment against concurrent frees.

[GitBox]

xintongsong opened a new pull request #15273:
URL: https://github.com/apache/flink/pull/15273


   
   
   ## What is the purpose of the change
   
   *(For example: This pull request makes task deployment go through the blob 
server, rather than through RPC. That way we avoid re-transferring them on each 
deployment (during recovery).)*
   
   
   ## Brief change log
   
   *(for example:)*
 - *The TaskInfo is stored in the blob store on job creation time as a 
persistent artifact*
 - *Deployments RPC transmits only the blob storage reference*
 - *TaskManagers retrieve the TaskInfo from the blob cache*
   
   
   ## Verifying this change
   
   *(Please pick either of the following options)*
   
   This change is a trivial rework / code cleanup without any test coverage.
   
   *(or)*
   
   This change is already covered by existing tests, such as *(please describe 
tests)*.
   
   *(or)*
   
   This change added tests and can be verified as follows:
   
   *(example:)*
 - *Added integration tests for end-to-end deployment with large payloads 
(100MB)*
 - *Extended integration test for recovery after master (JobManager) 
failure*
 - *Added test that validates that TaskInfo is transferred only once across 
recoveries*
 - *Manually verified the change by running a 4 node cluser with 2 
JobManagers and 4 TaskManagers, a stateful streaming program, and killing one 
JobManager and two TaskManagers during the execution, verifying that recovery 
happens correctly.*
   
   ## Does this pull request potentially affect one of the following parts:
   
 - Dependencies (does it add or upgrade a dependency): (yes / no)
 - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (yes / no)
 - The serializers: (yes / no / don't know)
 - The runtime per-record code paths (performance sensitive): (yes / no / 
don't know)
 - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't 
know)
 - The S3 file system connector: (yes / no / don't know)
   
   ## Documentation
   
 - Does this pull request introduce a new feature? (yes / no)
 - If yes, how is the feature documented? (not applicable / docs / JavaDocs 
/ not documented)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] KarmaGYZ commented on a change in pull request #15249: [FLINK-21794][metrics] Support retrieving slot details via rest api

[GitBox]

KarmaGYZ commented on a change in pull request #15249:
URL: https://github.com/apache/flink/pull/15249#discussion_r597364645



##
File path: 
flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/slotmanager/FineGrainedSlotManager.java
##
@@ -636,6 +637,15 @@ public ResourceProfile getFreeResourceOf(InstanceID 
instanceID) {
 return taskManagerTracker.getRegisteredResourceOf(instanceID);
 }
 
+@Override
+public Collection getAllocatedSlotsOf(InstanceID instanceID) {
+return taskManagerTracker.getRegisteredTaskManager(instanceID)
+.map(TaskManagerInfo::getAllocatedSlots).map(Map::values)
+.orElse(Collections.emptyList()).stream()
+.map(slot -> new SlotInfo(slot.getJobId(), 
slot.getResourceProfile()))
+.collect(Collectors.toList());

Review comment:
   I think how to expose that information to the user is depends on the 
WebUI and might be out of the scope of this PR. In current interface, we give 
the UI flexibility to choose how to aggregate it.
   Also, user might want to know the explicit resource of each task, especially 
for external resources.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Closed] (FLINK-21811) Support StreamExecJoin json serialization/deserialization

[godfrey he (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-21811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

godfrey he closed FLINK-21811.
--
Resolution: Fixed

Fixed in 1.13.0: 3582bb23ba3919b10f36c35d214048ecfaabe5c3

> Support StreamExecJoin json serialization/deserialization
> -
>
> Key: FLINK-21811
> URL: https://issues.apache.org/jira/browse/FLINK-21811
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table SQL / Planner
>Reporter: Terry Wang
>Assignee: Terry Wang
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.13.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] godfreyhe closed pull request #15239: [FLINK-21811][blink-table-planner]Support StreamExecJoin json serialization/deserialization

[GitBox]

godfreyhe closed pull request #15239:
URL: https://github.com/apache/flink/pull/15239


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] hehuiyuan closed pull request #15244: [FLINK-21829][Hive Connect]Fix to create HiveCatalog with custom hadoopconfdir first

[GitBox]

hehuiyuan closed pull request #15244:
URL: https://github.com/apache/flink/pull/15244


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] zuston commented on pull request #15185: [FLINK-21768][clients]Optimize system.exit() logic of CliFrontend

[GitBox]

zuston commented on pull request #15185:
URL: https://github.com/apache/flink/pull/15185#issuecomment-802467775


   > Thanks for creating this PR @zuston. LGTM. Merging once AZP gives green 
light.
   
   Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] zuston commented on pull request #15185: [FLINK-21768][clients]Optimize system.exit() logic of CliFrontend

[GitBox]

zuston commented on pull request #15185:
URL: https://github.com/apache/flink/pull/15185#issuecomment-802467665


   > @zuston nit: The commit message could be refined.
   > 
   > * Commit title`[clients]` -> `[client]`
   > * Remove the magic commit message content `Change-Id: 
Ia28bb5b7d389b3faedb11af15576a458a74d4e76`
   
   Done. 
   I share project with internal gerrit, so it will add change-id with gerrit.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [flink] vthinkxie commented on a change in pull request #15249: [FLINK-21794][metrics] Support retrieving slot details via rest api

[GitBox]

vthinkxie commented on a change in pull request #15249:
URL: https://github.com/apache/flink/pull/15249#discussion_r597360427



##
File path: 
flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/slotmanager/FineGrainedSlotManager.java
##
@@ -636,6 +637,15 @@ public ResourceProfile getFreeResourceOf(InstanceID 
instanceID) {
 return taskManagerTracker.getRegisteredResourceOf(instanceID);
 }
 
+@Override
+public Collection getAllocatedSlotsOf(InstanceID instanceID) {
+return taskManagerTracker.getRegisteredTaskManager(instanceID)
+.map(TaskManagerInfo::getAllocatedSlots).map(Map::values)
+.orElse(Collections.emptyList()).stream()
+.map(slot -> new SlotInfo(slot.getJobId(), 
slot.getResourceProfile()))
+.collect(Collectors.toList());

Review comment:
   "Job as has N slots on TM Z, where slot 1 uses X_1 memory any Y_1 cpus, 
and slot 2 uses X_2 memory and Y_2 cpus, and ..." could be aggregated and get 
"Job A has allocated X memory and Y CPUs on TM Z to run N tasks."
   while it cannot be pushed backward.
   Since TM and resources are implemented on a slots basis, the interface layer 
should expose atomic properties




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Closed] (FLINK-21744) Support StreamExecDeduplicate json serialization/deserialization

[godfrey he (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-21744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

godfrey he closed FLINK-21744.
--
Resolution: Fixed

Fixed in 1.13.0: 606ef6989c96ae376c969bf103d7275750323a8f

> Support StreamExecDeduplicate json serialization/deserialization
> 
>
> Key: FLINK-21744
> URL: https://issues.apache.org/jira/browse/FLINK-21744
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table SQL / Planner
>Reporter: Terry Wang
>Assignee: Terry Wang
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.13.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [flink] godfreyhe closed pull request #15220: [FLINK-21744][table-planner-blink]Support StreamExecDeduplicate json serialization/deserialization

[GitBox]

godfreyhe closed pull request #15220:
URL: https://github.com/apache/flink/pull/15220


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org