[GitHub] [flink] flinkbot edited a comment on pull request #14376: [FLINK-18202][PB format] New Format of protobuf
flinkbot edited a comment on pull request #14376: URL: https://github.com/apache/flink/pull/14376#issuecomment-744252765 ## CI report: * a721a9d5f663117a203628547a19b33b23927ac2 UNKNOWN * 8f1051527a46aeba13aadc2db3c5f34d8aff78d2 UNKNOWN * a3dd67481e55eec8a8d1a6dad219e1aa0ba10ec0 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27985) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14376: [FLINK-18202][PB format] New Format of protobuf
flinkbot edited a comment on pull request #14376: URL: https://github.com/apache/flink/pull/14376#issuecomment-744252765 ## CI report: * a721a9d5f663117a203628547a19b33b23927ac2 UNKNOWN * 8f1051527a46aeba13aadc2db3c5f34d8aff78d2 UNKNOWN * 5cf0fa22d2ad2e24c9b4c088097999f2eff8f106 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27772) * a3dd67481e55eec8a8d1a6dad219e1aa0ba10ec0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27985) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14376: [FLINK-18202][PB format] New Format of protobuf
flinkbot edited a comment on pull request #14376: URL: https://github.com/apache/flink/pull/14376#issuecomment-744252765 ## CI report: * a721a9d5f663117a203628547a19b33b23927ac2 UNKNOWN * 8f1051527a46aeba13aadc2db3c5f34d8aff78d2 UNKNOWN * 5cf0fa22d2ad2e24c9b4c088097999f2eff8f106 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27772) * a3dd67481e55eec8a8d1a6dad219e1aa0ba10ec0 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14376: [FLINK-18202][PB format] New Format of protobuf
flinkbot edited a comment on pull request #14376: URL: https://github.com/apache/flink/pull/14376#issuecomment-744252765 ## CI report: * a721a9d5f663117a203628547a19b33b23927ac2 UNKNOWN * 8f1051527a46aeba13aadc2db3c5f34d8aff78d2 UNKNOWN * 5cf0fa22d2ad2e24c9b4c088097999f2eff8f106 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27772) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14376: [FLINK-18202][PB format] New Format of protobuf
flinkbot edited a comment on pull request #14376: URL: https://github.com/apache/flink/pull/14376#issuecomment-744252765 ## CI report: * a721a9d5f663117a203628547a19b33b23927ac2 UNKNOWN * 8f1051527a46aeba13aadc2db3c5f34d8aff78d2 UNKNOWN * 5cf0fa22d2ad2e24c9b4c088097999f2eff8f106 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27772) * a3dd67481e55eec8a8d1a6dad219e1aa0ba10ec0 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14376: [FLINK-18202][PB format] New Format of protobuf
flinkbot edited a comment on pull request #14376: URL: https://github.com/apache/flink/pull/14376#issuecomment-744252765 ## CI report: * a721a9d5f663117a203628547a19b33b23927ac2 UNKNOWN * 8f1051527a46aeba13aadc2db3c5f34d8aff78d2 UNKNOWN * 5cf0fa22d2ad2e24c9b4c088097999f2eff8f106 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27772) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14376: [FLINK-18202][PB format] New Format of protobuf
flinkbot edited a comment on pull request #14376: URL: https://github.com/apache/flink/pull/14376#issuecomment-744252765 ## CI report: * a721a9d5f663117a203628547a19b33b23927ac2 UNKNOWN * 8f1051527a46aeba13aadc2db3c5f34d8aff78d2 UNKNOWN * 5cf0fa22d2ad2e24c9b4c088097999f2eff8f106 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27772) * a3dd67481e55eec8a8d1a6dad219e1aa0ba10ec0 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14376: [FLINK-18202][PB format] New Format of protobuf
flinkbot edited a comment on pull request #14376: URL: https://github.com/apache/flink/pull/14376#issuecomment-744252765 ## CI report: * a721a9d5f663117a203628547a19b33b23927ac2 UNKNOWN * 8f1051527a46aeba13aadc2db3c5f34d8aff78d2 UNKNOWN * 5cf0fa22d2ad2e24c9b4c088097999f2eff8f106 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27772) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14376: [FLINK-18202][PB format] New Format of protobuf
flinkbot edited a comment on pull request #14376: URL: https://github.com/apache/flink/pull/14376#issuecomment-744252765 ## CI report: * a721a9d5f663117a203628547a19b33b23927ac2 UNKNOWN * 8f1051527a46aeba13aadc2db3c5f34d8aff78d2 UNKNOWN * 5cf0fa22d2ad2e24c9b4c088097999f2eff8f106 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27772) * a3dd67481e55eec8a8d1a6dad219e1aa0ba10ec0 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14376: [FLINK-18202][PB format] New Format of protobuf
flinkbot edited a comment on pull request #14376: URL: https://github.com/apache/flink/pull/14376#issuecomment-744252765 ## CI report: * a721a9d5f663117a203628547a19b33b23927ac2 UNKNOWN * 8f1051527a46aeba13aadc2db3c5f34d8aff78d2 UNKNOWN * 5cf0fa22d2ad2e24c9b4c088097999f2eff8f106 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27772) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] maosuhan commented on a change in pull request #14376: [FLINK-18202][PB format] New Format of protobuf
maosuhan commented on a change in pull request #14376:
URL: https://github.com/apache/flink/pull/14376#discussion_r767101531
##
File path:
flink-formats/flink-protobuf/src/main/java/org/apache/flink/formats/protobuf/PbFormatUtils.java
##
@@ -0,0 +1,166 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.formats.protobuf;
+
+import org.apache.flink.table.types.logical.ArrayType;
+import org.apache.flink.table.types.logical.LogicalType;
+import org.apache.flink.table.types.logical.MapType;
+import org.apache.flink.util.FlinkRuntimeException;
+
+import com.google.protobuf.Descriptors;
+import org.apache.commons.lang3.StringUtils;
+
+/** Protobuf function util. */
+public class PbFormatUtils {
+
+/**
+ * protobuf code has a bug that, f_abc_7d will be convert to fAbc7d in
{@code
+ * com.google.protobuf.Descriptors.FileDescriptor.getJsonName()}, but
actually we need fAbc7D.
+ */
+public static String fieldNameToJsonName(String name) {
+final int length = name.length();
+StringBuilder result = new StringBuilder(length);
+boolean isNextUpperCase = false;
+for (int i = 0; i < length; i++) {
+char ch = name.charAt(i);
+if (ch == '_') {
+isNextUpperCase = true;
+} else if (isNextUpperCase) {
+if ('a' <= ch && ch <= 'z') {
+ch = (char) (ch - 'a' + 'A');
+isNextUpperCase = false;
+}
+result.append(ch);
+} else {
+result.append(ch);
+}
+}
+return result.toString();
+}
+
+private static String getJavaPackageFromProtoFile(Descriptors.Descriptor
descriptor) {
+boolean hasJavaPackage =
descriptor.getFile().getOptions().hasJavaPackage();
+if (hasJavaPackage) {
+String javaPackage =
descriptor.getFile().getOptions().getJavaPackage();
+if (StringUtils.isBlank(javaPackage)) {
+throw new FlinkRuntimeException("java_package cannot be blank
string");
+}
+return javaPackage;
+} else {
+String packageName = descriptor.getFile().getPackage();
+if (StringUtils.isBlank(packageName)) {
+throw new FlinkRuntimeException("package and java_package
cannot both be empty");
+}
+return packageName;
+}
+}
+
+public static String getFullJavaName(Descriptors.Descriptor descriptor) {
+String javaPackageName = getJavaPackageFromProtoFile(descriptor);
+if (descriptor.getFile().getOptions().getJavaMultipleFiles()) {
+// multiple_files=true
+if (null != descriptor.getContainingType()) {
+// nested type
+String parentJavaFullName =
getFullJavaName(descriptor.getContainingType());
+return parentJavaFullName + "." + descriptor.getName();
+} else {
+// top level message
+return javaPackageName + "." + descriptor.getName();
+}
+} else {
+// multiple_files=false
+if (null != descriptor.getContainingType()) {
+// nested type
+String parentJavaFullName =
getFullJavaName(descriptor.getContainingType());
+return parentJavaFullName + "." + descriptor.getName();
+} else {
+// top level message
+if
(!descriptor.getFile().getOptions().hasJavaOuterClassname()) {
+// user do not define outer class name in proto file
+return javaPackageName
Review comment:
@MyLanPangzi Thanks for your findings. Yes, it is a bug. I found that if
the proto file name is test.proto and the content is as below, the outermost
class name will be TestOuterClass because the message name is the same with
file name.
```
syntax = "proto2";
message Test {
}
```
It is a little tricky to get java full class name from the
`Descriptors.Descriptor`, so I use a simpler way that I extract the outer
prefix name from
[GitHub] [flink] flinkbot edited a comment on pull request #14376: [FLINK-18202][PB format] New Format of protobuf
flinkbot edited a comment on pull request #14376: URL: https://github.com/apache/flink/pull/14376#issuecomment-744252765 ## CI report: * a721a9d5f663117a203628547a19b33b23927ac2 UNKNOWN * 8f1051527a46aeba13aadc2db3c5f34d8aff78d2 UNKNOWN * 5cf0fa22d2ad2e24c9b4c088097999f2eff8f106 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27772) * a3dd67481e55eec8a8d1a6dad219e1aa0ba10ec0 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14376: [FLINK-18202][PB format] New Format of protobuf
flinkbot edited a comment on pull request #14376: URL: https://github.com/apache/flink/pull/14376#issuecomment-744252765 ## CI report: * a721a9d5f663117a203628547a19b33b23927ac2 UNKNOWN * 8f1051527a46aeba13aadc2db3c5f34d8aff78d2 UNKNOWN * 5cf0fa22d2ad2e24c9b4c088097999f2eff8f106 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27772) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14376: [FLINK-18202][PB format] New Format of protobuf
flinkbot edited a comment on pull request #14376: URL: https://github.com/apache/flink/pull/14376#issuecomment-744252765 ## CI report: * a721a9d5f663117a203628547a19b33b23927ac2 UNKNOWN * 8f1051527a46aeba13aadc2db3c5f34d8aff78d2 UNKNOWN * 5cf0fa22d2ad2e24c9b4c088097999f2eff8f106 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27772) * a3dd67481e55eec8a8d1a6dad219e1aa0ba10ec0 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] deadwind4 commented on pull request #18077: [FLINK-25240][core] update log4j2 version to 2.15.0
deadwind4 commented on pull request #18077: URL: https://github.com/apache/flink/pull/18077#issuecomment-991417465 > @EronWright committer and reviewer will backport to 1.13/1.14 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18083: [Draft][FLIP-198] Introduce working directory for Flink processes
flinkbot edited a comment on pull request #18083: URL: https://github.com/apache/flink/pull/18083#issuecomment-991128050 ## CI report: * 4c19e62271afe1533bd9cf2fa2036f2ebe0075d5 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27981) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] EronWright commented on pull request #18077: [FLINK-25240][core] update log4j2 version to 2.15.0
EronWright commented on pull request #18077: URL: https://github.com/apache/flink/pull/18077#issuecomment-991383576 @deadwind4 do you plan to backport to 1.13/1.14? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] JingGe commented on pull request #18082: [FLINK-24077][hbase2/HBaseConnectorITCase] fix flaky tests backport to 1.12
JingGe commented on pull request #18082: URL: https://github.com/apache/flink/pull/18082#issuecomment-991367079 @AHeise would you please help merging it? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18083: [Draft][FLIP-198] Introduce working directory for Flink processes
flinkbot edited a comment on pull request #18083: URL: https://github.com/apache/flink/pull/18083#issuecomment-991128050 ## CI report: * 8caa0c3160fd57ea0476c0dc0d54a1422ee8bcef Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27980) * 4c19e62271afe1533bd9cf2fa2036f2ebe0075d5 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27981) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18083: [Draft][FLIP-198] Introduce working directory for Flink processes
flinkbot edited a comment on pull request #18083: URL: https://github.com/apache/flink/pull/18083#issuecomment-991128050 ## CI report: * 2d966d5e2317c9a4bcbd398e0ce4d7d2d4800973 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27979) * 8caa0c3160fd57ea0476c0dc0d54a1422ee8bcef Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27980) * 4c19e62271afe1533bd9cf2fa2036f2ebe0075d5 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27981) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18083: [Draft][FLIP-198] Introduce working directory for Flink processes
flinkbot edited a comment on pull request #18083: URL: https://github.com/apache/flink/pull/18083#issuecomment-991128050 ## CI report: * 2d966d5e2317c9a4bcbd398e0ce4d7d2d4800973 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27979) * 8caa0c3160fd57ea0476c0dc0d54a1422ee8bcef Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27980) * 4c19e62271afe1533bd9cf2fa2036f2ebe0075d5 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-5263) Add docs about JVM sharing between tasks
[ https://issues.apache.org/jira/browse/FLINK-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-5263: -- Labels: auto-deprioritized-major stale-minor (was: auto-deprioritized-major) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Add docs about JVM sharing between tasks > > > Key: FLINK-5263 > URL: https://issues.apache.org/jira/browse/FLINK-5263 > Project: Flink > Issue Type: Improvement > Components: Documentation >Reporter: Ufuk Celebi >Priority: Minor > Labels: auto-deprioritized-major, stale-minor > > I've seen the question about why/how JVMs are shared between tasks and jobs > pop up on the mailing lists multiple times now. It makes sense to write down > which config options affect this, how to effectively configure one JVM per > task and job, and other related issues like PROs and CONs of Flink's approach > to things. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-5409) Separate Job API from the Runtime
[ https://issues.apache.org/jira/browse/FLINK-5409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-5409: -- Labels: auto-deprioritized-major stale-minor (was: auto-deprioritized-major) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Separate Job API from the Runtime > - > > Key: FLINK-5409 > URL: https://issues.apache.org/jira/browse/FLINK-5409 > Project: Flink > Issue Type: Improvement > Components: API / DataStream >Reporter: Matt Zimmer >Priority: Minor > Labels: auto-deprioritized-major, stale-minor > > Currently, all of the Flink runtime is visible to jobs. It will make > classpath management easier if only the minimum needed to create processing > jobs is loaded in the job ClassLoader. This would ideally be limited to the > job API and jars placed in a folder designated for sharing across jobs (as > /lib is now, but it contains the flink-dist_*.jar) and would exclude flink > runtime support classes and jars. > I've discussed this with [~till.rohrmann] and [~rmetzger]. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-8417) Support STSAssumeRoleSessionCredentialsProvider in FlinkKinesisConsumer
[
https://issues.apache.org/jira/browse/FLINK-8417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-8417:
--
Labels: auto-deprioritized-major auto-deprioritized-minor usability
(was: auto-deprioritized-major stale-minor usability)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Support STSAssumeRoleSessionCredentialsProvider in FlinkKinesisConsumer
> ---
>
> Key: FLINK-8417
> URL: https://issues.apache.org/jira/browse/FLINK-8417
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Kinesis
>Reporter: Tzu-Li (Gordon) Tai
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor,
> usability
>
> As discussed in ML:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kinesis-Connectors-With-Temporary-Credentials-td17734.html.
> Users need the functionality to access cross-account AWS Kinesis streams,
> using AWS Temporary Credentials [1].
> We should add support for
> {{AWSConfigConstants.CredentialsProvider.STSAssumeRole}}, which internally
> would use the {{STSAssumeRoleSessionCredentialsProvider}} [2] in
> {{AWSUtil#getCredentialsProvider(Properties)}}.
> [1] https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html
> [2]
> https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/STSAssumeRoleSessionCredentialsProvider.html
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-5301) Can't upload job via Web UI when using a proxy
[ https://issues.apache.org/jira/browse/FLINK-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-5301: -- Labels: auto-deprioritized-major stale-minor (was: auto-deprioritized-major) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Can't upload job via Web UI when using a proxy > -- > > Key: FLINK-5301 > URL: https://issues.apache.org/jira/browse/FLINK-5301 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Reporter: Mischa Krüger >Priority: Minor > Labels: auto-deprioritized-major, stale-minor > > Using DC/OS with Flink service in current development > (https://github.com/mesosphere/dcos-flink-service). For reproduction: > 1. Install a DC/OS cluster > 2. Follow the instruction on mentioned repo for setting up a universe server > with the flink app. > 3. Install the flink app via the universe > 4. Access the Web UI > 5. Upload a job > Experience: > The upload reaches 100%, and then says "Saving..." forever. > Upload works when using ssh forwarding to access the node directly serving > the Flink Web UI. > DC/OS uses a proxy to access the Web UI. The webpage is delivered by a > component called the "Admin Router". > Side note: > Interestingly also the new favicon does not appear. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-5351) Make the TypeExtractor support functions with more than 2 inputs
[ https://issues.apache.org/jira/browse/FLINK-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-5351: -- Labels: auto-deprioritized-major stale-minor (was: auto-deprioritized-major) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Make the TypeExtractor support functions with more than 2 inputs > > > Key: FLINK-5351 > URL: https://issues.apache.org/jira/browse/FLINK-5351 > Project: Flink > Issue Type: Improvement > Components: API / Type Serialization System, Library / Graph > Processing (Gelly) >Reporter: Vasia Kalavri >Priority: Minor > Labels: auto-deprioritized-major, stale-minor > > Currently, the The TypeExtractor doesn't support functions with more than 2 > inputs. We found that adding such support would be a useful feature for Gelly > in FLINK-5097. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-5139) consum kafka can set offset
[ https://issues.apache.org/jira/browse/FLINK-5139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-5139: -- Labels: auto-deprioritized-major stale-minor (was: auto-deprioritized-major) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > consum kafka can set offset > --- > > Key: FLINK-5139 > URL: https://issues.apache.org/jira/browse/FLINK-5139 > Project: Flink > Issue Type: Improvement > Components: Connectors / Kafka >Affects Versions: 1.1.3 >Reporter: jing lining >Priority: Minor > Labels: auto-deprioritized-major, stale-minor > > now get offset by this > org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08#getInvalidOffsetBehavior > class > whether can put in offset which is number -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-5264) Improve error message when assigning uid to intermediate operator
[ https://issues.apache.org/jira/browse/FLINK-5264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-5264: -- Labels: auto-deprioritized-major stale-minor (was: auto-deprioritized-major) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Improve error message when assigning uid to intermediate operator > - > > Key: FLINK-5264 > URL: https://issues.apache.org/jira/browse/FLINK-5264 > Project: Flink > Issue Type: Bug > Components: API / DataStream >Affects Versions: 1.1.3 >Reporter: Kostas Kloudas >Priority: Minor > Labels: auto-deprioritized-major, stale-minor > > Currently when trying to assign uid to an intermediate operator in a chain, > the error message just says that it is not the right place, without any > further information. > We could improve the message by telling explicitly to the user on which > operator to assign the uid. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-5279) Improve error message when trying to access keyed state in non-keyed operator
[
https://issues.apache.org/jira/browse/FLINK-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-5279:
--
Labels: auto-deprioritized-major stale-minor (was:
auto-deprioritized-major)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Improve error message when trying to access keyed state in non-keyed operator
> -
>
> Key: FLINK-5279
> URL: https://issues.apache.org/jira/browse/FLINK-5279
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
>Affects Versions: 1.1.3
>Reporter: Ufuk Celebi
>Priority: Minor
> Labels: auto-deprioritized-major, stale-minor
>
> When trying to access keyed state in a non-keyed operator, the error message
> is not very helpful. You get a trace like this:
> {code}
> java.lang.RuntimeException: Error while getting state
> ...
> Caused by: java.lang.RuntimeException: State key serializer has not been
> configured in the config. This operation cannot use partitioned state.
> {code}
> It will be helpful to users if this is more explicit to users, stating that
> the API can only be used on keyed streams, etc.
> If this applies to the current master as well, we should fix it there, too.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-5334) outdated scala SBT quickstart example
[ https://issues.apache.org/jira/browse/FLINK-5334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-5334: -- Labels: auto-deprioritized-major stale-minor (was: auto-deprioritized-major) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > outdated scala SBT quickstart example > - > > Key: FLINK-5334 > URL: https://issues.apache.org/jira/browse/FLINK-5334 > Project: Flink > Issue Type: Bug > Components: Documentation, Quickstarts >Affects Versions: 1.7.0, 1.8.2, 1.9.0 >Reporter: Nico Kruber >Priority: Minor > Labels: auto-deprioritized-major, stale-minor > > The scala quickstart set up via sbt-quickstart.sh or from the repository at > https://github.com/tillrohrmann/flink-project seems outdated compared to what > is set up with the maven archetype, e.g. Job.scala vs. BatchJob.scala and > StreamingJob.scala. This should probably be updated and also the hard-coded > example in sbt-quickstart.sh on the web page could be removed and download > the newest version instead as the mvn command does. > see > https://ci.apache.org/projects/flink/flink-docs-release-1.2/quickstart/scala_api_quickstart.html > for these two paths (SBT vs. Maven) -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-8302) Support shift_left and shift_right in TableAPI
[ https://issues.apache.org/jira/browse/FLINK-8302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-8302: -- Labels: auto-deprioritized-major auto-deprioritized-minor features pull-request-available (was: auto-deprioritized-major features pull-request-available stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Support shift_left and shift_right in TableAPI > -- > > Key: FLINK-8302 > URL: https://issues.apache.org/jira/browse/FLINK-8302 > Project: Flink > Issue Type: Improvement > Components: Table SQL / API >Reporter: DuBin >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > features, pull-request-available > Original Estimate: 48h > Remaining Estimate: 48h > > Add shift_left and shift_right support in TableAPI, shift_left(input, n) act > as input << n, shift_right(input, n) act as input >> n. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-8377) Document DataViews for user-defined aggregate functions
[
https://issues.apache.org/jira/browse/FLINK-8377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-8377:
--
Labels: auto-deprioritized-major auto-deprioritized-minor (was:
auto-deprioritized-major stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Document DataViews for user-defined aggregate functions
> ---
>
> Key: FLINK-8377
> URL: https://issues.apache.org/jira/browse/FLINK-8377
> Project: Flink
> Issue Type: Improvement
> Components: Documentation, Table SQL / API
>Reporter: Timo Walther
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor
>
> The {{DataView}} feature that has been implemented with FLINK-7206 has not
> been documented on our website. We should add some examples and explain the
> differences and limitations (where Flink state is used and where regular Java
> data structures are used).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-15642) Support to set JobManager readiness and liveness check
[
https://issues.apache.org/jira/browse/FLINK-15642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-15642:
---
Labels: auto-deprioritized-major auto-deprioritized-minor (was:
auto-deprioritized-major stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Support to set JobManager readiness and liveness check
> --
>
> Key: FLINK-15642
> URL: https://issues.apache.org/jira/browse/FLINK-15642
> Project: Flink
> Issue Type: New Feature
> Components: Deployment / Kubernetes
>Reporter: Yang Wang
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor
>
> The liveness of TaskManager will be controlled by Flink Master. When it
> failed, timeout, a new pod will be started to replace. We need to add a
> liveness check for JobManager.
>
> It just like what we could do in the yaml.
> {code:java}
> ...
> livenessProbe:
> tcpSocket:
> port: 6123
> initialDelaySeconds: 30
> periodSeconds: 60
> ...{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-15596) Support key-value messages for KafkaTableSink for flink SQL \Table
[ https://issues.apache.org/jira/browse/FLINK-15596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-15596: --- Labels: auto-deprioritized-major auto-deprioritized-minor (was: auto-deprioritized-major stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Support key-value messages for KafkaTableSink for flink SQL \Table > > > Key: FLINK-15596 > URL: https://issues.apache.org/jira/browse/FLINK-15596 > Project: Flink > Issue Type: Improvement > Components: Connectors / Kafka >Reporter: hehuiyuan >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor > Attachments: image-2020-01-15-13-58-00-535.png, > image-2020-01-15-13-58-39-073.png, image-2020-01-15-13-59-05-399.png > > > !image-2020-01-15-13-58-00-535.png|width=732,height=570! > !image-2020-01-15-13-58-39-073.png|width=583,height=115! > > !image-2020-01-15-13-59-05-399.png|width=782,height=306! > The data type in table is Row , we can add key indexes to get key from the > row. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-8358) Hostname used by DataDog metric reporter is not configurable
[
https://issues.apache.org/jira/browse/FLINK-8358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-8358:
--
Labels: auto-deprioritized-major auto-deprioritized-minor (was:
auto-deprioritized-major stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Hostname used by DataDog metric reporter is not configurable
>
>
> Key: FLINK-8358
> URL: https://issues.apache.org/jira/browse/FLINK-8358
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics
>Affects Versions: 1.4.0
>Reporter: Elias Levy
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor
>
> The hostname used by the DataDog metric reporter to report metrics is not
> configurable. This can problematic if the hostname that Flink uses is
> different from the hostname used by the system's DataDog agent.
> For instance, in our environment we use Chef, and using the DataDog Chef
> Handler, certain metadata such a host roles is associated with the hostname
> in the DataDog service. The hostname used to submit this metadata is the
> name we have given the host. But as Flink picks up the default name given by
> EC2 to the instance, metrics submitted by Flink to DataDog using that
> hostname are not associated with the tags derived from Chef.
> In the Job Manager we can avoid this issue by explicitly setting the config
> {{jobmanager.rpc.address}} to the hostname we desire. I attempted to do the
> name on the Task Manager by setting the {{taskmanager.hostname}} config, but
> DataDog does not seem to pick up that value.
> Digging through the code it seem the DD metric reporter get the hostname from
> the {{TaskManagerMetricGroup}} host variable, which seems to be set from
> {{taskManagerLocation.getHostname}}. That in turn seems to be by calling
> {{this.inetAddress.getCanonicalHostName()}}, which merely perform a reverse
> lookup on the IP address, and then calling {{NetUtils.getHostnameFromFQDN}}
> on the result. The later is further problematic because it result is a
> non-fully qualified hostname.
> More generally, there seems to be a need to specify the hostname of a JM or
> TM node that be reused across Flink components.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-5156) Consolidate streaming FieldAccessor functionality
[ https://issues.apache.org/jira/browse/FLINK-5156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-5156: -- Labels: auto-deprioritized-major stale-minor (was: auto-deprioritized-major) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Consolidate streaming FieldAccessor functionality > - > > Key: FLINK-5156 > URL: https://issues.apache.org/jira/browse/FLINK-5156 > Project: Flink > Issue Type: Technical Debt > Components: API / DataSet, API / DataStream, API / Type > Serialization System >Reporter: Márton Balassi >Priority: Minor > Labels: auto-deprioritized-major, stale-minor > > The streaming FieldAccessors (keyedStream.keyBy(...)) have slightly different > semantics compared to their batch counterparts. > Currently the streaming ones allow selecting a field within an array (which > might be dangerous as the array typeinfo does not contain the length of the > array, thus leading to a potential index out of bounds) and accept not only > "*", but also "0" to select a whole type. > This functionality should be either removed or documented. The latter can be > achieved by effectively reverting [1]. Note that said commit was squashed > before merging. > [1] > https://github.com/mbalassi/flink/commit/237f07eb113508703c980b14587d66970e7f6251 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-8444) Rework dependency setup docs
[
https://issues.apache.org/jira/browse/FLINK-8444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-8444:
--
Labels: auto-deprioritized-major auto-deprioritized-minor (was:
auto-deprioritized-major stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Rework dependency setup docs
>
>
> Key: FLINK-8444
> URL: https://issues.apache.org/jira/browse/FLINK-8444
> Project: Flink
> Issue Type: Improvement
> Components: Documentation
>Affects Versions: 1.4.0, 1.5.0
>Reporter: Chesnay Schepler
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor
>
> Taken from https://github.com/apache/flink/pull/5303:
> {quote}
> I would suggest to start thinking about the dependencies the following way:
> There are pure user-code projects where the Flink runtime is "provided"
> and they are started using an existing Flink setup (bin/flink run or REST
> entry point). This is the Framework Style.
> In the future, we will have "Flink as a Library" deployments, where users
> add something like flink-dist as a library to their program and then simply
> dockerize that Java application.
> Code can be run in the IDE or other similar style embedded forms. This is
> in some sense also a "Flink as a Library" deployment, but with selective
> (fewer) dependencies. The RocksDB issue applies only to this scenario here.
> To make this simpler for the users, it would be great to have not N different
> models that we talk about, but ideally only two: Framework Style and Library
> Style. We could for example start to advocate and document that users should
> always use flink-dist as their standard dependency - "provided" in the
> framework style deployment, "compile" in the library style deployment. That
> might be a really easy way to work with that. The only problem for the time
> being is that flink-dist is quite big and contains for example also optional
> dependencies like flink-table, which makes it more heavyweight for
> quickstarts. Maybe we can accept that as a trade-off for dependency
> simplicity.
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-8380) Dynamic BucketingSink paths based on ingested Kafka topics
[
https://issues.apache.org/jira/browse/FLINK-8380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-8380:
--
Labels: auto-deprioritized-major auto-deprioritized-minor (was:
auto-deprioritized-major stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Dynamic BucketingSink paths based on ingested Kafka topics
> --
>
> Key: FLINK-8380
> URL: https://issues.apache.org/jira/browse/FLINK-8380
> Project: Flink
> Issue Type: Improvement
> Components: API / DataStream
>Affects Versions: 1.4.0
>Reporter: Kyle Hamlin
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor
>
> Flink 1.4 released a feature that allows Kafka consumers to dynamically
> ingest topics based on a regex pattern. If a user wanted to use Flink as a
> simple (no transformations) but dynamic (auto topic discovery & auto output
> path generation) data persister they would currently only have half the tools
> to do so. I believe it would be a beneficial feature to allow users to not
> only define automatic topic discovery but also a way to dynamically
> incorporate those topics into a BucketingSink output path. For example:
> If I had three Kafka topics
> {code:java}
> select-topic-1
> ignore-topic-1
> select-topic-2
> {code}
> And my Kafka consumers regex only selected two topics
> {code:java}
> val consumer = new
> FlinkKafkaConsumer010[GenericRecord](Pattern.compile("select-.*?"), new
> MyDeserializer(), props)
> {code}
> Then the selected topics would appended to the beginning of the BucketingSink
> output path and any Bucketers partitions would follow
> {code:java}
> val sink = new BucketingSink[GenericRecord]("s3://my-bucket/")
> sink.setBucketer(new DateTimeBucketer[GenericRecord]("MMdd"))
> {code}
> The resulting output paths would be
> {code:java}
> s3://my-bucket/selected-topic1/MMdd/
> s3://my-bucket/selected-topic2/MMdd/
> {code}
> As new topics are discovered via the regex pattern (while the app is running)
> the set of BucketingSink output paths would grow.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-15678) Optimize producing primary key without row number in special Top 1
[
https://issues.apache.org/jira/browse/FLINK-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-15678:
---
Labels: auto-deprioritized-major auto-deprioritized-minor (was:
auto-deprioritized-major stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Optimize producing primary key without row number in special Top 1
> --
>
> Key: FLINK-15678
> URL: https://issues.apache.org/jira/browse/FLINK-15678
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
>Reporter: Jingsong Lee
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor
>
> {code:java}
> SELECT c1, c2, c3, c4
> FROM (
> SELECT *,ROW_NUMBER() OVER (PARTITION BY c1, c2, c3 ORDER BY c4 DESC) AS
> rownum
> FROM t
> ) WHERE rownum <= 1
> {code}
> This SQL is Top 1, Top N produce stream with primary keys contains row
> number, but your sql didn't select row number, so there is not primary key.
> But for Top 1, we can produce primary key, because row number is always 1.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-5422) Check for name collisions in the Rolling/BucketingSink
[ https://issues.apache.org/jira/browse/FLINK-5422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-5422: -- Labels: auto-deprioritized-major stale-minor (was: auto-deprioritized-major) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Check for name collisions in the Rolling/BucketingSink > -- > > Key: FLINK-5422 > URL: https://issues.apache.org/jira/browse/FLINK-5422 > Project: Flink > Issue Type: Bug > Components: Connectors / FileSystem >Affects Versions: 1.2.0 >Reporter: Kostas Kloudas >Priority: Minor > Labels: auto-deprioritized-major, stale-minor > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-5260) Add docs about how to react to cancellation
[ https://issues.apache.org/jira/browse/FLINK-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-5260: -- Labels: auto-deprioritized-major stale-minor (was: auto-deprioritized-major) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Add docs about how to react to cancellation > --- > > Key: FLINK-5260 > URL: https://issues.apache.org/jira/browse/FLINK-5260 > Project: Flink > Issue Type: Improvement > Components: Documentation >Reporter: Ufuk Celebi >Priority: Minor > Labels: auto-deprioritized-major, stale-minor > > Task cancellation with operators that work against external services can be a > source of confusion for users. Since users need to cooperate for this, I > would like to see some docs added about how the user can do this (which > callbacks are available, how to react to interrupts, etc.) -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-8416) Kinesis consumer doc examples should demonstrate preferred default credentials provider
[ https://issues.apache.org/jira/browse/FLINK-8416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-8416: -- Labels: auto-deprioritized-major auto-deprioritized-minor (was: auto-deprioritized-major stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Kinesis consumer doc examples should demonstrate preferred default > credentials provider > --- > > Key: FLINK-8416 > URL: https://issues.apache.org/jira/browse/FLINK-8416 > Project: Flink > Issue Type: Improvement > Components: Connectors / Kinesis, Documentation >Reporter: Tzu-Li (Gordon) Tai >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor > > The Kinesis consumer docs > [here](https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/connectors/kinesis.html#kinesis-consumer) > demonstrate providing credentials by explicitly supplying the AWS Access ID > and Key. > The always preferred approach for AWS, unless running locally, is to > automatically fetch the shipped credentials from the AWS environment. > That is actually the default behaviour of the Kinesis consumer, so the docs > should demonstrate that more clearly. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18083: [Draft][FLIP-198] Introduce working directory for Flink processes
flinkbot edited a comment on pull request #18083: URL: https://github.com/apache/flink/pull/18083#issuecomment-991128050 ## CI report: * 2d966d5e2317c9a4bcbd398e0ce4d7d2d4800973 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27979) * 8caa0c3160fd57ea0476c0dc0d54a1422ee8bcef Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27980) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18083: [Draft][FLIP-198] Introduce working directory for Flink processes
flinkbot edited a comment on pull request #18083: URL: https://github.com/apache/flink/pull/18083#issuecomment-991128050 ## CI report: * 737062727b800d068521ec30b89d5fd3a03fa2a3 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27977) * 2d966d5e2317c9a4bcbd398e0ce4d7d2d4800973 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27979) * 8caa0c3160fd57ea0476c0dc0d54a1422ee8bcef Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27980) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18083: [Draft][FLIP-198] Introduce working directory for Flink processes
flinkbot edited a comment on pull request #18083: URL: https://github.com/apache/flink/pull/18083#issuecomment-991128050 ## CI report: * 737062727b800d068521ec30b89d5fd3a03fa2a3 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27977) * 2d966d5e2317c9a4bcbd398e0ce4d7d2d4800973 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27979) * 8caa0c3160fd57ea0476c0dc0d54a1422ee8bcef UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17774: [FLINK-24611] Prevent JM from discarding state on checkpoint abortion
flinkbot edited a comment on pull request #17774: URL: https://github.com/apache/flink/pull/17774#issuecomment-966933536 ## CI report: * 6445d1c6fb469ef7ca57bdee129a52f0e5bb369b Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27975) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-web] sjwiesman commented on a change in pull request #488: add concrete flink configuration to CVE blog post
sjwiesman commented on a change in pull request #488: URL: https://github.com/apache/flink-web/pull/488#discussion_r766932872 ## File path: _posts/2021-12-10-log4j-cve.md ## @@ -13,8 +13,15 @@ It is by now tracked under [CVE-2021-44228](https://nvd.nist.gov/vuln/detail/CVE Apache Flink is bundling a version of Log4j that is affected by this vulnerability. We recommend users to follow the [advisory](https://logging.apache.org/log4j/2.x/security.html) of the Apache Log4j Community. -For Apache Flink this currently translates to "setting system property `log4j2.formatMsgNoLookups` to `true`" until Log4j has been upgraded to 2.15.0 in Apache Flink. +For Apache Flink this currently translates to setting the following property in your flink-conf.yaml: +```yaml +env.java.opts: -Dlog4j2.formatMsgNoLookups=true +``` + +If you are already setting `env.java.opts.jobmanager` or `env.java.opts.taskmanager`, you should instead add the system change to those existing parameter lists. Review comment: ```suggestion If you are already setting `env.java.opts.jobmanager`, `env.java.opts.taskmanager`, `env.java.opts.client`, or `env.java.opts.historyserver` you should instead add the system change to those existing parameter lists. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18083: [Draft][FLIP-198] Introduce working directory for Flink processes
flinkbot edited a comment on pull request #18083: URL: https://github.com/apache/flink/pull/18083#issuecomment-991128050 ## CI report: * 737062727b800d068521ec30b89d5fd3a03fa2a3 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27977) * 2d966d5e2317c9a4bcbd398e0ce4d7d2d4800973 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27979) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18083: [Draft][FLIP-198] Introduce working directory for Flink processes
flinkbot edited a comment on pull request #18083: URL: https://github.com/apache/flink/pull/18083#issuecomment-991128050 ## CI report: * 737062727b800d068521ec30b89d5fd3a03fa2a3 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27977) * 2d966d5e2317c9a4bcbd398e0ce4d7d2d4800973 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18017: [FLINK-25171] Validation of duplicate fields in ddl sql
flinkbot edited a comment on pull request #18017: URL: https://github.com/apache/flink/pull/18017#issuecomment-986500405 ## CI report: * ac179499cd39e4d3c8abea70dbb6209a45f6dee3 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27978) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18083: [Draft][FLIP-198] Introduce working directory for Flink processes
flinkbot edited a comment on pull request #18083: URL: https://github.com/apache/flink/pull/18083#issuecomment-991128050 ## CI report: * 737062727b800d068521ec30b89d5fd3a03fa2a3 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27977) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18082: [FLINK-24077][hbase2/HBaseConnectorITCase] fix flaky tests backport to 1.12
flinkbot edited a comment on pull request #18082: URL: https://github.com/apache/flink/pull/18082#issuecomment-990938904 ## CI report: * c9c237e13e5e43f0f2e5a70967ce3bf331c3bac8 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27965) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18082: [FLINK-24077][hbase2/HBaseConnectorITCase] fix flaky tests backport to 1.12
flinkbot edited a comment on pull request #18082: URL: https://github.com/apache/flink/pull/18082#issuecomment-990938904 ## CI report: * c9c237e13e5e43f0f2e5a70967ce3bf331c3bac8 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27965) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18082: [FLINK-24077][hbase2/HBaseConnectorITCase] fix flaky tests backport to 1.12
flinkbot edited a comment on pull request #18082: URL: https://github.com/apache/flink/pull/18082#issuecomment-990938904 ## CI report: * c9c237e13e5e43f0f2e5a70967ce3bf331c3bac8 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27965) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] JingGe commented on pull request #18082: [FLINK-24077][hbase2/HBaseConnectorITCase] fix flaky tests backport to 1.12
JingGe commented on pull request #18082: URL: https://github.com/apache/flink/pull/18082#issuecomment-991193943 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] pnowojski commented on a change in pull request #17774: [FLINK-24611] Prevent JM from discarding state on checkpoint abortion
pnowojski commented on a change in pull request #17774:
URL: https://github.com/apache/flink/pull/17774#discussion_r766826678
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java
##
@@ -1084,6 +1083,15 @@ public boolean receiveAcknowledgeMessage(
final PendingCheckpoint checkpoint =
pendingCheckpoints.get(checkpointId);
+if (message.getSubtaskState() != null) {
+// Register shared state regardless of checkpoint state and
task ACK state.
+// This way, shared state is
+// 1. kept if the message is late or state will be used by the
task otherwise
+// 2. removed eventually upon checkpoint subsumption (or job
cancellation)
+message.getSubtaskState()
+
.registerSharedStates(completedCheckpointStore.getRegistry(), checkpointId);
+}
+
Review comment:
Is this a valid change with the current (from the perspective of this
commit) reference counting method?
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/SharedStateRegistryImpl.java
##
@@ -95,48 +100,36 @@ public Result registerReference(
state,
entry.stateHandle);
}
-entry.increaseReferenceCount();
+entry.lastUsedCheckpointID = Math.max(checkpointID,
entry.lastUsedCheckpointID);
Review comment:
nitty nit: I would extra this to a method inside `SharedStateEntry` for
the sake of encapsulation.
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/DeactivatedCheckpointCompletedCheckpointStore.java
##
@@ -69,6 +69,11 @@ public SharedStateRegistry getSharedStateRegistry() {
throw unsupportedOperationException();
Review comment:
Is the javadoc of this class incorrect?
```
* This class represents a {@link CompletedCheckpointStore} if checkpointing
has been enabled.
```
Shouldn't it be
```
* This class represents a {@link CompletedCheckpointStore} if checkpointing
has been disabled.
```
?
Note that adding `getRegistry()` the method also brakes that java doc even
further.
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinatorTest.java
##
@@ -142,6 +145,49 @@
/** Tests for the checkpoint coordinator. */
public class CheckpointCoordinatorTest extends TestLogger {
+@Test
+public void testSharedStateNotDiscaredOnAbort() throws Exception {
+JobVertexID v1 = new JobVertexID(), v2 = new JobVertexID();
+
+ExecutionGraph graph =
+new
CheckpointCoordinatorTestingUtils.CheckpointExecutionGraphBuilder()
+.addJobVertex(v1)
+.addJobVertex(v2)
+.build();
+
+CheckpointCoordinator coordinator =
+new CheckpointCoordinatorBuilder()
+.setExecutionGraph(graph)
+.setTimer(manuallyTriggeredScheduledExecutor)
+.build();
+coordinator.startCheckpointScheduler();
+
+CompletableFuture cpFuture =
coordinator.triggerCheckpoint(true);
+manuallyTriggeredScheduledExecutor.triggerAll();
+cpFuture.getNow(null);
+
+TestingStreamStateHandle metaState = handle();
+TestingStreamStateHandle privateState = handle();
+TestingStreamStateHandle sharedState = handle();
+
+ack(1L, coordinator, v1, graph, metaState, privateState, sharedState);
+nack(1L, coordinator, v2, graph);
Review comment:
Can you rename the methods to something more descriptive?
`acknowledgeCheckpoint()` `declineCheckpoint()`?
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CompletedCheckpointStore.java
##
@@ -113,4 +113,6 @@ default long getLatestCheckpointId() {
/** Returns the {@link SharedStateRegistry} used to register the shared
state. */
SharedStateRegistry getSharedStateRegistry();
+
+SharedStateRegistry getRegistry();
Review comment:
Why the two? It looks like they are returning the same thing, except of
`DeactivatedCheckpointCompletedCheckpointStore`.
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/DeactivatedCheckpointCompletedCheckpointStore.java
##
@@ -69,6 +69,11 @@ public SharedStateRegistry getSharedStateRegistry() {
throw unsupportedOperationException();
}
+@Override
+public SharedStateRegistry getRegistry() {
+return SharedStateRegistry.NO_OP;
+}
+
Review comment:
Why do we need this `NO_OP` value here? Why not throw an exception like
in the existing
`DeactivatedCheckpointCompletedCheckpointStore#getSharedStateRegistry`?
##
[GitHub] [flink] flinkbot edited a comment on pull request #18082: [FLINK-24077][hbase2/HBaseConnectorITCase] fix flaky tests backport to 1.12
flinkbot edited a comment on pull request #18082: URL: https://github.com/apache/flink/pull/18082#issuecomment-990938904 ## CI report: * c9c237e13e5e43f0f2e5a70967ce3bf331c3bac8 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27965) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18071: [FLINK-24077][hbase2/HBaseConnectorITCase] fix flaky tests backport to 1.13
flinkbot edited a comment on pull request #18071: URL: https://github.com/apache/flink/pull/18071#issuecomment-990316009 ## CI report: * d72c4a290de9373b3868237940976276499b100b Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27966) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17785: [FLINK-24580][Connectors/Kinesis] Make ConnectTimeoutException recoverable
flinkbot edited a comment on pull request #17785: URL: https://github.com/apache/flink/pull/17785#issuecomment-967470963 ## CI report: * fc88ab25f463422610629a737ad087e427c01828 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27974) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17774: [FLINK-24611] Prevent JM from discarding state on checkpoint abortion
flinkbot edited a comment on pull request #17774: URL: https://github.com/apache/flink/pull/17774#issuecomment-966933536 ## CI report: * 70f0705383e67a4ffa9f577a48165a0657a8dcb7 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27972) * 6445d1c6fb469ef7ca57bdee129a52f0e5bb369b Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27975) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-22775) CassandraConnectorITCase.testCassandraTableSink Fail
[
https://issues.apache.org/jira/browse/FLINK-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17457284#comment-17457284
]
Till Rohrmann commented on FLINK-22775:
---
Can we backport the fix to {{1.14}} and {{1.13}} [~echauchot]?
> CassandraConnectorITCase.testCassandraTableSink Fail
>
>
> Key: FLINK-22775
> URL: https://issues.apache.org/jira/browse/FLINK-22775
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Cassandra
>Affects Versions: 1.14.0
>Reporter: Guowei Ma
>Assignee: Etienne Chauchot
>Priority: Minor
> Labels: auto-deprioritized-major, pull-request-available,
> test-stability
> Fix For: 1.15.0, 1.14.1
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=18328=logs=ba53eb01-1462-56a3-8e98-0dd97fbcaab5=bfbc6239-57a0-5db0-63f3-41551b4f7d51=14105
> {code:java}
> 2021-05-25T23:03:44.0756266Z May 25 23:03:44 [ERROR]
> testCassandraTableSink(org.apache.flink.streaming.connectors.cassandra.CassandraConnectorITCase)
> Time elapsed: 13.673 s <<< ERROR!
> 2021-05-25T23:03:44.0757635Z May 25 23:03:44
> java.util.concurrent.ExecutionException:
> org.apache.flink.table.api.TableException: Failed to wait job finish
> 2021-05-25T23:03:44.0760262Z May 25 23:03:44 at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> 2021-05-25T23:03:44.0761504Z May 25 23:03:44 at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
> 2021-05-25T23:03:44.0762906Z May 25 23:03:44 at
> org.apache.flink.table.api.internal.TableResultImpl.awaitInternal(TableResultImpl.java:129)
> 2021-05-25T23:03:44.0763878Z May 25 23:03:44 at
> org.apache.flink.table.api.internal.TableResultImpl.await(TableResultImpl.java:92)
> 2021-05-25T23:03:44.0764918Z May 25 23:03:44 at
> org.apache.flink.streaming.connectors.cassandra.CassandraConnectorITCase.testCassandraTableSink(CassandraConnectorITCase.java:520)
> 2021-05-25T23:03:44.0768225Z May 25 23:03:44 at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2021-05-25T23:03:44.0769100Z May 25 23:03:44 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2021-05-25T23:03:44.0769917Z May 25 23:03:44 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2021-05-25T23:03:44.0770645Z May 25 23:03:44 at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2021-05-25T23:03:44.0771387Z May 25 23:03:44 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 2021-05-25T23:03:44.0772228Z May 25 23:03:44 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 2021-05-25T23:03:44.0773541Z May 25 23:03:44 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 2021-05-25T23:03:44.0774367Z May 25 23:03:44 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 2021-05-25T23:03:44.0775246Z May 25 23:03:44 at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 2021-05-25T23:03:44.0776088Z May 25 23:03:44 at
> org.apache.flink.testutils.junit.RetryRule$RetryOnExceptionStatement.evaluate(RetryRule.java:192)
> 2021-05-25T23:03:44.0776946Z May 25 23:03:44 at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> 2021-05-25T23:03:44.0777685Z May 25 23:03:44 at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 2021-05-25T23:03:44.0778447Z May 25 23:03:44 at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 2021-05-25T23:03:44.0779110Z May 25 23:03:44 at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 2021-05-25T23:03:44.0779893Z May 25 23:03:44 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 2021-05-25T23:03:44.0780744Z May 25 23:03:44 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 2021-05-25T23:03:44.0781493Z May 25 23:03:44 at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2021-05-25T23:03:44.0782154Z May 25 23:03:44 at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2021-05-25T23:03:44.0782899Z May 25 23:03:44 at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2021-05-25T23:03:44.0783576Z May 25 23:03:44 at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 2021-05-25T23:03:44.0784312Z May 25 23:03:44 at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 2021-05-25T23:03:44.0785020Z May 25 23:03:44 at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 2021-05-25T23:03:44.0785815Z May 25 23:03:44 at
>
[GitHub] [flink] deadwind4 commented on pull request #16635: [hotfix][connector-jdbc] fix postgres unit test typo
deadwind4 commented on pull request #16635: URL: https://github.com/apache/flink/pull/16635#issuecomment-991145925 @MartijnVisser please review it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] deadwind4 removed a comment on pull request #18059: [FLINK-25224][filesystem] Bump Hadoop version to 2.8.4
deadwind4 removed a comment on pull request #18059: URL: https://github.com/apache/flink/pull/18059#issuecomment-991140514 LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] jelly-1203 commented on a change in pull request #18017: [FLINK-25171] Validation of duplicate fields in ddl sql
jelly-1203 commented on a change in pull request #18017:
URL: https://github.com/apache/flink/pull/18017#discussion_r766554432
##
File path:
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/operations/MergeTableLikeUtil.java
##
@@ -426,6 +433,14 @@ private void appendDerivedColumns(
}
}
+Set physicalFieldNames =
physicalFieldNamesToTypes.keySet();
+Set metadataFieldNames =
metadataFieldNamesToTypes.keySet();
+final Set result = new
LinkedHashSet<>(physicalFieldNames);
+result.retainAll(metadataFieldNames);
+if (!result.isEmpty()) {
+throw new ValidationException(
+"A field name conflict exists between a field
of the regular type and a field of the Metadata type.");
+}
Review comment:
> may be we can just check duplication when put the new Column to the
columns, at the end of this function?
hi @wenlong88
Thanks for your review and comments, I think we can adjust the determination
of whether a SqlRegularColumn type field is duplicated with other types of
fields to check if a new column is duplicated when put into the column at the
end of this function.
But accessibleFieldNamesToTypes heavy logic to this place, I think it is
necessary, because, when creating computecolcolumn if metadata columns in the
former, repetitive physical column in the bottom, In the use of
accessibleFieldNamesToTypes putAll, metadata list field covers physical column,
if the user want, according to the calculated regularColumn you don't need to
do so.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] jelly-1203 commented on a change in pull request #18017: [FLINK-25171] Validation of duplicate fields in ddl sql
jelly-1203 commented on a change in pull request #18017:
URL: https://github.com/apache/flink/pull/18017#discussion_r766554531
##
File path:
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/operations/MergeTableLikeUtil.java
##
@@ -426,6 +433,14 @@ private void appendDerivedColumns(
}
}
+Set physicalFieldNames =
physicalFieldNamesToTypes.keySet();
+Set metadataFieldNames =
metadataFieldNamesToTypes.keySet();
+final Set result = new
LinkedHashSet<>(physicalFieldNames);
+result.retainAll(metadataFieldNames);
+if (!result.isEmpty()) {
+throw new ValidationException(
+"A field name conflict exists between a field
of the regular type and a field of the Metadata type.");
+}
Review comment:
> may be we can just check duplication when put the new Column to the
columns, at the end of this function?
hi @wenlong88
Thanks for your review and comment. I do not think duplication can be
checked when putting the new Column to the columns, at the end of this
function. The reasons are as follows:
1. If computeColumn or MetadataColumn uses overwrite's Merge strategy,
duplicate fields are allowed for the same type.
2. To add physical accessibleFieldNamesToTypes columns and metadata columns,
if metadataColumn before, and repeated physical column, leads to putAll,
Metadata columns overwrite duplicate physical columns, which can result in a
generated computeColumn that is not as expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] deadwind4 commented on pull request #18059: [FLINK-25224][filesystem] Bump Hadoop version to 2.8.4
deadwind4 commented on pull request #18059: URL: https://github.com/apache/flink/pull/18059#issuecomment-991140514 LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18017: [FLINK-25171] Validation of duplicate fields in ddl sql
flinkbot edited a comment on pull request #18017: URL: https://github.com/apache/flink/pull/18017#issuecomment-986500405 ## CI report: * a3995c2e238f415d94360e70df3fc08b3c15e382 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27951) * ac179499cd39e4d3c8abea70dbb6209a45f6dee3 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27978) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18017: [FLINK-25171] Validation of duplicate fields in ddl sql
flinkbot edited a comment on pull request #18017: URL: https://github.com/apache/flink/pull/18017#issuecomment-986500405 ## CI report: * a3995c2e238f415d94360e70df3fc08b3c15e382 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27951) * ac179499cd39e4d3c8abea70dbb6209a45f6dee3 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18077: [FLINK-25240][core] update log4j2 version to 2.15.0
flinkbot edited a comment on pull request #18077: URL: https://github.com/apache/flink/pull/18077#issuecomment-990863888 ## CI report: * aec502fbe28c7dc9cf457b10972f2570582bd551 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27968) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18083: [Draft][FLIP-198] Introduce working directory for Flink processes
flinkbot edited a comment on pull request #18083: URL: https://github.com/apache/flink/pull/18083#issuecomment-991128050 ## CI report: * 737062727b800d068521ec30b89d5fd3a03fa2a3 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27977) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #18083: [Draft][FLIP-198] Introduce working directory for Flink processes
flinkbot commented on pull request #18083: URL: https://github.com/apache/flink/pull/18083#issuecomment-991129881 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 737062727b800d068521ec30b89d5fd3a03fa2a3 (Fri Dec 10 16:49:12 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **Invalid pull request title: No valid Jira ID provided** Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #18083: [Draft][FLIP-198] Introduce working directory for Flink processes
flinkbot commented on pull request #18083: URL: https://github.com/apache/flink/pull/18083#issuecomment-991128050 ## CI report: * 737062727b800d068521ec30b89d5fd3a03fa2a3 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] tillrohrmann opened a new pull request #18083: [Draft][FLIP-198] Introduce working directory for Flink processes
tillrohrmann opened a new pull request #18083: URL: https://github.com/apache/flink/pull/18083 This PR is a draft PR for [FLIP-198](https://cwiki.apache.org/confluence/x/ZZiqCw). It consists of several commits that implement the proposed changes: e5b561457cd15d641f891e00067f33276d69c450: Make JobManager process resource id configurable via jobmanager.resource-id This commit introduces a new configuration option `jobmanager.resource-id` to control the `ResourceID` of the JobManager process. 34e666722a01173a6f82c7c26779111dbc4bbfae: Introduce ClusterOptions.PROCESS_WORKING_DIR_BASE for configuring a process directory This commit introduces the configuration options `process.working-dir`, `process.jobmanager.working-dir` and `process.taskmanager.working-dir` to configure a working directory for the JobManager and TaskManager process. The resulting working directory will be /jm_ and /tm_. e284bfd733ee61a1eb8535bba766b1ec6e231024: Clear tmp directory when setting up the working directory This commit introduces a `tmp` directory that will be created in each working directory. Moreover, this tmp directory is cleaned whenever the process restarts. 799e753786d320f7dfb3af3057900d35a8d45c32: Let taskmanager.state.local.root-dirs default to /localState This commit makes `/localState` the default for `taskmanager.state.local.root-dirs`. That way the local state will be stored in the working directory if not explicitly configured. e74644ac5eb0c9896414b1b7185301e86700f2d6: Set state.backend.rocksdb.localdir to default to /tmp This commit makes `/tmp` the default for `state.backend.rocksdb.localdir` if not explicitly configured. This is a behaviour change because before the RocksDB local dir would default to io.tmp.dirs. With this change, the default RocksDB working directory is colocated on the same volume as the local state directory. 6d30d93d196aa582117ef3cac5c3232c8f63f425: Let blob.storage.directory default to /blobStorage This commit makes `/blobStorage` the default for the `blob.storage.directory` configuration option. This allows to reuse stored blobs across process restarts (assuming that neither the `BlobServer` nor the `BlobCacheService` delete this directory). ed8dceab45774b6099bf62167ada575f7e640c01: Introduce borrowed and owned storage directory to BlobServer/Cache for better clean up This commit allows to pass a borrowed storage directory to the BlobServer/Cache. Borrowed storage directories won't be automatically cleaned up when shutting the BlobServer/Cache down. This allows to move the lifecycle management of these directories to the owner of the BlobServer/Cache. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-25257) Make the TaskExecutor not depend on concrete BlobServiceCache implementation
[
https://issues.apache.org/jira/browse/FLINK-25257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann closed FLINK-25257.
-
Resolution: Fixed
Fixed via cf1e8c39111378735e4c05a5edb3bd713229bb08
> Make the TaskExecutor not depend on concrete BlobServiceCache implementation
>
>
> Key: FLINK-25257
> URL: https://issues.apache.org/jira/browse/FLINK-25257
> Project: Flink
> Issue Type: Technical Debt
> Components: Runtime / Coordination
>Affects Versions: 1.15.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.0
>
>
> In order to better evolve the {{TaskExecutor}} component, I suggest to not
> make it depend on the concrete {{BlobServiceCache}} implementation. Instead I
> suggest to introduce a {{TaskExecutorBlobCacheService}} interface that can be
> implemented by the {{BlobServiceCache}}. Additionally, should make the
> {{TaskExecutor}} independent of the {{PermanentBlobCache}} and
> {{TransientBlobCache}} implementations.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-25257) Make the TaskExecutor not depend on concrete BlobServiceCache implementation
[
https://issues.apache.org/jira/browse/FLINK-25257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann updated FLINK-25257:
--
Issue Type: Technical Debt (was: Improvement)
> Make the TaskExecutor not depend on concrete BlobServiceCache implementation
>
>
> Key: FLINK-25257
> URL: https://issues.apache.org/jira/browse/FLINK-25257
> Project: Flink
> Issue Type: Technical Debt
> Components: Runtime / Coordination
>Affects Versions: 1.15.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.0
>
>
> In order to better evolve the {{TaskExecutor}} component, I suggest to not
> make it depend on the concrete {{BlobServiceCache}} implementation. Instead I
> suggest to introduce a {{TaskExecutorBlobCacheService}} interface that can be
> implemented by the {{BlobServiceCache}}. Additionally, should make the
> {{TaskExecutor}} independent of the {{PermanentBlobCache}} and
> {{TransientBlobCache}} implementations.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] tillrohrmann closed pull request #18081: [FLINK-25257][TE] Let TaskExecutor use TaskExecutorBlobService interface instead of concrete BlobCacheService implementation
tillrohrmann closed pull request #18081: URL: https://github.com/apache/flink/pull/18081 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18078: [FLINK-20195][coordination] Deduplicate jobs for overview
flinkbot edited a comment on pull request #18078: URL: https://github.com/apache/flink/pull/18078#issuecomment-990888233 ## CI report: * 787da020d7ccc3ed2190090a5bb64e7f86efe8a1 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27964) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-web] zentol commented on a change in pull request #487: Add blog post about Log4j Zero Day
zentol commented on a change in pull request #487: URL: https://github.com/apache/flink-web/pull/487#discussion_r766811104 ## File path: _posts/2021-12-10-log4j-cve.md ## @@ -0,0 +1,21 @@ +--- +layout: post +title: "Advise on Apache Log4j Zero Day (CVE-2021-44228)" +date: 2021-12-10 00:00:00 +authors: +- knaufk: + name: "Konstantin Knauf" +excerpt: "Advise on Apache Log4j Zero Day (CVE-2021-44228)" +--- + +Yesterday, a new Zero Day for Apache Log4j was [reported](https://www.cyberkendra.com/2021/12/apache-log4j-vulnerability-details-and.html). +It is by now tracked under [CVE-2021-44228](https://nvd.nist.gov/vuln/detail/CVE-2021-44228). + +Apache Flink is bundling a version of Log4j that is affeced by this vulnerability. +We recommend users to follow the [adivsory](https://logging.apache.org/log4j/2.x/security.html) of the Apache Log4j Community. Review comment: adivsory -> advisory -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18051: [FLINK-25159][tests] Streamline E2E surefire setup
flinkbot edited a comment on pull request #18051: URL: https://github.com/apache/flink/pull/18051#issuecomment-988626420 ## CI report: * 4aa8e0b8440840e20dcc57f82acf73dd26f499c3 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27963) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-web] knaufk merged pull request #487: Add blog post about Log4j Zero Day
knaufk merged pull request #487: URL: https://github.com/apache/flink-web/pull/487 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-web] knaufk commented on a change in pull request #487: Add blog post about Log4j Zero Day
knaufk commented on a change in pull request #487: URL: https://github.com/apache/flink-web/pull/487#discussion_r766806463 ## File path: _posts/2021-12-10-log4j-cve.md ## @@ -0,0 +1,21 @@ +--- +layout: post +title: "Advise on Apache Log4j Zero Day (CVE-2021-44228)" +date: 2021-12-10 00:00:00 +authors: +- knaufk: + name: "Konstantin Knauf" +excerpt: "Advise on Apache Log4j Zero Day (CVE-2021-44228)" +--- + +Yesterday, a new Zero Day for Apache Log4j was [reported](https://www.cyberkendra.com/2021/12/apache-log4j-vulnerability-details-and.html). +It is by now tracked under [CVE-2021-44228](https://nvd.nist.gov/vuln/detail/CVE-2021-44228). + +Apache Flink is bundling a version of Log4j that is affeced by this vulnerability. +We recommend users to follow the [adivsory](https://logging.apache.org/log4j/2.x/security.html) of the Apache Log4j Community. Review comment: ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-25238) flink iceberg source reading array types fail with Cast Exception
[ https://issues.apache.org/jira/browse/FLINK-25238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Wiesman closed FLINK-25238. Resolution: Won't Fix > flink iceberg source reading array types fail with Cast Exception > - > > Key: FLINK-25238 > URL: https://issues.apache.org/jira/browse/FLINK-25238 > Project: Flink > Issue Type: Bug > Components: Table SQL / Runtime >Affects Versions: 1.13.2 >Reporter: Praneeth Ramesh >Priority: Major > Attachments: Screen Shot 2021-12-09 at 6.58.56 PM.png, Screen Shot > 2021-12-09 at 7.04.10 PM.png > > > I have a stream with iceberg table as a source. I have few columns of array > types in the table. > I try to read using iceberg connector. > Flink Version : 1.13.2 > Iceberg Flink Version: 0.12.1 > > I see the error as below. > java.lang.ClassCastException: class > org.apache.iceberg.flink.data.FlinkParquetReaders$ReusableArrayData cannot be > cast to class org.apache.flink.table.data.ColumnarArrayData > (org.apache.iceberg.flink.data.FlinkParquetReaders$ReusableArrayData and > org.apache.flink.table.data.ColumnarArrayData are in unnamed module of loader > 'app') > at > org.apache.flink.table.runtime.typeutils.ArrayDataSerializer.copy(ArrayDataSerializer.java:90) > at > org.apache.flink.table.runtime.typeutils.ArrayDataSerializer.copy(ArrayDataSerializer.java:47) > at > org.apache.flink.table.runtime.typeutils.RowDataSerializer.copyRowData(RowDataSerializer.java:170) > at > org.apache.flink.table.runtime.typeutils.RowDataSerializer.copy(RowDataSerializer.java:131) > at > org.apache.flink.table.runtime.typeutils.RowDataSerializer.copy(RowDataSerializer.java:48) > at > org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:69) > at > org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:46) > at > org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:26) > at > org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:50) > at > org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:28) > at > org.apache.flink.streaming.api.operators.StreamSourceContexts$ManualWatermarkContext.processAndCollect(StreamSourceContexts.java:317) > at > org.apache.flink.streaming.api.operators.StreamSourceContexts$WatermarkContext.collect(StreamSourceContexts.java:411) > at > org.apache.iceberg.flink.source.StreamingReaderOperator.processSplits(StreamingReaderOperator.java:155) > at > org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) > at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) > at > org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:359) > at > org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:323) > at > org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:202) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:681) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.executeInvoke(StreamTask.java:636) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:647) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:620) > at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:779) > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566) > at java.base/java.lang.Thread.run(Thread.java:834) > > Could be same issue as https://issues.apache.org/jira/browse/FLINK-21247 > except it happening for another type. > I see that Iceberg use custom types other than the types from > org.apache.flink.table.data like > org.apache.iceberg.flink.data.FlinkParquetReaders.ReusableArrayData and these > types are not handled in > org.apache.flink.table.runtime.typeutils.ArrayDataSerializer > !Screen Shot 2021-12-09 at 6.58.56 PM.png! > Just to try I changed the above code to handle the iceberg type as a binary > Array and built it locally and used in my application and that worked. > > !Screen Shot 2021-12-09 at 7.04.10 PM.png! > Not sure if this is already handled in some newer versions. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-25238) flink iceberg source reading array types fail with Cast Exception
[ https://issues.apache.org/jira/browse/FLINK-25238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17457235#comment-17457235 ] Seth Wiesman commented on FLINK-25238: -- The Iceberg Sink is maintained by the Apache Iceberg community. I am going to close this ticket and recommend you reach out to them for assistance. > flink iceberg source reading array types fail with Cast Exception > - > > Key: FLINK-25238 > URL: https://issues.apache.org/jira/browse/FLINK-25238 > Project: Flink > Issue Type: Bug > Components: Table SQL / Runtime >Affects Versions: 1.13.2 >Reporter: Praneeth Ramesh >Priority: Major > Attachments: Screen Shot 2021-12-09 at 6.58.56 PM.png, Screen Shot > 2021-12-09 at 7.04.10 PM.png > > > I have a stream with iceberg table as a source. I have few columns of array > types in the table. > I try to read using iceberg connector. > Flink Version : 1.13.2 > Iceberg Flink Version: 0.12.1 > > I see the error as below. > java.lang.ClassCastException: class > org.apache.iceberg.flink.data.FlinkParquetReaders$ReusableArrayData cannot be > cast to class org.apache.flink.table.data.ColumnarArrayData > (org.apache.iceberg.flink.data.FlinkParquetReaders$ReusableArrayData and > org.apache.flink.table.data.ColumnarArrayData are in unnamed module of loader > 'app') > at > org.apache.flink.table.runtime.typeutils.ArrayDataSerializer.copy(ArrayDataSerializer.java:90) > at > org.apache.flink.table.runtime.typeutils.ArrayDataSerializer.copy(ArrayDataSerializer.java:47) > at > org.apache.flink.table.runtime.typeutils.RowDataSerializer.copyRowData(RowDataSerializer.java:170) > at > org.apache.flink.table.runtime.typeutils.RowDataSerializer.copy(RowDataSerializer.java:131) > at > org.apache.flink.table.runtime.typeutils.RowDataSerializer.copy(RowDataSerializer.java:48) > at > org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:69) > at > org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:46) > at > org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:26) > at > org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:50) > at > org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:28) > at > org.apache.flink.streaming.api.operators.StreamSourceContexts$ManualWatermarkContext.processAndCollect(StreamSourceContexts.java:317) > at > org.apache.flink.streaming.api.operators.StreamSourceContexts$WatermarkContext.collect(StreamSourceContexts.java:411) > at > org.apache.iceberg.flink.source.StreamingReaderOperator.processSplits(StreamingReaderOperator.java:155) > at > org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) > at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) > at > org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:359) > at > org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:323) > at > org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:202) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:681) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.executeInvoke(StreamTask.java:636) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:647) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:620) > at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:779) > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566) > at java.base/java.lang.Thread.run(Thread.java:834) > > Could be same issue as https://issues.apache.org/jira/browse/FLINK-21247 > except it happening for another type. > I see that Iceberg use custom types other than the types from > org.apache.flink.table.data like > org.apache.iceberg.flink.data.FlinkParquetReaders.ReusableArrayData and these > types are not handled in > org.apache.flink.table.runtime.typeutils.ArrayDataSerializer > !Screen Shot 2021-12-09 at 6.58.56 PM.png! > Just to try I changed the above code to handle the iceberg type as a binary > Array and built it locally and used in my application and that worked. > > !Screen Shot 2021-12-09 at 7.04.10 PM.png! > Not sure if this is already handled in some newer versions. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18055: [docs] Tutorial: Write Your First Flink SQL program
flinkbot edited a comment on pull request #18055: URL: https://github.com/apache/flink/pull/18055#issuecomment-988788303 ## CI report: * 9763d55b86ff8ef5f68fb8520360197f7a66c044 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27969) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17774: [FLINK-24611] Prevent JM from discarding state on checkpoint abortion
flinkbot edited a comment on pull request #17774: URL: https://github.com/apache/flink/pull/17774#issuecomment-966933536 ## CI report: * 6731722ca4886ba8c584d541219a7d6b980b3028 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26655) * 70f0705383e67a4ffa9f577a48165a0657a8dcb7 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27972) * 6445d1c6fb469ef7ca57bdee129a52f0e5bb369b Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27975) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18063: [FLINK-24413] Apply trimming & padding when CASTing to CHAR/VARCHAR
flinkbot edited a comment on pull request #18063: URL: https://github.com/apache/flink/pull/18063#issuecomment-989830995 ## CI report: * 1d2ab2ff32663de8d43339b76686e4775d52f22f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27961) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17774: [FLINK-24611] Prevent JM from discarding state on checkpoint abortion
flinkbot edited a comment on pull request #17774: URL: https://github.com/apache/flink/pull/17774#issuecomment-966933536 ## CI report: * 6731722ca4886ba8c584d541219a7d6b980b3028 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26655) * 70f0705383e67a4ffa9f577a48165a0657a8dcb7 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27972) * 6445d1c6fb469ef7ca57bdee129a52f0e5bb369b UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17774: [FLINK-24611] Prevent JM from discarding state on checkpoint abortion
flinkbot edited a comment on pull request #17774: URL: https://github.com/apache/flink/pull/17774#issuecomment-966933536 ## CI report: * 6731722ca4886ba8c584d541219a7d6b980b3028 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26655) * 70f0705383e67a4ffa9f577a48165a0657a8dcb7 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27972) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17774: [FLINK-24611] Prevent JM from discarding state on checkpoint abortion
flinkbot edited a comment on pull request #17774: URL: https://github.com/apache/flink/pull/17774#issuecomment-966933536 ## CI report: * 6731722ca4886ba8c584d541219a7d6b980b3028 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26655) * 70f0705383e67a4ffa9f577a48165a0657a8dcb7 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27972) * 6445d1c6fb469ef7ca57bdee129a52f0e5bb369b UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17785: [FLINK-24580][Connectors/Kinesis] Make ConnectTimeoutException recoverable
flinkbot edited a comment on pull request #17785: URL: https://github.com/apache/flink/pull/17785#issuecomment-967470963 ## CI report: * 134c805f9271cfac8059656d4d49b5362538409b Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27913) * fc88ab25f463422610629a737ad087e427c01828 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27974) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17774: [FLINK-24611] Prevent JM from discarding state on checkpoint abortion
flinkbot edited a comment on pull request #17774: URL: https://github.com/apache/flink/pull/17774#issuecomment-966933536 ## CI report: * 6731722ca4886ba8c584d541219a7d6b980b3028 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26655) * 70f0705383e67a4ffa9f577a48165a0657a8dcb7 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27972) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17785: [FLINK-24580][Connectors/Kinesis] Make ConnectTimeoutException recoverable
flinkbot edited a comment on pull request #17785: URL: https://github.com/apache/flink/pull/17785#issuecomment-967470963 ## CI report: * 134c805f9271cfac8059656d4d49b5362538409b Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27913) * fc88ab25f463422610629a737ad087e427c01828 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17774: [FLINK-24611] Prevent JM from discarding state on checkpoint abortion
flinkbot edited a comment on pull request #17774: URL: https://github.com/apache/flink/pull/17774#issuecomment-966933536 ## CI report: * 6731722ca4886ba8c584d541219a7d6b980b3028 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26655) * 70f0705383e67a4ffa9f577a48165a0657a8dcb7 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27972) * 6445d1c6fb469ef7ca57bdee129a52f0e5bb369b UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17774: [FLINK-24611] Prevent JM from discarding state on checkpoint abortion
flinkbot edited a comment on pull request #17774: URL: https://github.com/apache/flink/pull/17774#issuecomment-966933536 ## CI report: * 6731722ca4886ba8c584d541219a7d6b980b3028 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26655) * 70f0705383e67a4ffa9f577a48165a0657a8dcb7 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27972) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-web] zentol commented on a change in pull request #487: Add blog post about Log4j Zero Day
zentol commented on a change in pull request #487: URL: https://github.com/apache/flink-web/pull/487#discussion_r766778688 ## File path: _posts/2021-12-10-log4j-cve.md ## @@ -0,0 +1,21 @@ +--- +layout: post +title: "Advise on Apache Log4j Zero Day (CVE-2021-44228)" +date: 2021-12-10 00:00:00 +authors: +- knaufk: + name: "Konstantin Knauf" +excerpt: "Advise on Apache Log4j Zero Day (CVE-2021-44228)" +--- + +Yesterday, a new Zero Day for Apache Log4j was [reported](https://www.cyberkendra.com/2021/12/apache-log4j-vulnerability-details-and.html). +It is by now tracked under [CVE-2021-44228](https://nvd.nist.gov/vuln/detail/CVE-2021-44228). + +Apache Flink is bundling a version of Log4j that is affeced by this vulnerability. +We recommend users to follow the [adivsory](https://logging.apache.org/log4j/2.x/security.html) of the Apache Log4j Community. +For Apache Flink this currently translates to "setting system property "log4j2.formatMsgNoLookups" to 'true'" until Log4j has been upgraded to 2.15.0 in Apache Flink. Review comment: ```suggestion For Apache Flink this currently translates to "setting system property `log4j2.formatMsgNoLookups` to `true`" until Log4j has been upgraded to 2.15.0 in Apache Flink. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-web] zentol commented on a change in pull request #487: Add blog post about Log4j Zero Day
zentol commented on a change in pull request #487: URL: https://github.com/apache/flink-web/pull/487#discussion_r766776759 ## File path: _posts/2021-12-10-log4j-cve.md ## @@ -0,0 +1,21 @@ +--- +layout: post +title: "Advise on Apache Log4j Zero Day (CVE-2021-44228)" +date: 2021-12-10 00:00:00 +authors: +- knaufk: + name: "Konstantin Knauf" +excerpt: "Advise on Apache Log4j Zero Day (CVE-2021-44228)" +--- + +Yesterday, a new Zero Day for Apache Log4j was [reported](https://www.cyberkendra.com/2021/12/apache-log4j-vulnerability-details-and.html). +It is by now tracked under [CVE-2021-44228](https://nvd.nist.gov/vuln/detail/CVE-2021-44228). + +Apache Flink is bundling a version of Log4j that is affeced by this vulnerability. +We recommend users to follow the [adivsory](https://logging.apache.org/log4j/2.x/security.html) of the Apache Log4j Community. Review comment: ```suggestion We recommend users to follow the [adivsory](https://logging.apache.org/log4j/2.x/security.html) of the Apache Log4j Community. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18055: [docs] Tutorial: Write Your First Flink SQL program
flinkbot edited a comment on pull request #18055: URL: https://github.com/apache/flink/pull/18055#issuecomment-988788303 ## CI report: * aa96631ae9ae4e1d26a350026bbac73e7785b7ff Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27962) * 9763d55b86ff8ef5f68fb8520360197f7a66c044 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27969) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17774: [FLINK-24611] Prevent JM from discarding state on checkpoint abortion
flinkbot edited a comment on pull request #17774: URL: https://github.com/apache/flink/pull/17774#issuecomment-966933536 ## CI report: * 6731722ca4886ba8c584d541219a7d6b980b3028 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26655) * 70f0705383e67a4ffa9f577a48165a0657a8dcb7 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27972) * 6445d1c6fb469ef7ca57bdee129a52f0e5bb369b UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17774: [FLINK-24611] Prevent JM from discarding state on checkpoint abortion
flinkbot edited a comment on pull request #17774: URL: https://github.com/apache/flink/pull/17774#issuecomment-966933536 ## CI report: * 6731722ca4886ba8c584d541219a7d6b980b3028 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26655) * 70f0705383e67a4ffa9f577a48165a0657a8dcb7 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27972) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18077: [FLINK-25240][core] update log4j2 version to 2.15.0
flinkbot edited a comment on pull request #18077: URL: https://github.com/apache/flink/pull/18077#issuecomment-990863888 ## CI report: * 917836dd8f25d3cc29c5e71f2cc19bab3593256b Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27960) * aec502fbe28c7dc9cf457b10972f2570582bd551 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27968) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
