[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669836#comment-16669836 ] Timo Walther commented on FLINK-10166: -- Fixed in 1.6.3: 2657868eb704f148d5f3f1a24eaa07ad2243b189 Fixed in 1.5.6: 78525bfb2b9524a6ba02ad0b1fe09a6b0a328bdd > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Assignee: Timo Walther >Priority: Blocker > Labels: pull-request-available > Fix For: 1.5.6, 1.6.3, 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669806#comment-16669806 ] ASF GitHub Bot commented on FLINK-10166: twalthr closed pull request #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/codegen/AggregationCodeGenerator.scala b/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/codegen/AggregationCodeGenerator.scala index 11c00080329..c7a6a64ab94 100644 --- a/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/codegen/AggregationCodeGenerator.scala +++ b/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/codegen/AggregationCodeGenerator.scala @@ -21,7 +21,6 @@ import java.lang.reflect.Modifier import java.lang.{Iterable => JIterable} import org.apache.calcite.rex.RexLiteral -import org.apache.commons.codec.binary.Base64 import org.apache.flink.api.common.state.{ListStateDescriptor, MapStateDescriptor, State, StateDescriptor} import org.apache.flink.api.common.typeinfo.{BasicTypeInfo, TypeInformation} import org.apache.flink.api.java.typeutils.RowTypeInfo @@ -330,7 +329,7 @@ class AggregationCodeGenerator( s""" |$descClassQualifier $descFieldTerm = ($descClassQualifier) | org.apache.flink.util.InstantiationUtil.deserializeObject( - | org.apache.commons.codec.binary.Base64.decodeBase64("$serializedData"), + | java.util.Base64.getUrlDecoder().decode("$serializedData".getBytes("UTF-8")), | $contextTerm.getUserCodeClassLoader()); |""".stripMargin val createDataView = if (dataViewField.getType == classOf[MapView[_, _]]) { @@ -774,6 +773,6 @@ class AggregationCodeGenerator( @throws[Exception] def serializeStateDescriptor(stateDescriptor: StateDescriptor[_, _]): String = { val byteArray = InstantiationUtil.serializeObject(stateDescriptor) -Base64.encodeBase64URLSafeString(byteArray) +new String(java.util.Base64.getUrlEncoder.withoutPadding().encode(byteArray), "UTF-8") } } diff --git a/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/descriptors/DescriptorProperties.scala b/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/descriptors/DescriptorProperties.scala index 9328c806913..4177c3a1620 100644 --- a/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/descriptors/DescriptorProperties.scala +++ b/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/descriptors/DescriptorProperties.scala @@ -26,7 +26,6 @@ import java.util.function.{Consumer, Supplier} import java.util.regex.Pattern import java.util.{Optional, List => JList, Map => JMap} -import org.apache.commons.codec.binary.Base64 import org.apache.commons.lang.StringEscapeUtils import org.apache.flink.api.common.typeinfo.TypeInformation import org.apache.flink.api.java.tuple.{Tuple2 => JTuple2} @@ -1356,7 +1355,7 @@ object DescriptorProperties { } try { val byteArray = InstantiationUtil.serializeObject(obj) - Base64.encodeBase64URLSafeString(byteArray) + new String(java.util.Base64.getUrlEncoder.withoutPadding().encode(byteArray), "UTF-8") } catch { case e: Exception => throw new ValidationException( @@ -1366,7 +1365,7 @@ object DescriptorProperties { def deserialize[T](data: String, expected: Class[T]): T = { try { - val byteData = Base64.decodeBase64(data) + val byteData = java.util.Base64.getUrlDecoder.decode(data.getBytes("UTF-8")) val obj = InstantiationUtil.deserializeObject[T]( byteData, Thread.currentThread.getContextClassLoader) diff --git a/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/functions/utils/UserDefinedFunctionUtils.scala b/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/functions/utils/UserDefinedFunctionUtils.scala index 2c08001d20c..7e5128cf939 100644 --- a/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/functions/utils/UserDefinedFunctionUtils.scala +++ b/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/functions/utils/UserDefinedFunctionUtils.scala @@ -24,7 +24,6 @@ import java.lang.{Integer => JInt, Long => JLong} import java.lang.reflect.{Method, Modifier} import java.sql.{Date, Time, Timestamp} -import org.apache.commons.codec.binary.Base64 import com.google.common.primitives.Primitives import org.apache.calcite.re
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669805#comment-16669805 ] ASF GitHub Bot commented on FLINK-10166: twalthr commented on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-434614175 I tried to fix this issue once and for all in #6966. This PR had to be rebased to the new `flink-table-common` module anyway. I will close this PR. Sorry, for the inconvenience. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Assignee: Timo Walther >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669790#comment-16669790 ] ASF GitHub Bot commented on FLINK-10166: asfgit closed pull request #6966: [FLINK-10166] [table] Reduce dependencies by removing org.apache.commons URL: https://github.com/apache/flink/pull/6966 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/flink-libraries/flink-table-common/src/main/java/org/apache/flink/table/descriptors/DescriptorProperties.java b/flink-libraries/flink-table-common/src/main/java/org/apache/flink/table/descriptors/DescriptorProperties.java index 0d2f835958f..c4769814728 100644 --- a/flink-libraries/flink-table-common/src/main/java/org/apache/flink/table/descriptors/DescriptorProperties.java +++ b/flink-libraries/flink-table-common/src/main/java/org/apache/flink/table/descriptors/DescriptorProperties.java @@ -1326,7 +1326,7 @@ private int extractMaxIndex(String key, String suffixPattern) { } public static String toString(String str) { - return EncodingUtils.escapeJava(str).replace("\\/", "/"); // '/' must not be escaped + return EncodingUtils.escapeJava(str); } public static String toString(String key, String value) { diff --git a/flink-libraries/flink-table-common/src/main/java/org/apache/flink/table/utils/EncodingUtils.java b/flink-libraries/flink-table-common/src/main/java/org/apache/flink/table/utils/EncodingUtils.java index 41fa58ef055..47aac25e897 100644 --- a/flink-libraries/flink-table-common/src/main/java/org/apache/flink/table/utils/EncodingUtils.java +++ b/flink-libraries/flink-table-common/src/main/java/org/apache/flink/table/utils/EncodingUtils.java @@ -19,6 +19,7 @@ package org.apache.flink.table.utils; import org.apache.flink.annotation.Internal; +import org.apache.flink.table.api.TableException; import org.apache.flink.table.api.ValidationException; import org.apache.flink.util.InstantiationUtil; @@ -26,9 +27,12 @@ import java.io.Serializable; import java.io.StringWriter; import java.io.Writer; -import java.nio.charset.StandardCharsets; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; import java.util.Base64; +import static java.nio.charset.StandardCharsets.UTF_8; + /** * General utilities for string-encoding. This class is used to avoid additional dependencies * to other projects. @@ -40,6 +44,8 @@ private static final Base64.Decoder BASE64_DECODER = java.util.Base64.getUrlDecoder(); + private static final char[] HEX_CHARS = "0123456789abcdef".toCharArray(); + private EncodingUtils() { // do not instantiate } @@ -47,7 +53,7 @@ private EncodingUtils() { public static String encodeObjectToString(Serializable obj) { try { final byte[] bytes = InstantiationUtil.serializeObject(obj); - return new String(BASE64_ENCODER.encode(bytes), StandardCharsets.UTF_8); + return new String(BASE64_ENCODER.encode(bytes), UTF_8); } catch (Exception e) { throw new ValidationException( "Unable to serialize object '" + obj.toString() + "' of class '" + obj.getClass().getName() + "'."); @@ -60,7 +66,7 @@ public static String encodeObjectToString(Serializable obj) { public static T decodeStringToObject(String base64String, Class baseClass, ClassLoader classLoader) { try { - final byte[] bytes = BASE64_DECODER.decode(base64String.getBytes(StandardCharsets.UTF_8)); + final byte[] bytes = BASE64_DECODER.decode(base64String.getBytes(UTF_8)); final T instance = InstantiationUtil.deserializeObject(bytes, classLoader); if (instance != null && !baseClass.isAssignableFrom(instance.getClass())) { throw new ValidationException( @@ -87,10 +93,138 @@ public static String encodeObjectToString(Serializable obj) { return loadClass(qualifiedName, Thread.currentThread().getContextClassLoader()); } + public static String encodeStringToBase64(String string) { + return new String(java.util.Base64.getEncoder().encode(string.getBytes(UTF_8)), UTF_8); + } + + public static String decodeBase64ToString(String base64) { + return new String(java.util.Base64.getDecoder().decode(base64.getBytes(UTF_8)), UTF_8); + } + + public static byte[] md5(String string) { + try { +
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669786#comment-16669786 ] Timo Walther commented on FLINK-10166: -- Fixed in 1.7.0: 08d875eadec779f68baa1cc7003dab7bdd51640a > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Assignee: Timo Walther >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669767#comment-16669767 ] ASF GitHub Bot commented on FLINK-10166: twalthr commented on issue #6966: [FLINK-10166] [table] Reduce dependencies by removing org.apache.commons URL: https://github.com/apache/flink/pull/6966#issuecomment-434605691 Thank you @igalshilman. I will merge this... This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Assignee: Timo Walther >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669766#comment-16669766 ] ASF GitHub Bot commented on FLINK-10166: igalshilman commented on issue #6966: [FLINK-10166] [table] Reduce dependencies by removing org.apache.commons URL: https://github.com/apache/flink/pull/6966#issuecomment-434604624 LGTM 👍 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Assignee: Timo Walther >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669765#comment-16669765 ] ASF GitHub Bot commented on FLINK-10166: igalshilman commented on a change in pull request #6966: [FLINK-10166] [table] Reduce dependencies by removing org.apache.commons URL: https://github.com/apache/flink/pull/6966#discussion_r229603180 ## File path: flink-libraries/flink-table-common/src/main/java/org/apache/flink/table/utils/EncodingUtils.java ## @@ -40,14 +44,18 @@ private static final Base64.Decoder BASE64_DECODER = java.util.Base64.getUrlDecoder(); + private static final MessageDigest MD5_MESSAGE_DIGEST = getMd5MessageDigest(); Review comment: Thanks for addressing this. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Assignee: Timo Walther >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669764#comment-16669764 ] ASF GitHub Bot commented on FLINK-10166: igalshilman commented on a change in pull request #6966: [FLINK-10166] [table] Reduce dependencies by removing org.apache.commons URL: https://github.com/apache/flink/pull/6966#discussion_r229603056 ## File path: flink-libraries/flink-table-common/src/main/java/org/apache/flink/table/utils/EncodingUtils.java ## @@ -87,10 +95,145 @@ public static String encodeObjectToString(Serializable obj) { return loadClass(qualifiedName, Thread.currentThread().getContextClassLoader()); } + public static String encodeStringToBase64(String string) { + return new String(java.util.Base64.getEncoder().encode(string.getBytes(UTF_8)), UTF_8); + } + + public static String decodeBase64ToString(String base64) { + return new String(java.util.Base64.getDecoder().decode(base64.getBytes(UTF_8)), UTF_8); + } + + public static byte[] md5(String string) { + if (MD5_MESSAGE_DIGEST == null) { + throw new TableException("Unsupported MD5 algorithm."); + } + return MD5_MESSAGE_DIGEST.digest(string.getBytes(UTF_8)); + } + + public static String hex(String string) { + return hex(string.getBytes(UTF_8)); + } + + public static String hex(byte[] bytes) { + // adopted from https://stackoverflow.com/a/9855338 + final char[] hexChars = new char[bytes.length * 2]; + for (int j = 0; j < bytes.length; j++) { + final int v = bytes[j] & 0xFF; + hexChars[j * 2] = HEX_CHARS[v >>> 4]; + hexChars[j * 2 + 1] = HEX_CHARS[v & 0x0F]; + } + return new String(hexChars); + } + + private static MessageDigest getMd5MessageDigest() { + try { + return MessageDigest.getInstance("MD5"); + } catch (NoSuchAlgorithmException e) { + return null; + } + } + + // + // Java String Repetition + // + // copied from o.a.commons.lang3.StringUtils (commons-lang3:3.3.2) + // + + private static final String EMPTY = ""; + + /** +* The maximum size to which the padding constant(s) can expand. +*/ + private static final int PAD_LIMIT = 8192; + + /** +* Repeat a String {@code repeat} times to form a new String. +* +* +* StringUtils.repeat(null, 2) = null +* StringUtils.repeat("", 0) = "" +* StringUtils.repeat("", 2) = "" +* StringUtils.repeat("a", 3) = "aaa" +* StringUtils.repeat("ab", 2) = "abab" +* StringUtils.repeat("a", -2) = "" +* +* +* @param strthe String to repeat, may be null +* @param repeat number of times to repeat str, negative treated as zero +* @return a new String consisting of the original String repeated, {@code null} if null String input +*/ + public static String repeat(final String str, final int repeat) { + // Performance tuned for 2.0 (JDK1.4) + + if (str == null) { + return null; + } + if (repeat <= 0) { + return EMPTY; + } + final int inputLength = str.length(); + if (repeat == 1 || inputLength == 0) { + return str; + } + if (inputLength == 1 && repeat <= PAD_LIMIT) { + return repeat(str.charAt(0), repeat); + } + + final int outputLength = inputLength * repeat; + switch (inputLength) { + case 1: + return repeat(str.charAt(0), repeat); + case 2: + final char ch0 = str.charAt(0); + final char ch1 = str.charAt(1); + final char[] output2 = new char[outputLength]; + for (int i = repeat * 2 - 2; i >= 0; i--, i--) { + output2[i] = ch0; + output2[i + 1] = ch1; + } + return new String(output2); + default: + final StringBuilder buf = new StringBuilder(outpu
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669741#comment-16669741 ] ASF GitHub Bot commented on FLINK-10166: twalthr commented on a change in pull request #6966: [FLINK-10166] [table] Reduce dependencies by removing org.apache.commons URL: https://github.com/apache/flink/pull/6966#discussion_r229599636 ## File path: flink-libraries/flink-table-common/src/main/java/org/apache/flink/table/utils/EncodingUtils.java ## @@ -87,10 +95,145 @@ public static String encodeObjectToString(Serializable obj) { return loadClass(qualifiedName, Thread.currentThread().getContextClassLoader()); } + public static String encodeStringToBase64(String string) { + return new String(java.util.Base64.getEncoder().encode(string.getBytes(UTF_8)), UTF_8); + } + + public static String decodeBase64ToString(String base64) { + return new String(java.util.Base64.getDecoder().decode(base64.getBytes(UTF_8)), UTF_8); + } + + public static byte[] md5(String string) { + if (MD5_MESSAGE_DIGEST == null) { + throw new TableException("Unsupported MD5 algorithm."); + } + return MD5_MESSAGE_DIGEST.digest(string.getBytes(UTF_8)); + } + + public static String hex(String string) { + return hex(string.getBytes(UTF_8)); + } + + public static String hex(byte[] bytes) { + // adopted from https://stackoverflow.com/a/9855338 + final char[] hexChars = new char[bytes.length * 2]; + for (int j = 0; j < bytes.length; j++) { + final int v = bytes[j] & 0xFF; + hexChars[j * 2] = HEX_CHARS[v >>> 4]; + hexChars[j * 2 + 1] = HEX_CHARS[v & 0x0F]; + } + return new String(hexChars); + } + + private static MessageDigest getMd5MessageDigest() { + try { + return MessageDigest.getInstance("MD5"); + } catch (NoSuchAlgorithmException e) { + return null; + } + } + + // + // Java String Repetition + // + // copied from o.a.commons.lang3.StringUtils (commons-lang3:3.3.2) + // + + private static final String EMPTY = ""; + + /** +* The maximum size to which the padding constant(s) can expand. +*/ + private static final int PAD_LIMIT = 8192; + + /** +* Repeat a String {@code repeat} times to form a new String. +* +* +* StringUtils.repeat(null, 2) = null +* StringUtils.repeat("", 0) = "" +* StringUtils.repeat("", 2) = "" +* StringUtils.repeat("a", 3) = "aaa" +* StringUtils.repeat("ab", 2) = "abab" +* StringUtils.repeat("a", -2) = "" +* +* +* @param strthe String to repeat, may be null +* @param repeat number of times to repeat str, negative treated as zero +* @return a new String consisting of the original String repeated, {@code null} if null String input +*/ + public static String repeat(final String str, final int repeat) { + // Performance tuned for 2.0 (JDK1.4) + + if (str == null) { + return null; + } + if (repeat <= 0) { + return EMPTY; + } + final int inputLength = str.length(); + if (repeat == 1 || inputLength == 0) { + return str; + } + if (inputLength == 1 && repeat <= PAD_LIMIT) { + return repeat(str.charAt(0), repeat); + } + + final int outputLength = inputLength * repeat; + switch (inputLength) { + case 1: + return repeat(str.charAt(0), repeat); + case 2: + final char ch0 = str.charAt(0); + final char ch1 = str.charAt(1); + final char[] output2 = new char[outputLength]; + for (int i = repeat * 2 - 2; i >= 0; i--, i--) { + output2[i] = ch0; + output2[i + 1] = ch1; + } + return new String(output2); + default: + final StringBuilder buf = new StringBuilder(outputLen
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669740#comment-16669740 ] ASF GitHub Bot commented on FLINK-10166: twalthr commented on a change in pull request #6966: [FLINK-10166] [table] Reduce dependencies by removing org.apache.commons URL: https://github.com/apache/flink/pull/6966#discussion_r229599636 ## File path: flink-libraries/flink-table-common/src/main/java/org/apache/flink/table/utils/EncodingUtils.java ## @@ -87,10 +95,145 @@ public static String encodeObjectToString(Serializable obj) { return loadClass(qualifiedName, Thread.currentThread().getContextClassLoader()); } + public static String encodeStringToBase64(String string) { + return new String(java.util.Base64.getEncoder().encode(string.getBytes(UTF_8)), UTF_8); + } + + public static String decodeBase64ToString(String base64) { + return new String(java.util.Base64.getDecoder().decode(base64.getBytes(UTF_8)), UTF_8); + } + + public static byte[] md5(String string) { + if (MD5_MESSAGE_DIGEST == null) { + throw new TableException("Unsupported MD5 algorithm."); + } + return MD5_MESSAGE_DIGEST.digest(string.getBytes(UTF_8)); + } + + public static String hex(String string) { + return hex(string.getBytes(UTF_8)); + } + + public static String hex(byte[] bytes) { + // adopted from https://stackoverflow.com/a/9855338 + final char[] hexChars = new char[bytes.length * 2]; + for (int j = 0; j < bytes.length; j++) { + final int v = bytes[j] & 0xFF; + hexChars[j * 2] = HEX_CHARS[v >>> 4]; + hexChars[j * 2 + 1] = HEX_CHARS[v & 0x0F]; + } + return new String(hexChars); + } + + private static MessageDigest getMd5MessageDigest() { + try { + return MessageDigest.getInstance("MD5"); + } catch (NoSuchAlgorithmException e) { + return null; + } + } + + // + // Java String Repetition + // + // copied from o.a.commons.lang3.StringUtils (commons-lang3:3.3.2) + // + + private static final String EMPTY = ""; + + /** +* The maximum size to which the padding constant(s) can expand. +*/ + private static final int PAD_LIMIT = 8192; + + /** +* Repeat a String {@code repeat} times to form a new String. +* +* +* StringUtils.repeat(null, 2) = null +* StringUtils.repeat("", 0) = "" +* StringUtils.repeat("", 2) = "" +* StringUtils.repeat("a", 3) = "aaa" +* StringUtils.repeat("ab", 2) = "abab" +* StringUtils.repeat("a", -2) = "" +* +* +* @param strthe String to repeat, may be null +* @param repeat number of times to repeat str, negative treated as zero +* @return a new String consisting of the original String repeated, {@code null} if null String input +*/ + public static String repeat(final String str, final int repeat) { + // Performance tuned for 2.0 (JDK1.4) + + if (str == null) { + return null; + } + if (repeat <= 0) { + return EMPTY; + } + final int inputLength = str.length(); + if (repeat == 1 || inputLength == 0) { + return str; + } + if (inputLength == 1 && repeat <= PAD_LIMIT) { + return repeat(str.charAt(0), repeat); + } + + final int outputLength = inputLength * repeat; + switch (inputLength) { + case 1: + return repeat(str.charAt(0), repeat); + case 2: + final char ch0 = str.charAt(0); + final char ch1 = str.charAt(1); + final char[] output2 = new char[outputLength]; + for (int i = repeat * 2 - 2; i >= 0; i--, i--) { + output2[i] = ch0; + output2[i + 1] = ch1; + } + return new String(output2); + default: + final StringBuilder buf = new StringBuilder(outputLen
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669738#comment-16669738 ] ASF GitHub Bot commented on FLINK-10166: twalthr commented on a change in pull request #6966: [FLINK-10166] [table] Reduce dependencies by removing org.apache.commons URL: https://github.com/apache/flink/pull/6966#discussion_r229598889 ## File path: flink-libraries/flink-table-common/src/main/java/org/apache/flink/table/utils/EncodingUtils.java ## @@ -40,14 +44,18 @@ private static final Base64.Decoder BASE64_DECODER = java.util.Base64.getUrlDecoder(); + private static final MessageDigest MD5_MESSAGE_DIGEST = getMd5MessageDigest(); Review comment: Usually this doesn't matter for translation (which is single threaded) but we are on the safer side if this method is thread-safe in the future. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Assignee: Timo Walther >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669365#comment-16669365 ] ASF GitHub Bot commented on FLINK-10166: igalshilman commented on a change in pull request #6966: [FLINK-10166] [table] Reduce dependencies by removing org.apache.commons URL: https://github.com/apache/flink/pull/6966#discussion_r229473954 ## File path: flink-libraries/flink-table-common/src/main/java/org/apache/flink/table/utils/EncodingUtils.java ## @@ -40,14 +44,18 @@ private static final Base64.Decoder BASE64_DECODER = java.util.Base64.getUrlDecoder(); + private static final MessageDigest MD5_MESSAGE_DIGEST = getMd5MessageDigest(); Review comment: Would MD5_MESSAGE_DIGEST be accessed concurrently? afaik this is not thread safe. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Assignee: Timo Walther >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669363#comment-16669363 ] ASF GitHub Bot commented on FLINK-10166: igalshilman commented on a change in pull request #6966: [FLINK-10166] [table] Reduce dependencies by removing org.apache.commons URL: https://github.com/apache/flink/pull/6966#discussion_r229481767 ## File path: flink-libraries/flink-table-common/src/main/java/org/apache/flink/table/utils/EncodingUtils.java ## @@ -87,10 +95,145 @@ public static String encodeObjectToString(Serializable obj) { return loadClass(qualifiedName, Thread.currentThread().getContextClassLoader()); } + public static String encodeStringToBase64(String string) { + return new String(java.util.Base64.getEncoder().encode(string.getBytes(UTF_8)), UTF_8); + } + + public static String decodeBase64ToString(String base64) { + return new String(java.util.Base64.getDecoder().decode(base64.getBytes(UTF_8)), UTF_8); + } + + public static byte[] md5(String string) { + if (MD5_MESSAGE_DIGEST == null) { + throw new TableException("Unsupported MD5 algorithm."); + } + return MD5_MESSAGE_DIGEST.digest(string.getBytes(UTF_8)); + } + + public static String hex(String string) { + return hex(string.getBytes(UTF_8)); + } + + public static String hex(byte[] bytes) { + // adopted from https://stackoverflow.com/a/9855338 + final char[] hexChars = new char[bytes.length * 2]; + for (int j = 0; j < bytes.length; j++) { + final int v = bytes[j] & 0xFF; + hexChars[j * 2] = HEX_CHARS[v >>> 4]; + hexChars[j * 2 + 1] = HEX_CHARS[v & 0x0F]; + } + return new String(hexChars); + } + + private static MessageDigest getMd5MessageDigest() { + try { + return MessageDigest.getInstance("MD5"); + } catch (NoSuchAlgorithmException e) { + return null; + } + } + + // + // Java String Repetition + // + // copied from o.a.commons.lang3.StringUtils (commons-lang3:3.3.2) + // + + private static final String EMPTY = ""; + + /** +* The maximum size to which the padding constant(s) can expand. +*/ + private static final int PAD_LIMIT = 8192; + + /** +* Repeat a String {@code repeat} times to form a new String. +* +* +* StringUtils.repeat(null, 2) = null +* StringUtils.repeat("", 0) = "" +* StringUtils.repeat("", 2) = "" +* StringUtils.repeat("a", 3) = "aaa" +* StringUtils.repeat("ab", 2) = "abab" +* StringUtils.repeat("a", -2) = "" +* +* +* @param strthe String to repeat, may be null +* @param repeat number of times to repeat str, negative treated as zero +* @return a new String consisting of the original String repeated, {@code null} if null String input +*/ + public static String repeat(final String str, final int repeat) { + // Performance tuned for 2.0 (JDK1.4) + + if (str == null) { + return null; + } + if (repeat <= 0) { + return EMPTY; + } + final int inputLength = str.length(); + if (repeat == 1 || inputLength == 0) { + return str; + } + if (inputLength == 1 && repeat <= PAD_LIMIT) { + return repeat(str.charAt(0), repeat); + } + + final int outputLength = inputLength * repeat; + switch (inputLength) { + case 1: + return repeat(str.charAt(0), repeat); + case 2: + final char ch0 = str.charAt(0); + final char ch1 = str.charAt(1); + final char[] output2 = new char[outputLength]; + for (int i = repeat * 2 - 2; i >= 0; i--, i--) { + output2[i] = ch0; + output2[i + 1] = ch1; + } + return new String(output2); + default: + final StringBuilder buf = new StringBuilder(outpu
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669364#comment-16669364 ] ASF GitHub Bot commented on FLINK-10166: igalshilman commented on a change in pull request #6966: [FLINK-10166] [table] Reduce dependencies by removing org.apache.commons URL: https://github.com/apache/flink/pull/6966#discussion_r229487810 ## File path: flink-libraries/flink-table-common/src/main/java/org/apache/flink/table/utils/EncodingUtils.java ## @@ -87,10 +95,145 @@ public static String encodeObjectToString(Serializable obj) { return loadClass(qualifiedName, Thread.currentThread().getContextClassLoader()); } + public static String encodeStringToBase64(String string) { + return new String(java.util.Base64.getEncoder().encode(string.getBytes(UTF_8)), UTF_8); + } + + public static String decodeBase64ToString(String base64) { + return new String(java.util.Base64.getDecoder().decode(base64.getBytes(UTF_8)), UTF_8); + } + + public static byte[] md5(String string) { + if (MD5_MESSAGE_DIGEST == null) { + throw new TableException("Unsupported MD5 algorithm."); + } + return MD5_MESSAGE_DIGEST.digest(string.getBytes(UTF_8)); + } + + public static String hex(String string) { + return hex(string.getBytes(UTF_8)); + } + + public static String hex(byte[] bytes) { + // adopted from https://stackoverflow.com/a/9855338 + final char[] hexChars = new char[bytes.length * 2]; + for (int j = 0; j < bytes.length; j++) { + final int v = bytes[j] & 0xFF; + hexChars[j * 2] = HEX_CHARS[v >>> 4]; + hexChars[j * 2 + 1] = HEX_CHARS[v & 0x0F]; + } + return new String(hexChars); + } + + private static MessageDigest getMd5MessageDigest() { + try { + return MessageDigest.getInstance("MD5"); + } catch (NoSuchAlgorithmException e) { + return null; + } + } + + // + // Java String Repetition + // + // copied from o.a.commons.lang3.StringUtils (commons-lang3:3.3.2) + // + + private static final String EMPTY = ""; + + /** +* The maximum size to which the padding constant(s) can expand. +*/ + private static final int PAD_LIMIT = 8192; + + /** +* Repeat a String {@code repeat} times to form a new String. +* +* +* StringUtils.repeat(null, 2) = null +* StringUtils.repeat("", 0) = "" +* StringUtils.repeat("", 2) = "" +* StringUtils.repeat("a", 3) = "aaa" +* StringUtils.repeat("ab", 2) = "abab" +* StringUtils.repeat("a", -2) = "" +* +* +* @param strthe String to repeat, may be null +* @param repeat number of times to repeat str, negative treated as zero +* @return a new String consisting of the original String repeated, {@code null} if null String input +*/ + public static String repeat(final String str, final int repeat) { + // Performance tuned for 2.0 (JDK1.4) + + if (str == null) { + return null; + } + if (repeat <= 0) { + return EMPTY; + } + final int inputLength = str.length(); + if (repeat == 1 || inputLength == 0) { + return str; + } + if (inputLength == 1 && repeat <= PAD_LIMIT) { + return repeat(str.charAt(0), repeat); + } + + final int outputLength = inputLength * repeat; + switch (inputLength) { + case 1: + return repeat(str.charAt(0), repeat); + case 2: + final char ch0 = str.charAt(0); + final char ch1 = str.charAt(1); + final char[] output2 = new char[outputLength]; + for (int i = repeat * 2 - 2; i >= 0; i--, i--) { + output2[i] = ch0; + output2[i + 1] = ch1; + } + return new String(output2); + default: + final StringBuilder buf = new StringBuilder(outpu
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669014#comment-16669014 ] ASF GitHub Bot commented on FLINK-10166: twalthr opened a new pull request #6966: [FLINK-10166] [table] Reduce dependencies by removing org.apache.commons URL: https://github.com/apache/flink/pull/6966 ## What is the purpose of the change This PR removes all dependencies to `org.apache.commons` libraries. In the past we only used a couple of methods that were partially pulled in from Hadoop causing the issues mentioned in the JIRA ticket. ## Brief change log - Add more utility method to the Flink code base in order to reduce external dependencies ## Verifying this change This change is already covered by existing tests. I added additional test such as `EncodingUtilsTest` and `TypeCheckUtilsTest`. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): yes - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659268#comment-16659268 ] ASF GitHub Bot commented on FLINK-10166: leanken removed a comment on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431779580 Calling @dawidwys for review. ^_^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659267#comment-16659267 ] ASF GitHub Bot commented on FLINK-10166: leanken removed a comment on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431779913 > Kindly remind. @dawidwys > Build turned green, ready to go. Calling @dawidwys for review. ^_^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659270#comment-16659270 ] ASF GitHub Bot commented on FLINK-10166: leanken removed a comment on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431779714 Calling @dawidwys for review. ^_^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659266#comment-16659266 ] ASF GitHub Bot commented on FLINK-10166: leanken removed a comment on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431782634 CALLING @dawidwys for review. ^_^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659272#comment-16659272 ] ASF GitHub Bot commented on FLINK-10166: leanken removed a comment on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431778970 Calling @dawidwys ^_^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659269#comment-16659269 ] ASF GitHub Bot commented on FLINK-10166: leanken removed a comment on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431779272 Calling @dawidwys . ^_^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659271#comment-16659271 ] ASF GitHub Bot commented on FLINK-10166: leanken removed a comment on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431779068 Calling @dawidwys ^_^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658851#comment-16658851 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431782634 CALLING @dawidwys for review. ^_^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658829#comment-16658829 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431779714 Calling @dawidwys for review. ^_^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658830#comment-16658830 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431779913 > Kindly remind. @dawidwys > Build turned green, ready to go. Calling @dawidwys for review. ^_^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658828#comment-16658828 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431779580 Calling @dawidwys for review. ^_^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658824#comment-16658824 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431779272 Calling @dawidwys . ^_^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658822#comment-16658822 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431779188 Calling @dawidwys . ^_^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658820#comment-16658820 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431779068 Calling @dawidwys ^_^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658816#comment-16658816 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431778789 Calling @dawidwys ^_^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658819#comment-16658819 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431778970 Calling @dawidwys ^_^ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656537#comment-16656537 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431308873 Kindly remind. @dawidwys Build turned green, ready to go. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656183#comment-16656183 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-431223615 let me fix TestCase build break first. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653401#comment-16653401 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-430594978 Just to confirm that the change did works. I catch the GeneratedCode from debugging. ``` function_org$apache$flink$table$functions$aggfunctions$CountAggFunction$f4346869d1df07833c4f2ba55982f5ce.open(new org.apache.flink.table.functions.FunctionContext(ctx)); org.apache.flink.api.common.state.StateDescriptor acc0_distinctValueMap_dataview_desc = (org.apache.flink.api.common.state.StateDescriptor) org.apache.flink.util.InstantiationUtil.deserializeObject( java.util.Base64.getUrlDecoder().decode("rO0ABXNyADRvcmcuYXBhY2hlLmZsaW5rLmFwaS5jb21tb24uc3RhdGUuTWFwU3RhdGVEZXNjcmlwdG9yAAECAAB4cgAxb3JnLmFwYWNoZS5mbGluay5hcGkuY29tbW9uLnN0YXRlLlN0YXRlRGVzY3JpcHRvcgABAwAFTAAEbmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEnF1ZXJ5YWJsZVN0YXRlTmFtZXEAfgACTAAKc2VyaWFsaXplcnQANkxvcmcvYXBhY2hlL2ZsaW5rL2FwaS9jb21tb24vdHlwZXV0aWxzL1R5cGVTZXJpYWxpe ``` the origin **org.apache.commons.codec.binary.Base64.decodeBase64** did switch into **java.util.Base64.getUrlDecoder().decode** This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653400#comment-16653400 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166][table] Replace commons.codec.binary.Base64 with java.util.Base64 URL: https://github.com/apache/flink/pull/6863#issuecomment-430594457 cc @dawidwys Updated commit. Replaced commons.codec.binary.Base64 with java.util.Base64, and basically I've search all reference of commons.codec.binary.Base64, all of them have been replaced with java.util.Base64. And the origin encodeBase64URLSafeString should use Base64.getUrlEncode as replace candidate. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653134#comment-16653134 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166][table] skip relocation of commons-codec and commons-lang3 from flink-table URL: https://github.com/apache/flink/pull/6863#issuecomment-430522770 > 1. As far as I know `Base64` is the only class from those packages used, and therefore cutting it should be the preferred way in this case, as it simplifies our dependencies. > 2. I agree, but I would say this is a bug of `flink-binary-with-hadoop`, rather than something that we should follow. ok. I see your point. I will update in the next commit for Base64 replacement. thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653129#comment-16653129 ] ASF GitHub Bot commented on FLINK-10166: dawidwys commented on issue #6863: [FLINK-10166][table] skip relocation of commons-codec and commons-lang3 from flink-table URL: https://github.com/apache/flink/pull/6863#issuecomment-430522013 1) As far as I know `Base64` is the only class from those packages used, and therefore cutting it should be the preferred way in this case, as it simplifies our dependencies. 2) I agree, but I would say this is a bug of `flink-binary-with-hadoop`, rather than something that we should follow. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653128#comment-16653128 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166][table] skip relocation of commons-codec and commons-lang3 from flink-table URL: https://github.com/apache/flink/pull/6863#issuecomment-430521953 @dawidwys I will first have a look at the candidate replacement for for further discussion. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653125#comment-16653125 ] ASF GitHub Bot commented on FLINK-10166: leanken edited a comment on issue #6863: [FLINK-10166][table] skip relocation of commons-codec and commons-lang3 from flink-table URL: https://github.com/apache/flink/pull/6863#issuecomment-430521113 > Hi @leanken . Thank you for your contribution. I am afraid though it is not the right way to proceed with this bug. I think we should rather cut off usage of `Base64` (replace with our own implementation) than add hard external dependencies. "reply: for Base64 replacement. I think it's ok for just this single case, what if there are even more cases like Base64, in future, we might need to do the replacement case by case" > > Also just to clarify why we relocate dependencies. The reason is not just so they do not clash with other dependencies of ours (e.g. from calcite as you said), but also so that Flink's users can use any arbitrary version of popular libraries. "reply: the reason that you mentioned, user can use any version of org.apache.commons prefix library, I dont think it is now working, because in flink-binary-with-hadoop common-codec library already included in classpath, even if FlinkUserJar contains its own common-codec jar, it might be conflict during runtime" This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653123#comment-16653123 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166][table] skip relocation of commons-codec and commons-lang3 from flink-table URL: https://github.com/apache/flink/pull/6863#issuecomment-430521113 > Hi @leanken . Thank you for your contribution. I am afraid though it is not the right way to proceed with this bug. I think we should rather cut off usage of `Base64` (replace with our own implementation) than add hard external dependencies. "reply: for Base64 replacement. I think it's ok for just this single case, what if there are even more cases like Base64, in future, we might need to do the replacement case by case" > > Also just to clarify why we relocate dependencies. The reason is not just so they do not clash with other dependencies of ours (e.g. from calcite as you said), but also so that Flink's users can use any arbitrary version of popular libraries. "reply: the reason that you mentioned, user can use any version of org.apache.commons prefix library, I dont think it is now working, because in flink-binary-with-hadoop common-codec library already included in classpath, even if FlinkUserJar contains its own common-codec jar, it might be conflict during runtime" This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653113#comment-16653113 ] ASF GitHub Bot commented on FLINK-10166: dawidwys commented on issue #6863: [FLINK-10166] skip relocation of commons-codes and commons-lang3 from flink-table URL: https://github.com/apache/flink/pull/6863#issuecomment-430518728 Hi @leanken . Thank you for your contribution. I am afraid though it is not the right way to proceed with this bug. I think we should rather cut off usage of `Base64` (replace with our own implementation) than add hard external dependencies. Also just to clarify why we relocate dependencies. The reason is not just so they do not clash with other dependencies of ours (e.g. from calcite as you said), but also so that Flink's users can use any arbitrary version of popular libraries. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653023#comment-16653023 ] ASF GitHub Bot commented on FLINK-10166: leanken commented on issue #6863: [FLINK-10166] skip relocation of commons-codes and commons-lang3 from flink-table URL: https://github.com/apache/flink/pull/6863#issuecomment-430502926 cc @StefanRRichter @tillrohrmann . Plz help review on the fix of hadoopless sqlClient classNotFound issue. thanks This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653022#comment-16653022 ] ASF GitHub Bot commented on FLINK-10166: leanken opened a new pull request #6863: [FLINK-10166] skip relocation of commons-codes and commons-lang3 from flink-table URL: https://github.com/apache/flink/pull/6863 ## What is the purpose of the change See. [FLINK-10166](https://issues.apache.org/jira/browse/FLINK-10166). Fix sqlClient **org.apache.commons.codec.binary.Base64** classNotFound issue on hadoopless binary. ## Diagnose The Exception stacktrace is quite straight-forward. ``` ## Exception Message execute SQL statement. Reason: org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown variable or type "org.apache.commons.codec.binary.Base64" ``` As I mentioned above, this error only happens at hadoopless binary because the **TaskManagerRunner** start without **flink-shaded-hadoop2-uber-1.6.0.jar**, and as for the **flink-table** component has a shaded relocation on org.apache.commons prefix. so in this case, neither the TaskManagerRunner nor the FlinkUserJar supply valid class definition for **org.apache.commons.codec.binary.Base64** ``` org.apache.commons org.apache.flink.table.shaded.org.apache.commons ``` ## Proposal I've went through the history code of the flink-table relocation. Supposed the purpose of the origin author is that, to avoid version conflict that Calcite might bring. * org.apache.commons.codec * org.apache.commons.lang * org.apache.commons.lang3 But, i have double check on the sub-dependency of calcite-core, only **org.apache.commons.lang** is sub-dependency of calcite-core, **org.apache.commons.codec** and **org.apache.commons.lang3** in fact are inherited from flink-parent, so i think these two namespace no need to relocation. In genernal, my proposal is to not relocation **org.apache.commons.codec** and **org.apache.commons.lang3** ## Brief change log * not relocation **org.apache.commons.codec** and **org.apache.commons.lang3** ## Verifying this change Since i have change the relocation behavior of **flink-table** component, its relative dependency are: * flink-sql-client * flink-examples-table I've also ran the example case under these two component to make sure the change did works. * sqlClient with hadoopless binary on `select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name)` [SUCCESS] * org.apache.flink.table.examples.scala.WordCountTable [SUCCESS] * org.apache.flink.table.examples.scala.WordCountSQL [SUCCESS] * org.apache.flink.table.examples.scala.StreamTableExample [SUCCESS] * org.apache.flink.table.examples.scala.StreamSQLExample [SUCCESS] ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > Labels: pull-request-available > Fix For: 1.7.0 > > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613326#comment-16613326 ] Timo Walther commented on FLINK-10166: -- [~dangdangdang] Throwing a hard exception is valid here. This is a bug in our dependency management that needs to be fixed. > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Assignee: Shimin Yang >Priority: Blocker > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613318#comment-16613318 ] Shimin Yang commented on FLINK-10166: - Hi [~twalthr]. Well, not many times. Maybe we could add some hint in the log instead of throw a exception with message "Table program cannot be compiled. This is a bug. Please file an issue.". > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Assignee: Shimin Yang >Priority: Blocker > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613151#comment-16613151 ] Timo Walther commented on FLINK-10166: -- [~dangdangdang] can you check where and how many time \{{commons-codec}} is used? I think we use much more than just base64 functionality. If it is not too much effort, we should try to get rid of this dependency. Otherwise we should relocate it to avoid dependency conflicts. > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Assignee: Shimin Yang >Priority: Blocker > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613136#comment-16613136 ] Shimin Yang commented on FLINK-10166: - Hi [~dawidwys] [~StephanEwen], I think a more elegant way to solve this problem is to not relocate commons-codec. What do you think? > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Blocker > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16585706#comment-16585706 ] Stephan Ewen commented on FLINK-10166: -- Looks like an dependency that is used, but not declared (only implicitly inherited in some cases). To guard against that in the future, there is an enforcer rule to prevent referencing transitive dependencies: https://maven.apache.org/enforcer/enforcer-rules/banTransitiveDependencies.html In this specific case, can we avoid using commons.codec and have a Base64 coder in Flink? Seems like a very small piece of functionality, and avoiding too many dependencies makes life easier for users of systems like Flink. > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Major > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10166) Dependency problems when executing SQL query in sql-client
[ https://issues.apache.org/jira/browse/FLINK-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583794#comment-16583794 ] Dawid Wysakowicz commented on FLINK-10166: -- I found out it happens only for the hadoopless binary. For others the {{commons-codec}} dependency is shipped with hadoop shaded uber jar. > Dependency problems when executing SQL query in sql-client > -- > > Key: FLINK-10166 > URL: https://issues.apache.org/jira/browse/FLINK-10166 > Project: Flink > Issue Type: Bug > Components: Table API & SQL >Affects Versions: 1.6.0 >Reporter: Dawid Wysakowicz >Priority: Major > > When tried to run query: > {code} > select count(distinct name) from (Values ('a'), ('b')) AS NameTable(name) > {code} > in {{sql-client.sh}} I got: > {code} > [ERROR] Could not execute SQL statement. Reason: > org.codehaus.commons.compiler.CompileException: Line 43, Column 10: Unknown > variable or type "org.apache.commons.codec.binary.Base64" > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)