[jira] [Commented] (FLINK-21142) Flink guava Dependence problem
[ https://issues.apache.org/jira/browse/FLINK-21142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17352456#comment-17352456 ] Timo Walther commented on FLINK-21142: -- I have created a PR for improving this issue. Please let me know what you think there. > Flink guava Dependence problem > -- > > Key: FLINK-21142 > URL: https://issues.apache.org/jira/browse/FLINK-21142 > Project: Flink > Issue Type: Bug > Components: Connectors / Hadoop Compatibility, Connectors / Hive >Affects Versions: 1.12.0 >Reporter: YUJIANBO >Assignee: Timo Walther >Priority: Minor > > We set up a new Hadoop cluster, and we use the flink1.12.0 compiled by the > previous release-1.12.0 branch.If I add hive jar to flink/lib/, it will > report errors. > *Operating environment：* > flink1.12.0 > Hadoop 3.3.0 > hive 3.1.2 > *Flink run official demo shell： /tmp/yjb/buildjar/flink1.12.0/bin/flink run > -m yarn-cluster /usr/local/flink1.12.0/examples/streaming/WordCount.jar* > If I put one of the jar *flink-sql-connector-hive-3.1.2_2.11-1.12.0.jar* or > *hive-exec-3.1.2.jar* in the Lib directory and execute the above shell, an > error will be reported java.lang.NoSuchMethodError : com.google.common . > base.Preconditions.checkArgument (ZLjava/lang/String;Ljava/lang/Object;)V. > *We can see that it's the dependency conflict of guava.* > *My cluster guava‘s version:* > /usr/local/hadoop-3.3.0/share/hadoop/yarn/csi/lib/guava-20.0.jar > /usr/local/hadoop-3.3.0/share/hadoop/common/lib/guava-27.0-jre.jar > /usr/local/apache-hive-3.1.2-bin/lib/guava-20.0.jar > /usr/local/apache-hive-3.1.2-bin/lib/jersey-guava-2.25.1.jar > /usr/local/spark-3.0.1-bin-hadoop3.2/jars/guava-14.0.1.jar > *Can you give me some advice？* > Thank you！ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-21142) Flink guava Dependence problem
[ https://issues.apache.org/jira/browse/FLINK-21142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335933#comment-17335933 ] Flink Jira Bot commented on FLINK-21142: This issue was labeled "stale-major" 7 ago and has not received any updates so it is being deprioritized. If this ticket is actually Major, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Flink guava Dependence problem > -- > > Key: FLINK-21142 > URL: https://issues.apache.org/jira/browse/FLINK-21142 > Project: Flink > Issue Type: Bug > Components: Connectors / Hadoop Compatibility, Connectors / Hive >Affects Versions: 1.12.0 >Reporter: YUJIANBO >Priority: Major > Labels: stale-major > > We set up a new Hadoop cluster, and we use the flink1.12.0 compiled by the > previous release-1.12.0 branch.If I add hive jar to flink/lib/, it will > report errors. > *Operating environment：* > flink1.12.0 > Hadoop 3.3.0 > hive 3.1.2 > *Flink run official demo shell： /tmp/yjb/buildjar/flink1.12.0/bin/flink run > -m yarn-cluster /usr/local/flink1.12.0/examples/streaming/WordCount.jar* > If I put one of the jar *flink-sql-connector-hive-3.1.2_2.11-1.12.0.jar* or > *hive-exec-3.1.2.jar* in the Lib directory and execute the above shell, an > error will be reported java.lang.NoSuchMethodError : com.google.common . > base.Preconditions.checkArgument (ZLjava/lang/String;Ljava/lang/Object;)V. > *We can see that it's the dependency conflict of guava.* > *My cluster guava‘s version:* > /usr/local/hadoop-3.3.0/share/hadoop/yarn/csi/lib/guava-20.0.jar > /usr/local/hadoop-3.3.0/share/hadoop/common/lib/guava-27.0-jre.jar > /usr/local/apache-hive-3.1.2-bin/lib/guava-20.0.jar > /usr/local/apache-hive-3.1.2-bin/lib/jersey-guava-2.25.1.jar > /usr/local/spark-3.0.1-bin-hadoop3.2/jars/guava-14.0.1.jar > *Can you give me some advice？* > Thank you！ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-21142) Flink guava Dependence problem
[ https://issues.apache.org/jira/browse/FLINK-21142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17327379#comment-17327379 ] Flink Jira Bot commented on FLINK-21142: This major issue is unassigned and itself and all of its Sub-Tasks have not been updated for 30 days. So, it has been labeled "stale-major". If this ticket is indeed "major", please either assign yourself or give an update. Afterwards, please remove the label. In 7 days the issue will be deprioritized. > Flink guava Dependence problem > -- > > Key: FLINK-21142 > URL: https://issues.apache.org/jira/browse/FLINK-21142 > Project: Flink > Issue Type: Bug > Components: Connectors / Hadoop Compatibility, Connectors / Hive >Affects Versions: 1.12.0 >Reporter: YUJIANBO >Priority: Major > Labels: stale-major > > We set up a new Hadoop cluster, and we use the flink1.12.0 compiled by the > previous release-1.12.0 branch.If I add hive jar to flink/lib/, it will > report errors. > *Operating environment：* > flink1.12.0 > Hadoop 3.3.0 > hive 3.1.2 > *Flink run official demo shell： /tmp/yjb/buildjar/flink1.12.0/bin/flink run > -m yarn-cluster /usr/local/flink1.12.0/examples/streaming/WordCount.jar* > If I put one of the jar *flink-sql-connector-hive-3.1.2_2.11-1.12.0.jar* or > *hive-exec-3.1.2.jar* in the Lib directory and execute the above shell, an > error will be reported java.lang.NoSuchMethodError : com.google.common . > base.Preconditions.checkArgument (ZLjava/lang/String;Ljava/lang/Object;)V. > *We can see that it's the dependency conflict of guava.* > *My cluster guava‘s version:* > /usr/local/hadoop-3.3.0/share/hadoop/yarn/csi/lib/guava-20.0.jar > /usr/local/hadoop-3.3.0/share/hadoop/common/lib/guava-27.0-jre.jar > /usr/local/apache-hive-3.1.2-bin/lib/guava-20.0.jar > /usr/local/apache-hive-3.1.2-bin/lib/jersey-guava-2.25.1.jar > /usr/local/spark-3.0.1-bin-hadoop3.2/jars/guava-14.0.1.jar > *Can you give me some advice？* > Thank you！ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-21142) Flink guava Dependence problem
[ https://issues.apache.org/jira/browse/FLINK-21142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17271953#comment-17271953 ] Rui Li commented on FLINK-21142: [~YUJIANBO] That depends on how the guava jars are used in your job. The principle is to avoid different versions of the same guava class in the class path. So you don't need to replace if it's not added to class path. > Flink guava Dependence problem > -- > > Key: FLINK-21142 > URL: https://issues.apache.org/jira/browse/FLINK-21142 > Project: Flink > Issue Type: Bug > Components: Connectors / Hadoop Compatibility, Connectors / Hive >Affects Versions: 1.12.0 >Reporter: YUJIANBO >Priority: Major > > We set up a new Hadoop cluster, and we use the flink1.12.0 compiled by the > previous release-1.12.0 branch.If I add hive jar to flink/lib/, it will > report errors. > *Operating environment：* > flink1.12.0 > Hadoop 3.3.0 > hive 3.1.2 > *Flink run official demo shell： /tmp/yjb/buildjar/flink1.12.0/bin/flink run > -m yarn-cluster /usr/local/flink1.12.0/examples/streaming/WordCount.jar* > If I put one of the jar *flink-sql-connector-hive-3.1.2_2.11-1.12.0.jar* or > *hive-exec-3.1.2.jar* in the Lib directory and execute the above shell, an > error will be reported java.lang.NoSuchMethodError : com.google.common . > base.Preconditions.checkArgument (ZLjava/lang/String;Ljava/lang/Object;)V. > *We can see that it's the dependency conflict of guava.* > *My cluster guava‘s version:* > /usr/local/hadoop-3.3.0/share/hadoop/yarn/csi/lib/guava-20.0.jar > /usr/local/hadoop-3.3.0/share/hadoop/common/lib/guava-27.0-jre.jar > /usr/local/apache-hive-3.1.2-bin/lib/guava-20.0.jar > /usr/local/apache-hive-3.1.2-bin/lib/jersey-guava-2.25.1.jar > /usr/local/spark-3.0.1-bin-hadoop3.2/jars/guava-14.0.1.jar > *Can you give me some advice？* > Thank you！ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-21142) Flink guava Dependence problem
[ https://issues.apache.org/jira/browse/FLINK-21142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17271912#comment-17271912 ] YUJIANBO commented on FLINK-21142: -- [~lirui]That's also my question. I didn't notice your letter. Thank you very much. I have quesiton, I will try first method to rebuild hive-exe, but my cluster guava' version next. Should I replace all guava dependencies with 27.0-jre's version？ /usr/local/hadoop-3.3.0/share/hadoop/yarn/csi/lib/guava-20.0.jar /usr/local/hadoop-3.3.0/share/hadoop/common/lib/guava-27.0-jre.jar /usr/local/apache-hive-3.1.2-bin/lib/guava-20.0.jar /usr/local/apache-hive-3.1.2-bin/lib/jersey-guava-2.25.1.jar /usr/local/spark-3.0.1-bin-hadoop3.2/jars/guava-14.0.1.jar > Flink guava Dependence problem > -- > > Key: FLINK-21142 > URL: https://issues.apache.org/jira/browse/FLINK-21142 > Project: Flink > Issue Type: Bug > Components: Connectors / Hadoop Compatibility, Connectors / Hive >Affects Versions: 1.12.0 >Reporter: YUJIANBO >Priority: Major > > We set up a new Hadoop cluster, and we use the flink1.12.0 compiled by the > previous release-1.12.0 branch.If I add hive jar to flink/lib/, it will > report errors. > *Operating environment：* > flink1.12.0 > Hadoop 3.3.0 > hive 3.1.2 > *Flink run official demo shell： /tmp/yjb/buildjar/flink1.12.0/bin/flink run > -m yarn-cluster /usr/local/flink1.12.0/examples/streaming/WordCount.jar* > If I put one of the jar *flink-sql-connector-hive-3.1.2_2.11-1.12.0.jar* or > *hive-exec-3.1.2.jar* in the Lib directory and execute the above shell, an > error will be reported java.lang.NoSuchMethodError : com.google.common . > base.Preconditions.checkArgument (ZLjava/lang/String;Ljava/lang/Object;)V. > *We can see that it's the dependency conflict of guava.* > *My cluster guava‘s version:* > /usr/local/hadoop-3.3.0/share/hadoop/yarn/csi/lib/guava-20.0.jar > /usr/local/hadoop-3.3.0/share/hadoop/common/lib/guava-27.0-jre.jar > /usr/local/apache-hive-3.1.2-bin/lib/guava-20.0.jar > /usr/local/apache-hive-3.1.2-bin/lib/jersey-guava-2.25.1.jar > /usr/local/spark-3.0.1-bin-hadoop3.2/jars/guava-14.0.1.jar > *Can you give me some advice？* > Thank you！ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-21142) Flink guava Dependence problem
[ https://issues.apache.org/jira/browse/FLINK-21142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17271899#comment-17271899 ] Rui Li commented on FLINK-21142: Hi [~YUJIANBO], the same issue has been reported in the [mailing list|http://apache-flink.147419.n8.nabble.com/Caused-by-java-lang-NoSuchMethodError-com-google-common-base-Preconditions-checkArgument-ZLjava-langV-td10474.html], and you can check out my reply there. The root cause is your hadoop version is incompatible with your hive version. Hive-3.1.2 depends on hadoop-3.1.0, so choosing a hadoop version <=3.1.0 would be safer. > Flink guava Dependence problem > -- > > Key: FLINK-21142 > URL: https://issues.apache.org/jira/browse/FLINK-21142 > Project: Flink > Issue Type: Bug > Components: Connectors / Hadoop Compatibility, Connectors / Hive >Affects Versions: 1.12.0 >Reporter: YUJIANBO >Priority: Major > > We set up a new Hadoop cluster, and we use the flink1.12.0 compiled by the > previous release-1.12.0 branch.If I add hive jar to flink/lib/, it will > report errors. > *Operating environment：* > flink1.12.0 > Hadoop 3.3.0 > hive 3.1.2 > *Flink run official demo shell： /tmp/yjb/buildjar/flink1.12.0/bin/flink run > -m yarn-cluster /usr/local/flink1.12.0/examples/streaming/WordCount.jar* > If I put one of the jar *flink-sql-connector-hive-3.1.2_2.11-1.12.0.jar* or > *hive-exec-3.1.2.jar* in the Lib directory and execute the above shell, an > error will be reported java.lang.NoSuchMethodError : com.google.common . > base.Preconditions.checkArgument (ZLjava/lang/String;Ljava/lang/Object;)V. > *We can see that it's the dependency conflict of guava.* > *My cluster guava‘s version:* > /usr/local/hadoop-3.3.0/share/hadoop/yarn/csi/lib/guava-20.0.jar > /usr/local/hadoop-3.3.0/share/hadoop/common/lib/guava-27.0-jre.jar > /usr/local/apache-hive-3.1.2-bin/lib/guava-20.0.jar > /usr/local/apache-hive-3.1.2-bin/lib/jersey-guava-2.25.1.jar > /usr/local/spark-3.0.1-bin-hadoop3.2/jars/guava-14.0.1.jar > *Can you give me some advice？* > Thank you！ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-21142) Flink guava Dependence problem
[ https://issues.apache.org/jira/browse/FLINK-21142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17271881#comment-17271881 ] YUJIANBO commented on FLINK-21142: -- [~jark] [~lirui] Hello！Can you give me some advice? > Flink guava Dependence problem > -- > > Key: FLINK-21142 > URL: https://issues.apache.org/jira/browse/FLINK-21142 > Project: Flink > Issue Type: Bug > Components: Connectors / Hadoop Compatibility, Connectors / Hive >Affects Versions: 1.12.0 >Reporter: YUJIANBO >Priority: Major > > We set up a new Hadoop cluster, and we use the flink1.12.0 compiled by the > previous release-1.12.0 branch.If I add hive jar to flink/lib/, it will > report errors. > *Operating environment：* > flink1.12.0 > Hadoop 3.3.0 > hive 3.1.2 > *Flink run official demo shell： /tmp/yjb/buildjar/flink1.12.0/bin/flink run > -m yarn-cluster /usr/local/flink1.12.0/examples/streaming/WordCount.jar* > If I put one of the jar *flink-sql-connector-hive-3.1.2_2.11-1.12.0.jar* or > *hive-exec-3.1.2.jar* in the Lib directory and execute the above command, an > error will be reported java.lang.NoSuchMethodError : com.google.common . > base.Preconditions.checkArgument (ZLjava/lang/String;Ljava/lang/Object;)V. > *We can see that it's the dependency conflict of guava.* > *My cluster guava‘s version:* > /usr/local/hadoop-3.3.0/share/hadoop/yarn/csi/lib/guava-20.0.jar > /usr/local/hadoop-3.3.0/share/hadoop/common/lib/guava-27.0-jre.jar > /usr/local/apache-hive-3.1.2-bin/lib/guava-20.0.jar > /usr/local/apache-hive-3.1.2-bin/lib/jersey-guava-2.25.1.jar > /usr/local/spark-3.0.1-bin-hadoop3.2/jars/guava-14.0.1.jar > *Can you give me some advice？* > Thank you！ -- This message was sent by Atlassian Jira (v8.3.4#803005)
