Re: Return of Flink shading problems in 1.2.0
Here is the JIRA: https://issues.apache.org/jira/browse/FLINK-6125 On Mon, Mar 20, 2017 at 10:27 AM, Robert Metzger wrote: > Hi Craig, > > I was able to reproduce the issue with maven 3.3 in Flink 1.2. I'll look > into it. > > On Fri, Mar 17, 2017 at 11:56 PM, Foster, Craig > wrote: > >> Ping. So I’ve built with 3.0.5 and it does give proper shading. So it >> does get me yet another workaround where my only recourse is to use a max >> version of Maven. Still, I feel there should be a long-term fix at some >> point in time. >> >> >> >> I also believe there is a regression in Flink 1.2.0 for Maven 3.3.x with >> the process as documented, so hoping someone can at least duplicate or let >> me know of a new workaround for 3.3.x. >> >> >> >> Thanks! >> >> Craig >> >> >> >> *From: *"Foster, Craig" >> *Reply-To: *"user@flink.apache.org" >> *Date: *Friday, March 17, 2017 at 7:23 AM >> *To: *"user@flink.apache.org" >> *Cc: *Ufuk Celebi , Robert Metzger , >> Stephan Ewen >> *Subject: *Re: Return of Flink shading problems in 1.2.0 >> >> >> >> Hey Stephen: >> >> I am building twice in every case described in my previous mail. Well, >> building then rebuilding the flink-dist submodule. >> >> >> >> This was fixed in BigTop but I started seeing this issue again with Flink >> 1.2.0. I was wondering if there's something else in the environment that >> could prevent the shading from working because it isn't now even with the >> workaround. >> >> >> On Mar 17, 2017, at 4:08 AM, Stephan Ewen wrote: >> >> Hi Craig! >> >> >> >> Maven 3.3.x has a shading problem. You need to build two times, once from >> root, once inside "flink-dist". Have a look here: >> >> >> >> https://ci.apache.org/projects/flink/flink-docs-release-1.2/ >> setup/building.html#dependency-shading >> >> >> >> Maybe that way missed in BigTop? >> >> >> >> I am wondering if we should actually throw an error if building with >> Maven 3.3.x - too many users run into that issue. >> >> >> >> Stephan >> >> >> >> >> >> >> >> On Fri, Mar 17, 2017 at 8:14 AM, Ufuk Celebi wrote: >> >> Pulling in Robert and Stephan who know the project's shading setup the >> best. >> >> >> On Fri, Mar 17, 2017 at 6:52 AM, Foster, Craig >> wrote: >> > Hi: >> > >> > A few months ago, I was building Flink and ran into shading issues for >> > flink-dist as described in your docs. We resolved this in BigTop by >> adding >> > the correct way to build flink-dist in the do-component-build script and >> > everything was fine after that. >> > >> > >> > >> > Now, I’m running into issues doing the same now in Flink 1.2.0 and I’m >> > trying to figure out what’s changed and how to fix it. Here’s how the >> > flink-dist jar looks with proper shading: >> > >> > >> > >> > jar -tvf /usr/lib/flink/lib/flink-dist_2.10-1.1.4.jar | grep >> > HttpConnectionParams >> > 2485 Tue Jan 01 00:00:00 UTC 1980 >> > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient >> /params/HttpConnectionParams.class >> > 3479 Tue Jan 01 00:00:00 UTC 1980 >> > org/apache/flink/hadoop/shaded/org/apache/http/params/HttpCo >> nnectionParams.class >> > >> > >> > >> > When I build Flink 1.2.0 in BigTop, here’s shading for the jar found in >> the >> > RPM: >> > >> > >> > >> > jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams >> > 2392 Tue Jan 01 00:00:00 GMT 1980 >> > org/apache/commons/httpclient/params/HttpConnectionParams.class >> > 2485 Tue Jan 01 00:00:00 GMT 1980 >> > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient >> /params/HttpConnectionParams.class >> > 3479 Tue Jan 01 00:00:00 GMT 1980 >> > org/apache/flink/hadoop/shaded/org/apache/http/params/HttpCo >> nnectionParams.class >> > 2868 Tue Jan 01 00:00:00 GMT 1980 >> > org/apache/http/params/HttpConnectionParams.class >> > >> > >> > >> > I thought maybe it was some strange thing going on with BigTop, so then >> I >> > tried just straight building Flink 1.2.0 (outside BigTop) and get the >> same >> > shading: >> > >
Re: Return of Flink shading problems in 1.2.0
Hi Craig, I was able to reproduce the issue with maven 3.3 in Flink 1.2. I'll look into it. On Fri, Mar 17, 2017 at 11:56 PM, Foster, Craig wrote: > Ping. So I’ve built with 3.0.5 and it does give proper shading. So it does > get me yet another workaround where my only recourse is to use a max > version of Maven. Still, I feel there should be a long-term fix at some > point in time. > > > > I also believe there is a regression in Flink 1.2.0 for Maven 3.3.x with > the process as documented, so hoping someone can at least duplicate or let > me know of a new workaround for 3.3.x. > > > > Thanks! > > Craig > > > > *From: *"Foster, Craig" > *Reply-To: *"user@flink.apache.org" > *Date: *Friday, March 17, 2017 at 7:23 AM > *To: *"user@flink.apache.org" > *Cc: *Ufuk Celebi , Robert Metzger , > Stephan Ewen > *Subject: *Re: Return of Flink shading problems in 1.2.0 > > > > Hey Stephen: > > I am building twice in every case described in my previous mail. Well, > building then rebuilding the flink-dist submodule. > > > > This was fixed in BigTop but I started seeing this issue again with Flink > 1.2.0. I was wondering if there's something else in the environment that > could prevent the shading from working because it isn't now even with the > workaround. > > > On Mar 17, 2017, at 4:08 AM, Stephan Ewen wrote: > > Hi Craig! > > > > Maven 3.3.x has a shading problem. You need to build two times, once from > root, once inside "flink-dist". Have a look here: > > > > https://ci.apache.org/projects/flink/flink-docs- > release-1.2/setup/building.html#dependency-shading > > > > Maybe that way missed in BigTop? > > > > I am wondering if we should actually throw an error if building with Maven > 3.3.x - too many users run into that issue. > > > > Stephan > > > > > > > > On Fri, Mar 17, 2017 at 8:14 AM, Ufuk Celebi wrote: > > Pulling in Robert and Stephan who know the project's shading setup the > best. > > > On Fri, Mar 17, 2017 at 6:52 AM, Foster, Craig > wrote: > > Hi: > > > > A few months ago, I was building Flink and ran into shading issues for > > flink-dist as described in your docs. We resolved this in BigTop by > adding > > the correct way to build flink-dist in the do-component-build script and > > everything was fine after that. > > > > > > > > Now, I’m running into issues doing the same now in Flink 1.2.0 and I’m > > trying to figure out what’s changed and how to fix it. Here’s how the > > flink-dist jar looks with proper shading: > > > > > > > > jar -tvf /usr/lib/flink/lib/flink-dist_2.10-1.1.4.jar | grep > > HttpConnectionParams > > 2485 Tue Jan 01 00:00:00 UTC 1980 > > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/ > HttpConnectionParams.class > > 3479 Tue Jan 01 00:00:00 UTC 1980 > > org/apache/flink/hadoop/shaded/org/apache/http/params/ > HttpConnectionParams.class > > > > > > > > When I build Flink 1.2.0 in BigTop, here’s shading for the jar found in > the > > RPM: > > > > > > > > jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams > > 2392 Tue Jan 01 00:00:00 GMT 1980 > > org/apache/commons/httpclient/params/HttpConnectionParams.class > > 2485 Tue Jan 01 00:00:00 GMT 1980 > > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/ > HttpConnectionParams.class > > 3479 Tue Jan 01 00:00:00 GMT 1980 > > org/apache/flink/hadoop/shaded/org/apache/http/params/ > HttpConnectionParams.class > > 2868 Tue Jan 01 00:00:00 GMT 1980 > > org/apache/http/params/HttpConnectionParams.class > > > > > > > > I thought maybe it was some strange thing going on with BigTop, so then I > > tried just straight building Flink 1.2.0 (outside BigTop) and get the > same > > shading: > > > > > > > > jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams > > > > 2485 Fri Mar 17 05:41:16 GMT 2017 > > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/ > HttpConnectionParams.class > > > > 3479 Fri Mar 17 05:41:16 GMT 2017 > > org/apache/flink/hadoop/shaded/org/apache/http/params/ > HttpConnectionParams.class > > > > 2392 Fri Mar 17 05:41:24 GMT 2017 > > org/apache/commons/httpclient/params/HttpConnectionParams.class > > > > 2868 Fri Mar 17 05:41:24 GMT 2017 > > org/apache/http/params/HttpConnectionParams.class > > > > > > > > And, yes, this is after going into flink-dist and running mvn clean > install > > again since I am using Maven 3.3.x. > > > > > > > > Here’s a snippet from my Maven version: > > > > mvn -version > > > > Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; > > 2015-11-10T16:41:47+00:00) > > > > Maven home: /usr/local/apache-maven > > > > Java version: 1.8.0_121, vendor: Oracle Corporation > > > > > > > > Any ideas on what my problem might be here? > > > > > > > > Thanks, > > > > Craig > > > > > > > >
Re: Return of Flink shading problems in 1.2.0
Ping. So I’ve built with 3.0.5 and it does give proper shading. So it does get me yet another workaround where my only recourse is to use a max version of Maven. Still, I feel there should be a long-term fix at some point in time. I also believe there is a regression in Flink 1.2.0 for Maven 3.3.x with the process as documented, so hoping someone can at least duplicate or let me know of a new workaround for 3.3.x. Thanks! Craig From: "Foster, Craig" Reply-To: "user@flink.apache.org" Date: Friday, March 17, 2017 at 7:23 AM To: "user@flink.apache.org" Cc: Ufuk Celebi , Robert Metzger , Stephan Ewen Subject: Re: Return of Flink shading problems in 1.2.0 Hey Stephen: I am building twice in every case described in my previous mail. Well, building then rebuilding the flink-dist submodule. This was fixed in BigTop but I started seeing this issue again with Flink 1.2.0. I was wondering if there's something else in the environment that could prevent the shading from working because it isn't now even with the workaround. On Mar 17, 2017, at 4:08 AM, Stephan Ewen mailto:se...@apache.org>> wrote: Hi Craig! Maven 3.3.x has a shading problem. You need to build two times, once from root, once inside "flink-dist". Have a look here: https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/building.html#dependency-shading Maybe that way missed in BigTop? I am wondering if we should actually throw an error if building with Maven 3.3.x - too many users run into that issue. Stephan On Fri, Mar 17, 2017 at 8:14 AM, Ufuk Celebi mailto:u...@apache.org>> wrote: Pulling in Robert and Stephan who know the project's shading setup the best. On Fri, Mar 17, 2017 at 6:52 AM, Foster, Craig mailto:foscr...@amazon.com>> wrote: > Hi: > > A few months ago, I was building Flink and ran into shading issues for > flink-dist as described in your docs. We resolved this in BigTop by adding > the correct way to build flink-dist in the do-component-build script and > everything was fine after that. > > > > Now, I’m running into issues doing the same now in Flink 1.2.0 and I’m > trying to figure out what’s changed and how to fix it. Here’s how the > flink-dist jar looks with proper shading: > > > > jar -tvf /usr/lib/flink/lib/flink-dist_2.10-1.1.4.jar | grep > HttpConnectionParams > 2485 Tue Jan 01 00:00:00 UTC 1980 > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/HttpConnectionParams.class > 3479 Tue Jan 01 00:00:00 UTC 1980 > org/apache/flink/hadoop/shaded/org/apache/http/params/HttpConnectionParams.class > > > > When I build Flink 1.2.0 in BigTop, here’s shading for the jar found in the > RPM: > > > > jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams > 2392 Tue Jan 01 00:00:00 GMT 1980 > org/apache/commons/httpclient/params/HttpConnectionParams.class > 2485 Tue Jan 01 00:00:00 GMT 1980 > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/HttpConnectionParams.class > 3479 Tue Jan 01 00:00:00 GMT 1980 > org/apache/flink/hadoop/shaded/org/apache/http/params/HttpConnectionParams.class > 2868 Tue Jan 01 00:00:00 GMT 1980 > org/apache/http/params/HttpConnectionParams.class > > > > I thought maybe it was some strange thing going on with BigTop, so then I > tried just straight building Flink 1.2.0 (outside BigTop) and get the same > shading: > > > > jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams > > 2485 Fri Mar 17 05:41:16 GMT 2017 > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/HttpConnectionParams.class > > 3479 Fri Mar 17 05:41:16 GMT 2017 > org/apache/flink/hadoop/shaded/org/apache/http/params/HttpConnectionParams.class > > 2392 Fri Mar 17 05:41:24 GMT 2017 > org/apache/commons/httpclient/params/HttpConnectionParams.class > > 2868 Fri Mar 17 05:41:24 GMT 2017 > org/apache/http/params/HttpConnectionParams.class > > > > And, yes, this is after going into flink-dist and running mvn clean install > again since I am using Maven 3.3.x. > > > > Here’s a snippet from my Maven version: > > mvn -version > > Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; > 2015-11-10T16:41:47+00:00) > > Maven home: /usr/local/apache-maven > > Java version: 1.8.0_121, vendor: Oracle Corporation > > > > Any ideas on what my problem might be here? > > > > Thanks, > > Craig > >
Re: Return of Flink shading problems in 1.2.0
Hey Stephen: I am building twice in every case described in my previous mail. Well, building then rebuilding the flink-dist submodule. This was fixed in BigTop but I started seeing this issue again with Flink 1.2.0. I was wondering if there's something else in the environment that could prevent the shading from working because it isn't now even with the workaround. On Mar 17, 2017, at 4:08 AM, Stephan Ewen mailto:se...@apache.org>> wrote: Hi Craig! Maven 3.3.x has a shading problem. You need to build two times, once from root, once inside "flink-dist". Have a look here: https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/building.html#dependency-shading Maybe that way missed in BigTop? I am wondering if we should actually throw an error if building with Maven 3.3.x - too many users run into that issue. Stephan On Fri, Mar 17, 2017 at 8:14 AM, Ufuk Celebi mailto:u...@apache.org>> wrote: Pulling in Robert and Stephan who know the project's shading setup the best. On Fri, Mar 17, 2017 at 6:52 AM, Foster, Craig mailto:foscr...@amazon.com>> wrote: > Hi: > > A few months ago, I was building Flink and ran into shading issues for > flink-dist as described in your docs. We resolved this in BigTop by adding > the correct way to build flink-dist in the do-component-build script and > everything was fine after that. > > > > Now, I'm running into issues doing the same now in Flink 1.2.0 and I'm > trying to figure out what's changed and how to fix it. Here's how the > flink-dist jar looks with proper shading: > > > > jar -tvf /usr/lib/flink/lib/flink-dist_2.10-1.1.4.jar | grep > HttpConnectionParams > 2485 Tue Jan 01 00:00:00 UTC 1980 > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/HttpConnectionParams.class > 3479 Tue Jan 01 00:00:00 UTC 1980 > org/apache/flink/hadoop/shaded/org/apache/http/params/HttpConnectionParams.class > > > > When I build Flink 1.2.0 in BigTop, here's shading for the jar found in the > RPM: > > > > jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams > 2392 Tue Jan 01 00:00:00 GMT 1980 > org/apache/commons/httpclient/params/HttpConnectionParams.class > 2485 Tue Jan 01 00:00:00 GMT 1980 > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/HttpConnectionParams.class > 3479 Tue Jan 01 00:00:00 GMT 1980 > org/apache/flink/hadoop/shaded/org/apache/http/params/HttpConnectionParams.class > 2868 Tue Jan 01 00:00:00 GMT 1980 > org/apache/http/params/HttpConnectionParams.class > > > > I thought maybe it was some strange thing going on with BigTop, so then I > tried just straight building Flink 1.2.0 (outside BigTop) and get the same > shading: > > > > jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams > > 2485 Fri Mar 17 05:41:16 GMT 2017 > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/HttpConnectionParams.class > > 3479 Fri Mar 17 05:41:16 GMT 2017 > org/apache/flink/hadoop/shaded/org/apache/http/params/HttpConnectionParams.class > > 2392 Fri Mar 17 05:41:24 GMT 2017 > org/apache/commons/httpclient/params/HttpConnectionParams.class > > 2868 Fri Mar 17 05:41:24 GMT 2017 > org/apache/http/params/HttpConnectionParams.class > > > > And, yes, this is after going into flink-dist and running mvn clean install > again since I am using Maven 3.3.x. > > > > Here's a snippet from my Maven version: > > mvn -version > > Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; > 2015-11-10T16:41:47+00:00) > > Maven home: /usr/local/apache-maven > > Java version: 1.8.0_121, vendor: Oracle Corporation > > > > Any ideas on what my problem might be here? > > > > Thanks, > > Craig > >
Re: Return of Flink shading problems in 1.2.0
Hi Craig! Maven 3.3.x has a shading problem. You need to build two times, once from root, once inside "flink-dist". Have a look here: https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/building.html#dependency-shading Maybe that way missed in BigTop? I am wondering if we should actually throw an error if building with Maven 3.3.x - too many users run into that issue. Stephan On Fri, Mar 17, 2017 at 8:14 AM, Ufuk Celebi wrote: > Pulling in Robert and Stephan who know the project's shading setup the > best. > > On Fri, Mar 17, 2017 at 6:52 AM, Foster, Craig > wrote: > > Hi: > > > > A few months ago, I was building Flink and ran into shading issues for > > flink-dist as described in your docs. We resolved this in BigTop by > adding > > the correct way to build flink-dist in the do-component-build script and > > everything was fine after that. > > > > > > > > Now, I’m running into issues doing the same now in Flink 1.2.0 and I’m > > trying to figure out what’s changed and how to fix it. Here’s how the > > flink-dist jar looks with proper shading: > > > > > > > > jar -tvf /usr/lib/flink/lib/flink-dist_2.10-1.1.4.jar | grep > > HttpConnectionParams > > 2485 Tue Jan 01 00:00:00 UTC 1980 > > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/ > HttpConnectionParams.class > > 3479 Tue Jan 01 00:00:00 UTC 1980 > > org/apache/flink/hadoop/shaded/org/apache/http/params/ > HttpConnectionParams.class > > > > > > > > When I build Flink 1.2.0 in BigTop, here’s shading for the jar found in > the > > RPM: > > > > > > > > jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams > > 2392 Tue Jan 01 00:00:00 GMT 1980 > > org/apache/commons/httpclient/params/HttpConnectionParams.class > > 2485 Tue Jan 01 00:00:00 GMT 1980 > > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/ > HttpConnectionParams.class > > 3479 Tue Jan 01 00:00:00 GMT 1980 > > org/apache/flink/hadoop/shaded/org/apache/http/params/ > HttpConnectionParams.class > > 2868 Tue Jan 01 00:00:00 GMT 1980 > > org/apache/http/params/HttpConnectionParams.class > > > > > > > > I thought maybe it was some strange thing going on with BigTop, so then I > > tried just straight building Flink 1.2.0 (outside BigTop) and get the > same > > shading: > > > > > > > > jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams > > > > 2485 Fri Mar 17 05:41:16 GMT 2017 > > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/ > HttpConnectionParams.class > > > > 3479 Fri Mar 17 05:41:16 GMT 2017 > > org/apache/flink/hadoop/shaded/org/apache/http/params/ > HttpConnectionParams.class > > > > 2392 Fri Mar 17 05:41:24 GMT 2017 > > org/apache/commons/httpclient/params/HttpConnectionParams.class > > > > 2868 Fri Mar 17 05:41:24 GMT 2017 > > org/apache/http/params/HttpConnectionParams.class > > > > > > > > And, yes, this is after going into flink-dist and running mvn clean > install > > again since I am using Maven 3.3.x. > > > > > > > > Here’s a snippet from my Maven version: > > > > mvn -version > > > > Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; > > 2015-11-10T16:41:47+00:00) > > > > Maven home: /usr/local/apache-maven > > > > Java version: 1.8.0_121, vendor: Oracle Corporation > > > > > > > > Any ideas on what my problem might be here? > > > > > > > > Thanks, > > > > Craig > > > > >
Re: Return of Flink shading problems in 1.2.0
Pulling in Robert and Stephan who know the project's shading setup the best. On Fri, Mar 17, 2017 at 6:52 AM, Foster, Craig wrote: > Hi: > > A few months ago, I was building Flink and ran into shading issues for > flink-dist as described in your docs. We resolved this in BigTop by adding > the correct way to build flink-dist in the do-component-build script and > everything was fine after that. > > > > Now, I’m running into issues doing the same now in Flink 1.2.0 and I’m > trying to figure out what’s changed and how to fix it. Here’s how the > flink-dist jar looks with proper shading: > > > > jar -tvf /usr/lib/flink/lib/flink-dist_2.10-1.1.4.jar | grep > HttpConnectionParams > 2485 Tue Jan 01 00:00:00 UTC 1980 > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/HttpConnectionParams.class > 3479 Tue Jan 01 00:00:00 UTC 1980 > org/apache/flink/hadoop/shaded/org/apache/http/params/HttpConnectionParams.class > > > > When I build Flink 1.2.0 in BigTop, here’s shading for the jar found in the > RPM: > > > > jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams > 2392 Tue Jan 01 00:00:00 GMT 1980 > org/apache/commons/httpclient/params/HttpConnectionParams.class > 2485 Tue Jan 01 00:00:00 GMT 1980 > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/HttpConnectionParams.class > 3479 Tue Jan 01 00:00:00 GMT 1980 > org/apache/flink/hadoop/shaded/org/apache/http/params/HttpConnectionParams.class > 2868 Tue Jan 01 00:00:00 GMT 1980 > org/apache/http/params/HttpConnectionParams.class > > > > I thought maybe it was some strange thing going on with BigTop, so then I > tried just straight building Flink 1.2.0 (outside BigTop) and get the same > shading: > > > > jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams > > 2485 Fri Mar 17 05:41:16 GMT 2017 > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/HttpConnectionParams.class > > 3479 Fri Mar 17 05:41:16 GMT 2017 > org/apache/flink/hadoop/shaded/org/apache/http/params/HttpConnectionParams.class > > 2392 Fri Mar 17 05:41:24 GMT 2017 > org/apache/commons/httpclient/params/HttpConnectionParams.class > > 2868 Fri Mar 17 05:41:24 GMT 2017 > org/apache/http/params/HttpConnectionParams.class > > > > And, yes, this is after going into flink-dist and running mvn clean install > again since I am using Maven 3.3.x. > > > > Here’s a snippet from my Maven version: > > mvn -version > > Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; > 2015-11-10T16:41:47+00:00) > > Maven home: /usr/local/apache-maven > > Java version: 1.8.0_121, vendor: Oracle Corporation > > > > Any ideas on what my problem might be here? > > > > Thanks, > > Craig > >
Return of Flink shading problems in 1.2.0
Hi: A few months ago, I was building Flink and ran into shading issues for flink-dist as described in your docs. We resolved this in BigTop by adding the correct way to build flink-dist in the do-component-build script and everything was fine after that. Now, I’m running into issues doing the same now in Flink 1.2.0 and I’m trying to figure out what’s changed and how to fix it. Here’s how the flink-dist jar looks with proper shading: jar -tvf /usr/lib/flink/lib/flink-dist_2.10-1.1.4.jar | grep HttpConnectionParams 2485 Tue Jan 01 00:00:00 UTC 1980 org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/HttpConnectionParams.class 3479 Tue Jan 01 00:00:00 UTC 1980 org/apache/flink/hadoop/shaded/org/apache/http/params/HttpConnectionParams.class When I build Flink 1.2.0 in BigTop, here’s shading for the jar found in the RPM: jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams 2392 Tue Jan 01 00:00:00 GMT 1980 org/apache/commons/httpclient/params/HttpConnectionParams.class 2485 Tue Jan 01 00:00:00 GMT 1980 org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/HttpConnectionParams.class 3479 Tue Jan 01 00:00:00 GMT 1980 org/apache/flink/hadoop/shaded/org/apache/http/params/HttpConnectionParams.class 2868 Tue Jan 01 00:00:00 GMT 1980 org/apache/http/params/HttpConnectionParams.class I thought maybe it was some strange thing going on with BigTop, so then I tried just straight building Flink 1.2.0 (outside BigTop) and get the same shading: jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams 2485 Fri Mar 17 05:41:16 GMT 2017 org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/HttpConnectionParams.class 3479 Fri Mar 17 05:41:16 GMT 2017 org/apache/flink/hadoop/shaded/org/apache/http/params/HttpConnectionParams.class 2392 Fri Mar 17 05:41:24 GMT 2017 org/apache/commons/httpclient/params/HttpConnectionParams.class 2868 Fri Mar 17 05:41:24 GMT 2017 org/apache/http/params/HttpConnectionParams.class And, yes, this is after going into flink-dist and running mvn clean install again since I am using Maven 3.3.x. Here’s a snippet from my Maven version: mvn -version Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00) Maven home: /usr/local/apache-maven Java version: 1.8.0_121, vendor: Oracle Corporation Any ideas on what my problem might be here? Thanks, Craig