[GitHub] flink issue #3960: [FLINK-6654][build] let 'flink-dist' properly depend on '...
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/3960 I think this looks correct now. Merging for `master` and `release-1.3`... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3960: [FLINK-6654][build] let 'flink-dist' properly depend on '...
Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/3960 I'm not sure if the proposed fix is correct, because it will add the shaded hadoop uber project's dependencies into Flink dist. IIRC, this was the whole point of Stephan's recent refactoring of the dependencies. I did a quick check, just based on the flink-dist file size and content, and with the current rel 1.3 branch, we have the following file sizes: ``` -rw-r--r-- 1 robert robert 68M May 24 09:47 flink-dist_2.10-1.3-SNAPSHOT.jar -rw-r--r-- 1 robert robert 99K May 24 09:40 flink-python_2.10-1.3-SNAPSHOT.jar -rw-r--r-- 1 robert robert 23M May 24 09:47 flink-shaded-hadoop2-uber-1.3-SNAPSHOT.jar -rw-r--r-- 1 robert robert 479K Nov 17 2015 log4j-1.2.17.jar -rw-r--r-- 1 robert robert 8.7K Nov 17 2015 slf4j-log4j12-1.7.7.jar ``` after the change, the flink dist file grows significantly: ``` -rw-r--r-- 1 robert robert 87M May 24 09:51 flink-dist_2.10-1.3-SNAPSHOT.jar -rw-r--r-- 1 robert robert 99K May 24 09:40 flink-python_2.10-1.3-SNAPSHOT.jar -rw-r--r-- 1 robert robert 23M May 24 09:47 flink-shaded-hadoop2-uber-1.3-SNAPSHOT.jar -rw-r--r-- 1 robert robert 479K Nov 17 2015 log4j-1.2.17.jar -rw-r--r-- 1 robert robert 8.7K Nov 17 2015 slf4j-log4j12-1.7.7.jar ``` I quickly opened the file and I found all `org.apache.hadoop` classes in the dist again. I'm against doing a "quick fix" before the release here. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3960: [FLINK-6654][build] let 'flink-dist' properly depend on '...
Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/3960 Okay, I was able to reproduce the issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3960: [FLINK-6654][build] let 'flink-dist' properly depend on '...
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3960 you probably need to clean up your whole source with `mvn clean` and try from there - I'm not quite sure, my initial command will clean up everything that is not in the dependencies of the flink-dist project --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3960: [FLINK-6654][build] let 'flink-dist' properly depend on '...
Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/3960 Not sure about the initial issue. For me the command worked (without yoru changes on the 1.3 branch) ``` [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 10:12 min [INFO] Finished at: 2017-05-23T16:50:31+02:00 [INFO] Final Memory: 131M/441M [INFO] mvn clean install -pl flink-dist -am -DskipTests ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3960: [FLINK-6654][build] let 'flink-dist' properly depend on '...
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3960 I wasn't quite sure and went the safe way - can remove the exclusion though... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3960: [FLINK-6654][build] let 'flink-dist' properly depend on '...
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/3960 Thanks. The exclusion should not be necessary, since the `flink-shaded-hadoop2-uber` dependency shades everything into one jar and has no further dependencies. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
