This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/flink.git
commit f0d01903aaa517af9c1be26b1244a778189dce65 Author: Chesnay Schepler <ches...@apache.org> AuthorDate: Fri May 27 00:07:04 2022 +0200 [FLINK-28016][ci] Update CI to use Maven 3.8.6 --- .mvn/wrapper/maven-wrapper.properties | 2 +- README.md | 2 +- azure-pipelines.yml | 6 ++--- docs/content.zh/docs/flinkDev/building.md | 25 --------------------- docs/content/docs/flinkDev/building.md | 35 ----------------------------- pom.xml | 5 +---- tools/azure-pipelines/build-apache-repo.yml | 6 ++--- tools/ci/maven-utils.sh | 2 +- 8 files changed, 10 insertions(+), 73 deletions(-) diff --git a/.mvn/wrapper/maven-wrapper.properties b/.mvn/wrapper/maven-wrapper.properties index d1ff65c6a56..57bb584385e 100644 --- a/.mvn/wrapper/maven-wrapper.properties +++ b/.mvn/wrapper/maven-wrapper.properties @@ -14,5 +14,5 @@ # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. -distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.2.5/apache-maven-3.2.5-bin.zip +distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.8.6/apache-maven-3.8.6-bin.zip wrapperUrl=https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.1.0/maven-wrapper-3.1.0.jar diff --git a/README.md b/README.md index 9b6452bf8fd..b2691132ffa 100644 --- a/README.md +++ b/README.md @@ -69,7 +69,7 @@ Prerequisites for building Flink: * Unix-like environment (we use Linux, Mac OS X, Cygwin, WSL) * Git -* Maven (we recommend version 3.2.5 and require at least 3.1.1) +* Maven (we recommend version 3.8.6 and require at least 3.1.1) * Java 8 or 11 (Java 9 or 10 may work) ``` diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 76dbdc07bf2..1ff4d3ba266 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -36,10 +36,10 @@ trigger: resources: containers: - # Container with Maven 3.2.5, SSL to have the same environment everywhere. - # see https://github.com/flink-ci/flink-ci-docker + # Container with Maven 3.8.6, SSL to have the same environment everywhere. + # see https://github.com/apache/flink-connector-shared-utils/tree/ci_utils - container: flink-build-container - image: chesnay/flink-ci:java_8_11_17 + image: chesnay/flink-ci:java_8_11_17_maven_386 # On AZP provided machines, set this flag to allow writing coredumps in docker options: --privileged diff --git a/docs/content.zh/docs/flinkDev/building.md b/docs/content.zh/docs/flinkDev/building.md index 2b5b52b8853..a1cfafc676f 100644 --- a/docs/content.zh/docs/flinkDev/building.md +++ b/docs/content.zh/docs/flinkDev/building.md @@ -37,8 +37,6 @@ under the License. 还需要准备 **Maven 3** 和 **JDK** (Java开发套件)。Flink 依赖 **Java 8 (deprecated) 或 Java 11** 来进行构建。 -*注意:Maven 3.3.x 可以构建 Flink,但是不能正确地屏蔽掉指定的依赖。Maven 3.2.5 可以正确地构建库文件。 - 输入以下命令从 Git 克隆代码 ```bash @@ -117,29 +115,6 @@ python -m pip install apache-flink-libraries/dist/*.tar.gz python -m pip install dist/*.whl ``` -## 依赖屏蔽 - -Flink [屏蔽](https://maven.apache.org/plugins/maven-shade-plugin/)了一些它使用的包,这样做是为了避免与程序员自己引入的包的存在的可能的版本冲突。屏蔽掉的包包括 *Google Guava*,*Asm*,*Apache Curator*,*Apache HTTP Components*,*Netty* 等。 - -这种依赖屏蔽机制最近在 Maven 中有所改变。需要用户根据 Maven 的的不同版本来执行不同的命令。 - -**对于Maven 3.1.x and 3.2.x** -直接在 Flink 源码根目录执行命令 `mvn clean install -DskipTests` 就足够了。 - -**Maven 3.3.x** -如下的构建需要两步走:第一步需要在基础目录下执行编译构建;第二步需要在编译后的 flink-dist 目录下执行: - -```bash -mvn clean install -DskipTests -cd flink-dist -mvn clean install -``` - -*注意:* 运行 `mvn --version` 以查看Maven的版本。 - -{{< top >}} - - ## Scala 版本 {{< hint info >}} diff --git a/docs/content/docs/flinkDev/building.md b/docs/content/docs/flinkDev/building.md index 5159f1db92f..eb8fef01f99 100644 --- a/docs/content/docs/flinkDev/building.md +++ b/docs/content/docs/flinkDev/building.md @@ -35,8 +35,6 @@ In order to build Flink you need the source code. Either [download the source of In addition you need **Maven 3** and a **JDK** (Java Development Kit). Flink requires **Java 8 (deprecated) or Java 11** to build. -*NOTE: Maven 3.3.x can build Flink, but will not properly shade away certain dependencies. Maven 3.2.5 creates the libraries properly.* - To clone from git, enter: ```bash @@ -115,39 +113,6 @@ The sdist and wheel packages of `apache-flink` will be found under `./flink-pyth python -m pip install dist/*.whl ``` -## Dependency Shading - -Flink [shades away](https://maven.apache.org/plugins/maven-shade-plugin/) some of the libraries it uses, in order to avoid version clashes with user programs that use different versions of these libraries. Among the shaded libraries are *Google Guava*, *Asm*, *Apache Curator*, *Apache HTTP Components*, *Netty*, and others. - -The dependency shading mechanism was recently changed in Maven and requires users to build Flink slightly differently, depending on their Maven version: - -**Maven and 3.2.x** -It is sufficient to call `mvn clean install -DskipTests` in the root directory of Flink code base. - -**Maven 3.3.x** -The build has to be done in two steps: First in the base directory, then in shaded modules, such as the distribution and the filesystems: - -```bash -# build overall project -mvn clean install -DskipTests - -# build shaded modules used in dist again, for example: -cd flink-filesystems/flink-s3-fs-presto/ -mvn clean install -DskipTests -# ... and other modules - -# build dist again to include shaded modules -cd flink-dist -mvn clean install -``` - -*Note:* To check your Maven version, run `mvn --version`. - -*Note:* We recommend using the latest Maven 3.2.x version for building production-grade Flink distributions, as this is the version the Flink developers are using for the official releases and testing. - -{{< top >}} - - ## Scala Versions {{< hint info >}} diff --git a/pom.xml b/pom.xml index a881b395669..5e1d8a33b06 100644 --- a/pom.xml +++ b/pom.xml @@ -1336,8 +1336,7 @@ under the License. <configuration> <rules> <requireMavenVersion> - <!-- maven version must be lower than 3.3. See FLINK-3158 --> - <version>(,3.3)</version> + <version>3.8.6</version> </requireMavenVersion> <requireJavaVersion> <version>1.8.0</version> @@ -1593,8 +1592,6 @@ under the License. <exclude>tools/japicmp-output/**</exclude> <!-- artifacts created during release process --> <exclude>tools/releasing/release/**</exclude> - <!-- manually installed version on travis --> - <exclude>apache-maven-3.2.5/**</exclude> <!-- PyCharm --> <exclude>**/.idea/**</exclude> <!-- Generated code via Avro --> diff --git a/tools/azure-pipelines/build-apache-repo.yml b/tools/azure-pipelines/build-apache-repo.yml index 3102d18acc7..03e685337fa 100644 --- a/tools/azure-pipelines/build-apache-repo.yml +++ b/tools/azure-pipelines/build-apache-repo.yml @@ -36,10 +36,10 @@ trigger: resources: containers: - # Container with Maven 3.2.5, SSL to have the same environment everywhere. - # see https://github.com/flink-ci/flink-ci-docker + # Container with Maven 3.8.6, SSL to have the same environment everywhere. + # see https://github.com/apache/flink-connector-shared-utils/tree/ci_utils - container: flink-build-container - image: chesnay/flink-ci:java_8_11_17 + image: chesnay/flink-ci:java_8_11_17_maven_386 variables: MAVEN_CACHE_FOLDER: $(Pipeline.Workspace)/.m2/repository diff --git a/tools/ci/maven-utils.sh b/tools/ci/maven-utils.sh index a11b5482751..ed27b7ae208 100755 --- a/tools/ci/maven-utils.sh +++ b/tools/ci/maven-utils.sh @@ -83,7 +83,7 @@ function collect_coredumps { CI_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" -MAVEN_VERSION="3.2.5" +MAVEN_VERSION="3.8.6" MAVEN_CACHE_DIR=${HOME}/maven_cache MAVEN_VERSIONED_DIR=${MAVEN_CACHE_DIR}/apache-maven-${MAVEN_VERSION}