This is an automated email from the ASF dual-hosted git repository.
yuanzhou pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new 766bebf813 [GLUTEN-9860][VL] Upload nightly releases to apache nightly
site (#9888)
766bebf813 is described below
commit 766bebf813580d0c41b91012a67ce02930a3fbd5
Author: Yuan <[email protected]>
AuthorDate: Wed Jun 11 05:21:40 2025 +0100
[GLUTEN-9860][VL] Upload nightly releases to apache nightly site (#9888)
Upload nightly release jars to
https://nightlies.apache.org/gluten/
Signed-off-by: Yuan <[email protected]>
---
.github/workflows/velox_nightly.yml | 74 +++++++++++++++++++++++++++++++++----
README.md | 3 +-
docs/release.md | 14 ++++---
3 files changed, 78 insertions(+), 13 deletions(-)
diff --git a/.github/workflows/velox_nightly.yml
b/.github/workflows/velox_nightly.yml
index b880a9d329..0e5cb298f2 100644
--- a/.github/workflows/velox_nightly.yml
+++ b/.github/workflows/velox_nightly.yml
@@ -25,6 +25,7 @@ on:
env:
ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true
CCACHE_DIR: "${{ github.workspace }}/.ccache"
+ SETUP: 'bash .github/workflows/util/setup_helper.sh'
concurrency:
group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{
github.workflow }}
@@ -65,7 +66,7 @@ jobs:
path: .m2/repository/org/apache/arrow/
name: velox-arrow-jar-centos-7-${{github.sha}}
- build-bundle-package-centos8:
+ build-bundle-package-centos8-jdk8:
needs: build-native-lib
runs-on: ubuntu-22.04
container: centos:8
@@ -85,24 +86,83 @@ jobs:
run: |
sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-* && \
sed -i
's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g'
/etc/yum.repos.d/CentOS-* && \
- yum update -y && yum install -y java-1.8.0-openjdk-devel wget && \
- wget
https://downloads.apache.org/maven/maven-3/3.8.8/binaries/apache-maven-3.8.8-bin.tar.gz
&& \
- tar -xvf apache-maven-3.8.8-bin.tar.gz && \
- mv apache-maven-3.8.8 /usr/lib/maven
+ yum update -y && yum install -y java-1.8.0-openjdk-devel wget
+ $SETUP install_maven
- name: Get current date
id: date
run: echo "::set-output name=date::$(date +'%Y-%m-%d')"
- - name: Build for Spark 3.5
+ - name: Build package for Spark
run: |
cd $GITHUB_WORKSPACE/ && \
export MAVEN_HOME=/usr/lib/maven && \
export PATH=${PATH}:${MAVEN_HOME}/bin && \
+ mvn clean install -Pspark-3.2 -Pbackends-velox -Pceleborn -Puniffle
-DskipTests -Dmaven.source.skip
+ mvn clean install -Pspark-3.3 -Pbackends-velox -Pceleborn -Puniffle
-DskipTests -Dmaven.source.skip
+ mvn clean install -Pspark-3.4 -Pbackends-velox -Pceleborn -Puniffle
-DskipTests -Dmaven.source.skip
mvn clean install -Pspark-3.5 -Pbackends-velox -Pceleborn -Puniffle
-DskipTests -Dmaven.source.skip
- name: Upload bundle package
uses: actions/upload-artifact@v4
with:
- name: nightly-gluten-velox-bundle-package-spark35-${{
steps.date.outputs.date }}
+ name: nightly-gluten-velox-bundle-package-jdk8-${{
steps.date.outputs.date }}
path: package/target/gluten-velox-bundle-*.jar
retention-days: 7
+ - name: rsync to apache nightly
+ uses: burnett01/[email protected]
+ with:
+ switches: -avzr
+ path: package/target/gluten-velox-bundle-*.jar
+ remote_path: ${{ secrets.NIGHTLIES_RSYNC_PATH
}}/gluten/nightly-release-jdk8
+ remote_host: ${{ secrets.NIGHTLIES_RSYNC_HOST }}
+ remote_port: ${{ secrets.NIGHTLIES_RSYNC_PORT }}
+ remote_user: ${{ secrets.NIGHTLIES_RSYNC_USER }}
+ remote_key: ${{ secrets.NIGHTLIES_RSYNC_KEY }}
+ build-bundle-package-centos8-jdk17:
+ needs: build-native-lib
+ runs-on: ubuntu-22.04
+ container: centos:8
+ steps:
+ - uses: actions/checkout@v4
+ - name: Download All Artifacts
+ uses: actions/download-artifact@v4
+ with:
+ name: velox-native-lib-${{github.sha}}
+ path: ./cpp/build/releases
+ - name: Download All Arrow Jar Artifacts
+ uses: actions/download-artifact@v4
+ with:
+ name: velox-arrow-jar-centos-7-${{github.sha}}
+ path: /root/.m2/repository/org/apache/arrow/
+ - name: Setup java and maven
+ run: |
+ sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-* && \
+ sed -i
's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g'
/etc/yum.repos.d/CentOS-* && \
+ yum update -y && yum install -y java-17-openjdk-devel wget
+ $SETUP install_maven
+ - name: Get current date
+ id: date
+ run: echo "::set-output name=date::$(date +'%Y-%m-%d')"
+ - name: Build package for Spark
+ run: |
+ cd $GITHUB_WORKSPACE/ && \
+ export MAVEN_HOME=/usr/lib/maven && \
+ export PATH=${PATH}:${MAVEN_HOME}/bin && \
+ mvn clean install -Pspark-3.4 -Pjava-17 -Pbackends-velox -Pceleborn
-Puniffle -DskipTests -Dmaven.source.skip
+ mvn clean install -Pspark-3.5 -Pjava-17 -Pbackends-velox -Pceleborn
-Puniffle -DskipTests -Dmaven.source.skip
+ - name: Upload bundle package
+ uses: actions/upload-artifact@v4
+ with:
+ name: nightly-gluten-velox-bundle-package-jdk17-${{
steps.date.outputs.date }}
+ path: package/target/gluten-velox-bundle-*.jar
+ retention-days: 7
+ - name: rsync to apache nightly
+ uses: burnett01/[email protected]
+ with:
+ switches: -avzr
+ path: package/target/gluten-velox-bundle-*.jar
+ remote_path: ${{ secrets.NIGHTLIES_RSYNC_PATH
}}/gluten/nightly-release-jdk17
+ remote_host: ${{ secrets.NIGHTLIES_RSYNC_HOST }}
+ remote_port: ${{ secrets.NIGHTLIES_RSYNC_PORT }}
+ remote_user: ${{ secrets.NIGHTLIES_RSYNC_USER }}
+ remote_key: ${{ secrets.NIGHTLIES_RSYNC_KEY }}
diff --git a/README.md b/README.md
index e3fbf3524f..bc5911a918 100644
--- a/README.md
+++ b/README.md
@@ -76,7 +76,8 @@ There are two ways to acquire Gluten jar for the above
configuration.
### Use Released Jar
Please download a tar package
[here](https://downloads.apache.org/incubator/gluten/), then extract out Gluten
jar from it.
-It was verified on Centos-7, Centos-8, Ubuntu-20.04 and Ubuntu-22.04.
+Additionally, Gluten offers nightly builds based on the main branch, which are
available for early testing. You can find these release jars at this link:
[Apache Gluten Nightlies](https://nightlies.apache.org/gluten/).
+It was verified on Centos-7, Centos-8, Centos-9, Ubuntu-20.04 and Ubuntu-22.04.
### Build From Source
For **Velox** backend, please refer to [Velox.md](./docs/get-started/Velox.md)
and [build-guide.md](./docs/get-started/build-guide.md).
diff --git a/docs/release.md b/docs/release.md
index 99661c07c9..11f15c6360 100644
--- a/docs/release.md
+++ b/docs/release.md
@@ -7,16 +7,20 @@ nav_order: 11
[Gluten](https://github.com/apache/incubator-gluten) is a plugin for Apache
Spark to double SparkSQL's performance.
## Latest release for Velox backend
-*
[Gluten-1.2.0](https://github.com/apache/incubator-gluten/releases/tag/v1.2.0)
(Sep. 3 2024)
+*
[Gluten-1.4.0](https://github.com/apache/incubator-gluten/releases/tag/v1.4.0)
(Jun 2025)
+
+## Nightly release for Velox backend
+* [Apache Nightly](https://nightlies.apache.org/gluten/)
## Planned release for Velox backend
-* [Gluten-1.2.1] (Nov. 30 2024)
-* [Gluten-1.3.0] (Dec. 30 2024)
-* [Gluten-1.3.1] (Jan. 30 2025)
-* [Gluten-1.4.0] (Mar. 30 2025)
+* [Gluten-1.5.0] (Jul. 30 2025)
+* [Gluten-1.6.0] (Sep. 30 2025)
+* [Gluten-1.7.0] (Dec. 30 2025)
* To be updated
## Archived releases
+*
[Gluten-1.3.0](https://github.com/apache/incubator-gluten/releases/tag/v1.3.0)
(Jan. 24 2025)
+*
[Gluten-1.2.0](https://github.com/apache/incubator-gluten/releases/tag/v1.2.0)
(Sep. 3 2024)
*
[Gluten-1.1.1](https://github.com/apache/incubator-gluten/releases/tag/v1.1.1)
(Mar. 2 2024)
*
[Gluten-1.1.0](https://github.com/apache/incubator-gluten/releases/tag/v1.1.0)
(Nov. 30 2023)
*
[Gluten-1.0.0](https://github.com/apache/incubator-gluten/releases/tag/v1.0.0)
(Jul. 13 2023)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
