alinaliBQ commented on code in PR #55321:
URL: https://github.com/apache/arrow/pull/55321#discussion_r4556446996
##########
.github/workflows/package_odbc.yml:
##########
@@ -179,3 +179,250 @@ jobs:
gh release upload $env:GITHUB_REF_NAME `
--clobber `
Apache-Arrow-Flight-SQL-ODBC-*-win64.msi
+
+ # Upload unsigned non-universal DYLIBs and the ARM64 base PKG
+ odbc-macos-upload-dylib:
+ name: ODBC macOS Upload Unsigned ${{ matrix.architecture }} DYLIB and
ARM64 PKG
+ runs-on: macos-${{ matrix.macos-version }}
+ if: github.event_name == 'push'
+ timeout-minutes: 120
+ strategy:
+ fail-fast: false
+ matrix:
+ include:
+ - { architecture: AMD64, macos-version: "26-intel" }
+ - { architecture: ARM64, macos-version: "26" }
+ env:
+ ARROW_BUILD_TESTS: OFF
+ ARROW_BUILD_TYPE: release
+ ARROW_DEPENDENCY_SOURCE: BUNDLED
+ ARROW_DEPENDENCY_USE_SHARED: OFF
+ ARROW_FLIGHT_SQL_ODBC: ON
+ ARROW_FLIGHT_SQL_ODBC_INSTALLER: ON
+ ARROW_HOME: /tmp/local
+ ARROW_MIMALLOC: OFF
+ MACOSX_DEPLOYMENT_TARGET: "14.0"
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v7
+ with:
+ persist-credentials: false
+ fetch-depth: 0
+ submodules: recursive
+ - name: Build ODBC macOS
+ uses: ./.github/actions/odbc-mac
+ with:
+ architecture: ${{ matrix.architecture }}
+ generate-installer: ${{ matrix.architecture == 'ARM64' }}
+ - name: Copy Unsigned ODBC DYLIB
+ run: |
+ # The unversioned name libarrow_flight_sql_odbc.dylib is a symlink to
+ # libarrow_flight_sql_odbc.<version>.dylib. Copy the real file,
keeping
+ # its versioned name, so the upload doesn't hit a dangling symlink.
+ mkdir ${{ matrix.architecture }}-dylib
+ cp "$(realpath build/cpp/${{ env.ARROW_BUILD_TYPE
}}/libarrow_flight_sql_odbc.dylib)" \
+ ${{ matrix.architecture }}-dylib/
+ - name: Rename ARM64 base ODBC PKG
+ if: matrix.architecture == 'ARM64'
+ run: |
+ pkg_path="$(ls build/cpp/ArrowFlightSQLODBC-*.pkg)"
+ mv "${pkg_path}" "$(basename "${pkg_path}" .pkg)_ARM64_base.pkg"
+ - name: Upload ODBC DYLIB to the job
+ uses: actions/upload-artifact@v7
+ with:
+ name: flight-sql-odbc-dylib-${{ matrix.architecture }}
+ path: ${{ matrix.architecture
}}-dylib/libarrow_flight_sql_odbc.*.dylib
+ if-no-files-found: error
+ - name: Upload ARM64 base ODBC PKG to the job
+ if: matrix.architecture == 'ARM64'
+ uses: actions/upload-artifact@v7
+ with:
+ name: flight-sql-odbc-pkg-base-ARM64
+ path: ArrowFlightSQLODBC-*_ARM64_base.pkg
+ if-no-files-found: error
+
+ # Upload unsigned universal DYLIB from ARM and AMD DYLIBs
+ odbc-macos-upload-universal-dylib:
+ needs: odbc-macos-upload-dylib
+ name: ODBC macOS Create Unsigned Universal DYLIB
+ runs-on: macos-26
+ timeout-minutes: 30
+ steps:
+ - name: Download ARM64 DYLIB
+ uses: actions/download-artifact@v8
+ with:
+ name: flight-sql-odbc-dylib-ARM64
+ path: arm64-dylib
+ - name: Download AMD64 DYLIB
+ uses: actions/download-artifact@v8
+ with:
+ name: flight-sql-odbc-dylib-AMD64
+ path: amd64-dylib
+ - name: Create universal DYLIB
+ run: |
+ set -euo pipefail
+
+ arm64_dylib="$(ls arm64-dylib/libarrow_flight_sql_odbc.*.dylib)"
+ amd64_dylib="$(ls amd64-dylib/libarrow_flight_sql_odbc.*.dylib)"
+ dylib_name="$(basename "${arm64_dylib}")"
+ if [ "$(basename "${amd64_dylib}")" != "${dylib_name}" ]; then
+ echo "ARM64 and AMD64 DYLIB names differ: ${arm64_dylib} vs
${amd64_dylib}"
+ exit 1
+ fi
+
+ # Rename unsigned DYLIB
+ mkdir universal-dylib
+
unsigned_path="universal-dylib/libarrow_flight_sql_odbc_unsigned.${dylib_name#libarrow_flight_sql_odbc.}"
+ lipo -create "${arm64_dylib}" "${amd64_dylib}" -output
"${unsigned_path}"
+ unsigned_archs="$(lipo -archs "${unsigned_path}")"
+ echo "${unsigned_path}: ${unsigned_archs}"
+ - name: Upload universal DYLIB to the job
+ uses: actions/upload-artifact@v7
+ with:
+ name: flight-sql-odbc-dylib-universal
+ path: universal-dylib/libarrow_flight_sql_odbc_unsigned.*.dylib
+ if-no-files-found: error
+
+ odbc-dylib-base-pkg-release:
+ needs:
+ - odbc-macos-upload-dylib
+ - odbc-macos-upload-universal-dylib
+ name: Upload Unsigned ODBC DYLIB and Base PKG to Release
+ runs-on: ubuntu-latest
+ permissions:
+ # Upload to GitHub Release
+ contents: write
+ steps:
+ - name: Checkout Arrow
+ uses: actions/checkout@v7
+ with:
+ persist-credentials: false
+ fetch-depth: 0
+ submodules: recursive
+ - name: Download the universal DYLIB
+ uses: actions/download-artifact@v8
+ with:
+ name: flight-sql-odbc-dylib-universal
+ - name: Download the ARM64 base PKG
+ uses: actions/download-artifact@v8
+ with:
+ name: flight-sql-odbc-pkg-base-ARM64
+ - name: Wait for creating GitHub Release
+ env:
+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ run: |
+ dev/release/utils-watch-gh-workflow.sh \
+ ${GITHUB_REF_NAME} \
+ release_candidate.yml
+ - name: Upload the artifacts to GitHub Release
+ env:
+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ run: |
+ gh release upload ${GITHUB_REF_NAME} \
+ --clobber \
+ libarrow_flight_sql_odbc_unsigned.*.dylib \
+ ArrowFlightSQLODBC-*_ARM64_base.pkg
+
+ # Download and repackage the ARM64 base PKG and
+ # the signed universal dylib
+ odbc-macos-upload-pkg:
+ name: ODBC macOS Repackage & Upload Unsigned Universal PKG
+ runs-on: macos-26
+ if: inputs.odbc_release_step
+ timeout-minutes: 30
+ permissions:
+ # Upload to GitHub Release
+ contents: write
+ steps:
+ - name: Download ARM64 base PKG and signed universal ODBC DYLIB
+ env:
+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ run: |
+ gh release download ${GITHUB_REF_NAME} \
+ --repo ${{ github.repository }} \
+ --pattern 'ArrowFlightSQLODBC-*_ARM64_base.pkg' \
+ --pattern 'libarrow_flight_sql_odbc.*.dylib' \
+ --clobber
Review Comment:
Hi @raulcd, I'm happy to explain here. The workflow is follows the same flow
as the ODBC Windows msi installer which is sign ODBC library .dll first, then
sign the newly built .msi installer which contains the signed ODBC library
file. Difference is that at step where we prepare for unsigned installer, we do
repackaging for macOS, and we build from scratch for Windows.
> The sign and upload (this is expected to be done locally) with the gpg
signature?
At this point the artifacts are signed, right?
Only the universal ODBC .dylib file is signed at 1st stage.
The file `'ArrowFlightSQLODBC-*_ARM64_base.pkg'` is left unsigned, because
the signature will be cleared during repackaging. So if
`'ArrowFlightSQLODBC-*_ARM64_base.pkg'` is signed at this stage it won't be
helpful to us.
> ... generating a new artifact ... Doesn't this require signing too?
Yes it does. The new repackaged artifact will be an unsigned universal .pkg
installer that contains the signed universal ODBC .dylib file (from 1st stage).
Then the universal .pkg file should be signed in 2nd stage. This way we get a
signed universal .pkg containing a signed ODBC .dylib file.
--------
Another different design route is, the workflow generates and uploads the
universal .pkg installer with unsigned universal ODBC .dylib file to Git
Release in one go, and then let the signing script
`dev/release/07-flightsqlodbc-upload.sh` handle the rest steps:
1. Extract the unsigned ODBC .dylib file from .pkg file to sign,
2. do the repackaging to put the signed .dylib file back to universal .pkg,
3. sign the universal .pkg.
But this will make the signing script too complicated I think.. It's
probably better to let CI handle the repackaging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
