This is an automated email from the ASF dual-hosted git repository. msumit pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow-client-python.git
The following commit(s) were added to refs/heads/master by this push: new dd1b4ad Add README_RELEASE_CLIENT (#10) dd1b4ad is described below commit dd1b4adc261bbd99fb13804cdf30beb20d323eef Author: Sumit Maheshwari <msu...@users.noreply.github.com> AuthorDate: Mon Apr 12 16:34:33 2021 +0530 Add README_RELEASE_CLIENT (#10) --- .pre-commit-config.yaml | 4 +- dev/README_RELEASE_CLIENT.md | 269 +++++++++++++++++++++++++++++++++++++++++++ dev/sign.sh | 34 ++++++ requirements.txt | 29 ++--- test-requirements.txt | 29 ++--- 5 files changed, 335 insertions(+), 30 deletions(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index ea9d354..afda3cc 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -58,7 +58,7 @@ repos: - --fuzzy-match-generates-todo - id: insert-license name: Add license for all other files - exclude: ^\.github/.*$ + exclude: ^\.github/.*$|\.*LICENSE.txt$ args: - --comment-style - "|#|" @@ -66,4 +66,4 @@ repos: - license-templates/LICENSE.txt - --fuzzy-match-generates-todo files: > - \.properties$|\.cfg$|\.conf$|\.ini$$ \ No newline at end of file + \.properties$|\.cfg$|\.conf$|\.ini$|\.txt$$ \ No newline at end of file diff --git a/dev/README_RELEASE_CLIENT.md b/dev/README_RELEASE_CLIENT.md new file mode 100644 index 0000000..90d260e --- /dev/null +++ b/dev/README_RELEASE_CLIENT.md @@ -0,0 +1,269 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + --> + +You can find the prerequisites to release Apache Airflow Python Client in [README.md](README.md). + +# Prepare the Apache Airflow Python Client Package RC + +## Build RC artifacts + +The Release Candidate artifacts we vote upon should be the exact ones we vote against, without any modification than +renaming – i.e. the contents of the files must be the same between voted release candidate and final release. Because +of this the version in the built artifacts that will become the official Apache releases must not include the rcN suffix. + +- Set environment variables + + ```shell script + # Set Version + export VERSION=2.0.0rc1 + + + # Example after cloning + git clone https://github.com/apache/airflow-client-python.git airflow-client + cd airflow-client + export CLIENT_REPO_ROOT=$(pwd) + ``` + +- Set your version to 2.0.0 in `setup.py` (without the RC tag) +- Commit the version change. + +- Tag your release + + ```shell script + git tag -s ${VERSION} + ``` + +- Clean the checkout: the sdist step below will + + ```shell script + git clean -fxd + ``` + +- Tarball the repo + + ```shell script + git archive --format=tar.gz ${VERSION} -o airflow-client-${VERSION}-source.tar.gz + ``` + +- Generate sdist + + NOTE: Make sure your checkout is clean at this stage - any untracked or changed files will otherwise be included + in the file produced. + + ```shell script + python setup.py sdist bdist_wheel + ``` + +- Rename the sdist + + ```shell script + mv dist/airflow-client-${VERSION%rc?}.tar.gz airflow-client-${VERSION}-bin.tar.gz + mv dist/airflow_client-${VERSION%rc?}-py3-none-any.whl airflow_client-${VERSION}-py3-none-any.whl + ``` + +- Generate SHA512/ASC (If you have not generated a key yet, generate it by following instructions on + http://www.apache.org/dev/openpgp.html#key-gen-generate-key) + + ```shell script + ${CLIENT_REPO_ROOT}/dev/sign.sh airflow-client-${VERSION}-source.tar.gz + ${CLIENT_REPO_ROOT}/dev/sign.sh airflow-client-${VERSION}-bin.tar.gz + ${CLIENT_REPO_ROOT}/dev/sign.sh airflow_client-${VERSION}-py3-none-any.whl + ``` + +- Push the artifacts to ASF dev dist repo + +``` +# First clone the repo +svn checkout https://dist.apache.org/repos/dist/dev/airflow airflow-dev + +# Create new folder for the release +cd airflow-dev/clients/python +svn mkdir ${VERSION} + +# Move the artifacts to svn folder & commit +mv ${CLIENT_REPO_ROOT}/airflow{-,_}client-${VERSION}* ${VERSION}/ +cd ${VERSION} +svn add * +svn commit -m "Add artifacts for Airflow Python Client ${VERSION}" +``` + +## Prepare PyPI convenience "snapshot" packages + +At this point we have the artefact that we vote on, but as a convenience to developers we also want to +publish "snapshots" of the RC builds to pypi for installing via pip. + +To do this we need to + +- Build the package: + + ```shell script + cd ${CLIENT_REPO_ROOT} + python setup.py egg_info --tag-build "$(sed -e "s/^[0-9.]*//" <<<"$VERSION")" sdist bdist_wheel + ``` + +- Verify the artifacts that would be uploaded: + + ```shell script + twine check dist/* + ``` + +- Upload the package to PyPi's test environment: + + ```shell script + twine upload --repository-url=https://test.pypi.org/legacy/ dist/* + ``` + +- Verify that the test package looks good by downloading it and installing it into a virtual environment. The package + download link is available at: + https://test.pypi.org/project/airflow-client/#files + + Or via pypi + pip install -i https://test.pypi.org/simple/ airflow-client==$(VERSION) + +- Upload the package to PyPi's production environment: + `twine upload -r pypi dist/*` + +- Again, confirm that the package is available here: + https://pypi.python.org/pypi/airflow-client + +It is important to stress that this snapshot should not be named "release", and it +is not supposed to be used by and advertised to the end-users who do not read the devlist. + +- Push Tag for the release candidate + + ```shell script + git push origin ${VERSION} + ``` + +## Prepare Vote email on the Airflow Client release candidate +See Airflow process documented [here](https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#prepare-vote-email-on-the-apache-airflow-release-candidate) (just replace Airflow with Airflow Client). + +# Verify the release candidate by PMCs +See Airflow process documented [here](https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#verify-the-release-candidate-by-pmcs). + +## SVN check +See Airflow process documented [here](https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#svn-check) (just replace Airflow with Airflow Client). + +## Licence check +See Airflow process documented [here](https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#licence-check). + +## Signature check +See Airflow process documented [here](https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#signature-check). + + + +# Verify release candidates by Contributors +This can be done (and we encourage to) by any of the Contributors. In fact, it's best if the +actual users of Airflow Client test it in their own staging/test installations. Each release candidate +is available on PyPI apart from SVN packages, so everyone should be able to install +the release candidate version of Airflow Client via simply (<VERSION> is 2.0.0 for example, and <X> is +release candidate number 1,2,3,....). + +```shell script +pip install airflow-client==<VERSION>rc<X> +``` + +Once you install and run Airflow Client, you should perform any verification you see as necessary to check +that the client works as you expected. + +# Publish the final Airflow client release + +## Summarize the voting for the Airflow client release + +See Airflow process documented [here](https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#publish-the-final-apache-airflow-release) (just replace Airflow with Airflow Client). + +## Publish release to SVN + +```shell script +# First clone the repo +export RC=2.0.0rc1 +export VERSION=${RC/rc?/} +svn checkout https://dist.apache.org/repos/dist/release/airflow airflow-release + +# Create new folder for the release +cd airflow-release/clients/python +svn mkdir ${VERSION} +cd ${VERSION} + +# Move the artifacts to svn folder & commit +for f in ../../../airflow-dev/clients/python/$RC/*; do svn cp $f ${$(basename $f)/rc?/}; done +svn commit -m "Release Airflow Client ${VERSION} from ${RC}" + +# Remove old release +# http://www.apache.org/legal/release-policy.html#when-to-archive +cd .. +export PREVIOUS_VERSION=1.0.0 +svn rm ${PREVIOUS_VERSION} +svn commit -m "Remove old release: ${PREVIOUS_VERSION}" +``` + +Verify that the packages appear in [airflow](https://dist.apache.org/repos/dist/release/airflow/clients/python) + +## Prepare PyPI "release" packages + +At this point we release an official package: + +- Build the package: + + ```shell script + python setup.py sdist bdist_wheel + ``` + +- Verify the artifacts that would be uploaded: + + ```shell script + twine check dist/* + ``` + +- Upload the package to PyPi's test environment: + + ```shell script + twine upload --repository-url=https://test.pypi.org/legacy/ dist/* + ``` + +- Verify that the test package looks good by downloading it and installing it into a virtual environment. + The package download link is available at: https://test.pypi.org/project/airflow-client/#files + +- Upload the package to PyPi's production environment: + + ```shell script + twine upload -r pypi dist/* + ``` + +- Again, confirm that the package is available here: https://pypi.python.org/pypi/airflow-client + +## Update CHANGELOG.md + +- Get a diff between the last version and the current version: + + ```shell script + git log 1.0.0..2.0.0 --pretty=oneline + ``` + +- Update CHANGELOG.md with the details, and commit it. + +- Push Tag for the final version + + ```shell script + git push origin ${VERSION} + ``` + +## Notify developers of release + +See Airflow process documented [here](https://github.com/apache/airflow/blob/master/dev/README_RELEASE_AIRFLOW.md#notify-developers-of-release) (just replace Airflow with Airflow Client) \ No newline at end of file diff --git a/dev/sign.sh b/dev/sign.sh new file mode 100755 index 0000000..a1b6916 --- /dev/null +++ b/dev/sign.sh @@ -0,0 +1,34 @@ +#!/usr/bin/env bash +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +set -euo pipefail + +# Use this to sign the tar balls generated from +# python setup.py sdist --formats=gztar +# ie. sign.sh <my_tar_ball> +# you will still be required to type in your signing key password +# or it needs to be available in your keychain + +# Which key to sign releases with? This can be a (partial) email address or a +# key id. By default use any apache.org key +SIGN_WITH="${SIGN_WITH:-apache.org}" + +for name in "${@}" +do + gpg --armor --local-user "$SIGN_WITH" --output "${name}.asc" --detach-sig "${name}" + shasum -a 512 "${name}" > "${name}.sha512" +done diff --git a/requirements.txt b/requirements.txt index ce3b2f4..51a300c 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,19 +1,20 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# http://www.apache.org/licenses/LICENSE-2.0 # -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + python_dateutil >= 2.5.3 setuptools >= 21.0.0 urllib3 >= 1.25.3 diff --git a/test-requirements.txt b/test-requirements.txt index 8f2be00..2ccbc58 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -1,19 +1,20 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# http://www.apache.org/licenses/LICENSE-2.0 # -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + pytest-cov>=2.8.1 python_dateutil >= 2.5.3 urllib3 >= 1.25.3