potiuk commented on code in PR #36660: URL: https://github.com/apache/airflow/pull/36660#discussion_r1445135899
########## airflow/providers/amazon/provider.yaml: ########## @@ -81,23 +81,14 @@ dependencies: - apache-airflow-providers-common-sql>=1.3.1 - apache-airflow-providers-http # We should update minimum version of boto3 and here regularly to avoid `pip` backtracking with the number - # of candidates to consider. We should also make sure that all the below related packages have also the - # same minimum version specified. Boto3 1.28.0 has been released on July 6 2023. We should also make sure we - # set it to the version that `aiobotocore` supports (see `aiobotocore` optional dependency at the end - # of this file). Currently we set aiobotocore as minimum 2.5.3 - as this is was the first version - # that supported boto3 1.28. - # NOTE!!! BEFORE botocore 1.33 BOTOCORE VERSIONS ARE SHIFTED BY 3 MINOR VERSIONS - # NOTE!!! Make sure to update _MIN_BOTO3_VERSION in setup.py when you update it here - - boto3>=1.28.0 - # NOTE!!! BEFORE botocore 1.33 version is always shifted by 3 MINOR VERSIONS from boto3, see: - # - https://github.com/boto/boto3/issues/2702 - # - https://github.com/boto/boto3/blob/a3e710df58676c2567b8023dbe09025ad5e0e50f/.changes/1.33.0.json#L13-L15 - - botocore>=1.31.0 + # of candidates to consider. Make sure to configure boto3 version here as well as in all the tools below + # in the `devel-dependencies` section to be the same minimum version. + - boto3>=1.33.0 + - botocore>=1.33.0 - inflection>=0.5.1 - # Allow a wider range of watchtower versions for flexibility among users - - watchtower>=2.0.1,<4 + - watchtower>=3.0.0,<4 Review Comment: > Is there any reason we want to limit the watchtower version to v3.x? Is it required for updating any other dependency? While testing the new packaging for Airflow in https://github.com/apache/airflow/pull/36537 I just noticed that there was a long backtracking from `pip` and I decided to just upgrade a few obvious candidates. I did it one-by -one and I saw each of them was causing shorter back-tracking - I saw watchtower as one of the culprits, so I bumped it while testing. But eventually, I think watchtower is not the major contributor - it adds a little, But I think there were other reasons (some of teething problems in the new packaging that I fixed this afternoon) . I think the other bumps (especially botocore) could be the main cause for "longer" backtracking so I think it's not an issue to leave watchtower limited to 2.0.1. I will see when I merge all the oustanding PRs and rebase #36537 - this can be easily seen by the time "upgrade to newer dependencies" take. Particularly this step: https://github.com/apache/airflow/actions/runs/7449867636/job/20268244835?pr=36537 - It takes 2m20s for my last attempt. If we can keep it to few minutes rather than 20/30 then it is completely no issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
