+1 for bumping the major version of the few (3 or 4 or so) providers
that need this, and setting `apache-airflow>=2.1` in their requirements
then!
On Thu, 15 Apr, 2021 at 23:30, .... <[email protected]> wrote:
Yes. We want to use a feature that will only be available in Airflow
2.1. Do we allow such situations or do we want to try to maintain
backward compatibility with Airflow 2.0 at all cost? If we copy some
code to the providers we can still be backward compatible with Airflow
2.0.
Earlier, Jarek suggested that we can and should keep these compatible,
but then we came to an agreement that if we manage the package and
core versions properly, this may not be needed.
. Have I understood correctly, and more importantly, providers of v1
will continue to work with Airflow 2.1?
Yes. Since providers don't use a private API, we can move
_create_connection method to another package and make it public. This
is a safe operation.
czw., 15 kwi 2021 o 13:02 Ash Berlin-Taylor <[email protected]
<mailto:[email protected]>> napisał(a):
Bumping the major version of the providers is only half the story
-- what is the behaviour if someone has pins
apache-airflow-providers-foo==1.0.0 and installs apache-airflow==2.1
-- pip check wouldn't complain but the code might not work?
That's far from ideal.
Ah, looking at the PR linked up thread , I would call this is more
an issue off forward compat, rather than back compat, so the issue
is that we want to release providers that need a feature that only
exists in 2.1, and for any such provider, we will bump the major
version.
Have I understood correctly, and more importantly, providers of v1
will continue to work with Airflow 2.1?
-ash
On Wed, 14 Apr, 2021 at 22:31, Daniel Standish
<[email protected] <mailto:[email protected]>> wrote:
The proposal to bump major for all providers with every core minor
version seems like a reasonable solution to me, and it sounds like
there may be consensus on that? Though eventially the version
numbers may get pretty large :)
This discussion came to my attiontion from engagement with the
vault secrets enhancement. Looking at this as a test case for this
question, to implement this change while maintaining compatibility
with 2.0.0 would require an unacceptable amount of code duplication
and mess. And it's a relatively simple change (so we can imagine it
being much worse in other instances).
