potiuk edited a comment on pull request #12497:
URL: https://github.com/apache/airflow/pull/12497#issuecomment-730791402
I have no doubts that it is great to add extras even for those providers
that do not have any requirements. That's a good thing that was overlooked so
far and we shoudl do it definitely!
Regarding the refactor, I am a little on the fence here where it comes to
it. On one hand it removes duplication and might help with some errors
(elasticsearch being good example), on the other hand, so fart we've had all
extras in one place - as a nicely sorted list of extras and now we only see all
extras after they are combined (generated automatically in INSTALL docs which
is nice).
Until we remove some of the old ones in 2.1 those EXTRAS are a mixture of
old backwards-compatible "aliases" to the provider extras, extras like "devel",
and others. So we do not gain much.
However, maybe a better solution will be to further split those extras into
"ALIASES_REQUIREMENTS" (all those to be removed + kubernetes) and few other
groups for example: similar as we have in the reference docs and combine them
into single EXTRAS_REQUIREMENTS. We already have some "split" in the references
docs, but maybe we can review /resplit those groups as well (for example I
would rather see a "DEVEL_REQUIREMENTS" group and "DB_REQUIREMENTS",
"PROVIDERS" group rather than "Fundamentals", "Services" etc. Though - there is
also an overlap here ("postgres" provider and "postgres" db) which share the
same extra - this makes it a bit ambiguous.
We could even change the ALIASES ones to become real aliases and point to
the actual requirements rather than repeat the requirement arrays. That would
require a bit more logic to build the EXTRAS_REQUIREMENTS but might avoid
further errors. smth like that:
```
EXTRAS_REQUIREMENTS=.. combine the others ...
ALIAS_REQUIREMENTS: Dict[str, str] = {
"atlas": "apache.atlas",
"aws": "amazon",
....
}
EXTRAS_REQUIREMENTS.update({k,EXTRAS_REQUIREMENTS[k] for (k,v) in
ALIAS_REQUIREMENTS.items()})
```
Might be a little over-the-top but that would be "ultimate" duplication
removal and nice "structured" approach here.
@ashb @kaxil WDYT? Maybe you have another idea? Or maybe we should leave the
full list without any refactoring?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
