> > You can not put this document on the mailing list because the document > contains advanced formatting. > > I will try to put this document in the wiki in the near future to the > archival purpose. I would like to point out that this is not the final > document and contains the final recommendations. I wanted to publish final > recommendations as AIP on the wiki in the future. I treat this document as > an intermediate step to develop the final document. > > Fair enough. I do not think there is anything which could not have been explained via plain text.
> > > > Case #3 I agree about some form of submodules for large cloud > providers. > > It > > > > sounds helpful. One question: where would I contribute an > > > > S3_to_GCS_operator in some of the new models? > > > > > > > > > > > "Operators that integrate with two services will not change." > > > They will be in the package with the remaining operators. The specific > > > place depends on the adoption of other solutions. > > > > > > > I am mostly worried about dependencies. What if the single integration > > breaks the multiple integration operators. We do not have this issue yet, > > but it might crop up once we split them more evenly. > > > > I am not sure if I understood you correctly. Changes in imports should > not > cause pip dependency problems. The changes presented in the document are > not intended to divide Airflow into many pip packages. > Apologies, I got confused after reading more about namespaces and thought this is where we were going with this. But you are right it is a different and somewhat orthogonal concern. Please disregard my original comment. > > > > > > > Case #5 Are we talking about the class name or the file name? The > class > > > > name is fine to me, but we could remove _operator from the file name. > > > > > > > > > > Case #2 describes the change of file names > > > Case #5 describes the change of class names > > > > > > I added examples to two cases to better describe the changes. > > > > > > > Thank you. I guess I see it for file names, but I am wondering about the > > operators and name collisions. > > Say I need both the HiveOperator and I inline a custom operator for > which I > > need a hive hook. > > # Would you recommmend the following? Or does case #5 only apply to > > Operators? > > from airflow.operators.hive import Hive as HiveOperator > > from airfow.hooks.hive import Hive as HiveHook > > > > I was just thinking it might be nice to be able to import what I need > > without renaming and not worry too much about names shadowing others. > > Especially if I am new-ish to Apache Airflow. > > > > > I think that the operator should describe the behavior(verb), hook should > describe a service(noun). These are other parts of speech. > In my opinion, HiveOperator should be named HiveExecuteQuery. Hook will be > called Hive. Then there will be no name conflicts. > But we will still provide the backwards compatibility for a while with aliases to the old names, correct? I completely agree that the operator names have been a bit muddled, HiveExecuteQuery sounds better to me. Thanks again for taking the time, Kamil, I really appreciate it. Best, Arthur
