> I think that the operator should describe the behavior(verb), hook should > describe a service(noun). These are other parts of speech. > In my opinion, HiveOperator should be named HiveExecuteQuery. Hook will > be > called Hive. Then there will be no name conflicts
I'm not sure I like Hive as a bare class name - it's a little bit ... "bare"? (I think I prefer HiveHook or HiveService as a name). But I do like the verb/noun idea. That's a really nice way of thinking about what belongs where. On 18 April 2019 21:38:34 BST, "Kamil Breguła" <[email protected]> wrote: >Comments inline > >On Thu, Apr 18, 2019 at 10:18 PM Arthur Wiedmer ><[email protected]> >wrote: > >> Comments inline >> >> On Thu, Apr 18, 2019, 11:10 Kamil Breguła <[email protected]> >> wrote: >> >> > On Thu, Apr 18, 2019 at 7:30 PM Arthur Wiedmer ><[email protected] >> > >> > wrote: >> > >> > > Hi Kamil, >> > > >> > > Thank you for this doc, it was a lot of good work to put all of >this >> > > together. Could you open the doc for comment? >> > > May I recommend that once comments have been added to eventually >port >> the >> > > doc to the wiki and keep trace of the history there as well. >> > > >> > > I would prefer not to turn on comments in this document. This >will make >> > it >> > difficult to manage this document. Turned on comments are also the >> ability >> > to add your own suggestions. History of comments is the second >reason >> why I >> > did turn on it. Saving the discussion history in the comments will >be >> very >> > hard. The official communication channel is a mailing list. >> > >> >> Sure. But then why not send the entire proposal via email as well to >> preserve history? If you feel it is too much for email, maybe the >wiki >> would be a better place. >> >> In particular, comments in the mailing list might refer to a version >of the >> doc which is not available anymore. We will be missing context >eventually. >> >> You can not put this document on the mailing list because the >document >contains advanced formatting. > >I will try to put this document in the wiki in the near future to the >archival purpose. I would like to point out that this is not the final >document and contains the final recommendations. I wanted to publish >final >recommendations as AIP on the wiki in the future. I treat this document >as >an intermediate step to develop the final document. > > >> > Case #3 I agree about some form of submodules for large cloud >providers. >> It >> > > sounds helpful. One question: where would I contribute an >> > > S3_to_GCS_operator in some of the new models? >> > > >> > > >> > "Operators that integrate with two services will not change." >> > They will be in the package with the remaining operators. The >specific >> > place depends on the adoption of other solutions. >> > >> >> I am mostly worried about dependencies. What if the single >integration >> breaks the multiple integration operators. We do not have this issue >yet, >> but it might crop up once we split them more evenly. >> >> I am not sure if I understood you correctly. Changes in imports >should not >cause pip dependency problems. The changes presented in the document >are >not intended to divide Airflow into many pip packages. > >> >> > > Case #5 Are we talking about the class name or the file name? The >class >> > > name is fine to me, but we could remove _operator from the file >name. >> > > >> > >> > Case #2 describes the change of file names >> > Case #5 describes the change of class names >> > >> > I added examples to two cases to better describe the changes. >> > >> >> Thank you. I guess I see it for file names, but I am wondering about >the >> operators and name collisions. >> Say I need both the HiveOperator and I inline a custom operator for >which I >> need a hive hook. >> # Would you recommmend the following? Or does case #5 only apply to >> Operators? >> from airflow.operators.hive import Hive as HiveOperator >> from airfow.hooks.hive import Hive as HiveHook >> >> I was just thinking it might be nice to be able to import what I need >> without renaming and not worry too much about names shadowing others. >> Especially if I am new-ish to Apache Airflow. >> >> >I think that the operator should describe the behavior(verb), hook >should >describe a service(noun). These are other parts of speech. >In my opinion, HiveOperator should be named HiveExecuteQuery. Hook will >be >called Hive. Then there will be no name conflicts. > > > >> Best, >> Arthur >> > > >-- > >Kamil Breguła >Polidea <https://www.polidea.com/> | Software Engineer > >M: +48 505 458 451 <+48505458451> >E: [email protected] >[image: Polidea] <https://www.polidea.com/> > >We create human & business stories through technology. >Check out our projects! <https://www.polidea.com/our-work> >[image: Github] <https://github.com/Polidea> [image: Facebook] ><https://www.facebook.com/Polidea.Software> [image: Twitter] ><https://twitter.com/polidea> [image: Linkedin] ><https://www.linkedin.com/company/polidea> [image: Instagram] ><https://instagram.com/polidea> [image: Behance] ><https://www.behance.net/polidea>
