Re: Custom (mongodb) hook as a plugin

2016-10-03 Thread Jakob Homan
On 23 September 2016 at 15:06, Rob Goretsky wrote: > More generally, would it make sense to create an enhancement to the plugin > architecture that allows the developer of a new hook to also specify the > name of a new 'connection type' along with what fields would be required > for that connectio

Re: Custom (mongodb) hook as a plugin

2016-09-23 Thread Rob Goretsky
I have been building a plugin with hooks for Redshift and Teradata, and initially was also confused as to why my I didn't see Redshift and Teradata appear as "Connection Types" in the Connections admin web interface. I then realized, as Andrew Phillips pointed out here, that the list of connection

Re: Custom (mongodb) hook as a plugin

2016-09-23 Thread Maxime Beauchemin
False, your hook will still read from the centralized connection store, even if it lives outside the Airflow package. Max On Fri, Sep 23, 2016 at 2:05 PM, Shin Tai wrote: > Ok - thanks. Sounds like the sort of thing Airflow isn't meant to do right > now because it wouldn't be possible to config

Re: Custom (mongodb) hook as a plugin

2016-09-23 Thread Shin Tai
Ok - thanks. Sounds like the sort of thing Airflow isn't meant to do right now because it wouldn't be possible to configure the connector with usernames, passwords etc I may just have to fork for the time being. Thanks again. On 23 September 2016 at 17:40, Andrew Phillips wrote: > 6. Went to U

Re: Custom (mongodb) hook as a plugin

2016-09-23 Thread Andrew Phillips
6. Went to UI/Connections but couldn't find my connector/hook The experts on the list will hopefully set me right I'm my understanding is incorrect, but from what I can see the list of hook types available *in the UI* is fixed: https://github.com/apache/incubator-airflow/blob/f360414774f1ecb