Re: Custom Hook development guideline

2016-09-23 Thread Vikas Malhotra
Thanks Andrew and Max, I explored the docs and various code snippets shared by other developers. I have managed to use a custom hook in my dag. Regarding contributing back to the project, I would love to, I have explored the basic part of Adobe Analytics API as of now. I would add more

Re: Custom (mongodb) hook as a plugin

2016-09-23 Thread Maxime Beauchemin
False, your hook will still read from the centralized connection store, even if it lives outside the Airflow package. Max On Fri, Sep 23, 2016 at 2:05 PM, Shin Tai wrote: > Ok - thanks. Sounds like the sort of thing Airflow isn't meant to do right > now because it wouldn't

Re: Custom (mongodb) hook as a plugin

2016-09-23 Thread Shin Tai
Ok - thanks. Sounds like the sort of thing Airflow isn't meant to do right now because it wouldn't be possible to configure the connector with usernames, passwords etc I may just have to fork for the time being. Thanks again. On 23 September 2016 at 17:40, Andrew Phillips

Re: Custom Hook development guideline

2016-09-23 Thread Maxime Beauchemin
Any reason why you want to package it as a plugin? It's pretty easy to just have your custom hooks and operators live alongside your pipelines, maybe in a `common` folder. In your pipeline file you just import what you need, relative to your file, from the same repo. Just don't forget the

Re: Custom (mongodb) hook as a plugin

2016-09-23 Thread Andrew Phillips
6. Went to UI/Connections but couldn't find my connector/hook The experts on the list will hopefully set me right I'm my understanding is incorrect, but from what I can see the list of hook types available *in the UI* is fixed:

Re: Custom Hook development guideline

2016-09-23 Thread Andrew Phillips
Are the above steps correct or am I missing something? Is something not working as expected, e.g. are you unable to *use* the plugin from your DAGs? If so, could you provide a few more details on the error? Regards ap