Hey Andrew - My suggestions are similar to Marcel's.
I favor invoking the Lambda function with the invoke <https://boto3.readthedocs.io/en/latest/reference/services/lambda.html#Lambda.Client.invoke> call in boto3 in a PythonOperator, or using the AwsHook if that abstraction adds value here vs calling boto directly. I second that using the SimpleHttpOperator in Airflow to hit the Lambda function like a REST API is a reasonable alternative. Personally, I would also recommend using a serverless framework such as Zappa or Serverless as in my experience, there's a surprising amount of boilerplate involved in writing, deploying, updating, and maintaining serverless functions (+ API Gateway etc). If you're considering Lambda alternatives, I think the Zeit Now Serverless Docker Beta <https://zeit.co/blog/serverless-docker> is a pretty cool and very easy way to run serverless functions. There's a Hello World Flask example <https://github.com/zeit/now-examples/tree/master/python-flask> that shows how it all works in just a few lines. *Taylor Edmiston* Blog <https://blog.tedmiston.com/> | CV <https://stackoverflow.com/cv/taylor> | LinkedIn <https://www.linkedin.com/in/tedmiston/> | AngelList <https://angel.co/taylor> | Stack Overflow <https://stackoverflow.com/users/149428/taylor-edmiston> On Wed, Jul 11, 2018 at 3:07 PM Hamade, Marcel (GfK) <marcel.ham...@gfk.com> wrote: > The aws_lambda_hook it self uses the AWS_hook which is a wrapper arount > the boto3 library(the standard way to interact with AWS via python). If you > add the AWS connections correctly you can use the hook in one of your self > written operators to trigger your lambda functions > > Ano way would be to trigger Lambda functions via AWS API Gateway > https://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started-with-lambda-integration.html > For that you can use the HTTP-Operator as a starting point but of course > you have to handle HTTP-Authorisation. > > Cheers, > Marcel > > ________________________________________ > Von: Andrew Maguire <andrewm4...@gmail.com> > Gesendet: Mittwoch, 11. Juli 2018 13:33 > An: dev@airflow.incubator.apache.org > Betreff: Airflow & AWS Lambda Tutorials? > > Hi all, > > Does anyone know of any tutorials or examples of triggering an aws lambda > function from airflow? > > I googled a bit bit could not find much. I do see the aws_lambda_hook in > there but just a little unsure how to use it. > > Cheers, > Andy > > ________________________________ > > GfK SE, Nuremberg, Germany, commercial register at the local court > Amtsgericht Nuremberg HRB 25014; Management Board: Peter Feld (CEO), > Christian Bigatà Joseph (CFO). Supervisory Board Chairman: Ralf > Klein-Bölting. This email and any attachments may contain confidential or > privileged information. Please note that unauthorized copying, disclosure > or distribution of the material in this email is not permitted. > >
