Re: Encrypting Airflow Communications

2016-10-28 Thread Bolke de Bruin
You can setup ssl and encryption for all these connections. For celery you need a broker that supports encryption (i.e. Rabbitmq) and for the db both Postgres and MySQL support ssl connections. I don't think a change in airflow is needed for this except configuration. Bolke Sent from my

Re: Encrypting Airflow Communications

2016-10-27 Thread siddharth anand
This might be relevant as well if you want to learn more about VPC : https://www.youtube.com/watch?v=HexrVfuIY1k On Thu, Oct 27, 2016 at 6:39 PM, siddharth anand wrote: > Hmn.. it looks like traffic within AWS's VPC is not encrypted.. so using > TLS between all services is a

Re: Encrypting Airflow Communications

2016-10-27 Thread siddharth anand
I haven't looked into it but would welcome a PR if you were to propose one. We use SQL Alchemy for our ORM, so you may want to look at that for 1) above. We (Agari) run in AWS and run all our EC2-based services (e.g. Airflow servers and the DB) within a VPC. I suspect the folks running in GCP

Encrypting Airflow Communications

2016-10-27 Thread Brandon White
>From what I see, Airflow communicates with a couple sources: 1) SQL Store 2) Celery Broker Does Airflow have any configurations which make it easy to encrypt all of its communications or do we need to build custom solutions into Airflow? -- This e-mail is private and confidential and is for