@podunk We’ve used containers for nearly a decade in cloud deployments. It has been reliable and cheap for our ETL service (now running Hop).
Docker does have some overhead (and generally I’d like to see LXC mature and become our primary provider). Keep an open mind - we looked at Airflow many times as well. As an orchestration tool it's useful also - many good choices that provide flexibility! Good luck and let us know what you choose. Get Outlook for iOS<https://aka.ms/o0ukef> ________________________________ From: [email protected] <[email protected]> Sent: Tuesday, December 31, 2024 6:21:37 AM To: [email protected] <[email protected]> Subject: Re: Do you have any experience running Hop pipelines in Apache Airflow? EXTERNAL SENDER: Caution opening links or attachments Thank you - I'll take a look at it. I already know that I will throw away all this 'Docker' baggage – I hate this software. Sent: Monday, December 30, 2024 at 4:41 PM From: "Hans Van Akelyen" <[email protected]> To: [email protected] Subject: Re: Do you have any experience running Hop pipelines in Apache Airflow? There is a how-to guide on the website [1] Cheers, Hans [1] https://hop.apache.org//manual/latest/how-to-guides/run-hop-in-apache-airflow.html#_what_is_apache_airflow<https://urldefense.com/v3/__https://hop.apache.org/*manual/latest/how-to-guides/run-hop-in-apache-airflow.html*_what_is_apache_airflow__;LyM!!AQdq3sQhfUj4q8uUguY!hFeHwKrbPQ01g-YD-r69CoIy-7qM2GIry7DOGGx2-wOpx1kyUfYByKz7cCM13n2LBP3blT3Vv5XsO4jsGhiW3w$> On Mon, 30 Dec 2024 at 16:20, Ganesh T <[email protected]<mailto:[email protected]>> wrote: Hi, Yeah. We are taking care of ETLs running hop workflows. Regards Ganesh On Mon, 30 Dec 2024 at 8:16 PM, <[email protected]<mailto:[email protected]>> wrote: Hi, Do you have any experience running Hop pipelines in Apache Airflow? Specifically, I mean managing logging, reporting execution status, etc. Best, P
