o-nikolas commented on PR #34324: URL: https://github.com/apache/airflow/pull/34324#issuecomment-1718299739
Thanks for the feedback @potiuk, replies inline below :) > Small comment - not directly this but related and clearly visible in the rendered page. Can you please move the "Testing" documentation out of the executor hierarchy ? The two Howtos seems completely out of place for the document. I don't see a "Testing" doc in the executor hierarchy? I'm likely missing something though :eyes: > > I like the document. The few proposals I would like to propose: > > * Instead of "strongly recommend" I would write "Sequential Executor is not suitable for production. It does not allow for parallel task runnning and due to that, some Airflow features (for example running sensors waiting for outputs of another task are not working properly). This section actually already existed before I updated it (my changes start at "Writing Your Own Executor"), but I can make this change anyway while I'm here. > * Maybe it would be possible to Have a boilerplate template of an executor as part of the docs ? That would probably significantly decrease the friction of writing new executor (on the other hand, maybe we actually want some friction). Writing your own executor is not an easy / simple task that everyone should even attempt I did think of this, but unlike custom operators, it's quite a large piece of code to even have a boilerplate. I was thinking of adding a very pseudo code snippet, but wasn't convinced that it would be helpful beyond folks just having a look at any of the existing community executors which all implement the BaseExecutor interface. Interested to hear what others think! > * Speaking of which - I think we should explaine why somoene would like to write their own executor. Short paragraph describing a decision making process here and some example reasons why one would rather write their own executor than choose from the existing ones, would be great. Great suggestion. I'll add a section and come up with a list of bullets for why you'd want to create a custom executor. If you have any to add, please reply with them here! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
