potiuk commented on issue #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver URL: https://github.com/apache/airflow/pull/5594#issuecomment-513080601 Question: I am not familiar with that part of the code but I took a quick look and I have a few comments. Possibly I do not understand all consequences but let me do some thought experiment: 1) I assume the Dag loaded via DagBag is mainly used to load it as python object for structural analysis and later render the UI graphs/charts/tabs etc. 2) To display the code in UI we use the original source code rather than stringified version (so no impact of stringification here) - at least that looks like from the code :) 3) The "BaseOperator" problem will show up in the UI where we look for DagRuns, Gant Charts and other places where the DAG object structure is used. 4) When you get to "Rendered template" screen you get an error in case of lambda functions etc. 5) Custom fields are replaced in stringified form with <main.A object at 0x7f4121780828> - but it should not (?) have impact on anything displayed in the UI. So maybe those simple hacks will work: 1) That's already solved by stringified form that provides the structure 2) Here we have no problem as source code is used directly 3) Why don't we inject "airflow_class_name = 'ClassName'" field in the stringified BaseOperator while we stringify it and use it as the name displayed in the UI (and everywhere where needed)? I think that might be really simple solution. No problems with loading the external modules (cause there is no need - we only need name from what I understand) 4) Why don't we load the "real" DAG just to display the templated fields - when they are actually clicked ? That will surely cause delays when displaying the tab, but maybe it's better than error. It should be rather easy - you can create a separate DagBag with just the single Dag as "folder" and from what I know, it should load that DAG (and then you could figure out and load needed modules before you display it). Maybe that's a solution (but also maybe it's not that necessary to fix this particular tab in this - really - edge case. 5) I believe it's not a problem?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
