[PR] Include plugins in the architecture diagrams [airflow]

potiuk opened a new pull request, #36513: URL: https://github.com/apache/airflow/pull/36513 When it comes to access and management plugins management follows different pattern than DAG files. While DAG files can (and should) be modified by DAG authors, the whole idea of Plugins was to make

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on PR #36513: URL: https://github.com/apache/airflow/pull/36513#issuecomment-1872959349 This is part of preparation to the last stage of multi-tentant last AIP and documentation around it. It was also partially result of the issue https://github.com/apache/airflow/i

Re: [PR] Include plugins in the architecture diagrams [airflow]

BasPH commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1438889862 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -43,13 +43,26 @@ An Airflow installation generally consists of the following components: Basic airflow archit

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on PR #36513: URL: https://github.com/apache/airflow/pull/36513#issuecomment-1872962105 > I feel all the different arrow styles require a legend. What does blue vs red and solid vs dashed indicate? Yep. Will add it. There is a good reasoning for that and deliberate s

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1438892198 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -43,13 +43,26 @@ An Airflow installation generally consists of the following components: Basic airflow archi

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1438892404 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -43,13 +43,26 @@ An Airflow installation generally consists of the following components: Basic airflow archi

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1438892537 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -43,13 +43,26 @@ An Airflow installation generally consists of the following components: Basic airflow archi

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on PR #36513: URL: https://github.com/apache/airflow/pull/36513#issuecomment-1872970410 > All the crossed dashed arrows make it difficult to follow the arrow. Would stick with solid lines. Yep. I will work a bit on those. -- This is an automated message

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1438896755 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -43,13 +43,26 @@ An Airflow installation generally consists of the following components: Basic airflow archi

Re: [PR] Include plugins in the architecture diagrams [airflow]

jscheffl commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1438898796 ## docs/apache-airflow/img/diagram_basic_airflow_architecture.png: ## Review Comment: I dis-like the icon for the plugin-folder (in terms of visual style) and t

Re: [PR] Include plugins in the architecture diagrams [airflow]

jscheffl commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1438900041 ## docs/apache-airflow/img/diagram_basic_airflow_architecture.png: ## Review Comment: Just checking it seems this framework only contains "ugly" icons, then I o

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1438892537 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -43,13 +43,26 @@ An Airflow installation generally consists of the following components: Basic airflow archi

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1438904061 ## docs/apache-airflow/img/diagram_basic_airflow_architecture.png: ## Review Comment: You can provide custom icons - no problem - this is how I added "scalable p

Re: [PR] Include plugins in the architecture diagrams [airflow]

jscheffl commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1438907293 ## docs/apache-airflow/img/diagram_basic_airflow_architecture.png: ## Review Comment: How about "just using" https://openclipart.org/detail/211760/matticonsfol

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1438907524 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -43,13 +43,26 @@ An Airflow installation generally consists of the following components: Basic airflow archi

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1438907553 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -43,13 +43,26 @@ An Airflow installation generally consists of the following components: Basic airflow archi

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1438907595 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -43,13 +43,26 @@ An Airflow installation generally consists of the following components: Basic airflow archi

Re: [PR] Include plugins in the architecture diagrams [airflow]

jscheffl commented on PR #36513: URL: https://github.com/apache/airflow/pull/36513#issuecomment-1872996653 > While we're changing the diagram, how about displaying a generic "database" icon instead of the PostgreSQL logo? PostgreSQL is the most used database but we do support others too.

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1438910073 ## docs/apache-airflow/img/diagram_basic_airflow_architecture.png: ## Review Comment: Free for any usage, no attribution needed. Should be fine (matticons) --

Re: [PR] Include plugins in the architecture diagrams [airflow]

BasPH commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1438914237 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -43,13 +43,26 @@ An Airflow installation generally consists of the following components: Basic airflow archit

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on PR #36513: URL: https://github.com/apache/airflow/pull/36513#issuecomment-1873012436 Should be quite a bit nicer (I will still need to figure out how to do anchoring a bit better but the icons should be nicer now. -- This is an automated message from the Apache Git Se

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on PR #36513: URL: https://github.com/apache/airflow/pull/36513#issuecomment-1873023020 Right I think I found a nice-enough layout. Looks like "ortho" layout for edges is rather difficult for edge anchoring (even if Iike it most) - but with some graphviz tweaking seem

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on PR #36513: URL: https://github.com/apache/airflow/pull/36513#issuecomment-1873026702 And even a but nicer now :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1438929790 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -43,13 +43,26 @@ An Airflow installation generally consists of the following components: Basic airflow archi

Re: [PR] Include plugins in the architecture diagrams [airflow]

BasPH commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1439322639 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -18,49 +18,151 @@ Architecture Overview = -Airflow is a platform that lets you build an

Re: [PR] Include plugins in the architecture diagrams [airflow]

BasPH commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1439342399 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -18,49 +18,151 @@ Architecture Overview = -Airflow is a platform that lets you build an

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on PR #36513: URL: https://github.com/apache/airflow/pull/36513#issuecomment-1873975167 I will take a closer look at other comments but one thing first: > I see several roles being mentioned, but don't think it's currently clear what those are (Deployment manager and

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1440793892 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -18,49 +18,151 @@ Architecture Overview = -Airflow is a platform that lets you build a

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1440794816 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -18,49 +18,151 @@ Architecture Overview = -Airflow is a platform that lets you build a

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1440800421 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -18,49 +18,151 @@ Architecture Overview = -Airflow is a platform that lets you build a

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1440802365 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -18,49 +18,151 @@ Architecture Overview = -Airflow is a platform that lets you build a

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1440805208 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -18,49 +18,151 @@ Architecture Overview = -Airflow is a platform that lets you build a

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1440807358 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -18,49 +18,151 @@ Architecture Overview = -Airflow is a platform that lets you build a

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1440808429 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -18,49 +18,151 @@ Architecture Overview = -Airflow is a platform that lets you build a

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1440820704 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -18,49 +18,151 @@ Architecture Overview = -Airflow is a platform that lets you build a

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1440823172 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -18,49 +18,151 @@ Architecture Overview = -Airflow is a platform that lets you build a

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1440828944 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -18,49 +18,151 @@ Architecture Overview = -Airflow is a platform that lets you build a

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1440835976 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -18,49 +18,151 @@ Architecture Overview = -Airflow is a platform that lets you build a

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1440846010 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -18,49 +18,151 @@ Architecture Overview = -Airflow is a platform that lets you build a

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on code in PR #36513: URL: https://github.com/apache/airflow/pull/36513#discussion_r1440847798 ## docs/apache-airflow/core-concepts/overview.rst: ## @@ -18,49 +18,151 @@ Architecture Overview = -Airflow is a platform that lets you build a

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on PR #36513: URL: https://github.com/apache/airflow/pull/36513#issuecomment-1875866207 Applied the review comments @BasPH. I decided not to separate out the "roles" into separate document - but to briefly mention the roies and explicitly referring to security-model documen

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on PR #36513: URL: https://github.com/apache/airflow/pull/36513#issuecomment-1875878444 I think with all those changes, iterations it starts to look reallly good and rather professional: This: https://github.com/apache/airflow/assets/595491/dcfeac38-1f1d-4b6b-b

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on PR #36513: URL: https://github.com/apache/airflow/pull/36513#issuecomment-1875880879 And: https://github.com/apache/airflow/assets/595491/1a4a6d98-0819-4d8e-acec-44795013666c";> -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on PR #36513: URL: https://github.com/apache/airflow/pull/36513#issuecomment-1877679277 @BasPH -> do you think it's cool to go ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk commented on PR #36513: URL: https://github.com/apache/airflow/pull/36513#issuecomment-1878779756 Ok. Will merge it now - we can always fix things later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Include plugins in the architecture diagrams [airflow]

potiuk merged PR #36513: URL: https://github.com/apache/airflow/pull/36513 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a