[GitHub] [airflow] potiuk opened a new pull request #18547: Excludes rightfullhy unlicensed files from chart from RAT check

2021-09-27 Thread GitBox
potiuk opened a new pull request #18547: URL: https://github.com/apache/airflow/pull/18547 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more info

[airflow] branch main updated (fa1a088 -> 387c43f)

2021-09-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from fa1a088 Remove workaround for docker-compose-failures (#18539) add 387c43f Fix intermittent orphan test (#18530

[GitHub] [airflow] potiuk merged pull request #18530: Fix intermittent orphan test

2021-09-27 Thread GitBox
potiuk merged pull request #18530: URL: https://github.com/apache/airflow/pull/18530 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsub

[GitHub] [airflow] subkanthi commented on issue #18482: No SQL error shown when using the JDBC operator

2021-09-27 Thread GitBox
subkanthi commented on issue #18482: URL: https://github.com/apache/airflow/issues/18482#issuecomment-927903090 Im having trouble getting the jar working for the JDBC connection, unassigning. -- This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [airflow] enguerranchevalier commented on issue #18369: If the DAG contains Tasks Goups, Graph view does not properly display tasks statuses

2021-09-27 Thread GitBox
enguerranchevalier commented on issue #18369: URL: https://github.com/apache/airflow/issues/18369#issuecomment-927869596 Having this issue with `Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:92.0) Gecko/20100101 Firefox/92.0 `. @bbovenzi @foldvari On my side, the problem also appears wi

[GitHub] [airflow] JavierLopezT commented on a change in pull request #18447: add RedshiftStatementHook, RedshiftOperator

2021-09-27 Thread GitBox
JavierLopezT commented on a change in pull request #18447: URL: https://github.com/apache/airflow/pull/18447#discussion_r716684545 ## File path: airflow/providers/amazon/aws/hooks/redshift_statement.py ## @@ -0,0 +1,144 @@ +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] [airflow] JavierLopezT opened a new issue #18546: Show code of other files than DAG one

2021-09-27 Thread GitBox
JavierLopezT opened a new issue #18546: URL: https://github.com/apache/airflow/issues/18546 ### Description Showing code for several files besides the DAG code in the Code view of a DAG ### Use case/motivation Sometimes I have python scripts and sql files in the same fol

[GitHub] [airflow] potiuk commented on a change in pull request #18531: Workaround intermittently failing scheduler test

2021-09-27 Thread GitBox
potiuk commented on a change in pull request #18531: URL: https://github.com/apache/airflow/pull/18531#discussion_r716678182 ## File path: tests/jobs/test_scheduler_job.py ## @@ -2658,9 +2659,14 @@ def test_do_schedule_max_active_runs_dag_timed_out(self, dag_maker): a

[GitHub] [airflow] JavierLopezT edited a comment on issue #17473: New view "Operator arguments" between "Task Instance Details" and "Rendered"

2021-09-27 Thread GitBox
JavierLopezT edited a comment on issue #17473: URL: https://github.com/apache/airflow/issues/17473#issuecomment-927861831 Let's say you have a DAG with the following default_args: ``` default_args = { 'email': globals_variables['email_data'], 'file_format': globals_va

[GitHub] [airflow] JavierLopezT commented on issue #17473: New view "Operator arguments" between "Task Instance Details" and "Rendered"

2021-09-27 Thread GitBox
JavierLopezT commented on issue #17473: URL: https://github.com/apache/airflow/issues/17473#issuecomment-927861831 Let's say you have the following default_args: ``` default_args = { 'email': globals_variables['email_data'], 'file_format': globals_variables['file_form

[GitHub] [airflow] Jaxing commented on issue #16209: BigQueryInsertJobOperator required parameter is missing, but which?

2021-09-27 Thread GitBox
Jaxing commented on issue #16209: URL: https://github.com/apache/airflow/issues/16209#issuecomment-927823503 I found a solution to my problem. I could use googles QueryJobConfiguration to create the config. ``` db_copy_projects_query = "" conf = QueryJobConfig( destina

[GitHub] [airflow] ashb closed pull request #18476: Give MSSQL container more time to start up

2021-09-27 Thread GitBox
ashb closed pull request #18476: URL: https://github.com/apache/airflow/pull/18476 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #18439: Check and run migration in webserver startup if necessary

2021-09-27 Thread GitBox
ephraimbuddy commented on a change in pull request #18439: URL: https://github.com/apache/airflow/pull/18439#discussion_r716616556 ## File path: airflow/utils/db.py ## @@ -626,27 +629,89 @@ def check_migrations(timeout): :param timeout: Timeout for the migration in seconds

[GitHub] [airflow] ephraimbuddy commented on pull request #18439: Check and run migration in webserver startup if necessary

2021-09-27 Thread GitBox
ephraimbuddy commented on pull request #18439: URL: https://github.com/apache/airflow/pull/18439#issuecomment-927796920 > > One question: Would there be race conditions if multiple `airflow` commands are launched concurrently? i.e. `./breeze start-airflow` launches a tmux session and five

[airflow] branch constraints-main updated: Updating constraints. Build id:1277714044

2021-09-27 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 45e2970 Updating const

[GitHub] [airflow] foldvari commented on issue #18369: If the DAG contains Tasks Goups, Graph view does not properly display tasks statuses

2021-09-27 Thread GitBox
foldvari commented on issue #18369: URL: https://github.com/apache/airflow/issues/18369#issuecomment-927763080 > I'm still having issues replicating this on my end. I have one hunch though, what browsers are people experiencing this on? I can reproduce both in Edge (93.0.961.52) and

[GitHub] [airflow] uranusjr closed pull request #18503: When calling `dr.get_task_instance` automatically set `dag_run` relationship

2021-09-27 Thread GitBox
uranusjr closed pull request #18503: URL: https://github.com/apache/airflow/pull/18503 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-uns

[GitHub] [airflow] ashb commented on a change in pull request #18439: Check and run migration in webserver startup if necessary

2021-09-27 Thread GitBox
ashb commented on a change in pull request #18439: URL: https://github.com/apache/airflow/pull/18439#discussion_r716578846 ## File path: airflow/utils/db.py ## @@ -626,27 +629,102 @@ def check_migrations(timeout): :param timeout: Timeout for the migration in seconds :

[GitHub] [airflow] uranusjr commented on a change in pull request #17329: Split sql statements in DbApi run

2021-09-27 Thread GitBox
uranusjr commented on a change in pull request #17329: URL: https://github.com/apache/airflow/pull/17329#discussion_r716578805 ## File path: airflow/hooks/dbapi.py ## @@ -259,6 +266,27 @@ def get_cursor(self): """Returns a cursor""" return self.get_conn().curs

[GitHub] [airflow] github-actions[bot] commented on pull request #18530: Fix intermittent orphan test

2021-09-27 Thread GitBox
github-actions[bot] commented on pull request #18530: URL: https://github.com/apache/airflow/pull/18530#issuecomment-927748031 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

[GitHub] [airflow] uranusjr commented on issue #18541: Error when running dag & something to do with parsing the start date

2021-09-27 Thread GitBox
uranusjr commented on issue #18541: URL: https://github.com/apache/airflow/issues/18541#issuecomment-927741327 Yup, that one is unfortunately not backported to 2.1, see #18315. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] bhavaniravi commented on issue #18541: Error when running dag & something to do with parsing the start date

2021-09-27 Thread GitBox
bhavaniravi commented on issue #18541: URL: https://github.com/apache/airflow/issues/18541#issuecomment-927739981 @bbovenzi That check is not available in 2.1.3 codebase https://github.com/apache/airflow/blob/2.1.3/airflow/www/views.py#L856 In what case would `start_date` be `None

[GitHub] [airflow] uranusjr commented on pull request #18522: Coerce datetime to pendulum for timetable

2021-09-27 Thread GitBox
uranusjr commented on pull request #18522: URL: https://github.com/apache/airflow/pull/18522#issuecomment-927733537 Grrgh, the diffrent `str()` representation between `datetime.datetime` and `pendulum.DateTime` is super annoying. And yes, let's also add a test when we fix the failing one a

[GitHub] [airflow] potiuk commented on pull request #18516: Fix rendering nested task fields

2021-09-27 Thread GitBox
potiuk commented on pull request #18516: URL: https://github.com/apache/airflow/pull/18516#issuecomment-927708088 Rebased to make use of the more stable (hopefully) main with merged flakiness fixes :) -- This is an automated message from the Apache Git Service. To respond to the message,

[airflow] branch main updated: Remove workaround for docker-compose-failures (#18539)

2021-09-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new fa1a088 Remove workaround for docker-compose-failu

[GitHub] [airflow] potiuk merged pull request #18539: Remove workaround for docker-compose-failures

2021-09-27 Thread GitBox
potiuk merged pull request #18539: URL: https://github.com/apache/airflow/pull/18539 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsub

[GitHub] [airflow] ephraimbuddy commented on pull request #18499: Fix DetachedInstanceError when dag_run attrs are accessed from ti

2021-09-27 Thread GitBox
ephraimbuddy commented on pull request #18499: URL: https://github.com/apache/airflow/pull/18499#issuecomment-927699124 > @ashb, I applied this fix #18503 and it worked. > I also applied a fix where I attached dag_run to this ti: > https://github.com/apache/airflow/blob/2643345e4b7206

[GitHub] [airflow] bbovenzi commented on issue #18369: If the DAG contains Tasks Goups, Graph view does not properly display tasks statuses

2021-09-27 Thread GitBox
bbovenzi commented on issue #18369: URL: https://github.com/apache/airflow/issues/18369#issuecomment-927698312 I'm still having issues replicating this on my end. I have one hunch though, what browsers are people experiencing this on? -- This is an automated message from the Apache Git S

[GitHub] [airflow] pawsok commented on issue #18545: Unable to add a new user when logged using LDAP auth

2021-09-27 Thread GitBox
pawsok commented on issue #18545: URL: https://github.com/apache/airflow/issues/18545#issuecomment-927688511 > > Indeed sounds like a valid feature to have. The permissions in [#18290 (comment)](https://github.com/apache/airflow/discussions/18290#discussioncomment-1378607) seem to do the t

[GitHub] [airflow] ashb commented on a change in pull request #18439: Check and run migration in webserver startup if necessary

2021-09-27 Thread GitBox
ashb commented on a change in pull request #18439: URL: https://github.com/apache/airflow/pull/18439#discussion_r716502072 ## File path: airflow/utils/db.py ## @@ -626,27 +629,102 @@ def check_migrations(timeout): :param timeout: Timeout for the migration in seconds :

[GitHub] [airflow] ashb commented on a change in pull request #18439: Check and run migration in webserver startup if necessary

2021-09-27 Thread GitBox
ashb commented on a change in pull request #18439: URL: https://github.com/apache/airflow/pull/18439#discussion_r716502072 ## File path: airflow/utils/db.py ## @@ -626,27 +629,102 @@ def check_migrations(timeout): :param timeout: Timeout for the migration in seconds :

[GitHub] [airflow] ashb commented on a change in pull request #18439: Check and run migration in webserver startup if necessary

2021-09-27 Thread GitBox
ashb commented on a change in pull request #18439: URL: https://github.com/apache/airflow/pull/18439#discussion_r716499095 ## File path: airflow/utils/db.py ## @@ -626,27 +629,102 @@ def check_migrations(timeout): :param timeout: Timeout for the migration in seconds :

[GitHub] [airflow] potiuk commented on issue #18545: Unable to add a new user when logged using LDAP auth

2021-09-27 Thread GitBox
potiuk commented on issue #18545: URL: https://github.com/apache/airflow/issues/18545#issuecomment-927676396 > Indeed sounds like a valid feature to have. The permissions in [#18290 (comment)](https://github.com/apache/airflow/discussions/18290#discussioncomment-1378607) seem to do the tri

[GitHub] [airflow] BasPH commented on issue #18545: Unable to add a new user when logged using LDAP auth

2021-09-27 Thread GitBox
BasPH commented on issue #18545: URL: https://github.com/apache/airflow/issues/18545#issuecomment-927674994 Indeed sounds like a valid feature to have. The permissions in https://github.com/apache/airflow/discussions/18290#discussioncomment-1378607 seem to do the trick. @pawsok would you l

[GitHub] [airflow] jensgrnb edited a comment on issue #18495: apache-airflow-providers-sendgrid==2.0.1 doesn't show in the connections drop down UI

2021-09-27 Thread GitBox
jensgrnb edited a comment on issue #18495: URL: https://github.com/apache/airflow/issues/18495#issuecomment-927673400 Okay so it'll never be listed as a specific conn_type under my current version airflow version 2.1.4. How can I use http conn type for this? I tried setting the host

[GitHub] [airflow] jensgrnb commented on issue #18495: apache-airflow-providers-sendgrid==2.0.1 doesn't show in the connections drop down UI

2021-09-27 Thread GitBox
jensgrnb commented on issue #18495: URL: https://github.com/apache/airflow/issues/18495#issuecomment-927673400 Okay so it'll never be listed as a specific conn_type under my current version airflow version 2.1.4. How can I use http conn type for this? I tried setting the host as ht

[GitHub] [airflow] potiuk commented on issue #18545: Unable to add a new user when logged using LDAP auth

2021-09-27 Thread GitBox
potiuk commented on issue #18545: URL: https://github.com/apache/airflow/issues/18545#issuecomment-927671158 cc: @pawsok -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] potiuk commented on issue #18545: Unable to add a new user when logged using LDAP auth

2021-09-27 Thread GitBox
potiuk commented on issue #18545: URL: https://github.com/apache/airflow/issues/18545#issuecomment-927670785 After discussion in https://github.com/apache/airflow/discussions/18290 and similar stack overflow issue here: https://stackoverflow.com/questions/69343651/create-user-with-ldap-aut

[airflow] branch constraints-main updated: Updating constraints. Build id:1277354580

2021-09-27 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 626e220 Updating const

[GitHub] [airflow] potiuk opened a new issue #18545: Unable to add a new user when logged using LDAP auth

2021-09-27 Thread GitBox
potiuk opened a new issue #18545: URL: https://github.com/apache/airflow/issues/18545 ### Discussed in https://github.com/apache/airflow/discussions/18290 Originally posted by **pawsok** September 16, 2021 ### Apache Airflow version 2.1.4 (latest released) ###

[GitHub] [airflow] eladkal commented on issue #11919: Add how-to Guide for Slack operators

2021-09-27 Thread GitBox
eladkal commented on issue #11919: URL: https://github.com/apache/airflow/issues/11919#issuecomment-927662306 solved in https://github.com/apache/airflow/pull/18525 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] eladkal closed issue #11919: Add how-to Guide for Slack operators

2021-09-27 Thread GitBox
eladkal closed issue #11919: URL: https://github.com/apache/airflow/issues/11919 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr.

[GitHub] [airflow] nwalens commented on pull request #18249: Add support for securityContext per deployment

2021-09-27 Thread GitBox
nwalens commented on pull request #18249: URL: https://github.com/apache/airflow/pull/18249#issuecomment-927656408 @potiuk the code was rebased. I'm still testing it after applying the changes @jedcunningham has requested and I am having some issues with several tests. I'm currently che

[GitHub] [airflow] github-actions[bot] commented on pull request #18539: Remove workaround for docker-compose-failures

2021-09-27 Thread GitBox
github-actions[bot] commented on pull request #18539: URL: https://github.com/apache/airflow/pull/18539#issuecomment-927646796 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

[GitHub] [airflow] uranusjr closed issue #18543: Where is the algorithm file stored in Airflow,and how to find it

2021-09-27 Thread GitBox
uranusjr closed issue #18543: URL: https://github.com/apache/airflow/issues/18543 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

[airflow] branch main updated: Improve isolation of MSSQL files (#18538)

2021-09-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 932c3b5 Improve isolation of MSSQL files (#18538)

[GitHub] [airflow] potiuk merged pull request #18538: Improve isolation of MSSQL files

2021-09-27 Thread GitBox
potiuk merged pull request #18538: URL: https://github.com/apache/airflow/pull/18538 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsub

[GitHub] [airflow] github-actions[bot] commented on pull request #18538: Improve isolation of MSSQL files

2021-09-27 Thread GitBox
github-actions[bot] commented on pull request #18538: URL: https://github.com/apache/airflow/pull/18538#issuecomment-927624139 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

[GitHub] [airflow] potiuk commented on pull request #18538: Improve isolation of MSSQL files

2021-09-27 Thread GitBox
potiuk commented on pull request #18538: URL: https://github.com/apache/airflow/pull/18538#issuecomment-927622890 I have a quite plausible hypothesis why this could help: * When you look at this https://github.com/actions/runner/issues/510 - you will see that `--once` has a race cond

[airflow] branch main updated (2643345 -> dc3bf42)

2021-09-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 2643345 Add guide for Apache Druid operators (#18527) add dc3bf42 Fix running helm tests in parallel (#18533)

[GitHub] [airflow] xiaodouzi666 opened a new issue #18543: Where is the algorithm file stored in Airflow,and how to find it

2021-09-27 Thread GitBox
xiaodouzi666 opened a new issue #18543: URL: https://github.com/apache/airflow/issues/18543 ### Describe the issue with documentation Dear airflow's developers: I am a student doing a DAG parsing engine project. I hope to refer to how your algorithm realizes the logic of inputting

[GitHub] [airflow] boring-cyborg[bot] commented on issue #18543: Where is the algorithm file stored in Airflow,and how to find it

2021-09-27 Thread GitBox
boring-cyborg[bot] commented on issue #18543: URL: https://github.com/apache/airflow/issues/18543#issuecomment-927614856 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] potiuk merged pull request #18533: Fix running helm tests in parallel

2021-09-27 Thread GitBox
potiuk merged pull request #18533: URL: https://github.com/apache/airflow/pull/18533 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsub

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #18278: Updating miscellaneous provider DAGs to use TaskFlow API where applicable

2021-09-27 Thread GitBox
ephraimbuddy commented on a change in pull request #18278: URL: https://github.com/apache/airflow/pull/18278#discussion_r716435686 ## File path: airflow/providers/sqlite/example_dags/example_sqlite.py ## @@ -55,6 +54,7 @@ # [END howto_operator_sqlite] +@dag.task(task_id="i

[GitHub] [airflow] Jaxing edited a comment on issue #16209: BigQueryInsertJobOperator required parameter is missing, but which?

2021-09-27 Thread GitBox
Jaxing edited a comment on issue #16209: URL: https://github.com/apache/airflow/issues/16209#issuecomment-926634257 @mik-laj I think this might be due to a bug. This configuration of the operator works ```db_copy_projects = BigQueryInsertJobOperator( task_id="db_copy_pro

[GitHub] [airflow] bbovenzi commented on issue #18541: Error when running dag & something to do with parsing the start date

2021-09-27 Thread GitBox
bbovenzi commented on issue #18541: URL: https://github.com/apache/airflow/issues/18541#issuecomment-927608066 @uranusjr Shouldn't this be caught here? https://github.com/apache/airflow/blob/main/airflow/www/views.py#L917 -- This is an automated message from the Apache Git Service. To re

[GitHub] [airflow] mnojek commented on a change in pull request #18494: Fix part of Google system tests

2021-09-27 Thread GitBox
mnojek commented on a change in pull request #18494: URL: https://github.com/apache/airflow/pull/18494#discussion_r716376092 ## File path: airflow/providers/google/cloud/example_dags/example_cloud_sql.py ## @@ -48,8 +49,8 @@ from airflow.utils.dates import days_ago GCP_PROJ

[GitHub] [airflow] fredthomsen opened a new pull request #18542: Allow ingress multiple hostnames w/diff secrets

2021-09-27 Thread GitBox
fredthomsen opened a new pull request #18542: URL: https://github.com/apache/airflow/pull/18542 This allows the ingress to specify multiple hosts with multiple different secrets, rather than being contrainted to one secret for all hosts specified. Doing this now avoids what is likely

[GitHub] [airflow] vlieven commented on issue #18501: Scheduler overloaded when backfilling by clearing DAG history

2021-09-27 Thread GitBox
vlieven commented on issue #18501: URL: https://github.com/apache/airflow/issues/18501#issuecomment-927584865 Suppose you have a DAG with tasks X -> Y -> Z. Setting task_concurrency=2 and max_active_runs=3 allows you to start task X for dagrun n+2, while task Z is still running in dagruns

[GitHub] [airflow] dstandish commented on a change in pull request #18447: add RedshiftStatementHook, RedshiftOperator

2021-09-27 Thread GitBox
dstandish commented on a change in pull request #18447: URL: https://github.com/apache/airflow/pull/18447#discussion_r716128423 ## File path: airflow/providers/amazon/aws/hooks/redshift_statement.py ## @@ -0,0 +1,159 @@ +# +# Licensed to the Apache Software Foundation (ASF) und

<    1   2