[GitHub] [airflow] xinbinhuang edited a comment on pull request #15574: #12401 - Duplicating connections from UI

2021-05-03 Thread GitBox
xinbinhuang edited a comment on pull request #15574: URL: https://github.com/apache/airflow/pull/15574#issuecomment-831716841 I have the same feeling as @jhtimmins : should it be snakecase `_copy` instead? Another thing is: can you also a test case to cover the duplicate logic? O

[GitHub] [airflow] xinbinhuang commented on pull request #15574: #12401 - Duplicating connections from UI

2021-05-03 Thread GitBox
xinbinhuang commented on pull request #15574: URL: https://github.com/apache/airflow/pull/15574#issuecomment-831716841 I have the same feeling as @jhtimmins : should it be snakecase `_copy` instead? Another thing is: can you also a test case to cover the duplicate logic? -- This i

[airflow] branch master updated: Feature qubole hook support headers (#15615)

2021-05-03 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new 47a5539 Feature qubole hook support headers (

[GitHub] [airflow] eladkal merged pull request #15615: Feature qubole hook support headers

2021-05-03 Thread GitBox
eladkal merged pull request #15615: URL: https://github.com/apache/airflow/pull/15615 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, plea

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #15615: Feature qubole hook support headers

2021-05-03 Thread GitBox
boring-cyborg[bot] commented on pull request #15615: URL: https://github.com/apache/airflow/pull/15615#issuecomment-831715782 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] eladkal closed issue #15598: Qubole Hook Does Not Support 'include_headers'

2021-05-03 Thread GitBox
eladkal closed issue #15598: URL: https://github.com/apache/airflow/issues/15598 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please co

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #15637: Fix WasbHook.delete_file broken when using prefix

2021-05-03 Thread GitBox
ephraimbuddy commented on a change in pull request #15637: URL: https://github.com/apache/airflow/pull/15637#discussion_r625530376 ## File path: airflow/providers/microsoft/azure/hooks/wasb.py ## @@ -407,7 +407,7 @@ def delete_file( :type kwargs: object """

[GitHub] [airflow] saryeHaddadi edited a comment on issue #9926: Scheduler errors when using MSSQL as the backend database

2021-05-03 Thread GitBox
saryeHaddadi edited a comment on issue #9926: URL: https://github.com/apache/airflow/issues/9926#issuecomment-831525325 Hi, I can bring some informations on: `what versions do we support, should we add support to more versions.` 1. In general, ms-sql is quite robust in term of newer

[GitHub] [airflow] wolfier opened a new issue #15645: Sensor start_date gets updated for the same try in reschedule mode

2021-05-03 Thread GitBox
wolfier opened a new issue #15645: URL: https://github.com/apache/airflow/issues/15645 **Apache Airflow version**: 2.0.0 **What happened**: Sensor start_date gets updated to each time the task instance is rescheduled for the same try. **What you expected to happen**

[GitHub] [airflow] jhtimmins commented on a change in pull request #15631: Add setting up using VS Code

2021-05-03 Thread GitBox
jhtimmins commented on a change in pull request #15631: URL: https://github.com/apache/airflow/pull/15631#discussion_r625495972 ## File path: CONTRIBUTORS_QUICK_START.rst ## @@ -387,8 +391,9 @@ Using Breeze 3. Setup mysql database in -MySQL Workbench with Host ``127.

[GitHub] [airflow] jhtimmins commented on pull request #15632: Add link to Oracle Connection Docs

2021-05-03 Thread GitBox
jhtimmins commented on pull request #15632: URL: https://github.com/apache/airflow/pull/15632#issuecomment-831668478 @sunkickr This looks good. Can you include the oracle connection id type (I'm guessing either string or int) to those docblocks? -- This is an automated message from the

[GitHub] [airflow] vikramkoka commented on issue #15268: Airflow UI display error

2021-05-03 Thread GitBox
vikramkoka commented on issue #15268: URL: https://github.com/apache/airflow/issues/15268#issuecomment-831667789 I am sorry, it is not clear if this is a problem or not. Just to be precise, the Airflow UI in 1.10.x is NOT a real-time UI which automatically refreshes the status of the DA

[GitHub] [airflow] jhtimmins commented on a change in pull request #15385: [WIP] Added table to view providers in airflow ui under admin tab

2021-05-03 Thread GitBox
jhtimmins commented on a change in pull request #15385: URL: https://github.com/apache/airflow/pull/15385#discussion_r625488542 ## File path: airflow/security/permissions.py ## @@ -32,6 +32,7 @@ RESOURCE_JOB = "Jobs" RESOURCE_POOL = "Pools" RESOURCE_PLUGIN = "Plugins" +RESOU

[GitHub] [airflow] vikramkoka commented on issue #15641: Add documentation on what each parameter to a `sla_miss_callback` callable is

2021-05-03 Thread GitBox
vikramkoka commented on issue #15641: URL: https://github.com/apache/airflow/issues/15641#issuecomment-831664376 Thank you for reporting this. Here is the high level description for the SLA callback in the docs http://apache-airflow-docs.s3-website.eu-central-1.amazonaws.com/docs/apache

[GitHub] [airflow] aquam8 commented on issue #11979: Support helm install --wait for the chart

2021-05-03 Thread GitBox
aquam8 commented on issue #11979: URL: https://github.com/apache/airflow/issues/11979#issuecomment-831663761 Hi, First of all, let me thank you for this project and your involvement in this awesome project. Now onto this issue.. I have come to realise that this is also the pro

[GitHub] [airflow] jhtimmins commented on pull request #15574: #12401 - Duplicating connections from UI

2021-05-03 Thread GitBox
jhtimmins commented on pull request #15574: URL: https://github.com/apache/airflow/pull/15574#issuecomment-831661455 @ryanahamilton @ashb Bumping y'all on this since you're code owners here. I think the code looks solid (nice job on the string update logic @pateash). My only nitpick

[airflow] branch master updated: Bump ssri from 6.0.1 to 6.0.2 in /airflow/www (#15437)

2021-05-03 Thread jhtimmins
This is an automated email from the ASF dual-hosted git repository. jhtimmins pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new edbc89c Bump ssri from 6.0.1 to 6.0.2 in /a

[GitHub] [airflow] jhtimmins merged pull request #15437: Bump ssri from 6.0.1 to 6.0.2 in /airflow/www

2021-05-03 Thread GitBox
jhtimmins merged pull request #15437: URL: https://github.com/apache/airflow/pull/15437 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, pl

[GitHub] [airflow] github-actions[bot] commented on pull request #15490: Fix unsuccessful KubernetesPod final_state call when `is_delete_operator_pod=True`

2021-05-03 Thread GitBox
github-actions[bot] commented on pull request #15490: URL: https://github.com/apache/airflow/pull/15490#issuecomment-831632578 [The Workflow run](https://github.com/apache/airflow/actions/runs/808528525) is cancelling this PR. Building images for the PR has failed. Follow the workflow link

[GitHub] [airflow] MatthewRBruce commented on pull request #15490: Fix unsuccessful KubernetesPod final_state call when `is_delete_operator_pod=True`

2021-05-03 Thread GitBox
MatthewRBruce commented on pull request #15490: URL: https://github.com/apache/airflow/pull/15490#issuecomment-831620276 👍 There were legitimately were mocks I missed updating the return values for, but the issue in master may have been a problem too. -- This is an automated message fro

[GitHub] [airflow] github-actions[bot] commented on issue #12823: DAG callback handling not working properly on simultaneous executions

2021-05-03 Thread GitBox
github-actions[bot] commented on issue #12823: URL: https://github.com/apache/airflow/issues/12823#issuecomment-831609016 This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further

[GitHub] [airflow] github-actions[bot] commented on issue #12941: The scheduler does not appear to be running

2021-05-03 Thread GitBox
github-actions[bot] commented on issue #12941: URL: https://github.com/apache/airflow/issues/12941#issuecomment-831609008 This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further

[GitHub] [airflow] github-actions[bot] closed pull request #12637: Remove unsupported arguments from Slack webhook

2021-05-03 Thread GitBox
github-actions[bot] closed pull request #12637: URL: https://github.com/apache/airflow/pull/12637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this s

[GitHub] [airflow] github-actions[bot] closed issue #13188: having problems to start the weserver

2021-05-03 Thread GitBox
github-actions[bot] closed issue #13188: URL: https://github.com/apache/airflow/issues/13188 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this servic

[GitHub] [airflow] github-actions[bot] closed issue #13687: BashOperator's on_success_callback process may be executed twice.

2021-05-03 Thread GitBox
github-actions[bot] closed issue #13687: URL: https://github.com/apache/airflow/issues/13687 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this servic

[GitHub] [airflow] github-actions[bot] commented on pull request #13269: fix serialize_dag schema migration data column type

2021-05-03 Thread GitBox
github-actions[bot] commented on pull request #13269: URL: https://github.com/apache/airflow/pull/13269#issuecomment-831608996 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank y

[GitHub] [airflow] github-actions[bot] commented on issue #13188: having problems to start the weserver

2021-05-03 Thread GitBox
github-actions[bot] commented on issue #13188: URL: https://github.com/apache/airflow/issues/13188#issuecomment-831609003 This issue has been closed because it has not received response from the issue author. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] github-actions[bot] commented on issue #13687: BashOperator's on_success_callback process may be executed twice.

2021-05-03 Thread GitBox
github-actions[bot] commented on issue #13687: URL: https://github.com/apache/airflow/issues/13687#issuecomment-831608983 This issue has been closed because it has not received response from the issue author. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] github-actions[bot] closed pull request #14725: Update concepts.rst

2021-05-03 Thread GitBox
github-actions[bot] closed pull request #14725: URL: https://github.com/apache/airflow/pull/14725 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this s

[GitHub] [airflow] github-actions[bot] commented on pull request #15640: Allow chart to work with custom cluster domains

2021-05-03 Thread GitBox
github-actions[bot] commented on pull request #15640: URL: https://github.com/apache/airflow/pull/15640#issuecomment-831607254 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not m

[GitHub] [airflow] jedcunningham commented on pull request #15490: Fix unsuccessful KubernetesPod final_state call when `is_delete_operator_pod=True`

2021-05-03 Thread GitBox
jedcunningham commented on pull request #15490: URL: https://github.com/apache/airflow/pull/15490#issuecomment-831595919 @MatthewRBruce, might be worth rebasing on master if you still have k8s test failures - there was an issue in master that is now fixed. -- This is an automated message

[airflow] branch master updated (7318787 -> 5f43cd9)

2021-05-03 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 7318787 Fix add quotes to extraConfigMaps/extraSecrets (#15633) add 5f43cd9 Worker ServiceAccount is

[GitHub] [airflow] ephraimbuddy merged pull request #15642: Fix k8s tests in master: Worker ServiceAccount is needed for KubernetesExecutor

2021-05-03 Thread GitBox
ephraimbuddy merged pull request #15642: URL: https://github.com/apache/airflow/pull/15642 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [airflow] ephraimbuddy opened a new pull request #15644: Use the Stable REST API for Kubernetes executor integration tests

2021-05-03 Thread GitBox
ephraimbuddy opened a new pull request #15644: URL: https://github.com/apache/airflow/pull/15644 Currently, we use the experimental REST API to run the Kubernetes integration tests. This PR changes this to use the stable REST API for these tests --- **^ Add meaningful descr

[GitHub] [airflow] jedcunningham opened a new pull request #15643: Fix breeze k8s tests on mac

2021-05-03 Thread GitBox
jedcunningham opened a new pull request #15643: URL: https://github.com/apache/airflow/pull/15643 I'm not sure if there is a better fix, but I needed these to be able to run the k8s tests via breeze on my mac. Without these changes, I get: ``` scripts/ci/kubernetes/ci_run_kuber

[GitHub] [airflow] github-actions[bot] commented on pull request #15642: Fix k8s tests in master: Worker ServiceAccount is needed for KubernetesExecutor

2021-05-03 Thread GitBox
github-actions[bot] commented on pull request #15642: URL: https://github.com/apache/airflow/pull/15642#issuecomment-831573972 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not m

[GitHub] [airflow] monti-python edited a comment on issue #15636: WasbHook does not delete blobs with slash characters in prefix mode

2021-05-03 Thread GitBox
monti-python edited a comment on issue #15636: URL: https://github.com/apache/airflow/issues/15636#issuecomment-831398880 I reopen it cause even specifying `check_options = {'delimiter': ''}` the method still breaks when calling `delete_blobs`; `delimiter` is not accepted as kwarg here: h

[GitHub] [airflow] jedcunningham opened a new pull request #15642: Fix k8s tests in master: Worker ServiceAccount is needed for KubernetesExecutor

2021-05-03 Thread GitBox
jedcunningham opened a new pull request #15642: URL: https://github.com/apache/airflow/pull/15642 KubernetesExecutor is also in the list of executors who needs a worker ServiceAccount. This was introduced in #14152. (getting CI going now, but will take a look at unit tests t

[GitHub] [airflow] saryeHaddadi edited a comment on issue #9926: Scheduler errors when using MSSQL as the backend database

2021-05-03 Thread GitBox
saryeHaddadi edited a comment on issue #9926: URL: https://github.com/apache/airflow/issues/9926#issuecomment-831525325 Hi, I can bring some informations on: `what versions do we support, should we add support to more versions.` 1. In general, ms-sql is quite robust in term of newer

[GitHub] [airflow] saryeHaddadi edited a comment on issue #9926: Scheduler errors when using MSSQL as the backend database

2021-05-03 Thread GitBox
saryeHaddadi edited a comment on issue #9926: URL: https://github.com/apache/airflow/issues/9926#issuecomment-831525325 Hi, I can bring some informations on: `what versions do we support, should we add support to more versions.` 1. In general, ms-sql is quite robust in term of newer

[GitHub] [airflow] saryeHaddadi edited a comment on issue #9926: Scheduler errors when using MSSQL as the backend database

2021-05-03 Thread GitBox
saryeHaddadi edited a comment on issue #9926: URL: https://github.com/apache/airflow/issues/9926#issuecomment-831525325 Hi, I can bring some informations on: `what versions do we support, should we add support to more versions.` 1. In general, ms-sql is quite robust in term of newer

[GitHub] [airflow] zachliu commented on issue #14205: Scheduler "deadlocks" itself when max_active_runs_per_dag is reached by up_for_retry tasks

2021-05-03 Thread GitBox
zachliu commented on issue #14205: URL: https://github.com/apache/airflow/issues/14205#issuecomment-831552084 I forgot to ask this simple question: Since the scheduler is now HA compatible in 2.x, does it help if we spin up multiple schedulers? I'm hoping if one got deadlocked, the ot

[GitHub] [airflow] saryeHaddadi edited a comment on issue #9926: Scheduler errors when using MSSQL as the backend database

2021-05-03 Thread GitBox
saryeHaddadi edited a comment on issue #9926: URL: https://github.com/apache/airflow/issues/9926#issuecomment-831525325 Hi, I can bring some informations on: `what versions do we support, should we add support to more versions.` 1. In general, ms-sql is quite robust in term of newer

[GitHub] [airflow] CrossNox opened a new issue #15641: Add documentation on what each parameter to a `sla_miss_callback` callable is

2021-05-03 Thread GitBox
CrossNox opened a new issue #15641: URL: https://github.com/apache/airflow/issues/15641 I couldn't find any official documentation specifying what each parameter to a `sla_miss_callback` callable are. This would be a great addition to know how to properly format the messages sent. -- Th

[GitHub] [airflow] boring-cyborg[bot] commented on issue #15641: Add documentation on what each parameter to a `sla_miss_callback` callable is

2021-05-03 Thread GitBox
boring-cyborg[bot] commented on issue #15641: URL: https://github.com/apache/airflow/issues/15641#issuecomment-831542389 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] saryeHaddadi commented on issue #9926: Scheduler errors when using MSSQL as the backend database

2021-05-03 Thread GitBox
saryeHaddadi commented on issue #9926: URL: https://github.com/apache/airflow/issues/9926#issuecomment-831525325 Hi friends, I can bring you some informations on: `what versions do we support, should we add support to more versions.` 1. In general, ms-sql is quite robust in term of n

[GitHub] [airflow] SamWheating edited a comment on issue #15559: airflow dag success , but tasks not yet started,not scheduled.

2021-05-03 Thread GitBox
SamWheating edited a comment on issue #15559: URL: https://github.com/apache/airflow/issues/15559#issuecomment-831450306 We've been experiencing the same issues (Airflow 2.0.2, 2x schedulers). Furthermore, when a task has `depends_on_past=True` this will cause the DAG to completely

[GitHub] [airflow-ci-infra] mik-laj commented on a change in pull request #19: Add gcp cloud init

2021-05-03 Thread GitBox
mik-laj commented on a change in pull request #19: URL: https://github.com/apache/airflow-ci-infra/pull/19#discussion_r625356682 ## File path: github-runner-ami/packer/files/runner-supervisor.py ## @@ -137,7 +137,7 @@ def main(repo, output_folder, user): # Just keep tryi

[GitHub] [airflow-ci-infra] mik-laj commented on a change in pull request #19: Add gcp cloud init

2021-05-03 Thread GitBox
mik-laj commented on a change in pull request #19: URL: https://github.com/apache/airflow-ci-infra/pull/19#discussion_r625356257 ## File path: gcp/metrics/gcp_create_metrics_descriptor.py ## @@ -0,0 +1,56 @@ +#!/usr/bin/env python3 +# Licensed to the Apache Software Foundation

[airflow-ci-infra] branch add-gcp-cloud-init updated (2df17fd -> 37d17f3)

2021-05-03 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch add-gcp-cloud-init in repository https://gitbox.apache.org/repos/asf/airflow-ci-infra.git. discard 2df17fd Cloud init is adapted to run GCE instance add 37d17f3 Cloud init is adapted to r

[GitHub] [airflow] jedcunningham opened a new pull request #15640: Dont use the default k8s cluster domain in chart

2021-05-03 Thread GitBox
jedcunningham opened a new pull request #15640: URL: https://github.com/apache/airflow/pull/15640 K8s clusters can use a different cluster domain than the default `cluster.local`. This change will make the chart compatible with those clusters as well. -- This is an automated message

[GitHub] [airflow] turbaszek commented on a change in pull request #10343: Add extra links for google dataproc

2021-05-03 Thread GitBox
turbaszek commented on a change in pull request #10343: URL: https://github.com/apache/airflow/pull/10343#discussion_r625310534 ## File path: airflow/providers/google/cloud/operators/dataproc.py ## @@ -173,11 +218,11 @@ def __init__( properties: Optional[Dict] = None,

[GitHub] [airflow] MatthewRBruce commented on a change in pull request #15490: Fix unsuccessful KubernetesPod final_state call when `is_delete_operator_pod=True`

2021-05-03 Thread GitBox
MatthewRBruce commented on a change in pull request #15490: URL: https://github.com/apache/airflow/pull/15490#discussion_r625293615 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -365,7 +365,10 @@ def execute(self, context) -> Optional[str]:

[GitHub] [airflow] SamWheating commented on issue #15559: airflow dag success , but tasks not yet started,not scheduled.

2021-05-03 Thread GitBox
SamWheating commented on issue #15559: URL: https://github.com/apache/airflow/issues/15559#issuecomment-831450306 We've been experiencing the same issues (Airflow 2.0.2, 2x schedulers). Furthermore, when a task has `depends_on_past=True` this will cause the DAG to completely lock as

[GitHub] [airflow] dimberman commented on a change in pull request #15638: Fix issue with parsing error logs in the KPO

2021-05-03 Thread GitBox
dimberman commented on a change in pull request #15638: URL: https://github.com/apache/airflow/pull/15638#discussion_r625274718 ## File path: airflow/providers/cncf/kubernetes/utils/pod_launcher.py ## @@ -183,7 +186,12 @@ def parse_log_line(self, line: str) -> Tuple[str, str]:

[GitHub] [airflow] dimberman commented on a change in pull request #15638: Fix issue with parsing error logs in the KPO

2021-05-03 Thread GitBox
dimberman commented on a change in pull request #15638: URL: https://github.com/apache/airflow/pull/15638#discussion_r625274292 ## File path: airflow/providers/cncf/kubernetes/utils/pod_launcher.py ## @@ -183,7 +186,12 @@ def parse_log_line(self, line: str) -> Tuple[str, str]:

[GitHub] [airflow] ashb commented on issue #12728: KubePodOperator sometimes fails with `Unable to parse string [failed]`

2021-05-03 Thread GitBox
ashb commented on issue #12728: URL: https://github.com/apache/airflow/issues/12728#issuecomment-831420589 > @ashb it appears that the issue might be when we have log lines that come from errors (e.g. in this case the word was "failed" and in another case the word was "Unable"), this break

[GitHub] [airflow] ashb commented on a change in pull request #15638: Fix issue with parsing error logs in the KPO

2021-05-03 Thread GitBox
ashb commented on a change in pull request #15638: URL: https://github.com/apache/airflow/pull/15638#discussion_r625252905 ## File path: airflow/providers/cncf/kubernetes/utils/pod_launcher.py ## @@ -183,7 +186,12 @@ def parse_log_line(self, line: str) -> Tuple[str, str]:

[GitHub] [airflow] github-actions[bot] commented on pull request #15238: adds airflowConfigAnnotations field to allow for the setting of the main configmaps annotations

2021-05-03 Thread GitBox
github-actions[bot] commented on pull request #15238: URL: https://github.com/apache/airflow/pull/15238#issuecomment-831415142 [The Workflow run](https://github.com/apache/airflow/actions/runs/807577092) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build

[GitHub] [airflow] levyitay commented on pull request #15615: Feature qubole hook support headers

2021-05-03 Thread GitBox
levyitay commented on pull request #15615: URL: https://github.com/apache/airflow/pull/15615#issuecomment-831414253 @eladkal Done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [airflow] dimberman commented on issue #12728: KubePodOperator sometimes fails with `Unable to parse string [failed]`

2021-05-03 Thread GitBox
dimberman commented on issue #12728: URL: https://github.com/apache/airflow/issues/12728#issuecomment-831414223 https://github.com/apache/airflow/pull/15638 I believe this should solve the issue. Will add unit tests soon. -- This is an automated message from the Apache Git Service. To re

[GitHub] [airflow] dimberman opened a new pull request #15638: Fix issue with parsing error logs in the KPO

2021-05-03 Thread GitBox
dimberman opened a new pull request #15638: URL: https://github.com/apache/airflow/pull/15638 This fixes an issue where logs that do not have timestamps cause the KubernetesPodOperator to crash. Basically error logs created by airflow do not have timestamps, which was causing an unhand

[GitHub] [airflow] monti-python commented on issue #15636: WasbHook does not delete blobs with slash characters in prefix mode

2021-05-03 Thread GitBox
monti-python commented on issue #15636: URL: https://github.com/apache/airflow/issues/15636#issuecomment-831398880 I reopen it cause it feels a bit hacky having to specify `check_options` just to remove all files based on a given prefix. I think it's worth considering the change in #15637

[GitHub] [airflow] monti-python edited a comment on issue #15636: WasbHook does not delete blobs with slash characters in prefix mode

2021-05-03 Thread GitBox
monti-python edited a comment on issue #15636: URL: https://github.com/apache/airflow/issues/15636#issuecomment-831384476 It's possible to override default `delimiter` by using `check_options` -- This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [airflow] dimberman commented on issue #12728: KubePodOperator sometimes fails with `Unable to parse string [failed]`

2021-05-03 Thread GitBox
dimberman commented on issue #12728: URL: https://github.com/apache/airflow/issues/12728#issuecomment-831396814 @ashb it appears that the issue might be when we have log lines that come from errors (e.g. in this case the word was "failed" and in another case the word was "Unable"), this br

[GitHub] [airflow] monti-python closed pull request #15637: Update wasb.py

2021-05-03 Thread GitBox
monti-python closed pull request #15637: URL: https://github.com/apache/airflow/pull/15637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [airflow] monti-python closed issue #15636: WasbHook does not delete blobs with slash characters in prefix mode

2021-05-03 Thread GitBox
monti-python closed issue #15636: URL: https://github.com/apache/airflow/issues/15636 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, plea

[GitHub] [airflow] monti-python commented on issue #15636: WasbHook does not delete blobs with slash characters in prefix mode

2021-05-03 Thread GitBox
monti-python commented on issue #15636: URL: https://github.com/apache/airflow/issues/15636#issuecomment-831384476 Closing: It's possible to override default `delimiter` by using `check_options` -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [airflow] bfmcneill commented on issue #14379: Running Airflow in Docker - Tutorial dags backfill tutorial example fails with TypeError: cannot serialize '_io.TextIOWrapper' object

2021-05-03 Thread GitBox
bfmcneill commented on issue #14379: URL: https://github.com/apache/airflow/issues/14379#issuecomment-831382498 does this mean that python 36 is the problem? I pulled airflow 2.0.2 official docker image and it is still running on python36 -- This is an automated message from the Apache

[GitHub] [airflow] monti-python commented on issue #15636: WasbHook does not delete blobs with slash characters in prefix mode

2021-05-03 Thread GitBox
monti-python commented on issue #15636: URL: https://github.com/apache/airflow/issues/15636#issuecomment-831377567 Made a PR to fix it: #15637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [airflow] monti-python opened a new pull request #15637: Update wasb.py

2021-05-03 Thread GitBox
monti-python opened a new pull request #15637: URL: https://github.com/apache/airflow/pull/15637 closes: #15636 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #15637: Update wasb.py

2021-05-03 Thread GitBox
boring-cyborg[bot] commented on pull request #15637: URL: https://github.com/apache/airflow/pull/15637#issuecomment-831377004 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Con

[GitHub] [airflow] monti-python opened a new issue #15636: WasbHook does not delete blobs with slash characters in prefix mode

2021-05-03 Thread GitBox
monti-python opened a new issue #15636: URL: https://github.com/apache/airflow/issues/15636 **Apache Airflow version**: 2.0.1 **Environment**: - **Cloud provider or hardware configuration**: docker container - **OS** (e.g. from /etc/os-release): `Debian GNU/Linux 10 (bus

[GitHub] [airflow] boring-cyborg[bot] commented on issue #15636: WasbHook does not delete blobs with slash characters in prefix mode

2021-05-03 Thread GitBox
boring-cyborg[bot] commented on issue #15636: URL: https://github.com/apache/airflow/issues/15636#issuecomment-831374843 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] ashb commented on issue #12728: KubePodOperator sometimes fails with `Unable to parse string [failed]`

2021-05-03 Thread GitBox
ashb commented on issue #12728: URL: https://github.com/apache/airflow/issues/12728#issuecomment-831330496 @bmfisher thanks for those logs - unfortunately I need the pod logs to look at what the problematic line is, the airflow task logs just tell us where in the Airflow code the error is

[GitHub] [airflow] jsemric commented on issue #15623: Testing status of the Providers released 2021.05.01

2021-05-03 Thread GitBox
jsemric commented on issue #15623: URL: https://github.com/apache/airflow/issues/15623#issuecomment-831323378 tested the changes added by https://github.com/apache/airflow/pull/14642 involving docker provider -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] bbovenzi closed issue #15091: Add internationalization to the new UI

2021-05-03 Thread GitBox
bbovenzi closed issue #15091: URL: https://github.com/apache/airflow/issues/15091 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please c

[GitHub] [airflow] bbovenzi commented on issue #15091: Add internationalization to the new UI

2021-05-03 Thread GitBox
bbovenzi commented on issue #15091: URL: https://github.com/apache/airflow/issues/15091#issuecomment-831321890 This is nice to have but is not trivial. Without a firm commitment to support multiple languages, it isn't worth doing for now. -- This is an automated message from the Apache G

[GitHub] [airflow] github-actions[bot] commented on pull request #15615: Feature qubole hook support headers

2021-05-03 Thread GitBox
github-actions[bot] commented on pull request #15615: URL: https://github.com/apache/airflow/pull/15615#issuecomment-831291434 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not m

[GitHub] [airflow] andormarkus commented on issue #15635: ARM64 support in docker images

2021-05-03 Thread GitBox
andormarkus commented on issue #15635: URL: https://github.com/apache/airflow/issues/15635#issuecomment-831223265 We could install and run `numpy` on AWS ARM without issue, however installing `pyarrow` 2.0.0 and 3.0.0 was not possible without serious workarounds. Pyarrow 4.0.0 [support](h

[GitHub] [airflow] saryeHaddadi edited a comment on issue #9926: Scheduler errors when using MSSQL as the backend database

2021-05-03 Thread GitBox
saryeHaddadi edited a comment on issue #9926: URL: https://github.com/apache/airflow/issues/9926#issuecomment-831208627 Concerning this issue. The part of the query triggering the error is : `WHERE dag.is_paused IS 1` As part of [Ms-SQL Doc](https://docs.microsoft.com/fr-fr/sql/t-

[GitHub] [airflow] saryeHaddadi commented on issue #9926: Scheduler errors when using MSSQL as the backend database

2021-05-03 Thread GitBox
saryeHaddadi commented on issue #9926: URL: https://github.com/apache/airflow/issues/9926#issuecomment-831208627 Concerning this issue. The part of the query triggering the error is : `WHERE dag.is_paused IS 1` As part of [Ms-SQL Doc](https://docs.microsoft.com/fr-fr/sql/t-sql/que

[GitHub] [airflow] potiuk commented on pull request #15634: Fix multiple issues in Microsoft AzureContainerInstancesOperator

2021-05-03 Thread GitBox
potiuk commented on pull request #15634: URL: https://github.com/apache/airflow/pull/15634#issuecomment-831186252 Thanks @BKronenbitter. Would you mind adding/extending unit tests to cover the failure scenarios ? -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] potiuk commented on issue #15635: ARM64 support in docker images

2021-05-03 Thread GitBox
potiuk commented on issue #15635: URL: https://github.com/apache/airflow/issues/15635#issuecomment-831185123 Just checked pyarrow 4.0.0 has aarch wheels. We are using 3.0 now so maybe it's not yet solved. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] pateash commented on issue #15623: Testing status of the Providers released 2021.05.01

2021-05-03 Thread GitBox
pateash commented on issue #15623: URL: https://github.com/apache/airflow/issues/15623#issuecomment-831185090 @potiuk I have tested the changes for #15245 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] potiuk commented on issue #15635: ARM64 support in docker images

2021-05-03 Thread GitBox
potiuk commented on issue #15635: URL: https://github.com/apache/airflow/issues/15635#issuecomment-831184024 > Airflow needs to develop infrastructures to build the images. Correct. We have credits from GCP that we will be able to use to run Runners in the ARM infrastructure. We are

[GitHub] [airflow] potiuk commented on issue #15635: ARM64 support in docker images

2021-05-03 Thread GitBox
potiuk commented on issue #15635: URL: https://github.com/apache/airflow/issues/15635#issuecomment-831183055 > The biggest obstacle is likely not Airflow itself (which would likely run on ARM just fine), but the many dependencies it has. Either upstreams need to publish ARM wheels (althoug

[GitHub] [airflow] uranusjr edited a comment on issue #15635: ARM64 support in docker images

2021-05-03 Thread GitBox
uranusjr edited a comment on issue #15635: URL: https://github.com/apache/airflow/issues/15635#issuecomment-831158297 The biggest obstacle is likely not Airflow itself (which would likely run on ARM just fine), but the many dependencies it has. Either upstreams need to publish ARM wheels (

[GitHub] [airflow] uranusjr commented on issue #15635: ARM64 support in docker images

2021-05-03 Thread GitBox
uranusjr commented on issue #15635: URL: https://github.com/apache/airflow/issues/15635#issuecomment-831158297 The biggest obstacle is likely not Airflow itself (which would likely run on ARM just fine), but the many dependencies it has. For macOS, upstreams need to publish ARM wheels (alt

[GitHub] [airflow] potiuk commented on issue #15623: Testing status of the Providers released 2021.05.01

2021-05-03 Thread GitBox
potiuk commented on issue #15623: URL: https://github.com/apache/airflow/issues/15623#issuecomment-831156270 Looking forward to some testing :). Doing some testing today myself. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [airflow] potiuk commented on issue #15635: ARM64 support in docker images

2021-05-03 Thread GitBox
potiuk commented on issue #15635: URL: https://github.com/apache/airflow/issues/15635#issuecomment-831155734 Been on my radar for quite some time. We were mostly waiting for getting the docker into stable version for MacOS. And yeah I think this is the next 'big' thing for the images once

[GitHub] [airflow] potiuk commented on issue #15635: ARM64 support in docker images

2021-05-03 Thread GitBox
potiuk commented on issue #15635: URL: https://github.com/apache/airflow/issues/15635#issuecomment-831154219 Absolutely -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. Fo

[GitHub] [airflow] subashcanapathy commented on issue #14592: Unreachable Secrets Backend Causes Web Server Crash

2021-05-03 Thread GitBox
subashcanapathy commented on issue #14592: URL: https://github.com/apache/airflow/issues/14592#issuecomment-831120616 DAG serialization is turned on for John's use case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] boring-cyborg[bot] commented on issue #15635: ARM64 support in docker images

2021-05-03 Thread GitBox
boring-cyborg[bot] commented on issue #15635: URL: https://github.com/apache/airflow/issues/15635#issuecomment-831117088 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] andormarkus opened a new issue #15635: ARM64 support in docker images

2021-05-03 Thread GitBox
andormarkus opened a new issue #15635: URL: https://github.com/apache/airflow/issues/15635 **Description** ARM64 support in docker images **Use case / motivation** In the recent years ARM based cpu popularity hugely increased. The arm based cpus are offering better pric

[GitHub] [airflow] janvaca92 commented on issue #15482: Unable to log to Azure PVC

2021-05-03 Thread GitBox
janvaca92 commented on issue #15482: URL: https://github.com/apache/airflow/issues/15482#issuecomment-831105942 We have faced the same issue. The problem is that Azure fileshare does not allowed changing file permissions via os.chmod. With the managed disks is possible but they require Rea

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #15634: Fix multiple issues in Microsoft AzureContainerInstancesOperator

2021-05-03 Thread GitBox
boring-cyborg[bot] commented on pull request #15634: URL: https://github.com/apache/airflow/pull/15634#issuecomment-831103766 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Con

[GitHub] [airflow] BKronenbitter opened a new pull request #15634: Fix multiple issues in Microsoft AzureContainerInstancesOperator

2021-05-03 Thread GitBox
BKronenbitter opened a new pull request #15634: URL: https://github.com/apache/airflow/pull/15634 After having worked with with Airflow in Azure using the AzureContainerInstancesOperator I encountered some issues. - The sleep command seems to be indented wrongly and not run periodically

[GitHub] [airflow] eladkal commented on issue #15547: Jinja templating throws bug when using a PostgresOperator within the Taskflow API

2021-05-03 Thread GitBox
eladkal commented on issue #15547: URL: https://github.com/apache/airflow/issues/15547#issuecomment-831087945 I wrote this as an explanation why operators are not being templated inside PythonOperator. I agree completely that the prefered way is to use hooks (maybe also custom operator). T