[GitHub] [airflow] anilkulkarni87 opened a new issue #14111: Error: ModuleNotFoundError: No module named 'airflow' | Docker Image Windows WSL2

2021-02-05 Thread GitBox
anilkulkarni87 opened a new issue #14111: URL: https://github.com/apache/airflow/issues/14111 **Apache Airflow version**: 2.0.0 **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): NA **Environment**: Local Windows Laptop - **Cloud provi

[GitHub] [airflow] boring-cyborg[bot] commented on issue #14111: Error: ModuleNotFoundError: No module named 'airflow' | Docker Image Windows WSL2

2021-02-05 Thread GitBox
boring-cyborg[bot] commented on issue #14111: URL: https://github.com/apache/airflow/issues/14111#issuecomment-774418715 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automate

[GitHub] [airflow] chrismed commented on issue #14089: S3 Remote Logging Kubernetes Executor worker task keeps waiting to send log: "acquiring 0"

2021-02-05 Thread GitBox
chrismed commented on issue #14089: URL: https://github.com/apache/airflow/issues/14089#issuecomment-774397372 Another person (from Apache slack channel) also described the exact issue I saw: https://apache-airflow.slack.com/archives/CSS36QQS1/p1612336991103900 ---

[GitHub] [airflow] kaxil opened a new pull request #14109: Add a decorator to retry functions with DB transactions

2021-02-05 Thread GitBox
kaxil opened a new pull request #14109: URL: https://github.com/apache/airflow/pull/14109 This commit adds a decorator: `@retry_db_transaction` to retry Methods and functions when there is an `OperationalError`. This unifies the retry configs in a single place and the adds logging.

[GitHub] [airflow] kaxil edited a comment on pull request #13893: Fix race condition when using Dynamic DAGs

2021-02-05 Thread GitBox
kaxil edited a comment on pull request #13893: URL: https://github.com/apache/airflow/pull/13893#issuecomment-774374643 > I am still seeing a lot of errors in 2.0.0rc2 like > > ``` > airflow.exceptions.SerializedDagNotFound: DAG 'targeting_pipeline_dappack.shard_3' not found in se

[GitHub] [airflow] kaxil commented on pull request #13893: Fix race condition when using Dynamic DAGs

2021-02-05 Thread GitBox
kaxil commented on pull request #13893: URL: https://github.com/apache/airflow/pull/13893#issuecomment-774374643 > I am still seeing a lot of errors in 2.0.0rc2 like > > ``` > airflow.exceptions.SerializedDagNotFound: DAG 'targeting_pipeline_dappack.shard_3' not found in serialize

[GitHub] [airflow] mik-laj commented on a change in pull request #14033: Doc + example dag for: CloudDLPDeidentifyContentOperator, GCSObjectExistenceSensor, GCSObjectsWithPrefixExistenceSensor

2021-02-05 Thread GitBox
mik-laj commented on a change in pull request #14033: URL: https://github.com/apache/airflow/pull/14033#discussion_r571323573 ## File path: tests/providers/google/cloud/operators/test_gcs_system.py ## @@ -22,6 +22,7 @@ from tests.test_utils.gcp_system_helpers import CLOUD_DAG_

[GitHub] [airflow] mik-laj commented on a change in pull request #14033: Doc + example dag for: CloudDLPDeidentifyContentOperator, GCSObjectExistenceSensor, GCSObjectsWithPrefixExistenceSensor

2021-02-05 Thread GitBox
mik-laj commented on a change in pull request #14033: URL: https://github.com/apache/airflow/pull/14033#discussion_r571323095 ## File path: docs/spelling_wordlist.txt ## @@ -101,6 +101,7 @@ Davydov De Decrypt Decrypts +DeidentifyConfig Review comment: ```suggestion

[GitHub] [airflow] mik-laj commented on a change in pull request #14033: Doc + example dag for: CloudDLPDeidentifyContentOperator, GCSObjectExistenceSensor, GCSObjectsWithPrefixExistenceSensor

2021-02-05 Thread GitBox
mik-laj commented on a change in pull request #14033: URL: https://github.com/apache/airflow/pull/14033#discussion_r571322768 ## File path: docs/conf.py ## @@ -151,9 +151,7 @@ 'README.rst', ] elif PACKAGE_NAME.startswith('apache-airflow-providers-'): -exclude

[GitHub] [airflow] mik-laj commented on a change in pull request #14033: Doc + example dag for: CloudDLPDeidentifyContentOperator, GCSObjectExistenceSensor, GCSObjectsWithPrefixExistenceSensor

2021-02-05 Thread GitBox
mik-laj commented on a change in pull request #14033: URL: https://github.com/apache/airflow/pull/14033#discussion_r571322690 ## File path: docs/apache-airflow-providers-google/operators/cloud/gcs.rst ## @@ -111,37 +118,79 @@ More information See Google Cloud Storage insert do

[GitHub] [airflow] mik-laj commented on a change in pull request #14033: Doc + example dag for: CloudDLPDeidentifyContentOperator, GCSObjectExistenceSensor, GCSObjectsWithPrefixExistenceSensor

2021-02-05 Thread GitBox
mik-laj commented on a change in pull request #14033: URL: https://github.com/apache/airflow/pull/14033#discussion_r571322501 ## File path: docs/apache-airflow-providers-google/operators/cloud/gcs.rst ## @@ -111,37 +118,79 @@ More information See Google Cloud Storage insert do

[GitHub] [airflow] mik-laj commented on a change in pull request #14033: Doc + example dag for: CloudDLPDeidentifyContentOperator, GCSObjectExistenceSensor, GCSObjectsWithPrefixExistenceSensor

2021-02-05 Thread GitBox
mik-laj commented on a change in pull request #14033: URL: https://github.com/apache/airflow/pull/14033#discussion_r571321860 ## File path: docs/apache-airflow-providers-google/operators/cloud/data_loss_prevention.rst ## @@ -296,10 +296,25 @@ Unlike storage methods (Jobs) cont

[GitHub] [airflow] TobKed commented on issue #13262: Dataflow Flex Template Operator

2021-02-05 Thread GitBox
TobKed commented on issue #13262: URL: https://github.com/apache/airflow/issues/13262#issuecomment-774359290 Taking `airflow/providers/google/cloud/hooks/dataflow.py` from @terekete [fix](https://github.com/apache/airflow/pull/13478) and putting it locally (alongside operator code) will ca

[GitHub] [airflow] mik-laj commented on issue #14097: Typo in Sensor: GCSObjectsWtihPrefixExistenceSensor (should be GCSObjectsWithPrefixExistenceSensor)

2021-02-05 Thread GitBox
mik-laj commented on issue #14097: URL: https://github.com/apache/airflow/issues/14097#issuecomment-774357371 You should change the name and then add the code below for backward compatibility. ``` class GCSObjectsWtihPrefixExistenceSensor(GCSObjectsWithPrefixExistenceSensor):

[GitHub] [airflow] fjmacagno commented on pull request #13893: Fix race condition when using Dynamic DAGs

2021-02-05 Thread GitBox
fjmacagno commented on pull request #13893: URL: https://github.com/apache/airflow/pull/13893#issuecomment-774353483 I am still seeing a lot of errors in 2.0.0rc2 like ``` airflow.exceptions.SerializedDagNotFound: DAG 'targeting_pipeline_dappack.shard_3' not found in serialized_ da

[GitHub] [airflow] dimberman commented on pull request #14085: Ensure executors end method is called

2021-02-05 Thread GitBox
dimberman commented on pull request #14085: URL: https://github.com/apache/airflow/pull/14085#issuecomment-774342376 @jedcunningham this LGTM. With regards to documentation, @kaxil do we need to document this? Seems like it fixes a bug instead of modifying a functionality

[GitHub] [airflow] milton0825 edited a comment on issue #7935: scheduler gets stuck without a trace

2021-02-05 Thread GitBox
milton0825 edited a comment on issue #7935: URL: https://github.com/apache/airflow/issues/7935#issuecomment-774340918 Seeing this on 1.10.8 with Celery executor. We are running the scheduler with num duration 900 seconds. It would run fine for a couple of days then suddenly just freeze.

[GitHub] [airflow] milton0825 commented on issue #7935: scheduler gets stuck without a trace

2021-02-05 Thread GitBox
milton0825 commented on issue #7935: URL: https://github.com/apache/airflow/issues/7935#issuecomment-774340918 Seeing this on 1.10.8 with Celery executor ``` Thread 1 (idle): "MainThread" wait (threading.py:295) wait (threading.py:551) wait (multiprocessing/pool.py

[GitHub] [airflow] matthieucham commented on issue #13262: Dataflow Flex Template Operator

2021-02-05 Thread GitBox
matthieucham commented on issue #13262: URL: https://github.com/apache/airflow/issues/13262#issuecomment-774336942 Thanks for fixing this Is there a way around this bug ? I could not find one : wait_until_finished=True as operator argument does not seem to change anything :( -

[jira] [Commented] (AIRFLOW-6994) SparkSubmitOperator re launches spark driver even when original driver still running

2021-02-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17280039#comment-17280039 ] ASF GitHub Bot commented on AIRFLOW-6994: - tooptoop4 commented on pull request

[GitHub] [airflow] tooptoop4 commented on pull request #7637: [AIRFLOW-6994] SparkSubmitOperator re-launches spark driver even when original driver still running

2021-02-05 Thread GitBox
tooptoop4 commented on pull request #7637: URL: https://github.com/apache/airflow/pull/7637#issuecomment-774336284 nah' This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[airflow] branch master updated (8a731f5 -> d4c4db8)

2021-02-05 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 8a731f5 Improve Apache Beam operators - refactor operator - common Dataflow logic (#14094) add d4c4db8 Al

[GitHub] [airflow] dimberman closed issue #13348: env_vars of KubernetesPodOperator are not truly templated

2021-02-05 Thread GitBox
dimberman closed issue #13348: URL: https://github.com/apache/airflow/issues/13348 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [airflow] dimberman merged pull request #14083: Allow users of the KPO to template environment variables

2021-02-05 Thread GitBox
dimberman merged pull request #14083: URL: https://github.com/apache/airflow/pull/14083 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] dimberman closed issue #13522: Templated Variables being treated as String by KubernetesPodOperator in Airflow 2.0

2021-02-05 Thread GitBox
dimberman closed issue #13522: URL: https://github.com/apache/airflow/issues/13522 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [airflow] jedcunningham commented on pull request #14085: Ensure executors end method is called

2021-02-05 Thread GitBox
jedcunningham commented on pull request #14085: URL: https://github.com/apache/airflow/pull/14085#issuecomment-774321409 @dimberman, take another look. This is an automated message from the Apache Git Service. To respond to t

[GitHub] [airflow] jedcunningham commented on a change in pull request #14085: Ensure executors end method is called

2021-02-05 Thread GitBox
jedcunningham commented on a change in pull request #14085: URL: https://github.com/apache/airflow/pull/14085#discussion_r571283551 ## File path: airflow/jobs/scheduler_job.py ## @@ -1292,13 +1292,12 @@ def _execute(self) -> None: ) models.DAG.

[GitHub] [airflow] potiuk commented on pull request #13730: Run CI (Build Image and committer PRs) on self-hosted runner

2021-02-05 Thread GitBox
potiuk commented on pull request #13730: URL: https://github.com/apache/airflow/pull/13730#issuecomment-774317905 I will take a look tomorrow This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [airflow] ashb commented on pull request #13730: Run CI (Build Image and committer PRs) on self-hosted runner

2021-02-05 Thread GitBox
ashb commented on pull request #13730: URL: https://github.com/apache/airflow/pull/13730#issuecomment-774316238 (Doesn't look like tmpfs helps much with the "main" test suite) This is an automated message from the Apache Git

[GitHub] [airflow] potiuk merged pull request #14094: Improve Apache Beam operators - refactor operator - common Dataflow logic

2021-02-05 Thread GitBox
potiuk merged pull request #14094: URL: https://github.com/apache/airflow/pull/14094 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[airflow] branch master updated (212d5cd -> 8a731f5)

2021-02-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 212d5cd Restores flexible installation version, fixes manual tag build process. (#14107) add 8a731f5 Improve

[GitHub] [airflow] github-actions[bot] commented on pull request #13730: Run CI (Build Image and committer PRs) on self-hosted runner

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #13730: URL: https://github.com/apache/airflow/pull/13730#issuecomment-774307601 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

[GitHub] [airflow] potiuk commented on a change in pull request #13730: Run CI (Build Image and committer PRs) on self-hosted runner

2021-02-05 Thread GitBox
potiuk commented on a change in pull request #13730: URL: https://github.com/apache/airflow/pull/13730#discussion_r571267998 ## File path: .github/workflows/ci.yml ## @@ -105,7 +117,12 @@ jobs: needs-api-codegen: ${{ steps.selective-checks.outputs.needs-api-codegen }}

[GitHub] [airflow] ashb commented on pull request #13730: Run CI (Build Image and committer PRs) on self-hosted runner

2021-02-05 Thread GitBox
ashb commented on pull request #13730: URL: https://github.com/apache/airflow/pull/13730#issuecomment-774305020 I'm running now with tmpfs for `/var/lib/docker`, `/tmp` and the runner `_work` dir. No changes needed on our code side to support it, all just a machine config change. ---

[GitHub] [airflow] ClassyLion commented on issue #13805: Could not get scheduler_job_id

2021-02-05 Thread GitBox
ClassyLion commented on issue #13805: URL: https://github.com/apache/airflow/issues/13805#issuecomment-774297920 > So the solution here would be to give an arbitrary scheduler_job_id "a.k.a. 'backfill' or 'manual'" For these two use-cases. > > @yjwong @ClassyLion would either of you

[GitHub] [airflow] github-actions[bot] commented on pull request #13080: KubernetesExecutor overrides should only append to lists

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #13080: URL: https://github.com/apache/airflow/pull/13080#issuecomment-774282126 [The Workflow run](https://github.com/apache/airflow/actions/runs/541571186) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build

[airflow] branch master updated (5d7d46b -> 212d5cd)

2021-02-05 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 5d7d46b Improve GCSToSFTPOperator paths handling (#11284) add 212d5cd Restores flexible installation version, fi

[GitHub] [airflow] ashb merged pull request #14107: Restores flexible installation version, fixes manual tag build process.

2021-02-05 Thread GitBox
ashb merged pull request #14107: URL: https://github.com/apache/airflow/pull/14107 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [airflow] ashb commented on pull request #14107: Restores flexible installation version, fixes manual tag build process.

2021-02-05 Thread GitBox
ashb commented on pull request #14107: URL: https://github.com/apache/airflow/pull/14107#issuecomment-774280782 `ERROR: Invalid requirement: '.[devel_ci]=='` will be fixed by this right? This is an automated message from the

[GitHub] [airflow] jedcunningham commented on a change in pull request #14085: Ensure executors end method is called

2021-02-05 Thread GitBox
jedcunningham commented on a change in pull request #14085: URL: https://github.com/apache/airflow/pull/14085#discussion_r570601695 ## File path: airflow/jobs/scheduler_job.py ## @@ -1292,13 +1292,12 @@ def _execute(self) -> None: ) models.DAG.

[GitHub] [airflow] github-actions[bot] commented on pull request #14083: Allow users of the KPO to template environment variables

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #14083: URL: https://github.com/apache/airflow/pull/14083#issuecomment-774265544 [The Workflow run](https://github.com/apache/airflow/actions/runs/541199160) is cancelling this PR. Building images for the PR has failed. Follow the the workflow

[GitHub] [airflow] github-actions[bot] commented on pull request #13197: Fix documentation error in git_sync_template.yaml

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #13197: URL: https://github.com/apache/airflow/pull/13197#issuecomment-774265296 [The Workflow run](https://github.com/apache/airflow/actions/runs/541206810) is cancelling this PR. Building images for the PR has failed. Follow the the workflow

[GitHub] [airflow] github-actions[bot] commented on pull request #14105: Add Google leveldb hook (#13109)

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #14105: URL: https://github.com/apache/airflow/pull/14105#issuecomment-774256592 [The Workflow run](https://github.com/apache/airflow/actions/runs/540838117) is cancelling this PR. Building images for the PR has failed. Follow the the workflow

[GitHub] [airflow] github-actions[bot] commented on pull request #13929: Parse session from positional/kwargs argument

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #13929: URL: https://github.com/apache/airflow/pull/13929#issuecomment-774256201 [The Workflow run](https://github.com/apache/airflow/actions/runs/540832426) is cancelling this PR. Building images for the PR has failed. Follow the the workflow

[GitHub] [airflow] github-actions[bot] commented on pull request #14103: MongoDB color for MongoToS3Operator

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #14103: URL: https://github.com/apache/airflow/pull/14103#issuecomment-774256118 [The Workflow run](https://github.com/apache/airflow/actions/runs/540787789) is cancelling this PR. Building images for the PR has failed. Follow the the workflow

[GitHub] [airflow] github-actions[bot] commented on pull request #14080: Refactor redundant doc url logic to use utility

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #14080: URL: https://github.com/apache/airflow/pull/14080#issuecomment-774253658 [The Workflow run](https://github.com/apache/airflow/actions/runs/540729591) is cancelling this PR. Building images for the PR has failed. Follow the the workflow

[GitHub] [airflow] github-actions[bot] commented on pull request #14102: Remove reinstalling azure-storage steps from CI / Breeze

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #14102: URL: https://github.com/apache/airflow/pull/14102#issuecomment-774250852 [The Workflow run](https://github.com/apache/airflow/actions/runs/54060) is cancelling this PR. Building images for the PR has failed. Follow the the workflow

[GitHub] [airflow] github-actions[bot] commented on pull request #14085: Ensure executors end method is called

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #14085: URL: https://github.com/apache/airflow/pull/14085#issuecomment-774247460 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

[GitHub] [airflow] fuxiao224 commented on a change in pull request #13980: update BigQueryUpsertTableOperator class to support CMEK

2021-02-05 Thread GitBox
fuxiao224 commented on a change in pull request #13980: URL: https://github.com/apache/airflow/pull/13980#discussion_r571204620 ## File path: airflow/providers/google/cloud/operators/bigquery.py ## @@ -1862,6 +1862,13 @@ class BigQueryUpsertTableOperator(BaseOperator): :ty

[GitHub] [airflow] dimberman commented on issue #13805: Could not get scheduler_job_id

2021-02-05 Thread GitBox
dimberman commented on issue #13805: URL: https://github.com/apache/airflow/issues/13805#issuecomment-774241825 So the solution here would be to give an arbitrary scheduler_job_id "a.k.a. 'backfill' or 'manual'" For these two use-cases. @yjwong @ClassyLion would either of you be inte

[GitHub] [airflow] dimberman commented on issue #11982: Scheduler not exiting after SIGTERM with KubeExecutor

2021-02-05 Thread GitBox
dimberman commented on issue #11982: URL: https://github.com/apache/airflow/issues/11982#issuecomment-774238500 @jedcunningham looks good to me! Please rebase on master to get tests running and then we can merge. This is an

[GitHub] [airflow] github-actions[bot] commented on pull request #13197: Fix documentation error in git_sync_template.yaml

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #13197: URL: https://github.com/apache/airflow/pull/13197#issuecomment-774236018 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

[GitHub] [airflow] dimberman commented on issue #12728: DAG execution works some times but throws Unable to parse string [failed] error at other times

2021-02-05 Thread GitBox
dimberman commented on issue #12728: URL: https://github.com/apache/airflow/issues/12728#issuecomment-774235142 @m0bi5 I'd say for the moment let's just hack it in and run the DAGs so we can see what the response from k8s looks like when this fails, once that's set we can get you set up wi

[airflow] branch master updated (f92194b -> 5d7d46b)

2021-02-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from f92194b Update Airflow Releasing Guide with 1.10.x and 2.0 commands (#14098) add 5d7d46b Improve GCSToSFTPOp

[GitHub] [airflow] potiuk merged pull request #11284: Improve GCSToSFTPOperator paths handling

2021-02-05 Thread GitBox
potiuk merged pull request #11284: URL: https://github.com/apache/airflow/pull/11284 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] dimberman commented on pull request #10302: [BUGFIX] podMutations are now deployed correctly to airflow_local_settings.py

2021-02-05 Thread GitBox
dimberman commented on pull request #10302: URL: https://github.com/apache/airflow/pull/10302#issuecomment-774231741 @alikhtag can you please get the tests to pass? Would be glad to merge once they are passing :). This is an

[GitHub] [airflow] github-actions[bot] commented on pull request #14107: Restores flexible installation version, fixes manual tag build process.

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #14107: URL: https://github.com/apache/airflow/pull/14107#issuecomment-774230104 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

[GitHub] [airflow] potiuk merged pull request #14098: Update Airflow Releasing Guide with 1.10.x and 2.0 commands

2021-02-05 Thread GitBox
potiuk merged pull request #14098: URL: https://github.com/apache/airflow/pull/14098 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[airflow] branch master updated (10343ec -> f92194b)

2021-02-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 10343ec Corrections in docs and tools after releasing provider RCs (#14082) add f92194b Update Airflow Relea

[GitHub] [airflow] potiuk merged pull request #14082: Corrections in docs and tools after releasing provider RCs

2021-02-05 Thread GitBox
potiuk merged pull request #14082: URL: https://github.com/apache/airflow/pull/14082 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[airflow] branch master updated (fc67521 -> 10343ec)

2021-02-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from fc67521 Fix order of failed deps (#14036) add 10343ec Corrections in docs and tools after releasing provider

[GitHub] [airflow] potiuk commented on pull request #14107: Restores flexible installation version, fixes manual tag build process.

2021-02-05 Thread GitBox
potiuk commented on pull request #14107: URL: https://github.com/apache/airflow/pull/14107#issuecomment-774223676 Just did. Actually I think master is broken now before this one gets merged https://github.com/potiuk/airflow/actions/runs/541317863

[GitHub] [airflow] dimberman commented on pull request #8009: Airflow kubernetes cli command for generating pod templates

2021-02-05 Thread GitBox
dimberman commented on pull request #8009: URL: https://github.com/apache/airflow/pull/8009#issuecomment-774221839 Hi @Javier162380 are you interested in continuing with this task? Glad to help :). This is an automated messa

[jira] [Commented] (AIRFLOW-6994) SparkSubmitOperator re launches spark driver even when original driver still running

2021-02-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17279913#comment-17279913 ] ASF GitHub Bot commented on AIRFLOW-6994: - dimberman commented on pull request

[GitHub] [airflow] dimberman commented on pull request #7637: [AIRFLOW-6994] SparkSubmitOperator re-launches spark driver even when original driver still running

2021-02-05 Thread GitBox
dimberman commented on pull request #7637: URL: https://github.com/apache/airflow/pull/7637#issuecomment-774221367 @tooptoop4 are you able to add a test to this? It just needs to be something that if the user changes the logic we will be notified via test failure.

[jira] [Commented] (AIRFLOW-6829) Auto apply apply_default decorator to all sub-classess

2021-02-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17279911#comment-17279911 ] ASF GitHub Bot commented on AIRFLOW-6829: - mik-laj commented on pull request #7

[GitHub] [airflow] mik-laj commented on pull request #7450: [AIRFLOW-6829][WIP] Auto-apply apply_default

2021-02-05 Thread GitBox
mik-laj commented on pull request #7450: URL: https://github.com/apache/airflow/pull/7450#issuecomment-774220730 It is still relevant. This is an automated message from the Apache Git Service. To respond to the message, plea

[jira] [Commented] (AIRFLOW-6829) Auto apply apply_default decorator to all sub-classess

2021-02-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17279910#comment-17279910 ] ASF GitHub Bot commented on AIRFLOW-6829: - dimberman commented on pull request

[GitHub] [airflow] criccomini commented on a change in pull request #13980: update BigQueryUpsertTableOperator class to support CMEK

2021-02-05 Thread GitBox
criccomini commented on a change in pull request #13980: URL: https://github.com/apache/airflow/pull/13980#discussion_r571177335 ## File path: airflow/providers/google/cloud/operators/bigquery.py ## @@ -1912,8 +1920,11 @@ def __init__( self.gcp_conn_id = gcp_conn_id

[GitHub] [airflow] dimberman commented on pull request #7450: [AIRFLOW-6829][WIP] Auto-apply apply_default

2021-02-05 Thread GitBox
dimberman commented on pull request #7450: URL: https://github.com/apache/airflow/pull/7450#issuecomment-774220321 @mik-laj is this PR still relevant or can we close it? This is an automated message from the Apache Git Servic

[jira] [Commented] (AIRFLOW-4534) Remove redundant list around dict comprehensions

2021-02-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17279905#comment-17279905 ] ASF GitHub Bot commented on AIRFLOW-4534: - dimberman commented on pull request

[jira] [Commented] (AIRFLOW-5307) Move the BaseOperator to the operators package

2021-02-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17279909#comment-17279909 ] ASF GitHub Bot commented on AIRFLOW-5307: - dimberman closed pull request #5910:

[GitHub] [airflow] dimberman closed pull request #5910: [WIP][AIRFLOW-5307] Move BaseOperator to airflow.operators.base_operator

2021-02-05 Thread GitBox
dimberman closed pull request #5910: URL: https://github.com/apache/airflow/pull/5910 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [airflow] dimberman commented on pull request #5910: [WIP][AIRFLOW-5307] Move BaseOperator to airflow.operators.base_operator

2021-02-05 Thread GitBox
dimberman commented on pull request #5910: URL: https://github.com/apache/airflow/pull/5910#issuecomment-774219920 @BasPH I'm going to close this but please feel free to re-open if you feel this is still relevant. This is an

[jira] [Commented] (AIRFLOW-5307) Move the BaseOperator to the operators package

2021-02-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17279908#comment-17279908 ] ASF GitHub Bot commented on AIRFLOW-5307: - dimberman commented on pull request

[GitHub] [airflow] dimberman commented on pull request #5298: [WIP][AIRFLOW-4534] Remove redundant list around dict comprehensions

2021-02-05 Thread GitBox
dimberman commented on pull request #5298: URL: https://github.com/apache/airflow/pull/5298#issuecomment-774219534 @BasPH I'm going to close this but please feel free to re-open if you feel this is still relevant. This is an

[jira] [Commented] (AIRFLOW-4534) Remove redundant list around dict comprehensions

2021-02-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17279904#comment-17279904 ] ASF GitHub Bot commented on AIRFLOW-4534: - dimberman closed pull request #5298:

[GitHub] [airflow] dimberman closed pull request #5298: [WIP][AIRFLOW-4534] Remove redundant list around dict comprehensions

2021-02-05 Thread GitBox
dimberman closed pull request #5298: URL: https://github.com/apache/airflow/pull/5298 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [airflow] dimberman closed pull request #9845: Fix PodGenerator to handle Kubernetes resources

2021-02-05 Thread GitBox
dimberman closed pull request #9845: URL: https://github.com/apache/airflow/pull/9845 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [airflow] dimberman commented on pull request #9845: Fix PodGenerator to handle Kubernetes resources

2021-02-05 Thread GitBox
dimberman commented on pull request #9845: URL: https://github.com/apache/airflow/pull/9845#issuecomment-774218955 @odracci I'm going to close this issue but please let me know if this issue is still a problem in 1.10.14 and we can re-open/merge it. ---

[GitHub] [airflow] github-actions[bot] commented on pull request #14102: Remove reinstalling azure-storage steps from CI / Breeze

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #14102: URL: https://github.com/apache/airflow/pull/14102#issuecomment-774216340 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

[GitHub] [airflow] kaxil commented on pull request #14107: Restores flexible installation version, fixes manual tag build process.

2021-02-05 Thread GitBox
kaxil commented on pull request #14107: URL: https://github.com/apache/airflow/pull/14107#issuecomment-774213778 @potiuk Can you also run this one on your fork -- so we can merge it when it is green This is an automated mess

[GitHub] [airflow] github-actions[bot] commented on pull request #14029: Add statsd extraMappings

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #14029: URL: https://github.com/apache/airflow/pull/14029#issuecomment-774213689 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not m

[GitHub] [airflow] github-actions[bot] commented on pull request #14103: MongoDB color for MongoToS3Operator

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #14103: URL: https://github.com/apache/airflow/pull/14103#issuecomment-774210792 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not m

[GitHub] [airflow] potiuk commented on a change in pull request #13832: removing try-catch block to fix timeout exception getting ignored in aws batch operator

2021-02-05 Thread GitBox
potiuk commented on a change in pull request #13832: URL: https://github.com/apache/airflow/pull/13832#discussion_r571159300 ## File path: airflow/providers/amazon/aws/operators/batch.py ## @@ -177,29 +177,26 @@ def submit_job(self, context: Dict): # pylint: disable=unused-ar

[GitHub] [airflow] potiuk commented on a change in pull request #13832: removing try-catch block to fix timeout exception getting ignored in aws batch operator

2021-02-05 Thread GitBox
potiuk commented on a change in pull request #13832: URL: https://github.com/apache/airflow/pull/13832#discussion_r571159300 ## File path: airflow/providers/amazon/aws/operators/batch.py ## @@ -177,29 +177,26 @@ def submit_job(self, context: Dict): # pylint: disable=unused-ar

[GitHub] [airflow] potiuk commented on a change in pull request #13832: removing try-catch block to fix timeout exception getting ignored in aws batch operator

2021-02-05 Thread GitBox
potiuk commented on a change in pull request #13832: URL: https://github.com/apache/airflow/pull/13832#discussion_r571159300 ## File path: airflow/providers/amazon/aws/operators/batch.py ## @@ -177,29 +177,26 @@ def submit_job(self, context: Dict): # pylint: disable=unused-ar

[airflow] branch fix-env-from-1-10 updated (eeb3833 -> c1f5768)

2021-02-05 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a change to branch fix-env-from-1-10 in repository https://gitbox.apache.org/repos/asf/airflow.git. discard eeb3833 fix pod test discard 25d3afc simplify pod_mutation_backcompat discard b7df326 KubernetesE

[GitHub] [airflow] ayushchauhan0811 commented on a change in pull request #13832: removing try-catch block to fix timeout exception getting ignored in aws batch operator

2021-02-05 Thread GitBox
ayushchauhan0811 commented on a change in pull request #13832: URL: https://github.com/apache/airflow/pull/13832#discussion_r571154673 ## File path: airflow/providers/amazon/aws/operators/batch.py ## @@ -177,29 +177,26 @@ def submit_job(self, context: Dict): # pylint: disable

[GitHub] [airflow] potiuk commented on issue #7785: Airflow UI showing "No module named 'snowflake'" error after importing snowflake hook

2021-02-05 Thread GitBox
potiuk commented on issue #7785: URL: https://github.com/apache/airflow/issues/7785#issuecomment-774194370 We are just voting on new providers and snowflake providers are part of it (release candidates): * Backport providers for 1.10: https://pypi.org/project/apache-airflow-backport

[GitHub] [airflow] dimberman commented on pull request #14090: Fix merging of secrets and configmaps

2021-02-05 Thread GitBox
dimberman commented on pull request #14090: URL: https://github.com/apache/airflow/pull/14090#issuecomment-774194005 Agreed once we have a test we can merge this This is an automated message from the Apache Git Service. To re

[GitHub] [airflow] dstandish commented on a change in pull request #13832: removing try-catch block to fix timeout exception getting ignored in aws batch operator

2021-02-05 Thread GitBox
dstandish commented on a change in pull request #13832: URL: https://github.com/apache/airflow/pull/13832#discussion_r571151494 ## File path: airflow/providers/amazon/aws/operators/batch.py ## @@ -177,29 +177,26 @@ def submit_job(self, context: Dict): # pylint: disable=unused

[GitHub] [airflow] dstandish commented on a change in pull request #13832: removing try-catch block to fix timeout exception getting ignored in aws batch operator

2021-02-05 Thread GitBox
dstandish commented on a change in pull request #13832: URL: https://github.com/apache/airflow/pull/13832#discussion_r571151494 ## File path: airflow/providers/amazon/aws/operators/batch.py ## @@ -177,29 +177,26 @@ def submit_job(self, context: Dict): # pylint: disable=unused

[GitHub] [airflow] potiuk commented on a change in pull request #14107: Restores flexible installation version, fixes manual tag build process.

2021-02-05 Thread GitBox
potiuk commented on a change in pull request #14107: URL: https://github.com/apache/airflow/pull/14107#discussion_r571147783 ## File path: scripts/ci/images/ci_build_dockerhub.sh ## @@ -113,8 +113,8 @@ else export DOCKER_CACHE="local" # Name the image based on the TAG

[GitHub] [airflow] potiuk commented on pull request #14102: Remove reinstalling azure-storage steps from CI / Breeze

2021-02-05 Thread GitBox
potiuk commented on pull request #14102: URL: https://github.com/apache/airflow/pull/14102#issuecomment-774186891 Let's do it step-by step :) This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [airflow] potiuk commented on issue #14106: Lineage Backend removed for no reason

2021-02-05 Thread GitBox
potiuk commented on issue #14106: URL: https://github.com/apache/airflow/issues/14106#issuecomment-774186142 I think only @bolkedebruin so far was really pursuing the lineage, but I am sure if you would like to re-add it we can jointly review it and get it in 2.1, I think it is important t

[GitHub] [airflow] dinigo commented on issue #12611: Switch from Email to Notification

2021-02-05 Thread GitBox
dinigo commented on issue #12611: URL: https://github.com/apache/airflow/issues/12611#issuecomment-774184431 Thanks for the bump @eladkal We both, OP from #8686 struggle with integrating Airflow in a SRE control plane --

[GitHub] [airflow] potiuk commented on pull request #14107: Restores flexible installation version, fixes manual tag build process.

2021-02-05 Thread GitBox
potiuk commented on pull request #14107: URL: https://github.com/apache/airflow/pull/14107#issuecomment-774183338 See my comment https://github.com/apache/airflow/pull/14107#discussion_r571138631 - I think this version I proposed now is fine, and I can make a follow-up PR where I separate

[GitHub] [airflow] potiuk commented on a change in pull request #14107: Restores flexible installation version, fixes manual tag build process.

2021-02-05 Thread GitBox
potiuk commented on a change in pull request #14107: URL: https://github.com/apache/airflow/pull/14107#discussion_r571139040 ## File path: scripts/ci/images/ci_build_dockerhub.sh ## @@ -113,8 +113,8 @@ else export DOCKER_CACHE="local" # Name the image based on the TAG

[GitHub] [airflow] dstandish commented on issue #13785: Apache Airflow's --> EmrAddStepsOperator throws up error as "ERROR - You must specify a region"

2021-02-05 Thread GitBox
dstandish commented on issue #13785: URL: https://github.com/apache/airflow/issues/13785#issuecomment-774181236 nice i thought you could put it in extra but had trouble finding the spot in the code This is an automated messa

  1   2   3   4   5   >