[GitHub] [airflow] spektom opened a new pull request #5785: Add Azure Data Explorer (Kusto) operator

spektom opened a new pull request #5785: Add Azure Data Explorer (Kusto) operator URL: https://github.com/apache/airflow/pull/5785 Make sure you have checked _all_ steps below. ### Jira https://issues.apache.org/jira/browse/AIRFLOW-5176 ### Description This PR ad

[GitHub] [airflow] spektom closed pull request #5785: Add Azure Data Explorer (Kusto) operator

spektom closed pull request #5785: Add Azure Data Explorer (Kusto) operator URL: https://github.com/apache/airflow/pull/5785 This is an automated message from the Apache Git Service. To respond to the message, please log on t

[jira] [Created] (AIRFLOW-5182) "KubernetesOperator" isn't implemented

Esfahan created AIRFLOW-5182: Summary: "KubernetesOperator" isn't implemented Key: AIRFLOW-5182 URL: https://issues.apache.org/jira/browse/AIRFLOW-5182 Project: Apache Airflow Issue Type: Bug

[jira] [Updated] (AIRFLOW-5176) Add integration with Azure Data Explorer

[ https://issues.apache.org/jira/browse/AIRFLOW-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Spector updated AIRFLOW-5176: - Affects Version/s: 1.10.4 > Add integration with Azure Data Explorer >

[jira] [Created] (AIRFLOW-5181) Connection reset by peer during Dataproc job

Maulik Soneji created AIRFLOW-5181: -- Summary: Connection reset by peer during Dataproc job Key: AIRFLOW-5181 URL: https://issues.apache.org/jira/browse/AIRFLOW-5181 Project: Apache Airflow I

[GitHub] [airflow] mauliksoneji commented on issue #5781: [AIRFLOW-5165] make number of dataproc masters configurable

mauliksoneji commented on issue #5781: [AIRFLOW-5165] make number of dataproc masters configurable URL: https://github.com/apache/airflow/pull/5781#issuecomment-520692065 @mik-laj pylint is failing as the number of lines has exceeded, what do you suggest to do? ---

[GitHub] [airflow] mik-laj commented on a change in pull request #5546: [AIRFLOW-4908] BigQuery Hooks/Operators for update_dataset, patch_dataset, get_dataset

mik-laj commented on a change in pull request #5546: [AIRFLOW-4908] BigQuery Hooks/Operators for update_dataset, patch_dataset, get_dataset URL: https://github.com/apache/airflow/pull/5546#discussion_r313204322 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -17

[GitHub] [airflow] ryanyuan commented on a change in pull request #5546: [AIRFLOW-4908] BigQuery Hooks/Operators for update_dataset, patch_dataset, get_dataset

ryanyuan commented on a change in pull request #5546: [AIRFLOW-4908] BigQuery Hooks/Operators for update_dataset, patch_dataset, get_dataset URL: https://github.com/apache/airflow/pull/5546#discussion_r313200415 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -1

[GitHub] [airflow] mik-laj closed pull request #4619: [AIRFLOW- boyscout] Enforce delimiter for gcs_to_gcs operator using a flag, enforce_delimiter

mik-laj closed pull request #4619: [AIRFLOW- boyscout] Enforce delimiter for gcs_to_gcs operator using a flag, enforce_delimiter URL: https://github.com/apache/airflow/pull/4619 This is an automated message from the Apache G

[GitHub] [airflow] mik-laj commented on issue #4619: [AIRFLOW- boyscout] Enforce delimiter for gcs_to_gcs operator using a flag, enforce_delimiter

mik-laj commented on issue #4619: [AIRFLOW- boyscout] Enforce delimiter for gcs_to_gcs operator using a flag, enforce_delimiter URL: https://github.com/apache/airflow/pull/4619#issuecomment-520665056 It seems to me that this change is no longer applicable, so i close this PR. If I am wrong

[jira] [Closed] (AIRFLOW-5131) Create scopes property in GoogleCloudBaseHook

[ https://issues.apache.org/jira/browse/AIRFLOW-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula closed AIRFLOW-5131. -- Resolution: Fixed Fix Version/s: 1.10.5 > Create scopes property in GoogleCloudBaseHoo

[jira] [Commented] (AIRFLOW-5131) Create scopes property in GoogleCloudBaseHook

[ https://issues.apache.org/jira/browse/AIRFLOW-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905744#comment-16905744 ] ASF subversion and git services commented on AIRFLOW-5131: -- Co

[jira] [Commented] (AIRFLOW-5131) Create scopes property in GoogleCloudBaseHook

[ https://issues.apache.org/jira/browse/AIRFLOW-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905743#comment-16905743 ] ASF GitHub Bot commented on AIRFLOW-5131: - mik-laj commented on pull request #5

[GitHub] [airflow] mik-laj merged pull request #5745: [AIRFLOW-5131] Create scopes property in GoogleCloudBaseHook

mik-laj merged pull request #5745: [AIRFLOW-5131] Create scopes property in GoogleCloudBaseHook URL: https://github.com/apache/airflow/pull/5745 This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [airflow] mik-laj commented on a change in pull request #5602: [AIRFLOW-4983] Feature/dataflow operator python3

mik-laj commented on a change in pull request #5602: [AIRFLOW-4983] Feature/dataflow operator python3 URL: https://github.com/apache/airflow/pull/5602#discussion_r313198109 ## File path: airflow/contrib/hooks/gcp_dataflow_hook.py ## @@ -404,6 +405,7 @@ def start_python_dat

[GitHub] [airflow] mik-laj commented on a change in pull request #5602: [AIRFLOW-4983] Feature/dataflow operator python3

mik-laj commented on a change in pull request #5602: [AIRFLOW-4983] Feature/dataflow operator python3 URL: https://github.com/apache/airflow/pull/5602#discussion_r313197835 ## File path: airflow/contrib/hooks/gcp_dataflow_hook.py ## @@ -417,17 +419,26 @@ def start_python_d

[jira] [Closed] (AIRFLOW-4686) Make dags Pylint compatible

[ https://issues.apache.org/jira/browse/AIRFLOW-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula closed AIRFLOW-4686. -- Resolution: Fixed Fix Version/s: 1.10.5 > Make dags Pylint compatible > --

[jira] [Closed] (AIRFLOW-5104) "Schedule was not specified." error from GCS API when using gcp_transfer_operator.S3ToGoogleCloudStorageTransferOperator without specifying a schedule

[ https://issues.apache.org/jira/browse/AIRFLOW-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula closed AIRFLOW-5104. -- Resolution: Fixed Fix Version/s: 1.10.5 > "Schedule was not specified." error from GCS

[jira] [Commented] (AIRFLOW-5104) "Schedule was not specified." error from GCS API when using gcp_transfer_operator.S3ToGoogleCloudStorageTransferOperator without specifying a schedule

[ https://issues.apache.org/jira/browse/AIRFLOW-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905733#comment-16905733 ] ASF subversion and git services commented on AIRFLOW-5104: -- Co

[jira] [Commented] (AIRFLOW-5104) "Schedule was not specified." error from GCS API when using gcp_transfer_operator.S3ToGoogleCloudStorageTransferOperator without specifying a schedule

[ https://issues.apache.org/jira/browse/AIRFLOW-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905732#comment-16905732 ] ASF GitHub Bot commented on AIRFLOW-5104: - mik-laj commented on pull request #5

[GitHub] [airflow] mik-laj merged pull request #5726: [AIRFLOW-5104] Set default schedule for GCP Transfer operators

mik-laj merged pull request #5726: [AIRFLOW-5104] Set default schedule for GCP Transfer operators URL: https://github.com/apache/airflow/pull/5726 This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [airflow] mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313195761 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -1317,7 +1316,7

[GitHub] [airflow] mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313195696 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -969,11 +968,11

[GitHub] [airflow] mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313195778 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -1468,10 +1467,

[GitHub] [airflow] mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313195713 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -1118,11 +1117,

[GitHub] [airflow] mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313195754 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -1308,7 +1307,7

[GitHub] [airflow] mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313195741 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -1253,7 +1252,7

[GitHub] [airflow] mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313195673 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -857,14 +856,14

[GitHub] [airflow] mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313195794 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -1468,10 +1467,

[GitHub] [airflow] mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313195682 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -957,9 +956,9 @

[GitHub] [airflow] mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313195727 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -1118,11 +1117,

[GitHub] [airflow] mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313195658 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -844,8 +843,8 @

[GitHub] [airflow] mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313195642 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -492,7 +492,6 @

[GitHub] [airflow] mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313195423 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -1718,6 +1717,7

[GitHub] [airflow] mik-laj commented on a change in pull request #5565: [AIRFLOW-4899] Fix get_dataset_list from bigquery hook to return next…

mik-laj commented on a change in pull request #5565: [AIRFLOW-4899] Fix get_dataset_list from bigquery hook to return next… URL: https://github.com/apache/airflow/pull/5565#discussion_r313194987 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -1707,14 +1711,30 @

[GitHub] [airflow] mik-laj commented on a change in pull request #5546: [AIRFLOW-4908] BigQuery Hooks/Operators for update_dataset, patch_dataset, get_dataset

mik-laj commented on a change in pull request #5546: [AIRFLOW-4908] BigQuery Hooks/Operators for update_dataset, patch_dataset, get_dataset URL: https://github.com/apache/airflow/pull/5546#discussion_r313193597 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -17

[GitHub] [airflow] mik-laj commented on issue #5546: [AIRFLOW-4908] BigQuery Hooks/Operators for update_dataset, patch_dataset, get_dataset

mik-laj commented on issue #5546: [AIRFLOW-4908] BigQuery Hooks/Operators for update_dataset, patch_dataset, get_dataset URL: https://github.com/apache/airflow/pull/5546#issuecomment-520657003 Are you planning to continue working on this change? This week, I would like to deal with PR revi

[GitHub] [airflow] mik-laj commented on a change in pull request #5546: [AIRFLOW-4908] BigQuery Hooks/Operators for update_dataset, patch_dataset, get_dataset

mik-laj commented on a change in pull request #5546: [AIRFLOW-4908] BigQuery Hooks/Operators for update_dataset, patch_dataset, get_dataset URL: https://github.com/apache/airflow/pull/5546#discussion_r313193597 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -17

[GitHub] [airflow] mik-laj commented on issue #5753: [AIRFLOW-4686] Make dags Pylint compatible

mik-laj commented on issue #5753: [AIRFLOW-4686] Make dags Pylint compatible URL: https://github.com/apache/airflow/pull/5753#issuecomment-520653293 @feluelle Thanks. Great work. Fixing pylint errors is very important. This will allow for faster development of the project.

[jira] [Commented] (AIRFLOW-4686) Make dags Pylint compatible

[ https://issues.apache.org/jira/browse/AIRFLOW-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905716#comment-16905716 ] ASF subversion and git services commented on AIRFLOW-4686: -- Co

[GitHub] [airflow] mik-laj merged pull request #5753: [AIRFLOW-4686] Make dags Pylint compatible

mik-laj merged pull request #5753: [AIRFLOW-4686] Make dags Pylint compatible URL: https://github.com/apache/airflow/pull/5753 This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Commented] (AIRFLOW-4686) Make dags Pylint compatible

[ https://issues.apache.org/jira/browse/AIRFLOW-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905715#comment-16905715 ] ASF GitHub Bot commented on AIRFLOW-4686: - mik-laj commented on pull request #5

[GitHub] [airflow] lsowen commented on issue #5754: handle elasticsearch `json_format` as a boolean

lsowen commented on issue #5754: handle elasticsearch `json_format` as a boolean URL: https://github.com/apache/airflow/pull/5754#issuecomment-520647430 Do I need to do anything else to get this merged? Or is it good? (just waiting on flaky test?)

[jira] [Commented] (AIRFLOW-5180) Add static checks for Yaml files (yamllint + LICENCE check)

[ https://issues.apache.org/jira/browse/AIRFLOW-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905700#comment-16905700 ] ASF GitHub Bot commented on AIRFLOW-5180: - potiuk commented on pull request #57

[GitHub] [airflow] potiuk opened a new pull request #5790: [AIRFLOW-5180] Added static checks (yamllint) + auto-licences for yaml (depends on AIRFLOW-5161, AIRFLOW-5170)

potiuk opened a new pull request #5790: [AIRFLOW-5180] Added static checks (yamllint) + auto-licences for yaml (depends on AIRFLOW-5161, AIRFLOW-5170) URL: https://github.com/apache/airflow/pull/5790 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses

[jira] [Commented] (AIRFLOW-4222) Add cli autocomplete

[ https://issues.apache.org/jira/browse/AIRFLOW-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905673#comment-16905673 ] ASF GitHub Bot commented on AIRFLOW-4222: - kaxil commented on pull request #578

[GitHub] [airflow] kaxil opened a new pull request #5789: [AIRFLOW-4222] Add cli autocomplete for bash & zsh

kaxil opened a new pull request #5789: [AIRFLOW-4222] Add cli autocomplete for bash & zsh URL: https://github.com/apache/airflow/pull/5789 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/

[jira] [Assigned] (AIRFLOW-4222) Add cli autocomplete

[ https://issues.apache.org/jira/browse/AIRFLOW-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik reassigned AIRFLOW-4222: --- Assignee: Kaxil Naik (was: zhongjiajie) > Add cli autocomplete > > >

[jira] [Created] (AIRFLOW-5180) Add static checks for Yaml files (yamllint + LICENCE check)

Jarek Potiuk created AIRFLOW-5180: - Summary: Add static checks for Yaml files (yamllint + LICENCE check) Key: AIRFLOW-5180 URL: https://issues.apache.org/jira/browse/AIRFLOW-5180 Project: Apache Airfl

[GitHub] [airflow] benjamingrenier commented on issue #5565: [AIRFLOW-4899] Fix get_dataset_list from bigquery hook to return next…

benjamingrenier commented on issue #5565: [AIRFLOW-4899] Fix get_dataset_list from bigquery hook to return next… URL: https://github.com/apache/airflow/pull/5565#issuecomment-520626905 > > Travis is sad > > ``` > > ==

[GitHub] [airflow] milton0825 commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability

milton0825 commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability URL: https://github.com/apache/airflow/pull/5743#discussion_r313163517 ## File path: airflow/dag/serialization/serialized_baseoperator.py ###

[GitHub] [airflow] benjamingrenier commented on issue #5565: [AIRFLOW-4899] Fix get_dataset_list from bigquery hook to return next…

benjamingrenier commented on issue #5565: [AIRFLOW-4899] Fix get_dataset_list from bigquery hook to return next… URL: https://github.com/apache/airflow/pull/5565#issuecomment-520623740 > Travis is sad > > ``` > ==

[jira] [Created] (AIRFLOW-5179) Top level __init__.py breaks imports

Cedrik Neumann created AIRFLOW-5179: --- Summary: Top level __init__.py breaks imports Key: AIRFLOW-5179 URL: https://issues.apache.org/jira/browse/AIRFLOW-5179 Project: Apache Airflow Issue T

[jira] [Assigned] (AIRFLOW-5118) Airflow DataprocClusterCreateOperator does not currently support setting optional components

[ https://issues.apache.org/jira/browse/AIRFLOW-5118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik reassigned AIRFLOW-5118: --- Assignee: Kaxil Naik > Airflow DataprocClusterCreateOperator does not currently support se

[jira] [Updated] (AIRFLOW-5178) AWS Batch Operator improvement to support batch job parameters

[ https://issues.apache.org/jira/browse/AIRFLOW-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Mottershead updated AIRFLOW-5178: - Description: AWSBatchOperator does not currently support AWS Batch Job parameters. Whe

[jira] [Created] (AIRFLOW-5178) AWS Batch Operator improvement to support batch job parameters

Tim Mottershead created AIRFLOW-5178: Summary: AWS Batch Operator improvement to support batch job parameters Key: AIRFLOW-5178 URL: https://issues.apache.org/jira/browse/AIRFLOW-5178 Project: Apa

[GitHub] [airflow] feluelle commented on a change in pull request #5672: [AIRFLOW-5056] Add argument to filter mails in ImapHook and related operators

feluelle commented on a change in pull request #5672: [AIRFLOW-5056] Add argument to filter mails in ImapHook and related operators URL: https://github.com/apache/airflow/pull/5672#discussion_r313107767 ## File path: airflow/contrib/hooks/imap_hook.py ## @@ -30,72 +34,105

[GitHub] [airflow] feluelle commented on a change in pull request #5672: [AIRFLOW-5056] Add argument to filter mails in ImapHook and related operators

feluelle commented on a change in pull request #5672: [AIRFLOW-5056] Add argument to filter mails in ImapHook and related operators URL: https://github.com/apache/airflow/pull/5672#discussion_r313107767 ## File path: airflow/contrib/hooks/imap_hook.py ## @@ -30,72 +34,105

[GitHub] [airflow] mik-laj commented on issue #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

mik-laj commented on issue #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#issuecomment-520574126 Recently introduced a new development environment. have you tried them? we plan to create a new extension to

[GitHub] [airflow] mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

mik-laj commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313102815 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -1718,6 +1718,7

[jira] [Created] (AIRFLOW-5177) scheduler_job command passes scheduler dag location

James Coder created AIRFLOW-5177: Summary: scheduler_job command passes scheduler dag location Key: AIRFLOW-5177 URL: https://issues.apache.org/jira/browse/AIRFLOW-5177 Project: Apache Airflow

[GitHub] [airflow] benjamingrenier commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

benjamingrenier commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313096535 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -1718,6

[GitHub] [airflow] benjamingrenier commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

benjamingrenier commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313096351 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -1718,6

[GitHub] [airflow] benjamingrenier commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

benjamingrenier commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313096161 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -1718,6

[GitHub] [airflow] benjamingrenier commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

benjamingrenier commented on a change in pull request #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#discussion_r313096077 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -1718,6

[GitHub] [airflow] benjamingrenier edited a comment on issue #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

benjamingrenier edited a comment on issue #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#issuecomment-520567183 Yes I will work on it this week. I try to resolve my docker errors to run unit-tests locally.

[GitHub] [airflow] benjamingrenier commented on issue #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

benjamingrenier commented on issue #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#issuecomment-520567183 Yes I will work on it this week. I try to resolve my mistakes to run unit-tests locally. ---

[GitHub] [airflow] benjamingrenier commented on issue #5565: [AIRFLOW-4899] Fix get_dataset_list from bigquery hook to return next…

benjamingrenier commented on issue #5565: [AIRFLOW-4899] Fix get_dataset_list from bigquery hook to return next… URL: https://github.com/apache/airflow/pull/5565#issuecomment-520566509 Yes I will work on this change soon, but I have some mistakes with my docker to run unit tests locally.

[GitHub] [airflow] kaxil commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability

kaxil commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability URL: https://github.com/apache/airflow/pull/5743#discussion_r313093773 ## File path: airflow/models/serialized_dag.py ## @@ -0,0 +1,155 @@ +#

[GitHub] [airflow] feluelle commented on a change in pull request #5753: [AIRFLOW-4686] Make dags Pylint compatible

feluelle commented on a change in pull request #5753: [AIRFLOW-4686] Make dags Pylint compatible URL: https://github.com/apache/airflow/pull/5753#discussion_r313086546 ## File path: airflow/contrib/example_dags/example_emr_job_flow_manual_steps.py ## @@ -16,7 +16,14 @@ #

[GitHub] [airflow] TV4Fun commented on a change in pull request #5726: [AIRFLOW-5104] Set default schedule for GCP Transfer operators

TV4Fun commented on a change in pull request #5726: [AIRFLOW-5104] Set default schedule for GCP Transfer operators URL: https://github.com/apache/airflow/pull/5726#discussion_r313060643 ## File path: airflow/contrib/operators/gcp_transfer_operator.py ## @@ -97,7 +99,13 @@

[GitHub] [airflow] TV4Fun commented on a change in pull request #5727: [AIRFLOW-5114] Fix gcp_transfer_hook behavior with default operator arguments

TV4Fun commented on a change in pull request #5727: [AIRFLOW-5114] Fix gcp_transfer_hook behavior with default operator arguments URL: https://github.com/apache/airflow/pull/5727#discussion_r313057997 ## File path: airflow/contrib/hooks/gcp_transfer_hook.py ## @@ -370,7 +3

[GitHub] [airflow] mik-laj commented on a change in pull request #5753: [AIRFLOW-4686] Make dags Pylint compatible

mik-laj commented on a change in pull request #5753: [AIRFLOW-4686] Make dags Pylint compatible URL: https://github.com/apache/airflow/pull/5753#discussion_r313057386 ## File path: airflow/contrib/example_dags/example_emr_job_flow_manual_steps.py ## @@ -16,7 +16,14 @@ # K

[GitHub] [airflow] mik-laj commented on a change in pull request #5753: [AIRFLOW-4686] Make dags Pylint compatible

mik-laj commented on a change in pull request #5753: [AIRFLOW-4686] Make dags Pylint compatible URL: https://github.com/apache/airflow/pull/5753#discussion_r313055834 ## File path: airflow/contrib/example_dags/example_emr_job_flow_manual_steps.py ## @@ -16,7 +16,14 @@ # K

[GitHub] [airflow] feluelle commented on a change in pull request #5753: [AIRFLOW-4686] Make dags Pylint compatible

feluelle commented on a change in pull request #5753: [AIRFLOW-4686] Make dags Pylint compatible URL: https://github.com/apache/airflow/pull/5753#discussion_r313051117 ## File path: airflow/contrib/example_dags/example_emr_job_flow_manual_steps.py ## @@ -16,7 +16,14 @@ #

[GitHub] [airflow] kaxil commented on a change in pull request #5701: [AIRFLOW-5088][AIP-24] Add DAG serialization using JSON

kaxil commented on a change in pull request #5701: [AIRFLOW-5088][AIP-24] Add DAG serialization using JSON URL: https://github.com/apache/airflow/pull/5701#discussion_r313047066 ## File path: airflow/dag/serialization/serialized_baseoperator.py ## @@ -0,0 +1,76 @@ +# -*- c

[GitHub] [airflow] kaxil commented on a change in pull request #5701: [AIRFLOW-5088][AIP-24] Add DAG serialization using JSON

kaxil commented on a change in pull request #5701: [AIRFLOW-5088][AIP-24] Add DAG serialization using JSON URL: https://github.com/apache/airflow/pull/5701#discussion_r313046906 ## File path: airflow/dag/serialization/serialization.py ## @@ -0,0 +1,221 @@ +# -*- coding: ut

[GitHub] [airflow] kaxil commented on a change in pull request #5701: [AIRFLOW-5088][AIP-24] Add DAG serialization using JSON

kaxil commented on a change in pull request #5701: [AIRFLOW-5088][AIP-24] Add DAG serialization using JSON URL: https://github.com/apache/airflow/pull/5701#discussion_r313046952 ## File path: airflow/dag/serialization/serialized_baseoperator.py ## @@ -0,0 +1,76 @@ +# -*- c

[GitHub] [airflow] kaxil commented on a change in pull request #5701: [AIRFLOW-5088][AIP-24] Add DAG serialization using JSON

kaxil commented on a change in pull request #5701: [AIRFLOW-5088][AIP-24] Add DAG serialization using JSON URL: https://github.com/apache/airflow/pull/5701#discussion_r313046854 ## File path: airflow/dag/serialization/json_schema.py ## @@ -0,0 +1,140 @@ +# -*- coding: utf-

[GitHub] [airflow] kaxil commented on a change in pull request #5701: [AIRFLOW-5088][AIP-24] Add DAG serialization using JSON

kaxil commented on a change in pull request #5701: [AIRFLOW-5088][AIP-24] Add DAG serialization using JSON URL: https://github.com/apache/airflow/pull/5701#discussion_r313046998 ## File path: airflow/dag/serialization/serialized_baseoperator.py ## @@ -0,0 +1,76 @@ +# -*- c

[GitHub] [airflow] kaxil commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability

kaxil commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability URL: https://github.com/apache/airflow/pull/5743#discussion_r313043384 ## File path: airflow/models/serialized_dag.py ## @@ -0,0 +1,155 @@ +#

[GitHub] [airflow] kaxil commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability

kaxil commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability URL: https://github.com/apache/airflow/pull/5743#discussion_r313043300 ## File path: airflow/models/serialized_dag.py ## @@ -0,0 +1,155 @@ +#

[GitHub] [airflow] kaxil commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability

kaxil commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability URL: https://github.com/apache/airflow/pull/5743#discussion_r313042993 ## File path: airflow/models/dagbag.py ## @@ -107,28 +112,51 @@ def si

[GitHub] [airflow] kaxil commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability

kaxil commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability URL: https://github.com/apache/airflow/pull/5743#discussion_r313043065 ## File path: airflow/models/dagbag.py ## @@ -416,6 +449,27 @@ def col

[GitHub] [airflow] kaxil commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability

kaxil commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability URL: https://github.com/apache/airflow/pull/5743#discussion_r313043027 ## File path: airflow/models/dagbag.py ## @@ -107,28 +112,51 @@ def si

[GitHub] [airflow] kaxil commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability

kaxil commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability URL: https://github.com/apache/airflow/pull/5743#discussion_r313042590 ## File path: airflow/utils/log/logging_mixin.py ## @@ -46,9 +46,10 @@

[GitHub] [airflow] mik-laj commented on a change in pull request #5770: [AIRFLOW-5162] GCS Hook Upload Method Improvement

mik-laj commented on a change in pull request #5770: [AIRFLOW-5162] GCS Hook Upload Method Improvement URL: https://github.com/apache/airflow/pull/5770#discussion_r313042343 ## File path: airflow/contrib/hooks/gcs_hook.py ## @@ -175,40 +175,55 @@ def download(self, bucket_

[GitHub] [airflow] dimberman opened a new pull request #5788: proof-of-concept multi-threading using asyncio

dimberman opened a new pull request #5788: proof-of-concept multi-threading using asyncio URL: https://github.com/apache/airflow/pull/5788 Make sure you have checked _all_ steps below. This is a POC showing we can use the asyncio library for coroutines in the LocalExecutor. This mig

[GitHub] [airflow] coufon commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability

coufon commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability URL: https://github.com/apache/airflow/pull/5743#discussion_r313038646 ## File path: airflow/utils/log/logging_mixin.py ## @@ -46,9 +46,10 @

[GitHub] [airflow] coufon commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability

coufon commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability URL: https://github.com/apache/airflow/pull/5743#discussion_r313038646 ## File path: airflow/utils/log/logging_mixin.py ## @@ -46,9 +46,10 @

[GitHub] [airflow] coufon commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability

coufon commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability URL: https://github.com/apache/airflow/pull/5743#discussion_r313039084 ## File path: airflow/www/views.py ## @@ -1328,7 +1331,7 @@ def set_d

[GitHub] [airflow] houqp commented on a change in pull request #5731: [AIRFLOW-5117] support refreshing EKS api tokens

houqp commented on a change in pull request #5731: [AIRFLOW-5117] support refreshing EKS api tokens URL: https://github.com/apache/airflow/pull/5731#discussion_r313038378 ## File path: scripts/ci/pylint_todo.txt ## @@ -201,6 +201,7 @@ ./airflow/kubernetes/kubernetes_reque

[jira] [Updated] (AIRFLOW-3278) GZIP support for Operators loading to GCS

[ https://issues.apache.org/jira/browse/AIRFLOW-3278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula updated AIRFLOW-3278: --- Description: gzip flag was added to GoogleCloudStorageHook by PR: [https://issues.apache.o

[jira] [Resolved] (AIRFLOW-3278) GZIP support for Operators loading to GCS

[ https://issues.apache.org/jira/browse/AIRFLOW-3278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula resolved AIRFLOW-3278. Resolution: Fixed Fix Version/s: 1.10.5 All operators have support for GZIP now.

[jira] [Updated] (AIRFLOW-3278) GZIP support for Operators loading to GCS

[ https://issues.apache.org/jira/browse/AIRFLOW-3278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula updated AIRFLOW-3278: --- Description: gzip flag was added to GoogleCloudStorageHook by PR: [https://issues.apache.o

[jira] [Updated] (AIRFLOW-3278) GZIP support for Operators loading to GCS

[ https://issues.apache.org/jira/browse/AIRFLOW-3278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula updated AIRFLOW-3278: --- Description: gzip flag was added to GoogleCloudStorageHook by PR: [https://issues.apache.o

[jira] [Updated] (AIRFLOW-3278) GZIP support for Operators loading to GCS

[ https://issues.apache.org/jira/browse/AIRFLOW-3278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula updated AIRFLOW-3278: --- Description: gzip flag was added to GoogleCloudStorageHook by PR: [https://issues.apache.o

[jira] [Resolved] (AIRFLOW-5124) Gzip support for S3ToGoogleCloudStorageOperator

[ https://issues.apache.org/jira/browse/AIRFLOW-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula resolved AIRFLOW-5124. Resolution: Fixed Fix Version/s: 1.10.5 > Gzip support for S3ToGoogleCloudStorageO

[jira] [Commented] (AIRFLOW-5124) Gzip support for S3ToGoogleCloudStorageOperator

[ https://issues.apache.org/jira/browse/AIRFLOW-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905400#comment-16905400 ] ASF subversion and git services commented on AIRFLOW-5124: -- Co

[jira] [Commented] (AIRFLOW-5124) Gzip support for S3ToGoogleCloudStorageOperator

[ https://issues.apache.org/jira/browse/AIRFLOW-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905399#comment-16905399 ] ASF GitHub Bot commented on AIRFLOW-5124: - mik-laj commented on pull request #5

  1   2   3   >