[jira] [Resolved] (AIRFLOW-2300) Add S3 Select functionarity to S3ToHiveTransfer

[ https://issues.apache.org/jira/browse/AIRFLOW-2300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-2300. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request #3243 [

[jira] [Commented] (AIRFLOW-2300) Add S3 Select functionarity to S3ToHiveTransfer

[ https://issues.apache.org/jira/browse/AIRFLOW-2300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447644#comment-16447644 ] ASF subversion and git services commented on AIRFLOW-2300: -- Com

[jira] [Created] (AIRFLOW-2358) make kubernetes examples installed optionally

Ruslan Dautkhanov created AIRFLOW-2358: -- Summary: make kubernetes examples installed optionally Key: AIRFLOW-2358 URL: https://issues.apache.org/jira/browse/AIRFLOW-2358 Project: Apache Airflow

[jira] [Commented] (AIRFLOW-2300) Add S3 Select functionarity to S3ToHiveTransfer

[ https://issues.apache.org/jira/browse/AIRFLOW-2300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447645#comment-16447645 ] ASF subversion and git services commented on AIRFLOW-2300: -- Com

incubator-airflow git commit: [AIRFLOW-2300] Add S3 Select functionarity to S3ToHiveTransfer

Repository: incubator-airflow Updated Branches: refs/heads/master a15b7c5b7 -> 49826af10 [AIRFLOW-2300] Add S3 Select functionarity to S3ToHiveTransfer To improve efficiency and usability, this PR adds S3 Select functionarity to S3ToHiveTransfer. It also contains some minor fixes for documents

[jira] [Updated] (AIRFLOW-2351) timezone fix for @once DAGs

[ https://issues.apache.org/jira/browse/AIRFLOW-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruslan Dautkhanov updated AIRFLOW-2351: --- Summary: timezone fix for @once DAGs (was: timezone fix for @onca DAGs) > timezone

[jira] [Updated] (AIRFLOW-2353) Separate kerberos config per impersonated UNIX user

[ https://issues.apache.org/jira/browse/AIRFLOW-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Slawomir Krysiak updated AIRFLOW-2353: -- Description: Would be useful to be able to use different Kerberos principals per an i

[jira] [Updated] (AIRFLOW-2353) Separate kerberos config per impersonated UNIX user

[ https://issues.apache.org/jira/browse/AIRFLOW-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Slawomir Krysiak updated AIRFLOW-2353: -- Summary: Separate kerberos config per impersonated UNIX user (was: Separate kerberos

[jira] [Updated] (AIRFLOW-2352) Airflow isn't picking up earlier periods after DAG definition update

[ https://issues.apache.org/jira/browse/AIRFLOW-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Slawomir Krysiak updated AIRFLOW-2352: -- Description: Hi,   It would be nice to be able to modify the period range (a.k.a sta

[jira] [Updated] (AIRFLOW-2352) Airflow isn't picking up earlier periods after DAG definition update

[ https://issues.apache.org/jira/browse/AIRFLOW-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Slawomir Krysiak updated AIRFLOW-2352: -- Description: Hi,   It would be nice to be able to modify the period range (a.k.a sta

[jira] [Updated] (AIRFLOW-2352) Airflow isn't picking up earlier periods after DAG definition update

[ https://issues.apache.org/jira/browse/AIRFLOW-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Slawomir Krysiak updated AIRFLOW-2352: -- Description: Hi,   It would be nice to be able to modify the period range (a.k.a sta

[jira] [Created] (AIRFLOW-2357) Use a persisten volume for the Kubernetes logs

Fokko Driesprong created AIRFLOW-2357: - Summary: Use a persisten volume for the Kubernetes logs Key: AIRFLOW-2357 URL: https://issues.apache.org/jira/browse/AIRFLOW-2357 Project: Apache Airflow

[jira] [Commented] (AIRFLOW-1314) Airflow kubernetes integration

[ https://issues.apache.org/jira/browse/AIRFLOW-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447151#comment-16447151 ] ASF subversion and git services commented on AIRFLOW-1314: -- Com

[jira] [Commented] (AIRFLOW-1314) Airflow kubernetes integration

[ https://issues.apache.org/jira/browse/AIRFLOW-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447156#comment-16447156 ] ASF subversion and git services commented on AIRFLOW-1314: -- Com

[jira] [Commented] (AIRFLOW-1314) Airflow kubernetes integration

[ https://issues.apache.org/jira/browse/AIRFLOW-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447149#comment-16447149 ] ASF subversion and git services commented on AIRFLOW-1314: -- Com

[jira] [Commented] (AIRFLOW-1314) Airflow kubernetes integration

[ https://issues.apache.org/jira/browse/AIRFLOW-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447155#comment-16447155 ] ASF subversion and git services commented on AIRFLOW-1314: -- Com

[jira] [Commented] (AIRFLOW-1314) Airflow kubernetes integration

[ https://issues.apache.org/jira/browse/AIRFLOW-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447148#comment-16447148 ] ASF subversion and git services commented on AIRFLOW-1314: -- Com

[jira] [Commented] (AIRFLOW-1314) Airflow kubernetes integration

[ https://issues.apache.org/jira/browse/AIRFLOW-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447152#comment-16447152 ] ASF subversion and git services commented on AIRFLOW-1314: -- Com

[jira] [Commented] (AIRFLOW-1314) Airflow kubernetes integration

[ https://issues.apache.org/jira/browse/AIRFLOW-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447157#comment-16447157 ] ASF subversion and git services commented on AIRFLOW-1314: -- Com

[jira] [Commented] (AIRFLOW-1314) Airflow kubernetes integration

[ https://issues.apache.org/jira/browse/AIRFLOW-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447153#comment-16447153 ] ASF subversion and git services commented on AIRFLOW-1314: -- Com

[jira] [Commented] (AIRFLOW-1314) Airflow kubernetes integration

[ https://issues.apache.org/jira/browse/AIRFLOW-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447150#comment-16447150 ] ASF subversion and git services commented on AIRFLOW-1314: -- Com

[jira] [Commented] (AIRFLOW-1999) Service account integration for kubernetes executor

[ https://issues.apache.org/jira/browse/AIRFLOW-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447154#comment-16447154 ] ASF subversion and git services commented on AIRFLOW-1999: -- Com

[10/20] incubator-airflow git commit: [AIRFLOW-1314] Add executor_config and tests

[AIRFLOW-1314] Add executor_config and tests * Added in executor_config to the task_instance table and the base_operator table * Fix test; bump up number of examples * Fix up comments from PR * Exclude the kubernetes example dag from a test * Fix dict -> KubernetesExecutorConfig * fixed up e

[09/20] incubator-airflow git commit: [AIRFLOW-1314] Use VolumeClaim for transporting DAGs

[AIRFLOW-1314] Use VolumeClaim for transporting DAGs - fix issue where watcher process randomly dies - fixed alembic head, was pointing to two tips Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/a9d90

[18/20] incubator-airflow git commit: [AIRFLOW-1999] Add per-task GCP service account support

[AIRFLOW-1999] Add per-task GCP service account support Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/cdb43cb8 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/cdb43cb8 Diff: http:/

[14/20] incubator-airflow git commit: [AIRFLOW-1314] Rebasing against master

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/b9a87a07/airflow/contrib/operators/k8s_pod_operator/__init__.py -- diff --git a/airflow/contrib/operators/k8s_pod_operator/__init__.py b/airflow/contrib/operators/k8s_p

[15/20] incubator-airflow git commit: [AIRFLOW-1314] Rebasing against master

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/b9a87a07/airflow/bin/cli/cli_factory.py -- diff --git a/airflow/bin/cli/cli_factory.py b/airflow/bin/cli/cli_factory.py deleted file mode 100644 index 167643b..000 -

[19/20] incubator-airflow git commit: [AIRFLOW-1314] Polish some of the Kubernetes docs/config

[AIRFLOW-1314] Polish some of the Kubernetes docs/config Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/d807830f Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/d807830f Diff: http:

[07/20] incubator-airflow git commit: [AIRFLOW=1314] Basic Kubernetes Mode

[AIRFLOW=1314] Basic Kubernetes Mode Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/58213208 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/58213208 Diff: http://git-wip-us.apache.

[01/20] incubator-airflow git commit: [AIRFLOW-1314] Create integration testing environment

Repository: incubator-airflow Updated Branches: refs/heads/master f520990fe -> a15b7c5b7 [AIRFLOW-1314] Create integration testing environment Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/29daa58

[17/20] incubator-airflow git commit: [AIRFLOW-1314] Rebasing against master

[AIRFLOW-1314] Rebasing against master Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/b9a87a07 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/b9a87a07 Diff: http://git-wip-us.apach

[03/20] incubator-airflow git commit: [AIRFLOW-1314] Add support for volume mounts & Secrets in Kubernetes Executor

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/c177d6e8/airflow/bin/cli/cli_factory.py -- diff --git a/airflow/bin/cli/cli_factory.py b/airflow/bin/cli/cli_factory.py new file mode 100644 index 000..46efcf7 --- /

[12/20] incubator-airflow git commit: [AIRFLOW-1314] Improve k8s support

[AIRFLOW-1314] Improve k8s support Add kubernetes config section in airflow.cfg and Inject GCP secrets upon executor start. (#17) Update Airflow to Pass configuration to k8s containers, add some Py3 … (#9) * Update Airflow to Pass configuration to k8s containers, add some Py3 compat., create

[16/20] incubator-airflow git commit: [AIRFLOW-1314] Rebasing against master

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/b9a87a07/airflow/bin/cli/cli.py -- diff --git a/airflow/bin/cli/cli.py b/airflow/bin/cli/cli.py deleted file mode 100644 index 8cf35b3..000 --- a/airflow/bin/cli/cli

[08/20] incubator-airflow git commit: [AIRFLOW-1314] Git Mode to pull in DAGs for Kubernetes Executor

[AIRFLOW-1314] Git Mode to pull in DAGs for Kubernetes Executor Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/bb1e05c3 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/bb1e05c3 Diff

[04/20] incubator-airflow git commit: [AIRFLOW-1314] Add support for volume mounts & Secrets in Kubernetes Executor

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/c177d6e8/airflow/bin/cli/cli.py -- diff --git a/airflow/bin/cli/cli.py b/airflow/bin/cli/cli.py new file mode 100644 index 000..8cf35b3 --- /dev/null +++ b/airflow/b

[02/20] incubator-airflow git commit: [AIRFLOW-1314] Add support for volume mounts & Secrets in Kubernetes Executor

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/c177d6e8/kube/postgres.yaml -- diff --git a/kube/postgres.yaml b/kube/postgres.yaml new file mode 100644 index 000..a7298d1 --- /dev/null +++ b/kube/postgres.yaml @@

[13/20] incubator-airflow git commit: [AIRFLOW-1314] Small cleanup to address PR comments (#24)

[AIRFLOW-1314] Small cleanup to address PR comments (#24) * Small cleanup to address PR comments * Remove use of enum * Change back to 3.4 Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/309f764a Tre

[06/20] incubator-airflow git commit: [AIRFLOW=1314] Basic Kubernetes Mode

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/58213208/tests/contrib/kubernetes/__init__.py -- diff --git a/tests/contrib/kubernetes/__init__.py b/tests/contrib/kubernetes/__init__.py new file mode 100644 index 000

[05/20] incubator-airflow git commit: [AIRFLOW-1314] Add support for volume mounts & Secrets in Kubernetes Executor

[AIRFLOW-1314] Add support for volume mounts & Secrets in Kubernetes Executor Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/c177d6e8 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree

[20/20] incubator-airflow git commit: [AIRFLOW-1314] Cleanup the config

[AIRFLOW-1314] Cleanup the config Closes #2414 from bloomberg:airflow-kubernetes-executor Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/a15b7c5b Tree: http://git-wip-us.apache.org/repos/asf/incubator

[11/20] incubator-airflow git commit: [AIRFLOW-1314] Improve error handling

[AIRFLOW-1314] Improve error handling Handle too old resource versions and throw exceptions on errors - K8s API errors will now throw Airflow exceptions - Add scheduler uuid to worker pod labels to match the two Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http