[jira] [Commented] (AIRFLOW-6242) Introduce run_type in DAG Run table

2020-04-26 Thread Will Hudgins (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17092962#comment-17092962 ] Will Hudgins commented on AIRFLOW-6242: --- [~kamil.bregula] Would I be correct in assuming this

[jira] [Commented] (AIRFLOW-6458) Deprecation warning about dropping support for Python 2

2020-04-26 Thread Will Hudgins (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17092961#comment-17092961 ] Will Hudgins commented on AIRFLOW-6458: --- [~kamil.bregula] If you don't mind, I can work on this

[GitHub] [airflow] haoliang7 opened a new pull request #8573: Fix dags test error if try_number>=1

2020-04-26 Thread GitBox
haoliang7 opened a new pull request #8573: URL: https://github.com/apache/airflow/pull/8573 Current code will raise pop key error if run `airflow dags test dag_id` which has ti try_number >= 1 --- Make sure to mark the boxes below before creating PR: [x] - [ ] Description

[GitHub] [airflow] mik-laj commented on a change in pull request #8534: fix: aws hook should work without conn id

2020-04-26 Thread GitBox
mik-laj commented on a change in pull request #8534: URL: https://github.com/apache/airflow/pull/8534#discussion_r415446445 ## File path: airflow/providers/amazon/aws/hooks/base_aws.py ## @@ -62,16 +63,14 @@ class AwsBaseHook(BaseHook): def __init__( self,

[GitHub] [airflow] mik-laj commented on a change in pull request #8533: Stop DockerSwarmOperator from pulling Docker images

2020-04-26 Thread GitBox
mik-laj commented on a change in pull request #8533: URL: https://github.com/apache/airflow/pull/8533#discussion_r415206235 ## File path: airflow/providers/docker/operators/docker.py ## @@ -289,6 +279,18 @@ def execute(self, context): return self._run_image() +

[GitHub] [airflow] turbaszek commented on a change in pull request #8534: fix: aws hook should work without conn id

2020-04-26 Thread GitBox
turbaszek commented on a change in pull request #8534: URL: https://github.com/apache/airflow/pull/8534#discussion_r415427532 ## File path: airflow/providers/amazon/aws/hooks/base_aws.py ## @@ -62,16 +63,14 @@ class AwsBaseHook(BaseHook): def __init__(

[GitHub] [airflow] casassg commented on issue #8052: [AIP-31] Create XComArg model

2020-04-26 Thread GitBox
casassg commented on issue #8052: URL: https://github.com/apache/airflow/issues/8052#issuecomment-619635122 Sounds good. Regarding Jinja, maybe a way is to make it `xcom_arg.jinja` property instead of making it under __str__. But it's mostly a nitpick, so fine by me. If we keep

[GitHub] [airflow] casassg commented on issue #8058: [AIP-31] Add lazy way to get Airflow context from a wrapped function

2020-04-26 Thread GitBox
casassg commented on issue #8058: URL: https://github.com/apache/airflow/issues/8058#issuecomment-619634745 Saw it on the PR, lgtm as well on making it a function  This is an automated message from the Apache Git Service.

[GitHub] [airflow] ImadYIdrissi commented on issue #8541: "TypeError: can't pickle _thread.RLock objects" on usage of BigQueryOperator

2020-04-26 Thread GitBox
ImadYIdrissi commented on issue #8541: URL: https://github.com/apache/airflow/issues/8541#issuecomment-619627785 Yeah, it turns out I forgot to upload the SQL SCRIPT that the `BigQueryOperator` uses... I believe there should be a clear `FileNotFound` exception instead of a UI crash.

[GitHub] [airflow] oripwk opened a new pull request #8572: Improve template capabilities of EMR job and step operators

2020-04-26 Thread GitBox
oripwk opened a new pull request #8572: URL: https://github.com/apache/airflow/pull/8572 Allow `EmrCreateJobFlowOperator` and `EmrAddStepsOperator` to receive their `job_flow_overrides`, and `steps` arguments respectively as Jinja template filenames. This is similar to `BashOperator`'s

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #8572: Improve template capabilities of EMR job and step operators

2020-04-26 Thread GitBox
boring-cyborg[bot] commented on pull request #8572: URL: https://github.com/apache/airflow/pull/8572#issuecomment-619625273 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[jira] [Commented] (AIRFLOW-4438) Add Gzip compression to S3_hook

2020-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17092861#comment-17092861 ] ASF GitHub Bot commented on AIRFLOW-4438: - OmairK opened a new pull request #8571: URL:

[GitHub] [airflow] OmairK opened a new pull request #8571: [AIRFLOW-4438] Add Gzip compression to S3_hook

2020-04-26 Thread GitBox
OmairK opened a new pull request #8571: URL: https://github.com/apache/airflow/pull/8571 Fixes #7680 --- Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Unit tests coverage for changes (not needed for

[GitHub] [airflow] houqp commented on a change in pull request #8534: fix: aws hook should work without conn id

2020-04-26 Thread GitBox
houqp commented on a change in pull request #8534: URL: https://github.com/apache/airflow/pull/8534#discussion_r415375821 ## File path: airflow/providers/amazon/aws/hooks/base_aws.py ## @@ -62,16 +63,14 @@ class AwsBaseHook(BaseHook): def __init__( self, -

[GitHub] [airflow] turbaszek commented on pull request #8559: Monkey patch greenlet celery pools

2020-04-26 Thread GitBox
turbaszek commented on pull request #8559: URL: https://github.com/apache/airflow/pull/8559#issuecomment-619584216 Thanks @aamangeldi !  This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #8559: Monkey patch greenlet celery pools

2020-04-26 Thread GitBox
boring-cyborg[bot] commented on pull request #8559: URL: https://github.com/apache/airflow/pull/8559#issuecomment-619583437 Awesome work, congrats on your first merged pull request! This is an automated message from the

[GitHub] [airflow] rajasaur opened a new issue #8570: Marking Jobs as Failed doesnt consider tasks that are 'up_for_reschedule'

2020-04-26 Thread GitBox
rajasaur opened a new issue #8570: URL: https://github.com/apache/airflow/issues/8570 **Apache Airflow version**: 1.10.10 **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): No K8s **Environment**: Mac OSX , Python 2.7 with

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8570: Marking Jobs as Failed doesnt consider tasks that are 'up_for_reschedule'

2020-04-26 Thread GitBox
boring-cyborg[bot] commented on issue #8570: URL: https://github.com/apache/airflow/issues/8570#issuecomment-619583218 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] potiuk opened a new pull request #8569: File should be specified in docker compose before command

2020-04-26 Thread GitBox
potiuk opened a new pull request #8569: URL: https://github.com/apache/airflow/pull/8569 --- Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Unit tests coverage for changes (not needed for documentation

[jira] [Commented] (AIRFLOW-6062) Scheduler doesn't delete worker pods from namespaces different than the scheduler's namespace

2020-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17092772#comment-17092772 ] ASF GitHub Bot commented on AIRFLOW-6062: - boring-cyborg[bot] commented on pull request #8546:

[GitHub] [airflow] dimberman commented on pull request #8546: AIRFLOW-6062 Watch worker pods from all namespaces

2020-04-26 Thread GitBox
dimberman commented on pull request #8546: URL: https://github.com/apache/airflow/pull/8546#issuecomment-619579162 Thank you for adding this @mppetkov! This is an automated message from the Apache Git Service. To respond to

[jira] [Commented] (AIRFLOW-6062) Scheduler doesn't delete worker pods from namespaces different than the scheduler's namespace

2020-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17092773#comment-17092773 ] ASF GitHub Bot commented on AIRFLOW-6062: - dimberman commented on pull request #8546: URL:

[jira] [Commented] (AIRFLOW-6062) Scheduler doesn't delete worker pods from namespaces different than the scheduler's namespace

2020-04-26 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17092774#comment-17092774 ] ASF subversion and git services commented on AIRFLOW-6062: -- Commit

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #8546: AIRFLOW-6062 Watch worker pods from all namespaces

2020-04-26 Thread GitBox
boring-cyborg[bot] commented on pull request #8546: URL: https://github.com/apache/airflow/pull/8546#issuecomment-619579130 Awesome work, congrats on your first merged pull request! This is an automated message from the

[jira] [Commented] (AIRFLOW-5850) Make DockerSwarmOperator capture task logs

2020-04-26 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17092766#comment-17092766 ] ASF subversion and git services commented on AIRFLOW-5850: -- Commit

[jira] [Commented] (AIRFLOW-5850) Make DockerSwarmOperator capture task logs

2020-04-26 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17092767#comment-17092767 ] ASF subversion and git services commented on AIRFLOW-5850: -- Commit

[jira] [Commented] (AIRFLOW-5850) Make DockerSwarmOperator capture task logs

2020-04-26 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17092769#comment-17092769 ] ASF subversion and git services commented on AIRFLOW-5850: -- Commit

[jira] [Commented] (AIRFLOW-5850) Make DockerSwarmOperator capture task logs

2020-04-26 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17092768#comment-17092768 ] ASF subversion and git services commented on AIRFLOW-5850: -- Commit

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #8430: Improve idempodency in CloudDataTransferServiceCreateJobOperator

2020-04-26 Thread GitBox
boring-cyborg[bot] commented on pull request #8430: URL: https://github.com/apache/airflow/pull/8430#issuecomment-619576242 Awesome work, congrats on your first merged pull request! This is an automated message from the

[GitHub] [airflow] turbaszek commented on pull request #8430: Improve idempodency in CloudDataTransferServiceCreateJobOperator

2020-04-26 Thread GitBox
turbaszek commented on pull request #8430: URL: https://github.com/apache/airflow/pull/8430#issuecomment-619576283 Thanks @khyurri ! This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] khyurri commented on pull request #8430: Improve idempodency in CloudDataTransferServiceCreateJobOperator

2020-04-26 Thread GitBox
khyurri commented on pull request #8430: URL: https://github.com/apache/airflow/pull/8430#issuecomment-619575041 @turbaszek yes. I can retest, if needed This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] potiuk commented on pull request #8566: Add test_localtaskjob_maintain_heart_rate to quarantine

2020-04-26 Thread GitBox
potiuk commented on pull request #8566: URL: https://github.com/apache/airflow/pull/8566#issuecomment-619569575 Another flaky test to addd to quarantine. This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] mik-laj opened a new issue #8568: Rewrite Google Cloud Build integration to use the native library.

2020-04-26 Thread GitBox
mik-laj opened a new issue #8568: URL: https://github.com/apache/airflow/issues/8568 Hello, We have an operator that allows us to run build in the Cloud Build service.

[GitHub] [airflow] mik-laj opened a new issue #8567: Support YMAL input for CloudBuildCreateOperator

2020-04-26 Thread GitBox
mik-laj opened a new issue #8567: URL: https://github.com/apache/airflow/issues/8567 **Description** Hello, We have an operator that allows us to run build in the Cloud Build service.

[GitHub] [airflow] evgenyshulman commented on issue #8052: [AIP-31] Create XComArg model

2020-04-26 Thread GitBox
evgenyshulman commented on issue #8052: URL: https://github.com/apache/airflow/issues/8052#issuecomment-619564136 > Regarding templating, I understand why you say that now. An option we implemented internally to do this was to add a `map` function to XComArg which is lazy evaluated and

[GitHub] [airflow] mik-laj commented on a change in pull request #8534: fix: aws hook should work without conn id

2020-04-26 Thread GitBox
mik-laj commented on a change in pull request #8534: URL: https://github.com/apache/airflow/pull/8534#discussion_r415325115 ## File path: airflow/providers/amazon/aws/hooks/base_aws.py ## @@ -62,16 +63,14 @@ class AwsBaseHook(BaseHook): def __init__( self,

[GitHub] [airflow] mik-laj commented on a change in pull request #8534: fix: aws hook should work without conn id

2020-04-26 Thread GitBox
mik-laj commented on a change in pull request #8534: URL: https://github.com/apache/airflow/pull/8534#discussion_r415323023 ## File path: airflow/providers/amazon/aws/hooks/base_aws.py ## @@ -62,16 +63,14 @@ class AwsBaseHook(BaseHook): def __init__( self,

[GitHub] [airflow] mik-laj commented on a change in pull request #8534: fix: aws hook should work without conn id

2020-04-26 Thread GitBox
mik-laj commented on a change in pull request #8534: URL: https://github.com/apache/airflow/pull/8534#discussion_r415322742 ## File path: airflow/providers/amazon/aws/hooks/base_aws.py ## @@ -62,16 +63,14 @@ class AwsBaseHook(BaseHook): def __init__( self,

[GitHub] [airflow] mik-laj commented on a change in pull request #8534: fix: aws hook should work without conn id

2020-04-26 Thread GitBox
mik-laj commented on a change in pull request #8534: URL: https://github.com/apache/airflow/pull/8534#discussion_r415322742 ## File path: airflow/providers/amazon/aws/hooks/base_aws.py ## @@ -62,16 +63,14 @@ class AwsBaseHook(BaseHook): def __init__( self,

[GitHub] [airflow] mik-laj commented on a change in pull request #8534: fix: aws hook should work without conn id

2020-04-26 Thread GitBox
mik-laj commented on a change in pull request #8534: URL: https://github.com/apache/airflow/pull/8534#discussion_r415321895 ## File path: airflow/providers/amazon/aws/hooks/base_aws.py ## @@ -62,16 +63,14 @@ class AwsBaseHook(BaseHook): def __init__( self,

[GitHub] [airflow] mik-laj commented on a change in pull request #8534: fix: aws hook should work without conn id

2020-04-26 Thread GitBox
mik-laj commented on a change in pull request #8534: URL: https://github.com/apache/airflow/pull/8534#discussion_r415321895 ## File path: airflow/providers/amazon/aws/hooks/base_aws.py ## @@ -62,16 +63,14 @@ class AwsBaseHook(BaseHook): def __init__( self,

[GitHub] [airflow] akki commented on issue #7955: Passing network and config/secret to DockerSwarmOperator

2020-04-26 Thread GitBox
akki commented on issue #7955: URL: https://github.com/apache/airflow/issues/7955#issuecomment-619555793 I don't think passing network is currently supported by `DockerSwarmOperator`. Can you find this feature in [Python Docker

[GitHub] [airflow] turbaszek commented on a change in pull request #8534: fix: aws hook should work without conn id

2020-04-26 Thread GitBox
turbaszek commented on a change in pull request #8534: URL: https://github.com/apache/airflow/pull/8534#discussion_r415314647 ## File path: airflow/providers/amazon/aws/hooks/base_aws.py ## @@ -62,16 +63,14 @@ class AwsBaseHook(BaseHook): def __init__(

[GitHub] [airflow] mik-laj commented on a change in pull request #8534: fix: aws hook should work without conn id

2020-04-26 Thread GitBox
mik-laj commented on a change in pull request #8534: URL: https://github.com/apache/airflow/pull/8534#discussion_r415314105 ## File path: airflow/providers/amazon/aws/hooks/base_aws.py ## @@ -62,16 +63,14 @@ class AwsBaseHook(BaseHook): def __init__( self,

[GitHub] [airflow] turbaszek commented on a change in pull request #8534: fix: aws hook should work without conn id

2020-04-26 Thread GitBox
turbaszek commented on a change in pull request #8534: URL: https://github.com/apache/airflow/pull/8534#discussion_r415309631 ## File path: airflow/providers/amazon/aws/hooks/base_aws.py ## @@ -62,16 +63,14 @@ class AwsBaseHook(BaseHook): def __init__(

[GitHub] [airflow] turbaszek edited a comment on issue #8058: [AIP-31] Add lazy way to get Airflow context from a wrapped function

2020-04-26 Thread GitBox
turbaszek edited a comment on issue #8058: URL: https://github.com/apache/airflow/issues/8058#issuecomment-619543260 > Sounds great, but does this imply we also change the context being provided the old way? No, if someone uses `**context` then it implicitly suggest that it will be

[GitHub] [airflow] turbaszek edited a comment on issue #8058: [AIP-31] Add lazy way to get Airflow context from a wrapped function

2020-04-26 Thread GitBox
turbaszek edited a comment on issue #8058: URL: https://github.com/apache/airflow/issues/8058#issuecomment-619543260 > Sounds great, but does this imply we also change the context being provided the old way? No, if someone uses `**context` then it implicitly suggest that it will be

[GitHub] [airflow] turbaszek edited a comment on issue #8058: [AIP-31] Add lazy way to get Airflow context from a wrapped function

2020-04-26 Thread GitBox
turbaszek edited a comment on issue #8058: URL: https://github.com/apache/airflow/issues/8058#issuecomment-619543260 > Sounds great, but does this imply we also change the context being provided the old way? No, if someone uses `**context` then it implicitly suggest that it will be

[GitHub] [airflow] turbaszek commented on issue #8058: [AIP-31] Add lazy way to get Airflow context from a wrapped function

2020-04-26 Thread GitBox
turbaszek commented on issue #8058: URL: https://github.com/apache/airflow/issues/8058#issuecomment-619543260 > Sounds great, but does this imply we also change the context being provided the old way? No, if someone uses `**context` then it implicitly suggest that it will be a

[GitHub] [airflow] turbaszek commented on pull request #8430: Improve idempodency in CloudDataTransferServiceCreateJobOperator

2020-04-26 Thread GitBox
turbaszek commented on pull request #8430: URL: https://github.com/apache/airflow/pull/8430#issuecomment-619542229 @khyurri have you tested the example dag? This is an automated message from the Apache Git Service. To

[GitHub] [airflow] jonathanshir edited a comment on issue #8058: [AIP-31] Add lazy way to get Airflow context from a wrapped function

2020-04-26 Thread GitBox
jonathanshir edited a comment on issue #8058: URL: https://github.com/apache/airflow/issues/8058#issuecomment-619541957 Sounds great, but does this imply we also change the context being provided the old way? ``` @provide_context def task(**context): print(type(context)) #

[GitHub] [airflow] jonathanshir commented on issue #8058: [AIP-31] Add lazy way to get Airflow context from a wrapped function

2020-04-26 Thread GitBox
jonathanshir commented on issue #8058: URL: https://github.com/apache/airflow/issues/8058#issuecomment-619541957 Sounds great, does this imply we also change the context being provided the old way? ``` @provide_context def task(**context): print(type(context)) # ==

[GitHub] [airflow] turbaszek commented on issue #8058: [AIP-31] Add lazy way to get Airflow context from a wrapped function

2020-04-26 Thread GitBox
turbaszek commented on issue #8058: URL: https://github.com/apache/airflow/issues/8058#issuecomment-619540876 > We simply have a single execute context for each task instance, so my intended implementation will be a bit simpler: > > ``` > from airflow.current import

[GitHub] [airflow] jonathanshir commented on issue #8058: [AIP-31] Add lazy way to get Airflow context from a wrapped function

2020-04-26 Thread GitBox
jonathanshir commented on issue #8058: URL: https://github.com/apache/airflow/issues/8058#issuecomment-619539473 My suggested changes: The issue states we want to implement context similarly to flask.requests : ``` from airflow.task import context @task def context_task(...)

[GitHub] [airflow] potiuk commented on a change in pull request #8562: Fixed SOURCE_BRANCH set by tag names

2020-04-26 Thread GitBox
potiuk commented on a change in pull request #8562: URL: https://github.com/apache/airflow/pull/8562#discussion_r415279039 ## File path: scripts/ci/ci_build_dockerhub.sh ## @@ -20,6 +20,7 @@ # on CI to potentially rebuild (and refresh layers that # are not cached) Docker

[GitHub] [airflow] potiuk opened a new pull request #8566: Add test_localtaskjob_maintain_heart_rate to quarantine

2020-04-26 Thread GitBox
potiuk opened a new pull request #8566: URL: https://github.com/apache/airflow/pull/8566 --- Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Unit tests coverage for changes (not needed for documentation

[GitHub] [airflow] zhongjiajie commented on a change in pull request #7903: Add property conn_name and connection to dbapi_hook

2020-04-26 Thread GitBox
zhongjiajie commented on a change in pull request #7903: URL: https://github.com/apache/airflow/pull/7903#discussion_r415272770 ## File path: airflow/hooks/dbapi_hook.py ## @@ -62,15 +63,29 @@ def __init__(self, *args, **kwargs): raise

[GitHub] [airflow] potiuk commented on pull request #8561: Add test to guard against command arg help message regression

2020-04-26 Thread GitBox
potiuk commented on pull request #8561: URL: https://github.com/apache/airflow/pull/8561#issuecomment-619522232 Yep. I am moving unstable tests to quarantine now (and we will de-quarantine them soon I hope - I have some idea for a process and way to assure tests are ready to be

[GitHub] [airflow] potiuk commented on a change in pull request #8562: Fixed SOURCE_BRANCH set by tag names

2020-04-26 Thread GitBox
potiuk commented on a change in pull request #8562: URL: https://github.com/apache/airflow/pull/8562#discussion_r415268589 ## File path: scripts/ci/ci_build_dockerhub.sh ## @@ -20,6 +20,7 @@ # on CI to potentially rebuild (and refresh layers that # are not cached) Docker

[GitHub] [airflow] Fokko commented on pull request #8563: This test (test_mark_success_on_success_callback) fails often

2020-04-26 Thread GitBox
Fokko commented on pull request #8563: URL: https://github.com/apache/airflow/pull/8563#issuecomment-619519468 Great work @potiuk  This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] potiuk commented on pull request #8563: This test (test_mark_success_on_success_callback) fails often

2020-04-26 Thread GitBox
potiuk commented on pull request #8563: URL: https://github.com/apache/airflow/pull/8563#issuecomment-619518680 > Should we create a ticket to de-quarantine this issue? :) Actually - I am going to create a project "De-quarantine" where I am going to add all quarantined test. However

[GitHub] [airflow] jeffolsi opened a new issue #8565: SalesforceHook missing method to return dataframe

2020-04-26 Thread GitBox
jeffolsi opened a new issue #8565: URL: https://github.com/apache/airflow/issues/8565 Currently SalesforceHook has function `write_object_to_file` which query salesforce and save the output to file. On the way it also uses df. The function even return the df. the problem is that if

[GitHub] [airflow] Fokko commented on a change in pull request #8256: updated _write_args on PythonVirtualenvOperator

2020-04-26 Thread GitBox
Fokko commented on a change in pull request #8256: URL: https://github.com/apache/airflow/pull/8256#discussion_r415256343 ## File path: airflow/operators/python_operator.py ## @@ -330,13 +330,28 @@ def _write_string_args(self, filename): def _write_args(self,

[GitHub] [airflow] feluelle commented on a change in pull request #7903: Add property conn_name and connection to dbapi_hook

2020-04-26 Thread GitBox
feluelle commented on a change in pull request #7903: URL: https://github.com/apache/airflow/pull/7903#discussion_r415253247 ## File path: airflow/hooks/dbapi_hook.py ## @@ -62,15 +63,29 @@ def __init__(self, *args, **kwargs): raise

[GitHub] [airflow] feluelle commented on a change in pull request #7903: Add property conn_name and connection to dbapi_hook

2020-04-26 Thread GitBox
feluelle commented on a change in pull request #7903: URL: https://github.com/apache/airflow/pull/7903#discussion_r415251335 ## File path: airflow/hooks/dbapi_hook.py ## @@ -62,15 +63,29 @@ def __init__(self, *args, **kwargs): raise

[GitHub] [airflow] feluelle commented on a change in pull request #8562: Fixed SOURCE_BRANCH set by tag names

2020-04-26 Thread GitBox
feluelle commented on a change in pull request #8562: URL: https://github.com/apache/airflow/pull/8562#discussion_r415248182 ## File path: scripts/ci/ci_build_dockerhub.sh ## @@ -20,6 +20,7 @@ # on CI to potentially rebuild (and refresh layers that # are not cached) Docker

[GitHub] [airflow] feluelle commented on a change in pull request #8562: Fixed SOURCE_BRANCH set by tag names

2020-04-26 Thread GitBox
feluelle commented on a change in pull request #8562: URL: https://github.com/apache/airflow/pull/8562#discussion_r415245697 ## File path: scripts/ci/ci_build_dockerhub.sh ## @@ -20,6 +20,7 @@ # on CI to potentially rebuild (and refresh layers that # are not cached) Docker

[jira] [Commented] (AIRFLOW-6251) add config for max tasks per dag to prevent scheduler denial of service

2020-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17092526#comment-17092526 ] ASF GitHub Bot commented on AIRFLOW-6251: - mik-laj commented on pull request #7157: URL:

[GitHub] [airflow] mik-laj commented on pull request #7157: [AIRFLOW-6251] add config for max tasks per dag

2020-04-26 Thread GitBox
mik-laj commented on pull request #7157: URL: https://github.com/apache/airflow/pull/7157#issuecomment-619488454 Why not add this code as an example for a cluster policy? I don't think it has to be a separate option. https://airflow.readthedocs.io/en/latest/concepts.html#cluster-policy