pingzh commented on a change in pull request #10654:
URL: https://github.com/apache/airflow/pull/10654#discussion_r486788268
##
File path: airflow/executors/celery_kubernetes_executor.py
##
@@ -0,0 +1,157 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+#
baolsen opened a new issue #10874:
URL: https://github.com/apache/airflow/issues/10874
**Apache Airflow version**: 1.10.8
**Environment**:
- **Cloud provider or hardware configuration**: 4 VCPU 8GB RAM VM
- **OS** (e.g. from /etc/os-release): RHEL 7.7
- **Kernel** (e.g.
elwinarens edited a comment on pull request #9639:
URL: https://github.com/apache/airflow/pull/9639#issuecomment-690866681
@kaxil that option is not available, because this PR is not under a personal
user. Feel free to cherry pick and to put up a new PR. Let me know if you need
anything.
msumit commented on issue #7935:
URL: https://github.com/apache/airflow/issues/7935#issuecomment-690875882
All system vitals like the disk, cpu, and mem are absolutely fine whenever
the stuck happens for us. Whenever the process stuck, it doesn't respond to any
other kill signals except 9
elwinarens edited a comment on pull request #9639:
URL: https://github.com/apache/airflow/pull/9639#issuecomment-690866681
@kaxil that option is not available, because this PR is not under a personal
user. Feel free cherry pick and to put up a new PR. Let me know if you need
anything.
elwinarens commented on pull request #9639:
URL: https://github.com/apache/airflow/pull/9639#issuecomment-690866681
@kaxil that option is not available, because this PR is not under a personal
user. Feel free to put up a new PR and close this one.
elwinarens edited a comment on pull request #9639:
URL: https://github.com/apache/airflow/pull/9639#issuecomment-690866681
@kaxil that option is not available, because this PR is not under a personal
user. Feel free to put up a new PR.
elwinarens closed pull request #9639:
URL: https://github.com/apache/airflow/pull/9639
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
stale[bot] closed pull request #9778:
URL: https://github.com/apache/airflow/pull/9778
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
[
https://issues.apache.org/jira/browse/AIRFLOW-6585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17193974#comment-17193974
]
ASF GitHub Bot commented on AIRFLOW-6585:
-
stale[bot] closed pull request #7153:
URL:
stale[bot] closed pull request #7153:
URL: https://github.com/apache/airflow/pull/7153
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
jedcunningham commented on pull request #10663:
URL: https://github.com/apache/airflow/pull/10663#issuecomment-690846446
@ashb, changed the default to true, refactored so we only hit conf once, and
gave the tests a little more love. The only remaining thing I can think of is
stale[bot] commented on pull request #10011:
URL: https://github.com/apache/airflow/pull/10011#issuecomment-690838639
This issue has been automatically marked as stale because it has not had
recent activity. It will be closed if no further activity occurs. Thank you for
your
stale[bot] commented on pull request #9737:
URL: https://github.com/apache/airflow/pull/9737#issuecomment-690838643
This issue has been automatically marked as stale because it has not had
recent activity. It will be closed if no further activity occurs. Thank you for
your contributions.
dutta1kartik3 commented on issue #8759:
URL: https://github.com/apache/airflow/issues/8759#issuecomment-690836209
But this external log deletion dag / script doesn't seem to cause the
scheduler to go down
[
https://issues.apache.org/jira/browse/AIRFLOW-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17193953#comment-17193953
]
ASF GitHub Bot commented on AIRFLOW-5071:
-
yuqian90 edited a comment on issue #10790:
URL:
yuqian90 edited a comment on issue #10790:
URL: https://github.com/apache/airflow/issues/10790#issuecomment-690785232
> Ok @yuqian90 @sgrzemski-ias what is you setting for
core.dagbag_import_timeout ?
>
> As I'm hitting:
>
> Traceback (most recent call last): File
yuqian90 opened a new pull request #10153:
URL: https://github.com/apache/airflow/pull/10153
This PR introduces `TaskGroup`, which is a simple UI task grouping concept.
What this PR does:
- TaskGroups can be collapsed/expanded in Graph View when clicked
- TaskGroups can be
yuqian90 closed pull request #10153:
URL: https://github.com/apache/airflow/pull/10153
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
dimberman commented on pull request #10872:
URL: https://github.com/apache/airflow/pull/10872#issuecomment-690808867
@potiuk @turbaszek any idea how I can unit test this? This is a bit tricky
since I'm generating a string inside of a config_file instead of an actual yaml
object that helm
abhilash1in opened a new pull request #10873:
URL: https://github.com/apache/airflow/pull/10873
## Summary:
Add a description (string) field to store additional details about a
[connection](https://airflow.apache.org/docs/stable/howto/connection/index.html).
## UI:
dimberman opened a new pull request #10872:
URL: https://github.com/apache/airflow/pull/10872
Since we are deprecating most k8sexecutor arguments
we should use the pod_template_file when launching airflow
using the KubernetesExecutor
(cherry picked from commit
yuqian90 commented on issue #10790:
URL: https://github.com/apache/airflow/issues/10790#issuecomment-690785232
> Ok @yuqian90 @sgrzemski-ias what is you setting for
core.dagbag_import_timeout ?
>
> As I'm hitting:
>
> Traceback (most recent call last): File
[
https://issues.apache.org/jira/browse/AIRFLOW-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17193906#comment-17193906
]
ASF GitHub Bot commented on AIRFLOW-5071:
-
yuqian90 commented on issue #10790:
URL:
vitaly-krugl commented on issue #8759:
URL: https://github.com/apache/airflow/issues/8759#issuecomment-690775457
Our internal environment enforces mandatory log rotation automatically as
part of its log rotation and archiving mechanism. Since the description of this
ticket points out that
dimberman commented on a change in pull request #10654:
URL: https://github.com/apache/airflow/pull/10654#discussion_r486677273
##
File path: airflow/executors/celery_kubernetes_executor.py
##
@@ -0,0 +1,157 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
pingzh commented on pull request #10654:
URL: https://github.com/apache/airflow/pull/10654#issuecomment-690772877
Strange, i started to get this static check error after I rebased the master:
```
Run
pingzh closed pull request #10871:
URL: https://github.com/apache/airflow/pull/10871
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
pingzh closed pull request #10870:
URL: https://github.com/apache/airflow/pull/10870
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
pingzh opened a new pull request #10871:
URL: https://github.com/apache/airflow/pull/10871
```
Run
mypy..Failed
- hook id: mypy
- exit code: 1
pingzh opened a new pull request #10870:
URL: https://github.com/apache/airflow/pull/10870
it consists of CeleryExecutor and KubernetesExecutor, which allows users
to route their tasks to either Kubernetes or Celery based on the queue
defined on a task
See the discussion in the
kaxil commented on a change in pull request #10654:
URL: https://github.com/apache/airflow/pull/10654#discussion_r486653277
##
File path: airflow/executors/celery_kubernetes_executor.py
##
@@ -0,0 +1,157 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or
pingzh commented on a change in pull request #10654:
URL: https://github.com/apache/airflow/pull/10654#discussion_r486651631
##
File path: airflow/executors/celery_kubernetes_executor.py
##
@@ -0,0 +1,157 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+#
kaxil commented on pull request #9639:
URL: https://github.com/apache/airflow/pull/9639#issuecomment-690739141
@elwinarens We are very close to merging this can you please rebase, or just
checkmark "Allow Edit From Maintainers"
mik-laj commented on issue #7907:
URL: https://github.com/apache/airflow/issues/7907#issuecomment-690738103
@nikste What specific E2E tests would be of interest to you? In my case,
E2E tests mean the configuration of a separate test environment, services that
are used to monitor the
tszerszen commented on issue #10723:
URL: https://github.com/apache/airflow/issues/10723#issuecomment-690737114
My proposition is to change `params` to `prediction_params`, here's the
[PR](https://github.com/apache/airflow/pull/10869)
nikste commented on issue #7907:
URL: https://github.com/apache/airflow/issues/7907#issuecomment-690735648
@mik-laj thanks for the update, i think it would make a lot of sense to have
some easy end-to-end testing (I would definitely find it useful :) )
kaxil commented on pull request #9639:
URL: https://github.com/apache/airflow/pull/9639#issuecomment-690732567
> @kaxil If neither are able to continue the work, what is remaining to do
here? Does it need a hook? has config secrets been added to this backend yet? I
am already using this
tszerszen opened a new pull request #10869:
URL: https://github.com/apache/airflow/pull/10869
Proposition of solving issue: [#10723
](https://github.com/apache/airflow/issues/10723)
Fix parameter name collision in `AutoMLBatchPredictOperator` by changing
parameter `params` to
mik-laj commented on issue #10449:
URL: https://github.com/apache/airflow/issues/10449#issuecomment-690727096
https://github.com/apache/airflow/issues/10194
https://lists.apache.org/thread.html/2b12ae265795ff2e655a5161c972f5c7bbe60722a12849a0e2c5c55f%40%3Cdev.airflow.apache.org%3E
alexbegg edited a comment on pull request #9639:
URL: https://github.com/apache/airflow/pull/9639#issuecomment-690718440
@kaxil If neither are able to continue the work, what is remaining to do
here? Does it need a hook? has config secrets been added to this backend yet? I
am already
mik-laj edited a comment on issue #7907:
URL: https://github.com/apache/airflow/issues/7907#issuecomment-690723903
@nikste I know a some organizations are working on solutions internally, but
there is no community progress.
mik-laj commented on issue #7907:
URL: https://github.com/apache/airflow/issues/7907#issuecomment-690723903
@nikste I know a lot of organizations are working on solutions internally,
but there is no community progress.
This
alexbegg commented on pull request #9639:
URL: https://github.com/apache/airflow/pull/9639#issuecomment-690718440
@kaxil If neither are able to continue the work, what is remaining to do
here? Does it need a hook? has config secrets been added to this backend yet? I
am already using this
mik-laj edited a comment on pull request #10820:
URL: https://github.com/apache/airflow/pull/10820#issuecomment-690716090
@anmol-dhingra Is it worth adding unit tests to detect regressions?
@chattarajoy Thanks for a review.
mik-laj commented on pull request #10820:
URL: https://github.com/apache/airflow/pull/10820#issuecomment-690716090
@anmol-dhingra Is it worth adding automated tests to detect regressions?
@chattarajoy Thanks for a review.
turbaszek commented on a change in pull request #10654:
URL: https://github.com/apache/airflow/pull/10654#discussion_r486617206
##
File path: airflow/executors/celery_kubernetes_executor.py
##
@@ -0,0 +1,157 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
turbaszek commented on a change in pull request #10864:
URL: https://github.com/apache/airflow/pull/10864#discussion_r486614542
##
File path: airflow/providers/postgres/hooks/postgres.py
##
@@ -191,7 +203,7 @@ def get_iam_token(self, conn):
return login, token, port
turbaszek commented on a change in pull request #10864:
URL: https://github.com/apache/airflow/pull/10864#discussion_r486611993
##
File path: .gitignore
##
@@ -205,3 +205,22 @@ Chart.lock
# Might be generated when you build wheels
pip-wheel-metadata
+
+# tmp ...
Review
mik-laj commented on a change in pull request #10864:
URL: https://github.com/apache/airflow/pull/10864#discussion_r486604058
##
File path: .gitignore
##
@@ -205,3 +205,22 @@ Chart.lock
# Might be generated when you build wheels
pip-wheel-metadata
+
+# tmp ...
Review
mik-laj commented on issue #10868:
URL: https://github.com/apache/airflow/issues/10868#issuecomment-690688512
It' a real bug. See:
airflow.plugins_manager.load_plugins_from_plugin_directory This does not call
the on_load method.
mik-laj commented on issue #10868:
URL: https://github.com/apache/airflow/issues/10868#issuecomment-690687844
@maiorBoltach Yes. We are open to contributions. I assigned you to this
ticket.
Contributor guide:
https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst
mik-laj edited a comment on issue #10868:
URL: https://github.com/apache/airflow/issues/10868#issuecomment-690687844
@maiorBoltach Yes. We are open to contributions from everyone. I assigned
you to this ticket.
Contributor guide:
maiorBoltach commented on issue #10868:
URL: https://github.com/apache/airflow/issues/10868#issuecomment-690686788
If this is really a bug, can I fix it by myself?
This is an automated message from the Apache Git Service.
To
maiorBoltach edited a comment on issue #10868:
URL: https://github.com/apache/airflow/issues/10868#issuecomment-690686788
If this is really a bug, can I contribute?
This is an automated message from the Apache Git Service.
maiorBoltach opened a new issue #10868:
URL: https://github.com/apache/airflow/issues/10868
**Apache Airflow version**: 1.10.12
**Environment**:
- **Cloud provider or hardware configuration**: Google Cloud, custom-8-16384
- **OS** (e.g. from /etc/os-release): CentOS
mik-laj commented on issue #10786:
URL: https://github.com/apache/airflow/issues/10786#issuecomment-690684141
I assigned you to this ticket.
> If you create pull-request, you don't have to create an issue first, but
if you want, you can do it. Creating an issue will allow you to
turbaszek commented on pull request #10866:
URL: https://github.com/apache/airflow/pull/10866#issuecomment-690677618
@tszerszen can you please take a look at CI? We hit the magic number of
allowed arguments and we need to add `pylint: disable=too-many-arguments`
ashb commented on pull request #10806:
URL: https://github.com/apache/airflow/pull/10806#issuecomment-690673656
My quick thoughts are:
- more tools are supporting it over time
- this behaviour is supported/documented in python so a bug in tools that
don't
- the less special
mik-laj opened a new issue #10867:
URL: https://github.com/apache/airflow/issues/10867
Hello,
Currently, you can define a new connection/variable in several ways.
* Metabase
* Environment variable
* Secret backend
However, only the connections defined with the
mik-laj commented on issue #10840:
URL: https://github.com/apache/airflow/issues/10840#issuecomment-690628023
@abhilash1in I assigned you to this ticket.
This is an automated message from the Apache Git Service.
To respond
potiuk edited a comment on pull request #10806:
URL: https://github.com/apache/airflow/pull/10806#issuecomment-690612974
Question - do we actually __want__ it eventually ?
I know from pure python installation "versatilty" point of view it might be
a good idea, but I hardly imagine
potiuk commented on pull request #10806:
URL: https://github.com/apache/airflow/pull/10806#issuecomment-690612974
Question - do we actually __want__ it eventually ?
I know from pure python installation "versatilty" point of view it might be
a good idea, but I hardly imagine this
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch constraints-master
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/constraints-master by this
push:
new 9682084 Updating
This is an automated email from the ASF dual-hosted git repository.
dimberman pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/v1-10-test by this push:
new 589d907 Fix pod_mutation_hook for
This is an automated email from the ASF dual-hosted git repository.
dimberman pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/v1-10-test by this push:
new 589d907 Fix pod_mutation_hook for
dimberman merged pull request #10850:
URL: https://github.com/apache/airflow/pull/10850
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
abhilash1in commented on issue #10840:
URL: https://github.com/apache/airflow/issues/10840#issuecomment-690584658
@mik-laj I can give this a shot. Please assign it to me.
This is an automated message from the Apache Git
jedcunningham commented on a change in pull request #10663:
URL: https://github.com/apache/airflow/pull/10663#discussion_r486531848
##
File path: airflow/config_templates/config.yml
##
@@ -229,6 +229,21 @@
type: string
example: ~
default: "30"
+- name:
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 59e8341 Add new lint check to now allow realtive imports (#10825)
add b9dc3c5 Added Plexus as an Airflow
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 59e8341 Add new lint check to now allow realtive imports (#10825)
add b9dc3c5 Added Plexus as an Airflow
potiuk merged pull request #10591:
URL: https://github.com/apache/airflow/pull/10591
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
boring-cyborg[bot] commented on pull request #10591:
URL: https://github.com/apache/airflow/pull/10591#issuecomment-690581454
Awesome work, congrats on your first merged pull request!
This is an automated message from
neil90 commented on issue #10786:
URL: https://github.com/apache/airflow/issues/10786#issuecomment-690568471
Yes I have made the PR https://github.com/apache/airflow/pull/10787
This is an automated message from the Apache
ashb commented on a change in pull request #10663:
URL: https://github.com/apache/airflow/pull/10663#discussion_r486521676
##
File path: airflow/models/dagbag.py
##
@@ -269,7 +270,12 @@ def _load_modules_from_file(self, filepath, safe_mode):
return
ashb commented on a change in pull request #10393:
URL: https://github.com/apache/airflow/pull/10393#discussion_r486521086
##
File path: chart/values.yaml
##
@@ -541,32 +543,11 @@ config:
namespace: '{{ .Release.Namespace }}'
airflow_configmap: '{{ include
kaxil commented on pull request #9639:
URL: https://github.com/apache/airflow/pull/9639#issuecomment-690557596
ping @elwinarens @flolas
This is an automated message from the Apache Git Service.
To respond to the message,
dimberman commented on a change in pull request #10393:
URL: https://github.com/apache/airflow/pull/10393#discussion_r486513263
##
File path: chart/values.yaml
##
@@ -541,32 +543,11 @@ config:
namespace: '{{ .Release.Namespace }}'
airflow_configmap: '{{ include
jedcunningham commented on a change in pull request #10663:
URL: https://github.com/apache/airflow/pull/10663#discussion_r486513326
##
File path: airflow/models/dagbag.py
##
@@ -269,7 +270,12 @@ def _load_modules_from_file(self, filepath, safe_mode):
return
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 68cc727 Add on_kill method to DataprocSubmitJobOperator (#10847)
add 59e8341 Add new lint check to now
millertracy commented on pull request #10591:
URL: https://github.com/apache/airflow/pull/10591#issuecomment-690531779
@potiuk thanks for approving changes. 2 checks were cancelled, but it
doesn't look related to my code. When should i expect a merge into the base
branch?
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 68cc727 Add on_kill method to DataprocSubmitJobOperator (#10847)
add 59e8341 Add new lint check to now
This is an automated email from the ASF dual-hosted git repository.
turbaszek pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from e773f8b Refactor content to a markdown table (#10863)
add 68cc727 Add on_kill method to
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 68cc727 Add on_kill method to DataprocSubmitJobOperator (#10847)
add 59e8341 Add new lint check to now
This is an automated email from the ASF dual-hosted git repository.
turbaszek pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from e773f8b Refactor content to a markdown table (#10863)
add 68cc727 Add on_kill method to
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 68cc727 Add on_kill method to DataprocSubmitJobOperator (#10847)
add 59e8341 Add new lint check to now
This is an automated email from the ASF dual-hosted git repository.
turbaszek pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from e773f8b Refactor content to a markdown table (#10863)
add 68cc727 Add on_kill method to
kaxil merged pull request #10825:
URL: https://github.com/apache/airflow/pull/10825
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
turbaszek commented on issue #10382:
URL: https://github.com/apache/airflow/issues/10382#issuecomment-690523736
@tszerszen awesome! I will take a look
This is an automated message from the Apache Git Service.
To respond to
turbaszek merged pull request #10847:
URL: https://github.com/apache/airflow/pull/10847
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
turbaszek closed issue #10381:
URL: https://github.com/apache/airflow/issues/10381
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
boring-cyborg[bot] commented on pull request #10847:
URL: https://github.com/apache/airflow/pull/10847#issuecomment-690523633
Awesome work, congrats on your first merged pull request!
This is an automated message from
This is an automated email from the ASF dual-hosted git repository.
ash pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 63b6e53 Detect orphaned task instances by SchedulerJob id and
heartbeat (#10729)
add 1a95361 Fix and
dimberman commented on a change in pull request #10654:
URL: https://github.com/apache/airflow/pull/10654#discussion_r486500203
##
File path: airflow/executors/celery_kubernetes_executor.py
##
@@ -0,0 +1,156 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
kaxil commented on a change in pull request #10594:
URL: https://github.com/apache/airflow/pull/10594#discussion_r486500224
##
File path: docs/security/access-control.rst
##
@@ -96,21 +96,69 @@ DAG Level Role
is treated as a ``View`` which has two permissions associated with
This is an automated email from the ASF dual-hosted git repository.
ash pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 67af97a Rename "Beyond the Horizon" section and refactor content
(#10802)
add 63b6e53 Detect orphaned task
This is an automated email from the ASF dual-hosted git repository.
ash pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 67af97a Rename "Beyond the Horizon" section and refactor content
(#10802)
add 63b6e53 Detect orphaned task
ashb merged pull request #10862:
URL: https://github.com/apache/airflow/pull/10862
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
This is an automated email from the ASF dual-hosted git repository.
ryw pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from ff72327 Move parse_once to quarantine (#10857)
add 67af97a Rename "Beyond the Horizon" section and refactor
This is an automated email from the ASF dual-hosted git repository.
ryw pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from ff72327 Move parse_once to quarantine (#10857)
add 67af97a Rename "Beyond the Horizon" section and refactor
1 - 100 of 219 matches
Mail list logo
