jhtimmins commented on a change in pull request #10594:
URL: https://github.com/apache/airflow/pull/10594#discussion_r480819521
##
File path: airflow/www/security.py
##
@@ -505,6 +507,7 @@ def sync_roles(self):
:return: None.
"""
+# breakpoint()
jhtimmins commented on a change in pull request #10594:
URL: https://github.com/apache/airflow/pull/10594#discussion_r480818779
##
File path: airflow/www/security.py
##
@@ -521,6 +524,18 @@ def sync_roles(self):
self.update_admin_perm_view()
coopergillan opened a new pull request #10671:
URL: https://github.com/apache/airflow/pull/10671
Add type annotations to several more AWS hooks. Fill in the last annotation
for a few others as well.
related: #9708
---
**^ Add meaningful description above**
Read the
houqp commented on a change in pull request #10652:
URL: https://github.com/apache/airflow/pull/10652#discussion_r480680965
##
File path: airflow/api_connexion/openapi/v1.yaml
##
@@ -2716,6 +2716,15 @@ components:
# Reusable callbacks
callbacks: {}
+ securitySchemes:
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to tag nightly-master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
*** WARNING: tag nightly-master was modified! ***
from 4476a09 (commit)
to f40ac9b (commit)
from
scottrigby edited a comment on issue #10523:
URL: https://github.com/apache/airflow/issues/10523#issuecomment-684078575
@potiuk I'm happy to help ask chart users and maintainers to contribute to a
migration guide. The current stable/airflow maintainers are here:
potiuk merged pull request #9444:
URL: https://github.com/apache/airflow/pull/9444
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/master by this push:
new f40ac9b Add placement_strategy option (#9444)
potiuk commented on pull request #9444:
URL: https://github.com/apache/airflow/pull/9444#issuecomment-684106979
Cool! What's next :)?
This is an automated message from the Apache Git Service.
To respond to the message,
mik-laj commented on pull request #10638:
URL: https://github.com/apache/airflow/pull/10638#issuecomment-684105624
@potiuk. I added backward compatibility. We have two similar functions:
load_connection and load_connections_dict, but I would like all the functions
responsible for loading
kanga333 commented on pull request #9444:
URL: https://github.com/apache/airflow/pull/9444#issuecomment-684105173
@potiuk Thanks for your review. I've rebased to the latest master and
confirmed that the CI will pass.
This
potiuk commented on pull request #10670:
URL: https://github.com/apache/airflow/pull/10670#issuecomment-684104998
cc: @OmairK
This is an automated message from the Apache Git Service.
To respond to the message, please log
mik-laj commented on a change in pull request #10664:
URL: https://github.com/apache/airflow/pull/10664#discussion_r480472704
##
File path: chart/values.schema.json
##
@@ -0,0 +1,1250 @@
+{
+"$schema": "http://json-schema.org/draft-07/schema;,
+"description": "Default
mik-laj commented on a change in pull request #10664:
URL: https://github.com/apache/airflow/pull/10664#discussion_r480472378
##
File path: chart/values.schema.json
##
@@ -0,0 +1,1250 @@
+{
+"$schema": "http://json-schema.org/draft-07/schema;,
+"description": "Default
potiuk opened a new pull request #10670:
URL: https://github.com/apache/airflow/pull/10670
Inspired by the Google Shell Guide where they mentioned
separating package names with :: I realized that this was
one of the missing pieces in the bash scripts of ours.
While we already
alexbegg commented on pull request #2460:
URL: https://github.com/apache/airflow/pull/2460#issuecomment-684083145
Another snag I ran into is I tried to replicate this functionality on top of
the latest master branch changes:
I can't figure out how to access the `schedulable_at` and
[
https://issues.apache.org/jira/browse/AIRFLOW-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188038#comment-17188038
]
ASF GitHub Bot commented on AIRFLOW-1424:
-
alexbegg edited a comment on pull request #2460:
alexbegg edited a comment on pull request #2460:
URL: https://github.com/apache/airflow/pull/2460#issuecomment-684010238
@ultrabug Going over this whole thread, I want some clarification before I
dive deeper into development this:
1. Will this show the next execution date for a DAG
[
https://issues.apache.org/jira/browse/AIRFLOW-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188037#comment-17188037
]
ASF GitHub Bot commented on AIRFLOW-1424:
-
alexbegg commented on pull request #2460:
URL:
scottrigby commented on issue #10523:
URL: https://github.com/apache/airflow/issues/10523#issuecomment-684078575
@potiuk I'm happy to help ask chart users and maintainers to contribute to a
migration guide. The current stable/airflow maintainers are here:
potiuk commented on issue #10667:
URL: https://github.com/apache/airflow/issues/10667#issuecomment-684063437
Sure - I am perfectly ok with that - and documenting is a good thing.
I just think we are spinning in circles a bit - I thought we reached
conclusions and explained all
houqp commented on pull request #5:
URL: https://github.com/apache/airflow-client-go/pull/5#issuecomment-684057510
@feluelle it would be 1.13, that's when go has the module feature fully
backed into the official tooling.
ryw commented on issue #10667:
URL: https://github.com/apache/airflow/issues/10667#issuecomment-684030568
We should seek to eliminate category-x dependencies/references, if possible,
as the cleanest resolution.
For those that we cannot get rid of, or don't think we must get rid of
mik-laj opened a new pull request #10669:
URL: https://github.com/apache/airflow/pull/10669
Diff: https://www.diffchecker.com/IrrA57LV
---
**^ Add meaningful description above**
Read the **[Pull Request
potiuk edited a comment on pull request #10668:
URL: https://github.com/apache/airflow/pull/10668#issuecomment-684026184
Looing forward to result of Snyk with those removed :)
This is an automated message from the Apache Git
potiuk commented on pull request #10668:
URL: https://github.com/apache/airflow/pull/10668#issuecomment-684026184
Looing forward to result of Snyke with those removed :)
This is an automated message from the Apache Git
This is an automated email from the ASF dual-hosted git repository.
ryw pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from aff8c76 Add example on airflow users create --help (#10662)
add 901ecf7 Remove requirements from the project.
potiuk edited a comment on issue #10667:
URL: https://github.com/apache/airflow/issues/10667#issuecomment-684025212
Duplicate of #9898
Just to give a bit of context here: As an ASF project we are bound by the
https://www.apache.org/legal/resolved.html
We've discussed most
potiuk commented on issue #10667:
URL: https://github.com/apache/airflow/issues/10667#issuecomment-684025212
Duplicate of #9898
Just to give a bit of context here: As an ASF project we are bound by the
https://www.apache.org/legal/resolved.html
We've discussed most (if not
ryw commented on pull request #10668:
URL: https://github.com/apache/airflow/pull/10668#issuecomment-684025015
Yes scanners have been no fun for me either - but cleaning this up as much
as possible helps us to avoid probably hundreds of frustrating conversations :)
This is an automated email from the ASF dual-hosted git repository.
ryw pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from aff8c76 Add example on airflow users create --help (#10662)
add 901ecf7 Remove requirements from the project.
ryw merged pull request #10668:
URL: https://github.com/apache/airflow/pull/10668
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
potiuk edited a comment on pull request #10668:
URL: https://github.com/apache/airflow/pull/10668#issuecomment-684016811
Fun fact: I lost the trust in Black Duck when we saw a number of games (?)
detected in the source code of Android (where there was obviously none of them
present) and
potiuk commented on pull request #10668:
URL: https://github.com/apache/airflow/pull/10668#issuecomment-684016811
Fun fact: I lost the trust in Black Duck when we saw a number of games (?)
detected in the source code of Android (where there was obviously none of them
present) and when we
potiuk commented on pull request #10668:
URL: https://github.com/apache/airflow/pull/10668#issuecomment-684014007
> Looks good to me - will report back what Snyk thinks of Airflow w/ these
files removed, once this PR is merged :)
Yep. I am always interested in those scanners. I used
ryw closed pull request #10665:
URL: https://github.com/apache/airflow/pull/10665
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
ryw commented on pull request #10665:
URL: https://github.com/apache/airflow/pull/10665#issuecomment-684011576
Closing this in favor of #10668
This is an automated message from the Apache Git Service.
To respond to the
[
https://issues.apache.org/jira/browse/AIRFLOW-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187985#comment-17187985
]
ASF GitHub Bot commented on AIRFLOW-1424:
-
alexbegg commented on pull request #2460:
URL:
alexbegg commented on pull request #2460:
URL: https://github.com/apache/airflow/pull/2460#issuecomment-684010238
@ultrabug Going over this whole thread, I want some clarification before I
step in to development for this:
1. Will this show the next execution date for a DAG which has
potiuk commented on pull request #10665:
URL: https://github.com/apache/airflow/pull/10665#issuecomment-684010452
PR here: https://github.com/apache/airflow/pull/10668
This is an automated message from the Apache Git
potiuk commented on pull request #10668:
URL: https://github.com/apache/airflow/pull/10668#issuecomment-684009873
Since we already released 1.10.12 - we can now safely remove the requirement
files. The image building process for both 1.10 and master branch is not using
them any more.
potiuk opened a new pull request #10668:
URL: https://github.com/apache/airflow/pull/10668
The requirements are not needed any more. We replaced them
with a new, better "constraints" mechanism where constraints
are stored in a separate, orphaned branches in the repository
and they
potiuk edited a comment on pull request #10665:
URL: https://github.com/apache/airflow/pull/10665#issuecomment-684008648
BTW. The vulnerability scanner will likely still show those dependencies, if
it is actually working (I would love to hear if it does).
As explained, the
potiuk commented on pull request #10665:
URL: https://github.com/apache/airflow/pull/10665#issuecomment-684008648
BTW. The vulnerability scanner will likely still show those dependencies, if
it is actually working (I would love to hear if it does).
As explained, the dependencies in
pgagnon commented on pull request #9935:
URL: https://github.com/apache/airflow/pull/9935#issuecomment-684006824
@dimberman
```
executor_config={
"KubernetesExecutor": {
"annotations": {
"iam.amazonaws.com/role":
dimberman commented on pull request #9935:
URL: https://github.com/apache/airflow/pull/9935#issuecomment-684005713
@pgagnon can you please post the executor_config that worked before but is
now broken?
This is an automated
potiuk commented on pull request #10665:
URL: https://github.com/apache/airflow/pull/10665#issuecomment-684004562
The PR is coming as you speak. BTW. See the
https://github.com/apache/airflow/blob/master/requirements/REMOVE.md note that
I left before.
pgagnon commented on pull request #9935:
URL: https://github.com/apache/airflow/pull/9935#issuecomment-684003991
Yeah, I got:
```
[2020-08-31 18:05:26,015] {{scheduler_job.py:1401}} ERROR - Exception when
executing execute_helper
Traceback (most recent call last):
File
ryw commented on pull request #10665:
URL: https://github.com/apache/airflow/pull/10665#issuecomment-684003525
Thanks @potiuk who will remove these unused files + when?
I'm responding to the output of our vulnerability scanner.
potiuk commented on pull request #10665:
URL: https://github.com/apache/airflow/pull/10665#issuecomment-684001008
FYI. @ryw - those files are not used anymore (we were waiting for 1.10.12 to
be released to remove them). And the name is indeed misleading so no wonder
you thought those are
ryw opened a new issue #10667:
URL: https://github.com/apache/airflow/issues/10667
I've pulled together a list of questionable licenses that license scanners
will pick up for Airflow. Ideally we should look to remove the dependencies
from the project.
| Component | Project |
ryw commented on pull request #10665:
URL: https://github.com/apache/airflow/pull/10665#issuecomment-683999560
i'm trying to understand the licensing risks we have in the project - and
realized that perhaps we can just remove Unidecode from codebase - tests don't
fail on this branch - can
dimberman commented on pull request #9935:
URL: https://github.com/apache/airflow/pull/9935#issuecomment-683997363
@pgagnon has this broken the API for you?
These lines should allow that
https://github.com/apache/airflow/pull/9935/files#diff-868ed785b2a336f20cb6a577dde2502aR362-R363
pgagnon commented on pull request #9935:
URL: https://github.com/apache/airflow/pull/9935#issuecomment-683990760
FYI, this is a breaking API change.
Old syntax was: `"resources": {"request_memory": "2Gi",
"limit_memory": "2Gi"},`
But now requests and limits need
dimberman commented on pull request #10666:
URL: https://github.com/apache/airflow/pull/10666#issuecomment-683986627
cc: @jhtimmins
This is an automated message from the Apache Git Service.
To respond to the message, please
dimberman opened a new pull request #10666:
URL: https://github.com/apache/airflow/pull/10666
This PR ensures that when a user kills a KubernetesPodOperator task
in the airflow UI, that the associated pod is also killed using the
on_kill method.
---
**^ Add
gardnerdev commented on pull request #10600:
URL: https://github.com/apache/airflow/pull/10600#issuecomment-683978963
@potiuk thank you! "create_user_job" is here by mistake. I made pull request
here: #10291 . I'll try to do separate PR. Thank you
ryw opened a new pull request #10665:
URL: https://github.com/apache/airflow/pull/10665
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)**
for more
potiuk commented on pull request #10598:
URL: https://github.com/apache/airflow/pull/10598#issuecomment-683965427
I am closing this one in favour of #10600.
This is an automated message from the Apache Git Service.
To
potiuk closed pull request #10598:
URL: https://github.com/apache/airflow/pull/10598
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
potiuk commented on a change in pull request #10600:
URL: https://github.com/apache/airflow/pull/10600#discussion_r480323012
##
File path: chart/values.yaml
##
@@ -79,8 +79,8 @@ images:
tag: 0.11.0
pullPolicy: IfNotPresent
redis:
-repository: redis
-tag:
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 4fd81f5 Add howto doc for Salesforce connection (#10482)
add aff8c76 Add example on airflow users create
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 4fd81f5 Add howto doc for Salesforce connection (#10482)
add aff8c76 Add example on airflow users create
potiuk commented on pull request #10662:
URL: https://github.com/apache/airflow/pull/10662#issuecomment-683960785
. Always had to recreate it from memory.
This is an automated message from the Apache Git Service.
To
potiuk merged pull request #10662:
URL: https://github.com/apache/airflow/pull/10662
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
dimberman commented on issue #10586:
URL: https://github.com/apache/airflow/issues/10586#issuecomment-683959236
@SamWheating I'd say retrieve all logs in all cases. The fact that these are
truncated is a bug. Glad to help whenever you have a PR ready!
potiuk edited a comment on issue #10523:
URL: https://github.com/apache/airflow/issues/10523#issuecomment-683956090
> * donation: the entire `helm/charts` project - this includes each chart
there - is already licensed under an Apache License 2.0:
potiuk edited a comment on issue #10523:
URL: https://github.com/apache/airflow/issues/10523#issuecomment-683956090
> * donation: the entire `helm/charts` project - this includes each chart
there - is already licensed under an Apache License 2.0:
potiuk commented on issue #10523:
URL: https://github.com/apache/airflow/issues/10523#issuecomment-683956090
> * donation: the entire `helm/charts` project - this includes each chart
there - is already licensed under an Apache License 2.0:
SamWheating commented on issue #10586:
URL: https://github.com/apache/airflow/issues/10586#issuecomment-683955857
Sounds good. From the kubernetes python client source:
```
:param int tail_lines: If set, the number of lines from the end of the logs
to show. If not specified, logs are
michalmisiewicz commented on issue #10586:
URL: https://github.com/apache/airflow/issues/10586#issuecomment-683947340
> I'd be interested in making a PR to expose the tail_lines property to the
user in the k8sPodOperator since this would allow us to get back onto the
official version of
SamWheating commented on issue #10586:
URL: https://github.com/apache/airflow/issues/10586#issuecomment-683942214
Replicated the issue in Airflow 1.10.12 like so:
Created a DAG to really quickly log a bunch of stuff in a k8s pod:
```python
from datetime import timedelta
MahsaSeifikar commented on issue #10656:
URL: https://github.com/apache/airflow/issues/10656#issuecomment-683939649
> This looks like real issue. Do I have to assign you to this ticket? Would
you like to contribute a fix for this problem? I happy to help with review.
Yup, I know
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from aa2db70 Unify error messages and complete type field in response
(#10333)
add 4fd81f5 Add howto doc
mik-laj closed issue #10029:
URL: https://github.com/apache/airflow/issues/10029
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
mik-laj merged pull request #10482:
URL: https://github.com/apache/airflow/pull/10482
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
potiuk commented on pull request #10393:
URL: https://github.com/apache/airflow/pull/10393#issuecomment-683929756
On vacations now. But yes. I will be on a lookout :).
This is an automated message from the Apache Git
potiuk commented on a change in pull request #10413:
URL: https://github.com/apache/airflow/pull/10413#discussion_r480287828
##
File path: docs/howto/initialize-database.rst
##
@@ -48,11 +48,46 @@ SqlAlchemy backend. We recommend using **MySQL** or
**Postgres**.
want to
potiuk commented on a change in pull request #10413:
URL: https://github.com/apache/airflow/pull/10413#discussion_r480287444
##
File path: docs/howto/initialize-database.rst
##
@@ -48,11 +48,49 @@ SqlAlchemy backend. We recommend using **MySQL** or
**Postgres**.
want to
dimberman commented on pull request #10393:
URL: https://github.com/apache/airflow/pull/10393#issuecomment-683928708
@potiuk yes This is breaking for the KubernetesPodOperator so we should hold
off. I'm hoping to have this merged in the next day or two. Can I please get a
lookover? :)
potiuk commented on pull request #10393:
URL: https://github.com/apache/airflow/pull/10393#issuecomment-683927786
Hey @dimberman - > should we wait with releasing backports before this one
is merged ?
This is an automated
eladkal commented on pull request #10482:
URL: https://github.com/apache/airflow/pull/10482#issuecomment-683926536
@mik-laj @feluelle rebased
This is an automated message from the Apache Git Service.
To respond to the
SamWheating commented on issue #10586:
URL: https://github.com/apache/airflow/issues/10586#issuecomment-683926160
Yup, I'm just updating our clusters to 1.10.12 and then I'll try to
replicate the error case (a pod producing a lot of logs quickly and then
shutting down).
dstandish edited a comment on pull request #10621:
URL: https://github.com/apache/airflow/pull/10621#issuecomment-683912244
Thanks @ArgentFalcon for looking it over .
One more bit of info here...
When trying to understand tooltip a little better I found [this SO
eladkal commented on issue #10648:
URL: https://github.com/apache/airflow/issues/10648#issuecomment-683921278
Commented about it on Slack.
Your implementation seems to be a private case of
dstandish edited a comment on pull request #10621:
URL: https://github.com/apache/airflow/pull/10621#issuecomment-683912244
Thanks @ArgentFalcon for looking it over .
One more bit of info here...
When trying to understand tooltip a little better I found [this SO
mik-laj commented on pull request #10664:
URL: https://github.com/apache/airflow/pull/10664#issuecomment-683918932
We have some errors Can you look at it?
```
2020-08-31T16:29:32.9918043Z helm install airflow . --namespace airflow
--set defaultAirflowRepository=apache/airflow --set
mik-laj commented on a change in pull request #10662:
URL: https://github.com/apache/airflow/pull/10662#discussion_r480273552
##
File path: docs/spelling_wordlist.txt
##
@@ -76,12 +76,12 @@ DagRun
Dagbag
Review comment:
I added firstname and lastname and sort the file
dstandish commented on pull request #10621:
URL: https://github.com/apache/airflow/pull/10621#issuecomment-683912244
Thanks @ArgentFalcon for looking it over .
One more bit of info here...
When trying to understand tooltip a little better I found [this SO
dimberman commented on a change in pull request #10393:
URL: https://github.com/apache/airflow/pull/10393#discussion_r480264064
##
File path: UPDATING.md
##
@@ -153,6 +152,175 @@ The Old and New provider configuration keys that have
changed are as follows
For more
mik-laj commented on a change in pull request #10652:
URL: https://github.com/apache/airflow/pull/10652#discussion_r480262934
##
File path: airflow/api_connexion/openapi/v1.yaml
##
@@ -2716,6 +2716,15 @@ components:
# Reusable callbacks
callbacks: {}
+
mik-laj commented on pull request #10652:
URL: https://github.com/apache/airflow/pull/10652#issuecomment-683897169
New sections have appeared in the reference documentation. I think it's
worth hiding them because we have other documentation that is more precise.
