eladkal opened a new issue, #28289:
URL: https://github.com/apache/airflow/issues/28289
### Body
[Neptune](https://docs.aws.amazon.com/neptune/index.html) is a graph db
offered by AWS.
boto3 endpoints can be seen
[here](https://boto3.amazonaws.com/v1/documentation/api/latest/refer
ryangrose commented on code in PR #28021:
URL: https://github.com/apache/airflow/pull/28021#discussion_r1045165468
##
docs/apache-airflow/concepts/dags.rst:
##
@@ -796,32 +796,44 @@ the dependency graph.
The dependency detector is configurable, so you can implement your own log
VladaZakharova commented on code in PR #28284:
URL: https://github.com/apache/airflow/pull/28284#discussion_r1045151208
##
airflow/providers/google/cloud/transfers/gcs_to_bigquery.py:
##
@@ -300,6 +305,16 @@ def execute(self, context: Context):
impersonation_chain=s
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch constraints-main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/constraints-main by this push:
new 5284b9ab40 Updating con
o-nikolas commented on code in PR #28288:
URL: https://github.com/apache/airflow/pull/28288#discussion_r1045147675
##
airflow/executors/local_kubernetes_executor.py:
##
@@ -43,6 +43,8 @@ class LocalKubernetesExecutor(LoggingMixin):
KUBERNETES_QUEUE = conf.get("local_kuber
o-nikolas commented on issue #28276:
URL: https://github.com/apache/airflow/issues/28276#issuecomment-1345405031
Yupp, I think that's the way I'm leaning as well. I'll hold off on reviewing
those PRs and give some time for others to weigh in since it will change how we
fix the inconsistency
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
from 3b203bcb67 More robust cleanup of executors in
test_kubernetes_executor (#28281)
add 0db5e46ec1 Proper Python Hos
potiuk merged PR #28254:
URL: https://github.com/apache/airflow/pull/28254
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.a
pierrejeambrun opened a new pull request, #28288:
URL: https://github.com/apache/airflow/pull/28288
`LocalKurbernetesExecutor` does not extends the `BaseExecutor` class, and
therefore does not inherit `is_local` attribute.
cc: @dstandish @o-nikolas
--
This is an automated mess
quentin-sommer commented on code in PR #28284:
URL: https://github.com/apache/airflow/pull/28284#discussion_r1045141132
##
airflow/providers/google/cloud/transfers/gcs_to_bigquery.py:
##
@@ -300,6 +305,16 @@ def execute(self, context: Context):
impersonation_chain=s
VladaZakharova closed pull request #28284: Fix for issue with reading schema
fields for JSON files in GCSToBigQueryOperator
URL: https://github.com/apache/airflow/pull/28284
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
pierrejeambrun commented on code in PR #27953:
URL: https://github.com/apache/airflow/pull/27953#discussion_r1045134838
##
airflow/models/dag.py:
##
@@ -3204,6 +3204,13 @@ def __init__(self, concurrency=None, **kwargs):
def __repr__(self):
return f""
+@proper
boring-cyborg[bot] commented on issue #28287:
URL: https://github.com/apache/airflow/issues/28287#issuecomment-1345378555
Thanks for opening your first issue here! Be sure to follow the issue
template!
--
This is an automated message from the Apache Git Service.
To respond to the mess
bletvaska opened a new issue, #28287:
URL: https://github.com/apache/airflow/issues/28287
### Apache Airflow version
2.5.0
### What happened
Let's have following DAG with task `echo`:
```python
@dag(start_date=pendulum.now())
def playground():
@tas
Taragolis commented on PR #28282:
URL: https://github.com/apache/airflow/pull/28282#issuecomment-1345371994
@swapz-z thanks for contribution
Seems like you didn't not configure pre-commit and your change do not pass
statics check, some useful info how to do this
- [Contributor's Qu
VladaZakharova commented on PR #27052:
URL: https://github.com/apache/airflow/pull/27052#issuecomment-1345366891
@quentin-sommer
Hi Team :)
Please, check the PR that I have created for the fix to this issue:
https://github.com/apache/airflow/pull/28284
Thanks!
--
This is an
VladaZakharova opened a new pull request, #28284:
URL: https://github.com/apache/airflow/pull/28284
This fix includes changes for issue with incorrect reading schema_fields in
case of using JSON files.
---
**^ Add meaningful description above**
Read the **[Pull Request
sternr commented on code in PR #28128:
URL: https://github.com/apache/airflow/pull/28128#discussion_r1045124170
##
airflow/jobs/scheduler_job.py:
##
@@ -880,13 +880,24 @@ def _run_scheduler_loop(self) -> None:
# is finished to avoid concurrent access to the
swapz-z commented on issue #27979:
URL: https://github.com/apache/airflow/issues/27979#issuecomment-1345356191
Created my first PR. Perhaps the PR helps to understand the issue it solves
more clearly.
Could you please have a look and provide your feedback on this PR and the
issue it addr
potiuk commented on issue #8212:
URL: https://github.com/apache/airflow/issues/8212#issuecomment-1345307989
Hey @dimon222 @EinavDanielDX @aa3pankaj - can you please try my (new attempt
to) fix from #28283 - it's generally a "brute-force" kinda fix. It requires you
to set the `_AIRFLOW_PATCH
potiuk closed pull request #27546: Make sure pach_all for gevent worker happens
before everything else
URL: https://github.com/apache/airflow/pull/27546
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
potiuk commented on PR #27546:
URL: https://github.com/apache/airflow/pull/27546#issuecomment-1345305081
This was a dead-end
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
potiuk opened a new pull request, #28283:
URL: https://github.com/apache/airflow/pull/28283
Gevent needs to monkeypatch a number of system libraries as soon as possible
when Python interpreter starts, in order to avoid other libraries
monkey-patching them before. We should do it before any
boring-cyborg[bot] commented on PR #28282:
URL: https://github.com/apache/airflow/pull/28282#issuecomment-1345303587
Congratulations on your first Pull Request and welcome to the Apache Airflow
community! If you have any issues or are unsure about any anything please check
our Contribution
swapz-z opened a new pull request, #28282:
URL: https://github.com/apache/airflow/pull/28282
References issue [#27979](https://github.com/apache/airflow/issues/27979)
`EmrAddStepsOperator` doesn't kill the step on EMR when a clear is called on
UI. Unlike other operators where process
potiuk commented on issue #28120:
URL: https://github.com/apache/airflow/issues/28120#issuecomment-1345299465
Interesting and needs some more thorough look by someone who has Celery
knowledge/experience.
--
This is an automated message from the Apache Git Service.
To respond to the messag
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
from 886d9bd6c9 Speed up most Users/Role CLI commands (#28259)
add 3b203bcb67 More robust cleanup of executors in
test
potiuk merged PR #28281:
URL: https://github.com/apache/airflow/pull/28281
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.a
Taragolis commented on PR #28257:
URL: https://github.com/apache/airflow/pull/28257#issuecomment-1345282264
And... it doesn't help
https://github.com/apache/airflow/actions/runs/3664583959/jobs/6195181628 😠ðŸ˜
```console
Creating airflow-test-integration_pinot_1 ...
Creatin
potiuk closed pull request #25094: Initial POC of GRPC Internal API
URL: https://github.com/apache/airflow/pull/25094
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubsc
potiuk commented on PR #25094:
URL: https://github.com/apache/airflow/pull/25094#issuecomment-1345278654
Closing as the founding PR for AIP-44 was already merged #27892
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
potiuk commented on PR #28248:
URL: https://github.com/apache/airflow/pull/28248#issuecomment-1345278340
I will run a few time with this one and see if it will start triggering
problems (then I will also run it few times with Public Runners).
--
This is an automated message from the Apach
potiuk commented on PR #28254:
URL: https://github.com/apache/airflow/pull/28254#issuecomment-1345277559
This one wil (finally) cut down the time for static checks by 3 minutes for
all PRs when merged.
--
This is an automated message from the Apache Git Service.
To respond to the message,
potiuk commented on PR #28281:
URL: https://github.com/apache/airflow/pull/28281#issuecomment-1345277212
BTW. I found one case where the test did not have "task_done()" run because
of mocking and that would even stop K8S executor from being killed. should be
fixed now.
--
This is an auto
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
from 5f54009872 AWS EC2 system test: wait for instance to exist before
launching it (#28274)
add 886d9bd6c9 Speed up m
potiuk merged PR #28259:
URL: https://github.com/apache/airflow/pull/28259
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.a
potiuk commented on issue #28275:
URL: https://github.com/apache/airflow/issues/28275#issuecomment-1345263652
> As option we could try to run each quarantined test in separate process by
[pytest-forked](https://github.com/pytest-dev/pytest-forked)
Yep. We have just a few quarantined t
potiuk commented on issue #16364:
URL: https://github.com/apache/airflow/issues/16364#issuecomment-1345262802
> Airflow version 2.5.0 - Facing ssh command timed out for the long-running
tasks with SSHOperator connecting & executing java scripts in remote server
@jayam26 Not sure what
jayam26 commented on issue #16364:
URL: https://github.com/apache/airflow/issues/16364#issuecomment-1345259289
Airflow version 2.5.0 - Facing ssh command timed out for the long-running
tasks with SSHOperator connecting & executing java scripts in remote server
--
This is an automated m
pierrejeambrun commented on issue #28276:
URL: https://github.com/apache/airflow/issues/28276#issuecomment-1345255738
I think you resumed pretty well the options we have.
I naively went for 1) in a quickfix
https://github.com/apache/airflow/pull/28277 for `is_local`. (I would be glad
eladkal commented on PR #28123:
URL: https://github.com/apache/airflow/pull/28123#issuecomment-1345250857
@RishuGuru can you please add unit tests to cover this ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
Taragolis commented on issue #28272:
URL: https://github.com/apache/airflow/issues/28272#issuecomment-1345242904
Assigned you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
Taragolis commented on code in PR #28128:
URL: https://github.com/apache/airflow/pull/28128#discussion_r1045070732
##
airflow/jobs/scheduler_job.py:
##
@@ -880,13 +880,24 @@ def _run_scheduler_loop(self) -> None:
# is finished to avoid concurrent access to t
Taragolis commented on issue #28275:
URL: https://github.com/apache/airflow/issues/28275#issuecomment-1345236050
As option we could try to run each quarantined test in separate process by
[pytest-forked](https://github.com/pytest-dev/pytest-forked)
--
This is an automated message from the
potiuk commented on PR #28128:
URL: https://github.com/apache/airflow/pull/28128#issuecomment-1345235142
Static checks :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
T
potiuk commented on PR #28259:
URL: https://github.com/apache/airflow/pull/28259#issuecomment-1345234968
One more "cached_app()" to remove :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the s
Taragolis commented on issue #28275:
URL: https://github.com/apache/airflow/issues/28275#issuecomment-1345206538
Also some kind of side effect might exeist, especially in this code
https://github.com/apache/airflow/blob/5f54009872b04c389899d9690d23e2a2c4b3cffd/tests/core/test_impersonatio
Taragolis commented on code in PR #28279:
URL: https://github.com/apache/airflow/pull/28279#discussion_r1045040610
##
airflow/providers/amazon/aws/hooks/ecr.py:
##
@@ -0,0 +1,100 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agre
potiuk commented on issue #11514:
URL: https://github.com/apache/airflow/issues/11514#issuecomment-1345201816
Hard to say without seeing any logs or evidences of what happened and
knowing your deployment.
--
This is an automated message from the Apache Git Service.
To respond to the messa
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
from c739a6a087 KubernetesExecutor multi_namespace_mode can use namespace
list to avoid requiring cluster role (#28047)
potiuk merged PR #28274:
URL: https://github.com/apache/airflow/pull/28274
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.a
Taragolis commented on code in PR #28279:
URL: https://github.com/apache/airflow/pull/28279#discussion_r1045036909
##
airflow/providers/amazon/aws/hooks/ecr.py:
##
@@ -0,0 +1,100 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agre
potiuk commented on code in PR #28279:
URL: https://github.com/apache/airflow/pull/28279#discussion_r1045031787
##
airflow/providers/amazon/aws/hooks/ecr.py:
##
@@ -0,0 +1,100 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreeme
potiuk commented on PR #28047:
URL: https://github.com/apache/airflow/pull/28047#issuecomment-1345190002
Actually I think we need a bit more protection, otherwise the same situation
happens if for any reason those asserts will start to raise exceptions.
try/finally and making sure that we a
potiuk opened a new pull request, #28281:
URL: https://github.com/apache/airflow/pull/28281
As a follow up after #28047, this PR will make the test cleanup more robust
and resilient to any errors that might have caused kubernetes_executors left
behind.
wrapping start()/end() in try/f
55 matches
Mail list logo
