potiuk commented on PR #35097:
URL: https://github.com/apache/airflow/pull/35097#issuecomment-1879020534
Woohoo!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
potiuk merged PR #35097:
URL: https://github.com/apache/airflow/pull/35097
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail:
potiuk commented on PR #35097:
URL: https://github.com/apache/airflow/pull/35097#issuecomment-1874970706
Docs building issues :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
RNHTTR opened a new pull request, #35097:
URL: https://github.com/apache/airflow/pull/35097
I'm not sure the numpy example is still accurate. I tested out importing
numpy locally and it was more or less instantaneous. I think this can cause
some confusion implying that no imports should be
github-actions[bot] closed pull request #35097: Replace numpy example with
practical exercise demonstrating top-level code
URL: https://github.com/apache/airflow/pull/35097
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
github-actions[bot] commented on PR #35097:
URL: https://github.com/apache/airflow/pull/35097#issuecomment-1846298315
This pull request has been automatically marked as stale because it has not
had recent activity. It will be closed in 5 days if no further activity occurs.
Thank you for
potiuk commented on PR #35097:
URL: https://github.com/apache/airflow/pull/35097#issuecomment-1774812370
> FWIW I’ve been wondering if it’s worthwhile to implement some magic in
Dagprocessor to automatically move imports inside task functions so people can
write DAG files “normally” but
uranusjr commented on PR #35097:
URL: https://github.com/apache/airflow/pull/35097#issuecomment-1774724323
FWIW I’ve been wondering if it’s worthwhile to implement some magic in
Dagprocessor to automatically move imports inside task functions so people can
write DAG files “normally” but
potiuk commented on PR #35097:
URL: https://github.com/apache/airflow/pull/35097#issuecomment-1773816055
Yeah but it does matter and I think we should explain it to educate the
users. Similarly as the user who asked the question about PEP8 conflicting with
our advice, they might not
RNHTTR commented on PR #35097:
URL: https://github.com/apache/airflow/pull/35097#issuecomment-1773804072
I did add this blurb about imports:
> Note that import statements also count as top-level code. So, if you have
an import statement that takes a long time or the imported module
potiuk commented on PR #35097:
URL: https://github.com/apache/airflow/pull/35097#issuecomment-1773764486
One thing worth mentioning that even 0.3 s is pretty impactful. All our DAGs
are parsed in DAG file process in a separately forked processes - so if such
'pandas' import happens at top
potiuk commented on PR #35097:
URL: https://github.com/apache/airflow/pull/35097#issuecomment-1773761513
I like the idea of mentioning and being explicit about expensive APIs but I
think there is a value in mentioning the imports, because not many people are
aware how big of an impact
RNHTTR opened a new pull request, #35097:
URL: https://github.com/apache/airflow/pull/35097
I'm not sure the numpy example is still accurate. I tested out importing
numpy locally and it was more or less instantaneous. I think this can cause
some confusion implying that no imports should be
13 matches
Mail list logo
