[RESULT][VOTE] AIP-61: Hybrid Execution

[Oliveira, Niko]

Hey folks!

The voting for AIP-61: Hybrid Execution 
(https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-61+Hybrid+Execution) 
was completed on February 06, 2024 PST, and I am happy announce the following 
voting result:

*Binding (+8) Votes*
Jens Scheffler
Jarek Potiuk
Amogh Desai
Dennis Ferruzzi
Sumit Maheshwari
Hussein Awala
Andrey Anshin
Ash Berlin-Taylor


*Non-binding (+7) Votes*
Aritra Basu
Wei Lee
Igor Kholopov
Ryan Hatter
Shubham Mehta
Rajesh Bishundeo
Eugen Kosteev


I would like to thank all the above who participated in this voting!
Link to the vote thread: 
https://lists.apache.org/thread/mkdlskz6tb0rbw36vglh54kfghl69kxs

Re: [VOTE] Add the ability to report slack messages that don't meet code of conduct

[Amogh Desai]

I was trying on a rough POC but it looks like I need to deploy my
application against a webserver on the internet.

I also do not have the access to create a private channel on Airflow Slack.

Thanks & Regards,
Amogh Desai


On Wed, Feb 7, 2024 at 9:43 AM Amogh Desai  wrote:

> Thank you, I will try to work on some POC soon
>
> On Tue, Feb 6, 2024 at 10:47 PM Briana Okyere
>  wrote:
>
>> Would love your help on this Amogh!
>>
>> On Mon, Feb 5, 2024 at 9:40 PM Amogh Desai 
>> wrote:
>>
>> > Let us try to summarise and see what the bot has to do. Summarising:
>> > 1. The bot should allow users to report issues - anonymously too (or
>> maybe
>> > an option)
>> > 2. These issues should get redirected to an admin channel / private
>> channel
>> > 3. Some command in the bot that could allow users to find out the
>> people in
>> > that channel so that they can be contacted
>> > privately too
>> >
>> > Havent deployed a bot ever, but would be willing to try
>> >
>> > Thanks & Regards,
>> > Amogh Desai
>> >
>> > On Tue, Feb 6, 2024 at 12:58 AM Briana Okyere
>> >  wrote:
>> >
>> > > I agree with the bot idea- does anyone on this thread have the
>> ability to
>> > > implement it?
>> > >
>> > > On Mon, Feb 5, 2024 at 8:58 AM Bishundeo, Rajeshwar
>> > >  wrote:
>> > >
>> > > > The bot idea is great and being able to be anonymous is important!
>> I'm
>> > > not
>> > > > a fan of filling out forms and that could be a deterrent for people
>> > > trying
>> > > > expressing their concerns, especially if they misinterpret the form
>> > > > structure/criteria. So def in favor of a #2 type approach with a
>> bot.
>> > > >
>> > > > -- Rajesh
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On 2024-02-05, 10:13 AM, "Jarek Potiuk" > > > > ja...@potiuk.com>> wrote:
>> > > >
>> > > >
>> > > > CAUTION: This email originated from outside of the organization. Do
>> not
>> > > > click links or open attachments unless you can confirm the sender
>> and
>> > > know
>> > > > the content is safe.
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur
>> > externe.
>> > > > Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne
>> > > pouvez
>> > > > pas confirmer l’identité de l’expéditeur et si vous n’êtes pas
>> certain
>> > > que
>> > > > le contenu ne présente aucun risque.
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > I think the bot would be best. It would be great if the reports are
>> > made
>> > > to
>> > > > a private channel (the bot that Andrey posted seems to be doing just
>> > > that)
>> > > > where it can be discussed between the admin channels and action
>> taken.
>> > > Plus
>> > > > some way of knowing who is in the admin channel and invitation to
>> > contact
>> > > > them privately in case someone feels it's not good to post to the
>> > channel
>> > > > (reasons explained before).
>> > > >
>> > > >
>> > > > J.
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Sun, Feb 4, 2024 at 1:42 AM Wei Lee > > > > > weilee...@gmail.com>> wrote:
>> > > >
>> > > >
>> > > > > I believe that anonymity is important in this case. It would be
>> great
>> > > if
>> > > > > we could have the bot suggested by Aritra, and option #2 is
>> > favorable.
>> > > > > Alternatively, I would prefer option #1. Thanks!
>> > > > >
>> > > > > Best regards,
>> > > > > Wei Lee
>> > > > >
>> > > > > > On Feb 4, 2024, at 12:37 AM, Aritra Basu <
>> aritrabasu1...@gmail.com
>> > > > >
>> > > > > wrote:
>> > > > > >
>> > > > > > I'm very much for anonymity here, so if we could have some kind
>> of
>> > > bot
>> > > > > > that'd be great. Though even without that I think the
>> combination
>> > of
>> > > > > > channel and form also works out fine for talking to a person vs
>> > > wanting
>> > > > > > anonymity.
>> > > > > >
>> > > > > > --
>> > > > > > Regards,
>> > > > > > Aritra Basu
>> > > > > >
>> > > > > > On Sat, Feb 3, 2024, 9:12 PM Andrey Anshin <
>> > andrey.ans...@taragol.is
>> > > > >
>> > > > > wrote:
>> > > > > >
>> > > > > >> I’m also for some bot/application, after quick search I’ve
>> found
>> > > this
>> > > > > one
>> > > > > >> https://github.com/slackapi/slack-reporting-tool <
>> > > > https://github.com/slackapi/slack-reporting-tool>
>> > > > > >>
>> > > > > >>
>> > > > > >> I’m not sure is it suits our requirements and it required
>> instance
>> > > > where
>> > > > > >> this code would be deployed but it worthwhile to check it,
>> > > especially
>> > > > if
>> > > > > >> someone familiar with Typescript
>> > > > > >>
>> > > > > >>> On 3 Feb 2024, at 19:19, Amogh Desai <
>> amoghdesai@gmail.com
>> > > > > wrote:
>> > > > > >>>
>> > > > > >>> I would also like to go with #2.
>> > > > > >>>
>> > > > > >>> But just to bring up, how easy / hard is it to have a slack
>> bot
>> > > that
>> > > > > >> users
>> > > > > >>> can use 

Re: [VOTE] Add the ability to report slack messages that don't meet code of conduct

[Amogh Desai]

Thank you, I will try to work on some POC soon

On Tue, Feb 6, 2024 at 10:47 PM Briana Okyere
 wrote:

> Would love your help on this Amogh!
>
> On Mon, Feb 5, 2024 at 9:40 PM Amogh Desai 
> wrote:
>
> > Let us try to summarise and see what the bot has to do. Summarising:
> > 1. The bot should allow users to report issues - anonymously too (or
> maybe
> > an option)
> > 2. These issues should get redirected to an admin channel / private
> channel
> > 3. Some command in the bot that could allow users to find out the people
> in
> > that channel so that they can be contacted
> > privately too
> >
> > Havent deployed a bot ever, but would be willing to try
> >
> > Thanks & Regards,
> > Amogh Desai
> >
> > On Tue, Feb 6, 2024 at 12:58 AM Briana Okyere
> >  wrote:
> >
> > > I agree with the bot idea- does anyone on this thread have the ability
> to
> > > implement it?
> > >
> > > On Mon, Feb 5, 2024 at 8:58 AM Bishundeo, Rajeshwar
> > >  wrote:
> > >
> > > > The bot idea is great and being able to be anonymous is important!
> I'm
> > > not
> > > > a fan of filling out forms and that could be a deterrent for people
> > > trying
> > > > expressing their concerns, especially if they misinterpret the form
> > > > structure/criteria. So def in favor of a #2 type approach with a bot.
> > > >
> > > > -- Rajesh
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On 2024-02-05, 10:13 AM, "Jarek Potiuk"  > > > ja...@potiuk.com>> wrote:
> > > >
> > > >
> > > > CAUTION: This email originated from outside of the organization. Do
> not
> > > > click links or open attachments unless you can confirm the sender and
> > > know
> > > > the content is safe.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur
> > externe.
> > > > Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne
> > > pouvez
> > > > pas confirmer l’identité de l’expéditeur et si vous n’êtes pas
> certain
> > > que
> > > > le contenu ne présente aucun risque.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > I think the bot would be best. It would be great if the reports are
> > made
> > > to
> > > > a private channel (the bot that Andrey posted seems to be doing just
> > > that)
> > > > where it can be discussed between the admin channels and action
> taken.
> > > Plus
> > > > some way of knowing who is in the admin channel and invitation to
> > contact
> > > > them privately in case someone feels it's not good to post to the
> > channel
> > > > (reasons explained before).
> > > >
> > > >
> > > > J.
> > > >
> > > >
> > > >
> > > >
> > > > On Sun, Feb 4, 2024 at 1:42 AM Wei Lee  > > > weilee...@gmail.com>> wrote:
> > > >
> > > >
> > > > > I believe that anonymity is important in this case. It would be
> great
> > > if
> > > > > we could have the bot suggested by Aritra, and option #2 is
> > favorable.
> > > > > Alternatively, I would prefer option #1. Thanks!
> > > > >
> > > > > Best regards,
> > > > > Wei Lee
> > > > >
> > > > > > On Feb 4, 2024, at 12:37 AM, Aritra Basu <
> aritrabasu1...@gmail.com
> > > > >
> > > > > wrote:
> > > > > >
> > > > > > I'm very much for anonymity here, so if we could have some kind
> of
> > > bot
> > > > > > that'd be great. Though even without that I think the combination
> > of
> > > > > > channel and form also works out fine for talking to a person vs
> > > wanting
> > > > > > anonymity.
> > > > > >
> > > > > > --
> > > > > > Regards,
> > > > > > Aritra Basu
> > > > > >
> > > > > > On Sat, Feb 3, 2024, 9:12 PM Andrey Anshin <
> > andrey.ans...@taragol.is
> > > > >
> > > > > wrote:
> > > > > >
> > > > > >> I’m also for some bot/application, after quick search I’ve found
> > > this
> > > > > one
> > > > > >> https://github.com/slackapi/slack-reporting-tool <
> > > > https://github.com/slackapi/slack-reporting-tool>
> > > > > >>
> > > > > >>
> > > > > >> I’m not sure is it suits our requirements and it required
> instance
> > > > where
> > > > > >> this code would be deployed but it worthwhile to check it,
> > > especially
> > > > if
> > > > > >> someone familiar with Typescript
> > > > > >>
> > > > > >>> On 3 Feb 2024, at 19:19, Amogh Desai  > > > > wrote:
> > > > > >>>
> > > > > >>> I would also like to go with #2.
> > > > > >>>
> > > > > >>> But just to bring up, how easy / hard is it to have a slack bot
> > > that
> > > > > >> users
> > > > > >>> can use to report
> > > > > >>> bad users or spam users. Never done it before myself, but it
> > would
> > > > be a
> > > > > >>> great interface to report issues.
> > > > > >>>
> > > > > >>> The issues would then come to few admins who can take
> regulatory
> > > > > action.
> > > > > >>> (Might be an overkill, but thinking out loud here)
> > > > > >>>
> > > > > >>> Thanks & Regards,
> > > > > >>> Amogh Desai
> > > > > >>>
> > > > >  On Sat, Feb 3, 2024 at 2:55 AM Briana Okyere
> > > > > 

Re: [VOTE] "Require conversation resolution" setting in PRs as permanent solution

[Amogh Desai]

Love the vote from @Jed Cunningham  here!

On Wed, Feb 7, 2024 at 3:05 AM Jed Cunningham 
wrote:

> -0.5
>

Re: Idea for Discussion: custom TI dependencies

[Xiaodong (XD) DENG]

Hi all,

Thanks a lot for the feedback and inputs. Greatly greatly appreciated!

Let me try to answer the questions & share my findings one by one:

To Daniel’s question: the two examples I shared earlier may not be super solid. 
I actually “made up” them, and the use case we are having in mind may be more 
complex, like dynamically deciding execution order among TIs. The whole idea is 
about to allow a customized/flexible scheduling when needed.

A common concern I see from folks are potential impact on performance. I 
totally agree with Ash to limit it only to power users who truly know what they 
are doing, and Jarek was making a great point that clear guidance/examples 
should be provided if we add this as a feature.

I also tried to follow the approach Ash shared (set `deps` in operator class 
from `task_policy`), and then I realized a few points worth sharing here:
- Adding a custom `ti_dep` into `deps` in operator class from `task_policy` 
itself would fail. It actually fails the DAG parsing, and it requires register 
the custom `ti_dep` in the Plugins. That’s when I realize actually we already 
support custom `ti_dep` in Airflow 
(https://github.com/apache/airflow/pull/22698) since 2.3.0. Without adding your 
custom `ti_dep` via Plugin properly, you will not be able to set that in the 
operator class’s `deps` from `task_policy`.
- After adding my own dummy custom ti_dep via Airflow Plugin, it’s actually 
failing the whole scheduling, and may result in “Task deadlock” for TI 
execution.

The questions I have for the current implementation are:
- There is zero documentation about the “add custom ti_deps in Airflow Plugin” 
(https://github.com/apache/airflow/pull/22698). Was that a miss or was it 
intended?
- Setting `deps` in operator class from `task_policy` may not be the solution 
that I have been looking for: it means the custom ti_deps will be evaluated 
almost everywhere, in eluding when we serialize operator info (like 
`_serialize_node`), while we may only want to evaluate it at the TI scheduling 
stage.


I would love to hear more thoughts from you. Many thanks!



XD

Re: [VOTE] "Require conversation resolution" setting in PRs as permanent solution

[Jed Cunningham]

-0.5

Re: [VOTE] Add the ability to report slack messages that don't meet code of conduct

[Briana Okyere]

Would love your help on this Amogh!

On Mon, Feb 5, 2024 at 9:40 PM Amogh Desai  wrote:

> Let us try to summarise and see what the bot has to do. Summarising:
> 1. The bot should allow users to report issues - anonymously too (or maybe
> an option)
> 2. These issues should get redirected to an admin channel / private channel
> 3. Some command in the bot that could allow users to find out the people in
> that channel so that they can be contacted
> privately too
>
> Havent deployed a bot ever, but would be willing to try
>
> Thanks & Regards,
> Amogh Desai
>
> On Tue, Feb 6, 2024 at 12:58 AM Briana Okyere
>  wrote:
>
> > I agree with the bot idea- does anyone on this thread have the ability to
> > implement it?
> >
> > On Mon, Feb 5, 2024 at 8:58 AM Bishundeo, Rajeshwar
> >  wrote:
> >
> > > The bot idea is great and being able to be anonymous is important! I'm
> > not
> > > a fan of filling out forms and that could be a deterrent for people
> > trying
> > > expressing their concerns, especially if they misinterpret the form
> > > structure/criteria. So def in favor of a #2 type approach with a bot.
> > >
> > > -- Rajesh
> > >
> > >
> > >
> > >
> > >
> > >
> > > On 2024-02-05, 10:13 AM, "Jarek Potiuk"  > > ja...@potiuk.com>> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > > click links or open attachments unless you can confirm the sender and
> > know
> > > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur
> externe.
> > > Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne
> > pouvez
> > > pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain
> > que
> > > le contenu ne présente aucun risque.
> > >
> > >
> > >
> > >
> > >
> > >
> > > I think the bot would be best. It would be great if the reports are
> made
> > to
> > > a private channel (the bot that Andrey posted seems to be doing just
> > that)
> > > where it can be discussed between the admin channels and action taken.
> > Plus
> > > some way of knowing who is in the admin channel and invitation to
> contact
> > > them privately in case someone feels it's not good to post to the
> channel
> > > (reasons explained before).
> > >
> > >
> > > J.
> > >
> > >
> > >
> > >
> > > On Sun, Feb 4, 2024 at 1:42 AM Wei Lee  > > weilee...@gmail.com>> wrote:
> > >
> > >
> > > > I believe that anonymity is important in this case. It would be great
> > if
> > > > we could have the bot suggested by Aritra, and option #2 is
> favorable.
> > > > Alternatively, I would prefer option #1. Thanks!
> > > >
> > > > Best regards,
> > > > Wei Lee
> > > >
> > > > > On Feb 4, 2024, at 12:37 AM, Aritra Basu  > > >
> > > > wrote:
> > > > >
> > > > > I'm very much for anonymity here, so if we could have some kind of
> > bot
> > > > > that'd be great. Though even without that I think the combination
> of
> > > > > channel and form also works out fine for talking to a person vs
> > wanting
> > > > > anonymity.
> > > > >
> > > > > --
> > > > > Regards,
> > > > > Aritra Basu
> > > > >
> > > > > On Sat, Feb 3, 2024, 9:12 PM Andrey Anshin <
> andrey.ans...@taragol.is
> > > >
> > > > wrote:
> > > > >
> > > > >> I’m also for some bot/application, after quick search I’ve found
> > this
> > > > one
> > > > >> https://github.com/slackapi/slack-reporting-tool <
> > > https://github.com/slackapi/slack-reporting-tool>
> > > > >>
> > > > >>
> > > > >> I’m not sure is it suits our requirements and it required instance
> > > where
> > > > >> this code would be deployed but it worthwhile to check it,
> > especially
> > > if
> > > > >> someone familiar with Typescript
> > > > >>
> > > > >>> On 3 Feb 2024, at 19:19, Amogh Desai  > > > wrote:
> > > > >>>
> > > > >>> I would also like to go with #2.
> > > > >>>
> > > > >>> But just to bring up, how easy / hard is it to have a slack bot
> > that
> > > > >> users
> > > > >>> can use to report
> > > > >>> bad users or spam users. Never done it before myself, but it
> would
> > > be a
> > > > >>> great interface to report issues.
> > > > >>>
> > > > >>> The issues would then come to few admins who can take regulatory
> > > > action.
> > > > >>> (Might be an overkill, but thinking out loud here)
> > > > >>>
> > > > >>> Thanks & Regards,
> > > > >>> Amogh Desai
> > > > >>>
> > > >  On Sat, Feb 3, 2024 at 2:55 AM Briana Okyere
> > > >   > > briana.oky...@astronomer.io.inva>lid> wrote:
> > > > 
> > > >  Hey All,
> > > > 
> > > >  I'm breaking this out into another conversation because I think
> it
> > > > >> warrants
> > > >  its own decision before we move forward.
> > > > 
> > > >  Last week, I proposed we add a code of conduct for Airflow Slack
> > and
> > > >  in-person meetups.
> > > >  Thread: <
> > > > >> 

Re: [VOTE] "Require conversation resolution" setting in PRs as permanent solution

[Hussein Awala]

+1 (binding)

On Tuesday, February 6, 2024, Ash Berlin-Taylor  wrote:

> -1
>
> On 6 February 2024 06:38:04 GMT, Bolke de Bruin  wrote:
> >-1, binding.
> >
> >
> >Sent from my iPhone
> >
> >> On 5 Feb 2024, at 21:07, Scheffler Jens (XC-AS/EAE-ADA-T) <
> jens.scheff...@de.bosch.com.invalid> wrote:
> >>
> >> +1 binding
> >>
> >> Sent from Outlook for iOS
> >> 
> >> From: Pankaj Koti 
> >> Sent: Monday, February 5, 2024 3:36:41 PM
> >> To: dev@airflow.apache.org 
> >> Subject: Re: [VOTE] "Require conversation resolution" setting in PRs as
> permanent solution
> >>
> >> +1 (binding)
> >>
> >>
> >> Best regards,
> >>
> >> *Pankaj Koti*
> >> Senior Software Engineer (Airflow OSS Engineering team)
> >> Location: Pune, Maharashtra, India
> >> Timezone: Indian Standard Time (IST)
> >> Phone: +91 9730079985
> >>
> >>
> >>> On Mon, Feb 5, 2024 at 8:05 PM Amogh Desai 
> wrote:
> >>>
> >>> +1 from me too based on my earlier points.
> >>>
> >>> Wondering if we can make it a little more rigid to address the concerns
> >>> about hiding comments
> >>> from the email thread earlier
> >>>
> >>>
> >>> Thanks & Regards,
> >>> Amogh Desai
> >>>
>  On Mon, Feb 5, 2024 at 7:47 PM Jarek Potiuk  wrote:
> 
>  Hello,
> 
>  We have been discussing and arguing and it seems that we do not have a
>  consensus of whether "require conversation resolution" is a good
> idea, as
>  promised I am casting a VOTE on it.
> 
>  The discussion is in this thread:
>  https://eur03.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Flists.apache.org%2Fthread%2Fbzg00p8yolboqgwlwfnlkdfvnn99
> 6601=05%7C02%7CJens.Scheffler%40de.bosch.com%
> 7C4d1c2ca317cb4363c81808dc2657f4fc%7C0ae51e1907c84e4bbb6d648ee584
> 10f4%7C0%7C0%7C638427406408545249%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
> MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C=
> 6AR07ScNSFOISyREVAnA1nXS7hIXERuLG8urLYp%2FY%2BI%3D=0<
> https://lists.apache.org/thread/bzg00p8yolboqgwlwfnlkdfvnn996601>
> 
>  Voting is according to https://eur03.safelinks.
> protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%
> 2Ffoundation%2Fvoting.html=05%7C02%7CJens.Scheffler%40de.bosch.com%
> 7C4d1c2ca317cb4363c81808dc2657f4fc%7C0ae51e1907c84e4bbb6d648ee584
> 10f4%7C0%7C0%7C638427406408552098%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
> MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%
> 7C%7C=%2BI9en7Hu47lq%2FbSN8U21StyCeKmT05eB24p9Vd8bRWs%3D=0<
> https://www.apache.org/foundation/voting.html>
>  (procedural).
>  Committers have binding votes.
> 
>  The voting will last for 72 hrs - till Thursday 15:30 CET.
> 
>  It gets +1 from my side.
> 
>  J.
> 
> >>>
> >
> >-
> >To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
> >For additional commands, e-mail: dev-h...@airflow.apache.org
> >
>