Re: [DISCUSS] FLIP-312: Add Yarn ACLs to Flink Containers
Thanks for the FLIP, Archit. The motivation sounds reasonable and it looks like a straightforward proposal. +1 from me. Thanks, Jiangjie (Becket) Qin On Fri, May 12, 2023 at 1:30 AM Archit Goyal wrote: > Hi all, > > I am opening this thread to discuss the proposal to support Yarn ACLs to > Flink containers which has been documented in FLIP-312 < > https://cwiki.apache.org/confluence/display/FLINK/FLIP+312%3A+Add+Yarn+ACLs+to+Flink+Containers > >. > > This FLIP mentions about providing Yarn application ACL mechanism on Flink > containers to be able to provide specific rights to users other than the > one running the Flink application job. This will restrict other users in > two ways: > > * view logs through the Resource Manager job history > * kill the application > > Please feel free to reply to this email thread and share your opinions. > > Thanks, > Archit Goyal > >
Re: [DISCUSS] FLIP-312: Add Yarn ACLs to Flink Containers
Thanks for the FLIP, Archit. +1 from me as well. This would be very useful for us and others in the community given the same issue was raised earlier as well. Regards Venkata krishnan On Fri, May 12, 2023 at 4:03 PM Becket Qin wrote: > Thanks for the FLIP, Archit. > > The motivation sounds reasonable and it looks like a straightforward > proposal. +1 from me. > > Thanks, > > Jiangjie (Becket) Qin > > On Fri, May 12, 2023 at 1:30 AM Archit Goyal > > wrote: > > > Hi all, > > > > I am opening this thread to discuss the proposal to support Yarn ACLs to > > Flink containers which has been documented in FLIP-312 < > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/FLINK/FLIP*312*3A*Add*Yarn*ACLs*to*Flink*Containers__;KyUrKysrKys!!IKRxdwAv5BmarQ!bQiA3GX9bFf-w6A9M4Aez7RSMYLdvFtjZnlrOSf6N2nQUFuDdnoJ20uujW8RPY1VbLS9P4AfpnqPmkZZOuQ$ > > >. > > > > This FLIP mentions about providing Yarn application ACL mechanism on > Flink > > containers to be able to provide specific rights to users other than the > > one running the Flink application job. This will restrict other users in > > two ways: > > > > * view logs through the Resource Manager job history > > * kill the application > > > > Please feel free to reply to this email thread and share your opinions. > > > > Thanks, > > Archit Goyal > > > > >
Re: [DISCUSS] FLIP-312: Add Yarn ACLs to Flink Containers
Thanks for creating this FLIP. This sounds like a useful feature to make the Flink applications running on YARN cluster more securely. However, I think we still miss some important parts in the FLIP. 1. Which hadoop versions this FLIP relies on? 2. We need to describe a bit more about how the YARN ACLs works. 3. Does the ACLs only apply to the logs? How about the Flink JobManager UI? Best, Yang Venkatakrishnan Sowrirajan 于2023年5月13日周六 08:12写道： > Thanks for the FLIP, Archit. > > +1 from me as well. This would be very useful for us and others in the > community given the same issue was raised earlier as well. > > Regards > Venkata krishnan > > > On Fri, May 12, 2023 at 4:03 PM Becket Qin wrote: > > > Thanks for the FLIP, Archit. > > > > The motivation sounds reasonable and it looks like a straightforward > > proposal. +1 from me. > > > > Thanks, > > > > Jiangjie (Becket) Qin > > > > On Fri, May 12, 2023 at 1:30 AM Archit Goyal > > > > > wrote: > > > > > Hi all, > > > > > > I am opening this thread to discuss the proposal to support Yarn ACLs > to > > > Flink containers which has been documented in FLIP-312 < > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/FLINK/FLIP*312*3A*Add*Yarn*ACLs*to*Flink*Containers__;KyUrKysrKys!!IKRxdwAv5BmarQ!bQiA3GX9bFf-w6A9M4Aez7RSMYLdvFtjZnlrOSf6N2nQUFuDdnoJ20uujW8RPY1VbLS9P4AfpnqPmkZZOuQ$ > > > >. > > > > > > This FLIP mentions about providing Yarn application ACL mechanism on > > Flink > > > containers to be able to provide specific rights to users other than > the > > > one running the Flink application job. This will restrict other users > in > > > two ways: > > > > > > * view logs through the Resource Manager job history > > > * kill the application > > > > > > Please feel free to reply to this email thread and share your opinions. > > > > > > Thanks, > > > Archit Goyal > > > > > > > > >
Re: [DISCUSS] FLIP-312: Add Yarn ACLs to Flink Containers
Thanks Yang for review. 1. FLIP-312 relies on Hadoop version 2.6.0 or later. 2. I have updated the FLIP and made it more descriptive. 3. ACLs apply to logs as well as permissions to kill the application. Also, in the PR we are setting ACLs for Task Manager (createTaskExecutorContext) as well as Job Manager (startAppMaster). Thanks, Archit Goyal From: Yang Wang Date: Sunday, May 21, 2023 at 9:08 PM To: dev@flink.apache.org Subject: Re: [DISCUSS] FLIP-312: Add Yarn ACLs to Flink Containers Thanks for creating this FLIP. This sounds like a useful feature to make the Flink applications running on YARN cluster more securely. However, I think we still miss some important parts in the FLIP. 1. Which hadoop versions this FLIP relies on? 2. We need to describe a bit more about how the YARN ACLs works. 3. Does the ACLs only apply to the logs? How about the Flink JobManager UI? Best, Yang Venkatakrishnan Sowrirajan 于2023年5月13日周六 08:12写道： > Thanks for the FLIP, Archit. > > +1 from me as well. This would be very useful for us and others in the > community given the same issue was raised earlier as well. > > Regards > Venkata krishnan > > > On Fri, May 12, 2023 at 4:03 PM Becket Qin wrote: > > > Thanks for the FLIP, Archit. > > > > The motivation sounds reasonable and it looks like a straightforward > > proposal. +1 from me. > > > > Thanks, > > > > Jiangjie (Becket) Qin > > > > On Fri, May 12, 2023 at 1:30 AM Archit Goyal > > > > > wrote: > > > > > Hi all, > > > > > > I am opening this thread to discuss the proposal to support Yarn ACLs > to > > > Flink containers which has been documented in FLIP-312 < > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FFLINK%2FFLIP*312*3A*Add*Yarn*ACLs*to*Flink*Containers__%3BKyUrKysrKys!!IKRxdwAv5BmarQ!bQiA3GX9bFf-w6A9M4Aez7RSMYLdvFtjZnlrOSf6N2nQUFuDdnoJ20uujW8RPY1VbLS9P4AfpnqPmkZZOuQ%24&data=05%7C01%7Cargoyal%40linkedin.com%7C0337240314fb45444f5e08db5a7a277f%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C638203252947441598%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HS6QhFdRGtX7Yp7qCzEB7kOeDyqB0ePhd%2BUy7BAPsY8%3D&reserved=0<https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/FLINK/FLIP*312*3A*Add*Yarn*ACLs*to*Flink*Containers__;KyUrKysrKys!!IKRxdwAv5BmarQ!bQiA3GX9bFf-w6A9M4Aez7RSMYLdvFtjZnlrOSf6N2nQUFuDdnoJ20uujW8RPY1VbLS9P4AfpnqPmkZZOuQ$> > > > >. > > > > > > This FLIP mentions about providing Yarn application ACL mechanism on > > Flink > > > containers to be able to provide specific rights to users other than > the > > > one running the Flink application job. This will restrict other users > in > > > two ways: > > > > > > * view logs through the Resource Manager job history > > > * kill the application > > > > > > Please feel free to reply to this email thread and share your opinions. > > > > > > Thanks, > > > Archit Goyal > > > > > > > > >
Re: [DISCUSS] FLIP-312: Add Yarn ACLs to Flink Containers
Thanks Archit Goyal for the explanation and updating the FLIP. No more concerns from my part. +1 Best, Yang Archit Goyal 于2023年5月27日周六 05:19写道： > Thanks Yang for review. > > > 1. FLIP-312 relies on Hadoop version 2.6.0 or later. > 2. I have updated the FLIP and made it more descriptive. > 3. ACLs apply to logs as well as permissions to kill the application. > Also, in the PR we are setting ACLs for Task Manager > (createTaskExecutorContext) as well as Job Manager (startAppMaster). > > Thanks, > Archit Goyal > > From: Yang Wang > Date: Sunday, May 21, 2023 at 9:08 PM > To: dev@flink.apache.org > Subject: Re: [DISCUSS] FLIP-312: Add Yarn ACLs to Flink Containers > Thanks for creating this FLIP. > > This sounds like a useful feature to make the Flink applications running on > YARN cluster more securely. > > However, I think we still miss some important parts in the FLIP. > 1. Which hadoop versions this FLIP relies on? > 2. We need to describe a bit more about how the YARN ACLs works. > 3. Does the ACLs only apply to the logs? How about the Flink JobManager UI? > > Best, > Yang > > Venkatakrishnan Sowrirajan 于2023年5月13日周六 08:12写道： > > > Thanks for the FLIP, Archit. > > > > +1 from me as well. This would be very useful for us and others in the > > community given the same issue was raised earlier as well. > > > > Regards > > Venkata krishnan > > > > > > On Fri, May 12, 2023 at 4:03 PM Becket Qin wrote: > > > > > Thanks for the FLIP, Archit. > > > > > > The motivation sounds reasonable and it looks like a straightforward > > > proposal. +1 from me. > > > > > > Thanks, > > > > > > Jiangjie (Becket) Qin > > > > > > On Fri, May 12, 2023 at 1:30 AM Archit Goyal > > > > > > > > wrote: > > > > > > > Hi all, > > > > > > > > I am opening this thread to discuss the proposal to support Yarn ACLs > > to > > > > Flink containers which has been documented in FLIP-312 < > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FFLINK%2FFLIP*312*3A*Add*Yarn*ACLs*to*Flink*Containers__%3BKyUrKysrKys!!IKRxdwAv5BmarQ!bQiA3GX9bFf-w6A9M4Aez7RSMYLdvFtjZnlrOSf6N2nQUFuDdnoJ20uujW8RPY1VbLS9P4AfpnqPmkZZOuQ%24&data=05%7C01%7Cargoyal%40linkedin.com%7C0337240314fb45444f5e08db5a7a277f%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C638203252947441598%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HS6QhFdRGtX7Yp7qCzEB7kOeDyqB0ePhd%2BUy7BAPsY8%3D&reserved=0 > < > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/FLINK/FLIP*312*3A*Add*Yarn*ACLs*to*Flink*Containers__;KyUrKysrKys!!IKRxdwAv5BmarQ!bQiA3GX9bFf-w6A9M4Aez7RSMYLdvFtjZnlrOSf6N2nQUFuDdnoJ20uujW8RPY1VbLS9P4AfpnqPmkZZOuQ$ > > > > > > >. > > > > > > > > This FLIP mentions about providing Yarn application ACL mechanism on > > > Flink > > > > containers to be able to provide specific rights to users other than > > the > > > > one running the Flink application job. This will restrict other users > > in > > > > two ways: > > > > > > > > * view logs through the Resource Manager job history > > > > * kill the application > > > > > > > > Please feel free to reply to this email thread and share your > opinions. > > > > > > > > Thanks, > > > > Archit Goyal > > > > > > > > > > > > > >
Re: [DISCUSS] FLIP-312: Add Yarn ACLs to Flink Containers
Hi All, If there are no further concerns about this FLIP, I will start a vote thread next Monday. Thanks, Archit Goyal From: Yang Wang Date: Thursday, June 1, 2023 at 12:16 AM To: dev@flink.apache.org Subject: Re: [DISCUSS] FLIP-312: Add Yarn ACLs to Flink Containers Thanks Archit Goyal for the explanation and updating the FLIP. No more concerns from my part. +1 Best, Yang Archit Goyal 于2023年5月27日周六 05:19写道： > Thanks Yang for review. > > > 1. FLIP-312 relies on Hadoop version 2.6.0 or later. > 2. I have updated the FLIP and made it more descriptive. > 3. ACLs apply to logs as well as permissions to kill the application. > Also, in the PR we are setting ACLs for Task Manager > (createTaskExecutorContext) as well as Job Manager (startAppMaster). > > Thanks, > Archit Goyal > > From: Yang Wang > Date: Sunday, May 21, 2023 at 9:08 PM > To: dev@flink.apache.org > Subject: Re: [DISCUSS] FLIP-312: Add Yarn ACLs to Flink Containers > Thanks for creating this FLIP. > > This sounds like a useful feature to make the Flink applications running on > YARN cluster more securely. > > However, I think we still miss some important parts in the FLIP. > 1. Which hadoop versions this FLIP relies on? > 2. We need to describe a bit more about how the YARN ACLs works. > 3. Does the ACLs only apply to the logs? How about the Flink JobManager UI? > > Best, > Yang > > Venkatakrishnan Sowrirajan 于2023年5月13日周六 08:12写道： > > > Thanks for the FLIP, Archit. > > > > +1 from me as well. This would be very useful for us and others in the > > community given the same issue was raised earlier as well. > > > > Regards > > Venkata krishnan > > > > > > On Fri, May 12, 2023 at 4:03 PM Becket Qin wrote: > > > > > Thanks for the FLIP, Archit. > > > > > > The motivation sounds reasonable and it looks like a straightforward > > > proposal. +1 from me. > > > > > > Thanks, > > > > > > Jiangjie (Becket) Qin > > > > > > On Fri, May 12, 2023 at 1:30 AM Archit Goyal > > > > > > > > wrote: > > > > > > > Hi all, > > > > > > > > I am opening this thread to discuss the proposal to support Yarn ACLs > > to > > > > Flink containers which has been documented in FLIP-312 < > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FFLINK%2FFLIP*312*3A*Add*Yarn*ACLs*to*Flink*Containers__%3BKyUrKysrKys!!IKRxdwAv5BmarQ!bQiA3GX9bFf-w6A9M4Aez7RSMYLdvFtjZnlrOSf6N2nQUFuDdnoJ20uujW8RPY1VbLS9P4AfpnqPmkZZOuQ%24&data=05%7C01%7Cargoyal%40linkedin.com%7C7d32480195ea40aba79408db62701f05%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C638212005931641640%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=DXb4CbIGeqO66Pgtf%2BDUOJ3YG5g6bafiJL53Im9zGPE%3D&reserved=0<https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/FLINK/FLIP*312*3A*Add*Yarn*ACLs*to*Flink*Containers__;KyUrKysrKys!!IKRxdwAv5BmarQ!bQiA3GX9bFf-w6A9M4Aez7RSMYLdvFtjZnlrOSf6N2nQUFuDdnoJ20uujW8RPY1VbLS9P4AfpnqPmkZZOuQ$> > < > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FFLINK%2FFLIP*312*3A*Add*Yarn*ACLs*to*Flink*Containers__%3BKyUrKysrKys!!IKRxdwAv5BmarQ!bQiA3GX9bFf-w6A9M4Aez7RSMYLdvFtjZnlrOSf6N2nQUFuDdnoJ20uujW8RPY1VbLS9P4AfpnqPmkZZOuQ%24&data=05%7C01%7Cargoyal%40linkedin.com%7C7d32480195ea40aba79408db62701f05%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C638212005931641640%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=DXb4CbIGeqO66Pgtf%2BDUOJ3YG5g6bafiJL53Im9zGPE%3D&reserved=0<https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/FLINK/FLIP*312*3A*Add*Yarn*ACLs*to*Flink*Containers__;KyUrKysrKys!!IKRxdwAv5BmarQ!bQiA3GX9bFf-w6A9M4Aez7RSMYLdvFtjZnlrOSf6N2nQUFuDdnoJ20uujW8RPY1VbLS9P4AfpnqPmkZZOuQ$> > > > > > > >. > > > > > > > > This FLIP mentions about providing Yarn application ACL mechanism on > > > Flink > > > > containers to be able to provide specific rights to users other than > > the > > > > one running the Flink application job. This will restrict other users > > in > > > > two ways: > > > > > > > > * view logs through the Resource Manager job history > > > > * kill the application > > > > > > > > Please feel free to reply to this email thread and share your > opinions. > > > > > > > > Thanks, > > > > Archit Goyal > > > > > > > > > > > > > >
