+1, I agree here too, having iceberg expressions for row-filters and transforms for projections aka column masks seems the right way to go, especially how portable and dialect agnostic they are ! As said above, we can always model complex mask / row filters that require JOINs etc as catalog UDF and reference them in the expression and transform. Plus having this representation would be really helpful in merging the policies as well and perform some *basic* boolean simplification and constant folding specially with context resolution so that final projection list aka transforms along with the filters apply can be conveyed back,
It will be really nice to see this move forward ! Best, Prashant Singh On Mon, Jul 21, 2025 at 3:56 PM Ryan Blue <rdb...@gmail.com> wrote: > I agree with Russell. The proposal doesn't look too controversial given > previous discussions on how to support FGAC managed by the catalog. I also > agree that a more detailed proposal should use Iceberg expressions and > transforms for the row-level filters and column mask expressions, and > catalog-managed UDFs can handle more complex representations (like > Substrait mentioned in the proposal) and expressions (like EXISTS with a > sub-query). > > It would be great to see this move forward. I think it's a good approach. > > On Mon, Jul 21, 2025 at 2:17 PM Russell Spitzer <russellspit...@apache.org> > wrote: > >> I think this is a really interesting approach as we've talked about in a >> few community >> syncs as well as in some other chats. If I understand the proposal >> correctly, we are >> essentially formalizing a way to return the FGAC "protection expressions" >> from the catalog >> to a trusted engine for execution. >> >> The only aspects I'm a little more nervous about are allowing dialects >> into the api (Sql or Substrait) .. I think the UDFs being worked on can >> probably handle that sort of >> complexity when it is required and we can stick to Iceberg expressions >> and transforms (which could refer to udfs) at least for a first version of >> these end points. I believe the are already >> efforts and discussions around constraints using this approach as well as >> corresponding >> work within Spark to make these exchanges more straightforward. >> >> I think if we minimize the surface area here we can probably figure out >> this API relatively quickly. >> >> On 2025/06/18 14:47:19 Robert Stupp wrote: >> > Hi all, >> > >> > We, the authors of the proposal [1], have been thinking about support >> > for fine grained access control for quite some time and would like to >> > propose both row-level access control and column-level transformation >> > (“masking”) to the Iceberg REST catalog in an interoperable way. >> > >> > The three main drivers for the proposed approach are >> > * interoperability between catalogs and query engines >> > * securely applying the FGAC policies >> > * ability to integrate any query engines >> > >> > The proposal describes the general, high-level approach and does >> > intentionally not go into specific internal & technical details to >> focus >> > on the concept as a whole. If there is consensus on the concept >> > described in the proposal, we can start a follow-up proposal >> considering >> > all the feedback - that would include details about the REST >> > specification and the technical interaction and between catalogs and >> > query engines, as well as portable representation of the policies and >> > “protection instructions” (details what the latter is are in the >> proposal). >> > >> > We would love to get your feedback and are happy to answer any >> questions! >> > >> > Robert >> > >> > [1] Proposal document >> > >> https://docs.google.com/document/d/1A5EHXZoluvW7GtEth3GzQz6n5N-fErYLtUbf6B93Pmw >> > >> > >> > >> >
