Hey Alex,
Thank you for your interest in the thread, and appreciate your inputs !
It seems like there is a misunderstanding about the thread, my intention
was just to introduce these catalog evaluated functions or *variables*,
which are evaluated based on callers context, and are replaced inline, t
Hi all,
Catching up quite late with this discussion thread, my apologies.
Trying to sum up my impressions so far:
We should definitely avoid parsing SQL and altering view definitions.
Instead it seems that Eric's proposal of defining FGAC policies with a
clear syntax and spec is a much more robus
Hi guys
About FGAC, I think we have to be more pragmatic.
We can identify three parts in FGAC:
- column masking
- column filtering
- row filtering
I believe the easiest to start with is column masking.
Let's start with column masking. The cinematic would be:
1. The query engine calls the catalo
In terms of achieving this I was thinking of this as the following as well
:
1. role based column filtering as -> This should be achievable, without a
lot of lift shift.
2. simple row based filters
3. column mask and row mask (when Iceberg UDF are established)
And agree we just need to store the ma
It seems to me that the *easiest* to start with would be role-based column
filtering. There are no functions to grapple with, no dialect differences.
You simply grab the list of columns that a given principal role has access
to according to the FGAC policy attached to a given table.
--EM
> Let's work together on a proper design.
For sure ! will be reaching out soon for this !
Best,
Prashant Singh
As I mentioned in my previous reply:
"FGAC is a very complex topic. The right way would be to have a holistic
design and agreed-on approach. That does take time."
The proposed approach changes the observed behavior. It makes it
impossible to change the view later on. Plus the other concerns I
Hey Robert,
I believe you are quoting Iceberg view spec :
https://iceberg.apache.org/view-spec/#versions
1. All representations for a version should express the same underlying
definition (This holds true )
2. Immutable View versions : if the concern is that we are using the same
view version, we
Hi Prashanth,
I looked at the use cases you are trying to solve. Would having RBAC on the
view solve them?
Taking your example,
CREATE VIEW dynamic_vw ASSELECT *FROM ns1.layer1_tableWHERE
is_principal_role('ANALYST');
Can't this be solved by simply having the view as,
CREATE VIEW dynamic_vw AS
Hey Keith,
For this case it does, but if you see what if i want to define something
like this
CREATE VIEW dynamic_vw AS SELECT *FROM ns1.layer1_table WHERE
is_principal_role('ANALYST')
OR IS_MEMBER() <- Random UDF or JOIN;
Now if i don't have access to view since my principal is ANALYST i will
Ah, I misunderstood that this applies only to views generated based on an
FGAC policy. On this point I agree -- we probably don't want to change VIEW
definitions, or at least we need to be very cautious about doing so. Let's
not put this on the critical path for FGAC views.
--EM
On Tue, May 20, 2
This proposal _does_ change the view definition - it returns a
_different_ representation than the one that has been stored before.
This is a change that breaks the contract of the specification and it
changes the observed behavior.
FGAC is a very complex topic. The right way would be to have
I wouldn’t say that Polaris is changing a view definition, but per my
understanding Polaris is actually generating a view based on a Policy.
We will need a way for Polaris to embed some information into these views.
I don’t think this is a P0 to make FGAC work, and I don’t think this
necessarily n
I don't think that Polaris should change any view definition in any way,
but this is what the proposed approach does.
The approach breaks the contract (behavior defined by the specification)
and in turn the observed behavior, absolute no go's.
Practically speaking, the approach is blindly cha
Hey Ajanta,
Thank you so much for your feedback ! But my intention is different, what I
want to do is to evaluate the identity related context at catalog end and
not let it evaluate at the engine end.
> It seems to me that what you are trying to do is to store scalar UDF in
the polaris catalog.
Hi Prashant,
It seems to me that what you are trying to do is to store scalar UDF in the
polaris catalog.
Iceberg is the best place to standardize this and store it as Iceberg UDF
(not Polaris).
Regarding the SQL syntax interoperability, we still haven't
implemented this at Iceberg.
Since UDF can
Hey JB,
Thank you so much for the feedback, I would like to convince you, as to
what my thought process is, when i propose this :
> not do query engine work, but more interact with any query engines for
ex: TMS
I agree with this in principle, and we should specially not involve any
compute (for
Hi Prashant
Thanks for the proposal.
I understand the purpose (about FGAC which is something we plan to
work on), but I'm not sure if it's a good approach with this kind of
SQL functions.
Polaris, as a catalog, should:
1. not do query engine work, but more interact with any query engines
(same di
Hey Robert,
Thank you for your honest feedbacks, please let me try answering your
concerns :
> There are tons of SQL dialects out there, each requires its own fully
implemented lexer/parser/interpreter
That's true and we are not interpreting it either, we are just replacing
the sql text wherever
I'm brutally honest here:
I think we should really stay away from interpreting SQL or any other
kind of (view) definition in Polaris. There are tons of SQL dialects out
there, each requires its own fully implemented lexer/parser/interpreter
- plus views-in-views-in-views-in-views... constructs
Yes that's true, Engines will be the one executing the view, the idea is
not to break that part but the intention is that we don't want engines to
resolve the identity
it is something the catalog should do. For ex engine might be running with
its own user / groups etc which would not mean anything
Maybe I'm missing something, but aren't the engines the ones actually
interpreting/executing the views, not Polaris?
On Mon, May 19, 2025 at 10:27 AM Prashant Singh
wrote:
> Hi everyone,
>
> I’d like to propose adding *context-aware functions* to Apache Polaris so
> that view definitions can res
22 matches
Mail list logo
