snazy commented on PR #2048: URL: https://github.com/apache/polaris/pull/2048#issuecomment-3102321217
Iceberg Expressions **as of today** can IMHO *not* be used to represent a bunch of simple and none of the more complex use cases. AFAIU it is *not* possible to: * replace a column value with a constant expression * credit-card masking (substr + string concatenation) * perform any kind of calculation or rounding or trimming or interval calculation (and more) I would love to use a portable standard, but as of today (as of Jan 2022 actually) Iceberg Expressions are not yet suitable for this use case. Until we have a way to express even simple, standard use cases, I strongly believe we have to have to support SQL. Which means, that **requiring** Iceberg Expressions, as proposed here, is IMHO not a viable option to satisfy the needs of our users today. As a side note I also believe that the "front end" use case (provide protection instructions) should drive the "CRUD" parts, not the other way around. It is completely fine when users only rely on Iceberg Expressions, but it would be IMHO quite a mistake to intentionally make it impossible for users to have a much more flexible way. I also do not think that UDFs are the answer to everything, despite that Iceberg UDFs are not that concrete nor even implemented. But even if Iceberg UDFs would be there, you would still have to deal with data types and casting and null-handling. None of these three aspects is currently covered by Iceberg Expressions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
