Do I understand correctly that the expected behavior for the user if they want to create a view with Flink and Spark representations is:
1. CREATE VIEW my_view AS SELECT ... - in Flink 2. ALTER VIEW my_view AS SELECT ... - in Spark How can a Spark user know the Flink SQL the user used to create the view? I'm trying to put together a workflow for creating interoperable views and using SQL doesn't seem viable, so as Ryan mentioned the only option is Iceberg API for now. Péter Váry <[email protected]> ezt írta (időpont: 2026. szept. 5., Szo, 9:16): > Thanks Alex for bringing this up. > > We had a very similar discussion with Guo when he wanted to add a Flink > representation to a view: https://github.com/apache/iceberg/issues/15296. > Talat also has a fresh PR for this issue in Flink: > https://github.com/apache/iceberg/pull/17874 > > My main concern is consistency. How can Flink be sure that the Flink SQL > returns the same results as the already existing Spark SQL. > In the end we accept the users authority to state that the Flink SQL is > consistent with the existing Spark SQL. This means we accept that they are > able to parse both, and make decisions. > I do think we should allow a user to update one or more of the SQLs > without needing extra flags. > > > > On Sat, Sep 5, 2026, 04:43 Prashant Singh <[email protected]> > wrote: > >> Thanks for starting the thread Alex, >> >> I agree with Dan, an engine should only update / add their own dialect >> and not make guarantees about other engine dialects without passing through >> the other engine's compiler. >> I would recommend ALTER too, I know engines such as starrocks have came >> up with grammar [1] to be more specific that one is adding / modifying the >> dialect, given the dialect is of starrocks, that could be an option : >> >> >> >> >> >> >> *ALTER VIEW [<catalog>.<database>.]<view_name>( <column_name> [, >> <column_name>]){ ADD | MODIFY } DIALECT<query_statement>* >> >> >> [1] >> https://docs.starrocks.io/docs/data_source/catalog/iceberg/DDL/#alter-view-to-update-starrocks-dialect >> >> Best, >> Prashant Singh >> >> On Fri, Sep 4, 2026 at 4:33 PM Daniel Weeks <[email protected]> wrote: >> >>> I feel like if we want to support updating/adding a representation for a >>> specific engine, the right way is to allow 'ALTER VIEW' to update or add >>> the current engine's representation. >>> >>> I don't think we can expect an engine to add or update other engines' >>> representations because that would potentially require parsing a different >>> SQL syntax than is native to the engine. >>> >>> For the example you gave where there's an existing representation in >>> Trino, then using `ALTER VIEW . . .` would just set the spark >>> representation (as opposed to a `REPLACE` or `CREATE` that would create a >>> new representation with only the spake dialect). >>> >>> I don't like the idea of adding properties to toggle behavior though as >>> that makes something declarative subject to side-effects of the >>> configuration/session. >>> >>> -Dan >>> >>> On Fri, Sep 4, 2026 at 3:37 PM Alex Stephen via dev < >>> [email protected]> wrote: >>> >>>> I'm not sure if `CREATE VIEW` would be the correct choice, as much as >>>> `ALTER VIEW`. The user can see that a view exists through `SHOW >>>> VIEWS` (which just calls the underlying List Views API), so they know that >>>> they don't need to create a view as much as alter it. >>>> >>>> The Java implementation >>>> <https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/BaseView.java#L114-L130> >>>> surfaces >>>> the first representation when a matching one doesn't exist. This assumes >>>> representations are more-or-less compatible between engines, which isn't >>>> the case. >>>> >>>> I'm concerned about the user journey where a user attempts to query a >>>> view and receives an error message stating they cannot query it due to a >>>> wrong engine or invalid SQL syntax (because their engine is using the wrong >>>> representation). At this point, their only current recourse is to use the >>>> REST endpoints directly to add their new representation. >>>> >>>> On Fri, Sep 4, 2026 at 3:01 PM Ryan Blue <[email protected]> wrote: >>>> >>>>> I think the intent was for this to happen through API integration. >>>>> What is the user interaction that you're trying to achieve? Do you want >>>>> each `CREATE VIEW` call to automatically append a new representation? >>>>> >>>>> On Fri, Sep 4, 2026 at 2:32 PM Alex Stephen via dev < >>>>> [email protected]> wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> The view spec allows creating a single view with multiple >>>>>> engine-specific representations. This allows a single view to be queried >>>>>> by >>>>>> multiple engines, each using its own specific SQL dialect. >>>>>> >>>>>> In practice, this is difficult to achieve. Users can use the >>>>>> `replace.drop-dialect.allowed` view property to allow the replacing of an >>>>>> existing representation. However, there's no way to append a >>>>>> representation >>>>>> (by creating a new ViewVersion with the existing representations and a >>>>>> new >>>>>> one). This means that a direct REST API call is the only way to create a >>>>>> View with multiple representations. >>>>>> >>>>>> I'd like to propose the creation of a new view property called >>>>>> `replace.append-dialect.allowed` that allows users to append a new >>>>>> representation to an existing view. >>>>>> >>>>>> PR: https://github.com/apache/iceberg/pull/17930 >>>>>> >>>>>> Please take a look. I'd love to hear from the community whether there >>>>>> is value in this use case. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- Alex Stephen >>>>>> >>>>>
