Thanks Ajantha for digging up the thread on this discussion. I think the important takeaway here is understanding what Iceberg is trying to solve with views, and what it isn't.
We know that engines have different sql dialects and semantics. Iceberg views enable exposing metadata in a way that multiple engines can consume in their native dialect (the representation, not the semantics). However, there is no intention to automate the translation or ensure consistency across representations. This is an important detail that the user must handle. Translation frameworks or manual verification can be used, but achieving consistent behavior is complicated and outside the project's scope. If we think we need more convenient ways to update the syntax, that's worth exploring. However, coordinating representations and verifying correctness remains the responsibility of the view creator (via API or natively through the engine). It's easy to get this wrong, but the utility of views outweighs the burden we place on users. I see 'ALTER VIEW' support as a way to make defining the views more convenient (than, say, using PyIceberg which can't validate the syntax of other engines). It allows the native engine to parse, resolve all references, and ensure the view can be executed, but notably does nothing to ensure the results are consistent with any other representation. -Dan On Mon, Sep 7, 2026 at 11:47 PM Ajantha Bhat <[email protected]> wrote: > This discussion feels very similar to the earlier thread that I opened on > view interoperability a long time ago. > > One of the takeaways there was that we wanted to avoid each engine > manually implementing or maintaining dialect translations. Instead, the > preference was to have a centralized translation layer (at the time we > discussed exposing that through PyIceberg) rather than pushing that > responsibility into every engine. Sharing the previous discussion for > context, as it covers many of the same trade-offs: > https://lists.apache.org/thread/k6szpr5smyrh37sy563xpgjor4g6pr81 > <https://lists.apache.org/thread/k6szpr5smyrh37sy563xpgjor4g6pr81?utm_source=chatgpt.com> > > - Ajantha > > On Tue, Sep 8, 2026 at 12:09 PM Péter Váry <[email protected]> > wrote: > >> In my opinion, the ideal experience is for users to define both SQL >> statements when creating the view. This avoids any period during which the >> view is only partially defined and allows the entire workflow to be handled >> through a single authoring tool. >> >> That said, the Iceberg Java API is available, and engines are free to >> choose which capabilities they support and how they expose them to users. >> >> Talat Uyarer <[email protected]> ezt írta (időpont: 2026. szept. 7., H, >> 17:18): >> >>> Hi Péter, >>> >>> I think a SQL-only workflow is closer than it looks. a Spark user >>> doesn't need Spark to parse the Flink SQL, they need to read it, and >>> showing another engine's SQL as text is engine-agnostic. What's missing is >>> presentation, not translation. >>> >>> The workflow I'd propose: >>> >>> 1. Flink: CREATE VIEW v AS SELECT ... — version 1, with a `flink` >>> representation. >>> 2. Spark user: DESCRIBE EXTENDED VIEW— engines should display *all* >>> representations with their dialect labels. The strings are already in the >>> metadata, so this is purely a display convention, no spec change. This >>> answers "how can the Spark user know the Flink SQL". >>> 3. The user reads the Flink SQL and authors the Spark equivalent. >>> 4. Spark: ALTER VIEW v AS SELECT ... — upserts the `spark` >>> representation and carries the `flink` one forward unchanged. Issuing that >>> statement *is* the consistency assertion — the "user authority" you >>> described, made concrete. The version summary already records engine >>> name/version, so we get per-version provenance for free. >>> 5. On read, each engine executes exactly its own dialect. >>> >>> The rule that makes this flag-free is, I think, just a crisper version >>> of what Dan >>> said: >>> >>> ALTER VIEW ... AS = update/add my engine's representation, >>> preserve the rest >>> CREATE OR REPLACE VIEW = redefine the view; only my representation >>> remains >>> >>> Two intents, two existing syntaxes, no properties, no new grammar. >>> >>> When someone genuinely changes the view's meaning with CREATE OR >>> REPLACE, the other representations are gone, the next read from another >>> engine fails fast, and that engine's user re-adds their dialect via steps >>> 2–4. The failure is the signal, no cross-engine coordination protocol >>> needed. >>> >>> That only holds if engines actually fail fast, which is Alex's fallback >>> point: today sqlFor() silently returns the "closest" representation, which >>> is exactly how an engine ends up executing another engine's SQL. That >>> leniency was defensible while appending a dialect was impossible; with this >>> workflow it loses its justification. #17874 adds an opt-in >>> view-dialect-strict to Flink as a first step, and ideally the error message >>> itself teaches the workflow: "View v has no 'flink' >>> representation (found: spark). Use SHOW CREATE VIEW to see the existing >>> SQL and ALTER VIEW to add a flink representation." >>> >>> Talat >>> >>> >>> On 2026/09/07 14:38:10 Péter Váry wrote: >>> > Do I understand correctly that the expected behavior for the user if >>> they >>> > want to create a view with Flink and Spark representations is: >>> > >>> > 1. CREATE VIEW my_view AS SELECT ... - in Flink >>> > 2. ALTER VIEW my_view AS SELECT ... - in Spark >>> > >>> > How can a Spark user know the Flink SQL the user used to create the >>> view? >>> > >>> > I'm trying to put together a workflow for creating interoperable views >>> and >>> > using SQL doesn't seem viable, so as Ryan mentioned the only option is >>> > Iceberg API for now. >>> > >>> > >>> > Péter Váry <[email protected]> ezt írta (időpont: 2026. >>> szept. >>> > 5., Szo, 9:16): >>> > >>> > > Thanks Alex for bringing this up. >>> > > >>> > > We had a very similar discussion with Guo when he wanted to add a >>> Flink >>> > > representation to a view: >>> https://github.com/apache/iceberg/issues/15296. >>> > > Talat also has a fresh PR for this issue in Flink: >>> > > https://github.com/apache/iceberg/pull/17874 >>> > > >>> > > My main concern is consistency. How can Flink be sure that the Flink >>> SQL >>> > > returns the same results as the already existing Spark SQL. >>> > > In the end we accept the users authority to state that the Flink SQL >>> is >>> > > consistent with the existing Spark SQL. This means we accept that >>> they are >>> > > able to parse both, and make decisions. >>> > > I do think we should allow a user to update one or more of the SQLs >>> > > without needing extra flags. >>> > > >>> > > >>> > > >>> > > On Sat, Sep 5, 2026, 04:43 Prashant Singh <[email protected]> >>> > > wrote: >>> > > >>> > >> Thanks for starting the thread Alex, >>> > >> >>> > >> I agree with Dan, an engine should only update / add their own >>> dialect >>> > >> and not make guarantees about other engine dialects without passing >>> through >>> > >> the other engine's compiler. >>> > >> I would recommend ALTER too, I know engines such as starrocks have >>> came >>> > >> up with grammar [1] to be more specific that one is adding / >>> modifying the >>> > >> dialect, given the dialect is of starrocks, that could be an option >>> : >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> *ALTER VIEW [<catalog>.<database>.]<view_name>( <column_name> [, >>> > >> <column_name>]){ ADD | MODIFY } DIALECT<query_statement>* >>> > >> >>> > >> >>> > >> [1] >>> > >> >>> https://docs.starrocks.io/docs/data_source/catalog/iceberg/DDL/#alter-view-to-update-starrocks-dialect >>> > >> >>> > >> Best, >>> > >> Prashant Singh >>> > >> >>> > >> On Fri, Sep 4, 2026 at 4:33 PM Daniel Weeks <[email protected]> >>> wrote: >>> > >> >>> > >>> I feel like if we want to support updating/adding a representation >>> for a >>> > >>> specific engine, the right way is to allow 'ALTER VIEW' to update >>> or add >>> > >>> the current engine's representation. >>> > >>> >>> > >>> I don't think we can expect an engine to add or update other >>> engines' >>> > >>> representations because that would potentially require parsing a >>> different >>> > >>> SQL syntax than is native to the engine. >>> > >>> >>> > >>> For the example you gave where there's an existing representation >>> in >>> > >>> Trino, then using `ALTER VIEW . . .` would just set the spark >>> > >>> representation (as opposed to a `REPLACE` or `CREATE` that would >>> create a >>> > >>> new representation with only the spake dialect). >>> > >>> >>> > >>> I don't like the idea of adding properties to toggle behavior >>> though as >>> > >>> that makes something declarative subject to side-effects of the >>> > >>> configuration/session. >>> > >>> >>> > >>> -Dan >>> > >>> >>> > >>> On Fri, Sep 4, 2026 at 3:37 PM Alex Stephen via dev < >>> > >>> [email protected]> wrote: >>> > >>> >>> > >>>> I'm not sure if `CREATE VIEW` would be the correct choice, as >>> much as >>> > >>>> `ALTER VIEW`. The user can see that a view exists through `SHOW >>> > >>>> VIEWS` (which just calls the underlying List Views API), so they >>> know that >>> > >>>> they don't need to create a view as much as alter it. >>> > >>>> >>> > >>>> The Java implementation >>> > >>>> < >>> https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/BaseView.java#L114-L130> >>> surfaces >>> > >>>> the first representation when a matching one doesn't exist. This >>> assumes >>> > >>>> representations are more-or-less compatible between engines, >>> which isn't >>> > >>>> the case. >>> > >>>> >>> > >>>> I'm concerned about the user journey where a user attempts to >>> query a >>> > >>>> view and receives an error message stating they cannot query it >>> due to a >>> > >>>> wrong engine or invalid SQL syntax (because their engine is using >>> the wrong >>> > >>>> representation). At this point, their only current recourse is to >>> use the >>> > >>>> REST endpoints directly to add their new representation. >>> > >>>> >>> > >>>> On Fri, Sep 4, 2026 at 3:01 PM Ryan Blue <[email protected]> wrote: >>> > >>>> >>> > >>>>> I think the intent was for this to happen through API >>> integration. >>> > >>>>> What is the user interaction that you're trying to achieve? Do >>> you want >>> > >>>>> each `CREATE VIEW` call to automatically append a new >>> representation? >>> > >>>>> >>> > >>>>> On Fri, Sep 4, 2026 at 2:32 PM Alex Stephen via dev < >>> > >>>>> [email protected]> wrote: >>> > >>>>> >>> > >>>>>> Hello, >>> > >>>>>> >>> > >>>>>> The view spec allows creating a single view with multiple >>> > >>>>>> engine-specific representations. This allows a single view to >>> be queried by >>> > >>>>>> multiple engines, each using its own specific SQL dialect. >>> > >>>>>> >>> > >>>>>> In practice, this is difficult to achieve. Users can use the >>> > >>>>>> `replace.drop-dialect.allowed` view property to allow the >>> replacing of an >>> > >>>>>> existing representation. However, there's no way to append a >>> representation >>> > >>>>>> (by creating a new ViewVersion with the existing >>> representations and a new >>> > >>>>>> one). This means that a direct REST API call is the only way to >>> create a >>> > >>>>>> View with multiple representations. >>> > >>>>>> >>> > >>>>>> I'd like to propose the creation of a new view property called >>> > >>>>>> `replace.append-dialect.allowed` that allows users to append a >>> new >>> > >>>>>> representation to an existing view. >>> > >>>>>> >>> > >>>>>> PR: https://github.com/apache/iceberg/pull/17930 >>> > >>>>>> >>> > >>>>>> Please take a look. I'd love to hear from the community whether >>> there >>> > >>>>>> is value in this use case. >>> > >>>>>> >>> > >>>>>> Thanks! >>> > >>>>>> >>> > >>>>>> -- Alex Stephen >>> > >>>>>> >>> > >>>>> >>> > >>> >>
