Hi Gustavo, A data point for reason 1. On 2.3.0 the require-on-conflict check is skipped under upsert-materialize=FORCE, so a query whose upsert key differs from the sink's primary key plans under FORCE without ON CONFLICT, but fails under AUTO. My fix for that, FLINK-40924, is on release-2.4. With the default left at true, such a query will fail on 2.4.0 with no change to the SQL. With the default at false, it keeps working, and users who opt in still get the check under FORCE.
If the default changes, the "When is ON CONFLICT required?" section of insert.md (English and Chinese) and the option description both describe false as restoring the old behavior, so they need updating too. +1 (non-binding) Best, Sai Krishna Sepuri On Thu, Oct 29, 2026 at 9:01 PM Gustavo de Morais <[email protected]> wrote: > Hi everyone, > > Bumping this one. If there are no objections, I’ll take silence as > agreement and move forward in 1-2 weeks before we release 2.4. Of course, > if anyone needs more time to review, just let me know and I’m happy to > wait. > > Kind regards, > Gustavo > > On Fri, 23 Oct 2026 at 15:40, Gustavo de Morais <[email protected]> > wrote: > > > Hi everyone, > > > > I'd like to start a discussion about changing the default of > > "table.exec.sink.require-on-conflict" from "true" to "false" for 2.4. The > > ON CONFLICT functionality stays as is. This is only about whether users > > must explicitly provide the clause. > > > > Since 2.3 [1], a query fails at planning time when its upsert key differs > > from the sink's primary key and it has no ON CONFLICT clause. After > > collecting feedback from users, clients and tooling developers, I think > > making the clause opt-in would be a better default for the following > > reasons: > > > > 1. The obvious pain: upgrades break working queries. A query that ran > > fine on 2.2 can fail on 2.3 with no change to the SQL. We've seen a lot > of > > confusion and pain from users, clients and tools migrating from 2.2 to > 2.3, > > and we could avoid that for future migrations by changing the default. > > > > 2. The planner can't always prove the keys match. Upsert key > inference > > is conservative. Functions, casts or projections can drop the key even > when > > the query is correct, and the user gets an error for something the > planner > > couldn't work out. We're working on improving that, but there are still > > cases where the engine can't preserve the keys. > > > > 3. The clause only exists on INSERT INTO. CTAS, RTAS and CREATE > > MATERIALIZED TABLE have no way to set it except a session config. This > > makes the breaking change inconsistent, and users can't move a query from > > INSERT INTO to CTAS or a materialized table without falling back to that > > config. We should first make the clause available for all of them. > > > > Without the clause, Flink would deduplicate like it did before 2.3. Users > > who already added ON CONFLICT to their queries don't need to change > > anything. > > > > I want to be clear that I think the clause is very useful, and I'd still > > recommend requiring it for setups that mostly use INSERT INTO and where > > users understand deduplication and its effects. But as long as the clause > > is only available for INSERT INTO and not for every statement that writes > > to a table, requiring it by default doesn't feel right. > > > > Looking forward to hearing the community's opinion! > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-558:+Improvements+to+SinkUpsertMaterializer+and+changelog+disorder > > > > Kind regards, > > Gustavo > > >
