Hi, On further experimentation we found that the update in already running jobs not only marks the col3 as null for that primary key but it marks the col3 null for the entire table.
- SELECT * FROM wide_table; ----> (0, 1, 2, 3), (1, 11, 22, 33) - INSERT INTO stream_1 VALUES (0, 10); - SELECT * FROM wide_table; ---> (0, 10, 2, null), (1, 11, 22, null) On Mon, Jul 28, 2025 at 4:55 PM Ashish Khatkar <[email protected]> wrote: > Hi, > > I am trying to test out schema evolution with the partial update function. > Is downstream sink table schema update automatically picked up by queries > running INSERT on that table? Here is a detailed example of the problem > that I am observing. > > - CREATE TABLE stream_1 (id INT, col1 INT, PRIMARY KEY (id) NOT ENFORCED); > - CREATE TABLE stream_2 (id INT, col2 INT, PRIMARY KEY (id) NOT ENFORCED); > > - CREATE TABLE wide_table (id INT, col1 INT, col2 INT, PRIMARY KEY (id) > NOT ENFORCED) WITH ('merge-engine' = 'partial-update', 'changelog-producer' > = 'full-compaction'); > > - INSERT INTO wide_table (id, col1) SELECT id, col1 FROM stream_1; > - INSERT INTO wide_table (id, col2) SELECT id, col2 FROM stream_2; > > - INSERT INTO stream_1 VALUES (0, 1); > - INSERT INTO stream_2 VALUES (0, 2); > > - SELECT * FROM wide_table; ---> outputs 0, 1, 2 as expected > > - CREATE TABLE stream_3 (id INT, col3 INT, PRIMARY KEY (id) NOT ENFORCED); > > - ALTER TABLE wide_table ADD (col3 INT); > > - INSERT INTO stream_3 VALUES (0, 3); > > - SELECT * FROM wide_table; ---> outputs 0, 1, 2, 3 as expected > > - INSERT INTO stream_1 VALUES (0, 11); > > - SELECT * FROM wide_table; ---> outputs 0, 11, 2, <null> > > The last insert updated the value for col3 to null which I didn't expect. > If I restart the queries for col1 and col2 and then do the insert it works. > I tried using sequence groups as well and that also gives the same results. > > Thanks >
