reuvenlax commented on issue #27848: URL: https://github.com/apache/beam/issues/27848#issuecomment-1666640122
https://github.com/apache/beam/issues/27848 is a POC PR. Note that validating from the schema is a bit tricky. We can't simply convert the BQ schema to a proto descriptor and validate it matches, because the mapping is not 1:1. There are multiple valid proto descriptors that might all map to the same BQ schema. I would be inclined to not implement autoUpdateSchema for now - TableRows are naturally more dynamic, and a better fit for this (though we could in theory use the proto's unknown fieldset). I would also not implement automatic CDC here. The user can always do it themselves by adding the fields to their proto. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
