jrmccluskey commented on code in PR #40618:
URL: https://github.com/apache/beam/pull/40618#discussion_r4326437216
##########
sdks/python/apache_beam/io/gcp/bigquery_change_history.py:
##########
@@ -499,8 +511,7 @@ def _emit_query_ranges(
self,
start_ts: Timestamp,
end_ts: Timestamp,
- watermark_estimator: _PollWatermarkEstimator
- ) -> Iterable[TimestampedValue[_QueryRange]]:
Review Comment:
Were the typehints applied here incorrect?
##########
sdks/python/apache_beam/io/gcp/bigquery_change_history.py:
##########
@@ -1294,20 +1300,22 @@ def expand(self, pbegin: beam.pvalue.PBegin) ->
beam.PCollection:
# [start_time, now - buffer) in a single poll.
config = _PollConfig(start_time=start_time)
+ poll = beam.ParDo(
Review Comment:
Is there a reason for the refactor of defining the ParDos prior to the
pipeline definition both here and elsewhere?
##########
CHANGES.md:
##########
@@ -120,6 +120,9 @@
* (Python) DataFrame API: `agg` with a list that includes `'count'` or
`'size'` now labels those rows `count`/`size` instead of `sum`
([#40504](https://github.com/apache/beam/issues/40504)).
* (Python) `ConcatSource` no longer raises `ZeroDivisionError` when every
source bundle reports a weight of 0, e.g. bundles smaller than 1 MB
([#21316](https://github.com/apache/beam/issues/21316)).
* (Python) DataFrame API: `Series.value_counts(bins=...)` no longer always
fails with a `TypeError`
([#40503](https://github.com/apache/beam/issues/40503)).
+* (Python) `ReadBigQueryChangeHistory` no longer fails at pipeline
construction with `TypeCheckError: The transform 'ExecuteQueries' requires
PCollections of type ... _QueryRange` when pipeline type checking is enabled
(the default); a regression in 2.76.0
([#X](https://github.com/apache/beam/issues/X)).
+* (Python) `ReadBigQueryChangeHistory` with `change_function='CHANGES'` no
longer fails reading its temp table with a Storage Read API `500
InternalServerError`: the `_CHANGE_IS_FOR_UPDATE` pseudo-column is now excluded
from the query instead of being written to the temp table
([#X](https://github.com/apache/beam/issues/X)).
+* (Python) `ReadBigQueryChangeHistory` now backtick-quotes the column names it
puts in the generated CHANGES/APPENDS query (`columns`, `change_type_column`,
`change_timestamp_column`), so reserved words and names with spaces work, and
`columns=[]` selects only the pseudo-columns instead of generating invalid SQL
([#X](https://github.com/apache/beam/issues/X)).
Review Comment:
Missing issue links
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
