gemini-code-assist[bot] commented on code in PR #35737:
URL: https://github.com/apache/beam/pull/35737#discussion_r2241764205
##########
sdks/python/apache_beam/io/external/xlang_bigqueryio_it_test.py:
##########
@@ -96,17 +99,19 @@ class BigQueryXlangStorageWriteIT(unittest.TestCase):
"str": "d",
"bool": False,
"bytes": b'd',
- "timestamp": Timestamp(4000, 400)
+ "timestamp": Timestamp(4000, 400),
+ "event_time": Timestamp(1722383999)
}
]
ALL_TYPES_SCHEMA = (
"int:INTEGER,float:FLOAT,numeric:NUMERIC,str:STRING,"
- "bool:BOOLEAN,bytes:BYTES,timestamp:TIMESTAMP")
+ "bool:BOOLEAN,bytes:BYTES,timestamp:TIMESTAMP,event_time:TIMESTAMP")
def setUp(self):
self.test_pipeline = TestPipeline(is_integration_test=True)
self.args = self.test_pipeline.get_full_options_as_args()
- self.project = self.test_pipeline.get_option('project')
+ # self.project = self.test_pipeline.get_option('project')
+ self.project = "tanusharmaa"
Review Comment:
Hardcoding the project name will cause this test to fail for other
developers and in the CI environment. Please revert this to use the project
from the pipeline options.
```suggestion
self.project = self.test_pipeline.get_option('project')
```
##########
sdks/python/apache_beam/io/gcp/bigquery.py:
##########
@@ -2702,13 +2702,17 @@ def expand(self, input):
table = StorageWriteToBigQuery.DYNAMIC_DESTINATIONS
clustering_fields = []
+ time_partitioning_config= {}
Review Comment:
Initializing `time_partitioning_config` to an empty dictionary `{}` can lead
to a runtime error. If `self.additional_bq_parameters` is not provided,
`time_partitioning_config` will remain `{}`. This empty map will be passed to
the Java transform, which will likely interpret it as a non-null configuration
object with all fields null. This will cause an `IllegalArgumentException` in
`PortableBigQueryDestinations` because the partitioning `type` will be missing.
To fix this, please initialize `time_partitioning_config` to `None`.
```suggestion
time_partitioning_config = None
```
##########
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/providers/BigQueryManagedIT.java:
##########
@@ -73,20 +73,27 @@ public class BigQueryManagedIT {
Schema.Field.of("number", Schema.FieldType.INT64),
Schema.Field.of("dest", Schema.FieldType.INT64));
+ // Schema.Field.of("field_date", Schema.FieldType.DATETIME)
Review Comment:
This commented-out code, and the one on line 81, seem to be for debugging or
future work. Please remove them before merging.
##########
website/www/site/content/en/documentation/io/managed-io.md:
##########
@@ -811,6 +811,18 @@ For more information on table properties, please visit
https://iceberg.apache.or
Determines how often to 'commit' progress into BigQuery. Default is
every 5 seconds.
</td>
</tr>
+ <tr>
+ <td>
+ Time Partitiong
Review Comment:
There's a typo here. It should be 'Partitioning'.
```suggestion
Time Partitioning
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
