rworley-monster commented on a change in pull request #12389:
URL: https://github.com/apache/beam/pull/12389#discussion_r471599270
##########
File path:
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryUtils.java
##########
@@ -263,6 +267,18 @@ private static FieldType fromTableFieldSchemaType(
"SqlTimestampWithLocalTzType", FieldType.STRING, "",
FieldType.DATETIME) {});
case "STRUCT":
case "RECORD":
+ // check if record represents a map entry
Review comment:
Regarding the second concern, I wonder if a user with a { "key", "value"
} struct would always be satisfied to work with the field as a Map in Beam. If
not, then I suppose we would need to somehow tag the field schema with a marker
that a user would not reasonably collide with. The options I see are:
1. Field names (could be something like "beam_key" or "map_key").
Personally, I'm not a fan of affecting schema names like this.
2. Extra field (in addition to "key" and "value"), could be something like
"beam_map" with all true or null values. Similarly messy like above.
3. Tag in field description(s) with a warning for the user to not delete it.
Might be something like #beam-map-do-not-delete#.
Though these options require that the fields are created via Beam and not by
the user beforehand. Or the user would need to know to use these special
markers to enable the Beam map functionality.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
