[
https://issues.apache.org/jira/browse/BEAM-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17132298#comment-17132298
]
Beam JIRA Bot commented on BEAM-3501:
-------------------------------------
This issue is assigned but has not received an update in 30 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> BigQuery Partitioned table creation/write fails when destination has
> partition decorator
> ----------------------------------------------------------------------------------------
>
> Key: BEAM-3501
> URL: https://issues.apache.org/jira/browse/BEAM-3501
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Reporter: Darshan Mehta
> Assignee: Eugene Kirpichov
> Priority: P2
> Labels: stale-assigned
>
> Following is the code that writes to BigQuery:
> {code:java}
> BigQueryIO.writeTableRows()
> .to(destination)
> .withCreateDisposition(CREATE_IF_NEEDED)
> .withWriteDisposition(WRITE_APPEND)
> .withSchema(tableSchema)
> .expand(tableRows);{code}
>
> Here's the destination's implementation:
> {code:java}
> public TableDestination apply(ValueInSingleWindow<TableRow> input) {
> String partition = timestampExtractor.apply(input.getValue())
> .toString(DateTimeFormat.forPattern("yyyyMMdd").withZoneUTC());
> TableReference tableReference = new TableReference();
> tableReference.setDatasetId(dataset);
> tableReference.setProjectId(projectId);
> tableReference.setTableId(String.format("%s_%s", table, partition));
> log.debug("Will write to BigQuery table: %s", tableReference);
> return new TableDestination(tableReference, null);
> }{code}
>
> When the dataflow tries to write to this table, I see the following message:
> {code:java}
> "errors" : [ {
> "domain" : "global",
> "message" : "Cannot read partition information from a table that is not
> partitioned: <project_id>:<dataset>.<table>$19730522",
> "reason" : "invalid"
> } ]{code}
> So, it looks like it's not creating tables with partition in the first place?
> Apache beam version : 2.2.0
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
