[ https://issues.apache.org/jira/browse/BEAM-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eugene Kirpichov closed BEAM-2370. ---------------------------------- Resolution: Duplicate Fix Version/s: 2.2.0 > BigQuery Insert with Partition Decorator throwing error > ------------------------------------------------------- > > Key: BEAM-2370 > URL: https://issues.apache.org/jira/browse/BEAM-2370 > Project: Beam > Issue Type: Bug > Components: sdk-java-gcp > Affects Versions: 2.0.0 > Environment: DirectRunner > Reporter: Andre > Assignee: Reuven Lax > Fix For: 2.2.0 > > > Running a DataFlow job with the DirectRunner which is inserting data into a > partitioned table using decorators throws the following error multiple times > BUT still inserts records into the right partition. > {code:java|title=Error} > org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl executeWithRetries > INFO: Ignore the error and retry the request. > com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad > Request > { > "code" : 400, > "errors" : [ { > "domain" : "global", > "message" : "Invalid table ID \"mytable_orders$20170516\". Table IDs must > be alphanumeric (plus underscores) and must be at most 1024 characters long. > Also, Table decorators cannot be used.", > "reason" : "invalid" > } ], > "message" : "Invalid table ID \"mytable_orders$20170516\". Table IDs must > be alphanumeric (plus underscores) and must be at most 1024 characters long. > Also, Table decorators cannot be used." > } > at > com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146) > {code} > {code:java|title=Code} > // Write TableRows to BQ > rows.apply("TransformationStep", ParDo.of(new Outputter())) > .apply("WindowDaily", Window. <TableRow> into(CalendarWindows.days(1))) > .apply("WriteToBQ", BigQueryIO.writeTableRows() > .to(new SerializableFunction <ValueInSingleWindow<TableRow> , > TableDestination> () { > private static final long serialVersionUID = 8196602721734820219 L; > @Override > public TableDestination apply(ValueInSingleWindow <TableRow> value) { > String dayString = > DateTimeFormat.forPattern("yyyyMMdd").withZone(DateTimeZone.UTC) > .print(((IntervalWindow) value.getWindow()).start()); > TableDestination td = new > TableDestination("my-project:dataset.mytable_orders$" + dayString, ""); > return td; > } > }).withSchema(mySchema) > .withCreateDisposition(BigQueryIO.Write.CreateDisposition.CREATE_NEVER) > .withWriteDisposition(BigQueryIO.Write.WriteDisposition.WRITE_TRUNCATE)); > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)