[ https://issues.apache.org/jira/browse/BEAM-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dan Kruchinin updated BEAM-6434: -------------------------------- Description: Sometimes I see the following NPE in my streaming pipeline: {code:java} Caused by: java.lang.NullPointerException com.google.cloud.bigquery.StandardTableDefinition$StreamingBuffer.toPb(StandardTableDefinition.java:106) com.google.cloud.bigquery.StandardTableDefinition.toPb(StandardTableDefinition.java:230) com.google.cloud.bigquery.TableInfo.toPb(TableInfo.java:453) com.google.cloud.bigquery.BigQueryImpl.update(BigQueryImpl.java:383) com.google.cloud.bigquery.Table.update(Table.java:212) com.heysphere.nosegment.dofn.TableRowSchemaMutatorFn.processElement(TableRowSchemaMutatorFn.java:132){code} It happens extremely rarely and I'm not sure what can cause it. To add a bit more context, the pipeline performs `Table.update` when it handles failed inserts from BigQueryIO and detects that the problem can be fixed by adding missing columns (sort of like https://github.com/ryanmcdowell/dataflow-dynamic-schema ) Somewhat reminds BEAM-6076 was: Sometimes I see the following NPE in my streaming pipeline: {code:java} Caused by: java.lang.NullPointerException com.google.cloud.bigquery.StandardTableDefinition$StreamingBuffer.toPb(StandardTableDefinition.java:106) com.google.cloud.bigquery.StandardTableDefinition.toPb(StandardTableDefinition.java:230) com.google.cloud.bigquery.TableInfo.toPb(TableInfo.java:453) com.google.cloud.bigquery.BigQueryImpl.update(BigQueryImpl.java:383) com.google.cloud.bigquery.Table.update(Table.java:212) com.heysphere.nosegment.dofn.TableRowSchemaMutatorFn.processElement(TableRowSchemaMutatorFn.java:132){code} It happens extremely rarely and I'm not sure what can cause it. To add a bit more context, the pipeline performs `Table.update` when it handles failed inserts from BigQueryIO and detects that the problem can be fixed by adding missing columns (sort of like [https://github.com/ryanmcdowell/dataflow-dynamic-schema).] Somewhat reminds BEAM-6076 > NPE in BigQuery client library in StreamingBuffer.toPb > ------------------------------------------------------ > > Key: BEAM-6434 > URL: https://issues.apache.org/jira/browse/BEAM-6434 > Project: Beam > Issue Type: Bug > Components: testing > Affects Versions: 2.9.0 > Reporter: Dan Kruchinin > Assignee: Jason Kuster > Priority: Major > > Sometimes I see the following NPE in my streaming pipeline: > > > {code:java} > Caused by: java.lang.NullPointerException > com.google.cloud.bigquery.StandardTableDefinition$StreamingBuffer.toPb(StandardTableDefinition.java:106) > > com.google.cloud.bigquery.StandardTableDefinition.toPb(StandardTableDefinition.java:230) > com.google.cloud.bigquery.TableInfo.toPb(TableInfo.java:453) > com.google.cloud.bigquery.BigQueryImpl.update(BigQueryImpl.java:383) > com.google.cloud.bigquery.Table.update(Table.java:212) > com.heysphere.nosegment.dofn.TableRowSchemaMutatorFn.processElement(TableRowSchemaMutatorFn.java:132){code} > > > It happens extremely rarely and I'm not sure what can cause it. To add a bit > more context, the pipeline performs `Table.update` when it handles failed > inserts from BigQueryIO and detects that the problem can be fixed by adding > missing columns (sort of like > https://github.com/ryanmcdowell/dataflow-dynamic-schema ) > > Somewhat reminds BEAM-6076 > -- This message was sent by Atlassian JIRA (v7.6.3#76005)