Filed https://issues.apache.org/jira/browse/BEAM-3198 for the IllegalArgumentException Do you mind posting a little code snippet of how you build the BQ IO connector on BEAM-3198?
On Wed, Nov 15, 2017 at 12:18 PM, Arpan Jain <jainar...@gmail.com> wrote: > Hi, > > I am trying to use Method.FILE_LOADS for loading data into BQ in my > streaming pipeline using RC3 release of 2.2.2. It looks > like withNumFileShards needs to be also set for using this. Couple of > questions regarding this: > > * Any guidelines on what's a good value for this? FWIW my pipeline is > going to write > 100K messages/second to around 500 tables > using DynamicDestinations. I am trying to set a triggering frequency of 30 > minutes or 1 hour. > > * Can we throw a better error message when it is not set? The exception > thrown by the pipeline if this is not set is "Exception in thread "main" > java.lang.IllegalArgumentException" without any message. >
