Re: ElasticIO retry configuration exception

Hi Wout, Maye check your classpath http client versions (against https://github.com/apache/beam/blob/v2.7.0/sdks/java/io/elasticsearch/build.gradle for instance). Romain Manni-Bucau @rmannibucau | Blog | Old Blog

Re: How to use of BigQueryIO Method.FILE_LOADS when reading from a unbounded source?

Hi Wout, you are so right - I forgot the --tempLocation= parameter when launching and after that I also needed to set the number of shards by adding: .withNumFileShards(1) Thank you! Tobi On Wed, Oct 10, 2018 at 3:23 PM Wout Scheepers < wout.scheep...@vente-exclusive.com> wrote:

Re: ElasticIO retry configuration exception

Hey JB, Thanks for your fast reply. The elastic version we're using is 5.6.2. "version": { "number": "5.6.2", "build_hash": "57e20f3", "build_date": "2017-09-23T13:16:45.703Z", "build_snapshot": false, "lucene_version": "6.6.1" } Wout On 10/10/201

Re: ElasticIO retry configuration exception

Hi Wout, what's the elasticsearch version ? (just to try to reproduce) Thanks, Regards JB On 10/10/2018 15:31, Wout Scheepers wrote: > Hey all, > >   > > When using .withRetryConfiguration()for ElasticsearchIO, I get the > following stacktrace: > >   > > Caused by: com.fasterxml.jackson.data

ElasticIO retry configuration exception

Hey all, When using .withRetryConfiguration()for ElasticsearchIO, I get the following stacktrace: Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input at [Source: (org.apache.http.nio.entity.ContentInputStream); line: 1, column: 0]

Re: How to use of BigQueryIO Method.FILE_LOADS when reading from a unbounded source?

Hey Tobias, org.apache.beam.sdk.io.gcp.bigquery.BatchLoads.expandTriggered(BatchLoads.java:212) points to the following code snippet (starting from BatchLoads.java:210) : if (bigQueryServices == null) { try { GcsPath.fromUri(tempLocation); } catch (IllegalArgumentException e) { thr

How to use of BigQueryIO Method.FILE_LOADS when reading from a unbounded source?

I am trying to read from an unbounded source and using FILE_LOADS instead of streaming inserts towards BigQuery. If I don't have the following two lines .withMethod(BigQueryIO.Write.Method.FILE_LOADS) .withTriggeringFrequency(Duration.standardMinutes(10)) my code works just fine, but uses str

Re: Issue with GroupByKey in BeamSql using SparkRunner

It's maybe related: I have a pipeline (streaming with sliding windows) that works fine with Direct and Flink runners, but I don't have any result when using the Spark runner. I gonna investigate this using my beam-samples. Regards JB On 10/10/2018 11:16, Ismaël Mejía wrote: > Are you trying this

Re: Issue with GroupByKey in BeamSql using SparkRunner

Are you trying this in a particular spark distribution or just locally ? I ask this because there was a data corruption issue with Spark 2.3.1 (previous version used by Beam) https://issues.apache.org/jira/browse/SPARK-23243 Current Beam master (and next release) moves Spark to version 2.3.2 and t