Re: [DISCUSS] [BEAM-4126] Deleting Maven build files (pom.xml) grace period?

2018-06-06 Thread Ted Yu
+1 on this effort Original message From: Chamikara Jayalath Date: 6/6/18 2:09 PM (GMT-08:00) To: d...@beam.apache.org, user@beam.apache.org Subject: Re: [DISCUSS] [BEAM-4126] Deleting Maven build files (pom.xml) grace period? +1 for the overall effort. As Pablo mentioned, we

Re: BigQuery streaming insert errors

2018-04-02 Thread Ted Yu
Wouldn't the following code give you information about failed insertions (around line 790 in BigQueryServicesImpl) ? if (!allErrors.isEmpty()) { throw new IOException("Insert failed: " + allErrors); Cheers On Mon, Apr 2, 2018 at 7:16 AM, Carlos Alonso wrote: > Hi everyone!! > > I

Re: Table with field based partitioning must have a schema

2018-03-24 Thread Ted Yu
It seems an improvement can be made where if CREATE_NEVER is present, table with field based partitioning doesn't have to be associated with a schema. Cheers On Sat, Mar 24, 2018 at 2:30 PM, Carlos Alonso wrote: > Otherwise the BQ load job fails with the above error as well (Table with > field

Re: "Radically modular data ingestion APIs in Apache Beam" @ Strata - slides available

2018-03-08 Thread Ted Yu
, once more runners catch up with the support: currently we have Dataflow > and Flink. +Chamikara Jayalath is currently > working on implementing it using SDFs in the Python SDK. > > On Thu, Mar 8, 2018 at 4:34 PM Ted Yu wrote: > >> Eugene: >> Very informative talk. >

Re: "Radically modular data ingestion APIs in Apache Beam" @ Strata - slides available

2018-03-08 Thread Ted Yu
Eugene: Very informative talk. I looked at: sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/splittabledofn/OffsetRangeTrackerTest.java Is there some example showing how OffsetRangeTracker works with Kafka partition(s) ? Thanks On Thu, Mar 8, 2018 at 3:58 PM, Eugene Kirpichov wrote:

Re: Beam Spark Dataframe support

2018-02-13 Thread Ted Yu
As far as I can tell, there is no support. FYI On Tue, Feb 13, 2018 at 7:34 AM, ramesh krishnan muthusamy < ramkrish1...@gmail.com> wrote: > > > Hi Team, > Does Apache Beam provide API's for Apache Spark Dataframes as a part for > Spark 2.0. > > -Ramesh > >

Re: Deprecate and remove support for Kafka 0.9.x and older version

2018-02-02 Thread Ted Yu
There wouldn't be new Kafka 0.9.x release. +1 on dropping support for 0.9.x On Fri, Feb 2, 2018 at 4:12 PM, Raghu Angadi wrote: > Is anyone using Apache Beam with Kafka 0.9.x and older? > > I am thinking of deprecating 0.9.x and 0.10.0 in Beam 2.4 and remove > support in 2.5 or later. 0.10.1 an

Re: Strata Conference this March 6-8

2018-01-16 Thread Ted Yu
+1 to BoF On Tue, Jan 16, 2018 at 5:00 PM, Dmitry Demeshchuk wrote: > Probably won't be attending the conference, but totally down for a BoF. > > On Tue, Jan 16, 2018 at 4:58 PM, Holden Karau > wrote: > >> Do interested folks have any timing constraints around a BoF? >> >> On Tue, Jan 16, 2018

Re: [POLL] Dropping support for Java 7 in Beam 2.3

2018-01-04 Thread Ted Yu
+1 on dropping Java 7 support.