Ahmet offline for the next 3 weeks

Hi all, I will be on vacation starting tomorrow through first week of October. I would not be able to respond to most of the things. Happy Beaming, Ahmet

Re: [PROPOSAL] FileIO.write: a modular replacement for FileBasedSink/WriteFiles

PR out for review https://github.com/apache/beam/pull/3817 Next steps are clean it up (in this PR) and implement sinks for Text, XML and TFRecord (in subsequent PRs). On Thu, Sep 7, 2017 at 9:57 AM Robert Bradshaw wrote: > Huge +1. > > This brings things more in

Re: [jira] [Commented] (BEAM-2815) Python DirectRunner is unusable with input files in the 100-250MB range

Ad another data point, you could try using --runner=apache_beam.runners.portability.fn_api_runner.FnApiRunner On Thu, Sep 7, 2017 at 1:20 PM, Ahmet Altay (JIRA) wrote: > > [ >

Re: BEAM-407

+dev@beam.apache.org Findbugs analysis is now only run in the release profile; see: https://github.com/apache/beam/commit/fae52a3f8f4963524c7883450aa740aefaf4a9c6 Adding "-Prelease" to your maven invocation should repro the issue. On Wed, Sep 6, 2017 at 10:23 PM Justin T

Re: Merge branch DSL_SQL to master

+1 Thanks to all contributors/reviewers! On Thu, Sep 7, 2017 at 9:55 AM, Kai Jiang wrote: > +1 looking forward to this. > > On Thu, Sep 7, 2017, 09:53 Tyler Akidau > wrote: > > > +1, thanks for all the hard work to everyone that contributed! > >

Re: [PROPOSAL] FileIO.write: a modular replacement for FileBasedSink/WriteFiles

Huge +1. This brings things more in line with Python's FileBasedSink where one simply overrides write[_encoded]_record and, usually, open/close. We may want to consider aligning the APIs. (And, of course bringing things like DynamicDestinations to Python.) On Wed, Sep 6, 2017 at 9:24 PM,

Re: Merge branch DSL_SQL to master

+1 looking forward to this. On Thu, Sep 7, 2017, 09:53 Tyler Akidau wrote: > +1, thanks for all the hard work to everyone that contributed! > > -Tyler > > On Thu, Sep 7, 2017 at 2:39 AM Ismaël Mejía wrote: > > > +1 > > A nice feature to have on

Re: BigQueryIO withSchemaFromView

Please include the full exception and please show the code that produces it. See also https://beam.apache.org/documentation/programming-guide/#transforms-sideio section "Side inputs and windowing" - that might be sufficient to resolve your problem. On Thu, Sep 7, 2017 at 5:10 AM Chaim Turkel

BigQueryIO withSchemaFromView

Hi, I have a pipline that bases on documents from mongo updates the schema and then adds the records to mongo. Since i want a partitioned table, i have a dally window. How do i get the schema view to be a window, i get the exception of: Attempted to get side input window for GlobalWindow from

Re: Merge branch DSL_SQL to master

+1 A nice feature to have on Beam. Great work guys ! On Thu, Sep 7, 2017 at 10:21 AM, Pei HE wrote: > +1 > > On Thu, Sep 7, 2017 at 4:03 PM, tarush grover > wrote: > >> Thank you all, it was a great learning experience! >> >> Regards, >> Tarush >> >>

Re: Merge branch DSL_SQL to master

+1 On Thu, Sep 7, 2017 at 4:03 PM, tarush grover wrote: > Thank you all, it was a great learning experience! > > Regards, > Tarush > > On Thu, 7 Sep 2017 at 1:05 PM, Jean-Baptiste Onofré > wrote: > > > +1 > > > > Great work guys ! > > Ready to help

Re: Merge branch DSL_SQL to master

+1 Great work guys ! Ready to help for the merge and maintain ! Regards JB On 09/07/2017 08:48 AM, Mingmin Xu wrote: Hi all, On behalf of the virtual Beam SQL team[1], I'd like to propose to merge DSL_SQL branch into master (PR #3782 [2]) and include it in release version 2.2.0, which will

Build failed in Jenkins: beam_Release_NightlySnapshot #527

See Changes: [ekirpichov] [BEAM-2803] Improves performance of JdbcIO for a very large number of [ekirpichov] Implements JdbcIO.readAll() and improves validation messages [altay] Fix dataflow runner

Re: Merge branch DSL_SQL to master

+1. Glad to see it's coming finally. On Thu, Sep 7, 2017 at 2:48 PM Mingmin Xu wrote: > Hi all, > > On behalf of the virtual Beam SQL team[1], I'd like to propose to merge > DSL_SQL branch into master (PR #3782 [2]) and include it in release version > 2.2.0, which will give

Re: Beam 2.2.0 release

Thanks @JB, add you and Tyler as reviewer and is waiting for jenkins job. On Thu, Sep 7, 2017 at 12:25 PM, Jean-Baptiste Onofré wrote: > It sounds good to me. > > By the way, you will need my help to complete the release process (as you > need some permissions that you don't