Re: The Go SDK got accidentally merged - options to deal with the pain

I'm not sure what value there is in preserving this accidental merge in history, but all options proposed seem fine to me. We should resolve this (or at least unblock other dev work) quickly though. On Tue, Mar 6, 2018 at 3:16 PM Kenneth Knowles wrote: > My own vote is for

Re: The Go SDK got accidentally merged - options to deal with the pain

Looks like the only issue is a snapshot version in the pom. On Tue, Mar 6, 2018 at 3:28 PM Henning Rohde wrote: > I'm happy to deal with any needed fixup on the Go SDK side in either case. > > On Tue, Mar 6, 2018 at 3:20 PM, Lukasz Cwik wrote: > >> Can we

Re: The Go SDK got accidentally merged - options to deal with the pain

I'm happy to deal with any needed fixup on the Go SDK side in either case. On Tue, Mar 6, 2018 at 3:20 PM, Lukasz Cwik wrote: > Can we open up the pair of commits so that master gets reverted and the Go > SDK merges from master plus another rollback? > > On Tue, Mar 6, 2018 at

Re: The Go SDK got accidentally merged - options to deal with the pain

Can we open up the pair of commits so that master gets reverted and the Go SDK merges from master plus another rollback? On Tue, Mar 6, 2018 at 2:42 PM, Kenneth Knowles wrote: > Hi all, > > You may have noticed that our tests are red. A pull request that was meant > for the Go

Re: The Go SDK got accidentally merged - options to deal with the pain

My own vote is for leaving the history immutable, which is the case for the full rollback or leaving it there disabled. On Tue, Mar 6, 2018 at 3:01 PM Thomas Weise wrote: > +1 for (1), assuming it is straightforward to exclude from the build and > eventually will end up in

Re: The Go SDK got accidentally merged - options to deal with the pain

+1 for (1), assuming it is straightforward to exclude from the build and eventually will end up in master anyways. On Tue, Mar 6, 2018 at 2:59 PM, Robert Bradshaw wrote: > I would opt for (2), but I'm not sure who has permissions to do that. It > should be easy to re-merge

Re: The Go SDK got accidentally merged - options to deal with the pain

I would opt for (2), but I'm not sure who has permissions to do that. It should be easy to re-merge the couple of things that have gone in since then. On Tue, Mar 6, 2018 at 2:43 PM Kenneth Knowles wrote: > Hi all, > > You may have noticed that our tests are red. A pull

Re: The Go SDK got accidentally merged - options to deal with the pain

I'm reading "The next time..." as the Go developers needs to take some special precautions the next time they merge from master to go-sdk? If that's correct, that's more than reasonable, especially in light of this current breakage. Please do whatever's necessary to make things right again. If

The Go SDK got accidentally merged - options to deal with the pain

Hi all, You may have noticed that our tests are red. A pull request that was meant for the Go SDK branch accidentally got merged onto the master branch. Things have been merged to master since then. I've opened a revert at https://github.com/apache/beam/pull/4808 The next time there is a master

Re: When should ParDo advance output watermarks?

Hi Kenn, Thank you! Shen On Tue, Mar 6, 2018 at 5:21 PM, Kenneth Knowles wrote: > On Tue, Mar 6, 2018 at 1:06 PM Shen Li wrote: > >> Hi, >> >> Should ParDo advance output watermarks based on only main input or all >> inputs? Say if the watermark from a

Re: When should ParDo advance output watermarks?

On Tue, Mar 6, 2018 at 1:06 PM Shen Li wrote: > Hi, > > Should ParDo advance output watermarks based on only main input or all > inputs? Say if the watermark from a side input falls behind, should it > block the output watermark of the ParDo. > The rule is that if the

When should ParDo advance output watermarks?

Hi, Should ParDo advance output watermarks based on only main input or all inputs? Say if the watermark from a side input falls behind, should it block the output watermark of the ParDo. If there are pushed back elements, should the ParDo hold back its output watermarks until corresponding

Re: Any reason to not use [vfs]?

Just to share a bit more than a ticket here is a bootstrap impl https://github.com/apache/beam/pull/4803 Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github

Re: Beam 2.4.0

Hi guys, tried to reapply the waitUntilFinish fix - without breaking the compilation this time ;) - in https://github.com/apache/beam/pull/4790, anyone able to help to review and move that forward? Romain Manni-Bucau @rmannibucau | Blog

Re: Releases and user support

2018-03-02 18:12 GMT+01:00 Robert Bradshaw : > On Fri, Mar 2, 2018 at 8:45 AM Romain Manni-Bucau > wrote: > > > Hi guys, > > > I didn't find a page about beam release support. With the fast minor > release rrythm which is targetted by beam (see other

Re: Any reason to not use [vfs]?

Cool. Then for now we should create a separate Vfs-backed Filesystem impl. Once Vfs supports all we need, I think we can consider keeping only that. Keep in mind that the bulk operations Luke mentioned translate to native bulk operations for Gcs at least (BatchRequest is part of the Gcs API). I'm

Re: Any reason to not use [vfs]?

+1 for the discussion and tracking. Regards JB On 03/06/2018 12:07 PM, Romain Manni-Bucau wrote: > created https://issues.apache.org/jira/browse/BEAM-3786 to track the > discussion > (without putting too much details in the ticket for now) > > > Romain Manni-Bucau > @rmannibucau

Re: Any reason to not use [vfs]?

created https://issues.apache.org/jira/browse/BEAM-3786 to track the discussion (without putting too much details in the ticket for now) Romain Manni-Bucau @rmannibucau | Blog | Old Blog |

Re: Any reason to not use [vfs]?

@Reuven: this was what I had in mind yes. Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn |

Re: Any reason to not use [vfs]?

Part of the point of the current Filesystem class _is_ to handle these things (e.g. bulk delete/rename). If Vfs doesn't, then maybe the right answer is to keep Filesystem but put Vfs under it (and maybe that will eventually allow us to remove some of the current code). On Mon, Mar 5, 2018 at