Re: Phrase triggering jobs problem

Thanks Kenn. On Wed, Jul 10, 2019 at 3:31 PM Kenneth Knowles wrote: > Just noticed this thread. Infra turned off one of the GitHub plugins - the > one we use. I forwarded the announcement. I'll see if we can get it back on > for a bit so we can migrate off. I'm not sure if they have identical jo

Re: Phrase triggering jobs problem

Just noticed this thread. Infra turned off one of the GitHub plugins - the one we use. I forwarded the announcement. I'll see if we can get it back on for a bit so we can migrate off. I'm not sure if they have identical job DSL or not. On Wed, Jul 10, 2019 at 12:32 PM Udi Meiri wrote: > Still ha

Re: [VOTE] Vendored Dependencies Release

No, the classes weren't expected inside of Guava. Cancelling this release candidate. I don't believe the protos/certs/keys matter and were part of our prior 1.13.1 release as well[1]. I found out that we stopped validating the contents of the vendored jar as part of the release process and opened

New Design Doc for Cost Based Optimization

Dear Members of Beam Community, Previously I had shared a document discussing row count estimation for the source tables in a query. https://docs.google.com/document/d/1vi1PBBu5IqSy-qZl1Gk-49CcANOpbNs1UAud6LnOaiY/edit I wrote another document that discusses the Cost Model and Join Reordering, and

Re: Python Utilities

On Wed, Jul 10, 2019 at 9:56 AM Rui Wang wrote: > The second link points to the first join utility in Beam. The idea is > similar: people can use the utility to do joins without writing them own. > BeamSQL also uses it. > > The first link points to Schema API. I actually thought Schema API also >

Fwd: [NOTICE] Jenkins GHPRB deprecated, please switch :)

This applies to our jobs. Kenn -- Forwarded message - From: Daniel Gruno Date: Wed, Jul 10, 2019 at 1:33 PM Subject: [NOTICE] Jenkins GHPRB deprecated, please switch :) To: Hi folks, as part of some cleanup and consolidation (essentially we don't want to maintain two different

Re: [VOTE] Vendored Dependencies Release

pull/8357 proposes to vendor bytebuddy artifact. Is it possible to release "beam-vendor-bytebuddy-1_9_3" in next release candidate? Best, Kai On Wed, Jul 10, 2019 at 11:31 AM Kenneth Knowles wrote: > grpc: jar contains certs, keys, protos at the top le

Re: Hazelcast Jet Runner

Yes please! On Wed, Jul 10, 2019 at 8:38 PM Kenneth Knowles wrote: > > Just to make sure we have closed on the Jet runner, my understanding is: I > was the main person asking for "runners-jet-experimental" but I am convinced > to go with plain "runners-jet". It seems everyone else is already fi

Re: Phrase triggering jobs problem

Still happening for me too. On Wed, Jul 10, 2019 at 10:40 AM Lukasz Cwik wrote: > This has happened in the past. Usually there is some issue where Jenkins > isn't notified of new PRs by Github or doesn't see the PR phrases and hence > Jenkins sits around idle. This is usually fixed after a few h

Re: Hazelcast Jet Runner

Just to make sure we have closed on the Jet runner, my understanding is: I was the main person asking for "runners-jet-experimental" but I am convinced to go with plain "runners-jet". It seems everyone else is already fine with this, so go ahead? On Tue, Jul 9, 2019 at 1:23 PM Maximilian Michels

Re: [VOTE] Vendored Dependencies Release

grpc: jar contains certs, keys, protos at the top level; intended? guava: jar contains classes not in vendored prefix, with prefixes such as com/google/j2objc, org/codehaus/mojo, com/google/errorprone, org/checkerframework, javax/annotation On Tue, Jul 9, 2019 at 3:34 PM Lukasz Cwik wrote: > Pl

Re: Beam/Samza Ensuring At Least Once semantics

When you restart the application, are you resuming it from Samza's last commit? Since the exception is thrown after the GBK, all the data could be read from Kafka and forwarded to the GBK operator inside of Samza and checkpointed in Kafka before the exception is ever thrown. On Tue, Jul 9, 2019 a

[Discuss] Retractions in Beam

Hi Community, Retractions is a part of core Beam model [1]. I come up with a doc to discuss retractions about use cases, model and API (see the link below). This is a very beginning discussion on retractions but I do hope we can have a consensus and make retractions implemented in a useful way eve

Re: Phrase triggering jobs problem

This has happened in the past. Usually there is some issue where Jenkins isn't notified of new PRs by Github or doesn't see the PR phrases and hence Jenkins sits around idle. This is usually fixed after a few hours without any action on our part. On Wed, Jul 10, 2019 at 10:28 AM Katarzyna Kucharcz

Phrase triggering jobs problem

Hi all, Hope it's not duplicate but I can't find if any issue with phrase triggering in Jenkins was already here. Currently, I started third PR and no test were triggered there. I tried to trigger some tests manually, but with no effect. Am I missing something? Here are links to my problematic P

Re: Python Utilities

The second link points to the first join utility in Beam. The idea is similar: people can use the utility to do joins without writing them own. BeamSQL also uses it. The first link points to Schema API. I actually thought Schema API also uses the join utility, and turns out it doesn't (I am not su

Re: Python Utilities

So it seams that the Java SDK has two different Join libraries? With Schema: https://github.com/apache/beam/tree/77b295b1c2b0a206099b8f50c4d3180c248e252c/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms And Another one: https://github.com/apache/beam/blob/77b295b1c2b0a206099b8f5

Re: [Python] Read Hadoop Sequence File?

If I wanted to go ahead and include this within a new Java Pipeline, what would I be looking at for level of work to integrate? On Wed, Jul 3, 2019 at 3:54 AM Ismaël Mejía wrote: > That's great. I can help whenever you need. We just need to choose its > destination. Both the `hadoop-format` and

Re: pickling typing types in Python 3.5+

I looked into CloudPickle a while back, and would be supportive of the change. On Mon, Jul 1, 2019 at 11:06 PM Valentyn Tymofieiev wrote: > > I have checked that cloudpickle (an alternative to dill) is able to pickle > and unpickle typing types on Python 3.5, 3.6, which seems to be a recent > c

Re: [VOTE] Vendored Dependencies Release

+1 On 2019/07/09 22:33:48, Lukasz Cwik wrote: > Please review the release of the following artifacts that we vendor:> > * beam-vendor-grpc_1_21_0> > * beam-vendor-guava-26_0-jre> > > Hi everyone,> > Please review and vote on the release candidate #2 for the> > org.apache.beam:beam-vendor-grpc_1

Re: [DISCUSS] Contributor guidelines for iterating on PRs: when to squash commits.

On Wed, Jul 10, 2019 at 5:06 AM Kenneth Knowles wrote: > > My opinion: what is important is that we have a policy for what goes into the > master commit history. This is very simple IMO: each commit should clearly do > something that it states, and a commit should do just one thing. Exactly how