Re: Companies using Beam?

Hi, We already have some testimonials on Beam home page (I did the one about Beam use at Talend). It makes sense to have a dedicated section as it gives ideas about use case and production system running with Beam. Regards JB > Le 28 avr. 2020 à 23:42, Austin Bennett a écrit > : > > Hi

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

On Tue, Apr 28, 2020 at 4:46 PM Hannah Jiang wrote: > I guess I assumed there was some reason we needed "lightweight images" in >> our tests (because licenses take up a lot of space IIRC), but maybe not. >> Can you elaborate on the purpose of this option Hannah? > > Reducing image size is a good

Re: Companies using Beam?

> One danger I've seen is that such pages can grow stale/feel dated if not regularly updated/added to, so we should have a plan there. Definitely. Case in point: one of our existing testimonials is from data artisans, which isn't even that company's name anymore. On Tue, Apr 28, 2020 at 7:35 PM

Jenkins test triggers

Hi, Has anyone noticed any changes today (since ~6h ago) to how tests are triggered on PRs? Are they triggering always, some of the time, not at all? Are phrase comment triggers working? Thanks. (background: https://issues.apache.org/jira/browse/INFRA-19836) smime.p7s Description: S/MIME

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

> > I guess I assumed there was some reason we needed "lightweight images" in > our tests (because licenses take up a lot of space IIRC), but maybe not. > Can you elaborate on the purpose of this option Hannah? Reducing image size is a good reason to have the pull option. There were user requests

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

How about we pull/add dependencies when either: - The container build target is executed for a release branch [1]. - We are running the build target on Jenkins: - some environmental variable tells us that we are running on Jenkins - path to repo starts with /home/jenkins - perhaps there is

Re: Companies using Beam?

I think this is a great idea, as long as we can get critical mass. One danger I've seen is that such pages can grow stale/feel dated if not regularly updated/added to, so we should have a plan there. On Tue, Apr 28, 2020 at 4:21 PM Aizhamal Nurmamat kyzy wrote: > +1 on adding

Re: Companies using Beam?

+1 on adding testimonials/snippets from users either or both on GitHub and website. As part of the Beam website redesign proposal [1], we are suggesting to add a few pages like, "Powered by" or "Use Cases" which users can refer to and find information on who is using Beam and why. The proposal

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

> To overwrite, we need to pass the tag for ALL Jenkins tests that create docker images, which is quite a bunch of work. If it comes to that, I'd rather we do the work of passing tags instead of users. > Does anyone know if there is an easy way to pass the tag to many tests? or overwrite the

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

Fundamentally having license checking off by default is dangerous for releases, but having it on by default is annoying for developers. (On that note, however, pull seems bad for both.) Is it possible to make it a gradle target that only runs when something (specifically dependencies, or the files

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

Do you mean set the default mode to skip and overwrite it to pull mode with Jenkins test? To overwrite, we need to pass the tag for ALL Jenkins tests that create docker images, which is quite a bunch of work. Does anyone know if there is an easy way to pass the tag to many tests? or overwrite the

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

Can this be solved by enabling the license magic for the Jenkins jobs? I also think the default should be off for better local development experience. On Tue, Apr 28, 2020 at 3:05 PM Hannah Jiang wrote: > We need to make sure the release images contains licenses/notices in order > to avoid

Re: Companies using Beam?

I agree it'd be beneficial to the community to highlight users better. Rather than just a list, though, perhaps we can get some additional user testimonials (either by requesting them or linking to existing blog posts, etc). Testimonials are more effort to write, but I think it's more enticing and

Re: Greetings from Tyson

Welcome to the community! On Tue, Apr 28, 2020 at 2:45 PM Tyson Hamilton wrote: > Hello Beam Community, > > This is just a simple 'Hello' to introduce myself. I'm a Software Engineer > at Google and have worked with data processing languages and runtime > systems on and off during my career. I

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

We need to make sure the release images contains licenses/notices in order to avoid potential legal issues. The purpose of setting the default to pull is checking PRs which introduce new dependencies include thier licenses as well, in a way of auto pulling or tool pulling, to make sure all

Greetings from Tyson

Hello Beam Community, This is just a simple 'Hello' to introduce myself. I'm a Software Engineer at Google and have worked with data processing languages and runtime systems on and off during my career. I now have the pleasure of dedicating more time towards working with you lovely folks on

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

The different modes make sense. One disagreement: I think the default should be skip. I imagine few users would want to put licenses in their own images. On Tue, Apr 28, 2020 at 5:36 PM Hannah Jiang wrote: > The above 1,2,3,4 are merged to master. > Here I would like to change behaviors with

Companies using Beam?

Hi All, Have we considered getting onto our website or our our GitHub repo the ability for individuals to share that their company is using Beam? Seeing - what I believe to be a reasonable list of - companies productively using Beam would be helpful to point others to. For instance, a common

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

The above 1,2,3,4 are merged to master. Here I would like to change behaviors with *docker-pull-licenses* tag and would like to collect feedbacks from community before finalizing my PR. docker-pull-licenses can be set to one of ['add', 'pull', 'skip']. - *docker-pull-licenses=add* pulls

Re: Jacek's new Apache Beam Internals Project

The tweet URL for ref in case someone wants to like/RT https://twitter.com/jaceklaskowski/status/1255046717277376512?s=19 On Tue, Apr 28, 2020, 8:04 PM Holden Karau wrote: > Hi Folks, > > I just saw Jacek's tweet about his new Beam Internals project (he's done a > great job on his Spark

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

Adding +Nam Bui and +Karolina Rosół to follow up on questions. On Tue, Apr 28, 2020 at 11:34 AM Ahmet Altay wrote: > I am having trouble reviewing the staged version. What is the best way to > review this change? > > Do we expect any changes to markdown files, beyond some metadata? > > On

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

I am having trouble reviewing the staged version. What is the best way to review this change? Do we expect any changes to markdown files, beyond some metadata? On Tue, Apr 28, 2020 at 10:45 AM Robert Bradshaw wrote: > Thanks. It'll be great to better support more languages. > > I looked at the

Re: How to submit PRs for dependant changes?

I would prefer c. I suppose you already have 4 separate changes in 4 separate local branches, Each of a,b and c will require merging locally. So there shouldn't be much of a difference in effort in these 3 options. c would be easiest to review and will preserve information in the most meaningful

Jacek's new Apache Beam Internals Project

Hi Folks, I just saw Jacek's tweet about his new Beam Internals project (he's done a great job on his Spark Internals documentation and blog posts) and I figured I'd share the link https://leanpub.com/the-internals-of-apache-beam in case folks are interested :) Cheers, Holden :) -- Twitter:

Re: How to submit PRs for dependant changes?

I prefer (c) as well, rebasing as things get merged. I would do (a) if they're really prerequisites for one another. On Tue, Apr 28, 2020 at 10:40 AM Udi Meiri wrote: > (a) or (c) should work. (c) is preferred if you want faster reviews. > > For multiple JIRAs, I've seen both

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

Thanks. It'll be great to better support more languages. I looked at the PR and there seems to be no provenance/history. E.g. all the content seems to be entirely new files rather than diffs from the old. (There also seems to be a huge amount of auto-generated js code as well.) On Tue, Apr 28,

Re: [RESULT][VOTE] Accept the Firefly design donation as Beam Mascot - Deadline Mon April 6

I'll be happy to as well! On Sun, Apr 26, 2020 at 4:18 AM Maximilian Michels wrote: > Hey Maria, > > I can testify :) > > Cheers, > Max > > On 23.04.20 20:49, María Cruz wrote: > > Hi everyone! > > It is amazing to see how this process developed to collaboratively > > create Apache Beam's

Re: How to submit PRs for dependant changes?

(a) or (c) should work. (c) is preferred if you want faster reviews. For multiple JIRAs, I've seen both [BEAM-123,BEAM-456] and [BEAM-123][BEAM-456] formats. One of them works but I'm not sure which. :D You can always manually add a PR to a JIRA. On Sun, Apr 26, 2020 at 2:49 PM Reuven Lax

Re: Jenkins jobs not running for my PR 10438

Thanks Udi! I'll track for updates on this. > On 28 Apr 2020, at 19:16, Udi Meiri wrote: > > Alexey, what you're doing should be working (commits should trigger tests, as > should "retest this please" and other phrases). > > https://issues.apache.org/jira/browse/INFRA-19836 >

[REVIEW][please pause website changes] Migrated the Beam website to Hugo

Hello everybody, We are almost done migrating the Apache Beam website from Jekyll to Hugo. You can see the PR in [1], and we'd love to hear your feedback/comments on the PR. It includes detailed guidelines on contributing to the new Hugo-based website and adding translations to pages [2]. For

Re: Jenkins jobs not running for my PR 10438

Alexey, what you're doing should be working (commits should trigger tests, as should "retest this please" and other phrases). https://issues.apache.org/jira/browse/INFRA-19836 tracks this issue On Tue, Apr 28, 2020 at 10:04 AM Alexey Romanenko wrote: > Does anyone know the “golden rule” how to

Re: Jenkins jobs not running for my PR 10438

Does anyone know the “golden rule” how to trigger Jenkins tests? For example: https://github.com/apache/beam/pull/11341 I tried several times and it’s still not triggered. > On 28 Apr 2020, at 13:33, Ismaël Mejía wrote: > > done > > On Tue, Apr

Re: Support for Avro 1.9.X

There's no way to do this now. There was an attempt to upgrade to 1.9.1 here [1]. The hangup was just that the default datetime type was switched to java time (from joda time). My memory is we could get that PR working just by setting some configuration.. somewhere.. to make sure we use joda time

Support for Avro 1.9.X

Hi, The latest version of beam currently only supports Avro 1.8.2, however, my customer (Charles Schwab) is heavily using Avro 1.9.x which handles date fields much better. Is there any way they can upgrade from 1.8.2 to 1.9.x in Dataflow/beam? Thanks, Parth Desai Customer Engineer, Data

Re: Jenkins jobs not running for my PR 10438

done On Tue, Apr 28, 2020 at 12:47 PM Shoaib Zafar wrote: > Hello Beam Committers, > > I would appreciate if you could trigger precommit checks for the PR: > https://github.com/apache/beam/pull/11210 along with the python > post-commit check (Run Python 3.5 PostCommit). > > Thanks and Regards.

Re: Jenkins jobs not running for my PR 10438

Hello Beam Committers, I would appreciate if you could trigger precommit checks for the PR: https://github.com/apache/beam/pull/11210 along with the python post-commit check (Run Python 3.5 PostCommit). Thanks and Regards. *Shoaib Zafar* Software Engineering Lead Mobile: +92 333 274 6242 Skype:

Re: JIRA Committer Permissions

FWIW committer and contributor roles in Beam's JIRA have practically identical permissions[1]. The only different is the "Set Issue Security" and "Manage Watchers" permissions, both of which I have never used before. Thanks for updating the wiki page! -Max [1]