Hello Beam Community!

2020-03-12 Thread Brittany Hermann 
Hello Beam Community!

My name is Brittany Hermann and I recently joined the Open Source team in
Data Analytics at Google. As a Program Manager, I will be focusing on
community engagement while getting to work on Apache Beam and Airflow
projects! I have always thrived on creating healthy, diverse, and overall
happy communities and am excited to bring that to the team. For a fun fact,
I am a big Wisconsin Badgers Football fan and have a goldendoodle puppy
named Ollie!

I look forward to collaborating with you all!

Kind regards,

Brittany Hermann

Hello Beam Community!

2020-03-12 Thread Brittany Hermann 
Hello Beam Community!

My name is Brittany Hermann and I recently joined the Open Source team in
Data Analytics at Google. As a Program Manager, I will be focusing on
community engagement while getting to work on Apache Beam and Airflow
projects! I have always thrived on creating healthy, diverse, and overall
happy communities and am excited to bring that to the team. For a fun fact,
I am a big Wisconsin Badgers Football fan and have a goldendoodle puppy
named Ollie!

I look forward to collaborating with you all!

Kind regards,

Brittany Hermann

Re: Compile error on Java 11 when running :examples:java:test

2020-03-12 Thread Kyle Weaver 
I'm trying to set up Beam from scratch on a new Linux install, and I ran
into the same issue. Initially, I had my Intellij project set up with JDK
11, but when I tried switching to JDK 8 got the same error. When I run the
same Gradle command in the terminal, though, it worked. So I figure it must
be a problem with my Intellij setup.


On Fri, Feb 7, 2020 at 9:36 PM Jean-Baptiste Onofre  wrote:

> Hi,
>
> AFAIR I had the same issue on my Linux.
>
> Let me do a new run.
>
> Regards
> JB
>
> Le 7 févr. 2020 à 21:35, Kenneth Knowles  a écrit :
>
> The expected class file version 53 is for Java 9, I believe. So is the
> right javac being invoked?
>
> I hit some issues like this on mac a while back, unrelated to Java 11.
> Suspected something wonky in Mac's Java setup not working well with the
> Gradle wrapper. Never resolved them actually. Have been working on linux
> lately.
>
> Kenn
>
> On Fri, Feb 7, 2020 at 11:32 AM Jean-Baptiste Onofré 
> wrote:
>
>> Hi
>>
>> No jdk 11 is not yet fully supported.
>>
>> I?ve started to work on it but it?s not yet ready.
>>
>> Regards
>> JB
>>
>> Le ven. 7 f?vr. 2020 ? 20:20, David Cavazos  a
>> ?crit :
>>
>>> Hi Beamers,
>>>
>>> I'm trying to run the tests for the Java examples using Java 11 and
>>> there is a compilation error due to an incompatible version.
>>>
>>> I'm using the latest version of master.
>>>
>>> [image: image.png]
>>>
>>> If I downgrade to Java 8, it works. But isn't Java 11 supported?
>>>
>>> Thanks!
>>>
>>
>

Re: Install Jenkins AnsiColor plugin

2020-03-12 Thread Udi Meiri 
https://github.com/apache/beam/pull/2 is out, attempts to colorize
pytest output


On Sun, Mar 8, 2020 at 10:56 AM Chad Dombrova  wrote:

> I don’t believe that it was ever resolved.  I have a PR with a bunch of
> attempts to get it working but I never did figure it out.  IIRC there did
> seem to be some ansi plugin already installed but I couldn’t get it to
> work.
>
> -chad
>
>
> On Sun, Mar 8, 2020 at 10:52 AM Ismaël Mejía  wrote:
>
>> Did this ever happen? If not what is blocking it?
>>
>>
>>
>> On Tue, Oct 22, 2019 at 10:13 PM Udi Meiri  wrote:
>> >
>> > Your proposal will only affect the seed job (which doesn't do color
>> outputs AFAIK).
>> > I think you want to add colorizeOutput() here:
>> >
>> https://github.com/apache/beam/blob/bfebbd0d16361f61fa40bfdec2f0cb6f943f7c9a/.test-infra/jenkins/CommonJobProperties.groovy#L79-L95
>> >
>> > Otherwise no concerns from me.
>> >
>> > On Tue, Oct 22, 2019 at 12:01 PM Chad Dombrova 
>> wrote:
>> >>
>> >> thanks, so IIUC, I’m going to update job_00_seed.groovy like this:
>> >>
>> >>   wrappers {
>> >> colorizeOutput()
>> >> timeout {
>> >>   absolute(60)
>> >>   abortBuild()
>> >> }
>> >>   }
>> >>
>> >> Then add the comment run seed job
>> >>
>> >> Does anyone have any concerns with me trying this out now?
>> >>
>> >> -chad
>> >>
>> >>
>> >> On Tue, Oct 22, 2019 at 11:42 AM Udi Meiri  wrote:
>> >>>
>> >>> Also note that changing the job DSL doesn't take effect until the
>> "seed" job runs. (use the "run seed job" phrase)
>> >>>
>> >>> On Tue, Oct 22, 2019 at 11:06 AM Chad Dombrova 
>> wrote:
>> 
>>  Thanks, I'll look into this.  I have a PR I'm building up with a
>> handful of minor changes related to this.
>> 
>> 
>> 
>>  On Tue, Oct 22, 2019 at 10:45 AM Yifan Zou 
>> wrote:
>> >
>> > Thanks, Udi! The ansicolor plugin was applied to ASF Jenkins
>> universally. You might need to explicitly enable the coloroutput in your
>> jenkins dsl.
>> >
>> > On Tue, Oct 22, 2019 at 10:33 AM Udi Meiri 
>> wrote:
>> >>
>> >> Seems to be already installed:
>> https://issues.apache.org/jira/browse/INFRA-16944
>> >> Do we just need to enable it somehow?
>> >> This might work:
>> https://jenkinsci.github.io/job-dsl-plugin/#method/javaposse.jobdsl.dsl.helpers.wrapper.WrapperContext.colorizeOutput
>> >>
>> >> BTW, our Jenkins is maintained by ASF's Infrastructure team:
>> https://cwiki.apache.org/confluence/display/INFRA/Jenkins
>> >>
>> >> On Tue, Oct 22, 2019 at 10:23 AM Chad Dombrova 
>> wrote:
>> >>>
>> >>> Hi all,
>> >>> As a user trying to grok failures in jenkins I think it would be
>> a huge help to have color output support.  This is something that works out
>> of the box for CI tools like gitlab and travis, and it really helps bring
>> that 21st century feel to your logs :)
>> >>>
>> >>> There's a Jenkins plugin for colorizing ansi escape sequences
>> here:
>> >>> https://plugins.jenkins.io/ansicolor
>> >>>
>> >>> I think this is something that has to be deployed by a Jenkins
>> admin.
>> >>>
>> >>> -chad
>> >>>
>>
>


smime.p7s
Description: S/MIME Cryptographic Signature

Re: Request to be added as Jira contributor

2020-03-12 Thread Ahmet Altay 
Done. Welcome!

On Wed, Mar 11, 2020 at 1:51 PM Piotr Szuberski 
wrote:

> *Hi, my name is Piotr. I am interested in contributing java sdk to the
> Apache Beam SDK. I'd like to be added **as a Jira contributor so that I
> can assign issues to myself. My ASF Jira Username is piotr-szuberski .*
>

Re: [Newbie Contributing Question] How to add a dependency?

2020-03-12 Thread Tomo Suzuki 
Jacob,

It works fine for me.
https://gist.github.com/suztomo/d90951a2c028c7406d87b13bd43e6eaa
Do you want to share the output?


On Wed, Mar 11, 2020 at 8:49 PM Jacob Ferriero  wrote:

> Thanks that was helpful in getting my dependencies added!
> I am not having a an issue where the testCompile dependencies such as
> junit aren't getting resolved when running
> ```
>  ./gradlew -p sdks/java/io/google-cloud-platform/ test
> ```
>  I didn't touch these. Is there some place I have to add new test
> directory?
>
> On 2020/03/11 04:17:50, Tomo Suzuki  wrote:
> > Hi Jacob,>
> >
> > You'll need to modify BeamModulePlugin.groovy, which defines a big map
> of>
> > Maven artifacts. The map is referenced by each module such as the>
> > google-cloud-platform/build.gradle.>
> > Example PR to touch dependencies:>
> > https://github.com/apache/beam/pull/11063/files>
> >
> > On Tue, Mar 10, 2020 at 11:22 PM Jacob Ferriero >
> > wrote:>
> >
> > > Hi beam dev list,>
> > >>
> > > Hoping to find some pointers on how to best add a dependency for a new
> GCP>
> > > IO connector.>
> > > Specifically I want to add a dependency on the Cloud Healthcare API or
> the>
> > > equivalent of this section of a maven pom.xml:>
> > > ```xml>
> > > >
> > >
> v1alpha2-rev20190901-1.30.1>
> > >   1.30.1>
> > > >
> > > >
> > >   com.google.http-client>
> > >   google-http-client-gson>
> > >   ${http.client.version}>
> > > >
> > > >
> > >   com.google.apis>
> > >   google-api-services-healthcare>
> > >   ${healthcare.version}>
> > > 
> > > ```>
> > > I know I'm one step away looking at this build.gradle>
> > >
> https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/build.gradle#L30>
>
> > >>
> > > *Background:*>
> > > I'm a first time contributor working on a PR for BEAM-9468>
> > >  an
> issue>
> > > I opened to create a new GCP IO connector for a series of use cases we
> have>
> > > in the healthcare space.>
> > > I'm completely new to gradle. I started my development in a fork of>
> > > https://github.com/GoogleCloudPlatform/DataflowTemplates which uses>
> > > maven but then decided these would better be to contribute these IO>
> > > connectors upstream.>
> > >>
> > > -->
> > >>
> > > *Jacob Ferriero*>
> > >>
> > > Strategic Cloud Engineer: Data Engineering>
> > >>
> > > jferri...@google.com>
> > >>
> > > 617-714-2509 <(617)%20714-2509> <(617)%20714-2509>>
> > >>
> >
> >
> > -- >
> > Regards,>
> > Tomo>
> >
>


-- 
Regards,
Tomo

Re: [DISCUSS] Drop support for Flink 1.7

2020-03-12 Thread Maximilian Michels 
This is now done. We added first Flink 1.10 and then removed 1.7
support. This will be reflected in Beam 2.21.0.

Thanks Jincheng for your initiative. It was a pleasure working with you
on the PRs.

Cheers,
Max

On 12.03.20 02:33, jincheng sun wrote:
> Hi all,
> I would like to drop the flink 1.7 support soon, as flink 1.10 already
> supported in this
> commit.https://github.com/apache/beam/commit/f91b390c8bbab4afe14734c1266da51dcc7558c9
> 
> Best,
> Jincheng
> 
> 
> 
> jincheng sun  > 于2020年2月24日周一 上午11:22写道：
> 
> Thanks for all of your feedback. Will dropping the 1.7 support
> after https://github.com/apache/beam/pull/10945 has be merged.
> 
> Best,
> Jincheng
> 
> 
> David Morávek  > 于2020年2月19日周三 下午7:08写道：
> 
> +1 for dropping 1.7, once we have 1.10 support ready
> 
> D.
> 
> On Tue, Feb 18, 2020 at 7:01 PM  > wrote:
> 
> Hi Ismael,
> yes, sure. The proposal would be to have snapshot dependency
> in the feature branch. The snapshot must be changed to
> release before merge to master.
> Jan
> 
> Dne 18. 2. 2020 17:55 napsal uživatel Ismaël Mejía
> mailto:ieme...@gmail.com>>:
> 
> Just to be sure, you mean Flink 1.11.0-SNAPSHOT ONLY in
> the next branch dependency?
> We should not have any SNAPSHOT dependency from other
> projects in Beam.
> 
> On Tue, Feb 18, 2020 at 5:34 PM  > wrote:
> 
> Hi=Cr�Jincheng,
> I think there should be a "process" for this. I
> would propose to:
>  a) create new branch with support for new
> (snapshot) flink - currently that would mean flink 1.11
>  b) as part of this brach drop support for all
> version up to N - 3
> I think th!t that dropping all versions and adding
> new version should be atomic, otherwise we risk we
> release beam version with less than three supprted
> flink versions.
> I'd suggest to start with the 1.10 branch support,
> include the drop of 1.7 into that branch. Once 1.10
> gets merged, we should create 1.11 with snapshot
> dependency to be able to keep up with the release
> cadence of flink.
> WDYT?
>  Jan
> 
> Dne 18. 2. 2020 15:34 napsal uživatel jincheng sun
>  >:
> 
> Hi folks,
> 
> Apache Flink 1.10 has completed the
> release announcement [1]n Then we would like to
> add Flink 1.10 build target and make Flink
> Runner compatible with Flink 1.10 [2]. So, I
> would suggest that at most three versions of
> Flink runner for Apache Beam community according
> to the update Policy of Apache Flink releases
> [3] , i.e. I think it's better to maintain the
> three versions of 1.8/1.9/1.10 after add Flink
> 1.10 build target to Flink runner.
> 
> The current existence of Flink runner 1.7 will
> affect the upgrade of Flink runner 1.8x and 1.9x
> due to the code of Flink 1.7 is too old, more
> detail can be found in [4]. So,  we need to drop
> the support of Flink runner 1.7 as soon as possible.
> 
> This discussion also CC to @User, due to the
> change will affect our users. And I would
> appreciate it if you could review the PR [5].
> 
> Welcome any feedback!
> 
> Best,
> Jincheng
> 
> [1]
> 
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flin+-1-10-0-released-td37564.html
> 
> 
> [2] https://issues.apache.org/jira/browse/BEAM-9295
> [3]
> 
> https://flink.apache.org/downloads.html#update-policy-for-old-releases
> [4] https://issues.apache.org/jira/browse/BEAM-9299
> [5] https://github.com/apache/beam/pull/10884
> 
>