Re: Gradle Status [April 6]

[Kenneth Knowles]

Caching success/failure of a test is legitimate useful behavior. That said,
I'm not sure how it interacts with Idea.

Kenn

On Sun, Apr 15, 2018 at 12:23 AM Romain Manni-Bucau 
wrote:

> If you dont the diff is empty and gradle runs nothing, no? Saw it with
> gradle 4.6
>
> Le 15 avr. 2018 00:49, "Kenneth Knowles"  a écrit :
>
>> You shouldn't do :module:cleanTest. If that is necessary that's a major
>> bug in the build.
>>
>> Kenn
>>
>> On Fri, Apr 13, 2018 at 11:46 PM Romain Manni-Bucau <
>> rmannibu...@gmail.com> wrote:
>>
>>> There is a fake module xxx_test which should have the right classpath
>>> but since idea compilation is messed up you will still have to run
>>> :module:cleanTest :module:test --tests org...MyTest.myMethod, even with
>>> idea which leads to the same latency as the command line :(
>>>
>>> Le 13 avr. 2018 22:23, "Eugene Kirpichov"  a
>>> écrit :
>>>
 While we're talking about running tests in IntelliJ with Gradle...
 Anybody got advice on how to run a single NeedsRunner test in
 sdks-java-core, say, ParDoTest? With Maven, I used to just run the test in
 IntelliJ and specify "runners-direct-java" as the classpath; with Gradle,
 the best I got is to manually run the direct runner's needsRunnerTests task
 with specifying --tests=..., but it takes a long time, and IntelliJ treats
 that as just a gradle task run, not as a test run.

 On Fri, Apr 13, 2018 at 11:14 AM Reuven Lax  wrote:

> Is there a Jira for this 3 second delay? Also you're initial complaint
> was not about the 3 second delay, so it wasn't clear that's what you were
> complaining about.
>
> Reuven
>
> On Fri, Apr 13, 2018 at 4:42 AM Romain Manni-Bucau <
> rmannibu...@gmail.com> wrote:
>
>> When you launch a test with gradle runner it launches gradle which
>> makes loose 3s on a very fast computer and more on a slower (6 on my
>> personal one which is already fast but not as much as my work one). We 
>> are
>> 5 to see that regression at least. So there is a reason to not use the
>> gradle runner if possible cause when you work and need to debug you are
>> just stucked (that is why i switched back to mvn after 15mn, i was 
>> loosing
>> to much time).
>>
>> Switching back to native idea test run would fix it but tests just
>> dont work this way for me whatever setup i do :( - missing resources IIRC
>> in idea out dir.
>>
>> Le 13 avr. 2018 00:07, "Reuven Lax"  a écrit :
>>
>> I also don't quite understand what your question is, and it appears
>> like Dan spent considerable time trying to reproduce your issue. For the
>> record, I have had no issues running tests via Gradle in IntelliJ for the
>> past few weeks.
>>
>> Reuven
>>
>> On Thu, Apr 12, 2018 at 9:47 PM Daniel Oliveira <
>> danolive...@google.com> wrote:
>>
>>> Sorry Romain, I'm not quite sure what you're asking. Can you clarify?
>>>
>>> On Thu, Apr 12, 2018 at 12:22 PM Romain Manni-Bucau <
>>> rmannibu...@gmail.com> wrote:
>>>
 Well you are the only one to not have the drawbacks to use it so
 maybe dont do it? I know Luke is in holidays but anyone else with the
 knowledge of why we nees that noise compared to idea native 
 tooling/flow?

 Le 12 avr. 2018 20:16, "Daniel Oliveira" 
 a écrit :

> Ah, I did not. Thanks Romain.
>
> I tried it again, restarting in between, and still had no
> differences. Since it seems like there's no reason not to use "Gradle 
> Test
> Runner", I'll mention it in the contributor's guide.
>
> On Thu, Apr 12, 2018 at 10:31 AM Romain Manni-Bucau <
> rmannibu...@gmail.com> wrote:
>
>> @Daniel: did you restart in between? Otherwise it does nothing.
>> One launches JunitCoreRunner from idea and the other a gradle 
>> command.
>>
>> Le 12 avr. 2018 19:24, "Daniel Oliveira" 
>> a écrit :
>>
>>> I think it depends on what exactly switching to "Gradle Test
>>> Runner" from "Platform Test Runner" does. I tried it out on my 
>>> machine and
>>> they seem to act identically to each other. The IntelliJ 
>>> documentation says
>>> it determines what API to use to run the tests
>>> , so maybe
>>> it's usefulness depends on the user's machine, in which case a note 
>>> about
>>> that would be useful. Something like: "If your IDE has trouble 
>>> running
>>> tests via IDEA shortcuts, try the following steps: [...]"
>>>
>>> On Thu, Apr 12, 2018 at 3:29 AM 

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

If you dont the diff is empty and gradle runs nothing, no? Saw it with
gradle 4.6

Le 15 avr. 2018 00:49, "Kenneth Knowles"  a écrit :

> You shouldn't do :module:cleanTest. If that is necessary that's a major
> bug in the build.
>
> Kenn
>
> On Fri, Apr 13, 2018 at 11:46 PM Romain Manni-Bucau 
> wrote:
>
>> There is a fake module xxx_test which should have the right classpath but
>> since idea compilation is messed up you will still have to run
>> :module:cleanTest :module:test --tests org...MyTest.myMethod, even with
>> idea which leads to the same latency as the command line :(
>>
>> Le 13 avr. 2018 22:23, "Eugene Kirpichov"  a
>> écrit :
>>
>>> While we're talking about running tests in IntelliJ with Gradle...
>>> Anybody got advice on how to run a single NeedsRunner test in
>>> sdks-java-core, say, ParDoTest? With Maven, I used to just run the test in
>>> IntelliJ and specify "runners-direct-java" as the classpath; with Gradle,
>>> the best I got is to manually run the direct runner's needsRunnerTests task
>>> with specifying --tests=..., but it takes a long time, and IntelliJ treats
>>> that as just a gradle task run, not as a test run.
>>>
>>> On Fri, Apr 13, 2018 at 11:14 AM Reuven Lax  wrote:
>>>
 Is there a Jira for this 3 second delay? Also you're initial complaint
 was not about the 3 second delay, so it wasn't clear that's what you were
 complaining about.

 Reuven

 On Fri, Apr 13, 2018 at 4:42 AM Romain Manni-Bucau <
 rmannibu...@gmail.com> wrote:

> When you launch a test with gradle runner it launches gradle which
> makes loose 3s on a very fast computer and more on a slower (6 on my
> personal one which is already fast but not as much as my work one). We are
> 5 to see that regression at least. So there is a reason to not use the
> gradle runner if possible cause when you work and need to debug you are
> just stucked (that is why i switched back to mvn after 15mn, i was loosing
> to much time).
>
> Switching back to native idea test run would fix it but tests just
> dont work this way for me whatever setup i do :( - missing resources IIRC
> in idea out dir.
>
> Le 13 avr. 2018 00:07, "Reuven Lax"  a écrit :
>
> I also don't quite understand what your question is, and it appears
> like Dan spent considerable time trying to reproduce your issue. For the
> record, I have had no issues running tests via Gradle in IntelliJ for the
> past few weeks.
>
> Reuven
>
> On Thu, Apr 12, 2018 at 9:47 PM Daniel Oliveira <
> danolive...@google.com> wrote:
>
>> Sorry Romain, I'm not quite sure what you're asking. Can you clarify?
>>
>> On Thu, Apr 12, 2018 at 12:22 PM Romain Manni-Bucau <
>> rmannibu...@gmail.com> wrote:
>>
>>> Well you are the only one to not have the drawbacks to use it so
>>> maybe dont do it? I know Luke is in holidays but anyone else with the
>>> knowledge of why we nees that noise compared to idea native 
>>> tooling/flow?
>>>
>>> Le 12 avr. 2018 20:16, "Daniel Oliveira"  a
>>> écrit :
>>>
 Ah, I did not. Thanks Romain.

 I tried it again, restarting in between, and still had no
 differences. Since it seems like there's no reason not to use "Gradle 
 Test
 Runner", I'll mention it in the contributor's guide.

 On Thu, Apr 12, 2018 at 10:31 AM Romain Manni-Bucau <
 rmannibu...@gmail.com> wrote:

> @Daniel: did you restart in between? Otherwise it does nothing.
> One launches JunitCoreRunner from idea and the other a gradle command.
>
> Le 12 avr. 2018 19:24, "Daniel Oliveira" 
> a écrit :
>
>> I think it depends on what exactly switching to "Gradle Test
>> Runner" from "Platform Test Runner" does. I tried it out on my 
>> machine and
>> they seem to act identically to each other. The IntelliJ 
>> documentation says
>> it determines what API to use to run the tests
>> , so maybe it's
>> usefulness depends on the user's machine, in which case a note about 
>> that
>> would be useful. Something like: "If your IDE has trouble running 
>> tests via
>> IDEA shortcuts, try the following steps: [...]"
>>
>> On Thu, Apr 12, 2018 at 3:29 AM Alexey Romanenko <
>> aromanenko@gmail.com> wrote:
>>
>>> Daniel, actually I did run it with default IDEA JUnit test
>>> runner. Then, in “Settings > Build, Execution, Deployment > Build 
>>> Tools >
>>> Gradle > Runner" I selected “Gradle Test Runner” in “Run tests 
>>> using”
>>> 

Re: Gradle Status [April 6]

[Kenneth Knowles]

You shouldn't do :module:cleanTest. If that is necessary that's a major bug
in the build.

Kenn

On Fri, Apr 13, 2018 at 11:46 PM Romain Manni-Bucau 
wrote:

> There is a fake module xxx_test which should have the right classpath but
> since idea compilation is messed up you will still have to run
> :module:cleanTest :module:test --tests org...MyTest.myMethod, even with
> idea which leads to the same latency as the command line :(
>
> Le 13 avr. 2018 22:23, "Eugene Kirpichov"  a écrit :
>
>> While we're talking about running tests in IntelliJ with Gradle...
>> Anybody got advice on how to run a single NeedsRunner test in
>> sdks-java-core, say, ParDoTest? With Maven, I used to just run the test in
>> IntelliJ and specify "runners-direct-java" as the classpath; with Gradle,
>> the best I got is to manually run the direct runner's needsRunnerTests task
>> with specifying --tests=..., but it takes a long time, and IntelliJ treats
>> that as just a gradle task run, not as a test run.
>>
>> On Fri, Apr 13, 2018 at 11:14 AM Reuven Lax  wrote:
>>
>>> Is there a Jira for this 3 second delay? Also you're initial complaint
>>> was not about the 3 second delay, so it wasn't clear that's what you were
>>> complaining about.
>>>
>>> Reuven
>>>
>>> On Fri, Apr 13, 2018 at 4:42 AM Romain Manni-Bucau <
>>> rmannibu...@gmail.com> wrote:
>>>
 When you launch a test with gradle runner it launches gradle which
 makes loose 3s on a very fast computer and more on a slower (6 on my
 personal one which is already fast but not as much as my work one). We are
 5 to see that regression at least. So there is a reason to not use the
 gradle runner if possible cause when you work and need to debug you are
 just stucked (that is why i switched back to mvn after 15mn, i was loosing
 to much time).

 Switching back to native idea test run would fix it but tests just dont
 work this way for me whatever setup i do :( - missing resources IIRC in
 idea out dir.

 Le 13 avr. 2018 00:07, "Reuven Lax"  a écrit :

 I also don't quite understand what your question is, and it appears
 like Dan spent considerable time trying to reproduce your issue. For the
 record, I have had no issues running tests via Gradle in IntelliJ for the
 past few weeks.

 Reuven

 On Thu, Apr 12, 2018 at 9:47 PM Daniel Oliveira 
 wrote:

> Sorry Romain, I'm not quite sure what you're asking. Can you clarify?
>
> On Thu, Apr 12, 2018 at 12:22 PM Romain Manni-Bucau <
> rmannibu...@gmail.com> wrote:
>
>> Well you are the only one to not have the drawbacks to use it so
>> maybe dont do it? I know Luke is in holidays but anyone else with the
>> knowledge of why we nees that noise compared to idea native tooling/flow?
>>
>> Le 12 avr. 2018 20:16, "Daniel Oliveira"  a
>> écrit :
>>
>>> Ah, I did not. Thanks Romain.
>>>
>>> I tried it again, restarting in between, and still had no
>>> differences. Since it seems like there's no reason not to use "Gradle 
>>> Test
>>> Runner", I'll mention it in the contributor's guide.
>>>
>>> On Thu, Apr 12, 2018 at 10:31 AM Romain Manni-Bucau <
>>> rmannibu...@gmail.com> wrote:
>>>
 @Daniel: did you restart in between? Otherwise it does nothing. One
 launches JunitCoreRunner from idea and the other a gradle command.

 Le 12 avr. 2018 19:24, "Daniel Oliveira" 
 a écrit :

> I think it depends on what exactly switching to "Gradle Test
> Runner" from "Platform Test Runner" does. I tried it out on my 
> machine and
> they seem to act identically to each other. The IntelliJ 
> documentation says
> it determines what API to use to run the tests
> , so maybe it's
> usefulness depends on the user's machine, in which case a note about 
> that
> would be useful. Something like: "If your IDE has trouble running 
> tests via
> IDEA shortcuts, try the following steps: [...]"
>
> On Thu, Apr 12, 2018 at 3:29 AM Alexey Romanenko <
> aromanenko@gmail.com> wrote:
>
>> Daniel, actually I did run it with default IDEA JUnit test
>> runner. Then, in “Settings > Build, Execution, Deployment > Build 
>> Tools >
>> Gradle > Runner" I selected “Gradle Test Runner” in “Run tests using”
>> selectbox and it works ok when I run my tests with IDEA shortcuts. 
>> So,
>> probably, we should add this details on
>> https://beam.apache.org/contribute/intellij/ too.
>> What do you think?
>>
>> WBR,
>> Alexey
>>
>> 

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

There is a fake module xxx_test which should have the right classpath but
since idea compilation is messed up you will still have to run
:module:cleanTest :module:test --tests org...MyTest.myMethod, even with
idea which leads to the same latency as the command line :(

Le 13 avr. 2018 22:23, "Eugene Kirpichov"  a écrit :

> While we're talking about running tests in IntelliJ with Gradle...
> Anybody got advice on how to run a single NeedsRunner test in
> sdks-java-core, say, ParDoTest? With Maven, I used to just run the test in
> IntelliJ and specify "runners-direct-java" as the classpath; with Gradle,
> the best I got is to manually run the direct runner's needsRunnerTests task
> with specifying --tests=..., but it takes a long time, and IntelliJ treats
> that as just a gradle task run, not as a test run.
>
> On Fri, Apr 13, 2018 at 11:14 AM Reuven Lax  wrote:
>
>> Is there a Jira for this 3 second delay? Also you're initial complaint
>> was not about the 3 second delay, so it wasn't clear that's what you were
>> complaining about.
>>
>> Reuven
>>
>> On Fri, Apr 13, 2018 at 4:42 AM Romain Manni-Bucau 
>> wrote:
>>
>>> When you launch a test with gradle runner it launches gradle which makes
>>> loose 3s on a very fast computer and more on a slower (6 on my personal one
>>> which is already fast but not as much as my work one). We are 5 to see that
>>> regression at least. So there is a reason to not use the gradle runner if
>>> possible cause when you work and need to debug you are just stucked (that
>>> is why i switched back to mvn after 15mn, i was loosing to much time).
>>>
>>> Switching back to native idea test run would fix it but tests just dont
>>> work this way for me whatever setup i do :( - missing resources IIRC in
>>> idea out dir.
>>>
>>> Le 13 avr. 2018 00:07, "Reuven Lax"  a écrit :
>>>
>>> I also don't quite understand what your question is, and it appears like
>>> Dan spent considerable time trying to reproduce your issue. For the record,
>>> I have had no issues running tests via Gradle in IntelliJ for the past few
>>> weeks.
>>>
>>> Reuven
>>>
>>> On Thu, Apr 12, 2018 at 9:47 PM Daniel Oliveira 
>>> wrote:
>>>
 Sorry Romain, I'm not quite sure what you're asking. Can you clarify?

 On Thu, Apr 12, 2018 at 12:22 PM Romain Manni-Bucau <
 rmannibu...@gmail.com> wrote:

> Well you are the only one to not have the drawbacks to use it so maybe
> dont do it? I know Luke is in holidays but anyone else with the knowledge
> of why we nees that noise compared to idea native tooling/flow?
>
> Le 12 avr. 2018 20:16, "Daniel Oliveira"  a
> écrit :
>
>> Ah, I did not. Thanks Romain.
>>
>> I tried it again, restarting in between, and still had no
>> differences. Since it seems like there's no reason not to use "Gradle 
>> Test
>> Runner", I'll mention it in the contributor's guide.
>>
>> On Thu, Apr 12, 2018 at 10:31 AM Romain Manni-Bucau <
>> rmannibu...@gmail.com> wrote:
>>
>>> @Daniel: did you restart in between? Otherwise it does nothing. One
>>> launches JunitCoreRunner from idea and the other a gradle command.
>>>
>>> Le 12 avr. 2018 19:24, "Daniel Oliveira"  a
>>> écrit :
>>>
 I think it depends on what exactly switching to "Gradle Test
 Runner" from "Platform Test Runner" does. I tried it out on my machine 
 and
 they seem to act identically to each other. The IntelliJ documentation 
 says
 it determines what API to use to run the tests
 , so maybe it's
 usefulness depends on the user's machine, in which case a note about 
 that
 would be useful. Something like: "If your IDE has trouble running 
 tests via
 IDEA shortcuts, try the following steps: [...]"

 On Thu, Apr 12, 2018 at 3:29 AM Alexey Romanenko <
 aromanenko@gmail.com> wrote:

> Daniel, actually I did run it with default IDEA JUnit test runner.
> Then, in “Settings > Build, Execution, Deployment > Build Tools > 
> Gradle >
> Runner" I selected “Gradle Test Runner” in “Run tests using” 
> selectbox and
> it works ok when I run my tests with IDEA shortcuts. So, probably, we
> should add this details on https://beam.apache.org/
> contribute/intellij/ too.
> What do you think?
>
> WBR,
> Alexey
>
> On 11 Apr 2018, at 21:17, Daniel Oliveira 
> wrote:
>
> Alexey, are you referring to tests run with "./gradlew
> :beam-runners-direct-java:needsRunnerTests"? That command works
> fine for me in both versions of IDEA, but I believe 

Re: Gradle Status [April 6]

[Reuven Lax]

Is there a Jira for this 3 second delay? Also you're initial complaint was
not about the 3 second delay, so it wasn't clear that's what you were
complaining about.

Reuven

On Fri, Apr 13, 2018 at 4:42 AM Romain Manni-Bucau 
wrote:

> When you launch a test with gradle runner it launches gradle which makes
> loose 3s on a very fast computer and more on a slower (6 on my personal one
> which is already fast but not as much as my work one). We are 5 to see that
> regression at least. So there is a reason to not use the gradle runner if
> possible cause when you work and need to debug you are just stucked (that
> is why i switched back to mvn after 15mn, i was loosing to much time).
>
> Switching back to native idea test run would fix it but tests just dont
> work this way for me whatever setup i do :( - missing resources IIRC in
> idea out dir.
>
> Le 13 avr. 2018 00:07, "Reuven Lax"  a écrit :
>
> I also don't quite understand what your question is, and it appears like
> Dan spent considerable time trying to reproduce your issue. For the record,
> I have had no issues running tests via Gradle in IntelliJ for the past few
> weeks.
>
> Reuven
>
> On Thu, Apr 12, 2018 at 9:47 PM Daniel Oliveira 
> wrote:
>
>> Sorry Romain, I'm not quite sure what you're asking. Can you clarify?
>>
>> On Thu, Apr 12, 2018 at 12:22 PM Romain Manni-Bucau <
>> rmannibu...@gmail.com> wrote:
>>
>>> Well you are the only one to not have the drawbacks to use it so maybe
>>> dont do it? I know Luke is in holidays but anyone else with the knowledge
>>> of why we nees that noise compared to idea native tooling/flow?
>>>
>>> Le 12 avr. 2018 20:16, "Daniel Oliveira"  a
>>> écrit :
>>>
 Ah, I did not. Thanks Romain.

 I tried it again, restarting in between, and still had no differences.
 Since it seems like there's no reason not to use "Gradle Test Runner", I'll
 mention it in the contributor's guide.

 On Thu, Apr 12, 2018 at 10:31 AM Romain Manni-Bucau <
 rmannibu...@gmail.com> wrote:

> @Daniel: did you restart in between? Otherwise it does nothing. One
> launches JunitCoreRunner from idea and the other a gradle command.
>
> Le 12 avr. 2018 19:24, "Daniel Oliveira"  a
> écrit :
>
>> I think it depends on what exactly switching to "Gradle Test Runner"
>> from "Platform Test Runner" does. I tried it out on my machine and they
>> seem to act identically to each other. The IntelliJ documentation says it
>> determines what API to use to run the tests
>> , so maybe it's
>> usefulness depends on the user's machine, in which case a note about that
>> would be useful. Something like: "If your IDE has trouble running tests 
>> via
>> IDEA shortcuts, try the following steps: [...]"
>>
>> On Thu, Apr 12, 2018 at 3:29 AM Alexey Romanenko <
>> aromanenko@gmail.com> wrote:
>>
>>> Daniel, actually I did run it with default IDEA JUnit test runner.
>>> Then, in “Settings > Build, Execution, Deployment > Build Tools > 
>>> Gradle >
>>> Runner" I selected “Gradle Test Runner” in “Run tests using” selectbox 
>>> and
>>> it works ok when I run my tests with IDEA shortcuts. So, probably, we
>>> should add this details on
>>> https://beam.apache.org/contribute/intellij/ too.
>>> What do you think?
>>>
>>> WBR,
>>> Alexey
>>>
>>> On 11 Apr 2018, at 21:17, Daniel Oliveira 
>>> wrote:
>>>
>>> Alexey, are you referring to tests run with "./gradlew
>>> :beam-runners-direct-java:needsRunnerTests"? That command works fine 
>>> for me
>>> in both versions of IDEA, but I believe the same tests fail if you run 
>>> them
>>> directly through "./gradlew test".
>>>
>>> However, I am having issues with a bunch of validatesRunner tests,
>>> mostly be caused by
>>> :beam-runners-google-cloud-dataflow-java:validatesRunner. Not sure if 
>>> it's
>>> because of a code change or a gradle config, I'll keep looking into it.
>>>
>>> On Wed, Apr 11, 2018 at 11:01 AM Romain Manni-Bucau <
>>> rmannibu...@gmail.com> wrote:
>>>
 I got tests running rrconfiguring gradle (which was setup for
 another project but seems beam didnt like it) but latency is still 
 "high"
 using gradle runner for tests (like Etienne said ~3s on an i7 with 16G 
 vs a
 few ms with default idea test runner, would be great to solve that).

 I also find the integration quite fishy cause configurations are
 customs so idea is kind of forced to propose your modukes 3 times at 
 least
 when you select the classpath (x_test being generally the working one).

 Also the false positive you get if 

Re: Gradle Status [April 6]

[Daniel Oliveira]

Ah, I see. I was attempting to replicate Alexey's issues with DirectRunner
tests, so I wasn't checking for that 3s overhead. A quick test for me shows
that I also get that several second overhead when running tests with
Gradle, ranging from 3-6 sec. However, on my machine this delay is
constant, regardless of whether I use the Gradle Runner or Platform Runner.

I think getting more info from someone more knowledgeable about Gradle is a
good idea; I'm still very new to Gradle, which is why my ability to help
with the Gradle effort is mainly limited to helping with documentation.

On Thu, Apr 12, 2018 at 9:41 PM Romain Manni-Bucau 
wrote:

> When you launch a test with gradle runner it launches gradle which makes
> loose 3s on a very fast computer and more on a slower (6 on my personal one
> which is already fast but not as much as my work one). We are 5 to see that
> regression at least. So there is a reason to not use the gradle runner if
> possible cause when you work and need to debug you are just stucked (that
> is why i switched back to mvn after 15mn, i was loosing to much time).
>
> Switching back to native idea test run would fix it but tests just dont
> work this way for me whatever setup i do :( - missing resources IIRC in
> idea out dir.
>
> Le 13 avr. 2018 00:07, "Reuven Lax"  a écrit :
>
> I also don't quite understand what your question is, and it appears like
> Dan spent considerable time trying to reproduce your issue. For the record,
> I have had no issues running tests via Gradle in IntelliJ for the past few
> weeks.
>
> Reuven
>
> On Thu, Apr 12, 2018 at 9:47 PM Daniel Oliveira 
> wrote:
>
>> Sorry Romain, I'm not quite sure what you're asking. Can you clarify?
>>
>> On Thu, Apr 12, 2018 at 12:22 PM Romain Manni-Bucau <
>> rmannibu...@gmail.com> wrote:
>>
>>> Well you are the only one to not have the drawbacks to use it so maybe
>>> dont do it? I know Luke is in holidays but anyone else with the knowledge
>>> of why we nees that noise compared to idea native tooling/flow?
>>>
>>> Le 12 avr. 2018 20:16, "Daniel Oliveira"  a
>>> écrit :
>>>
 Ah, I did not. Thanks Romain.

 I tried it again, restarting in between, and still had no differences.
 Since it seems like there's no reason not to use "Gradle Test Runner", I'll
 mention it in the contributor's guide.

 On Thu, Apr 12, 2018 at 10:31 AM Romain Manni-Bucau <
 rmannibu...@gmail.com> wrote:

> @Daniel: did you restart in between? Otherwise it does nothing. One
> launches JunitCoreRunner from idea and the other a gradle command.
>
> Le 12 avr. 2018 19:24, "Daniel Oliveira"  a
> écrit :
>
>> I think it depends on what exactly switching to "Gradle Test Runner"
>> from "Platform Test Runner" does. I tried it out on my machine and they
>> seem to act identically to each other. The IntelliJ documentation says it
>> determines what API to use to run the tests
>> , so maybe it's
>> usefulness depends on the user's machine, in which case a note about that
>> would be useful. Something like: "If your IDE has trouble running tests 
>> via
>> IDEA shortcuts, try the following steps: [...]"
>>
>> On Thu, Apr 12, 2018 at 3:29 AM Alexey Romanenko <
>> aromanenko@gmail.com> wrote:
>>
>>> Daniel, actually I did run it with default IDEA JUnit test runner.
>>> Then, in “Settings > Build, Execution, Deployment > Build Tools > 
>>> Gradle >
>>> Runner" I selected “Gradle Test Runner” in “Run tests using” selectbox 
>>> and
>>> it works ok when I run my tests with IDEA shortcuts. So, probably, we
>>> should add this details on
>>> https://beam.apache.org/contribute/intellij/ too.
>>> What do you think?
>>>
>>> WBR,
>>> Alexey
>>>
>>> On 11 Apr 2018, at 21:17, Daniel Oliveira 
>>> wrote:
>>>
>>> Alexey, are you referring to tests run with "./gradlew
>>> :beam-runners-direct-java:needsRunnerTests"? That command works fine 
>>> for me
>>> in both versions of IDEA, but I believe the same tests fail if you run 
>>> them
>>> directly through "./gradlew test".
>>>
>>> However, I am having issues with a bunch of validatesRunner tests,
>>> mostly be caused by
>>> :beam-runners-google-cloud-dataflow-java:validatesRunner. Not sure if 
>>> it's
>>> because of a code change or a gradle config, I'll keep looking into it.
>>>
>>> On Wed, Apr 11, 2018 at 11:01 AM Romain Manni-Bucau <
>>> rmannibu...@gmail.com> wrote:
>>>
 I got tests running rrconfiguring gradle (which was setup for
 another project but seems beam didnt like it) but latency is still 
 "high"
 using gradle runner for tests (like Etienne said ~3s on an i7 

Re: Gradle Status [April 6]

[Reuven Lax]

I also don't quite understand what your question is, and it appears like
Dan spent considerable time trying to reproduce your issue. For the record,
I have had no issues running tests via Gradle in IntelliJ for the past few
weeks.

Reuven

On Thu, Apr 12, 2018 at 9:47 PM Daniel Oliveira 
wrote:

> Sorry Romain, I'm not quite sure what you're asking. Can you clarify?
>
> On Thu, Apr 12, 2018 at 12:22 PM Romain Manni-Bucau 
> wrote:
>
>> Well you are the only one to not have the drawbacks to use it so maybe
>> dont do it? I know Luke is in holidays but anyone else with the knowledge
>> of why we nees that noise compared to idea native tooling/flow?
>>
>> Le 12 avr. 2018 20:16, "Daniel Oliveira"  a
>> écrit :
>>
>>> Ah, I did not. Thanks Romain.
>>>
>>> I tried it again, restarting in between, and still had no differences.
>>> Since it seems like there's no reason not to use "Gradle Test Runner", I'll
>>> mention it in the contributor's guide.
>>>
>>> On Thu, Apr 12, 2018 at 10:31 AM Romain Manni-Bucau <
>>> rmannibu...@gmail.com> wrote:
>>>
 @Daniel: did you restart in between? Otherwise it does nothing. One
 launches JunitCoreRunner from idea and the other a gradle command.

 Le 12 avr. 2018 19:24, "Daniel Oliveira"  a
 écrit :

> I think it depends on what exactly switching to "Gradle Test Runner"
> from "Platform Test Runner" does. I tried it out on my machine and they
> seem to act identically to each other. The IntelliJ documentation says it
> determines what API to use to run the tests
> , so maybe it's
> usefulness depends on the user's machine, in which case a note about that
> would be useful. Something like: "If your IDE has trouble running tests 
> via
> IDEA shortcuts, try the following steps: [...]"
>
> On Thu, Apr 12, 2018 at 3:29 AM Alexey Romanenko <
> aromanenko@gmail.com> wrote:
>
>> Daniel, actually I did run it with default IDEA JUnit test runner.
>> Then, in “Settings > Build, Execution, Deployment > Build Tools > Gradle 
>> >
>> Runner" I selected “Gradle Test Runner” in “Run tests using” selectbox 
>> and
>> it works ok when I run my tests with IDEA shortcuts. So, probably, we
>> should add this details on
>> https://beam.apache.org/contribute/intellij/ too.
>> What do you think?
>>
>> WBR,
>> Alexey
>>
>> On 11 Apr 2018, at 21:17, Daniel Oliveira 
>> wrote:
>>
>> Alexey, are you referring to tests run with "./gradlew
>> :beam-runners-direct-java:needsRunnerTests"? That command works fine for 
>> me
>> in both versions of IDEA, but I believe the same tests fail if you run 
>> them
>> directly through "./gradlew test".
>>
>> However, I am having issues with a bunch of validatesRunner tests,
>> mostly be caused by
>> :beam-runners-google-cloud-dataflow-java:validatesRunner. Not sure if 
>> it's
>> because of a code change or a gradle config, I'll keep looking into it.
>>
>> On Wed, Apr 11, 2018 at 11:01 AM Romain Manni-Bucau <
>> rmannibu...@gmail.com> wrote:
>>
>>> I got tests running rrconfiguring gradle (which was setup for
>>> another project but seems beam didnt like it) but latency is still 
>>> "high"
>>> using gradle runner for tests (like Etienne said ~3s on an i7 with 16G 
>>> vs a
>>> few ms with default idea test runner, would be great to solve that).
>>>
>>> I also find the integration quite fishy cause configurations are
>>> customs so idea is kind of forced to propose your modukes 3 times at 
>>> least
>>> when you select the classpath (x_test being generally the working one).
>>>
>>> Also the false positive you get if you forget a cleanX is a bit
>>> annoying, maybe we should force a clean for test or at least when there 
>>> is
>>> a --tests to avoid gradle to not run it cause there is no diff.
>>>
>>> So it works but dev productivity is reduced a lot and it became slow
>>> enough to think if you should do a contribution or not - at least for 
>>> me.
>>>
>>> Le 11 avr. 2018 19:37, "Alexey Romanenko" 
>>> a écrit :
>>>
 I’ve managed to import a project as it’s described in documentation
 (starting from empty project) using Idea 2018 and run unit tests
 successfully.
 For some reasons, tests, that use DirectRunner to run a pipeline,
 were failed.

 WBR,
 Alexey

 On 11 Apr 2018, at 19:01, Daniel Oliveira 
 wrote:

 Hi everyone, I was the one who initially wrote the PR with Idea
 instructions . I was

Re: Gradle Status [April 6]

[Daniel Oliveira]

Sorry Romain, I'm not quite sure what you're asking. Can you clarify?

On Thu, Apr 12, 2018 at 12:22 PM Romain Manni-Bucau 
wrote:

> Well you are the only one to not have the drawbacks to use it so maybe
> dont do it? I know Luke is in holidays but anyone else with the knowledge
> of why we nees that noise compared to idea native tooling/flow?
>
> Le 12 avr. 2018 20:16, "Daniel Oliveira"  a
> écrit :
>
>> Ah, I did not. Thanks Romain.
>>
>> I tried it again, restarting in between, and still had no differences.
>> Since it seems like there's no reason not to use "Gradle Test Runner", I'll
>> mention it in the contributor's guide.
>>
>> On Thu, Apr 12, 2018 at 10:31 AM Romain Manni-Bucau <
>> rmannibu...@gmail.com> wrote:
>>
>>> @Daniel: did you restart in between? Otherwise it does nothing. One
>>> launches JunitCoreRunner from idea and the other a gradle command.
>>>
>>> Le 12 avr. 2018 19:24, "Daniel Oliveira"  a
>>> écrit :
>>>
 I think it depends on what exactly switching to "Gradle Test Runner"
 from "Platform Test Runner" does. I tried it out on my machine and they
 seem to act identically to each other. The IntelliJ documentation says it
 determines what API to use to run the tests
 , so maybe it's
 usefulness depends on the user's machine, in which case a note about that
 would be useful. Something like: "If your IDE has trouble running tests via
 IDEA shortcuts, try the following steps: [...]"

 On Thu, Apr 12, 2018 at 3:29 AM Alexey Romanenko <
 aromanenko@gmail.com> wrote:

> Daniel, actually I did run it with default IDEA JUnit test runner.
> Then, in “Settings > Build, Execution, Deployment > Build Tools > Gradle >
> Runner" I selected “Gradle Test Runner” in “Run tests using” selectbox and
> it works ok when I run my tests with IDEA shortcuts. So, probably, we
> should add this details on
> https://beam.apache.org/contribute/intellij/ too.
> What do you think?
>
> WBR,
> Alexey
>
> On 11 Apr 2018, at 21:17, Daniel Oliveira 
> wrote:
>
> Alexey, are you referring to tests run with "./gradlew
> :beam-runners-direct-java:needsRunnerTests"? That command works fine for 
> me
> in both versions of IDEA, but I believe the same tests fail if you run 
> them
> directly through "./gradlew test".
>
> However, I am having issues with a bunch of validatesRunner tests,
> mostly be caused by
> :beam-runners-google-cloud-dataflow-java:validatesRunner. Not sure if it's
> because of a code change or a gradle config, I'll keep looking into it.
>
> On Wed, Apr 11, 2018 at 11:01 AM Romain Manni-Bucau <
> rmannibu...@gmail.com> wrote:
>
>> I got tests running rrconfiguring gradle (which was setup for another
>> project but seems beam didnt like it) but latency is still "high" using
>> gradle runner for tests (like Etienne said ~3s on an i7 with 16G vs a few
>> ms with default idea test runner, would be great to solve that).
>>
>> I also find the integration quite fishy cause configurations are
>> customs so idea is kind of forced to propose your modukes 3 times at 
>> least
>> when you select the classpath (x_test being generally the working one).
>>
>> Also the false positive you get if you forget a cleanX is a bit
>> annoying, maybe we should force a clean for test or at least when there 
>> is
>> a --tests to avoid gradle to not run it cause there is no diff.
>>
>> So it works but dev productivity is reduced a lot and it became slow
>> enough to think if you should do a contribution or not - at least for me.
>>
>> Le 11 avr. 2018 19:37, "Alexey Romanenko" 
>> a écrit :
>>
>>> I’ve managed to import a project as it’s described in documentation
>>> (starting from empty project) using Idea 2018 and run unit tests
>>> successfully.
>>> For some reasons, tests, that use DirectRunner to run a pipeline,
>>> were failed.
>>>
>>> WBR,
>>> Alexey
>>>
>>> On 11 Apr 2018, at 19:01, Daniel Oliveira 
>>> wrote:
>>>
>>> Hi everyone, I was the one who initially wrote the PR with Idea
>>> instructions . I was
>>> using 2017.3 as well while writing it so all the instructions were 
>>> tested
>>> on that version. I'll try testing the instructions on 2018 to see if I 
>>> can
>>> reproduce the issues people are having.
>>>
>>> On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik 
>>> wrote:
>>>
 I use 2017.3 and it has been reliable for me. I haven't tried 2018
 yet.

 On Wed, Apr 11, 2018 at 11:30 AM Romain 

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

Well you are the only one to not have the drawbacks to use it so maybe dont
do it? I know Luke is in holidays but anyone else with the knowledge of why
we nees that noise compared to idea native tooling/flow?

Le 12 avr. 2018 20:16, "Daniel Oliveira"  a écrit :

> Ah, I did not. Thanks Romain.
>
> I tried it again, restarting in between, and still had no differences.
> Since it seems like there's no reason not to use "Gradle Test Runner", I'll
> mention it in the contributor's guide.
>
> On Thu, Apr 12, 2018 at 10:31 AM Romain Manni-Bucau 
> wrote:
>
>> @Daniel: did you restart in between? Otherwise it does nothing. One
>> launches JunitCoreRunner from idea and the other a gradle command.
>>
>> Le 12 avr. 2018 19:24, "Daniel Oliveira"  a
>> écrit :
>>
>>> I think it depends on what exactly switching to "Gradle Test Runner"
>>> from "Platform Test Runner" does. I tried it out on my machine and they
>>> seem to act identically to each other. The IntelliJ documentation says it
>>> determines what API to use to run the tests
>>> , so maybe it's
>>> usefulness depends on the user's machine, in which case a note about that
>>> would be useful. Something like: "If your IDE has trouble running tests via
>>> IDEA shortcuts, try the following steps: [...]"
>>>
>>> On Thu, Apr 12, 2018 at 3:29 AM Alexey Romanenko <
>>> aromanenko@gmail.com> wrote:
>>>
 Daniel, actually I did run it with default IDEA JUnit test runner.
 Then, in “Settings > Build, Execution, Deployment > Build Tools > Gradle >
 Runner" I selected “Gradle Test Runner” in “Run tests using” selectbox and
 it works ok when I run my tests with IDEA shortcuts. So, probably, we
 should add this details on https://beam.apache.org/contribute/intellij/
  too.
 What do you think?

 WBR,
 Alexey

 On 11 Apr 2018, at 21:17, Daniel Oliveira 
 wrote:

 Alexey, are you referring to tests run with "./gradlew
 :beam-runners-direct-java:needsRunnerTests"? That command works fine
 for me in both versions of IDEA, but I believe the same tests fail if you
 run them directly through "./gradlew test".

 However, I am having issues with a bunch of validatesRunner tests,
 mostly be caused by 
 :beam-runners-google-cloud-dataflow-java:validatesRunner.
 Not sure if it's because of a code change or a gradle config, I'll keep
 looking into it.

 On Wed, Apr 11, 2018 at 11:01 AM Romain Manni-Bucau <
 rmannibu...@gmail.com> wrote:

> I got tests running rrconfiguring gradle (which was setup for another
> project but seems beam didnt like it) but latency is still "high" using
> gradle runner for tests (like Etienne said ~3s on an i7 with 16G vs a few
> ms with default idea test runner, would be great to solve that).
>
> I also find the integration quite fishy cause configurations are
> customs so idea is kind of forced to propose your modukes 3 times at least
> when you select the classpath (x_test being generally the working one).
>
> Also the false positive you get if you forget a cleanX is a bit
> annoying, maybe we should force a clean for test or at least when there is
> a --tests to avoid gradle to not run it cause there is no diff.
>
> So it works but dev productivity is reduced a lot and it became slow
> enough to think if you should do a contribution or not - at least for me.
>
> Le 11 avr. 2018 19:37, "Alexey Romanenko" 
> a écrit :
>
>> I’ve managed to import a project as it’s described in documentation
>> (starting from empty project) using Idea 2018 and run unit tests
>> successfully.
>> For some reasons, tests, that use DirectRunner to run a pipeline,
>> were failed.
>>
>> WBR,
>> Alexey
>>
>> On 11 Apr 2018, at 19:01, Daniel Oliveira 
>> wrote:
>>
>> Hi everyone, I was the one who initially wrote the PR with Idea
>> instructions . I was
>> using 2017.3 as well while writing it so all the instructions were tested
>> on that version. I'll try testing the instructions on 2018 to see if I 
>> can
>> reproduce the issues people are having.
>>
>> On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik  wrote:
>>
>>> I use 2017.3 and it has been reliable for me. I haven't tried 2018
>>> yet.
>>>
>>> On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau <
>>> rmannibu...@gmail.com> wrote:
>>>
 Any of you using the idea 2018? the import works for me but then it
 is
 not as smooth as it seems for you. I'm just trying to see if it is a
 procedure thing or a version issue.

 Romain Manni-Bucau

Re: Gradle Status [April 6]

[Daniel Oliveira]

Ah, I did not. Thanks Romain.

I tried it again, restarting in between, and still had no differences.
Since it seems like there's no reason not to use "Gradle Test Runner", I'll
mention it in the contributor's guide.

On Thu, Apr 12, 2018 at 10:31 AM Romain Manni-Bucau 
wrote:

> @Daniel: did you restart in between? Otherwise it does nothing. One
> launches JunitCoreRunner from idea and the other a gradle command.
>
> Le 12 avr. 2018 19:24, "Daniel Oliveira"  a
> écrit :
>
>> I think it depends on what exactly switching to "Gradle Test Runner" from
>> "Platform Test Runner" does. I tried it out on my machine and they seem to
>> act identically to each other. The IntelliJ documentation says it
>> determines what API to use to run the tests
>> , so maybe it's
>> usefulness depends on the user's machine, in which case a note about that
>> would be useful. Something like: "If your IDE has trouble running tests via
>> IDEA shortcuts, try the following steps: [...]"
>>
>> On Thu, Apr 12, 2018 at 3:29 AM Alexey Romanenko <
>> aromanenko@gmail.com> wrote:
>>
>>> Daniel, actually I did run it with default IDEA JUnit test runner. Then,
>>> in “Settings > Build, Execution, Deployment > Build Tools > Gradle >
>>> Runner" I selected “Gradle Test Runner” in “Run tests using” selectbox and
>>> it works ok when I run my tests with IDEA shortcuts. So, probably, we
>>> should add this details on https://beam.apache.org/contribute/intellij/
>>>  too.
>>> What do you think?
>>>
>>> WBR,
>>> Alexey
>>>
>>> On 11 Apr 2018, at 21:17, Daniel Oliveira 
>>> wrote:
>>>
>>> Alexey, are you referring to tests run with "./gradlew
>>> :beam-runners-direct-java:needsRunnerTests"? That command works fine for me
>>> in both versions of IDEA, but I believe the same tests fail if you run them
>>> directly through "./gradlew test".
>>>
>>> However, I am having issues with a bunch of validatesRunner tests,
>>> mostly be caused by
>>> :beam-runners-google-cloud-dataflow-java:validatesRunner. Not sure if it's
>>> because of a code change or a gradle config, I'll keep looking into it.
>>>
>>> On Wed, Apr 11, 2018 at 11:01 AM Romain Manni-Bucau <
>>> rmannibu...@gmail.com> wrote:
>>>
 I got tests running rrconfiguring gradle (which was setup for another
 project but seems beam didnt like it) but latency is still "high" using
 gradle runner for tests (like Etienne said ~3s on an i7 with 16G vs a few
 ms with default idea test runner, would be great to solve that).

 I also find the integration quite fishy cause configurations are
 customs so idea is kind of forced to propose your modukes 3 times at least
 when you select the classpath (x_test being generally the working one).

 Also the false positive you get if you forget a cleanX is a bit
 annoying, maybe we should force a clean for test or at least when there is
 a --tests to avoid gradle to not run it cause there is no diff.

 So it works but dev productivity is reduced a lot and it became slow
 enough to think if you should do a contribution or not - at least for me.

 Le 11 avr. 2018 19:37, "Alexey Romanenko"  a
 écrit :

> I’ve managed to import a project as it’s described in documentation
> (starting from empty project) using Idea 2018 and run unit tests
> successfully.
> For some reasons, tests, that use DirectRunner to run a pipeline, were
> failed.
>
> WBR,
> Alexey
>
> On 11 Apr 2018, at 19:01, Daniel Oliveira 
> wrote:
>
> Hi everyone, I was the one who initially wrote the PR with Idea
> instructions . I was
> using 2017.3 as well while writing it so all the instructions were tested
> on that version. I'll try testing the instructions on 2018 to see if I can
> reproduce the issues people are having.
>
> On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik  wrote:
>
>> I use 2017.3 and it has been reliable for me. I haven't tried 2018
>> yet.
>>
>> On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau <
>> rmannibu...@gmail.com> wrote:
>>
>>> Any of you using the idea 2018? the import works for me but then it
>>> is
>>> not as smooth as it seems for you. I'm just trying to see if it is a
>>> procedure thing or a version issue.
>>>
>>> Romain Manni-Bucau
>>> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>>
>>>
>>> 2018-04-11 17:28 GMT+02:00 Kenneth Knowles :
>>> > The only reason I did "empty project then add a module" procedure
>>> was to get
>>> > all the IntelliJ files outside the source tree. IIRC directly
>>> creating from
>>> > existing sources didn't give the necessary configuration options.

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

@Daniel: did you restart in between? Otherwise it does nothing. One
launches JunitCoreRunner from idea and the other a gradle command.

Le 12 avr. 2018 19:24, "Daniel Oliveira"  a écrit :

> I think it depends on what exactly switching to "Gradle Test Runner" from
> "Platform Test Runner" does. I tried it out on my machine and they seem to
> act identically to each other. The IntelliJ documentation says it
> determines what API to use to run the tests
> , so maybe it's
> usefulness depends on the user's machine, in which case a note about that
> would be useful. Something like: "If your IDE has trouble running tests via
> IDEA shortcuts, try the following steps: [...]"
>
> On Thu, Apr 12, 2018 at 3:29 AM Alexey Romanenko 
> wrote:
>
>> Daniel, actually I did run it with default IDEA JUnit test runner. Then,
>> in “Settings > Build, Execution, Deployment > Build Tools > Gradle >
>> Runner" I selected “Gradle Test Runner” in “Run tests using” selectbox and
>> it works ok when I run my tests with IDEA shortcuts. So, probably, we
>> should add this details on https://beam.apache.org/contribute/intellij/
>>  too.
>> What do you think?
>>
>> WBR,
>> Alexey
>>
>> On 11 Apr 2018, at 21:17, Daniel Oliveira  wrote:
>>
>> Alexey, are you referring to tests run with "./gradlew
>> :beam-runners-direct-java:needsRunnerTests"? That command works fine for
>> me in both versions of IDEA, but I believe the same tests fail if you run
>> them directly through "./gradlew test".
>>
>> However, I am having issues with a bunch of validatesRunner tests, mostly
>> be caused by :beam-runners-google-cloud-dataflow-java:validatesRunner.
>> Not sure if it's because of a code change or a gradle config, I'll keep
>> looking into it.
>>
>> On Wed, Apr 11, 2018 at 11:01 AM Romain Manni-Bucau <
>> rmannibu...@gmail.com> wrote:
>>
>>> I got tests running rrconfiguring gradle (which was setup for another
>>> project but seems beam didnt like it) but latency is still "high" using
>>> gradle runner for tests (like Etienne said ~3s on an i7 with 16G vs a few
>>> ms with default idea test runner, would be great to solve that).
>>>
>>> I also find the integration quite fishy cause configurations are customs
>>> so idea is kind of forced to propose your modukes 3 times at least when you
>>> select the classpath (x_test being generally the working one).
>>>
>>> Also the false positive you get if you forget a cleanX is a bit
>>> annoying, maybe we should force a clean for test or at least when there is
>>> a --tests to avoid gradle to not run it cause there is no diff.
>>>
>>> So it works but dev productivity is reduced a lot and it became slow
>>> enough to think if you should do a contribution or not - at least for me.
>>>
>>> Le 11 avr. 2018 19:37, "Alexey Romanenko"  a
>>> écrit :
>>>
 I’ve managed to import a project as it’s described in documentation
 (starting from empty project) using Idea 2018 and run unit tests
 successfully.
 For some reasons, tests, that use DirectRunner to run a pipeline, were
 failed.

 WBR,
 Alexey

 On 11 Apr 2018, at 19:01, Daniel Oliveira 
 wrote:

 Hi everyone, I was the one who initially wrote the PR with Idea
 instructions . I was
 using 2017.3 as well while writing it so all the instructions were tested
 on that version. I'll try testing the instructions on 2018 to see if I can
 reproduce the issues people are having.

 On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik  wrote:

> I use 2017.3 and it has been reliable for me. I haven't tried 2018 yet.
>
> On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau <
> rmannibu...@gmail.com> wrote:
>
>> Any of you using the idea 2018? the import works for me but then it is
>> not as smooth as it seems for you. I'm just trying to see if it is a
>> procedure thing or a version issue.
>>
>> Romain Manni-Bucau
>> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>
>>
>> 2018-04-11 17:28 GMT+02:00 Kenneth Knowles :
>> > The only reason I did "empty project then add a module" procedure
>> was to get
>> > all the IntelliJ files outside the source tree. IIRC directly
>> creating from
>> > existing sources didn't give the necessary configuration options.
>> If you
>> > don't care about being able to `git clean` then you can do the
>> shorter
>> > version. Also the particular UI for creation might have improved
>> since I
>> > tried it. I'll do it again.
>> >
>> > On the pom, since it is only generated for machine reading, why
>> care if the
>> > parent is inlined? I actually prefer to avoid coupling with things
>> that you
>> 

Re: Gradle Status [April 6]

[Daniel Oliveira]

I think it depends on what exactly switching to "Gradle Test Runner" from
"Platform Test Runner" does. I tried it out on my machine and they seem to
act identically to each other. The IntelliJ documentation says it
determines what API to use to run the tests
, so maybe it's usefulness
depends on the user's machine, in which case a note about that would be
useful. Something like: "If your IDE has trouble running tests via IDEA
shortcuts, try the following steps: [...]"

On Thu, Apr 12, 2018 at 3:29 AM Alexey Romanenko 
wrote:

> Daniel, actually I did run it with default IDEA JUnit test runner. Then,
> in “Settings > Build, Execution, Deployment > Build Tools > Gradle >
> Runner" I selected “Gradle Test Runner” in “Run tests using” selectbox and
> it works ok when I run my tests with IDEA shortcuts. So, probably, we
> should add this details on https://beam.apache.org/contribute/intellij/
>  too.
> What do you think?
>
> WBR,
> Alexey
>
> On 11 Apr 2018, at 21:17, Daniel Oliveira  wrote:
>
> Alexey, are you referring to tests run with "./gradlew
> :beam-runners-direct-java:needsRunnerTests"? That command works fine for me
> in both versions of IDEA, but I believe the same tests fail if you run them
> directly through "./gradlew test".
>
> However, I am having issues with a bunch of validatesRunner tests, mostly
> be caused by :beam-runners-google-cloud-dataflow-java:validatesRunner. Not
> sure if it's because of a code change or a gradle config, I'll keep looking
> into it.
>
> On Wed, Apr 11, 2018 at 11:01 AM Romain Manni-Bucau 
> wrote:
>
>> I got tests running rrconfiguring gradle (which was setup for another
>> project but seems beam didnt like it) but latency is still "high" using
>> gradle runner for tests (like Etienne said ~3s on an i7 with 16G vs a few
>> ms with default idea test runner, would be great to solve that).
>>
>> I also find the integration quite fishy cause configurations are customs
>> so idea is kind of forced to propose your modukes 3 times at least when you
>> select the classpath (x_test being generally the working one).
>>
>> Also the false positive you get if you forget a cleanX is a bit annoying,
>> maybe we should force a clean for test or at least when there is a --tests
>> to avoid gradle to not run it cause there is no diff.
>>
>> So it works but dev productivity is reduced a lot and it became slow
>> enough to think if you should do a contribution or not - at least for me.
>>
>> Le 11 avr. 2018 19:37, "Alexey Romanenko"  a
>> écrit :
>>
>>> I’ve managed to import a project as it’s described in documentation
>>> (starting from empty project) using Idea 2018 and run unit tests
>>> successfully.
>>> For some reasons, tests, that use DirectRunner to run a pipeline, were
>>> failed.
>>>
>>> WBR,
>>> Alexey
>>>
>>> On 11 Apr 2018, at 19:01, Daniel Oliveira 
>>> wrote:
>>>
>>> Hi everyone, I was the one who initially wrote the PR with Idea
>>> instructions . I was
>>> using 2017.3 as well while writing it so all the instructions were tested
>>> on that version. I'll try testing the instructions on 2018 to see if I can
>>> reproduce the issues people are having.
>>>
>>> On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik  wrote:
>>>
 I use 2017.3 and it has been reliable for me. I haven't tried 2018 yet.

 On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau <
 rmannibu...@gmail.com> wrote:

> Any of you using the idea 2018? the import works for me but then it is
> not as smooth as it seems for you. I'm just trying to see if it is a
> procedure thing or a version issue.
>
> Romain Manni-Bucau
> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>
>
> 2018-04-11 17:28 GMT+02:00 Kenneth Knowles :
> > The only reason I did "empty project then add a module" procedure
> was to get
> > all the IntelliJ files outside the source tree. IIRC directly
> creating from
> > existing sources didn't give the necessary configuration options. If
> you
> > don't care about being able to `git clean` then you can do the
> shorter
> > version. Also the particular UI for creation might have improved
> since I
> > tried it. I'll do it again.
> >
> > On the pom, since it is only generated for machine reading, why care
> if the
> > parent is inlined? I actually prefer to avoid coupling with things
> that you
> > have to go and look up. Using inheritance is OK for human edited poms
> > (actually IMO it is still a mistake) but it doesn't change the
> semantics of
> > a shipped pom if they are all immutable, which they should be. It is
> better
> > to have all the info right there. Is there an actually effective
> 

Re: Gradle Status [April 6]

[Alexey Romanenko]

Daniel, actually I did run it with default IDEA JUnit test runner. Then, in 
“Settings > Build, Execution, Deployment > Build Tools > Gradle > Runner" I 
selected “Gradle Test Runner” in “Run tests using” selectbox and it works ok 
when I run my tests with IDEA shortcuts. So, probably, we should add this 
details on https://beam.apache.org/contribute/intellij/ 
 too.
What do you think?

WBR,
Alexey

> On 11 Apr 2018, at 21:17, Daniel Oliveira  wrote:
> 
> Alexey, are you referring to tests run with "./gradlew 
> :beam-runners-direct-java:needsRunnerTests"? That command works fine for me 
> in both versions of IDEA, but I believe the same tests fail if you run them 
> directly through "./gradlew test".
> 
> However, I am having issues with a bunch of validatesRunner tests, mostly be 
> caused by :beam-runners-google-cloud-dataflow-java:validatesRunner. Not sure 
> if it's because of a code change or a gradle config, I'll keep looking into 
> it.
> 
> On Wed, Apr 11, 2018 at 11:01 AM Romain Manni-Bucau  > wrote:
> I got tests running rrconfiguring gradle (which was setup for another project 
> but seems beam didnt like it) but latency is still "high" using gradle runner 
> for tests (like Etienne said ~3s on an i7 with 16G vs a few ms with default 
> idea test runner, would be great to solve that).
> 
> I also find the integration quite fishy cause configurations are customs so 
> idea is kind of forced to propose your modukes 3 times at least when you 
> select the classpath (x_test being generally the working one).
> 
> Also the false positive you get if you forget a cleanX is a bit annoying, 
> maybe we should force a clean for test or at least when there is a --tests to 
> avoid gradle to not run it cause there is no diff.
> 
> So it works but dev productivity is reduced a lot and it became slow enough 
> to think if you should do a contribution or not - at least for me.
> 
> Le 11 avr. 2018 19:37, "Alexey Romanenko"  > a écrit :
> I’ve managed to import a project as it’s described in documentation (starting 
> from empty project) using Idea 2018 and run unit tests successfully. 
> For some reasons, tests, that use DirectRunner to run a pipeline, were failed.
> 
> WBR,
> Alexey
> 
>> On 11 Apr 2018, at 19:01, Daniel Oliveira > > wrote:
>> 
>> Hi everyone, I was the one who initially wrote the PR with Idea instructions 
>> . I was using 2017.3 as well 
>> while writing it so all the instructions were tested on that version. I'll 
>> try testing the instructions on 2018 to see if I can reproduce the issues 
>> people are having.
>> 
>> On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik > > wrote:
>> I use 2017.3 and it has been reliable for me. I haven't tried 2018 yet.
>> 
>> On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau > > wrote:
>> Any of you using the idea 2018? the import works for me but then it is
>> not as smooth as it seems for you. I'm just trying to see if it is a
>> procedure thing or a version issue.
>> 
>> Romain Manni-Bucau
>> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>> 
>> 
>> 2018-04-11 17:28 GMT+02:00 Kenneth Knowles > >:
>> > The only reason I did "empty project then add a module" procedure was to 
>> > get
>> > all the IntelliJ files outside the source tree. IIRC directly creating from
>> > existing sources didn't give the necessary configuration options. If you
>> > don't care about being able to `git clean` then you can do the shorter
>> > version. Also the particular UI for creation might have improved since I
>> > tried it. I'll do it again.
>> >
>> > On the pom, since it is only generated for machine reading, why care if the
>> > parent is inlined? I actually prefer to avoid coupling with things that you
>> > have to go and look up. Using inheritance is OK for human edited poms
>> > (actually IMO it is still a mistake) but it doesn't change the semantics of
>> > a shipped pom if they are all immutable, which they should be. It is better
>> > to have all the info right there. Is there an actually effective 
>> > difference?
>> >
>> > Kenn
>> >
>> > On Wed, Apr 11, 2018 at 6:53 AM Etienne Chauchot > > >
>> > wrote:
>> >>
>> >> Hi all,
>> >> I just tested gradle environment from a fresh source clone with this
>> >> procedure with just a tiny change: I used "new project from existing
>> >> sources" rather than create empty project and then add module.
>> >>
>> >> It works fine and junit runs from intellij also work.  with gradle we pay
>> >> a 2s delay (on my machine) before running the 

Re: Gradle Status [April 6]

[Etienne Chauchot]

My test was also on version 2017, 2017.2.7 to be precise
Etienne
Le mercredi 11 avril 2018 à 17:01 +, Daniel Oliveira a écrit :
> Hi everyone, I was the one who initially wrote the PR with Idea instructions. 
> I was using 2017.3 as well while writing
> it so all the instructions were tested on that version. I'll try testing the 
> instructions on 2018 to see if I can
> reproduce the issues people are having.
> 
> On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik  wrote:
> > I use 2017.3 and it has been reliable for me. I haven't tried 2018 yet.
> > 
> > On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau  
> > wrote:
> > > Any of you using the idea 2018? the import works for me but then it is
> > > not as smooth as it seems for you. I'm just trying to see if it is a
> > > procedure thing or a version issue.
> > > 
> > > Romain Manni-Bucau
> > > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> > > 
> > > 
> > > 2018-04-11 17:28 GMT+02:00 Kenneth Knowles :
> > > > The only reason I did "empty project then add a module" procedure was 
> > > > to get
> > > > all the IntelliJ files outside the source tree. IIRC directly creating 
> > > > from
> > > > existing sources didn't give the necessary configuration options. If you
> > > > don't care about being able to `git clean` then you can do the shorter
> > > > version. Also the particular UI for creation might have improved since I
> > > > tried it. I'll do it again.
> > > >
> > > > On the pom, since it is only generated for machine reading, why care if 
> > > > the
> > > > parent is inlined? I actually prefer to avoid coupling with things that 
> > > > you
> > > > have to go and look up. Using inheritance is OK for human edited poms
> > > > (actually IMO it is still a mistake) but it doesn't change the 
> > > > semantics of
> > > > a shipped pom if they are all immutable, which they should be. It is 
> > > > better
> > > > to have all the info right there. Is there an actually effective 
> > > > difference?
> > > >
> > > > Kenn
> > > >
> > > > On Wed, Apr 11, 2018 at 6:53 AM Etienne Chauchot 
> > > > wrote:
> > > >>
> > > >> Hi all,
> > > >> I just tested gradle environment from a fresh source clone with this
> > > >> procedure with just a tiny change: I used "new project from existing
> > > >> sources" rather than create empty project and then add module.
> > > >>
> > > >> It works fine and junit runs from intellij also work.  with gradle we 
> > > >> pay
> > > >> a 2s delay (on my machine) before running the actual test for each 
> > > >> run. This
> > > >> delay seems quite constant no matter the module: I have run all the 
> > > >> tests at
> > > >> once in  runner-core and a single test in another module with a similar
> > > >> delay.
> > > >>
> > > >> I also tested a debug session from intellij and it runs fine also.
> > > >>
> > > >> I'll do some more testing and keep you posted.
> > > >>
> > > >> I still have some concerns about the potential difficulty for new
> > > >> contributors to have to learn gradle in addition to Beam itself 
> > > >> comparing to
> > > >> maven which is more mainstream for java developers. That could be
> > > >> discouraging for ex for part-time contributors
> > > >>
> > > >> Etienne
> > > >>
> > > >> Le mardi 10 avril 2018 à 14:38 +, Lukasz Cwik a écrit :
> > > >>
> > > >> beam-site PR/414 updates the instructions for using Intellij and how to
> > > >> import a module:
> > > >>
> > > >> 1. Create an empty IntelliJ project outside of the Beam source tree.
> > > >> 2. Under Project Structure > Project, select a Project SDK.
> > > >> 3. Under Project Structure > Modules, click the + sign to add a module 
> > > >> and
> > > >>    select "Import Module".
> > > >>     1. Select the directory containing the Beam source tree.
> > > >>     2. Tick the "Import module from external model" button and select
> > > >> Gradle
> > > >>        from the list.
> > > >>     3. Tick the following boxes.
> > > >>        * Use auto-import
> > > >>        * Create separate module per source set
> > > >>        * Store generated project files externally
> > > >>        * Use default gradle wrapper
> > > >> 4. Delegate build actions to Gradle by going to Settings > Build,
> > > >> Execution,
> > > >>    Deployment > Build Tools > Gradle and checking "Delegate IDE 
> > > >>build/run
> > > >>    actions to gradle".
> > > >>
> > > >> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré 
> > > >> 
> > > >> wrote:
> > > >>
> > > >> That's a very important issue for contribution. Up to now, I used Maven
> > > >> for setup IntelliJ (and it works just fine). If we remove the pom.xml,
> > > >> we have to support Eclipse and IntelliJ "smoothly".
> > > >>
> > > >> Let me try in IntelliJ.
> > > >>
> > > >> Regards
> > > >> JB
> > > >>
> > > >> On 10/04/2018 15:21, Romain Manni-Bucau wrote:
> > > >> > You dont have issue due to the build setup with that 

Re: Gradle Status [April 6]

[Lukasz Cwik]

Running Dataflow validates runner tests requires you to have permission to
launch jobs on the 'apache-beam-testing' project.
You'll need to override dataflowProject and dataflowTempRoot with a GCP
project and GCS bucket you have access to.

On Wed, Apr 11, 2018 at 3:17 PM Daniel Oliveira 
wrote:

> Alexey, are you referring to tests run with "./gradlew
> :beam-runners-direct-java:needsRunnerTests"? That command works fine for me
> in both versions of IDEA, but I believe the same tests fail if you run them
> directly through "./gradlew test".
>
> However, I am having issues with a bunch of validatesRunner tests, mostly
> be caused by :beam-runners-google-cloud-dataflow-java:validatesRunner. Not
> sure if it's because of a code change or a gradle config, I'll keep looking
> into it.
>
> On Wed, Apr 11, 2018 at 11:01 AM Romain Manni-Bucau 
> wrote:
>
>> I got tests running rrconfiguring gradle (which was setup for another
>> project but seems beam didnt like it) but latency is still "high" using
>> gradle runner for tests (like Etienne said ~3s on an i7 with 16G vs a few
>> ms with default idea test runner, would be great to solve that).
>>
>> I also find the integration quite fishy cause configurations are customs
>> so idea is kind of forced to propose your modukes 3 times at least when you
>> select the classpath (x_test being generally the working one).
>>
>> Also the false positive you get if you forget a cleanX is a bit annoying,
>> maybe we should force a clean for test or at least when there is a --tests
>> to avoid gradle to not run it cause there is no diff.
>>
>> So it works but dev productivity is reduced a lot and it became slow
>> enough to think if you should do a contribution or not - at least for me.
>>
>> Le 11 avr. 2018 19:37, "Alexey Romanenko"  a
>> écrit :
>>
>>> I’ve managed to import a project as it’s described in documentation
>>> (starting from empty project) using Idea 2018 and run unit tests
>>> successfully.
>>> For some reasons, tests, that use DirectRunner to run a pipeline, were
>>> failed.
>>>
>>> WBR,
>>> Alexey
>>>
>>> On 11 Apr 2018, at 19:01, Daniel Oliveira 
>>> wrote:
>>>
>>> Hi everyone, I was the one who initially wrote the PR with Idea
>>> instructions . I was
>>> using 2017.3 as well while writing it so all the instructions were tested
>>> on that version. I'll try testing the instructions on 2018 to see if I can
>>> reproduce the issues people are having.
>>>
>>> On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik  wrote:
>>>
 I use 2017.3 and it has been reliable for me. I haven't tried 2018 yet.

 On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau <
 rmannibu...@gmail.com> wrote:

> Any of you using the idea 2018? the import works for me but then it is
> not as smooth as it seems for you. I'm just trying to see if it is a
> procedure thing or a version issue.
>
> Romain Manni-Bucau
> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>
>
> 2018-04-11 17:28 GMT+02:00 Kenneth Knowles :
> > The only reason I did "empty project then add a module" procedure
> was to get
> > all the IntelliJ files outside the source tree. IIRC directly
> creating from
> > existing sources didn't give the necessary configuration options. If
> you
> > don't care about being able to `git clean` then you can do the
> shorter
> > version. Also the particular UI for creation might have improved
> since I
> > tried it. I'll do it again.
> >
> > On the pom, since it is only generated for machine reading, why care
> if the
> > parent is inlined? I actually prefer to avoid coupling with things
> that you
> > have to go and look up. Using inheritance is OK for human edited poms
> > (actually IMO it is still a mistake) but it doesn't change the
> semantics of
> > a shipped pom if they are all immutable, which they should be. It is
> better
> > to have all the info right there. Is there an actually effective
> difference?
> >
> > Kenn
> >
> > On Wed, Apr 11, 2018 at 6:53 AM Etienne Chauchot <
> echauc...@apache.org>
> > wrote:
> >>
> >> Hi all,
> >> I just tested gradle environment from a fresh source clone with this
> >> procedure with just a tiny change: I used "new project from existing
> >> sources" rather than create empty project and then add module.
> >>
> >> It works fine and junit runs from intellij also work.  with gradle
> we pay
> >> a 2s delay (on my machine) before running the actual test for each
> run. This
> >> delay seems quite constant no matter the module: I have run all the
> tests at
> >> once in  runner-core and a single test in another module with a
> similar
> >> delay.

Re: Gradle Status [April 6]

[Daniel Oliveira]

Alexey, are you referring to tests run with "./gradlew
:beam-runners-direct-java:needsRunnerTests"? That command works fine for me
in both versions of IDEA, but I believe the same tests fail if you run them
directly through "./gradlew test".

However, I am having issues with a bunch of validatesRunner tests, mostly
be caused by :beam-runners-google-cloud-dataflow-java:validatesRunner. Not
sure if it's because of a code change or a gradle config, I'll keep looking
into it.

On Wed, Apr 11, 2018 at 11:01 AM Romain Manni-Bucau 
wrote:

> I got tests running rrconfiguring gradle (which was setup for another
> project but seems beam didnt like it) but latency is still "high" using
> gradle runner for tests (like Etienne said ~3s on an i7 with 16G vs a few
> ms with default idea test runner, would be great to solve that).
>
> I also find the integration quite fishy cause configurations are customs
> so idea is kind of forced to propose your modukes 3 times at least when you
> select the classpath (x_test being generally the working one).
>
> Also the false positive you get if you forget a cleanX is a bit annoying,
> maybe we should force a clean for test or at least when there is a --tests
> to avoid gradle to not run it cause there is no diff.
>
> So it works but dev productivity is reduced a lot and it became slow
> enough to think if you should do a contribution or not - at least for me.
>
> Le 11 avr. 2018 19:37, "Alexey Romanenko"  a
> écrit :
>
>> I’ve managed to import a project as it’s described in documentation
>> (starting from empty project) using Idea 2018 and run unit tests
>> successfully.
>> For some reasons, tests, that use DirectRunner to run a pipeline, were
>> failed.
>>
>> WBR,
>> Alexey
>>
>> On 11 Apr 2018, at 19:01, Daniel Oliveira  wrote:
>>
>> Hi everyone, I was the one who initially wrote the PR with Idea
>> instructions . I was using
>> 2017.3 as well while writing it so all the instructions were tested on that
>> version. I'll try testing the instructions on 2018 to see if I can
>> reproduce the issues people are having.
>>
>> On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik  wrote:
>>
>>> I use 2017.3 and it has been reliable for me. I haven't tried 2018 yet.
>>>
>>> On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau <
>>> rmannibu...@gmail.com> wrote:
>>>
 Any of you using the idea 2018? the import works for me but then it is
 not as smooth as it seems for you. I'm just trying to see if it is a
 procedure thing or a version issue.

 Romain Manni-Bucau
 @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book


 2018-04-11 17:28 GMT+02:00 Kenneth Knowles :
 > The only reason I did "empty project then add a module" procedure was
 to get
 > all the IntelliJ files outside the source tree. IIRC directly
 creating from
 > existing sources didn't give the necessary configuration options. If
 you
 > don't care about being able to `git clean` then you can do the shorter
 > version. Also the particular UI for creation might have improved
 since I
 > tried it. I'll do it again.
 >
 > On the pom, since it is only generated for machine reading, why care
 if the
 > parent is inlined? I actually prefer to avoid coupling with things
 that you
 > have to go and look up. Using inheritance is OK for human edited poms
 > (actually IMO it is still a mistake) but it doesn't change the
 semantics of
 > a shipped pom if they are all immutable, which they should be. It is
 better
 > to have all the info right there. Is there an actually effective
 difference?
 >
 > Kenn
 >
 > On Wed, Apr 11, 2018 at 6:53 AM Etienne Chauchot <
 echauc...@apache.org>
 > wrote:
 >>
 >> Hi all,
 >> I just tested gradle environment from a fresh source clone with this
 >> procedure with just a tiny change: I used "new project from existing
 >> sources" rather than create empty project and then add module.
 >>
 >> It works fine and junit runs from intellij also work.  with gradle
 we pay
 >> a 2s delay (on my machine) before running the actual test for each
 run. This
 >> delay seems quite constant no matter the module: I have run all the
 tests at
 >> once in  runner-core and a single test in another module with a
 similar
 >> delay.
 >>
 >> I also tested a debug session from intellij and it runs fine also.
 >>
 >> I'll do some more testing and keep you posted.
 >>
 >> I still have some concerns about the potential difficulty for new
 >> contributors to have to learn gradle in addition to Beam itself
 comparing to
 >> maven which is more mainstream for java developers. That could be
 >> discouraging for ex for part-time contributors

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

I got tests running rrconfiguring gradle (which was setup for another
project but seems beam didnt like it) but latency is still "high" using
gradle runner for tests (like Etienne said ~3s on an i7 with 16G vs a few
ms with default idea test runner, would be great to solve that).

I also find the integration quite fishy cause configurations are customs so
idea is kind of forced to propose your modukes 3 times at least when you
select the classpath (x_test being generally the working one).

Also the false positive you get if you forget a cleanX is a bit annoying,
maybe we should force a clean for test or at least when there is a --tests
to avoid gradle to not run it cause there is no diff.

So it works but dev productivity is reduced a lot and it became slow enough
to think if you should do a contribution or not - at least for me.

Le 11 avr. 2018 19:37, "Alexey Romanenko"  a
écrit :

> I’ve managed to import a project as it’s described in documentation
> (starting from empty project) using Idea 2018 and run unit tests
> successfully.
> For some reasons, tests, that use DirectRunner to run a pipeline, were
> failed.
>
> WBR,
> Alexey
>
> On 11 Apr 2018, at 19:01, Daniel Oliveira  wrote:
>
> Hi everyone, I was the one who initially wrote the PR with Idea
> instructions . I was using
> 2017.3 as well while writing it so all the instructions were tested on that
> version. I'll try testing the instructions on 2018 to see if I can
> reproduce the issues people are having.
>
> On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik  wrote:
>
>> I use 2017.3 and it has been reliable for me. I haven't tried 2018 yet.
>>
>> On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau <
>> rmannibu...@gmail.com> wrote:
>>
>>> Any of you using the idea 2018? the import works for me but then it is
>>> not as smooth as it seems for you. I'm just trying to see if it is a
>>> procedure thing or a version issue.
>>>
>>> Romain Manni-Bucau
>>> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>>
>>>
>>> 2018-04-11 17:28 GMT+02:00 Kenneth Knowles :
>>> > The only reason I did "empty project then add a module" procedure was
>>> to get
>>> > all the IntelliJ files outside the source tree. IIRC directly creating
>>> from
>>> > existing sources didn't give the necessary configuration options. If
>>> you
>>> > don't care about being able to `git clean` then you can do the shorter
>>> > version. Also the particular UI for creation might have improved since
>>> I
>>> > tried it. I'll do it again.
>>> >
>>> > On the pom, since it is only generated for machine reading, why care
>>> if the
>>> > parent is inlined? I actually prefer to avoid coupling with things
>>> that you
>>> > have to go and look up. Using inheritance is OK for human edited poms
>>> > (actually IMO it is still a mistake) but it doesn't change the
>>> semantics of
>>> > a shipped pom if they are all immutable, which they should be. It is
>>> better
>>> > to have all the info right there. Is there an actually effective
>>> difference?
>>> >
>>> > Kenn
>>> >
>>> > On Wed, Apr 11, 2018 at 6:53 AM Etienne Chauchot >> >
>>> > wrote:
>>> >>
>>> >> Hi all,
>>> >> I just tested gradle environment from a fresh source clone with this
>>> >> procedure with just a tiny change: I used "new project from existing
>>> >> sources" rather than create empty project and then add module.
>>> >>
>>> >> It works fine and junit runs from intellij also work.  with gradle we
>>> pay
>>> >> a 2s delay (on my machine) before running the actual test for each
>>> run. This
>>> >> delay seems quite constant no matter the module: I have run all the
>>> tests at
>>> >> once in  runner-core and a single test in another module with a
>>> similar
>>> >> delay.
>>> >>
>>> >> I also tested a debug session from intellij and it runs fine also.
>>> >>
>>> >> I'll do some more testing and keep you posted.
>>> >>
>>> >> I still have some concerns about the potential difficulty for new
>>> >> contributors to have to learn gradle in addition to Beam itself
>>> comparing to
>>> >> maven which is more mainstream for java developers. That could be
>>> >> discouraging for ex for part-time contributors
>>> >>
>>> >> Etienne
>>> >>
>>> >> Le mardi 10 avril 2018 à 14:38 +, Lukasz Cwik a écrit :
>>> >>
>>> >> beam-site PR/414 updates the instructions for using Intellij and how
>>> to
>>> >> import a module:
>>> >>
>>> >> 1. Create an empty IntelliJ project outside of the Beam source tree.
>>> >> 2. Under Project Structure > Project, select a Project SDK.
>>> >> 3. Under Project Structure > Modules, click the + sign to add a
>>> module and
>>> >>select "Import Module".
>>> >> 1. Select the directory containing the Beam source tree.
>>> >> 2. Tick the "Import module from external model" button and select
>>> >> Gradle
>>> >>from the list.
>>> >> 3. 

Re: Gradle Status [April 6]

[Alexey Romanenko]

I’ve managed to import a project as it’s described in documentation (starting 
from empty project) using Idea 2018 and run unit tests successfully. 
For some reasons, tests, that use DirectRunner to run a pipeline, were failed.

WBR,
Alexey

> On 11 Apr 2018, at 19:01, Daniel Oliveira  wrote:
> 
> Hi everyone, I was the one who initially wrote the PR with Idea instructions 
> . I was using 2017.3 as well 
> while writing it so all the instructions were tested on that version. I'll 
> try testing the instructions on 2018 to see if I can reproduce the issues 
> people are having.
> 
> On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik  > wrote:
> I use 2017.3 and it has been reliable for me. I haven't tried 2018 yet.
> 
> On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau  > wrote:
> Any of you using the idea 2018? the import works for me but then it is
> not as smooth as it seems for you. I'm just trying to see if it is a
> procedure thing or a version issue.
> 
> Romain Manni-Bucau
> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> 
> 
> 2018-04-11 17:28 GMT+02:00 Kenneth Knowles  >:
> > The only reason I did "empty project then add a module" procedure was to get
> > all the IntelliJ files outside the source tree. IIRC directly creating from
> > existing sources didn't give the necessary configuration options. If you
> > don't care about being able to `git clean` then you can do the shorter
> > version. Also the particular UI for creation might have improved since I
> > tried it. I'll do it again.
> >
> > On the pom, since it is only generated for machine reading, why care if the
> > parent is inlined? I actually prefer to avoid coupling with things that you
> > have to go and look up. Using inheritance is OK for human edited poms
> > (actually IMO it is still a mistake) but it doesn't change the semantics of
> > a shipped pom if they are all immutable, which they should be. It is better
> > to have all the info right there. Is there an actually effective difference?
> >
> > Kenn
> >
> > On Wed, Apr 11, 2018 at 6:53 AM Etienne Chauchot  > >
> > wrote:
> >>
> >> Hi all,
> >> I just tested gradle environment from a fresh source clone with this
> >> procedure with just a tiny change: I used "new project from existing
> >> sources" rather than create empty project and then add module.
> >>
> >> It works fine and junit runs from intellij also work.  with gradle we pay
> >> a 2s delay (on my machine) before running the actual test for each run. 
> >> This
> >> delay seems quite constant no matter the module: I have run all the tests 
> >> at
> >> once in  runner-core and a single test in another module with a similar
> >> delay.
> >>
> >> I also tested a debug session from intellij and it runs fine also.
> >>
> >> I'll do some more testing and keep you posted.
> >>
> >> I still have some concerns about the potential difficulty for new
> >> contributors to have to learn gradle in addition to Beam itself comparing 
> >> to
> >> maven which is more mainstream for java developers. That could be
> >> discouraging for ex for part-time contributors
> >>
> >> Etienne
> >>
> >> Le mardi 10 avril 2018 à 14:38 +, Lukasz Cwik a écrit :
> >>
> >> beam-site PR/414 updates the instructions for using Intellij and how to
> >> import a module:
> >>
> >> 1. Create an empty IntelliJ project outside of the Beam source tree.
> >> 2. Under Project Structure > Project, select a Project SDK.
> >> 3. Under Project Structure > Modules, click the + sign to add a module and
> >>select "Import Module".
> >> 1. Select the directory containing the Beam source tree.
> >> 2. Tick the "Import module from external model" button and select
> >> Gradle
> >>from the list.
> >> 3. Tick the following boxes.
> >>* Use auto-import
> >>* Create separate module per source set
> >>* Store generated project files externally
> >>* Use default gradle wrapper
> >> 4. Delegate build actions to Gradle by going to Settings > Build,
> >> Execution,
> >>Deployment > Build Tools > Gradle and checking "Delegate IDE build/run
> >>actions to gradle".
> >>
> >> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré  >> >
> >> wrote:
> >>
> >> That's a very important issue for contribution. Up to now, I used Maven
> >> for setup IntelliJ (and it works just fine). If we remove the pom.xml,
> >> we have to support Eclipse and IntelliJ "smoothly".
> >>
> >> Let me try in IntelliJ.
> >>
> >> Regards
> >> JB
> >>
> >> On 10/04/2018 15:21, Romain Manni-Bucau wrote:
> >> > You dont have issue due to the build setup with that option. I get:
> >> >
> >> > avr. 10, 2018 3:20:10 PM
> >> > 

Re: Gradle Status [April 6]

[Daniel Oliveira]

Hi everyone, I was the one who initially wrote the PR with Idea instructions
. I was using 2017.3 as well
while writing it so all the instructions were tested on that version. I'll
try testing the instructions on 2018 to see if I can reproduce the issues
people are having.

On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik  wrote:

> I use 2017.3 and it has been reliable for me. I haven't tried 2018 yet.
>
> On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau 
> wrote:
>
>> Any of you using the idea 2018? the import works for me but then it is
>> not as smooth as it seems for you. I'm just trying to see if it is a
>> procedure thing or a version issue.
>>
>> Romain Manni-Bucau
>> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>
>>
>> 2018-04-11 17:28 GMT+02:00 Kenneth Knowles :
>> > The only reason I did "empty project then add a module" procedure was
>> to get
>> > all the IntelliJ files outside the source tree. IIRC directly creating
>> from
>> > existing sources didn't give the necessary configuration options. If you
>> > don't care about being able to `git clean` then you can do the shorter
>> > version. Also the particular UI for creation might have improved since I
>> > tried it. I'll do it again.
>> >
>> > On the pom, since it is only generated for machine reading, why care if
>> the
>> > parent is inlined? I actually prefer to avoid coupling with things that
>> you
>> > have to go and look up. Using inheritance is OK for human edited poms
>> > (actually IMO it is still a mistake) but it doesn't change the
>> semantics of
>> > a shipped pom if they are all immutable, which they should be. It is
>> better
>> > to have all the info right there. Is there an actually effective
>> difference?
>> >
>> > Kenn
>> >
>> > On Wed, Apr 11, 2018 at 6:53 AM Etienne Chauchot 
>> > wrote:
>> >>
>> >> Hi all,
>> >> I just tested gradle environment from a fresh source clone with this
>> >> procedure with just a tiny change: I used "new project from existing
>> >> sources" rather than create empty project and then add module.
>> >>
>> >> It works fine and junit runs from intellij also work.  with gradle we
>> pay
>> >> a 2s delay (on my machine) before running the actual test for each
>> run. This
>> >> delay seems quite constant no matter the module: I have run all the
>> tests at
>> >> once in  runner-core and a single test in another module with a similar
>> >> delay.
>> >>
>> >> I also tested a debug session from intellij and it runs fine also.
>> >>
>> >> I'll do some more testing and keep you posted.
>> >>
>> >> I still have some concerns about the potential difficulty for new
>> >> contributors to have to learn gradle in addition to Beam itself
>> comparing to
>> >> maven which is more mainstream for java developers. That could be
>> >> discouraging for ex for part-time contributors
>> >>
>> >> Etienne
>> >>
>> >> Le mardi 10 avril 2018 à 14:38 +, Lukasz Cwik a écrit :
>> >>
>> >> beam-site PR/414 updates the instructions for using Intellij and how to
>> >> import a module:
>> >>
>> >> 1. Create an empty IntelliJ project outside of the Beam source tree.
>> >> 2. Under Project Structure > Project, select a Project SDK.
>> >> 3. Under Project Structure > Modules, click the + sign to add a module
>> and
>> >>select "Import Module".
>> >> 1. Select the directory containing the Beam source tree.
>> >> 2. Tick the "Import module from external model" button and select
>> >> Gradle
>> >>from the list.
>> >> 3. Tick the following boxes.
>> >>* Use auto-import
>> >>* Create separate module per source set
>> >>* Store generated project files externally
>> >>* Use default gradle wrapper
>> >> 4. Delegate build actions to Gradle by going to Settings > Build,
>> >> Execution,
>> >>Deployment > Build Tools > Gradle and checking "Delegate IDE
>> build/run
>> >>actions to gradle".
>> >>
>> >> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré > >
>> >> wrote:
>> >>
>> >> That's a very important issue for contribution. Up to now, I used Maven
>> >> for setup IntelliJ (and it works just fine). If we remove the pom.xml,
>> >> we have to support Eclipse and IntelliJ "smoothly".
>> >>
>> >> Let me try in IntelliJ.
>> >>
>> >> Regards
>> >> JB
>> >>
>> >> On 10/04/2018 15:21, Romain Manni-Bucau wrote:
>> >> > You dont have issue due to the build setup with that option. I get:
>> >> >
>> >> > avr. 10, 2018 3:20:10 PM
>> >> > org.apache.beam.runners.direct.DirectTransformExecutor run
>> >> > GRAVE: Error occurred within
>> >> > org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a
>> >> > com.google.common.util.concurrent.ExecutionError:
>> >> > java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy
>> >> >
>> >> > ?
>> >> >
>> >> > Romain Manni-Bucau
>> >> > @rmannibucau |  Blog | Old Blog | Github 

Re: Gradle Status [April 6]

[Lukasz Cwik]

I use 2017.3 and it has been reliable for me. I haven't tried 2018 yet.

On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau 
wrote:

> Any of you using the idea 2018? the import works for me but then it is
> not as smooth as it seems for you. I'm just trying to see if it is a
> procedure thing or a version issue.
>
> Romain Manni-Bucau
> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>
>
> 2018-04-11 17:28 GMT+02:00 Kenneth Knowles :
> > The only reason I did "empty project then add a module" procedure was to
> get
> > all the IntelliJ files outside the source tree. IIRC directly creating
> from
> > existing sources didn't give the necessary configuration options. If you
> > don't care about being able to `git clean` then you can do the shorter
> > version. Also the particular UI for creation might have improved since I
> > tried it. I'll do it again.
> >
> > On the pom, since it is only generated for machine reading, why care if
> the
> > parent is inlined? I actually prefer to avoid coupling with things that
> you
> > have to go and look up. Using inheritance is OK for human edited poms
> > (actually IMO it is still a mistake) but it doesn't change the semantics
> of
> > a shipped pom if they are all immutable, which they should be. It is
> better
> > to have all the info right there. Is there an actually effective
> difference?
> >
> > Kenn
> >
> > On Wed, Apr 11, 2018 at 6:53 AM Etienne Chauchot 
> > wrote:
> >>
> >> Hi all,
> >> I just tested gradle environment from a fresh source clone with this
> >> procedure with just a tiny change: I used "new project from existing
> >> sources" rather than create empty project and then add module.
> >>
> >> It works fine and junit runs from intellij also work.  with gradle we
> pay
> >> a 2s delay (on my machine) before running the actual test for each run.
> This
> >> delay seems quite constant no matter the module: I have run all the
> tests at
> >> once in  runner-core and a single test in another module with a similar
> >> delay.
> >>
> >> I also tested a debug session from intellij and it runs fine also.
> >>
> >> I'll do some more testing and keep you posted.
> >>
> >> I still have some concerns about the potential difficulty for new
> >> contributors to have to learn gradle in addition to Beam itself
> comparing to
> >> maven which is more mainstream for java developers. That could be
> >> discouraging for ex for part-time contributors
> >>
> >> Etienne
> >>
> >> Le mardi 10 avril 2018 à 14:38 +, Lukasz Cwik a écrit :
> >>
> >> beam-site PR/414 updates the instructions for using Intellij and how to
> >> import a module:
> >>
> >> 1. Create an empty IntelliJ project outside of the Beam source tree.
> >> 2. Under Project Structure > Project, select a Project SDK.
> >> 3. Under Project Structure > Modules, click the + sign to add a module
> and
> >>select "Import Module".
> >> 1. Select the directory containing the Beam source tree.
> >> 2. Tick the "Import module from external model" button and select
> >> Gradle
> >>from the list.
> >> 3. Tick the following boxes.
> >>* Use auto-import
> >>* Create separate module per source set
> >>* Store generated project files externally
> >>* Use default gradle wrapper
> >> 4. Delegate build actions to Gradle by going to Settings > Build,
> >> Execution,
> >>Deployment > Build Tools > Gradle and checking "Delegate IDE
> build/run
> >>actions to gradle".
> >>
> >> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré 
> >> wrote:
> >>
> >> That's a very important issue for contribution. Up to now, I used Maven
> >> for setup IntelliJ (and it works just fine). If we remove the pom.xml,
> >> we have to support Eclipse and IntelliJ "smoothly".
> >>
> >> Let me try in IntelliJ.
> >>
> >> Regards
> >> JB
> >>
> >> On 10/04/2018 15:21, Romain Manni-Bucau wrote:
> >> > You dont have issue due to the build setup with that option. I get:
> >> >
> >> > avr. 10, 2018 3:20:10 PM
> >> > org.apache.beam.runners.direct.DirectTransformExecutor run
> >> > GRAVE: Error occurred within
> >> > org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a
> >> > com.google.common.util.concurrent.ExecutionError:
> >> > java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy
> >> >
> >> > ?
> >> >
> >> > Romain Manni-Bucau
> >> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >> >
> >> >
> >> > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik :
> >> >> I have found that the simplest setup is to delegate the build/test
> >> >> actions
> >> >> to Gradle. This allows you to run unit tests very easily and since
> its
> >> >> in
> >> >> the same manner that Gradle would have, you know that if its passing
> it
> >> >> will
> >> >> pass on the command line and on Jenkins. Here is one site that
> >> >> discusses how
> >> >> to set this up:
> >> >>
> >> >>
> 

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

Any of you using the idea 2018? the import works for me but then it is
not as smooth as it seems for you. I'm just trying to see if it is a
procedure thing or a version issue.

Romain Manni-Bucau
@rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book


2018-04-11 17:28 GMT+02:00 Kenneth Knowles :
> The only reason I did "empty project then add a module" procedure was to get
> all the IntelliJ files outside the source tree. IIRC directly creating from
> existing sources didn't give the necessary configuration options. If you
> don't care about being able to `git clean` then you can do the shorter
> version. Also the particular UI for creation might have improved since I
> tried it. I'll do it again.
>
> On the pom, since it is only generated for machine reading, why care if the
> parent is inlined? I actually prefer to avoid coupling with things that you
> have to go and look up. Using inheritance is OK for human edited poms
> (actually IMO it is still a mistake) but it doesn't change the semantics of
> a shipped pom if they are all immutable, which they should be. It is better
> to have all the info right there. Is there an actually effective difference?
>
> Kenn
>
> On Wed, Apr 11, 2018 at 6:53 AM Etienne Chauchot 
> wrote:
>>
>> Hi all,
>> I just tested gradle environment from a fresh source clone with this
>> procedure with just a tiny change: I used "new project from existing
>> sources" rather than create empty project and then add module.
>>
>> It works fine and junit runs from intellij also work.  with gradle we pay
>> a 2s delay (on my machine) before running the actual test for each run. This
>> delay seems quite constant no matter the module: I have run all the tests at
>> once in  runner-core and a single test in another module with a similar
>> delay.
>>
>> I also tested a debug session from intellij and it runs fine also.
>>
>> I'll do some more testing and keep you posted.
>>
>> I still have some concerns about the potential difficulty for new
>> contributors to have to learn gradle in addition to Beam itself comparing to
>> maven which is more mainstream for java developers. That could be
>> discouraging for ex for part-time contributors
>>
>> Etienne
>>
>> Le mardi 10 avril 2018 à 14:38 +, Lukasz Cwik a écrit :
>>
>> beam-site PR/414 updates the instructions for using Intellij and how to
>> import a module:
>>
>> 1. Create an empty IntelliJ project outside of the Beam source tree.
>> 2. Under Project Structure > Project, select a Project SDK.
>> 3. Under Project Structure > Modules, click the + sign to add a module and
>>select "Import Module".
>> 1. Select the directory containing the Beam source tree.
>> 2. Tick the "Import module from external model" button and select
>> Gradle
>>from the list.
>> 3. Tick the following boxes.
>>* Use auto-import
>>* Create separate module per source set
>>* Store generated project files externally
>>* Use default gradle wrapper
>> 4. Delegate build actions to Gradle by going to Settings > Build,
>> Execution,
>>Deployment > Build Tools > Gradle and checking "Delegate IDE build/run
>>actions to gradle".
>>
>> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré 
>> wrote:
>>
>> That's a very important issue for contribution. Up to now, I used Maven
>> for setup IntelliJ (and it works just fine). If we remove the pom.xml,
>> we have to support Eclipse and IntelliJ "smoothly".
>>
>> Let me try in IntelliJ.
>>
>> Regards
>> JB
>>
>> On 10/04/2018 15:21, Romain Manni-Bucau wrote:
>> > You dont have issue due to the build setup with that option. I get:
>> >
>> > avr. 10, 2018 3:20:10 PM
>> > org.apache.beam.runners.direct.DirectTransformExecutor run
>> > GRAVE: Error occurred within
>> > org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a
>> > com.google.common.util.concurrent.ExecutionError:
>> > java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy
>> >
>> > ?
>> >
>> > Romain Manni-Bucau
>> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>> >
>> >
>> > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik :
>> >> I have found that the simplest setup is to delegate the build/test
>> >> actions
>> >> to Gradle. This allows you to run unit tests very easily and since its
>> >> in
>> >> the same manner that Gradle would have, you know that if its passing it
>> >> will
>> >> pass on the command line and on Jenkins. Here is one site that
>> >> discusses how
>> >> to set this up:
>> >>
>> >> http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html
>> >>
>> >>
>> >> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau
>> >> 
>> >> wrote:
>> >>>
>> >>> What's the plan to make idea supporting gradle on beam project? Do we
>> >>> import the workaround mentionned in
>> >>> https://youtrack.jetbrains.com/issue/IDEA-175172?
>> >>> For the ones who didn't see this 

Re: Gradle Status [April 6]

[Kenneth Knowles]

The only reason I did "empty project then add a module" procedure was to
get all the IntelliJ files outside the source tree. IIRC directly creating
from existing sources didn't give the necessary configuration options. If
you don't care about being able to `git clean` then you can do the shorter
version. Also the particular UI for creation might have improved since I
tried it. I'll do it again.

On the pom, since it is only generated for machine reading, why care if the
parent is inlined? I actually prefer to avoid coupling with things that you
have to go and look up. Using inheritance is OK for human edited poms
(actually IMO it is still a mistake) but it doesn't change the semantics of
a shipped pom if they are all immutable, which they should be. It is better
to have all the info right there. Is there an actually effective difference?

Kenn

On Wed, Apr 11, 2018 at 6:53 AM Etienne Chauchot 
wrote:

> Hi all,
> I just tested gradle environment from a fresh source clone with this
> procedure with just a tiny change: I used "new project from existing
> sources" rather than create empty project and then add module.
>
> It works fine and junit runs from intellij also work.  with gradle we pay
> a 2s delay (on my machine) before running the actual test for each run.
> This delay seems quite constant no matter the module: I have run all the
> tests at once in  runner-core and a single test in another module with a
> similar delay.
>
> I also tested a debug session from intellij and it runs fine also.
>
> I'll do some more testing and keep you posted.
>
> I still have some concerns about the potential difficulty for new
> contributors to have to learn gradle in addition to Beam itself comparing
> to maven which is more mainstream for java developers. That could be
> discouraging for ex for part-time contributors
>
> Etienne
>
> Le mardi 10 avril 2018 à 14:38 +, Lukasz Cwik a écrit :
>
> beam-site PR/414 updates the instructions for using Intellij and how to
> import a module:
>
> 1. Create an empty IntelliJ project outside of the Beam source tree.
> 2. Under Project Structure > Project, select a Project SDK.
> 3. Under Project Structure > Modules, click the + sign to add a module and
>select "Import Module".
> 1. Select the directory containing the Beam source tree.
> 2. Tick the "Import module from external model" button and select
> Gradle
>from the list.
> 3. Tick the following boxes.
>* Use auto-import
>* Create separate module per source set
>* Store generated project files externally
>* Use default gradle wrapper
> 4. Delegate build actions to Gradle by going to Settings > Build,
> Execution,
>Deployment > Build Tools > Gradle and checking "Delegate IDE build/run
>actions to gradle".
>
> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré 
> wrote:
>
> That's a very important issue for contribution. Up to now, I used Maven
> for setup IntelliJ (and it works just fine). If we remove the pom.xml,
> we have to support Eclipse and IntelliJ "smoothly".
>
> Let me try in IntelliJ.
>
> Regards
> JB
>
> On 10/04/2018 15:21, Romain Manni-Bucau wrote:
> > You dont have issue due to the build setup with that option. I get:
> >
> > avr. 10, 2018 3:20:10 PM
> > org.apache.beam.runners.direct.DirectTransformExecutor run
> > GRAVE: Error occurred within
> > org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a
> > com.google.common.util.concurrent.ExecutionError:
> > java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy
> >
> > ?
> >
> > Romain Manni-Bucau
> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >
> >
> > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik :
> >> I have found that the simplest setup is to delegate the build/test
> actions
> >> to Gradle. This allows you to run unit tests very easily and since its
> in
> >> the same manner that Gradle would have, you know that if its passing it
> will
> >> pass on the command line and on Jenkins. Here is one site that
> discusses how
> >> to set this up:
> >>
> http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html
> >>
> >>
> >> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau <
> rmannibu...@gmail.com>
> >> wrote:
> >>>
> >>> What's the plan to make idea supporting gradle on beam project? Do we
> >>> import the workaround mentionned in
> >>> https://youtrack.jetbrains.com/issue/IDEA-175172?
> >>> For the ones who didn't see this issue in action: idea will compile in
> >>> out/ instead of build/ and you will just miss all the resources you
> >>> need like some SPI registration which are used by all our registrar =>
> >>> no way to run tests in idea without hacking the configuration quite
> >>> deeply :(
> >>>
> >>> Romain Manni-Bucau
> >>> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >>>
> >>>
> >>> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot :
>  

Re: Gradle Status [April 6]

[Etienne Chauchot]

Hi all,
I just tested gradle environment from a fresh source clone with this procedure 
with just a tiny change: I used "new
project from existing sources" rather than create empty project and then add 
module. 
It works fine and junit runs from intellij also work.  with gradle we pay a 2s 
delay (on my machine) before running the
actual test for each run. This delay seems quite constant no matter the module: 
I have run all the tests at once in
 runner-core and a single test in another module with a similar delay. 
I also tested a debug session from intellij and it runs fine also.
I'll do some more testing and keep you posted.
I still have some concerns about the potential difficulty for new contributors 
to have to learn gradle in addition to
Beam itself comparing to maven which is more mainstream for java developers. 
That could be discouraging for ex for part-
time contributors
Etienne
Le mardi 10 avril 2018 à 14:38 +, Lukasz Cwik a écrit :
> beam-site PR/414 updates the instructions for using Intellij and how to 
> import a module:
> 
> 1. Create an empty IntelliJ project outside of the Beam source tree.
> 2. Under Project Structure > Project, select a Project SDK.
> 3. Under Project Structure > Modules, click the + sign to add a module and
>    select "Import Module".
>     1. Select the directory containing the Beam source tree.
>     2. Tick the "Import module from external model" button and select Gradle
>        from the list.
>     3. Tick the following boxes.
>        * Use auto-import
>        * Create separate module per source set
>        * Store generated project files externally
>        * Use default gradle wrapper
> 4. Delegate build actions to Gradle by going to Settings > Build, Execution,
>    Deployment > Build Tools > Gradle and checking "Delegate IDE build/run
>    actions to gradle".
> 
> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré  
> wrote:
> > That's a very important issue for contribution. Up to now, I used Maven
> > for setup IntelliJ (and it works just fine). If we remove the pom.xml,
> > we have to support Eclipse and IntelliJ "smoothly".
> > 
> > Let me try in IntelliJ.
> > 
> > Regards
> > JB
> > 
> > On 10/04/2018 15:21, Romain Manni-Bucau wrote:
> > > You dont have issue due to the build setup with that option. I get:
> > >
> > > avr. 10, 2018 3:20:10 PM
> > > org.apache.beam.runners.direct.DirectTransformExecutor run
> > > GRAVE: Error occurred within
> > > org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a
> > > com.google.common.util.concurrent.ExecutionError:
> > > java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy
> > >
> > > ?
> > >
> > > Romain Manni-Bucau
> > > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> > >
> > >
> > > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik :
> > >> I have found that the simplest setup is to delegate the build/test 
> > >> actions
> > >> to Gradle. This allows you to run unit tests very easily and since its in
> > >> the same manner that Gradle would have, you know that if its passing it 
> > >> will
> > >> pass on the command line and on Jenkins. Here is one site that discusses 
> > >> how
> > >> to set this up:
> > >> http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html
> > >>
> > >>
> > >> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau 
> > >> 
> > >> wrote:
> > >>>
> > >>> What's the plan to make idea supporting gradle on beam project? Do we
> > >>> import the workaround mentionned in
> > >>> https://youtrack.jetbrains.com/issue/IDEA-175172?
> > >>> For the ones who didn't see this issue in action: idea will compile in
> > >>> out/ instead of build/ and you will just miss all the resources you
> > >>> need like some SPI registration which are used by all our registrar =>
> > >>> no way to run tests in idea without hacking the configuration quite
> > >>> deeply :(
> > >>>
> > >>> Romain Manni-Bucau
> > >>> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> > >>>
> > >>>
> > >>> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot :
> >  As a gradle beginner, I could not agree more !
> >  +1
> >  Etienne
> >  Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste Onofré a écrit :
> > 
> >  Hi all,
> > 
> >  I did multiple gradle build since last week and I would like to share
> >  one of my concern: it's about the communities.
> > 
> >  If I think our users won't see any change for them due to Gradle build
> >  (I think that most of our users will still use Maven with artifacts
> >  provided by Gradle), I'm more concerned by the dev community and the
> >  contribution.
> > 
> >  Maven is well known and straight forward for a large part of potential
> >  contributors. I think we have to keep in mind that we still have to 
> >  grow
> >  up our contributors community.
> > 
> >  Today, maybe I'm 

Re: Gradle Status [April 6]

[Lukasz Cwik]

Romain, it seems like you want us to generate an exact copy of the existing
poms that we have. There is a lot of stuff in there related to
building/testing the Apache Beam project which our users don't need
(integration/test profiles, test scope dependencies, parents, ...). Users
who want to build the project will need to use the same tools as the
community and build from the source tarball.

On Wed, Apr 11, 2018 at 8:48 AM Romain Manni-Bucau 
wrote:

> Hi guys,
>
> checked the snapshot pom today and there are issues:
>
> 1. no parent (so 2 is hard to handle, no properties etc)
> 2. too much noise informations (all the parent data should be in the
> parent)

3. wrong scopes (all is compile) - which is likely a leak of gradle scripts
>
4. missing important configuration (properties and or plugin) like the
> ones defining the java versions




> I didn't check the profiles but it makes already some work ;)
>
>
> Romain Manni-Bucau
> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>
>
> 2018-04-11 6:50 GMT+02:00 Romain Manni-Bucau :
> >
> >
> > Le 11 avr. 2018 02:30, "Reuven Lax"  a écrit :
> >
> > Actually I always found the right-click to run tests to only sometimes
> work
> > in Maven, especially if there were changes to dependent AutoValue classes
> > where code had to be generated. Too often it would fail, and I would then
> > need to use Maven to rebuild the whole project. It would be cool if
> Gradle
> > could do this more reliably than Maven did.
> >
> >
> > Hmm, i exactly experiment the opposite. Testes with idea 2017 and 2018.
> >
> >
> > Reuven
> >
> > On Tue, Apr 10, 2018 at 8:46 PM Romain Manni-Bucau <
> rmannibu...@gmail.com>
> > wrote:
> >>
> >> @jb: what did you change? I re-imported the project like 3 times earlier
> >> today and never got it working acceptably :(
> >>
> >> Personally if importing the project and right click on a test+debug
> works
> >> as good as maven in idea id be happy. I can manage other stuff in a
> console
> >> even if gradle reporting is not that efficient for me for now.
> >>
> >> Le 10 avr. 2018 21:37, "Reuven Lax"  a écrit :
> >>>
> >>> There are a lot of ideas on how to increase usability, but I think
> >>> they'll get lost in the thread. I suggest we try to capture them in
> Jiras.
> >>>
> >>> I suggest we also find out what common use patterns are (people on this
> >>> thread are probably sufficient), as different people will have
> different
> >>> workflows. We can then make sure that all common workflows are
> documented.
> >>> As an example, one task I often do is to run just checkstyle over a
> module
> >>> or the entire project.
> >>>
> >>> Reuven
> >>>
> >>> On Tue, Apr 10, 2018 at 7:18 PM Jean-Baptiste Onofré 
> >>> wrote:
> 
>  FYI, I did a new attempt and it works fine (pretty long). Previous try
>  failed.
> 
>  Regards
>  JB
> 
>  On 10/04/2018 19:52, Kenneth Knowles wrote:
>  > I've been on Idea+Gradle for ~two months, around the time I added
>  > https://github.com/apache/beam/pull/4583 and
>  > https://github.com/apache/beam/pull/4626 to make the import require
>  > zero
>  > user work. I have no fear of deleting my project any time and
>  > re-importing.
>  >
>  > I agree with not having auto-import on. It is just too slow. I can't
>  > remember if it was importing too often due to build outputs or if it
>  > was
>  > just that I was messing with the build.gradle files. Anyhow it
> doesn't
>  > really add much value.
>  >
>  > The gradle runner _is_ able to use submodules and run individual
> tests
>  > methods, and all that.
>  >
>  > Kenn
>  >
>  >
>  > On Tue, Apr 10, 2018 at 9:31 AM Romain Manni-Bucau
>  > > wrote:
>  >
>  > Runner a test doesnt have the right classpath (idea uses out/
>  > instead
>  > of build/) then when you switch on gradle runner the launching
>  > uses
>  > gradle which is not able to use submodules directly but
> reconsider
>  > the
>  > whole project which is quite slow for normal dev iterations
>  > compare to just run the test with the right classpath and a fast
>  > compile step if needed. I lost literally 1h for something simple
>  > with
>  > that tooling, this is way too much to be acceptable on my side
>  > since
>  > I'm sadly not paid to work on beam (one day maybe ;)).
>  >
>  > Romain Manni-Bucau
>  > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>  >
>  >
>  > 2018-04-10 18:27 GMT+02:00 Reuven Lax   > >:
>  >  > Romain,
>  >  >
>  >  > Can you detail what's not working. I switched my IntelliJ
> over
>  > to
>  >   

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

Hi guys,

checked the snapshot pom today and there are issues:

1. no parent (so 2 is hard to handle, no properties etc)
2. too much noise informations (all the parent data should be in the parent)
3. wrong scopes (all is compile) - which is likely a leak of gradle scripts
4. missing important configuration (properties and or plugin) like the
ones defining the java versions

I didn't check the profiles but it makes already some work ;)


Romain Manni-Bucau
@rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book


2018-04-11 6:50 GMT+02:00 Romain Manni-Bucau :
>
>
> Le 11 avr. 2018 02:30, "Reuven Lax"  a écrit :
>
> Actually I always found the right-click to run tests to only sometimes work
> in Maven, especially if there were changes to dependent AutoValue classes
> where code had to be generated. Too often it would fail, and I would then
> need to use Maven to rebuild the whole project. It would be cool if Gradle
> could do this more reliably than Maven did.
>
>
> Hmm, i exactly experiment the opposite. Testes with idea 2017 and 2018.
>
>
> Reuven
>
> On Tue, Apr 10, 2018 at 8:46 PM Romain Manni-Bucau 
> wrote:
>>
>> @jb: what did you change? I re-imported the project like 3 times earlier
>> today and never got it working acceptably :(
>>
>> Personally if importing the project and right click on a test+debug works
>> as good as maven in idea id be happy. I can manage other stuff in a console
>> even if gradle reporting is not that efficient for me for now.
>>
>> Le 10 avr. 2018 21:37, "Reuven Lax"  a écrit :
>>>
>>> There are a lot of ideas on how to increase usability, but I think
>>> they'll get lost in the thread. I suggest we try to capture them in Jiras.
>>>
>>> I suggest we also find out what common use patterns are (people on this
>>> thread are probably sufficient), as different people will have different
>>> workflows. We can then make sure that all common workflows are documented.
>>> As an example, one task I often do is to run just checkstyle over a module
>>> or the entire project.
>>>
>>> Reuven
>>>
>>> On Tue, Apr 10, 2018 at 7:18 PM Jean-Baptiste Onofré 
>>> wrote:

 FYI, I did a new attempt and it works fine (pretty long). Previous try
 failed.

 Regards
 JB

 On 10/04/2018 19:52, Kenneth Knowles wrote:
 > I've been on Idea+Gradle for ~two months, around the time I added
 > https://github.com/apache/beam/pull/4583 and
 > https://github.com/apache/beam/pull/4626 to make the import require
 > zero
 > user work. I have no fear of deleting my project any time and
 > re-importing.
 >
 > I agree with not having auto-import on. It is just too slow. I can't
 > remember if it was importing too often due to build outputs or if it
 > was
 > just that I was messing with the build.gradle files. Anyhow it doesn't
 > really add much value.
 >
 > The gradle runner _is_ able to use submodules and run individual tests
 > methods, and all that.
 >
 > Kenn
 >
 >
 > On Tue, Apr 10, 2018 at 9:31 AM Romain Manni-Bucau
 > > wrote:
 >
 > Runner a test doesnt have the right classpath (idea uses out/
 > instead
 > of build/) then when you switch on gradle runner the launching
 > uses
 > gradle which is not able to use submodules directly but reconsider
 > the
 > whole project which is quite slow for normal dev iterations
 > compare to just run the test with the right classpath and a fast
 > compile step if needed. I lost literally 1h for something simple
 > with
 > that tooling, this is way too much to be acceptable on my side
 > since
 > I'm sadly not paid to work on beam (one day maybe ;)).
 >
 > Romain Manni-Bucau
 > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
 >
 >
 > 2018-04-10 18:27 GMT+02:00 Reuven Lax  >:
 >  > Romain,
 >  >
 >  > Can you detail what's not working. I switched my IntelliJ over
 > to
 > Gradle
 >  > about two weeks ago, and haven't had any trouble.
 >  >
 >  > Reuven
 >  >
 >  > On Tue, Apr 10, 2018 at 4:20 PM Romain Manni-Bucau
 > >
 >  > wrote:
 >  >>
 >  >> Ok, didn't find a way to make it working properly (only
 > workaround
 >  >> with direct commands and no good idea integration for
 > debugging). I'm
 >  >> back with maven, if anyone knows how to properly solve it
 > let's
 > do it.
 >  >> If not I think JB point is to consider more than any other
 > criteria.
 >  >>
 >  >> Romain Manni-Bucau
 >  

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

Le 11 avr. 2018 02:30, "Reuven Lax"  a écrit :

Actually I always found the right-click to run tests to only sometimes work
in Maven, especially if there were changes to dependent AutoValue classes
where code had to be generated. Too often it would fail, and I would then
need to use Maven to rebuild the whole project. It would be cool if Gradle
could do this more reliably than Maven did.


Hmm, i exactly experiment the opposite. Testes with idea 2017 and 2018.


Reuven

On Tue, Apr 10, 2018 at 8:46 PM Romain Manni-Bucau 
wrote:

> @jb: what did you change? I re-imported the project like 3 times earlier
> today and never got it working acceptably :(
>
> Personally if importing the project and right click on a test+debug works
> as good as maven in idea id be happy. I can manage other stuff in a console
> even if gradle reporting is not that efficient for me for now.
>
> Le 10 avr. 2018 21:37, "Reuven Lax"  a écrit :
>
>> There are a lot of ideas on how to increase usability, but I think
>> they'll get lost in the thread. I suggest we try to capture them in Jiras.
>>
>> I suggest we also find out what common use patterns are (people on this
>> thread are probably sufficient), as different people will have different
>> workflows. We can then make sure that all common workflows are documented.
>> As an example, one task I often do is to run just checkstyle over a module
>> or the entire project.
>>
>> Reuven
>>
>> On Tue, Apr 10, 2018 at 7:18 PM Jean-Baptiste Onofré 
>> wrote:
>>
>>> FYI, I did a new attempt and it works fine (pretty long). Previous try
>>> failed.
>>>
>>> Regards
>>> JB
>>>
>>> On 10/04/2018 19:52, Kenneth Knowles wrote:
>>> > I've been on Idea+Gradle for ~two months, around the time I added
>>> > https://github.com/apache/beam/pull/4583 and
>>> > https://github.com/apache/beam/pull/4626 to make the import require
>>> zero
>>> > user work. I have no fear of deleting my project any time and
>>> re-importing.
>>> >
>>> > I agree with not having auto-import on. It is just too slow. I can't
>>> > remember if it was importing too often due to build outputs or if it
>>> was
>>> > just that I was messing with the build.gradle files. Anyhow it doesn't
>>> > really add much value.
>>> >
>>> > The gradle runner _is_ able to use submodules and run individual tests
>>> > methods, and all that.
>>> >
>>> > Kenn
>>> >
>>> >
>>> > On Tue, Apr 10, 2018 at 9:31 AM Romain Manni-Bucau
>>> > > wrote:
>>> >
>>> > Runner a test doesnt have the right classpath (idea uses out/
>>> instead
>>> > of build/) then when you switch on gradle runner the launching uses
>>> > gradle which is not able to use submodules directly but reconsider
>>> the
>>> > whole project which is quite slow for normal dev iterations
>>> > compare to just run the test with the right classpath and a fast
>>> > compile step if needed. I lost literally 1h for something simple
>>> with
>>> > that tooling, this is way too much to be acceptable on my side
>>> since
>>> > I'm sadly not paid to work on beam (one day maybe ;)).
>>> >
>>> > Romain Manni-Bucau
>>> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>> >
>>> >
>>> > 2018-04-10 18:27 GMT+02:00 Reuven Lax >> > >:
>>> >  > Romain,
>>> >  >
>>> >  > Can you detail what's not working. I switched my IntelliJ over
>>> to
>>> > Gradle
>>> >  > about two weeks ago, and haven't had any trouble.
>>> >  >
>>> >  > Reuven
>>> >  >
>>> >  > On Tue, Apr 10, 2018 at 4:20 PM Romain Manni-Bucau
>>> > >
>>> >  > wrote:
>>> >  >>
>>> >  >> Ok, didn't find a way to make it working properly (only
>>> workaround
>>> >  >> with direct commands and no good idea integration for
>>> > debugging). I'm
>>> >  >> back with maven, if anyone knows how to properly solve it let's
>>> > do it.
>>> >  >> If not I think JB point is to consider more than any other
>>> criteria.
>>> >  >>
>>> >  >> Romain Manni-Bucau
>>> >  >> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>> >  >>
>>> >  >>
>>> >  >> 2018-04-10 16:41 GMT+02:00 Romain Manni-Bucau
>>> > >:
>>> >  >> > side note: do NOT use auto-import until you are sure you can,
>>> > it locks
>>> >  >> > regularly on beam (pby too big for idea?) and makes idea
>>> ready
>>> > to be
>>> >  >> > killed :(
>>> >  >> >
>>> >  >> > Romain Manni-Bucau
>>> >  >> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>> >  >> >
>>> >  >> >
>>> >  >> > 2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré
>>> > >:
>>> >  >> >> It's what I did, 

Re: Gradle Status [April 6]

[Reuven Lax]

Actually I always found the right-click to run tests to only sometimes work
in Maven, especially if there were changes to dependent AutoValue classes
where code had to be generated. Too often it would fail, and I would then
need to use Maven to rebuild the whole project. It would be cool if Gradle
could do this more reliably than Maven did.

Reuven

On Tue, Apr 10, 2018 at 8:46 PM Romain Manni-Bucau 
wrote:

> @jb: what did you change? I re-imported the project like 3 times earlier
> today and never got it working acceptably :(
>
> Personally if importing the project and right click on a test+debug works
> as good as maven in idea id be happy. I can manage other stuff in a console
> even if gradle reporting is not that efficient for me for now.
>
> Le 10 avr. 2018 21:37, "Reuven Lax"  a écrit :
>
>> There are a lot of ideas on how to increase usability, but I think
>> they'll get lost in the thread. I suggest we try to capture them in Jiras.
>>
>> I suggest we also find out what common use patterns are (people on this
>> thread are probably sufficient), as different people will have different
>> workflows. We can then make sure that all common workflows are documented.
>> As an example, one task I often do is to run just checkstyle over a module
>> or the entire project.
>>
>> Reuven
>>
>> On Tue, Apr 10, 2018 at 7:18 PM Jean-Baptiste Onofré 
>> wrote:
>>
>>> FYI, I did a new attempt and it works fine (pretty long). Previous try
>>> failed.
>>>
>>> Regards
>>> JB
>>>
>>> On 10/04/2018 19:52, Kenneth Knowles wrote:
>>> > I've been on Idea+Gradle for ~two months, around the time I added
>>> > https://github.com/apache/beam/pull/4583 and
>>> > https://github.com/apache/beam/pull/4626 to make the import require
>>> zero
>>> > user work. I have no fear of deleting my project any time and
>>> re-importing.
>>> >
>>> > I agree with not having auto-import on. It is just too slow. I can't
>>> > remember if it was importing too often due to build outputs or if it
>>> was
>>> > just that I was messing with the build.gradle files. Anyhow it doesn't
>>> > really add much value.
>>> >
>>> > The gradle runner _is_ able to use submodules and run individual tests
>>> > methods, and all that.
>>> >
>>> > Kenn
>>> >
>>> >
>>> > On Tue, Apr 10, 2018 at 9:31 AM Romain Manni-Bucau
>>> > > wrote:
>>> >
>>> > Runner a test doesnt have the right classpath (idea uses out/
>>> instead
>>> > of build/) then when you switch on gradle runner the launching uses
>>> > gradle which is not able to use submodules directly but reconsider
>>> the
>>> > whole project which is quite slow for normal dev iterations
>>> > compare to just run the test with the right classpath and a fast
>>> > compile step if needed. I lost literally 1h for something simple
>>> with
>>> > that tooling, this is way too much to be acceptable on my side
>>> since
>>> > I'm sadly not paid to work on beam (one day maybe ;)).
>>> >
>>> > Romain Manni-Bucau
>>> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>> >
>>> >
>>> > 2018-04-10 18:27 GMT+02:00 Reuven Lax >> > >:
>>> >  > Romain,
>>> >  >
>>> >  > Can you detail what's not working. I switched my IntelliJ over
>>> to
>>> > Gradle
>>> >  > about two weeks ago, and haven't had any trouble.
>>> >  >
>>> >  > Reuven
>>> >  >
>>> >  > On Tue, Apr 10, 2018 at 4:20 PM Romain Manni-Bucau
>>> > >
>>> >  > wrote:
>>> >  >>
>>> >  >> Ok, didn't find a way to make it working properly (only
>>> workaround
>>> >  >> with direct commands and no good idea integration for
>>> > debugging). I'm
>>> >  >> back with maven, if anyone knows how to properly solve it let's
>>> > do it.
>>> >  >> If not I think JB point is to consider more than any other
>>> criteria.
>>> >  >>
>>> >  >> Romain Manni-Bucau
>>> >  >> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>> >  >>
>>> >  >>
>>> >  >> 2018-04-10 16:41 GMT+02:00 Romain Manni-Bucau
>>> > >:
>>> >  >> > side note: do NOT use auto-import until you are sure you can,
>>> > it locks
>>> >  >> > regularly on beam (pby too big for idea?) and makes idea
>>> ready
>>> > to be
>>> >  >> > killed :(
>>> >  >> >
>>> >  >> > Romain Manni-Bucau
>>> >  >> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>> >  >> >
>>> >  >> >
>>> >  >> > 2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré
>>> > >:
>>> >  >> >> It's what I did, I'm trying a complete reload now (maybe
>>> this
>>> > step
>>> >  >> >> failed).
>>> >  >> >>
>>> >  >> >> On 10/04/2018 

Re: Gradle Status [April 6]

[Kenneth Knowles]

Reuven's point is good.

Once we hit the bare minimum of having things working, let's collect
usability improvements and engineering improvements on a separate JIRA from
the main migration.

I filed https://issues.apache.org/jira/browse/BEAM-4045 for these less
critical issues to separate them from blockers on
https://issues.apache.org/jira/browse/BEAM-3249.

Once we are through the migration, I expect the subtasks might just be
flattened to top-level tasks, but this is a nice view of them.

Kenn

On Tue, Apr 10, 2018 at 1:46 PM Romain Manni-Bucau 
wrote:

> @jb: what did you change? I re-imported the project like 3 times earlier
> today and never got it working acceptably :(
>
> Personally if importing the project and right click on a test+debug works
> as good as maven in idea id be happy. I can manage other stuff in a console
> even if gradle reporting is not that efficient for me for now.
>
> Le 10 avr. 2018 21:37, "Reuven Lax"  a écrit :
>
>> There are a lot of ideas on how to increase usability, but I think
>> they'll get lost in the thread. I suggest we try to capture them in Jiras.
>>
>> I suggest we also find out what common use patterns are (people on this
>> thread are probably sufficient), as different people will have different
>> workflows. We can then make sure that all common workflows are documented.
>> As an example, one task I often do is to run just checkstyle over a module
>> or the entire project.
>>
>> Reuven
>>
>> On Tue, Apr 10, 2018 at 7:18 PM Jean-Baptiste Onofré 
>> wrote:
>>
>>> FYI, I did a new attempt and it works fine (pretty long). Previous try
>>> failed.
>>>
>>> Regards
>>> JB
>>>
>>> On 10/04/2018 19:52, Kenneth Knowles wrote:
>>> > I've been on Idea+Gradle for ~two months, around the time I added
>>> > https://github.com/apache/beam/pull/4583 and
>>> > https://github.com/apache/beam/pull/4626 to make the import require
>>> zero
>>> > user work. I have no fear of deleting my project any time and
>>> re-importing.
>>> >
>>> > I agree with not having auto-import on. It is just too slow. I can't
>>> > remember if it was importing too often due to build outputs or if it
>>> was
>>> > just that I was messing with the build.gradle files. Anyhow it doesn't
>>> > really add much value.
>>> >
>>> > The gradle runner _is_ able to use submodules and run individual tests
>>> > methods, and all that.
>>> >
>>> > Kenn
>>> >
>>> >
>>> > On Tue, Apr 10, 2018 at 9:31 AM Romain Manni-Bucau
>>> > > wrote:
>>> >
>>> > Runner a test doesnt have the right classpath (idea uses out/
>>> instead
>>> > of build/) then when you switch on gradle runner the launching uses
>>> > gradle which is not able to use submodules directly but reconsider
>>> the
>>> > whole project which is quite slow for normal dev iterations
>>> > compare to just run the test with the right classpath and a fast
>>> > compile step if needed. I lost literally 1h for something simple
>>> with
>>> > that tooling, this is way too much to be acceptable on my side
>>> since
>>> > I'm sadly not paid to work on beam (one day maybe ;)).
>>> >
>>> > Romain Manni-Bucau
>>> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>> >
>>> >
>>> > 2018-04-10 18:27 GMT+02:00 Reuven Lax >> > >:
>>> >  > Romain,
>>> >  >
>>> >  > Can you detail what's not working. I switched my IntelliJ over
>>> to
>>> > Gradle
>>> >  > about two weeks ago, and haven't had any trouble.
>>> >  >
>>> >  > Reuven
>>> >  >
>>> >  > On Tue, Apr 10, 2018 at 4:20 PM Romain Manni-Bucau
>>> > >
>>> >  > wrote:
>>> >  >>
>>> >  >> Ok, didn't find a way to make it working properly (only
>>> workaround
>>> >  >> with direct commands and no good idea integration for
>>> > debugging). I'm
>>> >  >> back with maven, if anyone knows how to properly solve it let's
>>> > do it.
>>> >  >> If not I think JB point is to consider more than any other
>>> criteria.
>>> >  >>
>>> >  >> Romain Manni-Bucau
>>> >  >> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>> >  >>
>>> >  >>
>>> >  >> 2018-04-10 16:41 GMT+02:00 Romain Manni-Bucau
>>> > >:
>>> >  >> > side note: do NOT use auto-import until you are sure you can,
>>> > it locks
>>> >  >> > regularly on beam (pby too big for idea?) and makes idea
>>> ready
>>> > to be
>>> >  >> > killed :(
>>> >  >> >
>>> >  >> > Romain Manni-Bucau
>>> >  >> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>> >  >> >
>>> >  >> >
>>> >  >> > 2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré
>>> > >:
>>> >  >> >> It's what 

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

@jb: what did you change? I re-imported the project like 3 times earlier
today and never got it working acceptably :(

Personally if importing the project and right click on a test+debug works
as good as maven in idea id be happy. I can manage other stuff in a console
even if gradle reporting is not that efficient for me for now.

Le 10 avr. 2018 21:37, "Reuven Lax"  a écrit :

> There are a lot of ideas on how to increase usability, but I think they'll
> get lost in the thread. I suggest we try to capture them in Jiras.
>
> I suggest we also find out what common use patterns are (people on this
> thread are probably sufficient), as different people will have different
> workflows. We can then make sure that all common workflows are documented.
> As an example, one task I often do is to run just checkstyle over a module
> or the entire project.
>
> Reuven
>
> On Tue, Apr 10, 2018 at 7:18 PM Jean-Baptiste Onofré 
> wrote:
>
>> FYI, I did a new attempt and it works fine (pretty long). Previous try
>> failed.
>>
>> Regards
>> JB
>>
>> On 10/04/2018 19:52, Kenneth Knowles wrote:
>> > I've been on Idea+Gradle for ~two months, around the time I added
>> > https://github.com/apache/beam/pull/4583 and
>> > https://github.com/apache/beam/pull/4626 to make the import require
>> zero
>> > user work. I have no fear of deleting my project any time and
>> re-importing.
>> >
>> > I agree with not having auto-import on. It is just too slow. I can't
>> > remember if it was importing too often due to build outputs or if it
>> was
>> > just that I was messing with the build.gradle files. Anyhow it doesn't
>> > really add much value.
>> >
>> > The gradle runner _is_ able to use submodules and run individual tests
>> > methods, and all that.
>> >
>> > Kenn
>> >
>> >
>> > On Tue, Apr 10, 2018 at 9:31 AM Romain Manni-Bucau
>> > > wrote:
>> >
>> > Runner a test doesnt have the right classpath (idea uses out/
>> instead
>> > of build/) then when you switch on gradle runner the launching uses
>> > gradle which is not able to use submodules directly but reconsider
>> the
>> > whole project which is quite slow for normal dev iterations
>> > compare to just run the test with the right classpath and a fast
>> > compile step if needed. I lost literally 1h for something simple
>> with
>> > that tooling, this is way too much to be acceptable on my side since
>> > I'm sadly not paid to work on beam (one day maybe ;)).
>> >
>> > Romain Manni-Bucau
>> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>> >
>> >
>> > 2018-04-10 18:27 GMT+02:00 Reuven Lax > > >:
>> >  > Romain,
>> >  >
>> >  > Can you detail what's not working. I switched my IntelliJ over to
>> > Gradle
>> >  > about two weeks ago, and haven't had any trouble.
>> >  >
>> >  > Reuven
>> >  >
>> >  > On Tue, Apr 10, 2018 at 4:20 PM Romain Manni-Bucau
>> > >
>> >  > wrote:
>> >  >>
>> >  >> Ok, didn't find a way to make it working properly (only
>> workaround
>> >  >> with direct commands and no good idea integration for
>> > debugging). I'm
>> >  >> back with maven, if anyone knows how to properly solve it let's
>> > do it.
>> >  >> If not I think JB point is to consider more than any other
>> criteria.
>> >  >>
>> >  >> Romain Manni-Bucau
>> >  >> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>> >  >>
>> >  >>
>> >  >> 2018-04-10 16:41 GMT+02:00 Romain Manni-Bucau
>> > >:
>> >  >> > side note: do NOT use auto-import until you are sure you can,
>> > it locks
>> >  >> > regularly on beam (pby too big for idea?) and makes idea ready
>> > to be
>> >  >> > killed :(
>> >  >> >
>> >  >> > Romain Manni-Bucau
>> >  >> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>> >  >> >
>> >  >> >
>> >  >> > 2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré
>> > >:
>> >  >> >> It's what I did, I'm trying a complete reload now (maybe this
>> > step
>> >  >> >> failed).
>> >  >> >>
>> >  >> >> On 10/04/2018 16:38, Lukasz Cwik wrote:
>> >  >> >>>
>> >  >> >>> beam-site PR/414 updates the instructions for using Intellij
>> > and how
>> >  >> >>> to
>> >  >> >>> import a module:
>> >  >> >>>
>> >  >> >>> 1. Create an empty IntelliJ project outside of the Beam
>> > source tree.
>> >  >> >>> 2. Under Project Structure > Project, select a Project SDK.
>> >  >> >>> 3. Under Project Structure > Modules, click the + sign to
>> > add a module
>> >  >> >>> and
>> >  >> >>> select "Import Module".
>> >  >> >>>  1. Select 

Re: Gradle Status [April 6]

[Reuven Lax]

There are a lot of ideas on how to increase usability, but I think they'll
get lost in the thread. I suggest we try to capture them in Jiras.

I suggest we also find out what common use patterns are (people on this
thread are probably sufficient), as different people will have different
workflows. We can then make sure that all common workflows are documented.
As an example, one task I often do is to run just checkstyle over a module
or the entire project.

Reuven

On Tue, Apr 10, 2018 at 7:18 PM Jean-Baptiste Onofré 
wrote:

> FYI, I did a new attempt and it works fine (pretty long). Previous try
> failed.
>
> Regards
> JB
>
> On 10/04/2018 19:52, Kenneth Knowles wrote:
> > I've been on Idea+Gradle for ~two months, around the time I added
> > https://github.com/apache/beam/pull/4583 and
> > https://github.com/apache/beam/pull/4626 to make the import require
> zero
> > user work. I have no fear of deleting my project any time and
> re-importing.
> >
> > I agree with not having auto-import on. It is just too slow. I can't
> > remember if it was importing too often due to build outputs or if it was
> > just that I was messing with the build.gradle files. Anyhow it doesn't
> > really add much value.
> >
> > The gradle runner _is_ able to use submodules and run individual tests
> > methods, and all that.
> >
> > Kenn
> >
> >
> > On Tue, Apr 10, 2018 at 9:31 AM Romain Manni-Bucau
> > > wrote:
> >
> > Runner a test doesnt have the right classpath (idea uses out/ instead
> > of build/) then when you switch on gradle runner the launching uses
> > gradle which is not able to use submodules directly but reconsider
> the
> > whole project which is quite slow for normal dev iterations
> > compare to just run the test with the right classpath and a fast
> > compile step if needed. I lost literally 1h for something simple with
> > that tooling, this is way too much to be acceptable on my side since
> > I'm sadly not paid to work on beam (one day maybe ;)).
> >
> > Romain Manni-Bucau
> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >
> >
> > 2018-04-10 18:27 GMT+02:00 Reuven Lax  > >:
> >  > Romain,
> >  >
> >  > Can you detail what's not working. I switched my IntelliJ over to
> > Gradle
> >  > about two weeks ago, and haven't had any trouble.
> >  >
> >  > Reuven
> >  >
> >  > On Tue, Apr 10, 2018 at 4:20 PM Romain Manni-Bucau
> > >
> >  > wrote:
> >  >>
> >  >> Ok, didn't find a way to make it working properly (only
> workaround
> >  >> with direct commands and no good idea integration for
> > debugging). I'm
> >  >> back with maven, if anyone knows how to properly solve it let's
> > do it.
> >  >> If not I think JB point is to consider more than any other
> criteria.
> >  >>
> >  >> Romain Manni-Bucau
> >  >> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >  >>
> >  >>
> >  >> 2018-04-10 16:41 GMT+02:00 Romain Manni-Bucau
> > >:
> >  >> > side note: do NOT use auto-import until you are sure you can,
> > it locks
> >  >> > regularly on beam (pby too big for idea?) and makes idea ready
> > to be
> >  >> > killed :(
> >  >> >
> >  >> > Romain Manni-Bucau
> >  >> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >  >> >
> >  >> >
> >  >> > 2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré
> > >:
> >  >> >> It's what I did, I'm trying a complete reload now (maybe this
> > step
> >  >> >> failed).
> >  >> >>
> >  >> >> On 10/04/2018 16:38, Lukasz Cwik wrote:
> >  >> >>>
> >  >> >>> beam-site PR/414 updates the instructions for using Intellij
> > and how
> >  >> >>> to
> >  >> >>> import a module:
> >  >> >>>
> >  >> >>> 1. Create an empty IntelliJ project outside of the Beam
> > source tree.
> >  >> >>> 2. Under Project Structure > Project, select a Project SDK.
> >  >> >>> 3. Under Project Structure > Modules, click the + sign to
> > add a module
> >  >> >>> and
> >  >> >>> select "Import Module".
> >  >> >>>  1. Select the directory containing the Beam source tree.
> >  >> >>>  2. Tick the "Import module from external model" button
> > and select
> >  >> >>> Gradle
> >  >> >>> from the list.
> >  >> >>>  3. Tick the following boxes.
> >  >> >>> * Use auto-import
> >  >> >>> * Create separate module per source set
> >  >> >>> * Store generated project files externally
> >  >> >>> * Use default gradle wrapper
> >  >> >>> 4. Delegate build actions to Gradle by 

Re: Gradle Status [April 6]

[Jean-Baptiste Onofré]

FYI, I did a new attempt and it works fine (pretty long). Previous try 
failed.


Regards
JB

On 10/04/2018 19:52, Kenneth Knowles wrote:
I've been on Idea+Gradle for ~two months, around the time I added 
https://github.com/apache/beam/pull/4583 and 
https://github.com/apache/beam/pull/4626 to make the import require zero 
user work. I have no fear of deleting my project any time and re-importing.


I agree with not having auto-import on. It is just too slow. I can't 
remember if it was importing too often due to build outputs or if it was 
just that I was messing with the build.gradle files. Anyhow it doesn't 
really add much value.


The gradle runner _is_ able to use submodules and run individual tests 
methods, and all that.


Kenn


On Tue, Apr 10, 2018 at 9:31 AM Romain Manni-Bucau 
> wrote:


Runner a test doesnt have the right classpath (idea uses out/ instead
of build/) then when you switch on gradle runner the launching uses
gradle which is not able to use submodules directly but reconsider the
whole project which is quite slow for normal dev iterations
compare to just run the test with the right classpath and a fast
compile step if needed. I lost literally 1h for something simple with
that tooling, this is way too much to be acceptable on my side since
I'm sadly not paid to work on beam (one day maybe ;)).

Romain Manni-Bucau
@rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book


2018-04-10 18:27 GMT+02:00 Reuven Lax >:
 > Romain,
 >
 > Can you detail what's not working. I switched my IntelliJ over to
Gradle
 > about two weeks ago, and haven't had any trouble.
 >
 > Reuven
 >
 > On Tue, Apr 10, 2018 at 4:20 PM Romain Manni-Bucau
>
 > wrote:
 >>
 >> Ok, didn't find a way to make it working properly (only workaround
 >> with direct commands and no good idea integration for
debugging). I'm
 >> back with maven, if anyone knows how to properly solve it let's
do it.
 >> If not I think JB point is to consider more than any other criteria.
 >>
 >> Romain Manni-Bucau
 >> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
 >>
 >>
 >> 2018-04-10 16:41 GMT+02:00 Romain Manni-Bucau
>:
 >> > side note: do NOT use auto-import until you are sure you can,
it locks
 >> > regularly on beam (pby too big for idea?) and makes idea ready
to be
 >> > killed :(
 >> >
 >> > Romain Manni-Bucau
 >> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
 >> >
 >> >
 >> > 2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré
>:
 >> >> It's what I did, I'm trying a complete reload now (maybe this
step
 >> >> failed).
 >> >>
 >> >> On 10/04/2018 16:38, Lukasz Cwik wrote:
 >> >>>
 >> >>> beam-site PR/414 updates the instructions for using Intellij
and how
 >> >>> to
 >> >>> import a module:
 >> >>>
 >> >>> 1. Create an empty IntelliJ project outside of the Beam
source tree.
 >> >>> 2. Under Project Structure > Project, select a Project SDK.
 >> >>> 3. Under Project Structure > Modules, click the + sign to
add a module
 >> >>> and
 >> >>>     select "Import Module".
 >> >>>      1. Select the directory containing the Beam source tree.
 >> >>>      2. Tick the "Import module from external model" button
and select
 >> >>> Gradle
 >> >>>         from the list.
 >> >>>      3. Tick the following boxes.
 >> >>>         * Use auto-import
 >> >>>         * Create separate module per source set
 >> >>>         * Store generated project files externally
 >> >>>         * Use default gradle wrapper
 >> >>> 4. Delegate build actions to Gradle by going to Settings >
Build,
 >> >>> Execution,
 >> >>>     Deployment > Build Tools > Gradle and checking "Delegate IDE
 >> >>> build/run
 >> >>>     actions to gradle".
 >> >>>
 >> >>> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré

 >> >>> >> wrote:
 >> >>>
 >> >>>     That's a very important issue for contribution. Up to
now, I used
 >> >>> Maven
 >> >>>     for setup IntelliJ (and it works just fine). If we
remove the
 >> >>> pom.xml,
 >> >>>     we have to support Eclipse and IntelliJ "smoothly".
 >> >>>
 >> >>>     Let me try in IntelliJ.
 >> >>>
 >> >>>     Regards
 >> >>>     JB
 >> >>>
 >> >>>     On 10/04/2018 15:21, Romain Manni-Bucau wrote:
 >> >>>      > You dont have issue due to the build setup with that
option. I

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

Le 10 avr. 2018 19:53, "Kenneth Knowles"  a écrit :

I've been on Idea+Gradle for ~two months, around the time I added
https://github.com/apache/beam/pull/4583 and https://github.com/apache/
beam/pull/4626 to make the import require zero user work. I have no fear of
deleting my project any time and re-importing.



The import works (is slow but works) but then i can run anything "normally".


I agree with not having auto-import on. It is just too slow. I can't
remember if it was importing too often due to build outputs or if it was
just that I was messing with the build.gradle files. Anyhow it doesn't
really add much value.

The gradle runner _is_ able to use submodules and run individual tests
methods, and all that.



If i run gradle from any io or any runner module it still loads the whole
project and checks it which is very slow compare to just run the
module...and i hope i dont need a 50chars long command specific each time
to do it, just "build" would be good.


Kenn


On Tue, Apr 10, 2018 at 9:31 AM Romain Manni-Bucau 
wrote:

> Runner a test doesnt have the right classpath (idea uses out/ instead
> of build/) then when you switch on gradle runner the launching uses
> gradle which is not able to use submodules directly but reconsider the
> whole project which is quite slow for normal dev iterations
> compare to just run the test with the right classpath and a fast
> compile step if needed. I lost literally 1h for something simple with
> that tooling, this is way too much to be acceptable on my side since
> I'm sadly not paid to work on beam (one day maybe ;)).
>
> Romain Manni-Bucau
> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>
>
> 2018-04-10 18:27 GMT+02:00 Reuven Lax :
> > Romain,
> >
> > Can you detail what's not working. I switched my IntelliJ over to Gradle
> > about two weeks ago, and haven't had any trouble.
> >
> > Reuven
> >
> > On Tue, Apr 10, 2018 at 4:20 PM Romain Manni-Bucau <
> rmannibu...@gmail.com>
> > wrote:
> >>
> >> Ok, didn't find a way to make it working properly (only workaround
> >> with direct commands and no good idea integration for debugging). I'm
> >> back with maven, if anyone knows how to properly solve it let's do it.
> >> If not I think JB point is to consider more than any other criteria.
> >>
> >> Romain Manni-Bucau
> >> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >>
> >>
> >> 2018-04-10 16:41 GMT+02:00 Romain Manni-Bucau :
> >> > side note: do NOT use auto-import until you are sure you can, it locks
> >> > regularly on beam (pby too big for idea?) and makes idea ready to be
> >> > killed :(
> >> >
> >> > Romain Manni-Bucau
> >> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >> >
> >> >
> >> > 2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré :
> >> >> It's what I did, I'm trying a complete reload now (maybe this step
> >> >> failed).
> >> >>
> >> >> On 10/04/2018 16:38, Lukasz Cwik wrote:
> >> >>>
> >> >>> beam-site PR/414 updates the instructions for using Intellij and how
> >> >>> to
> >> >>> import a module:
> >> >>>
> >> >>> 1. Create an empty IntelliJ project outside of the Beam source tree.
> >> >>> 2. Under Project Structure > Project, select a Project SDK.
> >> >>> 3. Under Project Structure > Modules, click the + sign to add a
> module
> >> >>> and
> >> >>> select "Import Module".
> >> >>>  1. Select the directory containing the Beam source tree.
> >> >>>  2. Tick the "Import module from external model" button and
> select
> >> >>> Gradle
> >> >>> from the list.
> >> >>>  3. Tick the following boxes.
> >> >>> * Use auto-import
> >> >>> * Create separate module per source set
> >> >>> * Store generated project files externally
> >> >>> * Use default gradle wrapper
> >> >>> 4. Delegate build actions to Gradle by going to Settings > Build,
> >> >>> Execution,
> >> >>> Deployment > Build Tools > Gradle and checking "Delegate IDE
> >> >>> build/run
> >> >>> actions to gradle".
> >> >>>
> >> >>> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré <
> j...@nanthrax.net
> >> >>> > wrote:
> >> >>>
> >> >>> That's a very important issue for contribution. Up to now, I
> used
> >> >>> Maven
> >> >>> for setup IntelliJ (and it works just fine). If we remove the
> >> >>> pom.xml,
> >> >>> we have to support Eclipse and IntelliJ "smoothly".
> >> >>>
> >> >>> Let me try in IntelliJ.
> >> >>>
> >> >>> Regards
> >> >>> JB
> >> >>>
> >> >>> On 10/04/2018 15:21, Romain Manni-Bucau wrote:
> >> >>>  > You dont have issue due to the build setup with that option.
> I
> >> >>> get:
> >> >>>  >
> >> >>>  > avr. 10, 2018 3:20:10 PM
> >> >>>  > org.apache.beam.runners.direct.DirectTransformExecutor run
> >> >>>  > GRAVE: Error occurred within
> >> >>>  > 

Re: Gradle Status [April 6]

[Kenneth Knowles]

I've been on Idea+Gradle for ~two months, around the time I added
https://github.com/apache/beam/pull/4583 and
https://github.com/apache/beam/pull/4626 to make the import require zero
user work. I have no fear of deleting my project any time and re-importing.

I agree with not having auto-import on. It is just too slow. I can't
remember if it was importing too often due to build outputs or if it was
just that I was messing with the build.gradle files. Anyhow it doesn't
really add much value.

The gradle runner _is_ able to use submodules and run individual tests
methods, and all that.

Kenn


On Tue, Apr 10, 2018 at 9:31 AM Romain Manni-Bucau 
wrote:

> Runner a test doesnt have the right classpath (idea uses out/ instead
> of build/) then when you switch on gradle runner the launching uses
> gradle which is not able to use submodules directly but reconsider the
> whole project which is quite slow for normal dev iterations
> compare to just run the test with the right classpath and a fast
> compile step if needed. I lost literally 1h for something simple with
> that tooling, this is way too much to be acceptable on my side since
> I'm sadly not paid to work on beam (one day maybe ;)).
>
> Romain Manni-Bucau
> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>
>
> 2018-04-10 18:27 GMT+02:00 Reuven Lax :
> > Romain,
> >
> > Can you detail what's not working. I switched my IntelliJ over to Gradle
> > about two weeks ago, and haven't had any trouble.
> >
> > Reuven
> >
> > On Tue, Apr 10, 2018 at 4:20 PM Romain Manni-Bucau <
> rmannibu...@gmail.com>
> > wrote:
> >>
> >> Ok, didn't find a way to make it working properly (only workaround
> >> with direct commands and no good idea integration for debugging). I'm
> >> back with maven, if anyone knows how to properly solve it let's do it.
> >> If not I think JB point is to consider more than any other criteria.
> >>
> >> Romain Manni-Bucau
> >> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >>
> >>
> >> 2018-04-10 16:41 GMT+02:00 Romain Manni-Bucau :
> >> > side note: do NOT use auto-import until you are sure you can, it locks
> >> > regularly on beam (pby too big for idea?) and makes idea ready to be
> >> > killed :(
> >> >
> >> > Romain Manni-Bucau
> >> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >> >
> >> >
> >> > 2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré :
> >> >> It's what I did, I'm trying a complete reload now (maybe this step
> >> >> failed).
> >> >>
> >> >> On 10/04/2018 16:38, Lukasz Cwik wrote:
> >> >>>
> >> >>> beam-site PR/414 updates the instructions for using Intellij and how
> >> >>> to
> >> >>> import a module:
> >> >>>
> >> >>> 1. Create an empty IntelliJ project outside of the Beam source tree.
> >> >>> 2. Under Project Structure > Project, select a Project SDK.
> >> >>> 3. Under Project Structure > Modules, click the + sign to add a
> module
> >> >>> and
> >> >>> select "Import Module".
> >> >>>  1. Select the directory containing the Beam source tree.
> >> >>>  2. Tick the "Import module from external model" button and
> select
> >> >>> Gradle
> >> >>> from the list.
> >> >>>  3. Tick the following boxes.
> >> >>> * Use auto-import
> >> >>> * Create separate module per source set
> >> >>> * Store generated project files externally
> >> >>> * Use default gradle wrapper
> >> >>> 4. Delegate build actions to Gradle by going to Settings > Build,
> >> >>> Execution,
> >> >>> Deployment > Build Tools > Gradle and checking "Delegate IDE
> >> >>> build/run
> >> >>> actions to gradle".
> >> >>>
> >> >>> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré <
> j...@nanthrax.net
> >> >>> > wrote:
> >> >>>
> >> >>> That's a very important issue for contribution. Up to now, I
> used
> >> >>> Maven
> >> >>> for setup IntelliJ (and it works just fine). If we remove the
> >> >>> pom.xml,
> >> >>> we have to support Eclipse and IntelliJ "smoothly".
> >> >>>
> >> >>> Let me try in IntelliJ.
> >> >>>
> >> >>> Regards
> >> >>> JB
> >> >>>
> >> >>> On 10/04/2018 15:21, Romain Manni-Bucau wrote:
> >> >>>  > You dont have issue due to the build setup with that option.
> I
> >> >>> get:
> >> >>>  >
> >> >>>  > avr. 10, 2018 3:20:10 PM
> >> >>>  > org.apache.beam.runners.direct.DirectTransformExecutor run
> >> >>>  > GRAVE: Error occurred within
> >> >>>  >
> org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a
> >> >>>  > com.google.common.util.concurrent.ExecutionError:
> >> >>>  > java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy
> >> >>>  >
> >> >>>  > ?
> >> >>>  >
> >> >>>  > Romain Manni-Bucau
> >> >>>  > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >> >>>  >
> >> >>>  >
> >> >>>  > 2018-04-10 15:13 GMT+02:00 

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

Runner a test doesnt have the right classpath (idea uses out/ instead
of build/) then when you switch on gradle runner the launching uses
gradle which is not able to use submodules directly but reconsider the
whole project which is quite slow for normal dev iterations
compare to just run the test with the right classpath and a fast
compile step if needed. I lost literally 1h for something simple with
that tooling, this is way too much to be acceptable on my side since
I'm sadly not paid to work on beam (one day maybe ;)).

Romain Manni-Bucau
@rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book


2018-04-10 18:27 GMT+02:00 Reuven Lax :
> Romain,
>
> Can you detail what's not working. I switched my IntelliJ over to Gradle
> about two weeks ago, and haven't had any trouble.
>
> Reuven
>
> On Tue, Apr 10, 2018 at 4:20 PM Romain Manni-Bucau 
> wrote:
>>
>> Ok, didn't find a way to make it working properly (only workaround
>> with direct commands and no good idea integration for debugging). I'm
>> back with maven, if anyone knows how to properly solve it let's do it.
>> If not I think JB point is to consider more than any other criteria.
>>
>> Romain Manni-Bucau
>> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>
>>
>> 2018-04-10 16:41 GMT+02:00 Romain Manni-Bucau :
>> > side note: do NOT use auto-import until you are sure you can, it locks
>> > regularly on beam (pby too big for idea?) and makes idea ready to be
>> > killed :(
>> >
>> > Romain Manni-Bucau
>> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>> >
>> >
>> > 2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré :
>> >> It's what I did, I'm trying a complete reload now (maybe this step
>> >> failed).
>> >>
>> >> On 10/04/2018 16:38, Lukasz Cwik wrote:
>> >>>
>> >>> beam-site PR/414 updates the instructions for using Intellij and how
>> >>> to
>> >>> import a module:
>> >>>
>> >>> 1. Create an empty IntelliJ project outside of the Beam source tree.
>> >>> 2. Under Project Structure > Project, select a Project SDK.
>> >>> 3. Under Project Structure > Modules, click the + sign to add a module
>> >>> and
>> >>> select "Import Module".
>> >>>  1. Select the directory containing the Beam source tree.
>> >>>  2. Tick the "Import module from external model" button and select
>> >>> Gradle
>> >>> from the list.
>> >>>  3. Tick the following boxes.
>> >>> * Use auto-import
>> >>> * Create separate module per source set
>> >>> * Store generated project files externally
>> >>> * Use default gradle wrapper
>> >>> 4. Delegate build actions to Gradle by going to Settings > Build,
>> >>> Execution,
>> >>> Deployment > Build Tools > Gradle and checking "Delegate IDE
>> >>> build/run
>> >>> actions to gradle".
>> >>>
>> >>> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré > >>> > wrote:
>> >>>
>> >>> That's a very important issue for contribution. Up to now, I used
>> >>> Maven
>> >>> for setup IntelliJ (and it works just fine). If we remove the
>> >>> pom.xml,
>> >>> we have to support Eclipse and IntelliJ "smoothly".
>> >>>
>> >>> Let me try in IntelliJ.
>> >>>
>> >>> Regards
>> >>> JB
>> >>>
>> >>> On 10/04/2018 15:21, Romain Manni-Bucau wrote:
>> >>>  > You dont have issue due to the build setup with that option. I
>> >>> get:
>> >>>  >
>> >>>  > avr. 10, 2018 3:20:10 PM
>> >>>  > org.apache.beam.runners.direct.DirectTransformExecutor run
>> >>>  > GRAVE: Error occurred within
>> >>>  > org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a
>> >>>  > com.google.common.util.concurrent.ExecutionError:
>> >>>  > java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy
>> >>>  >
>> >>>  > ?
>> >>>  >
>> >>>  > Romain Manni-Bucau
>> >>>  > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>> >>>  >
>> >>>  >
>> >>>  > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik > >>> >:
>> >>>  >> I have found that the simplest setup is to delegate the
>> >>> build/test actions
>> >>>  >> to Gradle. This allows you to run unit tests very easily and
>> >>> since its in
>> >>>  >> the same manner that Gradle would have, you know that if its
>> >>> passing it will
>> >>>  >> pass on the command line and on Jenkins. Here is one site that
>> >>> discusses how
>> >>>  >> to set this up:
>> >>>  >>
>> >>>
>> >>>
>> >>> http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html
>> >>>  >>
>> >>>  >>
>> >>>  >> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau
>> >>> >
>> >>>  >> wrote:
>> >>>  >>>
>> >>>  >>> What's the plan to make idea supporting gradle on beam
>> >>> project?

Re: Gradle Status [April 6]

[Reuven Lax]

Romain,

Can you detail what's not working. I switched my IntelliJ over to Gradle
about two weeks ago, and haven't had any trouble.

Reuven

On Tue, Apr 10, 2018 at 4:20 PM Romain Manni-Bucau 
wrote:

> Ok, didn't find a way to make it working properly (only workaround
> with direct commands and no good idea integration for debugging). I'm
> back with maven, if anyone knows how to properly solve it let's do it.
> If not I think JB point is to consider more than any other criteria.
>
> Romain Manni-Bucau
> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>
>
> 2018-04-10 16:41 GMT+02:00 Romain Manni-Bucau :
> > side note: do NOT use auto-import until you are sure you can, it locks
> > regularly on beam (pby too big for idea?) and makes idea ready to be
> > killed :(
> >
> > Romain Manni-Bucau
> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >
> >
> > 2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré :
> >> It's what I did, I'm trying a complete reload now (maybe this step
> failed).
> >>
> >> On 10/04/2018 16:38, Lukasz Cwik wrote:
> >>>
> >>> beam-site PR/414 updates the instructions for using Intellij and how to
> >>> import a module:
> >>>
> >>> 1. Create an empty IntelliJ project outside of the Beam source tree.
> >>> 2. Under Project Structure > Project, select a Project SDK.
> >>> 3. Under Project Structure > Modules, click the + sign to add a module
> and
> >>> select "Import Module".
> >>>  1. Select the directory containing the Beam source tree.
> >>>  2. Tick the "Import module from external model" button and select
> >>> Gradle
> >>> from the list.
> >>>  3. Tick the following boxes.
> >>> * Use auto-import
> >>> * Create separate module per source set
> >>> * Store generated project files externally
> >>> * Use default gradle wrapper
> >>> 4. Delegate build actions to Gradle by going to Settings > Build,
> >>> Execution,
> >>> Deployment > Build Tools > Gradle and checking "Delegate IDE
> build/run
> >>> actions to gradle".
> >>>
> >>> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré  >>> > wrote:
> >>>
> >>> That's a very important issue for contribution. Up to now, I used
> >>> Maven
> >>> for setup IntelliJ (and it works just fine). If we remove the
> pom.xml,
> >>> we have to support Eclipse and IntelliJ "smoothly".
> >>>
> >>> Let me try in IntelliJ.
> >>>
> >>> Regards
> >>> JB
> >>>
> >>> On 10/04/2018 15:21, Romain Manni-Bucau wrote:
> >>>  > You dont have issue due to the build setup with that option. I
> get:
> >>>  >
> >>>  > avr. 10, 2018 3:20:10 PM
> >>>  > org.apache.beam.runners.direct.DirectTransformExecutor run
> >>>  > GRAVE: Error occurred within
> >>>  > org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a
> >>>  > com.google.common.util.concurrent.ExecutionError:
> >>>  > java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy
> >>>  >
> >>>  > ?
> >>>  >
> >>>  > Romain Manni-Bucau
> >>>  > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >>>  >
> >>>  >
> >>>  > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik  >>> >:
> >>>  >> I have found that the simplest setup is to delegate the
> >>> build/test actions
> >>>  >> to Gradle. This allows you to run unit tests very easily and
> >>> since its in
> >>>  >> the same manner that Gradle would have, you know that if its
> >>> passing it will
> >>>  >> pass on the command line and on Jenkins. Here is one site that
> >>> discusses how
> >>>  >> to set this up:
> >>>  >>
> >>>
> >>>
> http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html
> >>>  >>
> >>>  >>
> >>>  >> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau
> >>> >
> >>>  >> wrote:
> >>>  >>>
> >>>  >>> What's the plan to make idea supporting gradle on beam
> project?
> >>> Do we
> >>>  >>> import the workaround mentionned in
> >>>  >>> https://youtrack.jetbrains.com/issue/IDEA-175172?
> >>>  >>> For the ones who didn't see this issue in action: idea will
> >>> compile in
> >>>  >>> out/ instead of build/ and you will just miss all the
> resources
> >>> you
> >>>  >>> need like some SPI registration which are used by all our
> >>> registrar =>
> >>>  >>> no way to run tests in idea without hacking the configuration
> >>> quite
> >>>  >>> deeply :(
> >>>  >>>
> >>>  >>> Romain Manni-Bucau
> >>>  >>> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >>>  >>>
> >>>  >>>
> >>>  >>> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot
> >>> >:
> >>>
> >>>

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

Ok, didn't find a way to make it working properly (only workaround
with direct commands and no good idea integration for debugging). I'm
back with maven, if anyone knows how to properly solve it let's do it.
If not I think JB point is to consider more than any other criteria.

Romain Manni-Bucau
@rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book


2018-04-10 16:41 GMT+02:00 Romain Manni-Bucau :
> side note: do NOT use auto-import until you are sure you can, it locks
> regularly on beam (pby too big for idea?) and makes idea ready to be
> killed :(
>
> Romain Manni-Bucau
> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>
>
> 2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré :
>> It's what I did, I'm trying a complete reload now (maybe this step failed).
>>
>> On 10/04/2018 16:38, Lukasz Cwik wrote:
>>>
>>> beam-site PR/414 updates the instructions for using Intellij and how to
>>> import a module:
>>>
>>> 1. Create an empty IntelliJ project outside of the Beam source tree.
>>> 2. Under Project Structure > Project, select a Project SDK.
>>> 3. Under Project Structure > Modules, click the + sign to add a module and
>>> select "Import Module".
>>>  1. Select the directory containing the Beam source tree.
>>>  2. Tick the "Import module from external model" button and select
>>> Gradle
>>> from the list.
>>>  3. Tick the following boxes.
>>> * Use auto-import
>>> * Create separate module per source set
>>> * Store generated project files externally
>>> * Use default gradle wrapper
>>> 4. Delegate build actions to Gradle by going to Settings > Build,
>>> Execution,
>>> Deployment > Build Tools > Gradle and checking "Delegate IDE build/run
>>> actions to gradle".
>>>
>>> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré >> > wrote:
>>>
>>> That's a very important issue for contribution. Up to now, I used
>>> Maven
>>> for setup IntelliJ (and it works just fine). If we remove the pom.xml,
>>> we have to support Eclipse and IntelliJ "smoothly".
>>>
>>> Let me try in IntelliJ.
>>>
>>> Regards
>>> JB
>>>
>>> On 10/04/2018 15:21, Romain Manni-Bucau wrote:
>>>  > You dont have issue due to the build setup with that option. I get:
>>>  >
>>>  > avr. 10, 2018 3:20:10 PM
>>>  > org.apache.beam.runners.direct.DirectTransformExecutor run
>>>  > GRAVE: Error occurred within
>>>  > org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a
>>>  > com.google.common.util.concurrent.ExecutionError:
>>>  > java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy
>>>  >
>>>  > ?
>>>  >
>>>  > Romain Manni-Bucau
>>>  > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>>  >
>>>  >
>>>  > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik >> >:
>>>  >> I have found that the simplest setup is to delegate the
>>> build/test actions
>>>  >> to Gradle. This allows you to run unit tests very easily and
>>> since its in
>>>  >> the same manner that Gradle would have, you know that if its
>>> passing it will
>>>  >> pass on the command line and on Jenkins. Here is one site that
>>> discusses how
>>>  >> to set this up:
>>>  >>
>>>
>>> http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html
>>>  >>
>>>  >>
>>>  >> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau
>>> >
>>>  >> wrote:
>>>  >>>
>>>  >>> What's the plan to make idea supporting gradle on beam project?
>>> Do we
>>>  >>> import the workaround mentionned in
>>>  >>> https://youtrack.jetbrains.com/issue/IDEA-175172?
>>>  >>> For the ones who didn't see this issue in action: idea will
>>> compile in
>>>  >>> out/ instead of build/ and you will just miss all the resources
>>> you
>>>  >>> need like some SPI registration which are used by all our
>>> registrar =>
>>>  >>> no way to run tests in idea without hacking the configuration
>>> quite
>>>  >>> deeply :(
>>>  >>>
>>>  >>> Romain Manni-Bucau
>>>  >>> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>>  >>>
>>>  >>>
>>>  >>> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot
>>> >:
>>>
>>>   As a gradle beginner, I could not agree more !
>>>   +1
>>>   Etienne
>>>   Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste Onofré a
>>> écrit :
>>>  
>>>   Hi all,
>>>  
>>>   I did multiple gradle build since last week and I would like
>>> to share
>>>   one of my concern: it's about the communities.
>>>  
>>>   If I think our users won't see any change for them 

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

side note: do NOT use auto-import until you are sure you can, it locks
regularly on beam (pby too big for idea?) and makes idea ready to be
killed :(

Romain Manni-Bucau
@rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book


2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré :
> It's what I did, I'm trying a complete reload now (maybe this step failed).
>
> On 10/04/2018 16:38, Lukasz Cwik wrote:
>>
>> beam-site PR/414 updates the instructions for using Intellij and how to
>> import a module:
>>
>> 1. Create an empty IntelliJ project outside of the Beam source tree.
>> 2. Under Project Structure > Project, select a Project SDK.
>> 3. Under Project Structure > Modules, click the + sign to add a module and
>> select "Import Module".
>>  1. Select the directory containing the Beam source tree.
>>  2. Tick the "Import module from external model" button and select
>> Gradle
>> from the list.
>>  3. Tick the following boxes.
>> * Use auto-import
>> * Create separate module per source set
>> * Store generated project files externally
>> * Use default gradle wrapper
>> 4. Delegate build actions to Gradle by going to Settings > Build,
>> Execution,
>> Deployment > Build Tools > Gradle and checking "Delegate IDE build/run
>> actions to gradle".
>>
>> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré > > wrote:
>>
>> That's a very important issue for contribution. Up to now, I used
>> Maven
>> for setup IntelliJ (and it works just fine). If we remove the pom.xml,
>> we have to support Eclipse and IntelliJ "smoothly".
>>
>> Let me try in IntelliJ.
>>
>> Regards
>> JB
>>
>> On 10/04/2018 15:21, Romain Manni-Bucau wrote:
>>  > You dont have issue due to the build setup with that option. I get:
>>  >
>>  > avr. 10, 2018 3:20:10 PM
>>  > org.apache.beam.runners.direct.DirectTransformExecutor run
>>  > GRAVE: Error occurred within
>>  > org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a
>>  > com.google.common.util.concurrent.ExecutionError:
>>  > java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy
>>  >
>>  > ?
>>  >
>>  > Romain Manni-Bucau
>>  > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>  >
>>  >
>>  > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik > >:
>>  >> I have found that the simplest setup is to delegate the
>> build/test actions
>>  >> to Gradle. This allows you to run unit tests very easily and
>> since its in
>>  >> the same manner that Gradle would have, you know that if its
>> passing it will
>>  >> pass on the command line and on Jenkins. Here is one site that
>> discusses how
>>  >> to set this up:
>>  >>
>>
>> http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html
>>  >>
>>  >>
>>  >> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau
>> >
>>  >> wrote:
>>  >>>
>>  >>> What's the plan to make idea supporting gradle on beam project?
>> Do we
>>  >>> import the workaround mentionned in
>>  >>> https://youtrack.jetbrains.com/issue/IDEA-175172?
>>  >>> For the ones who didn't see this issue in action: idea will
>> compile in
>>  >>> out/ instead of build/ and you will just miss all the resources
>> you
>>  >>> need like some SPI registration which are used by all our
>> registrar =>
>>  >>> no way to run tests in idea without hacking the configuration
>> quite
>>  >>> deeply :(
>>  >>>
>>  >>> Romain Manni-Bucau
>>  >>> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>  >>>
>>  >>>
>>  >>> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot
>> >:
>>
>>   As a gradle beginner, I could not agree more !
>>   +1
>>   Etienne
>>   Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste Onofré a
>> écrit :
>>  
>>   Hi all,
>>  
>>   I did multiple gradle build since last week and I would like
>> to share
>>   one of my concern: it's about the communities.
>>  
>>   If I think our users won't see any change for them due to
>> Gradle build
>>   (I think that most of our users will still use Maven with
>> artifacts
>>   provided by Gradle), I'm more concerned by the dev community
>> and the
>>   contribution.
>>  
>>   Maven is well known and straight forward for a large part of
>> potential
>>   contributors. I think we have to keep in mind that we still
>> have to grow
>>   up our contributors community.
>>  
>>   Today, maybe I'm wrong, but I have the feeling that 

Re: Gradle Status [April 6]

[Jean-Baptiste Onofré]

It's what I did, I'm trying a complete reload now (maybe this step failed).

On 10/04/2018 16:38, Lukasz Cwik wrote:
beam-site PR/414 updates the instructions for using Intellij and how to 
import a module:


1. Create an empty IntelliJ project outside of the Beam source tree.
2. Under Project Structure > Project, select a Project SDK.
3. Under Project Structure > Modules, click the + sign to add a module and
    select "Import Module".
     1. Select the directory containing the Beam source tree.
     2. Tick the "Import module from external model" button and select 
Gradle

        from the list.
     3. Tick the following boxes.
        * Use auto-import
        * Create separate module per source set
        * Store generated project files externally
        * Use default gradle wrapper
4. Delegate build actions to Gradle by going to Settings > Build, Execution,
    Deployment > Build Tools > Gradle and checking "Delegate IDE build/run
    actions to gradle".

On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré > wrote:


That's a very important issue for contribution. Up to now, I used Maven
for setup IntelliJ (and it works just fine). If we remove the pom.xml,
we have to support Eclipse and IntelliJ "smoothly".

Let me try in IntelliJ.

Regards
JB

On 10/04/2018 15:21, Romain Manni-Bucau wrote:
 > You dont have issue due to the build setup with that option. I get:
 >
 > avr. 10, 2018 3:20:10 PM
 > org.apache.beam.runners.direct.DirectTransformExecutor run
 > GRAVE: Error occurred within
 > org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a
 > com.google.common.util.concurrent.ExecutionError:
 > java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy
 >
 > ?
 >
 > Romain Manni-Bucau
 > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
 >
 >
 > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik >:
 >> I have found that the simplest setup is to delegate the
build/test actions
 >> to Gradle. This allows you to run unit tests very easily and
since its in
 >> the same manner that Gradle would have, you know that if its
passing it will
 >> pass on the command line and on Jenkins. Here is one site that
discusses how
 >> to set this up:
 >>

http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html
 >>
 >>
 >> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau
>
 >> wrote:
 >>>
 >>> What's the plan to make idea supporting gradle on beam project?
Do we
 >>> import the workaround mentionned in
 >>> https://youtrack.jetbrains.com/issue/IDEA-175172?
 >>> For the ones who didn't see this issue in action: idea will
compile in
 >>> out/ instead of build/ and you will just miss all the resources you
 >>> need like some SPI registration which are used by all our
registrar =>
 >>> no way to run tests in idea without hacking the configuration quite
 >>> deeply :(
 >>>
 >>> Romain Manni-Bucau
 >>> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
 >>>
 >>>
 >>> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot
>:
  As a gradle beginner, I could not agree more !
  +1
  Etienne
  Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste Onofré a
écrit :
 
  Hi all,
 
  I did multiple gradle build since last week and I would like
to share
  one of my concern: it's about the communities.
 
  If I think our users won't see any change for them due to
Gradle build
  (I think that most of our users will still use Maven with
artifacts
  provided by Gradle), I'm more concerned by the dev community
and the
  contribution.
 
  Maven is well known and straight forward for a large part of
potential
  contributors. I think we have to keep in mind that we still
have to grow
  up our contributors community.
 
  Today, maybe I'm wrong, but I have the feeling that gradle
build is not
  straight forward (build.gradle includes build_rules.gradle,
gathering
  all taks all together).
 
  I would like to add a task in the gradle "migration" process:
simplify
  the gradle structure and files, and document this.
 
  I know we already have a Jira about the documentation part,
but I would
  like to "polish" and use a clean structure for the Gradle
resources. As
  already quickly discussed, I think that having one gradle file
per tasks
  in the .gradle directory would be helpful.
 
  The 

Re: Gradle Status [April 6]

[Lukasz Cwik]

beam-site PR/414 updates the instructions for using Intellij and how to
import a module:

1. Create an empty IntelliJ project outside of the Beam source tree.
2. Under Project Structure > Project, select a Project SDK.
3. Under Project Structure > Modules, click the + sign to add a module and
   select "Import Module".
1. Select the directory containing the Beam source tree.
2. Tick the "Import module from external model" button and select Gradle
   from the list.
3. Tick the following boxes.
   * Use auto-import
   * Create separate module per source set
   * Store generated project files externally
   * Use default gradle wrapper
4. Delegate build actions to Gradle by going to Settings > Build, Execution,
   Deployment > Build Tools > Gradle and checking "Delegate IDE build/run
   actions to gradle".

On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré 
wrote:

> That's a very important issue for contribution. Up to now, I used Maven
> for setup IntelliJ (and it works just fine). If we remove the pom.xml,
> we have to support Eclipse and IntelliJ "smoothly".
>
> Let me try in IntelliJ.
>
> Regards
> JB
>
> On 10/04/2018 15:21, Romain Manni-Bucau wrote:
> > You dont have issue due to the build setup with that option. I get:
> >
> > avr. 10, 2018 3:20:10 PM
> > org.apache.beam.runners.direct.DirectTransformExecutor run
> > GRAVE: Error occurred within
> > org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a
> > com.google.common.util.concurrent.ExecutionError:
> > java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy
> >
> > ?
> >
> > Romain Manni-Bucau
> > @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >
> >
> > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik :
> >> I have found that the simplest setup is to delegate the build/test
> actions
> >> to Gradle. This allows you to run unit tests very easily and since its
> in
> >> the same manner that Gradle would have, you know that if its passing it
> will
> >> pass on the command line and on Jenkins. Here is one site that
> discusses how
> >> to set this up:
> >>
> http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html
> >>
> >>
> >> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau <
> rmannibu...@gmail.com>
> >> wrote:
> >>>
> >>> What's the plan to make idea supporting gradle on beam project? Do we
> >>> import the workaround mentionned in
> >>> https://youtrack.jetbrains.com/issue/IDEA-175172?
> >>> For the ones who didn't see this issue in action: idea will compile in
> >>> out/ instead of build/ and you will just miss all the resources you
> >>> need like some SPI registration which are used by all our registrar =>
> >>> no way to run tests in idea without hacking the configuration quite
> >>> deeply :(
> >>>
> >>> Romain Manni-Bucau
> >>> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >>>
> >>>
> >>> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot :
>  As a gradle beginner, I could not agree more !
>  +1
>  Etienne
>  Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste Onofré a écrit :
> 
>  Hi all,
> 
>  I did multiple gradle build since last week and I would like to share
>  one of my concern: it's about the communities.
> 
>  If I think our users won't see any change for them due to Gradle build
>  (I think that most of our users will still use Maven with artifacts
>  provided by Gradle), I'm more concerned by the dev community and the
>  contribution.
> 
>  Maven is well known and straight forward for a large part of potential
>  contributors. I think we have to keep in mind that we still have to
> grow
>  up our contributors community.
> 
>  Today, maybe I'm wrong, but I have the feeling that gradle build is
> not
>  straight forward (build.gradle includes build_rules.gradle, gathering
>  all taks all together).
> 
>  I would like to add a task in the gradle "migration" process: simplify
>  the gradle structure and files, and document this.
> 
>  I know we already have a Jira about the documentation part, but I
> would
>  like to "polish" and use a clean structure for the Gradle resources.
> As
>  already quickly discussed, I think that having one gradle file per
> tasks
>  in the .gradle directory would be helpful.
> 
>  The goal is really to simplify the contribution.
> 
>  Do you agree if I add a Jira about "Gradle polish" ?
>  Thoughts ?
> 
>  Regards
>  JB
> 
>  On 07/04/2018 04:52, Scott Wegner wrote:
> 
>  Here's an end-of-day update on migration work:
> 
>  * Snapshot unsigned dailies and signed release builds are working
> (!!).
>  PR/5048 [1] merges changes from Luke's branch
>  * python precommit failing... will investigate python precommit
>  Monday
>  * All Precommits are gradle only
>  

Re: Gradle Status [April 6]

[Jean-Baptiste Onofré]

That's a very important issue for contribution. Up to now, I used Maven 
for setup IntelliJ (and it works just fine). If we remove the pom.xml, 
we have to support Eclipse and IntelliJ "smoothly".


Let me try in IntelliJ.

Regards
JB

On 10/04/2018 15:21, Romain Manni-Bucau wrote:

You dont have issue due to the build setup with that option. I get:

avr. 10, 2018 3:20:10 PM
org.apache.beam.runners.direct.DirectTransformExecutor run
GRAVE: Error occurred within
org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a
com.google.common.util.concurrent.ExecutionError:
java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy

?

Romain Manni-Bucau
@rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book


2018-04-10 15:13 GMT+02:00 Lukasz Cwik :

I have found that the simplest setup is to delegate the build/test actions
to Gradle. This allows you to run unit tests very easily and since its in
the same manner that Gradle would have, you know that if its passing it will
pass on the command line and on Jenkins. Here is one site that discusses how
to set this up:
http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html


On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau 
wrote:


What's the plan to make idea supporting gradle on beam project? Do we
import the workaround mentionned in
https://youtrack.jetbrains.com/issue/IDEA-175172?
For the ones who didn't see this issue in action: idea will compile in
out/ instead of build/ and you will just miss all the resources you
need like some SPI registration which are used by all our registrar =>
no way to run tests in idea without hacking the configuration quite
deeply :(

Romain Manni-Bucau
@rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book


2018-04-10 10:08 GMT+02:00 Etienne Chauchot :

As a gradle beginner, I could not agree more !
+1
Etienne
Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste Onofré a écrit :

Hi all,

I did multiple gradle build since last week and I would like to share
one of my concern: it's about the communities.

If I think our users won't see any change for them due to Gradle build
(I think that most of our users will still use Maven with artifacts
provided by Gradle), I'm more concerned by the dev community and the
contribution.

Maven is well known and straight forward for a large part of potential
contributors. I think we have to keep in mind that we still have to grow
up our contributors community.

Today, maybe I'm wrong, but I have the feeling that gradle build is not
straight forward (build.gradle includes build_rules.gradle, gathering
all taks all together).

I would like to add a task in the gradle "migration" process: simplify
the gradle structure and files, and document this.

I know we already have a Jira about the documentation part, but I would
like to "polish" and use a clean structure for the Gradle resources. As
already quickly discussed, I think that having one gradle file per tasks
in the .gradle directory would be helpful.

The goal is really to simplify the contribution.

Do you agree if I add a Jira about "Gradle polish" ?
Thoughts ?

Regards
JB

On 07/04/2018 04:52, Scott Wegner wrote:

Here's an end-of-day update on migration work:

* Snapshot unsigned dailies and signed release builds are working (!!).
PR/5048 [1] merges changes from Luke's branch
* python precommit failing... will investigate python precommit
Monday
* All Precommits are gradle only
* All Postcommits except performance tests and Java_JDK_Versions_Test
use gradle (after PR/5047 [2] merged)
* Nightly snapshot release using gradle is ready; needs PR/5048 to be
merged before switching
* ValidatesRunner_Spark failing consistently; investigating

Thanks for another productive day of hacking. I'll pick up again on
Monday.

[1] https://github.com/apache/beam/pull/5048
[2] https://github.com/apache/beam/pull/5047


On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau
> wrote:

 Why building a zip per runner which its stack and just pointing out
 on that zip and let beam lazy load the runner:

 --runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=... (or
 the fromSystemProperties() if it gets merged a day ;))

 Le 6 avr. 2018 20:21, "Kenneth Knowles" > a écrit :

 I'm working on finding a solution for launching the Nexmark
 suite with each runner. This doesn't have to be done via Gradle,
 but we anyhow need built artifacts that don't require user
 classpath intervention.

 It looks to me like the examples are also missing this - they
 have separate configuration e.g. sparkRunnerPreCommit but that
 is overspecified compared to a free-form launching of a main()
 program with a runner profile.

 On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik 

Re: Gradle Status [April 6]

[Lukasz Cwik]

Romain, I haven't seen that error. At the very top of your test execution
log it gives you the tasks that it is running, for example:
6:41:33 AM: Executing tasks ':beam-sdks-java-core:cleanTest
:beam-sdks-java-core:test --tests
"org.apache.beam.sdk.coders.AvroCoderTest.testAvroCoderEncoding"'...

What task did it fail for you for?

On Tue, Apr 10, 2018 at 9:21 AM Romain Manni-Bucau 
wrote:

> You dont have issue due to the build setup with that option. I get:
>
> avr. 10, 2018 3:20:10 PM
> org.apache.beam.runners.direct.DirectTransformExecutor run
> GRAVE: Error occurred within
> org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a
> com.google.common.util.concurrent.ExecutionError:
> java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy
>
> ?
>
> Romain Manni-Bucau
> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>
>
> 2018-04-10 15:13 GMT+02:00 Lukasz Cwik :
> > I have found that the simplest setup is to delegate the build/test
> actions
> > to Gradle. This allows you to run unit tests very easily and since its in
> > the same manner that Gradle would have, you know that if its passing it
> will
> > pass on the command line and on Jenkins. Here is one site that discusses
> how
> > to set this up:
> >
> http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html
> >
> >
> > On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau <
> rmannibu...@gmail.com>
> > wrote:
> >>
> >> What's the plan to make idea supporting gradle on beam project? Do we
> >> import the workaround mentionned in
> >> https://youtrack.jetbrains.com/issue/IDEA-175172?
> >> For the ones who didn't see this issue in action: idea will compile in
> >> out/ instead of build/ and you will just miss all the resources you
> >> need like some SPI registration which are used by all our registrar =>
> >> no way to run tests in idea without hacking the configuration quite
> >> deeply :(
> >>
> >> Romain Manni-Bucau
> >> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
> >>
> >>
> >> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot :
> >> > As a gradle beginner, I could not agree more !
> >> > +1
> >> > Etienne
> >> > Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste Onofré a écrit :
> >> >
> >> > Hi all,
> >> >
> >> > I did multiple gradle build since last week and I would like to share
> >> > one of my concern: it's about the communities.
> >> >
> >> > If I think our users won't see any change for them due to Gradle build
> >> > (I think that most of our users will still use Maven with artifacts
> >> > provided by Gradle), I'm more concerned by the dev community and the
> >> > contribution.
> >> >
> >> > Maven is well known and straight forward for a large part of potential
> >> > contributors. I think we have to keep in mind that we still have to
> grow
> >> > up our contributors community.
> >> >
> >> > Today, maybe I'm wrong, but I have the feeling that gradle build is
> not
> >> > straight forward (build.gradle includes build_rules.gradle, gathering
> >> > all taks all together).
> >> >
> >> > I would like to add a task in the gradle "migration" process: simplify
> >> > the gradle structure and files, and document this.
> >> >
> >> > I know we already have a Jira about the documentation part, but I
> would
> >> > like to "polish" and use a clean structure for the Gradle resources.
> As
> >> > already quickly discussed, I think that having one gradle file per
> tasks
> >> > in the .gradle directory would be helpful.
> >> >
> >> > The goal is really to simplify the contribution.
> >> >
> >> > Do you agree if I add a Jira about "Gradle polish" ?
> >> > Thoughts ?
> >> >
> >> > Regards
> >> > JB
> >> >
> >> > On 07/04/2018 04:52, Scott Wegner wrote:
> >> >
> >> > Here's an end-of-day update on migration work:
> >> >
> >> > * Snapshot unsigned dailies and signed release builds are working
> (!!).
> >> > PR/5048 [1] merges changes from Luke's branch
> >> >* python precommit failing... will investigate python precommit
> >> > Monday
> >> > * All Precommits are gradle only
> >> > * All Postcommits except performance tests and Java_JDK_Versions_Test
> >> > use gradle (after PR/5047 [2] merged)
> >> > * Nightly snapshot release using gradle is ready; needs PR/5048 to be
> >> > merged before switching
> >> > * ValidatesRunner_Spark failing consistently; investigating
> >> >
> >> > Thanks for another productive day of hacking. I'll pick up again on
> >> > Monday.
> >> >
> >> > [1] https://github.com/apache/beam/pull/5048
> >> > [2] https://github.com/apache/beam/pull/5047
> >> >
> >> >
> >> > On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau
> >> > > wrote:
> >> >
> >> > Why building a zip per runner which its stack and just pointing
> out
> >> > on that zip and let beam lazy load the runner:
> >> >
> >> > --runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=...
> 

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

You dont have issue due to the build setup with that option. I get:

avr. 10, 2018 3:20:10 PM
org.apache.beam.runners.direct.DirectTransformExecutor run
GRAVE: Error occurred within
org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a
com.google.common.util.concurrent.ExecutionError:
java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy

?

Romain Manni-Bucau
@rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book


2018-04-10 15:13 GMT+02:00 Lukasz Cwik :
> I have found that the simplest setup is to delegate the build/test actions
> to Gradle. This allows you to run unit tests very easily and since its in
> the same manner that Gradle would have, you know that if its passing it will
> pass on the command line and on Jenkins. Here is one site that discusses how
> to set this up:
> http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html
>
>
> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau 
> wrote:
>>
>> What's the plan to make idea supporting gradle on beam project? Do we
>> import the workaround mentionned in
>> https://youtrack.jetbrains.com/issue/IDEA-175172?
>> For the ones who didn't see this issue in action: idea will compile in
>> out/ instead of build/ and you will just miss all the resources you
>> need like some SPI registration which are used by all our registrar =>
>> no way to run tests in idea without hacking the configuration quite
>> deeply :(
>>
>> Romain Manni-Bucau
>> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>>
>>
>> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot :
>> > As a gradle beginner, I could not agree more !
>> > +1
>> > Etienne
>> > Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste Onofré a écrit :
>> >
>> > Hi all,
>> >
>> > I did multiple gradle build since last week and I would like to share
>> > one of my concern: it's about the communities.
>> >
>> > If I think our users won't see any change for them due to Gradle build
>> > (I think that most of our users will still use Maven with artifacts
>> > provided by Gradle), I'm more concerned by the dev community and the
>> > contribution.
>> >
>> > Maven is well known and straight forward for a large part of potential
>> > contributors. I think we have to keep in mind that we still have to grow
>> > up our contributors community.
>> >
>> > Today, maybe I'm wrong, but I have the feeling that gradle build is not
>> > straight forward (build.gradle includes build_rules.gradle, gathering
>> > all taks all together).
>> >
>> > I would like to add a task in the gradle "migration" process: simplify
>> > the gradle structure and files, and document this.
>> >
>> > I know we already have a Jira about the documentation part, but I would
>> > like to "polish" and use a clean structure for the Gradle resources. As
>> > already quickly discussed, I think that having one gradle file per tasks
>> > in the .gradle directory would be helpful.
>> >
>> > The goal is really to simplify the contribution.
>> >
>> > Do you agree if I add a Jira about "Gradle polish" ?
>> > Thoughts ?
>> >
>> > Regards
>> > JB
>> >
>> > On 07/04/2018 04:52, Scott Wegner wrote:
>> >
>> > Here's an end-of-day update on migration work:
>> >
>> > * Snapshot unsigned dailies and signed release builds are working (!!).
>> > PR/5048 [1] merges changes from Luke's branch
>> >* python precommit failing... will investigate python precommit
>> > Monday
>> > * All Precommits are gradle only
>> > * All Postcommits except performance tests and Java_JDK_Versions_Test
>> > use gradle (after PR/5047 [2] merged)
>> > * Nightly snapshot release using gradle is ready; needs PR/5048 to be
>> > merged before switching
>> > * ValidatesRunner_Spark failing consistently; investigating
>> >
>> > Thanks for another productive day of hacking. I'll pick up again on
>> > Monday.
>> >
>> > [1] https://github.com/apache/beam/pull/5048
>> > [2] https://github.com/apache/beam/pull/5047
>> >
>> >
>> > On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau
>> > > wrote:
>> >
>> > Why building a zip per runner which its stack and just pointing out
>> > on that zip and let beam lazy load the runner:
>> >
>> > --runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=... (or
>> > the fromSystemProperties() if it gets merged a day ;))
>> >
>> > Le 6 avr. 2018 20:21, "Kenneth Knowles" > > > a écrit :
>> >
>> > I'm working on finding a solution for launching the Nexmark
>> > suite with each runner. This doesn't have to be done via Gradle,
>> > but we anyhow need built artifacts that don't require user
>> > classpath intervention.
>> >
>> > It looks to me like the examples are also missing this - they
>> > have separate configuration e.g. sparkRunnerPreCommit but that
>> > is overspecified compared to a 

Re: Gradle Status [April 6]

[Lukasz Cwik]

I have found that the simplest setup is to delegate the build/test actions
to Gradle. This allows you to run unit tests very easily and since its in
the same manner that Gradle would have, you know that if its passing it
will pass on the command line and on Jenkins. Here is one site that
discusses how to set this up:
http://mrhaki.blogspot.com/2016/11/gradle-goodness-delegate-build-and-run.html


On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau 
wrote:

> What's the plan to make idea supporting gradle on beam project? Do we
> import the workaround mentionned in
> https://youtrack.jetbrains.com/issue/IDEA-175172?
> For the ones who didn't see this issue in action: idea will compile in
> out/ instead of build/ and you will just miss all the resources you
> need like some SPI registration which are used by all our registrar =>
> no way to run tests in idea without hacking the configuration quite
> deeply :(
>
> Romain Manni-Bucau
> @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book
>
>
> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot :
> > As a gradle beginner, I could not agree more !
> > +1
> > Etienne
> > Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste Onofré a écrit :
> >
> > Hi all,
> >
> > I did multiple gradle build since last week and I would like to share
> > one of my concern: it's about the communities.
> >
> > If I think our users won't see any change for them due to Gradle build
> > (I think that most of our users will still use Maven with artifacts
> > provided by Gradle), I'm more concerned by the dev community and the
> > contribution.
> >
> > Maven is well known and straight forward for a large part of potential
> > contributors. I think we have to keep in mind that we still have to grow
> > up our contributors community.
> >
> > Today, maybe I'm wrong, but I have the feeling that gradle build is not
> > straight forward (build.gradle includes build_rules.gradle, gathering
> > all taks all together).
> >
> > I would like to add a task in the gradle "migration" process: simplify
> > the gradle structure and files, and document this.
> >
> > I know we already have a Jira about the documentation part, but I would
> > like to "polish" and use a clean structure for the Gradle resources. As
> > already quickly discussed, I think that having one gradle file per tasks
> > in the .gradle directory would be helpful.
> >
> > The goal is really to simplify the contribution.
> >
> > Do you agree if I add a Jira about "Gradle polish" ?
> > Thoughts ?
> >
> > Regards
> > JB
> >
> > On 07/04/2018 04:52, Scott Wegner wrote:
> >
> > Here's an end-of-day update on migration work:
> >
> > * Snapshot unsigned dailies and signed release builds are working (!!).
> > PR/5048 [1] merges changes from Luke's branch
> >* python precommit failing... will investigate python precommit Monday
> > * All Precommits are gradle only
> > * All Postcommits except performance tests and Java_JDK_Versions_Test
> > use gradle (after PR/5047 [2] merged)
> > * Nightly snapshot release using gradle is ready; needs PR/5048 to be
> > merged before switching
> > * ValidatesRunner_Spark failing consistently; investigating
> >
> > Thanks for another productive day of hacking. I'll pick up again on
> Monday.
> >
> > [1] https://github.com/apache/beam/pull/5048
> > [2] https://github.com/apache/beam/pull/5047
> >
> >
> > On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau
> > > wrote:
> >
> > Why building a zip per runner which its stack and just pointing out
> > on that zip and let beam lazy load the runner:
> >
> > --runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=... (or
> > the fromSystemProperties() if it gets merged a day ;))
> >
> > Le 6 avr. 2018 20:21, "Kenneth Knowles"  > > a écrit :
> >
> > I'm working on finding a solution for launching the Nexmark
> > suite with each runner. This doesn't have to be done via Gradle,
> > but we anyhow need built artifacts that don't require user
> > classpath intervention.
> >
> > It looks to me like the examples are also missing this - they
> > have separate configuration e.g. sparkRunnerPreCommit but that
> > is overspecified compared to a free-form launching of a main()
> > program with a runner profile.
> >
> > On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik  > > wrote:
> >
> > Romain, are you talking about the profiles that exist as
> > part of the archetype examples?
> >
> > If so, then those still exist and haven't been changed. If
> > not, can you provide a link to the profile in a pom file to
> > be clearer?
> >
> > On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau
> > 

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

What's the plan to make idea supporting gradle on beam project? Do we
import the workaround mentionned in
https://youtrack.jetbrains.com/issue/IDEA-175172?
For the ones who didn't see this issue in action: idea will compile in
out/ instead of build/ and you will just miss all the resources you
need like some SPI registration which are used by all our registrar =>
no way to run tests in idea without hacking the configuration quite
deeply :(

Romain Manni-Bucau
@rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book


2018-04-10 10:08 GMT+02:00 Etienne Chauchot :
> As a gradle beginner, I could not agree more !
> +1
> Etienne
> Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste Onofré a écrit :
>
> Hi all,
>
> I did multiple gradle build since last week and I would like to share
> one of my concern: it's about the communities.
>
> If I think our users won't see any change for them due to Gradle build
> (I think that most of our users will still use Maven with artifacts
> provided by Gradle), I'm more concerned by the dev community and the
> contribution.
>
> Maven is well known and straight forward for a large part of potential
> contributors. I think we have to keep in mind that we still have to grow
> up our contributors community.
>
> Today, maybe I'm wrong, but I have the feeling that gradle build is not
> straight forward (build.gradle includes build_rules.gradle, gathering
> all taks all together).
>
> I would like to add a task in the gradle "migration" process: simplify
> the gradle structure and files, and document this.
>
> I know we already have a Jira about the documentation part, but I would
> like to "polish" and use a clean structure for the Gradle resources. As
> already quickly discussed, I think that having one gradle file per tasks
> in the .gradle directory would be helpful.
>
> The goal is really to simplify the contribution.
>
> Do you agree if I add a Jira about "Gradle polish" ?
> Thoughts ?
>
> Regards
> JB
>
> On 07/04/2018 04:52, Scott Wegner wrote:
>
> Here's an end-of-day update on migration work:
>
> * Snapshot unsigned dailies and signed release builds are working (!!).
> PR/5048 [1] merges changes from Luke's branch
>* python precommit failing... will investigate python precommit Monday
> * All Precommits are gradle only
> * All Postcommits except performance tests and Java_JDK_Versions_Test
> use gradle (after PR/5047 [2] merged)
> * Nightly snapshot release using gradle is ready; needs PR/5048 to be
> merged before switching
> * ValidatesRunner_Spark failing consistently; investigating
>
> Thanks for another productive day of hacking. I'll pick up again on Monday.
>
> [1] https://github.com/apache/beam/pull/5048
> [2] https://github.com/apache/beam/pull/5047
>
>
> On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau
> > wrote:
>
> Why building a zip per runner which its stack and just pointing out
> on that zip and let beam lazy load the runner:
>
> --runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=... (or
> the fromSystemProperties() if it gets merged a day ;))
>
> Le 6 avr. 2018 20:21, "Kenneth Knowles"  > a écrit :
>
> I'm working on finding a solution for launching the Nexmark
> suite with each runner. This doesn't have to be done via Gradle,
> but we anyhow need built artifacts that don't require user
> classpath intervention.
>
> It looks to me like the examples are also missing this - they
> have separate configuration e.g. sparkRunnerPreCommit but that
> is overspecified compared to a free-form launching of a main()
> program with a runner profile.
>
> On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik  > wrote:
>
> Romain, are you talking about the profiles that exist as
> part of the archetype examples?
>
> If so, then those still exist and haven't been changed. If
> not, can you provide a link to the profile in a pom file to
> be clearer?
>
> On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau
> > wrote:
>
> Hi Scott,
>
> is it right that 2 doesn't handle the hierachy anymore
> and that it doesn't handle profiles for runners as it is
> currently with maven?
>
>
> Romain Manni-Bucau
> @rmannibucau  | Blog
>  | Old Blog
>  | Github
>  | LinkedIn
>  | Book
>
> 

Re: Gradle Status [April 6]

[Etienne Chauchot]

As a gradle beginner, I could not agree more ! 
+1
Etienne
Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste Onofré a écrit :
> Hi all,
> 
> I did multiple gradle build since last week and I would like to share 
> one of my concern: it's about the communities.
> 
> If I think our users won't see any change for them due to Gradle build 
> (I think that most of our users will still use Maven with artifacts 
> provided by Gradle), I'm more concerned by the dev community and the 
> contribution.
> 
> Maven is well known and straight forward for a large part of potential 
> contributors. I think we have to keep in mind that we still have to grow 
> up our contributors community.
> 
> Today, maybe I'm wrong, but I have the feeling that gradle build is not 
> straight forward (build.gradle includes build_rules.gradle, gathering 
> all taks all together).
> 
> I would like to add a task in the gradle "migration" process: simplify 
> the gradle structure and files, and document this.
> 
> I know we already have a Jira about the documentation part, but I would 
> like to "polish" and use a clean structure for the Gradle resources. As 
> already quickly discussed, I think that having one gradle file per tasks 
> in the .gradle directory would be helpful.
> 
> The goal is really to simplify the contribution.
> 
> Do you agree if I add a Jira about "Gradle polish" ?
> Thoughts ?
> 
> Regards
> JB
> 
> On 07/04/2018 04:52, Scott Wegner wrote:
> > 
> > Here's an end-of-day update on migration work:
> > 
> > * Snapshot unsigned dailies and signed release builds are working (!!). 
> > PR/5048 [1] merges changes from Luke's branch
> >    * python precommit failing... will investigate python precommit Monday
> > * All Precommits are gradle only
> > * All Postcommits except performance tests and Java_JDK_Versions_Test  
> > use gradle (after PR/5047 [2] merged)
> > * Nightly snapshot release using gradle is ready; needs PR/5048 to be 
> > merged before switching
> > * ValidatesRunner_Spark failing consistently; investigating
> > 
> > Thanks for another productive day of hacking. I'll pick up again on Monday.
> > 
> > [1] https://github.com/apache/beam/pull/5048
> > [2] https://github.com/apache/beam/pull/5047
> > 
> > 
> > On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau 
> > > wrote:
> > 
> > Why building a zip per runner which its stack and just pointing out
> > on that zip and let beam lazy load the runner:
> > 
> > --runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=... (or
> > the fromSystemProperties() if it gets merged a day ;))
> > 
> > Le 6 avr. 2018 20:21, "Kenneth Knowles"  > k...@google.com>> a écrit :
> > 
> > I'm working on finding a solution for launching the Nexmark
> > suite with each runner. This doesn't have to be done via Gradle,
> > but we anyhow need built artifacts that don't require user
> > classpath intervention.
> > 
> > It looks to me like the examples are also missing this - they
> > have separate configuration e.g. sparkRunnerPreCommit but that
> > is overspecified compared to a free-form launching of a main()
> > program with a runner profile.
> > 
> > On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik  > lc...@google.com>> wrote:
> > 
> > Romain, are you talking about the profiles that exist as
> > part of the archetype examples?
> > 
> > If so, then those still exist and haven't been changed. If
> > not, can you provide a link to the profile in a pom file to
> > be clearer?
> > 
> > On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau
> > > wrote:
> > 
> > Hi Scott,
> > 
> > is it right that 2 doesn't handle the hierachy anymore
> > and that it doesn't handle profiles for runners as it is
> > currently with maven?
> > 
> > 
> > Romain Manni-Bucau
> > @rmannibucau  | Blog
> >  | Old Blog
> >  | Github
> >  | LinkedIn
> >  | Book
> > 
> > 
> > 
> > 2018-04-06 18:32 GMT+02:00 Scott Wegner
> > >:
> > 
> > I wanted to start a thread to summarize the current
> > state of Gradle migration. We've made lots of good
> > progress so far this week. Here's the status from
> > what I can tell-- please add or correct anything I

Re: Gradle Status [April 6]

[Lukasz Cwik]

Simplifying examples/java precommit is encompassed in BEAM-4033. Both you
and Kenn pointed to the same thing. To my knowledge it is the only place
where we have such a loop that generates tasks.

On Mon, Apr 9, 2018 at 3:37 PM Romain Manni-Bucau 
wrote:

>
>
> Le 9 avr. 2018 21:04, "Lukasz Cwik"  a écrit :
>
> Romain, can you clarify by the weird task naming? (Give some examples
> using our current project and Gradle and what you would have expected.)
>
>
> Sure
>
>
> https://github.com/apache/beam/blob/fb6ba3bfd43605a0f4944828b5f19e75840fa7aa/examples/java/build.gradle#L78
> for instance
>
> It is way more natural to have one generic command you can select a runner
> (including a csv value) than N commands for this kind of things.
>
> Gradle allows to define custom tasks and "scopes". This must not be used
> at Xtrem until justifiable. We now have a lot of configurations not making
> much sense too until you scripted yourself the script or know beam well.
> This doesnt enable contribs :(.
>
> Hope it is clearer
>
>
> On Mon, Apr 9, 2018 at 2:49 PM Romain Manni-Bucau 
> wrote:
>
>> +1 gradle not being mainstream - and even more in beam scope than other
>> java scope - it must stay simple. I know that personally it takes me x2 or
>> 3 to say "ok ill fork" a project using gradle, in particular with custom
>> scripts and not only parent/module scripts.
>>
>> If needed we can write custom plugins to simply it and hide the
>> complexity and weird task naming can move to "standard" names with
>> options/parameters maybe?
>>
>> Le 9 avr. 2018 20:27, "Kenneth Knowles"  a écrit :
>>
>>> My phrasing sounded a bit too much like adding a blocking condition. I
>>> absolutely don't want to do that. Our maven build was far from clear, and
>>> had lots of tech debt and spooky action at a distance,
>>>
>>> I just want to emphatically agree with JB's sentiment that we have an
>>> opportunity to improve it before it ossifies in the current state. Maybe we
>>> won't really have time, and that's OK too and we can just improve as we go.
>>>
>>> Luke - I know we have different standards of clarity but I doubt we'll
>>> have specific disagreements on cleanups to the build as they happen.* I was
>>> mostly listing elementary programming concepts that apply to our build and
>>> tend to get forgotten in configs & build files. Being "natural gradle" is
>>> also important. Let's see how it goes.
>>>
>>> Kenn
>>>
>>> *FWIW I did know that we have some string-munged identifiers already,
>>> but mostly I am speaking from experience beyond & before Beam too. I'll
>>> likely tidy the examples build.gradle as I finish up nexmark in a similar
>>> way.
>>>
>>> On Mon, Apr 9, 2018 at 10:50 AM Lukasz Cwik  wrote:
>>>
 JB, learning the build system in a project is hopefully avoided by most
 users if the contribution guide can clearly explain what users need to do.
 But for everyone else who wants to change a dependency version or add a
 dependency it should be as simple as copy/paste (which I believe it already
 is). Note that people who want to do anything more complicated like add new
 jenkins job configurations for new integration test targets needs to learn
 how the build system works, how maven profiles work, how to plumb the
 required set of attributes from Jenkins into the test target via the build
 system.

 Kenn, I have to disagree with a lot in 1 and 2. For users who are
 unfamiliar with Maven, we didn't setup the Maven build in such a way that
 anyone unfamiliar with Maven knew what was going on or try to use concepts
 unnatural to Maven in an attempt to make it seem easier. I believe we
 should stick with Gradle/Groovy conventions. Users who are not familiar
 with how Gradle/Groovy works will either ask the community or look at
 StackOverflow and doing things like passing the project around into
 functions is extremely uncommon compared to using the current context and
 applying closures to it. For users who are familiar with Gradle, the builds
 should be natural Gradle.

 *- If you are going to refer to an identifier, it should have an
 explicit point of origin (not strings smashed together in a loop)*
 I assume your referring to how the examples java precommit is done. It
 is the only case of it to my knowledge and extra hands to migrate it to be
 like how the validates runner tests run would be useful. Filed
 https://issues.apache.org/jira/browse/BEAM-4033


 On Mon, Apr 9, 2018 at 1:24 PM Kenneth Knowles  wrote:

> Huge +1 to the idea of investing in simplification and polish of the
> Gradle files before considering the migration complete.
>
> 1. build.gradle files should be as close to straight-line
> configuration as possible:
>  - You should be able to understand a 

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

Le 9 avr. 2018 21:04, "Lukasz Cwik"  a écrit :

Romain, can you clarify by the weird task naming? (Give some examples using
our current project and Gradle and what you would have expected.)


Sure

https://github.com/apache/beam/blob/fb6ba3bfd43605a0f4944828b5f19e75840fa7aa/examples/java/build.gradle#L78
for instance

It is way more natural to have one generic command you can select a runner
(including a csv value) than N commands for this kind of things.

Gradle allows to define custom tasks and "scopes". This must not be used at
Xtrem until justifiable. We now have a lot of configurations not making
much sense too until you scripted yourself the script or know beam well.
This doesnt enable contribs :(.

Hope it is clearer


On Mon, Apr 9, 2018 at 2:49 PM Romain Manni-Bucau 
wrote:

> +1 gradle not being mainstream - and even more in beam scope than other
> java scope - it must stay simple. I know that personally it takes me x2 or
> 3 to say "ok ill fork" a project using gradle, in particular with custom
> scripts and not only parent/module scripts.
>
> If needed we can write custom plugins to simply it and hide the complexity
> and weird task naming can move to "standard" names with options/parameters
> maybe?
>
> Le 9 avr. 2018 20:27, "Kenneth Knowles"  a écrit :
>
>> My phrasing sounded a bit too much like adding a blocking condition. I
>> absolutely don't want to do that. Our maven build was far from clear, and
>> had lots of tech debt and spooky action at a distance,
>>
>> I just want to emphatically agree with JB's sentiment that we have an
>> opportunity to improve it before it ossifies in the current state. Maybe we
>> won't really have time, and that's OK too and we can just improve as we go.
>>
>> Luke - I know we have different standards of clarity but I doubt we'll
>> have specific disagreements on cleanups to the build as they happen.* I was
>> mostly listing elementary programming concepts that apply to our build and
>> tend to get forgotten in configs & build files. Being "natural gradle" is
>> also important. Let's see how it goes.
>>
>> Kenn
>>
>> *FWIW I did know that we have some string-munged identifiers already, but
>> mostly I am speaking from experience beyond & before Beam too. I'll likely
>> tidy the examples build.gradle as I finish up nexmark in a similar way.
>>
>> On Mon, Apr 9, 2018 at 10:50 AM Lukasz Cwik  wrote:
>>
>>> JB, learning the build system in a project is hopefully avoided by most
>>> users if the contribution guide can clearly explain what users need to do.
>>> But for everyone else who wants to change a dependency version or add a
>>> dependency it should be as simple as copy/paste (which I believe it already
>>> is). Note that people who want to do anything more complicated like add new
>>> jenkins job configurations for new integration test targets needs to learn
>>> how the build system works, how maven profiles work, how to plumb the
>>> required set of attributes from Jenkins into the test target via the build
>>> system.
>>>
>>> Kenn, I have to disagree with a lot in 1 and 2. For users who are
>>> unfamiliar with Maven, we didn't setup the Maven build in such a way that
>>> anyone unfamiliar with Maven knew what was going on or try to use concepts
>>> unnatural to Maven in an attempt to make it seem easier. I believe we
>>> should stick with Gradle/Groovy conventions. Users who are not familiar
>>> with how Gradle/Groovy works will either ask the community or look at
>>> StackOverflow and doing things like passing the project around into
>>> functions is extremely uncommon compared to using the current context and
>>> applying closures to it. For users who are familiar with Gradle, the builds
>>> should be natural Gradle.
>>>
>>> *- If you are going to refer to an identifier, it should have an
>>> explicit point of origin (not strings smashed together in a loop)*
>>> I assume your referring to how the examples java precommit is done. It
>>> is the only case of it to my knowledge and extra hands to migrate it to be
>>> like how the validates runner tests run would be useful. Filed
>>> https://issues.apache.org/jira/browse/BEAM-4033
>>>
>>>
>>> On Mon, Apr 9, 2018 at 1:24 PM Kenneth Knowles  wrote:
>>>
 Huge +1 to the idea of investing in simplification and polish of the
 Gradle files before considering the migration complete.

 1. build.gradle files should be as close to straight-line configuration
 as possible:
  - You should be able to understand a module's build easily, locally,
 without knowing Groovy
  - As little Groovy programming as possible, focused on providing
 well-defined utilities

 2. Explicit > implicit, especially for config files
  - Ditto about being able to understand a module's build locally
  - Minimize globally inherited config, in favor of explicitly calling
 utilities
  - Passing 

Re: Gradle Status [April 6]

[Jean-Baptiste Onofré]

Hi Luke,

let me take a concrete example.

I'm developing a Beam extension. Most of the time, I will ask myself:

1. How do I do the build.gradle and include my extension in Beam project
2. How do my extension is compile (the equivalent of 
maven-compiler-plugin) and how to deal with dependency and scope

3. How my extension is packaged
4. How my test are executed

Of course, we can document this when we are on the happy path.
Now, imagine, dependency is not correctly resolved, rat is failing, 
findbugs is complaining, etc. I have to check the "core" Beam gradle rules.
Today, I think it's not easy at all to find the corresponding tasks and 
configuration.


I saw several projects storing the tasks in .gradle (like Apache Geode 
for instance).


My point is that I think it's a right timing to polish/cleanup our 
build. Else, I'm afraid we will never do that in the future.


Regards
JB

On 09/04/2018 19:50, Lukasz Cwik wrote:
JB, learning the build system in a project is hopefully avoided by most 
users if the contribution guide can clearly explain what users need to 
do. But for everyone else who wants to change a dependency version or 
add a dependency it should be as simple as copy/paste (which I believe 
it already is). Note that people who want to do anything more 
complicated like add new jenkins job configurations for new integration 
test targets needs to learn how the build system works, how maven 
profiles work, how to plumb the required set of attributes from Jenkins 
into the test target via the build system.


Kenn, I have to disagree with a lot in 1 and 2. For users who are 
unfamiliar with Maven, we didn't setup the Maven build in such a way 
that anyone unfamiliar with Maven knew what was going on or try to use 
concepts unnatural to Maven in an attempt to make it seem easier. I 
believe we should stick with Gradle/Groovy conventions. Users who are 
not familiar with how Gradle/Groovy works will either ask the community 
or look at StackOverflow and doing things like passing the project 
around into functions is extremely uncommon compared to using the 
current context and applying closures to it. For users who are familiar 
with Gradle, the builds should be natural Gradle.


*- If you are going to refer to an identifier, it should have an 
explicit point of origin (not strings smashed together in a loop)*
I assume your referring to how the examples java precommit is done. It 
is the only case of it to my knowledge and extra hands to migrate it to 
be like how the validates runner tests run would be useful. Filed 
https://issues.apache.org/jira/browse/BEAM-4033



On Mon, Apr 9, 2018 at 1:24 PM Kenneth Knowles > wrote:


Huge +1 to the idea of investing in simplification and polish of the
Gradle files before considering the migration complete.

1. build.gradle files should be as close to straight-line
configuration as possible:
  - You should be able to understand a module's build easily,
locally, without knowing Groovy
  - As little Groovy programming as possible, focused on providing
well-defined utilities

2. Explicit > implicit, especially for config files
  - Ditto about being able to understand a module's build locally
  - Minimize globally inherited config, in favor of explicitly
calling utilities
  - Passing current project to a utility is better than trying to
make something the "closure"
  - If you are going to refer to an identifier, it should have an
explicit point of origin (not strings smashed together in a loop)

When in doubt, a little redundancy to improve readability is worth
it in this context.

Kenn

On Mon, Apr 9, 2018 at 9:47 AM Jean-Baptiste Onofré > wrote:

Hi all,

I did multiple gradle build since last week and I would like to
share
one of my concern: it's about the communities.

If I think our users won't see any change for them due to Gradle
build
(I think that most of our users will still use Maven with artifacts
provided by Gradle), I'm more concerned by the dev community and the
contribution.

Maven is well known and straight forward for a large part of
potential
contributors. I think we have to keep in mind that we still have
to grow
up our contributors community.

Today, maybe I'm wrong, but I have the feeling that gradle build
is not
straight forward (build.gradle includes build_rules.gradle,
gathering
all taks all together).

I would like to add a task in the gradle "migration" process:
simplify
the gradle structure and files, and document this.

I know we already have a Jira about the documentation part, but
I would
like to "polish" and use a clean structure for the Gradle
resources. 

Re: Gradle Status [April 6]

[Lukasz Cwik]

Romain, can you clarify by the weird task naming? (Give some examples using
our current project and Gradle and what you would have expected.)

On Mon, Apr 9, 2018 at 2:49 PM Romain Manni-Bucau 
wrote:

> +1 gradle not being mainstream - and even more in beam scope than other
> java scope - it must stay simple. I know that personally it takes me x2 or
> 3 to say "ok ill fork" a project using gradle, in particular with custom
> scripts and not only parent/module scripts.
>
> If needed we can write custom plugins to simply it and hide the complexity
> and weird task naming can move to "standard" names with options/parameters
> maybe?
>
> Le 9 avr. 2018 20:27, "Kenneth Knowles"  a écrit :
>
>> My phrasing sounded a bit too much like adding a blocking condition. I
>> absolutely don't want to do that. Our maven build was far from clear, and
>> had lots of tech debt and spooky action at a distance,
>>
>> I just want to emphatically agree with JB's sentiment that we have an
>> opportunity to improve it before it ossifies in the current state. Maybe we
>> won't really have time, and that's OK too and we can just improve as we go.
>>
>> Luke - I know we have different standards of clarity but I doubt we'll
>> have specific disagreements on cleanups to the build as they happen.* I was
>> mostly listing elementary programming concepts that apply to our build and
>> tend to get forgotten in configs & build files. Being "natural gradle" is
>> also important. Let's see how it goes.
>>
>> Kenn
>>
>> *FWIW I did know that we have some string-munged identifiers already, but
>> mostly I am speaking from experience beyond & before Beam too. I'll likely
>> tidy the examples build.gradle as I finish up nexmark in a similar way.
>>
>> On Mon, Apr 9, 2018 at 10:50 AM Lukasz Cwik  wrote:
>>
>>> JB, learning the build system in a project is hopefully avoided by most
>>> users if the contribution guide can clearly explain what users need to do.
>>> But for everyone else who wants to change a dependency version or add a
>>> dependency it should be as simple as copy/paste (which I believe it already
>>> is). Note that people who want to do anything more complicated like add new
>>> jenkins job configurations for new integration test targets needs to learn
>>> how the build system works, how maven profiles work, how to plumb the
>>> required set of attributes from Jenkins into the test target via the build
>>> system.
>>>
>>> Kenn, I have to disagree with a lot in 1 and 2. For users who are
>>> unfamiliar with Maven, we didn't setup the Maven build in such a way that
>>> anyone unfamiliar with Maven knew what was going on or try to use concepts
>>> unnatural to Maven in an attempt to make it seem easier. I believe we
>>> should stick with Gradle/Groovy conventions. Users who are not familiar
>>> with how Gradle/Groovy works will either ask the community or look at
>>> StackOverflow and doing things like passing the project around into
>>> functions is extremely uncommon compared to using the current context and
>>> applying closures to it. For users who are familiar with Gradle, the builds
>>> should be natural Gradle.
>>>
>>> *- If you are going to refer to an identifier, it should have an
>>> explicit point of origin (not strings smashed together in a loop)*
>>> I assume your referring to how the examples java precommit is done. It
>>> is the only case of it to my knowledge and extra hands to migrate it to be
>>> like how the validates runner tests run would be useful. Filed
>>> https://issues.apache.org/jira/browse/BEAM-4033
>>>
>>>
>>> On Mon, Apr 9, 2018 at 1:24 PM Kenneth Knowles  wrote:
>>>
 Huge +1 to the idea of investing in simplification and polish of the
 Gradle files before considering the migration complete.

 1. build.gradle files should be as close to straight-line configuration
 as possible:
  - You should be able to understand a module's build easily, locally,
 without knowing Groovy
  - As little Groovy programming as possible, focused on providing
 well-defined utilities

 2. Explicit > implicit, especially for config files
  - Ditto about being able to understand a module's build locally
  - Minimize globally inherited config, in favor of explicitly calling
 utilities
  - Passing current project to a utility is better than trying to make
 something the "closure"
  - If you are going to refer to an identifier, it should have an
 explicit point of origin (not strings smashed together in a loop)

 When in doubt, a little redundancy to improve readability is worth it
 in this context.

 Kenn

 On Mon, Apr 9, 2018 at 9:47 AM Jean-Baptiste Onofré 
 wrote:

> Hi all,
>
> I did multiple gradle build since last week and I would like to share
> one of my concern: it's about the communities.
>
> If I 

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

+1 gradle not being mainstream - and even more in beam scope than other
java scope - it must stay simple. I know that personally it takes me x2 or
3 to say "ok ill fork" a project using gradle, in particular with custom
scripts and not only parent/module scripts.

If needed we can write custom plugins to simply it and hide the complexity
and weird task naming can move to "standard" names with options/parameters
maybe?

Le 9 avr. 2018 20:27, "Kenneth Knowles"  a écrit :

> My phrasing sounded a bit too much like adding a blocking condition. I
> absolutely don't want to do that. Our maven build was far from clear, and
> had lots of tech debt and spooky action at a distance,
>
> I just want to emphatically agree with JB's sentiment that we have an
> opportunity to improve it before it ossifies in the current state. Maybe we
> won't really have time, and that's OK too and we can just improve as we go.
>
> Luke - I know we have different standards of clarity but I doubt we'll
> have specific disagreements on cleanups to the build as they happen.* I was
> mostly listing elementary programming concepts that apply to our build and
> tend to get forgotten in configs & build files. Being "natural gradle" is
> also important. Let's see how it goes.
>
> Kenn
>
> *FWIW I did know that we have some string-munged identifiers already, but
> mostly I am speaking from experience beyond & before Beam too. I'll likely
> tidy the examples build.gradle as I finish up nexmark in a similar way.
>
> On Mon, Apr 9, 2018 at 10:50 AM Lukasz Cwik  wrote:
>
>> JB, learning the build system in a project is hopefully avoided by most
>> users if the contribution guide can clearly explain what users need to do.
>> But for everyone else who wants to change a dependency version or add a
>> dependency it should be as simple as copy/paste (which I believe it already
>> is). Note that people who want to do anything more complicated like add new
>> jenkins job configurations for new integration test targets needs to learn
>> how the build system works, how maven profiles work, how to plumb the
>> required set of attributes from Jenkins into the test target via the build
>> system.
>>
>> Kenn, I have to disagree with a lot in 1 and 2. For users who are
>> unfamiliar with Maven, we didn't setup the Maven build in such a way that
>> anyone unfamiliar with Maven knew what was going on or try to use concepts
>> unnatural to Maven in an attempt to make it seem easier. I believe we
>> should stick with Gradle/Groovy conventions. Users who are not familiar
>> with how Gradle/Groovy works will either ask the community or look at
>> StackOverflow and doing things like passing the project around into
>> functions is extremely uncommon compared to using the current context and
>> applying closures to it. For users who are familiar with Gradle, the builds
>> should be natural Gradle.
>>
>> *- If you are going to refer to an identifier, it should have an explicit
>> point of origin (not strings smashed together in a loop)*
>> I assume your referring to how the examples java precommit is done. It is
>> the only case of it to my knowledge and extra hands to migrate it to be
>> like how the validates runner tests run would be useful. Filed
>> https://issues.apache.org/jira/browse/BEAM-4033
>>
>>
>> On Mon, Apr 9, 2018 at 1:24 PM Kenneth Knowles  wrote:
>>
>>> Huge +1 to the idea of investing in simplification and polish of the
>>> Gradle files before considering the migration complete.
>>>
>>> 1. build.gradle files should be as close to straight-line configuration
>>> as possible:
>>>  - You should be able to understand a module's build easily, locally,
>>> without knowing Groovy
>>>  - As little Groovy programming as possible, focused on providing
>>> well-defined utilities
>>>
>>> 2. Explicit > implicit, especially for config files
>>>  - Ditto about being able to understand a module's build locally
>>>  - Minimize globally inherited config, in favor of explicitly calling
>>> utilities
>>>  - Passing current project to a utility is better than trying to make
>>> something the "closure"
>>>  - If you are going to refer to an identifier, it should have an
>>> explicit point of origin (not strings smashed together in a loop)
>>>
>>> When in doubt, a little redundancy to improve readability is worth it in
>>> this context.
>>>
>>> Kenn
>>>
>>> On Mon, Apr 9, 2018 at 9:47 AM Jean-Baptiste Onofré 
>>> wrote:
>>>
 Hi all,

 I did multiple gradle build since last week and I would like to share
 one of my concern: it's about the communities.

 If I think our users won't see any change for them due to Gradle build
 (I think that most of our users will still use Maven with artifacts
 provided by Gradle), I'm more concerned by the dev community and the
 contribution.

 Maven is well known and straight forward for a large part of potential
 contributors. 

Re: Gradle Status [April 6]

[Kenneth Knowles]

My phrasing sounded a bit too much like adding a blocking condition. I
absolutely don't want to do that. Our maven build was far from clear, and
had lots of tech debt and spooky action at a distance,

I just want to emphatically agree with JB's sentiment that we have an
opportunity to improve it before it ossifies in the current state. Maybe we
won't really have time, and that's OK too and we can just improve as we go.

Luke - I know we have different standards of clarity but I doubt we'll have
specific disagreements on cleanups to the build as they happen.* I was
mostly listing elementary programming concepts that apply to our build and
tend to get forgotten in configs & build files. Being "natural gradle" is
also important. Let's see how it goes.

Kenn

*FWIW I did know that we have some string-munged identifiers already, but
mostly I am speaking from experience beyond & before Beam too. I'll likely
tidy the examples build.gradle as I finish up nexmark in a similar way.

On Mon, Apr 9, 2018 at 10:50 AM Lukasz Cwik  wrote:

> JB, learning the build system in a project is hopefully avoided by most
> users if the contribution guide can clearly explain what users need to do.
> But for everyone else who wants to change a dependency version or add a
> dependency it should be as simple as copy/paste (which I believe it already
> is). Note that people who want to do anything more complicated like add new
> jenkins job configurations for new integration test targets needs to learn
> how the build system works, how maven profiles work, how to plumb the
> required set of attributes from Jenkins into the test target via the build
> system.
>
> Kenn, I have to disagree with a lot in 1 and 2. For users who are
> unfamiliar with Maven, we didn't setup the Maven build in such a way that
> anyone unfamiliar with Maven knew what was going on or try to use concepts
> unnatural to Maven in an attempt to make it seem easier. I believe we
> should stick with Gradle/Groovy conventions. Users who are not familiar
> with how Gradle/Groovy works will either ask the community or look at
> StackOverflow and doing things like passing the project around into
> functions is extremely uncommon compared to using the current context and
> applying closures to it. For users who are familiar with Gradle, the builds
> should be natural Gradle.
>
> *- If you are going to refer to an identifier, it should have an explicit
> point of origin (not strings smashed together in a loop)*
> I assume your referring to how the examples java precommit is done. It is
> the only case of it to my knowledge and extra hands to migrate it to be
> like how the validates runner tests run would be useful. Filed
> https://issues.apache.org/jira/browse/BEAM-4033
>
>
> On Mon, Apr 9, 2018 at 1:24 PM Kenneth Knowles  wrote:
>
>> Huge +1 to the idea of investing in simplification and polish of the
>> Gradle files before considering the migration complete.
>>
>> 1. build.gradle files should be as close to straight-line configuration
>> as possible:
>>  - You should be able to understand a module's build easily, locally,
>> without knowing Groovy
>>  - As little Groovy programming as possible, focused on providing
>> well-defined utilities
>>
>> 2. Explicit > implicit, especially for config files
>>  - Ditto about being able to understand a module's build locally
>>  - Minimize globally inherited config, in favor of explicitly calling
>> utilities
>>  - Passing current project to a utility is better than trying to make
>> something the "closure"
>>  - If you are going to refer to an identifier, it should have an explicit
>> point of origin (not strings smashed together in a loop)
>>
>> When in doubt, a little redundancy to improve readability is worth it in
>> this context.
>>
>> Kenn
>>
>> On Mon, Apr 9, 2018 at 9:47 AM Jean-Baptiste Onofré 
>> wrote:
>>
>>> Hi all,
>>>
>>> I did multiple gradle build since last week and I would like to share
>>> one of my concern: it's about the communities.
>>>
>>> If I think our users won't see any change for them due to Gradle build
>>> (I think that most of our users will still use Maven with artifacts
>>> provided by Gradle), I'm more concerned by the dev community and the
>>> contribution.
>>>
>>> Maven is well known and straight forward for a large part of potential
>>> contributors. I think we have to keep in mind that we still have to grow
>>> up our contributors community.
>>>
>>> Today, maybe I'm wrong, but I have the feeling that gradle build is not
>>> straight forward (build.gradle includes build_rules.gradle, gathering
>>> all taks all together).
>>>
>>> I would like to add a task in the gradle "migration" process: simplify
>>> the gradle structure and files, and document this.
>>>
>>> I know we already have a Jira about the documentation part, but I would
>>> like to "polish" and use a clean structure for the Gradle resources. As
>>> already quickly discussed, I 

Re: Gradle Status [April 6]

[Lukasz Cwik]

Fixed up prior e-mail.

On Mon, Apr 9, 2018 at 1:50 PM Lukasz Cwik  wrote:

> JB, learning the build system in a project is hopefully avoided by most
> users if the contribution guide can clearly explain what users need to do.
> But for everyone else who wants to change a dependency version or add a
> dependency it should be as simple as copy/paste (which I believe it already
> is). Note that people who want to do anything more complicated like add new
> jenkins job configurations for new integration test targets needs to learn
> how the build system works, how to plumb the required set of attributes
> from Jenkins into the test target via the build system.
>
> Kenn, I have to disagree with a lot in 1 and 2. For users who are
> unfamiliar with Maven, we didn't setup the Maven build in such a way that
> anyone unfamiliar with Maven knew what was going on or try to use concepts
> unnatural to Maven in an attempt to make it seem easier. I believe we
> should stick with Gradle/Groovy conventions. Users who are not familiar
> with how Gradle/Groovy works will either ask the community or look at
> StackOverflow and doing things like passing the project around into
> functions is extremely uncommon compared to using the current context and
> applying closures to it. For users who are familiar with Gradle, the builds
> should be natural Gradle.
>
> *- If you are going to refer to an identifier, it should have an explicit
> point of origin (not strings smashed together in a loop)*
> I assume your referring to how the examples java precommit is done. It is
> the only case of it to my knowledge and extra hands to migrate it to be
> like how the validates runner tests run would be useful. Filed
> https://issues.apache.org/jira/browse/BEAM-4033
>
>
> On Mon, Apr 9, 2018 at 1:24 PM Kenneth Knowles  wrote:
>
>> Huge +1 to the idea of investing in simplification and polish of the
>> Gradle files before considering the migration complete.
>>
>> 1. build.gradle files should be as close to straight-line configuration
>> as possible:
>>  - You should be able to understand a module's build easily, locally,
>> without knowing Groovy
>>  - As little Groovy programming as possible, focused on providing
>> well-defined utilities
>>
>> 2. Explicit > implicit, especially for config files
>>  - Ditto about being able to understand a module's build locally
>>  - Minimize globally inherited config, in favor of explicitly calling
>> utilities
>>  - Passing current project to a utility is better than trying to make
>> something the "closure"
>>  - If you are going to refer to an identifier, it should have an explicit
>> point of origin (not strings smashed together in a loop)
>>
>> When in doubt, a little redundancy to improve readability is worth it in
>> this context.
>>
>> Kenn
>>
>> On Mon, Apr 9, 2018 at 9:47 AM Jean-Baptiste Onofré 
>> wrote:
>>
>>> Hi all,
>>>
>>> I did multiple gradle build since last week and I would like to share
>>> one of my concern: it's about the communities.
>>>
>>> If I think our users won't see any change for them due to Gradle build
>>> (I think that most of our users will still use Maven with artifacts
>>> provided by Gradle), I'm more concerned by the dev community and the
>>> contribution.
>>>
>>> Maven is well known and straight forward for a large part of potential
>>> contributors. I think we have to keep in mind that we still have to grow
>>> up our contributors community.
>>>
>>> Today, maybe I'm wrong, but I have the feeling that gradle build is not
>>> straight forward (build.gradle includes build_rules.gradle, gathering
>>> all taks all together).
>>>
>>> I would like to add a task in the gradle "migration" process: simplify
>>> the gradle structure and files, and document this.
>>>
>>> I know we already have a Jira about the documentation part, but I would
>>> like to "polish" and use a clean structure for the Gradle resources. As
>>> already quickly discussed, I think that having one gradle file per tasks
>>> in the .gradle directory would be helpful.
>>>
>>> The goal is really to simplify the contribution.
>>>
>>> Do you agree if I add a Jira about "Gradle polish" ?
>>> Thoughts ?
>>>
>>> Regards
>>> JB
>>>
>>> On 07/04/2018 04:52, Scott Wegner wrote:
>>> > Here's an end-of-day update on migration work:
>>> >
>>> > * Snapshot unsigned dailies and signed release builds are working (!!).
>>> > PR/5048 [1] merges changes from Luke's branch
>>> >* python precommit failing... will investigate python precommit
>>> Monday
>>> > * All Precommits are gradle only
>>> > * All Postcommits except performance tests and Java_JDK_Versions_Test
>>> > use gradle (after PR/5047 [2] merged)
>>> > * Nightly snapshot release using gradle is ready; needs PR/5048 to be
>>> > merged before switching
>>> > * ValidatesRunner_Spark failing consistently; investigating
>>> >
>>> > Thanks for another productive day of hacking. I'll pick up again on
>>> Monday.

Re: Gradle Status [April 6]

[Lukasz Cwik]

JB, learning the build system in a project is hopefully avoided by most
users if the contribution guide can clearly explain what users need to do.
But for everyone else who wants to change a dependency version or add a
dependency it should be as simple as copy/paste (which I believe it already
is). Note that people who want to do anything more complicated like add new
jenkins job configurations for new integration test targets needs to learn
how the build system works, how maven profiles work, how to plumb the
required set of attributes from Jenkins into the test target via the build
system.

Kenn, I have to disagree with a lot in 1 and 2. For users who are
unfamiliar with Maven, we didn't setup the Maven build in such a way that
anyone unfamiliar with Maven knew what was going on or try to use concepts
unnatural to Maven in an attempt to make it seem easier. I believe we
should stick with Gradle/Groovy conventions. Users who are not familiar
with how Gradle/Groovy works will either ask the community or look at
StackOverflow and doing things like passing the project around into
functions is extremely uncommon compared to using the current context and
applying closures to it. For users who are familiar with Gradle, the builds
should be natural Gradle.

*- If you are going to refer to an identifier, it should have an explicit
point of origin (not strings smashed together in a loop)*
I assume your referring to how the examples java precommit is done. It is
the only case of it to my knowledge and extra hands to migrate it to be
like how the validates runner tests run would be useful. Filed
https://issues.apache.org/jira/browse/BEAM-4033


On Mon, Apr 9, 2018 at 1:24 PM Kenneth Knowles  wrote:

> Huge +1 to the idea of investing in simplification and polish of the
> Gradle files before considering the migration complete.
>
> 1. build.gradle files should be as close to straight-line configuration as
> possible:
>  - You should be able to understand a module's build easily, locally,
> without knowing Groovy
>  - As little Groovy programming as possible, focused on providing
> well-defined utilities
>
> 2. Explicit > implicit, especially for config files
>  - Ditto about being able to understand a module's build locally
>  - Minimize globally inherited config, in favor of explicitly calling
> utilities
>  - Passing current project to a utility is better than trying to make
> something the "closure"
>  - If you are going to refer to an identifier, it should have an explicit
> point of origin (not strings smashed together in a loop)
>
> When in doubt, a little redundancy to improve readability is worth it in
> this context.
>
> Kenn
>
> On Mon, Apr 9, 2018 at 9:47 AM Jean-Baptiste Onofré 
> wrote:
>
>> Hi all,
>>
>> I did multiple gradle build since last week and I would like to share
>> one of my concern: it's about the communities.
>>
>> If I think our users won't see any change for them due to Gradle build
>> (I think that most of our users will still use Maven with artifacts
>> provided by Gradle), I'm more concerned by the dev community and the
>> contribution.
>>
>> Maven is well known and straight forward for a large part of potential
>> contributors. I think we have to keep in mind that we still have to grow
>> up our contributors community.
>>
>> Today, maybe I'm wrong, but I have the feeling that gradle build is not
>> straight forward (build.gradle includes build_rules.gradle, gathering
>> all taks all together).
>>
>> I would like to add a task in the gradle "migration" process: simplify
>> the gradle structure and files, and document this.
>>
>> I know we already have a Jira about the documentation part, but I would
>> like to "polish" and use a clean structure for the Gradle resources. As
>> already quickly discussed, I think that having one gradle file per tasks
>> in the .gradle directory would be helpful.
>>
>> The goal is really to simplify the contribution.
>>
>> Do you agree if I add a Jira about "Gradle polish" ?
>> Thoughts ?
>>
>> Regards
>> JB
>>
>> On 07/04/2018 04:52, Scott Wegner wrote:
>> > Here's an end-of-day update on migration work:
>> >
>> > * Snapshot unsigned dailies and signed release builds are working (!!).
>> > PR/5048 [1] merges changes from Luke's branch
>> >* python precommit failing... will investigate python precommit
>> Monday
>> > * All Precommits are gradle only
>> > * All Postcommits except performance tests and Java_JDK_Versions_Test
>> > use gradle (after PR/5047 [2] merged)
>> > * Nightly snapshot release using gradle is ready; needs PR/5048 to be
>> > merged before switching
>> > * ValidatesRunner_Spark failing consistently; investigating
>> >
>> > Thanks for another productive day of hacking. I'll pick up again on
>> Monday.
>> >
>> > [1] https://github.com/apache/beam/pull/5048
>> > [2] https://github.com/apache/beam/pull/5047
>> >
>> >
>> > On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau
>> > 

Re: Gradle Status [April 6]

[Kenneth Knowles]

Huge +1 to the idea of investing in simplification and polish of the Gradle
files before considering the migration complete.

1. build.gradle files should be as close to straight-line configuration as
possible:
 - You should be able to understand a module's build easily, locally,
without knowing Groovy
 - As little Groovy programming as possible, focused on providing
well-defined utilities

2. Explicit > implicit, especially for config files
 - Ditto about being able to understand a module's build locally
 - Minimize globally inherited config, in favor of explicitly calling
utilities
 - Passing current project to a utility is better than trying to make
something the "closure"
 - If you are going to refer to an identifier, it should have an explicit
point of origin (not strings smashed together in a loop)

When in doubt, a little redundancy to improve readability is worth it in
this context.

Kenn

On Mon, Apr 9, 2018 at 9:47 AM Jean-Baptiste Onofré  wrote:

> Hi all,
>
> I did multiple gradle build since last week and I would like to share
> one of my concern: it's about the communities.
>
> If I think our users won't see any change for them due to Gradle build
> (I think that most of our users will still use Maven with artifacts
> provided by Gradle), I'm more concerned by the dev community and the
> contribution.
>
> Maven is well known and straight forward for a large part of potential
> contributors. I think we have to keep in mind that we still have to grow
> up our contributors community.
>
> Today, maybe I'm wrong, but I have the feeling that gradle build is not
> straight forward (build.gradle includes build_rules.gradle, gathering
> all taks all together).
>
> I would like to add a task in the gradle "migration" process: simplify
> the gradle structure and files, and document this.
>
> I know we already have a Jira about the documentation part, but I would
> like to "polish" and use a clean structure for the Gradle resources. As
> already quickly discussed, I think that having one gradle file per tasks
> in the .gradle directory would be helpful.
>
> The goal is really to simplify the contribution.
>
> Do you agree if I add a Jira about "Gradle polish" ?
> Thoughts ?
>
> Regards
> JB
>
> On 07/04/2018 04:52, Scott Wegner wrote:
> > Here's an end-of-day update on migration work:
> >
> > * Snapshot unsigned dailies and signed release builds are working (!!).
> > PR/5048 [1] merges changes from Luke's branch
> >* python precommit failing... will investigate python precommit Monday
> > * All Precommits are gradle only
> > * All Postcommits except performance tests and Java_JDK_Versions_Test
> > use gradle (after PR/5047 [2] merged)
> > * Nightly snapshot release using gradle is ready; needs PR/5048 to be
> > merged before switching
> > * ValidatesRunner_Spark failing consistently; investigating
> >
> > Thanks for another productive day of hacking. I'll pick up again on
> Monday.
> >
> > [1] https://github.com/apache/beam/pull/5048
> > [2] https://github.com/apache/beam/pull/5047
> >
> >
> > On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau
> > > wrote:
> >
> > Why building a zip per runner which its stack and just pointing out
> > on that zip and let beam lazy load the runner:
> >
> > --runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=... (or
> > the fromSystemProperties() if it gets merged a day ;))
> >
> > Le 6 avr. 2018 20:21, "Kenneth Knowles"  > > a écrit :
> >
> > I'm working on finding a solution for launching the Nexmark
> > suite with each runner. This doesn't have to be done via Gradle,
> > but we anyhow need built artifacts that don't require user
> > classpath intervention.
> >
> > It looks to me like the examples are also missing this - they
> > have separate configuration e.g. sparkRunnerPreCommit but that
> > is overspecified compared to a free-form launching of a main()
> > program with a runner profile.
> >
> > On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik  > > wrote:
> >
> > Romain, are you talking about the profiles that exist as
> > part of the archetype examples?
> >
> > If so, then those still exist and haven't been changed. If
> > not, can you provide a link to the profile in a pom file to
> > be clearer?
> >
> > On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau
> > >
> wrote:
> >
> > Hi Scott,
> >
> > is it right that 2 doesn't handle the hierachy anymore
> > and that it doesn't handle profiles for runners as it is
> > currently with maven?
> >
> >
> > Romain Manni-Bucau
> > 

Re: Gradle Status [April 6]

[Jean-Baptiste Onofré]

Hi all,

I did multiple gradle build since last week and I would like to share 
one of my concern: it's about the communities.


If I think our users won't see any change for them due to Gradle build 
(I think that most of our users will still use Maven with artifacts 
provided by Gradle), I'm more concerned by the dev community and the 
contribution.


Maven is well known and straight forward for a large part of potential 
contributors. I think we have to keep in mind that we still have to grow 
up our contributors community.


Today, maybe I'm wrong, but I have the feeling that gradle build is not 
straight forward (build.gradle includes build_rules.gradle, gathering 
all taks all together).


I would like to add a task in the gradle "migration" process: simplify 
the gradle structure and files, and document this.


I know we already have a Jira about the documentation part, but I would 
like to "polish" and use a clean structure for the Gradle resources. As 
already quickly discussed, I think that having one gradle file per tasks 
in the .gradle directory would be helpful.


The goal is really to simplify the contribution.

Do you agree if I add a Jira about "Gradle polish" ?
Thoughts ?

Regards
JB

On 07/04/2018 04:52, Scott Wegner wrote:

Here's an end-of-day update on migration work:

* Snapshot unsigned dailies and signed release builds are working (!!). 
PR/5048 [1] merges changes from Luke's branch

   * python precommit failing... will investigate python precommit Monday
* All Precommits are gradle only
* All Postcommits except performance tests and Java_JDK_Versions_Test  
use gradle (after PR/5047 [2] merged)
* Nightly snapshot release using gradle is ready; needs PR/5048 to be 
merged before switching

* ValidatesRunner_Spark failing consistently; investigating

Thanks for another productive day of hacking. I'll pick up again on Monday.

[1] https://github.com/apache/beam/pull/5048
[2] https://github.com/apache/beam/pull/5047


On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau 
> wrote:


Why building a zip per runner which its stack and just pointing out
on that zip and let beam lazy load the runner:

--runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=... (or
the fromSystemProperties() if it gets merged a day ;))

Le 6 avr. 2018 20:21, "Kenneth Knowles" > a écrit :

I'm working on finding a solution for launching the Nexmark
suite with each runner. This doesn't have to be done via Gradle,
but we anyhow need built artifacts that don't require user
classpath intervention.

It looks to me like the examples are also missing this - they
have separate configuration e.g. sparkRunnerPreCommit but that
is overspecified compared to a free-form launching of a main()
program with a runner profile.

On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik > wrote:

Romain, are you talking about the profiles that exist as
part of the archetype examples?

If so, then those still exist and haven't been changed. If
not, can you provide a link to the profile in a pom file to
be clearer?

On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau
> wrote:

Hi Scott,

is it right that 2 doesn't handle the hierachy anymore
and that it doesn't handle profiles for runners as it is
currently with maven?


Romain Manni-Bucau
@rmannibucau  | Blog
 | Old Blog
 | Github
 | LinkedIn
 | Book



2018-04-06 18:32 GMT+02:00 Scott Wegner
>:

I wanted to start a thread to summarize the current
state of Gradle migration. We've made lots of good
progress so far this week. Here's the status from
what I can tell-- please add or correct anything I
missed:

* Release artifacts can be built and published for
Snapshot and officlal releases [1]
* Gradle-generated releases have been validated with
the the Apache Beam archetype generation quickstart;
still needs additional validation.
* Generated release pom files have correct project
metadata 

Re: Gradle Status [April 6]

[Dan Halperin]

On Sat, Apr 7, 2018 at 12:43 Reuven Lax  wrote:

> So if I understand correctly, we've migrated all precommit, most
> postcommits, and we have a working release process using Gradle. There are
> a few bugs left, but at this pace it sounds like we're close to fully
> migrated.
>
> I know that multiple people put it long hours last getting this done last
> week (just look at the Slack messages!). This is awesome progress, and a
> hearty thank you to everyone who put in their time.
>
> Reuven
>
> On Fri, Apr 6, 2018 at 7:52 PM Scott Wegner  wrote:
>
>> Here's an end-of-day update on migration work:
>>
>> * Snapshot unsigned dailies and signed release builds are working (!!).
>> PR/5048 [1] merges changes from Luke's branch
>>   * python precommit failing... will investigate python precommit Monday
>> * All Precommits are gradle only
>> * All Postcommits except performance tests and Java_JDK_Versions_Test
>> use gradle (after PR/5047 [2] merged)
>> * Nightly snapshot release using gradle is ready; needs PR/5048 to be
>> merged before switching
>> * ValidatesRunner_Spark failing consistently; investigating
>>
>> Thanks for another productive day of hacking. I'll pick up again on
>> Monday.
>>
>> [1] https://github.com/apache/beam/pull/5048
>> [2] https://github.com/apache/beam/pull/5047
>>
>>
This is really phenomenal work, and a huge boost to the community. Thanks,
everyone who participated!
Dan


>
>> On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau 
>> wrote:
>>
>>> Why building a zip per runner which its stack and just pointing out on
>>> that zip and let beam lazy load the runner:
>>>
>>> --runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=... (or the
>>> fromSystemProperties() if it gets merged a day ;))
>>>
>>> Le 6 avr. 2018 20:21, "Kenneth Knowles"  a écrit :
>>>
 I'm working on finding a solution for launching the Nexmark suite with
 each runner. This doesn't have to be done via Gradle, but we anyhow need
 built artifacts that don't require user classpath intervention.

 It looks to me like the examples are also missing this - they have
 separate configuration e.g. sparkRunnerPreCommit but that is overspecified
 compared to a free-form launching of a main() program with a runner 
 profile.

 On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik  wrote:

> Romain, are you talking about the profiles that exist as part of the
> archetype examples?
>
> If so, then those still exist and haven't been changed. If not, can
> you provide a link to the profile in a pom file to be clearer?
>
> On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau <
> rmannibu...@gmail.com> wrote:
>
>> Hi Scott,
>>
>> is it right that 2 doesn't handle the hierachy anymore and that it
>> doesn't handle profiles for runners as it is currently with maven?
>>
>>
>> Romain Manni-Bucau
>> @rmannibucau  |  Blog
>>  | Old Blog
>>  | Github
>>  | LinkedIn
>>  | Book
>> 
>>
>> 2018-04-06 18:32 GMT+02:00 Scott Wegner :
>>
>>> I wanted to start a thread to summarize the current state of Gradle
>>> migration. We've made lots of good progress so far this week. Here's the
>>> status from what I can tell-- please add or correct anything I missed:
>>>
>>> * Release artifacts can be built and published for Snapshot and
>>> officlal releases [1]
>>> * Gradle-generated releases have been validated with the the Apache
>>> Beam archetype generation quickstart; still needs additional validation.
>>> * Generated release pom files have correct project metadata [2]
>>> * The python pre-commits are now working in Gradle [3]
>>> * Ismaël has started a collaborative doc of Gradle tips [4] as we
>>> all learn the new system-- please add your own. This will eventually 
>>> feed
>>> into official documentation on the website.
>>> * Łukasz Gajowy is working on migrating performance testing
>>> framework [5]
>>> * Daniel is working on updating documentation to refer to Gradle
>>> instead of maven
>>>
>>> If I missed anything, please add it to this thread.
>>>
>>> The general roadmap we're working towards is:
>>> (a) Publish release artifacts with Gradle (SNAPSHOT and signed
>>> releases)
>>> (b) Postcommits migrated to Gradle
>>> (c) Migrate documentation from maven to Gradle
>>> (d) Migrate perfkit suites to use Gradle
>>>
>>> For those of you that are hacking: thanks for your help so far!
>>> Progress is being roughly tracked on the Kanban 

Re: Gradle Status [April 6]

[Reuven Lax]

So if I understand correctly, we've migrated all precommit, most
postcommits, and we have a working release process using Gradle. There are
a few bugs left, but at this pace it sounds like we're close to fully
migrated.

I know that multiple people put it long hours last getting this done last
week (just look at the Slack messages!). This is awesome progress, and a
hearty thank you to everyone who put in their time.

Reuven

On Fri, Apr 6, 2018 at 7:52 PM Scott Wegner  wrote:

> Here's an end-of-day update on migration work:
>
> * Snapshot unsigned dailies and signed release builds are working (!!).
> PR/5048 [1] merges changes from Luke's branch
>   * python precommit failing... will investigate python precommit Monday
> * All Precommits are gradle only
> * All Postcommits except performance tests and Java_JDK_Versions_Test  use
> gradle (after PR/5047 [2] merged)
> * Nightly snapshot release using gradle is ready; needs PR/5048 to be
> merged before switching
> * ValidatesRunner_Spark failing consistently; investigating
>
> Thanks for another productive day of hacking. I'll pick up again on Monday.
>
> [1] https://github.com/apache/beam/pull/5048
> [2] https://github.com/apache/beam/pull/5047
>
>
> On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau 
> wrote:
>
>> Why building a zip per runner which its stack and just pointing out on
>> that zip and let beam lazy load the runner:
>>
>> --runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=... (or the
>> fromSystemProperties() if it gets merged a day ;))
>>
>> Le 6 avr. 2018 20:21, "Kenneth Knowles"  a écrit :
>>
>>> I'm working on finding a solution for launching the Nexmark suite with
>>> each runner. This doesn't have to be done via Gradle, but we anyhow need
>>> built artifacts that don't require user classpath intervention.
>>>
>>> It looks to me like the examples are also missing this - they have
>>> separate configuration e.g. sparkRunnerPreCommit but that is overspecified
>>> compared to a free-form launching of a main() program with a runner profile.
>>>
>>> On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik  wrote:
>>>
 Romain, are you talking about the profiles that exist as part of the
 archetype examples?

 If so, then those still exist and haven't been changed. If not, can you
 provide a link to the profile in a pom file to be clearer?

 On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau <
 rmannibu...@gmail.com> wrote:

> Hi Scott,
>
> is it right that 2 doesn't handle the hierachy anymore and that it
> doesn't handle profiles for runners as it is currently with maven?
>
>
> Romain Manni-Bucau
> @rmannibucau  |  Blog
>  | Old Blog
>  | Github
>  | LinkedIn
>  | Book
> 
>
> 2018-04-06 18:32 GMT+02:00 Scott Wegner :
>
>> I wanted to start a thread to summarize the current state of Gradle
>> migration. We've made lots of good progress so far this week. Here's the
>> status from what I can tell-- please add or correct anything I missed:
>>
>> * Release artifacts can be built and published for Snapshot and
>> officlal releases [1]
>> * Gradle-generated releases have been validated with the the Apache
>> Beam archetype generation quickstart; still needs additional validation.
>> * Generated release pom files have correct project metadata [2]
>> * The python pre-commits are now working in Gradle [3]
>> * Ismaël has started a collaborative doc of Gradle tips [4] as we all
>> learn the new system-- please add your own. This will eventually feed 
>> into
>> official documentation on the website.
>> * Łukasz Gajowy is working on migrating performance testing framework
>> [5]
>> * Daniel is working on updating documentation to refer to Gradle
>> instead of maven
>>
>> If I missed anything, please add it to this thread.
>>
>> The general roadmap we're working towards is:
>> (a) Publish release artifacts with Gradle (SNAPSHOT and signed
>> releases)
>> (b) Postcommits migrated to Gradle
>> (c) Migrate documentation from maven to Gradle
>> (d) Migrate perfkit suites to use Gradle
>>
>> For those of you that are hacking: thanks for your help so far!
>> Progress is being roughly tracked on the Kanban [6]; please make sure the
>> issues assigned to you are up-to-date. Many of the changes are staged on
>> lukecwik's local branch [7]; we'll work on merging them back soon.
>>
>>
>> [1] https://github.com/lukecwik/incubator-beam/pull/7
>> [2] 

Re: Gradle Status [April 6]

[Scott Wegner]

Here's an end-of-day update on migration work:

* Snapshot unsigned dailies and signed release builds are working (!!).
PR/5048 [1] merges changes from Luke's branch
  * python precommit failing... will investigate python precommit Monday
* All Precommits are gradle only
* All Postcommits except performance tests and Java_JDK_Versions_Test  use
gradle (after PR/5047 [2] merged)
* Nightly snapshot release using gradle is ready; needs PR/5048 to be
merged before switching
* ValidatesRunner_Spark failing consistently; investigating

Thanks for another productive day of hacking. I'll pick up again on Monday.

[1] https://github.com/apache/beam/pull/5048
[2] https://github.com/apache/beam/pull/5047


On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau 
wrote:

> Why building a zip per runner which its stack and just pointing out on
> that zip and let beam lazy load the runner:
>
> --runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=... (or the
> fromSystemProperties() if it gets merged a day ;))
>
> Le 6 avr. 2018 20:21, "Kenneth Knowles"  a écrit :
>
>> I'm working on finding a solution for launching the Nexmark suite with
>> each runner. This doesn't have to be done via Gradle, but we anyhow need
>> built artifacts that don't require user classpath intervention.
>>
>> It looks to me like the examples are also missing this - they have
>> separate configuration e.g. sparkRunnerPreCommit but that is overspecified
>> compared to a free-form launching of a main() program with a runner profile.
>>
>> On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik  wrote:
>>
>>> Romain, are you talking about the profiles that exist as part of the
>>> archetype examples?
>>>
>>> If so, then those still exist and haven't been changed. If not, can you
>>> provide a link to the profile in a pom file to be clearer?
>>>
>>> On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau <
>>> rmannibu...@gmail.com> wrote:
>>>
 Hi Scott,

 is it right that 2 doesn't handle the hierachy anymore and that it
 doesn't handle profiles for runners as it is currently with maven?


 Romain Manni-Bucau
 @rmannibucau  |  Blog
  | Old Blog
  | Github
  | LinkedIn
  | Book
 

 2018-04-06 18:32 GMT+02:00 Scott Wegner :

> I wanted to start a thread to summarize the current state of Gradle
> migration. We've made lots of good progress so far this week. Here's the
> status from what I can tell-- please add or correct anything I missed:
>
> * Release artifacts can be built and published for Snapshot and
> officlal releases [1]
> * Gradle-generated releases have been validated with the the Apache
> Beam archetype generation quickstart; still needs additional validation.
> * Generated release pom files have correct project metadata [2]
> * The python pre-commits are now working in Gradle [3]
> * Ismaël has started a collaborative doc of Gradle tips [4] as we all
> learn the new system-- please add your own. This will eventually feed into
> official documentation on the website.
> * Łukasz Gajowy is working on migrating performance testing framework
> [5]
> * Daniel is working on updating documentation to refer to Gradle
> instead of maven
>
> If I missed anything, please add it to this thread.
>
> The general roadmap we're working towards is:
> (a) Publish release artifacts with Gradle (SNAPSHOT and signed
> releases)
> (b) Postcommits migrated to Gradle
> (c) Migrate documentation from maven to Gradle
> (d) Migrate perfkit suites to use Gradle
>
> For those of you that are hacking: thanks for your help so far!
> Progress is being roughly tracked on the Kanban [6]; please make sure the
> issues assigned to you are up-to-date. Many of the changes are staged on
> lukecwik's local branch [7]; we'll work on merging them back soon.
>
>
> [1] https://github.com/lukecwik/incubator-beam/pull/7
> [2] https://github.com/lukecwik/incubator-beam/pull/3
> [3] https://github.com/apache/beam/pull/5032
> [4]
> https://docs.google.com/document/d/1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit
> [5] https://github.com/apache/beam/pull/5003
> [6]
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=242
> [7] https://github.com/lukecwik/incubator-beam/tree/gradle
> --
>
>
> Got feedback? http://go/swegner-feedback
>

 --


Got feedback? http://go/swegner-feedback

Re: Gradle Status [April 6]

[Romain Manni-Bucau]

Le 6 avr. 2018 20:09, "Lukasz Cwik"  a écrit :

Romain, are you talking about the profiles that exist as part of the
archetype examples?


Was more thinking to this kind of profiles
https://github.com/apache/beam/blob/master/sdks/java/nexmark/pom.xml (which
should hit all IO at some point to ensure their portability)

Idea is to be able to extract the deps for a runner from a particular pom
since it sometimes requires some dependencies work for conflicts.





If so, then those still exist and haven't been changed. If not, can you
provide a link to the profile in a pom file to be clearer?

On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau 
wrote:

> Hi Scott,
>
> is it right that 2 doesn't handle the hierachy anymore and that it doesn't
> handle profiles for runners as it is currently with maven?
>
>
> Romain Manni-Bucau
> @rmannibucau  |  Blog
>  | Old Blog
>  | Github
>  | LinkedIn
>  | Book
> 
>
> 2018-04-06 18:32 GMT+02:00 Scott Wegner :
>
>> I wanted to start a thread to summarize the current state of Gradle
>> migration. We've made lots of good progress so far this week. Here's the
>> status from what I can tell-- please add or correct anything I missed:
>>
>> * Release artifacts can be built and published for Snapshot and officlal
>> releases [1]
>> * Gradle-generated releases have been validated with the the Apache Beam
>> archetype generation quickstart; still needs additional validation.
>> * Generated release pom files have correct project metadata [2]
>> * The python pre-commits are now working in Gradle [3]
>> * Ismaël has started a collaborative doc of Gradle tips [4] as we all
>> learn the new system-- please add your own. This will eventually feed into
>> official documentation on the website.
>> * Łukasz Gajowy is working on migrating performance testing framework [5]
>> * Daniel is working on updating documentation to refer to Gradle instead
>> of maven
>>
>> If I missed anything, please add it to this thread.
>>
>> The general roadmap we're working towards is:
>> (a) Publish release artifacts with Gradle (SNAPSHOT and signed releases)
>> (b) Postcommits migrated to Gradle
>> (c) Migrate documentation from maven to Gradle
>> (d) Migrate perfkit suites to use Gradle
>>
>> For those of you that are hacking: thanks for your help so far! Progress
>> is being roughly tracked on the Kanban [6]; please make sure the issues
>> assigned to you are up-to-date. Many of the changes are staged on
>> lukecwik's local branch [7]; we'll work on merging them back soon.
>>
>>
>> [1] https://github.com/lukecwik/incubator-beam/pull/7
>> [2] https://github.com/lukecwik/incubator-beam/pull/3
>> [3] https://github.com/apache/beam/pull/5032
>> [4] https://docs.google.com/document/d/1wR56Jef3XIPwj4DFzQKznuGPM3JDf
>> RDVkxzeDlbdVSQ/edit
>> [5] https://github.com/apache/beam/pull/5003
>> [6] https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=242
>> [7] https://github.com/lukecwik/incubator-beam/tree/gradle
>> --
>>
>>
>> Got feedback? http://go/swegner-feedback
>>
>
>

Re: Gradle Status [April 6]

[Kenneth Knowles]

I'm working on finding a solution for launching the Nexmark suite with each
runner. This doesn't have to be done via Gradle, but we anyhow need built
artifacts that don't require user classpath intervention.

It looks to me like the examples are also missing this - they have separate
configuration e.g. sparkRunnerPreCommit but that is overspecified compared
to a free-form launching of a main() program with a runner profile.

On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik  wrote:

> Romain, are you talking about the profiles that exist as part of the
> archetype examples?
>
> If so, then those still exist and haven't been changed. If not, can you
> provide a link to the profile in a pom file to be clearer?
>
> On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau 
> wrote:
>
>> Hi Scott,
>>
>> is it right that 2 doesn't handle the hierachy anymore and that it
>> doesn't handle profiles for runners as it is currently with maven?
>>
>>
>> Romain Manni-Bucau
>> @rmannibucau  |  Blog
>>  | Old Blog
>>  | Github
>>  | LinkedIn
>>  | Book
>> 
>>
>> 2018-04-06 18:32 GMT+02:00 Scott Wegner :
>>
>>> I wanted to start a thread to summarize the current state of Gradle
>>> migration. We've made lots of good progress so far this week. Here's the
>>> status from what I can tell-- please add or correct anything I missed:
>>>
>>> * Release artifacts can be built and published for Snapshot and officlal
>>> releases [1]
>>> * Gradle-generated releases have been validated with the the Apache Beam
>>> archetype generation quickstart; still needs additional validation.
>>> * Generated release pom files have correct project metadata [2]
>>> * The python pre-commits are now working in Gradle [3]
>>> * Ismaël has started a collaborative doc of Gradle tips [4] as we all
>>> learn the new system-- please add your own. This will eventually feed into
>>> official documentation on the website.
>>> * Łukasz Gajowy is working on migrating performance testing framework [5]
>>> * Daniel is working on updating documentation to refer to Gradle instead
>>> of maven
>>>
>>> If I missed anything, please add it to this thread.
>>>
>>> The general roadmap we're working towards is:
>>> (a) Publish release artifacts with Gradle (SNAPSHOT and signed releases)
>>> (b) Postcommits migrated to Gradle
>>> (c) Migrate documentation from maven to Gradle
>>> (d) Migrate perfkit suites to use Gradle
>>>
>>> For those of you that are hacking: thanks for your help so far! Progress
>>> is being roughly tracked on the Kanban [6]; please make sure the issues
>>> assigned to you are up-to-date. Many of the changes are staged on
>>> lukecwik's local branch [7]; we'll work on merging them back soon.
>>>
>>>
>>> [1] https://github.com/lukecwik/incubator-beam/pull/7
>>> [2] https://github.com/lukecwik/incubator-beam/pull/3
>>> [3] https://github.com/apache/beam/pull/5032
>>> [4]
>>> https://docs.google.com/document/d/1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit
>>> [5] https://github.com/apache/beam/pull/5003
>>> [6] https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=242
>>> [7] https://github.com/lukecwik/incubator-beam/tree/gradle
>>> --
>>>
>>>
>>> Got feedback? http://go/swegner-feedback
>>>
>>
>>

Re: Gradle Status [April 6]

[Lukasz Cwik]

Romain, are you talking about the profiles that exist as part of the
archetype examples?

If so, then those still exist and haven't been changed. If not, can you
provide a link to the profile in a pom file to be clearer?

On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau 
wrote:

> Hi Scott,
>
> is it right that 2 doesn't handle the hierachy anymore and that it doesn't
> handle profiles for runners as it is currently with maven?
>
>
> Romain Manni-Bucau
> @rmannibucau  |  Blog
>  | Old Blog
>  | Github
>  | LinkedIn
>  | Book
> 
>
> 2018-04-06 18:32 GMT+02:00 Scott Wegner :
>
>> I wanted to start a thread to summarize the current state of Gradle
>> migration. We've made lots of good progress so far this week. Here's the
>> status from what I can tell-- please add or correct anything I missed:
>>
>> * Release artifacts can be built and published for Snapshot and officlal
>> releases [1]
>> * Gradle-generated releases have been validated with the the Apache Beam
>> archetype generation quickstart; still needs additional validation.
>> * Generated release pom files have correct project metadata [2]
>> * The python pre-commits are now working in Gradle [3]
>> * Ismaël has started a collaborative doc of Gradle tips [4] as we all
>> learn the new system-- please add your own. This will eventually feed into
>> official documentation on the website.
>> * Łukasz Gajowy is working on migrating performance testing framework [5]
>> * Daniel is working on updating documentation to refer to Gradle instead
>> of maven
>>
>> If I missed anything, please add it to this thread.
>>
>> The general roadmap we're working towards is:
>> (a) Publish release artifacts with Gradle (SNAPSHOT and signed releases)
>> (b) Postcommits migrated to Gradle
>> (c) Migrate documentation from maven to Gradle
>> (d) Migrate perfkit suites to use Gradle
>>
>> For those of you that are hacking: thanks for your help so far! Progress
>> is being roughly tracked on the Kanban [6]; please make sure the issues
>> assigned to you are up-to-date. Many of the changes are staged on
>> lukecwik's local branch [7]; we'll work on merging them back soon.
>>
>>
>> [1] https://github.com/lukecwik/incubator-beam/pull/7
>> [2] https://github.com/lukecwik/incubator-beam/pull/3
>> [3] https://github.com/apache/beam/pull/5032
>> [4]
>> https://docs.google.com/document/d/1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit
>> [5] https://github.com/apache/beam/pull/5003
>> [6] https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=242
>> [7] https://github.com/lukecwik/incubator-beam/tree/gradle
>> --
>>
>>
>> Got feedback? http://go/swegner-feedback
>>
>
>

Gradle Status [April 6]

[Scott Wegner]

I wanted to start a thread to summarize the current state of Gradle
migration. We've made lots of good progress so far this week. Here's the
status from what I can tell-- please add or correct anything I missed:

* Release artifacts can be built and published for Snapshot and officlal
releases [1]
* Gradle-generated releases have been validated with the the Apache Beam
archetype generation quickstart; still needs additional validation.
* Generated release pom files have correct project metadata [2]
* The python pre-commits are now working in Gradle [3]
* Ismaël has started a collaborative doc of Gradle tips [4] as we all learn
the new system-- please add your own. This will eventually feed into
official documentation on the website.
* Łukasz Gajowy is working on migrating performance testing framework [5]
* Daniel is working on updating documentation to refer to Gradle instead of
maven

If I missed anything, please add it to this thread.

The general roadmap we're working towards is:
(a) Publish release artifacts with Gradle (SNAPSHOT and signed releases)
(b) Postcommits migrated to Gradle
(c) Migrate documentation from maven to Gradle
(d) Migrate perfkit suites to use Gradle

For those of you that are hacking: thanks for your help so far! Progress is
being roughly tracked on the Kanban [6]; please make sure the issues
assigned to you are up-to-date. Many of the changes are staged on
lukecwik's local branch [7]; we'll work on merging them back soon.


[1] https://github.com/lukecwik/incubator-beam/pull/7
[2] https://github.com/lukecwik/incubator-beam/pull/3
[3] https://github.com/apache/beam/pull/5032
[4]
https://docs.google.com/document/d/1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit
[5] https://github.com/apache/beam/pull/5003
[6] https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=242
[7] https://github.com/lukecwik/incubator-beam/tree/gradle
-- 


Got feedback? http://go/swegner-feedback