Re: One Pager - Test Command Line Discoverability in Beam

[Kenneth Knowles]

On Wed, May 26, 2021 at 5:05 PM Kyle Weaver  wrote:

> (I assume the "job_" prefix is arbitrary; I'm not sure why we use a
>> filename prefix instead of a proper subfolder).
>>
>
Tangent: Totally agree this is odd. I believe it was because we failed to
figure out how to import utility libraries such as
CommonJobProperties.groovy from another folder.


> In Jenkins, the job name is prefixed by "beam_" (I assume for historical
>> reasons) on Jenkins and postfixed for variants like "_cron" or "_phrase".
>>
>
Quick explanation why we need the _commit and the _phrase variant (for
precommits): the _commit variant only runs if certain files are touched.
Even with a phrase trigger it will choose not to run. So the _phrase
variant never runs by default, but has no such filter and will run when
requested. It is all encapsulated in the builder classes though, I hope.

Kenn



> 2. The Github comment trigger, e.g. "Run Java Dataflow V2 ValidatesRunner"
>> 3. The job name displayed on Github, e.g. "Google Cloud Dataflow Runner
>> V2 Java ValidatesRunner Tests"
>>
>> The latter two seem redundant.
>>
>> On Wed, May 26, 2021 at 3:47 PM Kenneth Knowles  wrote:
>>
>>> Yea, big picture I agree with Kyle. "./gradlew $MODULE:test" and
>>> "./gradlew $MODULE:integrationTest" should be catch-alls ideally. With
>>> reasonable error messages if integrationTest has required parameters.
>>>
>>> Also just naming Jenkins jobs exactly by the command to run them would
>>> go a long way for me, personally.
>>>
>>> Kenn
>>>
>>> On Tue, May 25, 2021 at 2:46 PM Austin Bennett <
>>> whatwouldausti...@gmail.com> wrote:
>>>
>>>> Cool; will be good to have and make things clearer!
>>>>
>>>> On Tue, May 25, 2021 at 2:39 PM Kyle Weaver 
>>>> wrote:
>>>>
>>>>> I left some comments. In summary, I think this is mostly a
>>>>> documentation problem. If running a test isn't as easy as "./gradlew
>>>>> $MODULE:integrationTest", there should be instructions in the test class's
>>>>> javadoc.
>>>>>
>>>>> On Tue, May 25, 2021 at 2:05 PM Udi Meiri  wrote:
>>>>>
>>>>>> My first place to go would be here:
>>>>>> https://cwiki.apache.org/confluence/display/BEAM/Java+Tips (although
>>>>>> it doesn't document your use-case)
>>>>>>
>>>>>> You are right that finding the correct gradle task or jenkins job is
>>>>>> not straightforward.
>>>>>>
>>>>>>
>>>>>> On Tue, May 25, 2021 at 12:48 PM Alex Amato 
>>>>>> wrote:
>>>>>>
>>>>>>> Friendly ping. I'll wait for more suggestions by the end of the
>>>>>>> week. Then close it out.
>>>>>>>
>>>>>>> -- Forwarded message -
>>>>>>> From: Alex Amato 
>>>>>>> Date: Fri, May 21, 2021 at 2:54 PM
>>>>>>> Subject: One Pager - Test Command Line Discoverability in Beam
>>>>>>> To: dev 
>>>>>>>
>>>>>>>
>>>>>>> Hi, I have had some issues determining how to run Beam tests. I have
>>>>>>> written a one pager for review and would like your feedback, to solve 
>>>>>>> the
>>>>>>> problem
>>>>>>> <https://docs.google.com/document/d/1qGkBmHUOIVM2KfPyKhJL6oH2nAgX9Vkl0z9ZydEoWzA/edit>
>>>>>>> :
>>>>>>>
>>>>>>> "A Beam developer is looking at a test file, such as
>>>>>>> “BigQueryTornadoesIT.java” and wants to run this test. But they do not 
>>>>>>> know
>>>>>>> the command line they need to type to run this test."
>>>>>>>
>>>>>>> I would like your feedback, to get toward a more concrete proposal.
>>>>>>> A few solutions are possible for this, mentioned in the proposal. But 
>>>>>>> any
>>>>>>> solution that makes it very easy to understand how to run the test is a
>>>>>>> viable option as well.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Alex
>>>>>>>
>>>>>>

Re: One Pager - Test Command Line Discoverability in Beam

[Kyle Weaver]

Filed a wish JIRA (my favorite kind): BEAM-12412

On Wed, May 26, 2021 at 4:34 PM Kyle Weaver  wrote:

> > Also just naming Jenkins jobs exactly by the command to run them would
> go a long way for me, personally.
>
> Good point. If I may complain provide some detail, AFAIK Jenkins jobs
> have up to three different names.
>
> 1. The job name stem, e.g. "PostCommit_Java_VR_Dataflow_V2". This is
> defined in the source file
> "job_PostCommit_Java_ValidatesRunner_Dataflow_V2.groovy" (I assume the
> "job_" prefix is arbitrary; I'm not sure why we use a filename prefix
> instead of a proper subfolder). In Jenkins, the job name is prefixed by
> "beam_" (I assume for historical reasons) on Jenkins and postfixed for
> variants like "_cron" or "_phrase".
> 2. The Github comment trigger, e.g. "Run Java Dataflow V2 ValidatesRunner"
> 3. The job name displayed on Github, e.g. "Google Cloud Dataflow Runner V2
> Java ValidatesRunner Tests"
>
> The latter two seem redundant.
>
> On Wed, May 26, 2021 at 3:47 PM Kenneth Knowles  wrote:
>
>> Yea, big picture I agree with Kyle. "./gradlew $MODULE:test" and
>> "./gradlew $MODULE:integrationTest" should be catch-alls ideally. With
>> reasonable error messages if integrationTest has required parameters.
>>
>> Also just naming Jenkins jobs exactly by the command to run them would go
>> a long way for me, personally.
>>
>> Kenn
>>
>> On Tue, May 25, 2021 at 2:46 PM Austin Bennett <
>> whatwouldausti...@gmail.com> wrote:
>>
>>> Cool; will be good to have and make things clearer!
>>>
>>> On Tue, May 25, 2021 at 2:39 PM Kyle Weaver  wrote:
>>>
>>>> I left some comments. In summary, I think this is mostly a
>>>> documentation problem. If running a test isn't as easy as "./gradlew
>>>> $MODULE:integrationTest", there should be instructions in the test class's
>>>> javadoc.
>>>>
>>>> On Tue, May 25, 2021 at 2:05 PM Udi Meiri  wrote:
>>>>
>>>>> My first place to go would be here:
>>>>> https://cwiki.apache.org/confluence/display/BEAM/Java+Tips (although
>>>>> it doesn't document your use-case)
>>>>>
>>>>> You are right that finding the correct gradle task or jenkins job is
>>>>> not straightforward.
>>>>>
>>>>>
>>>>> On Tue, May 25, 2021 at 12:48 PM Alex Amato 
>>>>> wrote:
>>>>>
>>>>>> Friendly ping. I'll wait for more suggestions by the end of the week.
>>>>>> Then close it out.
>>>>>>
>>>>>> -- Forwarded message -
>>>>>> From: Alex Amato 
>>>>>> Date: Fri, May 21, 2021 at 2:54 PM
>>>>>> Subject: One Pager - Test Command Line Discoverability in Beam
>>>>>> To: dev 
>>>>>>
>>>>>>
>>>>>> Hi, I have had some issues determining how to run Beam tests. I have
>>>>>> written a one pager for review and would like your feedback, to solve the
>>>>>> problem
>>>>>> <https://docs.google.com/document/d/1qGkBmHUOIVM2KfPyKhJL6oH2nAgX9Vkl0z9ZydEoWzA/edit>
>>>>>> :
>>>>>>
>>>>>> "A Beam developer is looking at a test file, such as
>>>>>> “BigQueryTornadoesIT.java” and wants to run this test. But they do not 
>>>>>> know
>>>>>> the command line they need to type to run this test."
>>>>>>
>>>>>> I would like your feedback, to get toward a more concrete proposal. A
>>>>>> few solutions are possible for this, mentioned in the proposal. But any
>>>>>> solution that makes it very easy to understand how to run the test is a
>>>>>> viable option as well.
>>>>>>
>>>>>> Cheers,
>>>>>> Alex
>>>>>>
>>>>>

Re: One Pager - Test Command Line Discoverability in Beam

[Kyle Weaver]

> Also just naming Jenkins jobs exactly by the command to run them would go
a long way for me, personally.

Good point. If I may complain provide some detail, AFAIK Jenkins jobs have
up to three different names.

1. The job name stem, e.g. "PostCommit_Java_VR_Dataflow_V2". This is
defined in the source file
"job_PostCommit_Java_ValidatesRunner_Dataflow_V2.groovy" (I assume the
"job_" prefix is arbitrary; I'm not sure why we use a filename prefix
instead of a proper subfolder). In Jenkins, the job name is prefixed by
"beam_" (I assume for historical reasons) on Jenkins and postfixed for
variants like "_cron" or "_phrase".
2. The Github comment trigger, e.g. "Run Java Dataflow V2 ValidatesRunner"
3. The job name displayed on Github, e.g. "Google Cloud Dataflow Runner V2
Java ValidatesRunner Tests"

The latter two seem redundant.

On Wed, May 26, 2021 at 3:47 PM Kenneth Knowles  wrote:

> Yea, big picture I agree with Kyle. "./gradlew $MODULE:test" and
> "./gradlew $MODULE:integrationTest" should be catch-alls ideally. With
> reasonable error messages if integrationTest has required parameters.
>
> Also just naming Jenkins jobs exactly by the command to run them would go
> a long way for me, personally.
>
> Kenn
>
> On Tue, May 25, 2021 at 2:46 PM Austin Bennett <
> whatwouldausti...@gmail.com> wrote:
>
>> Cool; will be good to have and make things clearer!
>>
>> On Tue, May 25, 2021 at 2:39 PM Kyle Weaver  wrote:
>>
>>> I left some comments. In summary, I think this is mostly a documentation
>>> problem. If running a test isn't as easy as "./gradlew
>>> $MODULE:integrationTest", there should be instructions in the test class's
>>> javadoc.
>>>
>>> On Tue, May 25, 2021 at 2:05 PM Udi Meiri  wrote:
>>>
>>>> My first place to go would be here:
>>>> https://cwiki.apache.org/confluence/display/BEAM/Java+Tips (although
>>>> it doesn't document your use-case)
>>>>
>>>> You are right that finding the correct gradle task or jenkins job is
>>>> not straightforward.
>>>>
>>>>
>>>> On Tue, May 25, 2021 at 12:48 PM Alex Amato  wrote:
>>>>
>>>>> Friendly ping. I'll wait for more suggestions by the end of the week.
>>>>> Then close it out.
>>>>>
>>>>> -- Forwarded message -
>>>>> From: Alex Amato 
>>>>> Date: Fri, May 21, 2021 at 2:54 PM
>>>>> Subject: One Pager - Test Command Line Discoverability in Beam
>>>>> To: dev 
>>>>>
>>>>>
>>>>> Hi, I have had some issues determining how to run Beam tests. I have
>>>>> written a one pager for review and would like your feedback, to solve the
>>>>> problem
>>>>> <https://docs.google.com/document/d/1qGkBmHUOIVM2KfPyKhJL6oH2nAgX9Vkl0z9ZydEoWzA/edit>
>>>>> :
>>>>>
>>>>> "A Beam developer is looking at a test file, such as
>>>>> “BigQueryTornadoesIT.java” and wants to run this test. But they do not 
>>>>> know
>>>>> the command line they need to type to run this test."
>>>>>
>>>>> I would like your feedback, to get toward a more concrete proposal. A
>>>>> few solutions are possible for this, mentioned in the proposal. But any
>>>>> solution that makes it very easy to understand how to run the test is a
>>>>> viable option as well.
>>>>>
>>>>> Cheers,
>>>>> Alex
>>>>>
>>>>

Re: One Pager - Test Command Line Discoverability in Beam

[Kenneth Knowles]

Yea, big picture I agree with Kyle. "./gradlew $MODULE:test" and "./gradlew
$MODULE:integrationTest" should be catch-alls ideally. With reasonable
error messages if integrationTest has required parameters.

Also just naming Jenkins jobs exactly by the command to run them would go a
long way for me, personally.

Kenn

On Tue, May 25, 2021 at 2:46 PM Austin Bennett 
wrote:

> Cool; will be good to have and make things clearer!
>
> On Tue, May 25, 2021 at 2:39 PM Kyle Weaver  wrote:
>
>> I left some comments. In summary, I think this is mostly a documentation
>> problem. If running a test isn't as easy as "./gradlew
>> $MODULE:integrationTest", there should be instructions in the test class's
>> javadoc.
>>
>> On Tue, May 25, 2021 at 2:05 PM Udi Meiri  wrote:
>>
>>> My first place to go would be here:
>>> https://cwiki.apache.org/confluence/display/BEAM/Java+Tips (although it
>>> doesn't document your use-case)
>>>
>>> You are right that finding the correct gradle task or jenkins job is not
>>> straightforward.
>>>
>>>
>>> On Tue, May 25, 2021 at 12:48 PM Alex Amato  wrote:
>>>
>>>> Friendly ping. I'll wait for more suggestions by the end of the week.
>>>> Then close it out.
>>>>
>>>> -- Forwarded message -
>>>> From: Alex Amato 
>>>> Date: Fri, May 21, 2021 at 2:54 PM
>>>> Subject: One Pager - Test Command Line Discoverability in Beam
>>>> To: dev 
>>>>
>>>>
>>>> Hi, I have had some issues determining how to run Beam tests. I have
>>>> written a one pager for review and would like your feedback, to solve the
>>>> problem
>>>> <https://docs.google.com/document/d/1qGkBmHUOIVM2KfPyKhJL6oH2nAgX9Vkl0z9ZydEoWzA/edit>
>>>> :
>>>>
>>>> "A Beam developer is looking at a test file, such as
>>>> “BigQueryTornadoesIT.java” and wants to run this test. But they do not know
>>>> the command line they need to type to run this test."
>>>>
>>>> I would like your feedback, to get toward a more concrete proposal. A
>>>> few solutions are possible for this, mentioned in the proposal. But any
>>>> solution that makes it very easy to understand how to run the test is a
>>>> viable option as well.
>>>>
>>>> Cheers,
>>>> Alex
>>>>
>>>

Re: One Pager - Test Command Line Discoverability in Beam

[Austin Bennett]

Cool; will be good to have and make things clearer!

On Tue, May 25, 2021 at 2:39 PM Kyle Weaver  wrote:

> I left some comments. In summary, I think this is mostly a documentation
> problem. If running a test isn't as easy as "./gradlew
> $MODULE:integrationTest", there should be instructions in the test class's
> javadoc.
>
> On Tue, May 25, 2021 at 2:05 PM Udi Meiri  wrote:
>
>> My first place to go would be here:
>> https://cwiki.apache.org/confluence/display/BEAM/Java+Tips (although it
>> doesn't document your use-case)
>>
>> You are right that finding the correct gradle task or jenkins job is not
>> straightforward.
>>
>>
>> On Tue, May 25, 2021 at 12:48 PM Alex Amato  wrote:
>>
>>> Friendly ping. I'll wait for more suggestions by the end of the week.
>>> Then close it out.
>>>
>>> ------ Forwarded message -----
>>> From: Alex Amato 
>>> Date: Fri, May 21, 2021 at 2:54 PM
>>> Subject: One Pager - Test Command Line Discoverability in Beam
>>> To: dev 
>>>
>>>
>>> Hi, I have had some issues determining how to run Beam tests. I have
>>> written a one pager for review and would like your feedback, to solve the
>>> problem
>>> <https://docs.google.com/document/d/1qGkBmHUOIVM2KfPyKhJL6oH2nAgX9Vkl0z9ZydEoWzA/edit>
>>> :
>>>
>>> "A Beam developer is looking at a test file, such as
>>> “BigQueryTornadoesIT.java” and wants to run this test. But they do not know
>>> the command line they need to type to run this test."
>>>
>>> I would like your feedback, to get toward a more concrete proposal. A
>>> few solutions are possible for this, mentioned in the proposal. But any
>>> solution that makes it very easy to understand how to run the test is a
>>> viable option as well.
>>>
>>> Cheers,
>>> Alex
>>>
>>

Re: One Pager - Test Command Line Discoverability in Beam

[Kyle Weaver]

I left some comments. In summary, I think this is mostly a documentation
problem. If running a test isn't as easy as "./gradlew
$MODULE:integrationTest", there should be instructions in the test class's
javadoc.

On Tue, May 25, 2021 at 2:05 PM Udi Meiri  wrote:

> My first place to go would be here:
> https://cwiki.apache.org/confluence/display/BEAM/Java+Tips (although it
> doesn't document your use-case)
>
> You are right that finding the correct gradle task or jenkins job is not
> straightforward.
>
>
> On Tue, May 25, 2021 at 12:48 PM Alex Amato  wrote:
>
>> Friendly ping. I'll wait for more suggestions by the end of the week.
>> Then close it out.
>>
>> -- Forwarded message -
>> From: Alex Amato 
>> Date: Fri, May 21, 2021 at 2:54 PM
>> Subject: One Pager - Test Command Line Discoverability in Beam
>> To: dev 
>>
>>
>> Hi, I have had some issues determining how to run Beam tests. I have
>> written a one pager for review and would like your feedback, to solve the
>> problem
>> <https://docs.google.com/document/d/1qGkBmHUOIVM2KfPyKhJL6oH2nAgX9Vkl0z9ZydEoWzA/edit>
>> :
>>
>> "A Beam developer is looking at a test file, such as
>> “BigQueryTornadoesIT.java” and wants to run this test. But they do not know
>> the command line they need to type to run this test."
>>
>> I would like your feedback, to get toward a more concrete proposal. A few
>> solutions are possible for this, mentioned in the proposal. But any
>> solution that makes it very easy to understand how to run the test is a
>> viable option as well.
>>
>> Cheers,
>> Alex
>>
>

Re: One Pager - Test Command Line Discoverability in Beam

[Udi Meiri]

My first place to go would be here:
https://cwiki.apache.org/confluence/display/BEAM/Java+Tips (although it
doesn't document your use-case)

You are right that finding the correct gradle task or jenkins job is not
straightforward.


On Tue, May 25, 2021 at 12:48 PM Alex Amato  wrote:

> Friendly ping. I'll wait for more suggestions by the end of the week. Then
> close it out.
>
> -- Forwarded message -
> From: Alex Amato 
> Date: Fri, May 21, 2021 at 2:54 PM
> Subject: One Pager - Test Command Line Discoverability in Beam
> To: dev 
>
>
> Hi, I have had some issues determining how to run Beam tests. I have
> written a one pager for review and would like your feedback, to solve the
> problem
> <https://docs.google.com/document/d/1qGkBmHUOIVM2KfPyKhJL6oH2nAgX9Vkl0z9ZydEoWzA/edit>
> :
>
> "A Beam developer is looking at a test file, such as
> “BigQueryTornadoesIT.java” and wants to run this test. But they do not know
> the command line they need to type to run this test."
>
> I would like your feedback, to get toward a more concrete proposal. A few
> solutions are possible for this, mentioned in the proposal. But any
> solution that makes it very easy to understand how to run the test is a
> viable option as well.
>
> Cheers,
> Alex
>


smime.p7s
Description: S/MIME Cryptographic Signature

Fwd: One Pager - Test Command Line Discoverability in Beam

[Alex Amato]

Friendly ping. I'll wait for more suggestions by the end of the week. Then
close it out.

-- Forwarded message -
From: Alex Amato 
Date: Fri, May 21, 2021 at 2:54 PM
Subject: One Pager - Test Command Line Discoverability in Beam
To: dev 


Hi, I have had some issues determining how to run Beam tests. I have
written a one pager for review and would like your feedback, to solve the
problem
<https://docs.google.com/document/d/1qGkBmHUOIVM2KfPyKhJL6oH2nAgX9Vkl0z9ZydEoWzA/edit>
:

"A Beam developer is looking at a test file, such as
“BigQueryTornadoesIT.java” and wants to run this test. But they do not know
the command line they need to type to run this test."

I would like your feedback, to get toward a more concrete proposal. A few
solutions are possible for this, mentioned in the proposal. But any
solution that makes it very easy to understand how to run the test is a
viable option as well.

Cheers,
Alex

One Pager - Test Command Line Discoverability in Beam

[Alex Amato]

Hi, I have had some issues determining how to run Beam tests. I have
written a one pager for review and would like your feedback, to solve the
problem

:

"A Beam developer is looking at a test file, such as
“BigQueryTornadoesIT.java” and wants to run this test. But they do not know
the command line they need to type to run this test."

I would like your feedback, to get toward a more concrete proposal. A few
solutions are possible for this, mentioned in the proposal. But any
solution that makes it very easy to understand how to run the test is a
viable option as well.

Cheers,
Alex