2018-01-23 23:44 GMT+01:00 Kenneth Knowles <k...@google.com>:
> Romain - really good point about docs getting out of date.
>
> > On 01/23/2018 09:29 AM, Romain Manni-Bucau wrote:
>> >> Yep,
>> >>
>> >> a compromise can be to ensure all custom tasks have a description
>> maybe:
>>
>
> This is a great idea.
>
> Let's do both! I think the thing that the comments in the gradle code
> cannot capture are the ways that you might combine them, like the way you
> override properties, etc.
>
> On Tue, Jan 23, 2018 at 9:13 AM, Ismaël Mejía <ieme...@gmail.com> wrote:
>
>> I used to test kafka
>> with a different version of the client quickly by doing this with
>> maven:
>>
>> mvn verify -Prelease -Dkafka.clients.version=0.10.0.1 -pl
>> 'sdks/java/io/kafka'
>>
>> but I don't have any idea of how to do this on Gradle.
>>
>
> Did you figure this out. Luke - can you suggest something?
>
>
> Of course everybody has a different workflow, but I am pretty sure
>> there are some common tasks that we can document for people (like me)
>> that are new to gradle.
>
>
> Yea, actually, it never even occurred to me that you would use the command
> line to test against other Kafka versions :-).
>
> I have private gists with lots of maven invocations for doing things like
> running ValidatesRunner or example ITs on just a particular runner. It
> always requires multiple maven commands that are both multiple lines. I was
> halfway for a while but I am now all gradle. I will start building the same
> thing.
>
>
>> - How to skip the Java/Python/Go build depending on your priorities.
>>
>
> IMO they should NOT run by default. Everything should be
> dependency-driven. When I ask to run Java SDK tests, or Java examples IT on
> the Flink Runner, it is *incorrect* for any Python or Go builds to run.
>
>
>> - How to run an individual IntegrationTest or ValidatesRunner test.
>>
>
> Yes, #1 use case
>
>
>> - How to skip findbugs,checkstyle,javadoc gneeration, etc. To have an
>> ultra quick build.
>>
>
> Again, I think they should be independent tasks. I should be able to run
> *any* of them without running *any* of the others. It is incorrect for
> anything to cause any other thing to run if it does not directly require
> its outputs.
>
> There can be an aggregated "verify" command but I will actually very
> rarely run that until I am done with a large chunk of work.
>
> As for what the aggregated "verify" command does, people kept arguing
> about what to make default. As long as we have a correct build for
> individual checks (aka not running extra things) then I am happy for the
> default to be long and slow, but we should still build profiles for both.
>
Hmm, did you read it right Kenn? I think the idea was to skip all
validation/sanity checks tasks at once (gradle xxxx -Pfast) instead of
doing it manually (gradle -x findbugs -x checkstyle etc...)
>
> Kenn
>
>
>
>> >>
>> >> diff --git a/examples/java/build.gradle b/examples/java/build.gradle
>> >> index 0fc0b17df..001bd8b38 100644
>> >> --- a/examples/java/build.gradle
>> >> +++ b/examples/java/build.gradle
>> >> @@ -130,7 +130,7 @@ def preCommitAdditionalFlags = [
>> >> dataflowStreamingRunner: [ "--streaming=true" ],
>> >> ]
>> >> for (String runner : preCommitRunners) {
>> >> - tasks.create(name: runner + "PreCommit", type: Test) {
>> >> + tasks.create(name: runner + "PreCommit", type: Test, description:
>> "Run tests
>> >> for runner ${runner.replace('Runner', '')}") {
>> >> def preCommitBeamTestPipelineOptions = [
>> >> "--project=apache-beam-testing",
>> >> "--tempRoot=gs://temp-storage-for-end-to-end-tests",
>> >>
>> >>
>> >>
>> >> Romain Manni-Bucau
>> >> @rmannibucau <https://twitter.com/rmannibucau> | Blog
>> >> <https://rmannibucau.metawerx.net/> | Old Blog
>> >> <http://rmannibucau.wordpress.com> | Github <
>> https://github.com/rmannibucau> |
>> >> LinkedIn <https://www.linkedin.com/in/rmannibucau>
>> >>
>> >> 2018-01-23 8:45 GMT+01:00 Jean-Baptiste Onofré <j...@nanthrax.net
>> >> <mailto:j...@nanthrax.net>>:
>> >>
>> >> Hi Romain,
>> >>
>> >> I think we are pretty close: agree to add some explicit tasks &
>> projects names.
>> >>
>> >> We can add additional tasks like skipAudit, for instance.
>> >>
>> >> As reminder, gradle tasks provides the list of tasks and gradle
>> projects
>> >> provides the list of projects/modules.
>> >>
>> >> Regards
>> >> JB
>> >>
>> >> On 01/23/2018 08:12 AM, Romain Manni-Bucau wrote:
>> >> > Hmm, I have to admit docs dont have my favor cause they are
>> easily outdated and
>> >> > hard to search but you hit a good point. Starting by renaming
>> properly the tasks
>> >> > and maybe writing what is done in build files - since it is code
>> and even "api
>> >> > for dev", it requires as much comments than the main api - can
>> be better to start.
>> >> >
>> >> > Also a big switch flag to bypass checkstyle/findbugs/... can be
>> good while in
>> >> > dev since these phases cost a looot for nothing while you
>> validates your code in
>> >> > runners modules for instance.
>> >> >
>> >> > Le 23 janv. 2018 07:15, "Kenneth Knowles" <k...@google.com
>> <mailto:k...@google.com>
>> >> > <mailto:k...@google.com <mailto:k...@google.com>>> a écrit :
>> >> >
>> >> > On Mon, Jan 22, 2018 at 10:03 PM, Romain Manni-Bucau <
>> rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>
>> >> > <mailto:rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>>>
>> wrote:
>> >> >
>> >> > @Kenneth: why not dropping the doc for a script with
>> comments in the
>> >> > project? A "RUNME.sh" ;).
>> >> >
>> >> >
>> >> > That's cool, too, but also any shell one liner can be a
>> gradle one
>> >> liner or
>> >> > mvn two/three liner :-). it is just trading one command that
>> you cannot
>> >> > guess easily for a different one that you still can't guess
>> easily.
>> >> >
>> >> > For example, are the SparkRunner ValidatesRunner tests in the
>> >> SparkRunner or
>> >> > the core SDK or a third module that integrates the two? And
>> why would you
>> >> > know that the example ITs are called "sparkRunnerPreCommit"?
>> It
>> >> doesn't even
>> >> > make sense really to have "precommit" or "postcommit" except
>> as aliases to
>> >> > make it easy to repro Jenkins' behavior - they have no other
>> intrinsic
>> >> meaning.
>> >> >
>> >> > So I was proposing a mapping from "full sentence +
>> description" to one
>> >> liner
>> >> > to help people navigate the targets that we set up. Some web
>> page or doc
>> >> > that people can just quickly scan to find out to do common
>> things, easier
>> >> > than groovy or XML.
>> >> >
>> >> > Kenn
>> >> >
>> >> >
>> >>
>> >> --
>> >> Jean-Baptiste Onofré
>> >> jbono...@apache.org <mailto:jbono...@apache.org>
>> >> http://blog.nanthrax.net
>> >> Talend - http://www.talend.com
>> >>
>> >>
>> >
>> > --
>> > Jean-Baptiste Onofré
>> > jbono...@apache.org
>> > http://blog.nanthrax.net
>> > Talend - http://www.talend.com
>>
>
>
