Le 10 avr. 2018 19:53, "Kenneth Knowles" <k...@google.com> a écrit :
I've been on Idea+Gradle for ~two months, around the time I added https://github.com/apache/beam/pull/4583 and https://github.com/apache/ beam/pull/4626 to make the import require zero user work. I have no fear of deleting my project any time and re-importing. The import works (is slow but works) but then i can run anything "normally". I agree with not having auto-import on. It is just too slow. I can't remember if it was importing too often due to build outputs or if it was just that I was messing with the build.gradle files. Anyhow it doesn't really add much value. The gradle runner _is_ able to use submodules and run individual tests methods, and all that. If i run gradle from any io or any runner module it still loads the whole project and checks it which is very slow compare to just run the module...and i hope i dont need a 50chars long command specific each time to do it, just "build" would be good. Kenn On Tue, Apr 10, 2018 at 9:31 AM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Runner a test doesnt have the right classpath (idea uses out/ instead > of build/) then when you switch on gradle runner the launching uses > gradle which is not able to use submodules directly but reconsider the > whole project which is quite slow for normal dev iterations > compare to just run the test with the right classpath and a fast > compile step if needed. I lost literally 1h for something simple with > that tooling, this is way too much to be acceptable on my side since > I'm sadly not paid to work on beam (one day maybe ;)). > > Romain Manni-Bucau > @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book > > > 2018-04-10 18:27 GMT+02:00 Reuven Lax <re...@google.com>: > > Romain, > > > > Can you detail what's not working. I switched my IntelliJ over to Gradle > > about two weeks ago, and haven't had any trouble. > > > > Reuven > > > > On Tue, Apr 10, 2018 at 4:20 PM Romain Manni-Bucau < > rmannibu...@gmail.com> > > wrote: > >> > >> Ok, didn't find a way to make it working properly (only workaround > >> with direct commands and no good idea integration for debugging). I'm > >> back with maven, if anyone knows how to properly solve it let's do it. > >> If not I think JB point is to consider more than any other criteria. > >> > >> Romain Manni-Bucau > >> @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book > >> > >> > >> 2018-04-10 16:41 GMT+02:00 Romain Manni-Bucau <rmannibu...@gmail.com>: > >> > side note: do NOT use auto-import until you are sure you can, it locks > >> > regularly on beam (pby too big for idea?) and makes idea ready to be > >> > killed :( > >> > > >> > Romain Manni-Bucau > >> > @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book > >> > > >> > > >> > 2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré <j...@nanthrax.net>: > >> >> It's what I did, I'm trying a complete reload now (maybe this step > >> >> failed). > >> >> > >> >> On 10/04/2018 16:38, Lukasz Cwik wrote: > >> >>> > >> >>> beam-site PR/414 updates the instructions for using Intellij and how > >> >>> to > >> >>> import a module: > >> >>> > >> >>> 1. Create an empty IntelliJ project outside of the Beam source tree. > >> >>> 2. Under Project Structure > Project, select a Project SDK. > >> >>> 3. Under Project Structure > Modules, click the + sign to add a > module > >> >>> and > >> >>> select "Import Module". > >> >>> 1. Select the directory containing the Beam source tree. > >> >>> 2. Tick the "Import module from external model" button and > select > >> >>> Gradle > >> >>> from the list. > >> >>> 3. Tick the following boxes. > >> >>> * Use auto-import > >> >>> * Create separate module per source set > >> >>> * Store generated project files externally > >> >>> * Use default gradle wrapper > >> >>> 4. Delegate build actions to Gradle by going to Settings > Build, > >> >>> Execution, > >> >>> Deployment > Build Tools > Gradle and checking "Delegate IDE > >> >>> build/run > >> >>> actions to gradle". > >> >>> > >> >>> On Tue, Apr 10, 2018 at 10:34 AM Jean-Baptiste Onofré < > j...@nanthrax.net > >> >>> <mailto:j...@nanthrax.net>> wrote: > >> >>> > >> >>> That's a very important issue for contribution. Up to now, I > used > >> >>> Maven > >> >>> for setup IntelliJ (and it works just fine). If we remove the > >> >>> pom.xml, > >> >>> we have to support Eclipse and IntelliJ "smoothly". > >> >>> > >> >>> Let me try in IntelliJ. > >> >>> > >> >>> Regards > >> >>> JB > >> >>> > >> >>> On 10/04/2018 15:21, Romain Manni-Bucau wrote: > >> >>> > You dont have issue due to the build setup with that option. > I > >> >>> get: > >> >>> > > >> >>> > avr. 10, 2018 3:20:10 PM > >> >>> > org.apache.beam.runners.direct.DirectTransformExecutor run > >> >>> > GRAVE: Error occurred within > >> >>> > org.apache.beam.runners.direct.DirectTransformExecutor@ > 66761b7a > >> >>> > com.google.common.util.concurrent.ExecutionError: > >> >>> > java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy > >> >>> > > >> >>> > ? > >> >>> > > >> >>> > Romain Manni-Bucau > >> >>> > @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book > >> >>> > > >> >>> > > >> >>> > 2018-04-10 15:13 GMT+02:00 Lukasz Cwik <lc...@google.com > >> >>> <mailto:lc...@google.com>>: > >> >>> >> I have found that the simplest setup is to delegate the > >> >>> build/test actions > >> >>> >> to Gradle. This allows you to run unit tests very easily and > >> >>> since its in > >> >>> >> the same manner that Gradle would have, you know that if its > >> >>> passing it will > >> >>> >> pass on the command line and on Jenkins. Here is one site > that > >> >>> discusses how > >> >>> >> to set this up: > >> >>> >> > >> >>> > >> >>> > >> >>> http://mrhaki.blogspot.com/2016/11/gradle-goodness- > delegate-build-and-run.html > >> >>> >> > >> >>> >> > >> >>> >> On Tue, Apr 10, 2018 at 8:45 AM Romain Manni-Bucau > >> >>> <rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>> > >> >>> >> wrote: > >> >>> >>> > >> >>> >>> What's the plan to make idea supporting gradle on beam > >> >>> project? > >> >>> Do we > >> >>> >>> import the workaround mentionned in > >> >>> >>> https://youtrack.jetbrains.com/issue/IDEA-175172? > >> >>> >>> For the ones who didn't see this issue in action: idea will > >> >>> compile in > >> >>> >>> out/ instead of build/ and you will just miss all the > >> >>> resources > >> >>> you > >> >>> >>> need like some SPI registration which are used by all our > >> >>> registrar => > >> >>> >>> no way to run tests in idea without hacking the > configuration > >> >>> quite > >> >>> >>> deeply :( > >> >>> >>> > >> >>> >>> Romain Manni-Bucau > >> >>> >>> @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book > >> >>> >>> > >> >>> >>> > >> >>> >>> 2018-04-10 10:08 GMT+02:00 Etienne Chauchot > >> >>> <echauc...@apache.org <mailto:echauc...@apache.org>>: > >> >>> > >> >>> >>>> As a gradle beginner, I could not agree more ! > >> >>> >>>> +1 > >> >>> >>>> Etienne > >> >>> >>>> Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste > Onofré a > >> >>> écrit : > >> >>> >>>> > >> >>> >>>> Hi all, > >> >>> >>>> > >> >>> >>>> I did multiple gradle build since last week and I would > like > >> >>> to share > >> >>> >>>> one of my concern: it's about the communities. > >> >>> >>>> > >> >>> >>>> If I think our users won't see any change for them due to > >> >>> Gradle build > >> >>> >>>> (I think that most of our users will still use Maven with > >> >>> artifacts > >> >>> >>>> provided by Gradle), I'm more concerned by the dev > community > >> >>> and the > >> >>> >>>> contribution. > >> >>> >>>> > >> >>> >>>> Maven is well known and straight forward for a large part > of > >> >>> potential > >> >>> >>>> contributors. I think we have to keep in mind that we > still > >> >>> have to grow > >> >>> >>>> up our contributors community. > >> >>> >>>> > >> >>> >>>> Today, maybe I'm wrong, but I have the feeling that gradle > >> >>> build is not > >> >>> >>>> straight forward (build.gradle includes > build_rules.gradle, > >> >>> gathering > >> >>> >>>> all taks all together). > >> >>> >>>> > >> >>> >>>> I would like to add a task in the gradle "migration" > >> >>> process: > >> >>> simplify > >> >>> >>>> the gradle structure and files, and document this. > >> >>> >>>> > >> >>> >>>> I know we already have a Jira about the documentation > part, > >> >>> but I would > >> >>> >>>> like to "polish" and use a clean structure for the Gradle > >> >>> resources. As > >> >>> >>>> already quickly discussed, I think that having one gradle > >> >>> file > >> >>> per tasks > >> >>> >>>> in the .gradle directory would be helpful. > >> >>> >>>> > >> >>> >>>> The goal is really to simplify the contribution. > >> >>> >>>> > >> >>> >>>> Do you agree if I add a Jira about "Gradle polish" ? > >> >>> >>>> Thoughts ? > >> >>> >>>> > >> >>> >>>> Regards > >> >>> >>>> JB > >> >>> >>>> > >> >>> >>>> On 07/04/2018 04:52, Scott Wegner wrote: > >> >>> >>>> > >> >>> >>>> Here's an end-of-day update on migration work: > >> >>> >>>> > >> >>> >>>> * Snapshot unsigned dailies and signed release builds are > >> >>> working (!!). > >> >>> >>>> PR/5048 [1] merges changes from Luke's branch > >> >>> >>>> * python precommit failing... will investigate python > >> >>> precommit > >> >>> >>>> Monday > >> >>> >>>> * All Precommits are gradle only > >> >>> >>>> * All Postcommits except performance tests and > >> >>> Java_JDK_Versions_Test > >> >>> >>>> use gradle (after PR/5047 [2] merged) > >> >>> >>>> * Nightly snapshot release using gradle is ready; needs > >> >>> PR/5048 to be > >> >>> >>>> merged before switching > >> >>> >>>> * ValidatesRunner_Spark failing consistently; > investigating > >> >>> >>>> > >> >>> >>>> Thanks for another productive day of hacking. I'll pick up > >> >>> again on > >> >>> >>>> Monday. > >> >>> >>>> > >> >>> >>>> [1] https://github.com/apache/beam/pull/5048 > >> >>> >>>> [2] https://github.com/apache/beam/pull/5047 > >> >>> >>>> > >> >>> >>>> > >> >>> >>>> On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau > >> >>> >>>> <rmannibu...@gmail.com <mailto:rmannibu...@gmail.com> > >> >>> <mailto:rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>>> > >> >>> wrote: > >> >>> >>>> > >> >>> >>>> Why building a zip per runner which its stack and > just > >> >>> pointing out > >> >>> >>>> on that zip and let beam lazy load the runner: > >> >>> >>>> > >> >>> >>>> --runner=LazyRunner --lazyRunnerDir=... > >> >>> --lazyRunnerOptions=... (or > >> >>> >>>> the fromSystemProperties() if it gets merged a day > ;)) > >> >>> >>>> > >> >>> >>>> Le 6 avr. 2018 20:21, "Kenneth Knowles" < > k...@google.com > >> >>> <mailto:k...@google.com> > >> >>> >>>> <mailto:k...@google.com <mailto:k...@google.com>>> a > >> >>> écrit : > >> >>> >>>> > >> >>> >>>> I'm working on finding a solution for launching > the > >> >>> Nexmark > >> >>> >>>> suite with each runner. This doesn't have to be > >> >>> done > >> >>> via Gradle, > >> >>> >>>> but we anyhow need built artifacts that don't > >> >>> require > >> >>> user > >> >>> >>>> classpath intervention. > >> >>> >>>> > >> >>> >>>> It looks to me like the examples are also missing > >> >>> this - they > >> >>> >>>> have separate configuration e.g. > >> >>> sparkRunnerPreCommit > >> >>> but that > >> >>> >>>> is overspecified compared to a free-form > launching > >> >>> of > >> >>> a main() > >> >>> >>>> program with a runner profile. > >> >>> >>>> > >> >>> >>>> On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik > >> >>> <lc...@google.com <mailto:lc...@google.com> > >> >>> >>>> <mailto:lc...@google.com > >> >>> <mailto:lc...@google.com>>> > >> >>> wrote: > >> >>> >>>> > >> >>> >>>> Romain, are you talking about the profiles > that > >> >>> exist as > >> >>> >>>> part of the archetype examples? > >> >>> >>>> > >> >>> >>>> If so, then those still exist and haven't > been > >> >>> changed. If > >> >>> >>>> not, can you provide a link to the profile > in a > >> >>> pom file to > >> >>> >>>> be clearer? > >> >>> >>>> > >> >>> >>>> On Fri, Apr 6, 2018 at 12:40 PM Romain > >> >>> Manni-Bucau > >> >>> >>>> <rmannibu...@gmail.com > >> >>> <mailto:rmannibu...@gmail.com> <mailto:rmannibu...@gmail.com > >> >>> <mailto:rmannibu...@gmail.com>>> > >> >>> >>>> wrote: > >> >>> >>>> > >> >>> >>>> Hi Scott, > >> >>> >>>> > >> >>> >>>> is it right that 2 doesn't handle the > >> >>> hierachy anymore > >> >>> >>>> and that it doesn't handle profiles for > >> >>> runners as it is > >> >>> >>>> currently with maven? > >> >>> >>>> > >> >>> >>>> > >> >>> >>>> Romain Manni-Bucau > >> >>> >>>> @rmannibucau > >> >>> <https://twitter.com/rmannibucau> | Blog > >> >>> >>>> <https://rmannibucau.metawerx.net/> | > Old > >> >>> Blog > >> >>> >>>> <http://rmannibucau.wordpress.com> | > Github > >> >>> >>>> <https://github.com/rmannibucau> | > LinkedIn > >> >>> >>>> <https://www.linkedin.com/in/rmannibucau> > | > >> >>> Book > >> >>> >>>> > >> >>> >>>> > >> >>> >>>> > >> >>> > >> >>> > >> >>> <https://www.packtpub.com/application-development/java- > ee-8-high-performance> > >> >>> >>>> > >> >>> >>>> 2018-04-06 18:32 GMT+02:00 Scott Wegner > >> >>> >>>> <sweg...@google.com > >> >>> <mailto:sweg...@google.com> <mailto:sweg...@google.com > >> >>> > >> >>> <mailto:sweg...@google.com>>>: > >> >>> >>>> > >> >>> >>>> I wanted to start a thread to > summarize > >> >>> the current > >> >>> >>>> state of Gradle migration. We've made > >> >>> lots of good > >> >>> >>>> progress so far this week. Here's the > >> >>> status from > >> >>> >>>> what I can tell-- please add or > correct > >> >>> anything I > >> >>> >>>> missed: > >> >>> >>>> > >> >>> >>>> * Release artifacts can be built and > >> >>> published for > >> >>> >>>> Snapshot and officlal releases [1] > >> >>> >>>> * Gradle-generated releases have been > >> >>> validated with > >> >>> >>>> the the Apache Beam archetype > >> >>> generation > >> >>> quickstart; > >> >>> >>>> still needs additional validation. > >> >>> >>>> * Generated release pom files have > >> >>> correct project > >> >>> >>>> metadata [2] > >> >>> >>>> * The python pre-commits are now > >> >>> working > >> >>> in Gradle > >> >>> >>>> [3] > >> >>> >>>> * Ismaël has started a collaborative > >> >>> doc > >> >>> of Gradle > >> >>> >>>> tips [4] as we all learn the new > >> >>> system-- > >> >>> please add > >> >>> >>>> your own. This will eventually feed > >> >>> into > >> >>> official > >> >>> >>>> documentation on the website. > >> >>> >>>> * Łukasz Gajowy is working on > migrating > >> >>> performance > >> >>> >>>> testing framework [5] > >> >>> >>>> * Daniel is working on updating > >> >>> documentation to > >> >>> >>>> refer to Gradle instead of maven > >> >>> >>>> > >> >>> >>>> If I missed anything, please add it > to > >> >>> this thread. > >> >>> >>>> > >> >>> >>>> The general roadmap we're working > >> >>> towards > >> >>> is: > >> >>> >>>> (a) Publish release artifacts with > >> >>> Gradle > >> >>> (SNAPSHOT > >> >>> >>>> and signed releases) > >> >>> >>>> (b) Postcommits migrated to Gradle > >> >>> >>>> (c) Migrate documentation from maven > to > >> >>> Gradle > >> >>> >>>> (d) Migrate perfkit suites to use > >> >>> Gradle > >> >>> >>>> > >> >>> >>>> For those of you that are hacking: > >> >>> thanks > >> >>> for your > >> >>> >>>> help so far! Progress is being > roughly > >> >>> tracked on > >> >>> >>>> the Kanban [6]; please make sure the > >> >>> issues assigned > >> >>> >>>> to you are up-to-date. Many of the > >> >>> changes are > >> >>> >>>> staged on lukecwik's local branch > [7]; > >> >>> we'll work on > >> >>> >>>> merging them back soon. > >> >>> >>>> > >> >>> >>>> > >> >>> >>>> [1] > >> >>> >>>> https://github.com/lukecwik/incubator-beam/pull/7 > >> >>> >>>> [2] > >> >>> >>>> https://github.com/lukecwik/incubator-beam/pull/3 > >> >>> >>>> [3] > >> >>> https://github.com/apache/beam/pull/5032 > >> >>> >>>> [4] > >> >>> >>>> > >> >>> >>>> > >> >>> >>>> > >> >>> > >> >>> > >> >>> https://docs.google.com/document/d/1wR56Jef3XIPwj4DFzQKznuGPM3JDf > RDVkxzeDlbdVSQ/edit > >> >>> >>>> [5] > >> >>> https://github.com/apache/beam/pull/5003 > >> >>> >>>> [6] > >> >>> >>>> > >> >>> >>>> > >> >>> > >> >>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=242 > >> >>> >>>> > >> >>> >>>> [7] > >> >>> >>>> > >> >>> >>>> https://github.com/lukecwik/incubator-beam/tree/gradle > >> >>> >>>> -- > >> >>> >>>> > >> >>> >>>> > >> >>> >>>> Got feedback? > >> >>> http://go/swegner-feedback > >> >>> >>>> > >> >>> >>>> > >> >>> > >> >> >
