thanks ted. apologies for complaining about maven here again, but this is the first time i seriously use it for development, and i am completely unfamiliar with it.
a few more issues: "mvn clean package -DskipTests" takes about 30 mins for me. thats painful since its needed for the tests. does anyone know any tricks to speed it up? (besides getting a better laptop). does zinc help? does scalastyle overwrite files? after i do "mvn package" emacs is completely confused and for every file it says "it has been edited" and i need to re-open it. not helpful for a development cycle. i think i am simply going to edit the pom and remove scalacheck for development. mvn test runs through the projects until one fails. then it skips the rest! since its very likely that i get a failure in some subproject, this means its nearly impossible to do a general test run and get a good sense of the status of the project. for example: [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Spark Project Parent POM .......................... SUCCESS [2.199s] [INFO] Spark Project Core ................................ SUCCESS [39:43.028s] [INFO] Spark Project Bagel ............................... SUCCESS [42.569s] [INFO] Spark Project GraphX .............................. SUCCESS [3:22.104s] [INFO] Spark Project Streaming ........................... SUCCESS [7:12.592s] [INFO] Spark Project ML Library .......................... SUCCESS [10:32.682s] [INFO] Spark Project Tools ............................... SUCCESS [17.070s] [INFO] Spark Project Catalyst ............................ SUCCESS [3:03.470s] [INFO] Spark Project SQL ................................. SUCCESS [5:23.993s] [INFO] Spark Project Hive ................................ FAILURE [2:08.387s] [INFO] Spark Project REPL ................................ SKIPPED [INFO] Spark Project Assembly ............................ SKIPPED [INFO] Spark Project External Twitter .................... SKIPPED [INFO] Spark Project External Kafka ...................... SKIPPED [INFO] Spark Project External Flume Sink ................. SKIPPED [INFO] Spark Project External Flume ...................... SKIPPED [INFO] Spark Project External ZeroMQ ..................... SKIPPED [INFO] Spark Project External MQTT ....................... SKIPPED [INFO] Spark Project Examples ............................ SKIPPED in this case i dont care about Hive, but i would have liked to see REPL run, and Kafka. On Thu, Oct 23, 2014 at 4:44 PM, Ted Yu <yuzhih...@gmail.com> wrote: > Created SPARK-4066 and attached patch there. > > On Thu, Oct 23, 2014 at 1:07 PM, Koert Kuipers <ko...@tresata.com> wrote: > >> great thanks i will do that >> >> On Thu, Oct 23, 2014 at 3:55 PM, Ted Yu <yuzhih...@gmail.com> wrote: >> >>> Koert: >>> If you have time, you can try this diff - with which you would be able >>> to specify the following on the command line: >>> -Dscalastyle.failonviolation=false >>> >>> diff --git a/pom.xml b/pom.xml >>> index 687cc63..108585e 100644 >>> --- a/pom.xml >>> +++ b/pom.xml >>> @@ -123,6 +123,7 @@ >>> <log4j.version>1.2.17</log4j.version> >>> <hadoop.version>1.0.4</hadoop.version> >>> <protobuf.version>2.4.1</protobuf.version> >>> + <scalastyle.failonviolation>true</scalastyle.failonviolation> >>> <yarn.version>${hadoop.version}</yarn.version> >>> <hbase.version>0.94.6</hbase.version> >>> <flume.version>1.4.0</flume.version> >>> @@ -1071,7 +1072,7 @@ >>> <version>0.4.0</version> >>> <configuration> >>> <verbose>false</verbose> >>> - <failOnViolation>true</failOnViolation> >>> + >>> <failOnViolation>${scalastyle.failonviolation}</failOnViolation> >>> <includeTestSourceDirectory>false</includeTestSourceDirectory> >>> <failOnWarning>false</failOnWarning> >>> <sourceDirectory>${basedir}/src/main/scala</sourceDirectory> >>> >>> >>> >>> On Thu, Oct 23, 2014 at 12:07 PM, Koert Kuipers <ko...@tresata.com> >>> wrote: >>> >>>> Hey Ted, >>>> i tried: >>>> mvn clean package -DskipTests -Dscalastyle.failOnViolation=false >>>> >>>> no luck, still get >>>> [ERROR] Failed to execute goal >>>> org.scalastyle:scalastyle-maven-plugin:0.4.0:check (default) on project >>>> spark-core_2.10: Failed during scalastyle execution: You have 3 Scalastyle >>>> violation(s). -> [Help 1] >>>> >>>> >>>> On Thu, Oct 23, 2014 at 2:14 PM, Ted Yu <yuzhih...@gmail.com> wrote: >>>> >>>>> Koert: >>>>> Have you tried adding the following on your commandline ? >>>>> >>>>> -Dscalastyle.failOnViolation=false >>>>> >>>>> Cheers >>>>> >>>>> On Thu, Oct 23, 2014 at 11:07 AM, Patrick Wendell <pwend...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hey Koert, >>>>>> >>>>>> I think disabling the style checks in maven package could be a good >>>>>> idea for the reason you point out. I was sort of mixed on that when it >>>>>> was proposed for this exact reason. It's just annoying to developers. >>>>>> >>>>>> In terms of changing the global limit, this is more religion than >>>>>> anything else, but there are other cases where the current limit is >>>>>> useful (e.g. if you have many windows open in a large screen). >>>>>> >>>>>> - Patrick >>>>>> >>>>>> On Thu, Oct 23, 2014 at 11:03 AM, Koert Kuipers <ko...@tresata.com> >>>>>> wrote: >>>>>> > 100 max width seems very restrictive to me. >>>>>> > >>>>>> > even the most restrictive environment i have for development (ssh >>>>>> with >>>>>> > emacs) i get a lot more characters to work with than that. >>>>>> > >>>>>> > personally i find the code harder to read, not easier. like i kept >>>>>> > wondering why there are weird newlines in the >>>>>> > middle of constructors and such, only to realise later it was >>>>>> because of >>>>>> > the 100 character limit. >>>>>> > >>>>>> > also, i find "mvn package" erroring out because of style errors >>>>>> somewhat >>>>>> > excessive. i understand that a pull request needs to conform to >>>>>> "the style" >>>>>> > before being accepted, but this means i cant even run tests on code >>>>>> that >>>>>> > does not conform to the style guide, which is a bit silly. >>>>>> > >>>>>> > i keep going out for coffee while package and tests run, only to >>>>>> come back >>>>>> > for an annoying error that my line is 101 characters and therefore >>>>>> nothing >>>>>> > ran. >>>>>> > >>>>>> > is there some maven switch to disable the style checks? >>>>>> > >>>>>> > best! koert >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >>>>>> For additional commands, e-mail: dev-h...@spark.apache.org >>>>>> >>>>>> >>>>> >>>> >>> >> >