oh i found some stuff about tests and how to continue them, gonna try that
now (-fae switch). should have googled before asking...
On Fri, Oct 24, 2014 at 3:59 PM, Koert Kuipers <ko...@tresata.com> wrote:
> thanks ted.
>
> apologies for complaining about maven here again, but this is the first
> time i seriously use it for development, and i am completely unfamiliar
> with it.
>
> a few more issues:
>
> "mvn clean package -DskipTests" takes about 30 mins for me. thats painful
> since its needed for the tests. does anyone know any tricks to speed it up?
> (besides getting a better laptop). does zinc help?
>
> does scalastyle overwrite files? after i do "mvn package" emacs is
> completely confused and for every file it says "it has been edited" and i
> need to re-open it. not helpful for a development cycle. i think i am
> simply going to edit the pom and remove scalacheck for development.
>
> mvn test runs through the projects until one fails. then it skips the
> rest! since its very likely that i get a failure in some subproject, this
> means its nearly impossible to do a general test run and get a good sense
> of the status of the project. for example:
>
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Spark Project Parent POM .......................... SUCCESS [2.199s]
> [INFO] Spark Project Core ................................ SUCCESS
> [39:43.028s]
> [INFO] Spark Project Bagel ............................... SUCCESS
> [42.569s]
> [INFO] Spark Project GraphX .............................. SUCCESS
> [3:22.104s]
> [INFO] Spark Project Streaming ........................... SUCCESS
> [7:12.592s]
> [INFO] Spark Project ML Library .......................... SUCCESS
> [10:32.682s]
> [INFO] Spark Project Tools ............................... SUCCESS
> [17.070s]
> [INFO] Spark Project Catalyst ............................ SUCCESS
> [3:03.470s]
> [INFO] Spark Project SQL ................................. SUCCESS
> [5:23.993s]
> [INFO] Spark Project Hive ................................ FAILURE
> [2:08.387s]
> [INFO] Spark Project REPL ................................ SKIPPED
> [INFO] Spark Project Assembly ............................ SKIPPED
> [INFO] Spark Project External Twitter .................... SKIPPED
> [INFO] Spark Project External Kafka ...................... SKIPPED
> [INFO] Spark Project External Flume Sink ................. SKIPPED
> [INFO] Spark Project External Flume ...................... SKIPPED
> [INFO] Spark Project External ZeroMQ ..................... SKIPPED
> [INFO] Spark Project External MQTT ....................... SKIPPED
> [INFO] Spark Project Examples ............................ SKIPPED
>
> in this case i dont care about Hive, but i would have liked to see REPL
> run, and Kafka.
>
>
>
>
> On Thu, Oct 23, 2014 at 4:44 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>
>> Created SPARK-4066 and attached patch there.
>>
>> On Thu, Oct 23, 2014 at 1:07 PM, Koert Kuipers <ko...@tresata.com> wrote:
>>
>>> great thanks i will do that
>>>
>>> On Thu, Oct 23, 2014 at 3:55 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>>>
>>>> Koert:
>>>> If you have time, you can try this diff - with which you would be able
>>>> to specify the following on the command line:
>>>> -Dscalastyle.failonviolation=false
>>>>
>>>> diff --git a/pom.xml b/pom.xml
>>>> index 687cc63..108585e 100644
>>>> --- a/pom.xml
>>>> +++ b/pom.xml
>>>> @@ -123,6 +123,7 @@
>>>> <log4j.version>1.2.17</log4j.version>
>>>> <hadoop.version>1.0.4</hadoop.version>
>>>> <protobuf.version>2.4.1</protobuf.version>
>>>> + <scalastyle.failonviolation>true</scalastyle.failonviolation>
>>>> <yarn.version>${hadoop.version}</yarn.version>
>>>> <hbase.version>0.94.6</hbase.version>
>>>> <flume.version>1.4.0</flume.version>
>>>> @@ -1071,7 +1072,7 @@
>>>> <version>0.4.0</version>
>>>> <configuration>
>>>> <verbose>false</verbose>
>>>> - <failOnViolation>true</failOnViolation>
>>>> +
>>>> <failOnViolation>${scalastyle.failonviolation}</failOnViolation>
>>>>
>>>> <includeTestSourceDirectory>false</includeTestSourceDirectory>
>>>> <failOnWarning>false</failOnWarning>
>>>> <sourceDirectory>${basedir}/src/main/scala</sourceDirectory>
>>>>
>>>>
>>>>
>>>> On Thu, Oct 23, 2014 at 12:07 PM, Koert Kuipers <ko...@tresata.com>
>>>> wrote:
>>>>
>>>>> Hey Ted,
>>>>> i tried:
>>>>> mvn clean package -DskipTests -Dscalastyle.failOnViolation=false
>>>>>
>>>>> no luck, still get
>>>>> [ERROR] Failed to execute goal
>>>>> org.scalastyle:scalastyle-maven-plugin:0.4.0:check (default) on project
>>>>> spark-core_2.10: Failed during scalastyle execution: You have 3 Scalastyle
>>>>> violation(s). -> [Help 1]
>>>>>
>>>>>
>>>>> On Thu, Oct 23, 2014 at 2:14 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>>>>>
>>>>>> Koert:
>>>>>> Have you tried adding the following on your commandline ?
>>>>>>
>>>>>> -Dscalastyle.failOnViolation=false
>>>>>>
>>>>>> Cheers
>>>>>>
>>>>>> On Thu, Oct 23, 2014 at 11:07 AM, Patrick Wendell <pwend...@gmail.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Hey Koert,
>>>>>>>
>>>>>>> I think disabling the style checks in maven package could be a good
>>>>>>> idea for the reason you point out. I was sort of mixed on that when
>>>>>>> it
>>>>>>> was proposed for this exact reason. It's just annoying to developers.
>>>>>>>
>>>>>>> In terms of changing the global limit, this is more religion than
>>>>>>> anything else, but there are other cases where the current limit is
>>>>>>> useful (e.g. if you have many windows open in a large screen).
>>>>>>>
>>>>>>> - Patrick
>>>>>>>
>>>>>>> On Thu, Oct 23, 2014 at 11:03 AM, Koert Kuipers <ko...@tresata.com>
>>>>>>> wrote:
>>>>>>> > 100 max width seems very restrictive to me.
>>>>>>> >
>>>>>>> > even the most restrictive environment i have for development (ssh
>>>>>>> with
>>>>>>> > emacs) i get a lot more characters to work with than that.
>>>>>>> >
>>>>>>> > personally i find the code harder to read, not easier. like i kept
>>>>>>> > wondering why there are weird newlines in the
>>>>>>> > middle of constructors and such, only to realise later it was
>>>>>>> because of
>>>>>>> > the 100 character limit.
>>>>>>> >
>>>>>>> > also, i find "mvn package" erroring out because of style errors
>>>>>>> somewhat
>>>>>>> > excessive. i understand that a pull request needs to conform to
>>>>>>> "the style"
>>>>>>> > before being accepted, but this means i cant even run tests on
>>>>>>> code that
>>>>>>> > does not conform to the style guide, which is a bit silly.
>>>>>>> >
>>>>>>> > i keep going out for coffee while package and tests run, only to
>>>>>>> come back
>>>>>>> > for an annoying error that my line is 101 characters and therefore
>>>>>>> nothing
>>>>>>> > ran.
>>>>>>> >
>>>>>>> > is there some maven switch to disable the style checks?
>>>>>>> >
>>>>>>> > best! koert
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
>>>>>>> For additional commands, e-mail: dev-h...@spark.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
