thanks ted.
apologies for complaining about maven here again, but this is the first
time i seriously use it for development, and i am completely unfamiliar
with it.
a few more issues:
"mvn clean package -DskipTests" takes about 30 mins for me. thats painful
since its needed for the tests. does anyone know any tricks to speed it up?
(besides getting a better laptop). does zinc help?
does scalastyle overwrite files? after i do "mvn package" emacs is
completely confused and for every file it says "it has been edited" and i
need to re-open it. not helpful for a development cycle. i think i am
simply going to edit the pom and remove scalacheck for development.
mvn test runs through the projects until one fails. then it skips the rest!
since its very likely that i get a failure in some subproject, this means
its nearly impossible to do a general test run and get a good sense of the
status of the project. for example:
[INFO]
------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Spark Project Parent POM .......................... SUCCESS [2.199s]
[INFO] Spark Project Core ................................ SUCCESS
[39:43.028s]
[INFO] Spark Project Bagel ............................... SUCCESS [42.569s]
[INFO] Spark Project GraphX .............................. SUCCESS
[3:22.104s]
[INFO] Spark Project Streaming ........................... SUCCESS
[7:12.592s]
[INFO] Spark Project ML Library .......................... SUCCESS
[10:32.682s]
[INFO] Spark Project Tools ............................... SUCCESS [17.070s]
[INFO] Spark Project Catalyst ............................ SUCCESS
[3:03.470s]
[INFO] Spark Project SQL ................................. SUCCESS
[5:23.993s]
[INFO] Spark Project Hive ................................ FAILURE
[2:08.387s]
[INFO] Spark Project REPL ................................ SKIPPED
[INFO] Spark Project Assembly ............................ SKIPPED
[INFO] Spark Project External Twitter .................... SKIPPED
[INFO] Spark Project External Kafka ...................... SKIPPED
[INFO] Spark Project External Flume Sink ................. SKIPPED
[INFO] Spark Project External Flume ...................... SKIPPED
[INFO] Spark Project External ZeroMQ ..................... SKIPPED
[INFO] Spark Project External MQTT ....................... SKIPPED
[INFO] Spark Project Examples ............................ SKIPPED
in this case i dont care about Hive, but i would have liked to see REPL
run, and Kafka.
On Thu, Oct 23, 2014 at 4:44 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> Created SPARK-4066 and attached patch there.
>
> On Thu, Oct 23, 2014 at 1:07 PM, Koert Kuipers <ko...@tresata.com> wrote:
>
>> great thanks i will do that
>>
>> On Thu, Oct 23, 2014 at 3:55 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>>
>>> Koert:
>>> If you have time, you can try this diff - with which you would be able
>>> to specify the following on the command line:
>>> -Dscalastyle.failonviolation=false
>>>
>>> diff --git a/pom.xml b/pom.xml
>>> index 687cc63..108585e 100644
>>> --- a/pom.xml
>>> +++ b/pom.xml
>>> @@ -123,6 +123,7 @@
>>> <log4j.version>1.2.17</log4j.version>
>>> <hadoop.version>1.0.4</hadoop.version>
>>> <protobuf.version>2.4.1</protobuf.version>
>>> + <scalastyle.failonviolation>true</scalastyle.failonviolation>
>>> <yarn.version>${hadoop.version}</yarn.version>
>>> <hbase.version>0.94.6</hbase.version>
>>> <flume.version>1.4.0</flume.version>
>>> @@ -1071,7 +1072,7 @@
>>> <version>0.4.0</version>
>>> <configuration>
>>> <verbose>false</verbose>
>>> - <failOnViolation>true</failOnViolation>
>>> +
>>> <failOnViolation>${scalastyle.failonviolation}</failOnViolation>
>>> <includeTestSourceDirectory>false</includeTestSourceDirectory>
>>> <failOnWarning>false</failOnWarning>
>>> <sourceDirectory>${basedir}/src/main/scala</sourceDirectory>
>>>
>>>
>>>
>>> On Thu, Oct 23, 2014 at 12:07 PM, Koert Kuipers <ko...@tresata.com>
>>> wrote:
>>>
>>>> Hey Ted,
>>>> i tried:
>>>> mvn clean package -DskipTests -Dscalastyle.failOnViolation=false
>>>>
>>>> no luck, still get
>>>> [ERROR] Failed to execute goal
>>>> org.scalastyle:scalastyle-maven-plugin:0.4.0:check (default) on project
>>>> spark-core_2.10: Failed during scalastyle execution: You have 3 Scalastyle
>>>> violation(s). -> [Help 1]
>>>>
>>>>
>>>> On Thu, Oct 23, 2014 at 2:14 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>>>>
>>>>> Koert:
>>>>> Have you tried adding the following on your commandline ?
>>>>>
>>>>> -Dscalastyle.failOnViolation=false
>>>>>
>>>>> Cheers
>>>>>
>>>>> On Thu, Oct 23, 2014 at 11:07 AM, Patrick Wendell <pwend...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hey Koert,
>>>>>>
>>>>>> I think disabling the style checks in maven package could be a good
>>>>>> idea for the reason you point out. I was sort of mixed on that when it
>>>>>> was proposed for this exact reason. It's just annoying to developers.
>>>>>>
>>>>>> In terms of changing the global limit, this is more religion than
>>>>>> anything else, but there are other cases where the current limit is
>>>>>> useful (e.g. if you have many windows open in a large screen).
>>>>>>
>>>>>> - Patrick
>>>>>>
>>>>>> On Thu, Oct 23, 2014 at 11:03 AM, Koert Kuipers <ko...@tresata.com>
>>>>>> wrote:
>>>>>> > 100 max width seems very restrictive to me.
>>>>>> >
>>>>>> > even the most restrictive environment i have for development (ssh
>>>>>> with
>>>>>> > emacs) i get a lot more characters to work with than that.
>>>>>> >
>>>>>> > personally i find the code harder to read, not easier. like i kept
>>>>>> > wondering why there are weird newlines in the
>>>>>> > middle of constructors and such, only to realise later it was
>>>>>> because of
>>>>>> > the 100 character limit.
>>>>>> >
>>>>>> > also, i find "mvn package" erroring out because of style errors
>>>>>> somewhat
>>>>>> > excessive. i understand that a pull request needs to conform to
>>>>>> "the style"
>>>>>> > before being accepted, but this means i cant even run tests on code
>>>>>> that
>>>>>> > does not conform to the style guide, which is a bit silly.
>>>>>> >
>>>>>> > i keep going out for coffee while package and tests run, only to
>>>>>> come back
>>>>>> > for an annoying error that my line is 101 characters and therefore
>>>>>> nothing
>>>>>> > ran.
>>>>>> >
>>>>>> > is there some maven switch to disable the style checks?
>>>>>> >
>>>>>> > best! koert
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
>>>>>> For additional commands, e-mail: dev-h...@spark.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
