Thank for your advise, I have tried your recommended configuration, but
SPARK_WORKER_CORES=32
SPARK_WORKER_INSTANCES=1
still work better, in the offical spark-standalone, about the parameter
'SPARK_WORKER_INSTANCES' /You can make this more than 1 if you have have
very large machines and would
On Thu, Sep 4, 2014 at 4:19 PM, shane knapp skn...@berkeley.edu wrote:
on a side note, this incident will be accelerating our plan to move the
entire jenkins infrastructure in to a managed datacenter environment.
this
will be our major push over the next couple of weeks. more details about
How early can MiMa checks be run? Before Spark is even built
https://github.com/apache/spark/blob/8cc70e7e15fd800f31b94e9102069506360289db/dev/run-tests#L118?
After the build but before the unit tests?
On Thu, Sep 25, 2014 at 6:06 PM, Patrick Wendell pwend...@gmail.com wrote:
Yeah we can also
as of this morning, i've got the new jenkins up, with all of the current
builds set up (but failing). i'm in the middle of playing setup/debug
whack-a-mole, but we're getting there. my guess would be early next week
for the switchover.
On Wed, Oct 1, 2014 at 12:53 PM, Nicholas Chammas
Please take a look at WhitespaceEndOfLineChecker under:
http://www.scalastyle.org/rules-0.1.0.html
Cheers
On Wed, Oct 1, 2014 at 2:01 PM, Nicholas Chammas nicholas.cham...@gmail.com
wrote:
As discussed here https://github.com/apache/spark/pull/2619, it would be
good to extend our Scala style
Hey Nick,
We can always take built-in rules. Back when we added this Prashant
Sharma actually did some great work that lets us write our own style
rules in cases where rules don't exist.
You can see some existing rules here:
Ah, since there appears to be a built-in rule for end-of-line whitespace,
Michael and Cheng, y'all should be able to add this in pretty easily.
Nick
On Wed, Oct 1, 2014 at 6:37 PM, Patrick Wendell pwend...@gmail.com wrote:
Hey Nick,
We can always take built-in rules. Back when we added this
The hard part here is updating the existing code base... which is going to
create merge conflicts with like all of the open PRs...
On Wed, Oct 1, 2014 at 6:13 PM, Nicholas Chammas nicholas.cham...@gmail.com
wrote:
Ah, since there appears to be a built-in rule for end-of-line whitespace,
Yeah, I remember that hell when I added PEP 8 to the build checks and fixed
all the outstanding Python style issues. I had to keep rebasing and
resolving merge conflicts until the PR was merged.
It's a rough process, but thankfully it's also a one-time process. I might
be able to help with that
Does anyone know if Scala has something equivalent to autopep8
https://pypi.python.org/pypi/autopep8? It would help patch up the
existing code base a lot quicker as we add in new style rules.
On Wed, Oct 1, 2014 at 9:24 PM, Nicholas Chammas nicholas.cham...@gmail.com
wrote:
Yeah, I remember
There is scalariform but it can be disruptive. Last time I ran it on Spark
it didn't compile due to some xml interpolation problem.
On Wednesday, October 1, 2014, Nicholas Chammas nicholas.cham...@gmail.com
wrote:
Does anyone know if Scala has something equivalent to autopep8
Since we can easily catch the list of all changed files in a PR, I think
we can start with adding the no trailing space check for newly changed
files only?
On 10/2/14 9:24 AM, Nicholas Chammas wrote:
Yeah, I remember that hell when I added PEP 8 to the build checks and fixed
all the
12 matches
Mail list logo
