thank you!
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/What-s-the-use-of-RangePartitioner-hashCode-tp18953p19037.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
--
We have too many (large) files. We have about 30k partitions with about 4
years worth data and we need to process entire history in a one time
monolithic job.
I would like to know how spark decides the number of executors requested.
I've seen testcases where the max executors count is Integer's M
Hi Reynold,
That's my point indeed. If non-determinism is a trait/property of a
class (objects really at runtime) it's by definition part of a trait
and only Nondeterministic expressions are...well...non-deterministic.
I'm yet to review the code regarding the trait, but I suspect that
there are p
+1 (non binding)
I compiled and tested on the following two systems.
- CentOS 7.2 / Oracle JDK 1.8.0_77 / R 3.3.1 with -Pyarn -Phadoop-2.7
-Pkinesis-asl -Phive -Phive-thriftserver -Dsparkr
- CentOS 7.2 / Open JDK 1.8.0_102 with -Pyarn -Phadoop-2.7 -Pkinesis-asl
-Phive -Phive-thriftserver
Bests,
Hi,
I keep asking myself why are you guys not including -Pmesos in your
builds? Is this on purpose or have you overlooked it?
Pozdrawiam,
Jacek Laskowski
https://medium.com/@jaceklaskowski/
Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark
Follow me at https://twitter.com/jacek
The binary artifact that's published here is built with -Pmesos. The
'real' artifact from a process standpoint is the source release,
however. That's why we do (should) test the source release foremost.
I suppose individuals are invited to test with a configuration that is
of interest to them, and
Hi all,
The R API documentation version error was reported in a separate thread.
I've built a release candidate (RC3) and will send out a new vote email in
a bit.
On Thu, Sep 22, 2016 at 11:01 PM, Reynold Xin wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 2.
Please vote on releasing the following candidate as Apache Spark version
2.0.1. The vote is open until Tue, Sep 27, 2016 at 15:30 PDT and passes if
a majority of at least 3+1 PMC votes are cast.
[ ] +1 Release this package as Apache Spark 2.0.1
[ ] -1 Do not release this package because ...
The
Thanks for reporting. I sent out an email for rc3 fixing the issue.
We have also automated the version number update for documentation pages so
this won't happen again in the future.
On Fri, Sep 23, 2016 at 12:25 AM, Jagadeesan As wrote:
> Hi Reyonld,
>
> While checking the documentation for '
There is no "mesos" profile in 2.0.1.
On Sat, Sep 24, 2016 at 2:19 PM, Jacek Laskowski wrote:
> Hi,
>
> I keep asking myself why are you guys not including -Pmesos in your
> builds? Is this on purpose or have you overlooked it?
>
> Pozdrawiam,
> Jacek Laskowski
>
> https://medium.com/@jacekl
10 matches
Mail list logo