Hi Matei,
3.2 will also include support for pushed based shuffle (spip SPARK-30602).
Regards,
Mridul
On Mon, Aug 9, 2021 at 9:26 PM Hyukjin Kwon wrote:
> > Are you referring to what version of Koala project? 1.8.1?
>
> Yes, the latest version 1.8.1.
>
> 2021년 8월 10일 (화) 오전 11:07, Igor Costa
> Are you referring to what version of Koala project? 1.8.1?
Yes, the latest version 1.8.1.
2021년 8월 10일 (화) 오전 11:07, Igor Costa 님이 작성:
> Hi Matei, nice update
>
>
> Just one question, when you mention “ We are working on Spark 3.2.0 as
> our next release, with a release candidate likely to
There is an SPIP passed and ready for Spark 3.2:
pandas API on Spark:
- JIRA: SPIP: Support pandas API layer on PySpark (
https://issues.apache.org/jira/browse/SPARK-34849)
- Vote: [VOTE] SPIP: Support pandas API layer on PySpark (
https://www.mail-archive.com/dev@spark.apache.org/msg27605.html)
Hi Matei, nice update
Just one question, when you mention “ We are working on Spark 3.2.0 as our
next release, with a release candidate likely to come soon. Spark 3.2
includes a new Pandas API for Apache Spark based on the Koalas project”
Are you referring to what version of Koala project?
It’s time for our quarterly report to the ASF board, which we need to send out
this Wednesday. I wrote the draft below based on community activity — let me
know if you’d like to add or change anything:
==
Description:
Apache Spark is a fast and general
Thank you, Shane!
Xiao
shane knapp ☠ 于2021年8月9日周一 下午1:26写道：
> turns out that minikube/k8s and friends were being oom-killed and this was
> causing all sorts of weirdnesses.
>
> i've upped the ram limits on all of the k8s jobs to 8G (from 6G), and
> we'll keep an eye on things and see how they
turns out that minikube/k8s and friends were being oom-killed and this was
causing all sorts of weirdnesses.
i've upped the ram limits on all of the k8s jobs to 8G (from 6G), and we'll
keep an eye on things and see how they go.
On Mon, Aug 9, 2021 at 12:02 PM shane knapp ☠ wrote:
> as workers
Hi,
I have a basic question to ask.
I am running a Google k8s cluster (AKA GKE) with three nodes each having
configuration below
e2-standard-2 (2 vCPUs, 8 GB memory)
spark-submit is launched from another node (actually a data proc single
node that I have just upgraded to e2-custom (4 vCPUs, 8
as workers are continuing to fail, i've stopped jenkins from accepting new
builds for the time being.
more updates as they come.
On Mon, Aug 9, 2021 at 9:17 AM shane knapp ☠ wrote:
> happy monday!
>
> the server gods did not smile upon us this weekend, and 4 of the workers
> are down. we'll
happy monday!
the server gods did not smile upon us this weekend, and 4 of the workers
are down. we'll most likely need to head to our colo some time today and
give them an in-person kick and see what's going on.
i'll send an update when they're back up.
shane
--
Shane Knapp
Computer Guy /
10 matches
Mail list logo
