Thanks Sean
Nasrulla
-Original Message-
From: Sean Owen
Sent: Monday, May 13, 2019 4:16 PM
To: Nasrulla Khan Haris
Cc: dev@spark.apache.org
Subject: Re: adding shutdownmanagerhook to spark.
Spark just adds a hook to the mechanism that Hadoop exposes. You can do the
same. You
Spark just adds a hook to the mechanism that Hadoop exposes. You can
do the same. You shouldn't use Spark's.
On Mon, May 13, 2019 at 6:11 PM Nasrulla Khan Haris
wrote:
>
> HI All,
>
>
>
> I am trying to add shutdown hook, but looks like shutdown manager object
> requires the package to be spark
HI All,
I am trying to add shutdown hook, but looks like shutdown manager object
requires the package to be spark only, is there any other API that can help me
to do this ?
https://github.com/apache/spark/blob/v2.4.0/core/src/main/scala/org/apache/spark/util/ShutdownHookManager.scala
I can see
I’d like to ask this vote period to be extended, I’m interested but I don’t
have the cycles to review it in detail and make an informed vote until the
25th.
On Tue, May 14, 2019 at 1:49 AM Xiangrui Meng wrote:
> My vote is 0. Since the updated SPIP focuses on ETL use cases, I don't
> feel
All,
FYI, in case you are not also subscribed to d...@avro.apache.org, there is a
vote thread currently in progress for Avro version 1.9.0-RC4, which is binary
and source incompatible with version 1.8.2
http://people.apache.org/~busbey/avro/1.9.0-RC4/1.8.2_to_1.9.0RC4_compat_report.html
Hi Mridul - thanks for taking the time to give us feedback! Thoughts on the
points that you mentioned:
The API is meant to work with the existing SortShuffleManager algorithm. There
aren't strict requirements on how other ShuffleManager implementations must
behave, so it seems impractical
HI All,
Is there a API for sparkContext where we can add our custom code before
stopping sparkcontext ?
Appreciate your help.
Thanks,
Nasrulla
My vote is 0. Since the updated SPIP focuses on ETL use cases, I don't feel
strongly about it. I would still suggest doing the following:
1. Link the POC mentioned in Q4. So people can verify the POC result.
2. List public APIs we plan to expose in Appendix A. I did a quick check.
Beside
It would be nice to get feedback from people who responded on the
other vote thread - Reynold, Matei, Xiangrui, does the new version
look good?
Thanks,
Tom
On Mon, May 13, 2019 at 8:22 AM Jason Lowe wrote:
>
> +1 (non-binding)
>
> Jason
>
> On Tue, May 7, 2019 at 1:37 PM Thomas graves wrote:
+1 (non-binding)
Jason
On Tue, May 7, 2019 at 1:37 PM Thomas graves wrote:
> Hi everyone,
>
> I'd like to call for another vote on SPARK-27396 - SPIP: Public APIs
> for extended Columnar Processing Support. The proposal is to extend
> the support to allow for more columnar processing. We had
10 matches
Mail list logo