Thanks Sean
Nasrulla
-Original Message-
From: Sean Owen
Sent: Monday, May 13, 2019 4:16 PM
To: Nasrulla Khan Haris
Cc: dev@spark.apache.org
Subject: Re: adding shutdownmanagerhook to spark.
Spark just adds a hook to the mechanism that Hadoop exposes. You can do the
same. You shouldn'
Spark just adds a hook to the mechanism that Hadoop exposes. You can
do the same. You shouldn't use Spark's.
On Mon, May 13, 2019 at 6:11 PM Nasrulla Khan Haris
wrote:
>
> HI All,
>
>
>
> I am trying to add shutdown hook, but looks like shutdown manager object
> requires the package to be spark
HI All,
I am trying to add shutdown hook, but looks like shutdown manager object
requires the package to be spark only, is there any other API that can help me
to do this ?
https://github.com/apache/spark/blob/v2.4.0/core/src/main/scala/org/apache/spark/util/ShutdownHookManager.scala
I can see
I’d like to ask this vote period to be extended, I’m interested but I don’t
have the cycles to review it in detail and make an informed vote until the
25th.
On Tue, May 14, 2019 at 1:49 AM Xiangrui Meng wrote:
> My vote is 0. Since the updated SPIP focuses on ETL use cases, I don't
> feel strong
All,
FYI, in case you are not also subscribed to d...@avro.apache.org, there is a
vote thread currently in progress for Avro version 1.9.0-RC4, which is binary
and source incompatible with version 1.8.2
http://people.apache.org/~busbey/avro/1.9.0-RC4/1.8.2_to_1.9.0RC4_compat_report.html
Hi Mridul - thanks for taking the time to give us feedback! Thoughts on the
points that you mentioned:
The API is meant to work with the existing SortShuffleManager algorithm. There
aren't strict requirements on how other ShuffleManager implementations must
behave, so it seems impractical to
HI All,
Is there a API for sparkContext where we can add our custom code before
stopping sparkcontext ?
Appreciate your help.
Thanks,
Nasrulla
My vote is 0. Since the updated SPIP focuses on ETL use cases, I don't feel
strongly about it. I would still suggest doing the following:
1. Link the POC mentioned in Q4. So people can verify the POC result.
2. List public APIs we plan to expose in Appendix A. I did a quick check.
Beside ColumnarB
It would be nice to get feedback from people who responded on the
other vote thread - Reynold, Matei, Xiangrui, does the new version
look good?
Thanks,
Tom
On Mon, May 13, 2019 at 8:22 AM Jason Lowe wrote:
>
> +1 (non-binding)
>
> Jason
>
> On Tue, May 7, 2019 at 1:37 PM Thomas graves wrote:
>>
+1 (non-binding)
Jason
On Tue, May 7, 2019 at 1:37 PM Thomas graves wrote:
> Hi everyone,
>
> I'd like to call for another vote on SPARK-27396 - SPIP: Public APIs
> for extended Columnar Processing Support. The proposal is to extend
> the support to allow for more columnar processing. We had
10 matches
Mail list logo