es?
>>
>>>
>>> ------------------
>>> *From:* shane knapp
>>> *Sent:* Friday, June 14, 2019 10:23:56 AM
>>> *To:* Bryan Cutler
>>> *Cc:* Dongjoon Hyun; Holden Karau; Hyukjin Kwon; dev
>>> *Subject:* Re: [DISCUSS]
es?
>>
>>>
>>> ------------------
>>> *From:* shane knapp
>>> *Sent:* Friday, June 14, 2019 10:23:56 AM
>>> *To:* Bryan Cutler
>>> *Cc:* Dongjoon Hyun; Holden Karau; Hyukjin Kwon; dev
>>> *Subject:* Re: [DISCUSS]
-
> *From:* Holden Karau
> *Sent:* Friday, June 14, 2019 11:06:15 AM
> *To:* Felix Cheung
> *Cc:* Bryan Cutler; Dongjoon Hyun; Hyukjin Kwon; dev; shane knapp
> *Subject:* Re: [DISCUSS] Increasing minimum supported version of Pandas
>
> Are there other Python dependencies we sh
How about pyArrow?
From: Holden Karau
Sent: Friday, June 14, 2019 11:06:15 AM
To: Felix Cheung
Cc: Bryan Cutler; Dongjoon Hyun; Hyukjin Kwon; dev; shane knapp
Subject: Re: [DISCUSS] Increasing minimum supported version of Pandas
Are there other Python
gt; we sign off?
>
We should maybe add this to the release instruction notes?
>
> --
> *From:* shane knapp
> *Sent:* Friday, June 14, 2019 10:23:56 AM
> *To:* Bryan Cutler
> *Cc:* Dongjoon Hyun; Holden Karau; Hyukjin Kwon; dev
> *Subject:* Re:
; Hyukjin Kwon; dev
Subject: Re: [DISCUSS] Increasing minimum supported version of Pandas
excellent. i shall not touch anything. :)
On Fri, Jun 14, 2019 at 10:22 AM Bryan Cutler
mailto:cutl...@gmail.com>> wrote:
Shane, I think 0.24.2 is probably more common right now, so if we were to pick
excellent. i shall not touch anything. :)
On Fri, Jun 14, 2019 at 10:22 AM Bryan Cutler wrote:
> Shane, I think 0.24.2 is probably more common right now, so if we were to
> pick one to test against, I still think it should be that one. Our Pandas
> usage in PySpark is pretty conservative, so i
Shane, I think 0.24.2 is probably more common right now, so if we were to
pick one to test against, I still think it should be that one. Our Pandas
usage in PySpark is pretty conservative, so it's pretty unlikely that we
will add something that would break 0.23.X.
On Fri, Jun 14, 2019 at 10:10 AM
ah, ok... should we downgrade the testing env on jenkins then? any
specific version?
shane, who is loathe (and i mean LOATHE) to touch python envs ;)
On Fri, Jun 14, 2019 at 10:08 AM Bryan Cutler wrote:
> I should have stated this earlier, but when the user does something that
> requires Pand
I should have stated this earlier, but when the user does something that
requires Pandas, the minimum version is checked against what was imported
and will raise an exception if it is a lower version. So I'm concerned that
using 0.24.2 might be a little too new for users running older clusters. To
just to everyone knows, our python 3.6 testing infra is currently on
0.24.2...
On Fri, Jun 14, 2019 at 9:16 AM Dongjoon Hyun
wrote:
> +1
>
> Thank you for this effort, Bryan!
>
> Bests,
> Dongjoon.
>
> On Fri, Jun 14, 2019 at 4:24 AM Holden Karau wrote:
>
>> I’m +1 for upgrading, although since
+1
Thank you for this effort, Bryan!
Bests,
Dongjoon.
On Fri, Jun 14, 2019 at 4:24 AM Holden Karau wrote:
> I’m +1 for upgrading, although since this is probably the last easy chance
> we’ll have to bump version numbers easily I’d suggest 0.24.2
>
>
> On Fri, Jun 14, 2019 at 4:38 AM Hyukjin Kw
I’m +1 for upgrading, although since this is probably the last easy chance
we’ll have to bump version numbers easily I’d suggest 0.24.2
On Fri, Jun 14, 2019 at 4:38 AM Hyukjin Kwon wrote:
> I am +1 to go for 0.23.2 - it brings some overhead to test PyArrow and
> pandas combinations. Spark 3 sho
I am +1 to go for 0.23.2 - it brings some overhead to test PyArrow and
pandas combinations. Spark 3 should be good time to increase.
2019년 6월 14일 (금) 오전 9:46, Bryan Cutler 님이 작성:
> Hi All,
>
> We would like to discuss increasing the minimum supported version of
> Pandas in Spark, which is current
14 matches
Mail list logo
