(That may be so, but it may still be correct to revert a change in Spark if
necessary to not be exposed to it in the short term. I have no idea whether
that's the right thing here or not, just answering the point about why we'd
care about a bug in another project. Also, not clear which Hive
Well, it is not a bug for Spark 2.4 but a bug for Hive 2.1.1 . My colleague
will report it on the Spark JIRA later.
Presto works fine when reading the ORC table created by Spark 2.4.
We've decided to fix it in Hive 2.1.1 . Since Hive 2.1.1 is widely used, I
suggest that we should keep a
Do you know which bug ORC 1.5.2 introduced? Or is it because Hive uses a
legacy version of ORC which has a bug?
On Thu, Feb 14, 2019 at 2:35 PM Darcy Shen wrote:
>
> We found that ORC table created by Spark 2.4 failed to be read by Hive
> 2.1.1.
>
>
> spark-sql -e 'CREATE TABLE tmp.orcTable2
We found that ORC table created by Spark 2.4 failed to be read by Hive 2.1.1.
spark-sql -e 'CREATE TABLE tmp.orcTable2 USING orc AS SELECT * FROM
tmp.orcTable1 limit 10;'
hive -e 'select * from tmp.orcTable2'
The ERROR messages by Hive:
Failed with exception
just filed a JIRA in https://issues.apache.org/jira/browse/SPARK-26862
'
this issue only happens in 2.4.0 but not in 2.3.2
anyone would help to look into that?
On Tue, Feb 12, 2019 at 10:41 AM DB Tsai wrote:
> Great. I'll prepare the release for voting. Thanks!
>
> DB Tsai | Siri Open
Great. I'll prepare the release for voting. Thanks!
DB Tsai | Siri Open Source Technologies [not a contribution] | Apple, Inc
> On Feb 12, 2019, at 4:11 AM, Wenchen Fan wrote:
>
> +1 for 2.4.1
>
> On Tue, Feb 12, 2019 at 7:55 PM Hyukjin Kwon wrote:
> +1 for 2.4.1
>
> 2019년 2월 12일 (화)
+1 for 2.4.1
On Tue, Feb 12, 2019 at 7:55 PM Hyukjin Kwon wrote:
> +1 for 2.4.1
>
> 2019년 2월 12일 (화) 오후 4:56, Dongjin Lee 님이 작성:
>
>> > SPARK-23539 is a non-trivial improvement, so probably would not be
>> back-ported to 2.4.x.
>>
>> Got it. It seems reasonable.
>>
>> Committers:
>>
>> Please
+1 for 2.4.1
2019년 2월 12일 (화) 오후 4:56, Dongjin Lee 님이 작성:
> > SPARK-23539 is a non-trivial improvement, so probably would not be
> back-ported to 2.4.x.
>
> Got it. It seems reasonable.
>
> Committers:
>
> Please don't omit SPARK-23539 from 2.5.0. Kafka community needs this
> feature.
>
>
> SPARK-23539 is a non-trivial improvement, so probably would not be
back-ported to 2.4.x.
Got it. It seems reasonable.
Committers:
Please don't omit SPARK-23539 from 2.5.0. Kafka community needs this
feature.
Thanks,
Dongjin
On Tue, Feb 12, 2019 at 1:50 PM Takeshi Yamamuro
wrote:
> +1,
+1, too.
branch-2.4 accumulates too many commits..:
https://github.com/apache/spark/compare/0a4c03f7d084f1d2aa48673b99f3b9496893ce8d...af3c7111efd22907976fc8bbd7810fe3cfd92092
On Tue, Feb 12, 2019 at 12:36 PM Dongjoon Hyun wrote:
> Thank you, DB.
>
> +1, Yes. It's time for preparing 2.4.1
Thank you, DB.
+1, Yes. It's time for preparing 2.4.1 release.
Bests,
Dongjoon.
On 2019/02/12 03:16:05, Sean Owen wrote:
> I support a 2.4.1 release now, yes.
>
> SPARK-23539 is a non-trivial improvement, so probably would not be
> back-ported to 2.4.x.SPARK-26154 does look like a bug whose
I support a 2.4.1 release now, yes.
SPARK-23539 is a non-trivial improvement, so probably would not be
back-ported to 2.4.x.SPARK-26154 does look like a bug whose fix could
be back-ported, but that's a big change. I wouldn't hold up 2.4.1 for
it, but it could go in if otherwise ready.
On Mon,
Hi DB,
Could you add SPARK-23539[^1] into 2.4.1? I opened the PR[^2] a little bit
ago, but it has not included in 2.3.0 nor get enough review.
Thanks,
Dongjin
[^1]: https://issues.apache.org/jira/browse/SPARK-23539
[^2]: https://github.com/apache/spark/pull/22282
On Tue, Feb 12, 2019 at 6:28
Given SPARK-26154 [1] is a correctness issue and PR [2] is submitted, I
hope it can be reviewed and included within Spark 2.4.1 - otherwise it will
be a long-live correctness issue.
Thanks,
Jungtaek Lim (HeartSaVioR)
1. https://issues.apache.org/jira/browse/SPARK-26154
2.
Hello all,
I am preparing to cut a new Apache 2.4.1 release as there are many bugs and
correctness issues fixed in branch-2.4.
The list of addressed issues are
15 matches
Mail list logo
