Re:Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

2024-04-15 Thread beliefer
+1 在 2024-04-15 15:54:07,"Peter Toth" 写道: +1 Wenchen Fan ezt írta (időpont: 2024. ápr. 15., H, 9:08): +1 On Sun, Apr 14, 2024 at 6:28 AM Dongjoon Hyun wrote: I'll start from my +1. Dongjoon. On 2024/04/13 22:22:05 Dongjoon Hyun wrote: > Please vote on SPARK-4 to use ANSI SQL

Re:[ANNOUNCE] Apache Spark 3.5.1 released

2024-02-28 Thread beliefer
Congratulations! At 2024-02-28 17:43:25, "Jungtaek Lim" wrote: Hi everyone, We are happy to announce the availability of Spark 3.5.1! Spark 3.5.1 is a maintenance release containing stability fixes. This release is based on the branch-3.5 maintenance branch of Spark. We strongly

Re:Re: [DISCUSS] Release Spark 3.5.1?

2024-02-04 Thread beliefer
+1 在 2024-02-04 15:26:13,"Dongjoon Hyun" 写道: +1 On Sat, Feb 3, 2024 at 9:18 PM yangjie01 wrote: +1 在 2024/2/4 13:13,“Kent Yao”mailto:y...@apache.org>> 写入: +1 Jungtaek Lim mailto:kabhwan.opensou...@gmail.com>> 于2024年2月3日周六 21:14写道: > > Hi dev, > > looks like there are a huge

Re:[ANNOUNCE] Apache Spark 3.4.2 released

2023-11-30 Thread beliefer
Congratulations! At 2023-12-01 01:23:55, "Dongjoon Hyun" wrote: We are happy to announce the availability of Apache Spark 3.4.2! Spark 3.4.2 is a maintenance release containing many fixes including security and correctness domains. This release is based on the branch-3.4 maintenance

Re:Re: [VOTE] Updating documentation hosted for EOL and maintenance releases

2023-09-26 Thread beliefer
+1 At 2023-09-26 13:03:56, "Ruifeng Zheng" wrote: +1 On Tue, Sep 26, 2023 at 12:51 PM Hyukjin Kwon wrote: Hi all, I would like to start the vote for updating documentation hosted for EOL and maintenance releases to improve the usability here, and in order for end users to read the

Re:Are DataFrame rows ordered without an explicit ordering clause?

2023-09-23 Thread beliefer
AFAIK, The order is free whether it's SQL without spcified ORDER BY clause or DataFrame without sort. The behavior is consistent between them. At 2023-09-18 23:47:40, "Nicholas Chammas" wrote: I’ve always considered DataFrames to be logically equivalent to SQL tables or queries. In

Re:[ANNOUNCE] Apache Spark 3.5.0 released

2023-09-17 Thread beliefer
Congratulations! Apache Spark. At 2023-09-16 01:01:40, "Yuanjian Li" wrote: Hi All, We are happy to announce the availability of Apache Spark 3.5.0! Apache Spark 3.5.0 is the sixth release of the 3.x line. To download Spark 3.5.0, head over to the download page:

Re:[ANNOUNCE] Apache Spark 3.4.1 released

2023-06-24 Thread beliefer
Thanks! Dongjoon Hyun. Congratulation too! At 2023-06-24 07:57:05, "Dongjoon Hyun" wrote: We are happy to announce the availability of Apache Spark 3.4.1! Spark 3.4.1 is a maintenance release containing stability fixes. This release is based on the branch-3.4 maintenance branch of Spark.

Re: Apache Spark 3.4.1 Release?

2023-06-12 Thread beliefer
Dongjoon. Thank you. There is a issue should be fixed. https://issues.apache.org/jira/browse/SPARK-44018 在 2023-06-12 13:22:30,"Dongjoon Hyun" 写道: Thank you all. I'll check and prepare `branch-3.4` for the target date, June 20th. Dongjoon. On Fri, Jun 9, 2023 at 10:47 PM yangjie01

Re:[VOTE] Release Apache Spark 3.4.0 (RC7)

2023-04-11 Thread beliefer
+1 At 2023-04-08 07:29:46, "Xinrong Meng" wrote: Please vote on releasing the following candidate(RC7) as Apache Spark version 3.4.0. The vote is open until 11:59pm Pacific time April 12th and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this

Re:Re: [VOTE] Release Apache Spark 3.4.0 (RC4)

2023-03-10 Thread beliefer
There is a bug fix. https://issues.apache.org/jira/browse/SPARK-42740 在 2023-03-10 20:48:30,"Xinrong Meng" 写道: https://issues.apache.org/jira/browse/SPARK-42745 can be a new release blocker, thanks @Peter Toth for reporting that. On Fri, Mar 10, 2023 at 8:21 PM Xinrong Meng wrote:

Re:[ANNOUNCE] Apache Spark 3.3.2 released

2023-02-18 Thread beliefer
Congratulations ! At 2023-02-17 16:58:22, "L. C. Hsieh" wrote: >We are happy to announce the availability of Apache Spark 3.3.2! > >Spark 3.3.2 is a maintenance release containing stability fixes. This >release is based on the branch-3.3 maintenance branch of Spark. We strongly >recommend

Re:[ANNOUNCE] Apache Spark 3.3.1 released

2022-10-26 Thread beliefer
Congratulations everyone have contributed to this release. At 2022-10-26 14:21:36, "Yuming Wang" wrote: We are happy to announce the availability of Apache Spark 3.3.1! Spark 3.3.1 is a maintenance release containing stability fixes. This release is based on the branch-3.3 maintenance

Re:[DISCUSS] Improve the performance of Spark Decimal.

2022-08-29 Thread beliefer
There is a WIP PR https://github.com/apache/spark/pull/37536, only implement the add operator of Decimal128 At 2022-08-24 12:09:03, "beliefer" wrote: Hi all, Recently, we found many SQL query could improve performance by replace Spark Decimal to Double. This is confirmed b

Re:Re: [VOTE][SPIP] Spark Connect

2022-06-14 Thread beliefer
+1 Yeah, I tried to use Apache Livy, so as we can runing interactive query. But the Spark Driver in Livy looks heavy. The SPIP may resolve the issue. At 2022-06-14 18:11:21, "Wenchen Fan" wrote: +1 On Tue, Jun 14, 2022 at 9:38 AM Ruifeng Zheng wrote: +1 -- 原始邮件

Re:回复: [VOTE] Release Spark 3.3.0 (RC6)

2022-06-13 Thread beliefer
+1 AFAIK, no blocking issues now. Glad to hear to release 3.3.0 ! 在 2022-06-14 09:38:35,"Ruifeng Zheng" 写道: +1 (non-binding) Maxim, thank you for driving this release! thanks, ruifeng -- 原始邮件 -- 发件人: "Chao Sun" ; 发送时间: 2022年6月14日(星期二) 上午8:45 收件人:

Re:Re: Unable to create view due to up cast error when migrating from Hive to Spark

2022-05-19 Thread beliefer
ly I think we need to do the same for every Hive view if we need to use it in Spark. On Wed, May 18, 2022 at 7:03 PM beliefer wrote: During the migration from hive to spark, there was a problem with the SQL used to create views in hive. The problem is that the SQL that legally creates a vi

Unable to create view due to up cast error when migrating from Hive to Spark

2022-05-18 Thread beliefer
During the migration from hive to spark, there was a problem with the SQL used to create views in hive. The problem is that the SQL that legally creates a view in hive will make an error when executed in spark SQL. The SQL is as follows: CREATE VIEW test_db.my_view AS select case when age > 12

Unable to create view due to up cast error when migrating from Hive to Spark

2022-05-17 Thread beliefer
During the migration from Hive to spark, there was a problem when the view created in Hive was used in Spark SQL. The origin Hive SQL show below: CREATE VIEW myView AS SELECT CASE WHEN age > 12 THEN CAST(gender * 0.3 - 0.1 AS double) END AS TT, gender, age FROM myTable; Users use Spark SQL

Re:Re: Re: Re: [VOTE] Release Spark 3.3.0 (RC2)

2022-05-17 Thread beliefer
OK. let it into 3.3.1 在 2022-05-17 18:59:13,"Hyukjin Kwon" 写道: I think most users won't be affected since aggregate pushdown is disabled by default. On Tue, 17 May 2022 at 19:53, beliefer wrote: If we not contains https://github.com/apache/spark/pull/36556, we will break c

Re:Re: Re: [VOTE] Release Spark 3.3.0 (RC2)

2022-05-17 Thread beliefer
RC2 passes. Since this is a new API/improvement, I would prefer to not block the release by that. On Tue, 17 May 2022 at 19:19, beliefer wrote: We need add https://github.com/apache/spark/pull/36556 to RC2. 在 2022-05-17 17:37:13,"Hyukjin Kwon" 写道: That seems like a test-only i

Re:Re: [VOTE] Release Spark 3.3.0 (RC2)

2022-05-17 Thread beliefer
We need add https://github.com/apache/spark/pull/36556 to RC2. 在 2022-05-17 17:37:13,"Hyukjin Kwon" 写道: That seems like a test-only issue. I made a quick followup at https://github.com/apache/spark/pull/36576. On Tue, 17 May 2022 at 03:56, Sean Owen wrote: I'm still seeing failures

Unable to create view due to up cast error when migrating from Hive to Spark

2022-05-17 Thread beliefer
During the migration from hive to spark, there was a problem with the SQL used to create views in hive. The problem is that the SQL that legally creates a view in hive will make an error when executed in spark SQL. The SQL is as follows: CREATE VIEW myView AS SELECT CASE WHEN age > 12 THEN

Re:[VOTE] Release Spark 3.3.0 (RC1)

2022-05-07 Thread beliefer
@Maxim Gekk Glad to hear that! But there is a bug https://github.com/apache/spark/pull/36457 I think we should merge it into 3.3.0 At 2022-05-05 19:00:27, "Maxim Gekk" wrote: Please vote on releasing the following candidate as Apache Spark version 3.3.0. The vote is open until

Re:Apache Spark 3.3 Release

2022-03-16 Thread beliefer
+1 Glad to see we will release 3.3.0. At 2022-03-04 02:44:37, "Maxim Gekk" wrote: Hello All, I would like to bring on the table the theme about the new Spark release 3.3. According to the public schedule at https://spark.apache.org/versioning-policy.html, we planned to start the code

Re:[ANNOUNCE] Apache Spark 3.2.1 released

2022-01-31 Thread beliefer
Thank you huaxin gao! Glad to see the release. At 2022-01-29 09:07:13, "huaxin gao" wrote: We are happy to announce the availability of Spark 3.2.1! Spark 3.2.1 is a maintenance release containing stability fixes. This release is based on the branch-3.2 maintenance branch of Spark. We

Re:[VOTE] Release Spark 3.2.1 (RC1)

2022-01-26 Thread beliefer
+1 At 2022-01-11 02:09:46, "huaxin gao" wrote: Please vote on releasing the following candidate as Apache Spark version 3.2.1. The vote is open until Jan. 13th at 12 PM PST (8 PM UTC) and passes if a majority +1 PMC votes are cast, with a minimum of 3 + 1 votes. [ ] +1 Release this

Re: Spark-3.0 - performance degradation

2020-03-19 Thread beliefer
I test it and cannot reproduce the issue. I build Spark-3.1.0 and Spark2.3.1. After many tests, it is found that there is little difference between them, and they win and lose each other. And from the view of event timeline, Spark-3.1.0 looks more accurate. -- Sent from:

Re: Spark-3.0 - performance degradation

2020-03-06 Thread beliefer
Can you provide configuration information? -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Spark-3.0 - performance degradation

2020-03-06 Thread beliefer
Can you provide configuration information? -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Spark-3.0 - performance degradation

2020-03-06 Thread beliefer
Can you show the running configuration information? -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org