Re: [DISCUSS] SPIP: Stored Procedures API for Catalogs

2024-05-09 Thread huaxin gao
Thanks Anton for the updated proposal -- it looks great! I appreciate the hard work put into refining it. I am looking forward to the upcoming vote and moving forward with this initiative. Thanks, Huaxin On Thu, May 9, 2024 at 7:30 PM L. C. Hsieh wrote: > Thanks Anton. Thank you, Wenchen,

Re: [DISCUSS] SPIP: Stored Procedures API for Catalogs

2024-05-09 Thread Wenchen Fan
Thanks for leading this project! Let's move forward. On Fri, May 10, 2024 at 10:31 AM L. C. Hsieh wrote: > Thanks Anton. Thank you, Wenchen, Dongjoon, Ryan, Serge, Allison and > others if I miss those who are participating in the discussion. > > I suppose we have reached a consensus or close to

Re: [DISCUSS] SPIP: Stored Procedures API for Catalogs

2024-05-09 Thread L. C. Hsieh
Thanks Anton. Thank you, Wenchen, Dongjoon, Ryan, Serge, Allison and others if I miss those who are participating in the discussion. I suppose we have reached a consensus or close to being in the design. If you have some more comments, please let us know. If not, I will go to start a vote soon

Re: [DISCUSS] SPIP: Stored Procedures API for Catalogs

2024-05-09 Thread Anton Okolnychyi
Thanks to everyone who commented on the design doc. I updated the proposal and it is ready for another look. I hope we can converge and move forward with this effort! - Anton пт, 19 квіт. 2024 р. о 15:54 Anton Okolnychyi пише: > Hi folks, > > I'd like to start a discussion on SPARK-44167 that

Re: [DISCUSS] Spark 4.0.0 release

2024-05-09 Thread Wenchen Fan
UPDATE: I've successfully uploaded the release packages: https://dist.apache.org/repos/dist/dev/spark/v4.0.0-preview1-rc1-bin/ (I skipped SparkR as I was not able to fix the errors, I'll get back to it later) However, there is a new issue with doc building:

Re: [DISCUSS] Spark 4.0.0 release

2024-05-09 Thread Dongjoon Hyun
Please re-try to upload, Wenchen. ASF Infra team bumped up our upload limit based on our request. > Your upload limit has been increased to 650MB Dongjoon. On Thu, May 9, 2024 at 8:12 AM Wenchen Fan wrote: > I've created a ticket: https://issues.apache.org/jira/browse/INFRA-25776 > > On

Re: [DISCUSS] Spark 4.0.0 release

2024-05-09 Thread Wenchen Fan
I've created a ticket: https://issues.apache.org/jira/browse/INFRA-25776 On Thu, May 9, 2024 at 11:06 PM Dongjoon Hyun wrote: > In addition, FYI, I was the latest release manager with Apache Spark 3.4.3 > (2024-04-15 Vote) > > According to my work log, I uploaded the following binaries to SVN

Re: [DISCUSS] Spark 4.0.0 release

2024-05-09 Thread Dongjoon Hyun
In addition, FYI, I was the latest release manager with Apache Spark 3.4.3 (2024-04-15 Vote) According to my work log, I uploaded the following binaries to SVN from EC2 (us-west-2) without any issues. -rw-r--r--. 1 centos centos 311384003 Apr 15 01:29 pyspark-3.4.3.tar.gz -rw-r--r--. 1 centos

Re: [DISCUSS] Spark 4.0.0 release

2024-05-09 Thread Dongjoon Hyun
Could you file an INFRA JIRA issue with the error message and context first, Wenchen? As you know, if we see something, we had better file a JIRA issue because it could be not only an Apache Spark project issue but also all ASF project issues. Dongjoon. On Thu, May 9, 2024 at 12:28 AM Wenchen

Re: [DISCUSS] Spark - How to improve our release processes

2024-05-09 Thread Wenchen Fan
Thanks for starting the discussion! To add a bit more color, we should at least add a test job to make sure the release script can produce the packages correctly. Today it's kind of being manually tested by the release manager each time, which slows down the release process. It's better if we can

Re: [DISCUSS] Spark - How to improve our release processes

2024-05-09 Thread Hussein Awala
Hello, I can answer some of your common questions with other Apache projects. > Who currently has permissions for Github actions? Is there a specific owner for that today or a different volunteer each time? The Apache organization owns Github Actions, and committers (contributors with write

[DISCUSS] Spark - How to improve our release processes

2024-05-09 Thread Nimrod Ofek
Following the conversation started with Spark 4.0.0 release, this is a thread to discuss improvements to our release processes. I'll Start by raising some questions that probably should have answers to start the discussion: 1. What is currently running in GitHub Actions? 2. Who currently

Re: [DISCUSS] Spark 4.0.0 release

2024-05-09 Thread Wenchen Fan
UPDATE: After resolving a few issues in the release scripts, I can finally build the release packages. However, I can't upload them to the staging SVN repo due to a transmitting error, and it seems like a limitation from the server side. I tried it on both my local laptop and remote AWS instance,