Re: Please take a look at the draft of the Spark 3.1.1 release notes

2021-03-01 Thread Hyukjin Kwon
Oh yeah. I plan to fix it for the final release. Thanks for pointing that out. On Tue, 2 Mar 2021, 12:14 Kazuaki Ishizaki, wrote: > Hi Hyukjin, > Thanks for your effort. > > One question: Do you automatically update the URLs to Spark documents in > "the change of Behavior section" ? Currently,

Re: Please take a look at the draft of the Spark 3.1.1 release notes

2021-03-01 Thread Kazuaki Ishizaki
Hi Hyukjin, Thanks for your effort. One question: Do you automatically update the URLs to Spark documents in "the change of Behavior section" ? Currently, they refer to " https://spark.apache.org/docs/3.0.0/...;. I think that they should refer to "https://spark.apache.org/docs/3.1.1/...;

Re: Please take a look at the draft of the Spark 3.1.1 release notes

2021-03-01 Thread Hyukjin Kwon
Thanks guys for suggestions and fixes. Now I feel pretty confident about the release notes :-). I will start uploading and preparing to announce Spark 3.1.1. 2021년 3월 2일 (화) 오전 7:29, Tom Graves 님이 작성: > Thanks Hyukjin, overall they look good to me. > > Tom > On Saturday, February 27, 2021,

Re: Please take a look at the draft of the Spark 3.1.1 release notes

2021-03-01 Thread Tom Graves
Thanks Hyukjin, overall they look good to me. TomOn Saturday, February 27, 2021, 05:00:42 PM CST, Jungtaek Lim wrote: Thanks Hyukjin! I've only looked into the SS part, and added a comment. Otherwise it looks great!  On Sat, Feb 27, 2021 at 7:12 PM Dongjoon Hyun wrote: Thank you for

Re: [DISCUSS] SPIP: FunctionCatalog

2021-03-01 Thread Ryan Blue
Thanks for adding your perspective, Erik! If the input is string type but the UDF implementation calls row.getLong(0), it returns wrong data I think this is misleading. It is true for UnsafeRow, but there is no reason why InternalRow should return incorrect values. The implementation in