Re: [DISCUSS] Multiple catalog support

I think the major issue is, now users have 2 ways to create a specific data source table: 1) use the USING syntax. 2) create the table in the specific catalog. It can be super confusing if users create a cassandra table in hbase data source. Also we can't drop the USING syntax as data source v1

Re: [SPARK-24950] issues running DateTimeUtilsSuite daysToMillis and millisToDays w/java 8 181-b13

cool, thanks. testing now. On Fri, Jul 27, 2018 at 1:04 PM, Sean Owen wrote: > I posted on the JIRA -- looks like timezone definitions for Kiribati were > fixed in the 2018d timezone data release, and that's the difference between > the JDK release that works and does not. > > On Fri, Jul 27,

Re: [SPARK-24950] issues running DateTimeUtilsSuite daysToMillis and millisToDays w/java 8 181-b13

I posted on the JIRA -- looks like timezone definitions for Kiribati were fixed in the 2018d timezone data release, and that's the difference between the JDK release that works and does not. On Fri, Jul 27, 2018 at 1:23 PM shane knapp wrote: > hey everyone! > > i'm making great progress on

Re: [Spark SQL] Future of CalendarInterval

The code freeze of the upcoming release Spark 2.4 is very close. How about revisiting this and explicitly defining the support scope of CalendarIntervalType in the next release (Spark 3.0)? Thanks, Xiao 2018-07-27 10:45 GMT-07:00 Reynold Xin : > CalendarInterval is definitely externally

[SPARK-24950] issues running DateTimeUtilsSuite daysToMillis and millisToDays w/java 8 181-b13

hey everyone! i'm making great progress on porting the spark builds to run under ubuntu 16.04LTS, but have hit a show-stopper in my testing. i am not a scala person by any definition of the term, and could really use some help in trying to figure out what's going on. details of what i'm running

Re: [Spark SQL] Future of CalendarInterval

CalendarInterval is definitely externally visible. E.g. sql("select interval 1 day").dtypes would return "Array[(String, String)] = Array((interval 1 days,CalendarIntervalType))" However, I'm not sure what it means to support casting. What are the semantics for casting from any other data type

[Spark SQL] Future of CalendarInterval

Hi Sparkers! (maybe Sparkles ?) I just wanted to bring up the apparently ?controversial? Calendar Interval topic. I worked on: https://issues.apache.org/jira/browse/SPARK-24702 , https://github.com/apache/spark/pull/21706

[DISCUSS] Adaptive execution in Spark SQL

Dear all, The initial support of adaptive execution[SPARK-9850] in Spark SQL has been there since Spark 1.6, but there is no more update since then. One of the key features in adaptive execution is to determine the number of reducer

Re: [VOTE] SPARK 2.3.2 (RC3)

The following blocker/important fixes have been merged to Spark 2.3 branch: https://issues.apache.org/jira/browse/SPARK-24927 https://issues.apache.org/jira/browse/SPARK-24867 https://issues.apache.org/jira/browse/SPARK-24891 *Saisai*, could you start the next RC? Thanks, Xiao 2018-07-20

Re: [build system] upped build retention for GHPRB builds

Hi, Shane, Thank you for your help! Xiao 2018-07-24 11:03 GMT-07:00 shane knapp : > the PRB was set to rotate build logs out after two weeks, but due to the > sheer number of builds (yay! a great thing!), i just went in and upped it > to 30 days. > > we've got plenty of disk space on our

Re: code freeze and branch cut for Apache Spark 2.4

Extending code freeze date would be great for me too, I am working on a PR for supporting scala 2.12, I am close but need some more time. We could get it into 2.4. Stavros On Fri, Jul 27, 2018 at 9:27 AM, Wenchen Fan wrote: > This seems fine to me. > > BTW Ryan Blue and I are working on some

Re: code freeze and branch cut for Apache Spark 2.4

This seems fine to me. BTW Ryan Blue and I are working on some data source v2 stuff and hopefully we can get more things done with one more week. Thanks, Wenchen On Thu, Jul 26, 2018 at 1:14 PM Xingbo Jiang wrote: > Xiangrui and I are leading an effort to implement a highly desirable >