Re: Ask for ARM CI for spark

Hi, all, FYI: >> @Yuming Wang the results in float8.sql are from PostgreSQL directly? >> Interesting if it also returns the same less accurate result, which >> might suggest it's more to do with underlying OS math libraries. You >> noted that these tests sometimes gave platform-dependent

Re: [Discuss] Follow ANSI SQL on table insertion

I don't agree with handling literal values specially. Although Postgres does it, I can't find anything about it in the SQL standard. And it introduces inconsistent behaviors which may be strange to users: * What about something like "INSERT INTO t SELECT float_col + 1.1"? * The same insert with a

Re: Ask for ARM CI for spark

Hi all, Thanks for your concern. Yeah, that's worth to also test in backend database. But need to note here, this issue is hit in Spark SQL, as we only test it with spark itself, not integrate other databases. Best Regards, ZhaoBo [image: Mailtrack]

Re: Apache Training contribution for Spark - Feedback welcome

On Fri, Jul 26, 2019 at 4:01 PM Lars Francke wrote: > I understand why it might be seen that way and we need to make sure to point > out that we have no intention of becoming "The official Apache Spark > training" because that's not our intention at all. Of course that's the intention; the

Re: Apache Training contribution for Spark - Feedback welcome

Sean, thanks for taking the time to comment. We've discussed those issues during the proposal stage for the Incubator as others brought them up as well. I can't remember all the details but let me go through your points inline. My reservation here is that as an Apache project, it might appear

Re: [DISCUSS] New sections in Github Pull Request description template

The k8s template is pretty good. Under the behavior change section, it would be good to add instructions to also describe previous and new behavior as Hyukjin proposed. On Tue, Jul 23, 2019 at 10:07 PM Reynold Xin wrote: > I like the spirit, but not sure about the exact proposal. Take a look at

Re: [Discuss] Follow ANSI SQL on table insertion

I don’t think this is a good idea. Following the ANSI standard is usually fine, but here it would *silently corrupt data*. >From your proposal doc, ANSI allows implicitly casting from long to int (any numeric type to any other numeric type) and inserts NULL when a value overflows. That would drop

Re: Apache Training contribution for Spark - Feedback welcome

Generally speaking, I think we want to encourage more training and tutorial content out there, for sure, so, the more the merrier. My reservation here is that as an Apache project, it might appear to 'bless' one set of materials as authoritative over all the others out there. And there are

Re: New Spark Datasource for Hive ACID tables

Hey Naresh, Thanks for your question. Yes it will work! Thanks, Abhishek Somani On Fri, Jul 26, 2019 at 7:08 PM naresh Goud wrote: > Thanks Abhishek. > > Will it work on hive acid table which is not compacted ? i.e table having > base and delta files? > > Let’s say hive acid table customer >

Apache Training contribution for Spark - Feedback welcome

Hi Spark community, you may or may not have heard of a new-ish (February 2019) project at Apache: Apache Training (incubating). We aim to develop training material about various projects inside and outside the ASF: < http://training.apache.org/> One of our users wants to contribute material on

Re: New Spark Datasource for Hive ACID tables

Thanks Abhishek. Will it work on hive acid table which is not compacted ? i.e table having base and delta files? Let’s say hive acid table customer Create table customer(customer_id int, customer_name string, customer_email string) cluster by customer_id buckets 10 location ‘/test/customer’

New Spark Datasource for Hive ACID tables

Hi All, We at Qubole have open sourced a datasource that will enable users to work on their Hive ACID Transactional Tables using Spark. Github: https://github.com/qubole/spark-acid Hive ACID tables

Re: Ask for ARM CI for spark

Interesting. I don't think log(3) is special, it's just that some differences in how it's implemented and floating-point values on aarch64 vs x86, or in the JVM, manifest at some values like this. It's still a little surprising! BTW Wolfram Alpha suggests that the correct value is more like

Re: Ask for ARM CI for spark

Hi, all Sorry to disturb again, there are several sql tests failed on arm64 instance: - pgSQL/float8.sql *** FAILED *** Expected "0.549306144334054[9]", but got "0.549306144334054[8]" Result did not match for query #56 SELECT atanh(double('0.5')) (SQLQueryTestSuite.scala:362) -