Re: Cleaning Spark releases from mirrors, and the flakiness of HiveExternalCatalogVersionsSuite

2018-07-16 Thread Hyukjin Kwon
+1 2018년 7월 17일 (화) 오전 7:34, Sean Owen 님이 작성: > Fix is committed to branches back through 2.2.x, where this test was added. > > There is still some issue; I'm seeing that archive.apache.org is > rate-limiting downloads and frequently returning 503 errors. > > We can help, I guess, by avoiding

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-16 Thread Saisai Shao
I will put my +1 on this RC. For the test failure fix, I will include it if there's another RC. Sean Owen 于2018年7月16日周一 下午10:47写道: > OK, hm, will try to get to the bottom of it. But if others can build this > module successfully, I give a +1 . The test failure is inevitable here and > should

Re: Cleaning Spark releases from mirrors, and the flakiness of HiveExternalCatalogVersionsSuite

2018-07-16 Thread Sean Owen
Fix is committed to branches back through 2.2.x, where this test was added. There is still some issue; I'm seeing that archive.apache.org is rate-limiting downloads and frequently returning 503 errors. We can help, I guess, by avoiding testing against non-current releases. Right now we should be

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-16 Thread Sean Owen
OK, hm, will try to get to the bottom of it. But if others can build this module successfully, I give a +1 . The test failure is inevitable here and should not block release. On Sun, Jul 15, 2018 at 9:39 PM Saisai Shao wrote: > Hi Sean, > > I just did a clean build with mvn/sbt on 2.3.2, I

Re: JDBC Data Source and customSchema option but DataFrameReader.assertNoSpecifiedSchema?

2018-07-16 Thread Joseph Torres
I guess the question is partly about the semantics of DataFrameReader.schema. If it's supposed to mean "the loaded dataframe will definitely have exactly this schema", that doesn't quite match the behavior of the customSchema option. If it's only meant to be an arbitrary schema input which the

JDBC Data Source and customSchema option but DataFrameReader.assertNoSpecifiedSchema?

2018-07-16 Thread Jacek Laskowski
Hi, I think there is a sort of inconsistency in how DataFrameReader.jdbc deals with a user-defined schema as it makes sure that there's no user-specified schema [1][2] yet allows for setting one using customSchema option [3]. Why is so? Has this been merely overlooked or similar? I think