Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/16337
Thanks! Merging to master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and w
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/16337
Go for it!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the f
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/16337
Will merge it if nobody objects it in the next 24 hours. : )
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does n
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/16337
I believe it's ready to do so. Thank you, @gatorsmile, for your relentless
effort to review this PR. This PR will serve as a baseline for subsequent PRs
we will submit for the remaining test cases on
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/16337
@gatorsmile @nsyca @kevinyu98 can we pull the trigger on this one?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/16337
I compared the results and confirmed the results are consistent. LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your proje
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70782/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16337
**[Test build #70782 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70782/testReport)**
for PR 16337 at commit
[`895bb35`](https://github.com/apache/spark/commit/8
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16337
**[Test build #70782 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70782/testReport)**
for PR 16337 at commit
[`895bb35`](https://github.com/apache/spark/commit/89
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70768/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16337
**[Test build #70768 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70768/testReport)**
for PR 16337 at commit
[`7c129d9`](https://github.com/apache/spark/commit/7
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16337
**[Test build #70768 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70768/testReport)**
for PR 16337 at commit
[`7c129d9`](https://github.com/apache/spark/commit/7c
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70535/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16337
**[Test build #70535 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70535/testReport)**
for PR 16337 at commit
[`1c1900a`](https://github.com/apache/spark/commit/1
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16337
**[Test build #70535 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70535/testReport)**
for PR 16337 at commit
[`1c1900a`](https://github.com/apache/spark/commit/1c
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/16337
Retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishe
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/16337
I just run build/sbt "test-only org.apache.spark.sql.streaming.StreamSuite"
on my local machine, also the whole sql suite, it works fine. Can you re-run
the test? Thanks
---
If your project is s
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70528/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16337
**[Test build #70528 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70528/testReport)**
for PR 16337 at commit
[`1c1900a`](https://github.com/apache/spark/commit/1
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/16337
Hello Nat, thanks for the result checking and detail explanation. Based on
your assessment, I have comment out 4 test cases for now, pending on the
investigation of those open JIRAs.
---
If yo
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16337
**[Test build #70528 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70528/testReport)**
for PR 16337 at commit
[`1c1900a`](https://github.com/apache/spark/commit/1c
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/16337
[Modified test file of simple-in.sql used to run on
DB2](https://github.com/apache/spark/files/667053/simple-in.sql.db2.txt)
[Output of the run result on
DB2](https://github.com/apache/spark/files
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/16337
Second installment: subquery/in-subquery/simple-in.sql
**Recommended action:**
Comment out TC 01.08 and 01.10
**The result checking**
- All except 2 test cases produce differen
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/16337
[Modified test file used to run on
DB2](https://github.com/apache/spark/files/666917/in-group-by.sql.db2.txt)
[Output of the run result on
DB2](https://github.com/apache/spark/files/666921/in-grou
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/16337
I will give an update on the result checking against a second database in
two installments.
The first one is for the file in-group-by.sql.
**Recommended action:**
Comment out TC 0
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70469/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16337
**[Test build #70469 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70469/testReport)**
for PR 16337 at commit
[`9c584fb`](https://github.com/apache/spark/commit/9
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16337
**[Test build #70469 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70469/testReport)**
for PR 16337 at commit
[`9c584fb`](https://github.com/apache/spark/commit/9c
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/16337
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/16337
when I click the link, it shows
Test Result (no failures), not sure why the Test Failed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on Gi
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70457/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/16337
Nat will run against DB2 and provide result.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fea
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/16337
What are the refernce query results we can compare? For example, from DB2
or Hive?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/16337
Hello All:
I have divided the test case to small groups based the discussion, and this
pr will be the first pr for the IN subquery, it covers the simple and group-by
cases.
This the
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16337
**[Test build #70457 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70457/testReport)**
for PR 16337 at commit
[`9c584fb`](https://github.com/apache/spark/commit/9c
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/16337
yea those sound great.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/16337
We can live with the uniqueness at this point. I'd like to get your
feedback (positive, I hope) on the structure of the directories and the naming
convention of the test files so we can move forward w
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/16337
If the uniqueness is a problem, we can also just update SQLQueryTestSuite
to use the path rather than just file name.
---
If your project is set up for it, you can reply to this email and have your
r
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/16337
I have tested a few runs on `SQLQueryTestSuite` to confirm it allows to
have sub-directories under
`sql/core/src/test/resources/sql-tests/[inputs|results]` to group test files
further. By reading th
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/16337
I actually don't mind having 01, 02, 03, etc, but still some higher level
grouping would be useful.
---
If your project is set up for it, you can reply to this email and have your
reply appear on Git
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/16337
Let me try to summarize the comments around the structure of the test files
here:
1. A single file of 200+ test cases are too big. We prefer smaller files
with logical groupings.
2. File name
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/16337
We will separate this file to multiple files.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this f
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/16337
Can we create some higher level logical grouping and put these queries into
multiple files? It's not as useful to dump 200 queries in one file.
---
If your project is set up for it, you can reply to
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/16337
Could we split the test suites based on the subquery types?
- [With
aliases](https://technet.microsoft.com/en-us/library/ms190410(v=sql.105).aspx).
- With IN or NOT IN
- [Sub
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/16337
In Spark, we can use the following way to send the whole query to the
underlying data source.
```SQL
CREATE TEMPORARY TABLE jdbcTable
USING jdbc
OPTIONS (
url "jdbc:postgre
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70362/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16337
**[Test build #70362 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70362/testReport)**
for PR 16337 at commit
[`c2ca009`](https://github.com/apache/spark/commit/c
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/16337
@hvanhovell Does Spark JDBC connector act purely as a JDBC driver which
supposes to send an entire SQL statement to run on the destination system? If
there are part of the SQL statement processed in S
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/16337
On 1 & 2 - it might be nice if we can use the spark JDBC connector to run
the tests directly on a different system. That would make comparisons trivial.
On 3 - what is the current run-ti
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16337
**[Test build #70362 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70362/testReport)**
for PR 16337 at commit
[`c2ca009`](https://github.com/apache/spark/commit/c2
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/16337
- On 1&2: We will show the output from a comparison of the result sets with
a second DB.
- On 3: We wrote them. It's labour intensive. However, we plan to work on a
better method of adding test co
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/16337
@nsyca @kevinyu98
### 1. Correctness
> how about running the same set of test cases against a second SQL system
and comparing the results?
Yeah, that works. I am just trying to make su
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/16337
@hvanhovell: On the correctness, how about running the same set of test
cases against a second SQL system and comparing the results? If the two systems
yield the same result, we can assume Spark has t
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/16337
@kevinyu98 this looks like pretty solid work. A couple of questions:
- This is a **huge** PR. This will be hard to review. Is there a change
that we can split this up in more digestible
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70350/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16337
**[Test build #70350 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70350/testReport)**
for PR 16337 at commit
[`ee1e14e`](https://github.com/apache/spark/commit/e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16337
**[Test build #70350 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70350/testReport)**
for PR 16337 at commit
[`ee1e14e`](https://github.com/apache/spark/commit/ee
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/16337
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/16337
cc @hvanhovell @nsyca
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishe
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/16337
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16337
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feat
69 matches
Mail list logo
