[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97893/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22219 **[Test build #97893 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97893/testReport)** for PR 22219 at commit [`136a4f9`](https://github.com/apache/spark/commit/136a4f9b4d205b0c94ae71f465c4790b759e6f1d). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22219 **[Test build #97893 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97893/testReport)** for PR 22219 at commit [`136a4f9`](https://github.com/apache/spark/commit/136a4f9b4d205b0c94ae71f465c4790b759e6f1d). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22219 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97530/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22219 **[Test build #97530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97530/testReport)** for PR 22219 at commit [`136a4f9`](https://github.com/apache/spark/commit/136a4f9b4d205b0c94ae71f465c4790b759e6f1d). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user Dooyoung-Hwang commented on the issue: https://github.com/apache/spark/pull/22219 I refactored collectionResultAsSeqView function with using implicit class. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22219 **[Test build #97530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97530/testReport)** for PR 22219 at commit [`136a4f9`](https://github.com/apache/spark/commit/136a4f9b4d205b0c94ae71f465c4790b759e6f1d). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97492/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22219 **[Test build #97492 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97492/testReport)** for PR 22219 at commit [`9158edc`](https://github.com/apache/spark/commit/9158edc52405b4c31127311735dbf41dbe6d17f7). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22219 **[Test build #97492 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97492/testReport)** for PR 22219 at commit [`9158edc`](https://github.com/apache/spark/commit/9158edc52405b4c31127311735dbf41dbe6d17f7). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22219 cc @srinathshankar @yuchenhuo --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97452/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22219 **[Test build #97452 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97452/testReport)** for PR 22219 at commit [`e5baa50`](https://github.com/apache/spark/commit/e5baa50daa6cd531399fe3c36242c8c06879b120). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user Dooyoung-Hwang commented on the issue: https://github.com/apache/spark/pull/22219 Dear reviewers (cc : @dongjoon-hyun ) I updated these. 1. No behavior changes, if the new config is off. So, [PR SPARK-25353](https://github.com/apache/spark/pull/22347) is not required for this PR anymore. 2. Apply review comments. (Define config as boolean, and do not add function to DataSet.) 3. Add test case to test fetch_next & fetch_first. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22219 **[Test build #97452 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97452/testReport)** for PR 22219 at commit [`e5baa50`](https://github.com/apache/spark/commit/e5baa50daa6cd531399fe3c36242c8c06879b120). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97388/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22219 **[Test build #97388 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97388/testReport)** for PR 22219 at commit [`2a41b70`](https://github.com/apache/spark/commit/2a41b704f3aaa80aeb2d6c5b1118ee7faea5bf79). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22219 **[Test build #97388 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97388/testReport)** for PR 22219 at commit [`2a41b70`](https://github.com/apache/spark/commit/2a41b704f3aaa80aeb2d6c5b1118ee7faea5bf79). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97376/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22219 **[Test build #97376 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97376/testReport)** for PR 22219 at commit [`f05570c`](https://github.com/apache/spark/commit/f05570c2a8fc88d35085ee8ca377785acc400cdd). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22219 **[Test build #97376 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97376/testReport)** for PR 22219 at commit [`f05570c`](https://github.com/apache/spark/commit/f05570c2a8fc88d35085ee8ca377785acc400cdd). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97363/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22219 **[Test build #97363 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97363/testReport)** for PR 22219 at commit [`ffafd62`](https://github.com/apache/spark/commit/ffafd62b6d5b916e286ee6870e0db168d36a09eb). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22219 **[Test build #97363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97363/testReport)** for PR 22219 at commit [`ffafd62`](https://github.com/apache/spark/commit/ffafd62b6d5b916e286ee6870e0db168d36a09eb). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22219 nope not yet. It needs some more review iterations. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22219 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user tooptoop4 commented on the issue: https://github.com/apache/spark/pull/22219 @HyukjinKwon can this be merged? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22219 @Dooyoung-Hwang Would it possible to add a test case to verify result with and without incrementalCollects by changing a value of `spark.sql.thriftServer.batchDeserializeLimit`? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user Dooyoung-Hwang commented on the issue: https://github.com/apache/spark/pull/22219 @kiszk @viirya @HyukjinKwon @cloud-fan Could you review this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user Dooyoung-Hwang commented on the issue: https://github.com/apache/spark/pull/22219 Change the accessor of collectCountAndIterator to private[sql]. And updated doc of feature that I define in ThriftServer. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user Dooyoung-Hwang commented on the issue: https://github.com/apache/spark/pull/22219 Add test cases. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22219 Would it possible to prepare test cases? IIUC, this feature can be enabled without thriftServer by writing some test code. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user Dooyoung-Hwang commented on the issue: https://github.com/apache/spark/pull/22219 Yes, I verified results of a variety of queries, and memory & performance. This patch passed all our query test. And there was no performance degradation in our test cases. And below is result of memory test. I checked **memory utilization of Old Gen in JVM Heap** when executes query of **2,481,284 rows**. (I Executed "jstat -gc thriftserver-pid" and checked OU field.) After patch : 283910.0KB -> 316108.3KB => 31.44MB increases Before patch : 279425.6KB -> 1511834.2KB => 1203.52MB increases Memory improvement is very large, because the size of compressed result buffer surprisingly smaller than I expected. **Decompressed InternalRows are collected immediately after sending them while Young GC is done**, so the usage of Old Gen Heap is much smaller than before. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22219 Did you verify this feature manually? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22219 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
