[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18409 LGTM, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18409 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18409 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78641/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18409 **[Test build #78641 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78641/testReport)** for PR 18409 at commit [`7878c49`](https://github.com/apache/spark/commit/7878c4985bc949fddefc1b93cec4ac0aff478ac8). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18409 **[Test build #78641 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78641/testReport)** for PR 18409 at commit [`7878c49`](https://github.com/apache/spark/commit/7878c4985bc949fddefc1b93cec4ac0aff478ac8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/18409 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18409 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18409 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78557/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18409 **[Test build #78557 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78557/testReport)** for PR 18409 at commit [`7878c49`](https://github.com/apache/spark/commit/7878c4985bc949fddefc1b93cec4ac0aff478ac8). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18409 **[Test build #78557 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78557/testReport)** for PR 18409 at commit [`7878c49`](https://github.com/apache/spark/commit/7878c4985bc949fddefc1b93cec4ac0aff478ac8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18409 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18409 cc @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18409 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18409 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78551/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18409 **[Test build #78551 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78551/testReport)** for PR 18409 at commit [`7878c49`](https://github.com/apache/spark/commit/7878c4985bc949fddefc1b93cec4ac0aff478ac8). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18409 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78550/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18409 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18409 **[Test build #78550 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78550/testReport)** for PR 18409 at commit [`a349962`](https://github.com/apache/spark/commit/a349962eb04054a94de01b976b5c6217ea72519b). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18409 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18409 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78548/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18409 **[Test build #78548 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78548/testReport)** for PR 18409 at commit [`42238b8`](https://github.com/apache/spark/commit/42238b862e7b6e984967139e68259d8a5caae20f). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18409 **[Test build #78551 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78551/testReport)** for PR 18409 at commit [`7878c49`](https://github.com/apache/spark/commit/7878c4985bc949fddefc1b93cec4ac0aff478ac8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18409 **[Test build #78550 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78550/testReport)** for PR 18409 at commit [`a349962`](https://github.com/apache/spark/commit/a349962eb04054a94de01b976b5c6217ea72519b). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18409 **[Test build #78548 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78548/testReport)** for PR 18409 at commit [`42238b8`](https://github.com/apache/spark/commit/42238b862e7b6e984967139e68259d8a5caae20f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/18409 @gatorsmile Thanks, I have revised the code --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/18409
Example:
```java
codegenStringSeq(sql("select 1").queryExecution.executedPlan)
```
The example will return Seq[(String, String)] of length 1, containing the
subtree as string and the corresponding generated code.
The subtree as string:
> (*Project [1 AS 1#0]
> +- Scan OneRowRelation[]
The generated code:
```java
/* 001 */ public Object generate(Object[] references) {
/* 002 */ return new GeneratedIterator(references);
/* 003 */ }
/* 004 */
/* 005 */ final class GeneratedIterator extends
org.apache.spark.sql.execution.BufferedRowIterator {
/* 006 */ private Object[] references;
/* 007 */ private scala.collection.Iterator[] inputs;
/* 008 */ private scala.collection.Iterator inputadapter_input;
/* 009 */ private UnsafeRow project_result;
/* 010 */ private
org.apache.spark.sql.catalyst.expressions.codegen.BufferHolder project_holder;
/* 011 */ private
org.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter
project_rowWriter;
/* 012 */
/* 013 */ public GeneratedIterator(Object[] references) {
/* 014 */ this.references = references;
/* 015 */ }
/* 016 */
/* 017 */ public void init(int index, scala.collection.Iterator[] inputs)
{
/* 018 */ partitionIndex = index;
/* 019 */ this.inputs = inputs;
/* 020 */ inputadapter_input = inputs[0];
/* 021 */ project_result = new UnsafeRow(1);
/* 022 */ project_holder = new
org.apache.spark.sql.catalyst.expressions.codegen.BufferHolder(project_result,
0);
/* 023 */ project_rowWriter = new
org.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter(project_holder,
1);
/* 024 */
/* 025 */ }
/* 026 */
/* 027 */ protected void processNext() throws java.io.IOException {
/* 028 */ while (inputadapter_input.hasNext() && !stopEarly()) {
/* 029 */ InternalRow inputadapter_row = (InternalRow)
inputadapter_input.next();
/* 030 */ project_rowWriter.write(0, 1);
/* 031 */ append(project_result);
/* 032 */ if (shouldStop()) return;
/* 033 */ }
/* 034 */ }
/* 035 */
/* 036 */ }
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18409 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18409 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78541/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18409 **[Test build #78541 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78541/testReport)** for PR 18409 at commit [`73ef241`](https://github.com/apache/spark/commit/73ef241530d805ef9dca76e5412f3f3488369fc6). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18409: [SPARK-21196] Split codegen info of query plan into sequ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18409 **[Test build #78541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78541/testReport)** for PR 18409 at commit [`73ef241`](https://github.com/apache/spark/commit/73ef241530d805ef9dca76e5412f3f3488369fc6). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
