[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12103 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-206057723 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-206057724 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55043/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-206057262 **[Test build #55043 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55043/consoleFull)** for PR 12103 at commit [`2002131`](https://github.com/apache/spark/commit/2002131156909e42d0617e684a7f6fa373699d92). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-206032413 **[Test build #55043 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55043/consoleFull)** for PR 12103 at commit [`2002131`](https://github.com/apache/spark/commit/2002131156909e42d0617e684a7f6fa373699d92). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user sameeragarwal commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-206030946 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-206025299 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-206025302 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55035/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-206025295 **[Test build #55035 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55035/consoleFull)** for PR 12103 at commit [`2002131`](https://github.com/apache/spark/commit/2002131156909e42d0617e684a7f6fa373699d92). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-206024319 **[Test build #55035 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55035/consoleFull)** for PR 12103 at commit [`2002131`](https://github.com/apache/spark/commit/2002131156909e42d0617e684a7f6fa373699d92). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-206023394 Seems the last test run was 4 days ago. I just triggered a new one. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-206023222 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user sameeragarwal commented on a diff in the pull request:
https://github.com/apache/spark/pull/12103#discussion_r58618637
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java
---
@@ -232,6 +234,96 @@ public MapData getMap(int ordinal) {
public Object get(int ordinal, DataType dataType) {
throw new NotImplementedException();
}
+
+@Override
+public void update(int ordinal, Object value) {
--- End diff --
yes, this shouldn't be on the hot path. It was just a method that needed to
be overridden while implementing `MutableRow`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user nongli commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-205991469 lgtm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user nongli commented on a diff in the pull request:
https://github.com/apache/spark/pull/12103#discussion_r58617460
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java
---
@@ -232,6 +234,96 @@ public MapData getMap(int ordinal) {
public Object get(int ordinal, DataType dataType) {
throw new NotImplementedException();
}
+
+@Override
+public void update(int ordinal, Object value) {
--- End diff --
Why did you have to implement this? We should try not to call this in the
hot path.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204673703 **[Test build #2731 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2731/consoleFull)** for PR 12103 at commit [`2002131`](https://github.com/apache/spark/commit/2002131156909e42d0617e684a7f6fa373699d92). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204664119 **[Test build #2731 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2731/consoleFull)** for PR 12103 at commit [`2002131`](https://github.com/apache/spark/commit/2002131156909e42d0617e684a7f6fa373699d92). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204635320 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204635321 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54748/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204635280 **[Test build #54748 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54748/consoleFull)** for PR 12103 at commit [`2002131`](https://github.com/apache/spark/commit/2002131156909e42d0617e684a7f6fa373699d92). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204624962 **[Test build #54748 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54748/consoleFull)** for PR 12103 at commit [`2002131`](https://github.com/apache/spark/commit/2002131156909e42d0617e684a7f6fa373699d92). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user sameeragarwal commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204624420 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204624299 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204624300 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54746/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204624259 **[Test build #54746 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54746/consoleFull)** for PR 12103 at commit [`2002131`](https://github.com/apache/spark/commit/2002131156909e42d0617e684a7f6fa373699d92). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/12103#discussion_r58283672 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/AggregateHashMap.java --- @@ -19,6 +19,8 @@ import java.util.Arrays; +import com.google.common.annotations.VisibleForTesting; --- End diff -- just to annotate `public int find(long key)` that is only used in `BenchmarkWholeStageCodegen`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user sameeragarwal commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204615772 Thanks, comments addressed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204615748 **[Test build #54746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54746/consoleFull)** for PR 12103 at commit [`2002131`](https://github.com/apache/spark/commit/2002131156909e42d0617e684a7f6fa373699d92). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user nongli commented on a diff in the pull request:
https://github.com/apache/spark/pull/12103#discussion_r58247692
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java
---
@@ -232,6 +233,56 @@ public MapData getMap(int ordinal) {
public Object get(int ordinal, DataType dataType) {
throw new NotImplementedException();
}
+
+@Override
+public void setNullAt(int ordinal) {
+ columns[ordinal].putNull(rowId);
--- End diff --
how do you set something to NULL?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/12103#discussion_r58247570 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/AggregateHashMap.java --- @@ -19,6 +19,8 @@ import java.util.Arrays; +import com.google.common.annotations.VisibleForTesting; --- End diff -- is this necessary? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user nongli commented on a diff in the pull request:
https://github.com/apache/spark/pull/12103#discussion_r58247531
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/BenchmarkWholeStageCodegen.scala
---
@@ -472,9 +472,8 @@ class BenchmarkWholeStageCodegen extends SparkFunSuite {
.add("value", LongType)
val map = new AggregateHashMap(schema)
while (i < numKeys) {
-val idx = map.findOrInsert(i.toLong)
-map.batch.column(1).putLong(map.buckets(idx),
- map.batch.column(1).getLong(map.buckets(idx)) + 1)
+val row = map.findOrInsert(i.toLong)
+row.setLong(1, row.getLong(1))
--- End diff --
row.getLong(1) + 1?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user nongli commented on a diff in the pull request:
https://github.com/apache/spark/pull/12103#discussion_r58247456
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java
---
@@ -232,6 +233,56 @@ public MapData getMap(int ordinal) {
public Object get(int ordinal, DataType dataType) {
throw new NotImplementedException();
}
+
+@Override
+public void setNullAt(int ordinal) {
+ columns[ordinal].putNull(rowId);
+}
+
+@Override
+public void update(int ordinal, Object value) {
+ throw new NotImplementedException();
+}
+
+@Override
+public void setBoolean(int ordinal, boolean value) {
+ columns[ordinal].putBoolean(rowId, value);
+}
+
+@Override
+public void setByte(int ordinal, byte value) {
+ columns[ordinal].putByte(rowId, value);
+}
+
+@Override
+public void setShort(int ordinal, short value) {
+ columns[ordinal].putShort(rowId, value);
+}
+
+@Override
+public void setInt(int ordinal, int value) {
+ columns[ordinal].putInt(rowId, value);
+}
+
+@Override
+public void setLong(int ordinal, long value) {
+ columns[ordinal].putLong(rowId, value);
+}
+
+@Override
+public void setFloat(int ordinal, float value) {
+ columns[ordinal].putFloat(rowId, value);
+}
+
+@Override
+public void setDouble(int ordinal, double value) {
+ columns[ordinal].putDouble(rowId, value);
+}
+
+@Override
+public void setDecimal(int ordinal, Decimal value, int precision) {
+ throw new NotImplementedException();
--- End diff --
why not?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user nongli commented on a diff in the pull request:
https://github.com/apache/spark/pull/12103#discussion_r58247399
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java
---
@@ -232,6 +233,56 @@ public MapData getMap(int ordinal) {
public Object get(int ordinal, DataType dataType) {
throw new NotImplementedException();
}
+
+@Override
+public void setNullAt(int ordinal) {
+ columns[ordinal].putNull(rowId);
--- End diff --
Let's assert it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user sameeragarwal commented on a diff in the pull request:
https://github.com/apache/spark/pull/12103#discussion_r58241926
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchSuite.scala
---
@@ -756,4 +756,33 @@ class ColumnarBatchSuite extends SparkFunSuite {
}}
}
}
+
+ test("mutable ColumnarBatch rows") {
+(MemoryMode.ON_HEAP :: MemoryMode.OFF_HEAP :: Nil).foreach { memMode
=> {
--- End diff --
Thank you SparkBot. Such awesomeness! :heart:
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user DeepSparkBot commented on a diff in the pull request:
https://github.com/apache/spark/pull/12103#discussion_r58240859
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchSuite.scala
---
@@ -756,4 +756,33 @@ class ColumnarBatchSuite extends SparkFunSuite {
}}
}
}
+
+ test("mutable ColumnarBatch rows") {
+(MemoryMode.ON_HEAP :: MemoryMode.OFF_HEAP :: Nil).foreach { memMode
=> {
--- End diff --
**Detected minor style violation.**
Suggested improvement:
```
(MemoryMode.ON_HEAP :: MemoryMode.OFF_HEAP :: Nil).foreach { memMode =>
...
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user sameeragarwal commented on a diff in the pull request:
https://github.com/apache/spark/pull/12103#discussion_r58238680
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java
---
@@ -232,6 +233,56 @@ public MapData getMap(int ordinal) {
public Object get(int ordinal, DataType dataType) {
throw new NotImplementedException();
}
+
+@Override
+public void setNullAt(int ordinal) {
+ columns[ordinal].putNull(rowId);
--- End diff --
hmm, good point. It'd be good safety measure, but might affect performance
if do this additional check for each row/column.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/12103#discussion_r58162424
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java
---
@@ -232,6 +233,56 @@ public MapData getMap(int ordinal) {
public Object get(int ordinal, DataType dataType) {
throw new NotImplementedException();
}
+
+@Override
+public void setNullAt(int ordinal) {
+ columns[ordinal].putNull(rowId);
--- End diff --
Is it better to add checks whether ColumnVector.isConstant == false?
cc @nongli
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204193375 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204193378 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54676/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204193208 **[Test build #54676 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54676/consoleFull)** for PR 12103 at commit [`a1fe9f8`](https://github.com/apache/spark/commit/a1fe9f83c182ea02b6c9a8825ba90f10a5e6d638). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * ` public static final class Row extends MutableRow ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204166988 **[Test build #54676 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54676/consoleFull)** for PR 12103 at commit [`a1fe9f8`](https://github.com/apache/spark/commit/a1fe9f83c182ea02b6c9a8825ba90f10a5e6d638). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14320][SQL] Make ColumnarBatch.Row muta...
Github user sameeragarwal commented on the pull request: https://github.com/apache/spark/pull/12103#issuecomment-204166342 cc @nongli --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
