[jira] [Commented] (SPARK-8230) complex function: size
[ https://issues.apache.org/jira/browse/SPARK-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630920#comment-14630920 ] Apache Spark commented on SPARK-8230: - User 'EntilZha' has created a pull request for this issue: https://github.com/apache/spark/pull/7462 complex function: size -- Key: SPARK-8230 URL: https://issues.apache.org/jira/browse/SPARK-8230 Project: Spark Issue Type: Sub-task Components: SQL Reporter: Reynold Xin Assignee: Cheng Hao size(MapK.V): int size(ArrayT): int return the number of elements in the map or array. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8230) complex function: size
[ https://issues.apache.org/jira/browse/SPARK-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630921#comment-14630921 ] Pedro Rodriguez commented on SPARK-8230: [~chenghao], code is ready for review here: https://github.com/apache/spark/pull/7462 complex function: size -- Key: SPARK-8230 URL: https://issues.apache.org/jira/browse/SPARK-8230 Project: Spark Issue Type: Sub-task Components: SQL Reporter: Reynold Xin Assignee: Cheng Hao size(MapK.V): int size(ArrayT): int return the number of elements in the map or array. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8230) complex function: size
[ https://issues.apache.org/jira/browse/SPARK-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630575#comment-14630575 ] Pedro Rodriguez commented on SPARK-8230: I took a look at that code as well as several other examples. I think I have a good idea of how to start implementing, I will update here with a link to code once I have something complex function: size -- Key: SPARK-8230 URL: https://issues.apache.org/jira/browse/SPARK-8230 Project: Spark Issue Type: Sub-task Components: SQL Reporter: Reynold Xin Assignee: Cheng Hao size(MapK.V): int size(ArrayT): int return the number of elements in the map or array. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8230) complex function: size
[ https://issues.apache.org/jira/browse/SPARK-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629107#comment-14629107 ] Cheng Hao commented on SPARK-8230: -- [~pedrorodriguez], actually [~TarekAuel] set a good example at https://github.com/apache/spark/pull/7214/files, you can start get understanding his PR first, and I would like to review your code once it's ready. complex function: size -- Key: SPARK-8230 URL: https://issues.apache.org/jira/browse/SPARK-8230 Project: Spark Issue Type: Sub-task Components: SQL Reporter: Reynold Xin Assignee: Cheng Hao size(MapK.V): int size(ArrayT): int return the number of elements in the map or array. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8230) complex function: size
[ https://issues.apache.org/jira/browse/SPARK-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629079#comment-14629079 ] Pedro Rodriguez commented on SPARK-8230: Moving to here instead of mailing list. I would be interested in implementing it. I am not certain of all the steps, thus far: 1. The code that actually does the work would go somewhere here: https://github.com/apache/spark/tree/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions 2. Which would be added here: https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/functions.scala 3. Then would have to be brought to the Java/Python APIs. I am not certain where in sql.catalyst.expressions this would go. On the parent JIRA it looks like this is labeled as a complex function (because of type signature accepting anything?), so in a new file complex.scala? I am also unsure what general form it would take. I have been looking through how the other functions are defined. As far as taking the column as input, explode (https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala) looks like it would provide a good starting point for getting the Array/Map, then look at any of the other functions which returns an integer to see how it should return a value. Lastly, I noticed there are two other complex functions: sort_array and array_contains, would these be related in some way (much as the mathematical operations all share similar code)? Thanks complex function: size -- Key: SPARK-8230 URL: https://issues.apache.org/jira/browse/SPARK-8230 Project: Spark Issue Type: Sub-task Components: SQL Reporter: Reynold Xin Assignee: Cheng Hao size(MapK.V): int size(ArrayT): int return the number of elements in the map or array. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8230) complex function: size
[ https://issues.apache.org/jira/browse/SPARK-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14579906#comment-14579906 ] Cheng Hao commented on SPARK-8230: -- I'll take this one. complex function: size -- Key: SPARK-8230 URL: https://issues.apache.org/jira/browse/SPARK-8230 Project: Spark Issue Type: Sub-task Components: SQL Reporter: Reynold Xin size(MapK.V): int size(ArrayT): int return the number of elements in the map or array. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
