[
https://issues.apache.org/jira/browse/DATAFU-154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255740#comment-17255740
]
Matthew Hayes commented on DATAFU-154:
--------------------------------------
+1
> Spark Explode Array method
> --------------------------
>
> Key: DATAFU-154
> URL: https://issues.apache.org/jira/browse/DATAFU-154
> Project: DataFu
> Issue Type: New Feature
> Environment: {noformat}
> *no* further _formatting_ is done here{noformat}
> {code:java}
>
> {code}
> {code:java}
>
> {code}
> Reporter: Eyal Allweil
> Priority: Major
> Attachments: DATAFU-154.patch
>
>
> Spark has an
> [explode|https://spark.apache.org/docs/latest/api/sql/index.html#explode]
> method that divides an array into multiple rows. It is sometimes convenient
> to divide an array into multiple columns.
> h5. For example, for input:
> |label|sentence_arr |
> |0.0 |[Hi, I heard, about, Spark] |
> |0.0 |[I wish, Java, could use, case classes] |
> |1.0 |[Logistic, regression, models, are neat]|
> h5. the output could be
> |label|sentence_arr |token0 |token1 |token2 |token3 |
> |0.0 |[Hi, I heard, about, Spark] |Hi |I heard |about |Spark |
> |0.0 |[I wish, Java, could use, case classes] |I wish |Java |could use|case
> classes|
> |1.0 |[Logistic, regression, models, are neat]|Logistic|regression|models
> |are neat |
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
