[ https://issues.apache.org/jira/browse/DATAFU-154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255740#comment-17255740 ]
Matthew Hayes commented on DATAFU-154: -------------------------------------- +1 > Spark Explode Array method > -------------------------- > > Key: DATAFU-154 > URL: https://issues.apache.org/jira/browse/DATAFU-154 > Project: DataFu > Issue Type: New Feature > Environment: {noformat} > *no* further _formatting_ is done here{noformat} > {code:java} > > {code} > {code:java} > > {code} > Reporter: Eyal Allweil > Priority: Major > Attachments: DATAFU-154.patch > > > Spark has an > [explode|https://spark.apache.org/docs/latest/api/sql/index.html#explode] > method that divides an array into multiple rows. It is sometimes convenient > to divide an array into multiple columns. > h5. For example, for input: > |label|sentence_arr | > |0.0 |[Hi, I heard, about, Spark] | > |0.0 |[I wish, Java, could use, case classes] | > |1.0 |[Logistic, regression, models, are neat]| > h5. the output could be > |label|sentence_arr |token0 |token1 |token2 |token3 | > |0.0 |[Hi, I heard, about, Spark] |Hi |I heard |about |Spark | > |0.0 |[I wish, Java, could use, case classes] |I wish |Java |could use|case > classes| > |1.0 |[Logistic, regression, models, are neat]|Logistic|regression|models > |are neat | > -- This message was sent by Atlassian Jira (v8.3.4#803005)