[ https://issues.apache.org/jira/browse/DATAFU-154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eyal Allweil updated DATAFU-154: -------------------------------- Description: Spark has an [explode|https://spark.apache.org/docs/latest/api/sql/index.html#explode] method that divides an array into multiple rows. It is sometimes convenient to divide an array into multiple columns. h5. For example, for input: |label|sentence_arr | |0.0 |[Hi, I heard, about, Spark] | |0.0 |[I wish, Java, could use, case classes] | |1.0 |[Logistic, regression, models, are neat]| h5. the output could be |label|sentence_arr |token0 |token1 |token2 |token3 | |0.0 |[Hi, I heard, about, Spark] |Hi |I heard |about |Spark | |0.0 |[I wish, Java, could use, case classes] |I wish |Java |could use|case classes| |1.0 |[Logistic, regression, models, are neat]|Logistic|regression|models |are neat | was: Spark has an [explode|https://spark.apache.org/docs/latest/api/sql/index.html#explode] method that divides an array into multiple rows. It is sometimes convenient to divide an array into multiple columns. For example, for input +-----+----------------------------------------+ |label|sentence_arr | +-----+----------------------------------------+ |0.0 |[Hi, I heard, about, Spark] | |0.0 |[I wish, Java, could use, case classes] | |1.0 |[Logistic, regression, models, are neat]| +-----+----------------------------------------+ the output could be +-----+----------------------------------------+--------+----------+---------+------------+ |label|sentence_arr |token0 |token1 |token2 |token3 | +-----+----------------------------------------+--------+----------+---------+------------+ |0.0 |[Hi, I heard, about, Spark] |Hi |I heard |about |Spark | |0.0 |[I wish, Java, could use, case classes] |I wish |Java |could use|case classes| |1.0 |[Logistic, regression, models, are neat]|Logistic|regression|models |are neat | +-----+----------------------------------------+--------+----------+---------+------------+ > Spark Explode Array method > -------------------------- > > Key: DATAFU-154 > URL: https://issues.apache.org/jira/browse/DATAFU-154 > Project: DataFu > Issue Type: New Feature > Environment: {noformat} > *no* further _formatting_ is done here{noformat} > {code:java} > > {code} > {code:java} > > {code} > Reporter: Eyal Allweil > Priority: Major > > Spark has an > [explode|https://spark.apache.org/docs/latest/api/sql/index.html#explode] > method that divides an array into multiple rows. It is sometimes convenient > to divide an array into multiple columns. > h5. For example, for input: > |label|sentence_arr | > |0.0 |[Hi, I heard, about, Spark] | > |0.0 |[I wish, Java, could use, case classes] | > |1.0 |[Logistic, regression, models, are neat]| > h5. the output could be > |label|sentence_arr |token0 |token1 |token2 |token3 | > |0.0 |[Hi, I heard, about, Spark] |Hi |I heard |about |Spark | > |0.0 |[I wish, Java, could use, case classes] |I wish |Java |could use|case > classes| > |1.0 |[Logistic, regression, models, are neat]|Logistic|regression|models > |are neat | > -- This message was sent by Atlassian Jira (v8.3.4#803005)