[jira] [Updated] (SPARK-6177) Add note in LDA example to remind possible coalesce
[ https://issues.apache.org/jira/browse/SPARK-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6177: - Priority: Trivial (was: Minor) Assignee: yuhao yang Add note in LDA example to remind possible coalesce Key: SPARK-6177 URL: https://issues.apache.org/jira/browse/SPARK-6177 Project: Spark Issue Type: Improvement Components: Examples, MLlib Affects Versions: 1.2.1 Reporter: yuhao yang Assignee: yuhao yang Priority: Trivial Fix For: 1.4.0 Original Estimate: 1h Remaining Estimate: 1h Add comment to introduce coalesce to LDA example to avoid the possible massive partitions from sc.textFile. sc.textFile will create RDD with one partition for each file, and the possible massive partitions downgrades LDA performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-6177) Add note for
[ https://issues.apache.org/jira/browse/SPARK-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-6177: -- Summary: Add note for (was: LDA should check partitions size of the input) Add note for - Key: SPARK-6177 URL: https://issues.apache.org/jira/browse/SPARK-6177 Project: Spark Issue Type: Improvement Components: Examples, MLlib Affects Versions: 1.2.1 Reporter: yuhao yang Priority: Minor Original Estimate: 1h Remaining Estimate: 1h Add comment to introduce coalesce to LDA example to avoid the possible massive partitions from sc.textFile. sc.textFile will create RDD with one partition for each file, and the possible massive partitions downgrades LDA performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-6177) Add note in LDA example to remind possible coalesce
[ https://issues.apache.org/jira/browse/SPARK-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-6177: -- Summary: Add note in LDA example to remind possible coalesce (was: Add note for ) Add note in LDA example to remind possible coalesce Key: SPARK-6177 URL: https://issues.apache.org/jira/browse/SPARK-6177 Project: Spark Issue Type: Improvement Components: Examples, MLlib Affects Versions: 1.2.1 Reporter: yuhao yang Priority: Minor Original Estimate: 1h Remaining Estimate: 1h Add comment to introduce coalesce to LDA example to avoid the possible massive partitions from sc.textFile. sc.textFile will create RDD with one partition for each file, and the possible massive partitions downgrades LDA performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
