Repository: spark Updated Branches: refs/heads/master 6db315746 -> c7ad08520
[SPARK-4130][MLlib] Fixing libSVM parser bug with extra whitespace This simple patch filters out extra whitespace entries. Author: Joseph E. Gonzalez <joseph.e.gonza...@gmail.com> Author: Joey <joseph.e.gonza...@gmail.com> Closes #2996 from jegonzal/loadLibSVM and squashes the following commits: e0227ab [Joey] improving readability e028e84 [Joseph E. Gonzalez] fixing whitespace bug in loadLibSVMFile when parsing libSVM files Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c7ad0852 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c7ad0852 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/c7ad0852 Branch: refs/heads/master Commit: c7ad0852084dc28f3ebc144adfd4928b23f1c8ea Parents: 6db3157 Author: Joseph E. Gonzalez <joseph.e.gonza...@gmail.com> Authored: Thu Oct 30 00:05:57 2014 -0700 Committer: Xiangrui Meng <m...@databricks.com> Committed: Thu Oct 30 00:05:57 2014 -0700 ---------------------------------------------------------------------- mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/c7ad0852/mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala ---------------------------------------------------------------------- diff --git a/mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala b/mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala index dce0adf..b88e08b 100644 --- a/mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala +++ b/mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala @@ -76,7 +76,7 @@ object MLUtils { .map { line => val items = line.split(' ') val label = items.head.toDouble - val (indices, values) = items.tail.map { item => + val (indices, values) = items.tail.filter(_.nonEmpty).map { item => val indexAndValue = item.split(':') val index = indexAndValue(0).toInt - 1 // Convert 1-based indices to 0-based. val value = indexAndValue(1).toDouble --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org